Merge tag 'drm-intel-next-2017-06-19' of git://anongit.freedesktop.org/git/drm-intel into drm-next

Final pile of features for 4.13 New uabi: - batch bo in first slot, for faster execbuf assembly in userspace (Chris Wilson) - (sub)slice getparam, needed for mesa perf support (Robert Bragg) First pile of patches for cnl/cfl support, maintained by Rodrigo but with lots of contributions from others. Still incomplete since public review still ongoing. Features/refactoring: - Make execbuf faster (Chris Wilson), a pile of series to make execbuf buffer handling have fewer passes, use less list walking, postpone more work to async workers and shuffle buffers less, all to make the common case much faster (in some cases at least). - cold boot support for glk dsi (Madhav Chauhan) - Clean up pipe A quirk and related old platform hacks (Ville) - perf sampling support for kbl/glk (Lionel) - perf cleanups (Robert Bragg) - wire atomic state to backlight code, to avoid pipe lookup hacks (Maarten) - reduce request waiting latency/overhead to remove the spinning and associated cpu cycle wasting (Chris) - fix 90/270 rotation wm computation (Ville) - new ddb allocation algo for skl (Kumar Mahesh) - fix regression due to system suspend optimiazatino (Imre) - the usual pile of small cleanups and refactors all over GVT updates contained in this tag: - optimization for per-VM mmio save/restore (Changbin) - optimization for mmio hash table (Changbin) - scheduler optimization with event (Ping) - vGPU reset refinement (Fred) - other misc refactor and cleanups, etc. * tag 'drm-intel-next-2017-06-19' of git://anongit.freedesktop.org/git/drm-intel: (170 commits) drm/i915: Update DRIVER_DATE to 20170619 drm/i915/cfl: Introduce Coffee Lake workarounds. drm/i915: Store 9 bits of PCI Device ID for platforms with a LP PCH drm/i915: Stash a pointer to the obj's resv in the vma drm/i915: Async GPU relocation processing drm/i915: Allow execbuffer to use the first object as the batch drm/i915: Wait upon userptr get-user-pages within execbuffer drm/i915: First try the previous execbuffer location drm/i915: Store a persistent reference for an object in the execbuffer cache drm/i915: Eliminate lots of iterations over the execobjects array drm/i915: Disable EXEC_OBJECT_ASYNC when doing relocations drm/i915: Pass vma to relocate entry drm/i915: Store a direct lookup from object handle to vma drm/i915: Fix retrieval of hangcheck stats drm/i915: Store i915_gem_object_is_coherent() as a bit next to cache-dirty drm/i915: Mark CPU cache as dirty on every transition for CPU writes drm/i915: Make i915_vma_destroy() static drm/i915: Actually attach the tv_format property to the SDVO connector Revert "drm/i915/skl: New ddb allocation algorithm" drm/i915/glk: Add cold boot sequence for GLK DSI ...

Dave Airlie committed 9 years ago

Commit 305b9ed2 parents eafae13 9ddb8e1108 files · +36839 −3301Browse files

Unified · Split

108 files

drivers/gpu/drm/i915/Makefile +10 −1
drivers/gpu/drm/i915/dvo_ch7xxx.c +4 −6
drivers/gpu/drm/i915/gvt/Makefile +1 −1
drivers/gpu/drm/i915/gvt/cmd_parser.c +3 −53
drivers/gpu/drm/i915/gvt/execlist.c +23 −33
drivers/gpu/drm/i915/gvt/firmware.c +2 −7
drivers/gpu/drm/i915/gvt/gtt.c +9 −6
drivers/gpu/drm/i915/gvt/gvt.c +4 −2
drivers/gpu/drm/i915/gvt/gvt.h +97 −3
drivers/gpu/drm/i915/gvt/handlers.c +177 −126
drivers/gpu/drm/i915/gvt/interrupt.c +10 −10
drivers/gpu/drm/i915/gvt/mmio.c +25 −96
drivers/gpu/drm/i915/gvt/mmio.h +16 −28
drivers/gpu/drm/i915/gvt/mpt.h +1 −2
drivers/gpu/drm/i915/gvt/render.c +40 −8
drivers/gpu/drm/i915/gvt/render.h +2 −2
drivers/gpu/drm/i915/gvt/sched_policy.c +22 −5
drivers/gpu/drm/i915/gvt/scheduler.c +32 −7
drivers/gpu/drm/i915/gvt/scheduler.h +4 −0
drivers/gpu/drm/i915/gvt/trace.h +129 −49
drivers/gpu/drm/i915/gvt/vgpu.c +7 −2
drivers/gpu/drm/i915/i915_debugfs.c +41 −27
drivers/gpu/drm/i915/i915_drv.c +41 −5
drivers/gpu/drm/i915/i915_drv.h +76 −30
drivers/gpu/drm/i915/i915_gem.c +102 −62
drivers/gpu/drm/i915/i915_gem_batch_pool.c +17 −2
drivers/gpu/drm/i915/i915_gem_clflush.c +8 −9
drivers/gpu/drm/i915/i915_gem_context.c +82 −4
drivers/gpu/drm/i915/i915_gem_context.h +26 −0
drivers/gpu/drm/i915/i915_gem_evict.c +65 −54
drivers/gpu/drm/i915/i915_gem_execbuffer.c +1631 −1123
drivers/gpu/drm/i915/i915_gem_gtt.c +13 −3
drivers/gpu/drm/i915/i915_gem_gtt.h +2 −0
drivers/gpu/drm/i915/i915_gem_internal.c +3 −1
drivers/gpu/drm/i915/i915_gem_object.h +18 −4
drivers/gpu/drm/i915/i915_gem_request.c +7 −13
drivers/gpu/drm/i915/i915_gem_shrinker.c +21 −7
drivers/gpu/drm/i915/i915_gem_stolen.c +1 −0
drivers/gpu/drm/i915/i915_gem_userptr.c +19 −5
drivers/gpu/drm/i915/i915_guc_submission.c +1 −1
drivers/gpu/drm/i915/i915_irq.c +4 −2
drivers/gpu/drm/i915/i915_oa_bdw.c +5376 −0
drivers/gpu/drm/i915/i915_oa_bdw.h +40 −0
drivers/gpu/drm/i915/i915_oa_bxt.c +2690 −0
drivers/gpu/drm/i915/i915_oa_bxt.h +40 −0
drivers/gpu/drm/i915/i915_oa_chv.c +2873 −0
drivers/gpu/drm/i915/i915_oa_chv.h +40 −0
drivers/gpu/drm/i915/i915_oa_glk.c +2602 −0
drivers/gpu/drm/i915/i915_oa_glk.h +40 −0
drivers/gpu/drm/i915/i915_oa_hsw.c +187 −76
drivers/gpu/drm/i915/i915_oa_hsw.h +3 −1
drivers/gpu/drm/i915/i915_oa_kblgt2.c +2991 −0
drivers/gpu/drm/i915/i915_oa_kblgt2.h +40 −0
drivers/gpu/drm/i915/i915_oa_kblgt3.c +3040 −0
drivers/gpu/drm/i915/i915_oa_kblgt3.h +40 −0
drivers/gpu/drm/i915/i915_oa_sklgt2.c +3479 −0
drivers/gpu/drm/i915/i915_oa_sklgt2.h +40 −0
drivers/gpu/drm/i915/i915_oa_sklgt3.c +3039 −0
drivers/gpu/drm/i915/i915_oa_sklgt3.h +40 −0
drivers/gpu/drm/i915/i915_oa_sklgt4.c +3093 −0
drivers/gpu/drm/i915/i915_oa_sklgt4.h +40 −0
drivers/gpu/drm/i915/i915_pci.c +57 −30
drivers/gpu/drm/i915/i915_perf.c +1032 −94
drivers/gpu/drm/i915/i915_pvinfo.h +2 −6
drivers/gpu/drm/i915/i915_reg.h +270 −17
drivers/gpu/drm/i915/i915_utils.h +5 −0
drivers/gpu/drm/i915/i915_vgpu.c +35 −19
drivers/gpu/drm/i915/i915_vma.c +25 −3
drivers/gpu/drm/i915/i915_vma.h +15 −5
drivers/gpu/drm/i915/intel_atomic.c +107 −30
drivers/gpu/drm/i915/intel_breadcrumbs.c +43 −30
drivers/gpu/drm/i915/intel_cdclk.c +350 −5
drivers/gpu/drm/i915/intel_csr.c +12 −5
drivers/gpu/drm/i915/intel_ddi.c +377 −15
drivers/gpu/drm/i915/intel_device_info.c +6 −5
drivers/gpu/drm/i915/intel_display.c +162 −158
drivers/gpu/drm/i915/intel_dp.c +48 −142
drivers/gpu/drm/i915/intel_dp_aux_backlight.c +9 −8
drivers/gpu/drm/i915/intel_dpll_mgr.c +435 −2
drivers/gpu/drm/i915/intel_dpll_mgr.h +4 −0
drivers/gpu/drm/i915/intel_drv.h +39 −20
drivers/gpu/drm/i915/intel_dsi.c +83 −84
drivers/gpu/drm/i915/intel_dsi_dcs_backlight.c +10 −12
drivers/gpu/drm/i915/intel_engine_cs.c +64 −22
drivers/gpu/drm/i915/intel_fifo_underrun.c +1 −1
drivers/gpu/drm/i915/intel_guc_loader.c +1 −1
drivers/gpu/drm/i915/intel_gvt.c +32 −11
drivers/gpu/drm/i915/intel_gvt.h +5 −0
drivers/gpu/drm/i915/intel_hdmi.c +38 −119
drivers/gpu/drm/i915/intel_huc.c +1 −1
drivers/gpu/drm/i915/intel_i2c.c +13 −2
drivers/gpu/drm/i915/intel_lrc.c +7 −0
drivers/gpu/drm/i915/intel_lvds.c +16 −57
drivers/gpu/drm/i915/intel_mocs.c +1 −1
drivers/gpu/drm/i915/intel_opregion.c +1 −1
drivers/gpu/drm/i915/intel_overlay.c +0 −1
drivers/gpu/drm/i915/intel_panel.c +186 −79
drivers/gpu/drm/i915/intel_pm.c +126 −22
drivers/gpu/drm/i915/intel_runtime_pm.c +308 −8
drivers/gpu/drm/i915/intel_sdvo.c +284 −256
drivers/gpu/drm/i915/intel_uc.c +5 −2
drivers/gpu/drm/i915/intel_uncore.c +0 −126
drivers/gpu/drm/i915/selftests/huge_gem_object.c +3 −1
drivers/gpu/drm/i915/selftests/i915_gem_evict.c +2 −2
drivers/gpu/drm/i915/selftests/i915_vma.c +3 −13
drivers/gpu/drm/i915/selftests/mock_context.c +11 −1
include/drm/i915_pciids.h +40 −0
include/uapi/drm/i915_drm.h +36 −8

drivers/gpu/drm/i915/Makefile+10−1

@@ -129,7 +129,16 @@ i915-y += i915_vgpu.o · 128 unmodified lines
129	129
130	130	# perf code
131	131	i915-y += i915_perf.o \
132		− i915_oa_hsw.o
	132	+ i915_oa_hsw.o \
	133	+ i915_oa_bdw.o \
	134	+ i915_oa_chv.o \
	135	+ i915_oa_sklgt2.o \
	136	+ i915_oa_sklgt3.o \
	137	+ i915_oa_sklgt4.o \
	138	+ i915_oa_bxt.o \
	139	+ i915_oa_kblgt2.o \
	140	+ i915_oa_kblgt3.o \
	141	+ i915_oa_glk.o
133	142
134	143	ifeq ($(CONFIG_DRM_I915_GVT),y)
135	144	i915-y += intel_gvt.o

drivers/gpu/drm/i915/dvo_ch7xxx.c+4−6

@@ -217,9 +217,8 @@ static bool ch7xxx_init(struct intel_dvo_device *dvo, · 216 unmodified lines
217	217
218	218	name = ch7xxx_get_id(vendor);
219	219	if (!name) {
220		− DRM_DEBUG_KMS("ch7xxx not detected; got 0x%02x from %s "
221		− "slave %d.\n",
222		− vendor, adapter->name, dvo->slave_addr);
	220	+ DRM_DEBUG_KMS("ch7xxx not detected; got VID 0x%02x from %s slave %d.\n",
	221	+ vendor, adapter->name, dvo->slave_addr);
223	222	goto out;
224	223	}
225	224
@@ -229,9 +228,8 @@ static bool ch7xxx_init(struct intel_dvo_device *dvo, · 3 unmodified lines
229	228
230	229	devid = ch7xxx_get_did(device);
231	230	if (!devid) {
232		− DRM_DEBUG_KMS("ch7xxx not detected; got 0x%02x from %s "
233		− "slave %d.\n",
234		− vendor, adapter->name, dvo->slave_addr);
	231	+ DRM_DEBUG_KMS("ch7xxx not detected; got DID 0x%02x from %s slave %d.\n",
	232	+ device, adapter->name, dvo->slave_addr);
235	233	goto out;
236	234	}
237	235

drivers/gpu/drm/i915/gvt/Makefile+1−1

@@ -3,6 +3,6 @@ GVT_SOURCE := gvt.o aperture_gm.o handlers.o vgpu.o trace_points.o firmware.o \ · 2 unmodified lines
3	3	interrupt.o gtt.o cfg_space.o opregion.o mmio.o display.o edid.o \
4	4	execlist.o scheduler.o sched_policy.o render.o cmd_parser.o
5	5
6		−ccflags-y += -I$(src) -I$(src)/$(GVT_DIR) -Wall
	6	+ccflags-y += -I$(src) -I$(src)/$(GVT_DIR)
7	7	i915-y += $(addprefix $(GVT_DIR)/, $(GVT_SOURCE))
8	8	obj-$(CONFIG_DRM_I915_GVT_KVMGT) += $(GVT_DIR)/kvmgt.o

drivers/gpu/drm/i915/gvt/cmd_parser.c+3−53

@@ -2414,53 +2414,13 @@ static void add_cmd_entry(struct intel_gvt gvt, struct cmd_entry e) · 2413 unmodified lines
2414	2414	hash_add(gvt->cmd_table, &e->hlist, e->info->opcode);
2415	2415	}
2416	2416
2417		−#define GVT_MAX_CMD_LENGTH 20 /* In Dword */
2418		−
2419		−static void trace_cs_command(struct parser_exec_state *s,
2420		− cycles_t cost_pre_cmd_handler, cycles_t cost_cmd_handler)
2421		−{
2422		− /* This buffer is used by ftrace to store all commands copied from
2423		− * guest gma space. Sometimes commands can cross pages, this should
2424		− * not be handled in ftrace logic. So this is just used as a
2425		− * 'bounce buffer'
2426		− */
2427		− u32 cmd_trace_buf[GVT_MAX_CMD_LENGTH];
2428		− int i;
2429		− u32 cmd_len = cmd_length(s);
2430		− /* The chosen value of GVT_MAX_CMD_LENGTH are just based on
2431		− * following two considerations:
2432		− * 1) From observation, most common ring commands is not that long.
2433		− * But there are execeptions. So it indeed makes sence to observe
2434		− * longer commands.
2435		− * 2) From the performance and debugging point of view, dumping all
2436		− * contents of very commands is not necessary.
2437		− * We mgith shrink GVT_MAX_CMD_LENGTH or remove this trace event in
2438		− * future for performance considerations.
2439		− */
2440		− if (unlikely(cmd_len > GVT_MAX_CMD_LENGTH)) {
2441		− gvt_dbg_cmd("cmd length exceed tracing limitation!\n");
2442		− cmd_len = GVT_MAX_CMD_LENGTH;
2443		− }
2444		−
2445		− for (i = 0; i < cmd_len; i++)
2446		− cmd_trace_buf[i] = cmd_val(s, i);
2447		−
2448		− trace_gvt_command(s->vgpu->id, s->ring_id, s->ip_gma, cmd_trace_buf,
2449		− cmd_len, s->buf_type == RING_BUFFER_INSTRUCTION,
2450		− cost_pre_cmd_handler, cost_cmd_handler);
2451		−}
2452		−
2453	2417	/* call the cmd handler, and advance ip */
2454	2418	static int cmd_parser_exec(struct parser_exec_state *s)
2455	2419	{
	2420	+ struct intel_vgpu *vgpu = s->vgpu;
2456	2421	struct cmd_info *info;
2457	2422	u32 cmd;
2458	2423	int ret = 0;
2459		− cycles_t t0, t1, t2;
2460		− struct parser_exec_state s_before_advance_custom;
2461		− struct intel_vgpu *vgpu = s->vgpu;
2462		−
2463		− t0 = get_cycles();
2464	2424
2465	2425	cmd = cmd_val(s, 0);
2466	2426
@@ -2471,13 +2431,10 @@ static int cmd_parser_exec(struct parser_exec_state *s) · 4 unmodified lines
2471	2431	return -EINVAL;
2472	2432	}
2473	2433
2474		− gvt_dbg_cmd("%s\n", info->name);
2475		−
2476	2434	s->info = info;
2477	2435
2478		− t1 = get_cycles();
2479		−
2480		− s_before_advance_custom = *s;
	2436	+ trace_gvt_command(vgpu->id, s->ring_id, s->ip_gma, s->ip_va,
	2437	+ cmd_length(s), s->buf_type);
2481	2438
2482	2439	if (info->handler) {
2483	2440	ret = info->handler(s);
@@ -2486,9 +2443,6 @@ static int cmd_parser_exec(struct parser_exec_state *s) · 2 unmodified lines
2486	2443	return ret;
2487	2444	}
2488	2445	}
2489		− t2 = get_cycles();
2490		−
2491		− trace_cs_command(&s_before_advance_custom, t1 - t0, t2 - t1);
2492	2446
2493	2447	if (!(info->flag & F_IP_ADVANCE_CUSTOM)) {
2494	2448	ret = cmd_advance_default(s);
@@ -2522,8 +2476,6 @@ static int command_scan(struct parser_exec_state *s, · 27 unmodified lines
2522	2476	gma_tail = rb_start + rb_tail;
2523	2477	gma_bottom = rb_start + rb_len;
2524	2478
2525		− gvt_dbg_cmd("scan_start: start=%lx end=%lx\n", gma_head, gma_tail);
2526		−
2527	2479	while (s->ip_gma != gma_tail) {
2528	2480	if (s->buf_type == RING_BUFFER_INSTRUCTION) {
2529	2481	if (!(s->ip_gma >= rb_start) \|\|
@@ -2552,8 +2504,6 @@ static int command_scan(struct parser_exec_state *s, · 22 unmodified lines
2552	2504	}
2553	2505	}
2554	2506
2555		− gvt_dbg_cmd("scan_end\n");
2556		−
2557	2507	return ret;
2558	2508	}
2559	2509

drivers/gpu/drm/i915/gvt/execlist.c+23−33

@@ -708,53 +708,43 @@ static int submit_context(struct intel_vgpu *vgpu, int ring_id, · 707 unmodified lines
708	708	int intel_vgpu_submit_execlist(struct intel_vgpu *vgpu, int ring_id)
709	709	{
710	710	struct intel_vgpu_execlist *execlist = &vgpu->execlist[ring_id];
711		− struct execlist_ctx_descriptor_format *desc[2], valid_desc[2];
712		− unsigned long valid_desc_bitmap = 0;
713		− bool emulate_schedule_in = true;
714		− int ret;
715		− int i;
	711	+ struct execlist_ctx_descriptor_format desc[2];
	712	+ int i, ret;
716	713
717		− memset(valid_desc, 0, sizeof(valid_desc));
	714	+ desc[0] = *get_desc_from_elsp_dwords(&execlist->elsp_dwords, 1);
	715	+ desc[1] = *get_desc_from_elsp_dwords(&execlist->elsp_dwords, 0);
718	716
719		− desc[0] = get_desc_from_elsp_dwords(&execlist->elsp_dwords, 1);
720		− desc[1] = get_desc_from_elsp_dwords(&execlist->elsp_dwords, 0);
	717	+ if (!desc[0].valid) {
	718	+ gvt_vgpu_err("invalid elsp submission, desc0 is invalid\n");
	719	+ goto inv_desc;
	720	+ }
721	721
722		− for (i = 0; i < 2; i++) {
723		− if (!desc[i]->valid)
	722	+ for (i = 0; i < ARRAY_SIZE(desc); i++) {
	723	+ if (!desc[i].valid)
724	724	continue;
725		−
726		− if (!desc[i]->privilege_access) {
	725	+ if (!desc[i].privilege_access) {
727	726	gvt_vgpu_err("unexpected GGTT elsp submission\n");
728		− return -EINVAL;
	727	+ goto inv_desc;
729	728	}
730		−
731		− /* TODO: add another guest context checks here. */
732		− set_bit(i, &valid_desc_bitmap);
733		− valid_desc[i] = *desc[i];
734		− }
735		−
736		− if (!valid_desc_bitmap) {
737		− gvt_vgpu_err("no valid desc in a elsp submission\n");
738		− return -EINVAL;
739		− }
740		−
741		− if (!test_bit(0, (void *)&valid_desc_bitmap) &&
742		− test_bit(1, (void *)&valid_desc_bitmap)) {
743		− gvt_vgpu_err("weird elsp submission, desc 0 is not valid\n");
744		− return -EINVAL;
745	729	}
746	730
747	731	/* submit workload */
748		− for_each_set_bit(i, (void *)&valid_desc_bitmap, 2) {
749		− ret = submit_context(vgpu, ring_id, &valid_desc[i],
750		− emulate_schedule_in);
	732	+ for (i = 0; i < ARRAY_SIZE(desc); i++) {
	733	+ if (!desc[i].valid)
	734	+ continue;
	735	+ ret = submit_context(vgpu, ring_id, &desc[i], i == 0);
751	736	if (ret) {
752		− gvt_vgpu_err("fail to schedule workload\n");
	737	+ gvt_vgpu_err("failed to submit desc %d\n", i);
753	738	return ret;
754	739	}
755		− emulate_schedule_in = false;
756	740	}
	741	+
757	742	return 0;
	743	+
	744	+inv_desc:
	745	+ gvt_vgpu_err("descriptors content: desc0 %08x %08x desc1 %08x %08x\n",
	746	+ desc[0].udw, desc[0].ldw, desc[1].udw, desc[1].ldw);
	747	+ return -EINVAL;
758	748	}
759	749
760	750	static void init_vgpu_execlist(struct intel_vgpu *vgpu, int ring_id)

drivers/gpu/drm/i915/gvt/firmware.c+2−7

@@ -102,13 +102,8 @@ static int expose_firmware_sysfs(struct intel_gvt *gvt) · 101 unmodified lines
102	102
103	103	p = firmware + h->mmio_offset;
104	104
105		− hash_for_each(gvt->mmio.mmio_info_table, i, e, node) {
106		− int j;
107		−
108		− for (j = 0; j < e->length; j += 4)
109		− (u32 )(p + e->offset + j) =
110		− I915_READ_NOTRACE(_MMIO(e->offset + j));
111		− }
	105	+ hash_for_each(gvt->mmio.mmio_info_table, i, e, node)
	106	+ (u32 )(p + e->offset) = I915_READ_NOTRACE(_MMIO(e->offset));
112	107
113	108	memcpy(gvt->firmware.mmio, p, info->mmio_size);
114	109

drivers/gpu/drm/i915/gvt/gtt.c+9−6

@@ -244,15 +244,19 @@ static u64 read_pte64(struct drm_i915_private *dev_priv, unsigned long index) · 243 unmodified lines
244	244	return readq(addr);
245	245	}
246	246
	247	+static void gtt_invalidate(struct drm_i915_private *dev_priv)
	248	+{
	249	+ mmio_hw_access_pre(dev_priv);
	250	+ I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
	251	+ mmio_hw_access_post(dev_priv);
	252	+}
	253	+
247	254	static void write_pte64(struct drm_i915_private *dev_priv,
248	255	unsigned long index, u64 pte)
249	256	{
250	257	void __iomem addr = (gen8_pte_t __iomem )dev_priv->ggtt.gsm + index;
251	258
252	259	writeq(pte, addr);
253		−
254		− I915_WRITE(GFX_FLSH_CNTL_GEN6, GFX_FLSH_CNTL_EN);
255		− POSTING_READ(GFX_FLSH_CNTL_GEN6);
256	260	}
257	261
258	262	static inline struct intel_gvt_gtt_entry gtt_get_entry64(void pt,
@@ -1849,6 +1853,7 @@ static int emulate_gtt_mmio_write(struct intel_vgpu *vgpu, unsigned int off, · 1590 unmodified lines
1849	1853	}
1850	1854
1851	1855	ggtt_set_shadow_entry(ggtt_mm, &m, g_gtt_index);
	1856	+ gtt_invalidate(gvt->dev_priv);
1852	1857	ggtt_set_guest_entry(ggtt_mm, &e, g_gtt_index);
1853	1858	return 0;
1854	1859	}
@@ -2301,8 +2306,6 @@ void intel_vgpu_reset_ggtt(struct intel_vgpu *vgpu) · 446 unmodified lines
2301	2306	u32 num_entries;
2302	2307	struct intel_gvt_gtt_entry e;
2303	2308
2304		− intel_runtime_pm_get(dev_priv);
2305		−
2306	2309	memset(&e, 0, sizeof(struct intel_gvt_gtt_entry));
2307	2310	e.type = GTT_TYPE_GGTT_PTE;
2308	2311	ops->set_pfn(&e, gvt->gtt.scratch_ggtt_mfn);
@@ -2318,7 +2321,7 @@ void intel_vgpu_reset_ggtt(struct intel_vgpu *vgpu) · 9 unmodified lines
2318	2321	for (offset = 0; offset < num_entries; offset++)
2319	2322	ops->set_entry(NULL, &e, index + offset, false, 0, vgpu);
2320	2323
2321		− intel_runtime_pm_put(dev_priv);
	2324	+ gtt_invalidate(dev_priv);
2322	2325	}
2323	2326
2324	2327	/**

drivers/gpu/drm/i915/gvt/gvt.c+4−2

@@ -147,7 +147,9 @@ static int gvt_service_thread(void *data) · 146 unmodified lines
147	147	mutex_unlock(&gvt->lock);
148	148	}
149	149
150		− if (test_and_clear_bit(INTEL_GVT_REQUEST_SCHED,
	150	+ if (test_bit(INTEL_GVT_REQUEST_SCHED,
	151	+ (void *)&gvt->service_request) \|\|
	152	+ test_bit(INTEL_GVT_REQUEST_EVENT_SCHED,
151	153	(void *)&gvt->service_request)) {
152	154	intel_gvt_schedule(gvt);
153	155	}
@@ -244,7 +246,7 @@ int intel_gvt_init_device(struct drm_i915_private *dev_priv) · 90 unmodified lines
244	246	gvt_dbg_core("init gvt device\n");
245	247
246	248	idr_init(&gvt->vgpu_idr);
247		−
	249	+ spin_lock_init(&gvt->scheduler.mmio_context_lock);
248	250	mutex_init(&gvt->lock);
249	251	gvt->dev_priv = dev_priv;
250	252

drivers/gpu/drm/i915/gvt/gvt.h+97−3

@@ -165,7 +165,6 @@ struct intel_vgpu { · 164 unmodified lines
165	165	struct list_head workload_q_head[I915_NUM_ENGINES];
166	166	struct kmem_cache *workloads;
167	167	atomic_t running_workload_num;
168		− ktime_t last_ctx_submit_time;
169	168	DECLARE_BITMAP(tlb_handle_pending, I915_NUM_ENGINES);
170	169	struct i915_gem_context *shadow_ctx;
171	170
@@ -196,11 +195,27 @@ struct intel_gvt_fence { · 24 unmodified lines
196	195	unsigned long vgpu_allocated_fence_num;
197	196	};
198	197
199		−#define INTEL_GVT_MMIO_HASH_BITS 9
	198	+#define INTEL_GVT_MMIO_HASH_BITS 11
200	199
201	200	struct intel_gvt_mmio {
202		− u32 *mmio_attribute;
	201	+ u8 *mmio_attribute;
	202	+/* Register contains RO bits */
	203	+#define F_RO (1 << 0)
	204	+/* Register contains graphics address */
	205	+#define F_GMADR (1 << 1)
	206	+/* Mode mask registers with high 16 bits as the mask bits */
	207	+#define F_MODE_MASK (1 << 2)
	208	+/* This reg can be accessed by GPU commands */
	209	+#define F_CMD_ACCESS (1 << 3)
	210	+/* This reg has been accessed by a VM */
	211	+#define F_ACCESSED (1 << 4)
	212	+/* This reg has been accessed through GPU commands */
	213	+#define F_CMD_ACCESSED (1 << 5)
	214	+/* This reg could be accessed by unaligned address */
	215	+#define F_UNALIGN (1 << 6)
	216	+
203	217	DECLARE_HASHTABLE(mmio_info_table, INTEL_GVT_MMIO_HASH_BITS);
	218	+ unsigned int num_tracked_mmio;
204	219	};
205	220
206	221	struct intel_gvt_firmware {
@@ -257,7 +272,12 @@ static inline struct intel_gvt to_gvt(struct drm_i915_private i915) · 50 unmodified lines
257	272
258	273	enum {
259	274	INTEL_GVT_REQUEST_EMULATE_VBLANK = 0,
	275	+
	276	+ /* Scheduling trigger by timer */
260	277	INTEL_GVT_REQUEST_SCHED = 1,
	278	+
	279	+ /* Scheduling trigger by event */
	280	+ INTEL_GVT_REQUEST_EVENT_SCHED = 2,
261	281	};
262	282
263	283	static inline void intel_gvt_request_service(struct intel_gvt *gvt,
@@ -473,6 +493,80 @@ enum { · 209 unmodified lines
473	493	GVT_FAILSAFE_INSUFFICIENT_RESOURCE,
474	494	};
475	495
	496	+static inline void mmio_hw_access_pre(struct drm_i915_private *dev_priv)
	497	+{
	498	+ intel_runtime_pm_get(dev_priv);
	499	+}
	500	+
	501	+static inline void mmio_hw_access_post(struct drm_i915_private *dev_priv)
	502	+{
	503	+ intel_runtime_pm_put(dev_priv);
	504	+}
	505	+
	506	+/**
	507	+ * intel_gvt_mmio_set_accessed - mark a MMIO has been accessed
	508	+ * @gvt: a GVT device
	509	+ * @offset: register offset
	510	+ *
	511	+ */
	512	+static inline void intel_gvt_mmio_set_accessed(
	513	+ struct intel_gvt *gvt, unsigned int offset)
	514	+{
	515	+ gvt->mmio.mmio_attribute[offset >> 2] \|= F_ACCESSED;
	516	+}
	517	+
	518	+/**
	519	+ * intel_gvt_mmio_is_cmd_accessed - mark a MMIO could be accessed by command
	520	+ * @gvt: a GVT device
	521	+ * @offset: register offset
	522	+ *
	523	+ */
	524	+static inline bool intel_gvt_mmio_is_cmd_access(
	525	+ struct intel_gvt *gvt, unsigned int offset)
	526	+{
	527	+ return gvt->mmio.mmio_attribute[offset >> 2] & F_CMD_ACCESS;
	528	+}
	529	+
	530	+/**
	531	+ * intel_gvt_mmio_is_unalign - mark a MMIO could be accessed unaligned
	532	+ * @gvt: a GVT device
	533	+ * @offset: register offset
	534	+ *
	535	+ */
	536	+static inline bool intel_gvt_mmio_is_unalign(
	537	+ struct intel_gvt *gvt, unsigned int offset)
	538	+{
	539	+ return gvt->mmio.mmio_attribute[offset >> 2] & F_UNALIGN;
	540	+}
	541	+
	542	+/**
	543	+ * intel_gvt_mmio_set_cmd_accessed - mark a MMIO has been accessed by command
	544	+ * @gvt: a GVT device
	545	+ * @offset: register offset
	546	+ *
	547	+ */
	548	+static inline void intel_gvt_mmio_set_cmd_accessed(
	549	+ struct intel_gvt *gvt, unsigned int offset)
	550	+{
	551	+ gvt->mmio.mmio_attribute[offset >> 2] \|= F_CMD_ACCESSED;
	552	+}
	553	+
	554	+/**
	555	+ * intel_gvt_mmio_has_mode_mask - if a MMIO has a mode mask
	556	+ * @gvt: a GVT device
	557	+ * @offset: register offset
	558	+ *
	559	+ * Returns:
	560	+ * True if a MMIO has a mode mask in its higher 16 bits, false if it isn't.
	561	+ *
	562	+ */
	563	+static inline bool intel_gvt_mmio_has_mode_mask(
	564	+ struct intel_gvt *gvt, unsigned int offset)
	565	+{
	566	+ return gvt->mmio.mmio_attribute[offset >> 2] & F_MODE_MASK;
	567	+}
	568	+
	569	+#include "trace.h"
476	570	#include "mpt.h"
477	571
478	572	#endif

drivers/gpu/drm/i915/gvt/handlers.c+177−126

@@ -47,21 +47,6 @@ · 46 unmodified lines
47	47	#define PCH_PP_OFF_DELAYS _MMIO(0xc720c)
48	48	#define PCH_PP_DIVISOR _MMIO(0xc7210)
49	49
50		−/* Register contains RO bits */
51		−#define F_RO (1 << 0)
52		−/* Register contains graphics address */
53		−#define F_GMADR (1 << 1)
54		−/* Mode mask registers with high 16 bits as the mask bits */
55		−#define F_MODE_MASK (1 << 2)
56		−/* This reg can be accessed by GPU commands */
57		−#define F_CMD_ACCESS (1 << 3)
58		−/* This reg has been accessed by a VM */
59		−#define F_ACCESSED (1 << 4)
60		−/* This reg has been accessed through GPU commands */
61		−#define F_CMD_ACCESSED (1 << 5)
62		−/* This reg could be accessed by unaligned address */
63		−#define F_UNALIGN (1 << 6)
64		−
65	50	unsigned long intel_gvt_get_device_type(struct intel_gvt *gvt)
66	51	{
67	52	if (IS_BROADWELL(gvt->dev_priv))
@@ -92,11 +77,22 @@ static void write_vreg(struct intel_vgpu *vgpu, unsigned int offset, · 24 unmodified lines
92	77	memcpy(&vgpu_vreg(vgpu, offset), p_data, bytes);
93	78	}
94	79
	80	+static struct intel_gvt_mmio_info find_mmio_info(struct intel_gvt gvt,
	81	+ unsigned int offset)
	82	+{
	83	+ struct intel_gvt_mmio_info *e;
	84	+
	85	+ hash_for_each_possible(gvt->mmio.mmio_info_table, e, node, offset) {
	86	+ if (e->offset == offset)
	87	+ return e;
	88	+ }
	89	+ return NULL;
	90	+}
	91	+
95	92	static int new_mmio_info(struct intel_gvt *gvt,
96		− u32 offset, u32 flags, u32 size,
	93	+ u32 offset, u8 flags, u32 size,
97	94	u32 addr_mask, u32 ro_mask, u32 device,
98		− int (read)(struct intel_vgpu , unsigned int, void *, unsigned int),
99		− int (write)(struct intel_vgpu , unsigned int, void *, unsigned int))
	95	+ gvt_mmio_func read, gvt_mmio_func write)
100	96	{
101	97	struct intel_gvt_mmio_info info, p;
102	98	u32 start, end, i;
@@ -116,13 +112,11 @@ static int new_mmio_info(struct intel_gvt *gvt, · 13 unmodified lines
116	112	return -ENOMEM;
117	113
118	114	info->offset = i;
119		− p = intel_gvt_find_mmio_info(gvt, info->offset);
	115	+ p = find_mmio_info(gvt, info->offset);
120	116	if (p)
121	117	gvt_err("dup mmio definition offset %x\n",
122	118	info->offset);
123		− info->size = size;
124		− info->length = (i + 4) < end ? 4 : (end - i);
125		− info->addr_mask = addr_mask;
	119	+
126	120	info->ro_mask = ro_mask;
127	121	info->device = device;
128	122	info->read = read ? read : intel_vgpu_default_mmio_read;
@@ -130,6 +124,7 @@ static int new_mmio_info(struct intel_gvt *gvt, · 1 unmodified line
130	124	gvt->mmio.mmio_attribute[info->offset / 4] = flags;
131	125	INIT_HLIST_NODE(&info->node);
132	126	hash_add(gvt->mmio.mmio_info_table, &info->node, info->offset);
	127	+ gvt->mmio.num_tracked_mmio++;
133	128	}
134	129	return 0;
135	130	}
@@ -209,6 +204,7 @@ static int fence_mmio_read(struct intel_vgpu *vgpu, unsigned int off, · 73 unmodified lines
209	204	static int fence_mmio_write(struct intel_vgpu *vgpu, unsigned int off,
210	205	void *p_data, unsigned int bytes)
211	206	{
	207	+ struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
212	208	unsigned int fence_num = offset_to_fence_num(off);
213	209	int ret;
214	210
@@ -217,8 +213,10 @@ static int fence_mmio_write(struct intel_vgpu *vgpu, unsigned int off, · 2 unmodified lines
217	213	return ret;
218	214	write_vreg(vgpu, off, p_data, bytes);
219	215
	216	+ mmio_hw_access_pre(dev_priv);
220	217	intel_vgpu_write_fence(vgpu, fence_num,
221	218	vgpu_vreg64(vgpu, fence_num_to_offset(fence_num)));
	219	+ mmio_hw_access_post(dev_priv);
222	220	return 0;
223	221	}
224	222
@@ -300,6 +298,9 @@ static int gdrst_mmio_write(struct intel_vgpu *vgpu, unsigned int offset, · 75 unmodified lines
300	298
301	299	intel_gvt_reset_vgpu_locked(vgpu, false, engine_mask);
302	300
	301	+ /* sw will wait for the device to ack the reset request */
	302	+ vgpu_vreg(vgpu, offset) = 0;
	303	+
303	304	return 0;
304	305	}
305	306
@@ -1265,7 +1266,10 @@ static int gen9_trtte_write(struct intel_vgpu *vgpu, unsigned int offset, · 959 unmodified lines
1265	1266	}
1266	1267	write_vreg(vgpu, offset, p_data, bytes);
1267	1268	/* TRTTE is not per-context */
	1269	+
	1270	+ mmio_hw_access_pre(dev_priv);
1268	1271	I915_WRITE(_MMIO(offset), vgpu_vreg(vgpu, offset));
	1272	+ mmio_hw_access_post(dev_priv);
1269	1273
1270	1274	return 0;
1271	1275	}
@@ -1278,7 +1282,9 @@ static int gen9_trtt_chicken_write(struct intel_vgpu *vgpu, unsigned int offset, · 6 unmodified lines
1278	1282
1279	1283	if (val & 1) {
1280	1284	/* unblock hw logic */
	1285	+ mmio_hw_access_pre(dev_priv);
1281	1286	I915_WRITE(_MMIO(offset), val);
	1287	+ mmio_hw_access_post(dev_priv);
1282	1288	}
1283	1289	write_vreg(vgpu, offset, p_data, bytes);
1284	1290	return 0;
@@ -1415,7 +1421,20 @@ static int ring_timestamp_mmio_read(struct intel_vgpu *vgpu, · 130 unmodified lines
1415	1421	{
1416	1422	struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
1417	1423
	1424	+ mmio_hw_access_pre(dev_priv);
1418	1425	vgpu_vreg(vgpu, offset) = I915_READ(_MMIO(offset));
	1426	+ mmio_hw_access_post(dev_priv);
	1427	+ return intel_vgpu_default_mmio_read(vgpu, offset, p_data, bytes);
	1428	+}
	1429	+
	1430	+static int instdone_mmio_read(struct intel_vgpu *vgpu,
	1431	+ unsigned int offset, void *p_data, unsigned int bytes)
	1432	+{
	1433	+ struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
	1434	+
	1435	+ mmio_hw_access_pre(dev_priv);
	1436	+ vgpu_vreg(vgpu, offset) = I915_READ(_MMIO(offset));
	1437	+ mmio_hw_access_post(dev_priv);
1419	1438	return intel_vgpu_default_mmio_read(vgpu, offset, p_data, bytes);
1420	1439	}
1421	1440
@@ -1434,7 +1453,6 @@ static int elsp_mmio_write(struct intel_vgpu *vgpu, unsigned int offset, · 12 unmodified lines
1434	1453
1435	1454	execlist->elsp_dwords.data[execlist->elsp_dwords.index] = data;
1436	1455	if (execlist->elsp_dwords.index == 3) {
1437		− vgpu->last_ctx_submit_time = ktime_get();
1438	1456	ret = intel_vgpu_submit_execlist(vgpu, ring_id);
1439	1457	if(ret)
1440	1458	gvt_vgpu_err("fail submit workload on ring %d\n",
@@ -1603,6 +1621,12 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 162 unmodified lines
1603	1621	MMIO_RING_DFH(RING_REG, D_ALL, F_CMD_ACCESS, NULL, NULL);
1604	1622	#undef RING_REG
1605	1623
	1624	+#define RING_REG(base) (base + 0x6c)
	1625	+ MMIO_RING_DFH(RING_REG, D_ALL, 0, instdone_mmio_read, NULL);
	1626	+ MMIO_DH(RING_REG(GEN8_BSD2_RING_BASE), D_ALL, instdone_mmio_read, NULL);
	1627	+#undef RING_REG
	1628	+ MMIO_DH(GEN7_SC_INSTDONE, D_BDW_PLUS, instdone_mmio_read, NULL);
	1629	+
1606	1630	MMIO_GM_RDR(0x2148, D_ALL, NULL, NULL);
1607	1631	MMIO_GM_RDR(CCID, D_ALL, NULL, NULL);
1608	1632	MMIO_GM_RDR(0x12198, D_ALL, NULL, NULL);
@@ -1779,10 +1803,6 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 170 unmodified lines
1779	1803	MMIO_D(SPRSCALE(PIPE_C), D_ALL);
1780	1804	MMIO_D(SPRSURFLIVE(PIPE_C), D_ALL);
1781	1805
1782		− MMIO_F(LGC_PALETTE(PIPE_A, 0), 4 * 256, 0, 0, 0, D_ALL, NULL, NULL);
1783		− MMIO_F(LGC_PALETTE(PIPE_B, 0), 4 * 256, 0, 0, 0, D_ALL, NULL, NULL);
1784		− MMIO_F(LGC_PALETTE(PIPE_C, 0), 4 * 256, 0, 0, 0, D_ALL, NULL, NULL);
1785		−
1786	1806	MMIO_D(HTOTAL(TRANSCODER_A), D_ALL);
1787	1807	MMIO_D(HBLANK(TRANSCODER_A), D_ALL);
1788	1808	MMIO_D(HSYNC(TRANSCODER_A), D_ALL);
@@ -2187,7 +2207,7 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 398 unmodified lines
2187	2207	MMIO_DFH(GTFIFODBG, D_ALL, F_CMD_ACCESS, NULL, NULL);
2188	2208	MMIO_DFH(GTFIFOCTL, D_ALL, F_CMD_ACCESS, NULL, NULL);
2189	2209	MMIO_DH(FORCEWAKE_MT, D_PRE_SKL, NULL, mul_force_wake_write);
2190		− MMIO_DH(FORCEWAKE_ACK_HSW, D_HSW \| D_BDW, NULL, NULL);
	2210	+ MMIO_DH(FORCEWAKE_ACK_HSW, D_BDW, NULL, NULL);
2191	2211	MMIO_D(ECOBUS, D_ALL);
2192	2212	MMIO_DH(GEN6_RC_CONTROL, D_ALL, NULL, NULL);
2193	2213	MMIO_DH(GEN6_RC_STATE, D_ALL, NULL, NULL);
@@ -2219,22 +2239,19 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 25 unmodified lines
2219	2239	MMIO_D(GEN6_RC6p_THRESHOLD, D_ALL);
2220	2240	MMIO_D(GEN6_RC6pp_THRESHOLD, D_ALL);
2221	2241	MMIO_D(GEN6_PMINTRMSK, D_ALL);
2222		− MMIO_DH(HSW_PWR_WELL_BIOS, D_HSW \| D_BDW, NULL, power_well_ctl_mmio_write);
2223		− MMIO_DH(HSW_PWR_WELL_DRIVER, D_HSW \| D_BDW, NULL, power_well_ctl_mmio_write);
2224		− MMIO_DH(HSW_PWR_WELL_KVMR, D_HSW \| D_BDW, NULL, power_well_ctl_mmio_write);
2225		− MMIO_DH(HSW_PWR_WELL_DEBUG, D_HSW \| D_BDW, NULL, power_well_ctl_mmio_write);
2226		− MMIO_DH(HSW_PWR_WELL_CTL5, D_HSW \| D_BDW, NULL, power_well_ctl_mmio_write);
2227		− MMIO_DH(HSW_PWR_WELL_CTL6, D_HSW \| D_BDW, NULL, power_well_ctl_mmio_write);
	2242	+ MMIO_DH(HSW_PWR_WELL_BIOS, D_BDW, NULL, power_well_ctl_mmio_write);
	2243	+ MMIO_DH(HSW_PWR_WELL_DRIVER, D_BDW, NULL, power_well_ctl_mmio_write);
	2244	+ MMIO_DH(HSW_PWR_WELL_KVMR, D_BDW, NULL, power_well_ctl_mmio_write);
	2245	+ MMIO_DH(HSW_PWR_WELL_DEBUG, D_BDW, NULL, power_well_ctl_mmio_write);
	2246	+ MMIO_DH(HSW_PWR_WELL_CTL5, D_BDW, NULL, power_well_ctl_mmio_write);
	2247	+ MMIO_DH(HSW_PWR_WELL_CTL6, D_BDW, NULL, power_well_ctl_mmio_write);
2228	2248
2229	2249	MMIO_D(RSTDBYCTL, D_ALL);
2230	2250
2231	2251	MMIO_DH(GEN6_GDRST, D_ALL, NULL, gdrst_mmio_write);
2232	2252	MMIO_F(FENCE_REG_GEN6_LO(0), 0x80, 0, 0, 0, D_ALL, fence_mmio_read, fence_mmio_write);
2233		− MMIO_F(VGT_PVINFO_PAGE, VGT_PVINFO_SIZE, F_UNALIGN, 0, 0, D_ALL, pvinfo_mmio_read, pvinfo_mmio_write);
2234	2253	MMIO_DH(CPU_VGACNTRL, D_ALL, NULL, vga_control_mmio_write);
2235	2254
2236		− MMIO_F(MCHBAR_MIRROR_BASE_SNB, 0x40000, 0, 0, 0, D_ALL, NULL, NULL);
2237		−
2238	2255	MMIO_D(TILECTL, D_ALL);
2239	2256
2240	2257	MMIO_D(GEN6_UCGCTL1, D_ALL);
@@ -2242,7 +2259,6 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 1 unmodified line
2242	2259
2243	2260	MMIO_F(0x4f000, 0x90, 0, 0, 0, D_ALL, NULL, NULL);
2244	2261
2245		− MMIO_D(GEN6_PCODE_MAILBOX, D_PRE_BDW);
2246	2262	MMIO_D(GEN6_PCODE_DATA, D_ALL);
2247	2263	MMIO_D(0x13812c, D_ALL);
2248	2264	MMIO_DH(GEN7_ERR_INT, D_ALL, NULL, NULL);
@@ -2321,14 +2337,13 @@ static int init_generic_mmio_info(struct intel_gvt *gvt) · 72 unmodified lines
2321	2337	MMIO_D(0x1a054, D_ALL);
2322	2338
2323	2339	MMIO_D(0x44070, D_ALL);
2324		− MMIO_DFH(0x215c, D_HSW_PLUS, F_CMD_ACCESS, NULL, NULL);
	2340	+ MMIO_DFH(0x215c, D_BDW_PLUS, F_CMD_ACCESS, NULL, NULL);
2325	2341	MMIO_DFH(0x2178, D_ALL, F_CMD_ACCESS, NULL, NULL);
2326	2342	MMIO_DFH(0x217c, D_ALL, F_CMD_ACCESS, NULL, NULL);
2327	2343	MMIO_DFH(0x12178, D_ALL, F_CMD_ACCESS, NULL, NULL);
2328	2344	MMIO_DFH(0x1217c, D_ALL, F_CMD_ACCESS, NULL, NULL);
2329	2345
2330		− MMIO_F(0x2290, 8, F_CMD_ACCESS, 0, 0, D_HSW_PLUS, NULL, NULL);
2331		− MMIO_DFH(GEN7_OACONTROL, D_HSW, F_CMD_ACCESS, NULL, NULL);
	2346	+ MMIO_F(0x2290, 8, F_CMD_ACCESS, 0, 0, D_BDW_PLUS, NULL, NULL);
2332	2347	MMIO_D(0x2b00, D_BDW_PLUS);
2333	2348	MMIO_D(0x2360, D_BDW_PLUS);
2334	2349	MMIO_F(0x5200, 32, F_CMD_ACCESS, 0, 0, D_ALL, NULL, NULL);
@@ -2766,7 +2781,6 @@ static int init_skl_mmio_info(struct intel_gvt *gvt) · 431 unmodified lines
2766	2781	MMIO_D(0x72380, D_SKL_PLUS);
2767	2782	MMIO_D(0x7039c, D_SKL_PLUS);
2768	2783
2769		− MMIO_F(0x80000, 0x3000, 0, 0, 0, D_SKL_PLUS, NULL, NULL);
2770	2784	MMIO_D(0x8f074, D_SKL \| D_KBL);
2771	2785	MMIO_D(0x8f004, D_SKL \| D_KBL);
2772	2786	MMIO_D(0x8f034, D_SKL \| D_KBL);
@@ -2840,26 +2854,36 @@ static int init_skl_mmio_info(struct intel_gvt *gvt) · 67 unmodified lines
2840	2854	return 0;
2841	2855	}
2842	2856
2843		−/**
2844		− * intel_gvt_find_mmio_info - find MMIO information entry by aligned offset
2845		− * @gvt: GVT device
2846		− * @offset: register offset
2847		− *
2848		− * This function is used to find the MMIO information entry from hash table
2849		− *
2850		− * Returns:
2851		− * pointer to MMIO information entry, NULL if not exists
2852		− */
2853		−struct intel_gvt_mmio_info intel_gvt_find_mmio_info(struct intel_gvt gvt,
2854		− unsigned int offset)
2855		−{
2856		− struct intel_gvt_mmio_info *e;
	2857	+/* Special MMIO blocks. */
	2858	+static struct gvt_mmio_block {
	2859	+ unsigned int device;
	2860	+ i915_reg_t offset;
	2861	+ unsigned int size;
	2862	+ gvt_mmio_func read;
	2863	+ gvt_mmio_func write;
	2864	+} gvt_mmio_blocks[] = {
	2865	+ {D_SKL_PLUS, _MMIO(CSR_MMIO_START_RANGE), 0x3000, NULL, NULL},
	2866	+ {D_ALL, _MMIO(MCHBAR_MIRROR_BASE_SNB), 0x40000, NULL, NULL},
	2867	+ {D_ALL, _MMIO(VGT_PVINFO_PAGE), VGT_PVINFO_SIZE,
	2868	+ pvinfo_mmio_read, pvinfo_mmio_write},
	2869	+ {D_ALL, LGC_PALETTE(PIPE_A, 0), 1024, NULL, NULL},
	2870	+ {D_ALL, LGC_PALETTE(PIPE_B, 0), 1024, NULL, NULL},
	2871	+ {D_ALL, LGC_PALETTE(PIPE_C, 0), 1024, NULL, NULL},
	2872	+};
2857	2873
2858		− WARN_ON(!IS_ALIGNED(offset, 4));
	2874	+static struct gvt_mmio_block find_mmio_block(struct intel_gvt gvt,
	2875	+ unsigned int offset)
	2876	+{
	2877	+ unsigned long device = intel_gvt_get_device_type(gvt);
	2878	+ struct gvt_mmio_block *block = gvt_mmio_blocks;
	2879	+ int i;
2859	2880
2860		− hash_for_each_possible(gvt->mmio.mmio_info_table, e, node, offset) {
2861		− if (e->offset == offset)
2862		− return e;
	2881	+ for (i = 0; i < ARRAY_SIZE(gvt_mmio_blocks); i++, block++) {
	2882	+ if (!(device & block->device))
	2883	+ continue;
	2884	+ if (offset >= INTEL_GVT_MMIO_OFFSET(block->offset) &&
	2885	+ offset < INTEL_GVT_MMIO_OFFSET(block->offset) + block->size)
	2886	+ return block;
2863	2887	}
2864	2888	return NULL;
2865	2889	}
@@ -2899,9 +2923,10 @@ int intel_gvt_setup_mmio_info(struct intel_gvt *gvt) · 33 unmodified lines
2899	2923	{
2900	2924	struct intel_gvt_device_info *info = &gvt->device_info;
2901	2925	struct drm_i915_private *dev_priv = gvt->dev_priv;
	2926	+ int size = info->mmio_size / 4 * sizeof(*gvt->mmio.mmio_attribute);
2902	2927	int ret;
2903	2928
2904		− gvt->mmio.mmio_attribute = vzalloc(info->mmio_size);
	2929	+ gvt->mmio.mmio_attribute = vzalloc(size);
2905	2930	if (!gvt->mmio.mmio_attribute)
2906	2931	return -ENOMEM;
2907	2932
@@ -2922,77 +2947,15 @@ int intel_gvt_setup_mmio_info(struct intel_gvt *gvt) · 14 unmodified lines
2922	2947	if (ret)
2923	2948	goto err;
2924	2949	}
	2950	+
	2951	+ gvt_dbg_mmio("traced %u virtual mmio registers\n",
	2952	+ gvt->mmio.num_tracked_mmio);
2925	2953	return 0;
2926	2954	err:
2927	2955	intel_gvt_clean_mmio_info(gvt);
2928	2956	return ret;
2929	2957	}
2930	2958
2931		−/**
2932		− * intel_gvt_mmio_set_accessed - mark a MMIO has been accessed
2933		− * @gvt: a GVT device
2934		− * @offset: register offset
2935		− *
2936		− */
2937		−void intel_gvt_mmio_set_accessed(struct intel_gvt *gvt, unsigned int offset)
2938		−{
2939		− gvt->mmio.mmio_attribute[offset >> 2] \|=
2940		− F_ACCESSED;
2941		−}
2942		−
2943		−/**
2944		− * intel_gvt_mmio_is_cmd_accessed - mark a MMIO could be accessed by command
2945		− * @gvt: a GVT device
2946		− * @offset: register offset
2947		− *
2948		− */
2949		−bool intel_gvt_mmio_is_cmd_access(struct intel_gvt *gvt,
2950		− unsigned int offset)
2951		−{
2952		− return gvt->mmio.mmio_attribute[offset >> 2] &
2953		− F_CMD_ACCESS;
2954		−}
2955		−
2956		−/**
2957		− * intel_gvt_mmio_is_unalign - mark a MMIO could be accessed unaligned
2958		− * @gvt: a GVT device
2959		− * @offset: register offset
2960		− *
2961		− */
2962		−bool intel_gvt_mmio_is_unalign(struct intel_gvt *gvt,
2963		− unsigned int offset)
2964		−{
2965		− return gvt->mmio.mmio_attribute[offset >> 2] &
2966		− F_UNALIGN;
2967		−}
2968		−
2969		−/**
2970		− * intel_gvt_mmio_set_cmd_accessed - mark a MMIO has been accessed by command
2971		− * @gvt: a GVT device
2972		− * @offset: register offset
2973		− *
2974		− */
2975		−void intel_gvt_mmio_set_cmd_accessed(struct intel_gvt *gvt,
2976		− unsigned int offset)
2977		−{
2978		− gvt->mmio.mmio_attribute[offset >> 2] \|=
2979		− F_CMD_ACCESSED;
2980		−}
2981		−
2982		−/**
2983		− * intel_gvt_mmio_has_mode_mask - if a MMIO has a mode mask
2984		− * @gvt: a GVT device
2985		− * @offset: register offset
2986		− *
2987		− * Returns:
2988		− * True if a MMIO has a mode mask in its higher 16 bits, false if it isn't.
2989		− *
2990		− */
2991		−bool intel_gvt_mmio_has_mode_mask(struct intel_gvt *gvt, unsigned int offset)
2992		−{
2993		− return gvt->mmio.mmio_attribute[offset >> 2] &
2994		− F_MODE_MASK;
2995		−}
2996	2959
2997	2960	/**
2998	2961	* intel_vgpu_default_mmio_read - default MMIO read handler
@@ -3044,3 +3007,91 @@ bool intel_gvt_in_force_nonpriv_whitelist(struct intel_gvt *gvt, · 45 unmodified lines
3044	3007	{
3045	3008	return in_whitelist(offset);
3046	3009	}
	3010	+
	3011	+/**
	3012	+ * intel_vgpu_mmio_reg_rw - emulate tracked mmio registers
	3013	+ * @vgpu: a vGPU
	3014	+ * @offset: register offset
	3015	+ * @pdata: data buffer
	3016	+ * @bytes: data length
	3017	+ *
	3018	+ * Returns:
	3019	+ * Zero on success, negative error code if failed.
	3020	+ */
	3021	+int intel_vgpu_mmio_reg_rw(struct intel_vgpu *vgpu, unsigned int offset,
	3022	+ void *pdata, unsigned int bytes, bool is_read)
	3023	+{
	3024	+ struct intel_gvt *gvt = vgpu->gvt;
	3025	+ struct intel_gvt_mmio_info *mmio_info;
	3026	+ struct gvt_mmio_block *mmio_block;
	3027	+ gvt_mmio_func func;
	3028	+ int ret;
	3029	+
	3030	+ if (WARN_ON(bytes > 4))
	3031	+ return -EINVAL;
	3032	+
	3033	+ /*
	3034	+ * Handle special MMIO blocks.
	3035	+ */
	3036	+ mmio_block = find_mmio_block(gvt, offset);
	3037	+ if (mmio_block) {
	3038	+ func = is_read ? mmio_block->read : mmio_block->write;
	3039	+ if (func)
	3040	+ return func(vgpu, offset, pdata, bytes);
	3041	+ goto default_rw;
	3042	+ }
	3043	+
	3044	+ /*
	3045	+ * Normal tracked MMIOs.
	3046	+ */
	3047	+ mmio_info = find_mmio_info(gvt, offset);
	3048	+ if (!mmio_info) {
	3049	+ if (!vgpu->mmio.disable_warn_untrack)
	3050	+ gvt_vgpu_err("untracked MMIO %08x len %d\n",
	3051	+ offset, bytes);
	3052	+ goto default_rw;
	3053	+ }
	3054	+
	3055	+ if (is_read)
	3056	+ return mmio_info->read(vgpu, offset, pdata, bytes);
	3057	+ else {
	3058	+ u64 ro_mask = mmio_info->ro_mask;
	3059	+ u32 old_vreg = 0, old_sreg = 0;
	3060	+ u64 data = 0;
	3061	+
	3062	+ if (intel_gvt_mmio_has_mode_mask(gvt, mmio_info->offset)) {
	3063	+ old_vreg = vgpu_vreg(vgpu, offset);
	3064	+ old_sreg = vgpu_sreg(vgpu, offset);
	3065	+ }
	3066	+
	3067	+ if (likely(!ro_mask))
	3068	+ ret = mmio_info->write(vgpu, offset, pdata, bytes);
	3069	+ else if (!~ro_mask) {
	3070	+ gvt_vgpu_err("try to write RO reg %x\n", offset);
	3071	+ return 0;
	3072	+ } else {
	3073	+ /* keep the RO bits in the virtual register */
	3074	+ memcpy(&data, pdata, bytes);
	3075	+ data &= ~ro_mask;
	3076	+ data \|= vgpu_vreg(vgpu, offset) & ro_mask;
	3077	+ ret = mmio_info->write(vgpu, offset, &data, bytes);
	3078	+ }
	3079	+
	3080	+ /* higher 16bits of mode ctl regs are mask bits for change */
	3081	+ if (intel_gvt_mmio_has_mode_mask(gvt, mmio_info->offset)) {
	3082	+ u32 mask = vgpu_vreg(vgpu, offset) >> 16;
	3083	+
	3084	+ vgpu_vreg(vgpu, offset) = (old_vreg & ~mask)
	3085	+ \| (vgpu_vreg(vgpu, offset) & mask);
	3086	+ vgpu_sreg(vgpu, offset) = (old_sreg & ~mask)
	3087	+ \| (vgpu_sreg(vgpu, offset) & mask);
	3088	+ }
	3089	+ }
	3090	+
	3091	+ return ret;
	3092	+
	3093	+default_rw:
	3094	+ return is_read ?
	3095	+ intel_vgpu_default_mmio_read(vgpu, offset, pdata, bytes) :
	3096	+ intel_vgpu_default_mmio_write(vgpu, offset, pdata, bytes);
	3097	+}

drivers/gpu/drm/i915/gvt/interrupt.c+10−10

@@ -31,6 +31,7 @@ · 30 unmodified lines
31	31
32	32	#include "i915_drv.h"
33	33	#include "gvt.h"
	34	+#include "trace.h"
34	35
35	36	/* common offset among interrupt control registers */
36	37	#define regbase_to_isr(base) (base)
@@ -178,8 +179,8 @@ int intel_vgpu_reg_imr_handler(struct intel_vgpu *vgpu, · 141 unmodified lines
178	179	struct intel_gvt_irq_ops *ops = gvt->irq.ops;
179	180	u32 imr = (u32 )p_data;
180	181
181		− gvt_dbg_irq("write IMR %x, new %08x, old %08x, changed %08x\n",
182		− reg, imr, vgpu_vreg(vgpu, reg), vgpu_vreg(vgpu, reg) ^ imr);
	182	+ trace_write_ir(vgpu->id, "IMR", reg, imr, vgpu_vreg(vgpu, reg),
	183	+ (vgpu_vreg(vgpu, reg) ^ imr));
183	184
184	185	vgpu_vreg(vgpu, reg) = imr;
185	186
@@ -209,8 +210,8 @@ int intel_vgpu_reg_master_irq_handler(struct intel_vgpu *vgpu, · 23 unmodified lines
209	210	u32 ier = (u32 )p_data;
210	211	u32 virtual_ier = vgpu_vreg(vgpu, reg);
211	212
212		− gvt_dbg_irq("write MASTER_IRQ %x, new %08x, old %08x, changed %08x\n",
213		− reg, ier, virtual_ier, virtual_ier ^ ier);
	213	+ trace_write_ir(vgpu->id, "MASTER_IRQ", reg, ier, virtual_ier,
	214	+ (virtual_ier ^ ier));
214	215
215	216	/*
216	217	* GEN8_MASTER_IRQ is a special irq register,
@@ -248,8 +249,8 @@ int intel_vgpu_reg_ier_handler(struct intel_vgpu *vgpu, · 31 unmodified lines
248	249	struct intel_gvt_irq_info *info;
249	250	u32 ier = (u32 )p_data;
250	251
251		− gvt_dbg_irq("write IER %x, new %08x, old %08x, changed %08x\n",
252		− reg, ier, vgpu_vreg(vgpu, reg), vgpu_vreg(vgpu, reg) ^ ier);
	252	+ trace_write_ir(vgpu->id, "IER", reg, ier, vgpu_vreg(vgpu, reg),
	253	+ (vgpu_vreg(vgpu, reg) ^ ier));
253	254
254	255	vgpu_vreg(vgpu, reg) = ier;
255	256
@@ -285,8 +286,8 @@ int intel_vgpu_reg_iir_handler(struct intel_vgpu *vgpu, unsigned int reg, · 29 unmodified lines
285	286	iir_to_regbase(reg));
286	287	u32 iir = (u32 )p_data;
287	288
288		− gvt_dbg_irq("write IIR %x, new %08x, old %08x, changed %08x\n",
289		− reg, iir, vgpu_vreg(vgpu, reg), vgpu_vreg(vgpu, reg) ^ iir);
	289	+ trace_write_ir(vgpu->id, "IIR", reg, iir, vgpu_vreg(vgpu, reg),
	290	+ (vgpu_vreg(vgpu, reg) ^ iir));
290	291
291	292	if (WARN_ON(!info))
292	293	return -EINVAL;
@@ -411,8 +412,7 @@ static void propagate_event(struct intel_gvt_irq *irq, · 118 unmodified lines
411	412
412	413	if (!test_bit(bit, (void *)&vgpu_vreg(vgpu,
413	414	regbase_to_imr(reg_base)))) {
414		− gvt_dbg_irq("set bit (%d) for (%s) for vgpu (%d)\n",
415		− bit, irq_name[event], vgpu->id);
	415	+ trace_propagate_event(vgpu->id, irq_name[event], bit);
416	416	set_bit(bit, (void *)&vgpu_vreg(vgpu,
417	417	regbase_to_iir(reg_base)));
418	418	}

drivers/gpu/drm/i915/gvt/mmio.c+25−96

@@ -123,7 +123,6 @@ int intel_vgpu_emulate_mmio_read(struct intel_vgpu *vgpu, uint64_t pa, · 122 unmodified lines
123	123	void *p_data, unsigned int bytes)
124	124	{
125	125	struct intel_gvt *gvt = vgpu->gvt;
126		− struct intel_gvt_mmio_info *mmio;
127	126	unsigned int offset = 0;
128	127	int ret = -EINVAL;
129	128
@@ -187,32 +186,8 @@ int intel_vgpu_emulate_mmio_read(struct intel_vgpu *vgpu, uint64_t pa, · 57 unmodified lines
187	186	goto err;
188	187	}
189	188
190		− mmio = intel_gvt_find_mmio_info(gvt, rounddown(offset, 4));
191		− if (mmio) {
192		− if (!intel_gvt_mmio_is_unalign(gvt, mmio->offset)) {
193		− if (WARN_ON(offset + bytes > mmio->offset + mmio->size))
194		− goto err;
195		− if (WARN_ON(mmio->offset != offset))
196		− goto err;
197		− }
198		− ret = mmio->read(vgpu, offset, p_data, bytes);
199		− } else {
200		− ret = intel_vgpu_default_mmio_read(vgpu, offset, p_data, bytes);
201		−
202		− if (!vgpu->mmio.disable_warn_untrack) {
203		− gvt_vgpu_err("read untracked MMIO %x(%dB) val %x\n",
204		− offset, bytes, (u32 )p_data);
205		−
206		− if (offset == 0x206c) {
207		− gvt_vgpu_err("------------------------------------------\n");
208		− gvt_vgpu_err("likely triggers a gfx reset\n");
209		− gvt_vgpu_err("------------------------------------------\n");
210		− vgpu->mmio.disable_warn_untrack = true;
211		− }
212		− }
213		− }
214		−
215		− if (ret)
	189	+ ret = intel_vgpu_mmio_reg_rw(vgpu, offset, p_data, bytes, true);
	190	+ if (ret < 0)
216	191	goto err;
217	192
218	193	intel_gvt_mmio_set_accessed(gvt, offset);
@@ -239,9 +214,7 @@ int intel_vgpu_emulate_mmio_write(struct intel_vgpu *vgpu, uint64_t pa, · 20 unmodified lines
239	214	void *p_data, unsigned int bytes)
240	215	{
241	216	struct intel_gvt *gvt = vgpu->gvt;
242		− struct intel_gvt_mmio_info *mmio;
243	217	unsigned int offset = 0;
244		− u32 old_vreg = 0, old_sreg = 0;
245	218	int ret = -EINVAL;
246	219
247	220	if (vgpu->failsafe) {
@@ -296,66 +269,10 @@ int intel_vgpu_emulate_mmio_write(struct intel_vgpu *vgpu, uint64_t pa, · 48 unmodified lines
296	269	return ret;
297	270	}
298	271
299		− mmio = intel_gvt_find_mmio_info(gvt, rounddown(offset, 4));
300		− if (!mmio && !vgpu->mmio.disable_warn_untrack)
301		− gvt_dbg_mmio("vgpu%d: write untracked MMIO %x len %d val %x\n",
302		− vgpu->id, offset, bytes, (u32 )p_data);
303		−
304		− if (!intel_gvt_mmio_is_unalign(gvt, offset)) {
305		− if (WARN_ON(!IS_ALIGNED(offset, bytes)))
306		− goto err;
307		− }
308		−
309		− if (mmio) {
310		− u64 ro_mask = mmio->ro_mask;
311		−
312		− if (!intel_gvt_mmio_is_unalign(gvt, mmio->offset)) {
313		− if (WARN_ON(offset + bytes > mmio->offset + mmio->size))
314		− goto err;
315		− if (WARN_ON(mmio->offset != offset))
316		− goto err;
317		− }
318		−
319		− if (intel_gvt_mmio_has_mode_mask(gvt, mmio->offset)) {
320		− old_vreg = vgpu_vreg(vgpu, offset);
321		− old_sreg = vgpu_sreg(vgpu, offset);
322		− }
323		−
324		− if (!ro_mask) {
325		− ret = mmio->write(vgpu, offset, p_data, bytes);
326		− } else {
327		− /* Protect RO bits like HW */
328		− u64 data = 0;
329		−
330		− /* all register bits are RO. */
331		− if (ro_mask == ~(u64)0) {
332		− gvt_vgpu_err("try to write RO reg %x\n",
333		− offset);
334		− ret = 0;
335		− goto out;
336		− }
337		− /* keep the RO bits in the virtual register */
338		− memcpy(&data, p_data, bytes);
339		− data &= ~mmio->ro_mask;
340		− data \|= vgpu_vreg(vgpu, offset) & mmio->ro_mask;
341		− ret = mmio->write(vgpu, offset, &data, bytes);
342		− }
343		−
344		− /* higher 16bits of mode ctl regs are mask bits for change */
345		− if (intel_gvt_mmio_has_mode_mask(gvt, mmio->offset)) {
346		− u32 mask = vgpu_vreg(vgpu, offset) >> 16;
347		−
348		− vgpu_vreg(vgpu, offset) = (old_vreg & ~mask)
349		− \| (vgpu_vreg(vgpu, offset) & mask);
350		− vgpu_sreg(vgpu, offset) = (old_sreg & ~mask)
351		− \| (vgpu_sreg(vgpu, offset) & mask);
352		− }
353		− } else
354		− ret = intel_vgpu_default_mmio_write(vgpu, offset, p_data,
355		− bytes);
356		− if (ret)
	272	+ ret = intel_vgpu_mmio_reg_rw(vgpu, offset, p_data, bytes, false);
	273	+ if (ret < 0)
357	274	goto err;
358		−out:
	275	+
359	276	intel_gvt_mmio_set_accessed(gvt, offset);
360	277	mutex_unlock(&gvt->lock);
361	278	return 0;
@@ -372,20 +289,32 @@ err: · 10 unmodified lines
372	289	* @vgpu: a vGPU
373	290	*
374	291	*/
375		−void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu)
	292	+void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu, bool dmlr)
376	293	{
377	294	struct intel_gvt *gvt = vgpu->gvt;
378	295	const struct intel_gvt_device_info *info = &gvt->device_info;
	296	+ void *mmio = gvt->firmware.mmio;
	297	+
	298	+ if (dmlr) {
	299	+ memcpy(vgpu->mmio.vreg, mmio, info->mmio_size);
	300	+ memcpy(vgpu->mmio.sreg, mmio, info->mmio_size);
379	301
380		− memcpy(vgpu->mmio.vreg, gvt->firmware.mmio, info->mmio_size);
381		− memcpy(vgpu->mmio.sreg, gvt->firmware.mmio, info->mmio_size);
	302	+ vgpu_vreg(vgpu, GEN6_GT_THREAD_STATUS_REG) = 0;
382	303
383		− vgpu_vreg(vgpu, GEN6_GT_THREAD_STATUS_REG) = 0;
	304	+ /* set the bit 0:2(Core C-State ) to C0 */
	305	+ vgpu_vreg(vgpu, GEN6_GT_CORE_STATUS) = 0;
384	306
385		− /* set the bit 0:2(Core C-State ) to C0 */
386		− vgpu_vreg(vgpu, GEN6_GT_CORE_STATUS) = 0;
	307	+ vgpu->mmio.disable_warn_untrack = false;
	308	+ } else {
	309	+#define GVT_GEN8_MMIO_RESET_OFFSET (0x44200)
	310	+ /* only reset the engine related, so starting with 0x44200
	311	+ * interrupt include DE,display mmio related will not be
	312	+ * touched
	313	+ */
	314	+ memcpy(vgpu->mmio.vreg, mmio, GVT_GEN8_MMIO_RESET_OFFSET);
	315	+ memcpy(vgpu->mmio.sreg, mmio, GVT_GEN8_MMIO_RESET_OFFSET);
	316	+ }
387	317
388		− vgpu->mmio.disable_warn_untrack = false;
389	318	}
390	319
391	320	/**
@@ -405,7 +334,7 @@ int intel_vgpu_init_mmio(struct intel_vgpu *vgpu) · 13 unmodified lines
405	334
406	335	vgpu->mmio.sreg = vgpu->mmio.vreg + info->mmio_size;
407	336
408		− intel_vgpu_reset_mmio(vgpu);
	337	+ intel_vgpu_reset_mmio(vgpu, true);
409	338
410	339	return 0;
411	340	}

drivers/gpu/drm/i915/gvt/mmio.h+16−28

@@ -39,36 +39,28 @@ · 38 unmodified lines
39	39	struct intel_gvt;
40	40	struct intel_vgpu;
41	41
42		−#define D_SNB (1 << 0)
43		−#define D_IVB (1 << 1)
44		−#define D_HSW (1 << 2)
45		−#define D_BDW (1 << 3)
46		−#define D_SKL (1 << 4)
47		−#define D_KBL (1 << 5)
	42	+#define D_BDW (1 << 0)
	43	+#define D_SKL (1 << 1)
	44	+#define D_KBL (1 << 2)
48	45
49	46	#define D_GEN9PLUS (D_SKL \| D_KBL)
50	47	#define D_GEN8PLUS (D_BDW \| D_SKL \| D_KBL)
51		−#define D_GEN75PLUS (D_HSW \| D_BDW \| D_SKL \| D_KBL)
52		−#define D_GEN7PLUS (D_IVB \| D_HSW \| D_BDW \| D_SKL \| D_KBL)
53	48
54	49	#define D_SKL_PLUS (D_SKL \| D_KBL)
55	50	#define D_BDW_PLUS (D_BDW \| D_SKL \| D_KBL)
56		−#define D_HSW_PLUS (D_HSW \| D_BDW \| D_SKL \| D_KBL)
57		−#define D_IVB_PLUS (D_IVB \| D_HSW \| D_BDW \| D_SKL \| D_KBL)
58	51
59		−#define D_PRE_BDW (D_SNB \| D_IVB \| D_HSW)
60		−#define D_PRE_SKL (D_SNB \| D_IVB \| D_HSW \| D_BDW)
61		−#define D_ALL (D_SNB \| D_IVB \| D_HSW \| D_BDW \| D_SKL \| D_KBL)
	52	+#define D_PRE_SKL (D_BDW)
	53	+#define D_ALL (D_BDW \| D_SKL \| D_KBL)
	54	+
	55	+typedef int (gvt_mmio_func)(struct intel_vgpu , unsigned int, void *,
	56	+ unsigned int);
62	57
63	58	struct intel_gvt_mmio_info {
64	59	u32 offset;
65		− u32 size;
66		− u32 length;
67		− u32 addr_mask;
68	60	u64 ro_mask;
69	61	u32 device;
70		− int (read)(struct intel_vgpu , unsigned int, void *, unsigned int);
71		− int (write)(struct intel_vgpu , unsigned int, void *, unsigned int);
	62	+ gvt_mmio_func read;
	63	+ gvt_mmio_func write;
72	64	u32 addr_range;
73	65	struct hlist_node node;
74	66	};
@@ -79,8 +71,6 @@ bool intel_gvt_match_device(struct intel_gvt *gvt, unsigned long device); · 4 unmodified lines
79	71	int intel_gvt_setup_mmio_info(struct intel_gvt *gvt);
80	72	void intel_gvt_clean_mmio_info(struct intel_gvt *gvt);
81	73
82		−struct intel_gvt_mmio_info intel_gvt_find_mmio_info(struct intel_gvt gvt,
83		− unsigned int offset);
84	74	#define INTEL_GVT_MMIO_OFFSET(reg) ({ \
85	75	typeof(reg) __reg = reg; \
86	76	u32 offset = (u32 )&__reg; \
@@ -88,7 +78,7 @@ struct intel_gvt_mmio_info intel_gvt_find_mmio_info(struct intel_gvt gvt, · 1 unmodified line
88	78	})
89	79
90	80	int intel_vgpu_init_mmio(struct intel_vgpu *vgpu);
91		−void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu);
	81	+void intel_vgpu_reset_mmio(struct intel_vgpu *vgpu, bool dmlr);
92	82	void intel_vgpu_clean_mmio(struct intel_vgpu *vgpu);
93	83
94	84	int intel_vgpu_gpa_to_mmio_offset(struct intel_vgpu *vgpu, u64 gpa);
@@ -97,13 +87,7 @@ int intel_vgpu_emulate_mmio_read(struct intel_vgpu *vgpu, u64 pa, · 2 unmodified lines
97	87	void *p_data, unsigned int bytes);
98	88	int intel_vgpu_emulate_mmio_write(struct intel_vgpu *vgpu, u64 pa,
99	89	void *p_data, unsigned int bytes);
100		−bool intel_gvt_mmio_is_cmd_access(struct intel_gvt *gvt,
101		− unsigned int offset);
102		−bool intel_gvt_mmio_is_unalign(struct intel_gvt *gvt, unsigned int offset);
103		−void intel_gvt_mmio_set_accessed(struct intel_gvt *gvt, unsigned int offset);
104		−void intel_gvt_mmio_set_cmd_accessed(struct intel_gvt *gvt,
105		− unsigned int offset);
106		−bool intel_gvt_mmio_has_mode_mask(struct intel_gvt *gvt, unsigned int offset);
	90	+
107	91	int intel_vgpu_default_mmio_read(struct intel_vgpu *vgpu, unsigned int offset,
108	92	void *p_data, unsigned int bytes);
109	93	int intel_vgpu_default_mmio_write(struct intel_vgpu *vgpu, unsigned int offset,
@@ -111,4 +95,8 @@ int intel_vgpu_default_mmio_write(struct intel_vgpu *vgpu, unsigned int offset, · 1 unmodified line
111	95
112	96	bool intel_gvt_in_force_nonpriv_whitelist(struct intel_gvt *gvt,
113	97	unsigned int offset);
	98	+
	99	+int intel_vgpu_mmio_reg_rw(struct intel_vgpu *vgpu, unsigned int offset,
	100	+ void *pdata, unsigned int bytes, bool is_read);
	101	+
114	102	#endif

drivers/gpu/drm/i915/gvt/mpt.h+1−2

@@ -133,8 +133,7 @@ static inline int intel_gvt_hypervisor_inject_msi(struct intel_vgpu *vgpu) · 132 unmodified lines
133	133	if (WARN(control & GENMASK(15, 1), "only support one MSI format\n"))
134	134	return -EINVAL;
135	135
136		− gvt_dbg_irq("vgpu%d: inject msi address %x data%x\n", vgpu->id, addr,
137		− data);
	136	+ trace_inject_msi(vgpu->id, addr, data);
138	137
139	138	ret = intel_gvt_host.mpt->inject_msi(vgpu->handle, addr, data);
140	139	if (ret)

drivers/gpu/drm/i915/gvt/render.c+40−8

@@ -35,6 +35,7 @@ · 34 unmodified lines
35	35
36	36	#include "i915_drv.h"
37	37	#include "gvt.h"
	38	+#include "trace.h"
38	39
39	40	struct render_mmio {
40	41	int ring_id;
@@ -260,7 +261,8 @@ static void restore_mocs(struct intel_vgpu *vgpu, int ring_id) · 219 unmodified lines
260	261
261	262	#define CTX_CONTEXT_CONTROL_VAL 0x03
262	263
263		−void intel_gvt_load_render_mmio(struct intel_vgpu *vgpu, int ring_id)
	264	+/* Switch ring mmio values (context) from host to a vgpu. */
	265	+static void switch_mmio_to_vgpu(struct intel_vgpu *vgpu, int ring_id)
264	266	{
265	267	struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
266	268	struct render_mmio *mmio;
@@ -305,14 +307,15 @@ void intel_gvt_load_render_mmio(struct intel_vgpu *vgpu, int ring_id) · 38 unmodified lines
305	307	I915_WRITE(mmio->reg, v);
306	308	POSTING_READ(mmio->reg);
307	309
308		− gvt_dbg_render("load reg %x old %x new %x\n",
309		− i915_mmio_reg_offset(mmio->reg),
310		− mmio->value, v);
	310	+ trace_render_mmio(vgpu->id, "load",
	311	+ i915_mmio_reg_offset(mmio->reg),
	312	+ mmio->value, v);
311	313	}
312	314	handle_tlb_pending_event(vgpu, ring_id);
313	315	}
314	316
315		−void intel_gvt_restore_render_mmio(struct intel_vgpu *vgpu, int ring_id)
	317	+/* Switch ring mmio values (context) from vgpu to host. */
	318	+static void switch_mmio_to_host(struct intel_vgpu *vgpu, int ring_id)
316	319	{
317	320	struct drm_i915_private *dev_priv = vgpu->gvt->dev_priv;
318	321	struct render_mmio *mmio;
@@ -346,8 +349,37 @@ void intel_gvt_restore_render_mmio(struct intel_vgpu *vgpu, int ring_id) · 27 unmodified lines
346	349	I915_WRITE(mmio->reg, v);
347	350	POSTING_READ(mmio->reg);
348	351
349		− gvt_dbg_render("restore reg %x old %x new %x\n",
350		− i915_mmio_reg_offset(mmio->reg),
351		− mmio->value, v);
	352	+ trace_render_mmio(vgpu->id, "restore",
	353	+ i915_mmio_reg_offset(mmio->reg),
	354	+ mmio->value, v);
352	355	}
353	356	}
	357	+
	358	+/**
	359	+ * intel_gvt_switch_render_mmio - switch mmio context of specific engine
	360	+ * @pre: the last vGPU that own the engine
	361	+ * @next: the vGPU to switch to
	362	+ * @ring_id: specify the engine
	363	+ *
	364	+ * If pre is null indicates that host own the engine. If next is null
	365	+ * indicates that we are switching to host workload.
	366	+ */
	367	+void intel_gvt_switch_mmio(struct intel_vgpu *pre,
	368	+ struct intel_vgpu *next, int ring_id)
	369	+{
	370	+ if (WARN_ON(!pre && !next))
	371	+ return;
	372	+
	373	+ gvt_dbg_render("switch ring %d from %s to %s\n", ring_id,
	374	+ pre ? "vGPU" : "host", next ? "vGPU" : "HOST");
	375	+
	376	+ /**
	377	+ * TODO: Optimize for vGPU to vGPU switch by merging
	378	+ * switch_mmio_to_host() and switch_mmio_to_vgpu().
	379	+ */
	380	+ if (pre)
	381	+ switch_mmio_to_host(pre, ring_id);
	382	+
	383	+ if (next)
	384	+ switch_mmio_to_vgpu(next, ring_id);
	385	+}

drivers/gpu/drm/i915/gvt/render.h+2−2

@@ -36,8 +36,8 @@ · 35 unmodified lines
36	36	#ifndef __GVT_RENDER_H__
37	37	#define __GVT_RENDER_H__
38	38
39		−void intel_gvt_load_render_mmio(struct intel_vgpu *vgpu, int ring_id);
	39	+void intel_gvt_switch_mmio(struct intel_vgpu *pre,
	40	+ struct intel_vgpu *next, int ring_id);
40	41
41		−void intel_gvt_restore_render_mmio(struct intel_vgpu *vgpu, int ring_id);
42	42
43	43	#endif

drivers/gpu/drm/i915/gvt/sched_policy.c+22−5

@@ -202,11 +202,6 @@ static void tbs_sched_func(struct gvt_sched_data *sched_data) · 201 unmodified lines
202	202	struct intel_gvt_workload_scheduler *scheduler = &gvt->scheduler;
203	203	struct vgpu_sched_data *vgpu_data;
204	204	struct intel_vgpu *vgpu = NULL;
205		− static uint64_t timer_check;
206		−
207		− if (!(timer_check++ % GVT_TS_BALANCE_PERIOD_MS))
208		− gvt_balance_timeslice(sched_data);
209		−
210	205	/* no active vgpu or has already had a target */
211	206	if (list_empty(&sched_data->lru_runq_head) \|\| scheduler->next_vgpu)
212	207	goto out;
@@ -231,9 +226,19 @@ out: · 18 unmodified lines
231	226	void intel_gvt_schedule(struct intel_gvt *gvt)
232	227	{
233	228	struct gvt_sched_data *sched_data = gvt->scheduler.sched_data;
	229	+ static uint64_t timer_check;
234	230
235	231	mutex_lock(&gvt->lock);
	232	+
	233	+ if (test_and_clear_bit(INTEL_GVT_REQUEST_SCHED,
	234	+ (void *)&gvt->service_request)) {
	235	+ if (!(timer_check++ % GVT_TS_BALANCE_PERIOD_MS))
	236	+ gvt_balance_timeslice(sched_data);
	237	+ }
	238	+ clear_bit(INTEL_GVT_REQUEST_EVENT_SCHED, (void *)&gvt->service_request);
	239	+
236	240	tbs_sched_func(sched_data);
	241	+
237	242	mutex_unlock(&gvt->lock);
238	243	}
239	244
@@ -303,8 +308,20 @@ static int tbs_sched_init_vgpu(struct intel_vgpu *vgpu) · 63 unmodified lines
303	308
304	309	static void tbs_sched_clean_vgpu(struct intel_vgpu *vgpu)
305	310	{
	311	+ struct intel_gvt_workload_scheduler *scheduler = &vgpu->gvt->scheduler;
	312	+ int ring_id;
	313	+
306	314	kfree(vgpu->sched_data);
307	315	vgpu->sched_data = NULL;
	316	+
	317	+ spin_lock_bh(&scheduler->mmio_context_lock);
	318	+ for (ring_id = 0; ring_id < I915_NUM_ENGINES; ring_id++) {
	319	+ if (scheduler->engine_owner[ring_id] == vgpu) {
	320	+ intel_gvt_switch_mmio(vgpu, NULL, ring_id);
	321	+ scheduler->engine_owner[ring_id] = NULL;
	322	+ }
	323	+ }
	324	+ spin_unlock_bh(&scheduler->mmio_context_lock);
308	325	}
309	326
310	327	static void tbs_sched_start_schedule(struct intel_vgpu *vgpu)

drivers/gpu/drm/i915/gvt/scheduler.c+32−7

@@ -138,21 +138,42 @@ static int shadow_context_status_change(struct notifier_block *nb, · 137 unmodified lines
138	138	struct intel_gvt *gvt = container_of(nb, struct intel_gvt,
139	139	shadow_ctx_notifier_block[req->engine->id]);
140	140	struct intel_gvt_workload_scheduler *scheduler = &gvt->scheduler;
141		− struct intel_vgpu_workload *workload =
142		− scheduler->current_workload[req->engine->id];
	141	+ enum intel_engine_id ring_id = req->engine->id;
	142	+ struct intel_vgpu_workload *workload;
	143	+
	144	+ if (!is_gvt_request(req)) {
	145	+ spin_lock_bh(&scheduler->mmio_context_lock);
	146	+ if (action == INTEL_CONTEXT_SCHEDULE_IN &&
	147	+ scheduler->engine_owner[ring_id]) {
	148	+ /* Switch ring from vGPU to host. */
	149	+ intel_gvt_switch_mmio(scheduler->engine_owner[ring_id],
	150	+ NULL, ring_id);
	151	+ scheduler->engine_owner[ring_id] = NULL;
	152	+ }
	153	+ spin_unlock_bh(&scheduler->mmio_context_lock);
143	154
144		− if (!is_gvt_request(req) \|\| unlikely(!workload))
	155	+ return NOTIFY_OK;
	156	+ }
	157	+
	158	+ workload = scheduler->current_workload[ring_id];
	159	+ if (unlikely(!workload))
145	160	return NOTIFY_OK;
146	161
147	162	switch (action) {
148	163	case INTEL_CONTEXT_SCHEDULE_IN:
149		− intel_gvt_load_render_mmio(workload->vgpu,
150		− workload->ring_id);
	164	+ spin_lock_bh(&scheduler->mmio_context_lock);
	165	+ if (workload->vgpu != scheduler->engine_owner[ring_id]) {
	166	+ /* Switch ring from host to vGPU or vGPU to vGPU. */
	167	+ intel_gvt_switch_mmio(scheduler->engine_owner[ring_id],
	168	+ workload->vgpu, ring_id);
	169	+ scheduler->engine_owner[ring_id] = workload->vgpu;
	170	+ } else
	171	+ gvt_dbg_sched("skip ring %d mmio switch for vgpu%d\n",
	172	+ ring_id, workload->vgpu->id);
	173	+ spin_unlock_bh(&scheduler->mmio_context_lock);
151	174	atomic_set(&workload->shadow_ctx_active, 1);
152	175	break;
153	176	case INTEL_CONTEXT_SCHEDULE_OUT:
154		− intel_gvt_restore_render_mmio(workload->vgpu,
155		− workload->ring_id);
156	177	/* If the status is -EINPROGRESS means this workload
157	178	* doesn't meet any issue during dispatching so when
158	179	* get the SCHEDULE_OUT set the status to be zero for
@@ -431,6 +452,10 @@ static void complete_current_workload(struct intel_gvt *gvt, int ring_id) · 272 unmodified lines
431	452
432	453	atomic_dec(&vgpu->running_workload_num);
433	454	wake_up(&scheduler->workload_complete_wq);
	455	+
	456	+ if (gvt->scheduler.need_reschedule)
	457	+ intel_gvt_request_service(gvt, INTEL_GVT_REQUEST_EVENT_SCHED);
	458	+
434	459	mutex_unlock(&gvt->lock);
435	460	}
436	461

drivers/gpu/drm/i915/gvt/scheduler.h+4−0

@@ -42,6 +42,10 @@ struct intel_gvt_workload_scheduler { · 41 unmodified lines
42	42	struct intel_vgpu_workload *current_workload[I915_NUM_ENGINES];
43	43	bool need_reschedule;
44	44
	45	+ spinlock_t mmio_context_lock;
	46	+ /* can be null when owner is host */
	47	+ struct intel_vgpu *engine_owner[I915_NUM_ENGINES];
	48	+
45	49	wait_queue_head_t workload_complete_wq;
46	50	struct task_struct *thread[I915_NUM_ENGINES];
47	51	wait_queue_head_t waitq[I915_NUM_ENGINES];

drivers/gpu/drm/i915/gvt/trace.h+129−49

@@ -224,58 +224,138 @@ TRACE_EVENT(oos_sync, · 223 unmodified lines
224	224	TP_printk("%s", __entry->buf)
225	225	);
226	226
227		−#define MAX_CMD_STR_LEN 256
228	227	TRACE_EVENT(gvt_command,
229		− TP_PROTO(u8 vm_id, u8 ring_id, u32 ip_gma, u32 *cmd_va, u32 cmd_len, bool ring_buffer_cmd, cycles_t cost_pre_cmd_handler, cycles_t cost_cmd_handler),
230		−
231		− TP_ARGS(vm_id, ring_id, ip_gma, cmd_va, cmd_len, ring_buffer_cmd, cost_pre_cmd_handler, cost_cmd_handler),
232		−
233		− TP_STRUCT__entry(
234		− __field(u8, vm_id)
235		− __field(u8, ring_id)
236		− __field(int, i)
237		− __array(char, tmp_buf, MAX_CMD_STR_LEN)
238		− __array(char, cmd_str, MAX_CMD_STR_LEN)
239		− ),
240		−
241		− TP_fast_assign(
242		− __entry->vm_id = vm_id;
243		− __entry->ring_id = ring_id;
244		− __entry->cmd_str[0] = '\0';
245		− snprintf(__entry->tmp_buf, MAX_CMD_STR_LEN, "VM(%d) Ring(%d): %s ip(%08x) pre handler cost (%llu), handler cost (%llu) ", vm_id, ring_id, ring_buffer_cmd ? "RB":"BB", ip_gma, cost_pre_cmd_handler, cost_cmd_handler);
246		− strcat(__entry->cmd_str, __entry->tmp_buf);
247		− entry->i = 0;
248		− while (cmd_len > 0) {
249		− if (cmd_len >= 8) {
250		− snprintf(__entry->tmp_buf, MAX_CMD_STR_LEN, "%08x %08x %08x %08x %08x %08x %08x %08x ",
251		− cmd_va[__entry->i], cmd_va[__entry->i+1], cmd_va[__entry->i+2], cmd_va[__entry->i+3],
252		− cmd_va[__entry->i+4], cmd_va[__entry->i+5], cmd_va[__entry->i+6], cmd_va[__entry->i+7]);
253		− __entry->i += 8;
254		− cmd_len -= 8;
255		− strcat(__entry->cmd_str, __entry->tmp_buf);
256		− } else if (cmd_len >= 4) {
257		− snprintf(__entry->tmp_buf, MAX_CMD_STR_LEN, "%08x %08x %08x %08x ",
258		− cmd_va[__entry->i], cmd_va[__entry->i+1], cmd_va[__entry->i+2], cmd_va[__entry->i+3]);
259		− __entry->i += 4;
260		− cmd_len -= 4;
261		− strcat(__entry->cmd_str, __entry->tmp_buf);
262		− } else if (cmd_len >= 2) {
263		− snprintf(__entry->tmp_buf, MAX_CMD_STR_LEN, "%08x %08x ", cmd_va[__entry->i], cmd_va[__entry->i+1]);
264		− __entry->i += 2;
265		− cmd_len -= 2;
266		− strcat(__entry->cmd_str, __entry->tmp_buf);
267		− } else if (cmd_len == 1) {
268		− snprintf(__entry->tmp_buf, MAX_CMD_STR_LEN, "%08x ", cmd_va[__entry->i]);
269		− __entry->i += 1;
270		− cmd_len -= 1;
271		− strcat(__entry->cmd_str, __entry->tmp_buf);
272		− }
273		− }
274		− strcat(__entry->cmd_str, "\n");
275		− ),
	228	+ TP_PROTO(u8 vgpu_id, u8 ring_id, u32 ip_gma, u32 *cmd_va, u32 cmd_len,
	229	+ u32 buf_type),
	230	+
	231	+ TP_ARGS(vgpu_id, ring_id, ip_gma, cmd_va, cmd_len, buf_type),
	232	+
	233	+ TP_STRUCT__entry(
	234	+ __field(u8, vgpu_id)
	235	+ __field(u8, ring_id)
	236	+ __field(u32, ip_gma)
	237	+ __field(u32, buf_type)
	238	+ __field(u32, cmd_len)
	239	+ __dynamic_array(u32, raw_cmd, cmd_len)
	240	+ ),
	241	+
	242	+ TP_fast_assign(
	243	+ __entry->vgpu_id = vgpu_id;
	244	+ __entry->ring_id = ring_id;
	245	+ __entry->ip_gma = ip_gma;
	246	+ __entry->buf_type = buf_type;
	247	+ __entry->cmd_len = cmd_len;
	248	+ memcpy(__get_dynamic_array(raw_cmd), cmd_va, cmd_len * sizeof(*cmd_va));
	249	+ ),
	250	+
	251	+
	252	+ TP_printk("vgpu%d ring %d: buf_type %u, ip_gma %08x, raw cmd %s",
	253	+ __entry->vgpu_id,
	254	+ __entry->ring_id,
	255	+ __entry->buf_type,
	256	+ __entry->ip_gma,
	257	+ __print_array(__get_dynamic_array(raw_cmd), __entry->cmd_len, 4))
	258	+);
	259	+
	260	+#define GVT_TEMP_STR_LEN 10
	261	+TRACE_EVENT(write_ir,
	262	+ TP_PROTO(int id, char *reg_name, unsigned int reg, unsigned int new_val,
	263	+ unsigned int old_val, bool changed),
	264	+
	265	+ TP_ARGS(id, reg_name, reg, new_val, old_val, changed),
	266	+
	267	+ TP_STRUCT__entry(
	268	+ __field(int, id)
	269	+ __array(char, buf, GVT_TEMP_STR_LEN)
	270	+ __field(unsigned int, reg)
	271	+ __field(unsigned int, new_val)
	272	+ __field(unsigned int, old_val)
	273	+ __field(bool, changed)
	274	+ ),
	275	+
	276	+ TP_fast_assign(
	277	+ __entry->id = id;
	278	+ snprintf(__entry->buf, GVT_TEMP_STR_LEN, "%s", reg_name);
	279	+ __entry->reg = reg;
	280	+ __entry->new_val = new_val;
	281	+ __entry->old_val = old_val;
	282	+ __entry->changed = changed;
	283	+ ),
	284	+
	285	+ TP_printk("VM%u write [%s] %x, new %08x, old %08x, changed %08x\n",
	286	+ __entry->id, __entry->buf, __entry->reg, __entry->new_val,
	287	+ __entry->old_val, __entry->changed)
	288	+);
	289	+
	290	+TRACE_EVENT(propagate_event,
	291	+ TP_PROTO(int id, const char *irq_name, int bit),
	292	+
	293	+ TP_ARGS(id, irq_name, bit),
	294	+
	295	+ TP_STRUCT__entry(
	296	+ __field(int, id)
	297	+ __array(char, buf, GVT_TEMP_STR_LEN)
	298	+ __field(int, bit)
	299	+ ),
276	300
277		− TP_printk("%s", __entry->cmd_str)
	301	+ TP_fast_assign(
	302	+ __entry->id = id;
	303	+ snprintf(__entry->buf, GVT_TEMP_STR_LEN, "%s", irq_name);
	304	+ __entry->bit = bit;
	305	+ ),
	306	+
	307	+ TP_printk("Set bit (%d) for (%s) for vgpu (%d)\n",
	308	+ __entry->bit, __entry->buf, __entry->id)
278	309	);
	310	+
	311	+TRACE_EVENT(inject_msi,
	312	+ TP_PROTO(int id, unsigned int address, unsigned int data),
	313	+
	314	+ TP_ARGS(id, address, data),
	315	+
	316	+ TP_STRUCT__entry(
	317	+ __field(int, id)
	318	+ __field(unsigned int, address)
	319	+ __field(unsigned int, data)
	320	+ ),
	321	+
	322	+ TP_fast_assign(
	323	+ __entry->id = id;
	324	+ __entry->address = address;
	325	+ __entry->data = data;
	326	+ ),
	327	+
	328	+ TP_printk("vgpu%d:inject msi address %x data %x\n",
	329	+ __entry->id, __entry->address, __entry->data)
	330	+);
	331	+
	332	+TRACE_EVENT(render_mmio,
	333	+ TP_PROTO(int id, char *action, unsigned int reg,
	334	+ unsigned int old_val, unsigned int new_val),
	335	+
	336	+ TP_ARGS(id, action, reg, new_val, old_val),
	337	+
	338	+ TP_STRUCT__entry(
	339	+ __field(int, id)
	340	+ __array(char, buf, GVT_TEMP_STR_LEN)
	341	+ __field(unsigned int, reg)
	342	+ __field(unsigned int, old_val)
	343	+ __field(unsigned int, new_val)
	344	+ ),
	345	+
	346	+ TP_fast_assign(
	347	+ __entry->id = id;
	348	+ snprintf(__entry->buf, GVT_TEMP_STR_LEN, "%s", action);
	349	+ __entry->reg = reg;
	350	+ __entry->old_val = old_val;
	351	+ __entry->new_val = new_val;
	352	+ ),
	353	+
	354	+ TP_printk("VM%u %s reg %x, old %08x new %08x\n",
	355	+ __entry->id, __entry->buf, __entry->reg,
	356	+ __entry->old_val, __entry->new_val)
	357	+);
	358	+
279	359	#endif /* _GVT_TRACE_H_ */
280	360
281	361	/* This part must be out of protection */

drivers/gpu/drm/i915/gvt/vgpu.c+7−2

@@ -501,9 +501,14 @@ void intel_gvt_reset_vgpu_locked(struct intel_vgpu *vgpu, bool dmlr, · 500 unmodified lines
501	501
502	502	/* full GPU reset or device model level reset */
503	503	if (engine_mask == ALL_ENGINES \|\| dmlr) {
	504	+
504	505	intel_vgpu_reset_gtt(vgpu, dmlr);
505		− intel_vgpu_reset_resource(vgpu);
506		− intel_vgpu_reset_mmio(vgpu);
	506	+
	507	+ /fence will not be reset during virtual reset /
	508	+ if (dmlr)
	509	+ intel_vgpu_reset_resource(vgpu);
	510	+
	511	+ intel_vgpu_reset_mmio(vgpu, dmlr);
507	512	populate_pvinfo_page(vgpu);
508	513	intel_vgpu_reset_display(vgpu);
509	514

drivers/gpu/drm/i915/i915_debugfs.c+41−27

@@ -1670,12 +1670,22 @@ static int i915_fbc_status(struct seq_file m, void unused) · 1669 unmodified lines
1670	1670	seq_printf(m, "FBC disabled: %s\n",
1671	1671	dev_priv->fbc.no_fbc_reason);
1672	1672
1673		− if (intel_fbc_is_active(dev_priv) && INTEL_GEN(dev_priv) >= 7) {
1674		− uint32_t mask = INTEL_GEN(dev_priv) >= 8 ?
1675		− BDW_FBC_COMPRESSION_MASK :
1676		− IVB_FBC_COMPRESSION_MASK;
1677		− seq_printf(m, "Compressing: %s\n",
1678		− yesno(I915_READ(FBC_STATUS2) & mask));
	1673	+ if (intel_fbc_is_active(dev_priv)) {
	1674	+ u32 mask;
	1675	+
	1676	+ if (INTEL_GEN(dev_priv) >= 8)
	1677	+ mask = I915_READ(IVB_FBC_STATUS2) & BDW_FBC_COMP_SEG_MASK;
	1678	+ else if (INTEL_GEN(dev_priv) >= 7)
	1679	+ mask = I915_READ(IVB_FBC_STATUS2) & IVB_FBC_COMP_SEG_MASK;
	1680	+ else if (INTEL_GEN(dev_priv) >= 5)
	1681	+ mask = I915_READ(ILK_DPFC_STATUS) & ILK_DPFC_COMP_SEG_MASK;
	1682	+ else if (IS_G4X(dev_priv))
	1683	+ mask = I915_READ(DPFC_STATUS) & DPFC_COMP_SEG_MASK;
	1684	+ else
	1685	+ mask = I915_READ(FBC_STATUS) & (FBC_STAT_COMPRESSING \|
	1686	+ FBC_STAT_COMPRESSED);
	1687	+
	1688	+ seq_printf(m, "Compressing: %s\n", yesno(mask));
1679	1689	}
1680	1690
1681	1691	mutex_unlock(&dev_priv->fbc.lock);
@@ -1684,7 +1694,7 @@ static int i915_fbc_status(struct seq_file m, void unused) · 2 unmodified lines
1684	1694	return 0;
1685	1695	}
1686	1696
1687		−static int i915_fbc_fc_get(void data, u64 val)
	1697	+static int i915_fbc_false_color_get(void data, u64 val)
1688	1698	{
1689	1699	struct drm_i915_private *dev_priv = data;
1690	1700
@@ -1696,7 +1706,7 @@ static int i915_fbc_fc_get(void data, u64 val) · 5 unmodified lines
1696	1706	return 0;
1697	1707	}
1698	1708
1699		−static int i915_fbc_fc_set(void *data, u64 val)
	1709	+static int i915_fbc_false_color_set(void *data, u64 val)
1700	1710	{
1701	1711	struct drm_i915_private *dev_priv = data;
1702	1712	u32 reg;
@@ -1717,8 +1727,8 @@ static int i915_fbc_fc_set(void *data, u64 val) · 14 unmodified lines
1717	1727	return 0;
1718	1728	}
1719	1729
1720		−DEFINE_SIMPLE_ATTRIBUTE(i915_fbc_fc_fops,
1721		− i915_fbc_fc_get, i915_fbc_fc_set,
	1730	+DEFINE_SIMPLE_ATTRIBUTE(i915_fbc_false_color_fops,
	1731	+ i915_fbc_false_color_get, i915_fbc_false_color_set,
1722	1732	"%llu\n");
1723	1733
1724	1734	static int i915_ips_status(struct seq_file m, void unused)
@@ -1988,6 +1998,12 @@ static int i915_context_status(struct seq_file m, void unused) · 263 unmodified lines
1988	1998	seq_putc(m, '\n');
1989	1999	}
1990	2000
	2001	+ seq_printf(m,
	2002	+ "\tvma hashtable size=%u (actual %lu), count=%u\n",
	2003	+ ctx->vma_lut.ht_size,
	2004	+ BIT(ctx->vma_lut.ht_bits),
	2005	+ ctx->vma_lut.ht_count);
	2006	+
1991	2007	seq_putc(m, '\n');
1992	2008	}
1993	2009
@@ -4289,26 +4305,27 @@ i915_drop_caches_set(void *data, u64 val) · 2295 unmodified lines
4289	4305	{
4290	4306	struct drm_i915_private *dev_priv = data;
4291	4307	struct drm_device *dev = &dev_priv->drm;
4292		− int ret;
	4308	+ int ret = 0;
4293	4309
4294	4310	DRM_DEBUG("Dropping caches: 0x%08llx\n", val);
4295	4311
4296	4312	/* No need to check and wait for gpu resets, only libdrm auto-restarts
4297	4313	* on ioctls on -EAGAIN. */
4298		− ret = mutex_lock_interruptible(&dev->struct_mutex);
4299		− if (ret)
4300		− return ret;
4301		−
4302		− if (val & DROP_ACTIVE) {
4303		− ret = i915_gem_wait_for_idle(dev_priv,
4304		− I915_WAIT_INTERRUPTIBLE \|
4305		− I915_WAIT_LOCKED);
	4314	+ if (val & (DROP_ACTIVE \| DROP_RETIRE)) {
	4315	+ ret = mutex_lock_interruptible(&dev->struct_mutex);
4306	4316	if (ret)
4307		− goto unlock;
4308		− }
	4317	+ return ret;
4309	4318
4310		− if (val & DROP_RETIRE)
4311		− i915_gem_retire_requests(dev_priv);
	4319	+ if (val & DROP_ACTIVE)
	4320	+ ret = i915_gem_wait_for_idle(dev_priv,
	4321	+ I915_WAIT_INTERRUPTIBLE \|
	4322	+ I915_WAIT_LOCKED);
	4323	+
	4324	+ if (val & DROP_RETIRE)
	4325	+ i915_gem_retire_requests(dev_priv);
	4326	+
	4327	+ mutex_unlock(&dev->struct_mutex);
	4328	+ }
4312	4329
4313	4330	lockdep_set_current_reclaim_state(GFP_KERNEL);
4314	4331	if (val & DROP_BOUND)
@@ -4321,9 +4338,6 @@ i915_drop_caches_set(void *data, u64 val) · 6 unmodified lines
4321	4338	i915_gem_shrink_all(dev_priv);
4322	4339	lockdep_clear_current_reclaim_state();
4323	4340
4324		−unlock:
4325		− mutex_unlock(&dev->struct_mutex);
4326		−
4327	4341	if (val & DROP_FREED) {
4328	4342	synchronize_rcu();
4329	4343	i915_gem_drain_freed_objects(dev_priv);
@@ -4861,7 +4875,7 @@ static const struct i915_debugfs_files { · 531 unmodified lines
4861	4875	{"i915_pri_wm_latency", &i915_pri_wm_latency_fops},
4862	4876	{"i915_spr_wm_latency", &i915_spr_wm_latency_fops},
4863	4877	{"i915_cur_wm_latency", &i915_cur_wm_latency_fops},
4864		− {"i915_fbc_false_color", &i915_fbc_fc_fops},
	4878	+ {"i915_fbc_false_color", &i915_fbc_false_color_fops},
4865	4879	{"i915_dp_test_data", &i915_displayport_test_data_fops},
4866	4880	{"i915_dp_test_type", &i915_displayport_test_type_fops},
4867	4881	{"i915_dp_test_active", &i915_displayport_test_active_fops},

drivers/gpu/drm/i915/i915_drv.c+41−5

@@ -139,6 +139,9 @@ static enum intel_pch intel_virt_detect_pch(struct drm_i915_private *dev_priv) · 138 unmodified lines
139	139	} else if (IS_SKYLAKE(dev_priv) \|\| IS_KABYLAKE(dev_priv)) {
140	140	ret = PCH_SPT;
141	141	DRM_DEBUG_KMS("Assuming SunrisePoint PCH\n");
	142	+ } else if (IS_COFFEELAKE(dev_priv) \|\| IS_CANNONLAKE(dev_priv)) {
	143	+ ret = PCH_CNP;
	144	+ DRM_DEBUG_KMS("Assuming CannonPoint PCH\n");
142	145	}
143	146
144	147	return ret;
@@ -170,24 +173,29 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) · 25 unmodified lines
170	173	while ((pch = pci_get_class(PCI_CLASS_BRIDGE_ISA << 8, pch))) {
171	174	if (pch->vendor == PCI_VENDOR_ID_INTEL) {
172	175	unsigned short id = pch->device & INTEL_PCH_DEVICE_ID_MASK;
173		− dev_priv->pch_id = id;
	176	+ unsigned short id_ext = pch->device &
	177	+ INTEL_PCH_DEVICE_ID_MASK_EXT;
174	178
175	179	if (id == INTEL_PCH_IBX_DEVICE_ID_TYPE) {
	180	+ dev_priv->pch_id = id;
176	181	dev_priv->pch_type = PCH_IBX;
177	182	DRM_DEBUG_KMS("Found Ibex Peak PCH\n");
178	183	WARN_ON(!IS_GEN5(dev_priv));
179	184	} else if (id == INTEL_PCH_CPT_DEVICE_ID_TYPE) {
	185	+ dev_priv->pch_id = id;
180	186	dev_priv->pch_type = PCH_CPT;
181	187	DRM_DEBUG_KMS("Found CougarPoint PCH\n");
182	188	WARN_ON(!(IS_GEN6(dev_priv) \|\|
183	189	IS_IVYBRIDGE(dev_priv)));
184	190	} else if (id == INTEL_PCH_PPT_DEVICE_ID_TYPE) {
185	191	/* PantherPoint is CPT compatible */
	192	+ dev_priv->pch_id = id;
186	193	dev_priv->pch_type = PCH_CPT;
187	194	DRM_DEBUG_KMS("Found PantherPoint PCH\n");
188	195	WARN_ON(!(IS_GEN6(dev_priv) \|\|
189	196	IS_IVYBRIDGE(dev_priv)));
190	197	} else if (id == INTEL_PCH_LPT_DEVICE_ID_TYPE) {
	198	+ dev_priv->pch_id = id;
191	199	dev_priv->pch_type = PCH_LPT;
192	200	DRM_DEBUG_KMS("Found LynxPoint PCH\n");
193	201	WARN_ON(!IS_HASWELL(dev_priv) &&
@@ -195,6 +203,7 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) · 1 unmodified line
195	203	WARN_ON(IS_HSW_ULT(dev_priv) \|\|
196	204	IS_BDW_ULT(dev_priv));
197	205	} else if (id == INTEL_PCH_LPT_LP_DEVICE_ID_TYPE) {
	206	+ dev_priv->pch_id = id;
198	207	dev_priv->pch_type = PCH_LPT;
199	208	DRM_DEBUG_KMS("Found LynxPoint LP PCH\n");
200	209	WARN_ON(!IS_HASWELL(dev_priv) &&
@@ -202,20 +211,35 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) · 1 unmodified line
202	211	WARN_ON(!IS_HSW_ULT(dev_priv) &&
203	212	!IS_BDW_ULT(dev_priv));
204	213	} else if (id == INTEL_PCH_SPT_DEVICE_ID_TYPE) {
	214	+ dev_priv->pch_id = id;
205	215	dev_priv->pch_type = PCH_SPT;
206	216	DRM_DEBUG_KMS("Found SunrisePoint PCH\n");
207	217	WARN_ON(!IS_SKYLAKE(dev_priv) &&
208	218	!IS_KABYLAKE(dev_priv));
209		− } else if (id == INTEL_PCH_SPT_LP_DEVICE_ID_TYPE) {
	219	+ } else if (id_ext == INTEL_PCH_SPT_LP_DEVICE_ID_TYPE) {
	220	+ dev_priv->pch_id = id_ext;
210	221	dev_priv->pch_type = PCH_SPT;
211	222	DRM_DEBUG_KMS("Found SunrisePoint LP PCH\n");
212	223	WARN_ON(!IS_SKYLAKE(dev_priv) &&
213	224	!IS_KABYLAKE(dev_priv));
214	225	} else if (id == INTEL_PCH_KBP_DEVICE_ID_TYPE) {
	226	+ dev_priv->pch_id = id;
215	227	dev_priv->pch_type = PCH_KBP;
216	228	DRM_DEBUG_KMS("Found KabyPoint PCH\n");
217	229	WARN_ON(!IS_SKYLAKE(dev_priv) &&
218	230	!IS_KABYLAKE(dev_priv));
	231	+ } else if (id == INTEL_PCH_CNP_DEVICE_ID_TYPE) {
	232	+ dev_priv->pch_id = id;
	233	+ dev_priv->pch_type = PCH_CNP;
	234	+ DRM_DEBUG_KMS("Found CannonPoint PCH\n");
	235	+ WARN_ON(!IS_CANNONLAKE(dev_priv) &&
	236	+ !IS_COFFEELAKE(dev_priv));
	237	+ } else if (id_ext == INTEL_PCH_CNP_LP_DEVICE_ID_TYPE) {
	238	+ dev_priv->pch_id = id_ext;
	239	+ dev_priv->pch_type = PCH_CNP;
	240	+ DRM_DEBUG_KMS("Found CannonPoint LP PCH\n");
	241	+ WARN_ON(!IS_CANNONLAKE(dev_priv) &&
	242	+ !IS_COFFEELAKE(dev_priv));
219	243	} else if ((id == INTEL_PCH_P2X_DEVICE_ID_TYPE) \|\|
220	244	(id == INTEL_PCH_P3X_DEVICE_ID_TYPE) \|\|
221	245	((id == INTEL_PCH_QEMU_DEVICE_ID_TYPE) &&
@@ -223,6 +247,7 @@ static void intel_detect_pch(struct drm_i915_private *dev_priv) · 1 unmodified line
223	247	PCI_SUBVENDOR_ID_REDHAT_QUMRANET &&
224	248	pch->subsystem_device ==
225	249	PCI_SUBDEVICE_ID_QEMU)) {
	250	+ dev_priv->pch_id = id;
226	251	dev_priv->pch_type =
227	252	intel_virt_detect_pch(dev_priv);
228	253	} else
@@ -351,6 +376,7 @@ static int i915_getparam(struct drm_device dev, void data, · 122 unmodified lines
351	376	case I915_PARAM_HAS_EXEC_ASYNC:
352	377	case I915_PARAM_HAS_EXEC_FENCE:
353	378	case I915_PARAM_HAS_EXEC_CAPTURE:
	379	+ case I915_PARAM_HAS_EXEC_BATCH_FIRST:
354	380	/* For the time being all of these are always true;
355	381	* if some supported hardware does not have one of these
356	382	* features this value needs to be provided from
@@ -358,6 +384,16 @@ static int i915_getparam(struct drm_device dev, void data, · 1 unmodified line
358	384	*/
359	385	value = 1;
360	386	break;
	387	+ case I915_PARAM_SLICE_MASK:
	388	+ value = INTEL_INFO(dev_priv)->sseu.slice_mask;
	389	+ if (!value)
	390	+ return -ENODEV;
	391	+ break;
	392	+ case I915_PARAM_SUBSLICE_MASK:
	393	+ value = INTEL_INFO(dev_priv)->sseu.subslice_mask;
	394	+ if (!value)
	395	+ return -ENODEV;
	396	+ break;
361	397	default:
362	398	DRM_DEBUG("Unknown parameter %d\n", param->param);
363	399	return -EINVAL;
@@ -553,6 +589,7 @@ static void i915_gem_fini(struct drm_i915_private *dev_priv) · 189 unmodified lines
553	589	intel_uc_fini_hw(dev_priv);
554	590	i915_gem_cleanup_engines(dev_priv);
555	591	i915_gem_context_fini(dev_priv);
	592	+ i915_gem_cleanup_userptr(dev_priv);
556	593	mutex_unlock(&dev_priv->drm.struct_mutex);
557	594
558	595	i915_gem_drain_freed_objects(dev_priv);
@@ -997,6 +1034,8 @@ static void intel_sanitize_options(struct drm_i915_private *dev_priv) · 438 unmodified lines
997	1034	DRM_DEBUG_DRIVER("use GPU semaphores? %s\n", yesno(i915.semaphores));
998	1035
999	1036	intel_uc_sanitize_options(dev_priv);
	1037	+
	1038	+ intel_gvt_sanitize_options(dev_priv);
1000	1039	}
1001	1040
1002	1041	/**
@@ -2459,9 +2498,6 @@ static int intel_runtime_resume(struct device *kdev) · 1456 unmodified lines
2459	2498
2460	2499	intel_guc_resume(dev_priv);
2461	2500
2462		− if (IS_GEN6(dev_priv))
2463		− intel_init_pch_refclk(dev_priv);
2464		−
2465	2501	if (IS_GEN9_LP(dev_priv)) {
2466	2502	bxt_disable_dc9(dev_priv);
2467	2503	bxt_display_core_init(dev_priv, true);

drivers/gpu/drm/i915/i915_drv.h+76−30

@@ -37,7 +37,7 @@ · 36 unmodified lines
37	37	#include <linux/i2c.h>
38	38	#include <linux/i2c-algo-bit.h>
39	39	#include <linux/backlight.h>
40		−#include <linux/hashtable.h>
	40	+#include <linux/hash.h>
41	41	#include <linux/intel-iommu.h>
42	42	#include <linux/kref.h>
43	43	#include <linux/pm_qos.h>
@@ -80,8 +80,8 @@ · 36 unmodified lines
80	80
81	81	#define DRIVER_NAME "i915"
82	82	#define DRIVER_DESC "Intel Graphics"
83		−#define DRIVER_DATE "20170529"
84		−#define DRIVER_TIMESTAMP 1496041258
	83	+#define DRIVER_DATE "20170619"
	84	+#define DRIVER_TIMESTAMP 1497857498
85	85
86	86	/* Use I915_STATE_WARN(x) and I915_STATE_WARN_ON() (rather than WARN() and
87	87	* WARN_ON()) for hw state sanity checks to check for unexpected conditions
@@ -752,7 +752,6 @@ struct intel_csr { · 664 unmodified lines
752	752	func(has_aliasing_ppgtt); \
753	753	func(has_csr); \
754	754	func(has_ddi); \
755		− func(has_decoupled_mmio); \
756	755	func(has_dp_mst); \
757	756	func(has_fbc); \
758	757	func(has_fpga_dbg); \
@@ -827,6 +826,8 @@ enum intel_platform { · 68 unmodified lines
827	826	INTEL_BROXTON,
828	827	INTEL_KABYLAKE,
829	828	INTEL_GEMINILAKE,
	829	+ INTEL_COFFEELAKE,
	830	+ INTEL_CANNONLAKE,
830	831	INTEL_MAX_PLATFORMS
831	832	};
832	833
@@ -1152,6 +1153,7 @@ enum intel_pch { · 319 unmodified lines
1152	1153	PCH_LPT, /* Lynxpoint PCH */
1153	1154	PCH_SPT, /* Sunrisepoint PCH */
1154	1155	PCH_KBP, /* Kabypoint PCH */
	1156	+ PCH_CNP, /* Cannonpoint PCH */
1155	1157	PCH_NOP,
1156	1158	};
1157	1159
@@ -1160,11 +1162,9 @@ enum intel_sbi_destination { · 2 unmodified lines
1160	1162	SBI_MPHY,
1161	1163	};
1162	1164
1163		−#define QUIRK_PIPEA_FORCE (1<<0)
1164	1165	#define QUIRK_LVDS_SSC_DISABLE (1<<1)
1165	1166	#define QUIRK_INVERT_BRIGHTNESS (1<<2)
1166	1167	#define QUIRK_BACKLIGHT_PRESENT (1<<3)
1167		−#define QUIRK_PIPEB_FORCE (1<<4)
1168	1168	#define QUIRK_PIN_SWIZZLED_PAGES (1<<5)
1169	1169
1170	1170	struct intel_fbdev;
@@ -1454,6 +1454,13 @@ struct i915_gem_mm { · 283 unmodified lines
1454	1454	/** LRU list of objects with fence regs on them. */
1455	1455	struct list_head fence_list;
1456	1456
	1457	+ /**
	1458	+ * Workqueue to fault in userptr pages, flushed by the execbuf
	1459	+ * when required but otherwise left to userspace to try again
	1460	+ * on EAGAIN.
	1461	+ */
	1462	+ struct workqueue_struct *userptr_wq;
	1463	+
1457	1464	u64 unordered_timeline;
1458	1465
1459	1466	/* the indicator for dispatch video commands on two BSD rings */
@@ -2017,9 +2024,17 @@ struct i915_oa_ops { · 557 unmodified lines
2017	2024	void (init_oa_buffer)(struct drm_i915_private dev_priv);
2018	2025
2019	2026	/**
2020		− * @enable_metric_set: Applies any MUX configuration to set up the
2021		− * Boolean and Custom (B/C) counters that are part of the counter
2022		− * reports being sampled. May apply system constraints such as
	2027	+ * @select_metric_set: The auto generated code that checks whether a
	2028	+ * requested OA config is applicable to the system and if so sets up
	2029	+ * the mux, oa and flex eu register config pointers according to the
	2030	+ * current dev_priv->perf.oa.metrics_set.
	2031	+ */
	2032	+ int (select_metric_set)(struct drm_i915_private dev_priv);
	2033	+
	2034	+ /**
	2035	+ * @enable_metric_set: Selects and applies any MUX configuration to set
	2036	+ * up the Boolean and Custom (B/C) counters that are part of the
	2037	+ * counter reports being sampled. May apply system constraints such as
2023	2038	* disabling EU clock gating as required.
2024	2039	*/
2025	2040	int (enable_metric_set)(struct drm_i915_private dev_priv);
@@ -2050,20 +2065,13 @@ struct i915_oa_ops { · 24 unmodified lines
2050	2065	size_t *offset);
2051	2066
2052	2067	/**
2053		− * @oa_buffer_check: Check for OA buffer data + update tail
2054		− *
2055		− * This is either called via fops or the poll check hrtimer (atomic
2056		− * ctx) without any locks taken.
	2068	+ * @oa_hw_tail_read: read the OA tail pointer register
2057	2069	*
2058		− * It's safe to read OA config state here unlocked, assuming that this
2059		− * is only called while the stream is enabled, while the global OA
2060		− * configuration can't be modified.
2061		− *
2062		− * Efficiency is more important than avoiding some false positives
2063		− * here, which will be handled gracefully - likely resulting in an
2064		− * %EAGAIN error for userspace.
	2070	+ * In particular this enables us to share all the fiddly code for
	2071	+ * handling the OA unit tail pointer race that affects multiple
	2072	+ * generations.
2065	2073	*/
2066		− bool (oa_buffer_check)(struct drm_i915_private dev_priv);
	2074	+ u32 (oa_hw_tail_read)(struct drm_i915_private dev_priv);
2067	2075	};
2068	2076
2069	2077	struct intel_cdclk_state {
@@ -2394,8 +2402,6 @@ struct drm_i915_private { · 324 unmodified lines
2394	2402	struct mutex lock;
2395	2403	struct list_head streams;
2396	2404
2397		− spinlock_t hook_lock;
2398		−
2399	2405	struct {
2400	2406	struct i915_perf_stream *exclusive_stream;
2401	2407
@@ -2413,17 +2419,23 @@ struct drm_i915_private { · 11 unmodified lines
2413	2419
2414	2420	bool periodic;
2415	2421	int period_exponent;
	2422	+ int timestamp_frequency;
2416	2423
2417	2424	int metrics_set;
2418	2425
2419		− const struct i915_oa_reg *mux_regs;
2420		− int mux_regs_len;
	2426	+ const struct i915_oa_reg *mux_regs[6];
	2427	+ int mux_regs_lens[6];
	2428	+ int n_mux_configs;
	2429	+
2421	2430	const struct i915_oa_reg *b_counter_regs;
2422	2431	int b_counter_regs_len;
	2432	+ const struct i915_oa_reg *flex_regs;
	2433	+ int flex_regs_len;
2423	2434
2424	2435	struct {
2425	2436	struct i915_vma *vma;
2426	2437	u8 *vaddr;
	2438	+ u32 last_ctx_id;
2427	2439	int format;
2428	2440	int format_size;
2429	2441
@@ -2493,6 +2505,15 @@ struct drm_i915_private { · 63 unmodified lines
2493	2505	} oa_buffer;
2494	2506
2495	2507	u32 gen7_latched_oastatus1;
	2508	+ u32 ctx_oactxctrl_offset;
	2509	+ u32 ctx_flexeu0_offset;
	2510	+
	2511	+ /**
	2512	+ * The RPT_ID/reason field for Gen8+ includes a bit
	2513	+ * to determine if the CTX ID in the report is valid
	2514	+ * but the specific bit differs between Gen 8 and 9
	2515	+ */
	2516	+ u32 gen8_valid_ctx_bit;
2496	2517
2497	2518	struct i915_oa_ops ops;
2498	2519	const struct i915_oa_format *oa_formats;
@@ -2768,6 +2789,8 @@ intel_info(const struct drm_i915_private *dev_priv) · 269 unmodified lines
2768	2789	#define IS_BROXTON(dev_priv) ((dev_priv)->info.platform == INTEL_BROXTON)
2769	2790	#define IS_KABYLAKE(dev_priv) ((dev_priv)->info.platform == INTEL_KABYLAKE)
2770	2791	#define IS_GEMINILAKE(dev_priv) ((dev_priv)->info.platform == INTEL_GEMINILAKE)
	2792	+#define IS_COFFEELAKE(dev_priv) ((dev_priv)->info.platform == INTEL_COFFEELAKE)
	2793	+#define IS_CANNONLAKE(dev_priv) ((dev_priv)->info.platform == INTEL_CANNONLAKE)
2771	2794	#define IS_MOBILE(dev_priv) ((dev_priv)->info.is_mobile)
2772	2795	#define IS_HSW_EARLY_SDV(dev_priv) (IS_HASWELL(dev_priv) && \
2773	2796	(INTEL_DEVID(dev_priv) & 0xFF00) == 0x0C00)
@@ -2803,10 +2826,18 @@ intel_info(const struct drm_i915_private *dev_priv) · 29 unmodified lines
2803	2826	#define IS_KBL_ULX(dev_priv) (INTEL_DEVID(dev_priv) == 0x590E \|\| \
2804	2827	INTEL_DEVID(dev_priv) == 0x5915 \|\| \
2805	2828	INTEL_DEVID(dev_priv) == 0x591E)
	2829	+#define IS_SKL_GT2(dev_priv) (IS_SKYLAKE(dev_priv) && \
	2830	+ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)
2806	2831	#define IS_SKL_GT3(dev_priv) (IS_SKYLAKE(dev_priv) && \
2807	2832	(INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)
2808	2833	#define IS_SKL_GT4(dev_priv) (IS_SKYLAKE(dev_priv) && \
2809	2834	(INTEL_DEVID(dev_priv) & 0x00F0) == 0x0030)
	2835	+#define IS_KBL_GT2(dev_priv) (IS_KABYLAKE(dev_priv) && \
	2836	+ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0010)
	2837	+#define IS_KBL_GT3(dev_priv) (IS_KABYLAKE(dev_priv) && \
	2838	+ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x0020)
	2839	+#define IS_CFL_ULT(dev_priv) (IS_COFFEELAKE(dev_priv) && \
	2840	+ (INTEL_DEVID(dev_priv) & 0x00F0) == 0x00A0)
2810	2841
2811	2842	#define IS_ALPHA_SUPPORT(intel_info) ((intel_info)->is_alpha_support)
2812	2843
@@ -2845,6 +2876,12 @@ intel_info(const struct drm_i915_private *dev_priv) · 32 unmodified lines
2845	2876	#define IS_GLK_REVID(dev_priv, since, until) \
2846	2877	(IS_GEMINILAKE(dev_priv) && IS_REVID(dev_priv, since, until))
2847	2878
	2879	+#define CNL_REVID_A0 0x0
	2880	+#define CNL_REVID_B0 0x1
	2881	+
	2882	+#define IS_CNL_REVID(p, since, until) \
	2883	+ (IS_CANNONLAKE(p) && IS_REVID(p, since, until))
	2884	+
2848	2885	/*
2849	2886	* The genX designation typically refers to the render engine, so render
2850	2887	* capability related checks should use IS_GEN, while display and other checks
@@ -2859,6 +2896,7 @@ intel_info(const struct drm_i915_private *dev_priv) · 8 unmodified lines
2859	2896	#define IS_GEN7(dev_priv) (!!((dev_priv)->info.gen_mask & BIT(6)))
2860	2897	#define IS_GEN8(dev_priv) (!!((dev_priv)->info.gen_mask & BIT(7)))
2861	2898	#define IS_GEN9(dev_priv) (!!((dev_priv)->info.gen_mask & BIT(8)))
	2899	+#define IS_GEN10(dev_priv) (!!((dev_priv)->info.gen_mask & BIT(9)))
2862	2900
2863	2901	#define IS_LP(dev_priv) (INTEL_INFO(dev_priv)->is_lp)
2864	2902	#define IS_GEN9_LP(dev_priv) (IS_GEN9(dev_priv) && IS_LP(dev_priv))
@@ -2959,6 +2997,7 @@ intel_info(const struct drm_i915_private *dev_priv) · 94 unmodified lines
2959	2997	#define HAS_POOLED_EU(dev_priv) ((dev_priv)->info.has_pooled_eu)
2960	2998
2961	2999	#define INTEL_PCH_DEVICE_ID_MASK 0xff00
	3000	+#define INTEL_PCH_DEVICE_ID_MASK_EXT 0xff80
2962	3001	#define INTEL_PCH_IBX_DEVICE_ID_TYPE 0x3b00
2963	3002	#define INTEL_PCH_CPT_DEVICE_ID_TYPE 0x1c00
2964	3003	#define INTEL_PCH_PPT_DEVICE_ID_TYPE 0x1e00
@@ -2967,11 +3006,16 @@ intel_info(const struct drm_i915_private *dev_priv) · 2 unmodified lines
2967	3006	#define INTEL_PCH_SPT_DEVICE_ID_TYPE 0xA100
2968	3007	#define INTEL_PCH_SPT_LP_DEVICE_ID_TYPE 0x9D00
2969	3008	#define INTEL_PCH_KBP_DEVICE_ID_TYPE 0xA200
	3009	+#define INTEL_PCH_CNP_DEVICE_ID_TYPE 0xA300
	3010	+#define INTEL_PCH_CNP_LP_DEVICE_ID_TYPE 0x9D80
2970	3011	#define INTEL_PCH_P2X_DEVICE_ID_TYPE 0x7100
2971	3012	#define INTEL_PCH_P3X_DEVICE_ID_TYPE 0x7000
2972	3013	#define INTEL_PCH_QEMU_DEVICE_ID_TYPE 0x2900 /* qemu q35 has 2918 */
2973	3014
2974	3015	#define INTEL_PCH_TYPE(dev_priv) ((dev_priv)->pch_type)
	3016	+#define HAS_PCH_CNP(dev_priv) (INTEL_PCH_TYPE(dev_priv) == PCH_CNP)
	3017	+#define HAS_PCH_CNP_LP(dev_priv) \
	3018	+ ((dev_priv)->pch_id == INTEL_PCH_CNP_LP_DEVICE_ID_TYPE)
2975	3019	#define HAS_PCH_KBP(dev_priv) (INTEL_PCH_TYPE(dev_priv) == PCH_KBP)
2976	3020	#define HAS_PCH_SPT(dev_priv) (INTEL_PCH_TYPE(dev_priv) == PCH_SPT)
2977	3021	#define HAS_PCH_LPT(dev_priv) (INTEL_PCH_TYPE(dev_priv) == PCH_LPT)
@@ -2986,7 +3030,7 @@ intel_info(const struct drm_i915_private *dev_priv) · 8 unmodified lines
2986	3030
2987	3031	#define HAS_GMCH_DISPLAY(dev_priv) ((dev_priv)->info.has_gmch_display)
2988	3032
2989		−#define HAS_LSPCON(dev_priv) (IS_GEN9(dev_priv))
	3033	+#define HAS_LSPCON(dev_priv) (INTEL_GEN(dev_priv) >= 9)
2990	3034
2991	3035	/* DPF == dynamic parity feature */
2992	3036	#define HAS_L3_DPF(dev_priv) ((dev_priv)->info.has_l3_dpf)
@@ -2996,8 +3040,6 @@ intel_info(const struct drm_i915_private *dev_priv) · 3 unmodified lines
2996	3040	#define GT_FREQUENCY_MULTIPLIER 50
2997	3041	#define GEN9_FREQ_SCALER 3
2998	3042
2999		−#define HAS_DECOUPLED_MMIO(dev_priv) (INTEL_INFO(dev_priv)->has_decoupled_mmio)
3000		−
3001	3043	#include "i915_trace.h"
3002	3044
3003	3045	static inline bool intel_vtd_active(void)
@@ -3194,7 +3236,8 @@ int i915_gem_set_tiling_ioctl(struct drm_device dev, void data, · 190 unmodified lines
3194	3236	struct drm_file *file_priv);
3195	3237	int i915_gem_get_tiling_ioctl(struct drm_device dev, void data,
3196	3238	struct drm_file *file_priv);
3197		−void i915_gem_init_userptr(struct drm_i915_private *dev_priv);
	3239	+int i915_gem_init_userptr(struct drm_i915_private *dev_priv);
	3240	+void i915_gem_cleanup_userptr(struct drm_i915_private *dev_priv);
3198	3241	int i915_gem_userptr_ioctl(struct drm_device dev, void data,
3199	3242	struct drm_file *file);
3200	3243	int i915_gem_get_aperture_ioctl(struct drm_device dev, void data,
@@ -3534,6 +3577,9 @@ i915_gem_context_lookup_timeline(struct i915_gem_context *ctx, · 333 unmodified lines
3534	3577
3535	3578	int i915_perf_open_ioctl(struct drm_device dev, void data,
3536	3579	struct drm_file *file);
	3580	+void i915_oa_init_reg_state(struct intel_engine_cs *engine,
	3581	+ struct i915_gem_context *ctx,
	3582	+ uint32_t *reg_state);
3537	3583
3538	3584	/* i915_gem_evict.c */
3539	3585	int __must_check i915_gem_evict_something(struct i915_address_space *vm,
@@ -3544,7 +3590,7 @@ int __must_check i915_gem_evict_something(struct i915_address_space *vm, · 4 unmodified lines
3544	3590	int __must_check i915_gem_evict_for_node(struct i915_address_space *vm,
3545	3591	struct drm_mm_node *node,
3546	3592	unsigned int flags);
3547		−int i915_gem_evict_vm(struct i915_address_space *vm, bool do_idle);
	3593	+int i915_gem_evict_vm(struct i915_address_space *vm);
3548	3594
3549	3595	/* belongs in i915_gem_gtt.h */
3550	3596	static inline void i915_gem_chipset_flush(struct drm_i915_private *dev_priv)

drivers/gpu/drm/i915/i915_gem.c+102−62

@@ -49,10 +49,10 @@ static void i915_gem_flush_free_objects(struct drm_i915_private *i915); · 48 unmodified lines
49	49
50	50	static bool cpu_write_needs_clflush(struct drm_i915_gem_object *obj)
51	51	{
52		− if (obj->base.write_domain == I915_GEM_DOMAIN_CPU)
	52	+ if (obj->cache_dirty)
53	53	return false;
54	54
55		− if (!i915_gem_object_is_coherent(obj))
	55	+ if (!obj->cache_coherent)
56	56	return true;
57	57
58	58	return obj->pin_display;
@@ -143,9 +143,9 @@ i915_gem_get_aperture_ioctl(struct drm_device dev, void data, · 84 unmodified lines
143	143	struct i915_ggtt *ggtt = &dev_priv->ggtt;
144	144	struct drm_i915_gem_get_aperture *args = data;
145	145	struct i915_vma *vma;
146		− size_t pinned;
	146	+ u64 pinned;
147	147
148		− pinned = 0;
	148	+ pinned = ggtt->base.reserved;
149	149	mutex_lock(&dev->struct_mutex);
150	150	list_for_each_entry(vma, &ggtt->base.active_list, vm_link)
151	151	if (i915_vma_is_pinned(vma))
@@ -233,6 +233,14 @@ err_phys: · 81 unmodified lines
233	233	return st;
234	234	}
235	235
	236	+static void __start_cpu_write(struct drm_i915_gem_object *obj)
	237	+{
	238	+ obj->base.read_domains = I915_GEM_DOMAIN_CPU;
	239	+ obj->base.write_domain = I915_GEM_DOMAIN_CPU;
	240	+ if (cpu_write_needs_clflush(obj))
	241	+ obj->cache_dirty = true;
	242	+}
	243	+
236	244	static void
237	245	__i915_gem_object_release_shmem(struct drm_i915_gem_object *obj,
238	246	struct sg_table *pages,
@@ -245,11 +253,10 @@ __i915_gem_object_release_shmem(struct drm_i915_gem_object *obj, · 6 unmodified lines
245	253
246	254	if (needs_clflush &&
247	255	(obj->base.read_domains & I915_GEM_DOMAIN_CPU) == 0 &&
248		− !i915_gem_object_is_coherent(obj))
	256	+ !obj->cache_coherent)
249	257	drm_clflush_sg(pages);
250	258
251		− obj->base.read_domains = I915_GEM_DOMAIN_CPU;
252		− obj->base.write_domain = I915_GEM_DOMAIN_CPU;
	259	+ __start_cpu_write(obj);
253	260	}
254	261
255	262	static void
@@ -684,6 +691,12 @@ i915_gem_dumb_create(struct drm_file *file, · 428 unmodified lines
684	691	args->size, &args->handle);
685	692	}
686	693
	694	+static bool gpu_write_needs_clflush(struct drm_i915_gem_object *obj)
	695	+{
	696	+ return !(obj->cache_level == I915_CACHE_NONE \|\|
	697	+ obj->cache_level == I915_CACHE_WT);
	698	+}
	699	+
687	700	/**
688	701	* Creates a new mm object and returns a handle to it.
689	702	* @dev: drm device pointer
@@ -753,6 +766,11 @@ flush_write_domain(struct drm_i915_gem_object *obj, unsigned int flush_domains) · 63 unmodified lines
753	766	case I915_GEM_DOMAIN_CPU:
754	767	i915_gem_clflush_object(obj, I915_CLFLUSH_SYNC);
755	768	break;
	769	+
	770	+ case I915_GEM_DOMAIN_RENDER:
	771	+ if (gpu_write_needs_clflush(obj))
	772	+ obj->cache_dirty = true;
	773	+ break;
756	774	}
757	775
758	776	obj->base.write_domain = 0;
@@ -838,8 +856,7 @@ int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj, · 79 unmodified lines
838	856	if (ret)
839	857	return ret;
840	858
841		− if (i915_gem_object_is_coherent(obj) \|\|
842		− !static_cpu_has(X86_FEATURE_CLFLUSH)) {
	859	+ if (obj->cache_coherent \|\| !static_cpu_has(X86_FEATURE_CLFLUSH)) {
843	860	ret = i915_gem_object_set_to_cpu_domain(obj, false);
844	861	if (ret)
845	862	goto err_unpin;
@@ -854,7 +871,8 @@ int i915_gem_obj_prepare_shmem_read(struct drm_i915_gem_object *obj, · 8 unmodified lines
854	871	* optimizes for the case when the gpu will dirty the data
855	872	* anyway again before the next pread happens.
856	873	*/
857		− if (!(obj->base.read_domains & I915_GEM_DOMAIN_CPU))
	874	+ if (!obj->cache_dirty &&
	875	+ !(obj->base.read_domains & I915_GEM_DOMAIN_CPU))
858	876	*needs_clflush = CLFLUSH_BEFORE;
859	877
860	878	out:
@@ -890,8 +908,7 @@ int i915_gem_obj_prepare_shmem_write(struct drm_i915_gem_object *obj, · 29 unmodified lines
890	908	if (ret)
891	909	return ret;
892	910
893		− if (i915_gem_object_is_coherent(obj) \|\|
894		− !static_cpu_has(X86_FEATURE_CLFLUSH)) {
	911	+ if (obj->cache_coherent \|\| !static_cpu_has(X86_FEATURE_CLFLUSH)) {
895	912	ret = i915_gem_object_set_to_cpu_domain(obj, true);
896	913	if (ret)
897	914	goto err_unpin;
@@ -906,14 +923,16 @@ int i915_gem_obj_prepare_shmem_write(struct drm_i915_gem_object *obj, · 8 unmodified lines
906	923	* This optimizes for the case when the gpu will use the data
907	924	* right away and we therefore have to clflush anyway.
908	925	*/
909		− if (obj->base.write_domain != I915_GEM_DOMAIN_CPU)
	926	+ if (!obj->cache_dirty) {
910	927	*needs_clflush \|= CLFLUSH_AFTER;
911	928
912		− /* Same trick applies to invalidate partially written cachelines read
913		− * before writing.
914		− */
915		− if (!(obj->base.read_domains & I915_GEM_DOMAIN_CPU))
916		− *needs_clflush \|= CLFLUSH_BEFORE;
	929	+ /*
	930	+ * Same trick applies to invalidate partially written
	931	+ * cachelines read before writing.
	932	+ */
	933	+ if (!(obj->base.read_domains & I915_GEM_DOMAIN_CPU))
	934	+ *needs_clflush \|= CLFLUSH_BEFORE;
	935	+ }
917	936
918	937	out:
919	938	intel_fb_obj_invalidate(obj, ORIGIN_CPU);
@@ -2337,8 +2356,8 @@ i915_gem_object_get_pages_gtt(struct drm_i915_gem_object *obj) · 1417 unmodified lines
2337	2356	struct page *page;
2338	2357	unsigned long last_pfn = 0; /* suppress gcc warning */
2339	2358	unsigned int max_segment;
	2359	+ gfp_t noreclaim;
2340	2360	int ret;
2341		− gfp_t gfp;
2342	2361
2343	2362	/* Assert that the object is not currently in any GPU domain. As it
2344	2363	* wasn't in the GTT, there shouldn't be any way it could have been in
@@ -2367,22 +2386,30 @@ rebuild_st: · 22 unmodified lines
2367	2386	* Fail silently without starting the shrinker
2368	2387	*/
2369	2388	mapping = obj->base.filp->f_mapping;
2370		− gfp = mapping_gfp_constraint(mapping, ~(__GFP_IO \| __GFP_RECLAIM));
2371		− gfp \|= __GFP_NORETRY \| __GFP_NOWARN;
	2389	+ noreclaim = mapping_gfp_constraint(mapping, ~__GFP_RECLAIM);
	2390	+ noreclaim \|= __GFP_NORETRY \| __GFP_NOWARN;
	2391	+
2372	2392	sg = st->sgl;
2373	2393	st->nents = 0;
2374	2394	for (i = 0; i < page_count; i++) {
2375		− page = shmem_read_mapping_page_gfp(mapping, i, gfp);
2376		− if (unlikely(IS_ERR(page))) {
2377		− i915_gem_shrink(dev_priv,
2378		− page_count,
2379		− I915_SHRINK_BOUND \|
2380		− I915_SHRINK_UNBOUND \|
2381		− I915_SHRINK_PURGEABLE);
	2395	+ const unsigned int shrink[] = {
	2396	+ I915_SHRINK_BOUND \| I915_SHRINK_UNBOUND \| I915_SHRINK_PURGEABLE,
	2397	+ 0,
	2398	+ }, *s = shrink;
	2399	+ gfp_t gfp = noreclaim;
	2400	+
	2401	+ do {
2382	2402	page = shmem_read_mapping_page_gfp(mapping, i, gfp);
2383		− }
2384		− if (unlikely(IS_ERR(page))) {
2385		− gfp_t reclaim;
	2403	+ if (likely(!IS_ERR(page)))
	2404	+ break;
	2405	+
	2406	+ if (!*s) {
	2407	+ ret = PTR_ERR(page);
	2408	+ goto err_sg;
	2409	+ }
	2410	+
	2411	+ i915_gem_shrink(dev_priv, 2 * page_count, *s++);
	2412	+ cond_resched();
2386	2413
2387	2414	/* We've tried hard to allocate the memory by reaping
2388	2415	* our own buffer, now let the real VM do its job and
@@ -2392,15 +2419,26 @@ rebuild_st: · 3 unmodified lines
2392	2419	* defer the oom here by reporting the ENOMEM back
2393	2420	* to userspace.
2394	2421	*/
2395		− reclaim = mapping_gfp_mask(mapping);
2396		− reclaim \|= __GFP_NORETRY; /* reclaim, but no oom */
2397		−
2398		− page = shmem_read_mapping_page_gfp(mapping, i, reclaim);
2399		− if (IS_ERR(page)) {
2400		− ret = PTR_ERR(page);
2401		− goto err_sg;
	2422	+ if (!*s) {
	2423	+ /* reclaim and warn, but no oom */
	2424	+ gfp = mapping_gfp_mask(mapping);
	2425	+
	2426	+ /* Our bo are always dirty and so we require
	2427	+ * kswapd to reclaim our pages (direct reclaim
	2428	+ * does not effectively begin pageout of our
	2429	+ * buffers on its own). However, direct reclaim
	2430	+ * only waits for kswapd when under allocation
	2431	+ * congestion. So as a result __GFP_RECLAIM is
	2432	+ * unreliable and fails to actually reclaim our
	2433	+ * dirty pages -- unless you try over and over
	2434	+ * again with !__GFP_NORETRY. However, we still
	2435	+ * want to fail this allocation rather than
	2436	+ * trigger the out-of-memory killer and for
	2437	+ * this we want the future __GFP_MAYFAIL.
	2438	+ */
2402	2439	}
2403		− }
	2440	+ } while (1);
	2441	+
2404	2442	if (!i \|\|
2405	2443	sg->length >= max_segment \|\|
2406	2444	page_to_pfn(page) != last_pfn + 1) {
@@ -3223,6 +3261,10 @@ void i915_gem_close_object(struct drm_gem_object gem, struct drm_file file) · 816 unmodified lines
3223	3261	if (vma->vm->file == fpriv)
3224	3262	i915_vma_close(vma);
3225	3263
	3264	+ vma = obj->vma_hashed;
	3265	+ if (vma && vma->ctx->file_priv == fpriv)
	3266	+ i915_vma_unlink_ctx(vma);
	3267	+
3226	3268	if (i915_gem_object_is_active(obj) &&
3227	3269	!i915_gem_object_has_active_reference(obj)) {
3228	3270	i915_gem_object_set_active_reference(obj);
@@ -3376,10 +3418,13 @@ int i915_gem_wait_for_idle(struct drm_i915_private *i915, unsigned int flags) · 147 unmodified lines
3376	3418
3377	3419	static void __i915_gem_object_flush_for_display(struct drm_i915_gem_object *obj)
3378	3420	{
3379		− if (obj->base.write_domain != I915_GEM_DOMAIN_CPU && !obj->cache_dirty)
3380		− return;
3381		−
3382		− i915_gem_clflush_object(obj, I915_CLFLUSH_FORCE);
	3421	+ /*
	3422	+ * We manually flush the CPU domain so that we can override and
	3423	+ * force the flush for the display, and perform it asyncrhonously.
	3424	+ */
	3425	+ flush_write_domain(obj, ~I915_GEM_DOMAIN_CPU);
	3426	+ if (obj->cache_dirty)
	3427	+ i915_gem_clflush_object(obj, I915_CLFLUSH_FORCE);
3383	3428	obj->base.write_domain = 0;
3384	3429	}
3385	3430
@@ -3638,13 +3683,11 @@ restart: · 252 unmodified lines
3638	3683	}
3639	3684	}
3640	3685
3641		− if (obj->base.write_domain == I915_GEM_DOMAIN_CPU &&
3642		− i915_gem_object_is_coherent(obj))
3643		− obj->cache_dirty = true;
3644		−
3645	3686	list_for_each_entry(vma, &obj->vma_list, obj_link)
3646	3687	vma->node.color = cache_level;
3647	3688	obj->cache_level = cache_level;
	3689	+ obj->cache_coherent = i915_gem_object_is_coherent(obj);
	3690	+ obj->cache_dirty = true; /* Always invalidate stale cachelines */
3648	3691
3649	3692	return 0;
3650	3693	}
@@ -3866,9 +3909,6 @@ i915_gem_object_set_to_cpu_domain(struct drm_i915_gem_object *obj, bool write) · 215 unmodified lines
3866	3909	if (ret)
3867	3910	return ret;
3868	3911
3869		− if (obj->base.write_domain == I915_GEM_DOMAIN_CPU)
3870		− return 0;
3871		−
3872	3912	flush_write_domain(obj, ~I915_GEM_DOMAIN_CPU);
3873	3913
3874	3914	/* Flush the CPU cache if it's still invalid. */
@@ -3880,15 +3920,13 @@ i915_gem_object_set_to_cpu_domain(struct drm_i915_gem_object *obj, bool write) · 5 unmodified lines
3880	3920	/* It should now be out of any other write domains, and we can update
3881	3921	* the domain values for our changes.
3882	3922	*/
3883		− GEM_BUG_ON((obj->base.write_domain & ~I915_GEM_DOMAIN_CPU) != 0);
	3923	+ GEM_BUG_ON(obj->base.write_domain & ~I915_GEM_DOMAIN_CPU);
3884	3924
3885	3925	/* If we're writing through the CPU, then the GPU read domains will
3886	3926	* need to be invalidated at next use.
3887	3927	*/
3888		− if (write) {
3889		− obj->base.read_domains = I915_GEM_DOMAIN_CPU;
3890		− obj->base.write_domain = I915_GEM_DOMAIN_CPU;
3891		− }
	3928	+ if (write)
	3929	+ __start_cpu_write(obj);
3892	3930
3893	3931	return 0;
3894	3932	}
@@ -4220,7 +4258,6 @@ void i915_gem_object_init(struct drm_i915_gem_object *obj, · 325 unmodified lines
4220	4258
4221	4259	INIT_LIST_HEAD(&obj->global_link);
4222	4260	INIT_LIST_HEAD(&obj->userfault_link);
4223		− INIT_LIST_HEAD(&obj->obj_exec_link);
4224	4261	INIT_LIST_HEAD(&obj->vma_list);
4225	4262	INIT_LIST_HEAD(&obj->batch_pool_link);
4226	4263
@@ -4285,6 +4322,7 @@ i915_gem_object_create(struct drm_i915_private *dev_priv, u64 size) · 58 unmodified lines
4285	4322
4286	4323	mapping = obj->base.filp->f_mapping;
4287	4324	mapping_set_gfp_mask(mapping, mask);
	4325	+ GEM_BUG_ON(!(mapping_gfp_mask(mapping) & __GFP_RECLAIM));
4288	4326
4289	4327	i915_gem_object_init(obj, &i915_gem_object_ops);
4290	4328
@@ -4308,6 +4346,9 @@ i915_gem_object_create(struct drm_i915_private *dev_priv, u64 size) · 17 unmodified lines
4308	4346	} else
4309	4347	obj->cache_level = I915_CACHE_NONE;
4310	4348
	4349	+ obj->cache_coherent = i915_gem_object_is_coherent(obj);
	4350	+ obj->cache_dirty = !obj->cache_coherent;
	4351	+
4311	4352	trace_i915_gem_object_create(obj);
4312	4353
4313	4354	return obj;
@@ -4356,7 +4397,6 @@ static void __i915_gem_free_objects(struct drm_i915_private *i915, · 42 unmodified lines
4356	4397	GEM_BUG_ON(i915_gem_object_is_active(obj));
4357	4398	list_for_each_entry_safe(vma, vn,
4358	4399	&obj->vma_list, obj_link) {
4359		− GEM_BUG_ON(!i915_vma_is_ggtt(vma));
4360	4400	GEM_BUG_ON(i915_vma_is_active(vma));
4361	4401	vma->flags &= ~I915_VMA_PIN_MASK;
4362	4402	i915_vma_close(vma);
@@ -4763,7 +4803,9 @@ int i915_gem_init(struct drm_i915_private *dev_priv) · 400 unmodified lines
4763	4803	*/
4764	4804	intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
4765	4805
4766		− i915_gem_init_userptr(dev_priv);
	4806	+ ret = i915_gem_init_userptr(dev_priv);
	4807	+ if (ret)
	4808	+ goto out_unlock;
4767	4809
4768	4810	ret = i915_gem_init_ggtt(dev_priv);
4769	4811	if (ret)
@@ -4974,10 +5016,8 @@ int i915_gem_freeze_late(struct drm_i915_private *dev_priv) · 204 unmodified lines
4974	5016
4975	5017	mutex_lock(&dev_priv->drm.struct_mutex);
4976	5018	for (p = phases; *p; p++) {
4977		− list_for_each_entry(obj, *p, global_link) {
4978		− obj->base.read_domains = I915_GEM_DOMAIN_CPU;
4979		− obj->base.write_domain = I915_GEM_DOMAIN_CPU;
4980		− }
	5019	+ list_for_each_entry(obj, *p, global_link)
	5020	+ __start_cpu_write(obj);
4981	5021	}
4982	5022	mutex_unlock(&dev_priv->drm.struct_mutex);
4983	5023

drivers/gpu/drm/i915/i915_gem_batch_pool.c+17−2

@@ -114,12 +114,27 @@ i915_gem_batch_pool_get(struct i915_gem_batch_pool *pool, · 113 unmodified lines
114	114	list_for_each_entry(obj, list, batch_pool_link) {
115	115	/* The batches are strictly LRU ordered */
116	116	if (i915_gem_object_is_active(obj)) {
117		− if (!reservation_object_test_signaled_rcu(obj->resv,
118		− true))
	117	+ struct reservation_object *resv = obj->resv;
	118	+
	119	+ if (!reservation_object_test_signaled_rcu(resv, true))
119	120	break;
120	121
121	122	i915_gem_retire_requests(pool->engine->i915);
122	123	GEM_BUG_ON(i915_gem_object_is_active(obj));
	124	+
	125	+ /*
	126	+ * The object is now idle, clear the array of shared
	127	+ * fences before we add a new request. Although, we
	128	+ * remain on the same engine, we may be on a different
	129	+ * timeline and so may continually grow the array,
	130	+ * trapping a reference to all the old fences, rather
	131	+ * than replace the existing fence.
	132	+ */
	133	+ if (rcu_access_pointer(resv->fence)) {
	134	+ reservation_object_lock(resv, NULL);
	135	+ reservation_object_add_excl_fence(resv, NULL);
	136	+ reservation_object_unlock(resv);
	137	+ }
123	138	}
124	139
125	140	GEM_BUG_ON(!reservation_object_test_signaled_rcu(obj->resv,

drivers/gpu/drm/i915/i915_gem_clflush.c+8−9

@@ -71,8 +71,6 @@ static const struct dma_fence_ops i915_clflush_ops = { · 70 unmodified lines
71	71	static void __i915_do_clflush(struct drm_i915_gem_object *obj)
72	72	{
73	73	drm_clflush_sg(obj->mm.pages);
74		− obj->cache_dirty = false;
75		−
76	74	intel_fb_obj_flush(obj, ORIGIN_CPU);
77	75	}
78	76
@@ -81,9 +79,6 @@ static void i915_clflush_work(struct work_struct *work) · 2 unmodified lines
81	79	struct clflush clflush = container_of(work, typeof(clflush), work);
82	80	struct drm_i915_gem_object *obj = clflush->obj;
83	81
84		− if (!obj->cache_dirty)
85		− goto out;
86		−
87	82	if (i915_gem_object_pin_pages(obj)) {
88	83	DRM_ERROR("Failed to acquire obj->pages for clflushing\n");
89	84	goto out;
@@ -131,10 +126,10 @@ void i915_gem_clflush_object(struct drm_i915_gem_object *obj, · 41 unmodified lines
131	126	* anything not backed by physical memory we consider to be always
132	127	* coherent and not need clflushing.
133	128	*/
134		− if (!i915_gem_object_has_struct_page(obj))
	129	+ if (!i915_gem_object_has_struct_page(obj)) {
	130	+ obj->cache_dirty = false;
135	131	return;
136		−
137		− obj->cache_dirty = true;
	132	+ }
138	133
139	134	/* If the GPU is snooping the contents of the CPU cache,
140	135	* we do not need to manually clear the CPU cache lines. However,
@@ -144,7 +139,7 @@ void i915_gem_clflush_object(struct drm_i915_gem_object *obj, · 3 unmodified lines
144	139	* snooping behaviour occurs naturally as the result of our domain
145	140	* tracking.
146	141	*/
147		− if (!(flags & I915_CLFLUSH_FORCE) && i915_gem_object_is_coherent(obj))
	142	+ if (!(flags & I915_CLFLUSH_FORCE) && obj->cache_coherent)
148	143	return;
149	144
150	145	trace_i915_gem_object_clflush(obj);
@@ -153,6 +148,8 @@ void i915_gem_clflush_object(struct drm_i915_gem_object *obj, · 2 unmodified lines
153	148	if (!(flags & I915_CLFLUSH_SYNC))
154	149	clflush = kmalloc(sizeof(*clflush), GFP_KERNEL);
155	150	if (clflush) {
	151	+ GEM_BUG_ON(!obj->cache_dirty);
	152	+
156	153	dma_fence_init(&clflush->dma,
157	154	&i915_clflush_ops,
158	155	&clflush_lock,
@@ -180,4 +177,6 @@ void i915_gem_clflush_object(struct drm_i915_gem_object *obj, · 21 unmodified lines
180	177	} else {
181	178	GEM_BUG_ON(obj->base.write_domain != I915_GEM_DOMAIN_CPU);
182	179	}
	180	+
	181	+ obj->cache_dirty = false;
183	182	}

drivers/gpu/drm/i915/i915_gem_context.c+82−4

@@ -85,6 +85,7 @@ · 84 unmodified lines
85	85	*
86	86	*/
87	87
	88	+#include <linux/log2.h>
88	89	#include <drm/drmP.h>
89	90	#include <drm/i915_drm.h>
90	91	#include "i915_drv.h"
@@ -92,6 +93,71 @@ · 1 unmodified line
92	93
93	94	#define ALL_L3_SLICES(dev) (1 << NUM_L3_SLICES(dev)) - 1
94	95
	96	+/* Initial size (as log2) to preallocate the handle->object hashtable */
	97	+#define VMA_HT_BITS 2u /* 4 x 2 pointers, 64 bytes minimum */
	98	+
	99	+static void resize_vma_ht(struct work_struct *work)
	100	+{
	101	+ struct i915_gem_context_vma_lut *lut =
	102	+ container_of(work, typeof(*lut), resize);
	103	+ unsigned int bits, new_bits, size, i;
	104	+ struct hlist_head *new_ht;
	105	+
	106	+ GEM_BUG_ON(!(lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS));
	107	+
	108	+ bits = 1 + ilog2(4*lut->ht_count/3 + 1);
	109	+ new_bits = min_t(unsigned int,
	110	+ max(bits, VMA_HT_BITS),
	111	+ sizeof(unsigned int) * BITS_PER_BYTE - 1);
	112	+ if (new_bits == lut->ht_bits)
	113	+ goto out;
	114	+
	115	+ new_ht = kzalloc(sizeof(*new_ht)<<new_bits, GFP_KERNEL \| __GFP_NOWARN);
	116	+ if (!new_ht)
	117	+ new_ht = vzalloc(sizeof(*new_ht)<<new_bits);
	118	+ if (!new_ht)
	119	+ /* Pretend resize succeeded and stop calling us for a bit! */
	120	+ goto out;
	121	+
	122	+ size = BIT(lut->ht_bits);
	123	+ for (i = 0; i < size; i++) {
	124	+ struct i915_vma *vma;
	125	+ struct hlist_node *tmp;
	126	+
	127	+ hlist_for_each_entry_safe(vma, tmp, &lut->ht[i], ctx_node)
	128	+ hlist_add_head(&vma->ctx_node,
	129	+ &new_ht[hash_32(vma->ctx_handle,
	130	+ new_bits)]);
	131	+ }
	132	+ kvfree(lut->ht);
	133	+ lut->ht = new_ht;
	134	+ lut->ht_bits = new_bits;
	135	+out:
	136	+ smp_store_release(&lut->ht_size, BIT(bits));
	137	+ GEM_BUG_ON(lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS);
	138	+}
	139	+
	140	+static void vma_lut_free(struct i915_gem_context *ctx)
	141	+{
	142	+ struct i915_gem_context_vma_lut *lut = &ctx->vma_lut;
	143	+ unsigned int i, size;
	144	+
	145	+ if (lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS)
	146	+ cancel_work_sync(&lut->resize);
	147	+
	148	+ size = BIT(lut->ht_bits);
	149	+ for (i = 0; i < size; i++) {
	150	+ struct i915_vma *vma;
	151	+
	152	+ hlist_for_each_entry(vma, &lut->ht[i], ctx_node) {
	153	+ vma->obj->vma_hashed = NULL;
	154	+ vma->ctx = NULL;
	155	+ i915_vma_put(vma);
	156	+ }
	157	+ }
	158	+ kvfree(lut->ht);
	159	+}
	160	+
95	161	void i915_gem_context_free(struct kref *ctx_ref)
96	162	{
97	163	struct i915_gem_context ctx = container_of(ctx_ref, typeof(ctx), ref);
@@ -101,6 +167,7 @@ void i915_gem_context_free(struct kref *ctx_ref) · 3 unmodified lines
101	167	trace_i915_context_free(ctx);
102	168	GEM_BUG_ON(!i915_gem_context_is_closed(ctx));
103	169
	170	+ vma_lut_free(ctx);
104	171	i915_ppgtt_put(ctx->ppgtt);
105	172
106	173	for (i = 0; i < I915_NUM_ENGINES; i++) {
@@ -118,6 +185,7 @@ void i915_gem_context_free(struct kref *ctx_ref) · 11 unmodified lines
118	185
119	186	kfree(ctx->name);
120	187	put_pid(ctx->pid);
	188	+
121	189	list_del(&ctx->link);
122	190
123	191	ida_simple_remove(&ctx->i915->context_hw_ida, ctx->hw_id);
@@ -201,13 +269,24 @@ __create_hw_context(struct drm_i915_private *dev_priv, · 77 unmodified lines
201	269	ctx->i915 = dev_priv;
202	270	ctx->priority = I915_PRIORITY_NORMAL;
203	271
	272	+ ctx->vma_lut.ht_bits = VMA_HT_BITS;
	273	+ ctx->vma_lut.ht_size = BIT(VMA_HT_BITS);
	274	+ BUILD_BUG_ON(BIT(VMA_HT_BITS) == I915_CTX_RESIZE_IN_PROGRESS);
	275	+ ctx->vma_lut.ht = kcalloc(ctx->vma_lut.ht_size,
	276	+ sizeof(*ctx->vma_lut.ht),
	277	+ GFP_KERNEL);
	278	+ if (!ctx->vma_lut.ht)
	279	+ goto err_out;
	280	+
	281	+ INIT_WORK(&ctx->vma_lut.resize, resize_vma_ht);
	282	+
204	283	/* Default context will never have a file_priv */
205	284	ret = DEFAULT_CONTEXT_HANDLE;
206	285	if (file_priv) {
207	286	ret = idr_alloc(&file_priv->context_idr, ctx,
208	287	DEFAULT_CONTEXT_HANDLE, 0, GFP_KERNEL);
209	288	if (ret < 0)
210		− goto err_out;
	289	+ goto err_lut;
211	290	}
212	291	ctx->user_handle = ret;
213	292
@@ -248,6 +327,8 @@ __create_hw_context(struct drm_i915_private *dev_priv, · 34 unmodified lines
248	327	err_pid:
249	328	put_pid(ctx->pid);
250	329	idr_remove(&file_priv->context_idr, ctx->user_handle);
	330	+err_lut:
	331	+ kvfree(ctx->vma_lut.ht);
251	332	err_out:
252	333	context_close(ctx);
253	334	return ERR_PTR(ret);
@@ -1034,9 +1115,6 @@ int i915_gem_context_reset_stats_ioctl(struct drm_device *dev, · 780 unmodified lines
1034	1115	if (args->flags \|\| args->pad)
1035	1116	return -EINVAL;
1036	1117
1037		− if (args->ctx_id == DEFAULT_CONTEXT_HANDLE && !capable(CAP_SYS_ADMIN))
1038		− return -EPERM;
1039		−
1040	1118	ret = i915_mutex_lock_interruptible(dev);
1041	1119	if (ret)
1042	1120	return ret;

drivers/gpu/drm/i915/i915_gem_context.h+26−0

@@ -143,6 +143,32 @@ struct i915_gem_context { · 142 unmodified lines
143	143	/** ggtt_offset_bias: placement restriction for context objects */
144	144	u32 ggtt_offset_bias;
145	145
	146	+ struct i915_gem_context_vma_lut {
	147	+ /** ht_size: last request size to allocate the hashtable for. */
	148	+ unsigned int ht_size;
	149	+#define I915_CTX_RESIZE_IN_PROGRESS BIT(0)
	150	+ /** ht_bits: real log2(size) of hashtable. */
	151	+ unsigned int ht_bits;
	152	+ /** ht_count: current number of entries inside the hashtable */
	153	+ unsigned int ht_count;
	154	+
	155	+ /** ht: the array of buckets comprising the simple hashtable */
	156	+ struct hlist_head *ht;
	157	+
	158	+ /**
	159	+ * resize: After an execbuf completes, we check the load factor
	160	+ * of the hashtable. If the hashtable is too full, or too empty,
	161	+ * we schedule a task to resize the hashtable. During the
	162	+ * resize, the entries are moved between different buckets and
	163	+ * so we cannot simultaneously read the hashtable as it is
	164	+ * being resized (unlike rhashtable). Therefore we treat the
	165	+ * active work as a strong barrier, pausing a subsequent
	166	+ * execbuf to wait for the resize worker to complete, if
	167	+ * required.
	168	+ */
	169	+ struct work_struct resize;
	170	+ } vma_lut;
	171	+
146	172	/** engine: per-engine logical HW state */
147	173	struct intel_context {
148	174	struct i915_vma *state;

drivers/gpu/drm/i915/i915_gem_evict.c+65−54

@@ -50,6 +50,29 @@ static bool ggtt_is_idle(struct drm_i915_private *dev_priv) · 49 unmodified lines
50	50	return true;
51	51	}
52	52
	53	+static int ggtt_flush(struct drm_i915_private *i915)
	54	+{
	55	+ int err;
	56	+
	57	+ /* Not everything in the GGTT is tracked via vma (otherwise we
	58	+ * could evict as required with minimal stalling) so we are forced
	59	+ * to idle the GPU and explicitly retire outstanding requests in
	60	+ * the hopes that we can then remove contexts and the like only
	61	+ * bound by their active reference.
	62	+ */
	63	+ err = i915_gem_switch_to_kernel_context(i915);
	64	+ if (err)
	65	+ return err;
	66	+
	67	+ err = i915_gem_wait_for_idle(i915,
	68	+ I915_WAIT_INTERRUPTIBLE \|
	69	+ I915_WAIT_LOCKED);
	70	+ if (err)
	71	+ return err;
	72	+
	73	+ return 0;
	74	+}
	75	+
53	76	static bool
54	77	mark_free(struct drm_mm_scan *scan,
55	78	struct i915_vma *vma,
@@ -59,13 +82,10 @@ mark_free(struct drm_mm_scan *scan, · 3 unmodified lines
59	82	if (i915_vma_is_pinned(vma))
60	83	return false;
61	84
62		− if (WARN_ON(!list_empty(&vma->exec_list)))
63		− return false;
64		−
65	85	if (flags & PIN_NONFAULT && !list_empty(&vma->obj->userfault_link))
66	86	return false;
67	87
68		− list_add(&vma->exec_list, unwind);
	88	+ list_add(&vma->evict_link, unwind);
69	89	return drm_mm_scan_add_block(scan, &vma->node);
70	90	}
71	91
@@ -157,11 +177,9 @@ search_again: · 85 unmodified lines
157	177	} while (*++phase);
158	178
159	179	/* Nothing found, clean up and bail out! */
160		− list_for_each_entry_safe(vma, next, &eviction_list, exec_list) {
	180	+ list_for_each_entry_safe(vma, next, &eviction_list, evict_link) {
161	181	ret = drm_mm_scan_remove_block(&scan, &vma->node);
162	182	BUG_ON(ret);
163		−
164		− INIT_LIST_HEAD(&vma->exec_list);
165	183	}
166	184
167	185	/* Can we unpin some objects such as idle hw contents,
@@ -180,19 +198,7 @@ search_again: · 12 unmodified lines
180	198	return intel_has_pending_fb_unpin(dev_priv) ? -EAGAIN : -ENOSPC;
181	199	}
182	200
183		− /* Not everything in the GGTT is tracked via vma (otherwise we
184		− * could evict as required with minimal stalling) so we are forced
185		− * to idle the GPU and explicitly retire outstanding requests in
186		− * the hopes that we can then remove contexts and the like only
187		− * bound by their active reference.
188		− */
189		− ret = i915_gem_switch_to_kernel_context(dev_priv);
190		− if (ret)
191		− return ret;
192		−
193		− ret = i915_gem_wait_for_idle(dev_priv,
194		− I915_WAIT_INTERRUPTIBLE \|
195		− I915_WAIT_LOCKED);
	201	+ ret = ggtt_flush(dev_priv);
196	202	if (ret)
197	203	return ret;
198	204
@@ -205,21 +211,16 @@ found: · 6 unmodified lines
205	211	* calling unbind (which may remove the active reference
206	212	* of any of our objects, thus corrupting the list).
207	213	*/
208		− list_for_each_entry_safe(vma, next, &eviction_list, exec_list) {
	214	+ list_for_each_entry_safe(vma, next, &eviction_list, evict_link) {
209	215	if (drm_mm_scan_remove_block(&scan, &vma->node))
210	216	__i915_vma_pin(vma);
211	217	else
212		− list_del_init(&vma->exec_list);
	218	+ list_del(&vma->evict_link);
213	219	}
214	220
215	221	/* Unbinding will emit any required flushes */
216	222	ret = 0;
217		− while (!list_empty(&eviction_list)) {
218		− vma = list_first_entry(&eviction_list,
219		− struct i915_vma,
220		− exec_list);
221		−
222		− list_del_init(&vma->exec_list);
	223	+ list_for_each_entry_safe(vma, next, &eviction_list, evict_link) {
223	224	__i915_vma_unpin(vma);
224	225	if (ret == 0)
225	226	ret = i915_vma_unbind(vma);
@@ -315,7 +316,7 @@ int i915_gem_evict_for_node(struct i915_address_space *vm, · 89 unmodified lines
315	316	}
316	317
317	318	/* Overlap of objects in the same batch? */
318		− if (i915_vma_is_pinned(vma) \|\| !list_empty(&vma->exec_list)) {
	319	+ if (i915_vma_is_pinned(vma)) {
319	320	ret = -ENOSPC;
320	321	if (vma->exec_entry &&
321	322	vma->exec_entry->flags & EXEC_OBJECT_PINNED)
@@ -332,11 +333,10 @@ int i915_gem_evict_for_node(struct i915_address_space *vm, · 10 unmodified lines
332	333	* reference) another in our eviction list.
333	334	*/
334	335	__i915_vma_pin(vma);
335		− list_add(&vma->exec_list, &eviction_list);
	336	+ list_add(&vma->evict_link, &eviction_list);
336	337	}
337	338
338		− list_for_each_entry_safe(vma, next, &eviction_list, exec_list) {
339		− list_del_init(&vma->exec_list);
	339	+ list_for_each_entry_safe(vma, next, &eviction_list, evict_link) {
340	340	__i915_vma_unpin(vma);
341	341	if (ret == 0)
342	342	ret = i915_vma_unbind(vma);
@@ -348,10 +348,8 @@ int i915_gem_evict_for_node(struct i915_address_space *vm, · 5 unmodified lines
348	348	/**
349	349	* i915_gem_evict_vm - Evict all idle vmas from a vm
350	350	* @vm: Address space to cleanse
351		− * @do_idle: Boolean directing whether to idle first.
352	351	*
353		− * This function evicts all idles vmas from a vm. If all unpinned vmas should be
354		− * evicted the @do_idle needs to be set to true.
	352	+ * This function evicts all vmas from a vm.
355	353	*
356	354	* This is used by the execbuf code as a last-ditch effort to defragment the
357	355	* address space.
@@ -359,37 +357,50 @@ int i915_gem_evict_for_node(struct i915_address_space *vm, · 1 unmodified line
359	357	* To clarify: This is for freeing up virtual address space, not for freeing
360	358	* memory in e.g. the shrinker.
361	359	*/
362		−int i915_gem_evict_vm(struct i915_address_space *vm, bool do_idle)
	360	+int i915_gem_evict_vm(struct i915_address_space *vm)
363	361	{
	362	+ struct list_head *phases[] = {
	363	+ &vm->inactive_list,
	364	+ &vm->active_list,
	365	+ NULL
	366	+ }, **phase;
	367	+ struct list_head eviction_list;
364	368	struct i915_vma vma, next;
365	369	int ret;
366	370
367	371	lockdep_assert_held(&vm->i915->drm.struct_mutex);
368	372	trace_i915_gem_evict_vm(vm);
369	373
370		− if (do_idle) {
371		− struct drm_i915_private *dev_priv = vm->i915;
372		−
373		− if (i915_is_ggtt(vm)) {
374		− ret = i915_gem_switch_to_kernel_context(dev_priv);
375		− if (ret)
376		− return ret;
377		− }
378		−
379		− ret = i915_gem_wait_for_idle(dev_priv,
380		− I915_WAIT_INTERRUPTIBLE \|
381		− I915_WAIT_LOCKED);
	374	+ /* Switch back to the default context in order to unpin
	375	+ * the existing context objects. However, such objects only
	376	+ * pin themselves inside the global GTT and performing the
	377	+ * switch otherwise is ineffective.
	378	+ */
	379	+ if (i915_is_ggtt(vm)) {
	380	+ ret = ggtt_flush(vm->i915);
382	381	if (ret)
383	382	return ret;
384		−
385		− WARN_ON(!list_empty(&vm->active_list));
386	383	}
387	384
388		− list_for_each_entry_safe(vma, next, &vm->inactive_list, vm_link)
389		− if (!i915_vma_is_pinned(vma))
390		− WARN_ON(i915_vma_unbind(vma));
	385	+ INIT_LIST_HEAD(&eviction_list);
	386	+ phase = phases;
	387	+ do {
	388	+ list_for_each_entry(vma, *phase, vm_link) {
	389	+ if (i915_vma_is_pinned(vma))
	390	+ continue;
391	391
392		− return 0;
	392	+ __i915_vma_pin(vma);
	393	+ list_add(&vma->evict_link, &eviction_list);
	394	+ }
	395	+ } while (*++phase);
	396	+
	397	+ ret = 0;
	398	+ list_for_each_entry_safe(vma, next, &eviction_list, evict_link) {
	399	+ __i915_vma_unpin(vma);
	400	+ if (ret == 0)
	401	+ ret = i915_vma_unbind(vma);
	402	+ }
	403	+ return ret;
393	404	}
394	405
395	406	#if IS_ENABLED(CONFIG_DRM_I915_SELFTEST)

drivers/gpu/drm/i915/i915_gem_execbuffer.c+1631−1123

@@ -40,143 +40,726 @@ · 39 unmodified lines
40	40	#include "intel_drv.h"
41	41	#include "intel_frontbuffer.h"
42	42
43		−#define DBG_USE_CPU_RELOC 0 /* -1 force GTT relocs; 1 force CPU relocs */
	43	+enum {
	44	+ FORCE_CPU_RELOC = 1,
	45	+ FORCE_GTT_RELOC,
	46	+ FORCE_GPU_RELOC,
	47	+#define DBG_FORCE_RELOC 0 /* choose one of the above! */
	48	+};
	49	+
	50	+#define __EXEC_OBJECT_HAS_REF BIT(31)
	51	+#define __EXEC_OBJECT_HAS_PIN BIT(30)
	52	+#define __EXEC_OBJECT_HAS_FENCE BIT(29)
	53	+#define __EXEC_OBJECT_NEEDS_MAP BIT(28)
	54	+#define __EXEC_OBJECT_NEEDS_BIAS BIT(27)
	55	+#define __EXEC_OBJECT_INTERNAL_FLAGS (~0u << 27) /* all of the above */
	56	+#define __EXEC_OBJECT_RESERVED (__EXEC_OBJECT_HAS_PIN \| __EXEC_OBJECT_HAS_FENCE)
44	57
45		−#define __EXEC_OBJECT_HAS_PIN (1<<31)
46		−#define __EXEC_OBJECT_HAS_FENCE (1<<30)
47		−#define __EXEC_OBJECT_NEEDS_MAP (1<<29)
48		−#define __EXEC_OBJECT_NEEDS_BIAS (1<<28)
49		−#define __EXEC_OBJECT_INTERNAL_FLAGS (0xf<<28) /* all of the above */
	58	+#define __EXEC_HAS_RELOC BIT(31)
	59	+#define __EXEC_VALIDATED BIT(30)
	60	+#define UPDATE PIN_OFFSET_FIXED
50	61
51	62	#define BATCH_OFFSET_BIAS (256*1024)
52	63
53		−struct i915_execbuffer_params {
54		− struct drm_device *dev;
55		− struct drm_file *file;
56		− struct i915_vma *batch;
57		− u32 dispatch_flags;
58		− u32 args_batch_start_offset;
59		− struct intel_engine_cs *engine;
60		− struct i915_gem_context *ctx;
61		− struct drm_i915_gem_request *request;
62		−};
	64	+#define __I915_EXEC_ILLEGAL_FLAGS \
	65	+ (__I915_EXEC_UNKNOWN_FLAGS \| I915_EXEC_CONSTANTS_MASK)
	66	+
	67	+/**
	68	+ * DOC: User command execution
	69	+ *
	70	+ * Userspace submits commands to be executed on the GPU as an instruction
	71	+ * stream within a GEM object we call a batchbuffer. This instructions may
	72	+ * refer to other GEM objects containing auxiliary state such as kernels,
	73	+ * samplers, render targets and even secondary batchbuffers. Userspace does
	74	+ * not know where in the GPU memory these objects reside and so before the
	75	+ * batchbuffer is passed to the GPU for execution, those addresses in the
	76	+ * batchbuffer and auxiliary objects are updated. This is known as relocation,
	77	+ * or patching. To try and avoid having to relocate each object on the next
	78	+ * execution, userspace is told the location of those objects in this pass,
	79	+ * but this remains just a hint as the kernel may choose a new location for
	80	+ * any object in the future.
	81	+ *
	82	+ * Processing an execbuf ioctl is conceptually split up into a few phases.
	83	+ *
	84	+ * 1. Validation - Ensure all the pointers, handles and flags are valid.
	85	+ * 2. Reservation - Assign GPU address space for every object
	86	+ * 3. Relocation - Update any addresses to point to the final locations
	87	+ * 4. Serialisation - Order the request with respect to its dependencies
	88	+ * 5. Construction - Construct a request to execute the batchbuffer
	89	+ * 6. Submission (at some point in the future execution)
	90	+ *
	91	+ * Reserving resources for the execbuf is the most complicated phase. We
	92	+ * neither want to have to migrate the object in the address space, nor do
	93	+ * we want to have to update any relocations pointing to this object. Ideally,
	94	+ * we want to leave the object where it is and for all the existing relocations
	95	+ * to match. If the object is given a new address, or if userspace thinks the
	96	+ * object is elsewhere, we have to parse all the relocation entries and update
	97	+ * the addresses. Userspace can set the I915_EXEC_NORELOC flag to hint that
	98	+ * all the target addresses in all of its objects match the value in the
	99	+ * relocation entries and that they all match the presumed offsets given by the
	100	+ * list of execbuffer objects. Using this knowledge, we know that if we haven't
	101	+ * moved any buffers, all the relocation entries are valid and we can skip
	102	+ * the update. (If userspace is wrong, the likely outcome is an impromptu GPU
	103	+ * hang.) The requirement for using I915_EXEC_NO_RELOC are:
	104	+ *
	105	+ * The addresses written in the objects must match the corresponding
	106	+ * reloc.presumed_offset which in turn must match the corresponding
	107	+ * execobject.offset.
	108	+ *
	109	+ * Any render targets written to in the batch must be flagged with
	110	+ * EXEC_OBJECT_WRITE.
	111	+ *
	112	+ * To avoid stalling, execobject.offset should match the current
	113	+ * address of that object within the active context.
	114	+ *
	115	+ * The reservation is done is multiple phases. First we try and keep any
	116	+ * object already bound in its current location - so as long as meets the
	117	+ * constraints imposed by the new execbuffer. Any object left unbound after the
	118	+ * first pass is then fitted into any available idle space. If an object does
	119	+ * not fit, all objects are removed from the reservation and the process rerun
	120	+ * after sorting the objects into a priority order (more difficult to fit
	121	+ * objects are tried first). Failing that, the entire VM is cleared and we try
	122	+ * to fit the execbuf once last time before concluding that it simply will not
	123	+ * fit.
	124	+ *
	125	+ * A small complication to all of this is that we allow userspace not only to
	126	+ * specify an alignment and a size for the object in the address space, but
	127	+ * we also allow userspace to specify the exact offset. This objects are
	128	+ * simpler to place (the location is known a priori) all we have to do is make
	129	+ * sure the space is available.
	130	+ *
	131	+ * Once all the objects are in place, patching up the buried pointers to point
	132	+ * to the final locations is a fairly simple job of walking over the relocation
	133	+ * entry arrays, looking up the right address and rewriting the value into
	134	+ * the object. Simple! ... The relocation entries are stored in user memory
	135	+ * and so to access them we have to copy them into a local buffer. That copy
	136	+ * has to avoid taking any pagefaults as they may lead back to a GEM object
	137	+ * requiring the struct_mutex (i.e. recursive deadlock). So once again we split
	138	+ * the relocation into multiple passes. First we try to do everything within an
	139	+ * atomic context (avoid the pagefaults) which requires that we never wait. If
	140	+ * we detect that we may wait, or if we need to fault, then we have to fallback
	141	+ * to a slower path. The slowpath has to drop the mutex. (Can you hear alarm
	142	+ * bells yet?) Dropping the mutex means that we lose all the state we have
	143	+ * built up so far for the execbuf and we must reset any global data. However,
	144	+ * we do leave the objects pinned in their final locations - which is a
	145	+ * potential issue for concurrent execbufs. Once we have left the mutex, we can
	146	+ * allocate and copy all the relocation entries into a large array at our
	147	+ * leisure, reacquire the mutex, reclaim all the objects and other state and
	148	+ * then proceed to update any incorrect addresses with the objects.
	149	+ *
	150	+ * As we process the relocation entries, we maintain a record of whether the
	151	+ * object is being written to. Using NORELOC, we expect userspace to provide
	152	+ * this information instead. We also check whether we can skip the relocation
	153	+ * by comparing the expected value inside the relocation entry with the target's
	154	+ * final address. If they differ, we have to map the current object and rewrite
	155	+ * the 4 or 8 byte pointer within.
	156	+ *
	157	+ * Serialising an execbuf is quite simple according to the rules of the GEM
	158	+ * ABI. Execution within each context is ordered by the order of submission.
	159	+ * Writes to any GEM object are in order of submission and are exclusive. Reads
	160	+ * from a GEM object are unordered with respect to other reads, but ordered by
	161	+ * writes. A write submitted after a read cannot occur before the read, and
	162	+ * similarly any read submitted after a write cannot occur before the write.
	163	+ * Writes are ordered between engines such that only one write occurs at any
	164	+ * time (completing any reads beforehand) - using semaphores where available
	165	+ * and CPU serialisation otherwise. Other GEM access obey the same rules, any
	166	+ * write (either via mmaps using set-domain, or via pwrite) must flush all GPU
	167	+ * reads before starting, and any read (either using set-domain or pread) must
	168	+ * flush all GPU writes before starting. (Note we only employ a barrier before,
	169	+ * we currently rely on userspace not concurrently starting a new execution
	170	+ * whilst reading or writing to an object. This may be an advantage or not
	171	+ * depending on how much you trust userspace not to shoot themselves in the
	172	+ * foot.) Serialisation may just result in the request being inserted into
	173	+ * a DAG awaiting its turn, but most simple is to wait on the CPU until
	174	+ * all dependencies are resolved.
	175	+ *
	176	+ * After all of that, is just a matter of closing the request and handing it to
	177	+ * the hardware (well, leaving it in a queue to be executed). However, we also
	178	+ * offer the ability for batchbuffers to be run with elevated privileges so
	179	+ * that they access otherwise hidden registers. (Used to adjust L3 cache etc.)
	180	+ * Before any batch is given extra privileges we first must check that it
	181	+ * contains no nefarious instructions, we check that each instruction is from
	182	+ * our whitelist and all registers are also from an allowed list. We first
	183	+ * copy the user's batchbuffer to a shadow (so that the user doesn't have
	184	+ * access to it, either by the CPU or GPU as we scan it) and then parse each
	185	+ * instruction. If everything is ok, we set a flag telling the hardware to run
	186	+ * the batchbuffer in trusted mode, otherwise the ioctl is rejected.
	187	+ */
	188	+
	189	+struct i915_execbuffer {
	190	+ struct drm_i915_private i915; /* i915 backpointer */
	191	+ struct drm_file file; /* per-file lookup tables and limits */
	192	+ struct drm_i915_gem_execbuffer2 args; /* ioctl parameters */
	193	+ struct drm_i915_gem_exec_object2 exec; /* ioctl execobj[] */
	194	+
	195	+ struct intel_engine_cs engine; /* engine to queue the request to */
	196	+ struct i915_gem_context ctx; /* context for building the request */
	197	+ struct i915_address_space vm; /* GTT and vma for the request */
	198	+
	199	+ struct drm_i915_gem_request request; /* our request to build */
	200	+ struct i915_vma batch; /* identity of the batch obj/vma */
	201	+
	202	+ /** actual size of execobj[] as we may extend it for the cmdparser */
	203	+ unsigned int buffer_count;
63	204
64		−struct eb_vmas {
65		− struct drm_i915_private *i915;
66		− struct list_head vmas;
67		− int and;
68		− union {
69		− struct i915_vma *lut[0];
70		− struct hlist_head buckets[0];
71		− };
	205	+ /** list of vma not yet bound during reservation phase */
	206	+ struct list_head unbound;
	207	+
	208	+ /** list of vma that have execobj.relocation_count */
	209	+ struct list_head relocs;
	210	+
	211	+ /**
	212	+ * Track the most recently used object for relocations, as we
	213	+ * frequently have to perform multiple relocations within the same
	214	+ * obj/page
	215	+ */
	216	+ struct reloc_cache {
	217	+ struct drm_mm_node node; /** temporary GTT binding */
	218	+ unsigned long vaddr; /** Current kmap address */
	219	+ unsigned long page; /** Currently mapped page index */
	220	+ unsigned int gen; /** Cached value of INTEL_GEN */
	221	+ bool use_64bit_reloc : 1;
	222	+ bool has_llc : 1;
	223	+ bool has_fence : 1;
	224	+ bool needs_unfenced : 1;
	225	+
	226	+ struct drm_i915_gem_request *rq;
	227	+ u32 *rq_cmd;
	228	+ unsigned int rq_size;
	229	+ } reloc_cache;
	230	+
	231	+ u64 invalid_flags; /** Set of execobj.flags that are invalid */
	232	+ u32 context_flags; /** Set of execobj.flags to insert from the ctx */
	233	+
	234	+ u32 batch_start_offset; /** Location within object of batch */
	235	+ u32 batch_len; /** Length of batch within object */
	236	+ u32 batch_flags; /** Flags composed for emit_bb_start() */
	237	+
	238	+ /**
	239	+ * Indicate either the size of the hastable used to resolve
	240	+ * relocation handles, or if negative that we are using a direct
	241	+ * index into the execobj[].
	242	+ */
	243	+ int lut_size;
	244	+ struct hlist_head buckets; /* ht for relocation handles */
72	245	};
73	246
74		−static struct eb_vmas *
75		−eb_create(struct drm_i915_private *i915,
76		− struct drm_i915_gem_execbuffer2 *args)
	247	+/*
	248	+ * As an alternative to creating a hashtable of handle-to-vma for a batch,
	249	+ * we used the last available reserved field in the execobject[] and stash
	250	+ * a link from the execobj to its vma.
	251	+ */
	252	+#define __exec_to_vma(ee) (ee)->rsvd2
	253	+#define exec_to_vma(ee) u64_to_ptr(struct i915_vma, __exec_to_vma(ee))
	254	+
	255	+/*
	256	+ * Used to convert any address to canonical form.
	257	+ * Starting from gen8, some commands (e.g. STATE_BASE_ADDRESS,
	258	+ * MI_LOAD_REGISTER_MEM and others, see Broadwell PRM Vol2a) require the
	259	+ * addresses to be in a canonical form:
	260	+ * "GraphicsAddress[63:48] are ignored by the HW and assumed to be in correct
	261	+ * canonical form [63:48] == [47]."
	262	+ */
	263	+#define GEN8_HIGH_ADDRESS_BIT 47
	264	+static inline u64 gen8_canonical_addr(u64 address)
77	265	{
78		− struct eb_vmas *eb = NULL;
79		−
80		− if (args->flags & I915_EXEC_HANDLE_LUT) {
81		− unsigned size = args->buffer_count;
82		− size = sizeof(struct i915_vma );
83		− size += sizeof(struct eb_vmas);
84		− eb = kmalloc(size, GFP_TEMPORARY \| __GFP_NOWARN \| __GFP_NORETRY);
85		− }
86		−
87		− if (eb == NULL) {
88		− unsigned size = args->buffer_count;
89		− unsigned count = PAGE_SIZE / sizeof(struct hlist_head) / 2;
90		− BUILD_BUG_ON_NOT_POWER_OF_2(PAGE_SIZE / sizeof(struct hlist_head));
91		− while (count > 2*size)
92		− count >>= 1;
93		− eb = kzalloc(count*sizeof(struct hlist_head) +
94		− sizeof(struct eb_vmas),
95		− GFP_TEMPORARY);
96		− if (eb == NULL)
97		− return eb;
98		−
99		− eb->and = count - 1;
100		− } else
101		− eb->and = -args->buffer_count;
	266	+ return sign_extend64(address, GEN8_HIGH_ADDRESS_BIT);
	267	+}
102	268
103		− eb->i915 = i915;
104		− INIT_LIST_HEAD(&eb->vmas);
105		− return eb;
	269	+static inline u64 gen8_noncanonical_addr(u64 address)
	270	+{
	271	+ return address & GENMASK_ULL(GEN8_HIGH_ADDRESS_BIT, 0);
106	272	}
107	273
108		−static void
109		−eb_reset(struct eb_vmas *eb)
	274	+static int eb_create(struct i915_execbuffer *eb)
110	275	{
111		− if (eb->and >= 0)
112		− memset(eb->buckets, 0, (eb->and+1)*sizeof(struct hlist_head));
	276	+ if (!(eb->args->flags & I915_EXEC_HANDLE_LUT)) {
	277	+ unsigned int size = 1 + ilog2(eb->buffer_count);
	278	+
	279	+ /*
	280	+ * Without a 1:1 association between relocation handles and
	281	+ * the execobject[] index, we instead create a hashtable.
	282	+ * We size it dynamically based on available memory, starting
	283	+ * first with 1:1 assocative hash and scaling back until
	284	+ * the allocation succeeds.
	285	+ *
	286	+ * Later on we use a positive lut_size to indicate we are
	287	+ * using this hashtable, and a negative value to indicate a
	288	+ * direct lookup.
	289	+ */
	290	+ do {
	291	+ eb->buckets = kzalloc(sizeof(struct hlist_head) << size,
	292	+ GFP_TEMPORARY \|
	293	+ __GFP_NORETRY \|
	294	+ __GFP_NOWARN);
	295	+ if (eb->buckets)
	296	+ break;
	297	+ } while (--size);
	298	+
	299	+ if (unlikely(!eb->buckets)) {
	300	+ eb->buckets = kzalloc(sizeof(struct hlist_head),
	301	+ GFP_TEMPORARY);
	302	+ if (unlikely(!eb->buckets))
	303	+ return -ENOMEM;
	304	+ }
	305	+
	306	+ eb->lut_size = size;
	307	+ } else {
	308	+ eb->lut_size = -eb->buffer_count;
	309	+ }
	310	+
	311	+ return 0;
113	312	}
114	313
115		−static struct i915_vma *
116		−eb_get_batch(struct eb_vmas *eb)
	314	+static bool
	315	+eb_vma_misplaced(const struct drm_i915_gem_exec_object2 *entry,
	316	+ const struct i915_vma *vma)
	317	+{
	318	+ if (!(entry->flags & __EXEC_OBJECT_HAS_PIN))
	319	+ return true;
	320	+
	321	+ if (vma->node.size < entry->pad_to_size)
	322	+ return true;
	323	+
	324	+ if (entry->alignment && !IS_ALIGNED(vma->node.start, entry->alignment))
	325	+ return true;
	326	+
	327	+ if (entry->flags & EXEC_OBJECT_PINNED &&
	328	+ vma->node.start != entry->offset)
	329	+ return true;
	330	+
	331	+ if (entry->flags & __EXEC_OBJECT_NEEDS_BIAS &&
	332	+ vma->node.start < BATCH_OFFSET_BIAS)
	333	+ return true;
	334	+
	335	+ if (!(entry->flags & EXEC_OBJECT_SUPPORTS_48B_ADDRESS) &&
	336	+ (vma->node.start + vma->node.size - 1) >> 32)
	337	+ return true;
	338	+
	339	+ return false;
	340	+}
	341	+
	342	+static inline void
	343	+eb_pin_vma(struct i915_execbuffer *eb,
	344	+ struct drm_i915_gem_exec_object2 *entry,
	345	+ struct i915_vma *vma)
	346	+{
	347	+ u64 flags;
	348	+
	349	+ if (vma->node.size)
	350	+ flags = vma->node.start;
	351	+ else
	352	+ flags = entry->offset & PIN_OFFSET_MASK;
	353	+
	354	+ flags \|= PIN_USER \| PIN_NOEVICT \| PIN_OFFSET_FIXED;
	355	+ if (unlikely(entry->flags & EXEC_OBJECT_NEEDS_GTT))
	356	+ flags \|= PIN_GLOBAL;
	357	+
	358	+ if (unlikely(i915_vma_pin(vma, 0, 0, flags)))
	359	+ return;
	360	+
	361	+ if (unlikely(entry->flags & EXEC_OBJECT_NEEDS_FENCE)) {
	362	+ if (unlikely(i915_vma_get_fence(vma))) {
	363	+ i915_vma_unpin(vma);
	364	+ return;
	365	+ }
	366	+
	367	+ if (i915_vma_pin_fence(vma))
	368	+ entry->flags \|= __EXEC_OBJECT_HAS_FENCE;
	369	+ }
	370	+
	371	+ entry->flags \|= __EXEC_OBJECT_HAS_PIN;
	372	+}
	373	+
	374	+static inline void
	375	+__eb_unreserve_vma(struct i915_vma *vma,
	376	+ const struct drm_i915_gem_exec_object2 *entry)
	377	+{
	378	+ GEM_BUG_ON(!(entry->flags & __EXEC_OBJECT_HAS_PIN));
	379	+
	380	+ if (unlikely(entry->flags & __EXEC_OBJECT_HAS_FENCE))
	381	+ i915_vma_unpin_fence(vma);
	382	+
	383	+ __i915_vma_unpin(vma);
	384	+}
	385	+
	386	+static inline void
	387	+eb_unreserve_vma(struct i915_vma *vma,
	388	+ struct drm_i915_gem_exec_object2 *entry)
	389	+{
	390	+ if (!(entry->flags & __EXEC_OBJECT_HAS_PIN))
	391	+ return;
	392	+
	393	+ __eb_unreserve_vma(vma, entry);
	394	+ entry->flags &= ~__EXEC_OBJECT_RESERVED;
	395	+}
	396	+
	397	+static int
	398	+eb_validate_vma(struct i915_execbuffer *eb,
	399	+ struct drm_i915_gem_exec_object2 *entry,
	400	+ struct i915_vma *vma)
117	401	{
118		− struct i915_vma vma = list_entry(eb->vmas.prev, typeof(vma), exec_list);
	402	+ if (unlikely(entry->flags & eb->invalid_flags))
	403	+ return -EINVAL;
	404	+
	405	+ if (unlikely(entry->alignment && !is_power_of_2(entry->alignment)))
	406	+ return -EINVAL;
119	407
120	408	/*
121		− * SNA is doing fancy tricks with compressing batch buffers, which leads
122		− * to negative relocation deltas. Usually that works out ok since the
123		− * relocate address is still positive, except when the batch is placed
124		− * very low in the GTT. Ensure this doesn't happen.
125		− *
126		− * Note that actual hangs have only been observed on gen7, but for
127		− * paranoia do it everywhere.
	409	+ * Offset can be used as input (EXEC_OBJECT_PINNED), reject
	410	+ * any non-page-aligned or non-canonical addresses.
128	411	*/
129		− if ((vma->exec_entry->flags & EXEC_OBJECT_PINNED) == 0)
130		− vma->exec_entry->flags \|= __EXEC_OBJECT_NEEDS_BIAS;
	412	+ if (unlikely(entry->flags & EXEC_OBJECT_PINNED &&
	413	+ entry->offset != gen8_canonical_addr(entry->offset & PAGE_MASK)))
	414	+ return -EINVAL;
131	415
132		− return vma;
	416	+ /* pad_to_size was once a reserved field, so sanitize it */
	417	+ if (entry->flags & EXEC_OBJECT_PAD_TO_SIZE) {
	418	+ if (unlikely(offset_in_page(entry->pad_to_size)))
	419	+ return -EINVAL;
	420	+ } else {
	421	+ entry->pad_to_size = 0;
	422	+ }
	423	+
	424	+ if (unlikely(vma->exec_entry)) {
	425	+ DRM_DEBUG("Object [handle %d, index %d] appears more than once in object list\n",
	426	+ entry->handle, (int)(entry - eb->exec));
	427	+ return -EINVAL;
	428	+ }
	429	+
	430	+ /*
	431	+ * From drm_mm perspective address space is continuous,
	432	+ * so from this point we're always using non-canonical
	433	+ * form internally.
	434	+ */
	435	+ entry->offset = gen8_noncanonical_addr(entry->offset);
	436	+
	437	+ return 0;
133	438	}
134	439
135	440	static int
136		−eb_lookup_vmas(struct eb_vmas *eb,
137		− struct drm_i915_gem_exec_object2 *exec,
138		− const struct drm_i915_gem_execbuffer2 *args,
139		− struct i915_address_space *vm,
140		− struct drm_file *file)
	441	+eb_add_vma(struct i915_execbuffer *eb,
	442	+ struct drm_i915_gem_exec_object2 *entry,
	443	+ struct i915_vma *vma)
141	444	{
142		− struct drm_i915_gem_object *obj;
143		− struct list_head objects;
144		− int i, ret;
	445	+ int err;
145	446
146		− INIT_LIST_HEAD(&objects);
147		− spin_lock(&file->table_lock);
148		− /* Grab a reference to the object and release the lock so we can lookup
149		− * or create the VMA without using GFP_ATOMIC */
150		− for (i = 0; i < args->buffer_count; i++) {
151		− obj = to_intel_bo(idr_find(&file->object_idr, exec[i].handle));
152		− if (obj == NULL) {
153		− spin_unlock(&file->table_lock);
154		− DRM_DEBUG("Invalid object handle %d at index %d\n",
155		− exec[i].handle, i);
156		− ret = -ENOENT;
157		− goto err;
	447	+ GEM_BUG_ON(i915_vma_is_closed(vma));
	448	+
	449	+ if (!(eb->args->flags & __EXEC_VALIDATED)) {
	450	+ err = eb_validate_vma(eb, entry, vma);
	451	+ if (unlikely(err))
	452	+ return err;
	453	+ }
	454	+
	455	+ if (eb->lut_size >= 0) {
	456	+ vma->exec_handle = entry->handle;
	457	+ hlist_add_head(&vma->exec_node,
	458	+ &eb->buckets[hash_32(entry->handle,
	459	+ eb->lut_size)]);
	460	+ }
	461	+
	462	+ if (entry->relocation_count)
	463	+ list_add_tail(&vma->reloc_link, &eb->relocs);
	464	+
	465	+ if (!eb->reloc_cache.has_fence) {
	466	+ entry->flags &= ~EXEC_OBJECT_NEEDS_FENCE;
	467	+ } else {
	468	+ if ((entry->flags & EXEC_OBJECT_NEEDS_FENCE \|\|
	469	+ eb->reloc_cache.needs_unfenced) &&
	470	+ i915_gem_object_is_tiled(vma->obj))
	471	+ entry->flags \|= EXEC_OBJECT_NEEDS_GTT \| __EXEC_OBJECT_NEEDS_MAP;
	472	+ }
	473	+
	474	+ if (!(entry->flags & EXEC_OBJECT_PINNED))
	475	+ entry->flags \|= eb->context_flags;
	476	+
	477	+ /*
	478	+ * Stash a pointer from the vma to execobj, so we can query its flags,
	479	+ * size, alignment etc as provided by the user. Also we stash a pointer
	480	+ * to the vma inside the execobj so that we can use a direct lookup
	481	+ * to find the right target VMA when doing relocations.
	482	+ */
	483	+ vma->exec_entry = entry;
	484	+ __exec_to_vma(entry) = (uintptr_t)vma;
	485	+
	486	+ err = 0;
	487	+ eb_pin_vma(eb, entry, vma);
	488	+ if (eb_vma_misplaced(entry, vma)) {
	489	+ eb_unreserve_vma(vma, entry);
	490	+
	491	+ list_add_tail(&vma->exec_link, &eb->unbound);
	492	+ if (drm_mm_node_allocated(&vma->node))
	493	+ err = i915_vma_unbind(vma);
	494	+ } else {
	495	+ if (entry->offset != vma->node.start) {
	496	+ entry->offset = vma->node.start \| UPDATE;
	497	+ eb->args->flags \|= __EXEC_HAS_RELOC;
	498	+ }
	499	+ }
	500	+ return err;
	501	+}
	502	+
	503	+static inline int use_cpu_reloc(const struct reloc_cache *cache,
	504	+ const struct drm_i915_gem_object *obj)
	505	+{
	506	+ if (!i915_gem_object_has_struct_page(obj))
	507	+ return false;
	508	+
	509	+ if (DBG_FORCE_RELOC == FORCE_CPU_RELOC)
	510	+ return true;
	511	+
	512	+ if (DBG_FORCE_RELOC == FORCE_GTT_RELOC)
	513	+ return false;
	514	+
	515	+ return (cache->has_llc \|\|
	516	+ obj->cache_dirty \|\|
	517	+ obj->cache_level != I915_CACHE_NONE);
	518	+}
	519	+
	520	+static int eb_reserve_vma(const struct i915_execbuffer *eb,
	521	+ struct i915_vma *vma)
	522	+{
	523	+ struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
	524	+ u64 flags;
	525	+ int err;
	526	+
	527	+ flags = PIN_USER \| PIN_NONBLOCK;
	528	+ if (entry->flags & EXEC_OBJECT_NEEDS_GTT)
	529	+ flags \|= PIN_GLOBAL;
	530	+
	531	+ /*
	532	+ * Wa32bitGeneralStateOffset & Wa32bitInstructionBaseOffset,
	533	+ * limit address to the first 4GBs for unflagged objects.
	534	+ */
	535	+ if (!(entry->flags & EXEC_OBJECT_SUPPORTS_48B_ADDRESS))
	536	+ flags \|= PIN_ZONE_4G;
	537	+
	538	+ if (entry->flags & __EXEC_OBJECT_NEEDS_MAP)
	539	+ flags \|= PIN_MAPPABLE;
	540	+
	541	+ if (entry->flags & EXEC_OBJECT_PINNED) {
	542	+ flags \|= entry->offset \| PIN_OFFSET_FIXED;
	543	+ flags &= ~PIN_NONBLOCK; /* force overlapping PINNED checks */
	544	+ } else if (entry->flags & __EXEC_OBJECT_NEEDS_BIAS) {
	545	+ flags \|= BATCH_OFFSET_BIAS \| PIN_OFFSET_BIAS;
	546	+ }
	547	+
	548	+ err = i915_vma_pin(vma, entry->pad_to_size, entry->alignment, flags);
	549	+ if (err)
	550	+ return err;
	551	+
	552	+ if (entry->offset != vma->node.start) {
	553	+ entry->offset = vma->node.start \| UPDATE;
	554	+ eb->args->flags \|= __EXEC_HAS_RELOC;
	555	+ }
	556	+
	557	+ entry->flags \|= __EXEC_OBJECT_HAS_PIN;
	558	+ GEM_BUG_ON(eb_vma_misplaced(entry, vma));
	559	+
	560	+ if (unlikely(entry->flags & EXEC_OBJECT_NEEDS_FENCE)) {
	561	+ err = i915_vma_get_fence(vma);
	562	+ if (unlikely(err)) {
	563	+ i915_vma_unpin(vma);
	564	+ return err;
	565	+ }
	566	+
	567	+ if (i915_vma_pin_fence(vma))
	568	+ entry->flags \|= __EXEC_OBJECT_HAS_FENCE;
	569	+ }
	570	+
	571	+ return 0;
	572	+}
	573	+
	574	+static int eb_reserve(struct i915_execbuffer *eb)
	575	+{
	576	+ const unsigned int count = eb->buffer_count;
	577	+ struct list_head last;
	578	+ struct i915_vma *vma;
	579	+ unsigned int i, pass;
	580	+ int err;
	581	+
	582	+ /*
	583	+ * Attempt to pin all of the buffers into the GTT.
	584	+ * This is done in 3 phases:
	585	+ *
	586	+ * 1a. Unbind all objects that do not match the GTT constraints for
	587	+ * the execbuffer (fenceable, mappable, alignment etc).
	588	+ * 1b. Increment pin count for already bound objects.
	589	+ * 2. Bind new objects.
	590	+ * 3. Decrement pin count.
	591	+ *
	592	+ * This avoid unnecessary unbinding of later objects in order to make
	593	+ * room for the earlier objects unless we need to defragment.
	594	+ */
	595	+
	596	+ pass = 0;
	597	+ err = 0;
	598	+ do {
	599	+ list_for_each_entry(vma, &eb->unbound, exec_link) {
	600	+ err = eb_reserve_vma(eb, vma);
	601	+ if (err)
	602	+ break;
	603	+ }
	604	+ if (err != -ENOSPC)
	605	+ return err;
	606	+
	607	+ /* Resort all the objects into priority order */
	608	+ INIT_LIST_HEAD(&eb->unbound);
	609	+ INIT_LIST_HEAD(&last);
	610	+ for (i = 0; i < count; i++) {
	611	+ struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
	612	+
	613	+ if (entry->flags & EXEC_OBJECT_PINNED &&
	614	+ entry->flags & __EXEC_OBJECT_HAS_PIN)
	615	+ continue;
	616	+
	617	+ vma = exec_to_vma(entry);
	618	+ eb_unreserve_vma(vma, entry);
	619	+
	620	+ if (entry->flags & EXEC_OBJECT_PINNED)
	621	+ list_add(&vma->exec_link, &eb->unbound);
	622	+ else if (entry->flags & __EXEC_OBJECT_NEEDS_MAP)
	623	+ list_add_tail(&vma->exec_link, &eb->unbound);
	624	+ else
	625	+ list_add_tail(&vma->exec_link, &last);
	626	+ }
	627	+ list_splice_tail(&last, &eb->unbound);
	628	+
	629	+ switch (pass++) {
	630	+ case 0:
	631	+ break;
	632	+
	633	+ case 1:
	634	+ /* Too fragmented, unbind everything and retry */
	635	+ err = i915_gem_evict_vm(eb->vm);
	636	+ if (err)
	637	+ return err;
	638	+ break;
	639	+
	640	+ default:
	641	+ return -ENOSPC;
	642	+ }
	643	+ } while (1);
	644	+}
	645	+
	646	+static inline struct hlist_head *
	647	+ht_head(const struct i915_gem_context_vma_lut *lut, u32 handle)
	648	+{
	649	+ return &lut->ht[hash_32(handle, lut->ht_bits)];
	650	+}
	651	+
	652	+static inline bool
	653	+ht_needs_resize(const struct i915_gem_context_vma_lut *lut)
	654	+{
	655	+ return (4lut->ht_count > 3lut->ht_size \|\|
	656	+ 4*lut->ht_count + 1 < lut->ht_size);
	657	+}
	658	+
	659	+static unsigned int eb_batch_index(const struct i915_execbuffer *eb)
	660	+{
	661	+ if (eb->args->flags & I915_EXEC_BATCH_FIRST)
	662	+ return 0;
	663	+ else
	664	+ return eb->buffer_count - 1;
	665	+}
	666	+
	667	+static int eb_select_context(struct i915_execbuffer *eb)
	668	+{
	669	+ struct i915_gem_context *ctx;
	670	+
	671	+ ctx = i915_gem_context_lookup(eb->file->driver_priv, eb->args->rsvd1);
	672	+ if (unlikely(IS_ERR(ctx)))
	673	+ return PTR_ERR(ctx);
	674	+
	675	+ if (unlikely(i915_gem_context_is_banned(ctx))) {
	676	+ DRM_DEBUG("Context %u tried to submit while banned\n",
	677	+ ctx->user_handle);
	678	+ return -EIO;
	679	+ }
	680	+
	681	+ eb->ctx = i915_gem_context_get(ctx);
	682	+ eb->vm = ctx->ppgtt ? &ctx->ppgtt->base : &eb->i915->ggtt.base;
	683	+
	684	+ eb->context_flags = 0;
	685	+ if (ctx->flags & CONTEXT_NO_ZEROMAP)
	686	+ eb->context_flags \|= __EXEC_OBJECT_NEEDS_BIAS;
	687	+
	688	+ return 0;
	689	+}
	690	+
	691	+static int eb_lookup_vmas(struct i915_execbuffer *eb)
	692	+{
	693	+#define INTERMEDIATE BIT(0)
	694	+ const unsigned int count = eb->buffer_count;
	695	+ struct i915_gem_context_vma_lut *lut = &eb->ctx->vma_lut;
	696	+ struct i915_vma *vma;
	697	+ struct idr *idr;
	698	+ unsigned int i;
	699	+ int slow_pass = -1;
	700	+ int err;
	701	+
	702	+ INIT_LIST_HEAD(&eb->relocs);
	703	+ INIT_LIST_HEAD(&eb->unbound);
	704	+
	705	+ if (unlikely(lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS))
	706	+ flush_work(&lut->resize);
	707	+ GEM_BUG_ON(lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS);
	708	+
	709	+ for (i = 0; i < count; i++) {
	710	+ __exec_to_vma(&eb->exec[i]) = 0;
	711	+
	712	+ hlist_for_each_entry(vma,
	713	+ ht_head(lut, eb->exec[i].handle),
	714	+ ctx_node) {
	715	+ if (vma->ctx_handle != eb->exec[i].handle)
	716	+ continue;
	717	+
	718	+ err = eb_add_vma(eb, &eb->exec[i], vma);
	719	+ if (unlikely(err))
	720	+ return err;
	721	+
	722	+ goto next_vma;
158	723	}
159	724
160		− if (!list_empty(&obj->obj_exec_link)) {
161		− spin_unlock(&file->table_lock);
162		− DRM_DEBUG("Object %p [handle %d, index %d] appears more than once in object list\n",
163		− obj, exec[i].handle, i);
164		− ret = -EINVAL;
	725	+ if (slow_pass < 0)
	726	+ slow_pass = i;
	727	+next_vma: ;
	728	+ }
	729	+
	730	+ if (slow_pass < 0)
	731	+ goto out;
	732	+
	733	+ spin_lock(&eb->file->table_lock);
	734	+ /*
	735	+ * Grab a reference to the object and release the lock so we can lookup
	736	+ * or create the VMA without using GFP_ATOMIC
	737	+ */
	738	+ idr = &eb->file->object_idr;
	739	+ for (i = slow_pass; i < count; i++) {
	740	+ struct drm_i915_gem_object *obj;
	741	+
	742	+ if (__exec_to_vma(&eb->exec[i]))
	743	+ continue;
	744	+
	745	+ obj = to_intel_bo(idr_find(idr, eb->exec[i].handle));
	746	+ if (unlikely(!obj)) {
	747	+ spin_unlock(&eb->file->table_lock);
	748	+ DRM_DEBUG("Invalid object handle %d at index %d\n",
	749	+ eb->exec[i].handle, i);
	750	+ err = -ENOENT;
165	751	goto err;
166	752	}
167	753
168		− i915_gem_object_get(obj);
169		− list_add_tail(&obj->obj_exec_link, &objects);
	754	+ __exec_to_vma(&eb->exec[i]) = INTERMEDIATE \| (uintptr_t)obj;
170	755	}
171		− spin_unlock(&file->table_lock);
	756	+ spin_unlock(&eb->file->table_lock);
172	757
173		− i = 0;
174		− while (!list_empty(&objects)) {
175		− struct i915_vma *vma;
	758	+ for (i = slow_pass; i < count; i++) {
	759	+ struct drm_i915_gem_object *obj;
176	760
177		− obj = list_first_entry(&objects,
178		− struct drm_i915_gem_object,
179		− obj_exec_link);
	761	+ if (!(__exec_to_vma(&eb->exec[i]) & INTERMEDIATE))
	762	+ continue;
180	763
181	764	/*
182	765	* NOTE: We can leak any vmas created here when something fails
@@ -186,59 +769,93 @@ eb_lookup_vmas(struct eb_vmas *eb, · 3 unmodified lines
186	769	* from the (obj, vm) we don't run the risk of creating
187	770	* duplicated vmas for the same vm.
188	771	*/
189		− vma = i915_vma_instance(obj, vm, NULL);
	772	+ obj = u64_to_ptr(typeof(*obj),
	773	+ __exec_to_vma(&eb->exec[i]) & ~INTERMEDIATE);
	774	+ vma = i915_vma_instance(obj, eb->vm, NULL);
190	775	if (unlikely(IS_ERR(vma))) {
191	776	DRM_DEBUG("Failed to lookup VMA\n");
192		− ret = PTR_ERR(vma);
	777	+ err = PTR_ERR(vma);
193	778	goto err;
194	779	}
195	780
196		− /* Transfer ownership from the objects list to the vmas list. */
197		− list_add_tail(&vma->exec_list, &eb->vmas);
198		− list_del_init(&obj->obj_exec_link);
	781	+ /* First come, first served */
	782	+ if (!vma->ctx) {
	783	+ vma->ctx = eb->ctx;
	784	+ vma->ctx_handle = eb->exec[i].handle;
	785	+ hlist_add_head(&vma->ctx_node,
	786	+ ht_head(lut, eb->exec[i].handle));
	787	+ lut->ht_count++;
	788	+ lut->ht_size \|= I915_CTX_RESIZE_IN_PROGRESS;
	789	+ if (i915_vma_is_ggtt(vma)) {
	790	+ GEM_BUG_ON(obj->vma_hashed);
	791	+ obj->vma_hashed = vma;
	792	+ }
199	793
200		− vma->exec_entry = &exec[i];
201		− if (eb->and < 0) {
202		− eb->lut[i] = vma;
203		− } else {
204		− uint32_t handle = args->flags & I915_EXEC_HANDLE_LUT ? i : exec[i].handle;
205		− vma->exec_handle = handle;
206		− hlist_add_head(&vma->exec_node,
207		− &eb->buckets[handle & eb->and]);
	794	+ i915_vma_get(vma);
	795	+ }
	796	+
	797	+ err = eb_add_vma(eb, &eb->exec[i], vma);
	798	+ if (unlikely(err))
	799	+ goto err;
	800	+
	801	+ /* Only after we validated the user didn't use our bits */
	802	+ if (vma->ctx != eb->ctx) {
	803	+ i915_vma_get(vma);
	804	+ eb->exec[i].flags \|= __EXEC_OBJECT_HAS_REF;
208	805	}
209		− ++i;
210	806	}
211	807
212		− return 0;
	808	+ if (lut->ht_size & I915_CTX_RESIZE_IN_PROGRESS) {
	809	+ if (ht_needs_resize(lut))
	810	+ queue_work(system_highpri_wq, &lut->resize);
	811	+ else
	812	+ lut->ht_size &= ~I915_CTX_RESIZE_IN_PROGRESS;
	813	+ }
	814	+
	815	+out:
	816	+ /* take note of the batch buffer before we might reorder the lists */
	817	+ i = eb_batch_index(eb);
	818	+ eb->batch = exec_to_vma(&eb->exec[i]);
	819	+
	820	+ /*
	821	+ * SNA is doing fancy tricks with compressing batch buffers, which leads
	822	+ * to negative relocation deltas. Usually that works out ok since the
	823	+ * relocate address is still positive, except when the batch is placed
	824	+ * very low in the GTT. Ensure this doesn't happen.
	825	+ *
	826	+ * Note that actual hangs have only been observed on gen7, but for
	827	+ * paranoia do it everywhere.
	828	+ */
	829	+ if (!(eb->exec[i].flags & EXEC_OBJECT_PINNED))
	830	+ eb->exec[i].flags \|= __EXEC_OBJECT_NEEDS_BIAS;
	831	+ if (eb->reloc_cache.has_fence)
	832	+ eb->exec[i].flags \|= EXEC_OBJECT_NEEDS_FENCE;
213	833
	834	+ eb->args->flags \|= __EXEC_VALIDATED;
	835	+ return eb_reserve(eb);
214	836
215	837	err:
216		− while (!list_empty(&objects)) {
217		− obj = list_first_entry(&objects,
218		− struct drm_i915_gem_object,
219		− obj_exec_link);
220		− list_del_init(&obj->obj_exec_link);
221		− i915_gem_object_put(obj);
	838	+ for (i = slow_pass; i < count; i++) {
	839	+ if (__exec_to_vma(&eb->exec[i]) & INTERMEDIATE)
	840	+ __exec_to_vma(&eb->exec[i]) = 0;
222	841	}
223		− /*
224		− * Objects already transfered to the vmas list will be unreferenced by
225		− * eb_destroy.
226		− */
227		−
228		− return ret;
	842	+ lut->ht_size &= ~I915_CTX_RESIZE_IN_PROGRESS;
	843	+ return err;
	844	+#undef INTERMEDIATE
229	845	}
230	846
231		−static struct i915_vma eb_get_vma(struct eb_vmas eb, unsigned long handle)
	847	+static struct i915_vma *
	848	+eb_get_vma(const struct i915_execbuffer *eb, unsigned long handle)
232	849	{
233		− if (eb->and < 0) {
234		− if (handle >= -eb->and)
	850	+ if (eb->lut_size < 0) {
	851	+ if (handle >= -eb->lut_size)
235	852	return NULL;
236		− return eb->lut[handle];
	853	+ return exec_to_vma(&eb->exec[handle]);
237	854	} else {
238	855	struct hlist_head *head;
239	856	struct i915_vma *vma;
240	857
241		− head = &eb->buckets[handle & eb->and];
	858	+ head = &eb->buckets[hash_32(handle, eb->lut_size)];
242	859	hlist_for_each_entry(vma, head, exec_node) {
243	860	if (vma->exec_handle == handle)
244	861	return vma;
@@ -247,96 +864,69 @@ static struct i915_vma eb_get_vma(struct eb_vmas eb, unsigned long handle) · 2 unmodified lines
247	864	}
248	865	}
249	866
250		−static void
251		−i915_gem_execbuffer_unreserve_vma(struct i915_vma *vma)
	867	+static void eb_release_vmas(const struct i915_execbuffer *eb)
252	868	{
253		− struct drm_i915_gem_exec_object2 *entry;
	869	+ const unsigned int count = eb->buffer_count;
	870	+ unsigned int i;
254	871
255		− if (!drm_mm_node_allocated(&vma->node))
256		− return;
257		−
258		− entry = vma->exec_entry;
	872	+ for (i = 0; i < count; i++) {
	873	+ struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
	874	+ struct i915_vma *vma = exec_to_vma(entry);
259	875
260		− if (entry->flags & __EXEC_OBJECT_HAS_FENCE)
261		− i915_vma_unpin_fence(vma);
	876	+ if (!vma)
	877	+ continue;
262	878
263		− if (entry->flags & __EXEC_OBJECT_HAS_PIN)
264		− __i915_vma_unpin(vma);
	879	+ GEM_BUG_ON(vma->exec_entry != entry);
	880	+ vma->exec_entry = NULL;
265	881
266		− entry->flags &= ~(__EXEC_OBJECT_HAS_FENCE \| __EXEC_OBJECT_HAS_PIN);
267		−}
	882	+ if (entry->flags & __EXEC_OBJECT_HAS_PIN)
	883	+ __eb_unreserve_vma(vma, entry);
268	884
269		−static void eb_destroy(struct eb_vmas *eb)
270		−{
271		− while (!list_empty(&eb->vmas)) {
272		− struct i915_vma *vma;
	885	+ if (entry->flags & __EXEC_OBJECT_HAS_REF)
	886	+ i915_vma_put(vma);
273	887
274		− vma = list_first_entry(&eb->vmas,
275		− struct i915_vma,
276		− exec_list);
277		− list_del_init(&vma->exec_list);
278		− i915_gem_execbuffer_unreserve_vma(vma);
279		− vma->exec_entry = NULL;
280		− i915_vma_put(vma);
	888	+ entry->flags &=
	889	+ ~(__EXEC_OBJECT_RESERVED \| __EXEC_OBJECT_HAS_REF);
281	890	}
282		− kfree(eb);
283	891	}
284	892
285		−static inline int use_cpu_reloc(struct drm_i915_gem_object *obj)
	893	+static void eb_reset_vmas(const struct i915_execbuffer *eb)
286	894	{
287		− if (!i915_gem_object_has_struct_page(obj))
288		− return false;
289		−
290		− if (DBG_USE_CPU_RELOC)
291		− return DBG_USE_CPU_RELOC > 0;
292		−
293		− return (HAS_LLC(to_i915(obj->base.dev)) \|\|
294		− obj->base.write_domain == I915_GEM_DOMAIN_CPU \|\|
295		− obj->cache_level != I915_CACHE_NONE);
	895	+ eb_release_vmas(eb);
	896	+ if (eb->lut_size >= 0)
	897	+ memset(eb->buckets, 0,
	898	+ sizeof(struct hlist_head) << eb->lut_size);
296	899	}
297	900
298		−/* Used to convert any address to canonical form.
299		− * Starting from gen8, some commands (e.g. STATE_BASE_ADDRESS,
300		− * MI_LOAD_REGISTER_MEM and others, see Broadwell PRM Vol2a) require the
301		− * addresses to be in a canonical form:
302		− * "GraphicsAddress[63:48] are ignored by the HW and assumed to be in correct
303		− * canonical form [63:48] == [47]."
304		− */
305		−#define GEN8_HIGH_ADDRESS_BIT 47
306		−static inline uint64_t gen8_canonical_addr(uint64_t address)
	901	+static void eb_destroy(const struct i915_execbuffer *eb)
307	902	{
308		− return sign_extend64(address, GEN8_HIGH_ADDRESS_BIT);
309		−}
	903	+ GEM_BUG_ON(eb->reloc_cache.rq);
310	904
311		−static inline uint64_t gen8_noncanonical_addr(uint64_t address)
312		−{
313		− return address & ((1ULL << (GEN8_HIGH_ADDRESS_BIT + 1)) - 1);
	905	+ if (eb->lut_size >= 0)
	906	+ kfree(eb->buckets);
314	907	}
315	908
316		−static inline uint64_t
	909	+static inline u64
317	910	relocation_target(const struct drm_i915_gem_relocation_entry *reloc,
318		− uint64_t target_offset)
	911	+ const struct i915_vma *target)
319	912	{
320		− return gen8_canonical_addr((int)reloc->delta + target_offset);
	913	+ return gen8_canonical_addr((int)reloc->delta + target->node.start);
321	914	}
322	915
323		−struct reloc_cache {
324		− struct drm_i915_private *i915;
325		− struct drm_mm_node node;
326		− unsigned long vaddr;
327		− unsigned int page;
328		− bool use_64bit_reloc;
329		−};
330		−
331	916	static void reloc_cache_init(struct reloc_cache *cache,
332	917	struct drm_i915_private *i915)
333	918	{
334	919	cache->page = -1;
335	920	cache->vaddr = 0;
336		− cache->i915 = i915;
337	921	/* Must be a variable in the struct to allow GCC to unroll. */
	922	+ cache->gen = INTEL_GEN(i915);
	923	+ cache->has_llc = HAS_LLC(i915);
338	924	cache->use_64bit_reloc = HAS_64BIT_RELOC(i915);
	925	+ cache->has_fence = cache->gen < 4;
	926	+ cache->needs_unfenced = INTEL_INFO(i915)->unfenced_needs_alignment;
339	927	cache->node.allocated = false;
	928	+ cache->rq = NULL;
	929	+ cache->rq_size = 0;
340	930	}
341	931
342	932	static inline void *unmask_page(unsigned long p)
@@ -351,10 +941,31 @@ static inline unsigned int unmask_flags(unsigned long p) · 8 unmodified lines
351	941
352	942	#define KMAP 0x4 /* after CLFLUSH_FLAGS */
353	943
354		−static void reloc_cache_fini(struct reloc_cache *cache)
	944	+static inline struct i915_ggtt cache_to_ggtt(struct reloc_cache cache)
	945	+{
	946	+ struct drm_i915_private *i915 =
	947	+ container_of(cache, struct i915_execbuffer, reloc_cache)->i915;
	948	+ return &i915->ggtt;
	949	+}
	950	+
	951	+static void reloc_gpu_flush(struct reloc_cache *cache)
	952	+{
	953	+ GEM_BUG_ON(cache->rq_size >= cache->rq->batch->obj->base.size / sizeof(u32));
	954	+ cache->rq_cmd[cache->rq_size] = MI_BATCH_BUFFER_END;
	955	+ i915_gem_object_unpin_map(cache->rq->batch->obj);
	956	+ i915_gem_chipset_flush(cache->rq->i915);
	957	+
	958	+ __i915_add_request(cache->rq, true);
	959	+ cache->rq = NULL;
	960	+}
	961	+
	962	+static void reloc_cache_reset(struct reloc_cache *cache)
355	963	{
356	964	void *vaddr;
357	965
	966	+ if (cache->rq)
	967	+ reloc_gpu_flush(cache);
	968	+
358	969	if (!cache->vaddr)
359	970	return;
360	971
@@ -369,7 +980,7 @@ static void reloc_cache_fini(struct reloc_cache *cache) · 8 unmodified lines
369	980	wmb();
370	981	io_mapping_unmap_atomic((void __iomem *)vaddr);
371	982	if (cache->node.allocated) {
372		− struct i915_ggtt *ggtt = &cache->i915->ggtt;
	983	+ struct i915_ggtt *ggtt = cache_to_ggtt(cache);
373	984
374	985	ggtt->base.clear_range(&ggtt->base,
375	986	cache->node.start,
@@ -379,11 +990,14 @@ static void reloc_cache_fini(struct reloc_cache *cache) · 3 unmodified lines
379	990	i915_vma_unpin((struct i915_vma *)cache->node.mm);
380	991	}
381	992	}
	993	+
	994	+ cache->vaddr = 0;
	995	+ cache->page = -1;
382	996	}
383	997
384	998	static void reloc_kmap(struct drm_i915_gem_object obj,
385	999	struct reloc_cache *cache,
386		− int page)
	1000	+ unsigned long page)
387	1001	{
388	1002	void *vaddr;
389	1003
@@ -391,11 +1005,11 @@ static void reloc_kmap(struct drm_i915_gem_object obj, · 1 unmodified line
391	1005	kunmap_atomic(unmask_page(cache->vaddr));
392	1006	} else {
393	1007	unsigned int flushes;
394		− int ret;
	1008	+ int err;
395	1009
396		− ret = i915_gem_obj_prepare_shmem_write(obj, &flushes);
397		− if (ret)
398		− return ERR_PTR(ret);
	1010	+ err = i915_gem_obj_prepare_shmem_write(obj, &flushes);
	1011	+ if (err)
	1012	+ return ERR_PTR(err);
399	1013
400	1014	BUILD_BUG_ON(KMAP & CLFLUSH_FLAGS);
401	1015	BUILD_BUG_ON((KMAP \| CLFLUSH_FLAGS) & PAGE_MASK);
@@ -415,9 +1029,9 @@ static void reloc_kmap(struct drm_i915_gem_object obj, · 13 unmodified lines
415	1029
416	1030	static void reloc_iomap(struct drm_i915_gem_object obj,
417	1031	struct reloc_cache *cache,
418		− int page)
	1032	+ unsigned long page)
419	1033	{
420		− struct i915_ggtt *ggtt = &cache->i915->ggtt;
	1034	+ struct i915_ggtt *ggtt = cache_to_ggtt(cache);
421	1035	unsigned long offset;
422	1036	void *vaddr;
423	1037
@@ -425,31 +1039,31 @@ static void reloc_iomap(struct drm_i915_gem_object obj, · 1 unmodified line
425	1039	io_mapping_unmap_atomic((void __force __iomem *) unmask_page(cache->vaddr));
426	1040	} else {
427	1041	struct i915_vma *vma;
428		− int ret;
	1042	+ int err;
429	1043
430		− if (use_cpu_reloc(obj))
	1044	+ if (use_cpu_reloc(cache, obj))
431	1045	return NULL;
432	1046
433		− ret = i915_gem_object_set_to_gtt_domain(obj, true);
434		− if (ret)
435		− return ERR_PTR(ret);
	1047	+ err = i915_gem_object_set_to_gtt_domain(obj, true);
	1048	+ if (err)
	1049	+ return ERR_PTR(err);
436	1050
437	1051	vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0,
438	1052	PIN_MAPPABLE \| PIN_NONBLOCK);
439	1053	if (IS_ERR(vma)) {
440	1054	memset(&cache->node, 0, sizeof(cache->node));
441		− ret = drm_mm_insert_node_in_range
	1055	+ err = drm_mm_insert_node_in_range
442	1056	(&ggtt->base.mm, &cache->node,
443	1057	PAGE_SIZE, 0, I915_COLOR_UNEVICTABLE,
444	1058	0, ggtt->mappable_end,
445	1059	DRM_MM_INSERT_LOW);
446		− if (ret) /* no inactive aperture space, use cpu reloc */
	1060	+ if (err) /* no inactive aperture space, use cpu reloc */
447	1061	return NULL;
448	1062	} else {
449		− ret = i915_vma_put_fence(vma);
450		− if (ret) {
	1063	+ err = i915_vma_put_fence(vma);
	1064	+ if (err) {
451	1065	i915_vma_unpin(vma);
452		− return ERR_PTR(ret);
	1066	+ return ERR_PTR(err);
453	1067	}
454	1068
455	1069	cache->node.start = vma->node.start;
@@ -467,7 +1081,8 @@ static void reloc_iomap(struct drm_i915_gem_object obj, · 11 unmodified lines
467	1081	offset += page << PAGE_SHIFT;
468	1082	}
469	1083
470		− vaddr = (void __force *) io_mapping_map_atomic_wc(&cache->i915->ggtt.mappable, offset);
	1084	+ vaddr = (void __force *)io_mapping_map_atomic_wc(&ggtt->mappable,
	1085	+ offset);
471	1086	cache->page = page;
472	1087	cache->vaddr = (unsigned long)vaddr;
473	1088
@@ -476,7 +1091,7 @@ static void reloc_iomap(struct drm_i915_gem_object obj, · 2 unmodified lines
476	1091
477	1092	static void reloc_vaddr(struct drm_i915_gem_object obj,
478	1093	struct reloc_cache *cache,
479		− int page)
	1094	+ unsigned long page)
480	1095	{
481	1096	void *vaddr;
482	1097
@@ -503,7 +1118,8 @@ static void clflush_write32(u32 *addr, u32 value, unsigned int flushes) · 20 unmodified lines
503	1118
504	1119	*addr = value;
505	1120
506		− /* Writes to the same cacheline are serialised by the CPU
	1121	+ /*
	1122	+ * Writes to the same cacheline are serialised by the CPU
507	1123	* (including clflush). On the write path, we only require
508	1124	* that it hits memory in an orderly fashion and place
509	1125	* mb barriers at the start and end of the relocation phase
@@ -515,25 +1131,201 @@ static void clflush_write32(u32 *addr, u32 value, unsigned int flushes) · 5 unmodified lines
515	1131	*addr = value;
516	1132	}
517	1133
518		−static int
519		−relocate_entry(struct drm_i915_gem_object *obj,
	1134	+static int __reloc_gpu_alloc(struct i915_execbuffer *eb,
	1135	+ struct i915_vma *vma,
	1136	+ unsigned int len)
	1137	+{
	1138	+ struct reloc_cache *cache = &eb->reloc_cache;
	1139	+ struct drm_i915_gem_object *obj;
	1140	+ struct drm_i915_gem_request *rq;
	1141	+ struct i915_vma *batch;
	1142	+ u32 *cmd;
	1143	+ int err;
	1144	+
	1145	+ GEM_BUG_ON(vma->obj->base.write_domain & I915_GEM_DOMAIN_CPU);
	1146	+
	1147	+ obj = i915_gem_batch_pool_get(&eb->engine->batch_pool, PAGE_SIZE);
	1148	+ if (IS_ERR(obj))
	1149	+ return PTR_ERR(obj);
	1150	+
	1151	+ cmd = i915_gem_object_pin_map(obj,
	1152	+ cache->has_llc ? I915_MAP_WB : I915_MAP_WC);
	1153	+ i915_gem_object_unpin_pages(obj);
	1154	+ if (IS_ERR(cmd))
	1155	+ return PTR_ERR(cmd);
	1156	+
	1157	+ err = i915_gem_object_set_to_wc_domain(obj, false);
	1158	+ if (err)
	1159	+ goto err_unmap;
	1160	+
	1161	+ batch = i915_vma_instance(obj, vma->vm, NULL);
	1162	+ if (IS_ERR(batch)) {
	1163	+ err = PTR_ERR(batch);
	1164	+ goto err_unmap;
	1165	+ }
	1166	+
	1167	+ err = i915_vma_pin(batch, 0, 0, PIN_USER \| PIN_NONBLOCK);
	1168	+ if (err)
	1169	+ goto err_unmap;
	1170	+
	1171	+ rq = i915_gem_request_alloc(eb->engine, eb->ctx);
	1172	+ if (IS_ERR(rq)) {
	1173	+ err = PTR_ERR(rq);
	1174	+ goto err_unpin;
	1175	+ }
	1176	+
	1177	+ err = i915_gem_request_await_object(rq, vma->obj, true);
	1178	+ if (err)
	1179	+ goto err_request;
	1180	+
	1181	+ err = eb->engine->emit_flush(rq, EMIT_INVALIDATE);
	1182	+ if (err)
	1183	+ goto err_request;
	1184	+
	1185	+ err = i915_switch_context(rq);
	1186	+ if (err)
	1187	+ goto err_request;
	1188	+
	1189	+ err = eb->engine->emit_bb_start(rq,
	1190	+ batch->node.start, PAGE_SIZE,
	1191	+ cache->gen > 5 ? 0 : I915_DISPATCH_SECURE);
	1192	+ if (err)
	1193	+ goto err_request;
	1194	+
	1195	+ GEM_BUG_ON(!reservation_object_test_signaled_rcu(batch->resv, true));
	1196	+ i915_vma_move_to_active(batch, rq, 0);
	1197	+ reservation_object_lock(batch->resv, NULL);
	1198	+ reservation_object_add_excl_fence(batch->resv, &rq->fence);
	1199	+ reservation_object_unlock(batch->resv);
	1200	+ i915_vma_unpin(batch);
	1201	+
	1202	+ i915_vma_move_to_active(vma, rq, true);
	1203	+ reservation_object_lock(vma->resv, NULL);
	1204	+ reservation_object_add_excl_fence(vma->resv, &rq->fence);
	1205	+ reservation_object_unlock(vma->resv);
	1206	+
	1207	+ rq->batch = batch;
	1208	+
	1209	+ cache->rq = rq;
	1210	+ cache->rq_cmd = cmd;
	1211	+ cache->rq_size = 0;
	1212	+
	1213	+ /* Return with batch mapping (cmd) still pinned */
	1214	+ return 0;
	1215	+
	1216	+err_request:
	1217	+ i915_add_request(rq);
	1218	+err_unpin:
	1219	+ i915_vma_unpin(batch);
	1220	+err_unmap:
	1221	+ i915_gem_object_unpin_map(obj);
	1222	+ return err;
	1223	+}
	1224	+
	1225	+static u32 reloc_gpu(struct i915_execbuffer eb,
	1226	+ struct i915_vma *vma,
	1227	+ unsigned int len)
	1228	+{
	1229	+ struct reloc_cache *cache = &eb->reloc_cache;
	1230	+ u32 *cmd;
	1231	+
	1232	+ if (cache->rq_size > PAGE_SIZE/sizeof(u32) - (len + 1))
	1233	+ reloc_gpu_flush(cache);
	1234	+
	1235	+ if (unlikely(!cache->rq)) {
	1236	+ int err;
	1237	+
	1238	+ err = __reloc_gpu_alloc(eb, vma, len);
	1239	+ if (unlikely(err))
	1240	+ return ERR_PTR(err);
	1241	+ }
	1242	+
	1243	+ cmd = cache->rq_cmd + cache->rq_size;
	1244	+ cache->rq_size += len;
	1245	+
	1246	+ return cmd;
	1247	+}
	1248	+
	1249	+static u64
	1250	+relocate_entry(struct i915_vma *vma,
520	1251	const struct drm_i915_gem_relocation_entry *reloc,
521		− struct reloc_cache *cache,
522		− u64 target_offset)
	1252	+ struct i915_execbuffer *eb,
	1253	+ const struct i915_vma *target)
523	1254	{
524	1255	u64 offset = reloc->offset;
525		− bool wide = cache->use_64bit_reloc;
	1256	+ u64 target_offset = relocation_target(reloc, target);
	1257	+ bool wide = eb->reloc_cache.use_64bit_reloc;
526	1258	void *vaddr;
527	1259
528		− target_offset = relocation_target(reloc, target_offset);
	1260	+ if (!eb->reloc_cache.vaddr &&
	1261	+ (DBG_FORCE_RELOC == FORCE_GPU_RELOC \|\|
	1262	+ !reservation_object_test_signaled_rcu(vma->resv, true))) {
	1263	+ const unsigned int gen = eb->reloc_cache.gen;
	1264	+ unsigned int len;
	1265	+ u32 *batch;
	1266	+ u64 addr;
	1267	+
	1268	+ if (wide)
	1269	+ len = offset & 7 ? 8 : 5;
	1270	+ else if (gen >= 4)
	1271	+ len = 4;
	1272	+ else if (gen >= 3)
	1273	+ len = 3;
	1274	+ else /* On gen2 MI_STORE_DWORD_IMM uses a physical address */
	1275	+ goto repeat;
	1276	+
	1277	+ batch = reloc_gpu(eb, vma, len);
	1278	+ if (IS_ERR(batch))
	1279	+ goto repeat;
	1280	+
	1281	+ addr = gen8_canonical_addr(vma->node.start + offset);
	1282	+ if (wide) {
	1283	+ if (offset & 7) {
	1284	+ *batch++ = MI_STORE_DWORD_IMM_GEN4;
	1285	+ *batch++ = lower_32_bits(addr);
	1286	+ *batch++ = upper_32_bits(addr);
	1287	+ *batch++ = lower_32_bits(target_offset);
	1288	+
	1289	+ addr = gen8_canonical_addr(addr + 4);
	1290	+
	1291	+ *batch++ = MI_STORE_DWORD_IMM_GEN4;
	1292	+ *batch++ = lower_32_bits(addr);
	1293	+ *batch++ = upper_32_bits(addr);
	1294	+ *batch++ = upper_32_bits(target_offset);
	1295	+ } else {
	1296	+ *batch++ = (MI_STORE_DWORD_IMM_GEN4 \| (1 << 21)) + 1;
	1297	+ *batch++ = lower_32_bits(addr);
	1298	+ *batch++ = upper_32_bits(addr);
	1299	+ *batch++ = lower_32_bits(target_offset);
	1300	+ *batch++ = upper_32_bits(target_offset);
	1301	+ }
	1302	+ } else if (gen >= 6) {
	1303	+ *batch++ = MI_STORE_DWORD_IMM_GEN4;
	1304	+ *batch++ = 0;
	1305	+ *batch++ = addr;
	1306	+ *batch++ = target_offset;
	1307	+ } else if (gen >= 4) {
	1308	+ *batch++ = MI_STORE_DWORD_IMM_GEN4 \| MI_USE_GGTT;
	1309	+ *batch++ = 0;
	1310	+ *batch++ = addr;
	1311	+ *batch++ = target_offset;
	1312	+ } else {
	1313	+ *batch++ = MI_STORE_DWORD_IMM \| MI_MEM_VIRTUAL;
	1314	+ *batch++ = addr;
	1315	+ *batch++ = target_offset;
	1316	+ }
	1317	+
	1318	+ goto out;
	1319	+ }
	1320	+
529	1321	repeat:
530		− vaddr = reloc_vaddr(obj, cache, offset >> PAGE_SHIFT);
	1322	+ vaddr = reloc_vaddr(vma->obj, &eb->reloc_cache, offset >> PAGE_SHIFT);
531	1323	if (IS_ERR(vaddr))
532	1324	return PTR_ERR(vaddr);
533	1325
534	1326	clflush_write32(vaddr + offset_in_page(offset),
535	1327	lower_32_bits(target_offset),
536		− cache->vaddr);
	1328	+ eb->reloc_cache.vaddr);
537	1329
538	1330	if (wide) {
539	1331	offset += sizeof(u32);
@@ -542,48 +1334,29 @@ repeat: · 2 unmodified lines
542	1334	goto repeat;
543	1335	}
544	1336
545		− return 0;
	1337	+out:
	1338	+ return target->node.start \| UPDATE;
546	1339	}
547	1340
548		−static int
549		−i915_gem_execbuffer_relocate_entry(struct drm_i915_gem_object *obj,
550		− struct eb_vmas *eb,
551		− struct drm_i915_gem_relocation_entry *reloc,
552		− struct reloc_cache *cache)
	1341	+static u64
	1342	+eb_relocate_entry(struct i915_execbuffer *eb,
	1343	+ struct i915_vma *vma,
	1344	+ const struct drm_i915_gem_relocation_entry *reloc)
553	1345	{
554		− struct drm_i915_private *dev_priv = to_i915(obj->base.dev);
555		− struct drm_gem_object *target_obj;
556		− struct drm_i915_gem_object *target_i915_obj;
557		− struct i915_vma *target_vma;
558		− uint64_t target_offset;
559		− int ret;
	1346	+ struct i915_vma *target;
	1347	+ int err;
560	1348
561	1349	/* we've already hold a reference to all valid objects */
562		− target_vma = eb_get_vma(eb, reloc->target_handle);
563		− if (unlikely(target_vma == NULL))
	1350	+ target = eb_get_vma(eb, reloc->target_handle);
	1351	+ if (unlikely(!target))
564	1352	return -ENOENT;
565		− target_i915_obj = target_vma->obj;
566		− target_obj = &target_vma->obj->base;
567		−
568		− target_offset = gen8_canonical_addr(target_vma->node.start);
569		−
570		− /* Sandybridge PPGTT errata: We need a global gtt mapping for MI and
571		− * pipe_control writes because the gpu doesn't properly redirect them
572		− * through the ppgtt for non_secure batchbuffers. */
573		− if (unlikely(IS_GEN6(dev_priv) &&
574		− reloc->write_domain == I915_GEM_DOMAIN_INSTRUCTION)) {
575		− ret = i915_vma_bind(target_vma, target_i915_obj->cache_level,
576		− PIN_GLOBAL);
577		− if (WARN_ONCE(ret, "Unexpected failure to bind target VMA!"))
578		− return ret;
579		− }
580	1353
581	1354	/* Validate that the target is in a valid r/w GPU domain */
582	1355	if (unlikely(reloc->write_domain & (reloc->write_domain - 1))) {
583	1356	DRM_DEBUG("reloc with multiple write domains: "
584		− "obj %p target %d offset %d "
	1357	+ "target %d offset %d "
585	1358	"read %08x write %08x",
586		− obj, reloc->target_handle,
	1359	+ reloc->target_handle,
587	1360	(int) reloc->offset,
588	1361	reloc->read_domains,
589	1362	reloc->write_domain);
@@ -592,75 +1365,103 @@ i915_gem_execbuffer_relocate_entry(struct drm_i915_gem_object *obj, · 2 unmodified lines
592	1365	if (unlikely((reloc->write_domain \| reloc->read_domains)
593	1366	& ~I915_GEM_GPU_DOMAINS)) {
594	1367	DRM_DEBUG("reloc with read/write non-GPU domains: "
595		− "obj %p target %d offset %d "
	1368	+ "target %d offset %d "
596	1369	"read %08x write %08x",
597		− obj, reloc->target_handle,
	1370	+ reloc->target_handle,
598	1371	(int) reloc->offset,
599	1372	reloc->read_domains,
600	1373	reloc->write_domain);
601	1374	return -EINVAL;
602	1375	}
603	1376
604		− target_obj->pending_read_domains \|= reloc->read_domains;
605		− target_obj->pending_write_domain \|= reloc->write_domain;
	1377	+ if (reloc->write_domain) {
	1378	+ target->exec_entry->flags \|= EXEC_OBJECT_WRITE;
	1379	+
	1380	+ /*
	1381	+ * Sandybridge PPGTT errata: We need a global gtt mapping
	1382	+ * for MI and pipe_control writes because the gpu doesn't
	1383	+ * properly redirect them through the ppgtt for non_secure
	1384	+ * batchbuffers.
	1385	+ */
	1386	+ if (reloc->write_domain == I915_GEM_DOMAIN_INSTRUCTION &&
	1387	+ IS_GEN6(eb->i915)) {
	1388	+ err = i915_vma_bind(target, target->obj->cache_level,
	1389	+ PIN_GLOBAL);
	1390	+ if (WARN_ONCE(err,
	1391	+ "Unexpected failure to bind target VMA!"))
	1392	+ return err;
	1393	+ }
	1394	+ }
606	1395
607		− /* If the relocation already has the right value in it, no
	1396	+ /*
	1397	+ * If the relocation already has the right value in it, no
608	1398	* more work needs to be done.
609	1399	*/
610		− if (target_offset == reloc->presumed_offset)
	1400	+ if (!DBG_FORCE_RELOC &&
	1401	+ gen8_canonical_addr(target->node.start) == reloc->presumed_offset)
611	1402	return 0;
612	1403
613	1404	/* Check that the relocation address is valid... */
614	1405	if (unlikely(reloc->offset >
615		− obj->base.size - (cache->use_64bit_reloc ? 8 : 4))) {
	1406	+ vma->size - (eb->reloc_cache.use_64bit_reloc ? 8 : 4))) {
616	1407	DRM_DEBUG("Relocation beyond object bounds: "
617		− "obj %p target %d offset %d size %d.\n",
618		− obj, reloc->target_handle,
619		− (int) reloc->offset,
620		− (int) obj->base.size);
	1408	+ "target %d offset %d size %d.\n",
	1409	+ reloc->target_handle,
	1410	+ (int)reloc->offset,
	1411	+ (int)vma->size);
621	1412	return -EINVAL;
622	1413	}
623	1414	if (unlikely(reloc->offset & 3)) {
624	1415	DRM_DEBUG("Relocation not 4-byte aligned: "
625		− "obj %p target %d offset %d.\n",
626		− obj, reloc->target_handle,
627		− (int) reloc->offset);
	1416	+ "target %d offset %d.\n",
	1417	+ reloc->target_handle,
	1418	+ (int)reloc->offset);
628	1419	return -EINVAL;
629	1420	}
630	1421
631		− ret = relocate_entry(obj, reloc, cache, target_offset);
632		− if (ret)
633		− return ret;
	1422	+ /*
	1423	+ * If we write into the object, we need to force the synchronisation
	1424	+ * barrier, either with an asynchronous clflush or if we executed the
	1425	+ * patching using the GPU (though that should be serialised by the
	1426	+ * timeline). To be completely sure, and since we are required to
	1427	+ * do relocations we are already stalling, disable the user's opt
	1428	+ * of our synchronisation.
	1429	+ */
	1430	+ vma->exec_entry->flags &= ~EXEC_OBJECT_ASYNC;
634	1431
635	1432	/* and update the user's relocation entry */
636		− reloc->presumed_offset = target_offset;
637		− return 0;
	1433	+ return relocate_entry(vma, reloc, eb, target);
638	1434	}
639	1435
640		−static int
641		−i915_gem_execbuffer_relocate_vma(struct i915_vma *vma,
642		− struct eb_vmas *eb)
	1436	+static int eb_relocate_vma(struct i915_execbuffer eb, struct i915_vma vma)
643	1437	{
644	1438	#define N_RELOC(x) ((x) / sizeof(struct drm_i915_gem_relocation_entry))
645		− struct drm_i915_gem_relocation_entry stack_reloc[N_RELOC(512)];
646		− struct drm_i915_gem_relocation_entry __user *user_relocs;
647		− struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
648		− struct reloc_cache cache;
649		− int remain, ret = 0;
650		−
651		− user_relocs = u64_to_user_ptr(entry->relocs_ptr);
652		− reloc_cache_init(&cache, eb->i915);
	1439	+ struct drm_i915_gem_relocation_entry stack[N_RELOC(512)];
	1440	+ struct drm_i915_gem_relocation_entry __user *urelocs;
	1441	+ const struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
	1442	+ unsigned int remain;
653	1443
	1444	+ urelocs = u64_to_user_ptr(entry->relocs_ptr);
654	1445	remain = entry->relocation_count;
655		− while (remain) {
656		− struct drm_i915_gem_relocation_entry *r = stack_reloc;
657		− unsigned long unwritten;
658		− unsigned int count;
	1446	+ if (unlikely(remain > N_RELOC(ULONG_MAX)))
	1447	+ return -EINVAL;
659	1448
660		− count = min_t(unsigned int, remain, ARRAY_SIZE(stack_reloc));
661		− remain -= count;
	1449	+ /*
	1450	+ * We must check that the entire relocation array is safe
	1451	+ * to read. However, if the array is not writable the user loses
	1452	+ * the updated relocation values.
	1453	+ */
	1454	+ if (unlikely(!access_ok(VERIFY_READ, urelocs, remain*sizeof(urelocs))))
	1455	+ return -EFAULT;
	1456	+
	1457	+ do {
	1458	+ struct drm_i915_gem_relocation_entry *r = stack;
	1459	+ unsigned int count =
	1460	+ min_t(unsigned int, remain, ARRAY_SIZE(stack));
	1461	+ unsigned int copied;
662	1462
663		− /* This is the fast path and we cannot handle a pagefault
	1463	+ /*
	1464	+ * This is the fast path and we cannot handle a pagefault
664	1465	* whilst holding the struct mutex lest the user pass in the
665	1466	* relocations contained within a mmaped bo. For in such a case
666	1467	* we, the page fault handler would call i915_gem_fault() and
@@ -668,489 +1469,408 @@ i915_gem_execbuffer_relocate_vma(struct i915_vma *vma, · 1 unmodified line
668	1469	* this is bad and so lockdep complains vehemently.
669	1470	*/
670	1471	pagefault_disable();
671		− unwritten = __copy_from_user_inatomic(r, user_relocs, count*sizeof(r[0]));
	1472	+ copied = __copy_from_user_inatomic(r, urelocs, count * sizeof(r[0]));
672	1473	pagefault_enable();
673		− if (unlikely(unwritten)) {
674		− ret = -EFAULT;
	1474	+ if (unlikely(copied)) {
	1475	+ remain = -EFAULT;
675	1476	goto out;
676	1477	}
677	1478
	1479	+ remain -= count;
678	1480	do {
679		− u64 offset = r->presumed_offset;
	1481	+ u64 offset = eb_relocate_entry(eb, vma, r);
680	1482
681		− ret = i915_gem_execbuffer_relocate_entry(vma->obj, eb, r, &cache);
682		− if (ret)
	1483	+ if (likely(offset == 0)) {
	1484	+ } else if ((s64)offset < 0) {
	1485	+ remain = (int)offset;
683	1486	goto out;
684		−
685		− if (r->presumed_offset != offset) {
686		− pagefault_disable();
687		− unwritten = __put_user(r->presumed_offset,
688		− &user_relocs->presumed_offset);
689		− pagefault_enable();
690		− if (unlikely(unwritten)) {
691		− /* Note that reporting an error now
692		− * leaves everything in an inconsistent
693		− * state as we have already changed
694		− * the relocation value inside the
695		− * object. As we have not changed the
696		− * reloc.presumed_offset or will not
697		− * change the execobject.offset, on the
698		− * call we may not rewrite the value
699		− * inside the object, leaving it
700		− * dangling and causing a GPU hang.
701		− */
702		− ret = -EFAULT;
703		− goto out;
704		− }
	1487	+ } else {
	1488	+ /*
	1489	+ * Note that reporting an error now
	1490	+ * leaves everything in an inconsistent
	1491	+ * state as we have already changed
	1492	+ * the relocation value inside the
	1493	+ * object. As we have not changed the
	1494	+ * reloc.presumed_offset or will not
	1495	+ * change the execobject.offset, on the
	1496	+ * call we may not rewrite the value
	1497	+ * inside the object, leaving it
	1498	+ * dangling and causing a GPU hang. Unless
	1499	+ * userspace dynamically rebuilds the
	1500	+ * relocations on each execbuf rather than
	1501	+ * presume a static tree.
	1502	+ *
	1503	+ * We did previously check if the relocations
	1504	+ * were writable (access_ok), an error now
	1505	+ * would be a strange race with mprotect,
	1506	+ * having already demonstrated that we
	1507	+ * can read from this userspace address.
	1508	+ */
	1509	+ offset = gen8_canonical_addr(offset & ~UPDATE);
	1510	+ __put_user(offset,
	1511	+ &urelocs[r-stack].presumed_offset);
705	1512	}
706		−
707		− user_relocs++;
708		− r++;
709		− } while (--count);
710		− }
711		−
	1513	+ } while (r++, --count);
	1514	+ urelocs += ARRAY_SIZE(stack);
	1515	+ } while (remain);
712	1516	out:
713		− reloc_cache_fini(&cache);
714		− return ret;
715		−#undef N_RELOC
	1517	+ reloc_cache_reset(&eb->reloc_cache);
	1518	+ return remain;
716	1519	}
717	1520
718	1521	static int
719		−i915_gem_execbuffer_relocate_vma_slow(struct i915_vma *vma,
720		− struct eb_vmas *eb,
721		− struct drm_i915_gem_relocation_entry *relocs)
	1522	+eb_relocate_vma_slow(struct i915_execbuffer eb, struct i915_vma vma)
722	1523	{
723	1524	const struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
724		− struct reloc_cache cache;
725		− int i, ret = 0;
	1525	+ struct drm_i915_gem_relocation_entry *relocs =
	1526	+ u64_to_ptr(typeof(*relocs), entry->relocs_ptr);
	1527	+ unsigned int i;
	1528	+ int err;
726	1529
727		− reloc_cache_init(&cache, eb->i915);
728	1530	for (i = 0; i < entry->relocation_count; i++) {
729		− ret = i915_gem_execbuffer_relocate_entry(vma->obj, eb, &relocs[i], &cache);
730		− if (ret)
731		− break;
732		− }
733		− reloc_cache_fini(&cache);
	1531	+ u64 offset = eb_relocate_entry(eb, vma, &relocs[i]);
734	1532
735		− return ret;
	1533	+ if ((s64)offset < 0) {
	1534	+ err = (int)offset;
	1535	+ goto err;
	1536	+ }
	1537	+ }
	1538	+ err = 0;
	1539	+err:
	1540	+ reloc_cache_reset(&eb->reloc_cache);
	1541	+ return err;
736	1542	}
737	1543
738		−static int
739		−i915_gem_execbuffer_relocate(struct eb_vmas *eb)
	1544	+static int check_relocations(const struct drm_i915_gem_exec_object2 *entry)
740	1545	{
741		− struct i915_vma *vma;
742		− int ret = 0;
	1546	+ const char __user addr, end;
	1547	+ unsigned long size;
	1548	+ char __maybe_unused c;
743	1549
744		− list_for_each_entry(vma, &eb->vmas, exec_list) {
745		− ret = i915_gem_execbuffer_relocate_vma(vma, eb);
746		− if (ret)
747		− break;
748		− }
	1550	+ size = entry->relocation_count;
	1551	+ if (size == 0)
	1552	+ return 0;
749	1553
750		− return ret;
751		−}
	1554	+ if (size > N_RELOC(ULONG_MAX))
	1555	+ return -EINVAL;
752	1556
753		−static bool only_mappable_for_reloc(unsigned int flags)
754		−{
755		− return (flags & (EXEC_OBJECT_NEEDS_FENCE \| __EXEC_OBJECT_NEEDS_MAP)) ==
756		− __EXEC_OBJECT_NEEDS_MAP;
	1557	+ addr = u64_to_user_ptr(entry->relocs_ptr);
	1558	+ size *= sizeof(struct drm_i915_gem_relocation_entry);
	1559	+ if (!access_ok(VERIFY_READ, addr, size))
	1560	+ return -EFAULT;
	1561	+
	1562	+ end = addr + size;
	1563	+ for (; addr < end; addr += PAGE_SIZE) {
	1564	+ int err = __get_user(c, addr);
	1565	+ if (err)
	1566	+ return err;
	1567	+ }
	1568	+ return __get_user(c, end - 1);
757	1569	}
758	1570
759		−static int
760		−i915_gem_execbuffer_reserve_vma(struct i915_vma *vma,
761		− struct intel_engine_cs *engine,
762		− bool *need_reloc)
	1571	+static int eb_copy_relocations(const struct i915_execbuffer *eb)
763	1572	{
764		− struct drm_i915_gem_object *obj = vma->obj;
765		− struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
766		− uint64_t flags;
767		− int ret;
	1573	+ const unsigned int count = eb->buffer_count;
	1574	+ unsigned int i;
	1575	+ int err;
768	1576
769		− flags = PIN_USER;
770		− if (entry->flags & EXEC_OBJECT_NEEDS_GTT)
771		− flags \|= PIN_GLOBAL;
	1577	+ for (i = 0; i < count; i++) {
	1578	+ const unsigned int nreloc = eb->exec[i].relocation_count;
	1579	+ struct drm_i915_gem_relocation_entry __user *urelocs;
	1580	+ struct drm_i915_gem_relocation_entry *relocs;
	1581	+ unsigned long size;
	1582	+ unsigned long copied;
772	1583
773		− if (!drm_mm_node_allocated(&vma->node)) {
774		− /* Wa32bitGeneralStateOffset & Wa32bitInstructionBaseOffset,
775		− * limit address to the first 4GBs for unflagged objects.
776		− */
777		− if ((entry->flags & EXEC_OBJECT_SUPPORTS_48B_ADDRESS) == 0)
778		− flags \|= PIN_ZONE_4G;
779		− if (entry->flags & __EXEC_OBJECT_NEEDS_MAP)
780		− flags \|= PIN_GLOBAL \| PIN_MAPPABLE;
781		− if (entry->flags & __EXEC_OBJECT_NEEDS_BIAS)
782		− flags \|= BATCH_OFFSET_BIAS \| PIN_OFFSET_BIAS;
783		− if (entry->flags & EXEC_OBJECT_PINNED)
784		− flags \|= entry->offset \| PIN_OFFSET_FIXED;
785		− if ((flags & PIN_MAPPABLE) == 0)
786		− flags \|= PIN_HIGH;
787		− }
788		−
789		− ret = i915_vma_pin(vma,
790		− entry->pad_to_size,
791		− entry->alignment,
792		− flags);
793		− if ((ret == -ENOSPC \|\| ret == -E2BIG) &&
794		− only_mappable_for_reloc(entry->flags))
795		− ret = i915_vma_pin(vma,
796		− entry->pad_to_size,
797		− entry->alignment,
798		− flags & ~PIN_MAPPABLE);
799		− if (ret)
800		− return ret;
	1584	+ if (nreloc == 0)
	1585	+ continue;
801	1586
802		− entry->flags \|= __EXEC_OBJECT_HAS_PIN;
	1587	+ err = check_relocations(&eb->exec[i]);
	1588	+ if (err)
	1589	+ goto err;
803	1590
804		− if (entry->flags & EXEC_OBJECT_NEEDS_FENCE) {
805		− ret = i915_vma_get_fence(vma);
806		− if (ret)
807		− return ret;
	1591	+ urelocs = u64_to_user_ptr(eb->exec[i].relocs_ptr);
	1592	+ size = nreloc * sizeof(*relocs);
808	1593
809		− if (i915_vma_pin_fence(vma))
810		− entry->flags \|= __EXEC_OBJECT_HAS_FENCE;
811		− }
	1594	+ relocs = kvmalloc_array(size, 1, GFP_TEMPORARY);
	1595	+ if (!relocs) {
	1596	+ kvfree(relocs);
	1597	+ err = -ENOMEM;
	1598	+ goto err;
	1599	+ }
812	1600
813		− if (entry->offset != vma->node.start) {
814		− entry->offset = vma->node.start;
815		− *need_reloc = true;
816		− }
	1601	+ /* copy_from_user is limited to < 4GiB */
	1602	+ copied = 0;
	1603	+ do {
	1604	+ unsigned int len =
	1605	+ min_t(u64, BIT_ULL(31), size - copied);
	1606	+
	1607	+ if (__copy_from_user((char *)relocs + copied,
	1608	+ (char *)urelocs + copied,
	1609	+ len)) {
	1610	+ kvfree(relocs);
	1611	+ err = -EFAULT;
	1612	+ goto err;
	1613	+ }
	1614	+
	1615	+ copied += len;
	1616	+ } while (copied < size);
	1617	+
	1618	+ /*
	1619	+ * As we do not update the known relocation offsets after
	1620	+ * relocating (due to the complexities in lock handling),
	1621	+ * we need to mark them as invalid now so that we force the
	1622	+ * relocation processing next time. Just in case the target
	1623	+ * object is evicted and then rebound into its old
	1624	+ * presumed_offset before the next execbuffer - if that
	1625	+ * happened we would make the mistake of assuming that the
	1626	+ * relocations were valid.
	1627	+ */
	1628	+ user_access_begin();
	1629	+ for (copied = 0; copied < nreloc; copied++)
	1630	+ unsafe_put_user(-1,
	1631	+ &urelocs[copied].presumed_offset,
	1632	+ end_user);
	1633	+end_user:
	1634	+ user_access_end();
817	1635
818		− if (entry->flags & EXEC_OBJECT_WRITE) {
819		− obj->base.pending_read_domains = I915_GEM_DOMAIN_RENDER;
820		− obj->base.pending_write_domain = I915_GEM_DOMAIN_RENDER;
	1636	+ eb->exec[i].relocs_ptr = (uintptr_t)relocs;
821	1637	}
822	1638
823	1639	return 0;
	1640	+
	1641	+err:
	1642	+ while (i--) {
	1643	+ struct drm_i915_gem_relocation_entry *relocs =
	1644	+ u64_to_ptr(typeof(*relocs), eb->exec[i].relocs_ptr);
	1645	+ if (eb->exec[i].relocation_count)
	1646	+ kvfree(relocs);
	1647	+ }
	1648	+ return err;
824	1649	}
825	1650
826		−static bool
827		−need_reloc_mappable(struct i915_vma *vma)
	1651	+static int eb_prefault_relocations(const struct i915_execbuffer *eb)
828	1652	{
829		− struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
830		−
831		− if (entry->relocation_count == 0)
832		− return false;
	1653	+ const unsigned int count = eb->buffer_count;
	1654	+ unsigned int i;
833	1655
834		− if (!i915_vma_is_ggtt(vma))
835		− return false;
	1656	+ if (unlikely(i915.prefault_disable))
	1657	+ return 0;
836	1658
837		− /* See also use_cpu_reloc() */
838		− if (HAS_LLC(to_i915(vma->obj->base.dev)))
839		− return false;
	1659	+ for (i = 0; i < count; i++) {
	1660	+ int err;
840	1661
841		− if (vma->obj->base.write_domain == I915_GEM_DOMAIN_CPU)
842		− return false;
	1662	+ err = check_relocations(&eb->exec[i]);
	1663	+ if (err)
	1664	+ return err;
	1665	+ }
843	1666
844		− return true;
	1667	+ return 0;
845	1668	}
846	1669
847		−static bool
848		−eb_vma_misplaced(struct i915_vma *vma)
	1670	+static noinline int eb_relocate_slow(struct i915_execbuffer *eb)
849	1671	{
850		− struct drm_i915_gem_exec_object2 *entry = vma->exec_entry;
851		−
852		− WARN_ON(entry->flags & __EXEC_OBJECT_NEEDS_MAP &&
853		− !i915_vma_is_ggtt(vma));
854		−
855		− if (entry->alignment && !IS_ALIGNED(vma->node.start, entry->alignment))
856		− return true;
857		−
858		− if (vma->node.size < entry->pad_to_size)
859		− return true;
860		−
861		− if (entry->flags & EXEC_OBJECT_PINNED &&
862		− vma->node.start != entry->offset)
863		− return true;
864		−
865		− if (entry->flags & __EXEC_OBJECT_NEEDS_BIAS &&
866		− vma->node.start < BATCH_OFFSET_BIAS)
867		− return true;
868		−
869		− /* avoid costly ping-pong once a batch bo ended up non-mappable */
870		− if (entry->flags & __EXEC_OBJECT_NEEDS_MAP &&
871		− !i915_vma_is_map_and_fenceable(vma))
872		− return !only_mappable_for_reloc(entry->flags);
	1672	+ struct drm_device *dev = &eb->i915->drm;
	1673	+ bool have_copy = false;
	1674	+ struct i915_vma *vma;
	1675	+ int err = 0;
873	1676
874		− if ((entry->flags & EXEC_OBJECT_SUPPORTS_48B_ADDRESS) == 0 &&
875		− (vma->node.start + vma->node.size - 1) >> 32)
876		− return true;
	1677	+repeat:
	1678	+ if (signal_pending(current)) {
	1679	+ err = -ERESTARTSYS;
	1680	+ goto out;
	1681	+ }
877	1682
878		− return false;
879		−}
	1683	+ /* We may process another execbuffer during the unlock... */
	1684	+ eb_reset_vmas(eb);
	1685	+ mutex_unlock(&dev->struct_mutex);
880	1686
881		−static int
882		−i915_gem_execbuffer_reserve(struct intel_engine_cs *engine,
883		− struct list_head *vmas,
884		− struct i915_gem_context *ctx,
885		− bool *need_relocs)
886		−{
887		− struct drm_i915_gem_object *obj;
888		− struct i915_vma *vma;
889		− struct i915_address_space *vm;
890		− struct list_head ordered_vmas;
891		− struct list_head pinned_vmas;
892		− bool has_fenced_gpu_access = INTEL_GEN(engine->i915) < 4;
893		− bool needs_unfenced_map = INTEL_INFO(engine->i915)->unfenced_needs_alignment;
894		− int retry;
895		−
896		− vm = list_first_entry(vmas, struct i915_vma, exec_list)->vm;
897		−
898		− INIT_LIST_HEAD(&ordered_vmas);
899		− INIT_LIST_HEAD(&pinned_vmas);
900		− while (!list_empty(vmas)) {
901		− struct drm_i915_gem_exec_object2 *entry;
902		− bool need_fence, need_mappable;
903		−
904		− vma = list_first_entry(vmas, struct i915_vma, exec_list);
905		− obj = vma->obj;
906		− entry = vma->exec_entry;
907		−
908		− if (ctx->flags & CONTEXT_NO_ZEROMAP)
909		− entry->flags \|= __EXEC_OBJECT_NEEDS_BIAS;
910		−
911		− if (!has_fenced_gpu_access)
912		− entry->flags &= ~EXEC_OBJECT_NEEDS_FENCE;
913		− need_fence =
914		− (entry->flags & EXEC_OBJECT_NEEDS_FENCE \|\|
915		− needs_unfenced_map) &&
916		− i915_gem_object_is_tiled(obj);
917		− need_mappable = need_fence \|\| need_reloc_mappable(vma);
918		−
919		− if (entry->flags & EXEC_OBJECT_PINNED)
920		− list_move_tail(&vma->exec_list, &pinned_vmas);
921		− else if (need_mappable) {
922		− entry->flags \|= __EXEC_OBJECT_NEEDS_MAP;
923		− list_move(&vma->exec_list, &ordered_vmas);
924		− } else
925		− list_move_tail(&vma->exec_list, &ordered_vmas);
926		−
927		− obj->base.pending_read_domains = I915_GEM_GPU_DOMAINS & ~I915_GEM_DOMAIN_COMMAND;
928		− obj->base.pending_write_domain = 0;
929		− }
930		− list_splice(&ordered_vmas, vmas);
931		− list_splice(&pinned_vmas, vmas);
932		−
933		− /* Attempt to pin all of the buffers into the GTT.
934		− * This is done in 3 phases:
	1687	+ /*
	1688	+ * We take 3 passes through the slowpatch.
935	1689	*
936		− * 1a. Unbind all objects that do not match the GTT constraints for
937		− * the execbuffer (fenceable, mappable, alignment etc).
938		− * 1b. Increment pin count for already bound objects.
939		− * 2. Bind new objects.
940		− * 3. Decrement pin count.
	1690	+ * 1 - we try to just prefault all the user relocation entries and
	1691	+ * then attempt to reuse the atomic pagefault disabled fast path again.
941	1692	*
942		− * This avoid unnecessary unbinding of later objects in order to make
943		− * room for the earlier objects unless we need to defragment.
	1693	+ * 2 - we copy the user entries to a local buffer here outside of the
	1694	+ * local and allow ourselves to wait upon any rendering before
	1695	+ * relocations
	1696	+ *
	1697	+ * 3 - we already have a local copy of the relocation entries, but
	1698	+ * were interrupted (EAGAIN) whilst waiting for the objects, try again.
944	1699	*/
945		− retry = 0;
946		− do {
947		− int ret = 0;
948		−
949		− /* Unbind any ill-fitting objects or pin. */
950		− list_for_each_entry(vma, vmas, exec_list) {
951		− if (!drm_mm_node_allocated(&vma->node))
952		− continue;
953		−
954		− if (eb_vma_misplaced(vma))
955		− ret = i915_vma_unbind(vma);
956		− else
957		− ret = i915_gem_execbuffer_reserve_vma(vma,
958		− engine,
959		− need_relocs);
960		− if (ret)
961		− goto err;
962		− }
963		−
964		− /* Bind fresh objects */
965		− list_for_each_entry(vma, vmas, exec_list) {
966		− if (drm_mm_node_allocated(&vma->node))
967		− continue;
968		−
969		− ret = i915_gem_execbuffer_reserve_vma(vma, engine,
970		− need_relocs);
971		− if (ret)
972		− goto err;
973		− }
974		−
975		−err:
976		− if (ret != -ENOSPC \|\| retry++)
977		− return ret;
978		−
979		− /* Decrement pin count for bound objects */
980		− list_for_each_entry(vma, vmas, exec_list)
981		− i915_gem_execbuffer_unreserve_vma(vma);
	1700	+ if (!err) {
	1701	+ err = eb_prefault_relocations(eb);
	1702	+ } else if (!have_copy) {
	1703	+ err = eb_copy_relocations(eb);
	1704	+ have_copy = err == 0;
	1705	+ } else {
	1706	+ cond_resched();
	1707	+ err = 0;
	1708	+ }
	1709	+ if (err) {
	1710	+ mutex_lock(&dev->struct_mutex);
	1711	+ goto out;
	1712	+ }
982	1713
983		− ret = i915_gem_evict_vm(vm, true);
984		− if (ret)
985		− return ret;
986		− } while (1);
987		−}
	1714	+ /* A frequent cause for EAGAIN are currently unavailable client pages */
	1715	+ flush_workqueue(eb->i915->mm.userptr_wq);
988	1716
989		−static int
990		−i915_gem_execbuffer_relocate_slow(struct drm_device *dev,
991		− struct drm_i915_gem_execbuffer2 *args,
992		− struct drm_file *file,
993		− struct intel_engine_cs *engine,
994		− struct eb_vmas *eb,
995		− struct drm_i915_gem_exec_object2 *exec,
996		− struct i915_gem_context *ctx)
997		−{
998		− struct drm_i915_gem_relocation_entry *reloc;
999		− struct i915_address_space *vm;
1000		− struct i915_vma *vma;
1001		− bool need_relocs;
1002		− int *reloc_offset;
1003		− int i, total, ret;
1004		− unsigned count = args->buffer_count;
	1717	+ err = i915_mutex_lock_interruptible(dev);
	1718	+ if (err) {
	1719	+ mutex_lock(&dev->struct_mutex);
	1720	+ goto out;
	1721	+ }
1005	1722
1006		− vm = list_first_entry(&eb->vmas, struct i915_vma, exec_list)->vm;
	1723	+ /* reacquire the objects */
	1724	+ err = eb_lookup_vmas(eb);
	1725	+ if (err)
	1726	+ goto err;
1007	1727
1008		− /* We may process another execbuffer during the unlock... */
1009		− while (!list_empty(&eb->vmas)) {
1010		− vma = list_first_entry(&eb->vmas, struct i915_vma, exec_list);
1011		− list_del_init(&vma->exec_list);
1012		− i915_gem_execbuffer_unreserve_vma(vma);
1013		− i915_vma_put(vma);
	1728	+ list_for_each_entry(vma, &eb->relocs, reloc_link) {
	1729	+ if (!have_copy) {
	1730	+ pagefault_disable();
	1731	+ err = eb_relocate_vma(eb, vma);
	1732	+ pagefault_enable();
	1733	+ if (err)
	1734	+ goto repeat;
	1735	+ } else {
	1736	+ err = eb_relocate_vma_slow(eb, vma);
	1737	+ if (err)
	1738	+ goto err;
	1739	+ }
1014	1740	}
1015	1741
1016		− mutex_unlock(&dev->struct_mutex);
	1742	+ /*
	1743	+ * Leave the user relocations as are, this is the painfully slow path,
	1744	+ * and we want to avoid the complication of dropping the lock whilst
	1745	+ * having buffers reserved in the aperture and so causing spurious
	1746	+ * ENOSPC for random operations.
	1747	+ */
1017	1748
1018		− total = 0;
1019		− for (i = 0; i < count; i++)
1020		− total += exec[i].relocation_count;
	1749	+err:
	1750	+ if (err == -EAGAIN)
	1751	+ goto repeat;
1021	1752
1022		− reloc_offset = kvmalloc_array(count, sizeof(*reloc_offset), GFP_KERNEL);
1023		− reloc = kvmalloc_array(total, sizeof(*reloc), GFP_KERNEL);
1024		− if (reloc == NULL \|\| reloc_offset == NULL) {
1025		− kvfree(reloc);
1026		− kvfree(reloc_offset);
1027		− mutex_lock(&dev->struct_mutex);
1028		− return -ENOMEM;
1029		− }
	1753	+out:
	1754	+ if (have_copy) {
	1755	+ const unsigned int count = eb->buffer_count;
	1756	+ unsigned int i;
1030	1757
1031		− total = 0;
1032		− for (i = 0; i < count; i++) {
1033		− struct drm_i915_gem_relocation_entry __user *user_relocs;
1034		− u64 invalid_offset = (u64)-1;
1035		− int j;
	1758	+ for (i = 0; i < count; i++) {
	1759	+ const struct drm_i915_gem_exec_object2 *entry =
	1760	+ &eb->exec[i];
	1761	+ struct drm_i915_gem_relocation_entry *relocs;
1036	1762
1037		− user_relocs = u64_to_user_ptr(exec[i].relocs_ptr);
	1763	+ if (!entry->relocation_count)
	1764	+ continue;
1038	1765
1039		− if (copy_from_user(reloc+total, user_relocs,
1040		− exec[i].relocation_count * sizeof(*reloc))) {
1041		− ret = -EFAULT;
1042		− mutex_lock(&dev->struct_mutex);
1043		− goto err;
	1766	+ relocs = u64_to_ptr(typeof(*relocs), entry->relocs_ptr);
	1767	+ kvfree(relocs);
1044	1768	}
	1769	+ }
1045	1770
1046		− /* As we do not update the known relocation offsets after
1047		− * relocating (due to the complexities in lock handling),
1048		− * we need to mark them as invalid now so that we force the
1049		− * relocation processing next time. Just in case the target
1050		− * object is evicted and then rebound into its old
1051		− * presumed_offset before the next execbuffer - if that
1052		− * happened we would make the mistake of assuming that the
1053		− * relocations were valid.
1054		− */
1055		− for (j = 0; j < exec[i].relocation_count; j++) {
1056		− if (__copy_to_user(&user_relocs[j].presumed_offset,
1057		− &invalid_offset,
1058		− sizeof(invalid_offset))) {
1059		− ret = -EFAULT;
1060		− mutex_lock(&dev->struct_mutex);
1061		− goto err;
1062		− }
1063		− }
	1771	+ return err ?: have_copy;
	1772	+}
1064	1773
1065		− reloc_offset[i] = total;
1066		− total += exec[i].relocation_count;
1067		− }
	1774	+static int eb_relocate(struct i915_execbuffer *eb)
	1775	+{
	1776	+ if (eb_lookup_vmas(eb))
	1777	+ goto slow;
1068	1778
1069		− ret = i915_mutex_lock_interruptible(dev);
1070		− if (ret) {
1071		− mutex_lock(&dev->struct_mutex);
1072		− goto err;
	1779	+ /* The objects are in their final locations, apply the relocations. */
	1780	+ if (eb->args->flags & __EXEC_HAS_RELOC) {
	1781	+ struct i915_vma *vma;
	1782	+
	1783	+ list_for_each_entry(vma, &eb->relocs, reloc_link) {
	1784	+ if (eb_relocate_vma(eb, vma))
	1785	+ goto slow;
	1786	+ }
1073	1787	}
1074	1788
1075		− /* reacquire the objects */
1076		− eb_reset(eb);
1077		− ret = eb_lookup_vmas(eb, exec, args, vm, file);
1078		− if (ret)
1079		− goto err;
	1789	+ return 0;
1080	1790
1081		− need_relocs = (args->flags & I915_EXEC_NO_RELOC) == 0;
1082		− ret = i915_gem_execbuffer_reserve(engine, &eb->vmas, ctx,
1083		− &need_relocs);
1084		− if (ret)
1085		− goto err;
	1791	+slow:
	1792	+ return eb_relocate_slow(eb);
	1793	+}
1086	1794
1087		− list_for_each_entry(vma, &eb->vmas, exec_list) {
1088		− int offset = vma->exec_entry - exec;
1089		− ret = i915_gem_execbuffer_relocate_vma_slow(vma, eb,
1090		− reloc + reloc_offset[offset]);
1091		− if (ret)
1092		− goto err;
1093		− }
	1795	+static void eb_export_fence(struct i915_vma *vma,
	1796	+ struct drm_i915_gem_request *req,
	1797	+ unsigned int flags)
	1798	+{
	1799	+ struct reservation_object *resv = vma->resv;
1094	1800
1095		− /* Leave the user relocations as are, this is the painfully slow path,
1096		− * and we want to avoid the complication of dropping the lock whilst
1097		− * having buffers reserved in the aperture and so causing spurious
1098		− * ENOSPC for random operations.
	1801	+ /*
	1802	+ * Ignore errors from failing to allocate the new fence, we can't
	1803	+ * handle an error right now. Worst case should be missed
	1804	+ * synchronisation leading to rendering corruption.
1099	1805	*/
1100		−
1101		−err:
1102		− kvfree(reloc);
1103		− kvfree(reloc_offset);
1104		− return ret;
	1806	+ reservation_object_lock(resv, NULL);
	1807	+ if (flags & EXEC_OBJECT_WRITE)
	1808	+ reservation_object_add_excl_fence(resv, &req->fence);
	1809	+ else if (reservation_object_reserve_shared(resv) == 0)
	1810	+ reservation_object_add_shared_fence(resv, &req->fence);
	1811	+ reservation_object_unlock(resv);
1105	1812	}
1106	1813
1107		−static int
1108		−i915_gem_execbuffer_move_to_gpu(struct drm_i915_gem_request *req,
1109		− struct list_head *vmas)
	1814	+static int eb_move_to_gpu(struct i915_execbuffer *eb)
1110	1815	{
1111		− struct i915_vma *vma;
1112		− int ret;
	1816	+ const unsigned int count = eb->buffer_count;
	1817	+ unsigned int i;
	1818	+ int err;
1113	1819
1114		− list_for_each_entry(vma, vmas, exec_list) {
	1820	+ for (i = 0; i < count; i++) {
	1821	+ const struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
	1822	+ struct i915_vma *vma = exec_to_vma(entry);
1115	1823	struct drm_i915_gem_object *obj = vma->obj;
1116	1824
1117		− if (vma->exec_entry->flags & EXEC_OBJECT_CAPTURE) {
	1825	+ if (entry->flags & EXEC_OBJECT_CAPTURE) {
1118	1826	struct i915_gem_capture_list *capture;
1119	1827
1120	1828	capture = kmalloc(sizeof(*capture), GFP_KERNEL);
1121	1829	if (unlikely(!capture))
1122	1830	return -ENOMEM;
1123	1831
1124		− capture->next = req->capture_list;
	1832	+ capture->next = eb->request->capture_list;
1125	1833	capture->vma = vma;
1126		− req->capture_list = capture;
	1834	+ eb->request->capture_list = capture;
1127	1835	}
1128	1836
1129		− if (vma->exec_entry->flags & EXEC_OBJECT_ASYNC)
1130		− continue;
	1837	+ if (entry->flags & EXEC_OBJECT_ASYNC)
	1838	+ goto skip_flushes;
1131	1839
1132		− if (obj->base.write_domain & I915_GEM_DOMAIN_CPU) {
	1840	+ if (unlikely(obj->cache_dirty && !obj->cache_coherent))
1133	1841	i915_gem_clflush_object(obj, 0);
1134		− obj->base.write_domain = 0;
1135		− }
1136	1842
1137		− ret = i915_gem_request_await_object
1138		− (req, obj, obj->base.pending_write_domain);
1139		− if (ret)
1140		− return ret;
	1843	+ err = i915_gem_request_await_object
	1844	+ (eb->request, obj, entry->flags & EXEC_OBJECT_WRITE);
	1845	+ if (err)
	1846	+ return err;
	1847	+
	1848	+skip_flushes:
	1849	+ i915_vma_move_to_active(vma, eb->request, entry->flags);
	1850	+ __eb_unreserve_vma(vma, entry);
	1851	+ vma->exec_entry = NULL;
	1852	+ }
	1853	+
	1854	+ for (i = 0; i < count; i++) {
	1855	+ const struct drm_i915_gem_exec_object2 *entry = &eb->exec[i];
	1856	+ struct i915_vma *vma = exec_to_vma(entry);
	1857	+
	1858	+ eb_export_fence(vma, eb->request, entry->flags);
	1859	+ if (unlikely(entry->flags & __EXEC_OBJECT_HAS_REF))
	1860	+ i915_vma_put(vma);
1141	1861	}
	1862	+ eb->exec = NULL;
1142	1863
1143	1864	/* Unconditionally flush any chipset caches (for streaming writes). */
1144		− i915_gem_chipset_flush(req->engine->i915);
	1865	+ i915_gem_chipset_flush(eb->i915);
1145	1866
1146	1867	/* Unconditionally invalidate GPU caches and TLBs. */
1147		− return req->engine->emit_flush(req, EMIT_INVALIDATE);
	1868	+ return eb->engine->emit_flush(eb->request, EMIT_INVALIDATE);
1148	1869	}
1149	1870
1150		−static bool
1151		−i915_gem_check_execbuffer(struct drm_i915_gem_execbuffer2 *exec)
	1871	+static bool i915_gem_check_execbuffer(struct drm_i915_gem_execbuffer2 *exec)
1152	1872	{
1153		− if (exec->flags & __I915_EXEC_UNKNOWN_FLAGS)
	1873	+ if (exec->flags & __I915_EXEC_ILLEGAL_FLAGS)
1154	1874	return false;
1155	1875
1156	1876	/* Kernel clipping was a DRI1 misfeature */
@@ -1170,107 +1890,6 @@ i915_gem_check_execbuffer(struct drm_i915_gem_execbuffer2 *exec) · 13 unmodified lines
1170	1890	return true;
1171	1891	}
1172	1892
1173		−static int
1174		−validate_exec_list(struct drm_device *dev,
1175		− struct drm_i915_gem_exec_object2 *exec,
1176		− int count)
1177		−{
1178		− unsigned relocs_total = 0;
1179		− unsigned relocs_max = UINT_MAX / sizeof(struct drm_i915_gem_relocation_entry);
1180		− unsigned invalid_flags;
1181		− int i;
1182		−
1183		− /* INTERNAL flags must not overlap with external ones */
1184		− BUILD_BUG_ON(__EXEC_OBJECT_INTERNAL_FLAGS & ~__EXEC_OBJECT_UNKNOWN_FLAGS);
1185		−
1186		− invalid_flags = __EXEC_OBJECT_UNKNOWN_FLAGS;
1187		− if (USES_FULL_PPGTT(dev))
1188		− invalid_flags \|= EXEC_OBJECT_NEEDS_GTT;
1189		−
1190		− for (i = 0; i < count; i++) {
1191		− char __user *ptr = u64_to_user_ptr(exec[i].relocs_ptr);
1192		− int length; /* limited by fault_in_pages_readable() */
1193		−
1194		− if (exec[i].flags & invalid_flags)
1195		− return -EINVAL;
1196		−
1197		− /* Offset can be used as input (EXEC_OBJECT_PINNED), reject
1198		− * any non-page-aligned or non-canonical addresses.
1199		− */
1200		− if (exec[i].flags & EXEC_OBJECT_PINNED) {
1201		− if (exec[i].offset !=
1202		− gen8_canonical_addr(exec[i].offset & PAGE_MASK))
1203		− return -EINVAL;
1204		− }
1205		−
1206		− /* From drm_mm perspective address space is continuous,
1207		− * so from this point we're always using non-canonical
1208		− * form internally.
1209		− */
1210		− exec[i].offset = gen8_noncanonical_addr(exec[i].offset);
1211		−
1212		− if (exec[i].alignment && !is_power_of_2(exec[i].alignment))
1213		− return -EINVAL;
1214		−
1215		− /* pad_to_size was once a reserved field, so sanitize it */
1216		− if (exec[i].flags & EXEC_OBJECT_PAD_TO_SIZE) {
1217		− if (offset_in_page(exec[i].pad_to_size))
1218		− return -EINVAL;
1219		− } else {
1220		− exec[i].pad_to_size = 0;
1221		− }
1222		−
1223		− /* First check for malicious input causing overflow in
1224		− * the worst case where we need to allocate the entire
1225		− * relocation tree as a single array.
1226		− */
1227		− if (exec[i].relocation_count > relocs_max - relocs_total)
1228		− return -EINVAL;
1229		− relocs_total += exec[i].relocation_count;
1230		−
1231		− length = exec[i].relocation_count *
1232		− sizeof(struct drm_i915_gem_relocation_entry);
1233		− /*
1234		− * We must check that the entire relocation array is safe
1235		− * to read, but since we may need to update the presumed
1236		− * offsets during execution, check for full write access.
1237		− */
1238		− if (!access_ok(VERIFY_WRITE, ptr, length))
1239		− return -EFAULT;
1240		−
1241		− if (likely(!i915.prefault_disable)) {
1242		− if (fault_in_pages_readable(ptr, length))
1243		− return -EFAULT;
1244		− }
1245		− }
1246		−
1247		− return 0;
1248		−}
1249		−
1250		−static struct i915_gem_context *
1251		−i915_gem_validate_context(struct drm_device dev, struct drm_file file,
1252		− struct intel_engine_cs *engine, const u32 ctx_id)
1253		−{
1254		− struct i915_gem_context *ctx;
1255		−
1256		− ctx = i915_gem_context_lookup(file->driver_priv, ctx_id);
1257		− if (IS_ERR(ctx))
1258		− return ctx;
1259		−
1260		− if (i915_gem_context_is_banned(ctx)) {
1261		− DRM_DEBUG("Context %u tried to submit while banned\n", ctx_id);
1262		− return ERR_PTR(-EIO);
1263		− }
1264		−
1265		− return ctx;
1266		−}
1267		−
1268		−static bool gpu_write_needs_clflush(struct drm_i915_gem_object *obj)
1269		−{
1270		− return !(obj->cache_level == I915_CACHE_NONE \|\|
1271		− obj->cache_level == I915_CACHE_WT);
1272		−}
1273		−
1274	1893	void i915_vma_move_to_active(struct i915_vma *vma,
1275	1894	struct drm_i915_gem_request *req,
1276	1895	unsigned int flags)
@@ -1281,7 +1900,8 @@ void i915_vma_move_to_active(struct i915_vma *vma, · 4 unmodified lines
1281	1900	lockdep_assert_held(&req->i915->drm.struct_mutex);
1282	1901	GEM_BUG_ON(!drm_mm_node_allocated(&vma->node));
1283	1902
1284		− /* Add a reference if we're newly entering the active list.
	1903	+ /*
	1904	+ * Add a reference if we're newly entering the active list.
1285	1905	* The order in which we add operations to the retirement queue is
1286	1906	* vital here: mark_active adds to the start of the callback list,
1287	1907	* such that subsequent callbacks are called first. Therefore we
@@ -1294,61 +1914,22 @@ void i915_vma_move_to_active(struct i915_vma *vma, · 6 unmodified lines
1294	1914	i915_gem_active_set(&vma->last_read[idx], req);
1295	1915	list_move_tail(&vma->vm_link, &vma->vm->active_list);
1296	1916
	1917	+ obj->base.write_domain = 0;
1297	1918	if (flags & EXEC_OBJECT_WRITE) {
	1919	+ obj->base.write_domain = I915_GEM_DOMAIN_RENDER;
	1920	+
1298	1921	if (intel_fb_obj_invalidate(obj, ORIGIN_CS))
1299	1922	i915_gem_active_set(&obj->frontbuffer_write, req);
1300	1923
1301		− /* update for the implicit flush after a batch */
1302		− obj->base.write_domain &= ~I915_GEM_GPU_DOMAINS;
1303		− if (!obj->cache_dirty && gpu_write_needs_clflush(obj))
1304		− obj->cache_dirty = true;
	1924	+ obj->base.read_domains = 0;
1305	1925	}
	1926	+ obj->base.read_domains \|= I915_GEM_GPU_DOMAINS;
1306	1927
1307	1928	if (flags & EXEC_OBJECT_NEEDS_FENCE)
1308	1929	i915_gem_active_set(&vma->last_fence, req);
1309	1930	}
1310	1931
1311		−static void eb_export_fence(struct drm_i915_gem_object *obj,
1312		− struct drm_i915_gem_request *req,
1313		− unsigned int flags)
1314		−{
1315		− struct reservation_object *resv = obj->resv;
1316		−
1317		− /* Ignore errors from failing to allocate the new fence, we can't
1318		− * handle an error right now. Worst case should be missed
1319		− * synchronisation leading to rendering corruption.
1320		− */
1321		− reservation_object_lock(resv, NULL);
1322		− if (flags & EXEC_OBJECT_WRITE)
1323		− reservation_object_add_excl_fence(resv, &req->fence);
1324		− else if (reservation_object_reserve_shared(resv) == 0)
1325		− reservation_object_add_shared_fence(resv, &req->fence);
1326		− reservation_object_unlock(resv);
1327		−}
1328		−
1329		−static void
1330		−i915_gem_execbuffer_move_to_active(struct list_head *vmas,
1331		− struct drm_i915_gem_request *req)
1332		−{
1333		− struct i915_vma *vma;
1334		−
1335		− list_for_each_entry(vma, vmas, exec_list) {
1336		− struct drm_i915_gem_object *obj = vma->obj;
1337		−
1338		− obj->base.write_domain = obj->base.pending_write_domain;
1339		− if (obj->base.write_domain)
1340		− vma->exec_entry->flags \|= EXEC_OBJECT_WRITE;
1341		− else
1342		− obj->base.pending_read_domains \|= obj->base.read_domains;
1343		− obj->base.read_domains = obj->base.pending_read_domains;
1344		−
1345		− i915_vma_move_to_active(vma, req, vma->exec_entry->flags);
1346		− eb_export_fence(obj, req, vma->exec_entry->flags);
1347		− }
1348		−}
1349		−
1350		−static int
1351		−i915_reset_gen7_sol_offsets(struct drm_i915_gem_request *req)
	1932	+static int i915_reset_gen7_sol_offsets(struct drm_i915_gem_request *req)
1352	1933	{
1353	1934	u32 *cs;
1354	1935	int i;
@@ -1358,50 +1939,43 @@ i915_reset_gen7_sol_offsets(struct drm_i915_gem_request *req) · 3 unmodified lines
1358	1939	return -EINVAL;
1359	1940	}
1360	1941
1361		− cs = intel_ring_begin(req, 4 * 3);
	1942	+ cs = intel_ring_begin(req, 4 * 2 + 2);
1362	1943	if (IS_ERR(cs))
1363	1944	return PTR_ERR(cs);
1364	1945
	1946	+ *cs++ = MI_LOAD_REGISTER_IMM(4);
1365	1947	for (i = 0; i < 4; i++) {
1366		− *cs++ = MI_LOAD_REGISTER_IMM(1);
1367	1948	*cs++ = i915_mmio_reg_offset(GEN7_SO_WRITE_OFFSET(i));
1368	1949	*cs++ = 0;
1369	1950	}
1370		−
	1951	+ *cs++ = MI_NOOP;
1371	1952	intel_ring_advance(req, cs);
1372	1953
1373	1954	return 0;
1374	1955	}
1375	1956
1376		−static struct i915_vma *
1377		−i915_gem_execbuffer_parse(struct intel_engine_cs *engine,
1378		− struct drm_i915_gem_exec_object2 *shadow_exec_entry,
1379		− struct drm_i915_gem_object *batch_obj,
1380		− struct eb_vmas *eb,
1381		− u32 batch_start_offset,
1382		− u32 batch_len,
1383		− bool is_master)
	1957	+static struct i915_vma eb_parse(struct i915_execbuffer eb, bool is_master)
1384	1958	{
1385	1959	struct drm_i915_gem_object *shadow_batch_obj;
1386	1960	struct i915_vma *vma;
1387		− int ret;
	1961	+ int err;
1388	1962
1389		− shadow_batch_obj = i915_gem_batch_pool_get(&engine->batch_pool,
1390		− PAGE_ALIGN(batch_len));
	1963	+ shadow_batch_obj = i915_gem_batch_pool_get(&eb->engine->batch_pool,
	1964	+ PAGE_ALIGN(eb->batch_len));
1391	1965	if (IS_ERR(shadow_batch_obj))
1392	1966	return ERR_CAST(shadow_batch_obj);
1393	1967
1394		− ret = intel_engine_cmd_parser(engine,
1395		− batch_obj,
	1968	+ err = intel_engine_cmd_parser(eb->engine,
	1969	+ eb->batch->obj,
1396	1970	shadow_batch_obj,
1397		− batch_start_offset,
1398		− batch_len,
	1971	+ eb->batch_start_offset,
	1972	+ eb->batch_len,
1399	1973	is_master);
1400		− if (ret) {
1401		− if (ret == -EACCES) /* unhandled chained batch */
	1974	+ if (err) {
	1975	+ if (err == -EACCES) /* unhandled chained batch */
1402	1976	vma = NULL;
1403	1977	else
1404		− vma = ERR_PTR(ret);
	1978	+ vma = ERR_PTR(err);
1405	1979	goto out;
1406	1980	}
1407	1981
@@ -1409,12 +1983,11 @@ i915_gem_execbuffer_parse(struct intel_engine_cs *engine, · 1 unmodified line
1409	1983	if (IS_ERR(vma))
1410	1984	goto out;
1411	1985
1412		− memset(shadow_exec_entry, 0, sizeof(*shadow_exec_entry));
1413		−
1414		− vma->exec_entry = shadow_exec_entry;
1415		− vma->exec_entry->flags = __EXEC_OBJECT_HAS_PIN;
1416		− i915_gem_object_get(shadow_batch_obj);
1417		− list_add_tail(&vma->exec_list, &eb->vmas);
	1986	+ vma->exec_entry =
	1987	+ memset(&eb->exec[eb->buffer_count++],
	1988	+ 0, sizeof(*vma->exec_entry));
	1989	+ vma->exec_entry->flags = __EXEC_OBJECT_HAS_PIN \| __EXEC_OBJECT_HAS_REF;
	1990	+ __exec_to_vma(vma->exec_entry) = (uintptr_t)i915_vma_get(vma);
1418	1991
1419	1992	out:
1420	1993	i915_gem_object_unpin_pages(shadow_batch_obj);
@@ -1422,54 +1995,37 @@ out: · 1 unmodified line
1422	1995	}
1423	1996
1424	1997	static void
1425		−add_to_client(struct drm_i915_gem_request *req,
1426		− struct drm_file *file)
	1998	+add_to_client(struct drm_i915_gem_request req, struct drm_file file)
1427	1999	{
1428	2000	req->file_priv = file->driver_priv;
1429	2001	list_add_tail(&req->client_link, &req->file_priv->mm.request_list);
1430	2002	}
1431	2003
1432		−static int
1433		−execbuf_submit(struct i915_execbuffer_params *params,
1434		− struct drm_i915_gem_execbuffer2 *args,
1435		− struct list_head *vmas)
	2004	+static int eb_submit(struct i915_execbuffer *eb)
1436	2005	{
1437		− u64 exec_start, exec_len;
1438		− int ret;
1439		−
1440		− ret = i915_gem_execbuffer_move_to_gpu(params->request, vmas);
1441		− if (ret)
1442		− return ret;
	2006	+ int err;
1443	2007
1444		− ret = i915_switch_context(params->request);
1445		− if (ret)
1446		− return ret;
	2008	+ err = eb_move_to_gpu(eb);
	2009	+ if (err)
	2010	+ return err;
1447	2011
1448		− if (args->flags & I915_EXEC_CONSTANTS_MASK) {
1449		− DRM_DEBUG("I915_EXEC_CONSTANTS_* unsupported\n");
1450		− return -EINVAL;
1451		− }
	2012	+ err = i915_switch_context(eb->request);
	2013	+ if (err)
	2014	+ return err;
1452	2015
1453		− if (args->flags & I915_EXEC_GEN7_SOL_RESET) {
1454		− ret = i915_reset_gen7_sol_offsets(params->request);
1455		− if (ret)
1456		− return ret;
	2016	+ if (eb->args->flags & I915_EXEC_GEN7_SOL_RESET) {
	2017	+ err = i915_reset_gen7_sol_offsets(eb->request);
	2018	+ if (err)
	2019	+ return err;
1457	2020	}
1458	2021
1459		− exec_len = args->batch_len;
1460		− exec_start = params->batch->node.start +
1461		− params->args_batch_start_offset;
1462		−
1463		− if (exec_len == 0)
1464		− exec_len = params->batch->size - params->args_batch_start_offset;
1465		−
1466		− ret = params->engine->emit_bb_start(params->request,
1467		− exec_start, exec_len,
1468		− params->dispatch_flags);
1469		− if (ret)
1470		− return ret;
1471		−
1472		− i915_gem_execbuffer_move_to_active(vmas, params->request);
	2022	+ err = eb->engine->emit_bb_start(eb->request,
	2023	+ eb->batch->node.start +
	2024	+ eb->batch_start_offset,
	2025	+ eb->batch_len,
	2026	+ eb->batch_flags);
	2027	+ if (err)
	2028	+ return err;
1473	2029
1474	2030	return 0;
1475	2031	}
@@ -1551,66 +2107,62 @@ eb_select_engine(struct drm_i915_private *dev_priv, · 75 unmodified lines
1551	2107	}
1552	2108
1553	2109	static int
1554		−i915_gem_do_execbuffer(struct drm_device dev, void data,
	2110	+i915_gem_do_execbuffer(struct drm_device *dev,
1555	2111	struct drm_file *file,
1556	2112	struct drm_i915_gem_execbuffer2 *args,
1557	2113	struct drm_i915_gem_exec_object2 *exec)
1558	2114	{
1559		− struct drm_i915_private *dev_priv = to_i915(dev);
1560		− struct i915_ggtt *ggtt = &dev_priv->ggtt;
1561		− struct eb_vmas *eb;
1562		− struct drm_i915_gem_exec_object2 shadow_exec_entry;
1563		− struct intel_engine_cs *engine;
1564		− struct i915_gem_context *ctx;
1565		− struct i915_address_space *vm;
1566		− struct i915_execbuffer_params params_master; /* XXX: will be removed later */
1567		− struct i915_execbuffer_params *params = &params_master;
1568		− const u32 ctx_id = i915_execbuffer2_get_context_id(*args);
1569		− u32 dispatch_flags;
	2115	+ struct i915_execbuffer eb;
1570	2116	struct dma_fence *in_fence = NULL;
1571	2117	struct sync_file *out_fence = NULL;
1572	2118	int out_fence_fd = -1;
1573		− int ret;
1574		− bool need_relocs;
1575		−
1576		− if (!i915_gem_check_execbuffer(args))
1577		− return -EINVAL;
1578		−
1579		− ret = validate_exec_list(dev, exec, args->buffer_count);
1580		− if (ret)
1581		− return ret;
1582		−
1583		− dispatch_flags = 0;
	2119	+ int err;
	2120	+
	2121	+ BUILD_BUG_ON(__EXEC_OBJECT_INTERNAL_FLAGS &
	2122	+ ~__EXEC_OBJECT_UNKNOWN_FLAGS);
	2123	+
	2124	+ eb.i915 = to_i915(dev);
	2125	+ eb.file = file;
	2126	+ eb.args = args;
	2127	+ if (DBG_FORCE_RELOC \|\| !(args->flags & I915_EXEC_NO_RELOC))
	2128	+ args->flags \|= __EXEC_HAS_RELOC;
	2129	+ eb.exec = exec;
	2130	+ eb.ctx = NULL;
	2131	+ eb.invalid_flags = __EXEC_OBJECT_UNKNOWN_FLAGS;
	2132	+ if (USES_FULL_PPGTT(eb.i915))
	2133	+ eb.invalid_flags \|= EXEC_OBJECT_NEEDS_GTT;
	2134	+ reloc_cache_init(&eb.reloc_cache, eb.i915);
	2135	+
	2136	+ eb.buffer_count = args->buffer_count;
	2137	+ eb.batch_start_offset = args->batch_start_offset;
	2138	+ eb.batch_len = args->batch_len;
	2139	+
	2140	+ eb.batch_flags = 0;
1584	2141	if (args->flags & I915_EXEC_SECURE) {
1585	2142	if (!drm_is_current_master(file) \|\| !capable(CAP_SYS_ADMIN))
1586	2143	return -EPERM;
1587	2144
1588		− dispatch_flags \|= I915_DISPATCH_SECURE;
	2145	+ eb.batch_flags \|= I915_DISPATCH_SECURE;
1589	2146	}
1590	2147	if (args->flags & I915_EXEC_IS_PINNED)
1591		− dispatch_flags \|= I915_DISPATCH_PINNED;
1592		−
1593		− engine = eb_select_engine(dev_priv, file, args);
1594		− if (!engine)
1595		− return -EINVAL;
	2148	+ eb.batch_flags \|= I915_DISPATCH_PINNED;
1596	2149
1597		− if (args->buffer_count < 1) {
1598		− DRM_DEBUG("execbuf with %d buffers\n", args->buffer_count);
	2150	+ eb.engine = eb_select_engine(eb.i915, file, args);
	2151	+ if (!eb.engine)
1599	2152	return -EINVAL;
1600		− }
1601	2153
1602	2154	if (args->flags & I915_EXEC_RESOURCE_STREAMER) {
1603		− if (!HAS_RESOURCE_STREAMER(dev_priv)) {
	2155	+ if (!HAS_RESOURCE_STREAMER(eb.i915)) {
1604	2156	DRM_DEBUG("RS is only allowed for Haswell, Gen8 and above\n");
1605	2157	return -EINVAL;
1606	2158	}
1607		− if (engine->id != RCS) {
	2159	+ if (eb.engine->id != RCS) {
1608	2160	DRM_DEBUG("RS is not available on %s\n",
1609		− engine->name);
	2161	+ eb.engine->name);
1610	2162	return -EINVAL;
1611	2163	}
1612	2164
1613		− dispatch_flags \|= I915_DISPATCH_RS;
	2165	+ eb.batch_flags \|= I915_DISPATCH_RS;
1614	2166	}
1615	2167
1616	2168	if (args->flags & I915_EXEC_FENCE_IN) {
@@ -1622,102 +2174,62 @@ i915_gem_do_execbuffer(struct drm_device dev, void data, · 5 unmodified lines
1622	2174	if (args->flags & I915_EXEC_FENCE_OUT) {
1623	2175	out_fence_fd = get_unused_fd_flags(O_CLOEXEC);
1624	2176	if (out_fence_fd < 0) {
1625		− ret = out_fence_fd;
	2177	+ err = out_fence_fd;
1626	2178	goto err_in_fence;
1627	2179	}
1628	2180	}
1629	2181
1630		− /* Take a local wakeref for preparing to dispatch the execbuf as
	2182	+ if (eb_create(&eb))
	2183	+ return -ENOMEM;
	2184	+
	2185	+ /*
	2186	+ * Take a local wakeref for preparing to dispatch the execbuf as
1631	2187	* we expect to access the hardware fairly frequently in the
1632	2188	* process. Upon first dispatch, we acquire another prolonged
1633	2189	* wakeref that we hold until the GPU has been idle for at least
1634	2190	* 100ms.
1635	2191	*/
1636		− intel_runtime_pm_get(dev_priv);
	2192	+ intel_runtime_pm_get(eb.i915);
	2193	+ err = i915_mutex_lock_interruptible(dev);
	2194	+ if (err)
	2195	+ goto err_rpm;
1637	2196
1638		− ret = i915_mutex_lock_interruptible(dev);
1639		− if (ret)
1640		− goto pre_mutex_err;
	2197	+ err = eb_select_context(&eb);
	2198	+ if (unlikely(err))
	2199	+ goto err_unlock;
1641	2200
1642		− ctx = i915_gem_validate_context(dev, file, engine, ctx_id);
1643		− if (IS_ERR(ctx)) {
1644		− mutex_unlock(&dev->struct_mutex);
1645		− ret = PTR_ERR(ctx);
1646		− goto pre_mutex_err;
1647		− }
1648		−
1649		− i915_gem_context_get(ctx);
1650		−
1651		− if (ctx->ppgtt)
1652		− vm = &ctx->ppgtt->base;
1653		− else
1654		− vm = &ggtt->base;
1655		−
1656		− memset(&params_master, 0x00, sizeof(params_master));
1657		−
1658		− eb = eb_create(dev_priv, args);
1659		− if (eb == NULL) {
1660		− i915_gem_context_put(ctx);
1661		− mutex_unlock(&dev->struct_mutex);
1662		− ret = -ENOMEM;
1663		− goto pre_mutex_err;
1664		− }
1665		−
1666		− /* Look up object handles */
1667		− ret = eb_lookup_vmas(eb, exec, args, vm, file);
1668		− if (ret)
1669		− goto err;
1670		−
1671		− /* take note of the batch buffer before we might reorder the lists */
1672		− params->batch = eb_get_batch(eb);
1673		−
1674		− /* Move the objects en-masse into the GTT, evicting if necessary. */
1675		− need_relocs = (args->flags & I915_EXEC_NO_RELOC) == 0;
1676		− ret = i915_gem_execbuffer_reserve(engine, &eb->vmas, ctx,
1677		− &need_relocs);
1678		− if (ret)
1679		− goto err;
1680		−
1681		− /* The objects are in their final locations, apply the relocations. */
1682		− if (need_relocs)
1683		− ret = i915_gem_execbuffer_relocate(eb);
1684		− if (ret) {
1685		− if (ret == -EFAULT) {
1686		− ret = i915_gem_execbuffer_relocate_slow(dev, args, file,
1687		− engine,
1688		− eb, exec, ctx);
1689		− BUG_ON(!mutex_is_locked(&dev->struct_mutex));
1690		− }
1691		− if (ret)
1692		− goto err;
1693		− }
	2201	+ err = eb_relocate(&eb);
	2202	+ if (err)
	2203	+ /*
	2204	+ * If the user expects the execobject.offset and
	2205	+ * reloc.presumed_offset to be an exact match,
	2206	+ * as for using NO_RELOC, then we cannot update
	2207	+ * the execobject.offset until we have completed
	2208	+ * relocation.
	2209	+ */
	2210	+ args->flags &= ~__EXEC_HAS_RELOC;
	2211	+ if (err < 0)
	2212	+ goto err_vma;
1694	2213
1695		− /* Set the pending read domains for the batch buffer to COMMAND */
1696		− if (params->batch->obj->base.pending_write_domain) {
	2214	+ if (unlikely(eb.batch->exec_entry->flags & EXEC_OBJECT_WRITE)) {
1697	2215	DRM_DEBUG("Attempting to use self-modifying batch buffer\n");
1698		− ret = -EINVAL;
1699		− goto err;
	2216	+ err = -EINVAL;
	2217	+ goto err_vma;
1700	2218	}
1701		− if (args->batch_start_offset > params->batch->size \|\|
1702		− args->batch_len > params->batch->size - args->batch_start_offset) {
	2219	+ if (eb.batch_start_offset > eb.batch->size \|\|
	2220	+ eb.batch_len > eb.batch->size - eb.batch_start_offset) {
1703	2221	DRM_DEBUG("Attempting to use out-of-bounds batch\n");
1704		− ret = -EINVAL;
1705		− goto err;
	2222	+ err = -EINVAL;
	2223	+ goto err_vma;
1706	2224	}
1707	2225
1708		− params->args_batch_start_offset = args->batch_start_offset;
1709		− if (engine->needs_cmd_parser && args->batch_len) {
	2226	+ if (eb.engine->needs_cmd_parser && eb.batch_len) {
1710	2227	struct i915_vma *vma;
1711	2228
1712		− vma = i915_gem_execbuffer_parse(engine, &shadow_exec_entry,
1713		− params->batch->obj,
1714		− eb,
1715		− args->batch_start_offset,
1716		− args->batch_len,
1717		− drm_is_current_master(file));
	2229	+ vma = eb_parse(&eb, drm_is_current_master(file));
1718	2230	if (IS_ERR(vma)) {
1719		− ret = PTR_ERR(vma);
1720		− goto err;
	2231	+ err = PTR_ERR(vma);
	2232	+ goto err_vma;
1721	2233	}
1722	2234
1723	2235	if (vma) {
@@ -1730,19 +2242,20 @@ i915_gem_do_execbuffer(struct drm_device dev, void data, · 6 unmodified lines
1730	2242	* specifically don't want that set on batches the
1731	2243	* command parser has accepted.
1732	2244	*/
1733		− dispatch_flags \|= I915_DISPATCH_SECURE;
1734		− params->args_batch_start_offset = 0;
1735		− params->batch = vma;
	2245	+ eb.batch_flags \|= I915_DISPATCH_SECURE;
	2246	+ eb.batch_start_offset = 0;
	2247	+ eb.batch = vma;
1736	2248	}
1737	2249	}
1738	2250
1739		− params->batch->obj->base.pending_read_domains \|= I915_GEM_DOMAIN_COMMAND;
	2251	+ if (eb.batch_len == 0)
	2252	+ eb.batch_len = eb.batch->size - eb.batch_start_offset;
1740	2253
1741		− /* snb/ivb/vlv conflate the "batch in ppgtt" bit with the "non-secure
	2254	+ /*
	2255	+ * snb/ivb/vlv conflate the "batch in ppgtt" bit with the "non-secure
1742	2256	* batch" bit. Hence we need to pin secure batches into the global gtt.
1743	2257	* hsw should have this fixed, but bdw mucks it up again. */
1744		− if (dispatch_flags & I915_DISPATCH_SECURE) {
1745		− struct drm_i915_gem_object *obj = params->batch->obj;
	2258	+ if (eb.batch_flags & I915_DISPATCH_SECURE) {
1746	2259	struct i915_vma *vma;
1747	2260
1748	2261	/*
@@ -1755,66 +2268,56 @@ i915_gem_do_execbuffer(struct drm_device dev, void data, · 6 unmodified lines
1755	2268	* fitting due to fragmentation.
1756	2269	* So this is actually safe.
1757	2270	*/
1758		− vma = i915_gem_object_ggtt_pin(obj, NULL, 0, 0, 0);
	2271	+ vma = i915_gem_object_ggtt_pin(eb.batch->obj, NULL, 0, 0, 0);
1759	2272	if (IS_ERR(vma)) {
1760		− ret = PTR_ERR(vma);
1761		− goto err;
	2273	+ err = PTR_ERR(vma);
	2274	+ goto err_vma;
1762	2275	}
1763	2276
1764		− params->batch = vma;
	2277	+ eb.batch = vma;
1765	2278	}
1766	2279
	2280	+ /* All GPU relocation batches must be submitted prior to the user rq */
	2281	+ GEM_BUG_ON(eb.reloc_cache.rq);
	2282	+
1767	2283	/* Allocate a request for this batch buffer nice and early. */
1768		− params->request = i915_gem_request_alloc(engine, ctx);
1769		− if (IS_ERR(params->request)) {
1770		− ret = PTR_ERR(params->request);
	2284	+ eb.request = i915_gem_request_alloc(eb.engine, eb.ctx);
	2285	+ if (IS_ERR(eb.request)) {
	2286	+ err = PTR_ERR(eb.request);
1771	2287	goto err_batch_unpin;
1772	2288	}
1773	2289
1774	2290	if (in_fence) {
1775		− ret = i915_gem_request_await_dma_fence(params->request,
1776		− in_fence);
1777		− if (ret < 0)
	2291	+ err = i915_gem_request_await_dma_fence(eb.request, in_fence);
	2292	+ if (err < 0)
1778	2293	goto err_request;
1779	2294	}
1780	2295
1781	2296	if (out_fence_fd != -1) {
1782		− out_fence = sync_file_create(&params->request->fence);
	2297	+ out_fence = sync_file_create(&eb.request->fence);
1783	2298	if (!out_fence) {
1784		− ret = -ENOMEM;
	2299	+ err = -ENOMEM;
1785	2300	goto err_request;
1786	2301	}
1787	2302	}
1788	2303
1789		− /* Whilst this request exists, batch_obj will be on the
	2304	+ /*
	2305	+ * Whilst this request exists, batch_obj will be on the
1790	2306	* active_list, and so will hold the active reference. Only when this
1791	2307	* request is retired will the the batch_obj be moved onto the
1792	2308	* inactive_list and lose its active reference. Hence we do not need
1793	2309	* to explicitly hold another reference here.
1794	2310	*/
1795		− params->request->batch = params->batch;
1796		−
1797		− /*
1798		− * Save assorted stuff away to pass through to *_submission().
1799		− * NB: This data should be 'persistent' and not local as it will
1800		− * kept around beyond the duration of the IOCTL once the GPU
1801		− * scheduler arrives.
1802		− */
1803		− params->dev = dev;
1804		− params->file = file;
1805		− params->engine = engine;
1806		− params->dispatch_flags = dispatch_flags;
1807		− params->ctx = ctx;
	2311	+ eb.request->batch = eb.batch;
1808	2312
1809		− trace_i915_gem_request_queue(params->request, dispatch_flags);
1810		−
1811		− ret = execbuf_submit(params, args, &eb->vmas);
	2313	+ trace_i915_gem_request_queue(eb.request, eb.batch_flags);
	2314	+ err = eb_submit(&eb);
1812	2315	err_request:
1813		− __i915_add_request(params->request, ret == 0);
1814		− add_to_client(params->request, file);
	2316	+ __i915_add_request(eb.request, err == 0);
	2317	+ add_to_client(eb.request, file);
1815	2318
1816	2319	if (out_fence) {
1817		− if (ret == 0) {
	2320	+ if (err == 0) {
1818	2321	fd_install(out_fence_fd, out_fence->file);
1819	2322	args->rsvd2 &= GENMASK_ULL(0, 31); /* keep in-fence */
1820	2323	args->rsvd2 \|= (u64)out_fence_fd << 32;
@@ -1825,30 +2328,22 @@ err_request: · 4 unmodified lines
1825	2328	}
1826	2329
1827	2330	err_batch_unpin:
1828		− /*
1829		− * FIXME: We crucially rely upon the active tracking for the (ppgtt)
1830		− * batch vma for correctness. For less ugly and less fragility this
1831		− * needs to be adjusted to also track the ggtt batch vma properly as
1832		− * active.
1833		− */
1834		− if (dispatch_flags & I915_DISPATCH_SECURE)
1835		− i915_vma_unpin(params->batch);
1836		−err:
1837		− /* the request owns the ref now */
1838		− i915_gem_context_put(ctx);
1839		− eb_destroy(eb);
1840		−
	2331	+ if (eb.batch_flags & I915_DISPATCH_SECURE)
	2332	+ i915_vma_unpin(eb.batch);
	2333	+err_vma:
	2334	+ if (eb.exec)
	2335	+ eb_release_vmas(&eb);
	2336	+ i915_gem_context_put(eb.ctx);
	2337	+err_unlock:
1841	2338	mutex_unlock(&dev->struct_mutex);
1842		−
1843		−pre_mutex_err:
1844		− /* intel_gpu_busy should also get a ref, so it will free when the device
1845		− * is really idle. */
1846		− intel_runtime_pm_put(dev_priv);
	2339	+err_rpm:
	2340	+ intel_runtime_pm_put(eb.i915);
	2341	+ eb_destroy(&eb);
1847	2342	if (out_fence_fd != -1)
1848	2343	put_unused_fd(out_fence_fd);
1849	2344	err_in_fence:
1850	2345	dma_fence_put(in_fence);
1851		− return ret;
	2346	+ return err;
1852	2347	}
1853	2348
1854	2349	/*
@@ -1859,20 +2354,38 @@ int · 4 unmodified lines
1859	2354	i915_gem_execbuffer(struct drm_device dev, void data,
1860	2355	struct drm_file *file)
1861	2356	{
	2357	+ const size_t sz = sizeof(struct drm_i915_gem_exec_object2);
1862	2358	struct drm_i915_gem_execbuffer *args = data;
1863	2359	struct drm_i915_gem_execbuffer2 exec2;
1864	2360	struct drm_i915_gem_exec_object *exec_list = NULL;
1865	2361	struct drm_i915_gem_exec_object2 *exec2_list = NULL;
1866		− int ret, i;
	2362	+ unsigned int i;
	2363	+ int err;
1867	2364
1868		− if (args->buffer_count < 1) {
1869		− DRM_DEBUG("execbuf with %d buffers\n", args->buffer_count);
	2365	+ if (args->buffer_count < 1 \|\| args->buffer_count > SIZE_MAX / sz - 1) {
	2366	+ DRM_DEBUG("execbuf2 with %d buffers\n", args->buffer_count);
1870	2367	return -EINVAL;
1871	2368	}
1872	2369
	2370	+ exec2.buffers_ptr = args->buffers_ptr;
	2371	+ exec2.buffer_count = args->buffer_count;
	2372	+ exec2.batch_start_offset = args->batch_start_offset;
	2373	+ exec2.batch_len = args->batch_len;
	2374	+ exec2.DR1 = args->DR1;
	2375	+ exec2.DR4 = args->DR4;
	2376	+ exec2.num_cliprects = args->num_cliprects;
	2377	+ exec2.cliprects_ptr = args->cliprects_ptr;
	2378	+ exec2.flags = I915_EXEC_RENDER;
	2379	+ i915_execbuffer2_set_context_id(exec2, 0);
	2380	+
	2381	+ if (!i915_gem_check_execbuffer(&exec2))
	2382	+ return -EINVAL;
	2383	+
1873	2384	/* Copy in the exec list from userland */
1874		− exec_list = kvmalloc_array(sizeof(*exec_list), args->buffer_count, GFP_KERNEL);
1875		− exec2_list = kvmalloc_array(sizeof(*exec2_list), args->buffer_count, GFP_KERNEL);
	2385	+ exec_list = kvmalloc_array(args->buffer_count, sizeof(*exec_list),
	2386	+ __GFP_NOWARN \| GFP_TEMPORARY);
	2387	+ exec2_list = kvmalloc_array(args->buffer_count + 1, sz,
	2388	+ __GFP_NOWARN \| GFP_TEMPORARY);
1876	2389	if (exec_list == NULL \|\| exec2_list == NULL) {
1877	2390	DRM_DEBUG("Failed to allocate exec list for %d buffers\n",
1878	2391	args->buffer_count);
@@ -1880,12 +2393,12 @@ i915_gem_execbuffer(struct drm_device dev, void data, · 1 unmodified line
1880	2393	kvfree(exec2_list);
1881	2394	return -ENOMEM;
1882	2395	}
1883		− ret = copy_from_user(exec_list,
	2396	+ err = copy_from_user(exec_list,
1884	2397	u64_to_user_ptr(args->buffers_ptr),
1885	2398	sizeof(exec_list) args->buffer_count);
1886		− if (ret != 0) {
	2399	+ if (err) {
1887	2400	DRM_DEBUG("copy %d exec entries failed %d\n",
1888		− args->buffer_count, ret);
	2401	+ args->buffer_count, err);
1889	2402	kvfree(exec_list);
1890	2403	kvfree(exec2_list);
1891	2404	return -EFAULT;
@@ -1903,99 +2416,94 @@ i915_gem_execbuffer(struct drm_device dev, void data, · 11 unmodified lines
1903	2416	exec2_list[i].flags = 0;
1904	2417	}
1905	2418
1906		− exec2.buffers_ptr = args->buffers_ptr;
1907		− exec2.buffer_count = args->buffer_count;
1908		− exec2.batch_start_offset = args->batch_start_offset;
1909		− exec2.batch_len = args->batch_len;
1910		− exec2.DR1 = args->DR1;
1911		− exec2.DR4 = args->DR4;
1912		− exec2.num_cliprects = args->num_cliprects;
1913		− exec2.cliprects_ptr = args->cliprects_ptr;
1914		− exec2.flags = I915_EXEC_RENDER;
1915		− i915_execbuffer2_set_context_id(exec2, 0);
1916		−
1917		− ret = i915_gem_do_execbuffer(dev, data, file, &exec2, exec2_list);
1918		− if (!ret) {
	2419	+ err = i915_gem_do_execbuffer(dev, file, &exec2, exec2_list);
	2420	+ if (exec2.flags & __EXEC_HAS_RELOC) {
1919	2421	struct drm_i915_gem_exec_object __user *user_exec_list =
1920	2422	u64_to_user_ptr(args->buffers_ptr);
1921	2423
1922	2424	/* Copy the new buffer offsets back to the user's exec list. */
1923	2425	for (i = 0; i < args->buffer_count; i++) {
	2426	+ if (!(exec2_list[i].offset & UPDATE))
	2427	+ continue;
	2428	+
1924	2429	exec2_list[i].offset =
1925		− gen8_canonical_addr(exec2_list[i].offset);
1926		− ret = __copy_to_user(&user_exec_list[i].offset,
1927		− &exec2_list[i].offset,
1928		− sizeof(user_exec_list[i].offset));
1929		− if (ret) {
1930		− ret = -EFAULT;
1931		− DRM_DEBUG("failed to copy %d exec entries "
1932		− "back to user (%d)\n",
1933		− args->buffer_count, ret);
	2430	+ gen8_canonical_addr(exec2_list[i].offset & PIN_OFFSET_MASK);
	2431	+ exec2_list[i].offset &= PIN_OFFSET_MASK;
	2432	+ if (__copy_to_user(&user_exec_list[i].offset,
	2433	+ &exec2_list[i].offset,
	2434	+ sizeof(user_exec_list[i].offset)))
1934	2435	break;
1935		− }
1936	2436	}
1937	2437	}
1938	2438
1939	2439	kvfree(exec_list);
1940	2440	kvfree(exec2_list);
1941		− return ret;
	2441	+ return err;
1942	2442	}
1943	2443
1944	2444	int
1945	2445	i915_gem_execbuffer2(struct drm_device dev, void data,
1946	2446	struct drm_file *file)
1947	2447	{
	2448	+ const size_t sz = sizeof(struct drm_i915_gem_exec_object2);
1948	2449	struct drm_i915_gem_execbuffer2 *args = data;
1949		− struct drm_i915_gem_exec_object2 *exec2_list = NULL;
1950		− int ret;
	2450	+ struct drm_i915_gem_exec_object2 *exec2_list;
	2451	+ int err;
1951	2452
1952		− if (args->buffer_count < 1 \|\|
1953		− args->buffer_count > UINT_MAX / sizeof(*exec2_list)) {
	2453	+ if (args->buffer_count < 1 \|\| args->buffer_count > SIZE_MAX / sz - 1) {
1954	2454	DRM_DEBUG("execbuf2 with %d buffers\n", args->buffer_count);
1955	2455	return -EINVAL;
1956	2456	}
1957	2457
1958		− exec2_list = kvmalloc_array(args->buffer_count,
1959		− sizeof(*exec2_list),
1960		− GFP_TEMPORARY);
	2458	+ if (!i915_gem_check_execbuffer(args))
	2459	+ return -EINVAL;
	2460	+
	2461	+ /* Allocate an extra slot for use by the command parser */
	2462	+ exec2_list = kvmalloc_array(args->buffer_count + 1, sz,
	2463	+ __GFP_NOWARN \| GFP_TEMPORARY);
1961	2464	if (exec2_list == NULL) {
1962	2465	DRM_DEBUG("Failed to allocate exec list for %d buffers\n",
1963	2466	args->buffer_count);
1964	2467	return -ENOMEM;
1965	2468	}
1966		− ret = copy_from_user(exec2_list,
1967		− u64_to_user_ptr(args->buffers_ptr),
1968		− sizeof(exec2_list) args->buffer_count);
1969		− if (ret != 0) {
1970		− DRM_DEBUG("copy %d exec entries failed %d\n",
1971		− args->buffer_count, ret);
	2469	+ if (copy_from_user(exec2_list,
	2470	+ u64_to_user_ptr(args->buffers_ptr),
	2471	+ sizeof(exec2_list) args->buffer_count)) {
	2472	+ DRM_DEBUG("copy %d exec entries failed\n", args->buffer_count);
1972	2473	kvfree(exec2_list);
1973	2474	return -EFAULT;
1974	2475	}
1975	2476
1976		− ret = i915_gem_do_execbuffer(dev, data, file, args, exec2_list);
1977		− if (!ret) {
1978		− /* Copy the new buffer offsets back to the user's exec list. */
	2477	+ err = i915_gem_do_execbuffer(dev, file, args, exec2_list);
	2478	+
	2479	+ /*
	2480	+ * Now that we have begun execution of the batchbuffer, we ignore
	2481	+ * any new error after this point. Also given that we have already
	2482	+ * updated the associated relocations, we try to write out the current
	2483	+ * object locations irrespective of any error.
	2484	+ */
	2485	+ if (args->flags & __EXEC_HAS_RELOC) {
1979	2486	struct drm_i915_gem_exec_object2 __user *user_exec_list =
1980		− u64_to_user_ptr(args->buffers_ptr);
1981		− int i;
	2487	+ u64_to_user_ptr(args->buffers_ptr);
	2488	+ unsigned int i;
1982	2489
	2490	+ /* Copy the new buffer offsets back to the user's exec list. */
	2491	+ user_access_begin();
1983	2492	for (i = 0; i < args->buffer_count; i++) {
	2493	+ if (!(exec2_list[i].offset & UPDATE))
	2494	+ continue;
	2495	+
1984	2496	exec2_list[i].offset =
1985		− gen8_canonical_addr(exec2_list[i].offset);
1986		− ret = __copy_to_user(&user_exec_list[i].offset,
1987		− &exec2_list[i].offset,
1988		− sizeof(user_exec_list[i].offset));
1989		− if (ret) {
1990		− ret = -EFAULT;
1991		− DRM_DEBUG("failed to copy %d exec entries "
1992		− "back to user\n",
1993		− args->buffer_count);
1994		− break;
1995		− }
	2497	+ gen8_canonical_addr(exec2_list[i].offset & PIN_OFFSET_MASK);
	2498	+ unsafe_put_user(exec2_list[i].offset,
	2499	+ &user_exec_list[i].offset,
	2500	+ end_user);
1996	2501	}
	2502	+end_user:
	2503	+ user_access_end();
1997	2504	}
1998	2505
	2506	+ args->flags &= ~__I915_EXEC_UNKNOWN_FLAGS;
1999	2507	kvfree(exec2_list);
2000		− return ret;
	2508	+ return err;
2001	2509	}

drivers/gpu/drm/i915/i915_gem_gtt.c+13−3

@@ -1884,7 +1884,7 @@ static void gtt_write_workarounds(struct drm_i915_private *dev_priv) · 1883 unmodified lines
1884	1884	* called on driver load and after a GPU reset, so you can place
1885	1885	* workarounds here even if they get overwritten by GPU reset.
1886	1886	*/
1887		− /* WaIncreaseDefaultTLBEntries:chv,bdw,skl,bxt,kbl,glk */
	1887	+ /* WaIncreaseDefaultTLBEntries:chv,bdw,skl,bxt,kbl,glk,cfl */
1888	1888	if (IS_BROADWELL(dev_priv))
1889	1889	I915_WRITE(GEN8_L3_LRA_1_GPGPU, GEN8_L3_LRA_1_GPGPU_DEFAULT_VALUE_BDW);
1890	1890	else if (IS_CHERRYVIEW(dev_priv))
@@ -3095,13 +3095,17 @@ int i915_ggtt_enable_hw(struct drm_i915_private *dev_priv) · 1204 unmodified lines
3095	3095
3096	3096	void i915_ggtt_enable_guc(struct drm_i915_private *i915)
3097	3097	{
	3098	+ GEM_BUG_ON(i915->ggtt.invalidate != gen6_ggtt_invalidate);
	3099	+
3098	3100	i915->ggtt.invalidate = guc_ggtt_invalidate;
3099	3101	}
3100	3102
3101	3103	void i915_ggtt_disable_guc(struct drm_i915_private *i915)
3102	3104	{
3103		− if (i915->ggtt.invalidate == guc_ggtt_invalidate)
3104		− i915->ggtt.invalidate = gen6_ggtt_invalidate;
	3105	+ /* We should only be called after i915_ggtt_enable_guc() */
	3106	+ GEM_BUG_ON(i915->ggtt.invalidate != guc_ggtt_invalidate);
	3107	+
	3108	+ i915->ggtt.invalidate = gen6_ggtt_invalidate;
3105	3109	}
3106	3110
3107	3111	void i915_gem_restore_gtt_mappings(struct drm_i915_private *dev_priv)
@@ -3398,6 +3402,9 @@ int i915_gem_gtt_reserve(struct i915_address_space *vm, · 290 unmodified lines
3398	3402	if (err != -ENOSPC)
3399	3403	return err;
3400	3404
	3405	+ if (flags & PIN_NOEVICT)
	3406	+ return -ENOSPC;
	3407	+
3401	3408	err = i915_gem_evict_for_node(vm, node, flags);
3402	3409	if (err == 0)
3403	3410	err = drm_mm_reserve_node(&vm->mm, node);
@@ -3512,6 +3519,9 @@ int i915_gem_gtt_insert(struct i915_address_space *vm, · 108 unmodified lines
3512	3519	if (err != -ENOSPC)
3513	3520	return err;
3514	3521
	3522	+ if (flags & PIN_NOEVICT)
	3523	+ return -ENOSPC;
	3524	+
3515	3525	/* No free space, pick a slot at random.
3516	3526	*
3517	3527	* There is a pathological case here using a GTT shared between

drivers/gpu/drm/i915/i915_gem_gtt.h+2−0

@@ -255,6 +255,7 @@ struct i915_address_space { · 254 unmodified lines
255	255	struct drm_i915_file_private *file;
256	256	struct list_head global_link;
257	257	u64 total; /* size addr space maps (ex. 2GB for ggtt) */
	258	+ u64 reserved; /* size addr space reserved */
258	259
259	260	bool closed;
260	261
@@ -588,6 +589,7 @@ int i915_gem_gtt_insert(struct i915_address_space *vm, · 327 unmodified lines
588	589	#define PIN_MAPPABLE BIT(1)
589	590	#define PIN_ZONE_4G BIT(2)
590	591	#define PIN_NONFAULT BIT(3)
	592	+#define PIN_NOEVICT BIT(4)
591	593
592	594	#define PIN_MBZ BIT(5) /* I915_VMA_PIN_OVERFLOW */
593	595	#define PIN_GLOBAL BIT(6) /* I915_VMA_GLOBAL_BIND */

drivers/gpu/drm/i915/i915_gem_internal.c+3−1

@@ -188,9 +188,11 @@ i915_gem_object_create_internal(struct drm_i915_private *i915, · 187 unmodified lines
188	188	drm_gem_private_object_init(&i915->drm, &obj->base, size);
189	189	i915_gem_object_init(obj, &i915_gem_object_internal_ops);
190	190
191		− obj->base.write_domain = I915_GEM_DOMAIN_CPU;
192	191	obj->base.read_domains = I915_GEM_DOMAIN_CPU;
	192	+ obj->base.write_domain = I915_GEM_DOMAIN_CPU;
193	193	obj->cache_level = HAS_LLC(i915) ? I915_CACHE_LLC : I915_CACHE_NONE;
	194	+ obj->cache_coherent = i915_gem_object_is_coherent(obj);
	195	+ obj->cache_dirty = !obj->cache_coherent;
194	196
195	197	return obj;
196	198	}

drivers/gpu/drm/i915/i915_gem_object.h+18−4

@@ -68,9 +68,25 @@ struct drm_i915_gem_object { · 67 unmodified lines
68	68
69	69	const struct drm_i915_gem_object_ops *ops;
70	70
71		− /** List of VMAs backed by this object */
	71	+ /**
	72	+ * @vma_list: List of VMAs backed by this object
	73	+ *
	74	+ * The VMA on this list are ordered by type, all GGTT vma are placed
	75	+ * at the head and all ppGTT vma are placed at the tail. The different
	76	+ * types of GGTT vma are unordered between themselves, use the
	77	+ * @vma_tree (which has a defined order between all VMA) to find an
	78	+ * exact match.
	79	+ */
72	80	struct list_head vma_list;
	81	+ /**
	82	+ * @vma_tree: Ordered tree of VMAs backed by this object
	83	+ *
	84	+ * All VMA created for this object are placed in the @vma_tree for
	85	+ * fast retrieval via a binary search in i915_vma_instance().
	86	+ * They are also added to @vma_list for easy iteration.
	87	+ */
73	88	struct rb_root vma_tree;
	89	+ struct i915_vma *vma_hashed;
74	90
75	91	/** Stolen memory for this object, instead of being backed by shmem. */
76	92	struct drm_mm_node *stolen;
@@ -85,9 +101,6 @@ struct drm_i915_gem_object { · 8 unmodified lines
85	101	*/
86	102	struct list_head userfault_link;
87	103
88		− /** Used in execbuf to temporarily hold a ref */
89		− struct list_head obj_exec_link;
90		−
91	104	struct list_head batch_pool_link;
92	105	I915_SELFTEST_DECLARE(struct list_head st_link);
93	106
@@ -106,6 +119,7 @@ struct drm_i915_gem_object { · 12 unmodified lines
106	119	unsigned long gt_ro:1;
107	120	unsigned int cache_level:3;
108	121	unsigned int cache_dirty:1;
	122	+ unsigned int cache_coherent:1;
109	123
110	124	atomic_t frontbuffer_bits;
111	125	unsigned int frontbuffer_ggtt_origin; /* write once */

drivers/gpu/drm/i915/i915_gem_request.c+7−13

@@ -62,7 +62,7 @@ static bool i915_fence_enable_signaling(struct dma_fence *fence) · 61 unmodified lines
62	62	return false;
63	63
64	64	intel_engine_enable_signaling(to_request(fence), true);
65		− return true;
	65	+ return !i915_fence_signaled(fence);
66	66	}
67	67
68	68	static signed long i915_fence_wait(struct dma_fence *fence,
@@ -683,7 +683,6 @@ static int · 614 unmodified lines
683	683	i915_gem_request_await_request(struct drm_i915_gem_request *to,
684	684	struct drm_i915_gem_request *from)
685	685	{
686		− u32 seqno;
687	686	int ret;
688	687
689	688	GEM_BUG_ON(to == from);
@@ -707,18 +706,14 @@ i915_gem_request_await_request(struct drm_i915_gem_request *to, · 17 unmodified lines
707	706	return ret < 0 ? ret : 0;
708	707	}
709	708
710		− seqno = i915_gem_request_global_seqno(from);
711		− if (!seqno)
712		− goto await_dma_fence;
	709	+ if (to->engine->semaphore.sync_to) {
	710	+ u32 seqno;
713	711
714		− if (!to->engine->semaphore.sync_to) {
715		− if (!__i915_gem_request_started(from, seqno))
716		− goto await_dma_fence;
	712	+ GEM_BUG_ON(!from->engine->semaphore.signal);
717	713
718		− if (!__i915_spin_request(from, seqno, TASK_INTERRUPTIBLE, 2))
	714	+ seqno = i915_gem_request_global_seqno(from);
	715	+ if (!seqno)
719	716	goto await_dma_fence;
720		− } else {
721		− GEM_BUG_ON(!from->engine->semaphore.signal);
722	717
723	718	if (seqno <= to->timeline->global_sync[from->engine->id])
724	719	return 0;
@@ -729,10 +724,9 @@ i915_gem_request_await_request(struct drm_i915_gem_request *to, · 4 unmodified lines
729	724	return ret;
730	725
731	726	to->timeline->global_sync[from->engine->id] = seqno;
	727	+ return 0;
732	728	}
733	729
734		− return 0;
735		−
736	730	await_dma_fence:
737	731	ret = i915_sw_fence_await_dma_fence(&to->submit,
738	732	&from->fence, 0,

drivers/gpu/drm/i915/i915_gem_shrinker.c+21−7

@@ -38,16 +38,21 @@ · 37 unmodified lines
38	38	static bool shrinker_lock(struct drm_i915_private dev_priv, bool unlock)
39	39	{
40	40	switch (mutex_trylock_recursive(&dev_priv->drm.struct_mutex)) {
41		− case MUTEX_TRYLOCK_FAILED:
42		− return false;
43		−
44		− case MUTEX_TRYLOCK_SUCCESS:
45		− *unlock = true;
46		− return true;
47		−
48	41	case MUTEX_TRYLOCK_RECURSIVE:
49	42	*unlock = false;
50	43	return true;
	44	+
	45	+ case MUTEX_TRYLOCK_FAILED:
	46	+ do {
	47	+ cpu_relax();
	48	+ if (mutex_trylock(&dev_priv->drm.struct_mutex)) {
	49	+ case MUTEX_TRYLOCK_SUCCESS:
	50	+ *unlock = true;
	51	+ return true;
	52	+ }
	53	+ } while (!need_resched());
	54	+
	55	+ return false;
51	56	}
52	57
53	58	BUG();
@@ -332,6 +337,15 @@ i915_gem_shrinker_scan(struct shrinker shrinker, struct shrink_control sc) · 278 unmodified lines
332	337	sc->nr_to_scan - freed,
333	338	I915_SHRINK_BOUND \|
334	339	I915_SHRINK_UNBOUND);
	340	+ if (freed < sc->nr_to_scan && current_is_kswapd()) {
	341	+ intel_runtime_pm_get(dev_priv);
	342	+ freed += i915_gem_shrink(dev_priv,
	343	+ sc->nr_to_scan - freed,
	344	+ I915_SHRINK_ACTIVE \|
	345	+ I915_SHRINK_BOUND \|
	346	+ I915_SHRINK_UNBOUND);
	347	+ intel_runtime_pm_put(dev_priv);
	348	+ }
335	349
336	350	shrinker_unlock(dev_priv, unlock);
337	351

drivers/gpu/drm/i915/i915_gem_stolen.c+1−0

@@ -590,6 +590,7 @@ _i915_gem_object_create_stolen(struct drm_i915_private *dev_priv, · 589 unmodified lines
590	590	obj->stolen = stolen;
591	591	obj->base.read_domains = I915_GEM_DOMAIN_CPU \| I915_GEM_DOMAIN_GTT;
592	592	obj->cache_level = HAS_LLC(dev_priv) ? I915_CACHE_LLC : I915_CACHE_NONE;
	593	+ obj->cache_coherent = true; /* assumptions! more like cache_oblivious */
593	594
594	595	if (i915_gem_object_pin_pages(obj))
595	596	goto cleanup;

drivers/gpu/drm/i915/i915_gem_userptr.c+19−5

@@ -378,7 +378,7 @@ __i915_mm_struct_free(struct kref *kref) · 377 unmodified lines
378	378	mutex_unlock(&mm->i915->mm_lock);
379	379
380	380	INIT_WORK(&mm->work, __i915_mm_struct_free__worker);
381		− schedule_work(&mm->work);
	381	+ queue_work(mm->i915->mm.userptr_wq, &mm->work);
382	382	}
383	383
384	384	static void
@@ -598,7 +598,7 @@ __i915_gem_userptr_get_pages_schedule(struct drm_i915_gem_object *obj) · 213 unmodified lines
598	598	get_task_struct(work->task);
599	599
600	600	INIT_WORK(&work->work, __i915_gem_userptr_get_pages_worker);
601		− schedule_work(&work->work);
	601	+ queue_work(to_i915(obj->base.dev)->mm.userptr_wq, &work->work);
602	602
603	603	return ERR_PTR(-EAGAIN);
604	604	}
@@ -802,9 +802,11 @@ i915_gem_userptr_ioctl(struct drm_device dev, void data, struct drm_file *file · 197 unmodified lines
802	802
803	803	drm_gem_private_object_init(dev, &obj->base, args->user_size);
804	804	i915_gem_object_init(obj, &i915_gem_userptr_ops);
805		− obj->cache_level = I915_CACHE_LLC;
806		− obj->base.write_domain = I915_GEM_DOMAIN_CPU;
807	805	obj->base.read_domains = I915_GEM_DOMAIN_CPU;
	806	+ obj->base.write_domain = I915_GEM_DOMAIN_CPU;
	807	+ obj->cache_level = I915_CACHE_LLC;
	808	+ obj->cache_coherent = i915_gem_object_is_coherent(obj);
	809	+ obj->cache_dirty = !obj->cache_coherent;
808	810
809	811	obj->userptr.ptr = args->user_ptr;
810	812	obj->userptr.read_only = !!(args->flags & I915_USERPTR_READ_ONLY);
@@ -828,8 +830,20 @@ i915_gem_userptr_ioctl(struct drm_device dev, void data, struct drm_file *file · 17 unmodified lines
828	830	return 0;
829	831	}
830	832
831		−void i915_gem_init_userptr(struct drm_i915_private *dev_priv)
	833	+int i915_gem_init_userptr(struct drm_i915_private *dev_priv)
832	834	{
833	835	mutex_init(&dev_priv->mm_lock);
834	836	hash_init(dev_priv->mm_structs);
	837	+
	838	+ dev_priv->mm.userptr_wq =
	839	+ alloc_workqueue("i915-userptr-acquire", WQ_HIGHPRI, 0);
	840	+ if (!dev_priv->mm.userptr_wq)
	841	+ return -ENOMEM;
	842	+
	843	+ return 0;
	844	+}
	845	+
	846	+void i915_gem_cleanup_userptr(struct drm_i915_private *dev_priv)
	847	+{
	848	+ destroy_workqueue(dev_priv->mm.userptr_wq);
835	849	}

drivers/gpu/drm/i915/i915_guc_submission.c+1−1

@@ -105,7 +105,7 @@ static int __reserve_doorbell(struct i915_guc_client *client) · 104 unmodified lines
105	105	end += offset;
106	106	}
107	107
108		− id = find_next_zero_bit(client->guc->doorbell_bitmap, offset, end);
	108	+ id = find_next_zero_bit(client->guc->doorbell_bitmap, end, offset);
109	109	if (id == end)
110	110	return -ENOSPC;
111	111

drivers/gpu/drm/i915/i915_irq.c+4−2

@@ -2548,7 +2548,8 @@ gen8_de_irq_handler(struct drm_i915_private *dev_priv, u32 master_ctl) · 2547 unmodified lines
2548	2548	I915_WRITE(SDEIIR, iir);
2549	2549	ret = IRQ_HANDLED;
2550	2550
2551		− if (HAS_PCH_SPT(dev_priv) \|\| HAS_PCH_KBP(dev_priv))
	2551	+ if (HAS_PCH_SPT(dev_priv) \|\| HAS_PCH_KBP(dev_priv) \|\|
	2552	+ HAS_PCH_CNP(dev_priv))
2552	2553	spt_irq_handler(dev_priv, iir);
2553	2554	else
2554	2555	cpt_irq_handler(dev_priv, iir);
@@ -4289,7 +4290,8 @@ void intel_irq_init(struct drm_i915_private *dev_priv) · 1734 unmodified lines
4289	4290	dev->driver->disable_vblank = gen8_disable_vblank;
4290	4291	if (IS_GEN9_LP(dev_priv))
4291	4292	dev_priv->display.hpd_irq_setup = bxt_hpd_irq_setup;
4292		− else if (HAS_PCH_SPT(dev_priv) \|\| HAS_PCH_KBP(dev_priv))
	4293	+ else if (HAS_PCH_SPT(dev_priv) \|\| HAS_PCH_KBP(dev_priv) \|\|
	4294	+ HAS_PCH_CNP(dev_priv))
4293	4295	dev_priv->display.hpd_irq_setup = spt_hpd_irq_setup;
4294	4296	else
4295	4297	dev_priv->display.hpd_irq_setup = ilk_hpd_irq_setup;

drivers/gpu/drm/i915/i915_oa_bdw.c+5376−0

@@ -0,0 +1,5376 @@
	1	+/*
	2	+ * Autogenerated file by GPU Top : https://github.com/rib/gputop
	3	+ * DO NOT EDIT manually!
	4	+ *
	5	+ *
	6	+ * Copyright (c) 2015 Intel Corporation
	7	+ *
	8	+ * Permission is hereby granted, free of charge, to any person obtaining a
	9	+ * copy of this software and associated documentation files (the "Software"),
	10	+ * to deal in the Software without restriction, including without limitation
	11	+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
	12	+ * and/or sell copies of the Software, and to permit persons to whom the
	13	+ * Software is furnished to do so, subject to the following conditions:
	14	+ *
	15	+ * The above copyright notice and this permission notice (including the next
	16	+ * paragraph) shall be included in all copies or substantial portions of the
	17	+ * Software.
	18	+ *
	19	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
	20	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
	21	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
	22	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
	23	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
	24	+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
	25	+ * IN THE SOFTWARE.
	26	+ *
	27	+ */
	28	+
	29	+#include <linux/sysfs.h>
	30	+
	31	+#include "i915_drv.h"
	32	+#include "i915_oa_bdw.h"
	33	+
	34	+enum metric_set_id {
	35	+ METRIC_SET_ID_RENDER_BASIC = 1,
	36	+ METRIC_SET_ID_COMPUTE_BASIC,
	37	+ METRIC_SET_ID_RENDER_PIPE_PROFILE,
	38	+ METRIC_SET_ID_MEMORY_READS,
	39	+ METRIC_SET_ID_MEMORY_WRITES,
	40	+ METRIC_SET_ID_COMPUTE_EXTENDED,
	41	+ METRIC_SET_ID_COMPUTE_L3_CACHE,
	42	+ METRIC_SET_ID_DATA_PORT_READS_COALESCING,
	43	+ METRIC_SET_ID_DATA_PORT_WRITES_COALESCING,
	44	+ METRIC_SET_ID_HDC_AND_SF,
	45	+ METRIC_SET_ID_L3_1,
	46	+ METRIC_SET_ID_L3_2,
	47	+ METRIC_SET_ID_L3_3,
	48	+ METRIC_SET_ID_L3_4,
	49	+ METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND,
	50	+ METRIC_SET_ID_SAMPLER_1,
	51	+ METRIC_SET_ID_SAMPLER_2,
	52	+ METRIC_SET_ID_TDL_1,
	53	+ METRIC_SET_ID_TDL_2,
	54	+ METRIC_SET_ID_COMPUTE_EXTRA,
	55	+ METRIC_SET_ID_VME_PIPE,
	56	+ METRIC_SET_ID_TEST_OA,
	57	+};
	58	+
	59	+int i915_oa_n_builtin_metric_sets_bdw = 22;
	60	+
	61	+static const struct i915_oa_reg b_counter_config_render_basic[] = {
	62	+ { _MMIO(0x2710), 0x00000000 },
	63	+ { _MMIO(0x2714), 0x00800000 },
	64	+ { _MMIO(0x2720), 0x00000000 },
	65	+ { _MMIO(0x2724), 0x00800000 },
	66	+ { _MMIO(0x2740), 0x00000000 },
	67	+};
	68	+
	69	+static const struct i915_oa_reg flex_eu_config_render_basic[] = {
	70	+ { _MMIO(0xe458), 0x00005004 },
	71	+ { _MMIO(0xe558), 0x00010003 },
	72	+ { _MMIO(0xe658), 0x00012011 },
	73	+ { _MMIO(0xe758), 0x00015014 },
	74	+ { _MMIO(0xe45c), 0x00051050 },
	75	+ { _MMIO(0xe55c), 0x00053052 },
	76	+ { _MMIO(0xe65c), 0x00055054 },
	77	+};
	78	+
	79	+static const struct i915_oa_reg mux_config_render_basic_0_slices_0x01[] = {
	80	+ { _MMIO(0x9888), 0x143f000f },
	81	+ { _MMIO(0x9888), 0x14110014 },
	82	+ { _MMIO(0x9888), 0x14310014 },
	83	+ { _MMIO(0x9888), 0x14bf000f },
	84	+ { _MMIO(0x9888), 0x118a0317 },
	85	+ { _MMIO(0x9888), 0x13837be0 },
	86	+ { _MMIO(0x9888), 0x3b800060 },
	87	+ { _MMIO(0x9888), 0x3d800005 },
	88	+ { _MMIO(0x9888), 0x005c4000 },
	89	+ { _MMIO(0x9888), 0x065c8000 },
	90	+ { _MMIO(0x9888), 0x085cc000 },
	91	+ { _MMIO(0x9888), 0x003d8000 },
	92	+ { _MMIO(0x9888), 0x183d0800 },
	93	+ { _MMIO(0x9888), 0x0a3f0023 },
	94	+ { _MMIO(0x9888), 0x103f0000 },
	95	+ { _MMIO(0x9888), 0x00584000 },
	96	+ { _MMIO(0x9888), 0x08584000 },
	97	+ { _MMIO(0x9888), 0x0a5a4000 },
	98	+ { _MMIO(0x9888), 0x005b4000 },
	99	+ { _MMIO(0x9888), 0x0e5b8000 },
	100	+ { _MMIO(0x9888), 0x185b2400 },
	101	+ { _MMIO(0x9888), 0x0a1d4000 },
	102	+ { _MMIO(0x9888), 0x0c1f0800 },
	103	+ { _MMIO(0x9888), 0x0e1faa00 },
	104	+ { _MMIO(0x9888), 0x00384000 },
	105	+ { _MMIO(0x9888), 0x0e384000 },
	106	+ { _MMIO(0x9888), 0x16384000 },
	107	+ { _MMIO(0x9888), 0x18380001 },
	108	+ { _MMIO(0x9888), 0x00392000 },
	109	+ { _MMIO(0x9888), 0x06398000 },
	110	+ { _MMIO(0x9888), 0x0839a000 },
	111	+ { _MMIO(0x9888), 0x0a391000 },
	112	+ { _MMIO(0x9888), 0x00104000 },
	113	+ { _MMIO(0x9888), 0x08104000 },
	114	+ { _MMIO(0x9888), 0x00110030 },
	115	+ { _MMIO(0x9888), 0x08110031 },
	116	+ { _MMIO(0x9888), 0x10110000 },
	117	+ { _MMIO(0x9888), 0x00134000 },
	118	+ { _MMIO(0x9888), 0x16130020 },
	119	+ { _MMIO(0x9888), 0x06308000 },
	120	+ { _MMIO(0x9888), 0x08308000 },
	121	+ { _MMIO(0x9888), 0x06311800 },
	122	+ { _MMIO(0x9888), 0x08311880 },
	123	+ { _MMIO(0x9888), 0x10310000 },
	124	+ { _MMIO(0x9888), 0x0e334000 },
	125	+ { _MMIO(0x9888), 0x16330080 },
	126	+ { _MMIO(0x9888), 0x0abf1180 },
	127	+ { _MMIO(0x9888), 0x10bf0000 },
	128	+ { _MMIO(0x9888), 0x0ada8000 },
	129	+ { _MMIO(0x9888), 0x0a9d8000 },
	130	+ { _MMIO(0x9888), 0x109f0002 },
	131	+ { _MMIO(0x9888), 0x0ab94000 },
	132	+ { _MMIO(0x9888), 0x0d888000 },
	133	+ { _MMIO(0x9888), 0x038a0380 },
	134	+ { _MMIO(0x9888), 0x058a000e },
	135	+ { _MMIO(0x9888), 0x018a8000 },
	136	+ { _MMIO(0x9888), 0x0f8a8000 },
	137	+ { _MMIO(0x9888), 0x198a8000 },
	138	+ { _MMIO(0x9888), 0x1b8a00a0 },
	139	+ { _MMIO(0x9888), 0x078a0000 },
	140	+ { _MMIO(0x9888), 0x098a0000 },
	141	+ { _MMIO(0x9888), 0x238b2820 },
	142	+ { _MMIO(0x9888), 0x258b2550 },
	143	+ { _MMIO(0x9888), 0x198c1000 },
	144	+ { _MMIO(0x9888), 0x0b8d8000 },
	145	+ { _MMIO(0x9888), 0x1f85aa80 },
	146	+ { _MMIO(0x9888), 0x2185aaa0 },
	147	+ { _MMIO(0x9888), 0x2385002a },
	148	+ { _MMIO(0x9888), 0x0d831021 },
	149	+ { _MMIO(0x9888), 0x0f83572f },
	150	+ { _MMIO(0x9888), 0x01835680 },
	151	+ { _MMIO(0x9888), 0x0383002c },
	152	+ { _MMIO(0x9888), 0x11830000 },
	153	+ { _MMIO(0x9888), 0x19835400 },
	154	+ { _MMIO(0x9888), 0x1b830001 },
	155	+ { _MMIO(0x9888), 0x05830000 },
	156	+ { _MMIO(0x9888), 0x07834000 },
	157	+ { _MMIO(0x9888), 0x09834000 },
	158	+ { _MMIO(0x9888), 0x0184c000 },
	159	+ { _MMIO(0x9888), 0x07848000 },
	160	+ { _MMIO(0x9888), 0x0984c000 },
	161	+ { _MMIO(0x9888), 0x0b84c000 },
	162	+ { _MMIO(0x9888), 0x0d84c000 },
	163	+ { _MMIO(0x9888), 0x0f84c000 },
	164	+ { _MMIO(0x9888), 0x0384c000 },
	165	+ { _MMIO(0x9888), 0x05844000 },
	166	+ { _MMIO(0x9888), 0x1b80c137 },
	167	+ { _MMIO(0x9888), 0x1d80c147 },
	168	+ { _MMIO(0x9888), 0x21800000 },
	169	+ { _MMIO(0x9888), 0x1180c000 },
	170	+ { _MMIO(0x9888), 0x17808000 },
	171	+ { _MMIO(0x9888), 0x1980c000 },
	172	+ { _MMIO(0x9888), 0x1f80c000 },
	173	+ { _MMIO(0x9888), 0x1380c000 },
	174	+ { _MMIO(0x9888), 0x15804000 },
	175	+ { _MMIO(0x9888), 0x4d801110 },
	176	+ { _MMIO(0x9888), 0x4f800331 },
	177	+ { _MMIO(0x9888), 0x43800802 },
	178	+ { _MMIO(0x9888), 0x51800000 },
	179	+ { _MMIO(0x9888), 0x45801465 },
	180	+ { _MMIO(0x9888), 0x53801111 },
	181	+ { _MMIO(0x9888), 0x478014a5 },
	182	+ { _MMIO(0x9888), 0x31800000 },
	183	+ { _MMIO(0x9888), 0x3f800ca5 },
	184	+ { _MMIO(0x9888), 0x41800003 },
	185	+};
	186	+
	187	+static const struct i915_oa_reg mux_config_render_basic_1_slices_0x02[] = {
	188	+ { _MMIO(0x9888), 0x143f000f },
	189	+ { _MMIO(0x9888), 0x14bf000f },
	190	+ { _MMIO(0x9888), 0x14910014 },
	191	+ { _MMIO(0x9888), 0x14b10014 },
	192	+ { _MMIO(0x9888), 0x118a0317 },
	193	+ { _MMIO(0x9888), 0x13837be0 },
	194	+ { _MMIO(0x9888), 0x3b800060 },
	195	+ { _MMIO(0x9888), 0x3d800005 },
	196	+ { _MMIO(0x9888), 0x0a3f0023 },
	197	+ { _MMIO(0x9888), 0x103f0000 },
	198	+ { _MMIO(0x9888), 0x0a5a4000 },
	199	+ { _MMIO(0x9888), 0x0a1d4000 },
	200	+ { _MMIO(0x9888), 0x0e1f8000 },
	201	+ { _MMIO(0x9888), 0x0a391000 },
	202	+ { _MMIO(0x9888), 0x00dc4000 },
	203	+ { _MMIO(0x9888), 0x06dc8000 },
	204	+ { _MMIO(0x9888), 0x08dcc000 },
	205	+ { _MMIO(0x9888), 0x00bd8000 },
	206	+ { _MMIO(0x9888), 0x18bd0800 },
	207	+ { _MMIO(0x9888), 0x0abf1180 },
	208	+ { _MMIO(0x9888), 0x10bf0000 },
	209	+ { _MMIO(0x9888), 0x00d84000 },
	210	+ { _MMIO(0x9888), 0x08d84000 },
	211	+ { _MMIO(0x9888), 0x0ada8000 },
	212	+ { _MMIO(0x9888), 0x00db4000 },
	213	+ { _MMIO(0x9888), 0x0edb8000 },
	214	+ { _MMIO(0x9888), 0x18db2400 },
	215	+ { _MMIO(0x9888), 0x0a9d8000 },
	216	+ { _MMIO(0x9888), 0x0c9f0800 },
	217	+ { _MMIO(0x9888), 0x0e9f2a00 },
	218	+ { _MMIO(0x9888), 0x109f0002 },
	219	+ { _MMIO(0x9888), 0x00b84000 },
	220	+ { _MMIO(0x9888), 0x0eb84000 },
	221	+ { _MMIO(0x9888), 0x16b84000 },
	222	+ { _MMIO(0x9888), 0x18b80001 },
	223	+ { _MMIO(0x9888), 0x00b92000 },
	224	+ { _MMIO(0x9888), 0x06b98000 },
	225	+ { _MMIO(0x9888), 0x08b9a000 },
	226	+ { _MMIO(0x9888), 0x0ab94000 },
	227	+ { _MMIO(0x9888), 0x00904000 },
	228	+ { _MMIO(0x9888), 0x08904000 },
	229	+ { _MMIO(0x9888), 0x00910030 },
	230	+ { _MMIO(0x9888), 0x08910031 },
	231	+ { _MMIO(0x9888), 0x10910000 },
	232	+ { _MMIO(0x9888), 0x00934000 },
	233	+ { _MMIO(0x9888), 0x16930020 },
	234	+ { _MMIO(0x9888), 0x06b08000 },
	235	+ { _MMIO(0x9888), 0x08b08000 },
	236	+ { _MMIO(0x9888), 0x06b11800 },
	237	+ { _MMIO(0x9888), 0x08b11880 },
	238	+ { _MMIO(0x9888), 0x10b10000 },
	239	+ { _MMIO(0x9888), 0x0eb34000 },
	240	+ { _MMIO(0x9888), 0x16b30080 },
	241	+ { _MMIO(0x9888), 0x01888000 },
	242	+ { _MMIO(0x9888), 0x0d88b800 },
	243	+ { _MMIO(0x9888), 0x038a0380 },
	244	+ { _MMIO(0x9888), 0x058a000e },
	245	+ { _MMIO(0x9888), 0x1b8a0080 },
	246	+ { _MMIO(0x9888), 0x078a0000 },
	247	+ { _MMIO(0x9888), 0x098a0000 },
	248	+ { _MMIO(0x9888), 0x238b2840 },
	249	+ { _MMIO(0x9888), 0x258b26a0 },
	250	+ { _MMIO(0x9888), 0x018c4000 },
	251	+ { _MMIO(0x9888), 0x0f8c4000 },
	252	+ { _MMIO(0x9888), 0x178c2000 },
	253	+ { _MMIO(0x9888), 0x198c1100 },
	254	+ { _MMIO(0x9888), 0x018d2000 },
	255	+ { _MMIO(0x9888), 0x078d8000 },
	256	+ { _MMIO(0x9888), 0x098da000 },
	257	+ { _MMIO(0x9888), 0x0b8d8000 },
	258	+ { _MMIO(0x9888), 0x1f85aa80 },
	259	+ { _MMIO(0x9888), 0x2185aaa0 },
	260	+ { _MMIO(0x9888), 0x2385002a },
	261	+ { _MMIO(0x9888), 0x0d831021 },
	262	+ { _MMIO(0x9888), 0x0f83572f },
	263	+ { _MMIO(0x9888), 0x01835680 },
	264	+ { _MMIO(0x9888), 0x0383002c },
	265	+ { _MMIO(0x9888), 0x11830000 },
	266	+ { _MMIO(0x9888), 0x19835400 },
	267	+ { _MMIO(0x9888), 0x1b830001 },
	268	+ { _MMIO(0x9888), 0x05830000 },
	269	+ { _MMIO(0x9888), 0x07834000 },
	270	+ { _MMIO(0x9888), 0x09834000 },
	271	+ { _MMIO(0x9888), 0x0184c000 },
	272	+ { _MMIO(0x9888), 0x07848000 },
	273	+ { _MMIO(0x9888), 0x0984c000 },
	274	+ { _MMIO(0x9888), 0x0b84c000 },
	275	+ { _MMIO(0x9888), 0x0d84c000 },
	276	+ { _MMIO(0x9888), 0x0f84c000 },
	277	+ { _MMIO(0x9888), 0x0384c000 },
	278	+ { _MMIO(0x9888), 0x05844000 },
	279	+ { _MMIO(0x9888), 0x1b80c137 },
	280	+ { _MMIO(0x9888), 0x1d80c147 },
	281	+ { _MMIO(0x9888), 0x21800000 },
	282	+ { _MMIO(0x9888), 0x1180c000 },
	283	+ { _MMIO(0x9888), 0x17808000 },
	284	+ { _MMIO(0x9888), 0x1980c000 },
	285	+ { _MMIO(0x9888), 0x1f80c000 },
	286	+ { _MMIO(0x9888), 0x1380c000 },
	287	+ { _MMIO(0x9888), 0x15804000 },
	288	+ { _MMIO(0x9888), 0x4d801550 },
	289	+ { _MMIO(0x9888), 0x4f800331 },
	290	+ { _MMIO(0x9888), 0x43800802 },
	291	+ { _MMIO(0x9888), 0x51800400 },
	292	+ { _MMIO(0x9888), 0x458004a1 },
	293	+ { _MMIO(0x9888), 0x53805555 },
	294	+ { _MMIO(0x9888), 0x47800421 },
	295	+ { _MMIO(0x9888), 0x31800000 },
	296	+ { _MMIO(0x9888), 0x3f801421 },
	297	+ { _MMIO(0x9888), 0x41800845 },
	298	+};
	299	+
	300	+static int
	301	+get_render_basic_mux_config(struct drm_i915_private *dev_priv,
	302	+ const struct i915_oa_reg **regs,
	303	+ int *lens)
	304	+{
	305	+ int n = 0;
	306	+
	307	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 2);
	308	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 2);
	309	+
	310	+ if (INTEL_INFO(dev_priv)->sseu.slice_mask & 0x01) {
	311	+ regs[n] = mux_config_render_basic_0_slices_0x01;
	312	+ lens[n] = ARRAY_SIZE(mux_config_render_basic_0_slices_0x01);
	313	+ n++;
	314	+ }
	315	+ if (INTEL_INFO(dev_priv)->sseu.slice_mask & 0x02) {
	316	+ regs[n] = mux_config_render_basic_1_slices_0x02;
	317	+ lens[n] = ARRAY_SIZE(mux_config_render_basic_1_slices_0x02);
	318	+ n++;
	319	+ }
	320	+
	321	+ return n;
	322	+}
	323	+
	324	+static const struct i915_oa_reg b_counter_config_compute_basic[] = {
	325	+ { _MMIO(0x2710), 0x00000000 },
	326	+ { _MMIO(0x2714), 0x00800000 },
	327	+ { _MMIO(0x2720), 0x00000000 },
	328	+ { _MMIO(0x2724), 0x00800000 },
	329	+ { _MMIO(0x2740), 0x00000000 },
	330	+};
	331	+
	332	+static const struct i915_oa_reg flex_eu_config_compute_basic[] = {
	333	+ { _MMIO(0xe458), 0x00005004 },
	334	+ { _MMIO(0xe558), 0x00000003 },
	335	+ { _MMIO(0xe658), 0x00002001 },
	336	+ { _MMIO(0xe758), 0x00778008 },
	337	+ { _MMIO(0xe45c), 0x00088078 },
	338	+ { _MMIO(0xe55c), 0x00808708 },
	339	+ { _MMIO(0xe65c), 0x00a08908 },
	340	+};
	341	+
	342	+static const struct i915_oa_reg mux_config_compute_basic_0_slices_0x01[] = {
	343	+ { _MMIO(0x9888), 0x105c00e0 },
	344	+ { _MMIO(0x9888), 0x105800e0 },
	345	+ { _MMIO(0x9888), 0x103800e0 },
	346	+ { _MMIO(0x9888), 0x3580001a },
	347	+ { _MMIO(0x9888), 0x3b800060 },
	348	+ { _MMIO(0x9888), 0x3d800005 },
	349	+ { _MMIO(0x9888), 0x065c2100 },
	350	+ { _MMIO(0x9888), 0x0a5c0041 },
	351	+ { _MMIO(0x9888), 0x0c5c6600 },
	352	+ { _MMIO(0x9888), 0x005c6580 },
	353	+ { _MMIO(0x9888), 0x085c8000 },
	354	+ { _MMIO(0x9888), 0x0e5c8000 },
	355	+ { _MMIO(0x9888), 0x00580042 },
	356	+ { _MMIO(0x9888), 0x08582080 },
	357	+ { _MMIO(0x9888), 0x0c58004c },
	358	+ { _MMIO(0x9888), 0x0e582580 },
	359	+ { _MMIO(0x9888), 0x005b4000 },
	360	+ { _MMIO(0x9888), 0x185b1000 },
	361	+ { _MMIO(0x9888), 0x1a5b0104 },
	362	+ { _MMIO(0x9888), 0x0c1fa800 },
	363	+ { _MMIO(0x9888), 0x0e1faa00 },
	364	+ { _MMIO(0x9888), 0x101f02aa },
	365	+ { _MMIO(0x9888), 0x08380042 },
	366	+ { _MMIO(0x9888), 0x0a382080 },
	367	+ { _MMIO(0x9888), 0x0e38404c },
	368	+ { _MMIO(0x9888), 0x0238404b },
	369	+ { _MMIO(0x9888), 0x00384000 },
	370	+ { _MMIO(0x9888), 0x16380000 },
	371	+ { _MMIO(0x9888), 0x18381145 },
	372	+ { _MMIO(0x9888), 0x04380000 },
	373	+ { _MMIO(0x9888), 0x0039a000 },
	374	+ { _MMIO(0x9888), 0x06398000 },
	375	+ { _MMIO(0x9888), 0x0839a000 },
	376	+ { _MMIO(0x9888), 0x0a39a000 },
	377	+ { _MMIO(0x9888), 0x0c39a000 },
	378	+ { _MMIO(0x9888), 0x0e39a000 },
	379	+ { _MMIO(0x9888), 0x02392000 },
	380	+ { _MMIO(0x9888), 0x018a8000 },
	381	+ { _MMIO(0x9888), 0x0f8a8000 },
	382	+ { _MMIO(0x9888), 0x198a8000 },
	383	+ { _MMIO(0x9888), 0x1b8aaaa0 },
	384	+ { _MMIO(0x9888), 0x1d8a0002 },
	385	+ { _MMIO(0x9888), 0x038a8000 },
	386	+ { _MMIO(0x9888), 0x058a8000 },
	387	+ { _MMIO(0x9888), 0x238b02a0 },
	388	+ { _MMIO(0x9888), 0x258b5550 },
	389	+ { _MMIO(0x9888), 0x278b0015 },
	390	+ { _MMIO(0x9888), 0x1f850a80 },
	391	+ { _MMIO(0x9888), 0x2185aaa0 },
	392	+ { _MMIO(0x9888), 0x2385002a },
	393	+ { _MMIO(0x9888), 0x01834000 },
	394	+ { _MMIO(0x9888), 0x0f834000 },
	395	+ { _MMIO(0x9888), 0x19835400 },
	396	+ { _MMIO(0x9888), 0x1b830155 },
	397	+ { _MMIO(0x9888), 0x03834000 },
	398	+ { _MMIO(0x9888), 0x05834000 },
	399	+ { _MMIO(0x9888), 0x0184c000 },
	400	+ { _MMIO(0x9888), 0x07848000 },
	401	+ { _MMIO(0x9888), 0x0984c000 },
	402	+ { _MMIO(0x9888), 0x0b84c000 },
	403	+ { _MMIO(0x9888), 0x0d84c000 },
	404	+ { _MMIO(0x9888), 0x0f84c000 },
	405	+ { _MMIO(0x9888), 0x03844000 },
	406	+ { _MMIO(0x9888), 0x17808137 },
	407	+ { _MMIO(0x9888), 0x1980c147 },
	408	+ { _MMIO(0x9888), 0x1b80c0e5 },
	409	+ { _MMIO(0x9888), 0x1d80c0e3 },
	410	+ { _MMIO(0x9888), 0x21800000 },
	411	+ { _MMIO(0x9888), 0x1180c000 },
	412	+ { _MMIO(0x9888), 0x1f80c000 },
	413	+ { _MMIO(0x9888), 0x13804000 },
	414	+ { _MMIO(0x9888), 0x15800000 },
	415	+ { _MMIO(0xd24), 0x00000000 },
	416	+ { _MMIO(0x9888), 0x4d801000 },
	417	+ { _MMIO(0x9888), 0x4f800111 },
	418	+ { _MMIO(0x9888), 0x43800062 },
	419	+ { _MMIO(0x9888), 0x51800000 },
	420	+ { _MMIO(0x9888), 0x45800062 },
	421	+ { _MMIO(0x9888), 0x53800000 },
	422	+ { _MMIO(0x9888), 0x47800062 },
	423	+ { _MMIO(0x9888), 0x31800000 },
	424	+ { _MMIO(0x9888), 0x3f801062 },
	425	+ { _MMIO(0x9888), 0x41801084 },
	426	+};
	427	+
	428	+static const struct i915_oa_reg mux_config_compute_basic_2_slices_0x02[] = {
	429	+ { _MMIO(0x9888), 0x10dc00e0 },
	430	+ { _MMIO(0x9888), 0x10d800e0 },
	431	+ { _MMIO(0x9888), 0x10b800e0 },
	432	+ { _MMIO(0x9888), 0x3580001a },
	433	+ { _MMIO(0x9888), 0x3b800060 },
	434	+ { _MMIO(0x9888), 0x3d800005 },
	435	+ { _MMIO(0x9888), 0x06dc2100 },
	436	+ { _MMIO(0x9888), 0x0adc0041 },
	437	+ { _MMIO(0x9888), 0x0cdc6600 },
	438	+ { _MMIO(0x9888), 0x00dc6580 },
	439	+ { _MMIO(0x9888), 0x08dc8000 },
	440	+ { _MMIO(0x9888), 0x0edc8000 },
	441	+ { _MMIO(0x9888), 0x00d80042 },
	442	+ { _MMIO(0x9888), 0x08d82080 },
	443	+ { _MMIO(0x9888), 0x0cd8004c },
	444	+ { _MMIO(0x9888), 0x0ed82580 },
	445	+ { _MMIO(0x9888), 0x00db4000 },
	446	+ { _MMIO(0x9888), 0x18db1000 },
	447	+ { _MMIO(0x9888), 0x1adb0104 },
	448	+ { _MMIO(0x9888), 0x0c9fa800 },
	449	+ { _MMIO(0x9888), 0x0e9faa00 },
	450	+ { _MMIO(0x9888), 0x109f02aa },
	451	+ { _MMIO(0x9888), 0x08b80042 },
	452	+ { _MMIO(0x9888), 0x0ab82080 },
	453	+ { _MMIO(0x9888), 0x0eb8404c },
	454	+ { _MMIO(0x9888), 0x02b8404b },
	455	+ { _MMIO(0x9888), 0x00b84000 },
	456	+ { _MMIO(0x9888), 0x16b80000 },
	457	+ { _MMIO(0x9888), 0x18b81145 },
	458	+ { _MMIO(0x9888), 0x04b80000 },
	459	+ { _MMIO(0x9888), 0x00b9a000 },
	460	+ { _MMIO(0x9888), 0x06b98000 },
	461	+ { _MMIO(0x9888), 0x08b9a000 },
	462	+ { _MMIO(0x9888), 0x0ab9a000 },
	463	+ { _MMIO(0x9888), 0x0cb9a000 },
	464	+ { _MMIO(0x9888), 0x0eb9a000 },
	465	+ { _MMIO(0x9888), 0x02b92000 },
	466	+ { _MMIO(0x9888), 0x01888000 },
	467	+ { _MMIO(0x9888), 0x0d88f800 },
	468	+ { _MMIO(0x9888), 0x0f88000f },
	469	+ { _MMIO(0x9888), 0x03888000 },
	470	+ { _MMIO(0x9888), 0x05888000 },
	471	+ { _MMIO(0x9888), 0x238b0540 },
	472	+ { _MMIO(0x9888), 0x258baaa0 },
	473	+ { _MMIO(0x9888), 0x278b002a },
	474	+ { _MMIO(0x9888), 0x018c4000 },
	475	+ { _MMIO(0x9888), 0x0f8c4000 },
	476	+ { _MMIO(0x9888), 0x178c2000 },
	477	+ { _MMIO(0x9888), 0x198c5500 },
	478	+ { _MMIO(0x9888), 0x1b8c0015 },
	479	+ { _MMIO(0x9888), 0x038c4000 },
	480	+ { _MMIO(0x9888), 0x058c4000 },
	481	+ { _MMIO(0x9888), 0x018da000 },
	482	+ { _MMIO(0x9888), 0x078d8000 },
	483	+ { _MMIO(0x9888), 0x098da000 },
	484	+ { _MMIO(0x9888), 0x0b8da000 },
	485	+ { _MMIO(0x9888), 0x0d8da000 },
	486	+ { _MMIO(0x9888), 0x0f8da000 },
	487	+ { _MMIO(0x9888), 0x038d2000 },
	488	+ { _MMIO(0x9888), 0x1f850a80 },
	489	+ { _MMIO(0x9888), 0x2185aaa0 },
	490	+ { _MMIO(0x9888), 0x2385002a },
	491	+ { _MMIO(0x9888), 0x01834000 },
	492	+ { _MMIO(0x9888), 0x0f834000 },
	493	+ { _MMIO(0x9888), 0x19835400 },
	494	+ { _MMIO(0x9888), 0x1b830155 },
	495	+ { _MMIO(0x9888), 0x03834000 },
	496	+ { _MMIO(0x9888), 0x05834000 },
	497	+ { _MMIO(0x9888), 0x0184c000 },
	498	+ { _MMIO(0x9888), 0x07848000 },
	499	+ { _MMIO(0x9888), 0x0984c000 },
	500	+ { _MMIO(0x9888), 0x0b84c000 },
	501	+ { _MMIO(0x9888), 0x0d84c000 },
	502	+ { _MMIO(0x9888), 0x0f84c000 },
	503	+ { _MMIO(0x9888), 0x03844000 },
	504	+ { _MMIO(0x9888), 0x17808137 },
	505	+ { _MMIO(0x9888), 0x1980c147 },
	506	+ { _MMIO(0x9888), 0x1b80c0e5 },
	507	+ { _MMIO(0x9888), 0x1d80c0e3 },
	508	+ { _MMIO(0x9888), 0x21800000 },
	509	+ { _MMIO(0x9888), 0x1180c000 },
	510	+ { _MMIO(0x9888), 0x1f80c000 },
	511	+ { _MMIO(0x9888), 0x13804000 },
	512	+ { _MMIO(0x9888), 0x15800000 },
	513	+ { _MMIO(0xd24), 0x00000000 },
	514	+ { _MMIO(0x9888), 0x4d805000 },
	515	+ { _MMIO(0x9888), 0x4f800555 },
	516	+ { _MMIO(0x9888), 0x43800062 },
	517	+ { _MMIO(0x9888), 0x51800000 },
	518	+ { _MMIO(0x9888), 0x45800062 },
	519	+ { _MMIO(0x9888), 0x53800000 },
	520	+ { _MMIO(0x9888), 0x47800062 },
	521	+ { _MMIO(0x9888), 0x31800000 },
	522	+ { _MMIO(0x9888), 0x3f800062 },
	523	+ { _MMIO(0x9888), 0x41800000 },
	524	+};
	525	+
	526	+static int
	527	+get_compute_basic_mux_config(struct drm_i915_private *dev_priv,
	528	+ const struct i915_oa_reg **regs,
	529	+ int *lens)
	530	+{
	531	+ int n = 0;
	532	+
	533	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 2);
	534	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 2);
	535	+
	536	+ if (INTEL_INFO(dev_priv)->sseu.slice_mask & 0x01) {
	537	+ regs[n] = mux_config_compute_basic_0_slices_0x01;
	538	+ lens[n] = ARRAY_SIZE(mux_config_compute_basic_0_slices_0x01);
	539	+ n++;
	540	+ }
	541	+ if (INTEL_INFO(dev_priv)->sseu.slice_mask & 0x02) {
	542	+ regs[n] = mux_config_compute_basic_2_slices_0x02;
	543	+ lens[n] = ARRAY_SIZE(mux_config_compute_basic_2_slices_0x02);
	544	+ n++;
	545	+ }
	546	+
	547	+ return n;
	548	+}
	549	+
	550	+static const struct i915_oa_reg b_counter_config_render_pipe_profile[] = {
	551	+ { _MMIO(0x2724), 0xf0800000 },
	552	+ { _MMIO(0x2720), 0x00000000 },
	553	+ { _MMIO(0x2714), 0xf0800000 },
	554	+ { _MMIO(0x2710), 0x00000000 },
	555	+ { _MMIO(0x2740), 0x00000000 },
	556	+ { _MMIO(0x2770), 0x0007ffea },
	557	+ { _MMIO(0x2774), 0x00007ffc },
	558	+ { _MMIO(0x2778), 0x0007affa },
	559	+ { _MMIO(0x277c), 0x0000f5fd },
	560	+ { _MMIO(0x2780), 0x00079ffa },
	561	+ { _MMIO(0x2784), 0x0000f3fb },
	562	+ { _MMIO(0x2788), 0x0007bf7a },
	563	+ { _MMIO(0x278c), 0x0000f7e7 },
	564	+ { _MMIO(0x2790), 0x0007fefa },
	565	+ { _MMIO(0x2794), 0x0000f7cf },
	566	+ { _MMIO(0x2798), 0x00077ffa },
	567	+ { _MMIO(0x279c), 0x0000efdf },
	568	+ { _MMIO(0x27a0), 0x0006fffa },
	569	+ { _MMIO(0x27a4), 0x0000cfbf },
	570	+ { _MMIO(0x27a8), 0x0003fffa },
	571	+ { _MMIO(0x27ac), 0x00005f7f },
	572	+};
	573	+
	574	+static const struct i915_oa_reg flex_eu_config_render_pipe_profile[] = {
	575	+ { _MMIO(0xe458), 0x00005004 },
	576	+ { _MMIO(0xe558), 0x00015014 },
	577	+ { _MMIO(0xe658), 0x00025024 },
	578	+ { _MMIO(0xe758), 0x00035034 },
	579	+ { _MMIO(0xe45c), 0x00045044 },
	580	+ { _MMIO(0xe55c), 0x00055054 },
	581	+ { _MMIO(0xe65c), 0x00065064 },
	582	+};
	583	+
	584	+static const struct i915_oa_reg mux_config_render_pipe_profile[] = {
	585	+ { _MMIO(0x9888), 0x0a1e0000 },
	586	+ { _MMIO(0x9888), 0x0c1f000f },
	587	+ { _MMIO(0x9888), 0x10176800 },
	588	+ { _MMIO(0x9888), 0x1191001f },
	589	+ { _MMIO(0x9888), 0x0b880320 },
	590	+ { _MMIO(0x9888), 0x01890c40 },
	591	+ { _MMIO(0x9888), 0x118a1c00 },
	592	+ { _MMIO(0x9888), 0x118d7c00 },
	593	+ { _MMIO(0x9888), 0x118e0020 },
	594	+ { _MMIO(0x9888), 0x118f4c00 },
	595	+ { _MMIO(0x9888), 0x11900000 },
	596	+ { _MMIO(0x9888), 0x13900001 },
	597	+ { _MMIO(0x9888), 0x065c4000 },
	598	+ { _MMIO(0x9888), 0x0c3d8000 },
	599	+ { _MMIO(0x9888), 0x06584000 },
	600	+ { _MMIO(0x9888), 0x0c5b4000 },
	601	+ { _MMIO(0x9888), 0x081e0040 },
	602	+ { _MMIO(0x9888), 0x0e1e0000 },
	603	+ { _MMIO(0x9888), 0x021f5400 },
	604	+ { _MMIO(0x9888), 0x001f0000 },
	605	+ { _MMIO(0x9888), 0x101f0010 },
	606	+ { _MMIO(0x9888), 0x0e1f0080 },
	607	+ { _MMIO(0x9888), 0x0c384000 },
	608	+ { _MMIO(0x9888), 0x06392000 },
	609	+ { _MMIO(0x9888), 0x0c13c000 },
	610	+ { _MMIO(0x9888), 0x06164000 },
	611	+ { _MMIO(0x9888), 0x06170012 },
	612	+ { _MMIO(0x9888), 0x00170000 },
	613	+ { _MMIO(0x9888), 0x01910005 },
	614	+ { _MMIO(0x9888), 0x07880002 },
	615	+ { _MMIO(0x9888), 0x01880c00 },
	616	+ { _MMIO(0x9888), 0x0f880000 },
	617	+ { _MMIO(0x9888), 0x0d880000 },
	618	+ { _MMIO(0x9888), 0x05880000 },
	619	+ { _MMIO(0x9888), 0x09890032 },
	620	+ { _MMIO(0x9888), 0x078a0800 },
	621	+ { _MMIO(0x9888), 0x0f8a0a00 },
	622	+ { _MMIO(0x9888), 0x198a4000 },
	623	+ { _MMIO(0x9888), 0x1b8a2000 },
	624	+ { _MMIO(0x9888), 0x1d8a0000 },
	625	+ { _MMIO(0x9888), 0x038a4000 },
	626	+ { _MMIO(0x9888), 0x0b8a8000 },
	627	+ { _MMIO(0x9888), 0x0d8a8000 },
	628	+ { _MMIO(0x9888), 0x238b54c0 },
	629	+ { _MMIO(0x9888), 0x258baa55 },
	630	+ { _MMIO(0x9888), 0x278b0019 },
	631	+ { _MMIO(0x9888), 0x198c0100 },
	632	+ { _MMIO(0x9888), 0x058c4000 },
	633	+ { _MMIO(0x9888), 0x0f8d0015 },
	634	+ { _MMIO(0x9888), 0x018d1000 },
	635	+ { _MMIO(0x9888), 0x098d8000 },
	636	+ { _MMIO(0x9888), 0x0b8df000 },
	637	+ { _MMIO(0x9888), 0x0d8d3000 },
	638	+ { _MMIO(0x9888), 0x038de000 },
	639	+ { _MMIO(0x9888), 0x058d3000 },
	640	+ { _MMIO(0x9888), 0x0d8e0004 },
	641	+ { _MMIO(0x9888), 0x058e000c },
	642	+ { _MMIO(0x9888), 0x098e0000 },
	643	+ { _MMIO(0x9888), 0x078e0000 },
	644	+ { _MMIO(0x9888), 0x038e0000 },
	645	+ { _MMIO(0x9888), 0x0b8f0020 },
	646	+ { _MMIO(0x9888), 0x198f0c00 },
	647	+ { _MMIO(0x9888), 0x078f8000 },
	648	+ { _MMIO(0x9888), 0x098f4000 },
	649	+ { _MMIO(0x9888), 0x0b900980 },
	650	+ { _MMIO(0x9888), 0x03900d80 },
	651	+ { _MMIO(0x9888), 0x01900000 },
	652	+ { _MMIO(0x9888), 0x1f85aa80 },
	653	+ { _MMIO(0x9888), 0x2185aaaa },
	654	+ { _MMIO(0x9888), 0x2385002a },
	655	+ { _MMIO(0x9888), 0x01834000 },
	656	+ { _MMIO(0x9888), 0x0f834000 },
	657	+ { _MMIO(0x9888), 0x19835400 },
	658	+ { _MMIO(0x9888), 0x1b830155 },
	659	+ { _MMIO(0x9888), 0x03834000 },
	660	+ { _MMIO(0x9888), 0x05834000 },
	661	+ { _MMIO(0x9888), 0x07834000 },
	662	+ { _MMIO(0x9888), 0x09834000 },
	663	+ { _MMIO(0x9888), 0x0b834000 },
	664	+ { _MMIO(0x9888), 0x0d834000 },
	665	+ { _MMIO(0x9888), 0x0184c000 },
	666	+ { _MMIO(0x9888), 0x0784c000 },
	667	+ { _MMIO(0x9888), 0x0984c000 },
	668	+ { _MMIO(0x9888), 0x0b84c000 },
	669	+ { _MMIO(0x9888), 0x0d84c000 },
	670	+ { _MMIO(0x9888), 0x0f84c000 },
	671	+ { _MMIO(0x9888), 0x0384c000 },
	672	+ { _MMIO(0x9888), 0x0584c000 },
	673	+ { _MMIO(0x9888), 0x1180c000 },
	674	+ { _MMIO(0x9888), 0x1780c000 },
	675	+ { _MMIO(0x9888), 0x1980c000 },
	676	+ { _MMIO(0x9888), 0x1b80c000 },
	677	+ { _MMIO(0x9888), 0x1d80c000 },
	678	+ { _MMIO(0x9888), 0x1f80c000 },
	679	+ { _MMIO(0x9888), 0x1380c000 },
	680	+ { _MMIO(0x9888), 0x1580c000 },
	681	+ { _MMIO(0xd24), 0x00000000 },
	682	+ { _MMIO(0x9888), 0x4d801111 },
	683	+ { _MMIO(0x9888), 0x3d800800 },
	684	+ { _MMIO(0x9888), 0x4f801011 },
	685	+ { _MMIO(0x9888), 0x43800443 },
	686	+ { _MMIO(0x9888), 0x51801111 },
	687	+ { _MMIO(0x9888), 0x45800422 },
	688	+ { _MMIO(0x9888), 0x53801111 },
	689	+ { _MMIO(0x9888), 0x47800c60 },
	690	+ { _MMIO(0x9888), 0x21800000 },
	691	+ { _MMIO(0x9888), 0x31800000 },
	692	+ { _MMIO(0x9888), 0x3f800422 },
	693	+ { _MMIO(0x9888), 0x41800021 },
	694	+};
	695	+
	696	+static int
	697	+get_render_pipe_profile_mux_config(struct drm_i915_private *dev_priv,
	698	+ const struct i915_oa_reg **regs,
	699	+ int *lens)
	700	+{
	701	+ int n = 0;
	702	+
	703	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	704	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	705	+
	706	+ regs[n] = mux_config_render_pipe_profile;
	707	+ lens[n] = ARRAY_SIZE(mux_config_render_pipe_profile);
	708	+ n++;
	709	+
	710	+ return n;
	711	+}
	712	+
	713	+static const struct i915_oa_reg b_counter_config_memory_reads[] = {
	714	+ { _MMIO(0x2724), 0xf0800000 },
	715	+ { _MMIO(0x2720), 0x00000000 },
	716	+ { _MMIO(0x2714), 0xf0800000 },
	717	+ { _MMIO(0x2710), 0x00000000 },
	718	+ { _MMIO(0x274c), 0x86543210 },
	719	+ { _MMIO(0x2748), 0x86543210 },
	720	+ { _MMIO(0x2744), 0x00006667 },
	721	+ { _MMIO(0x2740), 0x00000000 },
	722	+ { _MMIO(0x275c), 0x86543210 },
	723	+ { _MMIO(0x2758), 0x86543210 },
	724	+ { _MMIO(0x2754), 0x00006465 },
	725	+ { _MMIO(0x2750), 0x00000000 },
	726	+ { _MMIO(0x2770), 0x0007f81a },
	727	+ { _MMIO(0x2774), 0x0000fe00 },
	728	+ { _MMIO(0x2778), 0x0007f82a },
	729	+ { _MMIO(0x277c), 0x0000fe00 },
	730	+ { _MMIO(0x2780), 0x0007f872 },
	731	+ { _MMIO(0x2784), 0x0000fe00 },
	732	+ { _MMIO(0x2788), 0x0007f8ba },
	733	+ { _MMIO(0x278c), 0x0000fe00 },
	734	+ { _MMIO(0x2790), 0x0007f87a },
	735	+ { _MMIO(0x2794), 0x0000fe00 },
	736	+ { _MMIO(0x2798), 0x0007f8ea },
	737	+ { _MMIO(0x279c), 0x0000fe00 },
	738	+ { _MMIO(0x27a0), 0x0007f8e2 },
	739	+ { _MMIO(0x27a4), 0x0000fe00 },
	740	+ { _MMIO(0x27a8), 0x0007f8f2 },
	741	+ { _MMIO(0x27ac), 0x0000fe00 },
	742	+};
	743	+
	744	+static const struct i915_oa_reg flex_eu_config_memory_reads[] = {
	745	+ { _MMIO(0xe458), 0x00005004 },
	746	+ { _MMIO(0xe558), 0x00015014 },
	747	+ { _MMIO(0xe658), 0x00025024 },
	748	+ { _MMIO(0xe758), 0x00035034 },
	749	+ { _MMIO(0xe45c), 0x00045044 },
	750	+ { _MMIO(0xe55c), 0x00055054 },
	751	+ { _MMIO(0xe65c), 0x00065064 },
	752	+};
	753	+
	754	+static const struct i915_oa_reg mux_config_memory_reads[] = {
	755	+ { _MMIO(0x9888), 0x198b0343 },
	756	+ { _MMIO(0x9888), 0x13845800 },
	757	+ { _MMIO(0x9888), 0x15840018 },
	758	+ { _MMIO(0x9888), 0x3580001a },
	759	+ { _MMIO(0x9888), 0x038b6300 },
	760	+ { _MMIO(0x9888), 0x058b6b62 },
	761	+ { _MMIO(0x9888), 0x078b006a },
	762	+ { _MMIO(0x9888), 0x118b0000 },
	763	+ { _MMIO(0x9888), 0x238b0000 },
	764	+ { _MMIO(0x9888), 0x258b0000 },
	765	+ { _MMIO(0x9888), 0x1f85a080 },
	766	+ { _MMIO(0x9888), 0x2185aaaa },
	767	+ { _MMIO(0x9888), 0x2385000a },
	768	+ { _MMIO(0x9888), 0x07834000 },
	769	+ { _MMIO(0x9888), 0x09834000 },
	770	+ { _MMIO(0x9888), 0x0b834000 },
	771	+ { _MMIO(0x9888), 0x0d834000 },
	772	+ { _MMIO(0x9888), 0x01840018 },
	773	+ { _MMIO(0x9888), 0x07844c80 },
	774	+ { _MMIO(0x9888), 0x09840d9a },
	775	+ { _MMIO(0x9888), 0x0b840e9c },
	776	+ { _MMIO(0x9888), 0x0d840f9e },
	777	+ { _MMIO(0x9888), 0x0f840010 },
	778	+ { _MMIO(0x9888), 0x11840000 },
	779	+ { _MMIO(0x9888), 0x03848000 },
	780	+ { _MMIO(0x9888), 0x0584c000 },
	781	+ { _MMIO(0x9888), 0x2f8000e5 },
	782	+ { _MMIO(0x9888), 0x138080e3 },
	783	+ { _MMIO(0x9888), 0x1580c0e1 },
	784	+ { _MMIO(0x9888), 0x21800000 },
	785	+ { _MMIO(0x9888), 0x11804000 },
	786	+ { _MMIO(0x9888), 0x1780c000 },
	787	+ { _MMIO(0x9888), 0x1980c000 },
	788	+ { _MMIO(0x9888), 0x1b80c000 },
	789	+ { _MMIO(0x9888), 0x1d80c000 },
	790	+ { _MMIO(0x9888), 0x1f804000 },
	791	+ { _MMIO(0xd24), 0x00000000 },
	792	+ { _MMIO(0x9888), 0x4d800000 },
	793	+ { _MMIO(0x9888), 0x3d800800 },
	794	+ { _MMIO(0x9888), 0x4f800000 },
	795	+ { _MMIO(0x9888), 0x43800842 },
	796	+ { _MMIO(0x9888), 0x51800000 },
	797	+ { _MMIO(0x9888), 0x45800842 },
	798	+ { _MMIO(0x9888), 0x53800000 },
	799	+ { _MMIO(0x9888), 0x47801042 },
	800	+ { _MMIO(0x9888), 0x31800000 },
	801	+ { _MMIO(0x9888), 0x3f800084 },
	802	+ { _MMIO(0x9888), 0x41800000 },
	803	+};
	804	+
	805	+static int
	806	+get_memory_reads_mux_config(struct drm_i915_private *dev_priv,
	807	+ const struct i915_oa_reg **regs,
	808	+ int *lens)
	809	+{
	810	+ int n = 0;
	811	+
	812	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	813	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	814	+
	815	+ regs[n] = mux_config_memory_reads;
	816	+ lens[n] = ARRAY_SIZE(mux_config_memory_reads);
	817	+ n++;
	818	+
	819	+ return n;
	820	+}
	821	+
	822	+static const struct i915_oa_reg b_counter_config_memory_writes[] = {
	823	+ { _MMIO(0x2724), 0xf0800000 },
	824	+ { _MMIO(0x2720), 0x00000000 },
	825	+ { _MMIO(0x2714), 0xf0800000 },
	826	+ { _MMIO(0x2710), 0x00000000 },
	827	+ { _MMIO(0x274c), 0x86543210 },
	828	+ { _MMIO(0x2748), 0x86543210 },
	829	+ { _MMIO(0x2744), 0x00006667 },
	830	+ { _MMIO(0x2740), 0x00000000 },
	831	+ { _MMIO(0x275c), 0x86543210 },
	832	+ { _MMIO(0x2758), 0x86543210 },
	833	+ { _MMIO(0x2754), 0x00006465 },
	834	+ { _MMIO(0x2750), 0x00000000 },
	835	+ { _MMIO(0x2770), 0x0007f81a },
	836	+ { _MMIO(0x2774), 0x0000fe00 },
	837	+ { _MMIO(0x2778), 0x0007f82a },
	838	+ { _MMIO(0x277c), 0x0000fe00 },
	839	+ { _MMIO(0x2780), 0x0007f822 },
	840	+ { _MMIO(0x2784), 0x0000fe00 },
	841	+ { _MMIO(0x2788), 0x0007f8ba },
	842	+ { _MMIO(0x278c), 0x0000fe00 },
	843	+ { _MMIO(0x2790), 0x0007f87a },
	844	+ { _MMIO(0x2794), 0x0000fe00 },
	845	+ { _MMIO(0x2798), 0x0007f8ea },
	846	+ { _MMIO(0x279c), 0x0000fe00 },
	847	+ { _MMIO(0x27a0), 0x0007f8e2 },
	848	+ { _MMIO(0x27a4), 0x0000fe00 },
	849	+ { _MMIO(0x27a8), 0x0007f8f2 },
	850	+ { _MMIO(0x27ac), 0x0000fe00 },
	851	+};
	852	+
	853	+static const struct i915_oa_reg flex_eu_config_memory_writes[] = {
	854	+ { _MMIO(0xe458), 0x00005004 },
	855	+ { _MMIO(0xe558), 0x00015014 },
	856	+ { _MMIO(0xe658), 0x00025024 },
	857	+ { _MMIO(0xe758), 0x00035034 },
	858	+ { _MMIO(0xe45c), 0x00045044 },
	859	+ { _MMIO(0xe55c), 0x00055054 },
	860	+ { _MMIO(0xe65c), 0x00065064 },
	861	+};
	862	+
	863	+static const struct i915_oa_reg mux_config_memory_writes[] = {
	864	+ { _MMIO(0x9888), 0x198b0343 },
	865	+ { _MMIO(0x9888), 0x13845400 },
	866	+ { _MMIO(0x9888), 0x3580001a },
	867	+ { _MMIO(0x9888), 0x3d800805 },
	868	+ { _MMIO(0x9888), 0x038b6300 },
	869	+ { _MMIO(0x9888), 0x058b6b62 },
	870	+ { _MMIO(0x9888), 0x078b006a },
	871	+ { _MMIO(0x9888), 0x118b0000 },
	872	+ { _MMIO(0x9888), 0x238b0000 },
	873	+ { _MMIO(0x9888), 0x258b0000 },
	874	+ { _MMIO(0x9888), 0x1f85a080 },
	875	+ { _MMIO(0x9888), 0x2185aaaa },
	876	+ { _MMIO(0x9888), 0x23850002 },
	877	+ { _MMIO(0x9888), 0x07834000 },
	878	+ { _MMIO(0x9888), 0x09834000 },
	879	+ { _MMIO(0x9888), 0x0b834000 },
	880	+ { _MMIO(0x9888), 0x0d834000 },
	881	+ { _MMIO(0x9888), 0x01840010 },
	882	+ { _MMIO(0x9888), 0x07844880 },
	883	+ { _MMIO(0x9888), 0x09840992 },
	884	+ { _MMIO(0x9888), 0x0b840a94 },
	885	+ { _MMIO(0x9888), 0x0d840b96 },
	886	+ { _MMIO(0x9888), 0x11840000 },
	887	+ { _MMIO(0x9888), 0x03848000 },
	888	+ { _MMIO(0x9888), 0x0584c000 },
	889	+ { _MMIO(0x9888), 0x2d800147 },
	890	+ { _MMIO(0x9888), 0x2f8000e5 },
	891	+ { _MMIO(0x9888), 0x138080e3 },
	892	+ { _MMIO(0x9888), 0x1580c0e1 },
	893	+ { _MMIO(0x9888), 0x21800000 },
	894	+ { _MMIO(0x9888), 0x11804000 },
	895	+ { _MMIO(0x9888), 0x1780c000 },
	896	+ { _MMIO(0x9888), 0x1980c000 },
	897	+ { _MMIO(0x9888), 0x1b80c000 },
	898	+ { _MMIO(0x9888), 0x1d80c000 },
	899	+ { _MMIO(0x9888), 0x1f800000 },
	900	+ { _MMIO(0xd24), 0x00000000 },
	901	+ { _MMIO(0x9888), 0x4d800000 },
	902	+ { _MMIO(0x9888), 0x4f800000 },
	903	+ { _MMIO(0x9888), 0x43800842 },
	904	+ { _MMIO(0x9888), 0x51800000 },
	905	+ { _MMIO(0x9888), 0x45800842 },
	906	+ { _MMIO(0x9888), 0x53800000 },
	907	+ { _MMIO(0x9888), 0x47801082 },
	908	+ { _MMIO(0x9888), 0x31800000 },
	909	+ { _MMIO(0x9888), 0x3f800084 },
	910	+ { _MMIO(0x9888), 0x41800000 },
	911	+};
	912	+
	913	+static int
	914	+get_memory_writes_mux_config(struct drm_i915_private *dev_priv,
	915	+ const struct i915_oa_reg **regs,
	916	+ int *lens)
	917	+{
	918	+ int n = 0;
	919	+
	920	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	921	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	922	+
	923	+ regs[n] = mux_config_memory_writes;
	924	+ lens[n] = ARRAY_SIZE(mux_config_memory_writes);
	925	+ n++;
	926	+
	927	+ return n;
	928	+}
	929	+
	930	+static const struct i915_oa_reg b_counter_config_compute_extended[] = {
	931	+ { _MMIO(0x2724), 0xf0800000 },
	932	+ { _MMIO(0x2720), 0x00000000 },
	933	+ { _MMIO(0x2714), 0xf0800000 },
	934	+ { _MMIO(0x2710), 0x00000000 },
	935	+ { _MMIO(0x2740), 0x00000000 },
	936	+ { _MMIO(0x2770), 0x0007fc2a },
	937	+ { _MMIO(0x2774), 0x0000bf00 },
	938	+ { _MMIO(0x2778), 0x0007fc6a },
	939	+ { _MMIO(0x277c), 0x0000bf00 },
	940	+ { _MMIO(0x2780), 0x0007fc92 },
	941	+ { _MMIO(0x2784), 0x0000bf00 },
	942	+ { _MMIO(0x2788), 0x0007fca2 },
	943	+ { _MMIO(0x278c), 0x0000bf00 },
	944	+ { _MMIO(0x2790), 0x0007fc32 },
	945	+ { _MMIO(0x2794), 0x0000bf00 },
	946	+ { _MMIO(0x2798), 0x0007fc9a },
	947	+ { _MMIO(0x279c), 0x0000bf00 },
	948	+ { _MMIO(0x27a0), 0x0007fe6a },
	949	+ { _MMIO(0x27a4), 0x0000bf00 },
	950	+ { _MMIO(0x27a8), 0x0007fe7a },
	951	+ { _MMIO(0x27ac), 0x0000bf00 },
	952	+};
	953	+
	954	+static const struct i915_oa_reg flex_eu_config_compute_extended[] = {
	955	+ { _MMIO(0xe458), 0x00005004 },
	956	+ { _MMIO(0xe558), 0x00000003 },
	957	+ { _MMIO(0xe658), 0x00002001 },
	958	+ { _MMIO(0xe758), 0x00778008 },
	959	+ { _MMIO(0xe45c), 0x00088078 },
	960	+ { _MMIO(0xe55c), 0x00808708 },
	961	+ { _MMIO(0xe65c), 0x00a08908 },
	962	+};
	963	+
	964	+static const struct i915_oa_reg mux_config_compute_extended_0_subslices_0x01[] = {
	965	+ { _MMIO(0x9888), 0x143d0160 },
	966	+ { _MMIO(0x9888), 0x163d2800 },
	967	+ { _MMIO(0x9888), 0x183d0120 },
	968	+ { _MMIO(0x9888), 0x105800e0 },
	969	+ { _MMIO(0x9888), 0x005cc000 },
	970	+ { _MMIO(0x9888), 0x065c8000 },
	971	+ { _MMIO(0x9888), 0x085cc000 },
	972	+ { _MMIO(0x9888), 0x0a5cc000 },
	973	+ { _MMIO(0x9888), 0x0c5cc000 },
	974	+ { _MMIO(0x9888), 0x0e5cc000 },
	975	+ { _MMIO(0x9888), 0x025cc000 },
	976	+ { _MMIO(0x9888), 0x045cc000 },
	977	+ { _MMIO(0x9888), 0x003d0011 },
	978	+ { _MMIO(0x9888), 0x063d0900 },
	979	+ { _MMIO(0x9888), 0x083d0a13 },
	980	+ { _MMIO(0x9888), 0x0a3d0b15 },
	981	+ { _MMIO(0x9888), 0x0c3d2317 },
	982	+ { _MMIO(0x9888), 0x043d21b7 },
	983	+ { _MMIO(0x9888), 0x103d0000 },
	984	+ { _MMIO(0x9888), 0x0e3d0000 },
	985	+ { _MMIO(0x9888), 0x1a3d0000 },
	986	+ { _MMIO(0x9888), 0x0e5825c1 },
	987	+ { _MMIO(0x9888), 0x00586100 },
	988	+ { _MMIO(0x9888), 0x0258204c },
	989	+ { _MMIO(0x9888), 0x06588000 },
	990	+ { _MMIO(0x9888), 0x0858c000 },
	991	+ { _MMIO(0x9888), 0x0a58c000 },
	992	+ { _MMIO(0x9888), 0x0c58c000 },
	993	+ { _MMIO(0x9888), 0x0458c000 },
	994	+ { _MMIO(0x9888), 0x005b4000 },
	995	+ { _MMIO(0x9888), 0x0e5b4000 },
	996	+ { _MMIO(0x9888), 0x185b5400 },
	997	+ { _MMIO(0x9888), 0x1a5b0155 },
	998	+ { _MMIO(0x9888), 0x025b4000 },
	999	+ { _MMIO(0x9888), 0x045b4000 },
	1000	+ { _MMIO(0x9888), 0x065b4000 },
	1001	+ { _MMIO(0x9888), 0x085b4000 },
	1002	+ { _MMIO(0x9888), 0x0a5b4000 },
	1003	+ { _MMIO(0x9888), 0x0c1fa800 },
	1004	+ { _MMIO(0x9888), 0x0e1faa2a },
	1005	+ { _MMIO(0x9888), 0x101f02aa },
	1006	+ { _MMIO(0x9888), 0x00384000 },
	1007	+ { _MMIO(0x9888), 0x0e384000 },
	1008	+ { _MMIO(0x9888), 0x16384000 },
	1009	+ { _MMIO(0x9888), 0x18381555 },
	1010	+ { _MMIO(0x9888), 0x02384000 },
	1011	+ { _MMIO(0x9888), 0x04384000 },
	1012	+ { _MMIO(0x9888), 0x06384000 },
	1013	+ { _MMIO(0x9888), 0x08384000 },
	1014	+ { _MMIO(0x9888), 0x0a384000 },
	1015	+ { _MMIO(0x9888), 0x0039a000 },
	1016	+ { _MMIO(0x9888), 0x06398000 },
	1017	+ { _MMIO(0x9888), 0x0839a000 },
	1018	+ { _MMIO(0x9888), 0x0a39a000 },
	1019	+ { _MMIO(0x9888), 0x0c39a000 },
	1020	+ { _MMIO(0x9888), 0x0e39a000 },
	1021	+ { _MMIO(0x9888), 0x0239a000 },
	1022	+ { _MMIO(0x9888), 0x0439a000 },
	1023	+ { _MMIO(0x9888), 0x018a8000 },
	1024	+ { _MMIO(0x9888), 0x0f8a8000 },
	1025	+ { _MMIO(0x9888), 0x198a8000 },
	1026	+ { _MMIO(0x9888), 0x1b8aaaa0 },
	1027	+ { _MMIO(0x9888), 0x1d8a0002 },
	1028	+ { _MMIO(0x9888), 0x038a8000 },
	1029	+ { _MMIO(0x9888), 0x058a8000 },
	1030	+ { _MMIO(0x9888), 0x078a8000 },
	1031	+ { _MMIO(0x9888), 0x098a8000 },
	1032	+ { _MMIO(0x9888), 0x0b8a8000 },
	1033	+ { _MMIO(0x9888), 0x238b2aa0 },
	1034	+ { _MMIO(0x9888), 0x258b5551 },
	1035	+ { _MMIO(0x9888), 0x278b0015 },
	1036	+ { _MMIO(0x9888), 0x1f85aa80 },
	1037	+ { _MMIO(0x9888), 0x2185aaa2 },
	1038	+ { _MMIO(0x9888), 0x2385002a },
	1039	+ { _MMIO(0x9888), 0x01834000 },
	1040	+ { _MMIO(0x9888), 0x0f834000 },
	1041	+ { _MMIO(0x9888), 0x19835400 },
	1042	+ { _MMIO(0x9888), 0x1b830155 },
	1043	+ { _MMIO(0x9888), 0x03834000 },
	1044	+ { _MMIO(0x9888), 0x05834000 },
	1045	+ { _MMIO(0x9888), 0x07834000 },
	1046	+ { _MMIO(0x9888), 0x09834000 },
	1047	+ { _MMIO(0x9888), 0x0b834000 },
	1048	+ { _MMIO(0x9888), 0x0184c000 },
	1049	+ { _MMIO(0x9888), 0x07848000 },
	1050	+ { _MMIO(0x9888), 0x0984c000 },
	1051	+ { _MMIO(0x9888), 0x0b84c000 },
	1052	+ { _MMIO(0x9888), 0x0d84c000 },
	1053	+ { _MMIO(0x9888), 0x0f84c000 },
	1054	+ { _MMIO(0x9888), 0x0384c000 },
	1055	+ { _MMIO(0x9888), 0x0584c000 },
	1056	+ { _MMIO(0x9888), 0x1180c000 },
	1057	+ { _MMIO(0x9888), 0x17808000 },
	1058	+ { _MMIO(0x9888), 0x1980c000 },
	1059	+ { _MMIO(0x9888), 0x1b80c000 },
	1060	+ { _MMIO(0x9888), 0x1d80c000 },
	1061	+ { _MMIO(0x9888), 0x1f80c000 },
	1062	+ { _MMIO(0x9888), 0x1380c000 },
	1063	+ { _MMIO(0x9888), 0x1580c000 },
	1064	+ { _MMIO(0xd24), 0x00000000 },
	1065	+ { _MMIO(0x9888), 0x4d800000 },
	1066	+ { _MMIO(0x9888), 0x3d800000 },
	1067	+ { _MMIO(0x9888), 0x4f800000 },
	1068	+ { _MMIO(0x9888), 0x43800000 },
	1069	+ { _MMIO(0x9888), 0x51800000 },
	1070	+ { _MMIO(0x9888), 0x45800000 },
	1071	+ { _MMIO(0x9888), 0x53800000 },
	1072	+ { _MMIO(0x9888), 0x47800420 },
	1073	+ { _MMIO(0x9888), 0x21800000 },
	1074	+ { _MMIO(0x9888), 0x31800000 },
	1075	+ { _MMIO(0x9888), 0x3f800421 },
	1076	+ { _MMIO(0x9888), 0x41800000 },
	1077	+};
	1078	+
	1079	+static const struct i915_oa_reg mux_config_compute_extended_2_subslices_0x02[] = {
	1080	+ { _MMIO(0x9888), 0x105c00e0 },
	1081	+ { _MMIO(0x9888), 0x145b0160 },
	1082	+ { _MMIO(0x9888), 0x165b2800 },
	1083	+ { _MMIO(0x9888), 0x185b0120 },
	1084	+ { _MMIO(0x9888), 0x0e5c25c1 },
	1085	+ { _MMIO(0x9888), 0x005c6100 },
	1086	+ { _MMIO(0x9888), 0x025c204c },
	1087	+ { _MMIO(0x9888), 0x065c8000 },
	1088	+ { _MMIO(0x9888), 0x085cc000 },
	1089	+ { _MMIO(0x9888), 0x0a5cc000 },
	1090	+ { _MMIO(0x9888), 0x0c5cc000 },
	1091	+ { _MMIO(0x9888), 0x045cc000 },
	1092	+ { _MMIO(0x9888), 0x005b0011 },
	1093	+ { _MMIO(0x9888), 0x065b0900 },
	1094	+ { _MMIO(0x9888), 0x085b0a13 },
	1095	+ { _MMIO(0x9888), 0x0a5b0b15 },
	1096	+ { _MMIO(0x9888), 0x0c5b2317 },
	1097	+ { _MMIO(0x9888), 0x045b21b7 },
	1098	+ { _MMIO(0x9888), 0x105b0000 },
	1099	+ { _MMIO(0x9888), 0x0e5b0000 },
	1100	+ { _MMIO(0x9888), 0x1a5b0000 },
	1101	+ { _MMIO(0x9888), 0x0c1fa800 },
	1102	+ { _MMIO(0x9888), 0x0e1faa2a },
	1103	+ { _MMIO(0x9888), 0x101f02aa },
	1104	+ { _MMIO(0x9888), 0x00384000 },
	1105	+ { _MMIO(0x9888), 0x0e384000 },
	1106	+ { _MMIO(0x9888), 0x16384000 },
	1107	+ { _MMIO(0x9888), 0x18381555 },
	1108	+ { _MMIO(0x9888), 0x02384000 },
	1109	+ { _MMIO(0x9888), 0x04384000 },
	1110	+ { _MMIO(0x9888), 0x06384000 },
	1111	+ { _MMIO(0x9888), 0x08384000 },
	1112	+ { _MMIO(0x9888), 0x0a384000 },
	1113	+ { _MMIO(0x9888), 0x0039a000 },
	1114	+ { _MMIO(0x9888), 0x06398000 },
	1115	+ { _MMIO(0x9888), 0x0839a000 },
	1116	+ { _MMIO(0x9888), 0x0a39a000 },
	1117	+ { _MMIO(0x9888), 0x0c39a000 },
	1118	+ { _MMIO(0x9888), 0x0e39a000 },
	1119	+ { _MMIO(0x9888), 0x0239a000 },
	1120	+ { _MMIO(0x9888), 0x0439a000 },
	1121	+ { _MMIO(0x9888), 0x018a8000 },
	1122	+ { _MMIO(0x9888), 0x0f8a8000 },
	1123	+ { _MMIO(0x9888), 0x198a8000 },
	1124	+ { _MMIO(0x9888), 0x1b8aaaa0 },
	1125	+ { _MMIO(0x9888), 0x1d8a0002 },
	1126	+ { _MMIO(0x9888), 0x038a8000 },
	1127	+ { _MMIO(0x9888), 0x058a8000 },
	1128	+ { _MMIO(0x9888), 0x078a8000 },
	1129	+ { _MMIO(0x9888), 0x098a8000 },
	1130	+ { _MMIO(0x9888), 0x0b8a8000 },
	1131	+ { _MMIO(0x9888), 0x238b2aa0 },
	1132	+ { _MMIO(0x9888), 0x258b5551 },
	1133	+ { _MMIO(0x9888), 0x278b0015 },
	1134	+ { _MMIO(0x9888), 0x1f85aa80 },
	1135	+ { _MMIO(0x9888), 0x2185aaa2 },
	1136	+ { _MMIO(0x9888), 0x2385002a },
	1137	+ { _MMIO(0x9888), 0x01834000 },
	1138	+ { _MMIO(0x9888), 0x0f834000 },
	1139	+ { _MMIO(0x9888), 0x19835400 },
	1140	+ { _MMIO(0x9888), 0x1b830155 },
	1141	+ { _MMIO(0x9888), 0x03834000 },
	1142	+ { _MMIO(0x9888), 0x05834000 },
	1143	+ { _MMIO(0x9888), 0x07834000 },
	1144	+ { _MMIO(0x9888), 0x09834000 },
	1145	+ { _MMIO(0x9888), 0x0b834000 },
	1146	+ { _MMIO(0x9888), 0x0184c000 },
	1147	+ { _MMIO(0x9888), 0x07848000 },
	1148	+ { _MMIO(0x9888), 0x0984c000 },
	1149	+ { _MMIO(0x9888), 0x0b84c000 },
	1150	+ { _MMIO(0x9888), 0x0d84c000 },
	1151	+ { _MMIO(0x9888), 0x0f84c000 },
	1152	+ { _MMIO(0x9888), 0x0384c000 },
	1153	+ { _MMIO(0x9888), 0x0584c000 },
	1154	+ { _MMIO(0x9888), 0x1180c000 },
	1155	+ { _MMIO(0x9888), 0x17808000 },
	1156	+ { _MMIO(0x9888), 0x1980c000 },
	1157	+ { _MMIO(0x9888), 0x1b80c000 },
	1158	+ { _MMIO(0x9888), 0x1d80c000 },
	1159	+ { _MMIO(0x9888), 0x1f80c000 },
	1160	+ { _MMIO(0x9888), 0x1380c000 },
	1161	+ { _MMIO(0x9888), 0x1580c000 },
	1162	+ { _MMIO(0xd24), 0x00000000 },
	1163	+ { _MMIO(0x9888), 0x4d800000 },
	1164	+ { _MMIO(0x9888), 0x3d800000 },
	1165	+ { _MMIO(0x9888), 0x4f800000 },
	1166	+ { _MMIO(0x9888), 0x43800000 },
	1167	+ { _MMIO(0x9888), 0x51800000 },
	1168	+ { _MMIO(0x9888), 0x45800000 },
	1169	+ { _MMIO(0x9888), 0x53800000 },
	1170	+ { _MMIO(0x9888), 0x47800420 },
	1171	+ { _MMIO(0x9888), 0x21800000 },
	1172	+ { _MMIO(0x9888), 0x31800000 },
	1173	+ { _MMIO(0x9888), 0x3f800421 },
	1174	+ { _MMIO(0x9888), 0x41800000 },
	1175	+};
	1176	+
	1177	+static const struct i915_oa_reg mux_config_compute_extended_4_subslices_0x04[] = {
	1178	+ { _MMIO(0x9888), 0x103800e0 },
	1179	+ { _MMIO(0x9888), 0x143a0160 },
	1180	+ { _MMIO(0x9888), 0x163a2800 },
	1181	+ { _MMIO(0x9888), 0x183a0120 },
	1182	+ { _MMIO(0x9888), 0x0c1fa800 },
	1183	+ { _MMIO(0x9888), 0x0e1faa2a },
	1184	+ { _MMIO(0x9888), 0x101f02aa },
	1185	+ { _MMIO(0x9888), 0x0e38a5c1 },
	1186	+ { _MMIO(0x9888), 0x0038a100 },
	1187	+ { _MMIO(0x9888), 0x0238204c },
	1188	+ { _MMIO(0x9888), 0x16388000 },
	1189	+ { _MMIO(0x9888), 0x183802aa },
	1190	+ { _MMIO(0x9888), 0x04380000 },
	1191	+ { _MMIO(0x9888), 0x06380000 },
	1192	+ { _MMIO(0x9888), 0x08388000 },
	1193	+ { _MMIO(0x9888), 0x0a388000 },
	1194	+ { _MMIO(0x9888), 0x0039a000 },
	1195	+ { _MMIO(0x9888), 0x06398000 },
	1196	+ { _MMIO(0x9888), 0x0839a000 },
	1197	+ { _MMIO(0x9888), 0x0a39a000 },
	1198	+ { _MMIO(0x9888), 0x0c39a000 },
	1199	+ { _MMIO(0x9888), 0x0e39a000 },
	1200	+ { _MMIO(0x9888), 0x0239a000 },
	1201	+ { _MMIO(0x9888), 0x0439a000 },
	1202	+ { _MMIO(0x9888), 0x003a0011 },
	1203	+ { _MMIO(0x9888), 0x063a0900 },
	1204	+ { _MMIO(0x9888), 0x083a0a13 },
	1205	+ { _MMIO(0x9888), 0x0a3a0b15 },
	1206	+ { _MMIO(0x9888), 0x0c3a2317 },
	1207	+ { _MMIO(0x9888), 0x043a21b7 },
	1208	+ { _MMIO(0x9888), 0x103a0000 },
	1209	+ { _MMIO(0x9888), 0x0e3a0000 },
	1210	+ { _MMIO(0x9888), 0x1a3a0000 },
	1211	+ { _MMIO(0x9888), 0x018a8000 },
	1212	+ { _MMIO(0x9888), 0x0f8a8000 },
	1213	+ { _MMIO(0x9888), 0x198a8000 },
	1214	+ { _MMIO(0x9888), 0x1b8aaaa0 },
	1215	+ { _MMIO(0x9888), 0x1d8a0002 },
	1216	+ { _MMIO(0x9888), 0x038a8000 },
	1217	+ { _MMIO(0x9888), 0x058a8000 },
	1218	+ { _MMIO(0x9888), 0x078a8000 },
	1219	+ { _MMIO(0x9888), 0x098a8000 },
	1220	+ { _MMIO(0x9888), 0x0b8a8000 },
	1221	+ { _MMIO(0x9888), 0x238b2aa0 },
	1222	+ { _MMIO(0x9888), 0x258b5551 },
	1223	+ { _MMIO(0x9888), 0x278b0015 },
	1224	+ { _MMIO(0x9888), 0x1f85aa80 },
	1225	+ { _MMIO(0x9888), 0x2185aaa2 },
	1226	+ { _MMIO(0x9888), 0x2385002a },
	1227	+ { _MMIO(0x9888), 0x01834000 },
	1228	+ { _MMIO(0x9888), 0x0f834000 },
	1229	+ { _MMIO(0x9888), 0x19835400 },
	1230	+ { _MMIO(0x9888), 0x1b830155 },
	1231	+ { _MMIO(0x9888), 0x03834000 },
	1232	+ { _MMIO(0x9888), 0x05834000 },
	1233	+ { _MMIO(0x9888), 0x07834000 },
	1234	+ { _MMIO(0x9888), 0x09834000 },
	1235	+ { _MMIO(0x9888), 0x0b834000 },
	1236	+ { _MMIO(0x9888), 0x0184c000 },
	1237	+ { _MMIO(0x9888), 0x07848000 },
	1238	+ { _MMIO(0x9888), 0x0984c000 },
	1239	+ { _MMIO(0x9888), 0x0b84c000 },
	1240	+ { _MMIO(0x9888), 0x0d84c000 },
	1241	+ { _MMIO(0x9888), 0x0f84c000 },
	1242	+ { _MMIO(0x9888), 0x0384c000 },
	1243	+ { _MMIO(0x9888), 0x0584c000 },
	1244	+ { _MMIO(0x9888), 0x1180c000 },
	1245	+ { _MMIO(0x9888), 0x17808000 },
	1246	+ { _MMIO(0x9888), 0x1980c000 },
	1247	+ { _MMIO(0x9888), 0x1b80c000 },
	1248	+ { _MMIO(0x9888), 0x1d80c000 },
	1249	+ { _MMIO(0x9888), 0x1f80c000 },
	1250	+ { _MMIO(0x9888), 0x1380c000 },
	1251	+ { _MMIO(0x9888), 0x1580c000 },
	1252	+ { _MMIO(0xd24), 0x00000000 },
	1253	+ { _MMIO(0x9888), 0x4d800000 },
	1254	+ { _MMIO(0x9888), 0x3d800000 },
	1255	+ { _MMIO(0x9888), 0x4f800000 },
	1256	+ { _MMIO(0x9888), 0x43800000 },
	1257	+ { _MMIO(0x9888), 0x51800000 },
	1258	+ { _MMIO(0x9888), 0x45800000 },
	1259	+ { _MMIO(0x9888), 0x53800000 },
	1260	+ { _MMIO(0x9888), 0x47800420 },
	1261	+ { _MMIO(0x9888), 0x21800000 },
	1262	+ { _MMIO(0x9888), 0x31800000 },
	1263	+ { _MMIO(0x9888), 0x3f800421 },
	1264	+ { _MMIO(0x9888), 0x41800000 },
	1265	+};
	1266	+
	1267	+static const struct i915_oa_reg mux_config_compute_extended_1_subslices_0x08[] = {
	1268	+ { _MMIO(0x9888), 0x14bd0160 },
	1269	+ { _MMIO(0x9888), 0x16bd2800 },
	1270	+ { _MMIO(0x9888), 0x18bd0120 },
	1271	+ { _MMIO(0x9888), 0x10d800e0 },
	1272	+ { _MMIO(0x9888), 0x00dcc000 },
	1273	+ { _MMIO(0x9888), 0x06dc8000 },
	1274	+ { _MMIO(0x9888), 0x08dcc000 },
	1275	+ { _MMIO(0x9888), 0x0adcc000 },
	1276	+ { _MMIO(0x9888), 0x0cdcc000 },
	1277	+ { _MMIO(0x9888), 0x0edcc000 },
	1278	+ { _MMIO(0x9888), 0x02dcc000 },
	1279	+ { _MMIO(0x9888), 0x04dcc000 },
	1280	+ { _MMIO(0x9888), 0x00bd0011 },
	1281	+ { _MMIO(0x9888), 0x06bd0900 },
	1282	+ { _MMIO(0x9888), 0x08bd0a13 },
	1283	+ { _MMIO(0x9888), 0x0abd0b15 },
	1284	+ { _MMIO(0x9888), 0x0cbd2317 },
	1285	+ { _MMIO(0x9888), 0x04bd21b7 },
	1286	+ { _MMIO(0x9888), 0x10bd0000 },
	1287	+ { _MMIO(0x9888), 0x0ebd0000 },
	1288	+ { _MMIO(0x9888), 0x1abd0000 },
	1289	+ { _MMIO(0x9888), 0x0ed825c1 },
	1290	+ { _MMIO(0x9888), 0x00d86100 },
	1291	+ { _MMIO(0x9888), 0x02d8204c },
	1292	+ { _MMIO(0x9888), 0x06d88000 },
	1293	+ { _MMIO(0x9888), 0x08d8c000 },
	1294	+ { _MMIO(0x9888), 0x0ad8c000 },
	1295	+ { _MMIO(0x9888), 0x0cd8c000 },
	1296	+ { _MMIO(0x9888), 0x04d8c000 },
	1297	+ { _MMIO(0x9888), 0x00db4000 },
	1298	+ { _MMIO(0x9888), 0x0edb4000 },
	1299	+ { _MMIO(0x9888), 0x18db5400 },
	1300	+ { _MMIO(0x9888), 0x1adb0155 },
	1301	+ { _MMIO(0x9888), 0x02db4000 },
	1302	+ { _MMIO(0x9888), 0x04db4000 },
	1303	+ { _MMIO(0x9888), 0x06db4000 },
	1304	+ { _MMIO(0x9888), 0x08db4000 },
	1305	+ { _MMIO(0x9888), 0x0adb4000 },
	1306	+ { _MMIO(0x9888), 0x0c9fa800 },
	1307	+ { _MMIO(0x9888), 0x0e9faa2a },
	1308	+ { _MMIO(0x9888), 0x109f02aa },
	1309	+ { _MMIO(0x9888), 0x00b84000 },
	1310	+ { _MMIO(0x9888), 0x0eb84000 },
	1311	+ { _MMIO(0x9888), 0x16b84000 },
	1312	+ { _MMIO(0x9888), 0x18b81555 },
	1313	+ { _MMIO(0x9888), 0x02b84000 },
	1314	+ { _MMIO(0x9888), 0x04b84000 },
	1315	+ { _MMIO(0x9888), 0x06b84000 },
	1316	+ { _MMIO(0x9888), 0x08b84000 },
	1317	+ { _MMIO(0x9888), 0x0ab84000 },
	1318	+ { _MMIO(0x9888), 0x00b9a000 },
	1319	+ { _MMIO(0x9888), 0x06b98000 },
	1320	+ { _MMIO(0x9888), 0x08b9a000 },
	1321	+ { _MMIO(0x9888), 0x0ab9a000 },
	1322	+ { _MMIO(0x9888), 0x0cb9a000 },
	1323	+ { _MMIO(0x9888), 0x0eb9a000 },
	1324	+ { _MMIO(0x9888), 0x02b9a000 },
	1325	+ { _MMIO(0x9888), 0x04b9a000 },
	1326	+ { _MMIO(0x9888), 0x01888000 },
	1327	+ { _MMIO(0x9888), 0x0d88f800 },
	1328	+ { _MMIO(0x9888), 0x0f88000f },
	1329	+ { _MMIO(0x9888), 0x03888000 },
	1330	+ { _MMIO(0x9888), 0x05888000 },
	1331	+ { _MMIO(0x9888), 0x07888000 },
	1332	+ { _MMIO(0x9888), 0x09888000 },
	1333	+ { _MMIO(0x9888), 0x0b888000 },
	1334	+ { _MMIO(0x9888), 0x238b5540 },
	1335	+ { _MMIO(0x9888), 0x258baaa2 },
	1336	+ { _MMIO(0x9888), 0x278b002a },
	1337	+ { _MMIO(0x9888), 0x018c4000 },
	1338	+ { _MMIO(0x9888), 0x0f8c4000 },
	1339	+ { _MMIO(0x9888), 0x178c2000 },
	1340	+ { _MMIO(0x9888), 0x198c5500 },
	1341	+ { _MMIO(0x9888), 0x1b8c0015 },
	1342	+ { _MMIO(0x9888), 0x038c4000 },
	1343	+ { _MMIO(0x9888), 0x058c4000 },
	1344	+ { _MMIO(0x9888), 0x078c4000 },
	1345	+ { _MMIO(0x9888), 0x098c4000 },
	1346	+ { _MMIO(0x9888), 0x0b8c4000 },
	1347	+ { _MMIO(0x9888), 0x018da000 },
	1348	+ { _MMIO(0x9888), 0x078d8000 },
	1349	+ { _MMIO(0x9888), 0x098da000 },
	1350	+ { _MMIO(0x9888), 0x0b8da000 },
	1351	+ { _MMIO(0x9888), 0x0d8da000 },
	1352	+ { _MMIO(0x9888), 0x0f8da000 },
	1353	+ { _MMIO(0x9888), 0x038da000 },
	1354	+ { _MMIO(0x9888), 0x058da000 },
	1355	+ { _MMIO(0x9888), 0x1f85aa80 },
	1356	+ { _MMIO(0x9888), 0x2185aaa2 },
	1357	+ { _MMIO(0x9888), 0x2385002a },
	1358	+ { _MMIO(0x9888), 0x01834000 },
	1359	+ { _MMIO(0x9888), 0x0f834000 },
	1360	+ { _MMIO(0x9888), 0x19835400 },
	1361	+ { _MMIO(0x9888), 0x1b830155 },
	1362	+ { _MMIO(0x9888), 0x03834000 },
	1363	+ { _MMIO(0x9888), 0x05834000 },
	1364	+ { _MMIO(0x9888), 0x07834000 },
	1365	+ { _MMIO(0x9888), 0x09834000 },
	1366	+ { _MMIO(0x9888), 0x0b834000 },
	1367	+ { _MMIO(0x9888), 0x0184c000 },
	1368	+ { _MMIO(0x9888), 0x07848000 },
	1369	+ { _MMIO(0x9888), 0x0984c000 },
	1370	+ { _MMIO(0x9888), 0x0b84c000 },
	1371	+ { _MMIO(0x9888), 0x0d84c000 },
	1372	+ { _MMIO(0x9888), 0x0f84c000 },
	1373	+ { _MMIO(0x9888), 0x0384c000 },
	1374	+ { _MMIO(0x9888), 0x0584c000 },
	1375	+ { _MMIO(0x9888), 0x1180c000 },
	1376	+ { _MMIO(0x9888), 0x17808000 },
	1377	+ { _MMIO(0x9888), 0x1980c000 },
	1378	+ { _MMIO(0x9888), 0x1b80c000 },
	1379	+ { _MMIO(0x9888), 0x1d80c000 },
	1380	+ { _MMIO(0x9888), 0x1f80c000 },
	1381	+ { _MMIO(0x9888), 0x1380c000 },
	1382	+ { _MMIO(0x9888), 0x1580c000 },
	1383	+ { _MMIO(0xd24), 0x00000000 },
	1384	+ { _MMIO(0x9888), 0x4d800000 },
	1385	+ { _MMIO(0x9888), 0x3d800000 },
	1386	+ { _MMIO(0x9888), 0x4f800000 },
	1387	+ { _MMIO(0x9888), 0x43800000 },
	1388	+ { _MMIO(0x9888), 0x51800000 },
	1389	+ { _MMIO(0x9888), 0x45800000 },
	1390	+ { _MMIO(0x9888), 0x53800000 },
	1391	+ { _MMIO(0x9888), 0x47800420 },
	1392	+ { _MMIO(0x9888), 0x21800000 },
	1393	+ { _MMIO(0x9888), 0x31800000 },
	1394	+ { _MMIO(0x9888), 0x3f800421 },
	1395	+ { _MMIO(0x9888), 0x41800000 },
	1396	+};
	1397	+
	1398	+static const struct i915_oa_reg mux_config_compute_extended_3_subslices_0x10[] = {
	1399	+ { _MMIO(0x9888), 0x10dc00e0 },
	1400	+ { _MMIO(0x9888), 0x14db0160 },
	1401	+ { _MMIO(0x9888), 0x16db2800 },
	1402	+ { _MMIO(0x9888), 0x18db0120 },
	1403	+ { _MMIO(0x9888), 0x0edc25c1 },
	1404	+ { _MMIO(0x9888), 0x00dc6100 },
	1405	+ { _MMIO(0x9888), 0x02dc204c },
	1406	+ { _MMIO(0x9888), 0x06dc8000 },
	1407	+ { _MMIO(0x9888), 0x08dcc000 },
	1408	+ { _MMIO(0x9888), 0x0adcc000 },
	1409	+ { _MMIO(0x9888), 0x0cdcc000 },
	1410	+ { _MMIO(0x9888), 0x04dcc000 },
	1411	+ { _MMIO(0x9888), 0x00db0011 },
	1412	+ { _MMIO(0x9888), 0x06db0900 },
	1413	+ { _MMIO(0x9888), 0x08db0a13 },
	1414	+ { _MMIO(0x9888), 0x0adb0b15 },
	1415	+ { _MMIO(0x9888), 0x0cdb2317 },
	1416	+ { _MMIO(0x9888), 0x04db21b7 },
	1417	+ { _MMIO(0x9888), 0x10db0000 },
	1418	+ { _MMIO(0x9888), 0x0edb0000 },
	1419	+ { _MMIO(0x9888), 0x1adb0000 },
	1420	+ { _MMIO(0x9888), 0x0c9fa800 },
	1421	+ { _MMIO(0x9888), 0x0e9faa2a },
	1422	+ { _MMIO(0x9888), 0x109f02aa },
	1423	+ { _MMIO(0x9888), 0x00b84000 },
	1424	+ { _MMIO(0x9888), 0x0eb84000 },
	1425	+ { _MMIO(0x9888), 0x16b84000 },
	1426	+ { _MMIO(0x9888), 0x18b81555 },
	1427	+ { _MMIO(0x9888), 0x02b84000 },
	1428	+ { _MMIO(0x9888), 0x04b84000 },
	1429	+ { _MMIO(0x9888), 0x06b84000 },
	1430	+ { _MMIO(0x9888), 0x08b84000 },
	1431	+ { _MMIO(0x9888), 0x0ab84000 },
	1432	+ { _MMIO(0x9888), 0x00b9a000 },
	1433	+ { _MMIO(0x9888), 0x06b98000 },
	1434	+ { _MMIO(0x9888), 0x08b9a000 },
	1435	+ { _MMIO(0x9888), 0x0ab9a000 },
	1436	+ { _MMIO(0x9888), 0x0cb9a000 },
	1437	+ { _MMIO(0x9888), 0x0eb9a000 },
	1438	+ { _MMIO(0x9888), 0x02b9a000 },
	1439	+ { _MMIO(0x9888), 0x04b9a000 },
	1440	+ { _MMIO(0x9888), 0x01888000 },
	1441	+ { _MMIO(0x9888), 0x0d88f800 },
	1442	+ { _MMIO(0x9888), 0x0f88000f },
	1443	+ { _MMIO(0x9888), 0x03888000 },
	1444	+ { _MMIO(0x9888), 0x05888000 },
	1445	+ { _MMIO(0x9888), 0x07888000 },
	1446	+ { _MMIO(0x9888), 0x09888000 },
	1447	+ { _MMIO(0x9888), 0x0b888000 },
	1448	+ { _MMIO(0x9888), 0x238b5540 },
	1449	+ { _MMIO(0x9888), 0x258baaa2 },
	1450	+ { _MMIO(0x9888), 0x278b002a },
	1451	+ { _MMIO(0x9888), 0x018c4000 },
	1452	+ { _MMIO(0x9888), 0x0f8c4000 },
	1453	+ { _MMIO(0x9888), 0x178c2000 },
	1454	+ { _MMIO(0x9888), 0x198c5500 },
	1455	+ { _MMIO(0x9888), 0x1b8c0015 },
	1456	+ { _MMIO(0x9888), 0x038c4000 },
	1457	+ { _MMIO(0x9888), 0x058c4000 },
	1458	+ { _MMIO(0x9888), 0x078c4000 },
	1459	+ { _MMIO(0x9888), 0x098c4000 },
	1460	+ { _MMIO(0x9888), 0x0b8c4000 },
	1461	+ { _MMIO(0x9888), 0x018da000 },
	1462	+ { _MMIO(0x9888), 0x078d8000 },
	1463	+ { _MMIO(0x9888), 0x098da000 },
	1464	+ { _MMIO(0x9888), 0x0b8da000 },
	1465	+ { _MMIO(0x9888), 0x0d8da000 },
	1466	+ { _MMIO(0x9888), 0x0f8da000 },
	1467	+ { _MMIO(0x9888), 0x038da000 },
	1468	+ { _MMIO(0x9888), 0x058da000 },
	1469	+ { _MMIO(0x9888), 0x1f85aa80 },
	1470	+ { _MMIO(0x9888), 0x2185aaa2 },
	1471	+ { _MMIO(0x9888), 0x2385002a },
	1472	+ { _MMIO(0x9888), 0x01834000 },
	1473	+ { _MMIO(0x9888), 0x0f834000 },
	1474	+ { _MMIO(0x9888), 0x19835400 },
	1475	+ { _MMIO(0x9888), 0x1b830155 },
	1476	+ { _MMIO(0x9888), 0x03834000 },
	1477	+ { _MMIO(0x9888), 0x05834000 },
	1478	+ { _MMIO(0x9888), 0x07834000 },
	1479	+ { _MMIO(0x9888), 0x09834000 },
	1480	+ { _MMIO(0x9888), 0x0b834000 },
	1481	+ { _MMIO(0x9888), 0x0184c000 },
	1482	+ { _MMIO(0x9888), 0x07848000 },
	1483	+ { _MMIO(0x9888), 0x0984c000 },
	1484	+ { _MMIO(0x9888), 0x0b84c000 },
	1485	+ { _MMIO(0x9888), 0x0d84c000 },
	1486	+ { _MMIO(0x9888), 0x0f84c000 },
	1487	+ { _MMIO(0x9888), 0x0384c000 },
	1488	+ { _MMIO(0x9888), 0x0584c000 },
	1489	+ { _MMIO(0x9888), 0x1180c000 },
	1490	+ { _MMIO(0x9888), 0x17808000 },
	1491	+ { _MMIO(0x9888), 0x1980c000 },
	1492	+ { _MMIO(0x9888), 0x1b80c000 },
	1493	+ { _MMIO(0x9888), 0x1d80c000 },
	1494	+ { _MMIO(0x9888), 0x1f80c000 },
	1495	+ { _MMIO(0x9888), 0x1380c000 },
	1496	+ { _MMIO(0x9888), 0x1580c000 },
	1497	+ { _MMIO(0xd24), 0x00000000 },
	1498	+ { _MMIO(0x9888), 0x4d800000 },
	1499	+ { _MMIO(0x9888), 0x3d800000 },
	1500	+ { _MMIO(0x9888), 0x4f800000 },
	1501	+ { _MMIO(0x9888), 0x43800000 },
	1502	+ { _MMIO(0x9888), 0x51800000 },
	1503	+ { _MMIO(0x9888), 0x45800000 },
	1504	+ { _MMIO(0x9888), 0x53800000 },
	1505	+ { _MMIO(0x9888), 0x47800420 },
	1506	+ { _MMIO(0x9888), 0x21800000 },
	1507	+ { _MMIO(0x9888), 0x31800000 },
	1508	+ { _MMIO(0x9888), 0x3f800421 },
	1509	+ { _MMIO(0x9888), 0x41800000 },
	1510	+};
	1511	+
	1512	+static const struct i915_oa_reg mux_config_compute_extended_5_subslices_0x20[] = {
	1513	+ { _MMIO(0x9888), 0x10b800e0 },
	1514	+ { _MMIO(0x9888), 0x14ba0160 },
	1515	+ { _MMIO(0x9888), 0x16ba2800 },
	1516	+ { _MMIO(0x9888), 0x18ba0120 },
	1517	+ { _MMIO(0x9888), 0x0c9fa800 },
	1518	+ { _MMIO(0x9888), 0x0e9faa2a },
	1519	+ { _MMIO(0x9888), 0x109f02aa },
	1520	+ { _MMIO(0x9888), 0x0eb8a5c1 },
	1521	+ { _MMIO(0x9888), 0x00b8a100 },
	1522	+ { _MMIO(0x9888), 0x02b8204c },
	1523	+ { _MMIO(0x9888), 0x16b88000 },
	1524	+ { _MMIO(0x9888), 0x18b802aa },
	1525	+ { _MMIO(0x9888), 0x04b80000 },
	1526	+ { _MMIO(0x9888), 0x06b80000 },
	1527	+ { _MMIO(0x9888), 0x08b88000 },
	1528	+ { _MMIO(0x9888), 0x0ab88000 },
	1529	+ { _MMIO(0x9888), 0x00b9a000 },
	1530	+ { _MMIO(0x9888), 0x06b98000 },
	1531	+ { _MMIO(0x9888), 0x08b9a000 },
	1532	+ { _MMIO(0x9888), 0x0ab9a000 },
	1533	+ { _MMIO(0x9888), 0x0cb9a000 },
	1534	+ { _MMIO(0x9888), 0x0eb9a000 },
	1535	+ { _MMIO(0x9888), 0x02b9a000 },
	1536	+ { _MMIO(0x9888), 0x04b9a000 },
	1537	+ { _MMIO(0x9888), 0x00ba0011 },
	1538	+ { _MMIO(0x9888), 0x06ba0900 },
	1539	+ { _MMIO(0x9888), 0x08ba0a13 },
	1540	+ { _MMIO(0x9888), 0x0aba0b15 },
	1541	+ { _MMIO(0x9888), 0x0cba2317 },
	1542	+ { _MMIO(0x9888), 0x04ba21b7 },
	1543	+ { _MMIO(0x9888), 0x10ba0000 },
	1544	+ { _MMIO(0x9888), 0x0eba0000 },
	1545	+ { _MMIO(0x9888), 0x1aba0000 },
	1546	+ { _MMIO(0x9888), 0x01888000 },
	1547	+ { _MMIO(0x9888), 0x0d88f800 },
	1548	+ { _MMIO(0x9888), 0x0f88000f },
	1549	+ { _MMIO(0x9888), 0x03888000 },
	1550	+ { _MMIO(0x9888), 0x05888000 },
	1551	+ { _MMIO(0x9888), 0x07888000 },
	1552	+ { _MMIO(0x9888), 0x09888000 },
	1553	+ { _MMIO(0x9888), 0x0b888000 },
	1554	+ { _MMIO(0x9888), 0x238b5540 },
	1555	+ { _MMIO(0x9888), 0x258baaa2 },
	1556	+ { _MMIO(0x9888), 0x278b002a },
	1557	+ { _MMIO(0x9888), 0x018c4000 },
	1558	+ { _MMIO(0x9888), 0x0f8c4000 },
	1559	+ { _MMIO(0x9888), 0x178c2000 },
	1560	+ { _MMIO(0x9888), 0x198c5500 },
	1561	+ { _MMIO(0x9888), 0x1b8c0015 },
	1562	+ { _MMIO(0x9888), 0x038c4000 },
	1563	+ { _MMIO(0x9888), 0x058c4000 },
	1564	+ { _MMIO(0x9888), 0x078c4000 },
	1565	+ { _MMIO(0x9888), 0x098c4000 },
	1566	+ { _MMIO(0x9888), 0x0b8c4000 },
	1567	+ { _MMIO(0x9888), 0x018da000 },
	1568	+ { _MMIO(0x9888), 0x078d8000 },
	1569	+ { _MMIO(0x9888), 0x098da000 },
	1570	+ { _MMIO(0x9888), 0x0b8da000 },
	1571	+ { _MMIO(0x9888), 0x0d8da000 },
	1572	+ { _MMIO(0x9888), 0x0f8da000 },
	1573	+ { _MMIO(0x9888), 0x038da000 },
	1574	+ { _MMIO(0x9888), 0x058da000 },
	1575	+ { _MMIO(0x9888), 0x1f85aa80 },
	1576	+ { _MMIO(0x9888), 0x2185aaa2 },
	1577	+ { _MMIO(0x9888), 0x2385002a },
	1578	+ { _MMIO(0x9888), 0x01834000 },
	1579	+ { _MMIO(0x9888), 0x0f834000 },
	1580	+ { _MMIO(0x9888), 0x19835400 },
	1581	+ { _MMIO(0x9888), 0x1b830155 },
	1582	+ { _MMIO(0x9888), 0x03834000 },
	1583	+ { _MMIO(0x9888), 0x05834000 },
	1584	+ { _MMIO(0x9888), 0x07834000 },
	1585	+ { _MMIO(0x9888), 0x09834000 },
	1586	+ { _MMIO(0x9888), 0x0b834000 },
	1587	+ { _MMIO(0x9888), 0x0184c000 },
	1588	+ { _MMIO(0x9888), 0x07848000 },
	1589	+ { _MMIO(0x9888), 0x0984c000 },
	1590	+ { _MMIO(0x9888), 0x0b84c000 },
	1591	+ { _MMIO(0x9888), 0x0d84c000 },
	1592	+ { _MMIO(0x9888), 0x0f84c000 },
	1593	+ { _MMIO(0x9888), 0x0384c000 },
	1594	+ { _MMIO(0x9888), 0x0584c000 },
	1595	+ { _MMIO(0x9888), 0x1180c000 },
	1596	+ { _MMIO(0x9888), 0x17808000 },
	1597	+ { _MMIO(0x9888), 0x1980c000 },
	1598	+ { _MMIO(0x9888), 0x1b80c000 },
	1599	+ { _MMIO(0x9888), 0x1d80c000 },
	1600	+ { _MMIO(0x9888), 0x1f80c000 },
	1601	+ { _MMIO(0x9888), 0x1380c000 },
	1602	+ { _MMIO(0x9888), 0x1580c000 },
	1603	+ { _MMIO(0xd24), 0x00000000 },
	1604	+ { _MMIO(0x9888), 0x4d800000 },
	1605	+ { _MMIO(0x9888), 0x3d800000 },
	1606	+ { _MMIO(0x9888), 0x4f800000 },
	1607	+ { _MMIO(0x9888), 0x43800000 },
	1608	+ { _MMIO(0x9888), 0x51800000 },
	1609	+ { _MMIO(0x9888), 0x45800000 },
	1610	+ { _MMIO(0x9888), 0x53800000 },
	1611	+ { _MMIO(0x9888), 0x47800420 },
	1612	+ { _MMIO(0x9888), 0x21800000 },
	1613	+ { _MMIO(0x9888), 0x31800000 },
	1614	+ { _MMIO(0x9888), 0x3f800421 },
	1615	+ { _MMIO(0x9888), 0x41800000 },
	1616	+};
	1617	+
	1618	+static int
	1619	+get_compute_extended_mux_config(struct drm_i915_private *dev_priv,
	1620	+ const struct i915_oa_reg **regs,
	1621	+ int *lens)
	1622	+{
	1623	+ int n = 0;
	1624	+
	1625	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 6);
	1626	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 6);
	1627	+
	1628	+ if (INTEL_INFO(dev_priv)->sseu.subslice_mask & 0x01) {
	1629	+ regs[n] = mux_config_compute_extended_0_subslices_0x01;
	1630	+ lens[n] = ARRAY_SIZE(mux_config_compute_extended_0_subslices_0x01);
	1631	+ n++;
	1632	+ }
	1633	+ if (INTEL_INFO(dev_priv)->sseu.subslice_mask & 0x08) {
	1634	+ regs[n] = mux_config_compute_extended_1_subslices_0x08;
	1635	+ lens[n] = ARRAY_SIZE(mux_config_compute_extended_1_subslices_0x08);
	1636	+ n++;
	1637	+ }
	1638	+ if (INTEL_INFO(dev_priv)->sseu.subslice_mask & 0x02) {
	1639	+ regs[n] = mux_config_compute_extended_2_subslices_0x02;
	1640	+ lens[n] = ARRAY_SIZE(mux_config_compute_extended_2_subslices_0x02);
	1641	+ n++;
	1642	+ }
	1643	+ if (INTEL_INFO(dev_priv)->sseu.subslice_mask & 0x10) {
	1644	+ regs[n] = mux_config_compute_extended_3_subslices_0x10;
	1645	+ lens[n] = ARRAY_SIZE(mux_config_compute_extended_3_subslices_0x10);
	1646	+ n++;
	1647	+ }
	1648	+ if (INTEL_INFO(dev_priv)->sseu.subslice_mask & 0x04) {
	1649	+ regs[n] = mux_config_compute_extended_4_subslices_0x04;
	1650	+ lens[n] = ARRAY_SIZE(mux_config_compute_extended_4_subslices_0x04);
	1651	+ n++;
	1652	+ }
	1653	+ if (INTEL_INFO(dev_priv)->sseu.subslice_mask & 0x20) {
	1654	+ regs[n] = mux_config_compute_extended_5_subslices_0x20;
	1655	+ lens[n] = ARRAY_SIZE(mux_config_compute_extended_5_subslices_0x20);
	1656	+ n++;
	1657	+ }
	1658	+
	1659	+ return n;
	1660	+}
	1661	+
	1662	+static const struct i915_oa_reg b_counter_config_compute_l3_cache[] = {
	1663	+ { _MMIO(0x2710), 0x00000000 },
	1664	+ { _MMIO(0x2714), 0x30800000 },
	1665	+ { _MMIO(0x2720), 0x00000000 },
	1666	+ { _MMIO(0x2724), 0x30800000 },
	1667	+ { _MMIO(0x2740), 0x00000000 },
	1668	+ { _MMIO(0x2770), 0x0007fffa },
	1669	+ { _MMIO(0x2774), 0x0000fefe },
	1670	+ { _MMIO(0x2778), 0x0007fffa },
	1671	+ { _MMIO(0x277c), 0x0000fefd },
	1672	+ { _MMIO(0x2790), 0x0007fffa },
	1673	+ { _MMIO(0x2794), 0x0000fbef },
	1674	+ { _MMIO(0x2798), 0x0007fffa },
	1675	+ { _MMIO(0x279c), 0x0000fbdf },
	1676	+};
	1677	+
	1678	+static const struct i915_oa_reg flex_eu_config_compute_l3_cache[] = {
	1679	+ { _MMIO(0xe458), 0x00005004 },
	1680	+ { _MMIO(0xe558), 0x00000003 },
	1681	+ { _MMIO(0xe658), 0x00002001 },
	1682	+ { _MMIO(0xe758), 0x00101100 },
	1683	+ { _MMIO(0xe45c), 0x00201200 },
	1684	+ { _MMIO(0xe55c), 0x00301300 },
	1685	+ { _MMIO(0xe65c), 0x00401400 },
	1686	+};
	1687	+
	1688	+static const struct i915_oa_reg mux_config_compute_l3_cache[] = {
	1689	+ { _MMIO(0x9888), 0x143f00b3 },
	1690	+ { _MMIO(0x9888), 0x14bf00b3 },
	1691	+ { _MMIO(0x9888), 0x138303c0 },
	1692	+ { _MMIO(0x9888), 0x3b800060 },
	1693	+ { _MMIO(0x9888), 0x3d800805 },
	1694	+ { _MMIO(0x9888), 0x003f0029 },
	1695	+ { _MMIO(0x9888), 0x063f1400 },
	1696	+ { _MMIO(0x9888), 0x083f1225 },
	1697	+ { _MMIO(0x9888), 0x0e3f1327 },
	1698	+ { _MMIO(0x9888), 0x103f0000 },
	1699	+ { _MMIO(0x9888), 0x005a4000 },
	1700	+ { _MMIO(0x9888), 0x065a8000 },
	1701	+ { _MMIO(0x9888), 0x085ac000 },
	1702	+ { _MMIO(0x9888), 0x0e5ac000 },
	1703	+ { _MMIO(0x9888), 0x001d4000 },
	1704	+ { _MMIO(0x9888), 0x061d8000 },
	1705	+ { _MMIO(0x9888), 0x081dc000 },
	1706	+ { _MMIO(0x9888), 0x0e1dc000 },
	1707	+ { _MMIO(0x9888), 0x0c1f0800 },
	1708	+ { _MMIO(0x9888), 0x0e1f2a00 },
	1709	+ { _MMIO(0x9888), 0x101f0280 },
	1710	+ { _MMIO(0x9888), 0x00391000 },
	1711	+ { _MMIO(0x9888), 0x06394000 },
	1712	+ { _MMIO(0x9888), 0x08395000 },
	1713	+ { _MMIO(0x9888), 0x0e395000 },
	1714	+ { _MMIO(0x9888), 0x0abf1429 },
	1715	+ { _MMIO(0x9888), 0x0cbf1225 },
	1716	+ { _MMIO(0x9888), 0x00bf1380 },
	1717	+ { _MMIO(0x9888), 0x02bf0026 },
	1718	+ { _MMIO(0x9888), 0x10bf0000 },
	1719	+ { _MMIO(0x9888), 0x0adac000 },
	1720	+ { _MMIO(0x9888), 0x0cdac000 },
	1721	+ { _MMIO(0x9888), 0x00da8000 },
	1722	+ { _MMIO(0x9888), 0x02da4000 },
	1723	+ { _MMIO(0x9888), 0x0a9dc000 },
	1724	+ { _MMIO(0x9888), 0x0c9dc000 },
	1725	+ { _MMIO(0x9888), 0x009d8000 },
	1726	+ { _MMIO(0x9888), 0x029d4000 },
	1727	+ { _MMIO(0x9888), 0x0e9f8000 },
	1728	+ { _MMIO(0x9888), 0x109f002a },
	1729	+ { _MMIO(0x9888), 0x0c9fa000 },
	1730	+ { _MMIO(0x9888), 0x0ab95000 },
	1731	+ { _MMIO(0x9888), 0x0cb95000 },
	1732	+ { _MMIO(0x9888), 0x00b94000 },
	1733	+ { _MMIO(0x9888), 0x02b91000 },
	1734	+ { _MMIO(0x9888), 0x0d88c000 },
	1735	+ { _MMIO(0x9888), 0x0f880003 },
	1736	+ { _MMIO(0x9888), 0x03888000 },
	1737	+ { _MMIO(0x9888), 0x05888000 },
	1738	+ { _MMIO(0x9888), 0x018a8000 },
	1739	+ { _MMIO(0x9888), 0x0f8a8000 },
	1740	+ { _MMIO(0x9888), 0x198a8000 },
	1741	+ { _MMIO(0x9888), 0x1b8a8020 },
	1742	+ { _MMIO(0x9888), 0x1d8a0002 },
	1743	+ { _MMIO(0x9888), 0x238b0520 },
	1744	+ { _MMIO(0x9888), 0x258ba950 },
	1745	+ { _MMIO(0x9888), 0x278b0016 },
	1746	+ { _MMIO(0x9888), 0x198c5400 },
	1747	+ { _MMIO(0x9888), 0x1b8c0001 },
	1748	+ { _MMIO(0x9888), 0x038c4000 },
	1749	+ { _MMIO(0x9888), 0x058c4000 },
	1750	+ { _MMIO(0x9888), 0x0b8da000 },
	1751	+ { _MMIO(0x9888), 0x0d8da000 },
	1752	+ { _MMIO(0x9888), 0x018d8000 },
	1753	+ { _MMIO(0x9888), 0x038d2000 },
	1754	+ { _MMIO(0x9888), 0x1f85aa80 },
	1755	+ { _MMIO(0x9888), 0x2185aaa0 },
	1756	+ { _MMIO(0x9888), 0x2385002a },
	1757	+ { _MMIO(0x9888), 0x03835180 },
	1758	+ { _MMIO(0x9888), 0x05834022 },
	1759	+ { _MMIO(0x9888), 0x11830000 },
	1760	+ { _MMIO(0x9888), 0x01834000 },
	1761	+ { _MMIO(0x9888), 0x0f834000 },
	1762	+ { _MMIO(0x9888), 0x19835400 },
	1763	+ { _MMIO(0x9888), 0x1b830155 },
	1764	+ { _MMIO(0x9888), 0x07830000 },
	1765	+ { _MMIO(0x9888), 0x09830000 },
	1766	+ { _MMIO(0x9888), 0x0184c000 },
	1767	+ { _MMIO(0x9888), 0x07848000 },
	1768	+ { _MMIO(0x9888), 0x0984c000 },
	1769	+ { _MMIO(0x9888), 0x0b84c000 },
	1770	+ { _MMIO(0x9888), 0x0d84c000 },
	1771	+ { _MMIO(0x9888), 0x0f84c000 },
	1772	+ { _MMIO(0x9888), 0x0384c000 },
	1773	+ { _MMIO(0x9888), 0x05844000 },
	1774	+ { _MMIO(0x9888), 0x1b80c137 },
	1775	+ { _MMIO(0x9888), 0x1d80c147 },
	1776	+ { _MMIO(0x9888), 0x21800000 },
	1777	+ { _MMIO(0x9888), 0x1180c000 },
	1778	+ { _MMIO(0x9888), 0x17808000 },
	1779	+ { _MMIO(0x9888), 0x1980c000 },
	1780	+ { _MMIO(0x9888), 0x1f80c000 },
	1781	+ { _MMIO(0x9888), 0x1380c000 },
	1782	+ { _MMIO(0x9888), 0x15804000 },
	1783	+ { _MMIO(0xd24), 0x00000000 },
	1784	+ { _MMIO(0x9888), 0x4d801000 },
	1785	+ { _MMIO(0x9888), 0x4f800111 },
	1786	+ { _MMIO(0x9888), 0x43800842 },
	1787	+ { _MMIO(0x9888), 0x51800000 },
	1788	+ { _MMIO(0x9888), 0x45800000 },
	1789	+ { _MMIO(0x9888), 0x53800000 },
	1790	+ { _MMIO(0x9888), 0x47800840 },
	1791	+ { _MMIO(0x9888), 0x31800000 },
	1792	+ { _MMIO(0x9888), 0x3f800800 },
	1793	+ { _MMIO(0x9888), 0x418014a2 },
	1794	+};
	1795	+
	1796	+static int
	1797	+get_compute_l3_cache_mux_config(struct drm_i915_private *dev_priv,
	1798	+ const struct i915_oa_reg **regs,
	1799	+ int *lens)
	1800	+{
	1801	+ int n = 0;
	1802	+
	1803	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1804	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1805	+
	1806	+ regs[n] = mux_config_compute_l3_cache;
	1807	+ lens[n] = ARRAY_SIZE(mux_config_compute_l3_cache);
	1808	+ n++;
	1809	+
	1810	+ return n;
	1811	+}
	1812	+
	1813	+static const struct i915_oa_reg b_counter_config_data_port_reads_coalescing[] = {
	1814	+ { _MMIO(0x2724), 0xf0800000 },
	1815	+ { _MMIO(0x2720), 0x00000000 },
	1816	+ { _MMIO(0x2714), 0xf0800000 },
	1817	+ { _MMIO(0x2710), 0x00000000 },
	1818	+ { _MMIO(0x274c), 0xba98ba98 },
	1819	+ { _MMIO(0x2748), 0xba98ba98 },
	1820	+ { _MMIO(0x2744), 0x00003377 },
	1821	+ { _MMIO(0x2740), 0x00000000 },
	1822	+ { _MMIO(0x2770), 0x0007fff2 },
	1823	+ { _MMIO(0x2774), 0x00007ff0 },
	1824	+ { _MMIO(0x2778), 0x0007ffe2 },
	1825	+ { _MMIO(0x277c), 0x00007ff0 },
	1826	+ { _MMIO(0x2780), 0x0007ffc2 },
	1827	+ { _MMIO(0x2784), 0x00007ff0 },
	1828	+ { _MMIO(0x2788), 0x0007ff82 },
	1829	+ { _MMIO(0x278c), 0x00007ff0 },
	1830	+ { _MMIO(0x2790), 0x0007fffa },
	1831	+ { _MMIO(0x2794), 0x0000bfef },
	1832	+ { _MMIO(0x2798), 0x0007fffa },
	1833	+ { _MMIO(0x279c), 0x0000bfdf },
	1834	+ { _MMIO(0x27a0), 0x0007fffa },
	1835	+ { _MMIO(0x27a4), 0x0000bfbf },
	1836	+ { _MMIO(0x27a8), 0x0007fffa },
	1837	+ { _MMIO(0x27ac), 0x0000bf7f },
	1838	+};
	1839	+
	1840	+static const struct i915_oa_reg flex_eu_config_data_port_reads_coalescing[] = {
	1841	+ { _MMIO(0xe458), 0x00005004 },
	1842	+ { _MMIO(0xe558), 0x00000003 },
	1843	+ { _MMIO(0xe658), 0x00002001 },
	1844	+ { _MMIO(0xe758), 0x00778008 },
	1845	+ { _MMIO(0xe45c), 0x00088078 },
	1846	+ { _MMIO(0xe55c), 0x00808708 },
	1847	+ { _MMIO(0xe65c), 0x00a08908 },
	1848	+};
	1849	+
	1850	+static const struct i915_oa_reg mux_config_data_port_reads_coalescing_0_subslices_0x01[] = {
	1851	+ { _MMIO(0x9888), 0x103d0005 },
	1852	+ { _MMIO(0x9888), 0x163d240b },
	1853	+ { _MMIO(0x9888), 0x1058022f },
	1854	+ { _MMIO(0x9888), 0x185b5520 },
	1855	+ { _MMIO(0x9888), 0x198b0003 },
	1856	+ { _MMIO(0x9888), 0x005cc000 },
	1857	+ { _MMIO(0x9888), 0x065cc000 },
	1858	+ { _MMIO(0x9888), 0x085cc000 },
	1859	+ { _MMIO(0x9888), 0x0a5cc000 },
	1860	+ { _MMIO(0x9888), 0x0c5cc000 },
	1861	+ { _MMIO(0x9888), 0x0e5cc000 },
	1862	+ { _MMIO(0x9888), 0x025c4000 },
	1863	+ { _MMIO(0x9888), 0x045c8000 },
	1864	+ { _MMIO(0x9888), 0x003d0000 },
	1865	+ { _MMIO(0x9888), 0x063d00b0 },
	1866	+ { _MMIO(0x9888), 0x083d0182 },
	1867	+ { _MMIO(0x9888), 0x0a3d10a0 },
	1868	+ { _MMIO(0x9888), 0x0c3d11a2 },
	1869	+ { _MMIO(0x9888), 0x0e3d0000 },
	1870	+ { _MMIO(0x9888), 0x183d0000 },
	1871	+ { _MMIO(0x9888), 0x1a3d0000 },
	1872	+ { _MMIO(0x9888), 0x0e582242 },
	1873	+ { _MMIO(0x9888), 0x00586700 },
	1874	+ { _MMIO(0x9888), 0x0258004f },
	1875	+ { _MMIO(0x9888), 0x0658c000 },
	1876	+ { _MMIO(0x9888), 0x0858c000 },
	1877	+ { _MMIO(0x9888), 0x0a58c000 },
	1878	+ { _MMIO(0x9888), 0x0c58c000 },
	1879	+ { _MMIO(0x9888), 0x045b6300 },
	1880	+ { _MMIO(0x9888), 0x105b0000 },
	1881	+ { _MMIO(0x9888), 0x005b4000 },
	1882	+ { _MMIO(0x9888), 0x0e5b4000 },
	1883	+ { _MMIO(0x9888), 0x1a5b0155 },
	1884	+ { _MMIO(0x9888), 0x025b4000 },
	1885	+ { _MMIO(0x9888), 0x0a5b0000 },
	1886	+ { _MMIO(0x9888), 0x0c5b4000 },
	1887	+ { _MMIO(0x9888), 0x0c1fa800 },
	1888	+ { _MMIO(0x9888), 0x0e1faaa0 },
	1889	+ { _MMIO(0x9888), 0x101f02aa },
	1890	+ { _MMIO(0x9888), 0x00384000 },
	1891	+ { _MMIO(0x9888), 0x0e384000 },
	1892	+ { _MMIO(0x9888), 0x16384000 },
	1893	+ { _MMIO(0x9888), 0x18381555 },
	1894	+ { _MMIO(0x9888), 0x02384000 },
	1895	+ { _MMIO(0x9888), 0x04384000 },
	1896	+ { _MMIO(0x9888), 0x0a384000 },
	1897	+ { _MMIO(0x9888), 0x0c384000 },
	1898	+ { _MMIO(0x9888), 0x0039a000 },
	1899	+ { _MMIO(0x9888), 0x0639a000 },
	1900	+ { _MMIO(0x9888), 0x0839a000 },
	1901	+ { _MMIO(0x9888), 0x0a39a000 },
	1902	+ { _MMIO(0x9888), 0x0c39a000 },
	1903	+ { _MMIO(0x9888), 0x0e39a000 },
	1904	+ { _MMIO(0x9888), 0x02392000 },
	1905	+ { _MMIO(0x9888), 0x04398000 },
	1906	+ { _MMIO(0x9888), 0x018a8000 },
	1907	+ { _MMIO(0x9888), 0x0f8a8000 },
	1908	+ { _MMIO(0x9888), 0x198a8000 },
	1909	+ { _MMIO(0x9888), 0x1b8aaaa0 },
	1910	+ { _MMIO(0x9888), 0x1d8a0002 },
	1911	+ { _MMIO(0x9888), 0x038a8000 },
	1912	+ { _MMIO(0x9888), 0x058a8000 },
	1913	+ { _MMIO(0x9888), 0x0b8a8000 },
	1914	+ { _MMIO(0x9888), 0x0d8a8000 },
	1915	+ { _MMIO(0x9888), 0x038b6300 },
	1916	+ { _MMIO(0x9888), 0x058b0062 },
	1917	+ { _MMIO(0x9888), 0x118b0000 },
	1918	+ { _MMIO(0x9888), 0x238b02a0 },
	1919	+ { _MMIO(0x9888), 0x258b5555 },
	1920	+ { _MMIO(0x9888), 0x278b0015 },
	1921	+ { _MMIO(0x9888), 0x1f85aa80 },
	1922	+ { _MMIO(0x9888), 0x2185aaaa },
	1923	+ { _MMIO(0x9888), 0x2385002a },
	1924	+ { _MMIO(0x9888), 0x01834000 },
	1925	+ { _MMIO(0x9888), 0x0f834000 },
	1926	+ { _MMIO(0x9888), 0x19835400 },
	1927	+ { _MMIO(0x9888), 0x1b830155 },
	1928	+ { _MMIO(0x9888), 0x03834000 },
	1929	+ { _MMIO(0x9888), 0x05834000 },
	1930	+ { _MMIO(0x9888), 0x07834000 },
	1931	+ { _MMIO(0x9888), 0x09834000 },
	1932	+ { _MMIO(0x9888), 0x0b834000 },
	1933	+ { _MMIO(0x9888), 0x0d834000 },
	1934	+ { _MMIO(0x9888), 0x0184c000 },
	1935	+ { _MMIO(0x9888), 0x0784c000 },
	1936	+ { _MMIO(0x9888), 0x0984c000 },
	1937	+ { _MMIO(0x9888), 0x0b84c000 },
	1938	+ { _MMIO(0x9888), 0x0d84c000 },
	1939	+ { _MMIO(0x9888), 0x0f84c000 },
	1940	+ { _MMIO(0x9888), 0x0384c000 },
	1941	+ { _MMIO(0x9888), 0x0584c000 },
	1942	+ { _MMIO(0x9888), 0x1180c000 },
	1943	+ { _MMIO(0x9888), 0x1780c000 },
	1944	+ { _MMIO(0x9888), 0x1980c000 },
	1945	+ { _MMIO(0x9888), 0x1b80c000 },
	1946	+ { _MMIO(0x9888), 0x1d80c000 },
	1947	+ { _MMIO(0x9888), 0x1f80c000 },
	1948	+ { _MMIO(0x9888), 0x1380c000 },
	1949	+ { _MMIO(0x9888), 0x1580c000 },
	1950	+ { _MMIO(0xd24), 0x00000000 },
	1951	+ { _MMIO(0x9888), 0x4d801000 },
	1952	+ { _MMIO(0x9888), 0x3d800000 },
	1953	+ { _MMIO(0x9888), 0x4f800001 },
	1954	+ { _MMIO(0x9888), 0x43800000 },
	1955	+ { _MMIO(0x9888), 0x51800000 },
	1956	+ { _MMIO(0x9888), 0x45800000 },
	1957	+ { _MMIO(0x9888), 0x53800000 },
	1958	+ { _MMIO(0x9888), 0x47800420 },
	1959	+ { _MMIO(0x9888), 0x21800000 },
	1960	+ { _MMIO(0x9888), 0x31800000 },
	1961	+ { _MMIO(0x9888), 0x3f800421 },
	1962	+ { _MMIO(0x9888), 0x41800041 },
	1963	+};
	1964	+
	1965	+static int
	1966	+get_data_port_reads_coalescing_mux_config(struct drm_i915_private *dev_priv,
	1967	+ const struct i915_oa_reg **regs,
	1968	+ int *lens)
	1969	+{
	1970	+ int n = 0;
	1971	+
	1972	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1973	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1974	+
	1975	+ if (INTEL_INFO(dev_priv)->sseu.subslice_mask & 0x01) {
	1976	+ regs[n] = mux_config_data_port_reads_coalescing_0_subslices_0x01;
	1977	+ lens[n] = ARRAY_SIZE(mux_config_data_port_reads_coalescing_0_subslices_0x01);
	1978	+ n++;
	1979	+ }
	1980	+
	1981	+ return n;
	1982	+}
	1983	+
	1984	+static const struct i915_oa_reg b_counter_config_data_port_writes_coalescing[] = {
	1985	+ { _MMIO(0x2724), 0xf0800000 },
	1986	+ { _MMIO(0x2720), 0x00000000 },
	1987	+ { _MMIO(0x2714), 0xf0800000 },
	1988	+ { _MMIO(0x2710), 0x00000000 },
	1989	+ { _MMIO(0x274c), 0xba98ba98 },
	1990	+ { _MMIO(0x2748), 0xba98ba98 },
	1991	+ { _MMIO(0x2744), 0x00003377 },
	1992	+ { _MMIO(0x2740), 0x00000000 },
	1993	+ { _MMIO(0x2770), 0x0007ff72 },
	1994	+ { _MMIO(0x2774), 0x0000bfd0 },
	1995	+ { _MMIO(0x2778), 0x0007ff62 },
	1996	+ { _MMIO(0x277c), 0x0000bfd0 },
	1997	+ { _MMIO(0x2780), 0x0007ff42 },
	1998	+ { _MMIO(0x2784), 0x0000bfd0 },
	1999	+ { _MMIO(0x2788), 0x0007ff02 },
	2000	+ { _MMIO(0x278c), 0x0000bfd0 },
	2001	+ { _MMIO(0x2790), 0x0005fff2 },
	2002	+ { _MMIO(0x2794), 0x0000bfd0 },
	2003	+ { _MMIO(0x2798), 0x0005ffe2 },
	2004	+ { _MMIO(0x279c), 0x0000bfd0 },
	2005	+ { _MMIO(0x27a0), 0x0005ffc2 },
	2006	+ { _MMIO(0x27a4), 0x0000bfd0 },
	2007	+ { _MMIO(0x27a8), 0x0005ff82 },
	2008	+ { _MMIO(0x27ac), 0x0000bfd0 },
	2009	+};
	2010	+
	2011	+static const struct i915_oa_reg flex_eu_config_data_port_writes_coalescing[] = {
	2012	+ { _MMIO(0xe458), 0x00005004 },
	2013	+ { _MMIO(0xe558), 0x00000003 },
	2014	+ { _MMIO(0xe658), 0x00002001 },
	2015	+ { _MMIO(0xe758), 0x00778008 },
	2016	+ { _MMIO(0xe45c), 0x00088078 },
	2017	+ { _MMIO(0xe55c), 0x00808708 },
	2018	+ { _MMIO(0xe65c), 0x00a08908 },
	2019	+};
	2020	+
	2021	+static const struct i915_oa_reg mux_config_data_port_writes_coalescing_0_subslices_0x01[] = {
	2022	+ { _MMIO(0x9888), 0x103d0005 },
	2023	+ { _MMIO(0x9888), 0x143d0120 },
	2024	+ { _MMIO(0x9888), 0x163d2400 },
	2025	+ { _MMIO(0x9888), 0x1058022f },
	2026	+ { _MMIO(0x9888), 0x105b0000 },
	2027	+ { _MMIO(0x9888), 0x198b0003 },
	2028	+ { _MMIO(0x9888), 0x005cc000 },
	2029	+ { _MMIO(0x9888), 0x065cc000 },
	2030	+ { _MMIO(0x9888), 0x085cc000 },
	2031	+ { _MMIO(0x9888), 0x0a5cc000 },
	2032	+ { _MMIO(0x9888), 0x0e5cc000 },
	2033	+ { _MMIO(0x9888), 0x025c4000 },
	2034	+ { _MMIO(0x9888), 0x045c8000 },
	2035	+ { _MMIO(0x9888), 0x003d0000 },
	2036	+ { _MMIO(0x9888), 0x063d0094 },
	2037	+ { _MMIO(0x9888), 0x083d0182 },
	2038	+ { _MMIO(0x9888), 0x0a3d1814 },
	2039	+ { _MMIO(0x9888), 0x0e3d0000 },
	2040	+ { _MMIO(0x9888), 0x183d0000 },
	2041	+ { _MMIO(0x9888), 0x1a3d0000 },
	2042	+ { _MMIO(0x9888), 0x0c3d0000 },
	2043	+ { _MMIO(0x9888), 0x0e582242 },
	2044	+ { _MMIO(0x9888), 0x00586700 },
	2045	+ { _MMIO(0x9888), 0x0258004f },
	2046	+ { _MMIO(0x9888), 0x0658c000 },
	2047	+ { _MMIO(0x9888), 0x0858c000 },
	2048	+ { _MMIO(0x9888), 0x0a58c000 },
	2049	+ { _MMIO(0x9888), 0x045b6a80 },
	2050	+ { _MMIO(0x9888), 0x005b4000 },
	2051	+ { _MMIO(0x9888), 0x0e5b4000 },
	2052	+ { _MMIO(0x9888), 0x185b5400 },
	2053	+ { _MMIO(0x9888), 0x1a5b0141 },
	2054	+ { _MMIO(0x9888), 0x025b4000 },
	2055	+ { _MMIO(0x9888), 0x0a5b0000 },
	2056	+ { _MMIO(0x9888), 0x0c5b4000 },
	2057	+ { _MMIO(0x9888), 0x0c1fa800 },
	2058	+ { _MMIO(0x9888), 0x0e1faaa0 },
	2059	+ { _MMIO(0x9888), 0x101f0282 },
	2060	+ { _MMIO(0x9888), 0x00384000 },
	2061	+ { _MMIO(0x9888), 0x0e384000 },
	2062	+ { _MMIO(0x9888), 0x16384000 },
	2063	+ { _MMIO(0x9888), 0x18381415 },
	2064	+ { _MMIO(0x9888), 0x02384000 },
	2065	+ { _MMIO(0x9888), 0x04384000 },
	2066	+ { _MMIO(0x9888), 0x0a384000 },
	2067	+ { _MMIO(0x9888), 0x0c384000 },
	2068	+ { _MMIO(0x9888), 0x0039a000 },
	2069	+ { _MMIO(0x9888), 0x0639a000 },
	2070	+ { _MMIO(0x9888), 0x0839a000 },
	2071	+ { _MMIO(0x9888), 0x0a39a000 },
	2072	+ { _MMIO(0x9888), 0x0e39a000 },
	2073	+ { _MMIO(0x9888), 0x02392000 },
	2074	+ { _MMIO(0x9888), 0x04398000 },
	2075	+ { _MMIO(0x9888), 0x018a8000 },
	2076	+ { _MMIO(0x9888), 0x0f8a8000 },
	2077	+ { _MMIO(0x9888), 0x198a8000 },
	2078	+ { _MMIO(0x9888), 0x1b8a82a0 },
	2079	+ { _MMIO(0x9888), 0x1d8a0002 },
	2080	+ { _MMIO(0x9888), 0x038a8000 },
	2081	+ { _MMIO(0x9888), 0x058a8000 },
	2082	+ { _MMIO(0x9888), 0x0b8a8000 },
	2083	+ { _MMIO(0x9888), 0x0d8a8000 },
	2084	+ { _MMIO(0x9888), 0x038b6300 },
	2085	+ { _MMIO(0x9888), 0x058b0062 },
	2086	+ { _MMIO(0x9888), 0x118b0000 },
	2087	+ { _MMIO(0x9888), 0x238b02a0 },
	2088	+ { _MMIO(0x9888), 0x258b1555 },
	2089	+ { _MMIO(0x9888), 0x278b0014 },
	2090	+ { _MMIO(0x9888), 0x1f85aa80 },
	2091	+ { _MMIO(0x9888), 0x21852aaa },
	2092	+ { _MMIO(0x9888), 0x23850028 },
	2093	+ { _MMIO(0x9888), 0x01834000 },
	2094	+ { _MMIO(0x9888), 0x0f834000 },
	2095	+ { _MMIO(0x9888), 0x19835400 },
	2096	+ { _MMIO(0x9888), 0x1b830141 },
	2097	+ { _MMIO(0x9888), 0x03834000 },
	2098	+ { _MMIO(0x9888), 0x05834000 },
	2099	+ { _MMIO(0x9888), 0x07834000 },
	2100	+ { _MMIO(0x9888), 0x09834000 },
	2101	+ { _MMIO(0x9888), 0x0b834000 },
	2102	+ { _MMIO(0x9888), 0x0d834000 },
	2103	+ { _MMIO(0x9888), 0x0184c000 },
	2104	+ { _MMIO(0x9888), 0x0784c000 },
	2105	+ { _MMIO(0x9888), 0x0984c000 },
	2106	+ { _MMIO(0x9888), 0x0b84c000 },
	2107	+ { _MMIO(0x9888), 0x0f84c000 },
	2108	+ { _MMIO(0x9888), 0x0384c000 },
	2109	+ { _MMIO(0x9888), 0x0584c000 },
	2110	+ { _MMIO(0x9888), 0x1180c000 },
	2111	+ { _MMIO(0x9888), 0x1780c000 },
	2112	+ { _MMIO(0x9888), 0x1980c000 },
	2113	+ { _MMIO(0x9888), 0x1b80c000 },
	2114	+ { _MMIO(0x9888), 0x1f80c000 },
	2115	+ { _MMIO(0x9888), 0x1380c000 },
	2116	+ { _MMIO(0x9888), 0x1580c000 },
	2117	+ { _MMIO(0xd24), 0x00000000 },
	2118	+ { _MMIO(0x9888), 0x4d801000 },
	2119	+ { _MMIO(0x9888), 0x3d800000 },
	2120	+ { _MMIO(0x9888), 0x4f800001 },
	2121	+ { _MMIO(0x9888), 0x43800000 },
	2122	+ { _MMIO(0x9888), 0x51800000 },
	2123	+ { _MMIO(0x9888), 0x45800000 },
	2124	+ { _MMIO(0x9888), 0x21800000 },
	2125	+ { _MMIO(0x9888), 0x31800000 },
	2126	+ { _MMIO(0x9888), 0x53800000 },
	2127	+ { _MMIO(0x9888), 0x47800420 },
	2128	+ { _MMIO(0x9888), 0x3f800421 },
	2129	+ { _MMIO(0x9888), 0x41800041 },
	2130	+};
	2131	+
	2132	+static int
	2133	+get_data_port_writes_coalescing_mux_config(struct drm_i915_private *dev_priv,
	2134	+ const struct i915_oa_reg **regs,
	2135	+ int *lens)
	2136	+{
	2137	+ int n = 0;
	2138	+
	2139	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	2140	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	2141	+
	2142	+ if (INTEL_INFO(dev_priv)->sseu.subslice_mask & 0x01) {
	2143	+ regs[n] = mux_config_data_port_writes_coalescing_0_subslices_0x01;
	2144	+ lens[n] = ARRAY_SIZE(mux_config_data_port_writes_coalescing_0_subslices_0x01);
	2145	+ n++;
	2146	+ }
	2147	+
	2148	+ return n;
	2149	+}
	2150	+
	2151	+static const struct i915_oa_reg b_counter_config_hdc_and_sf[] = {
	2152	+ { _MMIO(0x2740), 0x00000000 },
	2153	+ { _MMIO(0x2744), 0x00800000 },
	2154	+ { _MMIO(0x2710), 0x00000000 },
	2155	+ { _MMIO(0x2714), 0x10800000 },
	2156	+ { _MMIO(0x2720), 0x00000000 },
	2157	+ { _MMIO(0x2724), 0x00800000 },
	2158	+ { _MMIO(0x2770), 0x00000002 },
	2159	+ { _MMIO(0x2774), 0x0000fff7 },
	2160	+};
	2161	+
	2162	+static const struct i915_oa_reg flex_eu_config_hdc_and_sf[] = {
	2163	+ { _MMIO(0xe458), 0x00005004 },
	2164	+ { _MMIO(0xe558), 0x00010003 },
	2165	+ { _MMIO(0xe658), 0x00012011 },
	2166	+ { _MMIO(0xe758), 0x00015014 },
	2167	+ { _MMIO(0xe45c), 0x00051050 },
	2168	+ { _MMIO(0xe55c), 0x00053052 },
	2169	+ { _MMIO(0xe65c), 0x00055054 },
	2170	+};
	2171	+
	2172	+static const struct i915_oa_reg mux_config_hdc_and_sf[] = {
	2173	+ { _MMIO(0x9888), 0x105c0232 },
	2174	+ { _MMIO(0x9888), 0x10580232 },
	2175	+ { _MMIO(0x9888), 0x10380232 },
	2176	+ { _MMIO(0x9888), 0x10dc0232 },
	2177	+ { _MMIO(0x9888), 0x10d80232 },
	2178	+ { _MMIO(0x9888), 0x10b80232 },
	2179	+ { _MMIO(0x9888), 0x118e4400 },
	2180	+ { _MMIO(0x9888), 0x025c6080 },
	2181	+ { _MMIO(0x9888), 0x045c004b },
	2182	+ { _MMIO(0x9888), 0x005c8000 },
	2183	+ { _MMIO(0x9888), 0x00582080 },
	2184	+ { _MMIO(0x9888), 0x0258004b },
	2185	+ { _MMIO(0x9888), 0x025b4000 },
	2186	+ { _MMIO(0x9888), 0x045b4000 },
	2187	+ { _MMIO(0x9888), 0x0c1fa000 },
	2188	+ { _MMIO(0x9888), 0x0e1f00aa },
	2189	+ { _MMIO(0x9888), 0x04386080 },
	2190	+ { _MMIO(0x9888), 0x0638404b },
	2191	+ { _MMIO(0x9888), 0x02384000 },
	2192	+ { _MMIO(0x9888), 0x08384000 },
	2193	+ { _MMIO(0x9888), 0x0a380000 },
	2194	+ { _MMIO(0x9888), 0x0c380000 },
	2195	+ { _MMIO(0x9888), 0x00398000 },
	2196	+ { _MMIO(0x9888), 0x0239a000 },
	2197	+ { _MMIO(0x9888), 0x0439a000 },
	2198	+ { _MMIO(0x9888), 0x06392000 },
	2199	+ { _MMIO(0x9888), 0x0cdc25c1 },
	2200	+ { _MMIO(0x9888), 0x0adcc000 },
	2201	+ { _MMIO(0x9888), 0x0ad825c1 },
	2202	+ { _MMIO(0x9888), 0x18db4000 },
	2203	+ { _MMIO(0x9888), 0x1adb0001 },
	2204	+ { _MMIO(0x9888), 0x0e9f8000 },
	2205	+ { _MMIO(0x9888), 0x109f02aa },
	2206	+ { _MMIO(0x9888), 0x0eb825c1 },
	2207	+ { _MMIO(0x9888), 0x18b80154 },
	2208	+ { _MMIO(0x9888), 0x0ab9a000 },
	2209	+ { _MMIO(0x9888), 0x0cb9a000 },
	2210	+ { _MMIO(0x9888), 0x0eb9a000 },
	2211	+ { _MMIO(0x9888), 0x0d88c000 },
	2212	+ { _MMIO(0x9888), 0x0f88000f },
	2213	+ { _MMIO(0x9888), 0x038a8000 },
	2214	+ { _MMIO(0x9888), 0x058a8000 },
	2215	+ { _MMIO(0x9888), 0x078a8000 },
	2216	+ { _MMIO(0x9888), 0x098a8000 },
	2217	+ { _MMIO(0x9888), 0x0b8a8000 },
	2218	+ { _MMIO(0x9888), 0x0d8a8000 },
	2219	+ { _MMIO(0x9888), 0x258baa05 },
	2220	+ { _MMIO(0x9888), 0x278b002a },
	2221	+ { _MMIO(0x9888), 0x238b2a80 },
	2222	+ { _MMIO(0x9888), 0x198c5400 },
	2223	+ { _MMIO(0x9888), 0x1b8c0015 },
	2224	+ { _MMIO(0x9888), 0x098dc000 },
	2225	+ { _MMIO(0x9888), 0x0b8da000 },
	2226	+ { _MMIO(0x9888), 0x0d8da000 },
	2227	+ { _MMIO(0x9888), 0x0f8da000 },
	2228	+ { _MMIO(0x9888), 0x098e05c0 },
	2229	+ { _MMIO(0x9888), 0x058e0000 },
	2230	+ { _MMIO(0x9888), 0x198f0020 },
	2231	+ { _MMIO(0x9888), 0x2185aa0a },
	2232	+ { _MMIO(0x9888), 0x2385002a },
	2233	+ { _MMIO(0x9888), 0x1f85aa00 },
	2234	+ { _MMIO(0x9888), 0x19835000 },
	2235	+ { _MMIO(0x9888), 0x1b830155 },
	2236	+ { _MMIO(0x9888), 0x03834000 },
	2237	+ { _MMIO(0x9888), 0x05834000 },
	2238	+ { _MMIO(0x9888), 0x07834000 },
	2239	+ { _MMIO(0x9888), 0x09834000 },
	2240	+ { _MMIO(0x9888), 0x0b834000 },
	2241	+ { _MMIO(0x9888), 0x0d834000 },
	2242	+ { _MMIO(0x9888), 0x09848000 },
	2243	+ { _MMIO(0x9888), 0x0b84c000 },
	2244	+ { _MMIO(0x9888), 0x0d84c000 },
	2245	+ { _MMIO(0x9888), 0x0f84c000 },
	2246	+ { _MMIO(0x9888), 0x01848000 },
	2247	+ { _MMIO(0x9888), 0x0384c000 },
	2248	+ { _MMIO(0x9888), 0x0584c000 },
	2249	+ { _MMIO(0x9888), 0x07844000 },
	2250	+ { _MMIO(0x9888), 0x19808000 },
	2251	+ { _MMIO(0x9888), 0x1b80c000 },
	2252	+ { _MMIO(0x9888), 0x1d80c000 },
	2253	+ { _MMIO(0x9888), 0x1f80c000 },
	2254	+ { _MMIO(0x9888), 0x11808000 },
	2255	+ { _MMIO(0x9888), 0x1380c000 },
	2256	+ { _MMIO(0x9888), 0x1580c000 },
	2257	+ { _MMIO(0x9888), 0x17804000 },
	2258	+ { _MMIO(0x9888), 0x51800040 },
	2259	+ { _MMIO(0x9888), 0x43800400 },
	2260	+ { _MMIO(0x9888), 0x45800800 },
	2261	+ { _MMIO(0x9888), 0x53800000 },
	2262	+ { _MMIO(0x9888), 0x47800c62 },
	2263	+ { _MMIO(0x9888), 0x21800000 },
	2264	+ { _MMIO(0x9888), 0x31800000 },
	2265	+ { _MMIO(0x9888), 0x4d800000 },
	2266	+ { _MMIO(0x9888), 0x3f801042 },
	2267	+ { _MMIO(0x9888), 0x4f800000 },
	2268	+ { _MMIO(0x9888), 0x418014a4 },
	2269	+};
	2270	+
	2271	+static int
	2272	+get_hdc_and_sf_mux_config(struct drm_i915_private *dev_priv,
	2273	+ const struct i915_oa_reg **regs,
	2274	+ int *lens)
	2275	+{
	2276	+ int n = 0;
	2277	+
	2278	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	2279	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	2280	+
	2281	+ regs[n] = mux_config_hdc_and_sf;
	2282	+ lens[n] = ARRAY_SIZE(mux_config_hdc_and_sf);
	2283	+ n++;
	2284	+
	2285	+ return n;
	2286	+}
	2287	+
	2288	+static const struct i915_oa_reg b_counter_config_l3_1[] = {
	2289	+ { _MMIO(0x2740), 0x00000000 },
	2290	+ { _MMIO(0x2744), 0x00800000 },
	2291	+ { _MMIO(0x2710), 0x00000000 },
	2292	+ { _MMIO(0x2714), 0xf0800000 },
	2293	+ { _MMIO(0x2720), 0x00000000 },
	2294	+ { _MMIO(0x2724), 0xf0800000 },
	2295	+ { _MMIO(0x2770), 0x00100070 },
	2296	+ { _MMIO(0x2774), 0x0000fff1 },
	2297	+ { _MMIO(0x2778), 0x00014002 },
	2298	+ { _MMIO(0x277c), 0x0000c3ff },
	2299	+ { _MMIO(0x2780), 0x00010002 },
	2300	+ { _MMIO(0x2784), 0x0000c7ff },
	2301	+ { _MMIO(0x2788), 0x00004002 },
	2302	+ { _MMIO(0x278c), 0x0000d3ff },
	2303	+ { _MMIO(0x2790), 0x00100700 },
	2304	+ { _MMIO(0x2794), 0x0000ff1f },
	2305	+ { _MMIO(0x2798), 0x00001402 },
	2306	+ { _MMIO(0x279c), 0x0000fc3f },
	2307	+ { _MMIO(0x27a0), 0x00001002 },
	2308	+ { _MMIO(0x27a4), 0x0000fc7f },
	2309	+ { _MMIO(0x27a8), 0x00000402 },
	2310	+ { _MMIO(0x27ac), 0x0000fd3f },
	2311	+};
	2312	+
	2313	+static const struct i915_oa_reg flex_eu_config_l3_1[] = {
	2314	+ { _MMIO(0xe458), 0x00005004 },
	2315	+ { _MMIO(0xe558), 0x00010003 },
	2316	+ { _MMIO(0xe658), 0x00012011 },
	2317	+ { _MMIO(0xe758), 0x00015014 },
	2318	+ { _MMIO(0xe45c), 0x00051050 },
	2319	+ { _MMIO(0xe55c), 0x00053052 },
	2320	+ { _MMIO(0xe65c), 0x00055054 },
	2321	+};
	2322	+
	2323	+static const struct i915_oa_reg mux_config_l3_1[] = {
	2324	+ { _MMIO(0x9888), 0x10bf03da },
	2325	+ { _MMIO(0x9888), 0x14bf0001 },
	2326	+ { _MMIO(0x9888), 0x12980340 },
	2327	+ { _MMIO(0x9888), 0x12990340 },
	2328	+ { _MMIO(0x9888), 0x0cbf1187 },
	2329	+ { _MMIO(0x9888), 0x0ebf1205 },
	2330	+ { _MMIO(0x9888), 0x00bf0500 },
	2331	+ { _MMIO(0x9888), 0x02bf042b },
	2332	+ { _MMIO(0x9888), 0x04bf002c },
	2333	+ { _MMIO(0x9888), 0x0cdac000 },
	2334	+ { _MMIO(0x9888), 0x0edac000 },
	2335	+ { _MMIO(0x9888), 0x00da8000 },
	2336	+ { _MMIO(0x9888), 0x02dac000 },
	2337	+ { _MMIO(0x9888), 0x04da4000 },
	2338	+ { _MMIO(0x9888), 0x04983400 },
	2339	+ { _MMIO(0x9888), 0x10980000 },
	2340	+ { _MMIO(0x9888), 0x06990034 },
	2341	+ { _MMIO(0x9888), 0x10990000 },
	2342	+ { _MMIO(0x9888), 0x0c9dc000 },
	2343	+ { _MMIO(0x9888), 0x0e9dc000 },
	2344	+ { _MMIO(0x9888), 0x009d8000 },
	2345	+ { _MMIO(0x9888), 0x029dc000 },
	2346	+ { _MMIO(0x9888), 0x049d4000 },
	2347	+ { _MMIO(0x9888), 0x109f02a8 },
	2348	+ { _MMIO(0x9888), 0x0c9fa000 },
	2349	+ { _MMIO(0x9888), 0x0e9f00ba },
	2350	+ { _MMIO(0x9888), 0x0cb88000 },
	2351	+ { _MMIO(0x9888), 0x0cb95000 },
	2352	+ { _MMIO(0x9888), 0x0eb95000 },
	2353	+ { _MMIO(0x9888), 0x00b94000 },
	2354	+ { _MMIO(0x9888), 0x02b95000 },
	2355	+ { _MMIO(0x9888), 0x04b91000 },
	2356	+ { _MMIO(0x9888), 0x06b92000 },
	2357	+ { _MMIO(0x9888), 0x0cba4000 },
	2358	+ { _MMIO(0x9888), 0x0f88000f },
	2359	+ { _MMIO(0x9888), 0x03888000 },
	2360	+ { _MMIO(0x9888), 0x05888000 },
	2361	+ { _MMIO(0x9888), 0x07888000 },
	2362	+ { _MMIO(0x9888), 0x09888000 },
	2363	+ { _MMIO(0x9888), 0x0b888000 },
	2364	+ { _MMIO(0x9888), 0x0d880400 },
	2365	+ { _MMIO(0x9888), 0x258b800a },
	2366	+ { _MMIO(0x9888), 0x278b002a },
	2367	+ { _MMIO(0x9888), 0x238b5500 },
	2368	+ { _MMIO(0x9888), 0x198c4000 },
	2369	+ { _MMIO(0x9888), 0x1b8c0015 },
	2370	+ { _MMIO(0x9888), 0x038c4000 },
	2371	+ { _MMIO(0x9888), 0x058c4000 },
	2372	+ { _MMIO(0x9888), 0x078c4000 },
	2373	+ { _MMIO(0x9888), 0x098c4000 },
	2374	+ { _MMIO(0x9888), 0x0b8c4000 },
	2375	+ { _MMIO(0x9888), 0x0d8c4000 },
	2376	+ { _MMIO(0x9888), 0x0d8da000 },
	2377	+ { _MMIO(0x9888), 0x0f8da000 },
	2378	+ { _MMIO(0x9888), 0x018d8000 },
	2379	+ { _MMIO(0x9888), 0x038da000 },
	2380	+ { _MMIO(0x9888), 0x058da000 },
	2381	+ { _MMIO(0x9888), 0x078d2000 },
	2382	+ { _MMIO(0x9888), 0x2185800a },
	2383	+ { _MMIO(0x9888), 0x2385002a },
	2384	+ { _MMIO(0x9888), 0x1f85aa00 },
	2385	+ { _MMIO(0x9888), 0x1b830154 },
	2386	+ { _MMIO(0x9888), 0x03834000 },
	2387	+ { _MMIO(0x9888), 0x05834000 },
	2388	+ { _MMIO(0x9888), 0x07834000 },
	2389	+ { _MMIO(0x9888), 0x09834000 },
	2390	+ { _MMIO(0x9888), 0x0b834000 },
	2391	+ { _MMIO(0x9888), 0x0d834000 },
	2392	+ { _MMIO(0x9888), 0x0d84c000 },
	2393	+ { _MMIO(0x9888), 0x0f84c000 },
	2394	+ { _MMIO(0x9888), 0x01848000 },
	2395	+ { _MMIO(0x9888), 0x0384c000 },
	2396	+ { _MMIO(0x9888), 0x0584c000 },
	2397	+ { _MMIO(0x9888), 0x07844000 },
	2398	+ { _MMIO(0x9888), 0x1d80c000 },
	2399	+ { _MMIO(0x9888), 0x1f80c000 },
	2400	+ { _MMIO(0x9888), 0x11808000 },
	2401	+ { _MMIO(0x9888), 0x1380c000 },
	2402	+ { _MMIO(0x9888), 0x1580c000 },
	2403	+ { _MMIO(0x9888), 0x17804000 },
	2404	+ { _MMIO(0x9888), 0x53800000 },
	2405	+ { _MMIO(0x9888), 0x45800000 },
	2406	+ { _MMIO(0x9888), 0x47800000 },
	2407	+ { _MMIO(0x9888), 0x21800000 },
	2408	+ { _MMIO(0x9888), 0x31800000 },
	2409	+ { _MMIO(0x9888), 0x4d800000 },
	2410	+ { _MMIO(0x9888), 0x3f800000 },
	2411	+ { _MMIO(0x9888), 0x4f800000 },
	2412	+ { _MMIO(0x9888), 0x41800060 },
	2413	+};
	2414	+
	2415	+static int
	2416	+get_l3_1_mux_config(struct drm_i915_private *dev_priv,
	2417	+ const struct i915_oa_reg **regs,
	2418	+ int *lens)
	2419	+{
	2420	+ int n = 0;
	2421	+
	2422	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	2423	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	2424	+
	2425	+ regs[n] = mux_config_l3_1;
	2426	+ lens[n] = ARRAY_SIZE(mux_config_l3_1);
	2427	+ n++;
	2428	+
	2429	+ return n;
	2430	+}
	2431	+
	2432	+static const struct i915_oa_reg b_counter_config_l3_2[] = {
	2433	+ { _MMIO(0x2740), 0x00000000 },
	2434	+ { _MMIO(0x2744), 0x00800000 },
	2435	+ { _MMIO(0x2710), 0x00000000 },
	2436	+ { _MMIO(0x2714), 0xf0800000 },
	2437	+ { _MMIO(0x2720), 0x00000000 },
	2438	+ { _MMIO(0x2724), 0xf0800000 },
	2439	+ { _MMIO(0x2770), 0x00100070 },
	2440	+ { _MMIO(0x2774), 0x0000fff1 },
	2441	+ { _MMIO(0x2778), 0x00014002 },
	2442	+ { _MMIO(0x277c), 0x0000c3ff },
	2443	+ { _MMIO(0x2780), 0x00010002 },
	2444	+ { _MMIO(0x2784), 0x0000c7ff },
	2445	+ { _MMIO(0x2788), 0x00004002 },
	2446	+ { _MMIO(0x278c), 0x0000d3ff },
	2447	+ { _MMIO(0x2790), 0x00100700 },
	2448	+ { _MMIO(0x2794), 0x0000ff1f },
	2449	+ { _MMIO(0x2798), 0x00001402 },
	2450	+ { _MMIO(0x279c), 0x0000fc3f },
	2451	+ { _MMIO(0x27a0), 0x00001002 },
	2452	+ { _MMIO(0x27a4), 0x0000fc7f },
	2453	+ { _MMIO(0x27a8), 0x00000402 },
	2454	+ { _MMIO(0x27ac), 0x0000fd3f },
	2455	+};
	2456	+
	2457	+static const struct i915_oa_reg flex_eu_config_l3_2[] = {
	2458	+ { _MMIO(0xe458), 0x00005004 },
	2459	+ { _MMIO(0xe558), 0x00010003 },
	2460	+ { _MMIO(0xe658), 0x00012011 },
	2461	+ { _MMIO(0xe758), 0x00015014 },
	2462	+ { _MMIO(0xe45c), 0x00051050 },
	2463	+ { _MMIO(0xe55c), 0x00053052 },
	2464	+ { _MMIO(0xe65c), 0x00055054 },
	2465	+};
	2466	+
	2467	+static const struct i915_oa_reg mux_config_l3_2[] = {
	2468	+ { _MMIO(0x9888), 0x103f03da },
	2469	+ { _MMIO(0x9888), 0x143f0001 },
	2470	+ { _MMIO(0x9888), 0x12180340 },
	2471	+ { _MMIO(0x9888), 0x12190340 },
	2472	+ { _MMIO(0x9888), 0x0c3f1187 },
	2473	+ { _MMIO(0x9888), 0x0e3f1205 },
	2474	+ { _MMIO(0x9888), 0x003f0500 },
	2475	+ { _MMIO(0x9888), 0x023f042b },
	2476	+ { _MMIO(0x9888), 0x043f002c },
	2477	+ { _MMIO(0x9888), 0x0c5ac000 },
	2478	+ { _MMIO(0x9888), 0x0e5ac000 },
	2479	+ { _MMIO(0x9888), 0x005a8000 },
	2480	+ { _MMIO(0x9888), 0x025ac000 },
	2481	+ { _MMIO(0x9888), 0x045a4000 },
	2482	+ { _MMIO(0x9888), 0x04183400 },
	2483	+ { _MMIO(0x9888), 0x10180000 },
	2484	+ { _MMIO(0x9888), 0x06190034 },
	2485	+ { _MMIO(0x9888), 0x10190000 },
	2486	+ { _MMIO(0x9888), 0x0c1dc000 },
	2487	+ { _MMIO(0x9888), 0x0e1dc000 },
	2488	+ { _MMIO(0x9888), 0x001d8000 },
	2489	+ { _MMIO(0x9888), 0x021dc000 },
	2490	+ { _MMIO(0x9888), 0x041d4000 },
	2491	+ { _MMIO(0x9888), 0x101f02a8 },
	2492	+ { _MMIO(0x9888), 0x0c1fa000 },
	2493	+ { _MMIO(0x9888), 0x0e1f00ba },
	2494	+ { _MMIO(0x9888), 0x0c388000 },
	2495	+ { _MMIO(0x9888), 0x0c395000 },
	2496	+ { _MMIO(0x9888), 0x0e395000 },
	2497	+ { _MMIO(0x9888), 0x00394000 },
	2498	+ { _MMIO(0x9888), 0x02395000 },
	2499	+ { _MMIO(0x9888), 0x04391000 },
	2500	+ { _MMIO(0x9888), 0x06392000 },
	2501	+ { _MMIO(0x9888), 0x0c3a4000 },
	2502	+ { _MMIO(0x9888), 0x1b8aa800 },
	2503	+ { _MMIO(0x9888), 0x1d8a0002 },
	2504	+ { _MMIO(0x9888), 0x038a8000 },
	2505	+ { _MMIO(0x9888), 0x058a8000 },
	2506	+ { _MMIO(0x9888), 0x078a8000 },
	2507	+ { _MMIO(0x9888), 0x098a8000 },
	2508	+ { _MMIO(0x9888), 0x0b8a8000 },
	2509	+ { _MMIO(0x9888), 0x0d8a8000 },
	2510	+ { _MMIO(0x9888), 0x258b4005 },
	2511	+ { _MMIO(0x9888), 0x278b0015 },
	2512	+ { _MMIO(0x9888), 0x238b2a80 },
	2513	+ { _MMIO(0x9888), 0x2185800a },
	2514	+ { _MMIO(0x9888), 0x2385002a },
	2515	+ { _MMIO(0x9888), 0x1f85aa00 },
	2516	+ { _MMIO(0x9888), 0x1b830154 },
	2517	+ { _MMIO(0x9888), 0x03834000 },
	2518	+ { _MMIO(0x9888), 0x05834000 },
	2519	+ { _MMIO(0x9888), 0x07834000 },
	2520	+ { _MMIO(0x9888), 0x09834000 },
	2521	+ { _MMIO(0x9888), 0x0b834000 },
	2522	+ { _MMIO(0x9888), 0x0d834000 },
	2523	+ { _MMIO(0x9888), 0x0d84c000 },
	2524	+ { _MMIO(0x9888), 0x0f84c000 },
	2525	+ { _MMIO(0x9888), 0x01848000 },
	2526	+ { _MMIO(0x9888), 0x0384c000 },
	2527	+ { _MMIO(0x9888), 0x0584c000 },
	2528	+ { _MMIO(0x9888), 0x07844000 },
	2529	+ { _MMIO(0x9888), 0x1d80c000 },
	2530	+ { _MMIO(0x9888), 0x1f80c000 },
	2531	+ { _MMIO(0x9888), 0x11808000 },
	2532	+ { _MMIO(0x9888), 0x1380c000 },
	2533	+ { _MMIO(0x9888), 0x1580c000 },
	2534	+ { _MMIO(0x9888), 0x17804000 },
	2535	+ { _MMIO(0x9888), 0x53800000 },
	2536	+ { _MMIO(0x9888), 0x45800000 },
	2537	+ { _MMIO(0x9888), 0x47800000 },
	2538	+ { _MMIO(0x9888), 0x21800000 },
	2539	+ { _MMIO(0x9888), 0x31800000 },
	2540	+ { _MMIO(0x9888), 0x4d800000 },
	2541	+ { _MMIO(0x9888), 0x3f800000 },
	2542	+ { _MMIO(0x9888), 0x4f800000 },
	2543	+ { _MMIO(0x9888), 0x41800060 },
	2544	+};
	2545	+
	2546	+static int
	2547	+get_l3_2_mux_config(struct drm_i915_private *dev_priv,
	2548	+ const struct i915_oa_reg **regs,
	2549	+ int *lens)
	2550	+{
	2551	+ int n = 0;
	2552	+
	2553	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	2554	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	2555	+
	2556	+ regs[n] = mux_config_l3_2;
	2557	+ lens[n] = ARRAY_SIZE(mux_config_l3_2);
	2558	+ n++;
	2559	+
	2560	+ return n;
	2561	+}
	2562	+
	2563	+static const struct i915_oa_reg b_counter_config_l3_3[] = {
	2564	+ { _MMIO(0x2740), 0x00000000 },
	2565	+ { _MMIO(0x2744), 0x00800000 },
	2566	+ { _MMIO(0x2710), 0x00000000 },
	2567	+ { _MMIO(0x2714), 0xf0800000 },
	2568	+ { _MMIO(0x2720), 0x00000000 },
	2569	+ { _MMIO(0x2724), 0xf0800000 },
	2570	+ { _MMIO(0x2770), 0x00100070 },
	2571	+ { _MMIO(0x2774), 0x0000fff1 },
	2572	+ { _MMIO(0x2778), 0x00014002 },
	2573	+ { _MMIO(0x277c), 0x0000c3ff },
	2574	+ { _MMIO(0x2780), 0x00010002 },
	2575	+ { _MMIO(0x2784), 0x0000c7ff },
	2576	+ { _MMIO(0x2788), 0x00004002 },
	2577	+ { _MMIO(0x278c), 0x0000d3ff },
	2578	+ { _MMIO(0x2790), 0x00100700 },
	2579	+ { _MMIO(0x2794), 0x0000ff1f },
	2580	+ { _MMIO(0x2798), 0x00001402 },
	2581	+ { _MMIO(0x279c), 0x0000fc3f },
	2582	+ { _MMIO(0x27a0), 0x00001002 },
	2583	+ { _MMIO(0x27a4), 0x0000fc7f },
	2584	+ { _MMIO(0x27a8), 0x00000402 },
	2585	+ { _MMIO(0x27ac), 0x0000fd3f },
	2586	+};
	2587	+
	2588	+static const struct i915_oa_reg flex_eu_config_l3_3[] = {
	2589	+ { _MMIO(0xe458), 0x00005004 },
	2590	+ { _MMIO(0xe558), 0x00010003 },
	2591	+ { _MMIO(0xe658), 0x00012011 },
	2592	+ { _MMIO(0xe758), 0x00015014 },
	2593	+ { _MMIO(0xe45c), 0x00051050 },
	2594	+ { _MMIO(0xe55c), 0x00053052 },
	2595	+ { _MMIO(0xe65c), 0x00055054 },
	2596	+};
	2597	+
	2598	+static const struct i915_oa_reg mux_config_l3_3[] = {
	2599	+ { _MMIO(0x9888), 0x121b0340 },
	2600	+ { _MMIO(0x9888), 0x103f0274 },
	2601	+ { _MMIO(0x9888), 0x123f0000 },
	2602	+ { _MMIO(0x9888), 0x129b0340 },
	2603	+ { _MMIO(0x9888), 0x10bf0274 },
	2604	+ { _MMIO(0x9888), 0x12bf0000 },
	2605	+ { _MMIO(0x9888), 0x041b3400 },
	2606	+ { _MMIO(0x9888), 0x101b0000 },
	2607	+ { _MMIO(0x9888), 0x045c8000 },
	2608	+ { _MMIO(0x9888), 0x0a3d4000 },
	2609	+ { _MMIO(0x9888), 0x003f0080 },
	2610	+ { _MMIO(0x9888), 0x023f0793 },
	2611	+ { _MMIO(0x9888), 0x043f0014 },
	2612	+ { _MMIO(0x9888), 0x04588000 },
	2613	+ { _MMIO(0x9888), 0x005a8000 },
	2614	+ { _MMIO(0x9888), 0x025ac000 },
	2615	+ { _MMIO(0x9888), 0x045a4000 },
	2616	+ { _MMIO(0x9888), 0x0a5b4000 },
	2617	+ { _MMIO(0x9888), 0x001d8000 },
	2618	+ { _MMIO(0x9888), 0x021dc000 },
	2619	+ { _MMIO(0x9888), 0x041d4000 },
	2620	+ { _MMIO(0x9888), 0x0c1fa000 },
	2621	+ { _MMIO(0x9888), 0x0e1f002a },
	2622	+ { _MMIO(0x9888), 0x0a384000 },
	2623	+ { _MMIO(0x9888), 0x00394000 },
	2624	+ { _MMIO(0x9888), 0x02395000 },
	2625	+ { _MMIO(0x9888), 0x04399000 },
	2626	+ { _MMIO(0x9888), 0x069b0034 },
	2627	+ { _MMIO(0x9888), 0x109b0000 },
	2628	+ { _MMIO(0x9888), 0x06dc4000 },
	2629	+ { _MMIO(0x9888), 0x0cbd4000 },
	2630	+ { _MMIO(0x9888), 0x0cbf0981 },
	2631	+ { _MMIO(0x9888), 0x0ebf0a0f },
	2632	+ { _MMIO(0x9888), 0x06d84000 },
	2633	+ { _MMIO(0x9888), 0x0cdac000 },
	2634	+ { _MMIO(0x9888), 0x0edac000 },
	2635	+ { _MMIO(0x9888), 0x0cdb4000 },
	2636	+ { _MMIO(0x9888), 0x0c9dc000 },
	2637	+ { _MMIO(0x9888), 0x0e9dc000 },
	2638	+ { _MMIO(0x9888), 0x109f02a8 },
	2639	+ { _MMIO(0x9888), 0x0e9f0080 },
	2640	+ { _MMIO(0x9888), 0x0cb84000 },
	2641	+ { _MMIO(0x9888), 0x0cb95000 },
	2642	+ { _MMIO(0x9888), 0x0eb95000 },
	2643	+ { _MMIO(0x9888), 0x06b92000 },
	2644	+ { _MMIO(0x9888), 0x0f88000f },
	2645	+ { _MMIO(0x9888), 0x0d880400 },
	2646	+ { _MMIO(0x9888), 0x038a8000 },
	2647	+ { _MMIO(0x9888), 0x058a8000 },
	2648	+ { _MMIO(0x9888), 0x078a8000 },
	2649	+ { _MMIO(0x9888), 0x098a8000 },
	2650	+ { _MMIO(0x9888), 0x0b8a8000 },
	2651	+ { _MMIO(0x9888), 0x258b8009 },
	2652	+ { _MMIO(0x9888), 0x278b002a },
	2653	+ { _MMIO(0x9888), 0x238b2a80 },
	2654	+ { _MMIO(0x9888), 0x198c4000 },
	2655	+ { _MMIO(0x9888), 0x1b8c0015 },
	2656	+ { _MMIO(0x9888), 0x0d8c4000 },
	2657	+ { _MMIO(0x9888), 0x0d8da000 },
	2658	+ { _MMIO(0x9888), 0x0f8da000 },
	2659	+ { _MMIO(0x9888), 0x078d2000 },
	2660	+ { _MMIO(0x9888), 0x2185800a },
	2661	+ { _MMIO(0x9888), 0x2385002a },
	2662	+ { _MMIO(0x9888), 0x1f85aa00 },
	2663	+ { _MMIO(0x9888), 0x1b830154 },
	2664	+ { _MMIO(0x9888), 0x03834000 },
	2665	+ { _MMIO(0x9888), 0x05834000 },
	2666	+ { _MMIO(0x9888), 0x07834000 },
	2667	+ { _MMIO(0x9888), 0x09834000 },
	2668	+ { _MMIO(0x9888), 0x0b834000 },
	2669	+ { _MMIO(0x9888), 0x0d834000 },
	2670	+ { _MMIO(0x9888), 0x0d84c000 },
	2671	+ { _MMIO(0x9888), 0x0f84c000 },
	2672	+ { _MMIO(0x9888), 0x01848000 },
	2673	+ { _MMIO(0x9888), 0x0384c000 },
	2674	+ { _MMIO(0x9888), 0x0584c000 },
	2675	+ { _MMIO(0x9888), 0x07844000 },
	2676	+ { _MMIO(0x9888), 0x1d80c000 },
	2677	+ { _MMIO(0x9888), 0x1f80c000 },
	2678	+ { _MMIO(0x9888), 0x11808000 },
	2679	+ { _MMIO(0x9888), 0x1380c000 },
	2680	+ { _MMIO(0x9888), 0x1580c000 },
	2681	+ { _MMIO(0x9888), 0x17804000 },
	2682	+ { _MMIO(0x9888), 0x53800000 },
	2683	+ { _MMIO(0x9888), 0x45800c00 },
	2684	+ { _MMIO(0x9888), 0x47800c63 },
	2685	+ { _MMIO(0x9888), 0x21800000 },
	2686	+ { _MMIO(0x9888), 0x31800000 },
	2687	+ { _MMIO(0x9888), 0x4d800000 },
	2688	+ { _MMIO(0x9888), 0x3f8014a5 },
	2689	+ { _MMIO(0x9888), 0x4f800000 },
	2690	+ { _MMIO(0x9888), 0x41800045 },
	2691	+};
	2692	+
	2693	+static int
	2694	+get_l3_3_mux_config(struct drm_i915_private *dev_priv,
	2695	+ const struct i915_oa_reg **regs,
	2696	+ int *lens)
	2697	+{
	2698	+ int n = 0;
	2699	+
	2700	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	2701	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	2702	+
	2703	+ regs[n] = mux_config_l3_3;
	2704	+ lens[n] = ARRAY_SIZE(mux_config_l3_3);
	2705	+ n++;
	2706	+
	2707	+ return n;
	2708	+}
	2709	+
	2710	+static const struct i915_oa_reg b_counter_config_l3_4[] = {
	2711	+ { _MMIO(0x2740), 0x00000000 },
	2712	+ { _MMIO(0x2744), 0x00800000 },
	2713	+ { _MMIO(0x2710), 0x00000000 },
	2714	+ { _MMIO(0x2714), 0xf0800000 },
	2715	+ { _MMIO(0x2720), 0x00000000 },
	2716	+ { _MMIO(0x2724), 0xf0800000 },
	2717	+ { _MMIO(0x2770), 0x00100070 },
	2718	+ { _MMIO(0x2774), 0x0000fff1 },
	2719	+ { _MMIO(0x2778), 0x00014002 },
	2720	+ { _MMIO(0x277c), 0x0000c3ff },
	2721	+ { _MMIO(0x2780), 0x00010002 },
	2722	+ { _MMIO(0x2784), 0x0000c7ff },
	2723	+ { _MMIO(0x2788), 0x00004002 },
	2724	+ { _MMIO(0x278c), 0x0000d3ff },
	2725	+ { _MMIO(0x2790), 0x00100700 },
	2726	+ { _MMIO(0x2794), 0x0000ff1f },
	2727	+ { _MMIO(0x2798), 0x00001402 },
	2728	+ { _MMIO(0x279c), 0x0000fc3f },
	2729	+ { _MMIO(0x27a0), 0x00001002 },
	2730	+ { _MMIO(0x27a4), 0x0000fc7f },
	2731	+ { _MMIO(0x27a8), 0x00000402 },
	2732	+ { _MMIO(0x27ac), 0x0000fd3f },
	2733	+};
	2734	+
	2735	+static const struct i915_oa_reg flex_eu_config_l3_4[] = {
	2736	+ { _MMIO(0xe458), 0x00005004 },
	2737	+ { _MMIO(0xe558), 0x00010003 },
	2738	+ { _MMIO(0xe658), 0x00012011 },
	2739	+ { _MMIO(0xe758), 0x00015014 },
	2740	+ { _MMIO(0xe45c), 0x00051050 },
	2741	+ { _MMIO(0xe55c), 0x00053052 },
	2742	+ { _MMIO(0xe65c), 0x00055054 },
	2743	+};
	2744	+
	2745	+static const struct i915_oa_reg mux_config_l3_4[] = {
	2746	+ { _MMIO(0x9888), 0x121a0340 },
	2747	+ { _MMIO(0x9888), 0x103f0017 },
	2748	+ { _MMIO(0x9888), 0x123f0020 },
	2749	+ { _MMIO(0x9888), 0x129a0340 },
	2750	+ { _MMIO(0x9888), 0x10bf0017 },
	2751	+ { _MMIO(0x9888), 0x12bf0020 },
	2752	+ { _MMIO(0x9888), 0x041a3400 },
	2753	+ { _MMIO(0x9888), 0x101a0000 },
	2754	+ { _MMIO(0x9888), 0x043b8000 },
	2755	+ { _MMIO(0x9888), 0x0a3e0010 },
	2756	+ { _MMIO(0x9888), 0x003f0200 },
	2757	+ { _MMIO(0x9888), 0x023f0113 },
	2758	+ { _MMIO(0x9888), 0x043f0014 },
	2759	+ { _MMIO(0x9888), 0x02592000 },
	2760	+ { _MMIO(0x9888), 0x005a8000 },
	2761	+ { _MMIO(0x9888), 0x025ac000 },
	2762	+ { _MMIO(0x9888), 0x045a4000 },
	2763	+ { _MMIO(0x9888), 0x0a1c8000 },
	2764	+ { _MMIO(0x9888), 0x001d8000 },
	2765	+ { _MMIO(0x9888), 0x021dc000 },
	2766	+ { _MMIO(0x9888), 0x041d4000 },
	2767	+ { _MMIO(0x9888), 0x0a1e8000 },
	2768	+ { _MMIO(0x9888), 0x0c1fa000 },
	2769	+ { _MMIO(0x9888), 0x0e1f001a },
	2770	+ { _MMIO(0x9888), 0x00394000 },
	2771	+ { _MMIO(0x9888), 0x02395000 },
	2772	+ { _MMIO(0x9888), 0x04391000 },
	2773	+ { _MMIO(0x9888), 0x069a0034 },
	2774	+ { _MMIO(0x9888), 0x109a0000 },
	2775	+ { _MMIO(0x9888), 0x06bb4000 },
	2776	+ { _MMIO(0x9888), 0x0abe0040 },
	2777	+ { _MMIO(0x9888), 0x0cbf0984 },
	2778	+ { _MMIO(0x9888), 0x0ebf0a02 },
	2779	+ { _MMIO(0x9888), 0x02d94000 },
	2780	+ { _MMIO(0x9888), 0x0cdac000 },
	2781	+ { _MMIO(0x9888), 0x0edac000 },
	2782	+ { _MMIO(0x9888), 0x0c9c0400 },
	2783	+ { _MMIO(0x9888), 0x0c9dc000 },
	2784	+ { _MMIO(0x9888), 0x0e9dc000 },
	2785	+ { _MMIO(0x9888), 0x0c9e0400 },
	2786	+ { _MMIO(0x9888), 0x109f02a8 },
	2787	+ { _MMIO(0x9888), 0x0e9f0040 },
	2788	+ { _MMIO(0x9888), 0x0cb95000 },
	2789	+ { _MMIO(0x9888), 0x0eb95000 },
	2790	+ { _MMIO(0x9888), 0x0f88000f },
	2791	+ { _MMIO(0x9888), 0x0d880400 },
	2792	+ { _MMIO(0x9888), 0x038a8000 },
	2793	+ { _MMIO(0x9888), 0x058a8000 },
	2794	+ { _MMIO(0x9888), 0x078a8000 },
	2795	+ { _MMIO(0x9888), 0x098a8000 },
	2796	+ { _MMIO(0x9888), 0x0b8a8000 },
	2797	+ { _MMIO(0x9888), 0x258b8009 },
	2798	+ { _MMIO(0x9888), 0x278b002a },
	2799	+ { _MMIO(0x9888), 0x238b2a80 },
	2800	+ { _MMIO(0x9888), 0x198c4000 },
	2801	+ { _MMIO(0x9888), 0x1b8c0015 },
	2802	+ { _MMIO(0x9888), 0x0d8c4000 },
	2803	+ { _MMIO(0x9888), 0x0d8da000 },
	2804	+ { _MMIO(0x9888), 0x0f8da000 },
	2805	+ { _MMIO(0x9888), 0x078d2000 },
	2806	+ { _MMIO(0x9888), 0x2185800a },
	2807	+ { _MMIO(0x9888), 0x2385002a },
	2808	+ { _MMIO(0x9888), 0x1f85aa00 },
	2809	+ { _MMIO(0x9888), 0x1b830154 },
	2810	+ { _MMIO(0x9888), 0x03834000 },
	2811	+ { _MMIO(0x9888), 0x05834000 },
	2812	+ { _MMIO(0x9888), 0x07834000 },
	2813	+ { _MMIO(0x9888), 0x09834000 },
	2814	+ { _MMIO(0x9888), 0x0b834000 },
	2815	+ { _MMIO(0x9888), 0x0d834000 },
	2816	+ { _MMIO(0x9888), 0x0d84c000 },
	2817	+ { _MMIO(0x9888), 0x0f84c000 },
	2818	+ { _MMIO(0x9888), 0x01848000 },
	2819	+ { _MMIO(0x9888), 0x0384c000 },
	2820	+ { _MMIO(0x9888), 0x0584c000 },
	2821	+ { _MMIO(0x9888), 0x07844000 },
	2822	+ { _MMIO(0x9888), 0x1d80c000 },
	2823	+ { _MMIO(0x9888), 0x1f80c000 },
	2824	+ { _MMIO(0x9888), 0x11808000 },
	2825	+ { _MMIO(0x9888), 0x1380c000 },
	2826	+ { _MMIO(0x9888), 0x1580c000 },
	2827	+ { _MMIO(0x9888), 0x17804000 },
	2828	+ { _MMIO(0x9888), 0x53800000 },
	2829	+ { _MMIO(0x9888), 0x45800800 },
	2830	+ { _MMIO(0x9888), 0x47800842 },
	2831	+ { _MMIO(0x9888), 0x21800000 },
	2832	+ { _MMIO(0x9888), 0x31800000 },
	2833	+ { _MMIO(0x9888), 0x4d800000 },
	2834	+ { _MMIO(0x9888), 0x3f801084 },
	2835	+ { _MMIO(0x9888), 0x4f800000 },
	2836	+ { _MMIO(0x9888), 0x41800044 },
	2837	+};
	2838	+
	2839	+static int
	2840	+get_l3_4_mux_config(struct drm_i915_private *dev_priv,
	2841	+ const struct i915_oa_reg **regs,
	2842	+ int *lens)
	2843	+{
	2844	+ int n = 0;
	2845	+
	2846	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	2847	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	2848	+
	2849	+ regs[n] = mux_config_l3_4;
	2850	+ lens[n] = ARRAY_SIZE(mux_config_l3_4);
	2851	+ n++;
	2852	+
	2853	+ return n;
	2854	+}
	2855	+
	2856	+static const struct i915_oa_reg b_counter_config_rasterizer_and_pixel_backend[] = {
	2857	+ { _MMIO(0x2740), 0x00000000 },
	2858	+ { _MMIO(0x2744), 0x00800000 },
	2859	+ { _MMIO(0x2710), 0x00000000 },
	2860	+ { _MMIO(0x2714), 0xf0800000 },
	2861	+ { _MMIO(0x2720), 0x00000000 },
	2862	+ { _MMIO(0x2724), 0x30800000 },
	2863	+ { _MMIO(0x2770), 0x00006000 },
	2864	+ { _MMIO(0x2774), 0x0000f3ff },
	2865	+ { _MMIO(0x2778), 0x00001800 },
	2866	+ { _MMIO(0x277c), 0x0000fcff },
	2867	+ { _MMIO(0x2780), 0x00000600 },
	2868	+ { _MMIO(0x2784), 0x0000ff3f },
	2869	+ { _MMIO(0x2788), 0x00000180 },
	2870	+ { _MMIO(0x278c), 0x0000ffcf },
	2871	+ { _MMIO(0x2790), 0x00000060 },
	2872	+ { _MMIO(0x2794), 0x0000fff3 },
	2873	+ { _MMIO(0x2798), 0x00000018 },
	2874	+ { _MMIO(0x279c), 0x0000fffc },
	2875	+};
	2876	+
	2877	+static const struct i915_oa_reg flex_eu_config_rasterizer_and_pixel_backend[] = {
	2878	+ { _MMIO(0xe458), 0x00005004 },
	2879	+ { _MMIO(0xe558), 0x00010003 },
	2880	+ { _MMIO(0xe658), 0x00012011 },
	2881	+ { _MMIO(0xe758), 0x00015014 },
	2882	+ { _MMIO(0xe45c), 0x00051050 },
	2883	+ { _MMIO(0xe55c), 0x00053052 },
	2884	+ { _MMIO(0xe65c), 0x00055054 },
	2885	+};
	2886	+
	2887	+static const struct i915_oa_reg mux_config_rasterizer_and_pixel_backend[] = {
	2888	+ { _MMIO(0x9888), 0x143b000e },
	2889	+ { _MMIO(0x9888), 0x043c55c0 },
	2890	+ { _MMIO(0x9888), 0x0a1e0280 },
	2891	+ { _MMIO(0x9888), 0x0c1e0408 },
	2892	+ { _MMIO(0x9888), 0x10390000 },
	2893	+ { _MMIO(0x9888), 0x12397a1f },
	2894	+ { _MMIO(0x9888), 0x14bb000e },
	2895	+ { _MMIO(0x9888), 0x04bc5000 },
	2896	+ { _MMIO(0x9888), 0x0a9e0296 },
	2897	+ { _MMIO(0x9888), 0x0c9e0008 },
	2898	+ { _MMIO(0x9888), 0x10b90000 },
	2899	+ { _MMIO(0x9888), 0x12b97a1f },
	2900	+ { _MMIO(0x9888), 0x063b0042 },
	2901	+ { _MMIO(0x9888), 0x103b0000 },
	2902	+ { _MMIO(0x9888), 0x083c0000 },
	2903	+ { _MMIO(0x9888), 0x0a3e0040 },
	2904	+ { _MMIO(0x9888), 0x043f8000 },
	2905	+ { _MMIO(0x9888), 0x02594000 },
	2906	+ { _MMIO(0x9888), 0x045a8000 },
	2907	+ { _MMIO(0x9888), 0x0c1c0400 },
	2908	+ { _MMIO(0x9888), 0x041d8000 },
	2909	+ { _MMIO(0x9888), 0x081e02c0 },
	2910	+ { _MMIO(0x9888), 0x0e1e0000 },
	2911	+ { _MMIO(0x9888), 0x0c1fa800 },
	2912	+ { _MMIO(0x9888), 0x0e1f0260 },
	2913	+ { _MMIO(0x9888), 0x101f0014 },
	2914	+ { _MMIO(0x9888), 0x003905e0 },
	2915	+ { _MMIO(0x9888), 0x06390bc0 },
	2916	+ { _MMIO(0x9888), 0x02390018 },
	2917	+ { _MMIO(0x9888), 0x04394000 },
	2918	+ { _MMIO(0x9888), 0x04bb0042 },
	2919	+ { _MMIO(0x9888), 0x10bb0000 },
	2920	+ { _MMIO(0x9888), 0x02bc05c0 },
	2921	+ { _MMIO(0x9888), 0x08bc0000 },
	2922	+ { _MMIO(0x9888), 0x0abe0004 },
	2923	+ { _MMIO(0x9888), 0x02bf8000 },
	2924	+ { _MMIO(0x9888), 0x02d91000 },
	2925	+ { _MMIO(0x9888), 0x02da8000 },
	2926	+ { _MMIO(0x9888), 0x089c8000 },
	2927	+ { _MMIO(0x9888), 0x029d8000 },
	2928	+ { _MMIO(0x9888), 0x089e8000 },
	2929	+ { _MMIO(0x9888), 0x0e9e0000 },
	2930	+ { _MMIO(0x9888), 0x0e9fa806 },
	2931	+ { _MMIO(0x9888), 0x109f0142 },
	2932	+ { _MMIO(0x9888), 0x08b90617 },
	2933	+ { _MMIO(0x9888), 0x0ab90be0 },
	2934	+ { _MMIO(0x9888), 0x02b94000 },
	2935	+ { _MMIO(0x9888), 0x0d88f000 },
	2936	+ { _MMIO(0x9888), 0x0f88000c },
	2937	+ { _MMIO(0x9888), 0x07888000 },
	2938	+ { _MMIO(0x9888), 0x09888000 },
	2939	+ { _MMIO(0x9888), 0x018a8000 },
	2940	+ { _MMIO(0x9888), 0x0f8a8000 },
	2941	+ { _MMIO(0x9888), 0x1b8a2800 },
	2942	+ { _MMIO(0x9888), 0x038a8000 },
	2943	+ { _MMIO(0x9888), 0x058a8000 },
	2944	+ { _MMIO(0x9888), 0x0b8a8000 },
	2945	+ { _MMIO(0x9888), 0x0d8a8000 },
	2946	+ { _MMIO(0x9888), 0x238b52a0 },
	2947	+ { _MMIO(0x9888), 0x258b6a95 },
	2948	+ { _MMIO(0x9888), 0x278b0029 },
	2949	+ { _MMIO(0x9888), 0x178c2000 },
	2950	+ { _MMIO(0x9888), 0x198c1500 },
	2951	+ { _MMIO(0x9888), 0x1b8c0014 },
	2952	+ { _MMIO(0x9888), 0x078c4000 },
	2953	+ { _MMIO(0x9888), 0x098c4000 },
	2954	+ { _MMIO(0x9888), 0x098da000 },
	2955	+ { _MMIO(0x9888), 0x0b8da000 },
	2956	+ { _MMIO(0x9888), 0x0f8da000 },
	2957	+ { _MMIO(0x9888), 0x038d8000 },
	2958	+ { _MMIO(0x9888), 0x058d2000 },
	2959	+ { _MMIO(0x9888), 0x1f85aa80 },
	2960	+ { _MMIO(0x9888), 0x2185aaaa },
	2961	+ { _MMIO(0x9888), 0x2385002a },
	2962	+ { _MMIO(0x9888), 0x01834000 },
	2963	+ { _MMIO(0x9888), 0x0f834000 },
	2964	+ { _MMIO(0x9888), 0x19835400 },
	2965	+ { _MMIO(0x9888), 0x1b830155 },
	2966	+ { _MMIO(0x9888), 0x03834000 },
	2967	+ { _MMIO(0x9888), 0x05834000 },
	2968	+ { _MMIO(0x9888), 0x07834000 },
	2969	+ { _MMIO(0x9888), 0x09834000 },
	2970	+ { _MMIO(0x9888), 0x0b834000 },
	2971	+ { _MMIO(0x9888), 0x0d834000 },
	2972	+ { _MMIO(0x9888), 0x0184c000 },
	2973	+ { _MMIO(0x9888), 0x0784c000 },
	2974	+ { _MMIO(0x9888), 0x0984c000 },
	2975	+ { _MMIO(0x9888), 0x0b84c000 },
	2976	+ { _MMIO(0x9888), 0x0d84c000 },
	2977	+ { _MMIO(0x9888), 0x0f84c000 },
	2978	+ { _MMIO(0x9888), 0x0384c000 },
	2979	+ { _MMIO(0x9888), 0x0584c000 },
	2980	+ { _MMIO(0x9888), 0x1180c000 },
	2981	+ { _MMIO(0x9888), 0x1780c000 },
	2982	+ { _MMIO(0x9888), 0x1980c000 },
	2983	+ { _MMIO(0x9888), 0x1b80c000 },
	2984	+ { _MMIO(0x9888), 0x1d80c000 },
	2985	+ { _MMIO(0x9888), 0x1f80c000 },
	2986	+ { _MMIO(0x9888), 0x1380c000 },
	2987	+ { _MMIO(0x9888), 0x1580c000 },
	2988	+ { _MMIO(0x9888), 0x4d800444 },
	2989	+ { _MMIO(0x9888), 0x3d800000 },
	2990	+ { _MMIO(0x9888), 0x4f804000 },
	2991	+ { _MMIO(0x9888), 0x43801080 },
	2992	+ { _MMIO(0x9888), 0x51800000 },
	2993	+ { _MMIO(0x9888), 0x45800084 },
	2994	+ { _MMIO(0x9888), 0x53800044 },
	2995	+ { _MMIO(0x9888), 0x47801080 },
	2996	+ { _MMIO(0x9888), 0x21800000 },
	2997	+ { _MMIO(0x9888), 0x31800000 },
	2998	+ { _MMIO(0x9888), 0x3f800000 },
	2999	+ { _MMIO(0x9888), 0x41800840 },
	3000	+};
	3001	+
	3002	+static int
	3003	+get_rasterizer_and_pixel_backend_mux_config(struct drm_i915_private *dev_priv,
	3004	+ const struct i915_oa_reg **regs,
	3005	+ int *lens)
	3006	+{
	3007	+ int n = 0;
	3008	+
	3009	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	3010	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	3011	+
	3012	+ regs[n] = mux_config_rasterizer_and_pixel_backend;
	3013	+ lens[n] = ARRAY_SIZE(mux_config_rasterizer_and_pixel_backend);
	3014	+ n++;
	3015	+
	3016	+ return n;
	3017	+}
	3018	+
	3019	+static const struct i915_oa_reg b_counter_config_sampler_1[] = {
	3020	+ { _MMIO(0x2740), 0x00000000 },
	3021	+ { _MMIO(0x2744), 0x00800000 },
	3022	+ { _MMIO(0x2710), 0x00000000 },
	3023	+ { _MMIO(0x2714), 0x70800000 },
	3024	+ { _MMIO(0x2720), 0x00000000 },
	3025	+ { _MMIO(0x2724), 0x00800000 },
	3026	+ { _MMIO(0x2770), 0x0000c000 },
	3027	+ { _MMIO(0x2774), 0x0000e7ff },
	3028	+ { _MMIO(0x2778), 0x00003000 },
	3029	+ { _MMIO(0x277c), 0x0000f9ff },
	3030	+ { _MMIO(0x2780), 0x00000c00 },
	3031	+ { _MMIO(0x2784), 0x0000fe7f },
	3032	+};
	3033	+
	3034	+static const struct i915_oa_reg flex_eu_config_sampler_1[] = {
	3035	+ { _MMIO(0xe458), 0x00005004 },
	3036	+ { _MMIO(0xe558), 0x00010003 },
	3037	+ { _MMIO(0xe658), 0x00012011 },
	3038	+ { _MMIO(0xe758), 0x00015014 },
	3039	+ { _MMIO(0xe45c), 0x00051050 },
	3040	+ { _MMIO(0xe55c), 0x00053052 },
	3041	+ { _MMIO(0xe65c), 0x00055054 },
	3042	+};
	3043	+
	3044	+static const struct i915_oa_reg mux_config_sampler_1[] = {
	3045	+ { _MMIO(0x9888), 0x18921400 },
	3046	+ { _MMIO(0x9888), 0x149500ab },
	3047	+ { _MMIO(0x9888), 0x18b21400 },
	3048	+ { _MMIO(0x9888), 0x14b500ab },
	3049	+ { _MMIO(0x9888), 0x18d21400 },
	3050	+ { _MMIO(0x9888), 0x14d500ab },
	3051	+ { _MMIO(0x9888), 0x0cdc8000 },
	3052	+ { _MMIO(0x9888), 0x0edc4000 },
	3053	+ { _MMIO(0x9888), 0x02dcc000 },
	3054	+ { _MMIO(0x9888), 0x04dcc000 },
	3055	+ { _MMIO(0x9888), 0x1abd00a0 },
	3056	+ { _MMIO(0x9888), 0x0abd8000 },
	3057	+ { _MMIO(0x9888), 0x0cd88000 },
	3058	+ { _MMIO(0x9888), 0x0ed84000 },
	3059	+ { _MMIO(0x9888), 0x04d88000 },
	3060	+ { _MMIO(0x9888), 0x1adb0050 },
	3061	+ { _MMIO(0x9888), 0x04db8000 },
	3062	+ { _MMIO(0x9888), 0x06db8000 },
	3063	+ { _MMIO(0x9888), 0x08db8000 },
	3064	+ { _MMIO(0x9888), 0x0adb4000 },
	3065	+ { _MMIO(0x9888), 0x109f02a0 },
	3066	+ { _MMIO(0x9888), 0x0c9fa000 },
	3067	+ { _MMIO(0x9888), 0x0e9f00aa },
	3068	+ { _MMIO(0x9888), 0x18b82500 },
	3069	+ { _MMIO(0x9888), 0x02b88000 },
	3070	+ { _MMIO(0x9888), 0x04b84000 },
	3071	+ { _MMIO(0x9888), 0x06b84000 },
	3072	+ { _MMIO(0x9888), 0x08b84000 },
	3073	+ { _MMIO(0x9888), 0x0ab84000 },
	3074	+ { _MMIO(0x9888), 0x0cb88000 },
	3075	+ { _MMIO(0x9888), 0x0cb98000 },
	3076	+ { _MMIO(0x9888), 0x0eb9a000 },
	3077	+ { _MMIO(0x9888), 0x00b98000 },
	3078	+ { _MMIO(0x9888), 0x02b9a000 },
	3079	+ { _MMIO(0x9888), 0x04b9a000 },
	3080	+ { _MMIO(0x9888), 0x06b92000 },
	3081	+ { _MMIO(0x9888), 0x1aba0200 },
	3082	+ { _MMIO(0x9888), 0x02ba8000 },
	3083	+ { _MMIO(0x9888), 0x0cba8000 },
	3084	+ { _MMIO(0x9888), 0x04908000 },
	3085	+ { _MMIO(0x9888), 0x04918000 },
	3086	+ { _MMIO(0x9888), 0x04927300 },
	3087	+ { _MMIO(0x9888), 0x10920000 },
	3088	+ { _MMIO(0x9888), 0x1893000a },
	3089	+ { _MMIO(0x9888), 0x0a934000 },
	3090	+ { _MMIO(0x9888), 0x0a946000 },
	3091	+ { _MMIO(0x9888), 0x0c959000 },
	3092	+ { _MMIO(0x9888), 0x0e950098 },
	3093	+ { _MMIO(0x9888), 0x10950000 },
	3094	+ { _MMIO(0x9888), 0x04b04000 },
	3095	+ { _MMIO(0x9888), 0x04b14000 },
	3096	+ { _MMIO(0x9888), 0x04b20073 },
	3097	+ { _MMIO(0x9888), 0x10b20000 },
	3098	+ { _MMIO(0x9888), 0x04b38000 },
	3099	+ { _MMIO(0x9888), 0x06b38000 },
	3100	+ { _MMIO(0x9888), 0x08b34000 },
	3101	+ { _MMIO(0x9888), 0x04b4c000 },
	3102	+ { _MMIO(0x9888), 0x02b59890 },
	3103	+ { _MMIO(0x9888), 0x10b50000 },
	3104	+ { _MMIO(0x9888), 0x06d04000 },
	3105	+ { _MMIO(0x9888), 0x06d14000 },
	3106	+ { _MMIO(0x9888), 0x06d20073 },
	3107	+ { _MMIO(0x9888), 0x10d20000 },
	3108	+ { _MMIO(0x9888), 0x18d30020 },
	3109	+ { _MMIO(0x9888), 0x02d38000 },
	3110	+ { _MMIO(0x9888), 0x0cd34000 },
	3111	+ { _MMIO(0x9888), 0x0ad48000 },
	3112	+ { _MMIO(0x9888), 0x04d42000 },
	3113	+ { _MMIO(0x9888), 0x0ed59000 },
	3114	+ { _MMIO(0x9888), 0x00d59800 },
	3115	+ { _MMIO(0x9888), 0x10d50000 },
	3116	+ { _MMIO(0x9888), 0x0f88000e },
	3117	+ { _MMIO(0x9888), 0x03888000 },
	3118	+ { _MMIO(0x9888), 0x05888000 },
	3119	+ { _MMIO(0x9888), 0x07888000 },
	3120	+ { _MMIO(0x9888), 0x09888000 },
	3121	+ { _MMIO(0x9888), 0x0b888000 },
	3122	+ { _MMIO(0x9888), 0x0d880400 },
	3123	+ { _MMIO(0x9888), 0x278b002a },
	3124	+ { _MMIO(0x9888), 0x238b5500 },
	3125	+ { _MMIO(0x9888), 0x258b000a },
	3126	+ { _MMIO(0x9888), 0x1b8c0015 },
	3127	+ { _MMIO(0x9888), 0x038c4000 },
	3128	+ { _MMIO(0x9888), 0x058c4000 },
	3129	+ { _MMIO(0x9888), 0x078c4000 },
	3130	+ { _MMIO(0x9888), 0x098c4000 },
	3131	+ { _MMIO(0x9888), 0x0b8c4000 },
	3132	+ { _MMIO(0x9888), 0x0d8c4000 },
	3133	+ { _MMIO(0x9888), 0x0d8d8000 },
	3134	+ { _MMIO(0x9888), 0x0f8da000 },
	3135	+ { _MMIO(0x9888), 0x018d8000 },
	3136	+ { _MMIO(0x9888), 0x038da000 },
	3137	+ { _MMIO(0x9888), 0x058da000 },
	3138	+ { _MMIO(0x9888), 0x078d2000 },
	3139	+ { _MMIO(0x9888), 0x2385002a },
	3140	+ { _MMIO(0x9888), 0x1f85aa00 },
	3141	+ { _MMIO(0x9888), 0x2185000a },
	3142	+ { _MMIO(0x9888), 0x1b830150 },
	3143	+ { _MMIO(0x9888), 0x03834000 },
	3144	+ { _MMIO(0x9888), 0x05834000 },
	3145	+ { _MMIO(0x9888), 0x07834000 },
	3146	+ { _MMIO(0x9888), 0x09834000 },
	3147	+ { _MMIO(0x9888), 0x0b834000 },
	3148	+ { _MMIO(0x9888), 0x0d834000 },
	3149	+ { _MMIO(0x9888), 0x0d848000 },
	3150	+ { _MMIO(0x9888), 0x0f84c000 },
	3151	+ { _MMIO(0x9888), 0x01848000 },
	3152	+ { _MMIO(0x9888), 0x0384c000 },
	3153	+ { _MMIO(0x9888), 0x0584c000 },
	3154	+ { _MMIO(0x9888), 0x07844000 },
	3155	+ { _MMIO(0x9888), 0x1d808000 },
	3156	+ { _MMIO(0x9888), 0x1f80c000 },
	3157	+ { _MMIO(0x9888), 0x11808000 },
	3158	+ { _MMIO(0x9888), 0x1380c000 },
	3159	+ { _MMIO(0x9888), 0x1580c000 },
	3160	+ { _MMIO(0x9888), 0x17804000 },
	3161	+ { _MMIO(0x9888), 0x53800000 },
	3162	+ { _MMIO(0x9888), 0x47801021 },
	3163	+ { _MMIO(0x9888), 0x21800000 },
	3164	+ { _MMIO(0x9888), 0x31800000 },
	3165	+ { _MMIO(0x9888), 0x4d800000 },
	3166	+ { _MMIO(0x9888), 0x3f800c64 },
	3167	+ { _MMIO(0x9888), 0x4f800000 },
	3168	+ { _MMIO(0x9888), 0x41800c02 },
	3169	+};
	3170	+
	3171	+static int
	3172	+get_sampler_1_mux_config(struct drm_i915_private *dev_priv,
	3173	+ const struct i915_oa_reg **regs,
	3174	+ int *lens)
	3175	+{
	3176	+ int n = 0;
	3177	+
	3178	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	3179	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	3180	+
	3181	+ regs[n] = mux_config_sampler_1;
	3182	+ lens[n] = ARRAY_SIZE(mux_config_sampler_1);
	3183	+ n++;
	3184	+
	3185	+ return n;
	3186	+}
	3187	+
	3188	+static const struct i915_oa_reg b_counter_config_sampler_2[] = {
	3189	+ { _MMIO(0x2740), 0x00000000 },
	3190	+ { _MMIO(0x2744), 0x00800000 },
	3191	+ { _MMIO(0x2710), 0x00000000 },
	3192	+ { _MMIO(0x2714), 0x70800000 },
	3193	+ { _MMIO(0x2720), 0x00000000 },
	3194	+ { _MMIO(0x2724), 0x00800000 },
	3195	+ { _MMIO(0x2770), 0x0000c000 },
	3196	+ { _MMIO(0x2774), 0x0000e7ff },
	3197	+ { _MMIO(0x2778), 0x00003000 },
	3198	+ { _MMIO(0x277c), 0x0000f9ff },
	3199	+ { _MMIO(0x2780), 0x00000c00 },
	3200	+ { _MMIO(0x2784), 0x0000fe7f },
	3201	+};
	3202	+
	3203	+static const struct i915_oa_reg flex_eu_config_sampler_2[] = {
	3204	+ { _MMIO(0xe458), 0x00005004 },
	3205	+ { _MMIO(0xe558), 0x00010003 },
	3206	+ { _MMIO(0xe658), 0x00012011 },
	3207	+ { _MMIO(0xe758), 0x00015014 },
	3208	+ { _MMIO(0xe45c), 0x00051050 },
	3209	+ { _MMIO(0xe55c), 0x00053052 },
	3210	+ { _MMIO(0xe65c), 0x00055054 },
	3211	+};
	3212	+
	3213	+static const struct i915_oa_reg mux_config_sampler_2[] = {
	3214	+ { _MMIO(0x9888), 0x18121400 },
	3215	+ { _MMIO(0x9888), 0x141500ab },
	3216	+ { _MMIO(0x9888), 0x18321400 },
	3217	+ { _MMIO(0x9888), 0x143500ab },
	3218	+ { _MMIO(0x9888), 0x18521400 },
	3219	+ { _MMIO(0x9888), 0x145500ab },
	3220	+ { _MMIO(0x9888), 0x0c5c8000 },
	3221	+ { _MMIO(0x9888), 0x0e5c4000 },
	3222	+ { _MMIO(0x9888), 0x025cc000 },
	3223	+ { _MMIO(0x9888), 0x045cc000 },
	3224	+ { _MMIO(0x9888), 0x1a3d00a0 },
	3225	+ { _MMIO(0x9888), 0x0a3d8000 },
	3226	+ { _MMIO(0x9888), 0x0c588000 },
	3227	+ { _MMIO(0x9888), 0x0e584000 },
	3228	+ { _MMIO(0x9888), 0x04588000 },
	3229	+ { _MMIO(0x9888), 0x1a5b0050 },
	3230	+ { _MMIO(0x9888), 0x045b8000 },
	3231	+ { _MMIO(0x9888), 0x065b8000 },
	3232	+ { _MMIO(0x9888), 0x085b8000 },
	3233	+ { _MMIO(0x9888), 0x0a5b4000 },
	3234	+ { _MMIO(0x9888), 0x101f02a0 },
	3235	+ { _MMIO(0x9888), 0x0c1fa000 },
	3236	+ { _MMIO(0x9888), 0x0e1f00aa },
	3237	+ { _MMIO(0x9888), 0x18382500 },
	3238	+ { _MMIO(0x9888), 0x02388000 },
	3239	+ { _MMIO(0x9888), 0x04384000 },
	3240	+ { _MMIO(0x9888), 0x06384000 },
	3241	+ { _MMIO(0x9888), 0x08384000 },
	3242	+ { _MMIO(0x9888), 0x0a384000 },
	3243	+ { _MMIO(0x9888), 0x0c388000 },
	3244	+ { _MMIO(0x9888), 0x0c398000 },
	3245	+ { _MMIO(0x9888), 0x0e39a000 },
	3246	+ { _MMIO(0x9888), 0x00398000 },
	3247	+ { _MMIO(0x9888), 0x0239a000 },
	3248	+ { _MMIO(0x9888), 0x0439a000 },
	3249	+ { _MMIO(0x9888), 0x06392000 },
	3250	+ { _MMIO(0x9888), 0x1a3a0200 },
	3251	+ { _MMIO(0x9888), 0x023a8000 },
	3252	+ { _MMIO(0x9888), 0x0c3a8000 },
	3253	+ { _MMIO(0x9888), 0x04108000 },
	3254	+ { _MMIO(0x9888), 0x04118000 },
	3255	+ { _MMIO(0x9888), 0x04127300 },
	3256	+ { _MMIO(0x9888), 0x10120000 },
	3257	+ { _MMIO(0x9888), 0x1813000a },
	3258	+ { _MMIO(0x9888), 0x0a134000 },
	3259	+ { _MMIO(0x9888), 0x0a146000 },
	3260	+ { _MMIO(0x9888), 0x0c159000 },
	3261	+ { _MMIO(0x9888), 0x0e150098 },
	3262	+ { _MMIO(0x9888), 0x10150000 },
	3263	+ { _MMIO(0x9888), 0x04304000 },
	3264	+ { _MMIO(0x9888), 0x04314000 },
	3265	+ { _MMIO(0x9888), 0x04320073 },
	3266	+ { _MMIO(0x9888), 0x10320000 },
	3267	+ { _MMIO(0x9888), 0x04338000 },
	3268	+ { _MMIO(0x9888), 0x06338000 },
	3269	+ { _MMIO(0x9888), 0x08334000 },
	3270	+ { _MMIO(0x9888), 0x0434c000 },
	3271	+ { _MMIO(0x9888), 0x02359890 },
	3272	+ { _MMIO(0x9888), 0x10350000 },
	3273	+ { _MMIO(0x9888), 0x06504000 },
	3274	+ { _MMIO(0x9888), 0x06514000 },
	3275	+ { _MMIO(0x9888), 0x06520073 },
	3276	+ { _MMIO(0x9888), 0x10520000 },
	3277	+ { _MMIO(0x9888), 0x18530020 },
	3278	+ { _MMIO(0x9888), 0x02538000 },
	3279	+ { _MMIO(0x9888), 0x0c534000 },
	3280	+ { _MMIO(0x9888), 0x0a548000 },
	3281	+ { _MMIO(0x9888), 0x04542000 },
	3282	+ { _MMIO(0x9888), 0x0e559000 },
	3283	+ { _MMIO(0x9888), 0x00559800 },
	3284	+ { _MMIO(0x9888), 0x10550000 },
	3285	+ { _MMIO(0x9888), 0x1b8aa000 },
	3286	+ { _MMIO(0x9888), 0x1d8a0002 },
	3287	+ { _MMIO(0x9888), 0x038a8000 },
	3288	+ { _MMIO(0x9888), 0x058a8000 },
	3289	+ { _MMIO(0x9888), 0x078a8000 },
	3290	+ { _MMIO(0x9888), 0x098a8000 },
	3291	+ { _MMIO(0x9888), 0x0b8a8000 },
	3292	+ { _MMIO(0x9888), 0x0d8a8000 },
	3293	+ { _MMIO(0x9888), 0x278b0015 },
	3294	+ { _MMIO(0x9888), 0x238b2a80 },
	3295	+ { _MMIO(0x9888), 0x258b0005 },
	3296	+ { _MMIO(0x9888), 0x2385002a },
	3297	+ { _MMIO(0x9888), 0x1f85aa00 },
	3298	+ { _MMIO(0x9888), 0x2185000a },
	3299	+ { _MMIO(0x9888), 0x1b830150 },
	3300	+ { _MMIO(0x9888), 0x03834000 },
	3301	+ { _MMIO(0x9888), 0x05834000 },
	3302	+ { _MMIO(0x9888), 0x07834000 },
	3303	+ { _MMIO(0x9888), 0x09834000 },
	3304	+ { _MMIO(0x9888), 0x0b834000 },
	3305	+ { _MMIO(0x9888), 0x0d834000 },
	3306	+ { _MMIO(0x9888), 0x0d848000 },
	3307	+ { _MMIO(0x9888), 0x0f84c000 },
	3308	+ { _MMIO(0x9888), 0x01848000 },
	3309	+ { _MMIO(0x9888), 0x0384c000 },
	3310	+ { _MMIO(0x9888), 0x0584c000 },
	3311	+ { _MMIO(0x9888), 0x07844000 },
	3312	+ { _MMIO(0x9888), 0x1d808000 },
	3313	+ { _MMIO(0x9888), 0x1f80c000 },
	3314	+ { _MMIO(0x9888), 0x11808000 },
	3315	+ { _MMIO(0x9888), 0x1380c000 },
	3316	+ { _MMIO(0x9888), 0x1580c000 },
	3317	+ { _MMIO(0x9888), 0x17804000 },
	3318	+ { _MMIO(0x9888), 0x53800000 },
	3319	+ { _MMIO(0x9888), 0x47801021 },
	3320	+ { _MMIO(0x9888), 0x21800000 },
	3321	+ { _MMIO(0x9888), 0x31800000 },
	3322	+ { _MMIO(0x9888), 0x4d800000 },
	3323	+ { _MMIO(0x9888), 0x3f800c64 },
	3324	+ { _MMIO(0x9888), 0x4f800000 },
	3325	+ { _MMIO(0x9888), 0x41800c02 },
	3326	+};
	3327	+
	3328	+static int
	3329	+get_sampler_2_mux_config(struct drm_i915_private *dev_priv,
	3330	+ const struct i915_oa_reg **regs,
	3331	+ int *lens)
	3332	+{
	3333	+ int n = 0;
	3334	+
	3335	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	3336	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	3337	+
	3338	+ regs[n] = mux_config_sampler_2;
	3339	+ lens[n] = ARRAY_SIZE(mux_config_sampler_2);
	3340	+ n++;
	3341	+
	3342	+ return n;
	3343	+}
	3344	+
	3345	+static const struct i915_oa_reg b_counter_config_tdl_1[] = {
	3346	+ { _MMIO(0x2740), 0x00000000 },
	3347	+ { _MMIO(0x2744), 0x00800000 },
	3348	+ { _MMIO(0x2710), 0x00000000 },
	3349	+ { _MMIO(0x2714), 0xf0800000 },
	3350	+ { _MMIO(0x2720), 0x00000000 },
	3351	+ { _MMIO(0x2724), 0x30800000 },
	3352	+ { _MMIO(0x2770), 0x00000002 },
	3353	+ { _MMIO(0x2774), 0x0000fdff },
	3354	+ { _MMIO(0x2778), 0x00000000 },
	3355	+ { _MMIO(0x277c), 0x0000fe7f },
	3356	+ { _MMIO(0x2780), 0x00000002 },
	3357	+ { _MMIO(0x2784), 0x0000ffbf },
	3358	+ { _MMIO(0x2788), 0x00000000 },
	3359	+ { _MMIO(0x278c), 0x0000ffcf },
	3360	+ { _MMIO(0x2790), 0x00000002 },
	3361	+ { _MMIO(0x2794), 0x0000fff7 },
	3362	+ { _MMIO(0x2798), 0x00000000 },
	3363	+ { _MMIO(0x279c), 0x0000fff9 },
	3364	+};
	3365	+
	3366	+static const struct i915_oa_reg flex_eu_config_tdl_1[] = {
	3367	+ { _MMIO(0xe458), 0x00005004 },
	3368	+ { _MMIO(0xe558), 0x00010003 },
	3369	+ { _MMIO(0xe658), 0x00012011 },
	3370	+ { _MMIO(0xe758), 0x00015014 },
	3371	+ { _MMIO(0xe45c), 0x00051050 },
	3372	+ { _MMIO(0xe55c), 0x00053052 },
	3373	+ { _MMIO(0xe65c), 0x00055054 },
	3374	+};
	3375	+
	3376	+static const struct i915_oa_reg mux_config_tdl_1[] = {
	3377	+ { _MMIO(0x9888), 0x16154d60 },
	3378	+ { _MMIO(0x9888), 0x16352e60 },
	3379	+ { _MMIO(0x9888), 0x16554d60 },
	3380	+ { _MMIO(0x9888), 0x16950000 },
	3381	+ { _MMIO(0x9888), 0x16b50000 },
	3382	+ { _MMIO(0x9888), 0x16d50000 },
	3383	+ { _MMIO(0x9888), 0x005c8000 },
	3384	+ { _MMIO(0x9888), 0x045cc000 },
	3385	+ { _MMIO(0x9888), 0x065c4000 },
	3386	+ { _MMIO(0x9888), 0x083d8000 },
	3387	+ { _MMIO(0x9888), 0x0a3d8000 },
	3388	+ { _MMIO(0x9888), 0x0458c000 },
	3389	+ { _MMIO(0x9888), 0x025b8000 },
	3390	+ { _MMIO(0x9888), 0x085b4000 },
	3391	+ { _MMIO(0x9888), 0x0a5b4000 },
	3392	+ { _MMIO(0x9888), 0x0c5b8000 },
	3393	+ { _MMIO(0x9888), 0x0c1fa000 },
	3394	+ { _MMIO(0x9888), 0x0e1f00aa },
	3395	+ { _MMIO(0x9888), 0x02384000 },
	3396	+ { _MMIO(0x9888), 0x04388000 },
	3397	+ { _MMIO(0x9888), 0x06388000 },
	3398	+ { _MMIO(0x9888), 0x08384000 },
	3399	+ { _MMIO(0x9888), 0x0a384000 },
	3400	+ { _MMIO(0x9888), 0x0c384000 },
	3401	+ { _MMIO(0x9888), 0x00398000 },
	3402	+ { _MMIO(0x9888), 0x0239a000 },
	3403	+ { _MMIO(0x9888), 0x0439a000 },
	3404	+ { _MMIO(0x9888), 0x06392000 },
	3405	+ { _MMIO(0x9888), 0x043a8000 },
	3406	+ { _MMIO(0x9888), 0x063a8000 },
	3407	+ { _MMIO(0x9888), 0x08138000 },
	3408	+ { _MMIO(0x9888), 0x0a138000 },
	3409	+ { _MMIO(0x9888), 0x06143000 },
	3410	+ { _MMIO(0x9888), 0x0415cfc7 },
	3411	+ { _MMIO(0x9888), 0x10150000 },
	3412	+ { _MMIO(0x9888), 0x02338000 },
	3413	+ { _MMIO(0x9888), 0x0c338000 },
	3414	+ { _MMIO(0x9888), 0x04342000 },
	3415	+ { _MMIO(0x9888), 0x06344000 },
	3416	+ { _MMIO(0x9888), 0x0035c700 },
	3417	+ { _MMIO(0x9888), 0x063500cf },
	3418	+ { _MMIO(0x9888), 0x10350000 },
	3419	+ { _MMIO(0x9888), 0x04538000 },
	3420	+ { _MMIO(0x9888), 0x06538000 },
	3421	+ { _MMIO(0x9888), 0x0454c000 },
	3422	+ { _MMIO(0x9888), 0x0255cfc7 },
	3423	+ { _MMIO(0x9888), 0x10550000 },
	3424	+ { _MMIO(0x9888), 0x06dc8000 },
	3425	+ { _MMIO(0x9888), 0x08dc4000 },
	3426	+ { _MMIO(0x9888), 0x0cdcc000 },
	3427	+ { _MMIO(0x9888), 0x0edcc000 },
	3428	+ { _MMIO(0x9888), 0x1abd00a8 },
	3429	+ { _MMIO(0x9888), 0x0cd8c000 },
	3430	+ { _MMIO(0x9888), 0x0ed84000 },
	3431	+ { _MMIO(0x9888), 0x0edb8000 },
	3432	+ { _MMIO(0x9888), 0x18db0800 },
	3433	+ { _MMIO(0x9888), 0x1adb0254 },
	3434	+ { _MMIO(0x9888), 0x0e9faa00 },
	3435	+ { _MMIO(0x9888), 0x109f02aa },
	3436	+ { _MMIO(0x9888), 0x0eb84000 },
	3437	+ { _MMIO(0x9888), 0x16b84000 },
	3438	+ { _MMIO(0x9888), 0x18b8156a },
	3439	+ { _MMIO(0x9888), 0x06b98000 },
	3440	+ { _MMIO(0x9888), 0x08b9a000 },
	3441	+ { _MMIO(0x9888), 0x0ab9a000 },
	3442	+ { _MMIO(0x9888), 0x0cb9a000 },
	3443	+ { _MMIO(0x9888), 0x0eb9a000 },
	3444	+ { _MMIO(0x9888), 0x18baa000 },
	3445	+ { _MMIO(0x9888), 0x1aba0002 },
	3446	+ { _MMIO(0x9888), 0x16934000 },
	3447	+ { _MMIO(0x9888), 0x1893000a },
	3448	+ { _MMIO(0x9888), 0x0a947000 },
	3449	+ { _MMIO(0x9888), 0x0c95c5c1 },
	3450	+ { _MMIO(0x9888), 0x0e9500c3 },
	3451	+ { _MMIO(0x9888), 0x10950000 },
	3452	+ { _MMIO(0x9888), 0x0eb38000 },
	3453	+ { _MMIO(0x9888), 0x16b30040 },
	3454	+ { _MMIO(0x9888), 0x18b30020 },
	3455	+ { _MMIO(0x9888), 0x06b48000 },
	3456	+ { _MMIO(0x9888), 0x08b41000 },
	3457	+ { _MMIO(0x9888), 0x0ab48000 },
	3458	+ { _MMIO(0x9888), 0x06b5c500 },
	3459	+ { _MMIO(0x9888), 0x08b500c3 },
	3460	+ { _MMIO(0x9888), 0x0eb5c100 },
	3461	+ { _MMIO(0x9888), 0x10b50000 },
	3462	+ { _MMIO(0x9888), 0x16d31500 },
	3463	+ { _MMIO(0x9888), 0x08d4e000 },
	3464	+ { _MMIO(0x9888), 0x08d5c100 },
	3465	+ { _MMIO(0x9888), 0x0ad5c3c5 },
	3466	+ { _MMIO(0x9888), 0x10d50000 },
	3467	+ { _MMIO(0x9888), 0x0d88f800 },
	3468	+ { _MMIO(0x9888), 0x0f88000f },
	3469	+ { _MMIO(0x9888), 0x038a8000 },
	3470	+ { _MMIO(0x9888), 0x058a8000 },
	3471	+ { _MMIO(0x9888), 0x078a8000 },
	3472	+ { _MMIO(0x9888), 0x098a8000 },
	3473	+ { _MMIO(0x9888), 0x0b8a8000 },
	3474	+ { _MMIO(0x9888), 0x0d8a8000 },
	3475	+ { _MMIO(0x9888), 0x258baaa5 },
	3476	+ { _MMIO(0x9888), 0x278b002a },
	3477	+ { _MMIO(0x9888), 0x238b2a80 },
	3478	+ { _MMIO(0x9888), 0x0f8c4000 },
	3479	+ { _MMIO(0x9888), 0x178c2000 },
	3480	+ { _MMIO(0x9888), 0x198c5500 },
	3481	+ { _MMIO(0x9888), 0x1b8c0015 },
	3482	+ { _MMIO(0x9888), 0x078d8000 },
	3483	+ { _MMIO(0x9888), 0x098da000 },
	3484	+ { _MMIO(0x9888), 0x0b8da000 },
	3485	+ { _MMIO(0x9888), 0x0d8da000 },
	3486	+ { _MMIO(0x9888), 0x0f8da000 },
	3487	+ { _MMIO(0x9888), 0x2185aaaa },
	3488	+ { _MMIO(0x9888), 0x2385002a },
	3489	+ { _MMIO(0x9888), 0x1f85aa00 },
	3490	+ { _MMIO(0x9888), 0x0f834000 },
	3491	+ { _MMIO(0x9888), 0x19835400 },
	3492	+ { _MMIO(0x9888), 0x1b830155 },
	3493	+ { _MMIO(0x9888), 0x03834000 },
	3494	+ { _MMIO(0x9888), 0x05834000 },
	3495	+ { _MMIO(0x9888), 0x07834000 },
	3496	+ { _MMIO(0x9888), 0x09834000 },
	3497	+ { _MMIO(0x9888), 0x0b834000 },
	3498	+ { _MMIO(0x9888), 0x0d834000 },
	3499	+ { _MMIO(0x9888), 0x0784c000 },
	3500	+ { _MMIO(0x9888), 0x0984c000 },
	3501	+ { _MMIO(0x9888), 0x0b84c000 },
	3502	+ { _MMIO(0x9888), 0x0d84c000 },
	3503	+ { _MMIO(0x9888), 0x0f84c000 },
	3504	+ { _MMIO(0x9888), 0x01848000 },
	3505	+ { _MMIO(0x9888), 0x0384c000 },
	3506	+ { _MMIO(0x9888), 0x0584c000 },
	3507	+ { _MMIO(0x9888), 0x1780c000 },
	3508	+ { _MMIO(0x9888), 0x1980c000 },
	3509	+ { _MMIO(0x9888), 0x1b80c000 },
	3510	+ { _MMIO(0x9888), 0x1d80c000 },
	3511	+ { _MMIO(0x9888), 0x1f80c000 },
	3512	+ { _MMIO(0x9888), 0x11808000 },
	3513	+ { _MMIO(0x9888), 0x1380c000 },
	3514	+ { _MMIO(0x9888), 0x1580c000 },
	3515	+ { _MMIO(0x9888), 0x4f800000 },
	3516	+ { _MMIO(0x9888), 0x43800c42 },
	3517	+ { _MMIO(0x9888), 0x51800000 },
	3518	+ { _MMIO(0x9888), 0x45800063 },
	3519	+ { _MMIO(0x9888), 0x53800000 },
	3520	+ { _MMIO(0x9888), 0x47800800 },
	3521	+ { _MMIO(0x9888), 0x21800000 },
	3522	+ { _MMIO(0x9888), 0x31800000 },
	3523	+ { _MMIO(0x9888), 0x4d800000 },
	3524	+ { _MMIO(0x9888), 0x3f8014a4 },
	3525	+ { _MMIO(0x9888), 0x41801042 },
	3526	+};
	3527	+
	3528	+static int
	3529	+get_tdl_1_mux_config(struct drm_i915_private *dev_priv,
	3530	+ const struct i915_oa_reg **regs,
	3531	+ int *lens)
	3532	+{
	3533	+ int n = 0;
	3534	+
	3535	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	3536	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	3537	+
	3538	+ regs[n] = mux_config_tdl_1;
	3539	+ lens[n] = ARRAY_SIZE(mux_config_tdl_1);
	3540	+ n++;
	3541	+
	3542	+ return n;
	3543	+}
	3544	+
	3545	+static const struct i915_oa_reg b_counter_config_tdl_2[] = {
	3546	+ { _MMIO(0x2740), 0x00000000 },
	3547	+ { _MMIO(0x2744), 0x00800000 },
	3548	+ { _MMIO(0x2710), 0x00000000 },
	3549	+ { _MMIO(0x2714), 0xf0800000 },
	3550	+ { _MMIO(0x2720), 0x00000000 },
	3551	+ { _MMIO(0x2724), 0x30800000 },
	3552	+ { _MMIO(0x2770), 0x00000002 },
	3553	+ { _MMIO(0x2774), 0x0000fdff },
	3554	+ { _MMIO(0x2778), 0x00000000 },
	3555	+ { _MMIO(0x277c), 0x0000fe7f },
	3556	+ { _MMIO(0x2780), 0x00000000 },
	3557	+ { _MMIO(0x2784), 0x0000ff9f },
	3558	+ { _MMIO(0x2788), 0x00000000 },
	3559	+ { _MMIO(0x278c), 0x0000ffe7 },
	3560	+ { _MMIO(0x2790), 0x00000002 },
	3561	+ { _MMIO(0x2794), 0x0000fffb },
	3562	+ { _MMIO(0x2798), 0x00000002 },
	3563	+ { _MMIO(0x279c), 0x0000fffd },
	3564	+};
	3565	+
	3566	+static const struct i915_oa_reg flex_eu_config_tdl_2[] = {
	3567	+ { _MMIO(0xe458), 0x00005004 },
	3568	+ { _MMIO(0xe558), 0x00010003 },
	3569	+ { _MMIO(0xe658), 0x00012011 },
	3570	+ { _MMIO(0xe758), 0x00015014 },
	3571	+ { _MMIO(0xe45c), 0x00051050 },
	3572	+ { _MMIO(0xe55c), 0x00053052 },
	3573	+ { _MMIO(0xe65c), 0x00055054 },
	3574	+};
	3575	+
	3576	+static const struct i915_oa_reg mux_config_tdl_2[] = {
	3577	+ { _MMIO(0x9888), 0x16150000 },
	3578	+ { _MMIO(0x9888), 0x16350000 },
	3579	+ { _MMIO(0x9888), 0x16550000 },
	3580	+ { _MMIO(0x9888), 0x16952e60 },
	3581	+ { _MMIO(0x9888), 0x16b54d60 },
	3582	+ { _MMIO(0x9888), 0x16d52e60 },
	3583	+ { _MMIO(0x9888), 0x065c8000 },
	3584	+ { _MMIO(0x9888), 0x085cc000 },
	3585	+ { _MMIO(0x9888), 0x0a5cc000 },
	3586	+ { _MMIO(0x9888), 0x0c5c4000 },
	3587	+ { _MMIO(0x9888), 0x0e3d8000 },
	3588	+ { _MMIO(0x9888), 0x183da000 },
	3589	+ { _MMIO(0x9888), 0x06588000 },
	3590	+ { _MMIO(0x9888), 0x08588000 },
	3591	+ { _MMIO(0x9888), 0x0a584000 },
	3592	+ { _MMIO(0x9888), 0x0e5b4000 },
	3593	+ { _MMIO(0x9888), 0x185b5800 },
	3594	+ { _MMIO(0x9888), 0x1a5b000a },
	3595	+ { _MMIO(0x9888), 0x0e1faa00 },
	3596	+ { _MMIO(0x9888), 0x101f02aa },
	3597	+ { _MMIO(0x9888), 0x0e384000 },
	3598	+ { _MMIO(0x9888), 0x16384000 },
	3599	+ { _MMIO(0x9888), 0x18382a55 },
	3600	+ { _MMIO(0x9888), 0x06398000 },
	3601	+ { _MMIO(0x9888), 0x0839a000 },
	3602	+ { _MMIO(0x9888), 0x0a39a000 },
	3603	+ { _MMIO(0x9888), 0x0c39a000 },
	3604	+ { _MMIO(0x9888), 0x0e39a000 },
	3605	+ { _MMIO(0x9888), 0x1a3a02a0 },
	3606	+ { _MMIO(0x9888), 0x0e138000 },
	3607	+ { _MMIO(0x9888), 0x16130500 },
	3608	+ { _MMIO(0x9888), 0x06148000 },
	3609	+ { _MMIO(0x9888), 0x08146000 },
	3610	+ { _MMIO(0x9888), 0x0615c100 },
	3611	+ { _MMIO(0x9888), 0x0815c500 },
	3612	+ { _MMIO(0x9888), 0x0a1500c3 },
	3613	+ { _MMIO(0x9888), 0x10150000 },
	3614	+ { _MMIO(0x9888), 0x16335040 },
	3615	+ { _MMIO(0x9888), 0x08349000 },
	3616	+ { _MMIO(0x9888), 0x0a341000 },
	3617	+ { _MMIO(0x9888), 0x083500c1 },
	3618	+ { _MMIO(0x9888), 0x0a35c500 },
	3619	+ { _MMIO(0x9888), 0x0c3500c3 },
	3620	+ { _MMIO(0x9888), 0x10350000 },
	3621	+ { _MMIO(0x9888), 0x1853002a },
	3622	+ { _MMIO(0x9888), 0x0a54e000 },
	3623	+ { _MMIO(0x9888), 0x0c55c500 },
	3624	+ { _MMIO(0x9888), 0x0e55c1c3 },
	3625	+ { _MMIO(0x9888), 0x10550000 },
	3626	+ { _MMIO(0x9888), 0x00dc8000 },
	3627	+ { _MMIO(0x9888), 0x02dcc000 },
	3628	+ { _MMIO(0x9888), 0x04dc4000 },
	3629	+ { _MMIO(0x9888), 0x04bd8000 },
	3630	+ { _MMIO(0x9888), 0x06bd8000 },
	3631	+ { _MMIO(0x9888), 0x02d8c000 },
	3632	+ { _MMIO(0x9888), 0x02db8000 },
	3633	+ { _MMIO(0x9888), 0x04db4000 },
	3634	+ { _MMIO(0x9888), 0x06db4000 },
	3635	+ { _MMIO(0x9888), 0x08db8000 },
	3636	+ { _MMIO(0x9888), 0x0c9fa000 },
	3637	+ { _MMIO(0x9888), 0x0e9f00aa },
	3638	+ { _MMIO(0x9888), 0x02b84000 },
	3639	+ { _MMIO(0x9888), 0x04b84000 },
	3640	+ { _MMIO(0x9888), 0x06b84000 },
	3641	+ { _MMIO(0x9888), 0x08b84000 },
	3642	+ { _MMIO(0x9888), 0x0ab88000 },
	3643	+ { _MMIO(0x9888), 0x0cb88000 },
	3644	+ { _MMIO(0x9888), 0x00b98000 },
	3645	+ { _MMIO(0x9888), 0x02b9a000 },
	3646	+ { _MMIO(0x9888), 0x04b9a000 },
	3647	+ { _MMIO(0x9888), 0x06b92000 },
	3648	+ { _MMIO(0x9888), 0x0aba8000 },
	3649	+ { _MMIO(0x9888), 0x0cba8000 },
	3650	+ { _MMIO(0x9888), 0x04938000 },
	3651	+ { _MMIO(0x9888), 0x06938000 },
	3652	+ { _MMIO(0x9888), 0x0494c000 },
	3653	+ { _MMIO(0x9888), 0x0295cfc7 },
	3654	+ { _MMIO(0x9888), 0x10950000 },
	3655	+ { _MMIO(0x9888), 0x02b38000 },
	3656	+ { _MMIO(0x9888), 0x08b38000 },
	3657	+ { _MMIO(0x9888), 0x04b42000 },
	3658	+ { _MMIO(0x9888), 0x06b41000 },
	3659	+ { _MMIO(0x9888), 0x00b5c700 },
	3660	+ { _MMIO(0x9888), 0x04b500cf },
	3661	+ { _MMIO(0x9888), 0x10b50000 },
	3662	+ { _MMIO(0x9888), 0x0ad38000 },
	3663	+ { _MMIO(0x9888), 0x0cd38000 },
	3664	+ { _MMIO(0x9888), 0x06d46000 },
	3665	+ { _MMIO(0x9888), 0x04d5c700 },
	3666	+ { _MMIO(0x9888), 0x06d500cf },
	3667	+ { _MMIO(0x9888), 0x10d50000 },
	3668	+ { _MMIO(0x9888), 0x03888000 },
	3669	+ { _MMIO(0x9888), 0x05888000 },
	3670	+ { _MMIO(0x9888), 0x07888000 },
	3671	+ { _MMIO(0x9888), 0x09888000 },
	3672	+ { _MMIO(0x9888), 0x0b888000 },
	3673	+ { _MMIO(0x9888), 0x0d880400 },
	3674	+ { _MMIO(0x9888), 0x0f8a8000 },
	3675	+ { _MMIO(0x9888), 0x198a8000 },
	3676	+ { _MMIO(0x9888), 0x1b8aaaa0 },
	3677	+ { _MMIO(0x9888), 0x1d8a0002 },
	3678	+ { _MMIO(0x9888), 0x258b555a },
	3679	+ { _MMIO(0x9888), 0x278b0015 },
	3680	+ { _MMIO(0x9888), 0x238b5500 },
	3681	+ { _MMIO(0x9888), 0x038c4000 },
	3682	+ { _MMIO(0x9888), 0x058c4000 },
	3683	+ { _MMIO(0x9888), 0x078c4000 },
	3684	+ { _MMIO(0x9888), 0x098c4000 },
	3685	+ { _MMIO(0x9888), 0x0b8c4000 },
	3686	+ { _MMIO(0x9888), 0x0d8c4000 },
	3687	+ { _MMIO(0x9888), 0x018d8000 },
	3688	+ { _MMIO(0x9888), 0x038da000 },
	3689	+ { _MMIO(0x9888), 0x058da000 },
	3690	+ { _MMIO(0x9888), 0x078d2000 },
	3691	+ { _MMIO(0x9888), 0x2185aaaa },
	3692	+ { _MMIO(0x9888), 0x2385002a },
	3693	+ { _MMIO(0x9888), 0x1f85aa00 },
	3694	+ { _MMIO(0x9888), 0x0f834000 },
	3695	+ { _MMIO(0x9888), 0x19835400 },
	3696	+ { _MMIO(0x9888), 0x1b830155 },
	3697	+ { _MMIO(0x9888), 0x03834000 },
	3698	+ { _MMIO(0x9888), 0x05834000 },
	3699	+ { _MMIO(0x9888), 0x07834000 },
	3700	+ { _MMIO(0x9888), 0x09834000 },
	3701	+ { _MMIO(0x9888), 0x0b834000 },
	3702	+ { _MMIO(0x9888), 0x0d834000 },
	3703	+ { _MMIO(0x9888), 0x0784c000 },
	3704	+ { _MMIO(0x9888), 0x0984c000 },
	3705	+ { _MMIO(0x9888), 0x0b84c000 },
	3706	+ { _MMIO(0x9888), 0x0d84c000 },
	3707	+ { _MMIO(0x9888), 0x0f84c000 },
	3708	+ { _MMIO(0x9888), 0x01848000 },
	3709	+ { _MMIO(0x9888), 0x0384c000 },
	3710	+ { _MMIO(0x9888), 0x0584c000 },
	3711	+ { _MMIO(0x9888), 0x1780c000 },
	3712	+ { _MMIO(0x9888), 0x1980c000 },
	3713	+ { _MMIO(0x9888), 0x1b80c000 },
	3714	+ { _MMIO(0x9888), 0x1d80c000 },
	3715	+ { _MMIO(0x9888), 0x1f80c000 },
	3716	+ { _MMIO(0x9888), 0x11808000 },
	3717	+ { _MMIO(0x9888), 0x1380c000 },
	3718	+ { _MMIO(0x9888), 0x1580c000 },
	3719	+ { _MMIO(0x9888), 0x4f800000 },
	3720	+ { _MMIO(0x9888), 0x43800882 },
	3721	+ { _MMIO(0x9888), 0x51800000 },
	3722	+ { _MMIO(0x9888), 0x45801082 },
	3723	+ { _MMIO(0x9888), 0x53800000 },
	3724	+ { _MMIO(0x9888), 0x478014a5 },
	3725	+ { _MMIO(0x9888), 0x21800000 },
	3726	+ { _MMIO(0x9888), 0x31800000 },
	3727	+ { _MMIO(0x9888), 0x4d800000 },
	3728	+ { _MMIO(0x9888), 0x3f800002 },
	3729	+ { _MMIO(0x9888), 0x41800c62 },
	3730	+};
	3731	+
	3732	+static int
	3733	+get_tdl_2_mux_config(struct drm_i915_private *dev_priv,
	3734	+ const struct i915_oa_reg **regs,
	3735	+ int *lens)
	3736	+{
	3737	+ int n = 0;
	3738	+
	3739	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	3740	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	3741	+
	3742	+ regs[n] = mux_config_tdl_2;
	3743	+ lens[n] = ARRAY_SIZE(mux_config_tdl_2);
	3744	+ n++;
	3745	+
	3746	+ return n;
	3747	+}
	3748	+
	3749	+static const struct i915_oa_reg b_counter_config_compute_extra[] = {
	3750	+ { _MMIO(0x2740), 0x00000000 },
	3751	+ { _MMIO(0x2744), 0x00800000 },
	3752	+ { _MMIO(0x2710), 0x00000000 },
	3753	+ { _MMIO(0x2714), 0x00800000 },
	3754	+ { _MMIO(0x2720), 0x00000000 },
	3755	+ { _MMIO(0x2724), 0x00800000 },
	3756	+};
	3757	+
	3758	+static const struct i915_oa_reg flex_eu_config_compute_extra[] = {
	3759	+ { _MMIO(0xe458), 0x00001000 },
	3760	+ { _MMIO(0xe558), 0x00003002 },
	3761	+ { _MMIO(0xe658), 0x00005004 },
	3762	+ { _MMIO(0xe758), 0x00011010 },
	3763	+ { _MMIO(0xe45c), 0x00050012 },
	3764	+ { _MMIO(0xe55c), 0x00052051 },
	3765	+ { _MMIO(0xe65c), 0x00000008 },
	3766	+};
	3767	+
	3768	+static const struct i915_oa_reg mux_config_compute_extra[] = {
	3769	+ { _MMIO(0x9888), 0x161503e0 },
	3770	+ { _MMIO(0x9888), 0x163503e0 },
	3771	+ { _MMIO(0x9888), 0x165503e0 },
	3772	+ { _MMIO(0x9888), 0x169503e0 },
	3773	+ { _MMIO(0x9888), 0x16b503e0 },
	3774	+ { _MMIO(0x9888), 0x16d503e0 },
	3775	+ { _MMIO(0x9888), 0x045cc000 },
	3776	+ { _MMIO(0x9888), 0x083d8000 },
	3777	+ { _MMIO(0x9888), 0x04584000 },
	3778	+ { _MMIO(0x9888), 0x085b4000 },
	3779	+ { _MMIO(0x9888), 0x0a5b8000 },
	3780	+ { _MMIO(0x9888), 0x0e1f00a8 },
	3781	+ { _MMIO(0x9888), 0x08384000 },
	3782	+ { _MMIO(0x9888), 0x0a384000 },
	3783	+ { _MMIO(0x9888), 0x0c388000 },
	3784	+ { _MMIO(0x9888), 0x0439a000 },
	3785	+ { _MMIO(0x9888), 0x06392000 },
	3786	+ { _MMIO(0x9888), 0x0c3a8000 },
	3787	+ { _MMIO(0x9888), 0x08138000 },
	3788	+ { _MMIO(0x9888), 0x06141000 },
	3789	+ { _MMIO(0x9888), 0x041500c3 },
	3790	+ { _MMIO(0x9888), 0x10150000 },
	3791	+ { _MMIO(0x9888), 0x0a338000 },
	3792	+ { _MMIO(0x9888), 0x06342000 },
	3793	+ { _MMIO(0x9888), 0x0435c300 },
	3794	+ { _MMIO(0x9888), 0x10350000 },
	3795	+ { _MMIO(0x9888), 0x0c538000 },
	3796	+ { _MMIO(0x9888), 0x06544000 },
	3797	+ { _MMIO(0x9888), 0x065500c3 },
	3798	+ { _MMIO(0x9888), 0x10550000 },
	3799	+ { _MMIO(0x9888), 0x00dc8000 },
	3800	+ { _MMIO(0x9888), 0x02dc4000 },
	3801	+ { _MMIO(0x9888), 0x02bd8000 },
	3802	+ { _MMIO(0x9888), 0x00d88000 },
	3803	+ { _MMIO(0x9888), 0x02db4000 },
	3804	+ { _MMIO(0x9888), 0x04db8000 },
	3805	+ { _MMIO(0x9888), 0x0c9fa000 },
	3806	+ { _MMIO(0x9888), 0x0e9f0002 },
	3807	+ { _MMIO(0x9888), 0x02b84000 },
	3808	+ { _MMIO(0x9888), 0x04b84000 },
	3809	+ { _MMIO(0x9888), 0x06b88000 },
	3810	+ { _MMIO(0x9888), 0x00b98000 },
	3811	+ { _MMIO(0x9888), 0x02b9a000 },
	3812	+ { _MMIO(0x9888), 0x06ba8000 },
	3813	+ { _MMIO(0x9888), 0x02938000 },
	3814	+ { _MMIO(0x9888), 0x04942000 },
	3815	+ { _MMIO(0x9888), 0x0095c300 },
	3816	+ { _MMIO(0x9888), 0x10950000 },
	3817	+ { _MMIO(0x9888), 0x04b38000 },
	3818	+ { _MMIO(0x9888), 0x04b44000 },
	3819	+ { _MMIO(0x9888), 0x02b500c3 },
	3820	+ { _MMIO(0x9888), 0x10b50000 },
	3821	+ { _MMIO(0x9888), 0x06d38000 },
	3822	+ { _MMIO(0x9888), 0x04d48000 },
	3823	+ { _MMIO(0x9888), 0x02d5c300 },
	3824	+ { _MMIO(0x9888), 0x10d50000 },
	3825	+ { _MMIO(0x9888), 0x03888000 },
	3826	+ { _MMIO(0x9888), 0x05888000 },
	3827	+ { _MMIO(0x9888), 0x07888000 },
	3828	+ { _MMIO(0x9888), 0x098a8000 },
	3829	+ { _MMIO(0x9888), 0x0b8a8000 },
	3830	+ { _MMIO(0x9888), 0x0d8a8000 },
	3831	+ { _MMIO(0x9888), 0x238b3500 },
	3832	+ { _MMIO(0x9888), 0x258b0005 },
	3833	+ { _MMIO(0x9888), 0x038c4000 },
	3834	+ { _MMIO(0x9888), 0x058c4000 },
	3835	+ { _MMIO(0x9888), 0x078c4000 },
	3836	+ { _MMIO(0x9888), 0x018d8000 },
	3837	+ { _MMIO(0x9888), 0x038da000 },
	3838	+ { _MMIO(0x9888), 0x1f85aa00 },
	3839	+ { _MMIO(0x9888), 0x2185000a },
	3840	+ { _MMIO(0x9888), 0x03834000 },
	3841	+ { _MMIO(0x9888), 0x05834000 },
	3842	+ { _MMIO(0x9888), 0x07834000 },
	3843	+ { _MMIO(0x9888), 0x09834000 },
	3844	+ { _MMIO(0x9888), 0x0b834000 },
	3845	+ { _MMIO(0x9888), 0x0d834000 },
	3846	+ { _MMIO(0x9888), 0x01848000 },
	3847	+ { _MMIO(0x9888), 0x0384c000 },
	3848	+ { _MMIO(0x9888), 0x0584c000 },
	3849	+ { _MMIO(0x9888), 0x07844000 },
	3850	+ { _MMIO(0x9888), 0x11808000 },
	3851	+ { _MMIO(0x9888), 0x1380c000 },
	3852	+ { _MMIO(0x9888), 0x1580c000 },
	3853	+ { _MMIO(0x9888), 0x17804000 },
	3854	+ { _MMIO(0x9888), 0x21800000 },
	3855	+ { _MMIO(0x9888), 0x4d800000 },
	3856	+ { _MMIO(0x9888), 0x3f800c40 },
	3857	+ { _MMIO(0x9888), 0x4f800000 },
	3858	+ { _MMIO(0x9888), 0x41801482 },
	3859	+ { _MMIO(0x9888), 0x31800000 },
	3860	+};
	3861	+
	3862	+static int
	3863	+get_compute_extra_mux_config(struct drm_i915_private *dev_priv,
	3864	+ const struct i915_oa_reg **regs,
	3865	+ int *lens)
	3866	+{
	3867	+ int n = 0;
	3868	+
	3869	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	3870	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	3871	+
	3872	+ regs[n] = mux_config_compute_extra;
	3873	+ lens[n] = ARRAY_SIZE(mux_config_compute_extra);
	3874	+ n++;
	3875	+
	3876	+ return n;
	3877	+}
	3878	+
	3879	+static const struct i915_oa_reg b_counter_config_vme_pipe[] = {
	3880	+ { _MMIO(0x2740), 0x00000000 },
	3881	+ { _MMIO(0x2710), 0x00000000 },
	3882	+ { _MMIO(0x2714), 0xf0800000 },
	3883	+ { _MMIO(0x2720), 0x00000000 },
	3884	+ { _MMIO(0x2724), 0x30800000 },
	3885	+ { _MMIO(0x2770), 0x00100030 },
	3886	+ { _MMIO(0x2774), 0x0000fff9 },
	3887	+ { _MMIO(0x2778), 0x00000002 },
	3888	+ { _MMIO(0x277c), 0x0000fffc },
	3889	+ { _MMIO(0x2780), 0x00000002 },
	3890	+ { _MMIO(0x2784), 0x0000fff3 },
	3891	+ { _MMIO(0x2788), 0x00100180 },
	3892	+ { _MMIO(0x278c), 0x0000ffcf },
	3893	+ { _MMIO(0x2790), 0x00000002 },
	3894	+ { _MMIO(0x2794), 0x0000ffcf },
	3895	+ { _MMIO(0x2798), 0x00000002 },
	3896	+ { _MMIO(0x279c), 0x0000ff3f },
	3897	+};
	3898	+
	3899	+static const struct i915_oa_reg flex_eu_config_vme_pipe[] = {
	3900	+ { _MMIO(0xe458), 0x00005004 },
	3901	+ { _MMIO(0xe558), 0x00008003 },
	3902	+};
	3903	+
	3904	+static const struct i915_oa_reg mux_config_vme_pipe[] = {
	3905	+ { _MMIO(0x9888), 0x14100812 },
	3906	+ { _MMIO(0x9888), 0x14125800 },
	3907	+ { _MMIO(0x9888), 0x161200c0 },
	3908	+ { _MMIO(0x9888), 0x14300812 },
	3909	+ { _MMIO(0x9888), 0x14325800 },
	3910	+ { _MMIO(0x9888), 0x163200c0 },
	3911	+ { _MMIO(0x9888), 0x005c4000 },
	3912	+ { _MMIO(0x9888), 0x065c8000 },
	3913	+ { _MMIO(0x9888), 0x085cc000 },
	3914	+ { _MMIO(0x9888), 0x0a5cc000 },
	3915	+ { _MMIO(0x9888), 0x0c5cc000 },
	3916	+ { _MMIO(0x9888), 0x003d8000 },
	3917	+ { _MMIO(0x9888), 0x0e3d8000 },
	3918	+ { _MMIO(0x9888), 0x183d2800 },
	3919	+ { _MMIO(0x9888), 0x00584000 },
	3920	+ { _MMIO(0x9888), 0x06588000 },
	3921	+ { _MMIO(0x9888), 0x0858c000 },
	3922	+ { _MMIO(0x9888), 0x005b4000 },
	3923	+ { _MMIO(0x9888), 0x0e5b4000 },
	3924	+ { _MMIO(0x9888), 0x185b9400 },
	3925	+ { _MMIO(0x9888), 0x1a5b002a },
	3926	+ { _MMIO(0x9888), 0x0c1f0800 },
	3927	+ { _MMIO(0x9888), 0x0e1faa00 },
	3928	+ { _MMIO(0x9888), 0x101f002a },
	3929	+ { _MMIO(0x9888), 0x00384000 },
	3930	+ { _MMIO(0x9888), 0x0e384000 },
	3931	+ { _MMIO(0x9888), 0x16384000 },
	3932	+ { _MMIO(0x9888), 0x18380155 },
	3933	+ { _MMIO(0x9888), 0x00392000 },
	3934	+ { _MMIO(0x9888), 0x06398000 },
	3935	+ { _MMIO(0x9888), 0x0839a000 },
	3936	+ { _MMIO(0x9888), 0x0a39a000 },
	3937	+ { _MMIO(0x9888), 0x0c39a000 },
	3938	+ { _MMIO(0x9888), 0x00100047 },
	3939	+ { _MMIO(0x9888), 0x06101a80 },
	3940	+ { _MMIO(0x9888), 0x10100000 },
	3941	+ { _MMIO(0x9888), 0x0810c000 },
	3942	+ { _MMIO(0x9888), 0x0811c000 },
	3943	+ { _MMIO(0x9888), 0x08126151 },
	3944	+ { _MMIO(0x9888), 0x10120000 },
	3945	+ { _MMIO(0x9888), 0x00134000 },
	3946	+ { _MMIO(0x9888), 0x0e134000 },
	3947	+ { _MMIO(0x9888), 0x161300a0 },
	3948	+ { _MMIO(0x9888), 0x0a301ac7 },
	3949	+ { _MMIO(0x9888), 0x10300000 },
	3950	+ { _MMIO(0x9888), 0x0c30c000 },
	3951	+ { _MMIO(0x9888), 0x0c31c000 },
	3952	+ { _MMIO(0x9888), 0x0c326151 },
	3953	+ { _MMIO(0x9888), 0x10320000 },
	3954	+ { _MMIO(0x9888), 0x16332a00 },
	3955	+ { _MMIO(0x9888), 0x18330001 },
	3956	+ { _MMIO(0x9888), 0x018a8000 },
	3957	+ { _MMIO(0x9888), 0x0f8a8000 },
	3958	+ { _MMIO(0x9888), 0x198a8000 },
	3959	+ { _MMIO(0x9888), 0x1b8a2aa0 },
	3960	+ { _MMIO(0x9888), 0x238b0020 },
	3961	+ { _MMIO(0x9888), 0x258b5550 },
	3962	+ { _MMIO(0x9888), 0x278b0001 },
	3963	+ { _MMIO(0x9888), 0x1f850080 },
	3964	+ { _MMIO(0x9888), 0x2185aaa0 },
	3965	+ { _MMIO(0x9888), 0x23850002 },
	3966	+ { _MMIO(0x9888), 0x01834000 },
	3967	+ { _MMIO(0x9888), 0x0f834000 },
	3968	+ { _MMIO(0x9888), 0x19835400 },
	3969	+ { _MMIO(0x9888), 0x1b830015 },
	3970	+ { _MMIO(0x9888), 0x01844000 },
	3971	+ { _MMIO(0x9888), 0x07848000 },
	3972	+ { _MMIO(0x9888), 0x0984c000 },
	3973	+ { _MMIO(0x9888), 0x0b84c000 },
	3974	+ { _MMIO(0x9888), 0x0d84c000 },
	3975	+ { _MMIO(0x9888), 0x11804000 },
	3976	+ { _MMIO(0x9888), 0x17808000 },
	3977	+ { _MMIO(0x9888), 0x1980c000 },
	3978	+ { _MMIO(0x9888), 0x1b80c000 },
	3979	+ { _MMIO(0x9888), 0x1d80c000 },
	3980	+ { _MMIO(0x9888), 0x4d800000 },
	3981	+ { _MMIO(0x9888), 0x3d800800 },
	3982	+ { _MMIO(0x9888), 0x4f800000 },
	3983	+ { _MMIO(0x9888), 0x43800002 },
	3984	+ { _MMIO(0x9888), 0x51800000 },
	3985	+ { _MMIO(0x9888), 0x45800884 },
	3986	+ { _MMIO(0x9888), 0x53800000 },
	3987	+ { _MMIO(0x9888), 0x47800002 },
	3988	+ { _MMIO(0x9888), 0x21800000 },
	3989	+ { _MMIO(0x9888), 0x31800000 },
	3990	+};
	3991	+
	3992	+static int
	3993	+get_vme_pipe_mux_config(struct drm_i915_private *dev_priv,
	3994	+ const struct i915_oa_reg **regs,
	3995	+ int *lens)
	3996	+{
	3997	+ int n = 0;
	3998	+
	3999	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	4000	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	4001	+
	4002	+ regs[n] = mux_config_vme_pipe;
	4003	+ lens[n] = ARRAY_SIZE(mux_config_vme_pipe);
	4004	+ n++;
	4005	+
	4006	+ return n;
	4007	+}
	4008	+
	4009	+static const struct i915_oa_reg b_counter_config_test_oa[] = {
	4010	+ { _MMIO(0x2740), 0x00000000 },
	4011	+ { _MMIO(0x2744), 0x00800000 },
	4012	+ { _MMIO(0x2714), 0xf0800000 },
	4013	+ { _MMIO(0x2710), 0x00000000 },
	4014	+ { _MMIO(0x2724), 0xf0800000 },
	4015	+ { _MMIO(0x2720), 0x00000000 },
	4016	+ { _MMIO(0x2770), 0x00000004 },
	4017	+ { _MMIO(0x2774), 0x00000000 },
	4018	+ { _MMIO(0x2778), 0x00000003 },
	4019	+ { _MMIO(0x277c), 0x00000000 },
	4020	+ { _MMIO(0x2780), 0x00000007 },
	4021	+ { _MMIO(0x2784), 0x00000000 },
	4022	+ { _MMIO(0x2788), 0x00100002 },
	4023	+ { _MMIO(0x278c), 0x0000fff7 },
	4024	+ { _MMIO(0x2790), 0x00100002 },
	4025	+ { _MMIO(0x2794), 0x0000ffcf },
	4026	+ { _MMIO(0x2798), 0x00100082 },
	4027	+ { _MMIO(0x279c), 0x0000ffef },
	4028	+ { _MMIO(0x27a0), 0x001000c2 },
	4029	+ { _MMIO(0x27a4), 0x0000ffe7 },
	4030	+ { _MMIO(0x27a8), 0x00100001 },
	4031	+ { _MMIO(0x27ac), 0x0000ffe7 },
	4032	+};
	4033	+
	4034	+static const struct i915_oa_reg flex_eu_config_test_oa[] = {
	4035	+};
	4036	+
	4037	+static const struct i915_oa_reg mux_config_test_oa[] = {
	4038	+ { _MMIO(0x9888), 0x198b0000 },
	4039	+ { _MMIO(0x9888), 0x078b0066 },
	4040	+ { _MMIO(0x9888), 0x118b0000 },
	4041	+ { _MMIO(0x9888), 0x258b0000 },
	4042	+ { _MMIO(0x9888), 0x21850008 },
	4043	+ { _MMIO(0x9888), 0x0d834000 },
	4044	+ { _MMIO(0x9888), 0x07844000 },
	4045	+ { _MMIO(0x9888), 0x17804000 },
	4046	+ { _MMIO(0x9888), 0x21800000 },
	4047	+ { _MMIO(0x9888), 0x4f800000 },
	4048	+ { _MMIO(0x9888), 0x41800000 },
	4049	+ { _MMIO(0x9888), 0x31800000 },
	4050	+};
	4051	+
	4052	+static int
	4053	+get_test_oa_mux_config(struct drm_i915_private *dev_priv,
	4054	+ const struct i915_oa_reg **regs,
	4055	+ int *lens)
	4056	+{
	4057	+ int n = 0;
	4058	+
	4059	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	4060	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	4061	+
	4062	+ regs[n] = mux_config_test_oa;
	4063	+ lens[n] = ARRAY_SIZE(mux_config_test_oa);
	4064	+ n++;
	4065	+
	4066	+ return n;
	4067	+}
	4068	+
	4069	+int i915_oa_select_metric_set_bdw(struct drm_i915_private *dev_priv)
	4070	+{
	4071	+ dev_priv->perf.oa.n_mux_configs = 0;
	4072	+ dev_priv->perf.oa.b_counter_regs = NULL;
	4073	+ dev_priv->perf.oa.b_counter_regs_len = 0;
	4074	+ dev_priv->perf.oa.flex_regs = NULL;
	4075	+ dev_priv->perf.oa.flex_regs_len = 0;
	4076	+
	4077	+ switch (dev_priv->perf.oa.metrics_set) {
	4078	+ case METRIC_SET_ID_RENDER_BASIC:
	4079	+ dev_priv->perf.oa.n_mux_configs =
	4080	+ get_render_basic_mux_config(dev_priv,
	4081	+ dev_priv->perf.oa.mux_regs,
	4082	+ dev_priv->perf.oa.mux_regs_lens);
	4083	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4084	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RENDER_BASIC\" metric set\n");
	4085	+
	4086	+ /* EINVAL because *_register_sysfs already checked this
	4087	+ * and so it wouldn't have been advertised to userspace and
	4088	+ * so shouldn't have been requested
	4089	+ */
	4090	+ return -EINVAL;
	4091	+ }
	4092	+
	4093	+ dev_priv->perf.oa.b_counter_regs =
	4094	+ b_counter_config_render_basic;
	4095	+ dev_priv->perf.oa.b_counter_regs_len =
	4096	+ ARRAY_SIZE(b_counter_config_render_basic);
	4097	+
	4098	+ dev_priv->perf.oa.flex_regs =
	4099	+ flex_eu_config_render_basic;
	4100	+ dev_priv->perf.oa.flex_regs_len =
	4101	+ ARRAY_SIZE(flex_eu_config_render_basic);
	4102	+
	4103	+ return 0;
	4104	+ case METRIC_SET_ID_COMPUTE_BASIC:
	4105	+ dev_priv->perf.oa.n_mux_configs =
	4106	+ get_compute_basic_mux_config(dev_priv,
	4107	+ dev_priv->perf.oa.mux_regs,
	4108	+ dev_priv->perf.oa.mux_regs_lens);
	4109	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4110	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_BASIC\" metric set\n");
	4111	+
	4112	+ /* EINVAL because *_register_sysfs already checked this
	4113	+ * and so it wouldn't have been advertised to userspace and
	4114	+ * so shouldn't have been requested
	4115	+ */
	4116	+ return -EINVAL;
	4117	+ }
	4118	+
	4119	+ dev_priv->perf.oa.b_counter_regs =
	4120	+ b_counter_config_compute_basic;
	4121	+ dev_priv->perf.oa.b_counter_regs_len =
	4122	+ ARRAY_SIZE(b_counter_config_compute_basic);
	4123	+
	4124	+ dev_priv->perf.oa.flex_regs =
	4125	+ flex_eu_config_compute_basic;
	4126	+ dev_priv->perf.oa.flex_regs_len =
	4127	+ ARRAY_SIZE(flex_eu_config_compute_basic);
	4128	+
	4129	+ return 0;
	4130	+ case METRIC_SET_ID_RENDER_PIPE_PROFILE:
	4131	+ dev_priv->perf.oa.n_mux_configs =
	4132	+ get_render_pipe_profile_mux_config(dev_priv,
	4133	+ dev_priv->perf.oa.mux_regs,
	4134	+ dev_priv->perf.oa.mux_regs_lens);
	4135	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4136	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RENDER_PIPE_PROFILE\" metric set\n");
	4137	+
	4138	+ /* EINVAL because *_register_sysfs already checked this
	4139	+ * and so it wouldn't have been advertised to userspace and
	4140	+ * so shouldn't have been requested
	4141	+ */
	4142	+ return -EINVAL;
	4143	+ }
	4144	+
	4145	+ dev_priv->perf.oa.b_counter_regs =
	4146	+ b_counter_config_render_pipe_profile;
	4147	+ dev_priv->perf.oa.b_counter_regs_len =
	4148	+ ARRAY_SIZE(b_counter_config_render_pipe_profile);
	4149	+
	4150	+ dev_priv->perf.oa.flex_regs =
	4151	+ flex_eu_config_render_pipe_profile;
	4152	+ dev_priv->perf.oa.flex_regs_len =
	4153	+ ARRAY_SIZE(flex_eu_config_render_pipe_profile);
	4154	+
	4155	+ return 0;
	4156	+ case METRIC_SET_ID_MEMORY_READS:
	4157	+ dev_priv->perf.oa.n_mux_configs =
	4158	+ get_memory_reads_mux_config(dev_priv,
	4159	+ dev_priv->perf.oa.mux_regs,
	4160	+ dev_priv->perf.oa.mux_regs_lens);
	4161	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4162	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"MEMORY_READS\" metric set\n");
	4163	+
	4164	+ /* EINVAL because *_register_sysfs already checked this
	4165	+ * and so it wouldn't have been advertised to userspace and
	4166	+ * so shouldn't have been requested
	4167	+ */
	4168	+ return -EINVAL;
	4169	+ }
	4170	+
	4171	+ dev_priv->perf.oa.b_counter_regs =
	4172	+ b_counter_config_memory_reads;
	4173	+ dev_priv->perf.oa.b_counter_regs_len =
	4174	+ ARRAY_SIZE(b_counter_config_memory_reads);
	4175	+
	4176	+ dev_priv->perf.oa.flex_regs =
	4177	+ flex_eu_config_memory_reads;
	4178	+ dev_priv->perf.oa.flex_regs_len =
	4179	+ ARRAY_SIZE(flex_eu_config_memory_reads);
	4180	+
	4181	+ return 0;
	4182	+ case METRIC_SET_ID_MEMORY_WRITES:
	4183	+ dev_priv->perf.oa.n_mux_configs =
	4184	+ get_memory_writes_mux_config(dev_priv,
	4185	+ dev_priv->perf.oa.mux_regs,
	4186	+ dev_priv->perf.oa.mux_regs_lens);
	4187	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4188	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"MEMORY_WRITES\" metric set\n");
	4189	+
	4190	+ /* EINVAL because *_register_sysfs already checked this
	4191	+ * and so it wouldn't have been advertised to userspace and
	4192	+ * so shouldn't have been requested
	4193	+ */
	4194	+ return -EINVAL;
	4195	+ }
	4196	+
	4197	+ dev_priv->perf.oa.b_counter_regs =
	4198	+ b_counter_config_memory_writes;
	4199	+ dev_priv->perf.oa.b_counter_regs_len =
	4200	+ ARRAY_SIZE(b_counter_config_memory_writes);
	4201	+
	4202	+ dev_priv->perf.oa.flex_regs =
	4203	+ flex_eu_config_memory_writes;
	4204	+ dev_priv->perf.oa.flex_regs_len =
	4205	+ ARRAY_SIZE(flex_eu_config_memory_writes);
	4206	+
	4207	+ return 0;
	4208	+ case METRIC_SET_ID_COMPUTE_EXTENDED:
	4209	+ dev_priv->perf.oa.n_mux_configs =
	4210	+ get_compute_extended_mux_config(dev_priv,
	4211	+ dev_priv->perf.oa.mux_regs,
	4212	+ dev_priv->perf.oa.mux_regs_lens);
	4213	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4214	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_EXTENDED\" metric set\n");
	4215	+
	4216	+ /* EINVAL because *_register_sysfs already checked this
	4217	+ * and so it wouldn't have been advertised to userspace and
	4218	+ * so shouldn't have been requested
	4219	+ */
	4220	+ return -EINVAL;
	4221	+ }
	4222	+
	4223	+ dev_priv->perf.oa.b_counter_regs =
	4224	+ b_counter_config_compute_extended;
	4225	+ dev_priv->perf.oa.b_counter_regs_len =
	4226	+ ARRAY_SIZE(b_counter_config_compute_extended);
	4227	+
	4228	+ dev_priv->perf.oa.flex_regs =
	4229	+ flex_eu_config_compute_extended;
	4230	+ dev_priv->perf.oa.flex_regs_len =
	4231	+ ARRAY_SIZE(flex_eu_config_compute_extended);
	4232	+
	4233	+ return 0;
	4234	+ case METRIC_SET_ID_COMPUTE_L3_CACHE:
	4235	+ dev_priv->perf.oa.n_mux_configs =
	4236	+ get_compute_l3_cache_mux_config(dev_priv,
	4237	+ dev_priv->perf.oa.mux_regs,
	4238	+ dev_priv->perf.oa.mux_regs_lens);
	4239	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4240	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_L3_CACHE\" metric set\n");
	4241	+
	4242	+ /* EINVAL because *_register_sysfs already checked this
	4243	+ * and so it wouldn't have been advertised to userspace and
	4244	+ * so shouldn't have been requested
	4245	+ */
	4246	+ return -EINVAL;
	4247	+ }
	4248	+
	4249	+ dev_priv->perf.oa.b_counter_regs =
	4250	+ b_counter_config_compute_l3_cache;
	4251	+ dev_priv->perf.oa.b_counter_regs_len =
	4252	+ ARRAY_SIZE(b_counter_config_compute_l3_cache);
	4253	+
	4254	+ dev_priv->perf.oa.flex_regs =
	4255	+ flex_eu_config_compute_l3_cache;
	4256	+ dev_priv->perf.oa.flex_regs_len =
	4257	+ ARRAY_SIZE(flex_eu_config_compute_l3_cache);
	4258	+
	4259	+ return 0;
	4260	+ case METRIC_SET_ID_DATA_PORT_READS_COALESCING:
	4261	+ dev_priv->perf.oa.n_mux_configs =
	4262	+ get_data_port_reads_coalescing_mux_config(dev_priv,
	4263	+ dev_priv->perf.oa.mux_regs,
	4264	+ dev_priv->perf.oa.mux_regs_lens);
	4265	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4266	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"DATA_PORT_READS_COALESCING\" metric set\n");
	4267	+
	4268	+ /* EINVAL because *_register_sysfs already checked this
	4269	+ * and so it wouldn't have been advertised to userspace and
	4270	+ * so shouldn't have been requested
	4271	+ */
	4272	+ return -EINVAL;
	4273	+ }
	4274	+
	4275	+ dev_priv->perf.oa.b_counter_regs =
	4276	+ b_counter_config_data_port_reads_coalescing;
	4277	+ dev_priv->perf.oa.b_counter_regs_len =
	4278	+ ARRAY_SIZE(b_counter_config_data_port_reads_coalescing);
	4279	+
	4280	+ dev_priv->perf.oa.flex_regs =
	4281	+ flex_eu_config_data_port_reads_coalescing;
	4282	+ dev_priv->perf.oa.flex_regs_len =
	4283	+ ARRAY_SIZE(flex_eu_config_data_port_reads_coalescing);
	4284	+
	4285	+ return 0;
	4286	+ case METRIC_SET_ID_DATA_PORT_WRITES_COALESCING:
	4287	+ dev_priv->perf.oa.n_mux_configs =
	4288	+ get_data_port_writes_coalescing_mux_config(dev_priv,
	4289	+ dev_priv->perf.oa.mux_regs,
	4290	+ dev_priv->perf.oa.mux_regs_lens);
	4291	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4292	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"DATA_PORT_WRITES_COALESCING\" metric set\n");
	4293	+
	4294	+ /* EINVAL because *_register_sysfs already checked this
	4295	+ * and so it wouldn't have been advertised to userspace and
	4296	+ * so shouldn't have been requested
	4297	+ */
	4298	+ return -EINVAL;
	4299	+ }
	4300	+
	4301	+ dev_priv->perf.oa.b_counter_regs =
	4302	+ b_counter_config_data_port_writes_coalescing;
	4303	+ dev_priv->perf.oa.b_counter_regs_len =
	4304	+ ARRAY_SIZE(b_counter_config_data_port_writes_coalescing);
	4305	+
	4306	+ dev_priv->perf.oa.flex_regs =
	4307	+ flex_eu_config_data_port_writes_coalescing;
	4308	+ dev_priv->perf.oa.flex_regs_len =
	4309	+ ARRAY_SIZE(flex_eu_config_data_port_writes_coalescing);
	4310	+
	4311	+ return 0;
	4312	+ case METRIC_SET_ID_HDC_AND_SF:
	4313	+ dev_priv->perf.oa.n_mux_configs =
	4314	+ get_hdc_and_sf_mux_config(dev_priv,
	4315	+ dev_priv->perf.oa.mux_regs,
	4316	+ dev_priv->perf.oa.mux_regs_lens);
	4317	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4318	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"HDC_AND_SF\" metric set\n");
	4319	+
	4320	+ /* EINVAL because *_register_sysfs already checked this
	4321	+ * and so it wouldn't have been advertised to userspace and
	4322	+ * so shouldn't have been requested
	4323	+ */
	4324	+ return -EINVAL;
	4325	+ }
	4326	+
	4327	+ dev_priv->perf.oa.b_counter_regs =
	4328	+ b_counter_config_hdc_and_sf;
	4329	+ dev_priv->perf.oa.b_counter_regs_len =
	4330	+ ARRAY_SIZE(b_counter_config_hdc_and_sf);
	4331	+
	4332	+ dev_priv->perf.oa.flex_regs =
	4333	+ flex_eu_config_hdc_and_sf;
	4334	+ dev_priv->perf.oa.flex_regs_len =
	4335	+ ARRAY_SIZE(flex_eu_config_hdc_and_sf);
	4336	+
	4337	+ return 0;
	4338	+ case METRIC_SET_ID_L3_1:
	4339	+ dev_priv->perf.oa.n_mux_configs =
	4340	+ get_l3_1_mux_config(dev_priv,
	4341	+ dev_priv->perf.oa.mux_regs,
	4342	+ dev_priv->perf.oa.mux_regs_lens);
	4343	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4344	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_1\" metric set\n");
	4345	+
	4346	+ /* EINVAL because *_register_sysfs already checked this
	4347	+ * and so it wouldn't have been advertised to userspace and
	4348	+ * so shouldn't have been requested
	4349	+ */
	4350	+ return -EINVAL;
	4351	+ }
	4352	+
	4353	+ dev_priv->perf.oa.b_counter_regs =
	4354	+ b_counter_config_l3_1;
	4355	+ dev_priv->perf.oa.b_counter_regs_len =
	4356	+ ARRAY_SIZE(b_counter_config_l3_1);
	4357	+
	4358	+ dev_priv->perf.oa.flex_regs =
	4359	+ flex_eu_config_l3_1;
	4360	+ dev_priv->perf.oa.flex_regs_len =
	4361	+ ARRAY_SIZE(flex_eu_config_l3_1);
	4362	+
	4363	+ return 0;
	4364	+ case METRIC_SET_ID_L3_2:
	4365	+ dev_priv->perf.oa.n_mux_configs =
	4366	+ get_l3_2_mux_config(dev_priv,
	4367	+ dev_priv->perf.oa.mux_regs,
	4368	+ dev_priv->perf.oa.mux_regs_lens);
	4369	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4370	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_2\" metric set\n");
	4371	+
	4372	+ /* EINVAL because *_register_sysfs already checked this
	4373	+ * and so it wouldn't have been advertised to userspace and
	4374	+ * so shouldn't have been requested
	4375	+ */
	4376	+ return -EINVAL;
	4377	+ }
	4378	+
	4379	+ dev_priv->perf.oa.b_counter_regs =
	4380	+ b_counter_config_l3_2;
	4381	+ dev_priv->perf.oa.b_counter_regs_len =
	4382	+ ARRAY_SIZE(b_counter_config_l3_2);
	4383	+
	4384	+ dev_priv->perf.oa.flex_regs =
	4385	+ flex_eu_config_l3_2;
	4386	+ dev_priv->perf.oa.flex_regs_len =
	4387	+ ARRAY_SIZE(flex_eu_config_l3_2);
	4388	+
	4389	+ return 0;
	4390	+ case METRIC_SET_ID_L3_3:
	4391	+ dev_priv->perf.oa.n_mux_configs =
	4392	+ get_l3_3_mux_config(dev_priv,
	4393	+ dev_priv->perf.oa.mux_regs,
	4394	+ dev_priv->perf.oa.mux_regs_lens);
	4395	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4396	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_3\" metric set\n");
	4397	+
	4398	+ /* EINVAL because *_register_sysfs already checked this
	4399	+ * and so it wouldn't have been advertised to userspace and
	4400	+ * so shouldn't have been requested
	4401	+ */
	4402	+ return -EINVAL;
	4403	+ }
	4404	+
	4405	+ dev_priv->perf.oa.b_counter_regs =
	4406	+ b_counter_config_l3_3;
	4407	+ dev_priv->perf.oa.b_counter_regs_len =
	4408	+ ARRAY_SIZE(b_counter_config_l3_3);
	4409	+
	4410	+ dev_priv->perf.oa.flex_regs =
	4411	+ flex_eu_config_l3_3;
	4412	+ dev_priv->perf.oa.flex_regs_len =
	4413	+ ARRAY_SIZE(flex_eu_config_l3_3);
	4414	+
	4415	+ return 0;
	4416	+ case METRIC_SET_ID_L3_4:
	4417	+ dev_priv->perf.oa.n_mux_configs =
	4418	+ get_l3_4_mux_config(dev_priv,
	4419	+ dev_priv->perf.oa.mux_regs,
	4420	+ dev_priv->perf.oa.mux_regs_lens);
	4421	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4422	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_4\" metric set\n");
	4423	+
	4424	+ /* EINVAL because *_register_sysfs already checked this
	4425	+ * and so it wouldn't have been advertised to userspace and
	4426	+ * so shouldn't have been requested
	4427	+ */
	4428	+ return -EINVAL;
	4429	+ }
	4430	+
	4431	+ dev_priv->perf.oa.b_counter_regs =
	4432	+ b_counter_config_l3_4;
	4433	+ dev_priv->perf.oa.b_counter_regs_len =
	4434	+ ARRAY_SIZE(b_counter_config_l3_4);
	4435	+
	4436	+ dev_priv->perf.oa.flex_regs =
	4437	+ flex_eu_config_l3_4;
	4438	+ dev_priv->perf.oa.flex_regs_len =
	4439	+ ARRAY_SIZE(flex_eu_config_l3_4);
	4440	+
	4441	+ return 0;
	4442	+ case METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND:
	4443	+ dev_priv->perf.oa.n_mux_configs =
	4444	+ get_rasterizer_and_pixel_backend_mux_config(dev_priv,
	4445	+ dev_priv->perf.oa.mux_regs,
	4446	+ dev_priv->perf.oa.mux_regs_lens);
	4447	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4448	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RASTERIZER_AND_PIXEL_BACKEND\" metric set\n");
	4449	+
	4450	+ /* EINVAL because *_register_sysfs already checked this
	4451	+ * and so it wouldn't have been advertised to userspace and
	4452	+ * so shouldn't have been requested
	4453	+ */
	4454	+ return -EINVAL;
	4455	+ }
	4456	+
	4457	+ dev_priv->perf.oa.b_counter_regs =
	4458	+ b_counter_config_rasterizer_and_pixel_backend;
	4459	+ dev_priv->perf.oa.b_counter_regs_len =
	4460	+ ARRAY_SIZE(b_counter_config_rasterizer_and_pixel_backend);
	4461	+
	4462	+ dev_priv->perf.oa.flex_regs =
	4463	+ flex_eu_config_rasterizer_and_pixel_backend;
	4464	+ dev_priv->perf.oa.flex_regs_len =
	4465	+ ARRAY_SIZE(flex_eu_config_rasterizer_and_pixel_backend);
	4466	+
	4467	+ return 0;
	4468	+ case METRIC_SET_ID_SAMPLER_1:
	4469	+ dev_priv->perf.oa.n_mux_configs =
	4470	+ get_sampler_1_mux_config(dev_priv,
	4471	+ dev_priv->perf.oa.mux_regs,
	4472	+ dev_priv->perf.oa.mux_regs_lens);
	4473	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4474	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"SAMPLER_1\" metric set\n");
	4475	+
	4476	+ /* EINVAL because *_register_sysfs already checked this
	4477	+ * and so it wouldn't have been advertised to userspace and
	4478	+ * so shouldn't have been requested
	4479	+ */
	4480	+ return -EINVAL;
	4481	+ }
	4482	+
	4483	+ dev_priv->perf.oa.b_counter_regs =
	4484	+ b_counter_config_sampler_1;
	4485	+ dev_priv->perf.oa.b_counter_regs_len =
	4486	+ ARRAY_SIZE(b_counter_config_sampler_1);
	4487	+
	4488	+ dev_priv->perf.oa.flex_regs =
	4489	+ flex_eu_config_sampler_1;
	4490	+ dev_priv->perf.oa.flex_regs_len =
	4491	+ ARRAY_SIZE(flex_eu_config_sampler_1);
	4492	+
	4493	+ return 0;
	4494	+ case METRIC_SET_ID_SAMPLER_2:
	4495	+ dev_priv->perf.oa.n_mux_configs =
	4496	+ get_sampler_2_mux_config(dev_priv,
	4497	+ dev_priv->perf.oa.mux_regs,
	4498	+ dev_priv->perf.oa.mux_regs_lens);
	4499	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4500	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"SAMPLER_2\" metric set\n");
	4501	+
	4502	+ /* EINVAL because *_register_sysfs already checked this
	4503	+ * and so it wouldn't have been advertised to userspace and
	4504	+ * so shouldn't have been requested
	4505	+ */
	4506	+ return -EINVAL;
	4507	+ }
	4508	+
	4509	+ dev_priv->perf.oa.b_counter_regs =
	4510	+ b_counter_config_sampler_2;
	4511	+ dev_priv->perf.oa.b_counter_regs_len =
	4512	+ ARRAY_SIZE(b_counter_config_sampler_2);
	4513	+
	4514	+ dev_priv->perf.oa.flex_regs =
	4515	+ flex_eu_config_sampler_2;
	4516	+ dev_priv->perf.oa.flex_regs_len =
	4517	+ ARRAY_SIZE(flex_eu_config_sampler_2);
	4518	+
	4519	+ return 0;
	4520	+ case METRIC_SET_ID_TDL_1:
	4521	+ dev_priv->perf.oa.n_mux_configs =
	4522	+ get_tdl_1_mux_config(dev_priv,
	4523	+ dev_priv->perf.oa.mux_regs,
	4524	+ dev_priv->perf.oa.mux_regs_lens);
	4525	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4526	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TDL_1\" metric set\n");
	4527	+
	4528	+ /* EINVAL because *_register_sysfs already checked this
	4529	+ * and so it wouldn't have been advertised to userspace and
	4530	+ * so shouldn't have been requested
	4531	+ */
	4532	+ return -EINVAL;
	4533	+ }
	4534	+
	4535	+ dev_priv->perf.oa.b_counter_regs =
	4536	+ b_counter_config_tdl_1;
	4537	+ dev_priv->perf.oa.b_counter_regs_len =
	4538	+ ARRAY_SIZE(b_counter_config_tdl_1);
	4539	+
	4540	+ dev_priv->perf.oa.flex_regs =
	4541	+ flex_eu_config_tdl_1;
	4542	+ dev_priv->perf.oa.flex_regs_len =
	4543	+ ARRAY_SIZE(flex_eu_config_tdl_1);
	4544	+
	4545	+ return 0;
	4546	+ case METRIC_SET_ID_TDL_2:
	4547	+ dev_priv->perf.oa.n_mux_configs =
	4548	+ get_tdl_2_mux_config(dev_priv,
	4549	+ dev_priv->perf.oa.mux_regs,
	4550	+ dev_priv->perf.oa.mux_regs_lens);
	4551	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4552	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TDL_2\" metric set\n");
	4553	+
	4554	+ /* EINVAL because *_register_sysfs already checked this
	4555	+ * and so it wouldn't have been advertised to userspace and
	4556	+ * so shouldn't have been requested
	4557	+ */
	4558	+ return -EINVAL;
	4559	+ }
	4560	+
	4561	+ dev_priv->perf.oa.b_counter_regs =
	4562	+ b_counter_config_tdl_2;
	4563	+ dev_priv->perf.oa.b_counter_regs_len =
	4564	+ ARRAY_SIZE(b_counter_config_tdl_2);
	4565	+
	4566	+ dev_priv->perf.oa.flex_regs =
	4567	+ flex_eu_config_tdl_2;
	4568	+ dev_priv->perf.oa.flex_regs_len =
	4569	+ ARRAY_SIZE(flex_eu_config_tdl_2);
	4570	+
	4571	+ return 0;
	4572	+ case METRIC_SET_ID_COMPUTE_EXTRA:
	4573	+ dev_priv->perf.oa.n_mux_configs =
	4574	+ get_compute_extra_mux_config(dev_priv,
	4575	+ dev_priv->perf.oa.mux_regs,
	4576	+ dev_priv->perf.oa.mux_regs_lens);
	4577	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4578	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_EXTRA\" metric set\n");
	4579	+
	4580	+ /* EINVAL because *_register_sysfs already checked this
	4581	+ * and so it wouldn't have been advertised to userspace and
	4582	+ * so shouldn't have been requested
	4583	+ */
	4584	+ return -EINVAL;
	4585	+ }
	4586	+
	4587	+ dev_priv->perf.oa.b_counter_regs =
	4588	+ b_counter_config_compute_extra;
	4589	+ dev_priv->perf.oa.b_counter_regs_len =
	4590	+ ARRAY_SIZE(b_counter_config_compute_extra);
	4591	+
	4592	+ dev_priv->perf.oa.flex_regs =
	4593	+ flex_eu_config_compute_extra;
	4594	+ dev_priv->perf.oa.flex_regs_len =
	4595	+ ARRAY_SIZE(flex_eu_config_compute_extra);
	4596	+
	4597	+ return 0;
	4598	+ case METRIC_SET_ID_VME_PIPE:
	4599	+ dev_priv->perf.oa.n_mux_configs =
	4600	+ get_vme_pipe_mux_config(dev_priv,
	4601	+ dev_priv->perf.oa.mux_regs,
	4602	+ dev_priv->perf.oa.mux_regs_lens);
	4603	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4604	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"VME_PIPE\" metric set\n");
	4605	+
	4606	+ /* EINVAL because *_register_sysfs already checked this
	4607	+ * and so it wouldn't have been advertised to userspace and
	4608	+ * so shouldn't have been requested
	4609	+ */
	4610	+ return -EINVAL;
	4611	+ }
	4612	+
	4613	+ dev_priv->perf.oa.b_counter_regs =
	4614	+ b_counter_config_vme_pipe;
	4615	+ dev_priv->perf.oa.b_counter_regs_len =
	4616	+ ARRAY_SIZE(b_counter_config_vme_pipe);
	4617	+
	4618	+ dev_priv->perf.oa.flex_regs =
	4619	+ flex_eu_config_vme_pipe;
	4620	+ dev_priv->perf.oa.flex_regs_len =
	4621	+ ARRAY_SIZE(flex_eu_config_vme_pipe);
	4622	+
	4623	+ return 0;
	4624	+ case METRIC_SET_ID_TEST_OA:
	4625	+ dev_priv->perf.oa.n_mux_configs =
	4626	+ get_test_oa_mux_config(dev_priv,
	4627	+ dev_priv->perf.oa.mux_regs,
	4628	+ dev_priv->perf.oa.mux_regs_lens);
	4629	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	4630	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TEST_OA\" metric set\n");
	4631	+
	4632	+ /* EINVAL because *_register_sysfs already checked this
	4633	+ * and so it wouldn't have been advertised to userspace and
	4634	+ * so shouldn't have been requested
	4635	+ */
	4636	+ return -EINVAL;
	4637	+ }
	4638	+
	4639	+ dev_priv->perf.oa.b_counter_regs =
	4640	+ b_counter_config_test_oa;
	4641	+ dev_priv->perf.oa.b_counter_regs_len =
	4642	+ ARRAY_SIZE(b_counter_config_test_oa);
	4643	+
	4644	+ dev_priv->perf.oa.flex_regs =
	4645	+ flex_eu_config_test_oa;
	4646	+ dev_priv->perf.oa.flex_regs_len =
	4647	+ ARRAY_SIZE(flex_eu_config_test_oa);
	4648	+
	4649	+ return 0;
	4650	+ default:
	4651	+ return -ENODEV;
	4652	+ }
	4653	+}
	4654	+
	4655	+static ssize_t
	4656	+show_render_basic_id(struct device kdev, struct device_attribute attr, char *buf)
	4657	+{
	4658	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RENDER_BASIC);
	4659	+}
	4660	+
	4661	+static struct device_attribute dev_attr_render_basic_id = {
	4662	+ .attr = { .name = "id", .mode = 0444 },
	4663	+ .show = show_render_basic_id,
	4664	+ .store = NULL,
	4665	+};
	4666	+
	4667	+static struct attribute *attrs_render_basic[] = {
	4668	+ &dev_attr_render_basic_id.attr,
	4669	+ NULL,
	4670	+};
	4671	+
	4672	+static struct attribute_group group_render_basic = {
	4673	+ .name = "b541bd57-0e0f-4154-b4c0-5858010a2bf7",
	4674	+ .attrs = attrs_render_basic,
	4675	+};
	4676	+
	4677	+static ssize_t
	4678	+show_compute_basic_id(struct device kdev, struct device_attribute attr, char *buf)
	4679	+{
	4680	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_BASIC);
	4681	+}
	4682	+
	4683	+static struct device_attribute dev_attr_compute_basic_id = {
	4684	+ .attr = { .name = "id", .mode = 0444 },
	4685	+ .show = show_compute_basic_id,
	4686	+ .store = NULL,
	4687	+};
	4688	+
	4689	+static struct attribute *attrs_compute_basic[] = {
	4690	+ &dev_attr_compute_basic_id.attr,
	4691	+ NULL,
	4692	+};
	4693	+
	4694	+static struct attribute_group group_compute_basic = {
	4695	+ .name = "35fbc9b2-a891-40a6-a38d-022bb7057552",
	4696	+ .attrs = attrs_compute_basic,
	4697	+};
	4698	+
	4699	+static ssize_t
	4700	+show_render_pipe_profile_id(struct device kdev, struct device_attribute attr, char *buf)
	4701	+{
	4702	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RENDER_PIPE_PROFILE);
	4703	+}
	4704	+
	4705	+static struct device_attribute dev_attr_render_pipe_profile_id = {
	4706	+ .attr = { .name = "id", .mode = 0444 },
	4707	+ .show = show_render_pipe_profile_id,
	4708	+ .store = NULL,
	4709	+};
	4710	+
	4711	+static struct attribute *attrs_render_pipe_profile[] = {
	4712	+ &dev_attr_render_pipe_profile_id.attr,
	4713	+ NULL,
	4714	+};
	4715	+
	4716	+static struct attribute_group group_render_pipe_profile = {
	4717	+ .name = "233d0544-fff7-4281-8291-e02f222aff72",
	4718	+ .attrs = attrs_render_pipe_profile,
	4719	+};
	4720	+
	4721	+static ssize_t
	4722	+show_memory_reads_id(struct device kdev, struct device_attribute attr, char *buf)
	4723	+{
	4724	+ return sprintf(buf, "%d\n", METRIC_SET_ID_MEMORY_READS);
	4725	+}
	4726	+
	4727	+static struct device_attribute dev_attr_memory_reads_id = {
	4728	+ .attr = { .name = "id", .mode = 0444 },
	4729	+ .show = show_memory_reads_id,
	4730	+ .store = NULL,
	4731	+};
	4732	+
	4733	+static struct attribute *attrs_memory_reads[] = {
	4734	+ &dev_attr_memory_reads_id.attr,
	4735	+ NULL,
	4736	+};
	4737	+
	4738	+static struct attribute_group group_memory_reads = {
	4739	+ .name = "2b255d48-2117-4fef-a8f7-f151e1d25a2c",
	4740	+ .attrs = attrs_memory_reads,
	4741	+};
	4742	+
	4743	+static ssize_t
	4744	+show_memory_writes_id(struct device kdev, struct device_attribute attr, char *buf)
	4745	+{
	4746	+ return sprintf(buf, "%d\n", METRIC_SET_ID_MEMORY_WRITES);
	4747	+}
	4748	+
	4749	+static struct device_attribute dev_attr_memory_writes_id = {
	4750	+ .attr = { .name = "id", .mode = 0444 },
	4751	+ .show = show_memory_writes_id,
	4752	+ .store = NULL,
	4753	+};
	4754	+
	4755	+static struct attribute *attrs_memory_writes[] = {
	4756	+ &dev_attr_memory_writes_id.attr,
	4757	+ NULL,
	4758	+};
	4759	+
	4760	+static struct attribute_group group_memory_writes = {
	4761	+ .name = "f7fd3220-b466-4a4d-9f98-b0caf3f2394c",
	4762	+ .attrs = attrs_memory_writes,
	4763	+};
	4764	+
	4765	+static ssize_t
	4766	+show_compute_extended_id(struct device kdev, struct device_attribute attr, char *buf)
	4767	+{
	4768	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_EXTENDED);
	4769	+}
	4770	+
	4771	+static struct device_attribute dev_attr_compute_extended_id = {
	4772	+ .attr = { .name = "id", .mode = 0444 },
	4773	+ .show = show_compute_extended_id,
	4774	+ .store = NULL,
	4775	+};
	4776	+
	4777	+static struct attribute *attrs_compute_extended[] = {
	4778	+ &dev_attr_compute_extended_id.attr,
	4779	+ NULL,
	4780	+};
	4781	+
	4782	+static struct attribute_group group_compute_extended = {
	4783	+ .name = "e99ccaca-821c-4df9-97a7-96bdb7204e43",
	4784	+ .attrs = attrs_compute_extended,
	4785	+};
	4786	+
	4787	+static ssize_t
	4788	+show_compute_l3_cache_id(struct device kdev, struct device_attribute attr, char *buf)
	4789	+{
	4790	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_L3_CACHE);
	4791	+}
	4792	+
	4793	+static struct device_attribute dev_attr_compute_l3_cache_id = {
	4794	+ .attr = { .name = "id", .mode = 0444 },
	4795	+ .show = show_compute_l3_cache_id,
	4796	+ .store = NULL,
	4797	+};
	4798	+
	4799	+static struct attribute *attrs_compute_l3_cache[] = {
	4800	+ &dev_attr_compute_l3_cache_id.attr,
	4801	+ NULL,
	4802	+};
	4803	+
	4804	+static struct attribute_group group_compute_l3_cache = {
	4805	+ .name = "27a364dc-8225-4ecb-b607-d6f1925598d9",
	4806	+ .attrs = attrs_compute_l3_cache,
	4807	+};
	4808	+
	4809	+static ssize_t
	4810	+show_data_port_reads_coalescing_id(struct device kdev, struct device_attribute attr, char *buf)
	4811	+{
	4812	+ return sprintf(buf, "%d\n", METRIC_SET_ID_DATA_PORT_READS_COALESCING);
	4813	+}
	4814	+
	4815	+static struct device_attribute dev_attr_data_port_reads_coalescing_id = {
	4816	+ .attr = { .name = "id", .mode = 0444 },
	4817	+ .show = show_data_port_reads_coalescing_id,
	4818	+ .store = NULL,
	4819	+};
	4820	+
	4821	+static struct attribute *attrs_data_port_reads_coalescing[] = {
	4822	+ &dev_attr_data_port_reads_coalescing_id.attr,
	4823	+ NULL,
	4824	+};
	4825	+
	4826	+static struct attribute_group group_data_port_reads_coalescing = {
	4827	+ .name = "857fc630-2f09-4804-85f1-084adfadd5ab",
	4828	+ .attrs = attrs_data_port_reads_coalescing,
	4829	+};
	4830	+
	4831	+static ssize_t
	4832	+show_data_port_writes_coalescing_id(struct device kdev, struct device_attribute attr, char *buf)
	4833	+{
	4834	+ return sprintf(buf, "%d\n", METRIC_SET_ID_DATA_PORT_WRITES_COALESCING);
	4835	+}
	4836	+
	4837	+static struct device_attribute dev_attr_data_port_writes_coalescing_id = {
	4838	+ .attr = { .name = "id", .mode = 0444 },
	4839	+ .show = show_data_port_writes_coalescing_id,
	4840	+ .store = NULL,
	4841	+};
	4842	+
	4843	+static struct attribute *attrs_data_port_writes_coalescing[] = {
	4844	+ &dev_attr_data_port_writes_coalescing_id.attr,
	4845	+ NULL,
	4846	+};
	4847	+
	4848	+static struct attribute_group group_data_port_writes_coalescing = {
	4849	+ .name = "343ebc99-4a55-414c-8c17-d8e259cf5e20",
	4850	+ .attrs = attrs_data_port_writes_coalescing,
	4851	+};
	4852	+
	4853	+static ssize_t
	4854	+show_hdc_and_sf_id(struct device kdev, struct device_attribute attr, char *buf)
	4855	+{
	4856	+ return sprintf(buf, "%d\n", METRIC_SET_ID_HDC_AND_SF);
	4857	+}
	4858	+
	4859	+static struct device_attribute dev_attr_hdc_and_sf_id = {
	4860	+ .attr = { .name = "id", .mode = 0444 },
	4861	+ .show = show_hdc_and_sf_id,
	4862	+ .store = NULL,
	4863	+};
	4864	+
	4865	+static struct attribute *attrs_hdc_and_sf[] = {
	4866	+ &dev_attr_hdc_and_sf_id.attr,
	4867	+ NULL,
	4868	+};
	4869	+
	4870	+static struct attribute_group group_hdc_and_sf = {
	4871	+ .name = "7bdafd88-a4fa-4ed5-bc09-1a977aa5be3e",
	4872	+ .attrs = attrs_hdc_and_sf,
	4873	+};
	4874	+
	4875	+static ssize_t
	4876	+show_l3_1_id(struct device kdev, struct device_attribute attr, char *buf)
	4877	+{
	4878	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_1);
	4879	+}
	4880	+
	4881	+static struct device_attribute dev_attr_l3_1_id = {
	4882	+ .attr = { .name = "id", .mode = 0444 },
	4883	+ .show = show_l3_1_id,
	4884	+ .store = NULL,
	4885	+};
	4886	+
	4887	+static struct attribute *attrs_l3_1[] = {
	4888	+ &dev_attr_l3_1_id.attr,
	4889	+ NULL,
	4890	+};
	4891	+
	4892	+static struct attribute_group group_l3_1 = {
	4893	+ .name = "9385ebb2-f34f-4aa5-aec5-7e9cbbea0f0b",
	4894	+ .attrs = attrs_l3_1,
	4895	+};
	4896	+
	4897	+static ssize_t
	4898	+show_l3_2_id(struct device kdev, struct device_attribute attr, char *buf)
	4899	+{
	4900	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_2);
	4901	+}
	4902	+
	4903	+static struct device_attribute dev_attr_l3_2_id = {
	4904	+ .attr = { .name = "id", .mode = 0444 },
	4905	+ .show = show_l3_2_id,
	4906	+ .store = NULL,
	4907	+};
	4908	+
	4909	+static struct attribute *attrs_l3_2[] = {
	4910	+ &dev_attr_l3_2_id.attr,
	4911	+ NULL,
	4912	+};
	4913	+
	4914	+static struct attribute_group group_l3_2 = {
	4915	+ .name = "446ae59b-ff2e-41c9-b49e-0184a54bf00a",
	4916	+ .attrs = attrs_l3_2,
	4917	+};
	4918	+
	4919	+static ssize_t
	4920	+show_l3_3_id(struct device kdev, struct device_attribute attr, char *buf)
	4921	+{
	4922	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_3);
	4923	+}
	4924	+
	4925	+static struct device_attribute dev_attr_l3_3_id = {
	4926	+ .attr = { .name = "id", .mode = 0444 },
	4927	+ .show = show_l3_3_id,
	4928	+ .store = NULL,
	4929	+};
	4930	+
	4931	+static struct attribute *attrs_l3_3[] = {
	4932	+ &dev_attr_l3_3_id.attr,
	4933	+ NULL,
	4934	+};
	4935	+
	4936	+static struct attribute_group group_l3_3 = {
	4937	+ .name = "84a7956f-1ea4-4d0d-837f-e39a0376e38c",
	4938	+ .attrs = attrs_l3_3,
	4939	+};
	4940	+
	4941	+static ssize_t
	4942	+show_l3_4_id(struct device kdev, struct device_attribute attr, char *buf)
	4943	+{
	4944	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_4);
	4945	+}
	4946	+
	4947	+static struct device_attribute dev_attr_l3_4_id = {
	4948	+ .attr = { .name = "id", .mode = 0444 },
	4949	+ .show = show_l3_4_id,
	4950	+ .store = NULL,
	4951	+};
	4952	+
	4953	+static struct attribute *attrs_l3_4[] = {
	4954	+ &dev_attr_l3_4_id.attr,
	4955	+ NULL,
	4956	+};
	4957	+
	4958	+static struct attribute_group group_l3_4 = {
	4959	+ .name = "92b493d9-df18-4bed-be06-5cac6f2a6f5f",
	4960	+ .attrs = attrs_l3_4,
	4961	+};
	4962	+
	4963	+static ssize_t
	4964	+show_rasterizer_and_pixel_backend_id(struct device kdev, struct device_attribute attr, char *buf)
	4965	+{
	4966	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND);
	4967	+}
	4968	+
	4969	+static struct device_attribute dev_attr_rasterizer_and_pixel_backend_id = {
	4970	+ .attr = { .name = "id", .mode = 0444 },
	4971	+ .show = show_rasterizer_and_pixel_backend_id,
	4972	+ .store = NULL,
	4973	+};
	4974	+
	4975	+static struct attribute *attrs_rasterizer_and_pixel_backend[] = {
	4976	+ &dev_attr_rasterizer_and_pixel_backend_id.attr,
	4977	+ NULL,
	4978	+};
	4979	+
	4980	+static struct attribute_group group_rasterizer_and_pixel_backend = {
	4981	+ .name = "14345c35-cc46-40d0-bb04-6ed1fbb43679",
	4982	+ .attrs = attrs_rasterizer_and_pixel_backend,
	4983	+};
	4984	+
	4985	+static ssize_t
	4986	+show_sampler_1_id(struct device kdev, struct device_attribute attr, char *buf)
	4987	+{
	4988	+ return sprintf(buf, "%d\n", METRIC_SET_ID_SAMPLER_1);
	4989	+}
	4990	+
	4991	+static struct device_attribute dev_attr_sampler_1_id = {
	4992	+ .attr = { .name = "id", .mode = 0444 },
	4993	+ .show = show_sampler_1_id,
	4994	+ .store = NULL,
	4995	+};
	4996	+
	4997	+static struct attribute *attrs_sampler_1[] = {
	4998	+ &dev_attr_sampler_1_id.attr,
	4999	+ NULL,

377 more lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_bdw.h+40−0

@@ -0,0 +1,40 @@
	1	+/*
	2	+ * Autogenerated file by GPU Top : https://github.com/rib/gputop
	3	+ * DO NOT EDIT manually!
	4	+ *
	5	+ *
	6	+ * Copyright (c) 2015 Intel Corporation
	7	+ *
	8	+ * Permission is hereby granted, free of charge, to any person obtaining a
	9	+ * copy of this software and associated documentation files (the "Software"),
	10	+ * to deal in the Software without restriction, including without limitation
	11	+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
	12	+ * and/or sell copies of the Software, and to permit persons to whom the
	13	+ * Software is furnished to do so, subject to the following conditions:
	14	+ *
	15	+ * The above copyright notice and this permission notice (including the next
	16	+ * paragraph) shall be included in all copies or substantial portions of the
	17	+ * Software.
	18	+ *
	19	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
	20	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
	21	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
	22	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
	23	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
	24	+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
	25	+ * IN THE SOFTWARE.
	26	+ *
	27	+ */
	28	+
	29	+#ifndef __I915_OA_BDW_H__
	30	+#define __I915_OA_BDW_H__
	31	+
	32	+extern int i915_oa_n_builtin_metric_sets_bdw;
	33	+
	34	+extern int i915_oa_select_metric_set_bdw(struct drm_i915_private *dev_priv);
	35	+
	36	+extern int i915_perf_register_sysfs_bdw(struct drm_i915_private *dev_priv);
	37	+
	38	+extern void i915_perf_unregister_sysfs_bdw(struct drm_i915_private *dev_priv);
	39	+
	40	+#endif

drivers/gpu/drm/i915/i915_oa_bxt.c+2690−0

@@ -0,0 +1,2690 @@
	1	+/*
	2	+ * Autogenerated file by GPU Top : https://github.com/rib/gputop
	3	+ * DO NOT EDIT manually!
	4	+ *
	5	+ *
	6	+ * Copyright (c) 2015 Intel Corporation
	7	+ *
	8	+ * Permission is hereby granted, free of charge, to any person obtaining a
	9	+ * copy of this software and associated documentation files (the "Software"),
	10	+ * to deal in the Software without restriction, including without limitation
	11	+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
	12	+ * and/or sell copies of the Software, and to permit persons to whom the
	13	+ * Software is furnished to do so, subject to the following conditions:
	14	+ *
	15	+ * The above copyright notice and this permission notice (including the next
	16	+ * paragraph) shall be included in all copies or substantial portions of the
	17	+ * Software.
	18	+ *
	19	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
	20	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
	21	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
	22	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
	23	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
	24	+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
	25	+ * IN THE SOFTWARE.
	26	+ *
	27	+ */
	28	+
	29	+#include <linux/sysfs.h>
	30	+
	31	+#include "i915_drv.h"
	32	+#include "i915_oa_bxt.h"
	33	+
	34	+enum metric_set_id {
	35	+ METRIC_SET_ID_RENDER_BASIC = 1,
	36	+ METRIC_SET_ID_COMPUTE_BASIC,
	37	+ METRIC_SET_ID_RENDER_PIPE_PROFILE,
	38	+ METRIC_SET_ID_MEMORY_READS,
	39	+ METRIC_SET_ID_MEMORY_WRITES,
	40	+ METRIC_SET_ID_COMPUTE_EXTENDED,
	41	+ METRIC_SET_ID_COMPUTE_L3_CACHE,
	42	+ METRIC_SET_ID_HDC_AND_SF,
	43	+ METRIC_SET_ID_L3_1,
	44	+ METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND,
	45	+ METRIC_SET_ID_SAMPLER,
	46	+ METRIC_SET_ID_TDL_1,
	47	+ METRIC_SET_ID_TDL_2,
	48	+ METRIC_SET_ID_COMPUTE_EXTRA,
	49	+ METRIC_SET_ID_TEST_OA,
	50	+};
	51	+
	52	+int i915_oa_n_builtin_metric_sets_bxt = 15;
	53	+
	54	+static const struct i915_oa_reg b_counter_config_render_basic[] = {
	55	+ { _MMIO(0x2710), 0x00000000 },
	56	+ { _MMIO(0x2714), 0x00800000 },
	57	+ { _MMIO(0x2720), 0x00000000 },
	58	+ { _MMIO(0x2724), 0x00800000 },
	59	+ { _MMIO(0x2740), 0x00000000 },
	60	+};
	61	+
	62	+static const struct i915_oa_reg flex_eu_config_render_basic[] = {
	63	+ { _MMIO(0xe458), 0x00005004 },
	64	+ { _MMIO(0xe558), 0x00010003 },
	65	+ { _MMIO(0xe658), 0x00012011 },
	66	+ { _MMIO(0xe758), 0x00015014 },
	67	+ { _MMIO(0xe45c), 0x00051050 },
	68	+ { _MMIO(0xe55c), 0x00053052 },
	69	+ { _MMIO(0xe65c), 0x00055054 },
	70	+};
	71	+
	72	+static const struct i915_oa_reg mux_config_render_basic_0_sku_gte_0x03[] = {
	73	+ { _MMIO(0x9888), 0x166c00f0 },
	74	+ { _MMIO(0x9888), 0x12120280 },
	75	+ { _MMIO(0x9888), 0x12320280 },
	76	+ { _MMIO(0x9888), 0x11930317 },
	77	+ { _MMIO(0x9888), 0x159303df },
	78	+ { _MMIO(0x9888), 0x3f900c00 },
	79	+ { _MMIO(0x9888), 0x419000a0 },
	80	+ { _MMIO(0x9888), 0x002d1000 },
	81	+ { _MMIO(0x9888), 0x062d4000 },
	82	+ { _MMIO(0x9888), 0x082d5000 },
	83	+ { _MMIO(0x9888), 0x0a2d1000 },
	84	+ { _MMIO(0x9888), 0x0c2e0800 },
	85	+ { _MMIO(0x9888), 0x0e2e5900 },
	86	+ { _MMIO(0x9888), 0x0a4c8000 },
	87	+ { _MMIO(0x9888), 0x0c4c8000 },
	88	+ { _MMIO(0x9888), 0x0e4c4000 },
	89	+ { _MMIO(0x9888), 0x064e8000 },
	90	+ { _MMIO(0x9888), 0x084e8000 },
	91	+ { _MMIO(0x9888), 0x0a4e2000 },
	92	+ { _MMIO(0x9888), 0x1c4f0010 },
	93	+ { _MMIO(0x9888), 0x0a6c0053 },
	94	+ { _MMIO(0x9888), 0x106c0000 },
	95	+ { _MMIO(0x9888), 0x1c6c0000 },
	96	+ { _MMIO(0x9888), 0x1a0fcc00 },
	97	+ { _MMIO(0x9888), 0x1c0f0002 },
	98	+ { _MMIO(0x9888), 0x1c2c0040 },
	99	+ { _MMIO(0x9888), 0x00101000 },
	100	+ { _MMIO(0x9888), 0x04101000 },
	101	+ { _MMIO(0x9888), 0x00114000 },
	102	+ { _MMIO(0x9888), 0x08114000 },
	103	+ { _MMIO(0x9888), 0x00120020 },
	104	+ { _MMIO(0x9888), 0x08120021 },
	105	+ { _MMIO(0x9888), 0x00141000 },
	106	+ { _MMIO(0x9888), 0x08141000 },
	107	+ { _MMIO(0x9888), 0x02308000 },
	108	+ { _MMIO(0x9888), 0x04302000 },
	109	+ { _MMIO(0x9888), 0x06318000 },
	110	+ { _MMIO(0x9888), 0x08318000 },
	111	+ { _MMIO(0x9888), 0x06320800 },
	112	+ { _MMIO(0x9888), 0x08320840 },
	113	+ { _MMIO(0x9888), 0x00320000 },
	114	+ { _MMIO(0x9888), 0x06344000 },
	115	+ { _MMIO(0x9888), 0x08344000 },
	116	+ { _MMIO(0x9888), 0x0d931831 },
	117	+ { _MMIO(0x9888), 0x0f939f3f },
	118	+ { _MMIO(0x9888), 0x01939e80 },
	119	+ { _MMIO(0x9888), 0x039303bc },
	120	+ { _MMIO(0x9888), 0x0593000e },
	121	+ { _MMIO(0x9888), 0x1993002a },
	122	+ { _MMIO(0x9888), 0x07930000 },
	123	+ { _MMIO(0x9888), 0x09930000 },
	124	+ { _MMIO(0x9888), 0x1d900177 },
	125	+ { _MMIO(0x9888), 0x1f900187 },
	126	+ { _MMIO(0x9888), 0x35900000 },
	127	+ { _MMIO(0x9888), 0x13904000 },
	128	+ { _MMIO(0x9888), 0x21904000 },
	129	+ { _MMIO(0x9888), 0x23904000 },
	130	+ { _MMIO(0x9888), 0x25904000 },
	131	+ { _MMIO(0x9888), 0x27904000 },
	132	+ { _MMIO(0x9888), 0x2b904000 },
	133	+ { _MMIO(0x9888), 0x2d904000 },
	134	+ { _MMIO(0x9888), 0x2f904000 },
	135	+ { _MMIO(0x9888), 0x31904000 },
	136	+ { _MMIO(0x9888), 0x15904000 },
	137	+ { _MMIO(0x9888), 0x17904000 },
	138	+ { _MMIO(0x9888), 0x19904000 },
	139	+ { _MMIO(0x9888), 0x1b904000 },
	140	+ { _MMIO(0x9888), 0x53901110 },
	141	+ { _MMIO(0x9888), 0x43900423 },
	142	+ { _MMIO(0x9888), 0x55900111 },
	143	+ { _MMIO(0x9888), 0x47900c02 },
	144	+ { _MMIO(0x9888), 0x57900000 },
	145	+ { _MMIO(0x9888), 0x49900020 },
	146	+ { _MMIO(0x9888), 0x59901111 },
	147	+ { _MMIO(0x9888), 0x4b900421 },
	148	+ { _MMIO(0x9888), 0x37900000 },
	149	+ { _MMIO(0x9888), 0x33900000 },
	150	+ { _MMIO(0x9888), 0x4d900001 },
	151	+ { _MMIO(0x9888), 0x45900821 },
	152	+};
	153	+
	154	+static int
	155	+get_render_basic_mux_config(struct drm_i915_private *dev_priv,
	156	+ const struct i915_oa_reg **regs,
	157	+ int *lens)
	158	+{
	159	+ int n = 0;
	160	+
	161	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	162	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	163	+
	164	+ if (dev_priv->drm.pdev->revision >= 0x03) {
	165	+ regs[n] = mux_config_render_basic_0_sku_gte_0x03;
	166	+ lens[n] = ARRAY_SIZE(mux_config_render_basic_0_sku_gte_0x03);
	167	+ n++;
	168	+ }
	169	+
	170	+ return n;
	171	+}
	172	+
	173	+static const struct i915_oa_reg b_counter_config_compute_basic[] = {
	174	+ { _MMIO(0x2710), 0x00000000 },
	175	+ { _MMIO(0x2714), 0x00800000 },
	176	+ { _MMIO(0x2720), 0x00000000 },
	177	+ { _MMIO(0x2724), 0x00800000 },
	178	+ { _MMIO(0x2740), 0x00000000 },
	179	+};
	180	+
	181	+static const struct i915_oa_reg flex_eu_config_compute_basic[] = {
	182	+ { _MMIO(0xe458), 0x00005004 },
	183	+ { _MMIO(0xe558), 0x00000003 },
	184	+ { _MMIO(0xe658), 0x00002001 },
	185	+ { _MMIO(0xe758), 0x00778008 },
	186	+ { _MMIO(0xe45c), 0x00088078 },
	187	+ { _MMIO(0xe55c), 0x00808708 },
	188	+ { _MMIO(0xe65c), 0x00a08908 },
	189	+};
	190	+
	191	+static const struct i915_oa_reg mux_config_compute_basic[] = {
	192	+ { _MMIO(0x9888), 0x104f00e0 },
	193	+ { _MMIO(0x9888), 0x124f1c00 },
	194	+ { _MMIO(0x9888), 0x39900340 },
	195	+ { _MMIO(0x9888), 0x3f900c00 },
	196	+ { _MMIO(0x9888), 0x41900000 },
	197	+ { _MMIO(0x9888), 0x002d5000 },
	198	+ { _MMIO(0x9888), 0x062d4000 },
	199	+ { _MMIO(0x9888), 0x082d4000 },
	200	+ { _MMIO(0x9888), 0x0a2d1000 },
	201	+ { _MMIO(0x9888), 0x0c2d5000 },
	202	+ { _MMIO(0x9888), 0x0e2d4000 },
	203	+ { _MMIO(0x9888), 0x0c2e1400 },
	204	+ { _MMIO(0x9888), 0x0e2e5100 },
	205	+ { _MMIO(0x9888), 0x102e0114 },
	206	+ { _MMIO(0x9888), 0x044cc000 },
	207	+ { _MMIO(0x9888), 0x0a4c8000 },
	208	+ { _MMIO(0x9888), 0x0c4c8000 },
	209	+ { _MMIO(0x9888), 0x0e4c4000 },
	210	+ { _MMIO(0x9888), 0x104c8000 },
	211	+ { _MMIO(0x9888), 0x124c8000 },
	212	+ { _MMIO(0x9888), 0x164c2000 },
	213	+ { _MMIO(0x9888), 0x004ea000 },
	214	+ { _MMIO(0x9888), 0x064e8000 },
	215	+ { _MMIO(0x9888), 0x084e8000 },
	216	+ { _MMIO(0x9888), 0x0a4e2000 },
	217	+ { _MMIO(0x9888), 0x0c4ea000 },
	218	+ { _MMIO(0x9888), 0x0e4e8000 },
	219	+ { _MMIO(0x9888), 0x004f6b42 },
	220	+ { _MMIO(0x9888), 0x064f6200 },
	221	+ { _MMIO(0x9888), 0x084f4100 },
	222	+ { _MMIO(0x9888), 0x0a4f0061 },
	223	+ { _MMIO(0x9888), 0x0c4f6c4c },
	224	+ { _MMIO(0x9888), 0x0e4f4b00 },
	225	+ { _MMIO(0x9888), 0x1a4f0000 },
	226	+ { _MMIO(0x9888), 0x1c4f0000 },
	227	+ { _MMIO(0x9888), 0x180f5000 },
	228	+ { _MMIO(0x9888), 0x1a0f8800 },
	229	+ { _MMIO(0x9888), 0x1c0f08a2 },
	230	+ { _MMIO(0x9888), 0x182c4000 },
	231	+ { _MMIO(0x9888), 0x1c2c1451 },
	232	+ { _MMIO(0x9888), 0x1e2c0001 },
	233	+ { _MMIO(0x9888), 0x1a2c0010 },
	234	+ { _MMIO(0x9888), 0x01938000 },
	235	+ { _MMIO(0x9888), 0x0f938000 },
	236	+ { _MMIO(0x9888), 0x19938a28 },
	237	+ { _MMIO(0x9888), 0x03938000 },
	238	+ { _MMIO(0x9888), 0x19900177 },
	239	+ { _MMIO(0x9888), 0x1b900178 },
	240	+ { _MMIO(0x9888), 0x1d900125 },
	241	+ { _MMIO(0x9888), 0x1f900123 },
	242	+ { _MMIO(0x9888), 0x35900000 },
	243	+ { _MMIO(0x9888), 0x13904000 },
	244	+ { _MMIO(0x9888), 0x21904000 },
	245	+ { _MMIO(0x9888), 0x25904000 },
	246	+ { _MMIO(0x9888), 0x27904000 },
	247	+ { _MMIO(0x9888), 0x2b904000 },
	248	+ { _MMIO(0x9888), 0x2d904000 },
	249	+ { _MMIO(0x9888), 0x31904000 },
	250	+ { _MMIO(0x9888), 0x15904000 },
	251	+ { _MMIO(0x9888), 0x53901000 },
	252	+ { _MMIO(0x9888), 0x43900000 },
	253	+ { _MMIO(0x9888), 0x55900111 },
	254	+ { _MMIO(0x9888), 0x47900000 },
	255	+ { _MMIO(0x9888), 0x57900000 },
	256	+ { _MMIO(0x9888), 0x49900000 },
	257	+ { _MMIO(0x9888), 0x59900000 },
	258	+ { _MMIO(0x9888), 0x4b900000 },
	259	+ { _MMIO(0x9888), 0x37900000 },
	260	+ { _MMIO(0x9888), 0x33900000 },
	261	+ { _MMIO(0x9888), 0x4d900000 },
	262	+ { _MMIO(0x9888), 0x45900000 },
	263	+};
	264	+
	265	+static int
	266	+get_compute_basic_mux_config(struct drm_i915_private *dev_priv,
	267	+ const struct i915_oa_reg **regs,
	268	+ int *lens)
	269	+{
	270	+ int n = 0;
	271	+
	272	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	273	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	274	+
	275	+ regs[n] = mux_config_compute_basic;
	276	+ lens[n] = ARRAY_SIZE(mux_config_compute_basic);
	277	+ n++;
	278	+
	279	+ return n;
	280	+}
	281	+
	282	+static const struct i915_oa_reg b_counter_config_render_pipe_profile[] = {
	283	+ { _MMIO(0x2724), 0xf0800000 },
	284	+ { _MMIO(0x2720), 0x00000000 },
	285	+ { _MMIO(0x2714), 0xf0800000 },
	286	+ { _MMIO(0x2710), 0x00000000 },
	287	+ { _MMIO(0x2740), 0x00000000 },
	288	+ { _MMIO(0x2770), 0x0007ffea },
	289	+ { _MMIO(0x2774), 0x00007ffc },
	290	+ { _MMIO(0x2778), 0x0007affa },
	291	+ { _MMIO(0x277c), 0x0000f5fd },
	292	+ { _MMIO(0x2780), 0x00079ffa },
	293	+ { _MMIO(0x2784), 0x0000f3fb },
	294	+ { _MMIO(0x2788), 0x0007bf7a },
	295	+ { _MMIO(0x278c), 0x0000f7e7 },
	296	+ { _MMIO(0x2790), 0x0007fefa },
	297	+ { _MMIO(0x2794), 0x0000f7cf },
	298	+ { _MMIO(0x2798), 0x00077ffa },
	299	+ { _MMIO(0x279c), 0x0000efdf },
	300	+ { _MMIO(0x27a0), 0x0006fffa },
	301	+ { _MMIO(0x27a4), 0x0000cfbf },
	302	+ { _MMIO(0x27a8), 0x0003fffa },
	303	+ { _MMIO(0x27ac), 0x00005f7f },
	304	+};
	305	+
	306	+static const struct i915_oa_reg flex_eu_config_render_pipe_profile[] = {
	307	+ { _MMIO(0xe458), 0x00005004 },
	308	+ { _MMIO(0xe558), 0x00015014 },
	309	+ { _MMIO(0xe658), 0x00025024 },
	310	+ { _MMIO(0xe758), 0x00035034 },
	311	+ { _MMIO(0xe45c), 0x00045044 },
	312	+ { _MMIO(0xe55c), 0x00055054 },
	313	+ { _MMIO(0xe65c), 0x00065064 },
	314	+};
	315	+
	316	+static const struct i915_oa_reg mux_config_render_pipe_profile[] = {
	317	+ { _MMIO(0x9888), 0x0c2e001f },
	318	+ { _MMIO(0x9888), 0x0a2f0000 },
	319	+ { _MMIO(0x9888), 0x10186800 },
	320	+ { _MMIO(0x9888), 0x11810019 },
	321	+ { _MMIO(0x9888), 0x15810013 },
	322	+ { _MMIO(0x9888), 0x13820020 },
	323	+ { _MMIO(0x9888), 0x11830020 },
	324	+ { _MMIO(0x9888), 0x17840000 },
	325	+ { _MMIO(0x9888), 0x11860007 },
	326	+ { _MMIO(0x9888), 0x21860000 },
	327	+ { _MMIO(0x9888), 0x178703e0 },
	328	+ { _MMIO(0x9888), 0x0c2d8000 },
	329	+ { _MMIO(0x9888), 0x042d4000 },
	330	+ { _MMIO(0x9888), 0x062d1000 },
	331	+ { _MMIO(0x9888), 0x022e5400 },
	332	+ { _MMIO(0x9888), 0x002e0000 },
	333	+ { _MMIO(0x9888), 0x0e2e0080 },
	334	+ { _MMIO(0x9888), 0x082f0040 },
	335	+ { _MMIO(0x9888), 0x002f0000 },
	336	+ { _MMIO(0x9888), 0x06143000 },
	337	+ { _MMIO(0x9888), 0x06174000 },
	338	+ { _MMIO(0x9888), 0x06180012 },
	339	+ { _MMIO(0x9888), 0x00180000 },
	340	+ { _MMIO(0x9888), 0x0d804000 },
	341	+ { _MMIO(0x9888), 0x0f804000 },
	342	+ { _MMIO(0x9888), 0x05804000 },
	343	+ { _MMIO(0x9888), 0x09810200 },
	344	+ { _MMIO(0x9888), 0x0b810030 },
	345	+ { _MMIO(0x9888), 0x03810003 },
	346	+ { _MMIO(0x9888), 0x21819140 },
	347	+ { _MMIO(0x9888), 0x23819050 },
	348	+ { _MMIO(0x9888), 0x25810018 },
	349	+ { _MMIO(0x9888), 0x0b820980 },
	350	+ { _MMIO(0x9888), 0x03820d80 },
	351	+ { _MMIO(0x9888), 0x11820000 },
	352	+ { _MMIO(0x9888), 0x0182c000 },
	353	+ { _MMIO(0x9888), 0x07828000 },
	354	+ { _MMIO(0x9888), 0x09824000 },
	355	+ { _MMIO(0x9888), 0x0f828000 },
	356	+ { _MMIO(0x9888), 0x0d830004 },
	357	+ { _MMIO(0x9888), 0x0583000c },
	358	+ { _MMIO(0x9888), 0x0f831000 },
	359	+ { _MMIO(0x9888), 0x01848072 },
	360	+ { _MMIO(0x9888), 0x11840000 },
	361	+ { _MMIO(0x9888), 0x07848000 },
	362	+ { _MMIO(0x9888), 0x09844000 },
	363	+ { _MMIO(0x9888), 0x0f848000 },
	364	+ { _MMIO(0x9888), 0x07860000 },
	365	+ { _MMIO(0x9888), 0x09860092 },
	366	+ { _MMIO(0x9888), 0x0f860400 },
	367	+ { _MMIO(0x9888), 0x01869100 },
	368	+ { _MMIO(0x9888), 0x0f870065 },
	369	+ { _MMIO(0x9888), 0x01870000 },
	370	+ { _MMIO(0x9888), 0x19930800 },
	371	+ { _MMIO(0x9888), 0x0b938000 },
	372	+ { _MMIO(0x9888), 0x0d938000 },
	373	+ { _MMIO(0x9888), 0x1b952000 },
	374	+ { _MMIO(0x9888), 0x1d955055 },
	375	+ { _MMIO(0x9888), 0x1f951455 },
	376	+ { _MMIO(0x9888), 0x0992a000 },
	377	+ { _MMIO(0x9888), 0x0f928000 },
	378	+ { _MMIO(0x9888), 0x1192a800 },
	379	+ { _MMIO(0x9888), 0x1392028a },
	380	+ { _MMIO(0x9888), 0x0b92a000 },
	381	+ { _MMIO(0x9888), 0x0d922000 },
	382	+ { _MMIO(0x9888), 0x13908000 },
	383	+ { _MMIO(0x9888), 0x21908000 },
	384	+ { _MMIO(0x9888), 0x23908000 },
	385	+ { _MMIO(0x9888), 0x25908000 },
	386	+ { _MMIO(0x9888), 0x27908000 },
	387	+ { _MMIO(0x9888), 0x29908000 },
	388	+ { _MMIO(0x9888), 0x2b908000 },
	389	+ { _MMIO(0x9888), 0x2d904000 },
	390	+ { _MMIO(0x9888), 0x2f908000 },
	391	+ { _MMIO(0x9888), 0x31908000 },
	392	+ { _MMIO(0x9888), 0x15908000 },
	393	+ { _MMIO(0x9888), 0x17908000 },
	394	+ { _MMIO(0x9888), 0x19908000 },
	395	+ { _MMIO(0x9888), 0x1b908000 },
	396	+ { _MMIO(0x9888), 0x1d904000 },
	397	+ { _MMIO(0x9888), 0x1f904000 },
	398	+ { _MMIO(0x9888), 0x53900000 },
	399	+ { _MMIO(0x9888), 0x43900c01 },
	400	+ { _MMIO(0x9888), 0x55900000 },
	401	+ { _MMIO(0x9888), 0x47900000 },
	402	+ { _MMIO(0x9888), 0x57900000 },
	403	+ { _MMIO(0x9888), 0x49900863 },
	404	+ { _MMIO(0x9888), 0x59900000 },
	405	+ { _MMIO(0x9888), 0x4b900061 },
	406	+ { _MMIO(0x9888), 0x37900000 },
	407	+ { _MMIO(0x9888), 0x33900000 },
	408	+ { _MMIO(0x9888), 0x4d900000 },
	409	+ { _MMIO(0x9888), 0x45900c22 },
	410	+};
	411	+
	412	+static int
	413	+get_render_pipe_profile_mux_config(struct drm_i915_private *dev_priv,
	414	+ const struct i915_oa_reg **regs,
	415	+ int *lens)
	416	+{
	417	+ int n = 0;
	418	+
	419	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	420	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	421	+
	422	+ regs[n] = mux_config_render_pipe_profile;
	423	+ lens[n] = ARRAY_SIZE(mux_config_render_pipe_profile);
	424	+ n++;
	425	+
	426	+ return n;
	427	+}
	428	+
	429	+static const struct i915_oa_reg b_counter_config_memory_reads[] = {
	430	+ { _MMIO(0x272c), 0xffffffff },
	431	+ { _MMIO(0x2728), 0xffffffff },
	432	+ { _MMIO(0x2724), 0xf0800000 },
	433	+ { _MMIO(0x2720), 0x00000000 },
	434	+ { _MMIO(0x271c), 0xffffffff },
	435	+ { _MMIO(0x2718), 0xffffffff },
	436	+ { _MMIO(0x2714), 0xf0800000 },
	437	+ { _MMIO(0x2710), 0x00000000 },
	438	+ { _MMIO(0x274c), 0x86543210 },
	439	+ { _MMIO(0x2748), 0x86543210 },
	440	+ { _MMIO(0x2744), 0x00006667 },
	441	+ { _MMIO(0x2740), 0x00000000 },
	442	+ { _MMIO(0x275c), 0x86543210 },
	443	+ { _MMIO(0x2758), 0x86543210 },
	444	+ { _MMIO(0x2754), 0x00006465 },
	445	+ { _MMIO(0x2750), 0x00000000 },
	446	+ { _MMIO(0x2770), 0x0007f81a },
	447	+ { _MMIO(0x2774), 0x0000fe00 },
	448	+ { _MMIO(0x2778), 0x0007f82a },
	449	+ { _MMIO(0x277c), 0x0000fe00 },
	450	+ { _MMIO(0x2780), 0x0007f872 },
	451	+ { _MMIO(0x2784), 0x0000fe00 },
	452	+ { _MMIO(0x2788), 0x0007f8ba },
	453	+ { _MMIO(0x278c), 0x0000fe00 },
	454	+ { _MMIO(0x2790), 0x0007f87a },
	455	+ { _MMIO(0x2794), 0x0000fe00 },
	456	+ { _MMIO(0x2798), 0x0007f8ea },
	457	+ { _MMIO(0x279c), 0x0000fe00 },
	458	+ { _MMIO(0x27a0), 0x0007f8e2 },
	459	+ { _MMIO(0x27a4), 0x0000fe00 },
	460	+ { _MMIO(0x27a8), 0x0007f8f2 },
	461	+ { _MMIO(0x27ac), 0x0000fe00 },
	462	+};
	463	+
	464	+static const struct i915_oa_reg flex_eu_config_memory_reads[] = {
	465	+ { _MMIO(0xe458), 0x00005004 },
	466	+ { _MMIO(0xe558), 0x00015014 },
	467	+ { _MMIO(0xe658), 0x00025024 },
	468	+ { _MMIO(0xe758), 0x00035034 },
	469	+ { _MMIO(0xe45c), 0x00045044 },
	470	+ { _MMIO(0xe55c), 0x00055054 },
	471	+ { _MMIO(0xe65c), 0x00065064 },
	472	+};
	473	+
	474	+static const struct i915_oa_reg mux_config_memory_reads[] = {
	475	+ { _MMIO(0x9888), 0x19800343 },
	476	+ { _MMIO(0x9888), 0x39900340 },
	477	+ { _MMIO(0x9888), 0x3f901000 },
	478	+ { _MMIO(0x9888), 0x41900003 },
	479	+ { _MMIO(0x9888), 0x03803180 },
	480	+ { _MMIO(0x9888), 0x058035e2 },
	481	+ { _MMIO(0x9888), 0x0780006a },
	482	+ { _MMIO(0x9888), 0x11800000 },
	483	+ { _MMIO(0x9888), 0x2181a000 },
	484	+ { _MMIO(0x9888), 0x2381000a },
	485	+ { _MMIO(0x9888), 0x1d950550 },
	486	+ { _MMIO(0x9888), 0x0b928000 },
	487	+ { _MMIO(0x9888), 0x0d92a000 },
	488	+ { _MMIO(0x9888), 0x0f922000 },
	489	+ { _MMIO(0x9888), 0x13900170 },
	490	+ { _MMIO(0x9888), 0x21900171 },
	491	+ { _MMIO(0x9888), 0x23900172 },
	492	+ { _MMIO(0x9888), 0x25900173 },
	493	+ { _MMIO(0x9888), 0x27900174 },
	494	+ { _MMIO(0x9888), 0x29900175 },
	495	+ { _MMIO(0x9888), 0x2b900176 },
	496	+ { _MMIO(0x9888), 0x2d900177 },
	497	+ { _MMIO(0x9888), 0x2f90017f },
	498	+ { _MMIO(0x9888), 0x31900125 },
	499	+ { _MMIO(0x9888), 0x15900123 },
	500	+ { _MMIO(0x9888), 0x17900121 },
	501	+ { _MMIO(0x9888), 0x35900000 },
	502	+ { _MMIO(0x9888), 0x19908000 },
	503	+ { _MMIO(0x9888), 0x1b908000 },
	504	+ { _MMIO(0x9888), 0x1d908000 },
	505	+ { _MMIO(0x9888), 0x1f908000 },
	506	+ { _MMIO(0x9888), 0x53900000 },
	507	+ { _MMIO(0x9888), 0x43901084 },
	508	+ { _MMIO(0x9888), 0x55900000 },
	509	+ { _MMIO(0x9888), 0x47901080 },
	510	+ { _MMIO(0x9888), 0x57900000 },
	511	+ { _MMIO(0x9888), 0x49901084 },
	512	+ { _MMIO(0x9888), 0x59900000 },
	513	+ { _MMIO(0x9888), 0x4b901084 },
	514	+ { _MMIO(0x9888), 0x37900000 },
	515	+ { _MMIO(0x9888), 0x33900000 },
	516	+ { _MMIO(0x9888), 0x4d900004 },
	517	+ { _MMIO(0x9888), 0x45900000 },
	518	+};
	519	+
	520	+static int
	521	+get_memory_reads_mux_config(struct drm_i915_private *dev_priv,
	522	+ const struct i915_oa_reg **regs,
	523	+ int *lens)
	524	+{
	525	+ int n = 0;
	526	+
	527	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	528	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	529	+
	530	+ regs[n] = mux_config_memory_reads;
	531	+ lens[n] = ARRAY_SIZE(mux_config_memory_reads);
	532	+ n++;
	533	+
	534	+ return n;
	535	+}
	536	+
	537	+static const struct i915_oa_reg b_counter_config_memory_writes[] = {
	538	+ { _MMIO(0x272c), 0xffffffff },
	539	+ { _MMIO(0x2728), 0xffffffff },
	540	+ { _MMIO(0x2724), 0xf0800000 },
	541	+ { _MMIO(0x2720), 0x00000000 },
	542	+ { _MMIO(0x271c), 0xffffffff },
	543	+ { _MMIO(0x2718), 0xffffffff },
	544	+ { _MMIO(0x2714), 0xf0800000 },
	545	+ { _MMIO(0x2710), 0x00000000 },
	546	+ { _MMIO(0x274c), 0x86543210 },
	547	+ { _MMIO(0x2748), 0x86543210 },
	548	+ { _MMIO(0x2744), 0x00006667 },
	549	+ { _MMIO(0x2740), 0x00000000 },
	550	+ { _MMIO(0x275c), 0x86543210 },
	551	+ { _MMIO(0x2758), 0x86543210 },
	552	+ { _MMIO(0x2754), 0x00006465 },
	553	+ { _MMIO(0x2750), 0x00000000 },
	554	+ { _MMIO(0x2770), 0x0007f81a },
	555	+ { _MMIO(0x2774), 0x0000fe00 },
	556	+ { _MMIO(0x2778), 0x0007f82a },
	557	+ { _MMIO(0x277c), 0x0000fe00 },
	558	+ { _MMIO(0x2780), 0x0007f822 },
	559	+ { _MMIO(0x2784), 0x0000fe00 },
	560	+ { _MMIO(0x2788), 0x0007f8ba },
	561	+ { _MMIO(0x278c), 0x0000fe00 },
	562	+ { _MMIO(0x2790), 0x0007f87a },
	563	+ { _MMIO(0x2794), 0x0000fe00 },
	564	+ { _MMIO(0x2798), 0x0007f8ea },
	565	+ { _MMIO(0x279c), 0x0000fe00 },
	566	+ { _MMIO(0x27a0), 0x0007f8e2 },
	567	+ { _MMIO(0x27a4), 0x0000fe00 },
	568	+ { _MMIO(0x27a8), 0x0007f8f2 },
	569	+ { _MMIO(0x27ac), 0x0000fe00 },
	570	+};
	571	+
	572	+static const struct i915_oa_reg flex_eu_config_memory_writes[] = {
	573	+ { _MMIO(0xe458), 0x00005004 },
	574	+ { _MMIO(0xe558), 0x00015014 },
	575	+ { _MMIO(0xe658), 0x00025024 },
	576	+ { _MMIO(0xe758), 0x00035034 },
	577	+ { _MMIO(0xe45c), 0x00045044 },
	578	+ { _MMIO(0xe55c), 0x00055054 },
	579	+ { _MMIO(0xe65c), 0x00065064 },
	580	+};
	581	+
	582	+static const struct i915_oa_reg mux_config_memory_writes[] = {
	583	+ { _MMIO(0x9888), 0x19800343 },
	584	+ { _MMIO(0x9888), 0x39900340 },
	585	+ { _MMIO(0x9888), 0x3f900000 },
	586	+ { _MMIO(0x9888), 0x41900080 },
	587	+ { _MMIO(0x9888), 0x03803180 },
	588	+ { _MMIO(0x9888), 0x058035e2 },
	589	+ { _MMIO(0x9888), 0x0780006a },
	590	+ { _MMIO(0x9888), 0x11800000 },
	591	+ { _MMIO(0x9888), 0x2181a000 },
	592	+ { _MMIO(0x9888), 0x2381000a },
	593	+ { _MMIO(0x9888), 0x1d950550 },
	594	+ { _MMIO(0x9888), 0x0b928000 },
	595	+ { _MMIO(0x9888), 0x0d92a000 },
	596	+ { _MMIO(0x9888), 0x0f922000 },
	597	+ { _MMIO(0x9888), 0x13900180 },
	598	+ { _MMIO(0x9888), 0x21900181 },
	599	+ { _MMIO(0x9888), 0x23900182 },
	600	+ { _MMIO(0x9888), 0x25900183 },
	601	+ { _MMIO(0x9888), 0x27900184 },
	602	+ { _MMIO(0x9888), 0x29900185 },
	603	+ { _MMIO(0x9888), 0x2b900186 },
	604	+ { _MMIO(0x9888), 0x2d900187 },
	605	+ { _MMIO(0x9888), 0x2f900170 },
	606	+ { _MMIO(0x9888), 0x31900125 },
	607	+ { _MMIO(0x9888), 0x15900123 },
	608	+ { _MMIO(0x9888), 0x17900121 },
	609	+ { _MMIO(0x9888), 0x35900000 },
	610	+ { _MMIO(0x9888), 0x19908000 },
	611	+ { _MMIO(0x9888), 0x1b908000 },
	612	+ { _MMIO(0x9888), 0x1d908000 },
	613	+ { _MMIO(0x9888), 0x1f908000 },
	614	+ { _MMIO(0x9888), 0x53900000 },
	615	+ { _MMIO(0x9888), 0x43901084 },
	616	+ { _MMIO(0x9888), 0x55900000 },
	617	+ { _MMIO(0x9888), 0x47901080 },
	618	+ { _MMIO(0x9888), 0x57900000 },
	619	+ { _MMIO(0x9888), 0x49901084 },
	620	+ { _MMIO(0x9888), 0x59900000 },
	621	+ { _MMIO(0x9888), 0x4b901084 },
	622	+ { _MMIO(0x9888), 0x37900000 },
	623	+ { _MMIO(0x9888), 0x33900000 },
	624	+ { _MMIO(0x9888), 0x4d900004 },
	625	+ { _MMIO(0x9888), 0x45900000 },
	626	+};
	627	+
	628	+static int
	629	+get_memory_writes_mux_config(struct drm_i915_private *dev_priv,
	630	+ const struct i915_oa_reg **regs,
	631	+ int *lens)
	632	+{
	633	+ int n = 0;
	634	+
	635	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	636	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	637	+
	638	+ regs[n] = mux_config_memory_writes;
	639	+ lens[n] = ARRAY_SIZE(mux_config_memory_writes);
	640	+ n++;
	641	+
	642	+ return n;
	643	+}
	644	+
	645	+static const struct i915_oa_reg b_counter_config_compute_extended[] = {
	646	+ { _MMIO(0x2724), 0xf0800000 },
	647	+ { _MMIO(0x2720), 0x00000000 },
	648	+ { _MMIO(0x2714), 0xf0800000 },
	649	+ { _MMIO(0x2710), 0x00000000 },
	650	+ { _MMIO(0x2740), 0x00000000 },
	651	+ { _MMIO(0x2770), 0x0007fc2a },
	652	+ { _MMIO(0x2774), 0x0000bf00 },
	653	+ { _MMIO(0x2778), 0x0007fc6a },
	654	+ { _MMIO(0x277c), 0x0000bf00 },
	655	+ { _MMIO(0x2780), 0x0007fc92 },
	656	+ { _MMIO(0x2784), 0x0000bf00 },
	657	+ { _MMIO(0x2788), 0x0007fca2 },
	658	+ { _MMIO(0x278c), 0x0000bf00 },
	659	+ { _MMIO(0x2790), 0x0007fc32 },
	660	+ { _MMIO(0x2794), 0x0000bf00 },
	661	+ { _MMIO(0x2798), 0x0007fc9a },
	662	+ { _MMIO(0x279c), 0x0000bf00 },
	663	+ { _MMIO(0x27a0), 0x0007fe6a },
	664	+ { _MMIO(0x27a4), 0x0000bf00 },
	665	+ { _MMIO(0x27a8), 0x0007fe7a },
	666	+ { _MMIO(0x27ac), 0x0000bf00 },
	667	+};
	668	+
	669	+static const struct i915_oa_reg flex_eu_config_compute_extended[] = {
	670	+ { _MMIO(0xe458), 0x00005004 },
	671	+ { _MMIO(0xe558), 0x00000003 },
	672	+ { _MMIO(0xe658), 0x00002001 },
	673	+ { _MMIO(0xe758), 0x00778008 },
	674	+ { _MMIO(0xe45c), 0x00088078 },
	675	+ { _MMIO(0xe55c), 0x00808708 },
	676	+ { _MMIO(0xe65c), 0x00a08908 },
	677	+};
	678	+
	679	+static const struct i915_oa_reg mux_config_compute_extended[] = {
	680	+ { _MMIO(0x9888), 0x104f00e0 },
	681	+ { _MMIO(0x9888), 0x141c0160 },
	682	+ { _MMIO(0x9888), 0x161c0015 },
	683	+ { _MMIO(0x9888), 0x181c0120 },
	684	+ { _MMIO(0x9888), 0x002d5000 },
	685	+ { _MMIO(0x9888), 0x062d4000 },
	686	+ { _MMIO(0x9888), 0x082d5000 },
	687	+ { _MMIO(0x9888), 0x0a2d5000 },
	688	+ { _MMIO(0x9888), 0x0c2d5000 },
	689	+ { _MMIO(0x9888), 0x0e2d5000 },
	690	+ { _MMIO(0x9888), 0x022d5000 },
	691	+ { _MMIO(0x9888), 0x042d5000 },
	692	+ { _MMIO(0x9888), 0x0c2e5400 },
	693	+ { _MMIO(0x9888), 0x0e2e5515 },
	694	+ { _MMIO(0x9888), 0x102e0155 },
	695	+ { _MMIO(0x9888), 0x044cc000 },
	696	+ { _MMIO(0x9888), 0x0a4c8000 },
	697	+ { _MMIO(0x9888), 0x0c4cc000 },
	698	+ { _MMIO(0x9888), 0x0e4cc000 },
	699	+ { _MMIO(0x9888), 0x104c8000 },
	700	+ { _MMIO(0x9888), 0x124c8000 },
	701	+ { _MMIO(0x9888), 0x144c8000 },
	702	+ { _MMIO(0x9888), 0x164c2000 },
	703	+ { _MMIO(0x9888), 0x064cc000 },
	704	+ { _MMIO(0x9888), 0x084cc000 },
	705	+ { _MMIO(0x9888), 0x004ea000 },
	706	+ { _MMIO(0x9888), 0x064e8000 },
	707	+ { _MMIO(0x9888), 0x084ea000 },
	708	+ { _MMIO(0x9888), 0x0a4ea000 },
	709	+ { _MMIO(0x9888), 0x0c4ea000 },
	710	+ { _MMIO(0x9888), 0x0e4ea000 },
	711	+ { _MMIO(0x9888), 0x024ea000 },
	712	+ { _MMIO(0x9888), 0x044ea000 },
	713	+ { _MMIO(0x9888), 0x0e4f4b41 },
	714	+ { _MMIO(0x9888), 0x004f4200 },
	715	+ { _MMIO(0x9888), 0x024f404c },
	716	+ { _MMIO(0x9888), 0x1c4f0000 },
	717	+ { _MMIO(0x9888), 0x1a4f0000 },
	718	+ { _MMIO(0x9888), 0x001b4000 },
	719	+ { _MMIO(0x9888), 0x061b8000 },
	720	+ { _MMIO(0x9888), 0x081bc000 },
	721	+ { _MMIO(0x9888), 0x0a1bc000 },
	722	+ { _MMIO(0x9888), 0x0c1bc000 },
	723	+ { _MMIO(0x9888), 0x041bc000 },
	724	+ { _MMIO(0x9888), 0x001c0031 },
	725	+ { _MMIO(0x9888), 0x061c1900 },
	726	+ { _MMIO(0x9888), 0x081c1a33 },
	727	+ { _MMIO(0x9888), 0x0a1c1b35 },
	728	+ { _MMIO(0x9888), 0x0c1c3337 },
	729	+ { _MMIO(0x9888), 0x041c31c7 },
	730	+ { _MMIO(0x9888), 0x180f5000 },
	731	+ { _MMIO(0x9888), 0x1a0fa8aa },
	732	+ { _MMIO(0x9888), 0x1c0f0aaa },
	733	+ { _MMIO(0x9888), 0x182c8000 },
	734	+ { _MMIO(0x9888), 0x1c2c6aaa },
	735	+ { _MMIO(0x9888), 0x1e2c0001 },
	736	+ { _MMIO(0x9888), 0x1a2c2950 },
	737	+ { _MMIO(0x9888), 0x01938000 },
	738	+ { _MMIO(0x9888), 0x0f938000 },
	739	+ { _MMIO(0x9888), 0x1993aaaa },
	740	+ { _MMIO(0x9888), 0x03938000 },
	741	+ { _MMIO(0x9888), 0x05938000 },
	742	+ { _MMIO(0x9888), 0x07938000 },
	743	+ { _MMIO(0x9888), 0x09938000 },
	744	+ { _MMIO(0x9888), 0x0b938000 },
	745	+ { _MMIO(0x9888), 0x13904000 },
	746	+ { _MMIO(0x9888), 0x21904000 },
	747	+ { _MMIO(0x9888), 0x23904000 },
	748	+ { _MMIO(0x9888), 0x25904000 },
	749	+ { _MMIO(0x9888), 0x27904000 },
	750	+ { _MMIO(0x9888), 0x29904000 },
	751	+ { _MMIO(0x9888), 0x2b904000 },
	752	+ { _MMIO(0x9888), 0x2d904000 },
	753	+ { _MMIO(0x9888), 0x2f904000 },
	754	+ { _MMIO(0x9888), 0x31904000 },
	755	+ { _MMIO(0x9888), 0x15904000 },
	756	+ { _MMIO(0x9888), 0x17904000 },
	757	+ { _MMIO(0x9888), 0x19904000 },
	758	+ { _MMIO(0x9888), 0x1b904000 },
	759	+ { _MMIO(0x9888), 0x1d904000 },
	760	+ { _MMIO(0x9888), 0x53900000 },
	761	+ { _MMIO(0x9888), 0x43900420 },
	762	+ { _MMIO(0x9888), 0x55900000 },
	763	+ { _MMIO(0x9888), 0x47900000 },
	764	+ { _MMIO(0x9888), 0x57900000 },
	765	+ { _MMIO(0x9888), 0x49900000 },
	766	+ { _MMIO(0x9888), 0x59900000 },
	767	+ { _MMIO(0x9888), 0x4b900400 },
	768	+ { _MMIO(0x9888), 0x37900000 },
	769	+ { _MMIO(0x9888), 0x33900000 },
	770	+ { _MMIO(0x9888), 0x4d900001 },
	771	+ { _MMIO(0x9888), 0x45900001 },
	772	+};
	773	+
	774	+static int
	775	+get_compute_extended_mux_config(struct drm_i915_private *dev_priv,
	776	+ const struct i915_oa_reg **regs,
	777	+ int *lens)
	778	+{
	779	+ int n = 0;
	780	+
	781	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	782	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	783	+
	784	+ regs[n] = mux_config_compute_extended;
	785	+ lens[n] = ARRAY_SIZE(mux_config_compute_extended);
	786	+ n++;
	787	+
	788	+ return n;
	789	+}
	790	+
	791	+static const struct i915_oa_reg b_counter_config_compute_l3_cache[] = {
	792	+ { _MMIO(0x2710), 0x00000000 },
	793	+ { _MMIO(0x2714), 0x30800000 },
	794	+ { _MMIO(0x2720), 0x00000000 },
	795	+ { _MMIO(0x2724), 0x30800000 },
	796	+ { _MMIO(0x2740), 0x00000000 },
	797	+ { _MMIO(0x2770), 0x0007fffa },
	798	+ { _MMIO(0x2774), 0x0000fefe },
	799	+ { _MMIO(0x2778), 0x0007fffa },
	800	+ { _MMIO(0x277c), 0x0000fefd },
	801	+ { _MMIO(0x2790), 0x0007fffa },
	802	+ { _MMIO(0x2794), 0x0000fbef },
	803	+ { _MMIO(0x2798), 0x0007fffa },
	804	+ { _MMIO(0x279c), 0x0000fbdf },
	805	+};
	806	+
	807	+static const struct i915_oa_reg flex_eu_config_compute_l3_cache[] = {
	808	+ { _MMIO(0xe458), 0x00005004 },
	809	+ { _MMIO(0xe558), 0x00000003 },
	810	+ { _MMIO(0xe658), 0x00002001 },
	811	+ { _MMIO(0xe758), 0x00101100 },
	812	+ { _MMIO(0xe45c), 0x00201200 },
	813	+ { _MMIO(0xe55c), 0x00301300 },
	814	+ { _MMIO(0xe65c), 0x00401400 },
	815	+};
	816	+
	817	+static const struct i915_oa_reg mux_config_compute_l3_cache[] = {
	818	+ { _MMIO(0x9888), 0x166c03b0 },
	819	+ { _MMIO(0x9888), 0x1593001e },
	820	+ { _MMIO(0x9888), 0x3f900c00 },
	821	+ { _MMIO(0x9888), 0x41900000 },
	822	+ { _MMIO(0x9888), 0x002d1000 },
	823	+ { _MMIO(0x9888), 0x062d4000 },
	824	+ { _MMIO(0x9888), 0x082d5000 },
	825	+ { _MMIO(0x9888), 0x0e2d5000 },
	826	+ { _MMIO(0x9888), 0x0c2e0400 },
	827	+ { _MMIO(0x9888), 0x0e2e1500 },
	828	+ { _MMIO(0x9888), 0x102e0140 },
	829	+ { _MMIO(0x9888), 0x044c4000 },
	830	+ { _MMIO(0x9888), 0x0a4c8000 },
	831	+ { _MMIO(0x9888), 0x0c4cc000 },
	832	+ { _MMIO(0x9888), 0x144c8000 },
	833	+ { _MMIO(0x9888), 0x164c2000 },
	834	+ { _MMIO(0x9888), 0x004e2000 },
	835	+ { _MMIO(0x9888), 0x064e8000 },
	836	+ { _MMIO(0x9888), 0x084ea000 },
	837	+ { _MMIO(0x9888), 0x0e4ea000 },
	838	+ { _MMIO(0x9888), 0x1a4f4001 },
	839	+ { _MMIO(0x9888), 0x1c4f5005 },
	840	+ { _MMIO(0x9888), 0x006c0051 },
	841	+ { _MMIO(0x9888), 0x066c5000 },
	842	+ { _MMIO(0x9888), 0x086c5c5d },
	843	+ { _MMIO(0x9888), 0x0e6c5e5f },
	844	+ { _MMIO(0x9888), 0x106c0000 },
	845	+ { _MMIO(0x9888), 0x146c0000 },
	846	+ { _MMIO(0x9888), 0x1a6c0000 },
	847	+ { _MMIO(0x9888), 0x1c6c0000 },
	848	+ { _MMIO(0x9888), 0x180f1000 },
	849	+ { _MMIO(0x9888), 0x1a0fa800 },
	850	+ { _MMIO(0x9888), 0x1c0f0a00 },
	851	+ { _MMIO(0x9888), 0x182c4000 },
	852	+ { _MMIO(0x9888), 0x1c2c4015 },
	853	+ { _MMIO(0x9888), 0x1e2c0001 },
	854	+ { _MMIO(0x9888), 0x03931980 },
	855	+ { _MMIO(0x9888), 0x05930032 },
	856	+ { _MMIO(0x9888), 0x11930000 },
	857	+ { _MMIO(0x9888), 0x01938000 },
	858	+ { _MMIO(0x9888), 0x0f938000 },
	859	+ { _MMIO(0x9888), 0x1993a00a },
	860	+ { _MMIO(0x9888), 0x07930000 },
	861	+ { _MMIO(0x9888), 0x09930000 },
	862	+ { _MMIO(0x9888), 0x1d900177 },
	863	+ { _MMIO(0x9888), 0x1f900178 },
	864	+ { _MMIO(0x9888), 0x35900000 },
	865	+ { _MMIO(0x9888), 0x13904000 },
	866	+ { _MMIO(0x9888), 0x21904000 },
	867	+ { _MMIO(0x9888), 0x23904000 },
	868	+ { _MMIO(0x9888), 0x25904000 },
	869	+ { _MMIO(0x9888), 0x2f904000 },
	870	+ { _MMIO(0x9888), 0x31904000 },
	871	+ { _MMIO(0x9888), 0x19904000 },
	872	+ { _MMIO(0x9888), 0x1b904000 },
	873	+ { _MMIO(0x9888), 0x53901000 },
	874	+ { _MMIO(0x9888), 0x43900000 },
	875	+ { _MMIO(0x9888), 0x55900111 },
	876	+ { _MMIO(0x9888), 0x47900001 },
	877	+ { _MMIO(0x9888), 0x57900000 },
	878	+ { _MMIO(0x9888), 0x49900000 },
	879	+ { _MMIO(0x9888), 0x37900000 },
	880	+ { _MMIO(0x9888), 0x33900000 },
	881	+ { _MMIO(0x9888), 0x59900000 },
	882	+ { _MMIO(0x9888), 0x4b900000 },
	883	+ { _MMIO(0x9888), 0x4d900000 },
	884	+ { _MMIO(0x9888), 0x45900400 },
	885	+};
	886	+
	887	+static int
	888	+get_compute_l3_cache_mux_config(struct drm_i915_private *dev_priv,
	889	+ const struct i915_oa_reg **regs,
	890	+ int *lens)
	891	+{
	892	+ int n = 0;
	893	+
	894	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	895	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	896	+
	897	+ regs[n] = mux_config_compute_l3_cache;
	898	+ lens[n] = ARRAY_SIZE(mux_config_compute_l3_cache);
	899	+ n++;
	900	+
	901	+ return n;
	902	+}
	903	+
	904	+static const struct i915_oa_reg b_counter_config_hdc_and_sf[] = {
	905	+ { _MMIO(0x2740), 0x00000000 },
	906	+ { _MMIO(0x2744), 0x00800000 },
	907	+ { _MMIO(0x2710), 0x00000000 },
	908	+ { _MMIO(0x2714), 0x10800000 },
	909	+ { _MMIO(0x2720), 0x00000000 },
	910	+ { _MMIO(0x2724), 0x00800000 },
	911	+ { _MMIO(0x2770), 0x00000002 },
	912	+ { _MMIO(0x2774), 0x0000fdff },
	913	+};
	914	+
	915	+static const struct i915_oa_reg flex_eu_config_hdc_and_sf[] = {
	916	+ { _MMIO(0xe458), 0x00005004 },
	917	+ { _MMIO(0xe558), 0x00010003 },
	918	+ { _MMIO(0xe658), 0x00012011 },
	919	+ { _MMIO(0xe758), 0x00015014 },
	920	+ { _MMIO(0xe45c), 0x00051050 },
	921	+ { _MMIO(0xe55c), 0x00053052 },
	922	+ { _MMIO(0xe65c), 0x00055054 },
	923	+};
	924	+
	925	+static const struct i915_oa_reg mux_config_hdc_and_sf[] = {
	926	+ { _MMIO(0x9888), 0x104f0232 },
	927	+ { _MMIO(0x9888), 0x124f4640 },
	928	+ { _MMIO(0x9888), 0x11834400 },
	929	+ { _MMIO(0x9888), 0x022d4000 },
	930	+ { _MMIO(0x9888), 0x042d5000 },
	931	+ { _MMIO(0x9888), 0x062d1000 },
	932	+ { _MMIO(0x9888), 0x0e2e0055 },
	933	+ { _MMIO(0x9888), 0x064c8000 },
	934	+ { _MMIO(0x9888), 0x084cc000 },
	935	+ { _MMIO(0x9888), 0x0a4c4000 },
	936	+ { _MMIO(0x9888), 0x024e8000 },
	937	+ { _MMIO(0x9888), 0x044ea000 },
	938	+ { _MMIO(0x9888), 0x064e2000 },
	939	+ { _MMIO(0x9888), 0x024f6100 },
	940	+ { _MMIO(0x9888), 0x044f416b },
	941	+ { _MMIO(0x9888), 0x064f004b },
	942	+ { _MMIO(0x9888), 0x1a4f0000 },
	943	+ { _MMIO(0x9888), 0x1a0f02a8 },
	944	+ { _MMIO(0x9888), 0x1a2c5500 },
	945	+ { _MMIO(0x9888), 0x0f808000 },
	946	+ { _MMIO(0x9888), 0x25810020 },
	947	+ { _MMIO(0x9888), 0x0f8305c0 },
	948	+ { _MMIO(0x9888), 0x07938000 },
	949	+ { _MMIO(0x9888), 0x09938000 },
	950	+ { _MMIO(0x9888), 0x0b938000 },
	951	+ { _MMIO(0x9888), 0x0d938000 },
	952	+ { _MMIO(0x9888), 0x1f951000 },
	953	+ { _MMIO(0x9888), 0x13920200 },
	954	+ { _MMIO(0x9888), 0x31908000 },
	955	+ { _MMIO(0x9888), 0x19904000 },
	956	+ { _MMIO(0x9888), 0x1b904000 },
	957	+ { _MMIO(0x9888), 0x1d904000 },
	958	+ { _MMIO(0x9888), 0x1f904000 },
	959	+ { _MMIO(0x9888), 0x37900000 },
	960	+ { _MMIO(0x9888), 0x59900000 },
	961	+ { _MMIO(0x9888), 0x4d900003 },
	962	+ { _MMIO(0x9888), 0x53900000 },
	963	+ { _MMIO(0x9888), 0x45900000 },
	964	+ { _MMIO(0x9888), 0x55900000 },
	965	+ { _MMIO(0x9888), 0x47900000 },
	966	+ { _MMIO(0x9888), 0x33900000 },
	967	+};
	968	+
	969	+static int
	970	+get_hdc_and_sf_mux_config(struct drm_i915_private *dev_priv,
	971	+ const struct i915_oa_reg **regs,
	972	+ int *lens)
	973	+{
	974	+ int n = 0;
	975	+
	976	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	977	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	978	+
	979	+ regs[n] = mux_config_hdc_and_sf;
	980	+ lens[n] = ARRAY_SIZE(mux_config_hdc_and_sf);
	981	+ n++;
	982	+
	983	+ return n;
	984	+}
	985	+
	986	+static const struct i915_oa_reg b_counter_config_l3_1[] = {
	987	+ { _MMIO(0x2740), 0x00000000 },
	988	+ { _MMIO(0x2744), 0x00800000 },
	989	+ { _MMIO(0x2710), 0x00000000 },
	990	+ { _MMIO(0x2714), 0xf0800000 },
	991	+ { _MMIO(0x2720), 0x00000000 },
	992	+ { _MMIO(0x2724), 0xf0800000 },
	993	+ { _MMIO(0x2770), 0x00100070 },
	994	+ { _MMIO(0x2774), 0x0000fff1 },
	995	+ { _MMIO(0x2778), 0x00014002 },
	996	+ { _MMIO(0x277c), 0x0000c3ff },
	997	+ { _MMIO(0x2780), 0x00010002 },
	998	+ { _MMIO(0x2784), 0x0000c7ff },
	999	+ { _MMIO(0x2788), 0x00004002 },
	1000	+ { _MMIO(0x278c), 0x0000d3ff },
	1001	+ { _MMIO(0x2790), 0x00100700 },
	1002	+ { _MMIO(0x2794), 0x0000ff1f },
	1003	+ { _MMIO(0x2798), 0x00001402 },
	1004	+ { _MMIO(0x279c), 0x0000fc3f },
	1005	+ { _MMIO(0x27a0), 0x00001002 },
	1006	+ { _MMIO(0x27a4), 0x0000fc7f },
	1007	+ { _MMIO(0x27a8), 0x00000402 },
	1008	+ { _MMIO(0x27ac), 0x0000fd3f },
	1009	+};
	1010	+
	1011	+static const struct i915_oa_reg flex_eu_config_l3_1[] = {
	1012	+ { _MMIO(0xe458), 0x00005004 },
	1013	+ { _MMIO(0xe558), 0x00010003 },
	1014	+ { _MMIO(0xe658), 0x00012011 },
	1015	+ { _MMIO(0xe758), 0x00015014 },
	1016	+ { _MMIO(0xe45c), 0x00051050 },
	1017	+ { _MMIO(0xe55c), 0x00053052 },
	1018	+ { _MMIO(0xe65c), 0x00055054 },
	1019	+};
	1020	+
	1021	+static const struct i915_oa_reg mux_config_l3_1_0_sku_gte_0x03[] = {
	1022	+ { _MMIO(0x9888), 0x12643400 },
	1023	+ { _MMIO(0x9888), 0x12653400 },
	1024	+ { _MMIO(0x9888), 0x106c6800 },
	1025	+ { _MMIO(0x9888), 0x126c001e },
	1026	+ { _MMIO(0x9888), 0x166c0010 },
	1027	+ { _MMIO(0x9888), 0x0c2d5000 },
	1028	+ { _MMIO(0x9888), 0x0e2d5000 },
	1029	+ { _MMIO(0x9888), 0x002d4000 },
	1030	+ { _MMIO(0x9888), 0x022d5000 },
	1031	+ { _MMIO(0x9888), 0x042d5000 },
	1032	+ { _MMIO(0x9888), 0x062d1000 },
	1033	+ { _MMIO(0x9888), 0x102e0154 },
	1034	+ { _MMIO(0x9888), 0x0c2e5000 },
	1035	+ { _MMIO(0x9888), 0x0e2e0055 },
	1036	+ { _MMIO(0x9888), 0x104c8000 },
	1037	+ { _MMIO(0x9888), 0x124c8000 },
	1038	+ { _MMIO(0x9888), 0x144c8000 },
	1039	+ { _MMIO(0x9888), 0x164c2000 },
	1040	+ { _MMIO(0x9888), 0x044c8000 },
	1041	+ { _MMIO(0x9888), 0x064cc000 },
	1042	+ { _MMIO(0x9888), 0x084cc000 },
	1043	+ { _MMIO(0x9888), 0x0a4c4000 },
	1044	+ { _MMIO(0x9888), 0x0c4ea000 },
	1045	+ { _MMIO(0x9888), 0x0e4ea000 },
	1046	+ { _MMIO(0x9888), 0x004e8000 },
	1047	+ { _MMIO(0x9888), 0x024ea000 },
	1048	+ { _MMIO(0x9888), 0x044ea000 },
	1049	+ { _MMIO(0x9888), 0x064e2000 },
	1050	+ { _MMIO(0x9888), 0x1c4f5500 },
	1051	+ { _MMIO(0x9888), 0x1a4f1554 },
	1052	+ { _MMIO(0x9888), 0x0a640024 },
	1053	+ { _MMIO(0x9888), 0x10640000 },
	1054	+ { _MMIO(0x9888), 0x04640000 },
	1055	+ { _MMIO(0x9888), 0x0c650024 },
	1056	+ { _MMIO(0x9888), 0x10650000 },
	1057	+ { _MMIO(0x9888), 0x06650000 },
	1058	+ { _MMIO(0x9888), 0x0c6c5327 },
	1059	+ { _MMIO(0x9888), 0x0e6c5425 },
	1060	+ { _MMIO(0x9888), 0x006c2a00 },
	1061	+ { _MMIO(0x9888), 0x026c285b },
	1062	+ { _MMIO(0x9888), 0x046c005c },
	1063	+ { _MMIO(0x9888), 0x1c6c0000 },
	1064	+ { _MMIO(0x9888), 0x1a6c0900 },
	1065	+ { _MMIO(0x9888), 0x1c0f0aa0 },
	1066	+ { _MMIO(0x9888), 0x180f4000 },
	1067	+ { _MMIO(0x9888), 0x1a0f02aa },
	1068	+ { _MMIO(0x9888), 0x1c2c5400 },
	1069	+ { _MMIO(0x9888), 0x1e2c0001 },
	1070	+ { _MMIO(0x9888), 0x1a2c5550 },
	1071	+ { _MMIO(0x9888), 0x1993aa00 },
	1072	+ { _MMIO(0x9888), 0x03938000 },
	1073	+ { _MMIO(0x9888), 0x05938000 },
	1074	+ { _MMIO(0x9888), 0x07938000 },
	1075	+ { _MMIO(0x9888), 0x09938000 },
	1076	+ { _MMIO(0x9888), 0x0b938000 },
	1077	+ { _MMIO(0x9888), 0x0d938000 },
	1078	+ { _MMIO(0x9888), 0x2b904000 },
	1079	+ { _MMIO(0x9888), 0x2d904000 },
	1080	+ { _MMIO(0x9888), 0x2f904000 },
	1081	+ { _MMIO(0x9888), 0x31904000 },
	1082	+ { _MMIO(0x9888), 0x15904000 },
	1083	+ { _MMIO(0x9888), 0x17904000 },
	1084	+ { _MMIO(0x9888), 0x19904000 },
	1085	+ { _MMIO(0x9888), 0x1b904000 },
	1086	+ { _MMIO(0x9888), 0x1d904000 },
	1087	+ { _MMIO(0x9888), 0x1f904000 },
	1088	+ { _MMIO(0x9888), 0x59900000 },
	1089	+ { _MMIO(0x9888), 0x4b900421 },
	1090	+ { _MMIO(0x9888), 0x37900000 },
	1091	+ { _MMIO(0x9888), 0x33900000 },
	1092	+ { _MMIO(0x9888), 0x4d900001 },
	1093	+ { _MMIO(0x9888), 0x53900000 },
	1094	+ { _MMIO(0x9888), 0x43900420 },
	1095	+ { _MMIO(0x9888), 0x45900021 },
	1096	+ { _MMIO(0x9888), 0x55900000 },
	1097	+ { _MMIO(0x9888), 0x47900000 },
	1098	+};
	1099	+
	1100	+static const struct i915_oa_reg mux_config_l3_1_0_sku_lt_0x03[] = {
	1101	+ { _MMIO(0x9888), 0x14640340 },
	1102	+ { _MMIO(0x9888), 0x14650340 },
	1103	+ { _MMIO(0x9888), 0x106c6800 },
	1104	+ { _MMIO(0x9888), 0x126c001e },
	1105	+ { _MMIO(0x9888), 0x166c0010 },
	1106	+ { _MMIO(0x9888), 0x0c2d5000 },
	1107	+ { _MMIO(0x9888), 0x0e2d5000 },
	1108	+ { _MMIO(0x9888), 0x002d4000 },
	1109	+ { _MMIO(0x9888), 0x022d5000 },
	1110	+ { _MMIO(0x9888), 0x042d5000 },
	1111	+ { _MMIO(0x9888), 0x062d1000 },
	1112	+ { _MMIO(0x9888), 0x102e0154 },
	1113	+ { _MMIO(0x9888), 0x0c2e5000 },
	1114	+ { _MMIO(0x9888), 0x0e2e0055 },
	1115	+ { _MMIO(0x9888), 0x104c8000 },
	1116	+ { _MMIO(0x9888), 0x124c8000 },
	1117	+ { _MMIO(0x9888), 0x144c8000 },
	1118	+ { _MMIO(0x9888), 0x164c2000 },
	1119	+ { _MMIO(0x9888), 0x044c8000 },
	1120	+ { _MMIO(0x9888), 0x064cc000 },
	1121	+ { _MMIO(0x9888), 0x084cc000 },
	1122	+ { _MMIO(0x9888), 0x0a4c4000 },
	1123	+ { _MMIO(0x9888), 0x0c4ea000 },
	1124	+ { _MMIO(0x9888), 0x0e4ea000 },
	1125	+ { _MMIO(0x9888), 0x004e8000 },
	1126	+ { _MMIO(0x9888), 0x024ea000 },
	1127	+ { _MMIO(0x9888), 0x044ea000 },
	1128	+ { _MMIO(0x9888), 0x064e2000 },
	1129	+ { _MMIO(0x9888), 0x1c4f5500 },
	1130	+ { _MMIO(0x9888), 0x1a4f1554 },
	1131	+ { _MMIO(0x9888), 0x04642400 },
	1132	+ { _MMIO(0x9888), 0x22640000 },
	1133	+ { _MMIO(0x9888), 0x1a640000 },
	1134	+ { _MMIO(0x9888), 0x06650024 },
	1135	+ { _MMIO(0x9888), 0x22650000 },
	1136	+ { _MMIO(0x9888), 0x1c650000 },
	1137	+ { _MMIO(0x9888), 0x0c6c5327 },
	1138	+ { _MMIO(0x9888), 0x0e6c5425 },
	1139	+ { _MMIO(0x9888), 0x006c2a00 },
	1140	+ { _MMIO(0x9888), 0x026c285b },
	1141	+ { _MMIO(0x9888), 0x046c005c },
	1142	+ { _MMIO(0x9888), 0x1c6c0000 },
	1143	+ { _MMIO(0x9888), 0x1a6c0900 },
	1144	+ { _MMIO(0x9888), 0x1c0f0aa0 },
	1145	+ { _MMIO(0x9888), 0x180f4000 },
	1146	+ { _MMIO(0x9888), 0x1a0f02aa },
	1147	+ { _MMIO(0x9888), 0x1c2c5400 },
	1148	+ { _MMIO(0x9888), 0x1e2c0001 },
	1149	+ { _MMIO(0x9888), 0x1a2c5550 },
	1150	+ { _MMIO(0x9888), 0x1993aa00 },
	1151	+ { _MMIO(0x9888), 0x03938000 },
	1152	+ { _MMIO(0x9888), 0x05938000 },
	1153	+ { _MMIO(0x9888), 0x07938000 },
	1154	+ { _MMIO(0x9888), 0x09938000 },
	1155	+ { _MMIO(0x9888), 0x0b938000 },
	1156	+ { _MMIO(0x9888), 0x0d938000 },
	1157	+ { _MMIO(0x9888), 0x2b904000 },
	1158	+ { _MMIO(0x9888), 0x2d904000 },
	1159	+ { _MMIO(0x9888), 0x2f904000 },
	1160	+ { _MMIO(0x9888), 0x31904000 },
	1161	+ { _MMIO(0x9888), 0x15904000 },
	1162	+ { _MMIO(0x9888), 0x17904000 },
	1163	+ { _MMIO(0x9888), 0x19904000 },
	1164	+ { _MMIO(0x9888), 0x1b904000 },
	1165	+ { _MMIO(0x9888), 0x1d904000 },
	1166	+ { _MMIO(0x9888), 0x1f904000 },
	1167	+ { _MMIO(0x9888), 0x59900000 },
	1168	+ { _MMIO(0x9888), 0x4b900421 },
	1169	+ { _MMIO(0x9888), 0x37900000 },
	1170	+ { _MMIO(0x9888), 0x33900000 },
	1171	+ { _MMIO(0x9888), 0x4d900001 },
	1172	+ { _MMIO(0x9888), 0x53900000 },
	1173	+ { _MMIO(0x9888), 0x43900420 },
	1174	+ { _MMIO(0x9888), 0x45900021 },
	1175	+ { _MMIO(0x9888), 0x55900000 },
	1176	+ { _MMIO(0x9888), 0x47900000 },
	1177	+};
	1178	+
	1179	+static int
	1180	+get_l3_1_mux_config(struct drm_i915_private *dev_priv,
	1181	+ const struct i915_oa_reg **regs,
	1182	+ int *lens)
	1183	+{
	1184	+ int n = 0;
	1185	+
	1186	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 2);
	1187	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 2);
	1188	+
	1189	+ if (dev_priv->drm.pdev->revision >= 0x03) {
	1190	+ regs[n] = mux_config_l3_1_0_sku_gte_0x03;
	1191	+ lens[n] = ARRAY_SIZE(mux_config_l3_1_0_sku_gte_0x03);
	1192	+ n++;
	1193	+ }
	1194	+ if (dev_priv->drm.pdev->revision < 0x03) {
	1195	+ regs[n] = mux_config_l3_1_0_sku_lt_0x03;
	1196	+ lens[n] = ARRAY_SIZE(mux_config_l3_1_0_sku_lt_0x03);
	1197	+ n++;
	1198	+ }
	1199	+
	1200	+ return n;
	1201	+}
	1202	+
	1203	+static const struct i915_oa_reg b_counter_config_rasterizer_and_pixel_backend[] = {
	1204	+ { _MMIO(0x2740), 0x00000000 },
	1205	+ { _MMIO(0x2744), 0x00800000 },
	1206	+ { _MMIO(0x2710), 0x00000000 },
	1207	+ { _MMIO(0x2714), 0x30800000 },
	1208	+ { _MMIO(0x2720), 0x00000000 },
	1209	+ { _MMIO(0x2724), 0x00800000 },
	1210	+ { _MMIO(0x2770), 0x00000002 },
	1211	+ { _MMIO(0x2774), 0x0000efff },
	1212	+ { _MMIO(0x2778), 0x00006000 },
	1213	+ { _MMIO(0x277c), 0x0000f3ff },
	1214	+};
	1215	+
	1216	+static const struct i915_oa_reg flex_eu_config_rasterizer_and_pixel_backend[] = {
	1217	+ { _MMIO(0xe458), 0x00005004 },
	1218	+ { _MMIO(0xe558), 0x00010003 },
	1219	+ { _MMIO(0xe658), 0x00012011 },
	1220	+ { _MMIO(0xe758), 0x00015014 },
	1221	+ { _MMIO(0xe45c), 0x00051050 },
	1222	+ { _MMIO(0xe55c), 0x00053052 },
	1223	+ { _MMIO(0xe65c), 0x00055054 },
	1224	+};
	1225	+
	1226	+static const struct i915_oa_reg mux_config_rasterizer_and_pixel_backend[] = {
	1227	+ { _MMIO(0x9888), 0x102d7800 },
	1228	+ { _MMIO(0x9888), 0x122d79e0 },
	1229	+ { _MMIO(0x9888), 0x0c2f0004 },
	1230	+ { _MMIO(0x9888), 0x100e3800 },
	1231	+ { _MMIO(0x9888), 0x180f0005 },
	1232	+ { _MMIO(0x9888), 0x002d0940 },
	1233	+ { _MMIO(0x9888), 0x022d802f },
	1234	+ { _MMIO(0x9888), 0x042d4013 },
	1235	+ { _MMIO(0x9888), 0x062d1000 },
	1236	+ { _MMIO(0x9888), 0x0e2e0050 },
	1237	+ { _MMIO(0x9888), 0x022f0010 },
	1238	+ { _MMIO(0x9888), 0x002f0000 },
	1239	+ { _MMIO(0x9888), 0x084c8000 },
	1240	+ { _MMIO(0x9888), 0x0a4c4000 },
	1241	+ { _MMIO(0x9888), 0x044e8000 },
	1242	+ { _MMIO(0x9888), 0x064e2000 },
	1243	+ { _MMIO(0x9888), 0x040e0480 },
	1244	+ { _MMIO(0x9888), 0x000e0000 },
	1245	+ { _MMIO(0x9888), 0x060f0027 },
	1246	+ { _MMIO(0x9888), 0x100f0000 },
	1247	+ { _MMIO(0x9888), 0x1a0f0040 },
	1248	+ { _MMIO(0x9888), 0x03938000 },
	1249	+ { _MMIO(0x9888), 0x05938000 },
	1250	+ { _MMIO(0x9888), 0x07938000 },
	1251	+ { _MMIO(0x9888), 0x09938000 },
	1252	+ { _MMIO(0x9888), 0x0b938000 },
	1253	+ { _MMIO(0x9888), 0x0d938000 },
	1254	+ { _MMIO(0x9888), 0x15904000 },
	1255	+ { _MMIO(0x9888), 0x17904000 },
	1256	+ { _MMIO(0x9888), 0x19904000 },
	1257	+ { _MMIO(0x9888), 0x1b904000 },
	1258	+ { _MMIO(0x9888), 0x1d904000 },
	1259	+ { _MMIO(0x9888), 0x1f904000 },
	1260	+ { _MMIO(0x9888), 0x37900000 },
	1261	+ { _MMIO(0x9888), 0x53900000 },
	1262	+ { _MMIO(0x9888), 0x439014a0 },
	1263	+ { _MMIO(0x9888), 0x459000a4 },
	1264	+ { _MMIO(0x9888), 0x55900000 },
	1265	+ { _MMIO(0x9888), 0x47900001 },
	1266	+ { _MMIO(0x9888), 0x33900000 },
	1267	+};
	1268	+
	1269	+static int
	1270	+get_rasterizer_and_pixel_backend_mux_config(struct drm_i915_private *dev_priv,
	1271	+ const struct i915_oa_reg **regs,
	1272	+ int *lens)
	1273	+{
	1274	+ int n = 0;
	1275	+
	1276	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1277	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1278	+
	1279	+ regs[n] = mux_config_rasterizer_and_pixel_backend;
	1280	+ lens[n] = ARRAY_SIZE(mux_config_rasterizer_and_pixel_backend);
	1281	+ n++;
	1282	+
	1283	+ return n;
	1284	+}
	1285	+
	1286	+static const struct i915_oa_reg b_counter_config_sampler[] = {
	1287	+ { _MMIO(0x2740), 0x00000000 },
	1288	+ { _MMIO(0x2744), 0x00800000 },
	1289	+ { _MMIO(0x2710), 0x00000000 },
	1290	+ { _MMIO(0x2714), 0x70800000 },
	1291	+ { _MMIO(0x2720), 0x00000000 },
	1292	+ { _MMIO(0x2724), 0x00800000 },
	1293	+ { _MMIO(0x2770), 0x0000c000 },
	1294	+ { _MMIO(0x2774), 0x0000e7ff },
	1295	+ { _MMIO(0x2778), 0x00003000 },
	1296	+ { _MMIO(0x277c), 0x0000f9ff },
	1297	+ { _MMIO(0x2780), 0x00000c00 },
	1298	+ { _MMIO(0x2784), 0x0000fe7f },
	1299	+};
	1300	+
	1301	+static const struct i915_oa_reg flex_eu_config_sampler[] = {
	1302	+ { _MMIO(0xe458), 0x00005004 },
	1303	+ { _MMIO(0xe558), 0x00010003 },
	1304	+ { _MMIO(0xe658), 0x00012011 },
	1305	+ { _MMIO(0xe758), 0x00015014 },
	1306	+ { _MMIO(0xe45c), 0x00051050 },
	1307	+ { _MMIO(0xe55c), 0x00053052 },
	1308	+ { _MMIO(0xe65c), 0x00055054 },
	1309	+};
	1310	+
	1311	+static const struct i915_oa_reg mux_config_sampler[] = {
	1312	+ { _MMIO(0x9888), 0x121300a0 },
	1313	+ { _MMIO(0x9888), 0x141600ab },
	1314	+ { _MMIO(0x9888), 0x123300a0 },
	1315	+ { _MMIO(0x9888), 0x143600ab },
	1316	+ { _MMIO(0x9888), 0x125300a0 },
	1317	+ { _MMIO(0x9888), 0x145600ab },
	1318	+ { _MMIO(0x9888), 0x0c2d4000 },
	1319	+ { _MMIO(0x9888), 0x0e2d5000 },
	1320	+ { _MMIO(0x9888), 0x002d4000 },
	1321	+ { _MMIO(0x9888), 0x022d5000 },
	1322	+ { _MMIO(0x9888), 0x042d5000 },
	1323	+ { _MMIO(0x9888), 0x062d1000 },
	1324	+ { _MMIO(0x9888), 0x102e01a0 },
	1325	+ { _MMIO(0x9888), 0x0c2e5000 },
	1326	+ { _MMIO(0x9888), 0x0e2e0065 },
	1327	+ { _MMIO(0x9888), 0x164c2000 },
	1328	+ { _MMIO(0x9888), 0x044c8000 },
	1329	+ { _MMIO(0x9888), 0x064cc000 },
	1330	+ { _MMIO(0x9888), 0x084c4000 },
	1331	+ { _MMIO(0x9888), 0x0a4c4000 },
	1332	+ { _MMIO(0x9888), 0x0e4e8000 },
	1333	+ { _MMIO(0x9888), 0x004e8000 },
	1334	+ { _MMIO(0x9888), 0x024ea000 },
	1335	+ { _MMIO(0x9888), 0x044e2000 },
	1336	+ { _MMIO(0x9888), 0x064e2000 },
	1337	+ { _MMIO(0x9888), 0x1c0f0800 },
	1338	+ { _MMIO(0x9888), 0x180f4000 },
	1339	+ { _MMIO(0x9888), 0x1a0f023f },
	1340	+ { _MMIO(0x9888), 0x1e2c0003 },
	1341	+ { _MMIO(0x9888), 0x1a2cc030 },
	1342	+ { _MMIO(0x9888), 0x04132180 },
	1343	+ { _MMIO(0x9888), 0x02130000 },
	1344	+ { _MMIO(0x9888), 0x0c148000 },
	1345	+ { _MMIO(0x9888), 0x0e142000 },
	1346	+ { _MMIO(0x9888), 0x04148000 },
	1347	+ { _MMIO(0x9888), 0x1e150140 },
	1348	+ { _MMIO(0x9888), 0x1c150040 },
	1349	+ { _MMIO(0x9888), 0x0c163000 },
	1350	+ { _MMIO(0x9888), 0x0e160068 },
	1351	+ { _MMIO(0x9888), 0x10160000 },
	1352	+ { _MMIO(0x9888), 0x18160000 },
	1353	+ { _MMIO(0x9888), 0x0a164000 },
	1354	+ { _MMIO(0x9888), 0x04330043 },
	1355	+ { _MMIO(0x9888), 0x02330000 },
	1356	+ { _MMIO(0x9888), 0x0234a000 },
	1357	+ { _MMIO(0x9888), 0x04342000 },
	1358	+ { _MMIO(0x9888), 0x1c350015 },
	1359	+ { _MMIO(0x9888), 0x02363460 },
	1360	+ { _MMIO(0x9888), 0x10360000 },
	1361	+ { _MMIO(0x9888), 0x04360000 },
	1362	+ { _MMIO(0x9888), 0x06360000 },
	1363	+ { _MMIO(0x9888), 0x08364000 },
	1364	+ { _MMIO(0x9888), 0x06530043 },
	1365	+ { _MMIO(0x9888), 0x02530000 },
	1366	+ { _MMIO(0x9888), 0x0e548000 },
	1367	+ { _MMIO(0x9888), 0x00548000 },
	1368	+ { _MMIO(0x9888), 0x06542000 },
	1369	+ { _MMIO(0x9888), 0x1e550400 },
	1370	+ { _MMIO(0x9888), 0x1a552000 },
	1371	+ { _MMIO(0x9888), 0x1c550100 },
	1372	+ { _MMIO(0x9888), 0x0e563000 },
	1373	+ { _MMIO(0x9888), 0x00563400 },
	1374	+ { _MMIO(0x9888), 0x10560000 },
	1375	+ { _MMIO(0x9888), 0x18560000 },
	1376	+ { _MMIO(0x9888), 0x02560000 },
	1377	+ { _MMIO(0x9888), 0x0c564000 },
	1378	+ { _MMIO(0x9888), 0x1993a800 },
	1379	+ { _MMIO(0x9888), 0x03938000 },
	1380	+ { _MMIO(0x9888), 0x05938000 },
	1381	+ { _MMIO(0x9888), 0x07938000 },
	1382	+ { _MMIO(0x9888), 0x09938000 },
	1383	+ { _MMIO(0x9888), 0x0b938000 },
	1384	+ { _MMIO(0x9888), 0x0d938000 },
	1385	+ { _MMIO(0x9888), 0x2d904000 },
	1386	+ { _MMIO(0x9888), 0x2f904000 },
	1387	+ { _MMIO(0x9888), 0x31904000 },
	1388	+ { _MMIO(0x9888), 0x15904000 },
	1389	+ { _MMIO(0x9888), 0x17904000 },
	1390	+ { _MMIO(0x9888), 0x19904000 },
	1391	+ { _MMIO(0x9888), 0x1b904000 },
	1392	+ { _MMIO(0x9888), 0x1d904000 },
	1393	+ { _MMIO(0x9888), 0x1f904000 },
	1394	+ { _MMIO(0x9888), 0x59900000 },
	1395	+ { _MMIO(0x9888), 0x4b9014a0 },
	1396	+ { _MMIO(0x9888), 0x37900000 },
	1397	+ { _MMIO(0x9888), 0x33900000 },
	1398	+ { _MMIO(0x9888), 0x4d900001 },
	1399	+ { _MMIO(0x9888), 0x53900000 },
	1400	+ { _MMIO(0x9888), 0x43900820 },
	1401	+ { _MMIO(0x9888), 0x45901022 },
	1402	+ { _MMIO(0x9888), 0x55900000 },
	1403	+ { _MMIO(0x9888), 0x47900000 },
	1404	+};
	1405	+
	1406	+static int
	1407	+get_sampler_mux_config(struct drm_i915_private *dev_priv,
	1408	+ const struct i915_oa_reg **regs,
	1409	+ int *lens)
	1410	+{
	1411	+ int n = 0;
	1412	+
	1413	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1414	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1415	+
	1416	+ regs[n] = mux_config_sampler;
	1417	+ lens[n] = ARRAY_SIZE(mux_config_sampler);
	1418	+ n++;
	1419	+
	1420	+ return n;
	1421	+}
	1422	+
	1423	+static const struct i915_oa_reg b_counter_config_tdl_1[] = {
	1424	+ { _MMIO(0x2740), 0x00000000 },
	1425	+ { _MMIO(0x2744), 0x00800000 },
	1426	+ { _MMIO(0x2710), 0x00000000 },
	1427	+ { _MMIO(0x2714), 0xf0800000 },
	1428	+ { _MMIO(0x2720), 0x00000000 },
	1429	+ { _MMIO(0x2724), 0x30800000 },
	1430	+ { _MMIO(0x2770), 0x00000002 },
	1431	+ { _MMIO(0x2774), 0x00007fff },
	1432	+ { _MMIO(0x2778), 0x00000000 },
	1433	+ { _MMIO(0x277c), 0x00009fff },
	1434	+ { _MMIO(0x2780), 0x00000002 },
	1435	+ { _MMIO(0x2784), 0x0000efff },
	1436	+ { _MMIO(0x2788), 0x00000000 },
	1437	+ { _MMIO(0x278c), 0x0000f3ff },
	1438	+ { _MMIO(0x2790), 0x00000002 },
	1439	+ { _MMIO(0x2794), 0x0000fdff },
	1440	+ { _MMIO(0x2798), 0x00000000 },
	1441	+ { _MMIO(0x279c), 0x0000fe7f },
	1442	+};
	1443	+
	1444	+static const struct i915_oa_reg flex_eu_config_tdl_1[] = {
	1445	+ { _MMIO(0xe458), 0x00005004 },
	1446	+ { _MMIO(0xe558), 0x00010003 },
	1447	+ { _MMIO(0xe658), 0x00012011 },
	1448	+ { _MMIO(0xe758), 0x00015014 },
	1449	+ { _MMIO(0xe45c), 0x00051050 },
	1450	+ { _MMIO(0xe55c), 0x00053052 },
	1451	+ { _MMIO(0xe65c), 0x00055054 },
	1452	+};
	1453	+
	1454	+static const struct i915_oa_reg mux_config_tdl_1[] = {
	1455	+ { _MMIO(0x9888), 0x141a0000 },
	1456	+ { _MMIO(0x9888), 0x143a0000 },
	1457	+ { _MMIO(0x9888), 0x145a0000 },
	1458	+ { _MMIO(0x9888), 0x0c2d4000 },
	1459	+ { _MMIO(0x9888), 0x0e2d5000 },
	1460	+ { _MMIO(0x9888), 0x002d4000 },
	1461	+ { _MMIO(0x9888), 0x022d5000 },
	1462	+ { _MMIO(0x9888), 0x042d5000 },
	1463	+ { _MMIO(0x9888), 0x062d1000 },
	1464	+ { _MMIO(0x9888), 0x102e0150 },
	1465	+ { _MMIO(0x9888), 0x0c2e5000 },
	1466	+ { _MMIO(0x9888), 0x0e2e006a },
	1467	+ { _MMIO(0x9888), 0x124c8000 },
	1468	+ { _MMIO(0x9888), 0x144c8000 },
	1469	+ { _MMIO(0x9888), 0x164c2000 },
	1470	+ { _MMIO(0x9888), 0x044c8000 },
	1471	+ { _MMIO(0x9888), 0x064c4000 },
	1472	+ { _MMIO(0x9888), 0x0a4c4000 },
	1473	+ { _MMIO(0x9888), 0x0c4e8000 },
	1474	+ { _MMIO(0x9888), 0x0e4ea000 },
	1475	+ { _MMIO(0x9888), 0x004e8000 },
	1476	+ { _MMIO(0x9888), 0x024e2000 },
	1477	+ { _MMIO(0x9888), 0x064e2000 },
	1478	+ { _MMIO(0x9888), 0x1c0f0bc0 },
	1479	+ { _MMIO(0x9888), 0x180f4000 },
	1480	+ { _MMIO(0x9888), 0x1a0f0302 },
	1481	+ { _MMIO(0x9888), 0x1e2c0003 },
	1482	+ { _MMIO(0x9888), 0x1a2c00f0 },
	1483	+ { _MMIO(0x9888), 0x021a3080 },
	1484	+ { _MMIO(0x9888), 0x041a31e5 },
	1485	+ { _MMIO(0x9888), 0x02148000 },
	1486	+ { _MMIO(0x9888), 0x0414a000 },
	1487	+ { _MMIO(0x9888), 0x1c150054 },
	1488	+ { _MMIO(0x9888), 0x06168000 },
	1489	+ { _MMIO(0x9888), 0x08168000 },
	1490	+ { _MMIO(0x9888), 0x0a168000 },
	1491	+ { _MMIO(0x9888), 0x0c3a3280 },
	1492	+ { _MMIO(0x9888), 0x0e3a0063 },
	1493	+ { _MMIO(0x9888), 0x063a0061 },
	1494	+ { _MMIO(0x9888), 0x023a0000 },
	1495	+ { _MMIO(0x9888), 0x0c348000 },
	1496	+ { _MMIO(0x9888), 0x0e342000 },
	1497	+ { _MMIO(0x9888), 0x06342000 },
	1498	+ { _MMIO(0x9888), 0x1e350140 },
	1499	+ { _MMIO(0x9888), 0x1c350100 },
	1500	+ { _MMIO(0x9888), 0x18360028 },
	1501	+ { _MMIO(0x9888), 0x0c368000 },
	1502	+ { _MMIO(0x9888), 0x0e5a3080 },
	1503	+ { _MMIO(0x9888), 0x005a3280 },
	1504	+ { _MMIO(0x9888), 0x025a0063 },
	1505	+ { _MMIO(0x9888), 0x0e548000 },
	1506	+ { _MMIO(0x9888), 0x00548000 },
	1507	+ { _MMIO(0x9888), 0x02542000 },
	1508	+ { _MMIO(0x9888), 0x1e550400 },
	1509	+ { _MMIO(0x9888), 0x1a552000 },
	1510	+ { _MMIO(0x9888), 0x1c550001 },
	1511	+ { _MMIO(0x9888), 0x18560080 },
	1512	+ { _MMIO(0x9888), 0x02568000 },
	1513	+ { _MMIO(0x9888), 0x04568000 },
	1514	+ { _MMIO(0x9888), 0x1993a800 },
	1515	+ { _MMIO(0x9888), 0x03938000 },
	1516	+ { _MMIO(0x9888), 0x05938000 },
	1517	+ { _MMIO(0x9888), 0x07938000 },
	1518	+ { _MMIO(0x9888), 0x09938000 },
	1519	+ { _MMIO(0x9888), 0x0b938000 },
	1520	+ { _MMIO(0x9888), 0x0d938000 },
	1521	+ { _MMIO(0x9888), 0x2d904000 },
	1522	+ { _MMIO(0x9888), 0x2f904000 },
	1523	+ { _MMIO(0x9888), 0x31904000 },
	1524	+ { _MMIO(0x9888), 0x15904000 },
	1525	+ { _MMIO(0x9888), 0x17904000 },
	1526	+ { _MMIO(0x9888), 0x19904000 },
	1527	+ { _MMIO(0x9888), 0x1b904000 },
	1528	+ { _MMIO(0x9888), 0x1d904000 },
	1529	+ { _MMIO(0x9888), 0x1f904000 },
	1530	+ { _MMIO(0x9888), 0x59900000 },
	1531	+ { _MMIO(0x9888), 0x4b900420 },
	1532	+ { _MMIO(0x9888), 0x37900000 },
	1533	+ { _MMIO(0x9888), 0x33900000 },
	1534	+ { _MMIO(0x9888), 0x4d900000 },
	1535	+ { _MMIO(0x9888), 0x53900000 },
	1536	+ { _MMIO(0x9888), 0x43900000 },
	1537	+ { _MMIO(0x9888), 0x45901084 },
	1538	+ { _MMIO(0x9888), 0x55900000 },
	1539	+ { _MMIO(0x9888), 0x47900001 },
	1540	+};
	1541	+
	1542	+static int
	1543	+get_tdl_1_mux_config(struct drm_i915_private *dev_priv,
	1544	+ const struct i915_oa_reg **regs,
	1545	+ int *lens)
	1546	+{
	1547	+ int n = 0;
	1548	+
	1549	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1550	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1551	+
	1552	+ regs[n] = mux_config_tdl_1;
	1553	+ lens[n] = ARRAY_SIZE(mux_config_tdl_1);
	1554	+ n++;
	1555	+
	1556	+ return n;
	1557	+}
	1558	+
	1559	+static const struct i915_oa_reg b_counter_config_tdl_2[] = {
	1560	+ { _MMIO(0x2740), 0x00000000 },
	1561	+ { _MMIO(0x2744), 0x00800000 },
	1562	+ { _MMIO(0x2710), 0x00000000 },
	1563	+ { _MMIO(0x2714), 0x00800000 },
	1564	+ { _MMIO(0x2720), 0x00000000 },
	1565	+ { _MMIO(0x2724), 0x00800000 },
	1566	+};
	1567	+
	1568	+static const struct i915_oa_reg flex_eu_config_tdl_2[] = {
	1569	+ { _MMIO(0xe458), 0x00005004 },
	1570	+ { _MMIO(0xe558), 0x00010003 },
	1571	+ { _MMIO(0xe658), 0x00012011 },
	1572	+ { _MMIO(0xe758), 0x00015014 },
	1573	+ { _MMIO(0xe45c), 0x00051050 },
	1574	+ { _MMIO(0xe55c), 0x00053052 },
	1575	+ { _MMIO(0xe65c), 0x00055054 },
	1576	+};
	1577	+
	1578	+static const struct i915_oa_reg mux_config_tdl_2[] = {
	1579	+ { _MMIO(0x9888), 0x141a026b },
	1580	+ { _MMIO(0x9888), 0x143a0173 },
	1581	+ { _MMIO(0x9888), 0x145a026b },
	1582	+ { _MMIO(0x9888), 0x002d4000 },
	1583	+ { _MMIO(0x9888), 0x022d5000 },
	1584	+ { _MMIO(0x9888), 0x042d5000 },
	1585	+ { _MMIO(0x9888), 0x062d1000 },
	1586	+ { _MMIO(0x9888), 0x0c2e5000 },
	1587	+ { _MMIO(0x9888), 0x0e2e0069 },
	1588	+ { _MMIO(0x9888), 0x044c8000 },
	1589	+ { _MMIO(0x9888), 0x064cc000 },
	1590	+ { _MMIO(0x9888), 0x0a4c4000 },
	1591	+ { _MMIO(0x9888), 0x004e8000 },
	1592	+ { _MMIO(0x9888), 0x024ea000 },
	1593	+ { _MMIO(0x9888), 0x064e2000 },
	1594	+ { _MMIO(0x9888), 0x180f6000 },
	1595	+ { _MMIO(0x9888), 0x1a0f030a },
	1596	+ { _MMIO(0x9888), 0x1a2c03c0 },
	1597	+ { _MMIO(0x9888), 0x041a37e7 },
	1598	+ { _MMIO(0x9888), 0x021a0000 },
	1599	+ { _MMIO(0x9888), 0x0414a000 },
	1600	+ { _MMIO(0x9888), 0x1c150050 },
	1601	+ { _MMIO(0x9888), 0x08168000 },
	1602	+ { _MMIO(0x9888), 0x0a168000 },
	1603	+ { _MMIO(0x9888), 0x003a3380 },
	1604	+ { _MMIO(0x9888), 0x063a006f },
	1605	+ { _MMIO(0x9888), 0x023a0000 },
	1606	+ { _MMIO(0x9888), 0x00348000 },
	1607	+ { _MMIO(0x9888), 0x06342000 },
	1608	+ { _MMIO(0x9888), 0x1a352000 },
	1609	+ { _MMIO(0x9888), 0x1c350100 },
	1610	+ { _MMIO(0x9888), 0x02368000 },
	1611	+ { _MMIO(0x9888), 0x0c368000 },
	1612	+ { _MMIO(0x9888), 0x025a37e7 },
	1613	+ { _MMIO(0x9888), 0x0254a000 },
	1614	+ { _MMIO(0x9888), 0x1c550005 },
	1615	+ { _MMIO(0x9888), 0x04568000 },
	1616	+ { _MMIO(0x9888), 0x06568000 },
	1617	+ { _MMIO(0x9888), 0x03938000 },
	1618	+ { _MMIO(0x9888), 0x05938000 },
	1619	+ { _MMIO(0x9888), 0x07938000 },
	1620	+ { _MMIO(0x9888), 0x09938000 },
	1621	+ { _MMIO(0x9888), 0x0b938000 },
	1622	+ { _MMIO(0x9888), 0x0d938000 },
	1623	+ { _MMIO(0x9888), 0x15904000 },
	1624	+ { _MMIO(0x9888), 0x17904000 },
	1625	+ { _MMIO(0x9888), 0x19904000 },
	1626	+ { _MMIO(0x9888), 0x1b904000 },
	1627	+ { _MMIO(0x9888), 0x1d904000 },
	1628	+ { _MMIO(0x9888), 0x1f904000 },
	1629	+ { _MMIO(0x9888), 0x37900000 },
	1630	+ { _MMIO(0x9888), 0x53900000 },
	1631	+ { _MMIO(0x9888), 0x43900020 },
	1632	+ { _MMIO(0x9888), 0x45901080 },
	1633	+ { _MMIO(0x9888), 0x55900000 },
	1634	+ { _MMIO(0x9888), 0x47900001 },
	1635	+ { _MMIO(0x9888), 0x33900000 },
	1636	+};
	1637	+
	1638	+static int
	1639	+get_tdl_2_mux_config(struct drm_i915_private *dev_priv,
	1640	+ const struct i915_oa_reg **regs,
	1641	+ int *lens)
	1642	+{
	1643	+ int n = 0;
	1644	+
	1645	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1646	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1647	+
	1648	+ regs[n] = mux_config_tdl_2;
	1649	+ lens[n] = ARRAY_SIZE(mux_config_tdl_2);
	1650	+ n++;
	1651	+
	1652	+ return n;
	1653	+}
	1654	+
	1655	+static const struct i915_oa_reg b_counter_config_compute_extra[] = {
	1656	+ { _MMIO(0x2740), 0x00000000 },
	1657	+ { _MMIO(0x2744), 0x00800000 },
	1658	+ { _MMIO(0x2710), 0x00000000 },
	1659	+ { _MMIO(0x2714), 0x00800000 },
	1660	+ { _MMIO(0x2720), 0x00000000 },
	1661	+ { _MMIO(0x2724), 0x00800000 },
	1662	+};
	1663	+
	1664	+static const struct i915_oa_reg flex_eu_config_compute_extra[] = {
	1665	+ { _MMIO(0xe458), 0x00001000 },
	1666	+ { _MMIO(0xe558), 0x00003002 },
	1667	+ { _MMIO(0xe658), 0x00005004 },
	1668	+ { _MMIO(0xe758), 0x00011010 },
	1669	+ { _MMIO(0xe45c), 0x00050012 },
	1670	+ { _MMIO(0xe55c), 0x00052051 },
	1671	+ { _MMIO(0xe65c), 0x00000008 },
	1672	+};
	1673	+
	1674	+static const struct i915_oa_reg mux_config_compute_extra[] = {
	1675	+ { _MMIO(0x9888), 0x141a001f },
	1676	+ { _MMIO(0x9888), 0x143a001f },
	1677	+ { _MMIO(0x9888), 0x145a001f },
	1678	+ { _MMIO(0x9888), 0x042d5000 },
	1679	+ { _MMIO(0x9888), 0x062d1000 },
	1680	+ { _MMIO(0x9888), 0x0e2e0094 },
	1681	+ { _MMIO(0x9888), 0x084cc000 },
	1682	+ { _MMIO(0x9888), 0x044ea000 },
	1683	+ { _MMIO(0x9888), 0x1a0f00e0 },
	1684	+ { _MMIO(0x9888), 0x1a2c0c00 },
	1685	+ { _MMIO(0x9888), 0x061a0063 },
	1686	+ { _MMIO(0x9888), 0x021a0000 },
	1687	+ { _MMIO(0x9888), 0x06142000 },
	1688	+ { _MMIO(0x9888), 0x1c150100 },
	1689	+ { _MMIO(0x9888), 0x0c168000 },
	1690	+ { _MMIO(0x9888), 0x043a3180 },
	1691	+ { _MMIO(0x9888), 0x023a0000 },
	1692	+ { _MMIO(0x9888), 0x04348000 },
	1693	+ { _MMIO(0x9888), 0x1c350040 },
	1694	+ { _MMIO(0x9888), 0x0a368000 },
	1695	+ { _MMIO(0x9888), 0x045a0063 },
	1696	+ { _MMIO(0x9888), 0x025a0000 },
	1697	+ { _MMIO(0x9888), 0x04542000 },
	1698	+ { _MMIO(0x9888), 0x1c550010 },
	1699	+ { _MMIO(0x9888), 0x08568000 },
	1700	+ { _MMIO(0x9888), 0x09938000 },
	1701	+ { _MMIO(0x9888), 0x0b938000 },
	1702	+ { _MMIO(0x9888), 0x0d938000 },
	1703	+ { _MMIO(0x9888), 0x1b904000 },
	1704	+ { _MMIO(0x9888), 0x1d904000 },
	1705	+ { _MMIO(0x9888), 0x1f904000 },
	1706	+ { _MMIO(0x9888), 0x37900000 },
	1707	+ { _MMIO(0x9888), 0x55900000 },
	1708	+ { _MMIO(0x9888), 0x45900400 },
	1709	+ { _MMIO(0x9888), 0x47900004 },
	1710	+ { _MMIO(0x9888), 0x33900000 },
	1711	+};
	1712	+
	1713	+static int
	1714	+get_compute_extra_mux_config(struct drm_i915_private *dev_priv,
	1715	+ const struct i915_oa_reg **regs,
	1716	+ int *lens)
	1717	+{
	1718	+ int n = 0;
	1719	+
	1720	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1721	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1722	+
	1723	+ regs[n] = mux_config_compute_extra;
	1724	+ lens[n] = ARRAY_SIZE(mux_config_compute_extra);
	1725	+ n++;
	1726	+
	1727	+ return n;
	1728	+}
	1729	+
	1730	+static const struct i915_oa_reg b_counter_config_test_oa[] = {
	1731	+ { _MMIO(0x2740), 0x00000000 },
	1732	+ { _MMIO(0x2744), 0x00800000 },
	1733	+ { _MMIO(0x2714), 0xf0800000 },
	1734	+ { _MMIO(0x2710), 0x00000000 },
	1735	+ { _MMIO(0x2724), 0xf0800000 },
	1736	+ { _MMIO(0x2720), 0x00000000 },
	1737	+ { _MMIO(0x2770), 0x00000004 },
	1738	+ { _MMIO(0x2774), 0x00000000 },
	1739	+ { _MMIO(0x2778), 0x00000003 },
	1740	+ { _MMIO(0x277c), 0x00000000 },
	1741	+ { _MMIO(0x2780), 0x00000007 },
	1742	+ { _MMIO(0x2784), 0x00000000 },
	1743	+ { _MMIO(0x2788), 0x00100002 },
	1744	+ { _MMIO(0x278c), 0x0000fff7 },
	1745	+ { _MMIO(0x2790), 0x00100002 },
	1746	+ { _MMIO(0x2794), 0x0000ffcf },
	1747	+ { _MMIO(0x2798), 0x00100082 },
	1748	+ { _MMIO(0x279c), 0x0000ffef },
	1749	+ { _MMIO(0x27a0), 0x001000c2 },
	1750	+ { _MMIO(0x27a4), 0x0000ffe7 },
	1751	+ { _MMIO(0x27a8), 0x00100001 },
	1752	+ { _MMIO(0x27ac), 0x0000ffe7 },
	1753	+};
	1754	+
	1755	+static const struct i915_oa_reg flex_eu_config_test_oa[] = {
	1756	+};
	1757	+
	1758	+static const struct i915_oa_reg mux_config_test_oa[] = {
	1759	+ { _MMIO(0x9888), 0x19800000 },
	1760	+ { _MMIO(0x9888), 0x07800063 },
	1761	+ { _MMIO(0x9888), 0x11800000 },
	1762	+ { _MMIO(0x9888), 0x23810008 },
	1763	+ { _MMIO(0x9888), 0x1d950400 },
	1764	+ { _MMIO(0x9888), 0x0f922000 },
	1765	+ { _MMIO(0x9888), 0x1f908000 },
	1766	+ { _MMIO(0x9888), 0x37900000 },
	1767	+ { _MMIO(0x9888), 0x55900000 },
	1768	+ { _MMIO(0x9888), 0x47900000 },
	1769	+ { _MMIO(0x9888), 0x33900000 },
	1770	+};
	1771	+
	1772	+static int
	1773	+get_test_oa_mux_config(struct drm_i915_private *dev_priv,
	1774	+ const struct i915_oa_reg **regs,
	1775	+ int *lens)
	1776	+{
	1777	+ int n = 0;
	1778	+
	1779	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1780	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1781	+
	1782	+ regs[n] = mux_config_test_oa;
	1783	+ lens[n] = ARRAY_SIZE(mux_config_test_oa);
	1784	+ n++;
	1785	+
	1786	+ return n;
	1787	+}
	1788	+
	1789	+int i915_oa_select_metric_set_bxt(struct drm_i915_private *dev_priv)
	1790	+{
	1791	+ dev_priv->perf.oa.n_mux_configs = 0;
	1792	+ dev_priv->perf.oa.b_counter_regs = NULL;
	1793	+ dev_priv->perf.oa.b_counter_regs_len = 0;
	1794	+ dev_priv->perf.oa.flex_regs = NULL;
	1795	+ dev_priv->perf.oa.flex_regs_len = 0;
	1796	+
	1797	+ switch (dev_priv->perf.oa.metrics_set) {
	1798	+ case METRIC_SET_ID_RENDER_BASIC:
	1799	+ dev_priv->perf.oa.n_mux_configs =
	1800	+ get_render_basic_mux_config(dev_priv,
	1801	+ dev_priv->perf.oa.mux_regs,
	1802	+ dev_priv->perf.oa.mux_regs_lens);
	1803	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1804	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RENDER_BASIC\" metric set\n");
	1805	+
	1806	+ /* EINVAL because *_register_sysfs already checked this
	1807	+ * and so it wouldn't have been advertised to userspace and
	1808	+ * so shouldn't have been requested
	1809	+ */
	1810	+ return -EINVAL;
	1811	+ }
	1812	+
	1813	+ dev_priv->perf.oa.b_counter_regs =
	1814	+ b_counter_config_render_basic;
	1815	+ dev_priv->perf.oa.b_counter_regs_len =
	1816	+ ARRAY_SIZE(b_counter_config_render_basic);
	1817	+
	1818	+ dev_priv->perf.oa.flex_regs =
	1819	+ flex_eu_config_render_basic;
	1820	+ dev_priv->perf.oa.flex_regs_len =
	1821	+ ARRAY_SIZE(flex_eu_config_render_basic);
	1822	+
	1823	+ return 0;
	1824	+ case METRIC_SET_ID_COMPUTE_BASIC:
	1825	+ dev_priv->perf.oa.n_mux_configs =
	1826	+ get_compute_basic_mux_config(dev_priv,
	1827	+ dev_priv->perf.oa.mux_regs,
	1828	+ dev_priv->perf.oa.mux_regs_lens);
	1829	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1830	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_BASIC\" metric set\n");
	1831	+
	1832	+ /* EINVAL because *_register_sysfs already checked this
	1833	+ * and so it wouldn't have been advertised to userspace and
	1834	+ * so shouldn't have been requested
	1835	+ */
	1836	+ return -EINVAL;
	1837	+ }
	1838	+
	1839	+ dev_priv->perf.oa.b_counter_regs =
	1840	+ b_counter_config_compute_basic;
	1841	+ dev_priv->perf.oa.b_counter_regs_len =
	1842	+ ARRAY_SIZE(b_counter_config_compute_basic);
	1843	+
	1844	+ dev_priv->perf.oa.flex_regs =
	1845	+ flex_eu_config_compute_basic;
	1846	+ dev_priv->perf.oa.flex_regs_len =
	1847	+ ARRAY_SIZE(flex_eu_config_compute_basic);
	1848	+
	1849	+ return 0;
	1850	+ case METRIC_SET_ID_RENDER_PIPE_PROFILE:
	1851	+ dev_priv->perf.oa.n_mux_configs =
	1852	+ get_render_pipe_profile_mux_config(dev_priv,
	1853	+ dev_priv->perf.oa.mux_regs,
	1854	+ dev_priv->perf.oa.mux_regs_lens);
	1855	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1856	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RENDER_PIPE_PROFILE\" metric set\n");
	1857	+
	1858	+ /* EINVAL because *_register_sysfs already checked this
	1859	+ * and so it wouldn't have been advertised to userspace and
	1860	+ * so shouldn't have been requested
	1861	+ */
	1862	+ return -EINVAL;
	1863	+ }
	1864	+
	1865	+ dev_priv->perf.oa.b_counter_regs =
	1866	+ b_counter_config_render_pipe_profile;
	1867	+ dev_priv->perf.oa.b_counter_regs_len =
	1868	+ ARRAY_SIZE(b_counter_config_render_pipe_profile);
	1869	+
	1870	+ dev_priv->perf.oa.flex_regs =
	1871	+ flex_eu_config_render_pipe_profile;
	1872	+ dev_priv->perf.oa.flex_regs_len =
	1873	+ ARRAY_SIZE(flex_eu_config_render_pipe_profile);
	1874	+
	1875	+ return 0;
	1876	+ case METRIC_SET_ID_MEMORY_READS:
	1877	+ dev_priv->perf.oa.n_mux_configs =
	1878	+ get_memory_reads_mux_config(dev_priv,
	1879	+ dev_priv->perf.oa.mux_regs,
	1880	+ dev_priv->perf.oa.mux_regs_lens);
	1881	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1882	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"MEMORY_READS\" metric set\n");
	1883	+
	1884	+ /* EINVAL because *_register_sysfs already checked this
	1885	+ * and so it wouldn't have been advertised to userspace and
	1886	+ * so shouldn't have been requested
	1887	+ */
	1888	+ return -EINVAL;
	1889	+ }
	1890	+
	1891	+ dev_priv->perf.oa.b_counter_regs =
	1892	+ b_counter_config_memory_reads;
	1893	+ dev_priv->perf.oa.b_counter_regs_len =
	1894	+ ARRAY_SIZE(b_counter_config_memory_reads);
	1895	+
	1896	+ dev_priv->perf.oa.flex_regs =
	1897	+ flex_eu_config_memory_reads;
	1898	+ dev_priv->perf.oa.flex_regs_len =
	1899	+ ARRAY_SIZE(flex_eu_config_memory_reads);
	1900	+
	1901	+ return 0;
	1902	+ case METRIC_SET_ID_MEMORY_WRITES:
	1903	+ dev_priv->perf.oa.n_mux_configs =
	1904	+ get_memory_writes_mux_config(dev_priv,
	1905	+ dev_priv->perf.oa.mux_regs,
	1906	+ dev_priv->perf.oa.mux_regs_lens);
	1907	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1908	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"MEMORY_WRITES\" metric set\n");
	1909	+
	1910	+ /* EINVAL because *_register_sysfs already checked this
	1911	+ * and so it wouldn't have been advertised to userspace and
	1912	+ * so shouldn't have been requested
	1913	+ */
	1914	+ return -EINVAL;
	1915	+ }
	1916	+
	1917	+ dev_priv->perf.oa.b_counter_regs =
	1918	+ b_counter_config_memory_writes;
	1919	+ dev_priv->perf.oa.b_counter_regs_len =
	1920	+ ARRAY_SIZE(b_counter_config_memory_writes);
	1921	+
	1922	+ dev_priv->perf.oa.flex_regs =
	1923	+ flex_eu_config_memory_writes;
	1924	+ dev_priv->perf.oa.flex_regs_len =
	1925	+ ARRAY_SIZE(flex_eu_config_memory_writes);
	1926	+
	1927	+ return 0;
	1928	+ case METRIC_SET_ID_COMPUTE_EXTENDED:
	1929	+ dev_priv->perf.oa.n_mux_configs =
	1930	+ get_compute_extended_mux_config(dev_priv,
	1931	+ dev_priv->perf.oa.mux_regs,
	1932	+ dev_priv->perf.oa.mux_regs_lens);
	1933	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1934	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_EXTENDED\" metric set\n");
	1935	+
	1936	+ /* EINVAL because *_register_sysfs already checked this
	1937	+ * and so it wouldn't have been advertised to userspace and
	1938	+ * so shouldn't have been requested
	1939	+ */
	1940	+ return -EINVAL;
	1941	+ }
	1942	+
	1943	+ dev_priv->perf.oa.b_counter_regs =
	1944	+ b_counter_config_compute_extended;
	1945	+ dev_priv->perf.oa.b_counter_regs_len =
	1946	+ ARRAY_SIZE(b_counter_config_compute_extended);
	1947	+
	1948	+ dev_priv->perf.oa.flex_regs =
	1949	+ flex_eu_config_compute_extended;
	1950	+ dev_priv->perf.oa.flex_regs_len =
	1951	+ ARRAY_SIZE(flex_eu_config_compute_extended);
	1952	+
	1953	+ return 0;
	1954	+ case METRIC_SET_ID_COMPUTE_L3_CACHE:
	1955	+ dev_priv->perf.oa.n_mux_configs =
	1956	+ get_compute_l3_cache_mux_config(dev_priv,
	1957	+ dev_priv->perf.oa.mux_regs,
	1958	+ dev_priv->perf.oa.mux_regs_lens);
	1959	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1960	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_L3_CACHE\" metric set\n");
	1961	+
	1962	+ /* EINVAL because *_register_sysfs already checked this
	1963	+ * and so it wouldn't have been advertised to userspace and
	1964	+ * so shouldn't have been requested
	1965	+ */
	1966	+ return -EINVAL;
	1967	+ }
	1968	+
	1969	+ dev_priv->perf.oa.b_counter_regs =
	1970	+ b_counter_config_compute_l3_cache;
	1971	+ dev_priv->perf.oa.b_counter_regs_len =
	1972	+ ARRAY_SIZE(b_counter_config_compute_l3_cache);
	1973	+
	1974	+ dev_priv->perf.oa.flex_regs =
	1975	+ flex_eu_config_compute_l3_cache;
	1976	+ dev_priv->perf.oa.flex_regs_len =
	1977	+ ARRAY_SIZE(flex_eu_config_compute_l3_cache);
	1978	+
	1979	+ return 0;
	1980	+ case METRIC_SET_ID_HDC_AND_SF:
	1981	+ dev_priv->perf.oa.n_mux_configs =
	1982	+ get_hdc_and_sf_mux_config(dev_priv,
	1983	+ dev_priv->perf.oa.mux_regs,
	1984	+ dev_priv->perf.oa.mux_regs_lens);
	1985	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1986	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"HDC_AND_SF\" metric set\n");
	1987	+
	1988	+ /* EINVAL because *_register_sysfs already checked this
	1989	+ * and so it wouldn't have been advertised to userspace and
	1990	+ * so shouldn't have been requested
	1991	+ */
	1992	+ return -EINVAL;
	1993	+ }
	1994	+
	1995	+ dev_priv->perf.oa.b_counter_regs =
	1996	+ b_counter_config_hdc_and_sf;
	1997	+ dev_priv->perf.oa.b_counter_regs_len =
	1998	+ ARRAY_SIZE(b_counter_config_hdc_and_sf);
	1999	+
	2000	+ dev_priv->perf.oa.flex_regs =
	2001	+ flex_eu_config_hdc_and_sf;
	2002	+ dev_priv->perf.oa.flex_regs_len =
	2003	+ ARRAY_SIZE(flex_eu_config_hdc_and_sf);
	2004	+
	2005	+ return 0;
	2006	+ case METRIC_SET_ID_L3_1:
	2007	+ dev_priv->perf.oa.n_mux_configs =
	2008	+ get_l3_1_mux_config(dev_priv,
	2009	+ dev_priv->perf.oa.mux_regs,
	2010	+ dev_priv->perf.oa.mux_regs_lens);
	2011	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2012	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_1\" metric set\n");
	2013	+
	2014	+ /* EINVAL because *_register_sysfs already checked this
	2015	+ * and so it wouldn't have been advertised to userspace and
	2016	+ * so shouldn't have been requested
	2017	+ */
	2018	+ return -EINVAL;
	2019	+ }
	2020	+
	2021	+ dev_priv->perf.oa.b_counter_regs =
	2022	+ b_counter_config_l3_1;
	2023	+ dev_priv->perf.oa.b_counter_regs_len =
	2024	+ ARRAY_SIZE(b_counter_config_l3_1);
	2025	+
	2026	+ dev_priv->perf.oa.flex_regs =
	2027	+ flex_eu_config_l3_1;
	2028	+ dev_priv->perf.oa.flex_regs_len =
	2029	+ ARRAY_SIZE(flex_eu_config_l3_1);
	2030	+
	2031	+ return 0;
	2032	+ case METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND:
	2033	+ dev_priv->perf.oa.n_mux_configs =
	2034	+ get_rasterizer_and_pixel_backend_mux_config(dev_priv,
	2035	+ dev_priv->perf.oa.mux_regs,
	2036	+ dev_priv->perf.oa.mux_regs_lens);
	2037	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2038	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RASTERIZER_AND_PIXEL_BACKEND\" metric set\n");
	2039	+
	2040	+ /* EINVAL because *_register_sysfs already checked this
	2041	+ * and so it wouldn't have been advertised to userspace and
	2042	+ * so shouldn't have been requested
	2043	+ */
	2044	+ return -EINVAL;
	2045	+ }
	2046	+
	2047	+ dev_priv->perf.oa.b_counter_regs =
	2048	+ b_counter_config_rasterizer_and_pixel_backend;
	2049	+ dev_priv->perf.oa.b_counter_regs_len =
	2050	+ ARRAY_SIZE(b_counter_config_rasterizer_and_pixel_backend);
	2051	+
	2052	+ dev_priv->perf.oa.flex_regs =
	2053	+ flex_eu_config_rasterizer_and_pixel_backend;
	2054	+ dev_priv->perf.oa.flex_regs_len =
	2055	+ ARRAY_SIZE(flex_eu_config_rasterizer_and_pixel_backend);
	2056	+
	2057	+ return 0;
	2058	+ case METRIC_SET_ID_SAMPLER:
	2059	+ dev_priv->perf.oa.n_mux_configs =
	2060	+ get_sampler_mux_config(dev_priv,
	2061	+ dev_priv->perf.oa.mux_regs,
	2062	+ dev_priv->perf.oa.mux_regs_lens);
	2063	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2064	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"SAMPLER\" metric set\n");
	2065	+
	2066	+ /* EINVAL because *_register_sysfs already checked this
	2067	+ * and so it wouldn't have been advertised to userspace and
	2068	+ * so shouldn't have been requested
	2069	+ */
	2070	+ return -EINVAL;
	2071	+ }
	2072	+
	2073	+ dev_priv->perf.oa.b_counter_regs =
	2074	+ b_counter_config_sampler;
	2075	+ dev_priv->perf.oa.b_counter_regs_len =
	2076	+ ARRAY_SIZE(b_counter_config_sampler);
	2077	+
	2078	+ dev_priv->perf.oa.flex_regs =
	2079	+ flex_eu_config_sampler;
	2080	+ dev_priv->perf.oa.flex_regs_len =
	2081	+ ARRAY_SIZE(flex_eu_config_sampler);
	2082	+
	2083	+ return 0;
	2084	+ case METRIC_SET_ID_TDL_1:
	2085	+ dev_priv->perf.oa.n_mux_configs =
	2086	+ get_tdl_1_mux_config(dev_priv,
	2087	+ dev_priv->perf.oa.mux_regs,
	2088	+ dev_priv->perf.oa.mux_regs_lens);
	2089	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2090	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TDL_1\" metric set\n");
	2091	+
	2092	+ /* EINVAL because *_register_sysfs already checked this
	2093	+ * and so it wouldn't have been advertised to userspace and
	2094	+ * so shouldn't have been requested
	2095	+ */
	2096	+ return -EINVAL;
	2097	+ }
	2098	+
	2099	+ dev_priv->perf.oa.b_counter_regs =
	2100	+ b_counter_config_tdl_1;
	2101	+ dev_priv->perf.oa.b_counter_regs_len =
	2102	+ ARRAY_SIZE(b_counter_config_tdl_1);
	2103	+
	2104	+ dev_priv->perf.oa.flex_regs =
	2105	+ flex_eu_config_tdl_1;
	2106	+ dev_priv->perf.oa.flex_regs_len =
	2107	+ ARRAY_SIZE(flex_eu_config_tdl_1);
	2108	+
	2109	+ return 0;
	2110	+ case METRIC_SET_ID_TDL_2:
	2111	+ dev_priv->perf.oa.n_mux_configs =
	2112	+ get_tdl_2_mux_config(dev_priv,
	2113	+ dev_priv->perf.oa.mux_regs,
	2114	+ dev_priv->perf.oa.mux_regs_lens);
	2115	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2116	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TDL_2\" metric set\n");
	2117	+
	2118	+ /* EINVAL because *_register_sysfs already checked this
	2119	+ * and so it wouldn't have been advertised to userspace and
	2120	+ * so shouldn't have been requested
	2121	+ */
	2122	+ return -EINVAL;
	2123	+ }
	2124	+
	2125	+ dev_priv->perf.oa.b_counter_regs =
	2126	+ b_counter_config_tdl_2;
	2127	+ dev_priv->perf.oa.b_counter_regs_len =
	2128	+ ARRAY_SIZE(b_counter_config_tdl_2);
	2129	+
	2130	+ dev_priv->perf.oa.flex_regs =
	2131	+ flex_eu_config_tdl_2;
	2132	+ dev_priv->perf.oa.flex_regs_len =
	2133	+ ARRAY_SIZE(flex_eu_config_tdl_2);
	2134	+
	2135	+ return 0;
	2136	+ case METRIC_SET_ID_COMPUTE_EXTRA:
	2137	+ dev_priv->perf.oa.n_mux_configs =
	2138	+ get_compute_extra_mux_config(dev_priv,
	2139	+ dev_priv->perf.oa.mux_regs,
	2140	+ dev_priv->perf.oa.mux_regs_lens);
	2141	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2142	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_EXTRA\" metric set\n");
	2143	+
	2144	+ /* EINVAL because *_register_sysfs already checked this
	2145	+ * and so it wouldn't have been advertised to userspace and
	2146	+ * so shouldn't have been requested
	2147	+ */
	2148	+ return -EINVAL;
	2149	+ }
	2150	+
	2151	+ dev_priv->perf.oa.b_counter_regs =
	2152	+ b_counter_config_compute_extra;
	2153	+ dev_priv->perf.oa.b_counter_regs_len =
	2154	+ ARRAY_SIZE(b_counter_config_compute_extra);
	2155	+
	2156	+ dev_priv->perf.oa.flex_regs =
	2157	+ flex_eu_config_compute_extra;
	2158	+ dev_priv->perf.oa.flex_regs_len =
	2159	+ ARRAY_SIZE(flex_eu_config_compute_extra);
	2160	+
	2161	+ return 0;
	2162	+ case METRIC_SET_ID_TEST_OA:
	2163	+ dev_priv->perf.oa.n_mux_configs =
	2164	+ get_test_oa_mux_config(dev_priv,
	2165	+ dev_priv->perf.oa.mux_regs,
	2166	+ dev_priv->perf.oa.mux_regs_lens);
	2167	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2168	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TEST_OA\" metric set\n");
	2169	+
	2170	+ /* EINVAL because *_register_sysfs already checked this
	2171	+ * and so it wouldn't have been advertised to userspace and
	2172	+ * so shouldn't have been requested
	2173	+ */
	2174	+ return -EINVAL;
	2175	+ }
	2176	+
	2177	+ dev_priv->perf.oa.b_counter_regs =
	2178	+ b_counter_config_test_oa;
	2179	+ dev_priv->perf.oa.b_counter_regs_len =
	2180	+ ARRAY_SIZE(b_counter_config_test_oa);
	2181	+
	2182	+ dev_priv->perf.oa.flex_regs =
	2183	+ flex_eu_config_test_oa;
	2184	+ dev_priv->perf.oa.flex_regs_len =
	2185	+ ARRAY_SIZE(flex_eu_config_test_oa);
	2186	+
	2187	+ return 0;
	2188	+ default:
	2189	+ return -ENODEV;
	2190	+ }
	2191	+}
	2192	+
	2193	+static ssize_t
	2194	+show_render_basic_id(struct device kdev, struct device_attribute attr, char *buf)
	2195	+{
	2196	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RENDER_BASIC);
	2197	+}
	2198	+
	2199	+static struct device_attribute dev_attr_render_basic_id = {
	2200	+ .attr = { .name = "id", .mode = 0444 },
	2201	+ .show = show_render_basic_id,
	2202	+ .store = NULL,
	2203	+};
	2204	+
	2205	+static struct attribute *attrs_render_basic[] = {
	2206	+ &dev_attr_render_basic_id.attr,
	2207	+ NULL,
	2208	+};
	2209	+
	2210	+static struct attribute_group group_render_basic = {
	2211	+ .name = "22b9519a-e9ba-4c41-8b54-f4f8ca14fa0a",
	2212	+ .attrs = attrs_render_basic,
	2213	+};
	2214	+
	2215	+static ssize_t
	2216	+show_compute_basic_id(struct device kdev, struct device_attribute attr, char *buf)
	2217	+{
	2218	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_BASIC);
	2219	+}
	2220	+
	2221	+static struct device_attribute dev_attr_compute_basic_id = {
	2222	+ .attr = { .name = "id", .mode = 0444 },
	2223	+ .show = show_compute_basic_id,
	2224	+ .store = NULL,
	2225	+};
	2226	+
	2227	+static struct attribute *attrs_compute_basic[] = {
	2228	+ &dev_attr_compute_basic_id.attr,
	2229	+ NULL,
	2230	+};
	2231	+
	2232	+static struct attribute_group group_compute_basic = {
	2233	+ .name = "012d72cf-82a9-4d25-8ddf-74076fd30797",
	2234	+ .attrs = attrs_compute_basic,
	2235	+};
	2236	+
	2237	+static ssize_t
	2238	+show_render_pipe_profile_id(struct device kdev, struct device_attribute attr, char *buf)
	2239	+{
	2240	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RENDER_PIPE_PROFILE);
	2241	+}
	2242	+
	2243	+static struct device_attribute dev_attr_render_pipe_profile_id = {
	2244	+ .attr = { .name = "id", .mode = 0444 },
	2245	+ .show = show_render_pipe_profile_id,
	2246	+ .store = NULL,
	2247	+};
	2248	+
	2249	+static struct attribute *attrs_render_pipe_profile[] = {
	2250	+ &dev_attr_render_pipe_profile_id.attr,
	2251	+ NULL,
	2252	+};
	2253	+
	2254	+static struct attribute_group group_render_pipe_profile = {
	2255	+ .name = "ce416533-e49e-4211-80af-ec513590a914",
	2256	+ .attrs = attrs_render_pipe_profile,
	2257	+};
	2258	+
	2259	+static ssize_t
	2260	+show_memory_reads_id(struct device kdev, struct device_attribute attr, char *buf)
	2261	+{
	2262	+ return sprintf(buf, "%d\n", METRIC_SET_ID_MEMORY_READS);
	2263	+}
	2264	+
	2265	+static struct device_attribute dev_attr_memory_reads_id = {
	2266	+ .attr = { .name = "id", .mode = 0444 },
	2267	+ .show = show_memory_reads_id,
	2268	+ .store = NULL,
	2269	+};
	2270	+
	2271	+static struct attribute *attrs_memory_reads[] = {
	2272	+ &dev_attr_memory_reads_id.attr,
	2273	+ NULL,
	2274	+};
	2275	+
	2276	+static struct attribute_group group_memory_reads = {
	2277	+ .name = "398e2452-18d7-42d0-b241-e4d0a9148ada",
	2278	+ .attrs = attrs_memory_reads,
	2279	+};
	2280	+
	2281	+static ssize_t
	2282	+show_memory_writes_id(struct device kdev, struct device_attribute attr, char *buf)
	2283	+{
	2284	+ return sprintf(buf, "%d\n", METRIC_SET_ID_MEMORY_WRITES);
	2285	+}
	2286	+
	2287	+static struct device_attribute dev_attr_memory_writes_id = {
	2288	+ .attr = { .name = "id", .mode = 0444 },
	2289	+ .show = show_memory_writes_id,
	2290	+ .store = NULL,
	2291	+};
	2292	+
	2293	+static struct attribute *attrs_memory_writes[] = {
	2294	+ &dev_attr_memory_writes_id.attr,
	2295	+ NULL,
	2296	+};
	2297	+
	2298	+static struct attribute_group group_memory_writes = {
	2299	+ .name = "d324a0d6-7269-4847-a5c2-6f71ddc7fed5",
	2300	+ .attrs = attrs_memory_writes,
	2301	+};
	2302	+
	2303	+static ssize_t
	2304	+show_compute_extended_id(struct device kdev, struct device_attribute attr, char *buf)
	2305	+{
	2306	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_EXTENDED);
	2307	+}
	2308	+
	2309	+static struct device_attribute dev_attr_compute_extended_id = {
	2310	+ .attr = { .name = "id", .mode = 0444 },
	2311	+ .show = show_compute_extended_id,
	2312	+ .store = NULL,
	2313	+};
	2314	+
	2315	+static struct attribute *attrs_compute_extended[] = {
	2316	+ &dev_attr_compute_extended_id.attr,
	2317	+ NULL,
	2318	+};
	2319	+
	2320	+static struct attribute_group group_compute_extended = {
	2321	+ .name = "caf3596a-7bb1-4dec-b3b3-2a080d283b49",
	2322	+ .attrs = attrs_compute_extended,
	2323	+};
	2324	+
	2325	+static ssize_t
	2326	+show_compute_l3_cache_id(struct device kdev, struct device_attribute attr, char *buf)
	2327	+{
	2328	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_L3_CACHE);
	2329	+}
	2330	+
	2331	+static struct device_attribute dev_attr_compute_l3_cache_id = {
	2332	+ .attr = { .name = "id", .mode = 0444 },
	2333	+ .show = show_compute_l3_cache_id,
	2334	+ .store = NULL,
	2335	+};
	2336	+
	2337	+static struct attribute *attrs_compute_l3_cache[] = {
	2338	+ &dev_attr_compute_l3_cache_id.attr,
	2339	+ NULL,
	2340	+};
	2341	+
	2342	+static struct attribute_group group_compute_l3_cache = {
	2343	+ .name = "49b956e2-d5b9-47e0-9d8a-cee5e8cec527",
	2344	+ .attrs = attrs_compute_l3_cache,
	2345	+};
	2346	+
	2347	+static ssize_t
	2348	+show_hdc_and_sf_id(struct device kdev, struct device_attribute attr, char *buf)
	2349	+{
	2350	+ return sprintf(buf, "%d\n", METRIC_SET_ID_HDC_AND_SF);
	2351	+}
	2352	+
	2353	+static struct device_attribute dev_attr_hdc_and_sf_id = {
	2354	+ .attr = { .name = "id", .mode = 0444 },
	2355	+ .show = show_hdc_and_sf_id,
	2356	+ .store = NULL,
	2357	+};
	2358	+
	2359	+static struct attribute *attrs_hdc_and_sf[] = {
	2360	+ &dev_attr_hdc_and_sf_id.attr,
	2361	+ NULL,
	2362	+};
	2363	+
	2364	+static struct attribute_group group_hdc_and_sf = {
	2365	+ .name = "f64ef50a-bdba-4b35-8f09-203c13d8ee5a",
	2366	+ .attrs = attrs_hdc_and_sf,
	2367	+};
	2368	+
	2369	+static ssize_t
	2370	+show_l3_1_id(struct device kdev, struct device_attribute attr, char *buf)
	2371	+{
	2372	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_1);
	2373	+}
	2374	+
	2375	+static struct device_attribute dev_attr_l3_1_id = {
	2376	+ .attr = { .name = "id", .mode = 0444 },
	2377	+ .show = show_l3_1_id,
	2378	+ .store = NULL,
	2379	+};
	2380	+
	2381	+static struct attribute *attrs_l3_1[] = {
	2382	+ &dev_attr_l3_1_id.attr,
	2383	+ NULL,
	2384	+};
	2385	+
	2386	+static struct attribute_group group_l3_1 = {
	2387	+ .name = "00ad5a41-7eab-4f7a-9103-49d411c67219",
	2388	+ .attrs = attrs_l3_1,
	2389	+};
	2390	+
	2391	+static ssize_t
	2392	+show_rasterizer_and_pixel_backend_id(struct device kdev, struct device_attribute attr, char *buf)
	2393	+{
	2394	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND);
	2395	+}
	2396	+
	2397	+static struct device_attribute dev_attr_rasterizer_and_pixel_backend_id = {
	2398	+ .attr = { .name = "id", .mode = 0444 },
	2399	+ .show = show_rasterizer_and_pixel_backend_id,
	2400	+ .store = NULL,
	2401	+};
	2402	+
	2403	+static struct attribute *attrs_rasterizer_and_pixel_backend[] = {
	2404	+ &dev_attr_rasterizer_and_pixel_backend_id.attr,
	2405	+ NULL,
	2406	+};
	2407	+
	2408	+static struct attribute_group group_rasterizer_and_pixel_backend = {
	2409	+ .name = "46dc44ca-491c-4cc1-a951-e7b3e62bf02b",
	2410	+ .attrs = attrs_rasterizer_and_pixel_backend,
	2411	+};
	2412	+
	2413	+static ssize_t
	2414	+show_sampler_id(struct device kdev, struct device_attribute attr, char *buf)
	2415	+{
	2416	+ return sprintf(buf, "%d\n", METRIC_SET_ID_SAMPLER);
	2417	+}
	2418	+
	2419	+static struct device_attribute dev_attr_sampler_id = {
	2420	+ .attr = { .name = "id", .mode = 0444 },
	2421	+ .show = show_sampler_id,
	2422	+ .store = NULL,
	2423	+};
	2424	+
	2425	+static struct attribute *attrs_sampler[] = {
	2426	+ &dev_attr_sampler_id.attr,
	2427	+ NULL,
	2428	+};
	2429	+
	2430	+static struct attribute_group group_sampler = {
	2431	+ .name = "8364e2a8-af63-40af-b0d5-42969a255654",
	2432	+ .attrs = attrs_sampler,
	2433	+};
	2434	+
	2435	+static ssize_t
	2436	+show_tdl_1_id(struct device kdev, struct device_attribute attr, char *buf)
	2437	+{
	2438	+ return sprintf(buf, "%d\n", METRIC_SET_ID_TDL_1);
	2439	+}
	2440	+
	2441	+static struct device_attribute dev_attr_tdl_1_id = {
	2442	+ .attr = { .name = "id", .mode = 0444 },
	2443	+ .show = show_tdl_1_id,
	2444	+ .store = NULL,
	2445	+};
	2446	+
	2447	+static struct attribute *attrs_tdl_1[] = {
	2448	+ &dev_attr_tdl_1_id.attr,
	2449	+ NULL,
	2450	+};
	2451	+
	2452	+static struct attribute_group group_tdl_1 = {
	2453	+ .name = "175c8092-cb25-4d1e-8dc7-b4fdd39e2d92",
	2454	+ .attrs = attrs_tdl_1,
	2455	+};
	2456	+
	2457	+static ssize_t
	2458	+show_tdl_2_id(struct device kdev, struct device_attribute attr, char *buf)
	2459	+{
	2460	+ return sprintf(buf, "%d\n", METRIC_SET_ID_TDL_2);
	2461	+}
	2462	+
	2463	+static struct device_attribute dev_attr_tdl_2_id = {
	2464	+ .attr = { .name = "id", .mode = 0444 },
	2465	+ .show = show_tdl_2_id,
	2466	+ .store = NULL,
	2467	+};
	2468	+
	2469	+static struct attribute *attrs_tdl_2[] = {
	2470	+ &dev_attr_tdl_2_id.attr,
	2471	+ NULL,
	2472	+};
	2473	+
	2474	+static struct attribute_group group_tdl_2 = {
	2475	+ .name = "d260f03f-b34d-4b49-a44e-436819117332",
	2476	+ .attrs = attrs_tdl_2,
	2477	+};
	2478	+
	2479	+static ssize_t
	2480	+show_compute_extra_id(struct device kdev, struct device_attribute attr, char *buf)
	2481	+{
	2482	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_EXTRA);
	2483	+}
	2484	+
	2485	+static struct device_attribute dev_attr_compute_extra_id = {
	2486	+ .attr = { .name = "id", .mode = 0444 },
	2487	+ .show = show_compute_extra_id,
	2488	+ .store = NULL,
	2489	+};
	2490	+
	2491	+static struct attribute *attrs_compute_extra[] = {
	2492	+ &dev_attr_compute_extra_id.attr,
	2493	+ NULL,
	2494	+};
	2495	+
	2496	+static struct attribute_group group_compute_extra = {
	2497	+ .name = "fa6ecf21-2cb8-4d0b-9308-6e4a7b4ca87a",
	2498	+ .attrs = attrs_compute_extra,
	2499	+};
	2500	+
	2501	+static ssize_t
	2502	+show_test_oa_id(struct device kdev, struct device_attribute attr, char *buf)
	2503	+{
	2504	+ return sprintf(buf, "%d\n", METRIC_SET_ID_TEST_OA);
	2505	+}
	2506	+
	2507	+static struct device_attribute dev_attr_test_oa_id = {
	2508	+ .attr = { .name = "id", .mode = 0444 },
	2509	+ .show = show_test_oa_id,
	2510	+ .store = NULL,
	2511	+};
	2512	+
	2513	+static struct attribute *attrs_test_oa[] = {
	2514	+ &dev_attr_test_oa_id.attr,
	2515	+ NULL,
	2516	+};
	2517	+
	2518	+static struct attribute_group group_test_oa = {
	2519	+ .name = "5ee72f5c-092f-421e-8b70-225f7c3e9612",
	2520	+ .attrs = attrs_test_oa,
	2521	+};
	2522	+
	2523	+int
	2524	+i915_perf_register_sysfs_bxt(struct drm_i915_private *dev_priv)
	2525	+{
	2526	+ const struct i915_oa_reg *mux_regs[ARRAY_SIZE(dev_priv->perf.oa.mux_regs)];
	2527	+ int mux_lens[ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens)];
	2528	+ int ret = 0;
	2529	+
	2530	+ if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens)) {
	2531	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_render_basic);
	2532	+ if (ret)
	2533	+ goto error_render_basic;
	2534	+ }
	2535	+ if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens)) {
	2536	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
	2537	+ if (ret)
	2538	+ goto error_compute_basic;
	2539	+ }
	2540	+ if (get_render_pipe_profile_mux_config(dev_priv, mux_regs, mux_lens)) {
	2541	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_render_pipe_profile);
	2542	+ if (ret)
	2543	+ goto error_render_pipe_profile;
	2544	+ }
	2545	+ if (get_memory_reads_mux_config(dev_priv, mux_regs, mux_lens)) {
	2546	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_memory_reads);
	2547	+ if (ret)
	2548	+ goto error_memory_reads;
	2549	+ }
	2550	+ if (get_memory_writes_mux_config(dev_priv, mux_regs, mux_lens)) {
	2551	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_memory_writes);
	2552	+ if (ret)
	2553	+ goto error_memory_writes;
	2554	+ }
	2555	+ if (get_compute_extended_mux_config(dev_priv, mux_regs, mux_lens)) {
	2556	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_extended);
	2557	+ if (ret)
	2558	+ goto error_compute_extended;
	2559	+ }
	2560	+ if (get_compute_l3_cache_mux_config(dev_priv, mux_regs, mux_lens)) {
	2561	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_l3_cache);
	2562	+ if (ret)
	2563	+ goto error_compute_l3_cache;
	2564	+ }
	2565	+ if (get_hdc_and_sf_mux_config(dev_priv, mux_regs, mux_lens)) {
	2566	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_hdc_and_sf);
	2567	+ if (ret)
	2568	+ goto error_hdc_and_sf;
	2569	+ }
	2570	+ if (get_l3_1_mux_config(dev_priv, mux_regs, mux_lens)) {
	2571	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_l3_1);
	2572	+ if (ret)
	2573	+ goto error_l3_1;
	2574	+ }
	2575	+ if (get_rasterizer_and_pixel_backend_mux_config(dev_priv, mux_regs, mux_lens)) {
	2576	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_rasterizer_and_pixel_backend);
	2577	+ if (ret)
	2578	+ goto error_rasterizer_and_pixel_backend;
	2579	+ }
	2580	+ if (get_sampler_mux_config(dev_priv, mux_regs, mux_lens)) {
	2581	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_sampler);
	2582	+ if (ret)
	2583	+ goto error_sampler;
	2584	+ }
	2585	+ if (get_tdl_1_mux_config(dev_priv, mux_regs, mux_lens)) {
	2586	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_tdl_1);
	2587	+ if (ret)
	2588	+ goto error_tdl_1;
	2589	+ }
	2590	+ if (get_tdl_2_mux_config(dev_priv, mux_regs, mux_lens)) {
	2591	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_tdl_2);
	2592	+ if (ret)
	2593	+ goto error_tdl_2;
	2594	+ }
	2595	+ if (get_compute_extra_mux_config(dev_priv, mux_regs, mux_lens)) {
	2596	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_extra);
	2597	+ if (ret)
	2598	+ goto error_compute_extra;
	2599	+ }
	2600	+ if (get_test_oa_mux_config(dev_priv, mux_regs, mux_lens)) {
	2601	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_test_oa);
	2602	+ if (ret)
	2603	+ goto error_test_oa;
	2604	+ }
	2605	+
	2606	+ return 0;
	2607	+
	2608	+error_test_oa:
	2609	+ if (get_compute_extra_mux_config(dev_priv, mux_regs, mux_lens))
	2610	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_extra);
	2611	+error_compute_extra:
	2612	+ if (get_tdl_2_mux_config(dev_priv, mux_regs, mux_lens))
	2613	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_tdl_2);
	2614	+error_tdl_2:
	2615	+ if (get_tdl_1_mux_config(dev_priv, mux_regs, mux_lens))
	2616	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_tdl_1);
	2617	+error_tdl_1:
	2618	+ if (get_sampler_mux_config(dev_priv, mux_regs, mux_lens))
	2619	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_sampler);
	2620	+error_sampler:
	2621	+ if (get_rasterizer_and_pixel_backend_mux_config(dev_priv, mux_regs, mux_lens))
	2622	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_rasterizer_and_pixel_backend);
	2623	+error_rasterizer_and_pixel_backend:
	2624	+ if (get_l3_1_mux_config(dev_priv, mux_regs, mux_lens))
	2625	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_1);
	2626	+error_l3_1:
	2627	+ if (get_hdc_and_sf_mux_config(dev_priv, mux_regs, mux_lens))
	2628	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_hdc_and_sf);
	2629	+error_hdc_and_sf:
	2630	+ if (get_compute_l3_cache_mux_config(dev_priv, mux_regs, mux_lens))
	2631	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_l3_cache);
	2632	+error_compute_l3_cache:
	2633	+ if (get_compute_extended_mux_config(dev_priv, mux_regs, mux_lens))
	2634	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_extended);
	2635	+error_compute_extended:
	2636	+ if (get_memory_writes_mux_config(dev_priv, mux_regs, mux_lens))
	2637	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_memory_writes);
	2638	+error_memory_writes:
	2639	+ if (get_memory_reads_mux_config(dev_priv, mux_regs, mux_lens))
	2640	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_memory_reads);
	2641	+error_memory_reads:
	2642	+ if (get_render_pipe_profile_mux_config(dev_priv, mux_regs, mux_lens))
	2643	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_pipe_profile);
	2644	+error_render_pipe_profile:
	2645	+ if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens))
	2646	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
	2647	+error_compute_basic:
	2648	+ if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens))
	2649	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_basic);
	2650	+error_render_basic:
	2651	+ return ret;
	2652	+}
	2653	+
	2654	+void
	2655	+i915_perf_unregister_sysfs_bxt(struct drm_i915_private *dev_priv)
	2656	+{
	2657	+ const struct i915_oa_reg *mux_regs[ARRAY_SIZE(dev_priv->perf.oa.mux_regs)];
	2658	+ int mux_lens[ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens)];
	2659	+
	2660	+ if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens))
	2661	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_basic);
	2662	+ if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens))
	2663	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
	2664	+ if (get_render_pipe_profile_mux_config(dev_priv, mux_regs, mux_lens))
	2665	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_pipe_profile);
	2666	+ if (get_memory_reads_mux_config(dev_priv, mux_regs, mux_lens))
	2667	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_memory_reads);
	2668	+ if (get_memory_writes_mux_config(dev_priv, mux_regs, mux_lens))
	2669	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_memory_writes);
	2670	+ if (get_compute_extended_mux_config(dev_priv, mux_regs, mux_lens))
	2671	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_extended);
	2672	+ if (get_compute_l3_cache_mux_config(dev_priv, mux_regs, mux_lens))
	2673	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_l3_cache);
	2674	+ if (get_hdc_and_sf_mux_config(dev_priv, mux_regs, mux_lens))
	2675	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_hdc_and_sf);
	2676	+ if (get_l3_1_mux_config(dev_priv, mux_regs, mux_lens))
	2677	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_1);
	2678	+ if (get_rasterizer_and_pixel_backend_mux_config(dev_priv, mux_regs, mux_lens))
	2679	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_rasterizer_and_pixel_backend);
	2680	+ if (get_sampler_mux_config(dev_priv, mux_regs, mux_lens))
	2681	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_sampler);
	2682	+ if (get_tdl_1_mux_config(dev_priv, mux_regs, mux_lens))
	2683	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_tdl_1);
	2684	+ if (get_tdl_2_mux_config(dev_priv, mux_regs, mux_lens))
	2685	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_tdl_2);
	2686	+ if (get_compute_extra_mux_config(dev_priv, mux_regs, mux_lens))
	2687	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_extra);
	2688	+ if (get_test_oa_mux_config(dev_priv, mux_regs, mux_lens))
	2689	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_test_oa);
	2690	+}

drivers/gpu/drm/i915/i915_oa_bxt.h+40−0

@@ -0,0 +1,40 @@
	1	+/*
	2	+ * Autogenerated file by GPU Top : https://github.com/rib/gputop
	3	+ * DO NOT EDIT manually!
	4	+ *
	5	+ *
	6	+ * Copyright (c) 2015 Intel Corporation
	7	+ *
	8	+ * Permission is hereby granted, free of charge, to any person obtaining a
	9	+ * copy of this software and associated documentation files (the "Software"),
	10	+ * to deal in the Software without restriction, including without limitation
	11	+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
	12	+ * and/or sell copies of the Software, and to permit persons to whom the
	13	+ * Software is furnished to do so, subject to the following conditions:
	14	+ *
	15	+ * The above copyright notice and this permission notice (including the next
	16	+ * paragraph) shall be included in all copies or substantial portions of the
	17	+ * Software.
	18	+ *
	19	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
	20	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
	21	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
	22	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
	23	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
	24	+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
	25	+ * IN THE SOFTWARE.
	26	+ *
	27	+ */
	28	+
	29	+#ifndef __I915_OA_BXT_H__
	30	+#define __I915_OA_BXT_H__
	31	+
	32	+extern int i915_oa_n_builtin_metric_sets_bxt;
	33	+
	34	+extern int i915_oa_select_metric_set_bxt(struct drm_i915_private *dev_priv);
	35	+
	36	+extern int i915_perf_register_sysfs_bxt(struct drm_i915_private *dev_priv);
	37	+
	38	+extern void i915_perf_unregister_sysfs_bxt(struct drm_i915_private *dev_priv);
	39	+
	40	+#endif

drivers/gpu/drm/i915/i915_oa_chv.c+2873−0

@@ -0,0 +1,2873 @@
	1	+/*
	2	+ * Autogenerated file by GPU Top : https://github.com/rib/gputop
	3	+ * DO NOT EDIT manually!
	4	+ *
	5	+ *
	6	+ * Copyright (c) 2015 Intel Corporation
	7	+ *
	8	+ * Permission is hereby granted, free of charge, to any person obtaining a
	9	+ * copy of this software and associated documentation files (the "Software"),
	10	+ * to deal in the Software without restriction, including without limitation
	11	+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
	12	+ * and/or sell copies of the Software, and to permit persons to whom the
	13	+ * Software is furnished to do so, subject to the following conditions:
	14	+ *
	15	+ * The above copyright notice and this permission notice (including the next
	16	+ * paragraph) shall be included in all copies or substantial portions of the
	17	+ * Software.
	18	+ *
	19	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
	20	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
	21	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
	22	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
	23	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
	24	+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
	25	+ * IN THE SOFTWARE.
	26	+ *
	27	+ */
	28	+
	29	+#include <linux/sysfs.h>
	30	+
	31	+#include "i915_drv.h"
	32	+#include "i915_oa_chv.h"
	33	+
	34	+enum metric_set_id {
	35	+ METRIC_SET_ID_RENDER_BASIC = 1,
	36	+ METRIC_SET_ID_COMPUTE_BASIC,
	37	+ METRIC_SET_ID_RENDER_PIPE_PROFILE,
	38	+ METRIC_SET_ID_HDC_AND_SF,
	39	+ METRIC_SET_ID_L3_1,
	40	+ METRIC_SET_ID_L3_2,
	41	+ METRIC_SET_ID_L3_3,
	42	+ METRIC_SET_ID_L3_4,
	43	+ METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND,
	44	+ METRIC_SET_ID_SAMPLER_1,
	45	+ METRIC_SET_ID_SAMPLER_2,
	46	+ METRIC_SET_ID_TDL_1,
	47	+ METRIC_SET_ID_TDL_2,
	48	+ METRIC_SET_ID_TEST_OA,
	49	+};
	50	+
	51	+int i915_oa_n_builtin_metric_sets_chv = 14;
	52	+
	53	+static const struct i915_oa_reg b_counter_config_render_basic[] = {
	54	+ { _MMIO(0x2740), 0x00000000 },
	55	+ { _MMIO(0x2710), 0x00000000 },
	56	+ { _MMIO(0x2714), 0x00800000 },
	57	+ { _MMIO(0x2720), 0x00000000 },
	58	+ { _MMIO(0x2724), 0x00800000 },
	59	+};
	60	+
	61	+static const struct i915_oa_reg flex_eu_config_render_basic[] = {
	62	+ { _MMIO(0xe458), 0x00005004 },
	63	+ { _MMIO(0xe558), 0x00010003 },
	64	+ { _MMIO(0xe658), 0x00012011 },
	65	+ { _MMIO(0xe758), 0x00015014 },
	66	+ { _MMIO(0xe45c), 0x00051050 },
	67	+ { _MMIO(0xe55c), 0x00053052 },
	68	+ { _MMIO(0xe65c), 0x00055054 },
	69	+};
	70	+
	71	+static const struct i915_oa_reg mux_config_render_basic[] = {
	72	+ { _MMIO(0x9888), 0x59800000 },
	73	+ { _MMIO(0x9888), 0x59800001 },
	74	+ { _MMIO(0x9888), 0x285a0006 },
	75	+ { _MMIO(0x9888), 0x2c110014 },
	76	+ { _MMIO(0x9888), 0x2e110000 },
	77	+ { _MMIO(0x9888), 0x2c310014 },
	78	+ { _MMIO(0x9888), 0x2e310000 },
	79	+ { _MMIO(0x9888), 0x2b8303df },
	80	+ { _MMIO(0x9888), 0x3580024f },
	81	+ { _MMIO(0x9888), 0x00580888 },
	82	+ { _MMIO(0x9888), 0x1e5a0015 },
	83	+ { _MMIO(0x9888), 0x205a0014 },
	84	+ { _MMIO(0x9888), 0x045a0000 },
	85	+ { _MMIO(0x9888), 0x025a0000 },
	86	+ { _MMIO(0x9888), 0x02180500 },
	87	+ { _MMIO(0x9888), 0x00190555 },
	88	+ { _MMIO(0x9888), 0x021d0500 },
	89	+ { _MMIO(0x9888), 0x021f0a00 },
	90	+ { _MMIO(0x9888), 0x00380444 },
	91	+ { _MMIO(0x9888), 0x02390500 },
	92	+ { _MMIO(0x9888), 0x003a0666 },
	93	+ { _MMIO(0x9888), 0x00100111 },
	94	+ { _MMIO(0x9888), 0x06110030 },
	95	+ { _MMIO(0x9888), 0x0a110031 },
	96	+ { _MMIO(0x9888), 0x0e110046 },
	97	+ { _MMIO(0x9888), 0x04110000 },
	98	+ { _MMIO(0x9888), 0x00110000 },
	99	+ { _MMIO(0x9888), 0x00130111 },
	100	+ { _MMIO(0x9888), 0x00300444 },
	101	+ { _MMIO(0x9888), 0x08310030 },
	102	+ { _MMIO(0x9888), 0x0c310031 },
	103	+ { _MMIO(0x9888), 0x10310046 },
	104	+ { _MMIO(0x9888), 0x04310000 },
	105	+ { _MMIO(0x9888), 0x00310000 },
	106	+ { _MMIO(0x9888), 0x00330444 },
	107	+ { _MMIO(0x9888), 0x038a0a00 },
	108	+ { _MMIO(0x9888), 0x018b0fff },
	109	+ { _MMIO(0x9888), 0x038b0a00 },
	110	+ { _MMIO(0x9888), 0x01855000 },
	111	+ { _MMIO(0x9888), 0x03850055 },
	112	+ { _MMIO(0x9888), 0x13830021 },
	113	+ { _MMIO(0x9888), 0x15830020 },
	114	+ { _MMIO(0x9888), 0x1783002f },
	115	+ { _MMIO(0x9888), 0x1983002e },
	116	+ { _MMIO(0x9888), 0x1b83002d },
	117	+ { _MMIO(0x9888), 0x1d83002c },
	118	+ { _MMIO(0x9888), 0x05830000 },
	119	+ { _MMIO(0x9888), 0x01840555 },
	120	+ { _MMIO(0x9888), 0x03840500 },
	121	+ { _MMIO(0x9888), 0x23800074 },
	122	+ { _MMIO(0x9888), 0x2580007d },
	123	+ { _MMIO(0x9888), 0x05800000 },
	124	+ { _MMIO(0x9888), 0x01805000 },
	125	+ { _MMIO(0x9888), 0x03800055 },
	126	+ { _MMIO(0x9888), 0x01865000 },
	127	+ { _MMIO(0x9888), 0x03860055 },
	128	+ { _MMIO(0x9888), 0x01875000 },
	129	+ { _MMIO(0x9888), 0x03870055 },
	130	+ { _MMIO(0x9888), 0x418000aa },
	131	+ { _MMIO(0x9888), 0x4380000a },
	132	+ { _MMIO(0x9888), 0x45800000 },
	133	+ { _MMIO(0x9888), 0x4780000a },
	134	+ { _MMIO(0x9888), 0x49800000 },
	135	+ { _MMIO(0x9888), 0x4b800000 },
	136	+ { _MMIO(0x9888), 0x4d800000 },
	137	+ { _MMIO(0x9888), 0x4f800000 },
	138	+ { _MMIO(0x9888), 0x51800000 },
	139	+ { _MMIO(0x9888), 0x53800000 },
	140	+ { _MMIO(0x9888), 0x55800000 },
	141	+ { _MMIO(0x9888), 0x57800000 },
	142	+ { _MMIO(0x9888), 0x59800000 },
	143	+};
	144	+
	145	+static int
	146	+get_render_basic_mux_config(struct drm_i915_private *dev_priv,
	147	+ const struct i915_oa_reg **regs,
	148	+ int *lens)
	149	+{
	150	+ int n = 0;
	151	+
	152	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	153	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	154	+
	155	+ regs[n] = mux_config_render_basic;
	156	+ lens[n] = ARRAY_SIZE(mux_config_render_basic);
	157	+ n++;
	158	+
	159	+ return n;
	160	+}
	161	+
	162	+static const struct i915_oa_reg b_counter_config_compute_basic[] = {
	163	+ { _MMIO(0x2710), 0x00000000 },
	164	+ { _MMIO(0x2714), 0x00800000 },
	165	+ { _MMIO(0x2720), 0x00000000 },
	166	+ { _MMIO(0x2724), 0x00800000 },
	167	+};
	168	+
	169	+static const struct i915_oa_reg flex_eu_config_compute_basic[] = {
	170	+ { _MMIO(0xe458), 0x00005004 },
	171	+ { _MMIO(0xe558), 0x00000003 },
	172	+ { _MMIO(0xe658), 0x00002001 },
	173	+ { _MMIO(0xe758), 0x00778008 },
	174	+ { _MMIO(0xe45c), 0x00088078 },
	175	+ { _MMIO(0xe55c), 0x00808708 },
	176	+ { _MMIO(0xe65c), 0x00a08908 },
	177	+};
	178	+
	179	+static const struct i915_oa_reg mux_config_compute_basic[] = {
	180	+ { _MMIO(0x9888), 0x59800000 },
	181	+ { _MMIO(0x9888), 0x59800001 },
	182	+ { _MMIO(0x9888), 0x2e5800e0 },
	183	+ { _MMIO(0x9888), 0x2e3800e0 },
	184	+ { _MMIO(0x9888), 0x3580024f },
	185	+ { _MMIO(0x9888), 0x3d800140 },
	186	+ { _MMIO(0x9888), 0x08580042 },
	187	+ { _MMIO(0x9888), 0x0c580040 },
	188	+ { _MMIO(0x9888), 0x1058004c },
	189	+ { _MMIO(0x9888), 0x1458004b },
	190	+ { _MMIO(0x9888), 0x04580000 },
	191	+ { _MMIO(0x9888), 0x00580000 },
	192	+ { _MMIO(0x9888), 0x00195555 },
	193	+ { _MMIO(0x9888), 0x06380042 },
	194	+ { _MMIO(0x9888), 0x0a380040 },
	195	+ { _MMIO(0x9888), 0x0e38004c },
	196	+ { _MMIO(0x9888), 0x1238004b },
	197	+ { _MMIO(0x9888), 0x04380000 },
	198	+ { _MMIO(0x9888), 0x00384444 },
	199	+ { _MMIO(0x9888), 0x003a5555 },
	200	+ { _MMIO(0x9888), 0x018bffff },
	201	+ { _MMIO(0x9888), 0x01845555 },
	202	+ { _MMIO(0x9888), 0x17800074 },
	203	+ { _MMIO(0x9888), 0x1980007d },
	204	+ { _MMIO(0x9888), 0x1b80007c },
	205	+ { _MMIO(0x9888), 0x1d8000b6 },
	206	+ { _MMIO(0x9888), 0x1f8000b7 },
	207	+ { _MMIO(0x9888), 0x05800000 },
	208	+ { _MMIO(0x9888), 0x03800000 },
	209	+ { _MMIO(0x9888), 0x418000aa },
	210	+ { _MMIO(0x9888), 0x438000aa },
	211	+ { _MMIO(0x9888), 0x45800000 },
	212	+ { _MMIO(0x9888), 0x47800000 },
	213	+ { _MMIO(0x9888), 0x4980012a },
	214	+ { _MMIO(0x9888), 0x4b80012a },
	215	+ { _MMIO(0x9888), 0x4d80012a },
	216	+ { _MMIO(0x9888), 0x4f80012a },
	217	+ { _MMIO(0x9888), 0x518001ce },
	218	+ { _MMIO(0x9888), 0x538001ce },
	219	+ { _MMIO(0x9888), 0x5580000e },
	220	+ { _MMIO(0x9888), 0x59800000 },
	221	+};
	222	+
	223	+static int
	224	+get_compute_basic_mux_config(struct drm_i915_private *dev_priv,
	225	+ const struct i915_oa_reg **regs,
	226	+ int *lens)
	227	+{
	228	+ int n = 0;
	229	+
	230	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	231	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	232	+
	233	+ regs[n] = mux_config_compute_basic;
	234	+ lens[n] = ARRAY_SIZE(mux_config_compute_basic);
	235	+ n++;
	236	+
	237	+ return n;
	238	+}
	239	+
	240	+static const struct i915_oa_reg b_counter_config_render_pipe_profile[] = {
	241	+ { _MMIO(0x2724), 0xf0800000 },
	242	+ { _MMIO(0x2720), 0x00000000 },
	243	+ { _MMIO(0x2714), 0xf0800000 },
	244	+ { _MMIO(0x2710), 0x00000000 },
	245	+ { _MMIO(0x2770), 0x0007ffea },
	246	+ { _MMIO(0x2774), 0x00007ffc },
	247	+ { _MMIO(0x2778), 0x0007affa },
	248	+ { _MMIO(0x277c), 0x0000f5fd },
	249	+ { _MMIO(0x2780), 0x00079ffa },
	250	+ { _MMIO(0x2784), 0x0000f3fb },
	251	+ { _MMIO(0x2788), 0x0007bf7a },
	252	+ { _MMIO(0x278c), 0x0000f7e7 },
	253	+ { _MMIO(0x2790), 0x0007fefa },
	254	+ { _MMIO(0x2794), 0x0000f7cf },
	255	+ { _MMIO(0x2798), 0x00077ffa },
	256	+ { _MMIO(0x279c), 0x0000efdf },
	257	+ { _MMIO(0x27a0), 0x0006fffa },
	258	+ { _MMIO(0x27a4), 0x0000cfbf },
	259	+ { _MMIO(0x27a8), 0x0003fffa },
	260	+ { _MMIO(0x27ac), 0x00005f7f },
	261	+};
	262	+
	263	+static const struct i915_oa_reg flex_eu_config_render_pipe_profile[] = {
	264	+ { _MMIO(0xe458), 0x00005004 },
	265	+ { _MMIO(0xe558), 0x00015014 },
	266	+ { _MMIO(0xe658), 0x00025024 },
	267	+ { _MMIO(0xe758), 0x00035034 },
	268	+ { _MMIO(0xe45c), 0x00045044 },
	269	+ { _MMIO(0xe55c), 0x00055054 },
	270	+ { _MMIO(0xe65c), 0x00065064 },
	271	+};
	272	+
	273	+static const struct i915_oa_reg mux_config_render_pipe_profile[] = {
	274	+ { _MMIO(0x9888), 0x59800000 },
	275	+ { _MMIO(0x9888), 0x59800001 },
	276	+ { _MMIO(0x9888), 0x261e0000 },
	277	+ { _MMIO(0x9888), 0x281f000f },
	278	+ { _MMIO(0x9888), 0x2817001a },
	279	+ { _MMIO(0x9888), 0x2791001f },
	280	+ { _MMIO(0x9888), 0x27880019 },
	281	+ { _MMIO(0x9888), 0x2d890000 },
	282	+ { _MMIO(0x9888), 0x278a0007 },
	283	+ { _MMIO(0x9888), 0x298d001f },
	284	+ { _MMIO(0x9888), 0x278e0020 },
	285	+ { _MMIO(0x9888), 0x2b8f0012 },
	286	+ { _MMIO(0x9888), 0x29900000 },
	287	+ { _MMIO(0x9888), 0x00184000 },
	288	+ { _MMIO(0x9888), 0x02181000 },
	289	+ { _MMIO(0x9888), 0x02194000 },
	290	+ { _MMIO(0x9888), 0x141e0002 },
	291	+ { _MMIO(0x9888), 0x041e0000 },
	292	+ { _MMIO(0x9888), 0x001e0000 },
	293	+ { _MMIO(0x9888), 0x221f0015 },
	294	+ { _MMIO(0x9888), 0x041f0000 },
	295	+ { _MMIO(0x9888), 0x001f4000 },
	296	+ { _MMIO(0x9888), 0x021f0000 },
	297	+ { _MMIO(0x9888), 0x023a8000 },
	298	+ { _MMIO(0x9888), 0x0213c000 },
	299	+ { _MMIO(0x9888), 0x02164000 },
	300	+ { _MMIO(0x9888), 0x24170012 },
	301	+ { _MMIO(0x9888), 0x04170000 },
	302	+ { _MMIO(0x9888), 0x07910005 },
	303	+ { _MMIO(0x9888), 0x05910000 },
	304	+ { _MMIO(0x9888), 0x01911500 },
	305	+ { _MMIO(0x9888), 0x03910501 },
	306	+ { _MMIO(0x9888), 0x0d880002 },
	307	+ { _MMIO(0x9888), 0x1d880003 },
	308	+ { _MMIO(0x9888), 0x05880000 },
	309	+ { _MMIO(0x9888), 0x0b890032 },
	310	+ { _MMIO(0x9888), 0x1b890031 },
	311	+ { _MMIO(0x9888), 0x05890000 },
	312	+ { _MMIO(0x9888), 0x01890040 },
	313	+ { _MMIO(0x9888), 0x03890040 },
	314	+ { _MMIO(0x9888), 0x098a0000 },
	315	+ { _MMIO(0x9888), 0x198a0004 },
	316	+ { _MMIO(0x9888), 0x058a0000 },
	317	+ { _MMIO(0x9888), 0x018a8050 },
	318	+ { _MMIO(0x9888), 0x038a2050 },
	319	+ { _MMIO(0x9888), 0x018b95a9 },
	320	+ { _MMIO(0x9888), 0x038be5a9 },
	321	+ { _MMIO(0x9888), 0x018c1500 },
	322	+ { _MMIO(0x9888), 0x038c0501 },
	323	+ { _MMIO(0x9888), 0x178d0015 },
	324	+ { _MMIO(0x9888), 0x058d0000 },
	325	+ { _MMIO(0x9888), 0x138e0004 },
	326	+ { _MMIO(0x9888), 0x218e000c },
	327	+ { _MMIO(0x9888), 0x058e0000 },
	328	+ { _MMIO(0x9888), 0x018e0500 },
	329	+ { _MMIO(0x9888), 0x038e0101 },
	330	+ { _MMIO(0x9888), 0x0f8f0027 },
	331	+ { _MMIO(0x9888), 0x058f0000 },
	332	+ { _MMIO(0x9888), 0x018f0000 },
	333	+ { _MMIO(0x9888), 0x038f0001 },
	334	+ { _MMIO(0x9888), 0x11900013 },
	335	+ { _MMIO(0x9888), 0x1f900017 },
	336	+ { _MMIO(0x9888), 0x05900000 },
	337	+ { _MMIO(0x9888), 0x01900100 },
	338	+ { _MMIO(0x9888), 0x03900001 },
	339	+ { _MMIO(0x9888), 0x01845555 },
	340	+ { _MMIO(0x9888), 0x03845555 },
	341	+ { _MMIO(0x9888), 0x418000aa },
	342	+ { _MMIO(0x9888), 0x438000aa },
	343	+ { _MMIO(0x9888), 0x458000aa },
	344	+ { _MMIO(0x9888), 0x478000aa },
	345	+ { _MMIO(0x9888), 0x4980018c },
	346	+ { _MMIO(0x9888), 0x4b80014b },
	347	+ { _MMIO(0x9888), 0x4d800128 },
	348	+ { _MMIO(0x9888), 0x4f80012a },
	349	+ { _MMIO(0x9888), 0x51800187 },
	350	+ { _MMIO(0x9888), 0x5380014b },
	351	+ { _MMIO(0x9888), 0x55800149 },
	352	+ { _MMIO(0x9888), 0x5780010a },
	353	+ { _MMIO(0x9888), 0x59800000 },
	354	+};
	355	+
	356	+static int
	357	+get_render_pipe_profile_mux_config(struct drm_i915_private *dev_priv,
	358	+ const struct i915_oa_reg **regs,
	359	+ int *lens)
	360	+{
	361	+ int n = 0;
	362	+
	363	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	364	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	365	+
	366	+ regs[n] = mux_config_render_pipe_profile;
	367	+ lens[n] = ARRAY_SIZE(mux_config_render_pipe_profile);
	368	+ n++;
	369	+
	370	+ return n;
	371	+}
	372	+
	373	+static const struct i915_oa_reg b_counter_config_hdc_and_sf[] = {
	374	+ { _MMIO(0x2740), 0x00000000 },
	375	+ { _MMIO(0x2744), 0x00800000 },
	376	+ { _MMIO(0x2710), 0x00000000 },
	377	+ { _MMIO(0x2714), 0x10800000 },
	378	+ { _MMIO(0x2720), 0x00000000 },
	379	+ { _MMIO(0x2724), 0x00800000 },
	380	+ { _MMIO(0x2770), 0x00000002 },
	381	+ { _MMIO(0x2774), 0x0000fff7 },
	382	+};
	383	+
	384	+static const struct i915_oa_reg flex_eu_config_hdc_and_sf[] = {
	385	+ { _MMIO(0xe458), 0x00005004 },
	386	+ { _MMIO(0xe558), 0x00010003 },
	387	+ { _MMIO(0xe658), 0x00012011 },
	388	+ { _MMIO(0xe758), 0x00015014 },
	389	+ { _MMIO(0xe45c), 0x00051050 },
	390	+ { _MMIO(0xe55c), 0x00053052 },
	391	+ { _MMIO(0xe65c), 0x00055054 },
	392	+};
	393	+
	394	+static const struct i915_oa_reg mux_config_hdc_and_sf[] = {
	395	+ { _MMIO(0x9888), 0x105c0232 },
	396	+ { _MMIO(0x9888), 0x10580232 },
	397	+ { _MMIO(0x9888), 0x10380232 },
	398	+ { _MMIO(0x9888), 0x10dc0232 },
	399	+ { _MMIO(0x9888), 0x10d80232 },
	400	+ { _MMIO(0x9888), 0x10b80232 },
	401	+ { _MMIO(0x9888), 0x118e4400 },
	402	+ { _MMIO(0x9888), 0x025c6080 },
	403	+ { _MMIO(0x9888), 0x045c004b },
	404	+ { _MMIO(0x9888), 0x005c8000 },
	405	+ { _MMIO(0x9888), 0x00582080 },
	406	+ { _MMIO(0x9888), 0x0258004b },
	407	+ { _MMIO(0x9888), 0x025b4000 },
	408	+ { _MMIO(0x9888), 0x045b4000 },
	409	+ { _MMIO(0x9888), 0x0c1fa000 },
	410	+ { _MMIO(0x9888), 0x0e1f00aa },
	411	+ { _MMIO(0x9888), 0x04386080 },
	412	+ { _MMIO(0x9888), 0x0638404b },
	413	+ { _MMIO(0x9888), 0x02384000 },
	414	+ { _MMIO(0x9888), 0x08384000 },
	415	+ { _MMIO(0x9888), 0x0a380000 },
	416	+ { _MMIO(0x9888), 0x0c380000 },
	417	+ { _MMIO(0x9888), 0x00398000 },
	418	+ { _MMIO(0x9888), 0x0239a000 },
	419	+ { _MMIO(0x9888), 0x0439a000 },
	420	+ { _MMIO(0x9888), 0x06392000 },
	421	+ { _MMIO(0x9888), 0x0cdc25c1 },
	422	+ { _MMIO(0x9888), 0x0adcc000 },
	423	+ { _MMIO(0x9888), 0x0ad825c1 },
	424	+ { _MMIO(0x9888), 0x18db4000 },
	425	+ { _MMIO(0x9888), 0x1adb0001 },
	426	+ { _MMIO(0x9888), 0x0e9f8000 },
	427	+ { _MMIO(0x9888), 0x109f02aa },
	428	+ { _MMIO(0x9888), 0x0eb825c1 },
	429	+ { _MMIO(0x9888), 0x18b80154 },
	430	+ { _MMIO(0x9888), 0x0ab9a000 },
	431	+ { _MMIO(0x9888), 0x0cb9a000 },
	432	+ { _MMIO(0x9888), 0x0eb9a000 },
	433	+ { _MMIO(0x9888), 0x0d88c000 },
	434	+ { _MMIO(0x9888), 0x0f88000f },
	435	+ { _MMIO(0x9888), 0x038a8000 },
	436	+ { _MMIO(0x9888), 0x058a8000 },
	437	+ { _MMIO(0x9888), 0x078a8000 },
	438	+ { _MMIO(0x9888), 0x098a8000 },
	439	+ { _MMIO(0x9888), 0x0b8a8000 },
	440	+ { _MMIO(0x9888), 0x0d8a8000 },
	441	+ { _MMIO(0x9888), 0x258baa05 },
	442	+ { _MMIO(0x9888), 0x278b002a },
	443	+ { _MMIO(0x9888), 0x238b2a80 },
	444	+ { _MMIO(0x9888), 0x198c5400 },
	445	+ { _MMIO(0x9888), 0x1b8c0015 },
	446	+ { _MMIO(0x9888), 0x098dc000 },
	447	+ { _MMIO(0x9888), 0x0b8da000 },
	448	+ { _MMIO(0x9888), 0x0d8da000 },
	449	+ { _MMIO(0x9888), 0x0f8da000 },
	450	+ { _MMIO(0x9888), 0x098e05c0 },
	451	+ { _MMIO(0x9888), 0x058e0000 },
	452	+ { _MMIO(0x9888), 0x198f0020 },
	453	+ { _MMIO(0x9888), 0x2185aa0a },
	454	+ { _MMIO(0x9888), 0x2385002a },
	455	+ { _MMIO(0x9888), 0x1f85aa00 },
	456	+ { _MMIO(0x9888), 0x19835000 },
	457	+ { _MMIO(0x9888), 0x1b830155 },
	458	+ { _MMIO(0x9888), 0x03834000 },
	459	+ { _MMIO(0x9888), 0x05834000 },
	460	+ { _MMIO(0x9888), 0x07834000 },
	461	+ { _MMIO(0x9888), 0x09834000 },
	462	+ { _MMIO(0x9888), 0x0b834000 },
	463	+ { _MMIO(0x9888), 0x0d834000 },
	464	+ { _MMIO(0x9888), 0x09848000 },
	465	+ { _MMIO(0x9888), 0x0b84c000 },
	466	+ { _MMIO(0x9888), 0x0d84c000 },
	467	+ { _MMIO(0x9888), 0x0f84c000 },
	468	+ { _MMIO(0x9888), 0x01848000 },
	469	+ { _MMIO(0x9888), 0x0384c000 },
	470	+ { _MMIO(0x9888), 0x0584c000 },
	471	+ { _MMIO(0x9888), 0x07844000 },
	472	+ { _MMIO(0x9888), 0x19808000 },
	473	+ { _MMIO(0x9888), 0x1b80c000 },
	474	+ { _MMIO(0x9888), 0x1d80c000 },
	475	+ { _MMIO(0x9888), 0x1f80c000 },
	476	+ { _MMIO(0x9888), 0x11808000 },
	477	+ { _MMIO(0x9888), 0x1380c000 },
	478	+ { _MMIO(0x9888), 0x1580c000 },
	479	+ { _MMIO(0x9888), 0x17804000 },
	480	+ { _MMIO(0x9888), 0x51800040 },
	481	+ { _MMIO(0x9888), 0x43800400 },
	482	+ { _MMIO(0x9888), 0x45800800 },
	483	+ { _MMIO(0x9888), 0x53800000 },
	484	+ { _MMIO(0x9888), 0x47800c62 },
	485	+ { _MMIO(0x9888), 0x21800000 },
	486	+ { _MMIO(0x9888), 0x31800000 },
	487	+ { _MMIO(0x9888), 0x4d800000 },
	488	+ { _MMIO(0x9888), 0x3f801042 },
	489	+ { _MMIO(0x9888), 0x4f800000 },
	490	+ { _MMIO(0x9888), 0x418014a4 },
	491	+};
	492	+
	493	+static int
	494	+get_hdc_and_sf_mux_config(struct drm_i915_private *dev_priv,
	495	+ const struct i915_oa_reg **regs,
	496	+ int *lens)
	497	+{
	498	+ int n = 0;
	499	+
	500	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	501	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	502	+
	503	+ regs[n] = mux_config_hdc_and_sf;
	504	+ lens[n] = ARRAY_SIZE(mux_config_hdc_and_sf);
	505	+ n++;
	506	+
	507	+ return n;
	508	+}
	509	+
	510	+static const struct i915_oa_reg b_counter_config_l3_1[] = {
	511	+ { _MMIO(0x2740), 0x00000000 },
	512	+ { _MMIO(0x2744), 0x00800000 },
	513	+ { _MMIO(0x2710), 0x00000000 },
	514	+ { _MMIO(0x2714), 0xf0800000 },
	515	+ { _MMIO(0x2720), 0x00000000 },
	516	+ { _MMIO(0x2724), 0xf0800000 },
	517	+ { _MMIO(0x2770), 0x00100070 },
	518	+ { _MMIO(0x2774), 0x0000fff1 },
	519	+ { _MMIO(0x2778), 0x00014002 },
	520	+ { _MMIO(0x277c), 0x0000c3ff },
	521	+ { _MMIO(0x2780), 0x00010002 },
	522	+ { _MMIO(0x2784), 0x0000c7ff },
	523	+ { _MMIO(0x2788), 0x00004002 },
	524	+ { _MMIO(0x278c), 0x0000d3ff },
	525	+ { _MMIO(0x2790), 0x00100700 },
	526	+ { _MMIO(0x2794), 0x0000ff1f },
	527	+ { _MMIO(0x2798), 0x00001402 },
	528	+ { _MMIO(0x279c), 0x0000fc3f },
	529	+ { _MMIO(0x27a0), 0x00001002 },
	530	+ { _MMIO(0x27a4), 0x0000fc7f },
	531	+ { _MMIO(0x27a8), 0x00000402 },
	532	+ { _MMIO(0x27ac), 0x0000fd3f },
	533	+};
	534	+
	535	+static const struct i915_oa_reg flex_eu_config_l3_1[] = {
	536	+ { _MMIO(0xe458), 0x00005004 },
	537	+ { _MMIO(0xe558), 0x00010003 },
	538	+ { _MMIO(0xe658), 0x00012011 },
	539	+ { _MMIO(0xe758), 0x00015014 },
	540	+ { _MMIO(0xe45c), 0x00051050 },
	541	+ { _MMIO(0xe55c), 0x00053052 },
	542	+ { _MMIO(0xe65c), 0x00055054 },
	543	+};
	544	+
	545	+static const struct i915_oa_reg mux_config_l3_1[] = {
	546	+ { _MMIO(0x9888), 0x10bf03da },
	547	+ { _MMIO(0x9888), 0x14bf0001 },
	548	+ { _MMIO(0x9888), 0x12980340 },
	549	+ { _MMIO(0x9888), 0x12990340 },
	550	+ { _MMIO(0x9888), 0x0cbf1187 },
	551	+ { _MMIO(0x9888), 0x0ebf1205 },
	552	+ { _MMIO(0x9888), 0x00bf0500 },
	553	+ { _MMIO(0x9888), 0x02bf042b },
	554	+ { _MMIO(0x9888), 0x04bf002c },
	555	+ { _MMIO(0x9888), 0x0cdac000 },
	556	+ { _MMIO(0x9888), 0x0edac000 },
	557	+ { _MMIO(0x9888), 0x00da8000 },
	558	+ { _MMIO(0x9888), 0x02dac000 },
	559	+ { _MMIO(0x9888), 0x04da4000 },
	560	+ { _MMIO(0x9888), 0x04983400 },
	561	+ { _MMIO(0x9888), 0x10980000 },
	562	+ { _MMIO(0x9888), 0x06990034 },
	563	+ { _MMIO(0x9888), 0x10990000 },
	564	+ { _MMIO(0x9888), 0x0c9dc000 },
	565	+ { _MMIO(0x9888), 0x0e9dc000 },
	566	+ { _MMIO(0x9888), 0x009d8000 },
	567	+ { _MMIO(0x9888), 0x029dc000 },
	568	+ { _MMIO(0x9888), 0x049d4000 },
	569	+ { _MMIO(0x9888), 0x109f02a8 },
	570	+ { _MMIO(0x9888), 0x0c9fa000 },
	571	+ { _MMIO(0x9888), 0x0e9f00ba },
	572	+ { _MMIO(0x9888), 0x0cb88000 },
	573	+ { _MMIO(0x9888), 0x0cb95000 },
	574	+ { _MMIO(0x9888), 0x0eb95000 },
	575	+ { _MMIO(0x9888), 0x00b94000 },
	576	+ { _MMIO(0x9888), 0x02b95000 },
	577	+ { _MMIO(0x9888), 0x04b91000 },
	578	+ { _MMIO(0x9888), 0x06b92000 },
	579	+ { _MMIO(0x9888), 0x0cba4000 },
	580	+ { _MMIO(0x9888), 0x0f88000f },
	581	+ { _MMIO(0x9888), 0x03888000 },
	582	+ { _MMIO(0x9888), 0x05888000 },
	583	+ { _MMIO(0x9888), 0x07888000 },
	584	+ { _MMIO(0x9888), 0x09888000 },
	585	+ { _MMIO(0x9888), 0x0b888000 },
	586	+ { _MMIO(0x9888), 0x0d880400 },
	587	+ { _MMIO(0x9888), 0x258b800a },
	588	+ { _MMIO(0x9888), 0x278b002a },
	589	+ { _MMIO(0x9888), 0x238b5500 },
	590	+ { _MMIO(0x9888), 0x198c4000 },
	591	+ { _MMIO(0x9888), 0x1b8c0015 },
	592	+ { _MMIO(0x9888), 0x038c4000 },
	593	+ { _MMIO(0x9888), 0x058c4000 },
	594	+ { _MMIO(0x9888), 0x078c4000 },
	595	+ { _MMIO(0x9888), 0x098c4000 },
	596	+ { _MMIO(0x9888), 0x0b8c4000 },
	597	+ { _MMIO(0x9888), 0x0d8c4000 },
	598	+ { _MMIO(0x9888), 0x0d8da000 },
	599	+ { _MMIO(0x9888), 0x0f8da000 },
	600	+ { _MMIO(0x9888), 0x018d8000 },
	601	+ { _MMIO(0x9888), 0x038da000 },
	602	+ { _MMIO(0x9888), 0x058da000 },
	603	+ { _MMIO(0x9888), 0x078d2000 },
	604	+ { _MMIO(0x9888), 0x2185800a },
	605	+ { _MMIO(0x9888), 0x2385002a },
	606	+ { _MMIO(0x9888), 0x1f85aa00 },
	607	+ { _MMIO(0x9888), 0x1b830154 },
	608	+ { _MMIO(0x9888), 0x03834000 },
	609	+ { _MMIO(0x9888), 0x05834000 },
	610	+ { _MMIO(0x9888), 0x07834000 },
	611	+ { _MMIO(0x9888), 0x09834000 },
	612	+ { _MMIO(0x9888), 0x0b834000 },
	613	+ { _MMIO(0x9888), 0x0d834000 },
	614	+ { _MMIO(0x9888), 0x0d84c000 },
	615	+ { _MMIO(0x9888), 0x0f84c000 },
	616	+ { _MMIO(0x9888), 0x01848000 },
	617	+ { _MMIO(0x9888), 0x0384c000 },
	618	+ { _MMIO(0x9888), 0x0584c000 },
	619	+ { _MMIO(0x9888), 0x07844000 },
	620	+ { _MMIO(0x9888), 0x1d80c000 },
	621	+ { _MMIO(0x9888), 0x1f80c000 },
	622	+ { _MMIO(0x9888), 0x11808000 },
	623	+ { _MMIO(0x9888), 0x1380c000 },
	624	+ { _MMIO(0x9888), 0x1580c000 },
	625	+ { _MMIO(0x9888), 0x17804000 },
	626	+ { _MMIO(0x9888), 0x53800000 },
	627	+ { _MMIO(0x9888), 0x45800000 },
	628	+ { _MMIO(0x9888), 0x47800000 },
	629	+ { _MMIO(0x9888), 0x21800000 },
	630	+ { _MMIO(0x9888), 0x31800000 },
	631	+ { _MMIO(0x9888), 0x4d800000 },
	632	+ { _MMIO(0x9888), 0x3f800000 },
	633	+ { _MMIO(0x9888), 0x4f800000 },
	634	+ { _MMIO(0x9888), 0x41800060 },
	635	+};
	636	+
	637	+static int
	638	+get_l3_1_mux_config(struct drm_i915_private *dev_priv,
	639	+ const struct i915_oa_reg **regs,
	640	+ int *lens)
	641	+{
	642	+ int n = 0;
	643	+
	644	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	645	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	646	+
	647	+ regs[n] = mux_config_l3_1;
	648	+ lens[n] = ARRAY_SIZE(mux_config_l3_1);
	649	+ n++;
	650	+
	651	+ return n;
	652	+}
	653	+
	654	+static const struct i915_oa_reg b_counter_config_l3_2[] = {
	655	+ { _MMIO(0x2740), 0x00000000 },
	656	+ { _MMIO(0x2744), 0x00800000 },
	657	+ { _MMIO(0x2710), 0x00000000 },
	658	+ { _MMIO(0x2714), 0xf0800000 },
	659	+ { _MMIO(0x2720), 0x00000000 },
	660	+ { _MMIO(0x2724), 0xf0800000 },
	661	+ { _MMIO(0x2770), 0x00100070 },
	662	+ { _MMIO(0x2774), 0x0000fff1 },
	663	+ { _MMIO(0x2778), 0x00014002 },
	664	+ { _MMIO(0x277c), 0x0000c3ff },
	665	+ { _MMIO(0x2780), 0x00010002 },
	666	+ { _MMIO(0x2784), 0x0000c7ff },
	667	+ { _MMIO(0x2788), 0x00004002 },
	668	+ { _MMIO(0x278c), 0x0000d3ff },
	669	+ { _MMIO(0x2790), 0x00100700 },
	670	+ { _MMIO(0x2794), 0x0000ff1f },
	671	+ { _MMIO(0x2798), 0x00001402 },
	672	+ { _MMIO(0x279c), 0x0000fc3f },
	673	+ { _MMIO(0x27a0), 0x00001002 },
	674	+ { _MMIO(0x27a4), 0x0000fc7f },
	675	+ { _MMIO(0x27a8), 0x00000402 },
	676	+ { _MMIO(0x27ac), 0x0000fd3f },
	677	+};
	678	+
	679	+static const struct i915_oa_reg flex_eu_config_l3_2[] = {
	680	+ { _MMIO(0xe458), 0x00005004 },
	681	+ { _MMIO(0xe558), 0x00010003 },
	682	+ { _MMIO(0xe658), 0x00012011 },
	683	+ { _MMIO(0xe758), 0x00015014 },
	684	+ { _MMIO(0xe45c), 0x00051050 },
	685	+ { _MMIO(0xe55c), 0x00053052 },
	686	+ { _MMIO(0xe65c), 0x00055054 },
	687	+};
	688	+
	689	+static const struct i915_oa_reg mux_config_l3_2[] = {
	690	+ { _MMIO(0x9888), 0x103f03da },
	691	+ { _MMIO(0x9888), 0x143f0001 },
	692	+ { _MMIO(0x9888), 0x12180340 },
	693	+ { _MMIO(0x9888), 0x12190340 },
	694	+ { _MMIO(0x9888), 0x0c3f1187 },
	695	+ { _MMIO(0x9888), 0x0e3f1205 },
	696	+ { _MMIO(0x9888), 0x003f0500 },
	697	+ { _MMIO(0x9888), 0x023f042b },
	698	+ { _MMIO(0x9888), 0x043f002c },
	699	+ { _MMIO(0x9888), 0x0c5ac000 },
	700	+ { _MMIO(0x9888), 0x0e5ac000 },
	701	+ { _MMIO(0x9888), 0x005a8000 },
	702	+ { _MMIO(0x9888), 0x025ac000 },
	703	+ { _MMIO(0x9888), 0x045a4000 },
	704	+ { _MMIO(0x9888), 0x04183400 },
	705	+ { _MMIO(0x9888), 0x10180000 },
	706	+ { _MMIO(0x9888), 0x06190034 },
	707	+ { _MMIO(0x9888), 0x10190000 },
	708	+ { _MMIO(0x9888), 0x0c1dc000 },
	709	+ { _MMIO(0x9888), 0x0e1dc000 },
	710	+ { _MMIO(0x9888), 0x001d8000 },
	711	+ { _MMIO(0x9888), 0x021dc000 },
	712	+ { _MMIO(0x9888), 0x041d4000 },
	713	+ { _MMIO(0x9888), 0x101f02a8 },
	714	+ { _MMIO(0x9888), 0x0c1fa000 },
	715	+ { _MMIO(0x9888), 0x0e1f00ba },
	716	+ { _MMIO(0x9888), 0x0c388000 },
	717	+ { _MMIO(0x9888), 0x0c395000 },
	718	+ { _MMIO(0x9888), 0x0e395000 },
	719	+ { _MMIO(0x9888), 0x00394000 },
	720	+ { _MMIO(0x9888), 0x02395000 },
	721	+ { _MMIO(0x9888), 0x04391000 },
	722	+ { _MMIO(0x9888), 0x06392000 },
	723	+ { _MMIO(0x9888), 0x0c3a4000 },
	724	+ { _MMIO(0x9888), 0x1b8aa800 },
	725	+ { _MMIO(0x9888), 0x1d8a0002 },
	726	+ { _MMIO(0x9888), 0x038a8000 },
	727	+ { _MMIO(0x9888), 0x058a8000 },
	728	+ { _MMIO(0x9888), 0x078a8000 },
	729	+ { _MMIO(0x9888), 0x098a8000 },
	730	+ { _MMIO(0x9888), 0x0b8a8000 },
	731	+ { _MMIO(0x9888), 0x0d8a8000 },
	732	+ { _MMIO(0x9888), 0x258b4005 },
	733	+ { _MMIO(0x9888), 0x278b0015 },
	734	+ { _MMIO(0x9888), 0x238b2a80 },
	735	+ { _MMIO(0x9888), 0x2185800a },
	736	+ { _MMIO(0x9888), 0x2385002a },
	737	+ { _MMIO(0x9888), 0x1f85aa00 },
	738	+ { _MMIO(0x9888), 0x1b830154 },
	739	+ { _MMIO(0x9888), 0x03834000 },
	740	+ { _MMIO(0x9888), 0x05834000 },
	741	+ { _MMIO(0x9888), 0x07834000 },
	742	+ { _MMIO(0x9888), 0x09834000 },
	743	+ { _MMIO(0x9888), 0x0b834000 },
	744	+ { _MMIO(0x9888), 0x0d834000 },
	745	+ { _MMIO(0x9888), 0x0d84c000 },
	746	+ { _MMIO(0x9888), 0x0f84c000 },
	747	+ { _MMIO(0x9888), 0x01848000 },
	748	+ { _MMIO(0x9888), 0x0384c000 },
	749	+ { _MMIO(0x9888), 0x0584c000 },
	750	+ { _MMIO(0x9888), 0x07844000 },
	751	+ { _MMIO(0x9888), 0x1d80c000 },
	752	+ { _MMIO(0x9888), 0x1f80c000 },
	753	+ { _MMIO(0x9888), 0x11808000 },
	754	+ { _MMIO(0x9888), 0x1380c000 },
	755	+ { _MMIO(0x9888), 0x1580c000 },
	756	+ { _MMIO(0x9888), 0x17804000 },
	757	+ { _MMIO(0x9888), 0x53800000 },
	758	+ { _MMIO(0x9888), 0x45800000 },
	759	+ { _MMIO(0x9888), 0x47800000 },
	760	+ { _MMIO(0x9888), 0x21800000 },
	761	+ { _MMIO(0x9888), 0x31800000 },
	762	+ { _MMIO(0x9888), 0x4d800000 },
	763	+ { _MMIO(0x9888), 0x3f800000 },
	764	+ { _MMIO(0x9888), 0x4f800000 },
	765	+ { _MMIO(0x9888), 0x41800060 },
	766	+};
	767	+
	768	+static int
	769	+get_l3_2_mux_config(struct drm_i915_private *dev_priv,
	770	+ const struct i915_oa_reg **regs,
	771	+ int *lens)
	772	+{
	773	+ int n = 0;
	774	+
	775	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	776	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	777	+
	778	+ regs[n] = mux_config_l3_2;
	779	+ lens[n] = ARRAY_SIZE(mux_config_l3_2);
	780	+ n++;
	781	+
	782	+ return n;
	783	+}
	784	+
	785	+static const struct i915_oa_reg b_counter_config_l3_3[] = {
	786	+ { _MMIO(0x2740), 0x00000000 },
	787	+ { _MMIO(0x2744), 0x00800000 },
	788	+ { _MMIO(0x2710), 0x00000000 },
	789	+ { _MMIO(0x2714), 0xf0800000 },
	790	+ { _MMIO(0x2720), 0x00000000 },
	791	+ { _MMIO(0x2724), 0xf0800000 },
	792	+ { _MMIO(0x2770), 0x00100070 },
	793	+ { _MMIO(0x2774), 0x0000fff1 },
	794	+ { _MMIO(0x2778), 0x00014002 },
	795	+ { _MMIO(0x277c), 0x0000c3ff },
	796	+ { _MMIO(0x2780), 0x00010002 },
	797	+ { _MMIO(0x2784), 0x0000c7ff },
	798	+ { _MMIO(0x2788), 0x00004002 },
	799	+ { _MMIO(0x278c), 0x0000d3ff },
	800	+ { _MMIO(0x2790), 0x00100700 },
	801	+ { _MMIO(0x2794), 0x0000ff1f },
	802	+ { _MMIO(0x2798), 0x00001402 },
	803	+ { _MMIO(0x279c), 0x0000fc3f },
	804	+ { _MMIO(0x27a0), 0x00001002 },
	805	+ { _MMIO(0x27a4), 0x0000fc7f },
	806	+ { _MMIO(0x27a8), 0x00000402 },
	807	+ { _MMIO(0x27ac), 0x0000fd3f },
	808	+};
	809	+
	810	+static const struct i915_oa_reg flex_eu_config_l3_3[] = {
	811	+ { _MMIO(0xe458), 0x00005004 },
	812	+ { _MMIO(0xe558), 0x00010003 },
	813	+ { _MMIO(0xe658), 0x00012011 },
	814	+ { _MMIO(0xe758), 0x00015014 },
	815	+ { _MMIO(0xe45c), 0x00051050 },
	816	+ { _MMIO(0xe55c), 0x00053052 },
	817	+ { _MMIO(0xe65c), 0x00055054 },
	818	+};
	819	+
	820	+static const struct i915_oa_reg mux_config_l3_3[] = {
	821	+ { _MMIO(0x9888), 0x121b0340 },
	822	+ { _MMIO(0x9888), 0x103f0274 },
	823	+ { _MMIO(0x9888), 0x123f0000 },
	824	+ { _MMIO(0x9888), 0x129b0340 },
	825	+ { _MMIO(0x9888), 0x10bf0274 },
	826	+ { _MMIO(0x9888), 0x12bf0000 },
	827	+ { _MMIO(0x9888), 0x041b3400 },
	828	+ { _MMIO(0x9888), 0x101b0000 },
	829	+ { _MMIO(0x9888), 0x045c8000 },
	830	+ { _MMIO(0x9888), 0x0a3d4000 },
	831	+ { _MMIO(0x9888), 0x003f0080 },
	832	+ { _MMIO(0x9888), 0x023f0793 },
	833	+ { _MMIO(0x9888), 0x043f0014 },
	834	+ { _MMIO(0x9888), 0x04588000 },
	835	+ { _MMIO(0x9888), 0x005a8000 },
	836	+ { _MMIO(0x9888), 0x025ac000 },
	837	+ { _MMIO(0x9888), 0x045a4000 },
	838	+ { _MMIO(0x9888), 0x0a5b4000 },
	839	+ { _MMIO(0x9888), 0x001d8000 },
	840	+ { _MMIO(0x9888), 0x021dc000 },
	841	+ { _MMIO(0x9888), 0x041d4000 },
	842	+ { _MMIO(0x9888), 0x0c1fa000 },
	843	+ { _MMIO(0x9888), 0x0e1f002a },
	844	+ { _MMIO(0x9888), 0x0a384000 },
	845	+ { _MMIO(0x9888), 0x00394000 },
	846	+ { _MMIO(0x9888), 0x02395000 },
	847	+ { _MMIO(0x9888), 0x04399000 },
	848	+ { _MMIO(0x9888), 0x069b0034 },
	849	+ { _MMIO(0x9888), 0x109b0000 },
	850	+ { _MMIO(0x9888), 0x06dc4000 },
	851	+ { _MMIO(0x9888), 0x0cbd4000 },
	852	+ { _MMIO(0x9888), 0x0cbf0981 },
	853	+ { _MMIO(0x9888), 0x0ebf0a0f },
	854	+ { _MMIO(0x9888), 0x06d84000 },
	855	+ { _MMIO(0x9888), 0x0cdac000 },
	856	+ { _MMIO(0x9888), 0x0edac000 },
	857	+ { _MMIO(0x9888), 0x0cdb4000 },
	858	+ { _MMIO(0x9888), 0x0c9dc000 },
	859	+ { _MMIO(0x9888), 0x0e9dc000 },
	860	+ { _MMIO(0x9888), 0x109f02a8 },
	861	+ { _MMIO(0x9888), 0x0e9f0080 },
	862	+ { _MMIO(0x9888), 0x0cb84000 },
	863	+ { _MMIO(0x9888), 0x0cb95000 },
	864	+ { _MMIO(0x9888), 0x0eb95000 },
	865	+ { _MMIO(0x9888), 0x06b92000 },
	866	+ { _MMIO(0x9888), 0x0f88000f },
	867	+ { _MMIO(0x9888), 0x0d880400 },
	868	+ { _MMIO(0x9888), 0x038a8000 },
	869	+ { _MMIO(0x9888), 0x058a8000 },
	870	+ { _MMIO(0x9888), 0x078a8000 },
	871	+ { _MMIO(0x9888), 0x098a8000 },
	872	+ { _MMIO(0x9888), 0x0b8a8000 },
	873	+ { _MMIO(0x9888), 0x258b8009 },
	874	+ { _MMIO(0x9888), 0x278b002a },
	875	+ { _MMIO(0x9888), 0x238b2a80 },
	876	+ { _MMIO(0x9888), 0x198c4000 },
	877	+ { _MMIO(0x9888), 0x1b8c0015 },
	878	+ { _MMIO(0x9888), 0x0d8c4000 },
	879	+ { _MMIO(0x9888), 0x0d8da000 },
	880	+ { _MMIO(0x9888), 0x0f8da000 },
	881	+ { _MMIO(0x9888), 0x078d2000 },
	882	+ { _MMIO(0x9888), 0x2185800a },
	883	+ { _MMIO(0x9888), 0x2385002a },
	884	+ { _MMIO(0x9888), 0x1f85aa00 },
	885	+ { _MMIO(0x9888), 0x1b830154 },
	886	+ { _MMIO(0x9888), 0x03834000 },
	887	+ { _MMIO(0x9888), 0x05834000 },
	888	+ { _MMIO(0x9888), 0x07834000 },
	889	+ { _MMIO(0x9888), 0x09834000 },
	890	+ { _MMIO(0x9888), 0x0b834000 },
	891	+ { _MMIO(0x9888), 0x0d834000 },
	892	+ { _MMIO(0x9888), 0x0d84c000 },
	893	+ { _MMIO(0x9888), 0x0f84c000 },
	894	+ { _MMIO(0x9888), 0x01848000 },
	895	+ { _MMIO(0x9888), 0x0384c000 },
	896	+ { _MMIO(0x9888), 0x0584c000 },
	897	+ { _MMIO(0x9888), 0x07844000 },
	898	+ { _MMIO(0x9888), 0x1d80c000 },
	899	+ { _MMIO(0x9888), 0x1f80c000 },
	900	+ { _MMIO(0x9888), 0x11808000 },
	901	+ { _MMIO(0x9888), 0x1380c000 },
	902	+ { _MMIO(0x9888), 0x1580c000 },
	903	+ { _MMIO(0x9888), 0x17804000 },
	904	+ { _MMIO(0x9888), 0x53800000 },
	905	+ { _MMIO(0x9888), 0x45800c00 },
	906	+ { _MMIO(0x9888), 0x47800c63 },
	907	+ { _MMIO(0x9888), 0x21800000 },
	908	+ { _MMIO(0x9888), 0x31800000 },
	909	+ { _MMIO(0x9888), 0x4d800000 },
	910	+ { _MMIO(0x9888), 0x3f8014a5 },
	911	+ { _MMIO(0x9888), 0x4f800000 },
	912	+ { _MMIO(0x9888), 0x41800045 },
	913	+};
	914	+
	915	+static int
	916	+get_l3_3_mux_config(struct drm_i915_private *dev_priv,
	917	+ const struct i915_oa_reg **regs,
	918	+ int *lens)
	919	+{
	920	+ int n = 0;
	921	+
	922	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	923	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	924	+
	925	+ regs[n] = mux_config_l3_3;
	926	+ lens[n] = ARRAY_SIZE(mux_config_l3_3);
	927	+ n++;
	928	+
	929	+ return n;
	930	+}
	931	+
	932	+static const struct i915_oa_reg b_counter_config_l3_4[] = {
	933	+ { _MMIO(0x2740), 0x00000000 },
	934	+ { _MMIO(0x2744), 0x00800000 },
	935	+ { _MMIO(0x2710), 0x00000000 },
	936	+ { _MMIO(0x2714), 0xf0800000 },
	937	+ { _MMIO(0x2720), 0x00000000 },
	938	+ { _MMIO(0x2724), 0xf0800000 },
	939	+ { _MMIO(0x2770), 0x00100070 },
	940	+ { _MMIO(0x2774), 0x0000fff1 },
	941	+ { _MMIO(0x2778), 0x00014002 },
	942	+ { _MMIO(0x277c), 0x0000c3ff },
	943	+ { _MMIO(0x2780), 0x00010002 },
	944	+ { _MMIO(0x2784), 0x0000c7ff },
	945	+ { _MMIO(0x2788), 0x00004002 },
	946	+ { _MMIO(0x278c), 0x0000d3ff },
	947	+ { _MMIO(0x2790), 0x00100700 },
	948	+ { _MMIO(0x2794), 0x0000ff1f },
	949	+ { _MMIO(0x2798), 0x00001402 },
	950	+ { _MMIO(0x279c), 0x0000fc3f },
	951	+ { _MMIO(0x27a0), 0x00001002 },
	952	+ { _MMIO(0x27a4), 0x0000fc7f },
	953	+ { _MMIO(0x27a8), 0x00000402 },
	954	+ { _MMIO(0x27ac), 0x0000fd3f },
	955	+};
	956	+
	957	+static const struct i915_oa_reg flex_eu_config_l3_4[] = {
	958	+ { _MMIO(0xe458), 0x00005004 },
	959	+ { _MMIO(0xe558), 0x00010003 },
	960	+ { _MMIO(0xe658), 0x00012011 },
	961	+ { _MMIO(0xe758), 0x00015014 },
	962	+ { _MMIO(0xe45c), 0x00051050 },
	963	+ { _MMIO(0xe55c), 0x00053052 },
	964	+ { _MMIO(0xe65c), 0x00055054 },
	965	+};
	966	+
	967	+static const struct i915_oa_reg mux_config_l3_4[] = {
	968	+ { _MMIO(0x9888), 0x121a0340 },
	969	+ { _MMIO(0x9888), 0x103f0017 },
	970	+ { _MMIO(0x9888), 0x123f0020 },
	971	+ { _MMIO(0x9888), 0x129a0340 },
	972	+ { _MMIO(0x9888), 0x10bf0017 },
	973	+ { _MMIO(0x9888), 0x12bf0020 },
	974	+ { _MMIO(0x9888), 0x041a3400 },
	975	+ { _MMIO(0x9888), 0x101a0000 },
	976	+ { _MMIO(0x9888), 0x043b8000 },
	977	+ { _MMIO(0x9888), 0x0a3e0010 },
	978	+ { _MMIO(0x9888), 0x003f0200 },
	979	+ { _MMIO(0x9888), 0x023f0113 },
	980	+ { _MMIO(0x9888), 0x043f0014 },
	981	+ { _MMIO(0x9888), 0x02592000 },
	982	+ { _MMIO(0x9888), 0x005a8000 },
	983	+ { _MMIO(0x9888), 0x025ac000 },
	984	+ { _MMIO(0x9888), 0x045a4000 },
	985	+ { _MMIO(0x9888), 0x0a1c8000 },
	986	+ { _MMIO(0x9888), 0x001d8000 },
	987	+ { _MMIO(0x9888), 0x021dc000 },
	988	+ { _MMIO(0x9888), 0x041d4000 },
	989	+ { _MMIO(0x9888), 0x0a1e8000 },
	990	+ { _MMIO(0x9888), 0x0c1fa000 },
	991	+ { _MMIO(0x9888), 0x0e1f001a },
	992	+ { _MMIO(0x9888), 0x00394000 },
	993	+ { _MMIO(0x9888), 0x02395000 },
	994	+ { _MMIO(0x9888), 0x04391000 },
	995	+ { _MMIO(0x9888), 0x069a0034 },
	996	+ { _MMIO(0x9888), 0x109a0000 },
	997	+ { _MMIO(0x9888), 0x06bb4000 },
	998	+ { _MMIO(0x9888), 0x0abe0040 },
	999	+ { _MMIO(0x9888), 0x0cbf0984 },
	1000	+ { _MMIO(0x9888), 0x0ebf0a02 },
	1001	+ { _MMIO(0x9888), 0x02d94000 },
	1002	+ { _MMIO(0x9888), 0x0cdac000 },
	1003	+ { _MMIO(0x9888), 0x0edac000 },
	1004	+ { _MMIO(0x9888), 0x0c9c0400 },
	1005	+ { _MMIO(0x9888), 0x0c9dc000 },
	1006	+ { _MMIO(0x9888), 0x0e9dc000 },
	1007	+ { _MMIO(0x9888), 0x0c9e0400 },
	1008	+ { _MMIO(0x9888), 0x109f02a8 },
	1009	+ { _MMIO(0x9888), 0x0e9f0040 },
	1010	+ { _MMIO(0x9888), 0x0cb95000 },
	1011	+ { _MMIO(0x9888), 0x0eb95000 },
	1012	+ { _MMIO(0x9888), 0x0f88000f },
	1013	+ { _MMIO(0x9888), 0x0d880400 },
	1014	+ { _MMIO(0x9888), 0x038a8000 },
	1015	+ { _MMIO(0x9888), 0x058a8000 },
	1016	+ { _MMIO(0x9888), 0x078a8000 },
	1017	+ { _MMIO(0x9888), 0x098a8000 },
	1018	+ { _MMIO(0x9888), 0x0b8a8000 },
	1019	+ { _MMIO(0x9888), 0x258b8009 },
	1020	+ { _MMIO(0x9888), 0x278b002a },
	1021	+ { _MMIO(0x9888), 0x238b2a80 },
	1022	+ { _MMIO(0x9888), 0x198c4000 },
	1023	+ { _MMIO(0x9888), 0x1b8c0015 },
	1024	+ { _MMIO(0x9888), 0x0d8c4000 },
	1025	+ { _MMIO(0x9888), 0x0d8da000 },
	1026	+ { _MMIO(0x9888), 0x0f8da000 },
	1027	+ { _MMIO(0x9888), 0x078d2000 },
	1028	+ { _MMIO(0x9888), 0x2185800a },
	1029	+ { _MMIO(0x9888), 0x2385002a },
	1030	+ { _MMIO(0x9888), 0x1f85aa00 },
	1031	+ { _MMIO(0x9888), 0x1b830154 },
	1032	+ { _MMIO(0x9888), 0x03834000 },
	1033	+ { _MMIO(0x9888), 0x05834000 },
	1034	+ { _MMIO(0x9888), 0x07834000 },
	1035	+ { _MMIO(0x9888), 0x09834000 },
	1036	+ { _MMIO(0x9888), 0x0b834000 },
	1037	+ { _MMIO(0x9888), 0x0d834000 },
	1038	+ { _MMIO(0x9888), 0x0d84c000 },
	1039	+ { _MMIO(0x9888), 0x0f84c000 },
	1040	+ { _MMIO(0x9888), 0x01848000 },
	1041	+ { _MMIO(0x9888), 0x0384c000 },
	1042	+ { _MMIO(0x9888), 0x0584c000 },
	1043	+ { _MMIO(0x9888), 0x07844000 },
	1044	+ { _MMIO(0x9888), 0x1d80c000 },
	1045	+ { _MMIO(0x9888), 0x1f80c000 },
	1046	+ { _MMIO(0x9888), 0x11808000 },
	1047	+ { _MMIO(0x9888), 0x1380c000 },
	1048	+ { _MMIO(0x9888), 0x1580c000 },
	1049	+ { _MMIO(0x9888), 0x17804000 },
	1050	+ { _MMIO(0x9888), 0x53800000 },
	1051	+ { _MMIO(0x9888), 0x45800800 },
	1052	+ { _MMIO(0x9888), 0x47800842 },
	1053	+ { _MMIO(0x9888), 0x21800000 },
	1054	+ { _MMIO(0x9888), 0x31800000 },
	1055	+ { _MMIO(0x9888), 0x4d800000 },
	1056	+ { _MMIO(0x9888), 0x3f801084 },
	1057	+ { _MMIO(0x9888), 0x4f800000 },
	1058	+ { _MMIO(0x9888), 0x41800044 },
	1059	+};
	1060	+
	1061	+static int
	1062	+get_l3_4_mux_config(struct drm_i915_private *dev_priv,
	1063	+ const struct i915_oa_reg **regs,
	1064	+ int *lens)
	1065	+{
	1066	+ int n = 0;
	1067	+
	1068	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1069	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1070	+
	1071	+ regs[n] = mux_config_l3_4;
	1072	+ lens[n] = ARRAY_SIZE(mux_config_l3_4);
	1073	+ n++;
	1074	+
	1075	+ return n;
	1076	+}
	1077	+
	1078	+static const struct i915_oa_reg b_counter_config_rasterizer_and_pixel_backend[] = {
	1079	+ { _MMIO(0x2740), 0x00000000 },
	1080	+ { _MMIO(0x2744), 0x00800000 },
	1081	+ { _MMIO(0x2710), 0x00000000 },
	1082	+ { _MMIO(0x2714), 0xf0800000 },
	1083	+ { _MMIO(0x2720), 0x00000000 },
	1084	+ { _MMIO(0x2724), 0x30800000 },
	1085	+ { _MMIO(0x2770), 0x00006000 },
	1086	+ { _MMIO(0x2774), 0x0000f3ff },
	1087	+ { _MMIO(0x2778), 0x00001800 },
	1088	+ { _MMIO(0x277c), 0x0000fcff },
	1089	+ { _MMIO(0x2780), 0x00000600 },
	1090	+ { _MMIO(0x2784), 0x0000ff3f },
	1091	+ { _MMIO(0x2788), 0x00000180 },
	1092	+ { _MMIO(0x278c), 0x0000ffcf },
	1093	+ { _MMIO(0x2790), 0x00000060 },
	1094	+ { _MMIO(0x2794), 0x0000fff3 },
	1095	+ { _MMIO(0x2798), 0x00000018 },
	1096	+ { _MMIO(0x279c), 0x0000fffc },
	1097	+};
	1098	+
	1099	+static const struct i915_oa_reg flex_eu_config_rasterizer_and_pixel_backend[] = {
	1100	+ { _MMIO(0xe458), 0x00005004 },
	1101	+ { _MMIO(0xe558), 0x00010003 },
	1102	+ { _MMIO(0xe658), 0x00012011 },
	1103	+ { _MMIO(0xe758), 0x00015014 },
	1104	+ { _MMIO(0xe45c), 0x00051050 },
	1105	+ { _MMIO(0xe55c), 0x00053052 },
	1106	+ { _MMIO(0xe65c), 0x00055054 },
	1107	+};
	1108	+
	1109	+static const struct i915_oa_reg mux_config_rasterizer_and_pixel_backend[] = {
	1110	+ { _MMIO(0x9888), 0x143b000e },
	1111	+ { _MMIO(0x9888), 0x043c55c0 },
	1112	+ { _MMIO(0x9888), 0x0a1e0280 },
	1113	+ { _MMIO(0x9888), 0x0c1e0408 },
	1114	+ { _MMIO(0x9888), 0x10390000 },
	1115	+ { _MMIO(0x9888), 0x12397a1f },
	1116	+ { _MMIO(0x9888), 0x14bb000e },
	1117	+ { _MMIO(0x9888), 0x04bc5000 },
	1118	+ { _MMIO(0x9888), 0x0a9e0296 },
	1119	+ { _MMIO(0x9888), 0x0c9e0008 },
	1120	+ { _MMIO(0x9888), 0x10b90000 },
	1121	+ { _MMIO(0x9888), 0x12b97a1f },
	1122	+ { _MMIO(0x9888), 0x063b0042 },
	1123	+ { _MMIO(0x9888), 0x103b0000 },
	1124	+ { _MMIO(0x9888), 0x083c0000 },
	1125	+ { _MMIO(0x9888), 0x0a3e0040 },
	1126	+ { _MMIO(0x9888), 0x043f8000 },
	1127	+ { _MMIO(0x9888), 0x02594000 },
	1128	+ { _MMIO(0x9888), 0x045a8000 },
	1129	+ { _MMIO(0x9888), 0x0c1c0400 },
	1130	+ { _MMIO(0x9888), 0x041d8000 },
	1131	+ { _MMIO(0x9888), 0x081e02c0 },
	1132	+ { _MMIO(0x9888), 0x0e1e0000 },
	1133	+ { _MMIO(0x9888), 0x0c1fa800 },
	1134	+ { _MMIO(0x9888), 0x0e1f0260 },
	1135	+ { _MMIO(0x9888), 0x101f0014 },
	1136	+ { _MMIO(0x9888), 0x003905e0 },
	1137	+ { _MMIO(0x9888), 0x06390bc0 },
	1138	+ { _MMIO(0x9888), 0x02390018 },
	1139	+ { _MMIO(0x9888), 0x04394000 },
	1140	+ { _MMIO(0x9888), 0x04bb0042 },
	1141	+ { _MMIO(0x9888), 0x10bb0000 },
	1142	+ { _MMIO(0x9888), 0x02bc05c0 },
	1143	+ { _MMIO(0x9888), 0x08bc0000 },
	1144	+ { _MMIO(0x9888), 0x0abe0004 },
	1145	+ { _MMIO(0x9888), 0x02bf8000 },
	1146	+ { _MMIO(0x9888), 0x02d91000 },
	1147	+ { _MMIO(0x9888), 0x02da8000 },
	1148	+ { _MMIO(0x9888), 0x089c8000 },
	1149	+ { _MMIO(0x9888), 0x029d8000 },
	1150	+ { _MMIO(0x9888), 0x089e8000 },
	1151	+ { _MMIO(0x9888), 0x0e9e0000 },
	1152	+ { _MMIO(0x9888), 0x0e9fa806 },
	1153	+ { _MMIO(0x9888), 0x109f0142 },
	1154	+ { _MMIO(0x9888), 0x08b90617 },
	1155	+ { _MMIO(0x9888), 0x0ab90be0 },
	1156	+ { _MMIO(0x9888), 0x02b94000 },
	1157	+ { _MMIO(0x9888), 0x0d88f000 },
	1158	+ { _MMIO(0x9888), 0x0f88000c },
	1159	+ { _MMIO(0x9888), 0x07888000 },
	1160	+ { _MMIO(0x9888), 0x09888000 },
	1161	+ { _MMIO(0x9888), 0x018a8000 },
	1162	+ { _MMIO(0x9888), 0x0f8a8000 },
	1163	+ { _MMIO(0x9888), 0x1b8a2800 },
	1164	+ { _MMIO(0x9888), 0x038a8000 },
	1165	+ { _MMIO(0x9888), 0x058a8000 },
	1166	+ { _MMIO(0x9888), 0x0b8a8000 },
	1167	+ { _MMIO(0x9888), 0x0d8a8000 },
	1168	+ { _MMIO(0x9888), 0x238b52a0 },
	1169	+ { _MMIO(0x9888), 0x258b6a95 },
	1170	+ { _MMIO(0x9888), 0x278b0029 },
	1171	+ { _MMIO(0x9888), 0x178c2000 },
	1172	+ { _MMIO(0x9888), 0x198c1500 },
	1173	+ { _MMIO(0x9888), 0x1b8c0014 },
	1174	+ { _MMIO(0x9888), 0x078c4000 },
	1175	+ { _MMIO(0x9888), 0x098c4000 },
	1176	+ { _MMIO(0x9888), 0x098da000 },
	1177	+ { _MMIO(0x9888), 0x0b8da000 },
	1178	+ { _MMIO(0x9888), 0x0f8da000 },
	1179	+ { _MMIO(0x9888), 0x038d8000 },
	1180	+ { _MMIO(0x9888), 0x058d2000 },
	1181	+ { _MMIO(0x9888), 0x1f85aa80 },
	1182	+ { _MMIO(0x9888), 0x2185aaaa },
	1183	+ { _MMIO(0x9888), 0x2385002a },
	1184	+ { _MMIO(0x9888), 0x01834000 },
	1185	+ { _MMIO(0x9888), 0x0f834000 },
	1186	+ { _MMIO(0x9888), 0x19835400 },
	1187	+ { _MMIO(0x9888), 0x1b830155 },
	1188	+ { _MMIO(0x9888), 0x03834000 },
	1189	+ { _MMIO(0x9888), 0x05834000 },
	1190	+ { _MMIO(0x9888), 0x07834000 },
	1191	+ { _MMIO(0x9888), 0x09834000 },
	1192	+ { _MMIO(0x9888), 0x0b834000 },
	1193	+ { _MMIO(0x9888), 0x0d834000 },
	1194	+ { _MMIO(0x9888), 0x0184c000 },
	1195	+ { _MMIO(0x9888), 0x0784c000 },
	1196	+ { _MMIO(0x9888), 0x0984c000 },
	1197	+ { _MMIO(0x9888), 0x0b84c000 },
	1198	+ { _MMIO(0x9888), 0x0d84c000 },
	1199	+ { _MMIO(0x9888), 0x0f84c000 },
	1200	+ { _MMIO(0x9888), 0x0384c000 },
	1201	+ { _MMIO(0x9888), 0x0584c000 },
	1202	+ { _MMIO(0x9888), 0x1180c000 },
	1203	+ { _MMIO(0x9888), 0x1780c000 },
	1204	+ { _MMIO(0x9888), 0x1980c000 },
	1205	+ { _MMIO(0x9888), 0x1b80c000 },
	1206	+ { _MMIO(0x9888), 0x1d80c000 },
	1207	+ { _MMIO(0x9888), 0x1f80c000 },
	1208	+ { _MMIO(0x9888), 0x1380c000 },
	1209	+ { _MMIO(0x9888), 0x1580c000 },
	1210	+ { _MMIO(0x9888), 0x4d800444 },
	1211	+ { _MMIO(0x9888), 0x3d800000 },
	1212	+ { _MMIO(0x9888), 0x4f804000 },
	1213	+ { _MMIO(0x9888), 0x43801080 },
	1214	+ { _MMIO(0x9888), 0x51800000 },
	1215	+ { _MMIO(0x9888), 0x45800084 },
	1216	+ { _MMIO(0x9888), 0x53800044 },
	1217	+ { _MMIO(0x9888), 0x47801080 },
	1218	+ { _MMIO(0x9888), 0x21800000 },
	1219	+ { _MMIO(0x9888), 0x31800000 },
	1220	+ { _MMIO(0x9888), 0x3f800000 },
	1221	+ { _MMIO(0x9888), 0x41800840 },
	1222	+};
	1223	+
	1224	+static int
	1225	+get_rasterizer_and_pixel_backend_mux_config(struct drm_i915_private *dev_priv,
	1226	+ const struct i915_oa_reg **regs,
	1227	+ int *lens)
	1228	+{
	1229	+ int n = 0;
	1230	+
	1231	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1232	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1233	+
	1234	+ regs[n] = mux_config_rasterizer_and_pixel_backend;
	1235	+ lens[n] = ARRAY_SIZE(mux_config_rasterizer_and_pixel_backend);
	1236	+ n++;
	1237	+
	1238	+ return n;
	1239	+}
	1240	+
	1241	+static const struct i915_oa_reg b_counter_config_sampler_1[] = {
	1242	+ { _MMIO(0x2740), 0x00000000 },
	1243	+ { _MMIO(0x2744), 0x00800000 },
	1244	+ { _MMIO(0x2710), 0x00000000 },
	1245	+ { _MMIO(0x2714), 0x70800000 },
	1246	+ { _MMIO(0x2720), 0x00000000 },
	1247	+ { _MMIO(0x2724), 0x00800000 },
	1248	+ { _MMIO(0x2770), 0x0000c000 },
	1249	+ { _MMIO(0x2774), 0x0000e7ff },
	1250	+ { _MMIO(0x2778), 0x00003000 },
	1251	+ { _MMIO(0x277c), 0x0000f9ff },
	1252	+ { _MMIO(0x2780), 0x00000c00 },
	1253	+ { _MMIO(0x2784), 0x0000fe7f },
	1254	+};
	1255	+
	1256	+static const struct i915_oa_reg flex_eu_config_sampler_1[] = {
	1257	+ { _MMIO(0xe458), 0x00005004 },
	1258	+ { _MMIO(0xe558), 0x00010003 },
	1259	+ { _MMIO(0xe658), 0x00012011 },
	1260	+ { _MMIO(0xe758), 0x00015014 },
	1261	+ { _MMIO(0xe45c), 0x00051050 },
	1262	+ { _MMIO(0xe55c), 0x00053052 },
	1263	+ { _MMIO(0xe65c), 0x00055054 },
	1264	+};
	1265	+
	1266	+static const struct i915_oa_reg mux_config_sampler_1[] = {
	1267	+ { _MMIO(0x9888), 0x18921400 },
	1268	+ { _MMIO(0x9888), 0x149500ab },
	1269	+ { _MMIO(0x9888), 0x18b21400 },
	1270	+ { _MMIO(0x9888), 0x14b500ab },
	1271	+ { _MMIO(0x9888), 0x18d21400 },
	1272	+ { _MMIO(0x9888), 0x14d500ab },
	1273	+ { _MMIO(0x9888), 0x0cdc8000 },
	1274	+ { _MMIO(0x9888), 0x0edc4000 },
	1275	+ { _MMIO(0x9888), 0x02dcc000 },
	1276	+ { _MMIO(0x9888), 0x04dcc000 },
	1277	+ { _MMIO(0x9888), 0x1abd00a0 },
	1278	+ { _MMIO(0x9888), 0x0abd8000 },
	1279	+ { _MMIO(0x9888), 0x0cd88000 },
	1280	+ { _MMIO(0x9888), 0x0ed84000 },
	1281	+ { _MMIO(0x9888), 0x04d88000 },
	1282	+ { _MMIO(0x9888), 0x1adb0050 },
	1283	+ { _MMIO(0x9888), 0x04db8000 },
	1284	+ { _MMIO(0x9888), 0x06db8000 },
	1285	+ { _MMIO(0x9888), 0x08db8000 },
	1286	+ { _MMIO(0x9888), 0x0adb4000 },
	1287	+ { _MMIO(0x9888), 0x109f02a0 },
	1288	+ { _MMIO(0x9888), 0x0c9fa000 },
	1289	+ { _MMIO(0x9888), 0x0e9f00aa },
	1290	+ { _MMIO(0x9888), 0x18b82500 },
	1291	+ { _MMIO(0x9888), 0x02b88000 },
	1292	+ { _MMIO(0x9888), 0x04b84000 },
	1293	+ { _MMIO(0x9888), 0x06b84000 },
	1294	+ { _MMIO(0x9888), 0x08b84000 },
	1295	+ { _MMIO(0x9888), 0x0ab84000 },
	1296	+ { _MMIO(0x9888), 0x0cb88000 },
	1297	+ { _MMIO(0x9888), 0x0cb98000 },
	1298	+ { _MMIO(0x9888), 0x0eb9a000 },
	1299	+ { _MMIO(0x9888), 0x00b98000 },
	1300	+ { _MMIO(0x9888), 0x02b9a000 },
	1301	+ { _MMIO(0x9888), 0x04b9a000 },
	1302	+ { _MMIO(0x9888), 0x06b92000 },
	1303	+ { _MMIO(0x9888), 0x1aba0200 },
	1304	+ { _MMIO(0x9888), 0x02ba8000 },
	1305	+ { _MMIO(0x9888), 0x0cba8000 },
	1306	+ { _MMIO(0x9888), 0x04908000 },
	1307	+ { _MMIO(0x9888), 0x04918000 },
	1308	+ { _MMIO(0x9888), 0x04927300 },
	1309	+ { _MMIO(0x9888), 0x10920000 },
	1310	+ { _MMIO(0x9888), 0x1893000a },
	1311	+ { _MMIO(0x9888), 0x0a934000 },
	1312	+ { _MMIO(0x9888), 0x0a946000 },
	1313	+ { _MMIO(0x9888), 0x0c959000 },
	1314	+ { _MMIO(0x9888), 0x0e950098 },
	1315	+ { _MMIO(0x9888), 0x10950000 },
	1316	+ { _MMIO(0x9888), 0x04b04000 },
	1317	+ { _MMIO(0x9888), 0x04b14000 },
	1318	+ { _MMIO(0x9888), 0x04b20073 },
	1319	+ { _MMIO(0x9888), 0x10b20000 },
	1320	+ { _MMIO(0x9888), 0x04b38000 },
	1321	+ { _MMIO(0x9888), 0x06b38000 },
	1322	+ { _MMIO(0x9888), 0x08b34000 },
	1323	+ { _MMIO(0x9888), 0x04b4c000 },
	1324	+ { _MMIO(0x9888), 0x02b59890 },
	1325	+ { _MMIO(0x9888), 0x10b50000 },
	1326	+ { _MMIO(0x9888), 0x06d04000 },
	1327	+ { _MMIO(0x9888), 0x06d14000 },
	1328	+ { _MMIO(0x9888), 0x06d20073 },
	1329	+ { _MMIO(0x9888), 0x10d20000 },
	1330	+ { _MMIO(0x9888), 0x18d30020 },
	1331	+ { _MMIO(0x9888), 0x02d38000 },
	1332	+ { _MMIO(0x9888), 0x0cd34000 },
	1333	+ { _MMIO(0x9888), 0x0ad48000 },
	1334	+ { _MMIO(0x9888), 0x04d42000 },
	1335	+ { _MMIO(0x9888), 0x0ed59000 },
	1336	+ { _MMIO(0x9888), 0x00d59800 },
	1337	+ { _MMIO(0x9888), 0x10d50000 },
	1338	+ { _MMIO(0x9888), 0x0f88000e },
	1339	+ { _MMIO(0x9888), 0x03888000 },
	1340	+ { _MMIO(0x9888), 0x05888000 },
	1341	+ { _MMIO(0x9888), 0x07888000 },
	1342	+ { _MMIO(0x9888), 0x09888000 },
	1343	+ { _MMIO(0x9888), 0x0b888000 },
	1344	+ { _MMIO(0x9888), 0x0d880400 },
	1345	+ { _MMIO(0x9888), 0x278b002a },
	1346	+ { _MMIO(0x9888), 0x238b5500 },
	1347	+ { _MMIO(0x9888), 0x258b000a },
	1348	+ { _MMIO(0x9888), 0x1b8c0015 },
	1349	+ { _MMIO(0x9888), 0x038c4000 },
	1350	+ { _MMIO(0x9888), 0x058c4000 },
	1351	+ { _MMIO(0x9888), 0x078c4000 },
	1352	+ { _MMIO(0x9888), 0x098c4000 },
	1353	+ { _MMIO(0x9888), 0x0b8c4000 },
	1354	+ { _MMIO(0x9888), 0x0d8c4000 },
	1355	+ { _MMIO(0x9888), 0x0d8d8000 },
	1356	+ { _MMIO(0x9888), 0x0f8da000 },
	1357	+ { _MMIO(0x9888), 0x018d8000 },
	1358	+ { _MMIO(0x9888), 0x038da000 },
	1359	+ { _MMIO(0x9888), 0x058da000 },
	1360	+ { _MMIO(0x9888), 0x078d2000 },
	1361	+ { _MMIO(0x9888), 0x2385002a },
	1362	+ { _MMIO(0x9888), 0x1f85aa00 },
	1363	+ { _MMIO(0x9888), 0x2185000a },
	1364	+ { _MMIO(0x9888), 0x1b830150 },
	1365	+ { _MMIO(0x9888), 0x03834000 },
	1366	+ { _MMIO(0x9888), 0x05834000 },
	1367	+ { _MMIO(0x9888), 0x07834000 },
	1368	+ { _MMIO(0x9888), 0x09834000 },
	1369	+ { _MMIO(0x9888), 0x0b834000 },
	1370	+ { _MMIO(0x9888), 0x0d834000 },
	1371	+ { _MMIO(0x9888), 0x0d848000 },
	1372	+ { _MMIO(0x9888), 0x0f84c000 },
	1373	+ { _MMIO(0x9888), 0x01848000 },
	1374	+ { _MMIO(0x9888), 0x0384c000 },
	1375	+ { _MMIO(0x9888), 0x0584c000 },
	1376	+ { _MMIO(0x9888), 0x07844000 },
	1377	+ { _MMIO(0x9888), 0x1d808000 },
	1378	+ { _MMIO(0x9888), 0x1f80c000 },
	1379	+ { _MMIO(0x9888), 0x11808000 },
	1380	+ { _MMIO(0x9888), 0x1380c000 },
	1381	+ { _MMIO(0x9888), 0x1580c000 },
	1382	+ { _MMIO(0x9888), 0x17804000 },
	1383	+ { _MMIO(0x9888), 0x53800000 },
	1384	+ { _MMIO(0x9888), 0x47801021 },
	1385	+ { _MMIO(0x9888), 0x21800000 },
	1386	+ { _MMIO(0x9888), 0x31800000 },
	1387	+ { _MMIO(0x9888), 0x4d800000 },
	1388	+ { _MMIO(0x9888), 0x3f800c64 },
	1389	+ { _MMIO(0x9888), 0x4f800000 },
	1390	+ { _MMIO(0x9888), 0x41800c02 },
	1391	+};
	1392	+
	1393	+static int
	1394	+get_sampler_1_mux_config(struct drm_i915_private *dev_priv,
	1395	+ const struct i915_oa_reg **regs,
	1396	+ int *lens)
	1397	+{
	1398	+ int n = 0;
	1399	+
	1400	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1401	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1402	+
	1403	+ regs[n] = mux_config_sampler_1;
	1404	+ lens[n] = ARRAY_SIZE(mux_config_sampler_1);
	1405	+ n++;
	1406	+
	1407	+ return n;
	1408	+}
	1409	+
	1410	+static const struct i915_oa_reg b_counter_config_sampler_2[] = {
	1411	+ { _MMIO(0x2740), 0x00000000 },
	1412	+ { _MMIO(0x2744), 0x00800000 },
	1413	+ { _MMIO(0x2710), 0x00000000 },
	1414	+ { _MMIO(0x2714), 0x70800000 },
	1415	+ { _MMIO(0x2720), 0x00000000 },
	1416	+ { _MMIO(0x2724), 0x00800000 },
	1417	+ { _MMIO(0x2770), 0x0000c000 },
	1418	+ { _MMIO(0x2774), 0x0000e7ff },
	1419	+ { _MMIO(0x2778), 0x00003000 },
	1420	+ { _MMIO(0x277c), 0x0000f9ff },
	1421	+ { _MMIO(0x2780), 0x00000c00 },
	1422	+ { _MMIO(0x2784), 0x0000fe7f },
	1423	+};
	1424	+
	1425	+static const struct i915_oa_reg flex_eu_config_sampler_2[] = {
	1426	+ { _MMIO(0xe458), 0x00005004 },
	1427	+ { _MMIO(0xe558), 0x00010003 },
	1428	+ { _MMIO(0xe658), 0x00012011 },
	1429	+ { _MMIO(0xe758), 0x00015014 },
	1430	+ { _MMIO(0xe45c), 0x00051050 },
	1431	+ { _MMIO(0xe55c), 0x00053052 },
	1432	+ { _MMIO(0xe65c), 0x00055054 },
	1433	+};
	1434	+
	1435	+static const struct i915_oa_reg mux_config_sampler_2[] = {
	1436	+ { _MMIO(0x9888), 0x18121400 },
	1437	+ { _MMIO(0x9888), 0x141500ab },
	1438	+ { _MMIO(0x9888), 0x18321400 },
	1439	+ { _MMIO(0x9888), 0x143500ab },
	1440	+ { _MMIO(0x9888), 0x18521400 },
	1441	+ { _MMIO(0x9888), 0x145500ab },
	1442	+ { _MMIO(0x9888), 0x0c5c8000 },
	1443	+ { _MMIO(0x9888), 0x0e5c4000 },
	1444	+ { _MMIO(0x9888), 0x025cc000 },
	1445	+ { _MMIO(0x9888), 0x045cc000 },
	1446	+ { _MMIO(0x9888), 0x1a3d00a0 },
	1447	+ { _MMIO(0x9888), 0x0a3d8000 },
	1448	+ { _MMIO(0x9888), 0x0c588000 },
	1449	+ { _MMIO(0x9888), 0x0e584000 },
	1450	+ { _MMIO(0x9888), 0x04588000 },
	1451	+ { _MMIO(0x9888), 0x1a5b0050 },
	1452	+ { _MMIO(0x9888), 0x045b8000 },
	1453	+ { _MMIO(0x9888), 0x065b8000 },
	1454	+ { _MMIO(0x9888), 0x085b8000 },
	1455	+ { _MMIO(0x9888), 0x0a5b4000 },
	1456	+ { _MMIO(0x9888), 0x101f02a0 },
	1457	+ { _MMIO(0x9888), 0x0c1fa000 },
	1458	+ { _MMIO(0x9888), 0x0e1f00aa },
	1459	+ { _MMIO(0x9888), 0x18382500 },
	1460	+ { _MMIO(0x9888), 0x02388000 },
	1461	+ { _MMIO(0x9888), 0x04384000 },
	1462	+ { _MMIO(0x9888), 0x06384000 },
	1463	+ { _MMIO(0x9888), 0x08384000 },
	1464	+ { _MMIO(0x9888), 0x0a384000 },
	1465	+ { _MMIO(0x9888), 0x0c388000 },
	1466	+ { _MMIO(0x9888), 0x0c398000 },
	1467	+ { _MMIO(0x9888), 0x0e39a000 },
	1468	+ { _MMIO(0x9888), 0x00398000 },
	1469	+ { _MMIO(0x9888), 0x0239a000 },
	1470	+ { _MMIO(0x9888), 0x0439a000 },
	1471	+ { _MMIO(0x9888), 0x06392000 },
	1472	+ { _MMIO(0x9888), 0x1a3a0200 },
	1473	+ { _MMIO(0x9888), 0x023a8000 },
	1474	+ { _MMIO(0x9888), 0x0c3a8000 },
	1475	+ { _MMIO(0x9888), 0x04108000 },
	1476	+ { _MMIO(0x9888), 0x04118000 },
	1477	+ { _MMIO(0x9888), 0x04127300 },
	1478	+ { _MMIO(0x9888), 0x10120000 },
	1479	+ { _MMIO(0x9888), 0x1813000a },
	1480	+ { _MMIO(0x9888), 0x0a134000 },
	1481	+ { _MMIO(0x9888), 0x0a146000 },
	1482	+ { _MMIO(0x9888), 0x0c159000 },
	1483	+ { _MMIO(0x9888), 0x0e150098 },
	1484	+ { _MMIO(0x9888), 0x10150000 },
	1485	+ { _MMIO(0x9888), 0x04304000 },
	1486	+ { _MMIO(0x9888), 0x04314000 },
	1487	+ { _MMIO(0x9888), 0x04320073 },
	1488	+ { _MMIO(0x9888), 0x10320000 },
	1489	+ { _MMIO(0x9888), 0x04338000 },
	1490	+ { _MMIO(0x9888), 0x06338000 },
	1491	+ { _MMIO(0x9888), 0x08334000 },
	1492	+ { _MMIO(0x9888), 0x0434c000 },
	1493	+ { _MMIO(0x9888), 0x02359890 },
	1494	+ { _MMIO(0x9888), 0x10350000 },
	1495	+ { _MMIO(0x9888), 0x06504000 },
	1496	+ { _MMIO(0x9888), 0x06514000 },
	1497	+ { _MMIO(0x9888), 0x06520073 },
	1498	+ { _MMIO(0x9888), 0x10520000 },
	1499	+ { _MMIO(0x9888), 0x18530020 },
	1500	+ { _MMIO(0x9888), 0x02538000 },
	1501	+ { _MMIO(0x9888), 0x0c534000 },
	1502	+ { _MMIO(0x9888), 0x0a548000 },
	1503	+ { _MMIO(0x9888), 0x04542000 },
	1504	+ { _MMIO(0x9888), 0x0e559000 },
	1505	+ { _MMIO(0x9888), 0x00559800 },
	1506	+ { _MMIO(0x9888), 0x10550000 },
	1507	+ { _MMIO(0x9888), 0x1b8aa000 },
	1508	+ { _MMIO(0x9888), 0x1d8a0002 },
	1509	+ { _MMIO(0x9888), 0x038a8000 },
	1510	+ { _MMIO(0x9888), 0x058a8000 },
	1511	+ { _MMIO(0x9888), 0x078a8000 },
	1512	+ { _MMIO(0x9888), 0x098a8000 },
	1513	+ { _MMIO(0x9888), 0x0b8a8000 },
	1514	+ { _MMIO(0x9888), 0x0d8a8000 },
	1515	+ { _MMIO(0x9888), 0x278b0015 },
	1516	+ { _MMIO(0x9888), 0x238b2a80 },
	1517	+ { _MMIO(0x9888), 0x258b0005 },
	1518	+ { _MMIO(0x9888), 0x2385002a },
	1519	+ { _MMIO(0x9888), 0x1f85aa00 },
	1520	+ { _MMIO(0x9888), 0x2185000a },
	1521	+ { _MMIO(0x9888), 0x1b830150 },
	1522	+ { _MMIO(0x9888), 0x03834000 },
	1523	+ { _MMIO(0x9888), 0x05834000 },
	1524	+ { _MMIO(0x9888), 0x07834000 },
	1525	+ { _MMIO(0x9888), 0x09834000 },
	1526	+ { _MMIO(0x9888), 0x0b834000 },
	1527	+ { _MMIO(0x9888), 0x0d834000 },
	1528	+ { _MMIO(0x9888), 0x0d848000 },
	1529	+ { _MMIO(0x9888), 0x0f84c000 },
	1530	+ { _MMIO(0x9888), 0x01848000 },
	1531	+ { _MMIO(0x9888), 0x0384c000 },
	1532	+ { _MMIO(0x9888), 0x0584c000 },
	1533	+ { _MMIO(0x9888), 0x07844000 },
	1534	+ { _MMIO(0x9888), 0x1d808000 },
	1535	+ { _MMIO(0x9888), 0x1f80c000 },
	1536	+ { _MMIO(0x9888), 0x11808000 },
	1537	+ { _MMIO(0x9888), 0x1380c000 },
	1538	+ { _MMIO(0x9888), 0x1580c000 },
	1539	+ { _MMIO(0x9888), 0x17804000 },
	1540	+ { _MMIO(0x9888), 0x53800000 },
	1541	+ { _MMIO(0x9888), 0x47801021 },
	1542	+ { _MMIO(0x9888), 0x21800000 },
	1543	+ { _MMIO(0x9888), 0x31800000 },
	1544	+ { _MMIO(0x9888), 0x4d800000 },
	1545	+ { _MMIO(0x9888), 0x3f800c64 },
	1546	+ { _MMIO(0x9888), 0x4f800000 },
	1547	+ { _MMIO(0x9888), 0x41800c02 },
	1548	+};
	1549	+
	1550	+static int
	1551	+get_sampler_2_mux_config(struct drm_i915_private *dev_priv,
	1552	+ const struct i915_oa_reg **regs,
	1553	+ int *lens)
	1554	+{
	1555	+ int n = 0;
	1556	+
	1557	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1558	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1559	+
	1560	+ regs[n] = mux_config_sampler_2;
	1561	+ lens[n] = ARRAY_SIZE(mux_config_sampler_2);
	1562	+ n++;
	1563	+
	1564	+ return n;
	1565	+}
	1566	+
	1567	+static const struct i915_oa_reg b_counter_config_tdl_1[] = {
	1568	+ { _MMIO(0x2740), 0x00000000 },
	1569	+ { _MMIO(0x2744), 0x00800000 },
	1570	+ { _MMIO(0x2710), 0x00000000 },
	1571	+ { _MMIO(0x2714), 0xf0800000 },
	1572	+ { _MMIO(0x2720), 0x00000000 },
	1573	+ { _MMIO(0x2724), 0x30800000 },
	1574	+ { _MMIO(0x2770), 0x00000002 },
	1575	+ { _MMIO(0x2774), 0x0000fdff },
	1576	+ { _MMIO(0x2778), 0x00000000 },
	1577	+ { _MMIO(0x277c), 0x0000fe7f },
	1578	+ { _MMIO(0x2780), 0x00000002 },
	1579	+ { _MMIO(0x2784), 0x0000ffbf },
	1580	+ { _MMIO(0x2788), 0x00000000 },
	1581	+ { _MMIO(0x278c), 0x0000ffcf },
	1582	+ { _MMIO(0x2790), 0x00000002 },
	1583	+ { _MMIO(0x2794), 0x0000fff7 },
	1584	+ { _MMIO(0x2798), 0x00000000 },
	1585	+ { _MMIO(0x279c), 0x0000fff9 },
	1586	+};
	1587	+
	1588	+static const struct i915_oa_reg flex_eu_config_tdl_1[] = {
	1589	+ { _MMIO(0xe458), 0x00005004 },
	1590	+ { _MMIO(0xe558), 0x00010003 },
	1591	+ { _MMIO(0xe658), 0x00012011 },
	1592	+ { _MMIO(0xe758), 0x00015014 },
	1593	+ { _MMIO(0xe45c), 0x00051050 },
	1594	+ { _MMIO(0xe55c), 0x00053052 },
	1595	+ { _MMIO(0xe65c), 0x00055054 },
	1596	+};
	1597	+
	1598	+static const struct i915_oa_reg mux_config_tdl_1[] = {
	1599	+ { _MMIO(0x9888), 0x16154d60 },
	1600	+ { _MMIO(0x9888), 0x16352e60 },
	1601	+ { _MMIO(0x9888), 0x16554d60 },
	1602	+ { _MMIO(0x9888), 0x16950000 },
	1603	+ { _MMIO(0x9888), 0x16b50000 },
	1604	+ { _MMIO(0x9888), 0x16d50000 },
	1605	+ { _MMIO(0x9888), 0x005c8000 },
	1606	+ { _MMIO(0x9888), 0x045cc000 },
	1607	+ { _MMIO(0x9888), 0x065c4000 },
	1608	+ { _MMIO(0x9888), 0x083d8000 },
	1609	+ { _MMIO(0x9888), 0x0a3d8000 },
	1610	+ { _MMIO(0x9888), 0x0458c000 },
	1611	+ { _MMIO(0x9888), 0x025b8000 },
	1612	+ { _MMIO(0x9888), 0x085b4000 },
	1613	+ { _MMIO(0x9888), 0x0a5b4000 },
	1614	+ { _MMIO(0x9888), 0x0c5b8000 },
	1615	+ { _MMIO(0x9888), 0x0c1fa000 },
	1616	+ { _MMIO(0x9888), 0x0e1f00aa },
	1617	+ { _MMIO(0x9888), 0x02384000 },
	1618	+ { _MMIO(0x9888), 0x04388000 },
	1619	+ { _MMIO(0x9888), 0x06388000 },
	1620	+ { _MMIO(0x9888), 0x08384000 },
	1621	+ { _MMIO(0x9888), 0x0a384000 },
	1622	+ { _MMIO(0x9888), 0x0c384000 },
	1623	+ { _MMIO(0x9888), 0x00398000 },
	1624	+ { _MMIO(0x9888), 0x0239a000 },
	1625	+ { _MMIO(0x9888), 0x0439a000 },
	1626	+ { _MMIO(0x9888), 0x06392000 },
	1627	+ { _MMIO(0x9888), 0x043a8000 },
	1628	+ { _MMIO(0x9888), 0x063a8000 },
	1629	+ { _MMIO(0x9888), 0x08138000 },
	1630	+ { _MMIO(0x9888), 0x0a138000 },
	1631	+ { _MMIO(0x9888), 0x06143000 },
	1632	+ { _MMIO(0x9888), 0x0415cfc7 },
	1633	+ { _MMIO(0x9888), 0x10150000 },
	1634	+ { _MMIO(0x9888), 0x02338000 },
	1635	+ { _MMIO(0x9888), 0x0c338000 },
	1636	+ { _MMIO(0x9888), 0x04342000 },
	1637	+ { _MMIO(0x9888), 0x06344000 },
	1638	+ { _MMIO(0x9888), 0x0035c700 },
	1639	+ { _MMIO(0x9888), 0x063500cf },
	1640	+ { _MMIO(0x9888), 0x10350000 },
	1641	+ { _MMIO(0x9888), 0x04538000 },
	1642	+ { _MMIO(0x9888), 0x06538000 },
	1643	+ { _MMIO(0x9888), 0x0454c000 },
	1644	+ { _MMIO(0x9888), 0x0255cfc7 },
	1645	+ { _MMIO(0x9888), 0x10550000 },
	1646	+ { _MMIO(0x9888), 0x06dc8000 },
	1647	+ { _MMIO(0x9888), 0x08dc4000 },
	1648	+ { _MMIO(0x9888), 0x0cdcc000 },
	1649	+ { _MMIO(0x9888), 0x0edcc000 },
	1650	+ { _MMIO(0x9888), 0x1abd00a8 },
	1651	+ { _MMIO(0x9888), 0x0cd8c000 },
	1652	+ { _MMIO(0x9888), 0x0ed84000 },
	1653	+ { _MMIO(0x9888), 0x0edb8000 },
	1654	+ { _MMIO(0x9888), 0x18db0800 },
	1655	+ { _MMIO(0x9888), 0x1adb0254 },
	1656	+ { _MMIO(0x9888), 0x0e9faa00 },
	1657	+ { _MMIO(0x9888), 0x109f02aa },
	1658	+ { _MMIO(0x9888), 0x0eb84000 },
	1659	+ { _MMIO(0x9888), 0x16b84000 },
	1660	+ { _MMIO(0x9888), 0x18b8156a },
	1661	+ { _MMIO(0x9888), 0x06b98000 },
	1662	+ { _MMIO(0x9888), 0x08b9a000 },
	1663	+ { _MMIO(0x9888), 0x0ab9a000 },
	1664	+ { _MMIO(0x9888), 0x0cb9a000 },
	1665	+ { _MMIO(0x9888), 0x0eb9a000 },
	1666	+ { _MMIO(0x9888), 0x18baa000 },
	1667	+ { _MMIO(0x9888), 0x1aba0002 },
	1668	+ { _MMIO(0x9888), 0x16934000 },
	1669	+ { _MMIO(0x9888), 0x1893000a },
	1670	+ { _MMIO(0x9888), 0x0a947000 },
	1671	+ { _MMIO(0x9888), 0x0c95c5c1 },
	1672	+ { _MMIO(0x9888), 0x0e9500c3 },
	1673	+ { _MMIO(0x9888), 0x10950000 },
	1674	+ { _MMIO(0x9888), 0x0eb38000 },
	1675	+ { _MMIO(0x9888), 0x16b30040 },
	1676	+ { _MMIO(0x9888), 0x18b30020 },
	1677	+ { _MMIO(0x9888), 0x06b48000 },
	1678	+ { _MMIO(0x9888), 0x08b41000 },
	1679	+ { _MMIO(0x9888), 0x0ab48000 },
	1680	+ { _MMIO(0x9888), 0x06b5c500 },
	1681	+ { _MMIO(0x9888), 0x08b500c3 },
	1682	+ { _MMIO(0x9888), 0x0eb5c100 },
	1683	+ { _MMIO(0x9888), 0x10b50000 },
	1684	+ { _MMIO(0x9888), 0x16d31500 },
	1685	+ { _MMIO(0x9888), 0x08d4e000 },
	1686	+ { _MMIO(0x9888), 0x08d5c100 },
	1687	+ { _MMIO(0x9888), 0x0ad5c3c5 },
	1688	+ { _MMIO(0x9888), 0x10d50000 },
	1689	+ { _MMIO(0x9888), 0x0d88f800 },
	1690	+ { _MMIO(0x9888), 0x0f88000f },
	1691	+ { _MMIO(0x9888), 0x038a8000 },
	1692	+ { _MMIO(0x9888), 0x058a8000 },
	1693	+ { _MMIO(0x9888), 0x078a8000 },
	1694	+ { _MMIO(0x9888), 0x098a8000 },
	1695	+ { _MMIO(0x9888), 0x0b8a8000 },
	1696	+ { _MMIO(0x9888), 0x0d8a8000 },
	1697	+ { _MMIO(0x9888), 0x258baaa5 },
	1698	+ { _MMIO(0x9888), 0x278b002a },
	1699	+ { _MMIO(0x9888), 0x238b2a80 },
	1700	+ { _MMIO(0x9888), 0x0f8c4000 },
	1701	+ { _MMIO(0x9888), 0x178c2000 },
	1702	+ { _MMIO(0x9888), 0x198c5500 },
	1703	+ { _MMIO(0x9888), 0x1b8c0015 },
	1704	+ { _MMIO(0x9888), 0x078d8000 },
	1705	+ { _MMIO(0x9888), 0x098da000 },
	1706	+ { _MMIO(0x9888), 0x0b8da000 },
	1707	+ { _MMIO(0x9888), 0x0d8da000 },
	1708	+ { _MMIO(0x9888), 0x0f8da000 },
	1709	+ { _MMIO(0x9888), 0x2185aaaa },
	1710	+ { _MMIO(0x9888), 0x2385002a },
	1711	+ { _MMIO(0x9888), 0x1f85aa00 },
	1712	+ { _MMIO(0x9888), 0x0f834000 },
	1713	+ { _MMIO(0x9888), 0x19835400 },
	1714	+ { _MMIO(0x9888), 0x1b830155 },
	1715	+ { _MMIO(0x9888), 0x03834000 },
	1716	+ { _MMIO(0x9888), 0x05834000 },
	1717	+ { _MMIO(0x9888), 0x07834000 },
	1718	+ { _MMIO(0x9888), 0x09834000 },
	1719	+ { _MMIO(0x9888), 0x0b834000 },
	1720	+ { _MMIO(0x9888), 0x0d834000 },
	1721	+ { _MMIO(0x9888), 0x0784c000 },
	1722	+ { _MMIO(0x9888), 0x0984c000 },
	1723	+ { _MMIO(0x9888), 0x0b84c000 },
	1724	+ { _MMIO(0x9888), 0x0d84c000 },
	1725	+ { _MMIO(0x9888), 0x0f84c000 },
	1726	+ { _MMIO(0x9888), 0x01848000 },
	1727	+ { _MMIO(0x9888), 0x0384c000 },
	1728	+ { _MMIO(0x9888), 0x0584c000 },
	1729	+ { _MMIO(0x9888), 0x1780c000 },
	1730	+ { _MMIO(0x9888), 0x1980c000 },
	1731	+ { _MMIO(0x9888), 0x1b80c000 },
	1732	+ { _MMIO(0x9888), 0x1d80c000 },
	1733	+ { _MMIO(0x9888), 0x1f80c000 },
	1734	+ { _MMIO(0x9888), 0x11808000 },
	1735	+ { _MMIO(0x9888), 0x1380c000 },
	1736	+ { _MMIO(0x9888), 0x1580c000 },
	1737	+ { _MMIO(0x9888), 0x4f800000 },
	1738	+ { _MMIO(0x9888), 0x43800c42 },
	1739	+ { _MMIO(0x9888), 0x51800000 },
	1740	+ { _MMIO(0x9888), 0x45800063 },
	1741	+ { _MMIO(0x9888), 0x53800000 },
	1742	+ { _MMIO(0x9888), 0x47800800 },
	1743	+ { _MMIO(0x9888), 0x21800000 },
	1744	+ { _MMIO(0x9888), 0x31800000 },
	1745	+ { _MMIO(0x9888), 0x4d800000 },
	1746	+ { _MMIO(0x9888), 0x3f8014a4 },
	1747	+ { _MMIO(0x9888), 0x41801042 },
	1748	+};
	1749	+
	1750	+static int
	1751	+get_tdl_1_mux_config(struct drm_i915_private *dev_priv,
	1752	+ const struct i915_oa_reg **regs,
	1753	+ int *lens)
	1754	+{
	1755	+ int n = 0;
	1756	+
	1757	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1758	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1759	+
	1760	+ regs[n] = mux_config_tdl_1;
	1761	+ lens[n] = ARRAY_SIZE(mux_config_tdl_1);
	1762	+ n++;
	1763	+
	1764	+ return n;
	1765	+}
	1766	+
	1767	+static const struct i915_oa_reg b_counter_config_tdl_2[] = {
	1768	+ { _MMIO(0x2740), 0x00000000 },
	1769	+ { _MMIO(0x2744), 0x00800000 },
	1770	+ { _MMIO(0x2710), 0x00000000 },
	1771	+ { _MMIO(0x2714), 0xf0800000 },
	1772	+ { _MMIO(0x2720), 0x00000000 },
	1773	+ { _MMIO(0x2724), 0x30800000 },
	1774	+ { _MMIO(0x2770), 0x00000002 },
	1775	+ { _MMIO(0x2774), 0x0000fdff },
	1776	+ { _MMIO(0x2778), 0x00000000 },
	1777	+ { _MMIO(0x277c), 0x0000fe7f },
	1778	+ { _MMIO(0x2780), 0x00000000 },
	1779	+ { _MMIO(0x2784), 0x0000ff9f },
	1780	+ { _MMIO(0x2788), 0x00000000 },
	1781	+ { _MMIO(0x278c), 0x0000ffe7 },
	1782	+ { _MMIO(0x2790), 0x00000002 },
	1783	+ { _MMIO(0x2794), 0x0000fffb },
	1784	+ { _MMIO(0x2798), 0x00000002 },
	1785	+ { _MMIO(0x279c), 0x0000fffd },
	1786	+};
	1787	+
	1788	+static const struct i915_oa_reg flex_eu_config_tdl_2[] = {
	1789	+ { _MMIO(0xe458), 0x00005004 },
	1790	+ { _MMIO(0xe558), 0x00010003 },
	1791	+ { _MMIO(0xe658), 0x00012011 },
	1792	+ { _MMIO(0xe758), 0x00015014 },
	1793	+ { _MMIO(0xe45c), 0x00051050 },
	1794	+ { _MMIO(0xe55c), 0x00053052 },
	1795	+ { _MMIO(0xe65c), 0x00055054 },
	1796	+};
	1797	+
	1798	+static const struct i915_oa_reg mux_config_tdl_2[] = {
	1799	+ { _MMIO(0x9888), 0x16150000 },
	1800	+ { _MMIO(0x9888), 0x16350000 },
	1801	+ { _MMIO(0x9888), 0x16550000 },
	1802	+ { _MMIO(0x9888), 0x16952e60 },
	1803	+ { _MMIO(0x9888), 0x16b54d60 },
	1804	+ { _MMIO(0x9888), 0x16d52e60 },
	1805	+ { _MMIO(0x9888), 0x065c8000 },
	1806	+ { _MMIO(0x9888), 0x085cc000 },
	1807	+ { _MMIO(0x9888), 0x0a5cc000 },
	1808	+ { _MMIO(0x9888), 0x0c5c4000 },
	1809	+ { _MMIO(0x9888), 0x0e3d8000 },
	1810	+ { _MMIO(0x9888), 0x183da000 },
	1811	+ { _MMIO(0x9888), 0x06588000 },
	1812	+ { _MMIO(0x9888), 0x08588000 },
	1813	+ { _MMIO(0x9888), 0x0a584000 },
	1814	+ { _MMIO(0x9888), 0x0e5b4000 },
	1815	+ { _MMIO(0x9888), 0x185b5800 },
	1816	+ { _MMIO(0x9888), 0x1a5b000a },
	1817	+ { _MMIO(0x9888), 0x0e1faa00 },
	1818	+ { _MMIO(0x9888), 0x101f02aa },
	1819	+ { _MMIO(0x9888), 0x0e384000 },
	1820	+ { _MMIO(0x9888), 0x16384000 },
	1821	+ { _MMIO(0x9888), 0x18382a55 },
	1822	+ { _MMIO(0x9888), 0x06398000 },
	1823	+ { _MMIO(0x9888), 0x0839a000 },
	1824	+ { _MMIO(0x9888), 0x0a39a000 },
	1825	+ { _MMIO(0x9888), 0x0c39a000 },
	1826	+ { _MMIO(0x9888), 0x0e39a000 },
	1827	+ { _MMIO(0x9888), 0x1a3a02a0 },
	1828	+ { _MMIO(0x9888), 0x0e138000 },
	1829	+ { _MMIO(0x9888), 0x16130500 },
	1830	+ { _MMIO(0x9888), 0x06148000 },
	1831	+ { _MMIO(0x9888), 0x08146000 },
	1832	+ { _MMIO(0x9888), 0x0615c100 },
	1833	+ { _MMIO(0x9888), 0x0815c500 },
	1834	+ { _MMIO(0x9888), 0x0a1500c3 },
	1835	+ { _MMIO(0x9888), 0x10150000 },
	1836	+ { _MMIO(0x9888), 0x16335040 },
	1837	+ { _MMIO(0x9888), 0x08349000 },
	1838	+ { _MMIO(0x9888), 0x0a341000 },
	1839	+ { _MMIO(0x9888), 0x083500c1 },
	1840	+ { _MMIO(0x9888), 0x0a35c500 },
	1841	+ { _MMIO(0x9888), 0x0c3500c3 },
	1842	+ { _MMIO(0x9888), 0x10350000 },
	1843	+ { _MMIO(0x9888), 0x1853002a },
	1844	+ { _MMIO(0x9888), 0x0a54e000 },
	1845	+ { _MMIO(0x9888), 0x0c55c500 },
	1846	+ { _MMIO(0x9888), 0x0e55c1c3 },
	1847	+ { _MMIO(0x9888), 0x10550000 },
	1848	+ { _MMIO(0x9888), 0x00dc8000 },
	1849	+ { _MMIO(0x9888), 0x02dcc000 },
	1850	+ { _MMIO(0x9888), 0x04dc4000 },
	1851	+ { _MMIO(0x9888), 0x04bd8000 },
	1852	+ { _MMIO(0x9888), 0x06bd8000 },
	1853	+ { _MMIO(0x9888), 0x02d8c000 },
	1854	+ { _MMIO(0x9888), 0x02db8000 },
	1855	+ { _MMIO(0x9888), 0x04db4000 },
	1856	+ { _MMIO(0x9888), 0x06db4000 },
	1857	+ { _MMIO(0x9888), 0x08db8000 },
	1858	+ { _MMIO(0x9888), 0x0c9fa000 },
	1859	+ { _MMIO(0x9888), 0x0e9f00aa },
	1860	+ { _MMIO(0x9888), 0x02b84000 },
	1861	+ { _MMIO(0x9888), 0x04b84000 },
	1862	+ { _MMIO(0x9888), 0x06b84000 },
	1863	+ { _MMIO(0x9888), 0x08b84000 },
	1864	+ { _MMIO(0x9888), 0x0ab88000 },
	1865	+ { _MMIO(0x9888), 0x0cb88000 },
	1866	+ { _MMIO(0x9888), 0x00b98000 },
	1867	+ { _MMIO(0x9888), 0x02b9a000 },
	1868	+ { _MMIO(0x9888), 0x04b9a000 },
	1869	+ { _MMIO(0x9888), 0x06b92000 },
	1870	+ { _MMIO(0x9888), 0x0aba8000 },
	1871	+ { _MMIO(0x9888), 0x0cba8000 },
	1872	+ { _MMIO(0x9888), 0x04938000 },
	1873	+ { _MMIO(0x9888), 0x06938000 },
	1874	+ { _MMIO(0x9888), 0x0494c000 },
	1875	+ { _MMIO(0x9888), 0x0295cfc7 },
	1876	+ { _MMIO(0x9888), 0x10950000 },
	1877	+ { _MMIO(0x9888), 0x02b38000 },
	1878	+ { _MMIO(0x9888), 0x08b38000 },
	1879	+ { _MMIO(0x9888), 0x04b42000 },
	1880	+ { _MMIO(0x9888), 0x06b41000 },
	1881	+ { _MMIO(0x9888), 0x00b5c700 },
	1882	+ { _MMIO(0x9888), 0x04b500cf },
	1883	+ { _MMIO(0x9888), 0x10b50000 },
	1884	+ { _MMIO(0x9888), 0x0ad38000 },
	1885	+ { _MMIO(0x9888), 0x0cd38000 },
	1886	+ { _MMIO(0x9888), 0x06d46000 },
	1887	+ { _MMIO(0x9888), 0x04d5c700 },
	1888	+ { _MMIO(0x9888), 0x06d500cf },
	1889	+ { _MMIO(0x9888), 0x10d50000 },
	1890	+ { _MMIO(0x9888), 0x03888000 },
	1891	+ { _MMIO(0x9888), 0x05888000 },
	1892	+ { _MMIO(0x9888), 0x07888000 },
	1893	+ { _MMIO(0x9888), 0x09888000 },
	1894	+ { _MMIO(0x9888), 0x0b888000 },
	1895	+ { _MMIO(0x9888), 0x0d880400 },
	1896	+ { _MMIO(0x9888), 0x0f8a8000 },
	1897	+ { _MMIO(0x9888), 0x198a8000 },
	1898	+ { _MMIO(0x9888), 0x1b8aaaa0 },
	1899	+ { _MMIO(0x9888), 0x1d8a0002 },
	1900	+ { _MMIO(0x9888), 0x258b555a },
	1901	+ { _MMIO(0x9888), 0x278b0015 },
	1902	+ { _MMIO(0x9888), 0x238b5500 },
	1903	+ { _MMIO(0x9888), 0x038c4000 },
	1904	+ { _MMIO(0x9888), 0x058c4000 },
	1905	+ { _MMIO(0x9888), 0x078c4000 },
	1906	+ { _MMIO(0x9888), 0x098c4000 },
	1907	+ { _MMIO(0x9888), 0x0b8c4000 },
	1908	+ { _MMIO(0x9888), 0x0d8c4000 },
	1909	+ { _MMIO(0x9888), 0x018d8000 },
	1910	+ { _MMIO(0x9888), 0x038da000 },
	1911	+ { _MMIO(0x9888), 0x058da000 },
	1912	+ { _MMIO(0x9888), 0x078d2000 },
	1913	+ { _MMIO(0x9888), 0x2185aaaa },
	1914	+ { _MMIO(0x9888), 0x2385002a },
	1915	+ { _MMIO(0x9888), 0x1f85aa00 },
	1916	+ { _MMIO(0x9888), 0x0f834000 },
	1917	+ { _MMIO(0x9888), 0x19835400 },
	1918	+ { _MMIO(0x9888), 0x1b830155 },
	1919	+ { _MMIO(0x9888), 0x03834000 },
	1920	+ { _MMIO(0x9888), 0x05834000 },
	1921	+ { _MMIO(0x9888), 0x07834000 },
	1922	+ { _MMIO(0x9888), 0x09834000 },
	1923	+ { _MMIO(0x9888), 0x0b834000 },
	1924	+ { _MMIO(0x9888), 0x0d834000 },
	1925	+ { _MMIO(0x9888), 0x0784c000 },
	1926	+ { _MMIO(0x9888), 0x0984c000 },
	1927	+ { _MMIO(0x9888), 0x0b84c000 },
	1928	+ { _MMIO(0x9888), 0x0d84c000 },
	1929	+ { _MMIO(0x9888), 0x0f84c000 },
	1930	+ { _MMIO(0x9888), 0x01848000 },
	1931	+ { _MMIO(0x9888), 0x0384c000 },
	1932	+ { _MMIO(0x9888), 0x0584c000 },
	1933	+ { _MMIO(0x9888), 0x1780c000 },
	1934	+ { _MMIO(0x9888), 0x1980c000 },
	1935	+ { _MMIO(0x9888), 0x1b80c000 },
	1936	+ { _MMIO(0x9888), 0x1d80c000 },
	1937	+ { _MMIO(0x9888), 0x1f80c000 },
	1938	+ { _MMIO(0x9888), 0x11808000 },
	1939	+ { _MMIO(0x9888), 0x1380c000 },
	1940	+ { _MMIO(0x9888), 0x1580c000 },
	1941	+ { _MMIO(0x9888), 0x4f800000 },
	1942	+ { _MMIO(0x9888), 0x43800882 },
	1943	+ { _MMIO(0x9888), 0x51800000 },
	1944	+ { _MMIO(0x9888), 0x45801082 },
	1945	+ { _MMIO(0x9888), 0x53800000 },
	1946	+ { _MMIO(0x9888), 0x478014a5 },
	1947	+ { _MMIO(0x9888), 0x21800000 },
	1948	+ { _MMIO(0x9888), 0x31800000 },
	1949	+ { _MMIO(0x9888), 0x4d800000 },
	1950	+ { _MMIO(0x9888), 0x3f800002 },
	1951	+ { _MMIO(0x9888), 0x41800c62 },
	1952	+};
	1953	+
	1954	+static int
	1955	+get_tdl_2_mux_config(struct drm_i915_private *dev_priv,
	1956	+ const struct i915_oa_reg **regs,
	1957	+ int *lens)
	1958	+{
	1959	+ int n = 0;
	1960	+
	1961	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1962	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1963	+
	1964	+ regs[n] = mux_config_tdl_2;
	1965	+ lens[n] = ARRAY_SIZE(mux_config_tdl_2);
	1966	+ n++;
	1967	+
	1968	+ return n;
	1969	+}
	1970	+
	1971	+static const struct i915_oa_reg b_counter_config_test_oa[] = {
	1972	+ { _MMIO(0x2740), 0x00000000 },
	1973	+ { _MMIO(0x2744), 0x00800000 },
	1974	+ { _MMIO(0x2714), 0xf0800000 },
	1975	+ { _MMIO(0x2710), 0x00000000 },
	1976	+ { _MMIO(0x2724), 0xf0800000 },
	1977	+ { _MMIO(0x2720), 0x00000000 },
	1978	+ { _MMIO(0x2770), 0x00000004 },
	1979	+ { _MMIO(0x2774), 0x00000000 },
	1980	+ { _MMIO(0x2778), 0x00000003 },
	1981	+ { _MMIO(0x277c), 0x00000000 },
	1982	+ { _MMIO(0x2780), 0x00000007 },
	1983	+ { _MMIO(0x2784), 0x00000000 },
	1984	+ { _MMIO(0x2788), 0x00100002 },
	1985	+ { _MMIO(0x278c), 0x0000fff7 },
	1986	+ { _MMIO(0x2790), 0x00100002 },
	1987	+ { _MMIO(0x2794), 0x0000ffcf },
	1988	+ { _MMIO(0x2798), 0x00100082 },
	1989	+ { _MMIO(0x279c), 0x0000ffef },
	1990	+ { _MMIO(0x27a0), 0x001000c2 },
	1991	+ { _MMIO(0x27a4), 0x0000ffe7 },
	1992	+ { _MMIO(0x27a8), 0x00100001 },
	1993	+ { _MMIO(0x27ac), 0x0000ffe7 },
	1994	+};
	1995	+
	1996	+static const struct i915_oa_reg flex_eu_config_test_oa[] = {
	1997	+};
	1998	+
	1999	+static const struct i915_oa_reg mux_config_test_oa[] = {
	2000	+ { _MMIO(0x9888), 0x59800000 },
	2001	+ { _MMIO(0x9888), 0x59800001 },
	2002	+ { _MMIO(0x9888), 0x338b0000 },
	2003	+ { _MMIO(0x9888), 0x258b0066 },
	2004	+ { _MMIO(0x9888), 0x058b0000 },
	2005	+ { _MMIO(0x9888), 0x038b0000 },
	2006	+ { _MMIO(0x9888), 0x03844000 },
	2007	+ { _MMIO(0x9888), 0x47800080 },
	2008	+ { _MMIO(0x9888), 0x57800000 },
	2009	+ { _MMIO(0x1823a4), 0x00000000 },
	2010	+ { _MMIO(0x9888), 0x59800000 },
	2011	+};
	2012	+
	2013	+static int
	2014	+get_test_oa_mux_config(struct drm_i915_private *dev_priv,
	2015	+ const struct i915_oa_reg **regs,
	2016	+ int *lens)
	2017	+{
	2018	+ int n = 0;
	2019	+
	2020	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	2021	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	2022	+
	2023	+ regs[n] = mux_config_test_oa;
	2024	+ lens[n] = ARRAY_SIZE(mux_config_test_oa);
	2025	+ n++;
	2026	+
	2027	+ return n;
	2028	+}
	2029	+
	2030	+int i915_oa_select_metric_set_chv(struct drm_i915_private *dev_priv)
	2031	+{
	2032	+ dev_priv->perf.oa.n_mux_configs = 0;
	2033	+ dev_priv->perf.oa.b_counter_regs = NULL;
	2034	+ dev_priv->perf.oa.b_counter_regs_len = 0;
	2035	+ dev_priv->perf.oa.flex_regs = NULL;
	2036	+ dev_priv->perf.oa.flex_regs_len = 0;
	2037	+
	2038	+ switch (dev_priv->perf.oa.metrics_set) {
	2039	+ case METRIC_SET_ID_RENDER_BASIC:
	2040	+ dev_priv->perf.oa.n_mux_configs =
	2041	+ get_render_basic_mux_config(dev_priv,
	2042	+ dev_priv->perf.oa.mux_regs,
	2043	+ dev_priv->perf.oa.mux_regs_lens);
	2044	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2045	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RENDER_BASIC\" metric set\n");
	2046	+
	2047	+ /* EINVAL because *_register_sysfs already checked this
	2048	+ * and so it wouldn't have been advertised to userspace and
	2049	+ * so shouldn't have been requested
	2050	+ */
	2051	+ return -EINVAL;
	2052	+ }
	2053	+
	2054	+ dev_priv->perf.oa.b_counter_regs =
	2055	+ b_counter_config_render_basic;
	2056	+ dev_priv->perf.oa.b_counter_regs_len =
	2057	+ ARRAY_SIZE(b_counter_config_render_basic);
	2058	+
	2059	+ dev_priv->perf.oa.flex_regs =
	2060	+ flex_eu_config_render_basic;
	2061	+ dev_priv->perf.oa.flex_regs_len =
	2062	+ ARRAY_SIZE(flex_eu_config_render_basic);
	2063	+
	2064	+ return 0;
	2065	+ case METRIC_SET_ID_COMPUTE_BASIC:
	2066	+ dev_priv->perf.oa.n_mux_configs =
	2067	+ get_compute_basic_mux_config(dev_priv,
	2068	+ dev_priv->perf.oa.mux_regs,
	2069	+ dev_priv->perf.oa.mux_regs_lens);
	2070	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2071	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_BASIC\" metric set\n");
	2072	+
	2073	+ /* EINVAL because *_register_sysfs already checked this
	2074	+ * and so it wouldn't have been advertised to userspace and
	2075	+ * so shouldn't have been requested
	2076	+ */
	2077	+ return -EINVAL;
	2078	+ }
	2079	+
	2080	+ dev_priv->perf.oa.b_counter_regs =
	2081	+ b_counter_config_compute_basic;
	2082	+ dev_priv->perf.oa.b_counter_regs_len =
	2083	+ ARRAY_SIZE(b_counter_config_compute_basic);
	2084	+
	2085	+ dev_priv->perf.oa.flex_regs =
	2086	+ flex_eu_config_compute_basic;
	2087	+ dev_priv->perf.oa.flex_regs_len =
	2088	+ ARRAY_SIZE(flex_eu_config_compute_basic);
	2089	+
	2090	+ return 0;
	2091	+ case METRIC_SET_ID_RENDER_PIPE_PROFILE:
	2092	+ dev_priv->perf.oa.n_mux_configs =
	2093	+ get_render_pipe_profile_mux_config(dev_priv,
	2094	+ dev_priv->perf.oa.mux_regs,
	2095	+ dev_priv->perf.oa.mux_regs_lens);
	2096	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2097	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RENDER_PIPE_PROFILE\" metric set\n");
	2098	+
	2099	+ /* EINVAL because *_register_sysfs already checked this
	2100	+ * and so it wouldn't have been advertised to userspace and
	2101	+ * so shouldn't have been requested
	2102	+ */
	2103	+ return -EINVAL;
	2104	+ }
	2105	+
	2106	+ dev_priv->perf.oa.b_counter_regs =
	2107	+ b_counter_config_render_pipe_profile;
	2108	+ dev_priv->perf.oa.b_counter_regs_len =
	2109	+ ARRAY_SIZE(b_counter_config_render_pipe_profile);
	2110	+
	2111	+ dev_priv->perf.oa.flex_regs =
	2112	+ flex_eu_config_render_pipe_profile;
	2113	+ dev_priv->perf.oa.flex_regs_len =
	2114	+ ARRAY_SIZE(flex_eu_config_render_pipe_profile);
	2115	+
	2116	+ return 0;
	2117	+ case METRIC_SET_ID_HDC_AND_SF:
	2118	+ dev_priv->perf.oa.n_mux_configs =
	2119	+ get_hdc_and_sf_mux_config(dev_priv,
	2120	+ dev_priv->perf.oa.mux_regs,
	2121	+ dev_priv->perf.oa.mux_regs_lens);
	2122	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2123	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"HDC_AND_SF\" metric set\n");
	2124	+
	2125	+ /* EINVAL because *_register_sysfs already checked this
	2126	+ * and so it wouldn't have been advertised to userspace and
	2127	+ * so shouldn't have been requested
	2128	+ */
	2129	+ return -EINVAL;
	2130	+ }
	2131	+
	2132	+ dev_priv->perf.oa.b_counter_regs =
	2133	+ b_counter_config_hdc_and_sf;
	2134	+ dev_priv->perf.oa.b_counter_regs_len =
	2135	+ ARRAY_SIZE(b_counter_config_hdc_and_sf);
	2136	+
	2137	+ dev_priv->perf.oa.flex_regs =
	2138	+ flex_eu_config_hdc_and_sf;
	2139	+ dev_priv->perf.oa.flex_regs_len =
	2140	+ ARRAY_SIZE(flex_eu_config_hdc_and_sf);
	2141	+
	2142	+ return 0;
	2143	+ case METRIC_SET_ID_L3_1:
	2144	+ dev_priv->perf.oa.n_mux_configs =
	2145	+ get_l3_1_mux_config(dev_priv,
	2146	+ dev_priv->perf.oa.mux_regs,
	2147	+ dev_priv->perf.oa.mux_regs_lens);
	2148	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2149	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_1\" metric set\n");
	2150	+
	2151	+ /* EINVAL because *_register_sysfs already checked this
	2152	+ * and so it wouldn't have been advertised to userspace and
	2153	+ * so shouldn't have been requested
	2154	+ */
	2155	+ return -EINVAL;
	2156	+ }
	2157	+
	2158	+ dev_priv->perf.oa.b_counter_regs =
	2159	+ b_counter_config_l3_1;
	2160	+ dev_priv->perf.oa.b_counter_regs_len =
	2161	+ ARRAY_SIZE(b_counter_config_l3_1);
	2162	+
	2163	+ dev_priv->perf.oa.flex_regs =
	2164	+ flex_eu_config_l3_1;
	2165	+ dev_priv->perf.oa.flex_regs_len =
	2166	+ ARRAY_SIZE(flex_eu_config_l3_1);
	2167	+
	2168	+ return 0;
	2169	+ case METRIC_SET_ID_L3_2:
	2170	+ dev_priv->perf.oa.n_mux_configs =
	2171	+ get_l3_2_mux_config(dev_priv,
	2172	+ dev_priv->perf.oa.mux_regs,
	2173	+ dev_priv->perf.oa.mux_regs_lens);
	2174	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2175	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_2\" metric set\n");
	2176	+
	2177	+ /* EINVAL because *_register_sysfs already checked this
	2178	+ * and so it wouldn't have been advertised to userspace and
	2179	+ * so shouldn't have been requested
	2180	+ */
	2181	+ return -EINVAL;
	2182	+ }
	2183	+
	2184	+ dev_priv->perf.oa.b_counter_regs =
	2185	+ b_counter_config_l3_2;
	2186	+ dev_priv->perf.oa.b_counter_regs_len =
	2187	+ ARRAY_SIZE(b_counter_config_l3_2);
	2188	+
	2189	+ dev_priv->perf.oa.flex_regs =
	2190	+ flex_eu_config_l3_2;
	2191	+ dev_priv->perf.oa.flex_regs_len =
	2192	+ ARRAY_SIZE(flex_eu_config_l3_2);
	2193	+
	2194	+ return 0;
	2195	+ case METRIC_SET_ID_L3_3:
	2196	+ dev_priv->perf.oa.n_mux_configs =
	2197	+ get_l3_3_mux_config(dev_priv,
	2198	+ dev_priv->perf.oa.mux_regs,
	2199	+ dev_priv->perf.oa.mux_regs_lens);
	2200	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2201	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_3\" metric set\n");
	2202	+
	2203	+ /* EINVAL because *_register_sysfs already checked this
	2204	+ * and so it wouldn't have been advertised to userspace and
	2205	+ * so shouldn't have been requested
	2206	+ */
	2207	+ return -EINVAL;
	2208	+ }
	2209	+
	2210	+ dev_priv->perf.oa.b_counter_regs =
	2211	+ b_counter_config_l3_3;
	2212	+ dev_priv->perf.oa.b_counter_regs_len =
	2213	+ ARRAY_SIZE(b_counter_config_l3_3);
	2214	+
	2215	+ dev_priv->perf.oa.flex_regs =
	2216	+ flex_eu_config_l3_3;
	2217	+ dev_priv->perf.oa.flex_regs_len =
	2218	+ ARRAY_SIZE(flex_eu_config_l3_3);
	2219	+
	2220	+ return 0;
	2221	+ case METRIC_SET_ID_L3_4:
	2222	+ dev_priv->perf.oa.n_mux_configs =
	2223	+ get_l3_4_mux_config(dev_priv,
	2224	+ dev_priv->perf.oa.mux_regs,
	2225	+ dev_priv->perf.oa.mux_regs_lens);
	2226	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2227	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_4\" metric set\n");
	2228	+
	2229	+ /* EINVAL because *_register_sysfs already checked this
	2230	+ * and so it wouldn't have been advertised to userspace and
	2231	+ * so shouldn't have been requested
	2232	+ */
	2233	+ return -EINVAL;
	2234	+ }
	2235	+
	2236	+ dev_priv->perf.oa.b_counter_regs =
	2237	+ b_counter_config_l3_4;
	2238	+ dev_priv->perf.oa.b_counter_regs_len =
	2239	+ ARRAY_SIZE(b_counter_config_l3_4);
	2240	+
	2241	+ dev_priv->perf.oa.flex_regs =
	2242	+ flex_eu_config_l3_4;
	2243	+ dev_priv->perf.oa.flex_regs_len =
	2244	+ ARRAY_SIZE(flex_eu_config_l3_4);
	2245	+
	2246	+ return 0;
	2247	+ case METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND:
	2248	+ dev_priv->perf.oa.n_mux_configs =
	2249	+ get_rasterizer_and_pixel_backend_mux_config(dev_priv,
	2250	+ dev_priv->perf.oa.mux_regs,
	2251	+ dev_priv->perf.oa.mux_regs_lens);
	2252	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2253	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RASTERIZER_AND_PIXEL_BACKEND\" metric set\n");
	2254	+
	2255	+ /* EINVAL because *_register_sysfs already checked this
	2256	+ * and so it wouldn't have been advertised to userspace and
	2257	+ * so shouldn't have been requested
	2258	+ */
	2259	+ return -EINVAL;
	2260	+ }
	2261	+
	2262	+ dev_priv->perf.oa.b_counter_regs =
	2263	+ b_counter_config_rasterizer_and_pixel_backend;
	2264	+ dev_priv->perf.oa.b_counter_regs_len =
	2265	+ ARRAY_SIZE(b_counter_config_rasterizer_and_pixel_backend);
	2266	+
	2267	+ dev_priv->perf.oa.flex_regs =
	2268	+ flex_eu_config_rasterizer_and_pixel_backend;
	2269	+ dev_priv->perf.oa.flex_regs_len =
	2270	+ ARRAY_SIZE(flex_eu_config_rasterizer_and_pixel_backend);
	2271	+
	2272	+ return 0;
	2273	+ case METRIC_SET_ID_SAMPLER_1:
	2274	+ dev_priv->perf.oa.n_mux_configs =
	2275	+ get_sampler_1_mux_config(dev_priv,
	2276	+ dev_priv->perf.oa.mux_regs,
	2277	+ dev_priv->perf.oa.mux_regs_lens);
	2278	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2279	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"SAMPLER_1\" metric set\n");
	2280	+
	2281	+ /* EINVAL because *_register_sysfs already checked this
	2282	+ * and so it wouldn't have been advertised to userspace and
	2283	+ * so shouldn't have been requested
	2284	+ */
	2285	+ return -EINVAL;
	2286	+ }
	2287	+
	2288	+ dev_priv->perf.oa.b_counter_regs =
	2289	+ b_counter_config_sampler_1;
	2290	+ dev_priv->perf.oa.b_counter_regs_len =
	2291	+ ARRAY_SIZE(b_counter_config_sampler_1);
	2292	+
	2293	+ dev_priv->perf.oa.flex_regs =
	2294	+ flex_eu_config_sampler_1;
	2295	+ dev_priv->perf.oa.flex_regs_len =
	2296	+ ARRAY_SIZE(flex_eu_config_sampler_1);
	2297	+
	2298	+ return 0;
	2299	+ case METRIC_SET_ID_SAMPLER_2:
	2300	+ dev_priv->perf.oa.n_mux_configs =
	2301	+ get_sampler_2_mux_config(dev_priv,
	2302	+ dev_priv->perf.oa.mux_regs,
	2303	+ dev_priv->perf.oa.mux_regs_lens);
	2304	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2305	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"SAMPLER_2\" metric set\n");
	2306	+
	2307	+ /* EINVAL because *_register_sysfs already checked this
	2308	+ * and so it wouldn't have been advertised to userspace and
	2309	+ * so shouldn't have been requested
	2310	+ */
	2311	+ return -EINVAL;
	2312	+ }
	2313	+
	2314	+ dev_priv->perf.oa.b_counter_regs =
	2315	+ b_counter_config_sampler_2;
	2316	+ dev_priv->perf.oa.b_counter_regs_len =
	2317	+ ARRAY_SIZE(b_counter_config_sampler_2);
	2318	+
	2319	+ dev_priv->perf.oa.flex_regs =
	2320	+ flex_eu_config_sampler_2;
	2321	+ dev_priv->perf.oa.flex_regs_len =
	2322	+ ARRAY_SIZE(flex_eu_config_sampler_2);
	2323	+
	2324	+ return 0;
	2325	+ case METRIC_SET_ID_TDL_1:
	2326	+ dev_priv->perf.oa.n_mux_configs =
	2327	+ get_tdl_1_mux_config(dev_priv,
	2328	+ dev_priv->perf.oa.mux_regs,
	2329	+ dev_priv->perf.oa.mux_regs_lens);
	2330	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2331	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TDL_1\" metric set\n");
	2332	+
	2333	+ /* EINVAL because *_register_sysfs already checked this
	2334	+ * and so it wouldn't have been advertised to userspace and
	2335	+ * so shouldn't have been requested
	2336	+ */
	2337	+ return -EINVAL;
	2338	+ }
	2339	+
	2340	+ dev_priv->perf.oa.b_counter_regs =
	2341	+ b_counter_config_tdl_1;
	2342	+ dev_priv->perf.oa.b_counter_regs_len =
	2343	+ ARRAY_SIZE(b_counter_config_tdl_1);
	2344	+
	2345	+ dev_priv->perf.oa.flex_regs =
	2346	+ flex_eu_config_tdl_1;
	2347	+ dev_priv->perf.oa.flex_regs_len =
	2348	+ ARRAY_SIZE(flex_eu_config_tdl_1);
	2349	+
	2350	+ return 0;
	2351	+ case METRIC_SET_ID_TDL_2:
	2352	+ dev_priv->perf.oa.n_mux_configs =
	2353	+ get_tdl_2_mux_config(dev_priv,
	2354	+ dev_priv->perf.oa.mux_regs,
	2355	+ dev_priv->perf.oa.mux_regs_lens);
	2356	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2357	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TDL_2\" metric set\n");
	2358	+
	2359	+ /* EINVAL because *_register_sysfs already checked this
	2360	+ * and so it wouldn't have been advertised to userspace and
	2361	+ * so shouldn't have been requested
	2362	+ */
	2363	+ return -EINVAL;
	2364	+ }
	2365	+
	2366	+ dev_priv->perf.oa.b_counter_regs =
	2367	+ b_counter_config_tdl_2;
	2368	+ dev_priv->perf.oa.b_counter_regs_len =
	2369	+ ARRAY_SIZE(b_counter_config_tdl_2);
	2370	+
	2371	+ dev_priv->perf.oa.flex_regs =
	2372	+ flex_eu_config_tdl_2;
	2373	+ dev_priv->perf.oa.flex_regs_len =
	2374	+ ARRAY_SIZE(flex_eu_config_tdl_2);
	2375	+
	2376	+ return 0;
	2377	+ case METRIC_SET_ID_TEST_OA:
	2378	+ dev_priv->perf.oa.n_mux_configs =
	2379	+ get_test_oa_mux_config(dev_priv,
	2380	+ dev_priv->perf.oa.mux_regs,
	2381	+ dev_priv->perf.oa.mux_regs_lens);
	2382	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2383	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TEST_OA\" metric set\n");
	2384	+
	2385	+ /* EINVAL because *_register_sysfs already checked this
	2386	+ * and so it wouldn't have been advertised to userspace and
	2387	+ * so shouldn't have been requested
	2388	+ */
	2389	+ return -EINVAL;
	2390	+ }
	2391	+
	2392	+ dev_priv->perf.oa.b_counter_regs =
	2393	+ b_counter_config_test_oa;
	2394	+ dev_priv->perf.oa.b_counter_regs_len =
	2395	+ ARRAY_SIZE(b_counter_config_test_oa);
	2396	+
	2397	+ dev_priv->perf.oa.flex_regs =
	2398	+ flex_eu_config_test_oa;
	2399	+ dev_priv->perf.oa.flex_regs_len =
	2400	+ ARRAY_SIZE(flex_eu_config_test_oa);
	2401	+
	2402	+ return 0;
	2403	+ default:
	2404	+ return -ENODEV;
	2405	+ }
	2406	+}
	2407	+
	2408	+static ssize_t
	2409	+show_render_basic_id(struct device kdev, struct device_attribute attr, char *buf)
	2410	+{
	2411	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RENDER_BASIC);
	2412	+}
	2413	+
	2414	+static struct device_attribute dev_attr_render_basic_id = {
	2415	+ .attr = { .name = "id", .mode = 0444 },
	2416	+ .show = show_render_basic_id,
	2417	+ .store = NULL,
	2418	+};
	2419	+
	2420	+static struct attribute *attrs_render_basic[] = {
	2421	+ &dev_attr_render_basic_id.attr,
	2422	+ NULL,
	2423	+};
	2424	+
	2425	+static struct attribute_group group_render_basic = {
	2426	+ .name = "9d8a3af5-c02c-4a4a-b947-f1672469e0fb",
	2427	+ .attrs = attrs_render_basic,
	2428	+};
	2429	+
	2430	+static ssize_t
	2431	+show_compute_basic_id(struct device kdev, struct device_attribute attr, char *buf)
	2432	+{
	2433	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_BASIC);
	2434	+}
	2435	+
	2436	+static struct device_attribute dev_attr_compute_basic_id = {
	2437	+ .attr = { .name = "id", .mode = 0444 },
	2438	+ .show = show_compute_basic_id,
	2439	+ .store = NULL,
	2440	+};
	2441	+
	2442	+static struct attribute *attrs_compute_basic[] = {
	2443	+ &dev_attr_compute_basic_id.attr,
	2444	+ NULL,
	2445	+};
	2446	+
	2447	+static struct attribute_group group_compute_basic = {
	2448	+ .name = "f522a89c-ecd1-4522-8331-3383c54af5f5",
	2449	+ .attrs = attrs_compute_basic,
	2450	+};
	2451	+
	2452	+static ssize_t
	2453	+show_render_pipe_profile_id(struct device kdev, struct device_attribute attr, char *buf)
	2454	+{
	2455	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RENDER_PIPE_PROFILE);
	2456	+}
	2457	+
	2458	+static struct device_attribute dev_attr_render_pipe_profile_id = {
	2459	+ .attr = { .name = "id", .mode = 0444 },
	2460	+ .show = show_render_pipe_profile_id,
	2461	+ .store = NULL,
	2462	+};
	2463	+
	2464	+static struct attribute *attrs_render_pipe_profile[] = {
	2465	+ &dev_attr_render_pipe_profile_id.attr,
	2466	+ NULL,
	2467	+};
	2468	+
	2469	+static struct attribute_group group_render_pipe_profile = {
	2470	+ .name = "a9ccc03d-a943-4e6b-9cd6-13e063075927",
	2471	+ .attrs = attrs_render_pipe_profile,
	2472	+};
	2473	+
	2474	+static ssize_t
	2475	+show_hdc_and_sf_id(struct device kdev, struct device_attribute attr, char *buf)
	2476	+{
	2477	+ return sprintf(buf, "%d\n", METRIC_SET_ID_HDC_AND_SF);
	2478	+}
	2479	+
	2480	+static struct device_attribute dev_attr_hdc_and_sf_id = {
	2481	+ .attr = { .name = "id", .mode = 0444 },
	2482	+ .show = show_hdc_and_sf_id,
	2483	+ .store = NULL,
	2484	+};
	2485	+
	2486	+static struct attribute *attrs_hdc_and_sf[] = {
	2487	+ &dev_attr_hdc_and_sf_id.attr,
	2488	+ NULL,
	2489	+};
	2490	+
	2491	+static struct attribute_group group_hdc_and_sf = {
	2492	+ .name = "2cf0c064-68df-4fac-9b3f-57f51ca8a069",
	2493	+ .attrs = attrs_hdc_and_sf,
	2494	+};
	2495	+
	2496	+static ssize_t
	2497	+show_l3_1_id(struct device kdev, struct device_attribute attr, char *buf)
	2498	+{
	2499	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_1);
	2500	+}
	2501	+
	2502	+static struct device_attribute dev_attr_l3_1_id = {
	2503	+ .attr = { .name = "id", .mode = 0444 },
	2504	+ .show = show_l3_1_id,
	2505	+ .store = NULL,
	2506	+};
	2507	+
	2508	+static struct attribute *attrs_l3_1[] = {
	2509	+ &dev_attr_l3_1_id.attr,
	2510	+ NULL,
	2511	+};
	2512	+
	2513	+static struct attribute_group group_l3_1 = {
	2514	+ .name = "78a87ff9-543a-49ce-95ea-26d86071ea93",
	2515	+ .attrs = attrs_l3_1,
	2516	+};
	2517	+
	2518	+static ssize_t
	2519	+show_l3_2_id(struct device kdev, struct device_attribute attr, char *buf)
	2520	+{
	2521	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_2);
	2522	+}
	2523	+
	2524	+static struct device_attribute dev_attr_l3_2_id = {
	2525	+ .attr = { .name = "id", .mode = 0444 },
	2526	+ .show = show_l3_2_id,
	2527	+ .store = NULL,
	2528	+};
	2529	+
	2530	+static struct attribute *attrs_l3_2[] = {
	2531	+ &dev_attr_l3_2_id.attr,
	2532	+ NULL,
	2533	+};
	2534	+
	2535	+static struct attribute_group group_l3_2 = {
	2536	+ .name = "9f2cece5-7bfe-4320-ad66-8c7cc526bec5",
	2537	+ .attrs = attrs_l3_2,
	2538	+};
	2539	+
	2540	+static ssize_t
	2541	+show_l3_3_id(struct device kdev, struct device_attribute attr, char *buf)
	2542	+{
	2543	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_3);
	2544	+}
	2545	+
	2546	+static struct device_attribute dev_attr_l3_3_id = {
	2547	+ .attr = { .name = "id", .mode = 0444 },
	2548	+ .show = show_l3_3_id,
	2549	+ .store = NULL,
	2550	+};
	2551	+
	2552	+static struct attribute *attrs_l3_3[] = {
	2553	+ &dev_attr_l3_3_id.attr,
	2554	+ NULL,
	2555	+};
	2556	+
	2557	+static struct attribute_group group_l3_3 = {
	2558	+ .name = "d890ef38-d309-47e4-b8b5-aa779bb19ab0",
	2559	+ .attrs = attrs_l3_3,
	2560	+};
	2561	+
	2562	+static ssize_t
	2563	+show_l3_4_id(struct device kdev, struct device_attribute attr, char *buf)
	2564	+{
	2565	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_4);
	2566	+}
	2567	+
	2568	+static struct device_attribute dev_attr_l3_4_id = {
	2569	+ .attr = { .name = "id", .mode = 0444 },
	2570	+ .show = show_l3_4_id,
	2571	+ .store = NULL,
	2572	+};
	2573	+
	2574	+static struct attribute *attrs_l3_4[] = {
	2575	+ &dev_attr_l3_4_id.attr,
	2576	+ NULL,
	2577	+};
	2578	+
	2579	+static struct attribute_group group_l3_4 = {
	2580	+ .name = "5fdff4a6-9dc8-45e1-bfda-ef54869fbdd4",
	2581	+ .attrs = attrs_l3_4,
	2582	+};
	2583	+
	2584	+static ssize_t
	2585	+show_rasterizer_and_pixel_backend_id(struct device kdev, struct device_attribute attr, char *buf)
	2586	+{
	2587	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND);
	2588	+}
	2589	+
	2590	+static struct device_attribute dev_attr_rasterizer_and_pixel_backend_id = {
	2591	+ .attr = { .name = "id", .mode = 0444 },
	2592	+ .show = show_rasterizer_and_pixel_backend_id,
	2593	+ .store = NULL,
	2594	+};
	2595	+
	2596	+static struct attribute *attrs_rasterizer_and_pixel_backend[] = {
	2597	+ &dev_attr_rasterizer_and_pixel_backend_id.attr,
	2598	+ NULL,
	2599	+};
	2600	+
	2601	+static struct attribute_group group_rasterizer_and_pixel_backend = {
	2602	+ .name = "2c0e45e1-7e2c-4a14-ae00-0b7ec868b8aa",
	2603	+ .attrs = attrs_rasterizer_and_pixel_backend,
	2604	+};
	2605	+
	2606	+static ssize_t
	2607	+show_sampler_1_id(struct device kdev, struct device_attribute attr, char *buf)
	2608	+{
	2609	+ return sprintf(buf, "%d\n", METRIC_SET_ID_SAMPLER_1);
	2610	+}
	2611	+
	2612	+static struct device_attribute dev_attr_sampler_1_id = {
	2613	+ .attr = { .name = "id", .mode = 0444 },
	2614	+ .show = show_sampler_1_id,
	2615	+ .store = NULL,
	2616	+};
	2617	+
	2618	+static struct attribute *attrs_sampler_1[] = {
	2619	+ &dev_attr_sampler_1_id.attr,
	2620	+ NULL,
	2621	+};
	2622	+
	2623	+static struct attribute_group group_sampler_1 = {
	2624	+ .name = "71148d78-baf5-474f-878a-e23158d0265d",
	2625	+ .attrs = attrs_sampler_1,
	2626	+};
	2627	+
	2628	+static ssize_t
	2629	+show_sampler_2_id(struct device kdev, struct device_attribute attr, char *buf)
	2630	+{
	2631	+ return sprintf(buf, "%d\n", METRIC_SET_ID_SAMPLER_2);
	2632	+}
	2633	+
	2634	+static struct device_attribute dev_attr_sampler_2_id = {
	2635	+ .attr = { .name = "id", .mode = 0444 },
	2636	+ .show = show_sampler_2_id,
	2637	+ .store = NULL,
	2638	+};
	2639	+
	2640	+static struct attribute *attrs_sampler_2[] = {
	2641	+ &dev_attr_sampler_2_id.attr,
	2642	+ NULL,
	2643	+};
	2644	+
	2645	+static struct attribute_group group_sampler_2 = {
	2646	+ .name = "b996a2b7-c59c-492d-877a-8cd54fd6df84",
	2647	+ .attrs = attrs_sampler_2,
	2648	+};
	2649	+
	2650	+static ssize_t
	2651	+show_tdl_1_id(struct device kdev, struct device_attribute attr, char *buf)
	2652	+{
	2653	+ return sprintf(buf, "%d\n", METRIC_SET_ID_TDL_1);
	2654	+}
	2655	+
	2656	+static struct device_attribute dev_attr_tdl_1_id = {
	2657	+ .attr = { .name = "id", .mode = 0444 },
	2658	+ .show = show_tdl_1_id,
	2659	+ .store = NULL,
	2660	+};
	2661	+
	2662	+static struct attribute *attrs_tdl_1[] = {
	2663	+ &dev_attr_tdl_1_id.attr,
	2664	+ NULL,
	2665	+};
	2666	+
	2667	+static struct attribute_group group_tdl_1 = {
	2668	+ .name = "eb2fecba-b431-42e7-8261-fe9429a6e67a",
	2669	+ .attrs = attrs_tdl_1,
	2670	+};
	2671	+
	2672	+static ssize_t
	2673	+show_tdl_2_id(struct device kdev, struct device_attribute attr, char *buf)
	2674	+{
	2675	+ return sprintf(buf, "%d\n", METRIC_SET_ID_TDL_2);
	2676	+}
	2677	+
	2678	+static struct device_attribute dev_attr_tdl_2_id = {
	2679	+ .attr = { .name = "id", .mode = 0444 },
	2680	+ .show = show_tdl_2_id,
	2681	+ .store = NULL,
	2682	+};
	2683	+
	2684	+static struct attribute *attrs_tdl_2[] = {
	2685	+ &dev_attr_tdl_2_id.attr,
	2686	+ NULL,
	2687	+};
	2688	+
	2689	+static struct attribute_group group_tdl_2 = {
	2690	+ .name = "60749470-a648-4a4b-9f10-dbfe1e36e44d",
	2691	+ .attrs = attrs_tdl_2,
	2692	+};
	2693	+
	2694	+static ssize_t
	2695	+show_test_oa_id(struct device kdev, struct device_attribute attr, char *buf)
	2696	+{
	2697	+ return sprintf(buf, "%d\n", METRIC_SET_ID_TEST_OA);
	2698	+}
	2699	+
	2700	+static struct device_attribute dev_attr_test_oa_id = {
	2701	+ .attr = { .name = "id", .mode = 0444 },
	2702	+ .show = show_test_oa_id,
	2703	+ .store = NULL,
	2704	+};
	2705	+
	2706	+static struct attribute *attrs_test_oa[] = {
	2707	+ &dev_attr_test_oa_id.attr,
	2708	+ NULL,
	2709	+};
	2710	+
	2711	+static struct attribute_group group_test_oa = {
	2712	+ .name = "4a534b07-cba3-414d-8d60-874830e883aa",
	2713	+ .attrs = attrs_test_oa,
	2714	+};
	2715	+
	2716	+int
	2717	+i915_perf_register_sysfs_chv(struct drm_i915_private *dev_priv)
	2718	+{
	2719	+ const struct i915_oa_reg *mux_regs[ARRAY_SIZE(dev_priv->perf.oa.mux_regs)];
	2720	+ int mux_lens[ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens)];
	2721	+ int ret = 0;
	2722	+
	2723	+ if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens)) {
	2724	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_render_basic);
	2725	+ if (ret)
	2726	+ goto error_render_basic;
	2727	+ }
	2728	+ if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens)) {
	2729	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
	2730	+ if (ret)
	2731	+ goto error_compute_basic;
	2732	+ }
	2733	+ if (get_render_pipe_profile_mux_config(dev_priv, mux_regs, mux_lens)) {
	2734	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_render_pipe_profile);
	2735	+ if (ret)
	2736	+ goto error_render_pipe_profile;
	2737	+ }
	2738	+ if (get_hdc_and_sf_mux_config(dev_priv, mux_regs, mux_lens)) {
	2739	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_hdc_and_sf);
	2740	+ if (ret)
	2741	+ goto error_hdc_and_sf;
	2742	+ }
	2743	+ if (get_l3_1_mux_config(dev_priv, mux_regs, mux_lens)) {
	2744	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_l3_1);
	2745	+ if (ret)
	2746	+ goto error_l3_1;
	2747	+ }
	2748	+ if (get_l3_2_mux_config(dev_priv, mux_regs, mux_lens)) {
	2749	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_l3_2);
	2750	+ if (ret)
	2751	+ goto error_l3_2;
	2752	+ }
	2753	+ if (get_l3_3_mux_config(dev_priv, mux_regs, mux_lens)) {
	2754	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_l3_3);
	2755	+ if (ret)
	2756	+ goto error_l3_3;
	2757	+ }
	2758	+ if (get_l3_4_mux_config(dev_priv, mux_regs, mux_lens)) {
	2759	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_l3_4);
	2760	+ if (ret)
	2761	+ goto error_l3_4;
	2762	+ }
	2763	+ if (get_rasterizer_and_pixel_backend_mux_config(dev_priv, mux_regs, mux_lens)) {
	2764	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_rasterizer_and_pixel_backend);
	2765	+ if (ret)
	2766	+ goto error_rasterizer_and_pixel_backend;
	2767	+ }
	2768	+ if (get_sampler_1_mux_config(dev_priv, mux_regs, mux_lens)) {
	2769	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_sampler_1);
	2770	+ if (ret)
	2771	+ goto error_sampler_1;
	2772	+ }
	2773	+ if (get_sampler_2_mux_config(dev_priv, mux_regs, mux_lens)) {
	2774	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_sampler_2);
	2775	+ if (ret)
	2776	+ goto error_sampler_2;
	2777	+ }
	2778	+ if (get_tdl_1_mux_config(dev_priv, mux_regs, mux_lens)) {
	2779	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_tdl_1);
	2780	+ if (ret)
	2781	+ goto error_tdl_1;
	2782	+ }
	2783	+ if (get_tdl_2_mux_config(dev_priv, mux_regs, mux_lens)) {
	2784	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_tdl_2);
	2785	+ if (ret)
	2786	+ goto error_tdl_2;
	2787	+ }
	2788	+ if (get_test_oa_mux_config(dev_priv, mux_regs, mux_lens)) {
	2789	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_test_oa);
	2790	+ if (ret)
	2791	+ goto error_test_oa;
	2792	+ }
	2793	+
	2794	+ return 0;
	2795	+
	2796	+error_test_oa:
	2797	+ if (get_tdl_2_mux_config(dev_priv, mux_regs, mux_lens))
	2798	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_tdl_2);
	2799	+error_tdl_2:
	2800	+ if (get_tdl_1_mux_config(dev_priv, mux_regs, mux_lens))
	2801	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_tdl_1);
	2802	+error_tdl_1:
	2803	+ if (get_sampler_2_mux_config(dev_priv, mux_regs, mux_lens))
	2804	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_sampler_2);
	2805	+error_sampler_2:
	2806	+ if (get_sampler_1_mux_config(dev_priv, mux_regs, mux_lens))
	2807	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_sampler_1);
	2808	+error_sampler_1:
	2809	+ if (get_rasterizer_and_pixel_backend_mux_config(dev_priv, mux_regs, mux_lens))
	2810	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_rasterizer_and_pixel_backend);
	2811	+error_rasterizer_and_pixel_backend:
	2812	+ if (get_l3_4_mux_config(dev_priv, mux_regs, mux_lens))
	2813	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_4);
	2814	+error_l3_4:
	2815	+ if (get_l3_3_mux_config(dev_priv, mux_regs, mux_lens))
	2816	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_3);
	2817	+error_l3_3:
	2818	+ if (get_l3_2_mux_config(dev_priv, mux_regs, mux_lens))
	2819	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_2);
	2820	+error_l3_2:
	2821	+ if (get_l3_1_mux_config(dev_priv, mux_regs, mux_lens))
	2822	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_1);
	2823	+error_l3_1:
	2824	+ if (get_hdc_and_sf_mux_config(dev_priv, mux_regs, mux_lens))
	2825	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_hdc_and_sf);
	2826	+error_hdc_and_sf:
	2827	+ if (get_render_pipe_profile_mux_config(dev_priv, mux_regs, mux_lens))
	2828	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_pipe_profile);
	2829	+error_render_pipe_profile:
	2830	+ if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens))
	2831	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
	2832	+error_compute_basic:
	2833	+ if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens))
	2834	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_basic);
	2835	+error_render_basic:
	2836	+ return ret;
	2837	+}
	2838	+
	2839	+void
	2840	+i915_perf_unregister_sysfs_chv(struct drm_i915_private *dev_priv)
	2841	+{
	2842	+ const struct i915_oa_reg *mux_regs[ARRAY_SIZE(dev_priv->perf.oa.mux_regs)];
	2843	+ int mux_lens[ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens)];
	2844	+
	2845	+ if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens))
	2846	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_basic);
	2847	+ if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens))
	2848	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
	2849	+ if (get_render_pipe_profile_mux_config(dev_priv, mux_regs, mux_lens))
	2850	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_render_pipe_profile);
	2851	+ if (get_hdc_and_sf_mux_config(dev_priv, mux_regs, mux_lens))
	2852	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_hdc_and_sf);
	2853	+ if (get_l3_1_mux_config(dev_priv, mux_regs, mux_lens))
	2854	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_1);
	2855	+ if (get_l3_2_mux_config(dev_priv, mux_regs, mux_lens))
	2856	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_2);
	2857	+ if (get_l3_3_mux_config(dev_priv, mux_regs, mux_lens))
	2858	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_3);
	2859	+ if (get_l3_4_mux_config(dev_priv, mux_regs, mux_lens))
	2860	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_l3_4);
	2861	+ if (get_rasterizer_and_pixel_backend_mux_config(dev_priv, mux_regs, mux_lens))
	2862	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_rasterizer_and_pixel_backend);
	2863	+ if (get_sampler_1_mux_config(dev_priv, mux_regs, mux_lens))
	2864	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_sampler_1);
	2865	+ if (get_sampler_2_mux_config(dev_priv, mux_regs, mux_lens))
	2866	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_sampler_2);
	2867	+ if (get_tdl_1_mux_config(dev_priv, mux_regs, mux_lens))
	2868	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_tdl_1);
	2869	+ if (get_tdl_2_mux_config(dev_priv, mux_regs, mux_lens))
	2870	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_tdl_2);
	2871	+ if (get_test_oa_mux_config(dev_priv, mux_regs, mux_lens))
	2872	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_test_oa);
	2873	+}

drivers/gpu/drm/i915/i915_oa_chv.h+40−0

@@ -0,0 +1,40 @@
	1	+/*
	2	+ * Autogenerated file by GPU Top : https://github.com/rib/gputop
	3	+ * DO NOT EDIT manually!
	4	+ *
	5	+ *
	6	+ * Copyright (c) 2015 Intel Corporation
	7	+ *
	8	+ * Permission is hereby granted, free of charge, to any person obtaining a
	9	+ * copy of this software and associated documentation files (the "Software"),
	10	+ * to deal in the Software without restriction, including without limitation
	11	+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
	12	+ * and/or sell copies of the Software, and to permit persons to whom the
	13	+ * Software is furnished to do so, subject to the following conditions:
	14	+ *
	15	+ * The above copyright notice and this permission notice (including the next
	16	+ * paragraph) shall be included in all copies or substantial portions of the
	17	+ * Software.
	18	+ *
	19	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
	20	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
	21	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
	22	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
	23	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
	24	+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
	25	+ * IN THE SOFTWARE.
	26	+ *
	27	+ */
	28	+
	29	+#ifndef __I915_OA_CHV_H__
	30	+#define __I915_OA_CHV_H__
	31	+
	32	+extern int i915_oa_n_builtin_metric_sets_chv;
	33	+
	34	+extern int i915_oa_select_metric_set_chv(struct drm_i915_private *dev_priv);
	35	+
	36	+extern int i915_perf_register_sysfs_chv(struct drm_i915_private *dev_priv);
	37	+
	38	+extern void i915_perf_unregister_sysfs_chv(struct drm_i915_private *dev_priv);
	39	+
	40	+#endif

drivers/gpu/drm/i915/i915_oa_glk.c+2602−0

@@ -0,0 +1,2602 @@
	1	+/*
	2	+ * Autogenerated file by GPU Top : https://github.com/rib/gputop
	3	+ * DO NOT EDIT manually!
	4	+ *
	5	+ *
	6	+ * Copyright (c) 2015 Intel Corporation
	7	+ *
	8	+ * Permission is hereby granted, free of charge, to any person obtaining a
	9	+ * copy of this software and associated documentation files (the "Software"),
	10	+ * to deal in the Software without restriction, including without limitation
	11	+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
	12	+ * and/or sell copies of the Software, and to permit persons to whom the
	13	+ * Software is furnished to do so, subject to the following conditions:
	14	+ *
	15	+ * The above copyright notice and this permission notice (including the next
	16	+ * paragraph) shall be included in all copies or substantial portions of the
	17	+ * Software.
	18	+ *
	19	+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
	20	+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
	21	+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
	22	+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
	23	+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
	24	+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
	25	+ * IN THE SOFTWARE.
	26	+ *
	27	+ */
	28	+
	29	+#include <linux/sysfs.h>
	30	+
	31	+#include "i915_drv.h"
	32	+#include "i915_oa_glk.h"
	33	+
	34	+enum metric_set_id {
	35	+ METRIC_SET_ID_RENDER_BASIC = 1,
	36	+ METRIC_SET_ID_COMPUTE_BASIC,
	37	+ METRIC_SET_ID_RENDER_PIPE_PROFILE,
	38	+ METRIC_SET_ID_MEMORY_READS,
	39	+ METRIC_SET_ID_MEMORY_WRITES,
	40	+ METRIC_SET_ID_COMPUTE_EXTENDED,
	41	+ METRIC_SET_ID_COMPUTE_L3_CACHE,
	42	+ METRIC_SET_ID_HDC_AND_SF,
	43	+ METRIC_SET_ID_L3_1,
	44	+ METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND,
	45	+ METRIC_SET_ID_SAMPLER,
	46	+ METRIC_SET_ID_TDL_1,
	47	+ METRIC_SET_ID_TDL_2,
	48	+ METRIC_SET_ID_COMPUTE_EXTRA,
	49	+ METRIC_SET_ID_TEST_OA,
	50	+};
	51	+
	52	+int i915_oa_n_builtin_metric_sets_glk = 15;
	53	+
	54	+static const struct i915_oa_reg b_counter_config_render_basic[] = {
	55	+ { _MMIO(0x2710), 0x00000000 },
	56	+ { _MMIO(0x2714), 0x00800000 },
	57	+ { _MMIO(0x2720), 0x00000000 },
	58	+ { _MMIO(0x2724), 0x00800000 },
	59	+ { _MMIO(0x2740), 0x00000000 },
	60	+};
	61	+
	62	+static const struct i915_oa_reg flex_eu_config_render_basic[] = {
	63	+ { _MMIO(0xe458), 0x00005004 },
	64	+ { _MMIO(0xe558), 0x00010003 },
	65	+ { _MMIO(0xe658), 0x00012011 },
	66	+ { _MMIO(0xe758), 0x00015014 },
	67	+ { _MMIO(0xe45c), 0x00051050 },
	68	+ { _MMIO(0xe55c), 0x00053052 },
	69	+ { _MMIO(0xe65c), 0x00055054 },
	70	+};
	71	+
	72	+static const struct i915_oa_reg mux_config_render_basic[] = {
	73	+ { _MMIO(0x9888), 0x166c00f0 },
	74	+ { _MMIO(0x9888), 0x12120280 },
	75	+ { _MMIO(0x9888), 0x12320280 },
	76	+ { _MMIO(0x9888), 0x11930317 },
	77	+ { _MMIO(0x9888), 0x159303df },
	78	+ { _MMIO(0x9888), 0x3f900c00 },
	79	+ { _MMIO(0x9888), 0x419000a0 },
	80	+ { _MMIO(0x9888), 0x002d1000 },
	81	+ { _MMIO(0x9888), 0x062d4000 },
	82	+ { _MMIO(0x9888), 0x082d5000 },
	83	+ { _MMIO(0x9888), 0x0a2d1000 },
	84	+ { _MMIO(0x9888), 0x0c2e0800 },
	85	+ { _MMIO(0x9888), 0x0e2e5900 },
	86	+ { _MMIO(0x9888), 0x0a4c8000 },
	87	+ { _MMIO(0x9888), 0x0c4c8000 },
	88	+ { _MMIO(0x9888), 0x0e4c4000 },
	89	+ { _MMIO(0x9888), 0x064e8000 },
	90	+ { _MMIO(0x9888), 0x084e8000 },
	91	+ { _MMIO(0x9888), 0x0a4e2000 },
	92	+ { _MMIO(0x9888), 0x1c4f0010 },
	93	+ { _MMIO(0x9888), 0x0a6c0053 },
	94	+ { _MMIO(0x9888), 0x106c0000 },
	95	+ { _MMIO(0x9888), 0x1c6c0000 },
	96	+ { _MMIO(0x9888), 0x1a0fcc00 },
	97	+ { _MMIO(0x9888), 0x1c0f0002 },
	98	+ { _MMIO(0x9888), 0x1c2c0040 },
	99	+ { _MMIO(0x9888), 0x00101000 },
	100	+ { _MMIO(0x9888), 0x04101000 },
	101	+ { _MMIO(0x9888), 0x00114000 },
	102	+ { _MMIO(0x9888), 0x08114000 },
	103	+ { _MMIO(0x9888), 0x00120020 },
	104	+ { _MMIO(0x9888), 0x08120021 },
	105	+ { _MMIO(0x9888), 0x00141000 },
	106	+ { _MMIO(0x9888), 0x08141000 },
	107	+ { _MMIO(0x9888), 0x02308000 },
	108	+ { _MMIO(0x9888), 0x04302000 },
	109	+ { _MMIO(0x9888), 0x06318000 },
	110	+ { _MMIO(0x9888), 0x08318000 },
	111	+ { _MMIO(0x9888), 0x06320800 },
	112	+ { _MMIO(0x9888), 0x08320840 },
	113	+ { _MMIO(0x9888), 0x00320000 },
	114	+ { _MMIO(0x9888), 0x06344000 },
	115	+ { _MMIO(0x9888), 0x08344000 },
	116	+ { _MMIO(0x9888), 0x0d931831 },
	117	+ { _MMIO(0x9888), 0x0f939f3f },
	118	+ { _MMIO(0x9888), 0x01939e80 },
	119	+ { _MMIO(0x9888), 0x039303bc },
	120	+ { _MMIO(0x9888), 0x0593000e },
	121	+ { _MMIO(0x9888), 0x1993002a },
	122	+ { _MMIO(0x9888), 0x07930000 },
	123	+ { _MMIO(0x9888), 0x09930000 },
	124	+ { _MMIO(0x9888), 0x1d900177 },
	125	+ { _MMIO(0x9888), 0x1f900187 },
	126	+ { _MMIO(0x9888), 0x35900000 },
	127	+ { _MMIO(0x9888), 0x13904000 },
	128	+ { _MMIO(0x9888), 0x21904000 },
	129	+ { _MMIO(0x9888), 0x23904000 },
	130	+ { _MMIO(0x9888), 0x25904000 },
	131	+ { _MMIO(0x9888), 0x27904000 },
	132	+ { _MMIO(0x9888), 0x2b904000 },
	133	+ { _MMIO(0x9888), 0x2d904000 },
	134	+ { _MMIO(0x9888), 0x2f904000 },
	135	+ { _MMIO(0x9888), 0x31904000 },
	136	+ { _MMIO(0x9888), 0x15904000 },
	137	+ { _MMIO(0x9888), 0x17904000 },
	138	+ { _MMIO(0x9888), 0x19904000 },
	139	+ { _MMIO(0x9888), 0x1b904000 },
	140	+ { _MMIO(0x9888), 0x53901110 },
	141	+ { _MMIO(0x9888), 0x43900423 },
	142	+ { _MMIO(0x9888), 0x55900111 },
	143	+ { _MMIO(0x9888), 0x47900c02 },
	144	+ { _MMIO(0x9888), 0x57900000 },
	145	+ { _MMIO(0x9888), 0x49900020 },
	146	+ { _MMIO(0x9888), 0x59901111 },
	147	+ { _MMIO(0x9888), 0x4b900421 },
	148	+ { _MMIO(0x9888), 0x37900000 },
	149	+ { _MMIO(0x9888), 0x33900000 },
	150	+ { _MMIO(0x9888), 0x4d900001 },
	151	+ { _MMIO(0x9888), 0x45900821 },
	152	+};
	153	+
	154	+static int
	155	+get_render_basic_mux_config(struct drm_i915_private *dev_priv,
	156	+ const struct i915_oa_reg **regs,
	157	+ int *lens)
	158	+{
	159	+ int n = 0;
	160	+
	161	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	162	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	163	+
	164	+ regs[n] = mux_config_render_basic;
	165	+ lens[n] = ARRAY_SIZE(mux_config_render_basic);
	166	+ n++;
	167	+
	168	+ return n;
	169	+}
	170	+
	171	+static const struct i915_oa_reg b_counter_config_compute_basic[] = {
	172	+ { _MMIO(0x2710), 0x00000000 },
	173	+ { _MMIO(0x2714), 0x00800000 },
	174	+ { _MMIO(0x2720), 0x00000000 },
	175	+ { _MMIO(0x2724), 0x00800000 },
	176	+ { _MMIO(0x2740), 0x00000000 },
	177	+};
	178	+
	179	+static const struct i915_oa_reg flex_eu_config_compute_basic[] = {
	180	+ { _MMIO(0xe458), 0x00005004 },
	181	+ { _MMIO(0xe558), 0x00000003 },
	182	+ { _MMIO(0xe658), 0x00002001 },
	183	+ { _MMIO(0xe758), 0x00778008 },
	184	+ { _MMIO(0xe45c), 0x00088078 },
	185	+ { _MMIO(0xe55c), 0x00808708 },
	186	+ { _MMIO(0xe65c), 0x00a08908 },
	187	+};
	188	+
	189	+static const struct i915_oa_reg mux_config_compute_basic[] = {
	190	+ { _MMIO(0x9888), 0x104f00e0 },
	191	+ { _MMIO(0x9888), 0x124f1c00 },
	192	+ { _MMIO(0x9888), 0x39900340 },
	193	+ { _MMIO(0x9888), 0x3f900c00 },
	194	+ { _MMIO(0x9888), 0x41900000 },
	195	+ { _MMIO(0x9888), 0x002d5000 },
	196	+ { _MMIO(0x9888), 0x062d4000 },
	197	+ { _MMIO(0x9888), 0x082d4000 },
	198	+ { _MMIO(0x9888), 0x0a2d1000 },
	199	+ { _MMIO(0x9888), 0x0c2d5000 },
	200	+ { _MMIO(0x9888), 0x0e2d4000 },
	201	+ { _MMIO(0x9888), 0x0c2e1400 },
	202	+ { _MMIO(0x9888), 0x0e2e5100 },
	203	+ { _MMIO(0x9888), 0x102e0114 },
	204	+ { _MMIO(0x9888), 0x044cc000 },
	205	+ { _MMIO(0x9888), 0x0a4c8000 },
	206	+ { _MMIO(0x9888), 0x0c4c8000 },
	207	+ { _MMIO(0x9888), 0x0e4c4000 },
	208	+ { _MMIO(0x9888), 0x104c8000 },
	209	+ { _MMIO(0x9888), 0x124c8000 },
	210	+ { _MMIO(0x9888), 0x164c2000 },
	211	+ { _MMIO(0x9888), 0x004ea000 },
	212	+ { _MMIO(0x9888), 0x064e8000 },
	213	+ { _MMIO(0x9888), 0x084e8000 },
	214	+ { _MMIO(0x9888), 0x0a4e2000 },
	215	+ { _MMIO(0x9888), 0x0c4ea000 },
	216	+ { _MMIO(0x9888), 0x0e4e8000 },
	217	+ { _MMIO(0x9888), 0x004f6b42 },
	218	+ { _MMIO(0x9888), 0x064f6200 },
	219	+ { _MMIO(0x9888), 0x084f4100 },
	220	+ { _MMIO(0x9888), 0x0a4f0061 },
	221	+ { _MMIO(0x9888), 0x0c4f6c4c },
	222	+ { _MMIO(0x9888), 0x0e4f4b00 },
	223	+ { _MMIO(0x9888), 0x1a4f0000 },
	224	+ { _MMIO(0x9888), 0x1c4f0000 },
	225	+ { _MMIO(0x9888), 0x180f5000 },
	226	+ { _MMIO(0x9888), 0x1a0f8800 },
	227	+ { _MMIO(0x9888), 0x1c0f08a2 },
	228	+ { _MMIO(0x9888), 0x182c4000 },
	229	+ { _MMIO(0x9888), 0x1c2c1451 },
	230	+ { _MMIO(0x9888), 0x1e2c0001 },
	231	+ { _MMIO(0x9888), 0x1a2c0010 },
	232	+ { _MMIO(0x9888), 0x01938000 },
	233	+ { _MMIO(0x9888), 0x0f938000 },
	234	+ { _MMIO(0x9888), 0x19938a28 },
	235	+ { _MMIO(0x9888), 0x03938000 },
	236	+ { _MMIO(0x9888), 0x19900177 },
	237	+ { _MMIO(0x9888), 0x1b900178 },
	238	+ { _MMIO(0x9888), 0x1d900125 },
	239	+ { _MMIO(0x9888), 0x1f900123 },
	240	+ { _MMIO(0x9888), 0x35900000 },
	241	+ { _MMIO(0x9888), 0x13904000 },
	242	+ { _MMIO(0x9888), 0x21904000 },
	243	+ { _MMIO(0x9888), 0x25904000 },
	244	+ { _MMIO(0x9888), 0x27904000 },
	245	+ { _MMIO(0x9888), 0x2b904000 },
	246	+ { _MMIO(0x9888), 0x2d904000 },
	247	+ { _MMIO(0x9888), 0x31904000 },
	248	+ { _MMIO(0x9888), 0x15904000 },
	249	+ { _MMIO(0x9888), 0x53901000 },
	250	+ { _MMIO(0x9888), 0x43900000 },
	251	+ { _MMIO(0x9888), 0x55900111 },
	252	+ { _MMIO(0x9888), 0x47900000 },
	253	+ { _MMIO(0x9888), 0x57900000 },
	254	+ { _MMIO(0x9888), 0x49900000 },
	255	+ { _MMIO(0x9888), 0x59900000 },
	256	+ { _MMIO(0x9888), 0x4b900000 },
	257	+ { _MMIO(0x9888), 0x37900000 },
	258	+ { _MMIO(0x9888), 0x33900000 },
	259	+ { _MMIO(0x9888), 0x4d900000 },
	260	+ { _MMIO(0x9888), 0x45900000 },
	261	+};
	262	+
	263	+static int
	264	+get_compute_basic_mux_config(struct drm_i915_private *dev_priv,
	265	+ const struct i915_oa_reg **regs,
	266	+ int *lens)
	267	+{
	268	+ int n = 0;
	269	+
	270	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	271	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	272	+
	273	+ regs[n] = mux_config_compute_basic;
	274	+ lens[n] = ARRAY_SIZE(mux_config_compute_basic);
	275	+ n++;
	276	+
	277	+ return n;
	278	+}
	279	+
	280	+static const struct i915_oa_reg b_counter_config_render_pipe_profile[] = {
	281	+ { _MMIO(0x2724), 0xf0800000 },
	282	+ { _MMIO(0x2720), 0x00000000 },
	283	+ { _MMIO(0x2714), 0xf0800000 },
	284	+ { _MMIO(0x2710), 0x00000000 },
	285	+ { _MMIO(0x2740), 0x00000000 },
	286	+ { _MMIO(0x2770), 0x0007ffea },
	287	+ { _MMIO(0x2774), 0x00007ffc },
	288	+ { _MMIO(0x2778), 0x0007affa },
	289	+ { _MMIO(0x277c), 0x0000f5fd },
	290	+ { _MMIO(0x2780), 0x00079ffa },
	291	+ { _MMIO(0x2784), 0x0000f3fb },
	292	+ { _MMIO(0x2788), 0x0007bf7a },
	293	+ { _MMIO(0x278c), 0x0000f7e7 },
	294	+ { _MMIO(0x2790), 0x0007fefa },
	295	+ { _MMIO(0x2794), 0x0000f7cf },
	296	+ { _MMIO(0x2798), 0x00077ffa },
	297	+ { _MMIO(0x279c), 0x0000efdf },
	298	+ { _MMIO(0x27a0), 0x0006fffa },
	299	+ { _MMIO(0x27a4), 0x0000cfbf },
	300	+ { _MMIO(0x27a8), 0x0003fffa },
	301	+ { _MMIO(0x27ac), 0x00005f7f },
	302	+};
	303	+
	304	+static const struct i915_oa_reg flex_eu_config_render_pipe_profile[] = {
	305	+ { _MMIO(0xe458), 0x00005004 },
	306	+ { _MMIO(0xe558), 0x00015014 },
	307	+ { _MMIO(0xe658), 0x00025024 },
	308	+ { _MMIO(0xe758), 0x00035034 },
	309	+ { _MMIO(0xe45c), 0x00045044 },
	310	+ { _MMIO(0xe55c), 0x00055054 },
	311	+ { _MMIO(0xe65c), 0x00065064 },
	312	+};
	313	+
	314	+static const struct i915_oa_reg mux_config_render_pipe_profile[] = {
	315	+ { _MMIO(0x9888), 0x0c2e001f },
	316	+ { _MMIO(0x9888), 0x0a2f0000 },
	317	+ { _MMIO(0x9888), 0x10186800 },
	318	+ { _MMIO(0x9888), 0x11810019 },
	319	+ { _MMIO(0x9888), 0x15810013 },
	320	+ { _MMIO(0x9888), 0x13820020 },
	321	+ { _MMIO(0x9888), 0x11830020 },
	322	+ { _MMIO(0x9888), 0x17840000 },
	323	+ { _MMIO(0x9888), 0x11860007 },
	324	+ { _MMIO(0x9888), 0x21860000 },
	325	+ { _MMIO(0x9888), 0x178703e0 },
	326	+ { _MMIO(0x9888), 0x0c2d8000 },
	327	+ { _MMIO(0x9888), 0x042d4000 },
	328	+ { _MMIO(0x9888), 0x062d1000 },
	329	+ { _MMIO(0x9888), 0x022e5400 },
	330	+ { _MMIO(0x9888), 0x002e0000 },
	331	+ { _MMIO(0x9888), 0x0e2e0080 },
	332	+ { _MMIO(0x9888), 0x082f0040 },
	333	+ { _MMIO(0x9888), 0x002f0000 },
	334	+ { _MMIO(0x9888), 0x06143000 },
	335	+ { _MMIO(0x9888), 0x06174000 },
	336	+ { _MMIO(0x9888), 0x06180012 },
	337	+ { _MMIO(0x9888), 0x00180000 },
	338	+ { _MMIO(0x9888), 0x0d804000 },
	339	+ { _MMIO(0x9888), 0x0f804000 },
	340	+ { _MMIO(0x9888), 0x05804000 },
	341	+ { _MMIO(0x9888), 0x09810200 },
	342	+ { _MMIO(0x9888), 0x0b810030 },
	343	+ { _MMIO(0x9888), 0x03810003 },
	344	+ { _MMIO(0x9888), 0x21819140 },
	345	+ { _MMIO(0x9888), 0x23819050 },
	346	+ { _MMIO(0x9888), 0x25810018 },
	347	+ { _MMIO(0x9888), 0x0b820980 },
	348	+ { _MMIO(0x9888), 0x03820d80 },
	349	+ { _MMIO(0x9888), 0x11820000 },
	350	+ { _MMIO(0x9888), 0x0182c000 },
	351	+ { _MMIO(0x9888), 0x07828000 },
	352	+ { _MMIO(0x9888), 0x09824000 },
	353	+ { _MMIO(0x9888), 0x0f828000 },
	354	+ { _MMIO(0x9888), 0x0d830004 },
	355	+ { _MMIO(0x9888), 0x0583000c },
	356	+ { _MMIO(0x9888), 0x0f831000 },
	357	+ { _MMIO(0x9888), 0x01848072 },
	358	+ { _MMIO(0x9888), 0x11840000 },
	359	+ { _MMIO(0x9888), 0x07848000 },
	360	+ { _MMIO(0x9888), 0x09844000 },
	361	+ { _MMIO(0x9888), 0x0f848000 },
	362	+ { _MMIO(0x9888), 0x07860000 },
	363	+ { _MMIO(0x9888), 0x09860092 },
	364	+ { _MMIO(0x9888), 0x0f860400 },
	365	+ { _MMIO(0x9888), 0x01869100 },
	366	+ { _MMIO(0x9888), 0x0f870065 },
	367	+ { _MMIO(0x9888), 0x01870000 },
	368	+ { _MMIO(0x9888), 0x19930800 },
	369	+ { _MMIO(0x9888), 0x0b938000 },
	370	+ { _MMIO(0x9888), 0x0d938000 },
	371	+ { _MMIO(0x9888), 0x1b952000 },
	372	+ { _MMIO(0x9888), 0x1d955055 },
	373	+ { _MMIO(0x9888), 0x1f951455 },
	374	+ { _MMIO(0x9888), 0x0992a000 },
	375	+ { _MMIO(0x9888), 0x0f928000 },
	376	+ { _MMIO(0x9888), 0x1192a800 },
	377	+ { _MMIO(0x9888), 0x1392028a },
	378	+ { _MMIO(0x9888), 0x0b92a000 },
	379	+ { _MMIO(0x9888), 0x0d922000 },
	380	+ { _MMIO(0x9888), 0x13908000 },
	381	+ { _MMIO(0x9888), 0x21908000 },
	382	+ { _MMIO(0x9888), 0x23908000 },
	383	+ { _MMIO(0x9888), 0x25908000 },
	384	+ { _MMIO(0x9888), 0x27908000 },
	385	+ { _MMIO(0x9888), 0x29908000 },
	386	+ { _MMIO(0x9888), 0x2b908000 },
	387	+ { _MMIO(0x9888), 0x2d904000 },
	388	+ { _MMIO(0x9888), 0x2f908000 },
	389	+ { _MMIO(0x9888), 0x31908000 },
	390	+ { _MMIO(0x9888), 0x15908000 },
	391	+ { _MMIO(0x9888), 0x17908000 },
	392	+ { _MMIO(0x9888), 0x19908000 },
	393	+ { _MMIO(0x9888), 0x1b908000 },
	394	+ { _MMIO(0x9888), 0x1d904000 },
	395	+ { _MMIO(0x9888), 0x1f904000 },
	396	+ { _MMIO(0x9888), 0x53900000 },
	397	+ { _MMIO(0x9888), 0x43900c01 },
	398	+ { _MMIO(0x9888), 0x55900000 },
	399	+ { _MMIO(0x9888), 0x47900000 },
	400	+ { _MMIO(0x9888), 0x57900000 },
	401	+ { _MMIO(0x9888), 0x49900863 },
	402	+ { _MMIO(0x9888), 0x59900000 },
	403	+ { _MMIO(0x9888), 0x4b900061 },
	404	+ { _MMIO(0x9888), 0x37900000 },
	405	+ { _MMIO(0x9888), 0x33900000 },
	406	+ { _MMIO(0x9888), 0x4d900000 },
	407	+ { _MMIO(0x9888), 0x45900c22 },
	408	+};
	409	+
	410	+static int
	411	+get_render_pipe_profile_mux_config(struct drm_i915_private *dev_priv,
	412	+ const struct i915_oa_reg **regs,
	413	+ int *lens)
	414	+{
	415	+ int n = 0;
	416	+
	417	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	418	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	419	+
	420	+ regs[n] = mux_config_render_pipe_profile;
	421	+ lens[n] = ARRAY_SIZE(mux_config_render_pipe_profile);
	422	+ n++;
	423	+
	424	+ return n;
	425	+}
	426	+
	427	+static const struct i915_oa_reg b_counter_config_memory_reads[] = {
	428	+ { _MMIO(0x272c), 0xffffffff },
	429	+ { _MMIO(0x2728), 0xffffffff },
	430	+ { _MMIO(0x2724), 0xf0800000 },
	431	+ { _MMIO(0x2720), 0x00000000 },
	432	+ { _MMIO(0x271c), 0xffffffff },
	433	+ { _MMIO(0x2718), 0xffffffff },
	434	+ { _MMIO(0x2714), 0xf0800000 },
	435	+ { _MMIO(0x2710), 0x00000000 },
	436	+ { _MMIO(0x274c), 0x86543210 },
	437	+ { _MMIO(0x2748), 0x86543210 },
	438	+ { _MMIO(0x2744), 0x00006667 },
	439	+ { _MMIO(0x2740), 0x00000000 },
	440	+ { _MMIO(0x275c), 0x86543210 },
	441	+ { _MMIO(0x2758), 0x86543210 },
	442	+ { _MMIO(0x2754), 0x00006465 },
	443	+ { _MMIO(0x2750), 0x00000000 },
	444	+ { _MMIO(0x2770), 0x0007f81a },
	445	+ { _MMIO(0x2774), 0x0000fe00 },
	446	+ { _MMIO(0x2778), 0x0007f82a },
	447	+ { _MMIO(0x277c), 0x0000fe00 },
	448	+ { _MMIO(0x2780), 0x0007f872 },
	449	+ { _MMIO(0x2784), 0x0000fe00 },
	450	+ { _MMIO(0x2788), 0x0007f8ba },
	451	+ { _MMIO(0x278c), 0x0000fe00 },
	452	+ { _MMIO(0x2790), 0x0007f87a },
	453	+ { _MMIO(0x2794), 0x0000fe00 },
	454	+ { _MMIO(0x2798), 0x0007f8ea },
	455	+ { _MMIO(0x279c), 0x0000fe00 },
	456	+ { _MMIO(0x27a0), 0x0007f8e2 },
	457	+ { _MMIO(0x27a4), 0x0000fe00 },
	458	+ { _MMIO(0x27a8), 0x0007f8f2 },
	459	+ { _MMIO(0x27ac), 0x0000fe00 },
	460	+};
	461	+
	462	+static const struct i915_oa_reg flex_eu_config_memory_reads[] = {
	463	+ { _MMIO(0xe458), 0x00005004 },
	464	+ { _MMIO(0xe558), 0x00015014 },
	465	+ { _MMIO(0xe658), 0x00025024 },
	466	+ { _MMIO(0xe758), 0x00035034 },
	467	+ { _MMIO(0xe45c), 0x00045044 },
	468	+ { _MMIO(0xe55c), 0x00055054 },
	469	+ { _MMIO(0xe65c), 0x00065064 },
	470	+};
	471	+
	472	+static const struct i915_oa_reg mux_config_memory_reads[] = {
	473	+ { _MMIO(0x9888), 0x19800343 },
	474	+ { _MMIO(0x9888), 0x39900340 },
	475	+ { _MMIO(0x9888), 0x3f901000 },
	476	+ { _MMIO(0x9888), 0x41900003 },
	477	+ { _MMIO(0x9888), 0x03803180 },
	478	+ { _MMIO(0x9888), 0x058035e2 },
	479	+ { _MMIO(0x9888), 0x0780006a },
	480	+ { _MMIO(0x9888), 0x11800000 },
	481	+ { _MMIO(0x9888), 0x2181a000 },
	482	+ { _MMIO(0x9888), 0x2381000a },
	483	+ { _MMIO(0x9888), 0x1d950550 },
	484	+ { _MMIO(0x9888), 0x0b928000 },
	485	+ { _MMIO(0x9888), 0x0d92a000 },
	486	+ { _MMIO(0x9888), 0x0f922000 },
	487	+ { _MMIO(0x9888), 0x13900170 },
	488	+ { _MMIO(0x9888), 0x21900171 },
	489	+ { _MMIO(0x9888), 0x23900172 },
	490	+ { _MMIO(0x9888), 0x25900173 },
	491	+ { _MMIO(0x9888), 0x27900174 },
	492	+ { _MMIO(0x9888), 0x29900175 },
	493	+ { _MMIO(0x9888), 0x2b900176 },
	494	+ { _MMIO(0x9888), 0x2d900177 },
	495	+ { _MMIO(0x9888), 0x2f90017f },
	496	+ { _MMIO(0x9888), 0x31900125 },
	497	+ { _MMIO(0x9888), 0x15900123 },
	498	+ { _MMIO(0x9888), 0x17900121 },
	499	+ { _MMIO(0x9888), 0x35900000 },
	500	+ { _MMIO(0x9888), 0x19908000 },
	501	+ { _MMIO(0x9888), 0x1b908000 },
	502	+ { _MMIO(0x9888), 0x1d908000 },
	503	+ { _MMIO(0x9888), 0x1f908000 },
	504	+ { _MMIO(0x9888), 0x53900000 },
	505	+ { _MMIO(0x9888), 0x43901084 },
	506	+ { _MMIO(0x9888), 0x55900000 },
	507	+ { _MMIO(0x9888), 0x47901080 },
	508	+ { _MMIO(0x9888), 0x57900000 },
	509	+ { _MMIO(0x9888), 0x49901084 },
	510	+ { _MMIO(0x9888), 0x59900000 },
	511	+ { _MMIO(0x9888), 0x4b901084 },
	512	+ { _MMIO(0x9888), 0x37900000 },
	513	+ { _MMIO(0x9888), 0x33900000 },
	514	+ { _MMIO(0x9888), 0x4d900004 },
	515	+ { _MMIO(0x9888), 0x45900000 },
	516	+};
	517	+
	518	+static int
	519	+get_memory_reads_mux_config(struct drm_i915_private *dev_priv,
	520	+ const struct i915_oa_reg **regs,
	521	+ int *lens)
	522	+{
	523	+ int n = 0;
	524	+
	525	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	526	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	527	+
	528	+ regs[n] = mux_config_memory_reads;
	529	+ lens[n] = ARRAY_SIZE(mux_config_memory_reads);
	530	+ n++;
	531	+
	532	+ return n;
	533	+}
	534	+
	535	+static const struct i915_oa_reg b_counter_config_memory_writes[] = {
	536	+ { _MMIO(0x272c), 0xffffffff },
	537	+ { _MMIO(0x2728), 0xffffffff },
	538	+ { _MMIO(0x2724), 0xf0800000 },
	539	+ { _MMIO(0x2720), 0x00000000 },
	540	+ { _MMIO(0x271c), 0xffffffff },
	541	+ { _MMIO(0x2718), 0xffffffff },
	542	+ { _MMIO(0x2714), 0xf0800000 },
	543	+ { _MMIO(0x2710), 0x00000000 },
	544	+ { _MMIO(0x274c), 0x86543210 },
	545	+ { _MMIO(0x2748), 0x86543210 },
	546	+ { _MMIO(0x2744), 0x00006667 },
	547	+ { _MMIO(0x2740), 0x00000000 },
	548	+ { _MMIO(0x275c), 0x86543210 },
	549	+ { _MMIO(0x2758), 0x86543210 },
	550	+ { _MMIO(0x2754), 0x00006465 },
	551	+ { _MMIO(0x2750), 0x00000000 },
	552	+ { _MMIO(0x2770), 0x0007f81a },
	553	+ { _MMIO(0x2774), 0x0000fe00 },
	554	+ { _MMIO(0x2778), 0x0007f82a },
	555	+ { _MMIO(0x277c), 0x0000fe00 },
	556	+ { _MMIO(0x2780), 0x0007f822 },
	557	+ { _MMIO(0x2784), 0x0000fe00 },
	558	+ { _MMIO(0x2788), 0x0007f8ba },
	559	+ { _MMIO(0x278c), 0x0000fe00 },
	560	+ { _MMIO(0x2790), 0x0007f87a },
	561	+ { _MMIO(0x2794), 0x0000fe00 },
	562	+ { _MMIO(0x2798), 0x0007f8ea },
	563	+ { _MMIO(0x279c), 0x0000fe00 },
	564	+ { _MMIO(0x27a0), 0x0007f8e2 },
	565	+ { _MMIO(0x27a4), 0x0000fe00 },
	566	+ { _MMIO(0x27a8), 0x0007f8f2 },
	567	+ { _MMIO(0x27ac), 0x0000fe00 },
	568	+};
	569	+
	570	+static const struct i915_oa_reg flex_eu_config_memory_writes[] = {
	571	+ { _MMIO(0xe458), 0x00005004 },
	572	+ { _MMIO(0xe558), 0x00015014 },
	573	+ { _MMIO(0xe658), 0x00025024 },
	574	+ { _MMIO(0xe758), 0x00035034 },
	575	+ { _MMIO(0xe45c), 0x00045044 },
	576	+ { _MMIO(0xe55c), 0x00055054 },
	577	+ { _MMIO(0xe65c), 0x00065064 },
	578	+};
	579	+
	580	+static const struct i915_oa_reg mux_config_memory_writes[] = {
	581	+ { _MMIO(0x9888), 0x19800343 },
	582	+ { _MMIO(0x9888), 0x39900340 },
	583	+ { _MMIO(0x9888), 0x3f900000 },
	584	+ { _MMIO(0x9888), 0x41900080 },
	585	+ { _MMIO(0x9888), 0x03803180 },
	586	+ { _MMIO(0x9888), 0x058035e2 },
	587	+ { _MMIO(0x9888), 0x0780006a },
	588	+ { _MMIO(0x9888), 0x11800000 },
	589	+ { _MMIO(0x9888), 0x2181a000 },
	590	+ { _MMIO(0x9888), 0x2381000a },
	591	+ { _MMIO(0x9888), 0x1d950550 },
	592	+ { _MMIO(0x9888), 0x0b928000 },
	593	+ { _MMIO(0x9888), 0x0d92a000 },
	594	+ { _MMIO(0x9888), 0x0f922000 },
	595	+ { _MMIO(0x9888), 0x13900180 },
	596	+ { _MMIO(0x9888), 0x21900181 },
	597	+ { _MMIO(0x9888), 0x23900182 },
	598	+ { _MMIO(0x9888), 0x25900183 },
	599	+ { _MMIO(0x9888), 0x27900184 },
	600	+ { _MMIO(0x9888), 0x29900185 },
	601	+ { _MMIO(0x9888), 0x2b900186 },
	602	+ { _MMIO(0x9888), 0x2d900187 },
	603	+ { _MMIO(0x9888), 0x2f900170 },
	604	+ { _MMIO(0x9888), 0x31900125 },
	605	+ { _MMIO(0x9888), 0x15900123 },
	606	+ { _MMIO(0x9888), 0x17900121 },
	607	+ { _MMIO(0x9888), 0x35900000 },
	608	+ { _MMIO(0x9888), 0x19908000 },
	609	+ { _MMIO(0x9888), 0x1b908000 },
	610	+ { _MMIO(0x9888), 0x1d908000 },
	611	+ { _MMIO(0x9888), 0x1f908000 },
	612	+ { _MMIO(0x9888), 0x53900000 },
	613	+ { _MMIO(0x9888), 0x43901084 },
	614	+ { _MMIO(0x9888), 0x55900000 },
	615	+ { _MMIO(0x9888), 0x47901080 },
	616	+ { _MMIO(0x9888), 0x57900000 },
	617	+ { _MMIO(0x9888), 0x49901084 },
	618	+ { _MMIO(0x9888), 0x59900000 },
	619	+ { _MMIO(0x9888), 0x4b901084 },
	620	+ { _MMIO(0x9888), 0x37900000 },
	621	+ { _MMIO(0x9888), 0x33900000 },
	622	+ { _MMIO(0x9888), 0x4d900004 },
	623	+ { _MMIO(0x9888), 0x45900000 },
	624	+};
	625	+
	626	+static int
	627	+get_memory_writes_mux_config(struct drm_i915_private *dev_priv,
	628	+ const struct i915_oa_reg **regs,
	629	+ int *lens)
	630	+{
	631	+ int n = 0;
	632	+
	633	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	634	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	635	+
	636	+ regs[n] = mux_config_memory_writes;
	637	+ lens[n] = ARRAY_SIZE(mux_config_memory_writes);
	638	+ n++;
	639	+
	640	+ return n;
	641	+}
	642	+
	643	+static const struct i915_oa_reg b_counter_config_compute_extended[] = {
	644	+ { _MMIO(0x2724), 0xf0800000 },
	645	+ { _MMIO(0x2720), 0x00000000 },
	646	+ { _MMIO(0x2714), 0xf0800000 },
	647	+ { _MMIO(0x2710), 0x00000000 },
	648	+ { _MMIO(0x2740), 0x00000000 },
	649	+ { _MMIO(0x2770), 0x0007fc2a },
	650	+ { _MMIO(0x2774), 0x0000bf00 },
	651	+ { _MMIO(0x2778), 0x0007fc6a },
	652	+ { _MMIO(0x277c), 0x0000bf00 },
	653	+ { _MMIO(0x2780), 0x0007fc92 },
	654	+ { _MMIO(0x2784), 0x0000bf00 },
	655	+ { _MMIO(0x2788), 0x0007fca2 },
	656	+ { _MMIO(0x278c), 0x0000bf00 },
	657	+ { _MMIO(0x2790), 0x0007fc32 },
	658	+ { _MMIO(0x2794), 0x0000bf00 },
	659	+ { _MMIO(0x2798), 0x0007fc9a },
	660	+ { _MMIO(0x279c), 0x0000bf00 },
	661	+ { _MMIO(0x27a0), 0x0007fe6a },
	662	+ { _MMIO(0x27a4), 0x0000bf00 },
	663	+ { _MMIO(0x27a8), 0x0007fe7a },
	664	+ { _MMIO(0x27ac), 0x0000bf00 },
	665	+};
	666	+
	667	+static const struct i915_oa_reg flex_eu_config_compute_extended[] = {
	668	+ { _MMIO(0xe458), 0x00005004 },
	669	+ { _MMIO(0xe558), 0x00000003 },
	670	+ { _MMIO(0xe658), 0x00002001 },
	671	+ { _MMIO(0xe758), 0x00778008 },
	672	+ { _MMIO(0xe45c), 0x00088078 },
	673	+ { _MMIO(0xe55c), 0x00808708 },
	674	+ { _MMIO(0xe65c), 0x00a08908 },
	675	+};
	676	+
	677	+static const struct i915_oa_reg mux_config_compute_extended[] = {
	678	+ { _MMIO(0x9888), 0x104f00e0 },
	679	+ { _MMIO(0x9888), 0x141c0160 },
	680	+ { _MMIO(0x9888), 0x161c0015 },
	681	+ { _MMIO(0x9888), 0x181c0120 },
	682	+ { _MMIO(0x9888), 0x002d5000 },
	683	+ { _MMIO(0x9888), 0x062d4000 },
	684	+ { _MMIO(0x9888), 0x082d5000 },
	685	+ { _MMIO(0x9888), 0x0a2d5000 },
	686	+ { _MMIO(0x9888), 0x0c2d5000 },
	687	+ { _MMIO(0x9888), 0x0e2d5000 },
	688	+ { _MMIO(0x9888), 0x022d5000 },
	689	+ { _MMIO(0x9888), 0x042d5000 },
	690	+ { _MMIO(0x9888), 0x0c2e5400 },
	691	+ { _MMIO(0x9888), 0x0e2e5515 },
	692	+ { _MMIO(0x9888), 0x102e0155 },
	693	+ { _MMIO(0x9888), 0x044cc000 },
	694	+ { _MMIO(0x9888), 0x0a4c8000 },
	695	+ { _MMIO(0x9888), 0x0c4cc000 },
	696	+ { _MMIO(0x9888), 0x0e4cc000 },
	697	+ { _MMIO(0x9888), 0x104c8000 },
	698	+ { _MMIO(0x9888), 0x124c8000 },
	699	+ { _MMIO(0x9888), 0x144c8000 },
	700	+ { _MMIO(0x9888), 0x164c2000 },
	701	+ { _MMIO(0x9888), 0x064cc000 },
	702	+ { _MMIO(0x9888), 0x084cc000 },
	703	+ { _MMIO(0x9888), 0x004ea000 },
	704	+ { _MMIO(0x9888), 0x064e8000 },
	705	+ { _MMIO(0x9888), 0x084ea000 },
	706	+ { _MMIO(0x9888), 0x0a4ea000 },
	707	+ { _MMIO(0x9888), 0x0c4ea000 },
	708	+ { _MMIO(0x9888), 0x0e4ea000 },
	709	+ { _MMIO(0x9888), 0x024ea000 },
	710	+ { _MMIO(0x9888), 0x044ea000 },
	711	+ { _MMIO(0x9888), 0x0e4f4b41 },
	712	+ { _MMIO(0x9888), 0x004f4200 },
	713	+ { _MMIO(0x9888), 0x024f404c },
	714	+ { _MMIO(0x9888), 0x1c4f0000 },
	715	+ { _MMIO(0x9888), 0x1a4f0000 },
	716	+ { _MMIO(0x9888), 0x001b4000 },
	717	+ { _MMIO(0x9888), 0x061b8000 },
	718	+ { _MMIO(0x9888), 0x081bc000 },
	719	+ { _MMIO(0x9888), 0x0a1bc000 },
	720	+ { _MMIO(0x9888), 0x0c1bc000 },
	721	+ { _MMIO(0x9888), 0x041bc000 },
	722	+ { _MMIO(0x9888), 0x001c0031 },
	723	+ { _MMIO(0x9888), 0x061c1900 },
	724	+ { _MMIO(0x9888), 0x081c1a33 },
	725	+ { _MMIO(0x9888), 0x0a1c1b35 },
	726	+ { _MMIO(0x9888), 0x0c1c3337 },
	727	+ { _MMIO(0x9888), 0x041c31c7 },
	728	+ { _MMIO(0x9888), 0x180f5000 },
	729	+ { _MMIO(0x9888), 0x1a0fa8aa },
	730	+ { _MMIO(0x9888), 0x1c0f0aaa },
	731	+ { _MMIO(0x9888), 0x182c8000 },
	732	+ { _MMIO(0x9888), 0x1c2c6aaa },
	733	+ { _MMIO(0x9888), 0x1e2c0001 },
	734	+ { _MMIO(0x9888), 0x1a2c2950 },
	735	+ { _MMIO(0x9888), 0x01938000 },
	736	+ { _MMIO(0x9888), 0x0f938000 },
	737	+ { _MMIO(0x9888), 0x1993aaaa },
	738	+ { _MMIO(0x9888), 0x03938000 },
	739	+ { _MMIO(0x9888), 0x05938000 },
	740	+ { _MMIO(0x9888), 0x07938000 },
	741	+ { _MMIO(0x9888), 0x09938000 },
	742	+ { _MMIO(0x9888), 0x0b938000 },
	743	+ { _MMIO(0x9888), 0x13904000 },
	744	+ { _MMIO(0x9888), 0x21904000 },
	745	+ { _MMIO(0x9888), 0x23904000 },
	746	+ { _MMIO(0x9888), 0x25904000 },
	747	+ { _MMIO(0x9888), 0x27904000 },
	748	+ { _MMIO(0x9888), 0x29904000 },
	749	+ { _MMIO(0x9888), 0x2b904000 },
	750	+ { _MMIO(0x9888), 0x2d904000 },
	751	+ { _MMIO(0x9888), 0x2f904000 },
	752	+ { _MMIO(0x9888), 0x31904000 },
	753	+ { _MMIO(0x9888), 0x15904000 },
	754	+ { _MMIO(0x9888), 0x17904000 },
	755	+ { _MMIO(0x9888), 0x19904000 },
	756	+ { _MMIO(0x9888), 0x1b904000 },
	757	+ { _MMIO(0x9888), 0x1d904000 },
	758	+ { _MMIO(0x9888), 0x53900000 },
	759	+ { _MMIO(0x9888), 0x43900420 },
	760	+ { _MMIO(0x9888), 0x55900000 },
	761	+ { _MMIO(0x9888), 0x47900000 },
	762	+ { _MMIO(0x9888), 0x57900000 },
	763	+ { _MMIO(0x9888), 0x49900000 },
	764	+ { _MMIO(0x9888), 0x59900000 },
	765	+ { _MMIO(0x9888), 0x4b900400 },
	766	+ { _MMIO(0x9888), 0x37900000 },
	767	+ { _MMIO(0x9888), 0x33900000 },
	768	+ { _MMIO(0x9888), 0x4d900001 },
	769	+ { _MMIO(0x9888), 0x45900001 },
	770	+};
	771	+
	772	+static int
	773	+get_compute_extended_mux_config(struct drm_i915_private *dev_priv,
	774	+ const struct i915_oa_reg **regs,
	775	+ int *lens)
	776	+{
	777	+ int n = 0;
	778	+
	779	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	780	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	781	+
	782	+ regs[n] = mux_config_compute_extended;
	783	+ lens[n] = ARRAY_SIZE(mux_config_compute_extended);
	784	+ n++;
	785	+
	786	+ return n;
	787	+}
	788	+
	789	+static const struct i915_oa_reg b_counter_config_compute_l3_cache[] = {
	790	+ { _MMIO(0x2710), 0x00000000 },
	791	+ { _MMIO(0x2714), 0x30800000 },
	792	+ { _MMIO(0x2720), 0x00000000 },
	793	+ { _MMIO(0x2724), 0x30800000 },
	794	+ { _MMIO(0x2740), 0x00000000 },
	795	+ { _MMIO(0x2770), 0x0007fffa },
	796	+ { _MMIO(0x2774), 0x0000fefe },
	797	+ { _MMIO(0x2778), 0x0007fffa },
	798	+ { _MMIO(0x277c), 0x0000fefd },
	799	+ { _MMIO(0x2790), 0x0007fffa },
	800	+ { _MMIO(0x2794), 0x0000fbef },
	801	+ { _MMIO(0x2798), 0x0007fffa },
	802	+ { _MMIO(0x279c), 0x0000fbdf },
	803	+};
	804	+
	805	+static const struct i915_oa_reg flex_eu_config_compute_l3_cache[] = {
	806	+ { _MMIO(0xe458), 0x00005004 },
	807	+ { _MMIO(0xe558), 0x00000003 },
	808	+ { _MMIO(0xe658), 0x00002001 },
	809	+ { _MMIO(0xe758), 0x00101100 },
	810	+ { _MMIO(0xe45c), 0x00201200 },
	811	+ { _MMIO(0xe55c), 0x00301300 },
	812	+ { _MMIO(0xe65c), 0x00401400 },
	813	+};
	814	+
	815	+static const struct i915_oa_reg mux_config_compute_l3_cache[] = {
	816	+ { _MMIO(0x9888), 0x166c03b0 },
	817	+ { _MMIO(0x9888), 0x1593001e },
	818	+ { _MMIO(0x9888), 0x3f900c00 },
	819	+ { _MMIO(0x9888), 0x41900000 },
	820	+ { _MMIO(0x9888), 0x002d1000 },
	821	+ { _MMIO(0x9888), 0x062d4000 },
	822	+ { _MMIO(0x9888), 0x082d5000 },
	823	+ { _MMIO(0x9888), 0x0e2d5000 },
	824	+ { _MMIO(0x9888), 0x0c2e0400 },
	825	+ { _MMIO(0x9888), 0x0e2e1500 },
	826	+ { _MMIO(0x9888), 0x102e0140 },
	827	+ { _MMIO(0x9888), 0x044c4000 },
	828	+ { _MMIO(0x9888), 0x0a4c8000 },
	829	+ { _MMIO(0x9888), 0x0c4cc000 },
	830	+ { _MMIO(0x9888), 0x144c8000 },
	831	+ { _MMIO(0x9888), 0x164c2000 },
	832	+ { _MMIO(0x9888), 0x004e2000 },
	833	+ { _MMIO(0x9888), 0x064e8000 },
	834	+ { _MMIO(0x9888), 0x084ea000 },
	835	+ { _MMIO(0x9888), 0x0e4ea000 },
	836	+ { _MMIO(0x9888), 0x1a4f4001 },
	837	+ { _MMIO(0x9888), 0x1c4f5005 },
	838	+ { _MMIO(0x9888), 0x006c0051 },
	839	+ { _MMIO(0x9888), 0x066c5000 },
	840	+ { _MMIO(0x9888), 0x086c5c5d },
	841	+ { _MMIO(0x9888), 0x0e6c5e5f },
	842	+ { _MMIO(0x9888), 0x106c0000 },
	843	+ { _MMIO(0x9888), 0x146c0000 },
	844	+ { _MMIO(0x9888), 0x1a6c0000 },
	845	+ { _MMIO(0x9888), 0x1c6c0000 },
	846	+ { _MMIO(0x9888), 0x180f1000 },
	847	+ { _MMIO(0x9888), 0x1a0fa800 },
	848	+ { _MMIO(0x9888), 0x1c0f0a00 },
	849	+ { _MMIO(0x9888), 0x182c4000 },
	850	+ { _MMIO(0x9888), 0x1c2c4015 },
	851	+ { _MMIO(0x9888), 0x1e2c0001 },
	852	+ { _MMIO(0x9888), 0x03931980 },
	853	+ { _MMIO(0x9888), 0x05930032 },
	854	+ { _MMIO(0x9888), 0x11930000 },
	855	+ { _MMIO(0x9888), 0x01938000 },
	856	+ { _MMIO(0x9888), 0x0f938000 },
	857	+ { _MMIO(0x9888), 0x1993a00a },
	858	+ { _MMIO(0x9888), 0x07930000 },
	859	+ { _MMIO(0x9888), 0x09930000 },
	860	+ { _MMIO(0x9888), 0x1d900177 },
	861	+ { _MMIO(0x9888), 0x1f900178 },
	862	+ { _MMIO(0x9888), 0x35900000 },
	863	+ { _MMIO(0x9888), 0x13904000 },
	864	+ { _MMIO(0x9888), 0x21904000 },
	865	+ { _MMIO(0x9888), 0x23904000 },
	866	+ { _MMIO(0x9888), 0x25904000 },
	867	+ { _MMIO(0x9888), 0x2f904000 },
	868	+ { _MMIO(0x9888), 0x31904000 },
	869	+ { _MMIO(0x9888), 0x19904000 },
	870	+ { _MMIO(0x9888), 0x1b904000 },
	871	+ { _MMIO(0x9888), 0x53901000 },
	872	+ { _MMIO(0x9888), 0x43900000 },
	873	+ { _MMIO(0x9888), 0x55900111 },
	874	+ { _MMIO(0x9888), 0x47900001 },
	875	+ { _MMIO(0x9888), 0x57900000 },
	876	+ { _MMIO(0x9888), 0x49900000 },
	877	+ { _MMIO(0x9888), 0x37900000 },
	878	+ { _MMIO(0x9888), 0x33900000 },
	879	+ { _MMIO(0x9888), 0x59900000 },
	880	+ { _MMIO(0x9888), 0x4b900000 },
	881	+ { _MMIO(0x9888), 0x4d900000 },
	882	+ { _MMIO(0x9888), 0x45900400 },
	883	+};
	884	+
	885	+static int
	886	+get_compute_l3_cache_mux_config(struct drm_i915_private *dev_priv,
	887	+ const struct i915_oa_reg **regs,
	888	+ int *lens)
	889	+{
	890	+ int n = 0;
	891	+
	892	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	893	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	894	+
	895	+ regs[n] = mux_config_compute_l3_cache;
	896	+ lens[n] = ARRAY_SIZE(mux_config_compute_l3_cache);
	897	+ n++;
	898	+
	899	+ return n;
	900	+}
	901	+
	902	+static const struct i915_oa_reg b_counter_config_hdc_and_sf[] = {
	903	+ { _MMIO(0x2740), 0x00000000 },
	904	+ { _MMIO(0x2744), 0x00800000 },
	905	+ { _MMIO(0x2710), 0x00000000 },
	906	+ { _MMIO(0x2714), 0x10800000 },
	907	+ { _MMIO(0x2720), 0x00000000 },
	908	+ { _MMIO(0x2724), 0x00800000 },
	909	+ { _MMIO(0x2770), 0x00000002 },
	910	+ { _MMIO(0x2774), 0x0000fdff },
	911	+};
	912	+
	913	+static const struct i915_oa_reg flex_eu_config_hdc_and_sf[] = {
	914	+ { _MMIO(0xe458), 0x00005004 },
	915	+ { _MMIO(0xe558), 0x00010003 },
	916	+ { _MMIO(0xe658), 0x00012011 },
	917	+ { _MMIO(0xe758), 0x00015014 },
	918	+ { _MMIO(0xe45c), 0x00051050 },
	919	+ { _MMIO(0xe55c), 0x00053052 },
	920	+ { _MMIO(0xe65c), 0x00055054 },
	921	+};
	922	+
	923	+static const struct i915_oa_reg mux_config_hdc_and_sf[] = {
	924	+ { _MMIO(0x9888), 0x104f0232 },
	925	+ { _MMIO(0x9888), 0x124f4640 },
	926	+ { _MMIO(0x9888), 0x11834400 },
	927	+ { _MMIO(0x9888), 0x022d4000 },
	928	+ { _MMIO(0x9888), 0x042d5000 },
	929	+ { _MMIO(0x9888), 0x062d1000 },
	930	+ { _MMIO(0x9888), 0x0e2e0055 },
	931	+ { _MMIO(0x9888), 0x064c8000 },
	932	+ { _MMIO(0x9888), 0x084cc000 },
	933	+ { _MMIO(0x9888), 0x0a4c4000 },
	934	+ { _MMIO(0x9888), 0x024e8000 },
	935	+ { _MMIO(0x9888), 0x044ea000 },
	936	+ { _MMIO(0x9888), 0x064e2000 },
	937	+ { _MMIO(0x9888), 0x024f6100 },
	938	+ { _MMIO(0x9888), 0x044f416b },
	939	+ { _MMIO(0x9888), 0x064f004b },
	940	+ { _MMIO(0x9888), 0x1a4f0000 },
	941	+ { _MMIO(0x9888), 0x1a0f02a8 },
	942	+ { _MMIO(0x9888), 0x1a2c5500 },
	943	+ { _MMIO(0x9888), 0x0f808000 },
	944	+ { _MMIO(0x9888), 0x25810020 },
	945	+ { _MMIO(0x9888), 0x0f8305c0 },
	946	+ { _MMIO(0x9888), 0x07938000 },
	947	+ { _MMIO(0x9888), 0x09938000 },
	948	+ { _MMIO(0x9888), 0x0b938000 },
	949	+ { _MMIO(0x9888), 0x0d938000 },
	950	+ { _MMIO(0x9888), 0x1f951000 },
	951	+ { _MMIO(0x9888), 0x13920200 },
	952	+ { _MMIO(0x9888), 0x31908000 },
	953	+ { _MMIO(0x9888), 0x19904000 },
	954	+ { _MMIO(0x9888), 0x1b904000 },
	955	+ { _MMIO(0x9888), 0x1d904000 },
	956	+ { _MMIO(0x9888), 0x1f904000 },
	957	+ { _MMIO(0x9888), 0x37900000 },
	958	+ { _MMIO(0x9888), 0x59900000 },
	959	+ { _MMIO(0x9888), 0x4d900003 },
	960	+ { _MMIO(0x9888), 0x53900000 },
	961	+ { _MMIO(0x9888), 0x45900000 },
	962	+ { _MMIO(0x9888), 0x55900000 },
	963	+ { _MMIO(0x9888), 0x47900000 },
	964	+ { _MMIO(0x9888), 0x33900000 },
	965	+};
	966	+
	967	+static int
	968	+get_hdc_and_sf_mux_config(struct drm_i915_private *dev_priv,
	969	+ const struct i915_oa_reg **regs,
	970	+ int *lens)
	971	+{
	972	+ int n = 0;
	973	+
	974	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	975	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	976	+
	977	+ regs[n] = mux_config_hdc_and_sf;
	978	+ lens[n] = ARRAY_SIZE(mux_config_hdc_and_sf);
	979	+ n++;
	980	+
	981	+ return n;
	982	+}
	983	+
	984	+static const struct i915_oa_reg b_counter_config_l3_1[] = {
	985	+ { _MMIO(0x2740), 0x00000000 },
	986	+ { _MMIO(0x2744), 0x00800000 },
	987	+ { _MMIO(0x2710), 0x00000000 },
	988	+ { _MMIO(0x2714), 0xf0800000 },
	989	+ { _MMIO(0x2720), 0x00000000 },
	990	+ { _MMIO(0x2724), 0xf0800000 },
	991	+ { _MMIO(0x2770), 0x00100070 },
	992	+ { _MMIO(0x2774), 0x0000fff1 },
	993	+ { _MMIO(0x2778), 0x00014002 },
	994	+ { _MMIO(0x277c), 0x0000c3ff },
	995	+ { _MMIO(0x2780), 0x00010002 },
	996	+ { _MMIO(0x2784), 0x0000c7ff },
	997	+ { _MMIO(0x2788), 0x00004002 },
	998	+ { _MMIO(0x278c), 0x0000d3ff },
	999	+ { _MMIO(0x2790), 0x00100700 },
	1000	+ { _MMIO(0x2794), 0x0000ff1f },
	1001	+ { _MMIO(0x2798), 0x00001402 },
	1002	+ { _MMIO(0x279c), 0x0000fc3f },
	1003	+ { _MMIO(0x27a0), 0x00001002 },
	1004	+ { _MMIO(0x27a4), 0x0000fc7f },
	1005	+ { _MMIO(0x27a8), 0x00000402 },
	1006	+ { _MMIO(0x27ac), 0x0000fd3f },
	1007	+};
	1008	+
	1009	+static const struct i915_oa_reg flex_eu_config_l3_1[] = {
	1010	+ { _MMIO(0xe458), 0x00005004 },
	1011	+ { _MMIO(0xe558), 0x00010003 },
	1012	+ { _MMIO(0xe658), 0x00012011 },
	1013	+ { _MMIO(0xe758), 0x00015014 },
	1014	+ { _MMIO(0xe45c), 0x00051050 },
	1015	+ { _MMIO(0xe55c), 0x00053052 },
	1016	+ { _MMIO(0xe65c), 0x00055054 },
	1017	+};
	1018	+
	1019	+static const struct i915_oa_reg mux_config_l3_1[] = {
	1020	+ { _MMIO(0x9888), 0x12643400 },
	1021	+ { _MMIO(0x9888), 0x12653400 },
	1022	+ { _MMIO(0x9888), 0x106c6800 },
	1023	+ { _MMIO(0x9888), 0x126c001e },
	1024	+ { _MMIO(0x9888), 0x166c0010 },
	1025	+ { _MMIO(0x9888), 0x0c2d5000 },
	1026	+ { _MMIO(0x9888), 0x0e2d5000 },
	1027	+ { _MMIO(0x9888), 0x002d4000 },
	1028	+ { _MMIO(0x9888), 0x022d5000 },
	1029	+ { _MMIO(0x9888), 0x042d5000 },
	1030	+ { _MMIO(0x9888), 0x062d1000 },
	1031	+ { _MMIO(0x9888), 0x102e0154 },
	1032	+ { _MMIO(0x9888), 0x0c2e5000 },
	1033	+ { _MMIO(0x9888), 0x0e2e0055 },
	1034	+ { _MMIO(0x9888), 0x104c8000 },
	1035	+ { _MMIO(0x9888), 0x124c8000 },
	1036	+ { _MMIO(0x9888), 0x144c8000 },
	1037	+ { _MMIO(0x9888), 0x164c2000 },
	1038	+ { _MMIO(0x9888), 0x044c8000 },
	1039	+ { _MMIO(0x9888), 0x064cc000 },
	1040	+ { _MMIO(0x9888), 0x084cc000 },
	1041	+ { _MMIO(0x9888), 0x0a4c4000 },
	1042	+ { _MMIO(0x9888), 0x0c4ea000 },
	1043	+ { _MMIO(0x9888), 0x0e4ea000 },
	1044	+ { _MMIO(0x9888), 0x004e8000 },
	1045	+ { _MMIO(0x9888), 0x024ea000 },
	1046	+ { _MMIO(0x9888), 0x044ea000 },
	1047	+ { _MMIO(0x9888), 0x064e2000 },
	1048	+ { _MMIO(0x9888), 0x1c4f5500 },
	1049	+ { _MMIO(0x9888), 0x1a4f1554 },
	1050	+ { _MMIO(0x9888), 0x0a640024 },
	1051	+ { _MMIO(0x9888), 0x10640000 },
	1052	+ { _MMIO(0x9888), 0x04640000 },
	1053	+ { _MMIO(0x9888), 0x0c650024 },
	1054	+ { _MMIO(0x9888), 0x10650000 },
	1055	+ { _MMIO(0x9888), 0x06650000 },
	1056	+ { _MMIO(0x9888), 0x0c6c5327 },
	1057	+ { _MMIO(0x9888), 0x0e6c5425 },
	1058	+ { _MMIO(0x9888), 0x006c2a00 },
	1059	+ { _MMIO(0x9888), 0x026c285b },
	1060	+ { _MMIO(0x9888), 0x046c005c },
	1061	+ { _MMIO(0x9888), 0x1c6c0000 },
	1062	+ { _MMIO(0x9888), 0x1a6c0900 },
	1063	+ { _MMIO(0x9888), 0x1c0f0aa0 },
	1064	+ { _MMIO(0x9888), 0x180f4000 },
	1065	+ { _MMIO(0x9888), 0x1a0f02aa },
	1066	+ { _MMIO(0x9888), 0x1c2c5400 },
	1067	+ { _MMIO(0x9888), 0x1e2c0001 },
	1068	+ { _MMIO(0x9888), 0x1a2c5550 },
	1069	+ { _MMIO(0x9888), 0x1993aa00 },
	1070	+ { _MMIO(0x9888), 0x03938000 },
	1071	+ { _MMIO(0x9888), 0x05938000 },
	1072	+ { _MMIO(0x9888), 0x07938000 },
	1073	+ { _MMIO(0x9888), 0x09938000 },
	1074	+ { _MMIO(0x9888), 0x0b938000 },
	1075	+ { _MMIO(0x9888), 0x0d938000 },
	1076	+ { _MMIO(0x9888), 0x2b904000 },
	1077	+ { _MMIO(0x9888), 0x2d904000 },
	1078	+ { _MMIO(0x9888), 0x2f904000 },
	1079	+ { _MMIO(0x9888), 0x31904000 },
	1080	+ { _MMIO(0x9888), 0x15904000 },
	1081	+ { _MMIO(0x9888), 0x17904000 },
	1082	+ { _MMIO(0x9888), 0x19904000 },
	1083	+ { _MMIO(0x9888), 0x1b904000 },
	1084	+ { _MMIO(0x9888), 0x1d904000 },
	1085	+ { _MMIO(0x9888), 0x1f904000 },
	1086	+ { _MMIO(0x9888), 0x59900000 },
	1087	+ { _MMIO(0x9888), 0x4b900421 },
	1088	+ { _MMIO(0x9888), 0x37900000 },
	1089	+ { _MMIO(0x9888), 0x33900000 },
	1090	+ { _MMIO(0x9888), 0x4d900001 },
	1091	+ { _MMIO(0x9888), 0x53900000 },
	1092	+ { _MMIO(0x9888), 0x43900420 },
	1093	+ { _MMIO(0x9888), 0x45900021 },
	1094	+ { _MMIO(0x9888), 0x55900000 },
	1095	+ { _MMIO(0x9888), 0x47900000 },
	1096	+};
	1097	+
	1098	+static int
	1099	+get_l3_1_mux_config(struct drm_i915_private *dev_priv,
	1100	+ const struct i915_oa_reg **regs,
	1101	+ int *lens)
	1102	+{
	1103	+ int n = 0;
	1104	+
	1105	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1106	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1107	+
	1108	+ regs[n] = mux_config_l3_1;
	1109	+ lens[n] = ARRAY_SIZE(mux_config_l3_1);
	1110	+ n++;
	1111	+
	1112	+ return n;
	1113	+}
	1114	+
	1115	+static const struct i915_oa_reg b_counter_config_rasterizer_and_pixel_backend[] = {
	1116	+ { _MMIO(0x2740), 0x00000000 },
	1117	+ { _MMIO(0x2744), 0x00800000 },
	1118	+ { _MMIO(0x2710), 0x00000000 },
	1119	+ { _MMIO(0x2714), 0x30800000 },
	1120	+ { _MMIO(0x2720), 0x00000000 },
	1121	+ { _MMIO(0x2724), 0x00800000 },
	1122	+ { _MMIO(0x2770), 0x00000002 },
	1123	+ { _MMIO(0x2774), 0x0000efff },
	1124	+ { _MMIO(0x2778), 0x00006000 },
	1125	+ { _MMIO(0x277c), 0x0000f3ff },
	1126	+};
	1127	+
	1128	+static const struct i915_oa_reg flex_eu_config_rasterizer_and_pixel_backend[] = {
	1129	+ { _MMIO(0xe458), 0x00005004 },
	1130	+ { _MMIO(0xe558), 0x00010003 },
	1131	+ { _MMIO(0xe658), 0x00012011 },
	1132	+ { _MMIO(0xe758), 0x00015014 },
	1133	+ { _MMIO(0xe45c), 0x00051050 },
	1134	+ { _MMIO(0xe55c), 0x00053052 },
	1135	+ { _MMIO(0xe65c), 0x00055054 },
	1136	+};
	1137	+
	1138	+static const struct i915_oa_reg mux_config_rasterizer_and_pixel_backend[] = {
	1139	+ { _MMIO(0x9888), 0x102d7800 },
	1140	+ { _MMIO(0x9888), 0x122d79e0 },
	1141	+ { _MMIO(0x9888), 0x0c2f0004 },
	1142	+ { _MMIO(0x9888), 0x100e3800 },
	1143	+ { _MMIO(0x9888), 0x180f0005 },
	1144	+ { _MMIO(0x9888), 0x002d0940 },
	1145	+ { _MMIO(0x9888), 0x022d802f },
	1146	+ { _MMIO(0x9888), 0x042d4013 },
	1147	+ { _MMIO(0x9888), 0x062d1000 },
	1148	+ { _MMIO(0x9888), 0x0e2e0050 },
	1149	+ { _MMIO(0x9888), 0x022f0010 },
	1150	+ { _MMIO(0x9888), 0x002f0000 },
	1151	+ { _MMIO(0x9888), 0x084c8000 },
	1152	+ { _MMIO(0x9888), 0x0a4c4000 },
	1153	+ { _MMIO(0x9888), 0x044e8000 },
	1154	+ { _MMIO(0x9888), 0x064e2000 },
	1155	+ { _MMIO(0x9888), 0x040e0480 },
	1156	+ { _MMIO(0x9888), 0x000e0000 },
	1157	+ { _MMIO(0x9888), 0x060f0027 },
	1158	+ { _MMIO(0x9888), 0x100f0000 },
	1159	+ { _MMIO(0x9888), 0x1a0f0040 },
	1160	+ { _MMIO(0x9888), 0x03938000 },
	1161	+ { _MMIO(0x9888), 0x05938000 },
	1162	+ { _MMIO(0x9888), 0x07938000 },
	1163	+ { _MMIO(0x9888), 0x09938000 },
	1164	+ { _MMIO(0x9888), 0x0b938000 },
	1165	+ { _MMIO(0x9888), 0x0d938000 },
	1166	+ { _MMIO(0x9888), 0x15904000 },
	1167	+ { _MMIO(0x9888), 0x17904000 },
	1168	+ { _MMIO(0x9888), 0x19904000 },
	1169	+ { _MMIO(0x9888), 0x1b904000 },
	1170	+ { _MMIO(0x9888), 0x1d904000 },
	1171	+ { _MMIO(0x9888), 0x1f904000 },
	1172	+ { _MMIO(0x9888), 0x37900000 },
	1173	+ { _MMIO(0x9888), 0x53900000 },
	1174	+ { _MMIO(0x9888), 0x439014a0 },
	1175	+ { _MMIO(0x9888), 0x459000a4 },
	1176	+ { _MMIO(0x9888), 0x55900000 },
	1177	+ { _MMIO(0x9888), 0x47900001 },
	1178	+ { _MMIO(0x9888), 0x33900000 },
	1179	+};
	1180	+
	1181	+static int
	1182	+get_rasterizer_and_pixel_backend_mux_config(struct drm_i915_private *dev_priv,
	1183	+ const struct i915_oa_reg **regs,
	1184	+ int *lens)
	1185	+{
	1186	+ int n = 0;
	1187	+
	1188	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1189	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1190	+
	1191	+ regs[n] = mux_config_rasterizer_and_pixel_backend;
	1192	+ lens[n] = ARRAY_SIZE(mux_config_rasterizer_and_pixel_backend);
	1193	+ n++;
	1194	+
	1195	+ return n;
	1196	+}
	1197	+
	1198	+static const struct i915_oa_reg b_counter_config_sampler[] = {
	1199	+ { _MMIO(0x2740), 0x00000000 },
	1200	+ { _MMIO(0x2744), 0x00800000 },
	1201	+ { _MMIO(0x2710), 0x00000000 },
	1202	+ { _MMIO(0x2714), 0x70800000 },
	1203	+ { _MMIO(0x2720), 0x00000000 },
	1204	+ { _MMIO(0x2724), 0x00800000 },
	1205	+ { _MMIO(0x2770), 0x0000c000 },
	1206	+ { _MMIO(0x2774), 0x0000e7ff },
	1207	+ { _MMIO(0x2778), 0x00003000 },
	1208	+ { _MMIO(0x277c), 0x0000f9ff },
	1209	+ { _MMIO(0x2780), 0x00000c00 },
	1210	+ { _MMIO(0x2784), 0x0000fe7f },
	1211	+};
	1212	+
	1213	+static const struct i915_oa_reg flex_eu_config_sampler[] = {
	1214	+ { _MMIO(0xe458), 0x00005004 },
	1215	+ { _MMIO(0xe558), 0x00010003 },
	1216	+ { _MMIO(0xe658), 0x00012011 },
	1217	+ { _MMIO(0xe758), 0x00015014 },
	1218	+ { _MMIO(0xe45c), 0x00051050 },
	1219	+ { _MMIO(0xe55c), 0x00053052 },
	1220	+ { _MMIO(0xe65c), 0x00055054 },
	1221	+};
	1222	+
	1223	+static const struct i915_oa_reg mux_config_sampler[] = {
	1224	+ { _MMIO(0x9888), 0x121300a0 },
	1225	+ { _MMIO(0x9888), 0x141600ab },
	1226	+ { _MMIO(0x9888), 0x123300a0 },
	1227	+ { _MMIO(0x9888), 0x143600ab },
	1228	+ { _MMIO(0x9888), 0x125300a0 },
	1229	+ { _MMIO(0x9888), 0x145600ab },
	1230	+ { _MMIO(0x9888), 0x0c2d4000 },
	1231	+ { _MMIO(0x9888), 0x0e2d5000 },
	1232	+ { _MMIO(0x9888), 0x002d4000 },
	1233	+ { _MMIO(0x9888), 0x022d5000 },
	1234	+ { _MMIO(0x9888), 0x042d5000 },
	1235	+ { _MMIO(0x9888), 0x062d1000 },
	1236	+ { _MMIO(0x9888), 0x102e01a0 },
	1237	+ { _MMIO(0x9888), 0x0c2e5000 },
	1238	+ { _MMIO(0x9888), 0x0e2e0065 },
	1239	+ { _MMIO(0x9888), 0x164c2000 },
	1240	+ { _MMIO(0x9888), 0x044c8000 },
	1241	+ { _MMIO(0x9888), 0x064cc000 },
	1242	+ { _MMIO(0x9888), 0x084c4000 },
	1243	+ { _MMIO(0x9888), 0x0a4c4000 },
	1244	+ { _MMIO(0x9888), 0x0e4e8000 },
	1245	+ { _MMIO(0x9888), 0x004e8000 },
	1246	+ { _MMIO(0x9888), 0x024ea000 },
	1247	+ { _MMIO(0x9888), 0x044e2000 },
	1248	+ { _MMIO(0x9888), 0x064e2000 },
	1249	+ { _MMIO(0x9888), 0x1c0f0800 },
	1250	+ { _MMIO(0x9888), 0x180f4000 },
	1251	+ { _MMIO(0x9888), 0x1a0f023f },
	1252	+ { _MMIO(0x9888), 0x1e2c0003 },
	1253	+ { _MMIO(0x9888), 0x1a2cc030 },
	1254	+ { _MMIO(0x9888), 0x04132180 },
	1255	+ { _MMIO(0x9888), 0x02130000 },
	1256	+ { _MMIO(0x9888), 0x0c148000 },
	1257	+ { _MMIO(0x9888), 0x0e142000 },
	1258	+ { _MMIO(0x9888), 0x04148000 },
	1259	+ { _MMIO(0x9888), 0x1e150140 },
	1260	+ { _MMIO(0x9888), 0x1c150040 },
	1261	+ { _MMIO(0x9888), 0x0c163000 },
	1262	+ { _MMIO(0x9888), 0x0e160068 },
	1263	+ { _MMIO(0x9888), 0x10160000 },
	1264	+ { _MMIO(0x9888), 0x18160000 },
	1265	+ { _MMIO(0x9888), 0x0a164000 },
	1266	+ { _MMIO(0x9888), 0x04330043 },
	1267	+ { _MMIO(0x9888), 0x02330000 },
	1268	+ { _MMIO(0x9888), 0x0234a000 },
	1269	+ { _MMIO(0x9888), 0x04342000 },
	1270	+ { _MMIO(0x9888), 0x1c350015 },
	1271	+ { _MMIO(0x9888), 0x02363460 },
	1272	+ { _MMIO(0x9888), 0x10360000 },
	1273	+ { _MMIO(0x9888), 0x04360000 },
	1274	+ { _MMIO(0x9888), 0x06360000 },
	1275	+ { _MMIO(0x9888), 0x08364000 },
	1276	+ { _MMIO(0x9888), 0x06530043 },
	1277	+ { _MMIO(0x9888), 0x02530000 },
	1278	+ { _MMIO(0x9888), 0x0e548000 },
	1279	+ { _MMIO(0x9888), 0x00548000 },
	1280	+ { _MMIO(0x9888), 0x06542000 },
	1281	+ { _MMIO(0x9888), 0x1e550400 },
	1282	+ { _MMIO(0x9888), 0x1a552000 },
	1283	+ { _MMIO(0x9888), 0x1c550100 },
	1284	+ { _MMIO(0x9888), 0x0e563000 },
	1285	+ { _MMIO(0x9888), 0x00563400 },
	1286	+ { _MMIO(0x9888), 0x10560000 },
	1287	+ { _MMIO(0x9888), 0x18560000 },
	1288	+ { _MMIO(0x9888), 0x02560000 },
	1289	+ { _MMIO(0x9888), 0x0c564000 },
	1290	+ { _MMIO(0x9888), 0x1993a800 },
	1291	+ { _MMIO(0x9888), 0x03938000 },
	1292	+ { _MMIO(0x9888), 0x05938000 },
	1293	+ { _MMIO(0x9888), 0x07938000 },
	1294	+ { _MMIO(0x9888), 0x09938000 },
	1295	+ { _MMIO(0x9888), 0x0b938000 },
	1296	+ { _MMIO(0x9888), 0x0d938000 },
	1297	+ { _MMIO(0x9888), 0x2d904000 },
	1298	+ { _MMIO(0x9888), 0x2f904000 },
	1299	+ { _MMIO(0x9888), 0x31904000 },
	1300	+ { _MMIO(0x9888), 0x15904000 },
	1301	+ { _MMIO(0x9888), 0x17904000 },
	1302	+ { _MMIO(0x9888), 0x19904000 },
	1303	+ { _MMIO(0x9888), 0x1b904000 },
	1304	+ { _MMIO(0x9888), 0x1d904000 },
	1305	+ { _MMIO(0x9888), 0x1f904000 },
	1306	+ { _MMIO(0x9888), 0x59900000 },
	1307	+ { _MMIO(0x9888), 0x4b9014a0 },
	1308	+ { _MMIO(0x9888), 0x37900000 },
	1309	+ { _MMIO(0x9888), 0x33900000 },
	1310	+ { _MMIO(0x9888), 0x4d900001 },
	1311	+ { _MMIO(0x9888), 0x53900000 },
	1312	+ { _MMIO(0x9888), 0x43900820 },
	1313	+ { _MMIO(0x9888), 0x45901022 },
	1314	+ { _MMIO(0x9888), 0x55900000 },
	1315	+ { _MMIO(0x9888), 0x47900000 },
	1316	+};
	1317	+
	1318	+static int
	1319	+get_sampler_mux_config(struct drm_i915_private *dev_priv,
	1320	+ const struct i915_oa_reg **regs,
	1321	+ int *lens)
	1322	+{
	1323	+ int n = 0;
	1324	+
	1325	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1326	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1327	+
	1328	+ regs[n] = mux_config_sampler;
	1329	+ lens[n] = ARRAY_SIZE(mux_config_sampler);
	1330	+ n++;
	1331	+
	1332	+ return n;
	1333	+}
	1334	+
	1335	+static const struct i915_oa_reg b_counter_config_tdl_1[] = {
	1336	+ { _MMIO(0x2740), 0x00000000 },
	1337	+ { _MMIO(0x2744), 0x00800000 },
	1338	+ { _MMIO(0x2710), 0x00000000 },
	1339	+ { _MMIO(0x2714), 0xf0800000 },
	1340	+ { _MMIO(0x2720), 0x00000000 },
	1341	+ { _MMIO(0x2724), 0x30800000 },
	1342	+ { _MMIO(0x2770), 0x00000002 },
	1343	+ { _MMIO(0x2774), 0x00007fff },
	1344	+ { _MMIO(0x2778), 0x00000000 },
	1345	+ { _MMIO(0x277c), 0x00009fff },
	1346	+ { _MMIO(0x2780), 0x00000002 },
	1347	+ { _MMIO(0x2784), 0x0000efff },
	1348	+ { _MMIO(0x2788), 0x00000000 },
	1349	+ { _MMIO(0x278c), 0x0000f3ff },
	1350	+ { _MMIO(0x2790), 0x00000002 },
	1351	+ { _MMIO(0x2794), 0x0000fdff },
	1352	+ { _MMIO(0x2798), 0x00000000 },
	1353	+ { _MMIO(0x279c), 0x0000fe7f },
	1354	+};
	1355	+
	1356	+static const struct i915_oa_reg flex_eu_config_tdl_1[] = {
	1357	+ { _MMIO(0xe458), 0x00005004 },
	1358	+ { _MMIO(0xe558), 0x00010003 },
	1359	+ { _MMIO(0xe658), 0x00012011 },
	1360	+ { _MMIO(0xe758), 0x00015014 },
	1361	+ { _MMIO(0xe45c), 0x00051050 },
	1362	+ { _MMIO(0xe55c), 0x00053052 },
	1363	+ { _MMIO(0xe65c), 0x00055054 },
	1364	+};
	1365	+
	1366	+static const struct i915_oa_reg mux_config_tdl_1[] = {
	1367	+ { _MMIO(0x9888), 0x141a0000 },
	1368	+ { _MMIO(0x9888), 0x143a0000 },
	1369	+ { _MMIO(0x9888), 0x145a0000 },
	1370	+ { _MMIO(0x9888), 0x0c2d4000 },
	1371	+ { _MMIO(0x9888), 0x0e2d5000 },
	1372	+ { _MMIO(0x9888), 0x002d4000 },
	1373	+ { _MMIO(0x9888), 0x022d5000 },
	1374	+ { _MMIO(0x9888), 0x042d5000 },
	1375	+ { _MMIO(0x9888), 0x062d1000 },
	1376	+ { _MMIO(0x9888), 0x102e0150 },
	1377	+ { _MMIO(0x9888), 0x0c2e5000 },
	1378	+ { _MMIO(0x9888), 0x0e2e006a },
	1379	+ { _MMIO(0x9888), 0x124c8000 },
	1380	+ { _MMIO(0x9888), 0x144c8000 },
	1381	+ { _MMIO(0x9888), 0x164c2000 },
	1382	+ { _MMIO(0x9888), 0x044c8000 },
	1383	+ { _MMIO(0x9888), 0x064c4000 },
	1384	+ { _MMIO(0x9888), 0x0a4c4000 },
	1385	+ { _MMIO(0x9888), 0x0c4e8000 },
	1386	+ { _MMIO(0x9888), 0x0e4ea000 },
	1387	+ { _MMIO(0x9888), 0x004e8000 },
	1388	+ { _MMIO(0x9888), 0x024e2000 },
	1389	+ { _MMIO(0x9888), 0x064e2000 },
	1390	+ { _MMIO(0x9888), 0x1c0f0bc0 },
	1391	+ { _MMIO(0x9888), 0x180f4000 },
	1392	+ { _MMIO(0x9888), 0x1a0f0302 },
	1393	+ { _MMIO(0x9888), 0x1e2c0003 },
	1394	+ { _MMIO(0x9888), 0x1a2c00f0 },
	1395	+ { _MMIO(0x9888), 0x021a3080 },
	1396	+ { _MMIO(0x9888), 0x041a31e5 },
	1397	+ { _MMIO(0x9888), 0x02148000 },
	1398	+ { _MMIO(0x9888), 0x0414a000 },
	1399	+ { _MMIO(0x9888), 0x1c150054 },
	1400	+ { _MMIO(0x9888), 0x06168000 },
	1401	+ { _MMIO(0x9888), 0x08168000 },
	1402	+ { _MMIO(0x9888), 0x0a168000 },
	1403	+ { _MMIO(0x9888), 0x0c3a3280 },
	1404	+ { _MMIO(0x9888), 0x0e3a0063 },
	1405	+ { _MMIO(0x9888), 0x063a0061 },
	1406	+ { _MMIO(0x9888), 0x023a0000 },
	1407	+ { _MMIO(0x9888), 0x0c348000 },
	1408	+ { _MMIO(0x9888), 0x0e342000 },
	1409	+ { _MMIO(0x9888), 0x06342000 },
	1410	+ { _MMIO(0x9888), 0x1e350140 },
	1411	+ { _MMIO(0x9888), 0x1c350100 },
	1412	+ { _MMIO(0x9888), 0x18360028 },
	1413	+ { _MMIO(0x9888), 0x0c368000 },
	1414	+ { _MMIO(0x9888), 0x0e5a3080 },
	1415	+ { _MMIO(0x9888), 0x005a3280 },
	1416	+ { _MMIO(0x9888), 0x025a0063 },
	1417	+ { _MMIO(0x9888), 0x0e548000 },
	1418	+ { _MMIO(0x9888), 0x00548000 },
	1419	+ { _MMIO(0x9888), 0x02542000 },
	1420	+ { _MMIO(0x9888), 0x1e550400 },
	1421	+ { _MMIO(0x9888), 0x1a552000 },
	1422	+ { _MMIO(0x9888), 0x1c550001 },
	1423	+ { _MMIO(0x9888), 0x18560080 },
	1424	+ { _MMIO(0x9888), 0x02568000 },
	1425	+ { _MMIO(0x9888), 0x04568000 },
	1426	+ { _MMIO(0x9888), 0x1993a800 },
	1427	+ { _MMIO(0x9888), 0x03938000 },
	1428	+ { _MMIO(0x9888), 0x05938000 },
	1429	+ { _MMIO(0x9888), 0x07938000 },
	1430	+ { _MMIO(0x9888), 0x09938000 },
	1431	+ { _MMIO(0x9888), 0x0b938000 },
	1432	+ { _MMIO(0x9888), 0x0d938000 },
	1433	+ { _MMIO(0x9888), 0x2d904000 },
	1434	+ { _MMIO(0x9888), 0x2f904000 },
	1435	+ { _MMIO(0x9888), 0x31904000 },
	1436	+ { _MMIO(0x9888), 0x15904000 },
	1437	+ { _MMIO(0x9888), 0x17904000 },
	1438	+ { _MMIO(0x9888), 0x19904000 },
	1439	+ { _MMIO(0x9888), 0x1b904000 },
	1440	+ { _MMIO(0x9888), 0x1d904000 },
	1441	+ { _MMIO(0x9888), 0x1f904000 },
	1442	+ { _MMIO(0x9888), 0x59900000 },
	1443	+ { _MMIO(0x9888), 0x4b900420 },
	1444	+ { _MMIO(0x9888), 0x37900000 },
	1445	+ { _MMIO(0x9888), 0x33900000 },
	1446	+ { _MMIO(0x9888), 0x4d900000 },
	1447	+ { _MMIO(0x9888), 0x53900000 },
	1448	+ { _MMIO(0x9888), 0x43900000 },
	1449	+ { _MMIO(0x9888), 0x45901084 },
	1450	+ { _MMIO(0x9888), 0x55900000 },
	1451	+ { _MMIO(0x9888), 0x47900001 },
	1452	+};
	1453	+
	1454	+static int
	1455	+get_tdl_1_mux_config(struct drm_i915_private *dev_priv,
	1456	+ const struct i915_oa_reg **regs,
	1457	+ int *lens)
	1458	+{
	1459	+ int n = 0;
	1460	+
	1461	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1462	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1463	+
	1464	+ regs[n] = mux_config_tdl_1;
	1465	+ lens[n] = ARRAY_SIZE(mux_config_tdl_1);
	1466	+ n++;
	1467	+
	1468	+ return n;
	1469	+}
	1470	+
	1471	+static const struct i915_oa_reg b_counter_config_tdl_2[] = {
	1472	+ { _MMIO(0x2740), 0x00000000 },
	1473	+ { _MMIO(0x2744), 0x00800000 },
	1474	+ { _MMIO(0x2710), 0x00000000 },
	1475	+ { _MMIO(0x2714), 0x00800000 },
	1476	+ { _MMIO(0x2720), 0x00000000 },
	1477	+ { _MMIO(0x2724), 0x00800000 },
	1478	+};
	1479	+
	1480	+static const struct i915_oa_reg flex_eu_config_tdl_2[] = {
	1481	+ { _MMIO(0xe458), 0x00005004 },
	1482	+ { _MMIO(0xe558), 0x00010003 },
	1483	+ { _MMIO(0xe658), 0x00012011 },
	1484	+ { _MMIO(0xe758), 0x00015014 },
	1485	+ { _MMIO(0xe45c), 0x00051050 },
	1486	+ { _MMIO(0xe55c), 0x00053052 },
	1487	+ { _MMIO(0xe65c), 0x00055054 },
	1488	+};
	1489	+
	1490	+static const struct i915_oa_reg mux_config_tdl_2[] = {
	1491	+ { _MMIO(0x9888), 0x141a026b },
	1492	+ { _MMIO(0x9888), 0x143a0173 },
	1493	+ { _MMIO(0x9888), 0x145a026b },
	1494	+ { _MMIO(0x9888), 0x002d4000 },
	1495	+ { _MMIO(0x9888), 0x022d5000 },
	1496	+ { _MMIO(0x9888), 0x042d5000 },
	1497	+ { _MMIO(0x9888), 0x062d1000 },
	1498	+ { _MMIO(0x9888), 0x0c2e5000 },
	1499	+ { _MMIO(0x9888), 0x0e2e0069 },
	1500	+ { _MMIO(0x9888), 0x044c8000 },
	1501	+ { _MMIO(0x9888), 0x064cc000 },
	1502	+ { _MMIO(0x9888), 0x0a4c4000 },
	1503	+ { _MMIO(0x9888), 0x004e8000 },
	1504	+ { _MMIO(0x9888), 0x024ea000 },
	1505	+ { _MMIO(0x9888), 0x064e2000 },
	1506	+ { _MMIO(0x9888), 0x180f6000 },
	1507	+ { _MMIO(0x9888), 0x1a0f030a },
	1508	+ { _MMIO(0x9888), 0x1a2c03c0 },
	1509	+ { _MMIO(0x9888), 0x041a37e7 },
	1510	+ { _MMIO(0x9888), 0x021a0000 },
	1511	+ { _MMIO(0x9888), 0x0414a000 },
	1512	+ { _MMIO(0x9888), 0x1c150050 },
	1513	+ { _MMIO(0x9888), 0x08168000 },
	1514	+ { _MMIO(0x9888), 0x0a168000 },
	1515	+ { _MMIO(0x9888), 0x003a3380 },
	1516	+ { _MMIO(0x9888), 0x063a006f },
	1517	+ { _MMIO(0x9888), 0x023a0000 },
	1518	+ { _MMIO(0x9888), 0x00348000 },
	1519	+ { _MMIO(0x9888), 0x06342000 },
	1520	+ { _MMIO(0x9888), 0x1a352000 },
	1521	+ { _MMIO(0x9888), 0x1c350100 },
	1522	+ { _MMIO(0x9888), 0x02368000 },
	1523	+ { _MMIO(0x9888), 0x0c368000 },
	1524	+ { _MMIO(0x9888), 0x025a37e7 },
	1525	+ { _MMIO(0x9888), 0x0254a000 },
	1526	+ { _MMIO(0x9888), 0x1c550005 },
	1527	+ { _MMIO(0x9888), 0x04568000 },
	1528	+ { _MMIO(0x9888), 0x06568000 },
	1529	+ { _MMIO(0x9888), 0x03938000 },
	1530	+ { _MMIO(0x9888), 0x05938000 },
	1531	+ { _MMIO(0x9888), 0x07938000 },
	1532	+ { _MMIO(0x9888), 0x09938000 },
	1533	+ { _MMIO(0x9888), 0x0b938000 },
	1534	+ { _MMIO(0x9888), 0x0d938000 },
	1535	+ { _MMIO(0x9888), 0x15904000 },
	1536	+ { _MMIO(0x9888), 0x17904000 },
	1537	+ { _MMIO(0x9888), 0x19904000 },
	1538	+ { _MMIO(0x9888), 0x1b904000 },
	1539	+ { _MMIO(0x9888), 0x1d904000 },
	1540	+ { _MMIO(0x9888), 0x1f904000 },
	1541	+ { _MMIO(0x9888), 0x37900000 },
	1542	+ { _MMIO(0x9888), 0x53900000 },
	1543	+ { _MMIO(0x9888), 0x43900020 },
	1544	+ { _MMIO(0x9888), 0x45901080 },
	1545	+ { _MMIO(0x9888), 0x55900000 },
	1546	+ { _MMIO(0x9888), 0x47900001 },
	1547	+ { _MMIO(0x9888), 0x33900000 },
	1548	+};
	1549	+
	1550	+static int
	1551	+get_tdl_2_mux_config(struct drm_i915_private *dev_priv,
	1552	+ const struct i915_oa_reg **regs,
	1553	+ int *lens)
	1554	+{
	1555	+ int n = 0;
	1556	+
	1557	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1558	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1559	+
	1560	+ regs[n] = mux_config_tdl_2;
	1561	+ lens[n] = ARRAY_SIZE(mux_config_tdl_2);
	1562	+ n++;
	1563	+
	1564	+ return n;
	1565	+}
	1566	+
	1567	+static const struct i915_oa_reg b_counter_config_compute_extra[] = {
	1568	+ { _MMIO(0x2740), 0x00000000 },
	1569	+ { _MMIO(0x2744), 0x00800000 },
	1570	+ { _MMIO(0x2710), 0x00000000 },
	1571	+ { _MMIO(0x2714), 0x00800000 },
	1572	+ { _MMIO(0x2720), 0x00000000 },
	1573	+ { _MMIO(0x2724), 0x00800000 },
	1574	+};
	1575	+
	1576	+static const struct i915_oa_reg flex_eu_config_compute_extra[] = {
	1577	+ { _MMIO(0xe458), 0x00001000 },
	1578	+ { _MMIO(0xe558), 0x00003002 },
	1579	+ { _MMIO(0xe658), 0x00005004 },
	1580	+ { _MMIO(0xe758), 0x00011010 },
	1581	+ { _MMIO(0xe45c), 0x00050012 },
	1582	+ { _MMIO(0xe55c), 0x00052051 },
	1583	+ { _MMIO(0xe65c), 0x00000008 },
	1584	+};
	1585	+
	1586	+static const struct i915_oa_reg mux_config_compute_extra[] = {
	1587	+ { _MMIO(0x9888), 0x141a001f },
	1588	+ { _MMIO(0x9888), 0x143a001f },
	1589	+ { _MMIO(0x9888), 0x145a001f },
	1590	+ { _MMIO(0x9888), 0x042d5000 },
	1591	+ { _MMIO(0x9888), 0x062d1000 },
	1592	+ { _MMIO(0x9888), 0x0e2e0094 },
	1593	+ { _MMIO(0x9888), 0x084cc000 },
	1594	+ { _MMIO(0x9888), 0x044ea000 },
	1595	+ { _MMIO(0x9888), 0x1a0f00e0 },
	1596	+ { _MMIO(0x9888), 0x1a2c0c00 },
	1597	+ { _MMIO(0x9888), 0x061a0063 },
	1598	+ { _MMIO(0x9888), 0x021a0000 },
	1599	+ { _MMIO(0x9888), 0x06142000 },
	1600	+ { _MMIO(0x9888), 0x1c150100 },
	1601	+ { _MMIO(0x9888), 0x0c168000 },
	1602	+ { _MMIO(0x9888), 0x043a3180 },
	1603	+ { _MMIO(0x9888), 0x023a0000 },
	1604	+ { _MMIO(0x9888), 0x04348000 },
	1605	+ { _MMIO(0x9888), 0x1c350040 },
	1606	+ { _MMIO(0x9888), 0x0a368000 },
	1607	+ { _MMIO(0x9888), 0x045a0063 },
	1608	+ { _MMIO(0x9888), 0x025a0000 },
	1609	+ { _MMIO(0x9888), 0x04542000 },
	1610	+ { _MMIO(0x9888), 0x1c550010 },
	1611	+ { _MMIO(0x9888), 0x08568000 },
	1612	+ { _MMIO(0x9888), 0x09938000 },
	1613	+ { _MMIO(0x9888), 0x0b938000 },
	1614	+ { _MMIO(0x9888), 0x0d938000 },
	1615	+ { _MMIO(0x9888), 0x1b904000 },
	1616	+ { _MMIO(0x9888), 0x1d904000 },
	1617	+ { _MMIO(0x9888), 0x1f904000 },
	1618	+ { _MMIO(0x9888), 0x37900000 },
	1619	+ { _MMIO(0x9888), 0x55900000 },
	1620	+ { _MMIO(0x9888), 0x45900400 },
	1621	+ { _MMIO(0x9888), 0x47900004 },
	1622	+ { _MMIO(0x9888), 0x33900000 },
	1623	+};
	1624	+
	1625	+static int
	1626	+get_compute_extra_mux_config(struct drm_i915_private *dev_priv,
	1627	+ const struct i915_oa_reg **regs,
	1628	+ int *lens)
	1629	+{
	1630	+ int n = 0;
	1631	+
	1632	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1633	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1634	+
	1635	+ regs[n] = mux_config_compute_extra;
	1636	+ lens[n] = ARRAY_SIZE(mux_config_compute_extra);
	1637	+ n++;
	1638	+
	1639	+ return n;
	1640	+}
	1641	+
	1642	+static const struct i915_oa_reg b_counter_config_test_oa[] = {
	1643	+ { _MMIO(0x2740), 0x00000000 },
	1644	+ { _MMIO(0x2744), 0x00800000 },
	1645	+ { _MMIO(0x2714), 0xf0800000 },
	1646	+ { _MMIO(0x2710), 0x00000000 },
	1647	+ { _MMIO(0x2724), 0xf0800000 },
	1648	+ { _MMIO(0x2720), 0x00000000 },
	1649	+ { _MMIO(0x2770), 0x00000004 },
	1650	+ { _MMIO(0x2774), 0x00000000 },
	1651	+ { _MMIO(0x2778), 0x00000003 },
	1652	+ { _MMIO(0x277c), 0x00000000 },
	1653	+ { _MMIO(0x2780), 0x00000007 },
	1654	+ { _MMIO(0x2784), 0x00000000 },
	1655	+ { _MMIO(0x2788), 0x00100002 },
	1656	+ { _MMIO(0x278c), 0x0000fff7 },
	1657	+ { _MMIO(0x2790), 0x00100002 },
	1658	+ { _MMIO(0x2794), 0x0000ffcf },
	1659	+ { _MMIO(0x2798), 0x00100082 },
	1660	+ { _MMIO(0x279c), 0x0000ffef },
	1661	+ { _MMIO(0x27a0), 0x001000c2 },
	1662	+ { _MMIO(0x27a4), 0x0000ffe7 },
	1663	+ { _MMIO(0x27a8), 0x00100001 },
	1664	+ { _MMIO(0x27ac), 0x0000ffe7 },
	1665	+};
	1666	+
	1667	+static const struct i915_oa_reg flex_eu_config_test_oa[] = {
	1668	+};
	1669	+
	1670	+static const struct i915_oa_reg mux_config_test_oa[] = {
	1671	+ { _MMIO(0x9888), 0x19800000 },
	1672	+ { _MMIO(0x9888), 0x07800063 },
	1673	+ { _MMIO(0x9888), 0x11800000 },
	1674	+ { _MMIO(0x9888), 0x23810008 },
	1675	+ { _MMIO(0x9888), 0x1d950400 },
	1676	+ { _MMIO(0x9888), 0x0f922000 },
	1677	+ { _MMIO(0x9888), 0x1f908000 },
	1678	+ { _MMIO(0x9888), 0x37900000 },
	1679	+ { _MMIO(0x9888), 0x55900000 },
	1680	+ { _MMIO(0x9888), 0x47900000 },
	1681	+ { _MMIO(0x9888), 0x33900000 },
	1682	+};
	1683	+
	1684	+static int
	1685	+get_test_oa_mux_config(struct drm_i915_private *dev_priv,
	1686	+ const struct i915_oa_reg **regs,
	1687	+ int *lens)
	1688	+{
	1689	+ int n = 0;
	1690	+
	1691	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs) < 1);
	1692	+ BUILD_BUG_ON(ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens) < 1);
	1693	+
	1694	+ regs[n] = mux_config_test_oa;
	1695	+ lens[n] = ARRAY_SIZE(mux_config_test_oa);
	1696	+ n++;
	1697	+
	1698	+ return n;
	1699	+}
	1700	+
	1701	+int i915_oa_select_metric_set_glk(struct drm_i915_private *dev_priv)
	1702	+{
	1703	+ dev_priv->perf.oa.n_mux_configs = 0;
	1704	+ dev_priv->perf.oa.b_counter_regs = NULL;
	1705	+ dev_priv->perf.oa.b_counter_regs_len = 0;
	1706	+ dev_priv->perf.oa.flex_regs = NULL;
	1707	+ dev_priv->perf.oa.flex_regs_len = 0;
	1708	+
	1709	+ switch (dev_priv->perf.oa.metrics_set) {
	1710	+ case METRIC_SET_ID_RENDER_BASIC:
	1711	+ dev_priv->perf.oa.n_mux_configs =
	1712	+ get_render_basic_mux_config(dev_priv,
	1713	+ dev_priv->perf.oa.mux_regs,
	1714	+ dev_priv->perf.oa.mux_regs_lens);
	1715	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1716	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RENDER_BASIC\" metric set\n");
	1717	+
	1718	+ /* EINVAL because *_register_sysfs already checked this
	1719	+ * and so it wouldn't have been advertised to userspace and
	1720	+ * so shouldn't have been requested
	1721	+ */
	1722	+ return -EINVAL;
	1723	+ }
	1724	+
	1725	+ dev_priv->perf.oa.b_counter_regs =
	1726	+ b_counter_config_render_basic;
	1727	+ dev_priv->perf.oa.b_counter_regs_len =
	1728	+ ARRAY_SIZE(b_counter_config_render_basic);
	1729	+
	1730	+ dev_priv->perf.oa.flex_regs =
	1731	+ flex_eu_config_render_basic;
	1732	+ dev_priv->perf.oa.flex_regs_len =
	1733	+ ARRAY_SIZE(flex_eu_config_render_basic);
	1734	+
	1735	+ return 0;
	1736	+ case METRIC_SET_ID_COMPUTE_BASIC:
	1737	+ dev_priv->perf.oa.n_mux_configs =
	1738	+ get_compute_basic_mux_config(dev_priv,
	1739	+ dev_priv->perf.oa.mux_regs,
	1740	+ dev_priv->perf.oa.mux_regs_lens);
	1741	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1742	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_BASIC\" metric set\n");
	1743	+
	1744	+ /* EINVAL because *_register_sysfs already checked this
	1745	+ * and so it wouldn't have been advertised to userspace and
	1746	+ * so shouldn't have been requested
	1747	+ */
	1748	+ return -EINVAL;
	1749	+ }
	1750	+
	1751	+ dev_priv->perf.oa.b_counter_regs =
	1752	+ b_counter_config_compute_basic;
	1753	+ dev_priv->perf.oa.b_counter_regs_len =
	1754	+ ARRAY_SIZE(b_counter_config_compute_basic);
	1755	+
	1756	+ dev_priv->perf.oa.flex_regs =
	1757	+ flex_eu_config_compute_basic;
	1758	+ dev_priv->perf.oa.flex_regs_len =
	1759	+ ARRAY_SIZE(flex_eu_config_compute_basic);
	1760	+
	1761	+ return 0;
	1762	+ case METRIC_SET_ID_RENDER_PIPE_PROFILE:
	1763	+ dev_priv->perf.oa.n_mux_configs =
	1764	+ get_render_pipe_profile_mux_config(dev_priv,
	1765	+ dev_priv->perf.oa.mux_regs,
	1766	+ dev_priv->perf.oa.mux_regs_lens);
	1767	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1768	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RENDER_PIPE_PROFILE\" metric set\n");
	1769	+
	1770	+ /* EINVAL because *_register_sysfs already checked this
	1771	+ * and so it wouldn't have been advertised to userspace and
	1772	+ * so shouldn't have been requested
	1773	+ */
	1774	+ return -EINVAL;
	1775	+ }
	1776	+
	1777	+ dev_priv->perf.oa.b_counter_regs =
	1778	+ b_counter_config_render_pipe_profile;
	1779	+ dev_priv->perf.oa.b_counter_regs_len =
	1780	+ ARRAY_SIZE(b_counter_config_render_pipe_profile);
	1781	+
	1782	+ dev_priv->perf.oa.flex_regs =
	1783	+ flex_eu_config_render_pipe_profile;
	1784	+ dev_priv->perf.oa.flex_regs_len =
	1785	+ ARRAY_SIZE(flex_eu_config_render_pipe_profile);
	1786	+
	1787	+ return 0;
	1788	+ case METRIC_SET_ID_MEMORY_READS:
	1789	+ dev_priv->perf.oa.n_mux_configs =
	1790	+ get_memory_reads_mux_config(dev_priv,
	1791	+ dev_priv->perf.oa.mux_regs,
	1792	+ dev_priv->perf.oa.mux_regs_lens);
	1793	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1794	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"MEMORY_READS\" metric set\n");
	1795	+
	1796	+ /* EINVAL because *_register_sysfs already checked this
	1797	+ * and so it wouldn't have been advertised to userspace and
	1798	+ * so shouldn't have been requested
	1799	+ */
	1800	+ return -EINVAL;
	1801	+ }
	1802	+
	1803	+ dev_priv->perf.oa.b_counter_regs =
	1804	+ b_counter_config_memory_reads;
	1805	+ dev_priv->perf.oa.b_counter_regs_len =
	1806	+ ARRAY_SIZE(b_counter_config_memory_reads);
	1807	+
	1808	+ dev_priv->perf.oa.flex_regs =
	1809	+ flex_eu_config_memory_reads;
	1810	+ dev_priv->perf.oa.flex_regs_len =
	1811	+ ARRAY_SIZE(flex_eu_config_memory_reads);
	1812	+
	1813	+ return 0;
	1814	+ case METRIC_SET_ID_MEMORY_WRITES:
	1815	+ dev_priv->perf.oa.n_mux_configs =
	1816	+ get_memory_writes_mux_config(dev_priv,
	1817	+ dev_priv->perf.oa.mux_regs,
	1818	+ dev_priv->perf.oa.mux_regs_lens);
	1819	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1820	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"MEMORY_WRITES\" metric set\n");
	1821	+
	1822	+ /* EINVAL because *_register_sysfs already checked this
	1823	+ * and so it wouldn't have been advertised to userspace and
	1824	+ * so shouldn't have been requested
	1825	+ */
	1826	+ return -EINVAL;
	1827	+ }
	1828	+
	1829	+ dev_priv->perf.oa.b_counter_regs =
	1830	+ b_counter_config_memory_writes;
	1831	+ dev_priv->perf.oa.b_counter_regs_len =
	1832	+ ARRAY_SIZE(b_counter_config_memory_writes);
	1833	+
	1834	+ dev_priv->perf.oa.flex_regs =
	1835	+ flex_eu_config_memory_writes;
	1836	+ dev_priv->perf.oa.flex_regs_len =
	1837	+ ARRAY_SIZE(flex_eu_config_memory_writes);
	1838	+
	1839	+ return 0;
	1840	+ case METRIC_SET_ID_COMPUTE_EXTENDED:
	1841	+ dev_priv->perf.oa.n_mux_configs =
	1842	+ get_compute_extended_mux_config(dev_priv,
	1843	+ dev_priv->perf.oa.mux_regs,
	1844	+ dev_priv->perf.oa.mux_regs_lens);
	1845	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1846	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_EXTENDED\" metric set\n");
	1847	+
	1848	+ /* EINVAL because *_register_sysfs already checked this
	1849	+ * and so it wouldn't have been advertised to userspace and
	1850	+ * so shouldn't have been requested
	1851	+ */
	1852	+ return -EINVAL;
	1853	+ }
	1854	+
	1855	+ dev_priv->perf.oa.b_counter_regs =
	1856	+ b_counter_config_compute_extended;
	1857	+ dev_priv->perf.oa.b_counter_regs_len =
	1858	+ ARRAY_SIZE(b_counter_config_compute_extended);
	1859	+
	1860	+ dev_priv->perf.oa.flex_regs =
	1861	+ flex_eu_config_compute_extended;
	1862	+ dev_priv->perf.oa.flex_regs_len =
	1863	+ ARRAY_SIZE(flex_eu_config_compute_extended);
	1864	+
	1865	+ return 0;
	1866	+ case METRIC_SET_ID_COMPUTE_L3_CACHE:
	1867	+ dev_priv->perf.oa.n_mux_configs =
	1868	+ get_compute_l3_cache_mux_config(dev_priv,
	1869	+ dev_priv->perf.oa.mux_regs,
	1870	+ dev_priv->perf.oa.mux_regs_lens);
	1871	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1872	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_L3_CACHE\" metric set\n");
	1873	+
	1874	+ /* EINVAL because *_register_sysfs already checked this
	1875	+ * and so it wouldn't have been advertised to userspace and
	1876	+ * so shouldn't have been requested
	1877	+ */
	1878	+ return -EINVAL;
	1879	+ }
	1880	+
	1881	+ dev_priv->perf.oa.b_counter_regs =
	1882	+ b_counter_config_compute_l3_cache;
	1883	+ dev_priv->perf.oa.b_counter_regs_len =
	1884	+ ARRAY_SIZE(b_counter_config_compute_l3_cache);
	1885	+
	1886	+ dev_priv->perf.oa.flex_regs =
	1887	+ flex_eu_config_compute_l3_cache;
	1888	+ dev_priv->perf.oa.flex_regs_len =
	1889	+ ARRAY_SIZE(flex_eu_config_compute_l3_cache);
	1890	+
	1891	+ return 0;
	1892	+ case METRIC_SET_ID_HDC_AND_SF:
	1893	+ dev_priv->perf.oa.n_mux_configs =
	1894	+ get_hdc_and_sf_mux_config(dev_priv,
	1895	+ dev_priv->perf.oa.mux_regs,
	1896	+ dev_priv->perf.oa.mux_regs_lens);
	1897	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1898	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"HDC_AND_SF\" metric set\n");
	1899	+
	1900	+ /* EINVAL because *_register_sysfs already checked this
	1901	+ * and so it wouldn't have been advertised to userspace and
	1902	+ * so shouldn't have been requested
	1903	+ */
	1904	+ return -EINVAL;
	1905	+ }
	1906	+
	1907	+ dev_priv->perf.oa.b_counter_regs =
	1908	+ b_counter_config_hdc_and_sf;
	1909	+ dev_priv->perf.oa.b_counter_regs_len =
	1910	+ ARRAY_SIZE(b_counter_config_hdc_and_sf);
	1911	+
	1912	+ dev_priv->perf.oa.flex_regs =
	1913	+ flex_eu_config_hdc_and_sf;
	1914	+ dev_priv->perf.oa.flex_regs_len =
	1915	+ ARRAY_SIZE(flex_eu_config_hdc_and_sf);
	1916	+
	1917	+ return 0;
	1918	+ case METRIC_SET_ID_L3_1:
	1919	+ dev_priv->perf.oa.n_mux_configs =
	1920	+ get_l3_1_mux_config(dev_priv,
	1921	+ dev_priv->perf.oa.mux_regs,
	1922	+ dev_priv->perf.oa.mux_regs_lens);
	1923	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1924	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"L3_1\" metric set\n");
	1925	+
	1926	+ /* EINVAL because *_register_sysfs already checked this
	1927	+ * and so it wouldn't have been advertised to userspace and
	1928	+ * so shouldn't have been requested
	1929	+ */
	1930	+ return -EINVAL;
	1931	+ }
	1932	+
	1933	+ dev_priv->perf.oa.b_counter_regs =
	1934	+ b_counter_config_l3_1;
	1935	+ dev_priv->perf.oa.b_counter_regs_len =
	1936	+ ARRAY_SIZE(b_counter_config_l3_1);
	1937	+
	1938	+ dev_priv->perf.oa.flex_regs =
	1939	+ flex_eu_config_l3_1;
	1940	+ dev_priv->perf.oa.flex_regs_len =
	1941	+ ARRAY_SIZE(flex_eu_config_l3_1);
	1942	+
	1943	+ return 0;
	1944	+ case METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND:
	1945	+ dev_priv->perf.oa.n_mux_configs =
	1946	+ get_rasterizer_and_pixel_backend_mux_config(dev_priv,
	1947	+ dev_priv->perf.oa.mux_regs,
	1948	+ dev_priv->perf.oa.mux_regs_lens);
	1949	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1950	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"RASTERIZER_AND_PIXEL_BACKEND\" metric set\n");
	1951	+
	1952	+ /* EINVAL because *_register_sysfs already checked this
	1953	+ * and so it wouldn't have been advertised to userspace and
	1954	+ * so shouldn't have been requested
	1955	+ */
	1956	+ return -EINVAL;
	1957	+ }
	1958	+
	1959	+ dev_priv->perf.oa.b_counter_regs =
	1960	+ b_counter_config_rasterizer_and_pixel_backend;
	1961	+ dev_priv->perf.oa.b_counter_regs_len =
	1962	+ ARRAY_SIZE(b_counter_config_rasterizer_and_pixel_backend);
	1963	+
	1964	+ dev_priv->perf.oa.flex_regs =
	1965	+ flex_eu_config_rasterizer_and_pixel_backend;
	1966	+ dev_priv->perf.oa.flex_regs_len =
	1967	+ ARRAY_SIZE(flex_eu_config_rasterizer_and_pixel_backend);
	1968	+
	1969	+ return 0;
	1970	+ case METRIC_SET_ID_SAMPLER:
	1971	+ dev_priv->perf.oa.n_mux_configs =
	1972	+ get_sampler_mux_config(dev_priv,
	1973	+ dev_priv->perf.oa.mux_regs,
	1974	+ dev_priv->perf.oa.mux_regs_lens);
	1975	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	1976	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"SAMPLER\" metric set\n");
	1977	+
	1978	+ /* EINVAL because *_register_sysfs already checked this
	1979	+ * and so it wouldn't have been advertised to userspace and
	1980	+ * so shouldn't have been requested
	1981	+ */
	1982	+ return -EINVAL;
	1983	+ }
	1984	+
	1985	+ dev_priv->perf.oa.b_counter_regs =
	1986	+ b_counter_config_sampler;
	1987	+ dev_priv->perf.oa.b_counter_regs_len =
	1988	+ ARRAY_SIZE(b_counter_config_sampler);
	1989	+
	1990	+ dev_priv->perf.oa.flex_regs =
	1991	+ flex_eu_config_sampler;
	1992	+ dev_priv->perf.oa.flex_regs_len =
	1993	+ ARRAY_SIZE(flex_eu_config_sampler);
	1994	+
	1995	+ return 0;
	1996	+ case METRIC_SET_ID_TDL_1:
	1997	+ dev_priv->perf.oa.n_mux_configs =
	1998	+ get_tdl_1_mux_config(dev_priv,
	1999	+ dev_priv->perf.oa.mux_regs,
	2000	+ dev_priv->perf.oa.mux_regs_lens);
	2001	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2002	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TDL_1\" metric set\n");
	2003	+
	2004	+ /* EINVAL because *_register_sysfs already checked this
	2005	+ * and so it wouldn't have been advertised to userspace and
	2006	+ * so shouldn't have been requested
	2007	+ */
	2008	+ return -EINVAL;
	2009	+ }
	2010	+
	2011	+ dev_priv->perf.oa.b_counter_regs =
	2012	+ b_counter_config_tdl_1;
	2013	+ dev_priv->perf.oa.b_counter_regs_len =
	2014	+ ARRAY_SIZE(b_counter_config_tdl_1);
	2015	+
	2016	+ dev_priv->perf.oa.flex_regs =
	2017	+ flex_eu_config_tdl_1;
	2018	+ dev_priv->perf.oa.flex_regs_len =
	2019	+ ARRAY_SIZE(flex_eu_config_tdl_1);
	2020	+
	2021	+ return 0;
	2022	+ case METRIC_SET_ID_TDL_2:
	2023	+ dev_priv->perf.oa.n_mux_configs =
	2024	+ get_tdl_2_mux_config(dev_priv,
	2025	+ dev_priv->perf.oa.mux_regs,
	2026	+ dev_priv->perf.oa.mux_regs_lens);
	2027	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2028	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TDL_2\" metric set\n");
	2029	+
	2030	+ /* EINVAL because *_register_sysfs already checked this
	2031	+ * and so it wouldn't have been advertised to userspace and
	2032	+ * so shouldn't have been requested
	2033	+ */
	2034	+ return -EINVAL;
	2035	+ }
	2036	+
	2037	+ dev_priv->perf.oa.b_counter_regs =
	2038	+ b_counter_config_tdl_2;
	2039	+ dev_priv->perf.oa.b_counter_regs_len =
	2040	+ ARRAY_SIZE(b_counter_config_tdl_2);
	2041	+
	2042	+ dev_priv->perf.oa.flex_regs =
	2043	+ flex_eu_config_tdl_2;
	2044	+ dev_priv->perf.oa.flex_regs_len =
	2045	+ ARRAY_SIZE(flex_eu_config_tdl_2);
	2046	+
	2047	+ return 0;
	2048	+ case METRIC_SET_ID_COMPUTE_EXTRA:
	2049	+ dev_priv->perf.oa.n_mux_configs =
	2050	+ get_compute_extra_mux_config(dev_priv,
	2051	+ dev_priv->perf.oa.mux_regs,
	2052	+ dev_priv->perf.oa.mux_regs_lens);
	2053	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2054	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"COMPUTE_EXTRA\" metric set\n");
	2055	+
	2056	+ /* EINVAL because *_register_sysfs already checked this
	2057	+ * and so it wouldn't have been advertised to userspace and
	2058	+ * so shouldn't have been requested
	2059	+ */
	2060	+ return -EINVAL;
	2061	+ }
	2062	+
	2063	+ dev_priv->perf.oa.b_counter_regs =
	2064	+ b_counter_config_compute_extra;
	2065	+ dev_priv->perf.oa.b_counter_regs_len =
	2066	+ ARRAY_SIZE(b_counter_config_compute_extra);
	2067	+
	2068	+ dev_priv->perf.oa.flex_regs =
	2069	+ flex_eu_config_compute_extra;
	2070	+ dev_priv->perf.oa.flex_regs_len =
	2071	+ ARRAY_SIZE(flex_eu_config_compute_extra);
	2072	+
	2073	+ return 0;
	2074	+ case METRIC_SET_ID_TEST_OA:
	2075	+ dev_priv->perf.oa.n_mux_configs =
	2076	+ get_test_oa_mux_config(dev_priv,
	2077	+ dev_priv->perf.oa.mux_regs,
	2078	+ dev_priv->perf.oa.mux_regs_lens);
	2079	+ if (dev_priv->perf.oa.n_mux_configs == 0) {
	2080	+ DRM_DEBUG_DRIVER("No suitable MUX config for \"TEST_OA\" metric set\n");
	2081	+
	2082	+ /* EINVAL because *_register_sysfs already checked this
	2083	+ * and so it wouldn't have been advertised to userspace and
	2084	+ * so shouldn't have been requested
	2085	+ */
	2086	+ return -EINVAL;
	2087	+ }
	2088	+
	2089	+ dev_priv->perf.oa.b_counter_regs =
	2090	+ b_counter_config_test_oa;
	2091	+ dev_priv->perf.oa.b_counter_regs_len =
	2092	+ ARRAY_SIZE(b_counter_config_test_oa);
	2093	+
	2094	+ dev_priv->perf.oa.flex_regs =
	2095	+ flex_eu_config_test_oa;
	2096	+ dev_priv->perf.oa.flex_regs_len =
	2097	+ ARRAY_SIZE(flex_eu_config_test_oa);
	2098	+
	2099	+ return 0;
	2100	+ default:
	2101	+ return -ENODEV;
	2102	+ }
	2103	+}
	2104	+
	2105	+static ssize_t
	2106	+show_render_basic_id(struct device kdev, struct device_attribute attr, char *buf)
	2107	+{
	2108	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RENDER_BASIC);
	2109	+}
	2110	+
	2111	+static struct device_attribute dev_attr_render_basic_id = {
	2112	+ .attr = { .name = "id", .mode = 0444 },
	2113	+ .show = show_render_basic_id,
	2114	+ .store = NULL,
	2115	+};
	2116	+
	2117	+static struct attribute *attrs_render_basic[] = {
	2118	+ &dev_attr_render_basic_id.attr,
	2119	+ NULL,
	2120	+};
	2121	+
	2122	+static struct attribute_group group_render_basic = {
	2123	+ .name = "d72df5c7-5b4a-4274-a43f-00b0fd51fc68",
	2124	+ .attrs = attrs_render_basic,
	2125	+};
	2126	+
	2127	+static ssize_t
	2128	+show_compute_basic_id(struct device kdev, struct device_attribute attr, char *buf)
	2129	+{
	2130	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_BASIC);
	2131	+}
	2132	+
	2133	+static struct device_attribute dev_attr_compute_basic_id = {
	2134	+ .attr = { .name = "id", .mode = 0444 },
	2135	+ .show = show_compute_basic_id,
	2136	+ .store = NULL,
	2137	+};
	2138	+
	2139	+static struct attribute *attrs_compute_basic[] = {
	2140	+ &dev_attr_compute_basic_id.attr,
	2141	+ NULL,
	2142	+};
	2143	+
	2144	+static struct attribute_group group_compute_basic = {
	2145	+ .name = "814285f6-354d-41d2-ba49-e24e622714a0",
	2146	+ .attrs = attrs_compute_basic,
	2147	+};
	2148	+
	2149	+static ssize_t
	2150	+show_render_pipe_profile_id(struct device kdev, struct device_attribute attr, char *buf)
	2151	+{
	2152	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RENDER_PIPE_PROFILE);
	2153	+}
	2154	+
	2155	+static struct device_attribute dev_attr_render_pipe_profile_id = {
	2156	+ .attr = { .name = "id", .mode = 0444 },
	2157	+ .show = show_render_pipe_profile_id,
	2158	+ .store = NULL,
	2159	+};
	2160	+
	2161	+static struct attribute *attrs_render_pipe_profile[] = {
	2162	+ &dev_attr_render_pipe_profile_id.attr,
	2163	+ NULL,
	2164	+};
	2165	+
	2166	+static struct attribute_group group_render_pipe_profile = {
	2167	+ .name = "07d397a6-b3e6-49f6-9433-a4f293d55978",
	2168	+ .attrs = attrs_render_pipe_profile,
	2169	+};
	2170	+
	2171	+static ssize_t
	2172	+show_memory_reads_id(struct device kdev, struct device_attribute attr, char *buf)
	2173	+{
	2174	+ return sprintf(buf, "%d\n", METRIC_SET_ID_MEMORY_READS);
	2175	+}
	2176	+
	2177	+static struct device_attribute dev_attr_memory_reads_id = {
	2178	+ .attr = { .name = "id", .mode = 0444 },
	2179	+ .show = show_memory_reads_id,
	2180	+ .store = NULL,
	2181	+};
	2182	+
	2183	+static struct attribute *attrs_memory_reads[] = {
	2184	+ &dev_attr_memory_reads_id.attr,
	2185	+ NULL,
	2186	+};
	2187	+
	2188	+static struct attribute_group group_memory_reads = {
	2189	+ .name = "1a356946-5428-450b-a2f0-89f8783a302d",
	2190	+ .attrs = attrs_memory_reads,
	2191	+};
	2192	+
	2193	+static ssize_t
	2194	+show_memory_writes_id(struct device kdev, struct device_attribute attr, char *buf)
	2195	+{
	2196	+ return sprintf(buf, "%d\n", METRIC_SET_ID_MEMORY_WRITES);
	2197	+}
	2198	+
	2199	+static struct device_attribute dev_attr_memory_writes_id = {
	2200	+ .attr = { .name = "id", .mode = 0444 },
	2201	+ .show = show_memory_writes_id,
	2202	+ .store = NULL,
	2203	+};
	2204	+
	2205	+static struct attribute *attrs_memory_writes[] = {
	2206	+ &dev_attr_memory_writes_id.attr,
	2207	+ NULL,
	2208	+};
	2209	+
	2210	+static struct attribute_group group_memory_writes = {
	2211	+ .name = "5299be9d-7a61-4c99-9f81-f87e6c5aaca9",
	2212	+ .attrs = attrs_memory_writes,
	2213	+};
	2214	+
	2215	+static ssize_t
	2216	+show_compute_extended_id(struct device kdev, struct device_attribute attr, char *buf)
	2217	+{
	2218	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_EXTENDED);
	2219	+}
	2220	+
	2221	+static struct device_attribute dev_attr_compute_extended_id = {
	2222	+ .attr = { .name = "id", .mode = 0444 },
	2223	+ .show = show_compute_extended_id,
	2224	+ .store = NULL,
	2225	+};
	2226	+
	2227	+static struct attribute *attrs_compute_extended[] = {
	2228	+ &dev_attr_compute_extended_id.attr,
	2229	+ NULL,
	2230	+};
	2231	+
	2232	+static struct attribute_group group_compute_extended = {
	2233	+ .name = "bc9bcff2-459a-4cbc-986d-a84b077153f3",
	2234	+ .attrs = attrs_compute_extended,
	2235	+};
	2236	+
	2237	+static ssize_t
	2238	+show_compute_l3_cache_id(struct device kdev, struct device_attribute attr, char *buf)
	2239	+{
	2240	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_L3_CACHE);
	2241	+}
	2242	+
	2243	+static struct device_attribute dev_attr_compute_l3_cache_id = {
	2244	+ .attr = { .name = "id", .mode = 0444 },
	2245	+ .show = show_compute_l3_cache_id,
	2246	+ .store = NULL,
	2247	+};
	2248	+
	2249	+static struct attribute *attrs_compute_l3_cache[] = {
	2250	+ &dev_attr_compute_l3_cache_id.attr,
	2251	+ NULL,
	2252	+};
	2253	+
	2254	+static struct attribute_group group_compute_l3_cache = {
	2255	+ .name = "88ec931f-5b4a-453a-9db6-a61232b6143d",
	2256	+ .attrs = attrs_compute_l3_cache,
	2257	+};
	2258	+
	2259	+static ssize_t
	2260	+show_hdc_and_sf_id(struct device kdev, struct device_attribute attr, char *buf)
	2261	+{
	2262	+ return sprintf(buf, "%d\n", METRIC_SET_ID_HDC_AND_SF);
	2263	+}
	2264	+
	2265	+static struct device_attribute dev_attr_hdc_and_sf_id = {
	2266	+ .attr = { .name = "id", .mode = 0444 },
	2267	+ .show = show_hdc_and_sf_id,
	2268	+ .store = NULL,
	2269	+};
	2270	+
	2271	+static struct attribute *attrs_hdc_and_sf[] = {
	2272	+ &dev_attr_hdc_and_sf_id.attr,
	2273	+ NULL,
	2274	+};
	2275	+
	2276	+static struct attribute_group group_hdc_and_sf = {
	2277	+ .name = "530d176d-2a18-4014-adf8-1500c6c60835",
	2278	+ .attrs = attrs_hdc_and_sf,
	2279	+};
	2280	+
	2281	+static ssize_t
	2282	+show_l3_1_id(struct device kdev, struct device_attribute attr, char *buf)
	2283	+{
	2284	+ return sprintf(buf, "%d\n", METRIC_SET_ID_L3_1);
	2285	+}
	2286	+
	2287	+static struct device_attribute dev_attr_l3_1_id = {
	2288	+ .attr = { .name = "id", .mode = 0444 },
	2289	+ .show = show_l3_1_id,
	2290	+ .store = NULL,
	2291	+};
	2292	+
	2293	+static struct attribute *attrs_l3_1[] = {
	2294	+ &dev_attr_l3_1_id.attr,
	2295	+ NULL,
	2296	+};
	2297	+
	2298	+static struct attribute_group group_l3_1 = {
	2299	+ .name = "fdee5a5a-f23c-43d1-aa73-f6257c71671d",
	2300	+ .attrs = attrs_l3_1,
	2301	+};
	2302	+
	2303	+static ssize_t
	2304	+show_rasterizer_and_pixel_backend_id(struct device kdev, struct device_attribute attr, char *buf)
	2305	+{
	2306	+ return sprintf(buf, "%d\n", METRIC_SET_ID_RASTERIZER_AND_PIXEL_BACKEND);
	2307	+}
	2308	+
	2309	+static struct device_attribute dev_attr_rasterizer_and_pixel_backend_id = {
	2310	+ .attr = { .name = "id", .mode = 0444 },
	2311	+ .show = show_rasterizer_and_pixel_backend_id,
	2312	+ .store = NULL,
	2313	+};
	2314	+
	2315	+static struct attribute *attrs_rasterizer_and_pixel_backend[] = {
	2316	+ &dev_attr_rasterizer_and_pixel_backend_id.attr,
	2317	+ NULL,
	2318	+};
	2319	+
	2320	+static struct attribute_group group_rasterizer_and_pixel_backend = {
	2321	+ .name = "6617623e-ca73-4791-b2b7-ddedd0846a0c",
	2322	+ .attrs = attrs_rasterizer_and_pixel_backend,
	2323	+};
	2324	+
	2325	+static ssize_t
	2326	+show_sampler_id(struct device kdev, struct device_attribute attr, char *buf)
	2327	+{
	2328	+ return sprintf(buf, "%d\n", METRIC_SET_ID_SAMPLER);
	2329	+}
	2330	+
	2331	+static struct device_attribute dev_attr_sampler_id = {
	2332	+ .attr = { .name = "id", .mode = 0444 },
	2333	+ .show = show_sampler_id,
	2334	+ .store = NULL,
	2335	+};
	2336	+
	2337	+static struct attribute *attrs_sampler[] = {
	2338	+ &dev_attr_sampler_id.attr,
	2339	+ NULL,
	2340	+};
	2341	+
	2342	+static struct attribute_group group_sampler = {
	2343	+ .name = "f3b2ea63-e82e-4234-b418-44dd20dd34d0",
	2344	+ .attrs = attrs_sampler,
	2345	+};
	2346	+
	2347	+static ssize_t
	2348	+show_tdl_1_id(struct device kdev, struct device_attribute attr, char *buf)
	2349	+{
	2350	+ return sprintf(buf, "%d\n", METRIC_SET_ID_TDL_1);
	2351	+}
	2352	+
	2353	+static struct device_attribute dev_attr_tdl_1_id = {
	2354	+ .attr = { .name = "id", .mode = 0444 },
	2355	+ .show = show_tdl_1_id,
	2356	+ .store = NULL,
	2357	+};
	2358	+
	2359	+static struct attribute *attrs_tdl_1[] = {
	2360	+ &dev_attr_tdl_1_id.attr,
	2361	+ NULL,
	2362	+};
	2363	+
	2364	+static struct attribute_group group_tdl_1 = {
	2365	+ .name = "14411d35-cbf6-4f5e-b68b-190faf9a1a83",
	2366	+ .attrs = attrs_tdl_1,
	2367	+};
	2368	+
	2369	+static ssize_t
	2370	+show_tdl_2_id(struct device kdev, struct device_attribute attr, char *buf)
	2371	+{
	2372	+ return sprintf(buf, "%d\n", METRIC_SET_ID_TDL_2);
	2373	+}
	2374	+
	2375	+static struct device_attribute dev_attr_tdl_2_id = {
	2376	+ .attr = { .name = "id", .mode = 0444 },
	2377	+ .show = show_tdl_2_id,
	2378	+ .store = NULL,
	2379	+};
	2380	+
	2381	+static struct attribute *attrs_tdl_2[] = {
	2382	+ &dev_attr_tdl_2_id.attr,
	2383	+ NULL,
	2384	+};
	2385	+
	2386	+static struct attribute_group group_tdl_2 = {
	2387	+ .name = "ffa3f263-0478-4724-8c9f-c911c5ec0f1d",
	2388	+ .attrs = attrs_tdl_2,
	2389	+};
	2390	+
	2391	+static ssize_t
	2392	+show_compute_extra_id(struct device kdev, struct device_attribute attr, char *buf)
	2393	+{
	2394	+ return sprintf(buf, "%d\n", METRIC_SET_ID_COMPUTE_EXTRA);
	2395	+}
	2396	+
	2397	+static struct device_attribute dev_attr_compute_extra_id = {
	2398	+ .attr = { .name = "id", .mode = 0444 },
	2399	+ .show = show_compute_extra_id,
	2400	+ .store = NULL,
	2401	+};
	2402	+
	2403	+static struct attribute *attrs_compute_extra[] = {
	2404	+ &dev_attr_compute_extra_id.attr,
	2405	+ NULL,
	2406	+};
	2407	+
	2408	+static struct attribute_group group_compute_extra = {
	2409	+ .name = "15274c82-27d2-4819-876a-7cb1a2c59ba4",
	2410	+ .attrs = attrs_compute_extra,
	2411	+};
	2412	+
	2413	+static ssize_t
	2414	+show_test_oa_id(struct device kdev, struct device_attribute attr, char *buf)
	2415	+{
	2416	+ return sprintf(buf, "%d\n", METRIC_SET_ID_TEST_OA);
	2417	+}
	2418	+
	2419	+static struct device_attribute dev_attr_test_oa_id = {
	2420	+ .attr = { .name = "id", .mode = 0444 },
	2421	+ .show = show_test_oa_id,
	2422	+ .store = NULL,
	2423	+};
	2424	+
	2425	+static struct attribute *attrs_test_oa[] = {
	2426	+ &dev_attr_test_oa_id.attr,
	2427	+ NULL,
	2428	+};
	2429	+
	2430	+static struct attribute_group group_test_oa = {
	2431	+ .name = "dd3fd789-e783-4204-8cd0-b671bbccb0cf",
	2432	+ .attrs = attrs_test_oa,
	2433	+};
	2434	+
	2435	+int
	2436	+i915_perf_register_sysfs_glk(struct drm_i915_private *dev_priv)
	2437	+{
	2438	+ const struct i915_oa_reg *mux_regs[ARRAY_SIZE(dev_priv->perf.oa.mux_regs)];
	2439	+ int mux_lens[ARRAY_SIZE(dev_priv->perf.oa.mux_regs_lens)];
	2440	+ int ret = 0;
	2441	+
	2442	+ if (get_render_basic_mux_config(dev_priv, mux_regs, mux_lens)) {
	2443	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_render_basic);
	2444	+ if (ret)
	2445	+ goto error_render_basic;
	2446	+ }
	2447	+ if (get_compute_basic_mux_config(dev_priv, mux_regs, mux_lens)) {
	2448	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_basic);
	2449	+ if (ret)
	2450	+ goto error_compute_basic;
	2451	+ }
	2452	+ if (get_render_pipe_profile_mux_config(dev_priv, mux_regs, mux_lens)) {
	2453	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_render_pipe_profile);
	2454	+ if (ret)
	2455	+ goto error_render_pipe_profile;
	2456	+ }
	2457	+ if (get_memory_reads_mux_config(dev_priv, mux_regs, mux_lens)) {
	2458	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_memory_reads);
	2459	+ if (ret)
	2460	+ goto error_memory_reads;
	2461	+ }
	2462	+ if (get_memory_writes_mux_config(dev_priv, mux_regs, mux_lens)) {
	2463	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_memory_writes);
	2464	+ if (ret)
	2465	+ goto error_memory_writes;
	2466	+ }
	2467	+ if (get_compute_extended_mux_config(dev_priv, mux_regs, mux_lens)) {
	2468	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_extended);
	2469	+ if (ret)
	2470	+ goto error_compute_extended;
	2471	+ }
	2472	+ if (get_compute_l3_cache_mux_config(dev_priv, mux_regs, mux_lens)) {
	2473	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_l3_cache);
	2474	+ if (ret)
	2475	+ goto error_compute_l3_cache;
	2476	+ }
	2477	+ if (get_hdc_and_sf_mux_config(dev_priv, mux_regs, mux_lens)) {
	2478	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_hdc_and_sf);
	2479	+ if (ret)
	2480	+ goto error_hdc_and_sf;
	2481	+ }
	2482	+ if (get_l3_1_mux_config(dev_priv, mux_regs, mux_lens)) {
	2483	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_l3_1);
	2484	+ if (ret)
	2485	+ goto error_l3_1;
	2486	+ }
	2487	+ if (get_rasterizer_and_pixel_backend_mux_config(dev_priv, mux_regs, mux_lens)) {
	2488	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_rasterizer_and_pixel_backend);
	2489	+ if (ret)
	2490	+ goto error_rasterizer_and_pixel_backend;
	2491	+ }
	2492	+ if (get_sampler_mux_config(dev_priv, mux_regs, mux_lens)) {
	2493	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_sampler);
	2494	+ if (ret)
	2495	+ goto error_sampler;
	2496	+ }
	2497	+ if (get_tdl_1_mux_config(dev_priv, mux_regs, mux_lens)) {
	2498	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_tdl_1);
	2499	+ if (ret)
	2500	+ goto error_tdl_1;
	2501	+ }
	2502	+ if (get_tdl_2_mux_config(dev_priv, mux_regs, mux_lens)) {
	2503	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_tdl_2);
	2504	+ if (ret)
	2505	+ goto error_tdl_2;
	2506	+ }
	2507	+ if (get_compute_extra_mux_config(dev_priv, mux_regs, mux_lens)) {
	2508	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_compute_extra);
	2509	+ if (ret)
	2510	+ goto error_compute_extra;
	2511	+ }
	2512	+ if (get_test_oa_mux_config(dev_priv, mux_regs, mux_lens)) {
	2513	+ ret = sysfs_create_group(dev_priv->perf.metrics_kobj, &group_test_oa);
	2514	+ if (ret)
	2515	+ goto error_test_oa;
	2516	+ }
	2517	+
	2518	+ return 0;
	2519	+
	2520	+error_test_oa:
	2521	+ if (get_compute_extra_mux_config(dev_priv, mux_regs, mux_lens))
	2522	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_compute_extra);
	2523	+error_compute_extra:
	2524	+ if (get_tdl_2_mux_config(dev_priv, mux_regs, mux_lens))
	2525	+ sysfs_remove_group(dev_priv->perf.metrics_kobj, &group_tdl_2);
	2526	+error_tdl_2:

76 more lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_glk.h+40−0

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_hsw.c+187−76

This diff is large: its 499 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_hsw.h+3−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_kblgt2.c+2991−0

This diff is large: its 2992 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_kblgt2.h+40−0

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_kblgt3.c+3040−0

This diff is large: its 3041 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_kblgt3.h+40−0

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_sklgt2.c+3479−0

This diff is large: its 3480 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_sklgt2.h+40−0

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_sklgt3.c+3039−0

This diff is large: its 3040 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_sklgt3.h+40−0

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_sklgt4.c+3093−0

This diff is large: its 3094 lines aren’t shown here.

drivers/gpu/drm/i915/i915_oa_sklgt4.h+40−0

This diff is large: its 41 lines aren’t shown here.

drivers/gpu/drm/i915/i915_pci.c+57−30

This diff is large: its 133 lines aren’t shown here.

drivers/gpu/drm/i915/i915_perf.c+1032−94

This diff is large: its 1401 lines aren’t shown here.

drivers/gpu/drm/i915/i915_pvinfo.h+2−6

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/i915/i915_reg.h+270−17

This diff is large: its 445 lines aren’t shown here.

drivers/gpu/drm/i915/i915_utils.h+5−0

This diff is large: its 12 lines aren’t shown here.

drivers/gpu/drm/i915/i915_vgpu.c+35−19

This diff is large: its 122 lines aren’t shown here.

drivers/gpu/drm/i915/i915_vma.c+25−3

This diff is large: its 66 lines aren’t shown here.

drivers/gpu/drm/i915/i915_vma.h+15−5

This diff is large: its 48 lines aren’t shown here.

drivers/gpu/drm/i915/intel_atomic.c+107−30

This diff is large: its 160 lines aren’t shown here.

drivers/gpu/drm/i915/intel_breadcrumbs.c+43−30

This diff is large: its 155 lines aren’t shown here.

drivers/gpu/drm/i915/intel_cdclk.c+350−5

This diff is large: its 413 lines aren’t shown here.

drivers/gpu/drm/i915/intel_csr.c+12−5

This diff is large: its 47 lines aren’t shown here.

drivers/gpu/drm/i915/intel_ddi.c+377−15

This diff is large: its 515 lines aren’t shown here.

drivers/gpu/drm/i915/intel_device_info.c+6−5

This diff is large: its 38 lines aren’t shown here.

drivers/gpu/drm/i915/intel_display.c+162−158

This diff is large: its 555 lines aren’t shown here.

drivers/gpu/drm/i915/intel_dp.c+48−142

This diff is large: its 368 lines aren’t shown here.

drivers/gpu/drm/i915/intel_dp_aux_backlight.c+9−8

This diff is large: its 57 lines aren’t shown here.

drivers/gpu/drm/i915/intel_dpll_mgr.c+435−2

This diff is large: its 458 lines aren’t shown here.

drivers/gpu/drm/i915/intel_dpll_mgr.h+4−0

This diff is large: its 11 lines aren’t shown here.

drivers/gpu/drm/i915/intel_drv.h+39−20

This diff is large: its 148 lines aren’t shown here.

drivers/gpu/drm/i915/intel_dsi.c+83−84

This diff is large: its 286 lines aren’t shown here.

drivers/gpu/drm/i915/intel_dsi_dcs_backlight.c+10−12

This diff is large: its 56 lines aren’t shown here.

drivers/gpu/drm/i915/intel_engine_cs.c+64−22

This diff is large: its 179 lines aren’t shown here.

drivers/gpu/drm/i915/intel_fifo_underrun.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_guc_loader.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_gvt.c+32−11

This diff is large: its 59 lines aren’t shown here.

drivers/gpu/drm/i915/intel_gvt.h+5−0

This diff is large: its 19 lines aren’t shown here.

drivers/gpu/drm/i915/intel_hdmi.c+38−119

This diff is large: its 296 lines aren’t shown here.

drivers/gpu/drm/i915/intel_huc.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_i2c.c+13−2

This diff is large: its 33 lines aren’t shown here.

drivers/gpu/drm/i915/intel_lrc.c+7−0

This diff is large: its 28 lines aren’t shown here.

drivers/gpu/drm/i915/intel_lvds.c+16−57

This diff is large: its 144 lines aren’t shown here.

drivers/gpu/drm/i915/intel_mocs.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_opregion.c+1−1

This diff is large: its 9 lines aren’t shown here.

drivers/gpu/drm/i915/intel_overlay.c+0−1

This diff is large: its 8 lines aren’t shown here.

drivers/gpu/drm/i915/intel_panel.c+186−79

This diff is large: its 581 lines aren’t shown here.

drivers/gpu/drm/i915/intel_pm.c+126−22

This diff is large: its 252 lines aren’t shown here.

drivers/gpu/drm/i915/intel_runtime_pm.c+308−8

This diff is large: its 403 lines aren’t shown here.

drivers/gpu/drm/i915/intel_sdvo.c+284−256

This diff is large: its 834 lines aren’t shown here.

drivers/gpu/drm/i915/intel_uc.c+5−2

This diff is large: its 22 lines aren’t shown here.

drivers/gpu/drm/i915/intel_uncore.c+0−126

This diff is large: its 168 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/huge_gem_object.c+3−1

This diff is large: its 13 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_gem_evict.c+2−2

This diff is large: its 18 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/i915_vma.c+3−13

This diff is large: its 46 lines aren’t shown here.

drivers/gpu/drm/i915/selftests/mock_context.c+11−1

This diff is large: its 29 lines aren’t shown here.

include/drm/i915_pciids.h+40−0

This diff is large: its 45 lines aren’t shown here.

include/uapi/drm/i915_drm.h+36−8

This diff is large: its 65 lines aren’t shown here.

Merge tag 'drm-intel-next-2017-06-19' of git://anongit.freedesktop.org/git/drm-intel into drm-next

Keyboard shortcuts

General

Issues and pull requests

Issues, pull requests and Inbox

Repository code