UAPI Changes: - Add additional error components to xe drm_ras (Riana) - Drop 'force_execlist' module parameter (Roper) Cross-subsystem Changes: - Perf events: Export perf_allow_{cpu,tracepoint} to be used by Xe (John) Display Changes: - Skip FORCE_WC and vm_bound check for external dma-bufs (Auld) Driver Changes: - Gate observation streams with perf_allow_cpu (John Hubbard) - Documentation updates and fixes (Michal, Zhan) - Remove unreleased NVL-S GuC (Daniele) - Fix pcode init path (Michal) - RTP fixes and improvements (Gustavo, Violet, Thomas, Roper) - TLB invalidation fixes and improvements (Tilak) - PXP detachment from HuC for newer platforms (Daniele) - Multi-queue fix (Niranjana) - Improve Kconfig.profile help (Rodrigo) - Xe_drm_ras and hw_error updates and fixes (Raag, Riana) - NVL-S updated PCI-IDs and W/a (Gustavo, Nitin) - Fix dma_fence refcound (Wentao) - Madvise: optimize invalidation path (Arvind) - Fix a infinite gt-reset loop in the timeout recovery (Rodrigo) - Fix wa_oob codegen recipe for external module builds (Thomas) - Avoid global forcewake in cycle query path (Xin) - Update TTM device benefical_order (Brost) - Fix buffer overflow in guc capture (Tejas) - Page-table fixes and improvements (Brian, Francois, Auld, Brost) - Removing redundant check (Lu) - General MCR and MMIO clean-up and improvements (Michal) - Don't whitelist OA registers unconditionally (Ashutosh) - SVM error return fix (Brost) - Userptr fix and small related clean-ups (Shuicheng) - Don't attempt to process FAST_REQ or EVENT relays on PF (Michal) - Couple fdinfo improvements (Auld) - Drop manual VF check on i2c (Raag) - Probe info outside of xe_info_init functions (Gustavo) Signed-off-by: Dave Airlie <airlied@redhat.com>
Unified · Split
| @@ -251,6 +251,13 @@ Description: RO. Fan 2 speed in RPM. · 250 unmodified lines | |||
| 251 | 251 | ||
| 252 | Only supported for particular Intel Xe graphics platforms. | 252 | Only supported for particular Intel Xe graphics platforms. |
| 253 | 253 | ||
| 254 | On DG2 the driver always shows two fan channels, because the | ||
| 255 | FSC_READ_NUM_FANS command does not work on some cards. OEMs | ||
| 256 | decide how the fans map to tach channels, so two fans can share | ||
| 257 | one tach line. When that happens, the second channel | ||
| 258 | reads 0 RPM even though the fan is spinning. This is normal, not | ||
| 259 | a bug. | ||
| 260 | |||
| 254 | What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/fan3_input | 261 | What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/fan3_input |
| 255 | Date: March 2025 | 262 | Date: March 2025 |
| 256 | KernelVersion: 6.16 | 263 | KernelVersion: 6.16 |
| @@ -1,50 +1,71 @@ | |||
| 1 | # SPDX-License-Identifier: GPL-2.0-only | 1 | # SPDX-License-Identifier: GPL-2.0-only |
| 2 | config DRM_XE_JOB_TIMEOUT_MAX | 2 | config DRM_XE_JOB_TIMEOUT_MAX |
| 3 | int "Default max job timeout (ms)" | 3 | int "Hard upper limit for job timeout (ms)" |
| 4 | default 10000 # milliseconds | 4 | default 10000 # milliseconds |
| 5 | help | 5 | help |
| 6 | Configures the default max job timeout after which job will | 6 | Absolute upper bound (in milliseconds) for the per-engine-class job |
| 7 | be forcefully taken away from scheduler. | 7 | timeout. This is the maximum value that can be written to the sysfs |
| 8 | job_timeout_ms knob, regardless of privileges. To raise this ceiling, | ||
| 9 | increase this value and rebuild the kernel. | ||
| 8 | config DRM_XE_JOB_TIMEOUT_MIN | 10 | config DRM_XE_JOB_TIMEOUT_MIN |
| 9 | int "Default min job timeout (ms)" | 11 | int "Hard lower limit for job timeout (ms)" |
| 10 | default 1 # milliseconds | 12 | default 1 # milliseconds |
| 11 | help | 13 | help |
| 12 | Configures the default min job timeout after which job will | 14 | Absolute lower bound (in milliseconds) for the per-engine-class job |
| 13 | be forcefully taken away from scheduler. | 15 | timeout. This is the minimum value that can be written to the sysfs |
| 16 | job_timeout_ms knob, regardless of privileges. | ||
| 17 | |||
| 18 | Note: the job timeout default (5000 ms) is hardcoded in the driver | ||
| 19 | and is not configurable here. Use the sysfs job_timeout_ms knob at | ||
| 20 | runtime to change the engine-class default. | ||
| 14 | config DRM_XE_TIMESLICE_MAX | 21 | config DRM_XE_TIMESLICE_MAX |
| 15 | int "Default max timeslice duration (us)" | 22 | int "Hard upper limit for timeslice duration (us)" |
| 16 | default 10000000 # microseconds | 23 | default 10000000 # microseconds |
| 17 | help | 24 | help |
| 18 | Configures the default max timeslice duration between multiple | 25 | Absolute upper bound (in microseconds) for the timeslice duration. |
| 19 | contexts by guc scheduling. | 26 | This caps both the sysfs timeslice_duration_us knob and the value |
| 27 | accepted via the DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE UAPI for | ||
| 28 | processes with CAP_SYS_NICE when DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT | ||
| 29 | is enabled. | ||
| 20 | config DRM_XE_TIMESLICE_MIN | 30 | config DRM_XE_TIMESLICE_MIN |
| 21 | int "Default min timeslice duration (us)" | 31 | int "Hard lower limit for timeslice duration (us)" |
| 22 | default 1 # microseconds | 32 | default 1 # microseconds |
| 23 | help | 33 | help |
| 24 | Configures the default min timeslice duration between multiple | 34 | Absolute lower bound (in microseconds) for the timeslice duration. |
| 25 | contexts by guc scheduling. | 35 | This caps both the sysfs timeslice_duration_us knob and the value |
| 36 | accepted via the DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE UAPI for | ||
| 37 | processes with CAP_SYS_NICE when DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT | ||
| 38 | is enabled. | ||
| 26 | config DRM_XE_PREEMPT_TIMEOUT | 39 | config DRM_XE_PREEMPT_TIMEOUT |
| 27 | int "Preempt timeout (us, jiffy granularity)" | 40 | int "Default preempt timeout (us, jiffy granularity)" |
| 28 | default 640000 # microseconds | 41 | default 640000 # microseconds |
| 29 | help | 42 | help |
| 30 | How long to wait (in microseconds) for a preemption event to occur | 43 | Initial per-engine-class preemption timeout (in microseconds). This |
| 31 | when submitting a new context. If the current context does not hit | 44 | is the value the driver programs at boot; it can be changed at |
| 32 | an arbitration point and yield to HW before the timer expires, the | 45 | runtime via the sysfs preempt_timeout_us knob. |
| 33 | HW will be reset to allow the more important context to execute. | 46 | |
| 47 | This is how long the driver waits for the current context to reach | ||
| 48 | an arbitration point and yield the GPU voluntarily when a | ||
| 49 | higher-priority context becomes runnable. If the context does not | ||
| 50 | yield before the timer expires, the HW is reset to allow the | ||
| 51 | higher-priority context to execute. | ||
| 52 | |||
| 53 | The range userspace may write via sysfs is bounded by | ||
| 54 | DRM_XE_PREEMPT_TIMEOUT_MIN and DRM_XE_PREEMPT_TIMEOUT_MAX. | ||
| 34 | config DRM_XE_PREEMPT_TIMEOUT_MAX | 55 | config DRM_XE_PREEMPT_TIMEOUT_MAX |
| 35 | int "Default max preempt timeout (us)" | 56 | int "Hard upper limit for preempt timeout (us)" |
| 36 | default 10000000 # microseconds | 57 | default 10000000 # microseconds |
| 37 | help | 58 | help |
| 38 | Configures the default max preempt timeout after which context | 59 | Absolute upper bound (in microseconds) for the per-engine-class |
| 39 | will be forcefully taken away and higher priority context will | 60 | preemption timeout. This is the maximum value that can be written to |
| 40 | run. | 61 | the sysfs preempt_timeout_us knob, regardless of privileges. |
| 41 | config DRM_XE_PREEMPT_TIMEOUT_MIN | 62 | config DRM_XE_PREEMPT_TIMEOUT_MIN |
| 42 | int "Default min preempt timeout (us)" | 63 | int "Hard lower limit for preempt timeout (us)" |
| 43 | default 1 # microseconds | 64 | default 1 # microseconds |
| 44 | help | 65 | help |
| 45 | Configures the default min preempt timeout after which context | 66 | Absolute lower bound (in microseconds) for the per-engine-class |
| 46 | will be forcefully taken away and higher priority context will | 67 | preemption timeout. This is the minimum value that can be written to |
| 47 | run. | 68 | the sysfs preempt_timeout_us knob, regardless of privileges. |
| 48 | config DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT | 69 | config DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT |
| 49 | bool "Default configuration of limitation on scheduler timeout" | 70 | bool "Default configuration of limitation on scheduler timeout" |
| 50 | default y | 71 | default y |
| @@ -101,6 +101,7 @@ xe-y += xe_bb.o \ · 100 unmodified lines | |||
| 101 | xe_page_reclaim.o \ | 101 | xe_page_reclaim.o \ |
| 102 | xe_pat.o \ | 102 | xe_pat.o \ |
| 103 | xe_pci.o \ | 103 | xe_pci.o \ |
| 104 | xe_pci_error.o \ | ||
| 104 | xe_pci_rebar.o \ | 105 | xe_pci_rebar.o \ |
| 105 | xe_pcode.o \ | 106 | xe_pcode.o \ |
| 106 | xe_pm.o \ | 107 | xe_pm.o \ |
| @@ -9,5 +9,6 @@ obj-$(CONFIG_DRM_XE_KUNIT_TEST) += xe_test.o · 8 unmodified lines | |||
| 9 | xe_test-y = xe_test_mod.o \ | 9 | xe_test-y = xe_test_mod.o \ |
| 10 | xe_args_test.o \ | 10 | xe_args_test.o \ |
| 11 | xe_pci_test.o \ | 11 | xe_pci_test.o \ |
| 12 | xe_rtp_tables_test.o \ | ||
| 12 | xe_rtp_test.o \ | 13 | xe_rtp_test.o \ |
| 13 | xe_wa_test.o | 14 | xe_wa_test.o |
| @@ -311,40 +311,44 @@ const void *xe_pci_id_gen_param(struct kunit *test, const void *prev, char *desc · 310 unmodified lines | |||
| 311 | } | 311 | } |
| 312 | EXPORT_SYMBOL_IF_KUNIT(xe_pci_id_gen_param); | 312 | EXPORT_SYMBOL_IF_KUNIT(xe_pci_id_gen_param); |
| 313 | 313 | ||
| 314 | static void fake_init_devid(struct xe_device *xe) | 314 | static int fake_probe_info(struct xe_device *xe, |
| 315 | const struct xe_device_desc *desc, | ||
| 316 | struct xe_pci_fake_data *data, | ||
| 317 | struct xe_probed_info *probed_info) | ||
| 315 | { | 318 | { |
| 316 | /* Nothing to do, just keep zero. */ | 319 | probed_info->tile_count = 1 + desc->max_remote_tiles; |
| 317 | } | ||
| 318 | |||
| 319 | static int fake_read_gmdid(struct xe_device *xe, enum xe_gmdid_type type, | ||
| 320 | u32 *ver, u32 *revid) | ||
| 321 | { | ||
| 322 | struct kunit *test = kunit_get_current_test(); | ||
| 323 | struct xe_pci_fake_data *data = test->priv; | ||
| 324 | 320 | ||
| 325 | if (type == GMDID_MEDIA) { | 321 | if (!data || desc->pre_gmdid_graphics_ip) { |
| 326 | *ver = data->media_verx100; | 322 | probed_info->graphics_ip = desc->pre_gmdid_graphics_ip; |
| 327 | *revid = xe_step_to_gmdid(data->step.media); | 323 | probed_info->media_ip = desc->pre_gmdid_media_ip; |
| 328 | } else { | 324 | } else { |
| 329 | *ver = data->graphics_verx100; | 325 | probed_info->graphics_ip = find_graphics_ip(data->graphics_verx100); |
| 330 | *revid = xe_step_to_gmdid(data->step.graphics); | 326 | |
| 327 | if (data->media_verx100) { | ||
| 328 | probed_info->media_ip = find_media_ip(data->media_verx100); | ||
| 329 | xe_assert(xe, probed_info->media_ip); | ||
| 330 | } | ||
| 331 | } | 331 | } |
| 332 | 332 | ||
| 333 | return 0; | 333 | xe_assert(xe, probed_info->graphics_ip); |
| 334 | } | 334 | if (!probed_info->graphics_ip) |
| 335 | return -ENODEV; | ||
| 335 | 336 | ||
| 336 | static void fake_xe_info_probe_tile_count(struct xe_device *xe) | 337 | if (data) |
| 337 | { | 338 | probed_info->step = data->step; |
| 338 | /* Nothing to do, just use the statically defined value. */ | 339 | |
| 340 | return 0; | ||
| 339 | } | 341 | } |
| 340 | 342 | ||
| 341 | int xe_pci_fake_device_init(struct xe_device *xe) | 343 | int xe_pci_fake_device_init(struct xe_device *xe) |
| 342 | { | 344 | { |
| 343 | struct kunit *test = kunit_get_current_test(); | 345 | struct kunit *test = kunit_get_current_test(); |
| 344 | struct xe_pci_fake_data *data = test->priv; | 346 | struct xe_pci_fake_data *data = test->priv; |
| 347 | struct xe_probed_info probed_info = {}; | ||
| 345 | const struct pci_device_id *ent = pciidlist; | 348 | const struct pci_device_id *ent = pciidlist; |
| 346 | const struct xe_device_desc *desc; | 349 | const struct xe_device_desc *desc; |
| 347 | const struct xe_subplatform_desc *subplatform_desc; | 350 | const struct xe_subplatform_desc *subplatform_desc; |
| 351 | int err; | ||
| 348 | 352 | ||
| 349 | if (!data) { | 353 | if (!data) { |
| 350 | desc = (const void *)ent->driver_data; | 354 | desc = (const void *)ent->driver_data; |
| @@ -374,13 +378,12 @@ done: · 23 unmodified lines | |||
| 374 | xe->sriov.__mode = data && data->sriov_mode ? | 378 | xe->sriov.__mode = data && data->sriov_mode ? |
| 375 | data->sriov_mode : XE_SRIOV_MODE_NONE; | 379 | data->sriov_mode : XE_SRIOV_MODE_NONE; |
| 376 | 380 | ||
| 377 | kunit_activate_static_stub(test, init_devid, fake_init_devid); | 381 | err = fake_probe_info(xe, desc, data, &probed_info); |
| 378 | kunit_activate_static_stub(test, read_gmdid, fake_read_gmdid); | 382 | if (err) |
| 379 | kunit_activate_static_stub(test, xe_info_probe_tile_count, | 383 | return err; |
| 380 | fake_xe_info_probe_tile_count); | ||
| 381 | 384 | ||
| 382 | xe_info_init_early(xe, desc, subplatform_desc); | 385 | xe_info_init_early(xe, desc, subplatform_desc, &probed_info); |
| 383 | xe_info_init(xe, desc); | 386 | xe_info_init(xe, desc, &probed_info); |
| 384 | 387 | ||
| 385 | return 0; | 388 | return 0; |
| 386 | } | 389 | } |
| @@ -0,0 +1,112 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0 | ||
| 2 | /* | ||
| 3 | * Copyright © 2026 Intel Corporation | ||
| 4 | */ | ||
| 5 | |||
| 6 | #include <kunit/test.h> | ||
| 7 | |||
| 8 | #include "xe_reg_whitelist.h" | ||
| 9 | #include "xe_rtp_types.h" | ||
| 10 | #include "xe_tuning.h" | ||
| 11 | #include "xe_wa.h" | ||
| 12 | |||
| 13 | #define RTP_TABLE_PARAM(table) \ | ||
| 14 | static const void *table##_gen_params(struct kunit *test, \ | ||
| 15 | const void *prev, char *desc) \ | ||
| 16 | { \ | ||
| 17 | typeof((table.entries)[0]) *__next = prev ? \ | ||
| 18 | ((typeof(__next))prev) + 1 : (table.entries); \ | ||
| 19 | if (__next - table.entries < table.n_entries) { \ | ||
| 20 | scnprintf(desc, KUNIT_PARAM_DESC_SIZE, #table "/%s", __next->name); \ | ||
| 21 | return __next; \ | ||
| 22 | } \ | ||
| 23 | return NULL; \ | ||
| 24 | } | ||
| 25 | |||
| 26 | static void xe_rtp_table_gt_test(struct kunit *test) | ||
| 27 | { | ||
| 28 | const struct xe_rtp_entry_sr *entry = test->param_value; | ||
| 29 | |||
| 30 | for (int i = 0; i < entry->n_rules; i++) { | ||
| 31 | KUNIT_EXPECT_TRUE(test, | ||
| 32 | entry->rules[i].match_type != XE_RTP_MATCH_ENGINE_CLASS || | ||
| 33 | entry->flags & XE_RTP_ENTRY_FLAG_FOREACH_ENGINE); | ||
| 34 | KUNIT_EXPECT_TRUE(test, | ||
| 35 | entry->rules[i].match_type != XE_RTP_MATCH_NOT_ENGINE_CLASS || | ||
| 36 | entry->flags & XE_RTP_ENTRY_FLAG_FOREACH_ENGINE); | ||
| 37 | } | ||
| 38 | } | ||
| 39 | |||
| 40 | RTP_TABLE_PARAM(gt_was); | ||
| 41 | RTP_TABLE_PARAM(gt_tunings); | ||
| 42 | |||
| 43 | static void xe_rtp_table_oob_test(struct kunit *test) | ||
| 44 | { | ||
| 45 | const struct xe_rtp_entry *entry = test->param_value; | ||
| 46 | |||
| 47 | for (int i = 0; i < entry->n_rules; i++) { | ||
| 48 | u8 match_type = entry->rules[i].match_type; | ||
| 49 | |||
| 50 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_ENGINE_CLASS); | ||
| 51 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_NOT_ENGINE_CLASS); | ||
| 52 | } | ||
| 53 | } | ||
| 54 | |||
| 55 | RTP_TABLE_PARAM(oob_was); | ||
| 56 | |||
| 57 | static void xe_rtp_table_dev_oob_test(struct kunit *test) | ||
| 58 | { | ||
| 59 | const struct xe_rtp_entry *entry = test->param_value; | ||
| 60 | |||
| 61 | for (int i = 0; i < entry->n_rules; i++) { | ||
| 62 | u8 match_type = entry->rules[i].match_type; | ||
| 63 | |||
| 64 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_ENGINE_CLASS); | ||
| 65 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_NOT_ENGINE_CLASS); | ||
| 66 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_GRAPHICS_VERSION); | ||
| 67 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_GRAPHICS_VERSION_RANGE); | ||
| 68 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_GRAPHICS_VERSION_ANY_GT); | ||
| 69 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_GRAPHICS_STEP); | ||
| 70 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_MEDIA_VERSION); | ||
| 71 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_MEDIA_VERSION_RANGE); | ||
| 72 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_MEDIA_VERSION_ANY_GT); | ||
| 73 | KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_MEDIA_STEP); | ||
| 74 | } | ||
| 75 | } | ||
| 76 | |||
| 77 | RTP_TABLE_PARAM(device_oob_was); | ||
| 78 | |||
| 79 | static void xe_rtp_table_missing_upper_bound_test(struct kunit *test) | ||
| 80 | { | ||
| 81 | const struct xe_rtp_entry_sr *entry = test->param_value; | ||
| 82 | |||
| 83 | for (int i = 0; i < entry->n_rules; i++) { | ||
| 84 | u8 match_type = entry->rules[i].match_type; | ||
| 85 | |||
| 86 | KUNIT_EXPECT_FALSE(test, | ||
| 87 | match_type == XE_RTP_MATCH_GRAPHICS_VERSION_RANGE && | ||
| 88 | entry->rules[i].ver_end == XE_RTP_END_VERSION_UNDEFINED); | ||
| 89 | KUNIT_EXPECT_FALSE(test, | ||
| 90 | match_type == XE_RTP_MATCH_MEDIA_VERSION_RANGE && | ||
| 91 | entry->rules[i].ver_end == XE_RTP_END_VERSION_UNDEFINED); | ||
| 92 | } | ||
| 93 | } | ||
| 94 | |||
| 95 | RTP_TABLE_PARAM(register_whitelist); | ||
| 96 | |||
| 97 | static struct kunit_case xe_rtp_table_tests[] = { | ||
| 98 | KUNIT_CASE_PARAM(xe_rtp_table_gt_test, gt_was_gen_params), | ||
| 99 | KUNIT_CASE_PARAM(xe_rtp_table_gt_test, gt_tunings_gen_params), | ||
| 100 | KUNIT_CASE_PARAM(xe_rtp_table_oob_test, oob_was_gen_params), | ||
| 101 | KUNIT_CASE_PARAM(xe_rtp_table_dev_oob_test, device_oob_was_gen_params), | ||
| 102 | KUNIT_CASE_PARAM(xe_rtp_table_missing_upper_bound_test, | ||
| 103 | register_whitelist_gen_params), | ||
| 104 | {} | ||
| 105 | }; | ||
| 106 | |||
| 107 | static struct kunit_suite xe_rtp_tables_test_suite = { | ||
| 108 | .name = "xe_rtp_tables_test", | ||
| 109 | .test_cases = xe_rtp_table_tests, | ||
| 110 | }; | ||
| 111 | |||
| 112 | kunit_test_suite(xe_rtp_tables_test_suite); | ||
| @@ -280,6 +280,11 @@ static void xe_rtp_rules_tests(struct kunit *test) · 279 unmodified lines | |||
| 280 | KUNIT_EXPECT_EQ(test, err, param->expected_err); | 280 | KUNIT_EXPECT_EQ(test, err, param->expected_err); |
| 281 | } | 281 | } |
| 282 | 282 | ||
| 283 | static u32 bits_2_3_set(struct xe_gt *gt, struct xe_hw_engine *hwe) | ||
| 284 | { | ||
| 285 | return REG_BIT(2) | REG_BIT(3); | ||
| 286 | } | ||
| 287 | |||
| 283 | static const struct rtp_to_sr_test_case rtp_to_sr_cases[] = { | 288 | static const struct rtp_to_sr_test_case rtp_to_sr_cases[] = { |
| 284 | { | 289 | { |
| 285 | .name = "coalesce-same-reg", | 290 | .name = "coalesce-same-reg", |
| @@ -300,6 +305,29 @@ static const struct rtp_to_sr_test_case rtp_to_sr_cases[] = { · 14 unmodified lines | |||
| 300 | }, | 305 | }, |
| 301 | ), | 306 | ), |
| 302 | }, | 307 | }, |
| 308 | { | ||
| 309 | .name = "coalesce-same-reg-literal-and-func", | ||
| 310 | .expected_reg = REGULAR_REG1, | ||
| 311 | .expected_set_bits = REG_BIT(0) | REG_BIT(1) | REG_BIT(2) | REG_BIT(3), | ||
| 312 | .expected_clr_bits = REG_BIT(0) | REG_BIT(1) | REG_BIT(2) | REG_BIT(3), | ||
| 313 | .expected_active = BIT(0) | BIT(1), | ||
| 314 | .expected_count_sr_entries = 1, | ||
| 315 | /* Different bits on the same register: create a single entry */ | ||
| 316 | .table = XE_RTP_TABLE_SR( | ||
| 317 | { XE_RTP_NAME("basic-1"), | ||
| 318 | XE_RTP_RULES(FUNC(match_yes)), | ||
| 319 | XE_RTP_ACTIONS(FIELD_SET(REGULAR_REG1, | ||
| 320 | REG_BIT(0) | REG_BIT(1), | ||
| 321 | REG_BIT(0) | REG_BIT(1))) | ||
| 322 | }, | ||
| 323 | { XE_RTP_NAME("basic-2"), | ||
| 324 | XE_RTP_RULES(FUNC(match_yes)), | ||
| 325 | XE_RTP_ACTIONS(FIELD_SET_FUNC(REGULAR_REG1, | ||
| 326 | REG_BIT(2) | REG_BIT(3), | ||
| 327 | bits_2_3_set)) | ||
| 328 | }, | ||
| 329 | ), | ||
| 330 | }, | ||
| 303 | { | 331 | { |
| 304 | .name = "no-match-no-add", | 332 | .name = "no-match-no-add", |
| 305 | .expected_reg = REGULAR_REG1, | 333 | .expected_reg = REGULAR_REG1, |
| @@ -417,6 +445,30 @@ static const struct rtp_to_sr_test_case rtp_to_sr_cases[] = { · 111 unmodified lines | |||
| 417 | }, | 445 | }, |
| 418 | ), | 446 | ), |
| 419 | }, | 447 | }, |
| 448 | { | ||
| 449 | .name = "conflict-not-disjoint-literal-and-func", | ||
| 450 | .expected_reg = REGULAR_REG1, | ||
| 451 | .expected_set_bits = REG_BIT(1) | REG_BIT(2), | ||
| 452 | .expected_clr_bits = REG_BIT(1) | REG_BIT(2), | ||
| 453 | .expected_active = BIT(0) | BIT(1), | ||
| 454 | .expected_count_sr_entries = 1, | ||
| 455 | .expected_sr_errors = 1, | ||
| 456 | .table = XE_RTP_TABLE_SR( | ||
| 457 | { XE_RTP_NAME("basic-1"), | ||
| 458 | XE_RTP_RULES(FUNC(match_yes)), | ||
| 459 | XE_RTP_ACTIONS(FIELD_SET(REGULAR_REG1, | ||
| 460 | REG_BIT(1) | REG_BIT(2), | ||
| 461 | REG_BIT(1) | REG_BIT(2))) | ||
| 462 | }, | ||
| 463 | /* drop: bits are not disjoint with previous entries */ | ||
| 464 | { XE_RTP_NAME("basic-2"), | ||
| 465 | XE_RTP_RULES(FUNC(match_yes)), | ||
| 466 | XE_RTP_ACTIONS(FIELD_SET_FUNC(REGULAR_REG1, | ||
| 467 | REG_BIT(2) | REG_BIT(3), | ||
| 468 | bits_2_3_set)) | ||
| 469 | }, | ||
| 470 | ), | ||
| 471 | }, | ||
| 420 | { | 472 | { |
| 421 | .name = "conflict-reg-type", | 473 | .name = "conflict-reg-type", |
| 422 | .expected_reg = REGULAR_REG1, | 474 | .expected_reg = REGULAR_REG1, |
| @@ -43,9 +43,6 @@ static int xe_wa_test_init(struct kunit *test) · 42 unmodified lines | |||
| 43 | xe_gt_mmio_init(gt); | 43 | xe_gt_mmio_init(gt); |
| 44 | } | 44 | } |
| 45 | 45 | ||
| 46 | if (!param->graphics_verx100) | ||
| 47 | xe->info.step = param->step; | ||
| 48 | |||
| 49 | /* TODO: init hw engines for engine/LRC WAs */ | 46 | /* TODO: init hw engines for engine/LRC WAs */ |
| 50 | xe->drm.dev = dev; | 47 | xe->drm.dev = dev; |
| 51 | test->priv = xe; | 48 | test->priv = xe; |
| @@ -117,7 +117,6 @@ static int info(struct seq_file *m, void *data) · 116 unmodified lines | |||
| 117 | drm_printf(&p, "revid %d\n", xe->info.revid); | 117 | drm_printf(&p, "revid %d\n", xe->info.revid); |
| 118 | drm_printf(&p, "tile_count %d\n", xe->info.tile_count); | 118 | drm_printf(&p, "tile_count %d\n", xe->info.tile_count); |
| 119 | drm_printf(&p, "vm_max_level %d\n", xe->info.vm_max_level); | 119 | drm_printf(&p, "vm_max_level %d\n", xe->info.vm_max_level); |
| 120 | drm_printf(&p, "force_execlist %s\n", str_yes_no(xe->info.force_execlist)); | ||
| 121 | drm_printf(&p, "has_flat_ccs %s\n", str_yes_no(xe->info.has_flat_ccs)); | 120 | drm_printf(&p, "has_flat_ccs %s\n", str_yes_no(xe->info.has_flat_ccs)); |
| 122 | drm_printf(&p, "has_usm %s\n", str_yes_no(xe->info.has_usm)); | 121 | drm_printf(&p, "has_usm %s\n", str_yes_no(xe->info.has_usm)); |
| 123 | drm_printf(&p, "skip_guc_pc %s\n", str_yes_no(xe->info.skip_guc_pc)); | 122 | drm_printf(&p, "skip_guc_pc %s\n", str_yes_no(xe->info.skip_guc_pc)); |
| @@ -61,6 +61,7 @@ · 60 unmodified lines | |||
| 61 | #include "xe_psmi.h" | 61 | #include "xe_psmi.h" |
| 62 | #include "xe_pxp.h" | 62 | #include "xe_pxp.h" |
| 63 | #include "xe_query.h" | 63 | #include "xe_query.h" |
| 64 | #include "xe_ras.h" | ||
| 64 | #include "xe_shrinker.h" | 65 | #include "xe_shrinker.h" |
| 65 | #include "xe_soc_remapper.h" | 66 | #include "xe_soc_remapper.h" |
| 66 | #include "xe_survivability_mode.h" | 67 | #include "xe_survivability_mode.h" |
| @@ -738,9 +739,11 @@ static void vf_update_device_info(struct xe_device *xe) · 671 unmodified lines | |||
| 738 | xe->info.probe_display = 0; | 739 | xe->info.probe_display = 0; |
| 739 | xe->info.has_heci_cscfi = 0; | 740 | xe->info.has_heci_cscfi = 0; |
| 740 | xe->info.has_heci_gscfi = 0; | 741 | xe->info.has_heci_gscfi = 0; |
| 742 | xe->info.has_i2c = 0; | ||
| 741 | xe->info.has_late_bind = 0; | 743 | xe->info.has_late_bind = 0; |
| 742 | xe->info.skip_guc_pc = 1; | 744 | xe->info.skip_guc_pc = 1; |
| 743 | xe->info.skip_pcode = 1; | 745 | xe->info.skip_pcode = 1; |
| 746 | xe->info.has_drm_ras = false; | ||
| 744 | } | 747 | } |
| 745 | 748 | ||
| 746 | static int xe_device_vram_alloc(struct xe_device *xe) | 749 | static int xe_device_vram_alloc(struct xe_device *xe) |
| @@ -949,6 +952,15 @@ int xe_device_probe(struct xe_device *xe) · 202 unmodified lines | |||
| 949 | return err; | 952 | return err; |
| 950 | } | 953 | } |
| 951 | 954 | ||
| 955 | /* | ||
| 956 | * Wa_16029380221: The affected GT will always use non-coherent | ||
| 957 | * access to page tables, so we must do uncached writes from the | ||
| 958 | * CPU. | ||
| 959 | */ | ||
| 960 | for_each_gt(gt, xe, id) | ||
| 961 | if (XE_GT_WA(gt, 16029380221)) | ||
| 962 | xe->info.has_cached_pt = false; | ||
| 963 | |||
| 952 | for_each_tile(tile, xe, id) { | 964 | for_each_tile(tile, xe, id) { |
| 953 | err = xe_ggtt_init_early(tile->mem.ggtt); | 965 | err = xe_ggtt_init_early(tile->mem.ggtt); |
| 954 | if (err) | 966 | if (err) |
| @@ -989,6 +1001,16 @@ int xe_device_probe(struct xe_device *xe) · 34 unmodified lines | |||
| 989 | if (err) | 1001 | if (err) |
| 990 | return err; | 1002 | return err; |
| 991 | 1003 | ||
| 1004 | err = xe_soc_remapper_init(xe); | ||
| 1005 | if (err) | ||
| 1006 | return err; | ||
| 1007 | |||
| 1008 | err = xe_sysctrl_init(xe); | ||
| 1009 | if (err) | ||
| 1010 | return err; | ||
| 1011 | |||
| 1012 | xe_ras_init(xe); | ||
| 1013 | |||
| 992 | /* | 1014 | /* |
| 993 | * Now that GT is initialized (TTM in particular), | 1015 | * Now that GT is initialized (TTM in particular), |
| 994 | * we can try to init display, and inherit the initial fb. | 1016 | * we can try to init display, and inherit the initial fb. |
| @@ -1029,10 +1051,6 @@ int xe_device_probe(struct xe_device *xe) · 34 unmodified lines | |||
| 1029 | 1051 | ||
| 1030 | xe_nvm_init(xe); | 1052 | xe_nvm_init(xe); |
| 1031 | 1053 | ||
| 1032 | err = xe_soc_remapper_init(xe); | ||
| 1033 | if (err) | ||
| 1034 | return err; | ||
| 1035 | |||
| 1036 | err = xe_heci_gsc_init(xe); | 1054 | err = xe_heci_gsc_init(xe); |
| 1037 | if (err) | 1055 | if (err) |
| 1038 | return err; | 1056 | return err; |
| @@ -1071,10 +1089,6 @@ int xe_device_probe(struct xe_device *xe) · 32 unmodified lines | |||
| 1071 | if (err) | 1089 | if (err) |
| 1072 | goto err_unregister_display; | 1090 | goto err_unregister_display; |
| 1073 | 1091 | ||
| 1074 | err = xe_sysctrl_init(xe); | ||
| 1075 | if (err) | ||
| 1076 | goto err_unregister_display; | ||
| 1077 | |||
| 1078 | err = xe_device_sysfs_init(xe); | 1092 | err = xe_device_sysfs_init(xe); |
| 1079 | if (err) | 1093 | if (err) |
| 1080 | goto err_unregister_display; | 1094 | goto err_unregister_display; |
| @@ -116,7 +116,7 @@ static inline struct xe_mmio *xe_root_tile_mmio(struct xe_device *xe) · 115 unmodified lines | |||
| 116 | 116 | ||
| 117 | static inline bool xe_device_uc_enabled(struct xe_device *xe) | 117 | static inline bool xe_device_uc_enabled(struct xe_device *xe) |
| 118 | { | 118 | { |
| 119 | return !xe->info.force_execlist; | 119 | return true; |
| 120 | } | 120 | } |
| 121 | 121 | ||
| 122 | #define for_each_tile(tile__, xe__, id__) \ | 122 | #define for_each_tile(tile__, xe__, id__) \ |
| @@ -181,6 +181,21 @@ static inline bool xe_device_has_mert(const struct xe_device *xe) · 58 unmodified lines | |||
| 181 | return xe->info.has_mert; | 181 | return xe->info.has_mert; |
| 182 | } | 182 | } |
| 183 | 183 | ||
| 184 | static inline bool xe_device_is_in_reset(struct xe_device *xe) | ||
| 185 | { | ||
| 186 | return atomic_read(&xe->in_reset); | ||
| 187 | } | ||
| 188 | |||
| 189 | static inline void xe_device_set_in_reset(struct xe_device *xe) | ||
| 190 | { | ||
| 191 | atomic_set(&xe->in_reset, 1); | ||
| 192 | } | ||
| 193 | |||
| 194 | static inline void xe_device_clear_in_reset(struct xe_device *xe) | ||
| 195 | { | ||
| 196 | atomic_set(&xe->in_reset, 0); | ||
| 197 | } | ||
| 198 | |||
| 184 | u32 xe_device_ccs_bytes(struct xe_device *xe, u64 size); | 199 | u32 xe_device_ccs_bytes(struct xe_device *xe, u64 size); |
| 185 | 200 | ||
| 186 | void xe_device_snapshot_print(struct xe_device *xe, struct drm_printer *p); | 201 | void xe_device_snapshot_print(struct xe_device *xe, struct drm_printer *p); |
| @@ -144,8 +144,6 @@ struct xe_device { · 143 unmodified lines | |||
| 144 | * Keep all flags below alphabetically sorted | 144 | * Keep all flags below alphabetically sorted |
| 145 | */ | 145 | */ |
| 146 | 146 | ||
| 147 | /** @info.force_execlist: Forced execlist submission */ | ||
| 148 | u8 force_execlist:1; | ||
| 149 | /** @info.has_access_counter: Device supports access counter */ | 147 | /** @info.has_access_counter: Device supports access counter */ |
| 150 | u8 has_access_counter:1; | 148 | u8 has_access_counter:1; |
| 151 | /** @info.has_asid: Has address space ID */ | 149 | /** @info.has_asid: Has address space ID */ |
| @@ -156,6 +154,8 @@ struct xe_device { · 4 unmodified lines | |||
| 156 | u8 has_cached_pt:1; | 154 | u8 has_cached_pt:1; |
| 157 | /** @info.has_device_atomics_on_smem: Supports device atomics on SMEM */ | 155 | /** @info.has_device_atomics_on_smem: Supports device atomics on SMEM */ |
| 158 | u8 has_device_atomics_on_smem:1; | 156 | u8 has_device_atomics_on_smem:1; |
| 157 | /** @info.has_drm_ras: Device supports drm_ras (Reliability, Availability, Serviceability) */ | ||
| 158 | u8 has_drm_ras:1; | ||
| 159 | /** @info.has_fan_control: Device supports fan control */ | 159 | /** @info.has_fan_control: Device supports fan control */ |
| 160 | u8 has_fan_control:1; | 160 | u8 has_fan_control:1; |
| 161 | /** @info.has_flat_ccs: Whether flat CCS metadata is used */ | 161 | /** @info.has_flat_ccs: Whether flat CCS metadata is used */ |
| @@ -483,6 +483,9 @@ struct xe_device { · 321 unmodified lines | |||
| 483 | /** @needs_flr_on_fini: requests function-reset on fini */ | 483 | /** @needs_flr_on_fini: requests function-reset on fini */ |
| 484 | bool needs_flr_on_fini; | 484 | bool needs_flr_on_fini; |
| 485 | 485 | ||
| 486 | /** @in_reset: Indicates if device is in reset */ | ||
| 487 | atomic_t in_reset; | ||
| 488 | |||
| 486 | /** @wedged: Struct to control Wedged States and mode */ | 489 | /** @wedged: Struct to control Wedged States and mode */ |
| 487 | struct { | 490 | struct { |
| 488 | /** @wedged.flag: Xe device faced a critical error and is now blocked. */ | 491 | /** @wedged.flag: Xe device faced a critical error and is now blocked. */ |
| @@ -495,6 +498,9 @@ struct xe_device { · 6 unmodified lines | |||
| 495 | bool inconsistent_reset; | 498 | bool inconsistent_reset; |
| 496 | } wedged; | 499 | } wedged; |
| 497 | 500 | ||
| 501 | /** @devres_group: devres group */ | ||
| 502 | void *devres_group; | ||
| 503 | |||
| 498 | /** @bo_device: Struct to control async free of BOs */ | 504 | /** @bo_device: Struct to control async free of BOs */ |
| 499 | struct xe_bo_dev { | 505 | struct xe_bo_dev { |
| 500 | /** @bo_device.async_free: Free worker */ | 506 | /** @bo_device.async_free: Free worker */ |
| @@ -193,7 +193,7 @@ static void bo_meminfo(struct xe_bo *bo, · 192 unmodified lines | |||
| 193 | if (!dma_resv_test_signaled(bo->ttm.base.resv, | 193 | if (!dma_resv_test_signaled(bo->ttm.base.resv, |
| 194 | DMA_RESV_USAGE_BOOKKEEP)) | 194 | DMA_RESV_USAGE_BOOKKEEP)) |
| 195 | stats[mem_type].active += sz; | 195 | stats[mem_type].active += sz; |
| 196 | else if (mem_type == XE_PL_SYSTEM) | 196 | else if (mem_type == XE_PL_SYSTEM || xe_bo_madv_is_dontneed(bo)) |
| 197 | stats[mem_type].purgeable += sz; | 197 | stats[mem_type].purgeable += sz; |
| 198 | } | 198 | } |
| 199 | } | 199 | } |
| @@ -273,8 +273,7 @@ static void show_meminfo(struct drm_printer *p, struct drm_file *file) · 73 unmodified lines | |||
| 273 | &stats[mem_type], | 273 | &stats[mem_type], |
| 274 | DRM_GEM_OBJECT_ACTIVE | | 274 | DRM_GEM_OBJECT_ACTIVE | |
| 275 | DRM_GEM_OBJECT_RESIDENT | | 275 | DRM_GEM_OBJECT_RESIDENT | |
| 276 | (mem_type != XE_PL_SYSTEM ? 0 : | 276 | DRM_GEM_OBJECT_PURGEABLE, |
| 277 | DRM_GEM_OBJECT_PURGEABLE), | ||
| 278 | xe_mem_type_to_name[mem_type]); | 277 | xe_mem_type_to_name[mem_type]); |
| 279 | } | 278 | } |
| 280 | } | 279 | } |
| @@ -11,27 +11,46 @@ · 10 unmodified lines | |||
| 11 | 11 | ||
| 12 | #include "xe_device_types.h" | 12 | #include "xe_device_types.h" |
| 13 | #include "xe_drm_ras.h" | 13 | #include "xe_drm_ras.h" |
| 14 | #include "xe_ras.h" | ||
| 14 | 15 | ||
| 15 | static const char * const error_components[] = DRM_XE_RAS_ERROR_COMPONENT_NAMES; | 16 | static const char * const error_components[] = DRM_XE_RAS_ERROR_COMPONENT_NAMES; |
| 16 | static const char * const error_severity[] = DRM_XE_RAS_ERROR_SEVERITY_NAMES; | 17 | static const char * const error_severity[] = DRM_XE_RAS_ERROR_SEVERITY_NAMES; |
| 17 | 18 | ||
| 18 | static int hw_query_error_counter(struct xe_drm_ras_counter *info, | 19 | static int query_error_counter(struct xe_device *xe, |
| 19 | u32 error_id, const char **name, u32 *val) | 20 | enum drm_xe_ras_error_severity severity, |
| 21 | u32 error_id, const char **name, u32 *val) | ||
| 20 | { | 22 | { |
| 23 | struct xe_drm_ras *ras = &xe->ras; | ||
| 24 | struct xe_drm_ras_counter *info = ras->info[severity]; | ||
| 25 | |||
| 21 | if (!info || !info[error_id].name) | 26 | if (!info || !info[error_id].name) |
| 22 | return -ENOENT; | 27 | return -ENOENT; |
| 23 | 28 | ||
| 24 | *name = info[error_id].name; | 29 | *name = info[error_id].name; |
| 30 | |||
| 31 | /* Fetch counter from system controller if supported */ | ||
| 32 | if (xe->info.has_sysctrl) | ||
| 33 | return xe_ras_get_counter(xe, severity, error_id, val); | ||
| 34 | |||
| 25 | *val = atomic_read(&info[error_id].counter); | 35 | *val = atomic_read(&info[error_id].counter); |
| 26 | 36 | ||
| 27 | return 0; | 37 | return 0; |
| 28 | } | 38 | } |
| 29 | 39 | ||
| 30 | static int hw_clear_error_counter(struct xe_drm_ras_counter *info, u32 error_id) | 40 | static int clear_error_counter(struct xe_device *xe, |
| 41 | enum drm_xe_ras_error_severity severity, | ||
| 42 | u32 error_id) | ||
| 31 | { | 43 | { |
| 44 | struct xe_drm_ras *ras = &xe->ras; | ||
| 45 | struct xe_drm_ras_counter *info = ras->info[severity]; | ||
| 46 | |||
| 32 | if (!info || !info[error_id].name) | 47 | if (!info || !info[error_id].name) |
| 33 | return -ENOENT; | 48 | return -ENOENT; |
| 34 | 49 | ||
| 50 | /* Clear counter from system controller if supported */ | ||
| 51 | if (xe->info.has_sysctrl) | ||
| 52 | return xe_ras_clear_counter(xe, severity, error_id); | ||
| 53 | |||
| 35 | atomic_set(&info[error_id].counter, 0); | 54 | atomic_set(&info[error_id].counter, 0); |
| 36 | 55 | ||
| 37 | return 0; | 56 | return 0; |
| @@ -41,38 +60,30 @@ static int query_uncorrectable_error_counter(struct drm_ras_node *ep, u32 error_ · 3 unmodified lines | |||
| 41 | const char **name, u32 *val) | 60 | const char **name, u32 *val) |
| 42 | { | 61 | { |
| 43 | struct xe_device *xe = ep->priv; | 62 | struct xe_device *xe = ep->priv; |
| 44 | struct xe_drm_ras *ras = &xe->ras; | ||
| 45 | struct xe_drm_ras_counter *info = ras->info[DRM_XE_RAS_ERR_SEV_UNCORRECTABLE]; | ||
| 46 | 63 | ||
| 47 | return hw_query_error_counter(info, error_id, name, val); | 64 | return query_error_counter(xe, DRM_XE_RAS_ERR_SEV_UNCORRECTABLE, error_id, name, val); |
| 48 | } | 65 | } |
| 49 | 66 | ||
| 50 | static int clear_uncorrectable_error_counter(struct drm_ras_node *node, u32 error_id) | 67 | static int clear_uncorrectable_error_counter(struct drm_ras_node *node, u32 error_id) |
| 51 | { | 68 | { |
| 52 | struct xe_device *xe = node->priv; | 69 | struct xe_device *xe = node->priv; |
| 53 | struct xe_drm_ras *ras = &xe->ras; | ||
| 54 | struct xe_drm_ras_counter *info = ras->info[DRM_XE_RAS_ERR_SEV_UNCORRECTABLE]; | ||
| 55 | 70 | ||
| 56 | return hw_clear_error_counter(info, error_id); | 71 | return clear_error_counter(xe, DRM_XE_RAS_ERR_SEV_UNCORRECTABLE, error_id); |
| 57 | } | 72 | } |
| 58 | 73 | ||
| 59 | static int query_correctable_error_counter(struct drm_ras_node *ep, u32 error_id, | 74 | static int query_correctable_error_counter(struct drm_ras_node *ep, u32 error_id, |
| 60 | const char **name, u32 *val) | 75 | const char **name, u32 *val) |
| 61 | { | 76 | { |
| 62 | struct xe_device *xe = ep->priv; | 77 | struct xe_device *xe = ep->priv; |
| 63 | struct xe_drm_ras *ras = &xe->ras; | ||
| 64 | struct xe_drm_ras_counter *info = ras->info[DRM_XE_RAS_ERR_SEV_CORRECTABLE]; | ||
| 65 | 78 | ||
| 66 | return hw_query_error_counter(info, error_id, name, val); | 79 | return query_error_counter(xe, DRM_XE_RAS_ERR_SEV_CORRECTABLE, error_id, name, val); |
| 67 | } | 80 | } |
| 68 | 81 | ||
| 69 | static int clear_correctable_error_counter(struct drm_ras_node *node, u32 error_id) | 82 | static int clear_correctable_error_counter(struct drm_ras_node *node, u32 error_id) |
| 70 | { | 83 | { |
| 71 | struct xe_device *xe = node->priv; | 84 | struct xe_device *xe = node->priv; |
| 72 | struct xe_drm_ras *ras = &xe->ras; | ||
| 73 | struct xe_drm_ras_counter *info = ras->info[DRM_XE_RAS_ERR_SEV_CORRECTABLE]; | ||
| 74 | 85 | ||
| 75 | return hw_clear_error_counter(info, error_id); | 86 | return clear_error_counter(xe, DRM_XE_RAS_ERR_SEV_CORRECTABLE, error_id); |
| 76 | } | 87 | } |
| 77 | 88 | ||
| 78 | static struct xe_drm_ras_counter *allocate_and_copy_counters(struct xe_device *xe) | 89 | static struct xe_drm_ras_counter *allocate_and_copy_counters(struct xe_device *xe) |
| @@ -985,9 +985,10 @@ int xe_eu_stall_stream_open(struct drm_device *dev, u64 data, struct drm_file *f · 984 unmodified lines | |||
| 985 | return -ENODEV; | 985 | return -ENODEV; |
| 986 | } | 986 | } |
| 987 | 987 | ||
| 988 | if (xe_observation_paranoid && !perfmon_capable()) { | 988 | ret = xe_observation_paranoid_check(); |
| 989 | if (ret) { | ||
| 989 | drm_dbg(&xe->drm, "Insufficient privileges for EU stall monitoring\n"); | 990 | drm_dbg(&xe->drm, "Insufficient privileges for EU stall monitoring\n"); |
| 990 | return -EACCES; | 991 | return ret; |
| 991 | } | 992 | } |
| 992 | 993 | ||
| 993 | /* Initialize and set default values */ | 994 | /* Initialize and set default values */ |
| @@ -318,8 +318,6 @@ struct xe_exec_queue_ops { · 317 unmodified lines | |||
| 318 | void (*resume)(struct xe_exec_queue *q); | 318 | void (*resume)(struct xe_exec_queue *q); |
| 319 | /** @reset_status: check exec queue reset status */ | 319 | /** @reset_status: check exec queue reset status */ |
| 320 | bool (*reset_status)(struct xe_exec_queue *q); | 320 | bool (*reset_status)(struct xe_exec_queue *q); |
| 321 | /** @active: check exec queue is active */ | ||
| 322 | bool (*active)(struct xe_exec_queue *q); | ||
| 323 | }; | 321 | }; |
| 324 | 322 | ||
| 325 | #endif | 323 | #endif |
| @@ -458,12 +458,6 @@ static bool execlist_exec_queue_reset_status(struct xe_exec_queue *q) · 457 unmodified lines | |||
| 458 | return false; | 458 | return false; |
| 459 | } | 459 | } |
| 460 | 460 | ||
| 461 | static bool execlist_exec_queue_active(struct xe_exec_queue *q) | ||
| 462 | { | ||
| 463 | /* NIY */ | ||
| 464 | return false; | ||
| 465 | } | ||
| 466 | |||
| 467 | static const struct xe_exec_queue_ops execlist_exec_queue_ops = { | 461 | static const struct xe_exec_queue_ops execlist_exec_queue_ops = { |
| 468 | .init = execlist_exec_queue_init, | 462 | .init = execlist_exec_queue_init, |
| 469 | .kill = execlist_exec_queue_kill, | 463 | .kill = execlist_exec_queue_kill, |
| @@ -476,7 +470,6 @@ static const struct xe_exec_queue_ops execlist_exec_queue_ops = { · 6 unmodified lines | |||
| 476 | .suspend_wait = execlist_exec_queue_suspend_wait, | 470 | .suspend_wait = execlist_exec_queue_suspend_wait, |
| 477 | .resume = execlist_exec_queue_resume, | 471 | .resume = execlist_exec_queue_resume, |
| 478 | .reset_status = execlist_exec_queue_reset_status, | 472 | .reset_status = execlist_exec_queue_reset_status, |
| 479 | .active = execlist_exec_queue_active, | ||
| 480 | }; | 473 | }; |
| 481 | 474 | ||
| 482 | int xe_execlist_init(struct xe_gt *gt) | 475 | int xe_execlist_init(struct xe_gt *gt) |
| @@ -111,14 +111,14 @@ struct xe_ggtt_pt_ops { · 110 unmodified lines | |||
| 111 | struct xe_ggtt { | 111 | struct xe_ggtt { |
| 112 | /** @tile: Back pointer to tile where this GGTT belongs */ | 112 | /** @tile: Back pointer to tile where this GGTT belongs */ |
| 113 | struct xe_tile *tile; | 113 | struct xe_tile *tile; |
| 114 | /** @start: Start offset of GGTT */ | 114 | /** @start: Start offset of GGTT */ |
| 115 | u64 start; | 115 | u64 start; |
| 116 | /** @size: Total usable size of this GGTT */ | 116 | /** @size: Total usable size of this GGTT */ |
| 117 | u64 size; | 117 | u64 size; |
| 118 | |||
| 119 | /** | 118 | /** |
| 120 | * @flags: Flags for this GGTT | 119 | * @flags: Flags for this GGTT. |
| 121 | * Acceptable flags: | 120 | * Acceptable flags: |
| 121 | * | ||
| 122 | * - %XE_GGTT_FLAGS_64K - if PTE size is 64K. Otherwise, regular is 4K. | 122 | * - %XE_GGTT_FLAGS_64K - if PTE size is 64K. Otherwise, regular is 4K. |
| 123 | * - %XE_GGTT_FLAGS_ONLINE - is GGTT online, protected by ggtt->lock | 123 | * - %XE_GGTT_FLAGS_ONLINE - is GGTT online, protected by ggtt->lock |
| 124 | * after init | 124 | * after init |
| @@ -129,7 +129,7 @@ struct xe_ggtt { · 4 unmodified lines | |||
| 129 | /** @lock: Mutex lock to protect GGTT data */ | 129 | /** @lock: Mutex lock to protect GGTT data */ |
| 130 | struct mutex lock; | 130 | struct mutex lock; |
| 131 | /** | 131 | /** |
| 132 | * @gsm: The iomem pointer to the actual location of the translation | 132 | * @gsm: The iomem pointer to the actual location of the translation |
| 133 | * table located in the GSM for easy PTE manipulation | 133 | * table located in the GSM for easy PTE manipulation |
| 134 | */ | 134 | */ |
| 135 | u64 __iomem *gsm; | 135 | u64 __iomem *gsm; |
| @@ -917,6 +917,9 @@ static void gt_reset_worker(struct work_struct *w) · 916 unmodified lines | |||
| 917 | if (xe_device_wedged(gt_to_xe(gt))) | 917 | if (xe_device_wedged(gt_to_xe(gt))) |
| 918 | goto err_pm_put; | 918 | goto err_pm_put; |
| 919 | 919 | ||
| 920 | if (xe_device_is_in_reset(gt_to_xe(gt))) | ||
| 921 | goto err_pm_put; | ||
| 922 | |||
| 920 | /* We only support GT resets with GuC submission */ | 923 | /* We only support GT resets with GuC submission */ |
| 921 | if (!xe_device_uc_enabled(gt_to_xe(gt))) | 924 | if (!xe_device_uc_enabled(gt_to_xe(gt))) |
| 922 | goto err_pm_put; | 925 | goto err_pm_put; |
| @@ -977,18 +980,21 @@ err_pm_put: · 54 unmodified lines | |||
| 977 | 980 | ||
| 978 | void xe_gt_reset_async(struct xe_gt *gt) | 981 | void xe_gt_reset_async(struct xe_gt *gt) |
| 979 | { | 982 | { |
| 980 | xe_gt_info(gt, "trying reset from %ps\n", __builtin_return_address(0)); | 983 | struct xe_device *xe = gt_to_xe(gt); |
| 984 | |||
| 985 | if (xe_device_is_in_reset(xe)) | ||
| 986 | return; | ||
| 981 | 987 | ||
| 982 | /* Don't do a reset while one is already in flight */ | 988 | /* Don't do a reset while one is already in flight */ |
| 983 | if (!xe_fault_inject_gt_reset() && xe_uc_reset_prepare(>->uc)) | 989 | if (!xe_fault_inject_gt_reset() && xe_uc_reset_prepare(>->uc)) |
| 984 | return; | 990 | return; |
| 985 | 991 | ||
| 986 | xe_gt_info(gt, "reset queued\n"); | 992 | xe_gt_info(gt, "reset queued from %ps\n", __builtin_return_address(0)); |
| 987 | 993 | ||
| 988 | /* Pair with put in gt_reset_worker() if work is enqueued */ | 994 | /* Pair with put in gt_reset_worker() if work is enqueued */ |
| 989 | xe_pm_runtime_get_noresume(gt_to_xe(gt)); | 995 | xe_pm_runtime_get_noresume(xe); |
| 990 | if (!queue_work(gt->ordered_wq, >->reset.worker)) | 996 | if (!queue_work(gt->ordered_wq, >->reset.worker)) |
| 991 | xe_pm_runtime_put(gt_to_xe(gt)); | 997 | xe_pm_runtime_put(xe); |
| 992 | } | 998 | } |
| 993 | 999 | ||
| 994 | void xe_gt_suspend_prepare(struct xe_gt *gt) | 1000 | void xe_gt_suspend_prepare(struct xe_gt *gt) |
| @@ -404,8 +404,7 @@ fallback: · 403 unmodified lines | |||
| 404 | * Some older platforms don't have tables or don't have complete tables. | 404 | * Some older platforms don't have tables or don't have complete tables. |
| 405 | * Newer platforms should always have the required info. | 405 | * Newer platforms should always have the required info. |
| 406 | */ | 406 | */ |
| 407 | if (GRAPHICS_VERx100(gt_to_xe(gt)) >= 2000 && | 407 | if (GRAPHICS_VERx100(gt_to_xe(gt)) >= 2000) |
| 408 | !gt_to_xe(gt)->info.force_execlist) | ||
| 409 | xe_gt_err(gt, "Slice/Subslice counts missing from hwconfig table; using typical fallback values\n"); | 408 | xe_gt_err(gt, "Slice/Subslice counts missing from hwconfig table; using typical fallback values\n"); |
| 410 | 409 | ||
| 411 | if (gt_to_xe(gt)->info.platform == XE_PVC) | 410 | if (gt_to_xe(gt)->info.platform == XE_PVC) |
| @@ -507,7 +506,7 @@ void xe_gt_mcr_init_early(struct xe_gt *gt) · 95 unmodified lines | |||
| 507 | spin_lock_init(>->mcr_lock); | 506 | spin_lock_init(>->mcr_lock); |
| 508 | 507 | ||
| 509 | if (gt->info.type == XE_GT_TYPE_MEDIA) { | 508 | if (gt->info.type == XE_GT_TYPE_MEDIA) { |
| 510 | drm_WARN_ON(&xe->drm, MEDIA_VER(xe) < 13); | 509 | xe_gt_WARN_ON(gt, MEDIA_VER(xe) < 13); |
| 511 | 510 | ||
| 512 | if (MEDIA_VER(xe) >= 30) { | 511 | if (MEDIA_VER(xe) >= 30) { |
| 513 | gt->steering[OADDRM].ranges = xe2lpm_gpmxmt_steering_table; | 512 | gt->steering[OADDRM].ranges = xe2lpm_gpmxmt_steering_table; |
| @@ -662,9 +661,9 @@ bool xe_gt_mcr_get_nonterminated_steering(struct xe_gt *gt, · 148 unmodified lines | |||
| 662 | 661 | ||
| 663 | for (int type = 0; type < IMPLICIT_STEERING; type++) { | 662 | for (int type = 0; type < IMPLICIT_STEERING; type++) { |
| 664 | if (reg_in_steering_type_ranges(gt, reg, type)) { | 663 | if (reg_in_steering_type_ranges(gt, reg, type)) { |
| 665 | drm_WARN(>_to_xe(gt)->drm, !gt->steering[type].initialized, | 664 | xe_gt_WARN(gt, !gt->steering[type].initialized, |
| 666 | "Uninitialized usage of MCR register %s/%#x\n", | 665 | "Uninitialized usage of MCR register %s/%#x\n", |
| 667 | xe_steering_types[type].name, reg.addr); | 666 | xe_steering_types[type].name, reg.addr); |
| 668 | 667 | ||
| 669 | *group = gt->steering[type].group_target; | 668 | *group = gt->steering[type].group_target; |
| 670 | *instance = gt->steering[type].instance_target; | 669 | *instance = gt->steering[type].instance_target; |
| @@ -679,9 +678,9 @@ bool xe_gt_mcr_get_nonterminated_steering(struct xe_gt *gt, · 8 unmodified lines | |||
| 679 | * Not found in a steering table and not a register with implicit | 678 | * Not found in a steering table and not a register with implicit |
| 680 | * steering. Just steer to 0/0 as a guess and raise a warning. | 679 | * steering. Just steer to 0/0 as a guess and raise a warning. |
| 681 | */ | 680 | */ |
| 682 | drm_WARN(>_to_xe(gt)->drm, true, | 681 | xe_gt_WARN(gt, true, |
| 683 | "Did not find MCR register %#x in any MCR steering table\n", | 682 | "Did not find MCR register %#x in any MCR steering table\n", |
| 684 | reg.addr); | 683 | reg.addr); |
| 685 | *group = 0; | 684 | *group = 0; |
| 686 | *instance = 0; | 685 | *instance = 0; |
| 687 | 686 | ||
| @@ -710,7 +709,7 @@ static void mcr_lock(struct xe_gt *gt) __acquires(>->mcr_lock) · 22 unmodified lines | |||
| 710 | ret = xe_mmio_wait32(>->mmio, STEER_SEMAPHORE, 0x1, 0x1, 10, NULL, | 709 | ret = xe_mmio_wait32(>->mmio, STEER_SEMAPHORE, 0x1, 0x1, 10, NULL, |
| 711 | true); | 710 | true); |
| 712 | 711 | ||
| 713 | drm_WARN_ON_ONCE(&xe->drm, ret == -ETIMEDOUT); | 712 | xe_gt_WARN_ON_ONCE(gt, ret == -ETIMEDOUT); |
| 714 | } | 713 | } |
| 715 | 714 | ||
| 716 | static void mcr_unlock(struct xe_gt *gt) __releases(>->mcr_lock) | 715 | static void mcr_unlock(struct xe_gt *gt) __releases(>->mcr_lock) |
| @@ -1532,7 +1532,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · 1531 unmodified lines | |||
| 1532 | * If devcoredump not captured and GuC capture for the job is not ready | 1532 | * If devcoredump not captured and GuC capture for the job is not ready |
| 1533 | * do manual capture first and decide later if we need to use it | 1533 | * do manual capture first and decide later if we need to use it |
| 1534 | */ | 1534 | */ |
| 1535 | if (!exec_queue_killed(q) && !xe->devcoredump.captured && | 1535 | if (!xe_device_is_in_reset(xe) && !exec_queue_killed(q) && !xe->devcoredump.captured && |
| 1536 | !xe_guc_capture_get_matching_and_lock(q)) { | 1536 | !xe_guc_capture_get_matching_and_lock(q)) { |
| 1537 | /* take force wake before engine register manual capture */ | 1537 | /* take force wake before engine register manual capture */ |
| 1538 | CLASS(xe_force_wake, fw_ref)(gt_to_fw(q->gt), XE_FORCEWAKE_ALL); | 1538 | CLASS(xe_force_wake, fw_ref)(gt_to_fw(q->gt), XE_FORCEWAKE_ALL); |
| @@ -1554,8 +1554,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · 15 unmodified lines | |||
| 1554 | set_exec_queue_banned(q); | 1554 | set_exec_queue_banned(q); |
| 1555 | 1555 | ||
| 1556 | /* Kick job / queue off hardware */ | 1556 | /* Kick job / queue off hardware */ |
| 1557 | if (!wedged && (exec_queue_enabled(primary) || | 1557 | if (!xe_device_is_in_reset(xe) && !wedged && |
| 1558 | exec_queue_pending_disable(primary))) { | 1558 | (exec_queue_enabled(primary) || exec_queue_pending_disable(primary))) { |
| 1559 | int ret; | 1559 | int ret; |
| 1560 | 1560 | ||
| 1561 | if (exec_queue_reset(primary)) | 1561 | if (exec_queue_reset(primary)) |
| @@ -1623,7 +1623,8 @@ trigger_reset: · 61 unmodified lines | |||
| 1623 | 1623 | ||
| 1624 | trace_xe_sched_job_timedout(job); | 1624 | trace_xe_sched_job_timedout(job); |
| 1625 | 1625 | ||
| 1626 | if (!exec_queue_killed(q)) | 1626 | /* Do not access device if in reset */ |
| 1627 | if (!xe_device_is_in_reset(xe) && !exec_queue_killed(q)) | ||
| 1627 | xe_devcoredump(q, job, | 1628 | xe_devcoredump(q, job, |
| 1628 | "Timedout job - seqno=%u, lrc_seqno=%u, guc_id=%d, flags=0x%lx", | 1629 | "Timedout job - seqno=%u, lrc_seqno=%u, guc_id=%d, flags=0x%lx", |
| 1629 | xe_sched_job_seqno(job), xe_sched_job_lrc_seqno(job), | 1630 | xe_sched_job_seqno(job), xe_sched_job_lrc_seqno(job), |
| @@ -2244,14 +2245,6 @@ static bool guc_exec_queue_reset_status(struct xe_exec_queue *q) · 614 unmodified lines | |||
| 2244 | return exec_queue_reset(q) || exec_queue_killed_or_banned_or_wedged(q); | 2245 | return exec_queue_reset(q) || exec_queue_killed_or_banned_or_wedged(q); |
| 2245 | } | 2246 | } |
| 2246 | 2247 | ||
| 2247 | static bool guc_exec_queue_active(struct xe_exec_queue *q) | ||
| 2248 | { | ||
| 2249 | struct xe_exec_queue *primary = xe_exec_queue_multi_queue_primary(q); | ||
| 2250 | |||
| 2251 | return exec_queue_enabled(primary) && | ||
| 2252 | !exec_queue_pending_disable(primary); | ||
| 2253 | } | ||
| 2254 | |||
| 2255 | /* | 2248 | /* |
| 2256 | * All of these functions are an abstraction layer which other parts of Xe can | 2249 | * All of these functions are an abstraction layer which other parts of Xe can |
| 2257 | * use to trap into the GuC backend. All of these functions, aside from init, | 2250 | * use to trap into the GuC backend. All of these functions, aside from init, |
| @@ -2271,7 +2264,6 @@ static const struct xe_exec_queue_ops guc_exec_queue_ops = { · 13 unmodified lines | |||
| 2271 | .suspend_wait = guc_exec_queue_suspend_wait, | 2264 | .suspend_wait = guc_exec_queue_suspend_wait, |
| 2272 | .resume = guc_exec_queue_resume, | 2265 | .resume = guc_exec_queue_resume, |
| 2273 | .reset_status = guc_exec_queue_reset_status, | 2266 | .reset_status = guc_exec_queue_reset_status, |
| 2274 | .active = guc_exec_queue_active, | ||
| 2275 | }; | 2267 | }; |
| 2276 | 2268 | ||
| 2277 | static void guc_exec_queue_stop(struct xe_guc *guc, struct xe_exec_queue *q) | 2269 | static void guc_exec_queue_stop(struct xe_guc *guc, struct xe_exec_queue *q) |
| @@ -208,9 +208,6 @@ static int send_tlb_inval_asid_ppgtt(struct xe_tlb_inval *tlb_inval, u32 seqno, · 207 unmodified lines | |||
| 208 | 208 | ||
| 209 | lockdep_assert_held(&tlb_inval->seqno_lock); | 209 | lockdep_assert_held(&tlb_inval->seqno_lock); |
| 210 | 210 | ||
| 211 | if (guc_to_xe(guc)->info.force_execlist) | ||
| 212 | return -ECANCELED; | ||
| 213 | |||
| 214 | return send_tlb_inval_ppgtt(guc, seqno, start, end, asid, | 211 | return send_tlb_inval_ppgtt(guc, seqno, start, end, asid, |
| 215 | XE_GUC_TLB_INVAL_PAGE_SELECTIVE, prl_sa); | 212 | XE_GUC_TLB_INVAL_PAGE_SELECTIVE, prl_sa); |
| 216 | } | 213 | } |
| @@ -228,9 +225,6 @@ static int send_tlb_inval_ctx_ppgtt(struct xe_tlb_inval *tlb_inval, u32 seqno, · 11 unmodified lines | |||
| 228 | 225 | ||
| 229 | lockdep_assert_held(&tlb_inval->seqno_lock); | 226 | lockdep_assert_held(&tlb_inval->seqno_lock); |
| 230 | 227 | ||
| 231 | if (xe->info.force_execlist) | ||
| 232 | return -ECANCELED; | ||
| 233 | |||
| 234 | vm = xe_device_asid_to_vm(xe, asid); | 228 | vm = xe_device_asid_to_vm(xe, asid); |
| 235 | if (IS_ERR(vm)) | 229 | if (IS_ERR(vm)) |
| 236 | return PTR_ERR(vm); | 230 | return PTR_ERR(vm); |
| @@ -337,39 +337,41 @@ static bool xe_rtp_cfeg_wmtp_disabled(const struct xe_device *xe, · 336 unmodified lines | |||
| 337 | return xe_mmio_read32(&hwe->gt->mmio, XEHP_FUSE4) & CFEG_WMTP_DISABLE; | 337 | return xe_mmio_read32(&hwe->gt->mmio, XEHP_FUSE4) & CFEG_WMTP_DISABLE; |
| 338 | } | 338 | } |
| 339 | 339 | ||
| 340 | static u32 blit_cctl_val(struct xe_gt *gt, struct xe_hw_engine *hwe) | ||
| 341 | { | ||
| 342 | return REG_FIELD_PREP(BLIT_CCTL_DST_MOCS_MASK, gt->mocs.uc_index) | | ||
| 343 | REG_FIELD_PREP(BLIT_CCTL_SRC_MOCS_MASK, gt->mocs.uc_index); | ||
| 344 | } | ||
| 345 | |||
| 346 | static const struct xe_rtp_table_sr lrc_setup = XE_RTP_TABLE_SR( | ||
| 347 | /* | ||
| 348 | * Some blitter commands do not have a field for MOCS, those | ||
| 349 | * commands will use MOCS index pointed by BLIT_CCTL. | ||
| 350 | * BLIT_CCTL registers are needed to be programmed to un-cached. | ||
| 351 | */ | ||
| 352 | { XE_RTP_NAME("BLIT_CCTL_default_MOCS"), | ||
| 353 | XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1200, 1274), | ||
| 354 | ENGINE_CLASS(COPY)), | ||
| 355 | XE_RTP_ACTIONS(FIELD_SET_FUNC(BLIT_CCTL(0), | ||
| 356 | BLIT_CCTL_DST_MOCS_MASK | | ||
| 357 | BLIT_CCTL_SRC_MOCS_MASK, | ||
| 358 | blit_cctl_val, | ||
| 359 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 360 | }, | ||
| 361 | /* Disable WMTP if HW doesn't support it */ | ||
| 362 | { XE_RTP_NAME("DISABLE_WMTP_ON_UNSUPPORTED_HW"), | ||
| 363 | XE_RTP_RULES(FUNC(xe_rtp_cfeg_wmtp_disabled)), | ||
| 364 | XE_RTP_ACTIONS(FIELD_SET(CS_CHICKEN1(0), | ||
| 365 | PREEMPT_GPGPU_LEVEL_MASK, | ||
| 366 | PREEMPT_GPGPU_THREAD_GROUP_LEVEL)), | ||
| 367 | XE_RTP_ENTRY_FLAG(FOREACH_ENGINE) | ||
| 368 | }, | ||
| 369 | ); | ||
| 370 | |||
| 340 | static void | 371 | static void |
| 341 | hw_engine_setup_default_lrc_state(struct xe_hw_engine *hwe) | 372 | hw_engine_setup_default_lrc_state(struct xe_hw_engine *hwe) |
| 342 | { | 373 | { |
| 343 | struct xe_gt *gt = hwe->gt; | ||
| 344 | const u8 mocs_write_idx = gt->mocs.uc_index; | ||
| 345 | const u8 mocs_read_idx = gt->mocs.uc_index; | ||
| 346 | u32 blit_cctl_val = REG_FIELD_PREP(BLIT_CCTL_DST_MOCS_MASK, mocs_write_idx) | | ||
| 347 | REG_FIELD_PREP(BLIT_CCTL_SRC_MOCS_MASK, mocs_read_idx); | ||
| 348 | struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe); | 374 | struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe); |
| 349 | const struct xe_rtp_table_sr lrc_setup = XE_RTP_TABLE_SR( | ||
| 350 | /* | ||
| 351 | * Some blitter commands do not have a field for MOCS, those | ||
| 352 | * commands will use MOCS index pointed by BLIT_CCTL. | ||
| 353 | * BLIT_CCTL registers are needed to be programmed to un-cached. | ||
| 354 | */ | ||
| 355 | { XE_RTP_NAME("BLIT_CCTL_default_MOCS"), | ||
| 356 | XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1200, 1274), | ||
| 357 | ENGINE_CLASS(COPY)), | ||
| 358 | XE_RTP_ACTIONS(FIELD_SET(BLIT_CCTL(0), | ||
| 359 | BLIT_CCTL_DST_MOCS_MASK | | ||
| 360 | BLIT_CCTL_SRC_MOCS_MASK, | ||
| 361 | blit_cctl_val, | ||
| 362 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 363 | }, | ||
| 364 | /* Disable WMTP if HW doesn't support it */ | ||
| 365 | { XE_RTP_NAME("DISABLE_WMTP_ON_UNSUPPORTED_HW"), | ||
| 366 | XE_RTP_RULES(FUNC(xe_rtp_cfeg_wmtp_disabled)), | ||
| 367 | XE_RTP_ACTIONS(FIELD_SET(CS_CHICKEN1(0), | ||
| 368 | PREEMPT_GPGPU_LEVEL_MASK, | ||
| 369 | PREEMPT_GPGPU_THREAD_GROUP_LEVEL)), | ||
| 370 | XE_RTP_ENTRY_FLAG(FOREACH_ENGINE) | ||
| 371 | }, | ||
| 372 | ); | ||
| 373 | 375 | ||
| 374 | xe_rtp_process_to_sr(&ctx, &lrc_setup, &hwe->reg_lrc, true); | 376 | xe_rtp_process_to_sr(&ctx, &lrc_setup, &hwe->reg_lrc, true); |
| 375 | } | 377 | } |
| @@ -385,86 +387,92 @@ void xe_hw_engine_setup_reg_lrc(struct xe_hw_engine *hwe) · 9 unmodified lines | |||
| 385 | xe_tuning_process_lrc(hwe); | 387 | xe_tuning_process_lrc(hwe); |
| 386 | } | 388 | } |
| 387 | 389 | ||
| 388 | static void | 390 | /* |
| 389 | hw_engine_setup_default_state(struct xe_hw_engine *hwe) | 391 | * RING_CMD_CCTL specifies the default MOCS entry that will be |
| 392 | * used by the command streamer when executing commands that | ||
| 393 | * don't have a way to explicitly specify a MOCS setting. | ||
| 394 | * The default should usually reference whichever MOCS entry | ||
| 395 | * corresponds to uncached behavior, although use of a WB cached | ||
| 396 | * entry is recommended by the spec in certain circumstances on | ||
| 397 | * specific platforms. | ||
| 398 | * Bspec: 72161 | ||
| 399 | */ | ||
| 400 | static u32 ring_cmd_cctl_val(struct xe_gt *gt, struct xe_hw_engine *hwe) | ||
| 390 | { | 401 | { |
| 391 | struct xe_gt *gt = hwe->gt; | ||
| 392 | struct xe_device *xe = gt_to_xe(gt); | 402 | struct xe_device *xe = gt_to_xe(gt); |
| 403 | u8 mocs_read_idx = gt->mocs.uc_index; | ||
| 404 | |||
| 405 | if (hwe->class == XE_ENGINE_CLASS_COMPUTE && IS_DGFX(xe) && | ||
| 406 | (GRAPHICS_VER(xe) >= 20 || xe->info.platform == XE_PVC)) | ||
| 407 | mocs_read_idx = gt->mocs.wb_index; | ||
| 408 | |||
| 409 | return REG_FIELD_PREP(CMD_CCTL_WRITE_OVERRIDE_MASK, gt->mocs.uc_index) | | ||
| 410 | REG_FIELD_PREP(CMD_CCTL_READ_OVERRIDE_MASK, mocs_read_idx); | ||
| 411 | } | ||
| 412 | |||
| 413 | static const struct xe_rtp_table_sr engine_sr = XE_RTP_TABLE_SR( | ||
| 414 | { XE_RTP_NAME("RING_CMD_CCTL_default_MOCS"), | ||
| 415 | XE_RTP_RULES(FUNC(xe_rtp_match_always)), | ||
| 416 | XE_RTP_ACTIONS(FIELD_SET_FUNC(RING_CMD_CCTL(0), | ||
| 417 | CMD_CCTL_WRITE_OVERRIDE_MASK | | ||
| 418 | CMD_CCTL_READ_OVERRIDE_MASK, | ||
| 419 | ring_cmd_cctl_val, | ||
| 420 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 421 | }, | ||
| 422 | { XE_RTP_NAME("Disable HW status page updates for interrupts"), | ||
| 423 | XE_RTP_RULES(FUNC(xe_rtp_match_always)), | ||
| 424 | XE_RTP_ACTIONS(SET(RING_HWSTAM(0), ~0x0, | ||
| 425 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 426 | }, | ||
| 427 | { XE_RTP_NAME("Disable engine 'legacy' mode"), | ||
| 428 | XE_RTP_RULES(FUNC(xe_rtp_match_always)), | ||
| 429 | XE_RTP_ACTIONS(SET(GFX_MODE(0), GFX_DISABLE_LEGACY_MODE, | ||
| 430 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 431 | }, | ||
| 393 | /* | 432 | /* |
| 394 | * RING_CMD_CCTL specifies the default MOCS entry that will be | 433 | * To allow the GSC engine to go idle on MTL we need to enable |
| 395 | * used by the command streamer when executing commands that | 434 | * idle messaging and set the hysteresis value (we use 0xA=5us |
| 396 | * don't have a way to explicitly specify a MOCS setting. | 435 | * as recommended in spec). On platforms after MTL this is |
| 397 | * The default should usually reference whichever MOCS entry | 436 | * enabled by default. |
| 398 | * corresponds to uncached behavior, although use of a WB cached | ||
| 399 | * entry is recommended by the spec in certain circumstances on | ||
| 400 | * specific platforms. | ||
| 401 | * Bspec: 72161 | ||
| 402 | */ | 437 | */ |
| 403 | const u8 mocs_write_idx = gt->mocs.uc_index; | 438 | { XE_RTP_NAME("MTL GSCCS IDLE MSG enable"), |
| 404 | const u8 mocs_read_idx = hwe->class == XE_ENGINE_CLASS_COMPUTE && IS_DGFX(xe) && | 439 | XE_RTP_RULES(MEDIA_VERSION(1300), ENGINE_CLASS(OTHER)), |
| 405 | (GRAPHICS_VER(xe) >= 20 || xe->info.platform == XE_PVC) ? | 440 | XE_RTP_ACTIONS(CLR(RING_PSMI_CTL(0), |
| 406 | gt->mocs.wb_index : gt->mocs.uc_index; | 441 | IDLE_MSG_DISABLE, |
| 407 | u32 ring_cmd_cctl_val = REG_FIELD_PREP(CMD_CCTL_WRITE_OVERRIDE_MASK, mocs_write_idx) | | 442 | XE_RTP_ACTION_FLAG(ENGINE_BASE)), |
| 408 | REG_FIELD_PREP(CMD_CCTL_READ_OVERRIDE_MASK, mocs_read_idx); | 443 | FIELD_SET(RING_PWRCTX_MAXCNT(0), |
| 444 | IDLE_WAIT_TIME, | ||
| 445 | 0xA, | ||
| 446 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 447 | }, | ||
| 448 | /* Enable Priority Mem Read */ | ||
| 449 | { XE_RTP_NAME("Priority_Mem_Read"), | ||
| 450 | XE_RTP_RULES(GRAPHICS_VERSION_RANGE(2001, XE_RTP_END_VERSION_UNDEFINED)), | ||
| 451 | XE_RTP_ACTIONS(SET(CSFE_CHICKEN1(0), CS_PRIORITY_MEM_READ, | ||
| 452 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 453 | }, | ||
| 454 | { XE_RTP_NAME("Enable CCS Engine(s)"), | ||
| 455 | XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1255, XE_RTP_END_VERSION_UNDEFINED), | ||
| 456 | FUNC(xe_rtp_match_first_render_or_compute)), | ||
| 457 | XE_RTP_ACTIONS(SET(RCU_MODE, RCU_MODE_CCS_ENABLE)) | ||
| 458 | }, | ||
| 459 | /* Use Fixed slice CCS mode */ | ||
| 460 | { XE_RTP_NAME("RCU_MODE_FIXED_SLICE_CCS_MODE"), | ||
| 461 | XE_RTP_RULES(FUNC(xe_hw_engine_match_fixed_cslice_mode)), | ||
| 462 | XE_RTP_ACTIONS(FIELD_SET(RCU_MODE, RCU_MODE_FIXED_SLICE_CCS_MODE, | ||
| 463 | RCU_MODE_FIXED_SLICE_CCS_MODE)) | ||
| 464 | }, | ||
| 465 | { XE_RTP_NAME("Enable MSI-X interrupt support"), | ||
| 466 | XE_RTP_RULES(FUNC(xe_rtp_match_has_msix)), | ||
| 467 | XE_RTP_ACTIONS(SET(GFX_MODE(0), GFX_MSIX_INTERRUPT_ENABLE, | ||
| 468 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 469 | }, | ||
| 470 | ); | ||
| 471 | |||
| 472 | static void | ||
| 473 | hw_engine_setup_default_state(struct xe_hw_engine *hwe) | ||
| 474 | { | ||
| 409 | struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe); | 475 | struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe); |
| 410 | const struct xe_rtp_table_sr engine_sr = XE_RTP_TABLE_SR( | ||
| 411 | { XE_RTP_NAME("RING_CMD_CCTL_default_MOCS"), | ||
| 412 | XE_RTP_RULES(FUNC(xe_rtp_match_always)), | ||
| 413 | XE_RTP_ACTIONS(FIELD_SET(RING_CMD_CCTL(0), | ||
| 414 | CMD_CCTL_WRITE_OVERRIDE_MASK | | ||
| 415 | CMD_CCTL_READ_OVERRIDE_MASK, | ||
| 416 | ring_cmd_cctl_val, | ||
| 417 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 418 | }, | ||
| 419 | { XE_RTP_NAME("Disable HW status page updates for interrupts"), | ||
| 420 | XE_RTP_RULES(FUNC(xe_rtp_match_always)), | ||
| 421 | XE_RTP_ACTIONS(SET(RING_HWSTAM(0), ~0x0, | ||
| 422 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 423 | }, | ||
| 424 | { XE_RTP_NAME("Disable engine 'legacy' mode"), | ||
| 425 | XE_RTP_RULES(FUNC(xe_rtp_match_always)), | ||
| 426 | XE_RTP_ACTIONS(SET(GFX_MODE(0), GFX_DISABLE_LEGACY_MODE, | ||
| 427 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 428 | }, | ||
| 429 | /* | ||
| 430 | * To allow the GSC engine to go idle on MTL we need to enable | ||
| 431 | * idle messaging and set the hysteresis value (we use 0xA=5us | ||
| 432 | * as recommended in spec). On platforms after MTL this is | ||
| 433 | * enabled by default. | ||
| 434 | */ | ||
| 435 | { XE_RTP_NAME("MTL GSCCS IDLE MSG enable"), | ||
| 436 | XE_RTP_RULES(MEDIA_VERSION(1300), ENGINE_CLASS(OTHER)), | ||
| 437 | XE_RTP_ACTIONS(CLR(RING_PSMI_CTL(0), | ||
| 438 | IDLE_MSG_DISABLE, | ||
| 439 | XE_RTP_ACTION_FLAG(ENGINE_BASE)), | ||
| 440 | FIELD_SET(RING_PWRCTX_MAXCNT(0), | ||
| 441 | IDLE_WAIT_TIME, | ||
| 442 | 0xA, | ||
| 443 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 444 | }, | ||
| 445 | /* Enable Priority Mem Read */ | ||
| 446 | { XE_RTP_NAME("Priority_Mem_Read"), | ||
| 447 | XE_RTP_RULES(GRAPHICS_VERSION_RANGE(2001, XE_RTP_END_VERSION_UNDEFINED)), | ||
| 448 | XE_RTP_ACTIONS(SET(CSFE_CHICKEN1(0), CS_PRIORITY_MEM_READ, | ||
| 449 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 450 | }, | ||
| 451 | { XE_RTP_NAME("Enable CCS Engine(s)"), | ||
| 452 | XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1255, XE_RTP_END_VERSION_UNDEFINED), | ||
| 453 | FUNC(xe_rtp_match_first_render_or_compute)), | ||
| 454 | XE_RTP_ACTIONS(SET(RCU_MODE, RCU_MODE_CCS_ENABLE)) | ||
| 455 | }, | ||
| 456 | /* Use Fixed slice CCS mode */ | ||
| 457 | { XE_RTP_NAME("RCU_MODE_FIXED_SLICE_CCS_MODE"), | ||
| 458 | XE_RTP_RULES(FUNC(xe_hw_engine_match_fixed_cslice_mode)), | ||
| 459 | XE_RTP_ACTIONS(FIELD_SET(RCU_MODE, RCU_MODE_FIXED_SLICE_CCS_MODE, | ||
| 460 | RCU_MODE_FIXED_SLICE_CCS_MODE)) | ||
| 461 | }, | ||
| 462 | { XE_RTP_NAME("Enable MSI-X interrupt support"), | ||
| 463 | XE_RTP_RULES(FUNC(xe_rtp_match_has_msix)), | ||
| 464 | XE_RTP_ACTIONS(SET(GFX_MODE(0), GFX_MSIX_INTERRUPT_ENABLE, | ||
| 465 | XE_RTP_ACTION_FLAG(ENGINE_BASE))) | ||
| 466 | }, | ||
| 467 | ); | ||
| 468 | 476 | ||
| 469 | xe_rtp_process_to_sr(&ctx, &engine_sr, &hwe->reg_sr, false); | 477 | xe_rtp_process_to_sr(&ctx, &engine_sr, &hwe->reg_sr, false); |
| 470 | } | 478 | } |
| @@ -437,6 +437,16 @@ static void hw_error_source_handler(struct xe_tile *tile, const enum hardware_er · 436 unmodified lines | |||
| 437 | if (!IS_DGFX(xe)) | 437 | if (!IS_DGFX(xe)) |
| 438 | return; | 438 | return; |
| 439 | 439 | ||
| 440 | /* | ||
| 441 | * Hardware errors are reported through System Controller on the platforms that | ||
| 442 | * support it, and never routed as direct IRQ to SGUnit. So we should never be | ||
| 443 | * here for those platforms. | ||
| 444 | */ | ||
| 445 | if (xe->info.has_sysctrl) { | ||
| 446 | drm_err_ratelimited(&xe->drm, HW_ERR "Invalid error routing\n"); | ||
| 447 | return; | ||
| 448 | } | ||
| 449 | |||
| 440 | spin_lock_irqsave(&xe->irq.lock, flags); | 450 | spin_lock_irqsave(&xe->irq.lock, flags); |
| 441 | err_src = xe_mmio_read32(&tile->mmio, DEV_ERR_STAT_REG(hw_err)); | 451 | err_src = xe_mmio_read32(&tile->mmio, DEV_ERR_STAT_REG(hw_err)); |
| 442 | if (!err_src) { | 452 | if (!err_src) { |
| @@ -516,14 +526,6 @@ void xe_hw_error_irq_handler(struct xe_tile *tile, const u32 master_ctl) · 73 unmodified lines | |||
| 516 | } | 526 | } |
| 517 | } | 527 | } |
| 518 | 528 | ||
| 519 | static int hw_error_info_init(struct xe_device *xe) | ||
| 520 | { | ||
| 521 | if (xe->info.platform != XE_PVC) | ||
| 522 | return 0; | ||
| 523 | |||
| 524 | return xe_drm_ras_init(xe); | ||
| 525 | } | ||
| 526 | |||
| 527 | /* | 529 | /* |
| 528 | * Process hardware errors during boot | 530 | * Process hardware errors during boot |
| 529 | */ | 531 | */ |
| @@ -550,16 +552,11 @@ static void process_hw_errors(struct xe_device *xe) · 20 unmodified lines | |||
| 550 | void xe_hw_error_init(struct xe_device *xe) | 552 | void xe_hw_error_init(struct xe_device *xe) |
| 551 | { | 553 | { |
| 552 | struct xe_tile *tile = xe_device_get_root_tile(xe); | 554 | struct xe_tile *tile = xe_device_get_root_tile(xe); |
| 553 | int ret; | ||
| 554 | 555 | ||
| 555 | if (!IS_DGFX(xe) || IS_SRIOV_VF(xe)) | 556 | if (!IS_DGFX(xe) || IS_SRIOV_VF(xe)) |
| 556 | return; | 557 | return; |
| 557 | 558 | ||
| 558 | INIT_WORK(&tile->csc_hw_error_work, csc_hw_error_work); | 559 | INIT_WORK(&tile->csc_hw_error_work, csc_hw_error_work); |
| 559 | 560 | ||
| 560 | ret = hw_error_info_init(xe); | ||
| 561 | if (ret) | ||
| 562 | drm_err(&xe->drm, "Failed to initialize XE DRM RAS (%pe)\n", ERR_PTR(ret)); | ||
| 563 | |||
| 564 | process_hw_errors(xe); | 561 | process_hw_errors(xe); |
| 565 | } | 562 | } |
| @@ -334,9 +334,6 @@ int xe_i2c_probe(struct xe_device *xe) · 333 unmodified lines | |||
| 334 | if (!xe->info.has_i2c) | 334 | if (!xe->info.has_i2c) |
| 335 | return 0; | 335 | return 0; |
| 336 | 336 | ||
| 337 | if (IS_SRIOV_VF(xe)) | ||
| 338 | return 0; | ||
| 339 | |||
| 340 | xe_i2c_read_endpoint(xe_root_tile_mmio(xe), &ep); | 337 | xe_i2c_read_endpoint(xe_root_tile_mmio(xe), &ep); |
| 341 | if (ep.cookie != XE_I2C_EP_COOKIE_DEVICE) | 338 | if (ep.cookie != XE_I2C_EP_COOKIE_DEVICE) |
| 342 | return 0; | 339 | return 0; |
| @@ -2618,13 +2618,19 @@ void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot) · 2617 unmodified lines | |||
| 2618 | kfree(snapshot); | 2618 | kfree(snapshot); |
| 2619 | } | 2619 | } |
| 2620 | 2620 | ||
| 2621 | static bool engine_valid_for_utilization(struct xe_gt *gt, struct xe_hw_engine *hwe) | ||
| 2622 | { | ||
| 2623 | /* The USM-reserved copy engine runs kernel migrate contexts queried here */ | ||
| 2624 | return hwe && (!xe_hw_engine_is_reserved(hwe) || xe_gt_is_usm_hwe(gt, hwe)); | ||
| 2625 | } | ||
| 2626 | |||
| 2621 | static struct xe_hw_engine *engine_id_to_hwe(struct xe_gt *gt, u32 engine_id) | 2627 | static struct xe_hw_engine *engine_id_to_hwe(struct xe_gt *gt, u32 engine_id) |
| 2622 | { | 2628 | { |
| 2623 | u16 class = REG_FIELD_GET(ENGINE_CLASS_ID, engine_id); | 2629 | u16 class = REG_FIELD_GET(ENGINE_CLASS_ID, engine_id); |
| 2624 | u16 instance = REG_FIELD_GET(ENGINE_INSTANCE_ID, engine_id); | 2630 | u16 instance = REG_FIELD_GET(ENGINE_INSTANCE_ID, engine_id); |
| 2625 | struct xe_hw_engine *hwe = xe_gt_hw_engine(gt, class, instance, false); | 2631 | struct xe_hw_engine *hwe = xe_gt_hw_engine(gt, class, instance, false); |
| 2626 | 2632 | ||
| 2627 | if (xe_gt_WARN_ONCE(gt, !hwe || xe_hw_engine_is_reserved(hwe), | 2633 | if (xe_gt_WARN_ONCE(gt, !engine_valid_for_utilization(gt, hwe), |
| 2628 | "Unexpected engine class:instance %d:%d for utilization\n", | 2634 | "Unexpected engine class:instance %d:%d for utilization\n", |
| 2629 | class, instance)) | 2635 | class, instance)) |
| 2630 | return NULL; | 2636 | return NULL; |
| @@ -11,27 +11,18 @@ · 10 unmodified lines | |||
| 11 | #include <linux/pci.h> | 11 | #include <linux/pci.h> |
| 12 | 12 | ||
| 13 | #include <drm/drm_managed.h> | 13 | #include <drm/drm_managed.h> |
| 14 | #include <drm/drm_print.h> | ||
| 15 | 14 | ||
| 16 | #include "regs/xe_bars.h" | 15 | #include "regs/xe_bars.h" |
| 17 | #include "xe_device.h" | 16 | #include "xe_device.h" |
| 18 | #include "xe_gt_sriov_vf.h" | 17 | #include "xe_gt_sriov_vf.h" |
| 18 | #include "xe_printk.h" | ||
| 19 | #include "xe_sriov.h" | 19 | #include "xe_sriov.h" |
| 20 | #include "xe_tile_printk.h" | ||
| 20 | #include "xe_trace.h" | 21 | #include "xe_trace.h" |
| 21 | #include "xe_wa.h" | 22 | #include "xe_wa.h" |
| 22 | 23 | ||
| 23 | #include "generated/xe_device_wa_oob.h" | 24 | #include "generated/xe_device_wa_oob.h" |
| 24 | 25 | ||
| 25 | static void tiles_fini(void *arg) | ||
| 26 | { | ||
| 27 | struct xe_device *xe = arg; | ||
| 28 | struct xe_tile *tile; | ||
| 29 | int id; | ||
| 30 | |||
| 31 | for_each_remote_tile(tile, xe, id) | ||
| 32 | tile->mmio.regs = NULL; | ||
| 33 | } | ||
| 34 | |||
| 35 | /* | 26 | /* |
| 36 | * On multi-tile devices, partition the BAR space for MMIO on each tile, | 27 | * On multi-tile devices, partition the BAR space for MMIO on each tile, |
| 37 | * possibly accounting for register override on the number of tiles available. | 28 | * possibly accounting for register override on the number of tiles available. |
| @@ -56,50 +47,71 @@ static void mmio_multi_tile_setup(struct xe_device *xe, size_t tile_mmio_size) · 18 unmodified lines | |||
| 56 | struct xe_tile *tile; | 47 | struct xe_tile *tile; |
| 57 | u8 id; | 48 | u8 id; |
| 58 | 49 | ||
| 59 | /* | ||
| 60 | * Nothing to be done as tile 0 has already been setup earlier with the | ||
| 61 | * entire BAR mapped - see xe_mmio_probe_early() | ||
| 62 | */ | ||
| 63 | if (xe->info.tile_count == 1) | ||
| 64 | return; | ||
| 65 | |||
| 66 | for_each_remote_tile(tile, xe, id) | 50 | for_each_remote_tile(tile, xe, id) |
| 67 | xe_mmio_init(&tile->mmio, tile, xe->mmio.regs + id * tile_mmio_size, SZ_4M); | 51 | xe_mmio_init(&tile->mmio, tile, xe->mmio.regs + id * tile_mmio_size, SZ_4M); |
| 68 | } | 52 | } |
| 69 | 53 | ||
| 54 | /** | ||
| 55 | * xe_mmio_probe_tiles() - Initialize all tiles' MMIO | ||
| 56 | * @xe: the &xe_device | ||
| 57 | * | ||
| 58 | * Initialize the remaining tiles' MMIO instances. | ||
| 59 | * | ||
| 60 | * Return: 0 on success or a negative error code on failure. | ||
| 61 | */ | ||
| 70 | int xe_mmio_probe_tiles(struct xe_device *xe) | 62 | int xe_mmio_probe_tiles(struct xe_device *xe) |
| 71 | { | 63 | { |
| 72 | size_t tile_mmio_size = SZ_16M; | 64 | size_t tile_mmio_size = SZ_16M; |
| 73 | 65 | ||
| 74 | mmio_multi_tile_setup(xe, tile_mmio_size); | 66 | /* |
| 67 | * Nothing to be done as tile 0 has already been setup earlier with the | ||
| 68 | * entire BAR mapped - see xe_mmio_probe_early() | ||
| 69 | */ | ||
| 70 | if (xe->info.tile_count == 1) | ||
| 71 | return 0; | ||
| 75 | 72 | ||
| 76 | return devm_add_action_or_reset(xe->drm.dev, tiles_fini, xe); | 73 | if (xe->mmio.size < xe->info.tile_count * tile_mmio_size) { |
| 74 | xe_err(xe, "GTTMMADR_BAR is too small for %d tiles: %zu\n", | ||
| 75 | xe->info.tile_count, xe->mmio.size); | ||
| 76 | return -EIO; | ||
| 77 | } | ||
| 78 | |||
| 79 | mmio_multi_tile_setup(xe, tile_mmio_size); | ||
| 80 | return 0; | ||
| 77 | } | 81 | } |
| 78 | 82 | ||
| 79 | static void mmio_fini(void *arg) | 83 | static void mmio_fini(void *arg) |
| 80 | { | 84 | { |
| 81 | struct xe_device *xe = arg; | 85 | struct xe_device *xe = arg; |
| 82 | struct xe_tile *root_tile = xe_device_get_root_tile(xe); | ||
| 83 | 86 | ||
| 84 | pci_iounmap(to_pci_dev(xe->drm.dev), xe->mmio.regs); | ||
| 85 | xe->mmio.regs = NULL; | 87 | xe->mmio.regs = NULL; |
| 86 | root_tile->mmio.regs = NULL; | ||
| 87 | } | 88 | } |
| 88 | 89 | ||
| 90 | /** | ||
| 91 | * xe_mmio_probe_early() - Probe and initialize device's MMIO | ||
| 92 | * @xe: the &xe_device | ||
| 93 | * | ||
| 94 | * Map the entire GTTMMADR_BAR and initialize the first tile's MMIO instance. | ||
| 95 | * | ||
| 96 | * The first 16MB of the GTTMMADR_BAR always belongs to the root tile, and | ||
| 97 | * includes: registers (0-4MB), reserved space (4MB-8MB) and GGTT (8MB-16MB). | ||
| 98 | * | ||
| 99 | * Return: 0 on success or a negative error code on failure. | ||
| 100 | */ | ||
| 89 | int xe_mmio_probe_early(struct xe_device *xe) | 101 | int xe_mmio_probe_early(struct xe_device *xe) |
| 90 | { | 102 | { |
| 91 | struct xe_tile *root_tile = xe_device_get_root_tile(xe); | 103 | struct xe_tile *root_tile = xe_device_get_root_tile(xe); |
| 92 | struct pci_dev *pdev = to_pci_dev(xe->drm.dev); | 104 | struct pci_dev *pdev = to_pci_dev(xe->drm.dev); |
| 93 | 105 | ||
| 94 | /* | 106 | xe->mmio.regs = pcim_iomap(pdev, GTTMMADR_BAR, 0); |
| 95 | * Map the entire BAR. | ||
| 96 | * The first 16MB of the BAR, belong to the root tile, and include: | ||
| 97 | * registers (0-4MB), reserved space (4MB-8MB) and GGTT (8MB-16MB). | ||
| 98 | */ | ||
| 99 | xe->mmio.size = pci_resource_len(pdev, GTTMMADR_BAR); | ||
| 100 | xe->mmio.regs = pci_iomap(pdev, GTTMMADR_BAR, 0); | ||
| 101 | if (!xe->mmio.regs) { | 107 | if (!xe->mmio.regs) { |
| 102 | drm_err(&xe->drm, "failed to map registers\n"); | 108 | xe_err(xe, "Failed to map GTTMMADR_BAR\n"); |
| 109 | return -EIO; | ||
| 110 | } | ||
| 111 | |||
| 112 | xe->mmio.size = pci_resource_len(pdev, GTTMMADR_BAR); | ||
| 113 | if (xe->mmio.size < SZ_16M) { | ||
| 114 | xe_err(xe, "GTTMMADR_BAR is too small: %zu\n", xe->mmio.size); | ||
| 103 | return -EIO; | 115 | return -EIO; |
| 104 | } | 116 | } |
| 105 | 117 | ||
| @@ -128,6 +140,11 @@ void xe_mmio_init(struct xe_mmio *mmio, struct xe_tile *tile, void __iomem *ptr, · 22 unmodified lines | |||
| 128 | mmio->tile = tile; | 140 | mmio->tile = tile; |
| 129 | } | 141 | } |
| 130 | 142 | ||
| 143 | static bool mmio_available(struct xe_mmio *mmio) | ||
| 144 | { | ||
| 145 | return !xe_tile_WARN_ON_ONCE(mmio->tile, !mmio->tile->xe->mmio.regs); | ||
| 146 | } | ||
| 147 | |||
| 131 | static void mmio_flush_pending_writes(struct xe_mmio *mmio) | 148 | static void mmio_flush_pending_writes(struct xe_mmio *mmio) |
| 132 | { | 149 | { |
| 133 | #define DUMMY_REG_OFFSET 0x130030 | 150 | #define DUMMY_REG_OFFSET 0x130030 |
| @@ -146,6 +163,9 @@ u8 xe_mmio_read8(struct xe_mmio *mmio, struct xe_reg reg) · 12 unmodified lines | |||
| 146 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); | 163 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 147 | u8 val; | 164 | u8 val; |
| 148 | 165 | ||
| 166 | if (!mmio_available(mmio)) | ||
| 167 | return 0; | ||
| 168 | |||
| 149 | mmio_flush_pending_writes(mmio); | 169 | mmio_flush_pending_writes(mmio); |
| 150 | 170 | ||
| 151 | val = readb(mmio->regs + addr); | 171 | val = readb(mmio->regs + addr); |
| @@ -158,6 +178,9 @@ void xe_mmio_write8(struct xe_mmio *mmio, struct xe_reg reg, u8 val) · 6 unmodified lines | |||
| 158 | { | 178 | { |
| 159 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); | 179 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 160 | 180 | ||
| 181 | if (!mmio_available(mmio)) | ||
| 182 | return; | ||
| 183 | |||
| 161 | trace_xe_reg_rw(mmio, true, addr, val, sizeof(val)); | 184 | trace_xe_reg_rw(mmio, true, addr, val, sizeof(val)); |
| 162 | 185 | ||
| 163 | writeb(val, mmio->regs + addr); | 186 | writeb(val, mmio->regs + addr); |
| @@ -168,6 +191,9 @@ u16 xe_mmio_read16(struct xe_mmio *mmio, struct xe_reg reg) · 4 unmodified lines | |||
| 168 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); | 191 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 169 | u16 val; | 192 | u16 val; |
| 170 | 193 | ||
| 194 | if (!mmio_available(mmio)) | ||
| 195 | return 0; | ||
| 196 | |||
| 171 | mmio_flush_pending_writes(mmio); | 197 | mmio_flush_pending_writes(mmio); |
| 172 | 198 | ||
| 173 | val = readw(mmio->regs + addr); | 199 | val = readw(mmio->regs + addr); |
| @@ -180,6 +206,9 @@ void xe_mmio_write32(struct xe_mmio *mmio, struct xe_reg reg, u32 val) · 6 unmodified lines | |||
| 180 | { | 206 | { |
| 181 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); | 207 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 182 | 208 | ||
| 209 | if (!mmio_available(mmio)) | ||
| 210 | return; | ||
| 211 | |||
| 183 | trace_xe_reg_rw(mmio, true, addr, val, sizeof(val)); | 212 | trace_xe_reg_rw(mmio, true, addr, val, sizeof(val)); |
| 184 | 213 | ||
| 185 | if (!reg.vf && IS_SRIOV_VF(mmio->tile->xe)) | 214 | if (!reg.vf && IS_SRIOV_VF(mmio->tile->xe)) |
| @@ -194,6 +223,9 @@ u32 xe_mmio_read32(struct xe_mmio *mmio, struct xe_reg reg) · 8 unmodified lines | |||
| 194 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); | 223 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 195 | u32 val; | 224 | u32 val; |
| 196 | 225 | ||
| 226 | if (!mmio_available(mmio)) | ||
| 227 | return 0; | ||
| 228 | |||
| 197 | mmio_flush_pending_writes(mmio); | 229 | mmio_flush_pending_writes(mmio); |
| 198 | 230 | ||
| 199 | if (!reg.vf && IS_SRIOV_VF(mmio->tile->xe)) | 231 | if (!reg.vf && IS_SRIOV_VF(mmio->tile->xe)) |
| @@ -282,8 +314,8 @@ u64 xe_mmio_read64_2x32(struct xe_mmio *mmio, struct xe_reg reg) · 82 unmodified lines | |||
| 282 | oldudw = udw; | 314 | oldudw = udw; |
| 283 | } | 315 | } |
| 284 | 316 | ||
| 285 | drm_WARN(&mmio->tile->xe->drm, retries == 0, | 317 | xe_tile_WARN(mmio->tile, retries == 0, |
| 286 | "64-bit read of %#x did not stabilize\n", reg.addr); | 318 | "MMIO: 64-bit read of %#x did not stabilize\n", reg.addr); |
| 287 | 319 | ||
| 288 | return (u64)udw << 32 | ldw; | 320 | return (u64)udw << 32 | ldw; |
| 289 | } | 321 | } |
| @@ -36,9 +36,6 @@ module_param_named(svm_notifier_size, xe_modparam.svm_notifier_size, uint, 0600) · 35 unmodified lines | |||
| 36 | MODULE_PARM_DESC(svm_notifier_size, "Set the svm notifier size in MiB, must be power of 2 " | 36 | MODULE_PARM_DESC(svm_notifier_size, "Set the svm notifier size in MiB, must be power of 2 " |
| 37 | "[default=" __stringify(XE_DEFAULT_SVM_NOTIFIER_SIZE) "]"); | 37 | "[default=" __stringify(XE_DEFAULT_SVM_NOTIFIER_SIZE) "]"); |
| 38 | 38 | ||
| 39 | module_param_named_unsafe(force_execlist, xe_modparam.force_execlist, bool, 0444); | ||
| 40 | MODULE_PARM_DESC(force_execlist, "Force Execlist submission"); | ||
| 41 | |||
| 42 | #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY) | 39 | #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY) |
| 43 | module_param_named(probe_display, xe_modparam.probe_display, bool, 0444); | 40 | module_param_named(probe_display, xe_modparam.probe_display, bool, 0444); |
| 44 | MODULE_PARM_DESC(probe_display, "Probe display HW, otherwise it's left untouched " | 41 | MODULE_PARM_DESC(probe_display, "Probe display HW, otherwise it's left untouched " |
| @@ -10,7 +10,6 @@ · 9 unmodified lines | |||
| 10 | 10 | ||
| 11 | /* Module modprobe variables */ | 11 | /* Module modprobe variables */ |
| 12 | struct xe_modparam { | 12 | struct xe_modparam { |
| 13 | bool force_execlist; | ||
| 14 | bool probe_display; | 13 | bool probe_display; |
| 15 | int force_vram_bar_size; | 14 | int force_vram_bar_size; |
| 16 | int guc_log_level; | 15 | int guc_log_level; |
| @@ -1702,11 +1702,12 @@ static int xe_oa_release(struct inode *inode, struct file *file) · 1701 unmodified lines | |||
| 1702 | static int xe_oa_mmap(struct file *file, struct vm_area_struct *vma) | 1702 | static int xe_oa_mmap(struct file *file, struct vm_area_struct *vma) |
| 1703 | { | 1703 | { |
| 1704 | struct xe_oa_stream *stream = file->private_data; | 1704 | struct xe_oa_stream *stream = file->private_data; |
| 1705 | int ret = xe_observation_paranoid_check(); | ||
| 1705 | struct xe_bo *bo = stream->oa_buffer.bo; | 1706 | struct xe_bo *bo = stream->oa_buffer.bo; |
| 1706 | 1707 | ||
| 1707 | if (xe_observation_paranoid && !perfmon_capable()) { | 1708 | if (ret) { |
| 1708 | drm_dbg(&stream->oa->xe->drm, "Insufficient privilege to map OA buffer\n"); | 1709 | drm_dbg(&stream->oa->xe->drm, "Insufficient privilege to map OA buffer\n"); |
| 1709 | return -EACCES; | 1710 | return ret; |
| 1710 | } | 1711 | } |
| 1711 | 1712 | ||
| 1712 | /* Can mmap the entire OA buffer or nothing (no partial OA buffer mmaps) */ | 1713 | /* Can mmap the entire OA buffer or nothing (no partial OA buffer mmaps) */ |
| @@ -2080,10 +2081,12 @@ int xe_oa_stream_open_ioctl(struct drm_device *dev, u64 data, struct drm_file *f · 367 unmodified lines | |||
| 2080 | privileged_op = true; | 2081 | privileged_op = true; |
| 2081 | } | 2082 | } |
| 2082 | 2083 | ||
| 2083 | if (privileged_op && xe_observation_paranoid && !perfmon_capable()) { | 2084 | if (privileged_op) { |
| 2084 | drm_dbg(&oa->xe->drm, "Insufficient privileges to open xe OA stream\n"); | 2085 | ret = xe_observation_paranoid_check(); |
| 2085 | ret = -EACCES; | 2086 | if (ret) { |
| 2086 | goto err_exec_q; | 2087 | drm_dbg(&oa->xe->drm, "Insufficient privileges to open xe OA stream\n"); |
| 2088 | goto err_exec_q; | ||
| 2089 | } | ||
| 2087 | } | 2090 | } |
| 2088 | 2091 | ||
| 2089 | if (!param.exec_q && !param.sample) { | 2092 | if (!param.exec_q && !param.sample) { |
| @@ -2365,9 +2368,10 @@ int xe_oa_add_config_ioctl(struct drm_device *dev, u64 data, struct drm_file *fi · 275 unmodified lines | |||
| 2365 | return -ENODEV; | 2368 | return -ENODEV; |
| 2366 | } | 2369 | } |
| 2367 | 2370 | ||
| 2368 | if (xe_observation_paranoid && !perfmon_capable()) { | 2371 | err = xe_observation_paranoid_check(); |
| 2372 | if (err) { | ||
| 2369 | drm_dbg(&oa->xe->drm, "Insufficient privileges to add xe OA config\n"); | 2373 | drm_dbg(&oa->xe->drm, "Insufficient privileges to add xe OA config\n"); |
| 2370 | return -EACCES; | 2374 | return err; |
| 2371 | } | 2375 | } |
| 2372 | 2376 | ||
| 2373 | err = copy_from_user(¶m, u64_to_user_ptr(data), sizeof(param)); | 2377 | err = copy_from_user(¶m, u64_to_user_ptr(data), sizeof(param)); |
| @@ -2467,9 +2471,10 @@ int xe_oa_remove_config_ioctl(struct drm_device *dev, u64 data, struct drm_file · 93 unmodified lines | |||
| 2467 | return -ENODEV; | 2471 | return -ENODEV; |
| 2468 | } | 2472 | } |
| 2469 | 2473 | ||
| 2470 | if (xe_observation_paranoid && !perfmon_capable()) { | 2474 | ret = xe_observation_paranoid_check(); |
| 2475 | if (ret) { | ||
| 2471 | drm_dbg(&oa->xe->drm, "Insufficient privileges to remove xe OA config\n"); | 2476 | drm_dbg(&oa->xe->drm, "Insufficient privileges to remove xe OA config\n"); |
| 2472 | return -EACCES; | 2477 | return ret; |
| 2473 | } | 2478 | } |
| 2474 | 2479 | ||
| 2475 | ret = get_user(arg, ptr); | 2480 | ret = get_user(arg, ptr); |
| @@ -4,6 +4,7 @@ · 3 unmodified lines | |||
| 4 | */ | 4 | */ |
| 5 | 5 | ||
| 6 | #include <linux/errno.h> | 6 | #include <linux/errno.h> |
| 7 | #include <linux/perf_event.h> | ||
| 7 | #include <linux/sysctl.h> | 8 | #include <linux/sysctl.h> |
| 8 | 9 | ||
| 9 | #include <uapi/drm/xe_drm.h> | 10 | #include <uapi/drm/xe_drm.h> |
| @@ -12,9 +13,28 @@ · 2 unmodified lines | |||
| 12 | #include "xe_oa.h" | 13 | #include "xe_oa.h" |
| 13 | #include "xe_observation.h" | 14 | #include "xe_observation.h" |
| 14 | 15 | ||
| 15 | u32 xe_observation_paranoid = true; | 16 | static u32 xe_observation_paranoid = true; |
| 16 | static struct ctl_table_header *sysctl_header; | 17 | static struct ctl_table_header *sysctl_header; |
| 17 | 18 | ||
| 19 | /** | ||
| 20 | * xe_observation_paranoid_check - Gate access to xe observation streams. | ||
| 21 | * | ||
| 22 | * When the xe-specific observation_paranoid sysctl is enabled (the | ||
| 23 | * default), defer to perf_allow_cpu() so that access is governed by the | ||
| 24 | * same policy as system-wide perf CPU events: kernel.perf_event_paranoid | ||
| 25 | * plus the security_perf_event_open() LSM hook. When the sysctl has been | ||
| 26 | * cleared by a privileged user, observation is open to all callers. | ||
| 27 | * | ||
| 28 | * Return: 0 if access is permitted, a negative errno otherwise. | ||
| 29 | */ | ||
| 30 | int xe_observation_paranoid_check(void) | ||
| 31 | { | ||
| 32 | if (!xe_observation_paranoid) | ||
| 33 | return 0; | ||
| 34 | |||
| 35 | return perf_allow_cpu(); | ||
| 36 | } | ||
| 37 | |||
| 18 | static int xe_oa_ioctl(struct drm_device *dev, struct drm_xe_observation_param *arg, | 38 | static int xe_oa_ioctl(struct drm_device *dev, struct drm_xe_observation_param *arg, |
| 19 | struct drm_file *file) | 39 | struct drm_file *file) |
| 20 | { | 40 | { |
| @@ -83,11 +103,13 @@ static const struct ctl_table observation_ctl_table[] = { · 62 unmodified lines | |||
| 83 | }; | 103 | }; |
| 84 | 104 | ||
| 85 | /** | 105 | /** |
| 86 | * xe_observation_sysctl_register - Register xe_observation_paranoid sysctl | 106 | * xe_observation_sysctl_register - Register the observation_paranoid sysctl |
| 87 | * | 107 | * |
| 88 | * Normally only superuser/root can access observation stream | 108 | * When dev.xe.observation_paranoid is set (the default), access to |
| 89 | * data. However, superuser can set xe_observation_paranoid sysctl to 0 to | 109 | * observation streams follows the system-wide perf_allow_cpu() policy: |
| 90 | * allow non-privileged users to also access observation data. | 110 | * kernel.perf_event_paranoid plus the security_perf_event_open() LSM |
| 111 | * hook. A privileged user can clear the sysctl to bypass that gate and | ||
| 112 | * allow unprivileged access to observation data. | ||
| 91 | * | 113 | * |
| 92 | * Return: always returns 0 | 114 | * Return: always returns 0 |
| 93 | */ | 115 | */ |
| @@ -11,8 +11,7 @@ · 10 unmodified lines | |||
| 11 | struct drm_device; | 11 | struct drm_device; |
| 12 | struct drm_file; | 12 | struct drm_file; |
| 13 | 13 | ||
| 14 | extern u32 xe_observation_paranoid; | 14 | int xe_observation_paranoid_check(void); |
| 15 | |||
| 16 | int xe_observation_ioctl(struct drm_device *dev, void *data, struct drm_file *file); | 15 | int xe_observation_ioctl(struct drm_device *dev, void *data, struct drm_file *file); |
| 17 | int xe_observation_sysctl_register(void); | 16 | int xe_observation_sysctl_register(void); |
| 18 | void xe_observation_sysctl_unregister(void); | 17 | void xe_observation_sysctl_unregister(void); |
| @@ -26,6 +26,7 @@ · 25 unmodified lines | |||
| 26 | #include "xe_guc.h" | 26 | #include "xe_guc.h" |
| 27 | #include "xe_mmio.h" | 27 | #include "xe_mmio.h" |
| 28 | #include "xe_module.h" | 28 | #include "xe_module.h" |
| 29 | #include "xe_pci_error.h" | ||
| 29 | #include "xe_pci_rebar.h" | 30 | #include "xe_pci_rebar.h" |
| 30 | #include "xe_pci_sriov.h" | 31 | #include "xe_pci_sriov.h" |
| 31 | #include "xe_pci_types.h" | 32 | #include "xe_pci_types.h" |
| @@ -355,6 +356,7 @@ static const __maybe_unused struct xe_device_desc pvc_desc = { · 323 unmodified lines | |||
| 355 | PLATFORM(PVC), | 356 | PLATFORM(PVC), |
| 356 | .dma_mask_size = 52, | 357 | .dma_mask_size = 52, |
| 357 | .has_display = false, | 358 | .has_display = false, |
| 359 | .has_drm_ras = true, | ||
| 358 | .has_gsc_nvm = 1, | 360 | .has_gsc_nvm = 1, |
| 359 | .has_heci_gscfi = 1, | 361 | .has_heci_gscfi = 1, |
| 360 | .max_gt_per_tile = 1, | 362 | .max_gt_per_tile = 1, |
| @@ -457,6 +459,7 @@ static const struct xe_device_desc cri_desc = { · 96 unmodified lines | |||
| 457 | PLATFORM(CRESCENTISLAND), | 459 | PLATFORM(CRESCENTISLAND), |
| 458 | .dma_mask_size = 52, | 460 | .dma_mask_size = 52, |
| 459 | .has_display = false, | 461 | .has_display = false, |
| 462 | .has_drm_ras = true, | ||
| 460 | .has_flat_ccs = false, | 463 | .has_flat_ccs = false, |
| 461 | .has_gsc_nvm = 1, | 464 | .has_gsc_nvm = 1, |
| 462 | .has_i2c = true, | 465 | .has_i2c = true, |
| @@ -599,8 +602,6 @@ static int read_gmdid(struct xe_device *xe, enum xe_gmdid_type type, u32 *ver, u · 136 unmodified lines | |||
| 599 | struct xe_reg gmdid_reg = GMD_ID; | 602 | struct xe_reg gmdid_reg = GMD_ID; |
| 600 | u32 val; | 603 | u32 val; |
| 601 | 604 | ||
| 602 | KUNIT_STATIC_STUB_REDIRECT(read_gmdid, xe, type, ver, revid); | ||
| 603 | |||
| 604 | if (IS_SRIOV_VF(xe)) { | 605 | if (IS_SRIOV_VF(xe)) { |
| 605 | /* | 606 | /* |
| 606 | * To get the value of the GMDID register, VFs must obtain it | 607 | * To get the value of the GMDID register, VFs must obtain it |
| @@ -726,14 +727,30 @@ static int handle_gmdid(struct xe_device *xe, · 119 unmodified lines | |||
| 726 | return 0; | 727 | return 0; |
| 727 | } | 728 | } |
| 728 | 729 | ||
| 729 | static void init_devid(struct xe_device *xe) | 730 | struct xe_probed_info { |
| 731 | u16 devid; | ||
| 732 | u8 revid; | ||
| 733 | u8 tile_count; | ||
| 734 | struct xe_step_info step; | ||
| 735 | const struct xe_ip *graphics_ip; | ||
| 736 | const struct xe_ip *media_ip; | ||
| 737 | }; | ||
| 738 | |||
| 739 | /* | ||
| 740 | * Probe from the hardware the info required by xe_info_init_early(). | ||
| 741 | */ | ||
| 742 | static int xe_probe_info_early(struct xe_device *xe, | ||
| 743 | const struct xe_device_desc *desc, | ||
| 744 | struct xe_probed_info *probed_info) | ||
| 730 | { | 745 | { |
| 731 | struct pci_dev *pdev = to_pci_dev(xe->drm.dev); | 746 | struct pci_dev *pdev = to_pci_dev(xe->drm.dev); |
| 732 | 747 | ||
| 733 | KUNIT_STATIC_STUB_REDIRECT(init_devid, xe); | 748 | probed_info->devid = pdev->device; |
| 749 | probed_info->revid = pdev->revision; | ||
| 734 | 750 | ||
| 735 | xe->info.devid = pdev->device; | 751 | xe_step_platform_get(desc->platform, probed_info->revid, &probed_info->step); |
| 736 | xe->info.revid = pdev->revision; | 752 | |
| 753 | return 0; | ||
| 737 | } | 754 | } |
| 738 | 755 | ||
| 739 | /* | 756 | /* |
| @@ -742,17 +759,20 @@ static void init_devid(struct xe_device *xe) · 2 unmodified lines | |||
| 742 | */ | 759 | */ |
| 743 | static int xe_info_init_early(struct xe_device *xe, | 760 | static int xe_info_init_early(struct xe_device *xe, |
| 744 | const struct xe_device_desc *desc, | 761 | const struct xe_device_desc *desc, |
| 745 | const struct xe_subplatform_desc *subplatform_desc) | 762 | const struct xe_subplatform_desc *subplatform_desc, |
| 763 | struct xe_probed_info *probed_info) | ||
| 746 | { | 764 | { |
| 747 | int err; | 765 | int err; |
| 748 | 766 | ||
| 767 | xe->info.devid = probed_info->devid; | ||
| 768 | xe->info.revid = probed_info->revid; | ||
| 769 | xe->info.step.platform = probed_info->step.platform; | ||
| 770 | |||
| 749 | xe->info.platform_name = desc->platform_name; | 771 | xe->info.platform_name = desc->platform_name; |
| 750 | xe->info.platform = desc->platform; | 772 | xe->info.platform = desc->platform; |
| 751 | xe->info.subplatform = subplatform_desc ? | 773 | xe->info.subplatform = subplatform_desc ? |
| 752 | subplatform_desc->subplatform : XE_SUBPLATFORM_NONE; | 774 | subplatform_desc->subplatform : XE_SUBPLATFORM_NONE; |
| 753 | 775 | ||
| 754 | init_devid(xe); | ||
| 755 | |||
| 756 | xe->info.dma_mask_size = desc->dma_mask_size; | 776 | xe->info.dma_mask_size = desc->dma_mask_size; |
| 757 | xe->info.va_bits = desc->va_bits; | 777 | xe->info.va_bits = desc->va_bits; |
| 758 | xe->info.vm_max_level = desc->vm_max_level; | 778 | xe->info.vm_max_level = desc->vm_max_level; |
| @@ -760,6 +780,7 @@ static int xe_info_init_early(struct xe_device *xe, · 1 unmodified line | |||
| 760 | 780 | ||
| 761 | xe->info.is_dgfx = desc->is_dgfx; | 781 | xe->info.is_dgfx = desc->is_dgfx; |
| 762 | xe->info.has_cached_pt = desc->has_cached_pt; | 782 | xe->info.has_cached_pt = desc->has_cached_pt; |
| 783 | xe->info.has_drm_ras = desc->has_drm_ras; | ||
| 763 | xe->info.has_fan_control = desc->has_fan_control; | 784 | xe->info.has_fan_control = desc->has_fan_control; |
| 764 | /* runtime fusing may force flat_ccs to disabled later */ | 785 | /* runtime fusing may force flat_ccs to disabled later */ |
| 765 | xe->info.has_flat_ccs = desc->has_flat_ccs; | 786 | xe->info.has_flat_ccs = desc->has_flat_ccs; |
| @@ -789,14 +810,10 @@ static int xe_info_init_early(struct xe_device *xe, · 23 unmodified lines | |||
| 789 | xe->info.probe_display = IS_ENABLED(CONFIG_DRM_XE_DISPLAY) && | 810 | xe->info.probe_display = IS_ENABLED(CONFIG_DRM_XE_DISPLAY) && |
| 790 | xe_modparam.probe_display && | 811 | xe_modparam.probe_display && |
| 791 | desc->has_display; | 812 | desc->has_display; |
| 792 | xe->info.force_execlist = xe_modparam.force_execlist; | ||
| 793 | 813 | ||
| 794 | xe_assert(xe, desc->max_gt_per_tile > 0); | 814 | xe_assert(xe, desc->max_gt_per_tile > 0); |
| 795 | xe_assert(xe, desc->max_gt_per_tile <= XE_MAX_GT_PER_TILE); | 815 | xe_assert(xe, desc->max_gt_per_tile <= XE_MAX_GT_PER_TILE); |
| 796 | xe->info.max_gt_per_tile = desc->max_gt_per_tile; | 816 | xe->info.max_gt_per_tile = desc->max_gt_per_tile; |
| 797 | xe->info.tile_count = 1 + desc->max_remote_tiles; | ||
| 798 | |||
| 799 | xe_step_platform_get(xe); | ||
| 800 | 817 | ||
| 801 | err = xe_tile_init_early(xe_device_get_root_tile(xe), xe, 0); | 818 | err = xe_tile_init_early(xe_device_get_root_tile(xe), xe, 0); |
| 802 | if (err) | 819 | if (err) |
| @@ -805,22 +822,21 @@ static int xe_info_init_early(struct xe_device *xe, · 2 unmodified lines | |||
| 805 | return 0; | 822 | return 0; |
| 806 | } | 823 | } |
| 807 | 824 | ||
| 808 | /* | 825 | static void xe_probe_tile_count(struct xe_device *xe, |
| 809 | * Possibly override number of tile based on configuration register. | 826 | const struct xe_device_desc *desc, |
| 810 | */ | 827 | struct xe_probed_info *probed_info) |
| 811 | static void xe_info_probe_tile_count(struct xe_device *xe) | ||
| 812 | { | 828 | { |
| 813 | struct xe_mmio *mmio; | 829 | struct xe_mmio *mmio; |
| 814 | u8 tile_count; | 830 | u8 tile_count; |
| 815 | u32 mtcfg; | 831 | u32 mtcfg; |
| 816 | 832 | ||
| 817 | KUNIT_STATIC_STUB_REDIRECT(xe_info_probe_tile_count, xe); | 833 | probed_info->tile_count = 1 + desc->max_remote_tiles; |
| 818 | 834 | ||
| 819 | /* | 835 | /* |
| 820 | * Probe for tile count only for platforms that support multiple | 836 | * Probe for tile count only for platforms that support multiple |
| 821 | * tiles. | 837 | * tiles. |
| 822 | */ | 838 | */ |
| 823 | if (xe->info.tile_count == 1) | 839 | if (probed_info->tile_count == 1) |
| 824 | return; | 840 | return; |
| 825 | 841 | ||
| 826 | mmio = xe_root_tile_mmio(xe); | 842 | mmio = xe_root_tile_mmio(xe); |
| @@ -833,10 +849,10 @@ static void xe_info_probe_tile_count(struct xe_device *xe) · 6 unmodified lines | |||
| 833 | mtcfg = xe_mmio_read32(mmio, XEHP_MTCFG_ADDR); | 849 | mtcfg = xe_mmio_read32(mmio, XEHP_MTCFG_ADDR); |
| 834 | tile_count = REG_FIELD_GET(TILE_COUNT, mtcfg) + 1; | 850 | tile_count = REG_FIELD_GET(TILE_COUNT, mtcfg) + 1; |
| 835 | 851 | ||
| 836 | if (tile_count < xe->info.tile_count) { | 852 | if (tile_count < probed_info->tile_count) { |
| 837 | drm_info(&xe->drm, "tile_count: %d, reduced_tile_count %d\n", | 853 | drm_info(&xe->drm, "tile_count: %d, reduced_tile_count %d\n", |
| 838 | xe->info.tile_count, tile_count); | 854 | probed_info->tile_count, tile_count); |
| 839 | xe->info.tile_count = tile_count; | 855 | probed_info->tile_count = tile_count; |
| 840 | } | 856 | } |
| 841 | } | 857 | } |
| 842 | 858 | ||
| @@ -909,25 +925,10 @@ static struct xe_gt *alloc_media_gt(struct xe_tile *tile, · 66 unmodified lines | |||
| 909 | return gt; | 925 | return gt; |
| 910 | } | 926 | } |
| 911 | 927 | ||
| 912 | /* | 928 | static int xe_probe_ips(struct xe_device *xe, |
| 913 | * Initialize device info content that does require knowledge about | 929 | const struct xe_device_desc *desc, |
| 914 | * graphics / media IP version. | 930 | struct xe_probed_info *probed_info) |
| 915 | * Make sure that GT / tile structures allocated by the driver match the data | ||
| 916 | * present in device info. | ||
| 917 | */ | ||
| 918 | static int xe_info_init(struct xe_device *xe, | ||
| 919 | const struct xe_device_desc *desc) | ||
| 920 | { | 931 | { |
| 921 | u32 graphics_gmdid_revid = 0, media_gmdid_revid = 0; | ||
| 922 | const struct xe_ip *graphics_ip; | ||
| 923 | const struct xe_ip *media_ip; | ||
| 924 | const struct xe_graphics_desc *graphics_desc; | ||
| 925 | const struct xe_media_desc *media_desc; | ||
| 926 | struct xe_tile *tile; | ||
| 927 | struct xe_gt *gt; | ||
| 928 | int ret; | ||
| 929 | u8 id; | ||
| 930 | |||
| 931 | /* | 932 | /* |
| 932 | * If this platform supports GMD_ID, we'll detect the proper IP | 933 | * If this platform supports GMD_ID, we'll detect the proper IP |
| 933 | * descriptor to use from hardware registers. | 934 | * descriptor to use from hardware registers. |
| @@ -936,17 +937,21 @@ static int xe_info_init(struct xe_device *xe, · 2 unmodified lines | |||
| 936 | * versions are simply derived from that. | 937 | * versions are simply derived from that. |
| 937 | */ | 938 | */ |
| 938 | if (desc->pre_gmdid_graphics_ip) { | 939 | if (desc->pre_gmdid_graphics_ip) { |
| 939 | graphics_ip = desc->pre_gmdid_graphics_ip; | 940 | probed_info->graphics_ip = desc->pre_gmdid_graphics_ip; |
| 940 | media_ip = desc->pre_gmdid_media_ip; | 941 | probed_info->media_ip = desc->pre_gmdid_media_ip; |
| 941 | xe_step_pre_gmdid_get(xe); | 942 | xe_step_pre_gmdid_get(xe, &probed_info->step); |
| 942 | } else { | 943 | } else { |
| 944 | int err; | ||
| 945 | u32 graphics_revid, media_revid; | ||
| 946 | |||
| 943 | xe_assert(xe, !desc->pre_gmdid_media_ip); | 947 | xe_assert(xe, !desc->pre_gmdid_media_ip); |
| 944 | ret = handle_gmdid(xe, &graphics_ip, &media_ip, | ||
| 945 | &graphics_gmdid_revid, &media_gmdid_revid); | ||
| 946 | if (ret) | ||
| 947 | return ret; | ||
| 948 | 948 | ||
| 949 | xe_step_gmdid_get(xe, graphics_gmdid_revid, media_gmdid_revid); | 949 | err = handle_gmdid(xe, &probed_info->graphics_ip, &probed_info->media_ip, |
| 950 | &graphics_revid, &media_revid); | ||
| 951 | if (err) | ||
| 952 | return err; | ||
| 953 | |||
| 954 | xe_step_gmdid_get(xe, graphics_revid, media_revid, &probed_info->step); | ||
| 950 | } | 955 | } |
| 951 | 956 | ||
| 952 | /* | 957 | /* |
| @@ -954,9 +959,56 @@ static int xe_info_init(struct xe_device *xe, · 1 unmodified line | |||
| 954 | * error and we should abort driver load. Failing to detect media | 959 | * error and we should abort driver load. Failing to detect media |
| 955 | * IP is non-fatal; we'll just proceed without enabling media support. | 960 | * IP is non-fatal; we'll just proceed without enabling media support. |
| 956 | */ | 961 | */ |
| 957 | if (!graphics_ip) | 962 | if (!probed_info->graphics_ip) |
| 958 | return -ENODEV; | 963 | return -ENODEV; |
| 959 | 964 | ||
| 965 | return 0; | ||
| 966 | } | ||
| 967 | |||
| 968 | /* | ||
| 969 | * Probe from the hardware the info required by xe_info_init(). | ||
| 970 | */ | ||
| 971 | static int xe_probe_info(struct xe_device *xe, | ||
| 972 | const struct xe_device_desc *desc, | ||
| 973 | struct xe_probed_info *probed_info) | ||
| 974 | { | ||
| 975 | int err; | ||
| 976 | |||
| 977 | xe_probe_tile_count(xe, desc, probed_info); | ||
| 978 | |||
| 979 | err = xe_probe_ips(xe, desc, probed_info); | ||
| 980 | if (err) | ||
| 981 | return err; | ||
| 982 | |||
| 983 | return 0; | ||
| 984 | } | ||
| 985 | |||
| 986 | /* | ||
| 987 | * Initialize device info content that does require knowledge about | ||
| 988 | * graphics / media IP version. | ||
| 989 | * Make sure that GT / tile structures allocated by the driver match the data | ||
| 990 | * present in device info. | ||
| 991 | */ | ||
| 992 | static int xe_info_init(struct xe_device *xe, | ||
| 993 | const struct xe_device_desc *desc, | ||
| 994 | struct xe_probed_info *probed_info) | ||
| 995 | { | ||
| 996 | const struct xe_ip *graphics_ip; | ||
| 997 | const struct xe_ip *media_ip; | ||
| 998 | const struct xe_graphics_desc *graphics_desc; | ||
| 999 | const struct xe_media_desc *media_desc; | ||
| 1000 | struct xe_tile *tile; | ||
| 1001 | struct xe_gt *gt; | ||
| 1002 | u8 id; | ||
| 1003 | |||
| 1004 | graphics_ip = probed_info->graphics_ip; | ||
| 1005 | media_ip = probed_info->media_ip; | ||
| 1006 | |||
| 1007 | xe->info.tile_count = probed_info->tile_count; | ||
| 1008 | xe->info.step.basedie = probed_info->step.basedie; | ||
| 1009 | xe->info.step.graphics = probed_info->step.graphics; | ||
| 1010 | xe->info.step.media = probed_info->step.media; | ||
| 1011 | |||
| 960 | xe->info.graphics_verx100 = graphics_ip->verx100; | 1012 | xe->info.graphics_verx100 = graphics_ip->verx100; |
| 961 | xe->info.graphics_name = graphics_ip->name; | 1013 | xe->info.graphics_name = graphics_ip->name; |
| 962 | graphics_desc = graphics_ip->desc; | 1014 | graphics_desc = graphics_ip->desc; |
| @@ -988,8 +1040,6 @@ static int xe_info_init(struct xe_device *xe, · 25 unmodified lines | |||
| 988 | xe->info.has_soc_remapper_telem = 0; | 1040 | xe->info.has_soc_remapper_telem = 0; |
| 989 | } | 1041 | } |
| 990 | 1042 | ||
| 991 | xe_info_probe_tile_count(xe); | ||
| 992 | |||
| 993 | for_each_remote_tile(tile, xe, id) { | 1043 | for_each_remote_tile(tile, xe, id) { |
| 994 | int err; | 1044 | int err; |
| 995 | 1045 | ||
| @@ -1072,9 +1122,11 @@ static void xe_pci_remove(struct pci_dev *pdev) · 76 unmodified lines | |||
| 1072 | */ | 1122 | */ |
| 1073 | static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) | 1123 | static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) |
| 1074 | { | 1124 | { |
| 1125 | struct xe_probed_info probed_info = {}; | ||
| 1075 | const struct xe_device_desc *desc = (const void *)ent->driver_data; | 1126 | const struct xe_device_desc *desc = (const void *)ent->driver_data; |
| 1076 | const struct xe_subplatform_desc *subplatform_desc; | 1127 | const struct xe_subplatform_desc *subplatform_desc; |
| 1077 | struct xe_device *xe; | 1128 | struct xe_device *xe; |
| 1129 | void *group; | ||
| 1078 | int err; | 1130 | int err; |
| 1079 | 1131 | ||
| 1080 | subplatform_desc = find_subplatform(desc, pdev->device); | 1132 | subplatform_desc = find_subplatform(desc, pdev->device); |
| @@ -1102,6 +1154,11 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 21 unmodified lines | |||
| 1102 | if (xe_display_driver_probe_defer(pdev)) | 1154 | if (xe_display_driver_probe_defer(pdev)) |
| 1103 | return -EPROBE_DEFER; | 1155 | return -EPROBE_DEFER; |
| 1104 | 1156 | ||
| 1157 | /* Group all devres so xe_pci_error_slot_reset() can release them as a unit. */ | ||
| 1158 | group = devres_open_group(&pdev->dev, NULL, GFP_KERNEL); | ||
| 1159 | if (!group) | ||
| 1160 | return -ENOMEM; | ||
| 1161 | |||
| 1105 | err = pcim_enable_device(pdev); | 1162 | err = pcim_enable_device(pdev); |
| 1106 | if (err) | 1163 | if (err) |
| 1107 | return err; | 1164 | return err; |
| @@ -1110,13 +1167,19 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 2 unmodified lines | |||
| 1110 | if (IS_ERR(xe)) | 1167 | if (IS_ERR(xe)) |
| 1111 | return PTR_ERR(xe); | 1168 | return PTR_ERR(xe); |
| 1112 | 1169 | ||
| 1170 | xe->devres_group = group; | ||
| 1171 | |||
| 1113 | pci_set_drvdata(pdev, &xe->drm); | 1172 | pci_set_drvdata(pdev, &xe->drm); |
| 1114 | 1173 | ||
| 1115 | xe_pm_assert_unbounded_bridge(xe); | 1174 | xe_pm_assert_unbounded_bridge(xe); |
| 1116 | 1175 | ||
| 1117 | pci_set_master(pdev); | 1176 | pci_set_master(pdev); |
| 1118 | 1177 | ||
| 1119 | err = xe_info_init_early(xe, desc, subplatform_desc); | 1178 | err = xe_probe_info_early(xe, desc, &probed_info); |
| 1179 | if (err) | ||
| 1180 | return err; | ||
| 1181 | |||
| 1182 | err = xe_info_init_early(xe, desc, subplatform_desc, &probed_info); | ||
| 1120 | if (err) | 1183 | if (err) |
| 1121 | return err; | 1184 | return err; |
| 1122 | 1185 | ||
| @@ -1135,7 +1198,11 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 12 unmodified lines | |||
| 1135 | if (err) | 1198 | if (err) |
| 1136 | return err; | 1199 | return err; |
| 1137 | 1200 | ||
| 1138 | err = xe_info_init(xe, desc); | 1201 | err = xe_probe_info(xe, desc, &probed_info); |
| 1202 | if (err) | ||
| 1203 | return err; | ||
| 1204 | |||
| 1205 | err = xe_info_init(xe, desc, &probed_info); | ||
| 1139 | if (err) | 1206 | if (err) |
| 1140 | return err; | 1207 | return err; |
| 1141 | 1208 | ||
| @@ -1348,6 +1415,7 @@ static struct pci_driver xe_pci_driver = { · 206 unmodified lines | |||
| 1348 | .remove = xe_pci_remove, | 1415 | .remove = xe_pci_remove, |
| 1349 | .shutdown = xe_pci_shutdown, | 1416 | .shutdown = xe_pci_shutdown, |
| 1350 | .sriov_configure = xe_pci_sriov_configure, | 1417 | .sriov_configure = xe_pci_sriov_configure, |
| 1418 | .err_handler = &xe_pci_error_handlers, | ||
| 1351 | #ifdef CONFIG_PM_SLEEP | 1419 | #ifdef CONFIG_PM_SLEEP |
| 1352 | .driver.pm = &xe_pm_ops, | 1420 | .driver.pm = &xe_pm_ops, |
| 1353 | #endif | 1421 | #endif |
| @@ -0,0 +1,122 @@ | |||
| 1 | // SPDX-License-Identifier: MIT | ||
| 2 | /* | ||
| 3 | * Copyright © 2026 Intel Corporation | ||
| 4 | */ | ||
| 5 | |||
| 6 | #include <linux/pci.h> | ||
| 7 | |||
| 8 | #include "xe_device.h" | ||
| 9 | #include "xe_gt.h" | ||
| 10 | #include "xe_pci.h" | ||
| 11 | #include "xe_pm.h" | ||
| 12 | #include "xe_printk.h" | ||
| 13 | #include "xe_survivability_mode.h" | ||
| 14 | |||
| 15 | static void prepare_device_for_reset(struct pci_dev *pdev) | ||
| 16 | { | ||
| 17 | struct xe_device *xe = pdev_to_xe_device(pdev); | ||
| 18 | struct xe_gt *gt; | ||
| 19 | u8 id; | ||
| 20 | |||
| 21 | /* | ||
| 22 | * Wedge the device to prevent userspace access but do not send the uevent. | ||
| 23 | * xe_device_wedged_fini() releases runtime pm if wedged flag is set, so acquire a runtime | ||
| 24 | * pm reference to avoid underflow. | ||
| 25 | */ | ||
| 26 | if (!atomic_xchg(&xe->wedged.flag, 1)) | ||
| 27 | xe_pm_runtime_get_noresume(xe); | ||
| 28 | |||
| 29 | xe_device_set_in_reset(xe); | ||
| 30 | |||
| 31 | for_each_gt(gt, xe, id) | ||
| 32 | xe_gt_declare_wedged(gt); | ||
| 33 | |||
| 34 | pci_disable_device(pdev); | ||
| 35 | } | ||
| 36 | |||
| 37 | static pci_ers_result_t xe_pci_error_detected(struct pci_dev *pdev, pci_channel_state_t state) | ||
| 38 | { | ||
| 39 | struct xe_device *xe = pdev_to_xe_device(pdev); | ||
| 40 | |||
| 41 | xe_info(xe, "PCI error: detected state = %d\n", state); | ||
| 42 | |||
| 43 | if (state == pci_channel_io_perm_failure) | ||
| 44 | return PCI_ERS_RESULT_DISCONNECT; | ||
| 45 | |||
| 46 | /* If the device is already wedged or in survivability mode, do not attempt recovery */ | ||
| 47 | if (xe_survivability_mode_is_boot_enabled(xe) || xe_device_wedged(xe)) | ||
| 48 | return PCI_ERS_RESULT_DISCONNECT; | ||
| 49 | |||
| 50 | switch (state) { | ||
| 51 | case pci_channel_io_normal: | ||
| 52 | return PCI_ERS_RESULT_CAN_RECOVER; | ||
| 53 | case pci_channel_io_frozen: | ||
| 54 | prepare_device_for_reset(pdev); | ||
| 55 | return PCI_ERS_RESULT_NEED_RESET; | ||
| 56 | default: | ||
| 57 | xe_info(xe, "PCI error: unknown state %d\n", state); | ||
| 58 | return PCI_ERS_RESULT_DISCONNECT; | ||
| 59 | } | ||
| 60 | } | ||
| 61 | |||
| 62 | static pci_ers_result_t xe_pci_error_mmio_enabled(struct pci_dev *pdev) | ||
| 63 | { | ||
| 64 | struct xe_device *xe = pdev_to_xe_device(pdev); | ||
| 65 | |||
| 66 | xe_info(xe, "PCI error: MMIO enabled\n"); | ||
| 67 | |||
| 68 | /* TODO: Query system controller for the type of error and take appropriate action */ | ||
| 69 | return PCI_ERS_RESULT_RECOVERED; | ||
| 70 | } | ||
| 71 | |||
| 72 | static pci_ers_result_t xe_pci_error_slot_reset(struct pci_dev *pdev) | ||
| 73 | { | ||
| 74 | const struct pci_device_id *ent = pci_match_id(pdev->driver->id_table, pdev); | ||
| 75 | struct xe_device *xe = pdev_to_xe_device(pdev); | ||
| 76 | |||
| 77 | xe_info(xe, "PCI error: slot reset\n"); | ||
| 78 | |||
| 79 | pci_restore_state(pdev); | ||
| 80 | |||
| 81 | if (pci_enable_device(pdev)) { | ||
| 82 | xe_err(xe, "Cannot re-enable PCI device after reset\n"); | ||
| 83 | return PCI_ERS_RESULT_DISCONNECT; | ||
| 84 | } | ||
| 85 | |||
| 86 | /* | ||
| 87 | * Secondary Bus Reset causes all VRAM state to be lost along with | ||
| 88 | * hardware state. As an initial step, re-probe the device to | ||
| 89 | * re-initialize the driver and hardware. | ||
| 90 | * TODO: optimize by re-initializing only the hardware state and re-creating | ||
| 91 | * kernel BOs. | ||
| 92 | */ | ||
| 93 | xe_device_clear_in_reset(xe); | ||
| 94 | pdev->driver->remove(pdev); | ||
| 95 | devres_release_group(&pdev->dev, xe->devres_group); | ||
| 96 | |||
| 97 | if (pdev->driver->probe(pdev, ent)) | ||
| 98 | return PCI_ERS_RESULT_DISCONNECT; | ||
| 99 | |||
| 100 | xe = pdev_to_xe_device(pdev); | ||
| 101 | |||
| 102 | /* Wedge the device to prevent I/O operations till the resume callback */ | ||
| 103 | atomic_set(&xe->wedged.flag, 1); | ||
| 104 | |||
| 105 | return PCI_ERS_RESULT_RECOVERED; | ||
| 106 | } | ||
| 107 | |||
| 108 | static void xe_pci_error_resume(struct pci_dev *pdev) | ||
| 109 | { | ||
| 110 | struct xe_device *xe = pdev_to_xe_device(pdev); | ||
| 111 | |||
| 112 | xe_info(xe, "PCI error: resume\n"); | ||
| 113 | |||
| 114 | atomic_set(&xe->wedged.flag, 0); | ||
| 115 | } | ||
| 116 | |||
| 117 | const struct pci_error_handlers xe_pci_error_handlers = { | ||
| 118 | .error_detected = xe_pci_error_detected, | ||
| 119 | .mmio_enabled = xe_pci_error_mmio_enabled, | ||
| 120 | .slot_reset = xe_pci_error_slot_reset, | ||
| 121 | .resume = xe_pci_error_resume, | ||
| 122 | }; | ||
| @@ -0,0 +1,13 @@ | |||
| 1 | /* SPDX-License-Identifier: MIT */ | ||
| 2 | /* | ||
| 3 | * Copyright © 2026 Intel Corporation | ||
| 4 | */ | ||
| 5 | |||
| 6 | #ifndef _XE_PCI_ERROR_H_ | ||
| 7 | #define _XE_PCI_ERROR_H_ | ||
| 8 | |||
| 9 | struct pci_error_handlers; | ||
| 10 | |||
| 11 | extern const struct pci_error_handlers xe_pci_error_handlers; | ||
| 12 | |||
| 13 | #endif | ||
| @@ -40,6 +40,7 @@ struct xe_device_desc { · 39 unmodified lines | |||
| 40 | 40 | ||
| 41 | u8 has_cached_pt:1; | 41 | u8 has_cached_pt:1; |
| 42 | u8 has_display:1; | 42 | u8 has_display:1; |
| 43 | u8 has_drm_ras:1; | ||
| 43 | u8 has_fan_control:1; | 44 | u8 has_fan_control:1; |
| 44 | u8 has_flat_ccs:1; | 45 | u8 has_flat_ccs:1; |
| 45 | u8 has_gsc_nvm:1; | 46 | u8 has_gsc_nvm:1; |
| @@ -323,15 +323,17 @@ int xe_pcode_ready(struct xe_device *xe, bool locked) · 322 unmodified lines | |||
| 323 | } | 323 | } |
| 324 | 324 | ||
| 325 | /** | 325 | /** |
| 326 | * xe_pcode_init: initialize components of PCODE | 326 | * xe_pcode_init_early() - Initialize components of PCODE |
| 327 | * @tile: tile instance | 327 | * @tile: tile instance |
| 328 | * | 328 | * |
| 329 | * This function initializes the xe_pcode component. | 329 | * This function initializes the xe_pcode component. |
| 330 | * To be called once only during probe. | 330 | * To be called once only during probe. |
| 331 | * | ||
| 332 | * Return: 0 on success or a negative error code on failure. | ||
| 331 | */ | 333 | */ |
| 332 | void xe_pcode_init(struct xe_tile *tile) | 334 | int xe_pcode_init_early(struct xe_tile *tile) |
| 333 | { | 335 | { |
| 334 | drmm_mutex_init(&tile_to_xe(tile)->drm, &tile->pcode.lock); | 336 | return drmm_mutex_init(&tile_to_xe(tile)->drm, &tile->pcode.lock); |
| 335 | } | 337 | } |
| 336 | 338 | ||
| 337 | /** | 339 | /** |
| @@ -12,7 +12,7 @@ struct drm_device; · 11 unmodified lines | |||
| 12 | struct xe_device; | 12 | struct xe_device; |
| 13 | struct xe_tile; | 13 | struct xe_tile; |
| 14 | 14 | ||
| 15 | void xe_pcode_init(struct xe_tile *tile); | 15 | int xe_pcode_init_early(struct xe_tile *tile); |
| 16 | int xe_pcode_probe_early(struct xe_device *xe); | 16 | int xe_pcode_probe_early(struct xe_device *xe); |
| 17 | int xe_pcode_ready(struct xe_device *xe, bool locked); | 17 | int xe_pcode_ready(struct xe_device *xe, bool locked); |
| 18 | int xe_pcode_init_min_freq_table(struct xe_tile *tile, u32 min_gt_freq, | 18 | int xe_pcode_init_min_freq_table(struct xe_tile *tile, u32 min_gt_freq, |
| @@ -1026,12 +1026,22 @@ xe_vm_populate_pgtable(struct xe_migrate_pt_update *pt_update, struct xe_tile *t · 1025 unmodified lines | |||
| 1026 | u64 *ptr = data; | 1026 | u64 *ptr = data; |
| 1027 | u32 i; | 1027 | u32 i; |
| 1028 | 1028 | ||
| 1029 | /* | ||
| 1030 | * @qword_ofs is the absolute entry offset within the page table, while | ||
| 1031 | * @ptes is indexed relative to @update->ofs (its first entry). The GPU | ||
| 1032 | * path (write_pgtable) splits a single update into MAX_PTE_PER_SDI-sized | ||
| 1033 | * chunks, calling this with an advancing @qword_ofs but a fresh @data | ||
| 1034 | * pointer per chunk, so translate back into a @ptes index rather than | ||
| 1035 | * assuming the chunk starts at ptes[0]. | ||
| 1036 | */ | ||
| 1029 | for (i = 0; i < num_qwords; i++) { | 1037 | for (i = 0; i < num_qwords; i++) { |
| 1038 | u32 idx = qword_ofs - update->ofs + i; | ||
| 1039 | |||
| 1030 | if (map) | 1040 | if (map) |
| 1031 | xe_map_wr(tile_to_xe(tile), map, (qword_ofs + i) * | 1041 | xe_map_wr(tile_to_xe(tile), map, (qword_ofs + i) * |
| 1032 | sizeof(u64), u64, ptes[i].pte); | 1042 | sizeof(u64), u64, ptes[idx].pte); |
| 1033 | else | 1043 | else |
| 1034 | ptr[i] = ptes[i].pte; | 1044 | ptr[i] = ptes[idx].pte; |
| 1035 | } | 1045 | } |
| 1036 | } | 1046 | } |
| 1037 | 1047 | ||
| @@ -2070,6 +2080,9 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile, · 1032 unmodified lines | |||
| 2070 | * automatically when the context is re-enabled by the rebind worker, | 2080 | * automatically when the context is re-enabled by the rebind worker, |
| 2071 | * or in fault mode it was invalidated on PTE zapping. | 2081 | * or in fault mode it was invalidated on PTE zapping. |
| 2072 | * | 2082 | * |
| 2083 | * If rebind, we have to invalidate TLB on context based TLB invalidation | ||
| 2084 | * LR vms, as they cannot be relied on context re-enable. | ||
| 2085 | * | ||
| 2073 | * If !rebind, and scratch enabled VMs, there is a chance the scratch | 2086 | * If !rebind, and scratch enabled VMs, there is a chance the scratch |
| 2074 | * PTE is already cached in the TLB so it needs to be invalidated. | 2087 | * PTE is already cached in the TLB so it needs to be invalidated. |
| 2075 | * On !LR VMs this is done in the ring ops preceding a batch, but on | 2088 | * On !LR VMs this is done in the ring ops preceding a batch, but on |
| @@ -2079,6 +2092,9 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile, · 3 unmodified lines | |||
| 2079 | if ((!pt_op->rebind && xe_vm_has_scratch(vm) && | 2092 | if ((!pt_op->rebind && xe_vm_has_scratch(vm) && |
| 2080 | xe_vm_in_lr_mode(vm))) | 2093 | xe_vm_in_lr_mode(vm))) |
| 2081 | pt_update_ops->needs_invalidation = true; | 2094 | pt_update_ops->needs_invalidation = true; |
| 2095 | else if (pt_op->rebind && xe_vm_in_preempt_fence_mode(vm) && | ||
| 2096 | vm->xe->info.has_ctx_tlb_inval) | ||
| 2097 | pt_update_ops->needs_invalidation = true; | ||
| 2082 | else if (pt_op->rebind && !xe_vm_in_lr_mode(vm)) | 2098 | else if (pt_op->rebind && !xe_vm_in_lr_mode(vm)) |
| 2083 | /* We bump also if batch_invalidate_tlb is true */ | 2099 | /* We bump also if batch_invalidate_tlb is true */ |
| 2084 | vm->tlb_flush_seqno++; | 2100 | vm->tlb_flush_seqno++; |
| @@ -59,6 +59,7 @@ bool xe_pxp_is_enabled(const struct xe_pxp *pxp) · 58 unmodified lines | |||
| 59 | static bool pxp_prerequisites_done(const struct xe_pxp *pxp) | 59 | static bool pxp_prerequisites_done(const struct xe_pxp *pxp) |
| 60 | { | 60 | { |
| 61 | struct xe_gt *gt = pxp->gt; | 61 | struct xe_gt *gt = pxp->gt; |
| 62 | bool huc_ok; | ||
| 62 | bool ready; | 63 | bool ready; |
| 63 | 64 | ||
| 64 | CLASS(xe_force_wake, fw_ref)(gt_to_fw(gt), XE_FORCEWAKE_ALL); | 65 | CLASS(xe_force_wake, fw_ref)(gt_to_fw(gt), XE_FORCEWAKE_ALL); |
| @@ -73,9 +74,14 @@ static bool pxp_prerequisites_done(const struct xe_pxp *pxp) · 8 unmodified lines | |||
| 73 | */ | 74 | */ |
| 74 | XE_WARN_ON(!xe_force_wake_ref_has_domain(fw_ref.domains, XE_FORCEWAKE_ALL)); | 75 | XE_WARN_ON(!xe_force_wake_ref_has_domain(fw_ref.domains, XE_FORCEWAKE_ALL)); |
| 75 | 76 | ||
| 76 | /* PXP requires both HuC authentication via GSC and GSC proxy initialized */ | 77 | /* |
| 77 | ready = xe_huc_is_authenticated(>->uc.huc, XE_HUC_AUTH_VIA_GSC) && | 78 | * PXP requires GSC proxy to be initialized. On platforms where the HuC |
| 78 | xe_gsc_proxy_init_done(>->uc.gsc); | 79 | * is loaded by the kernel driver (i.e., pre media 35) PXP also requires |
| 80 | * the HuC to be authenticated by GSC. | ||
| 81 | */ | ||
| 82 | huc_ok = MEDIA_VER(gt_to_xe(gt)) >= 35 || | ||
| 83 | xe_huc_is_authenticated(>->uc.huc, XE_HUC_AUTH_VIA_GSC); | ||
| 84 | ready = huc_ok && xe_gsc_proxy_init_done(>->uc.gsc); | ||
| 79 | 85 | ||
| 80 | return ready; | 86 | return ready; |
| 81 | } | 87 | } |
| @@ -97,9 +103,13 @@ int xe_pxp_get_readiness_status(struct xe_pxp *pxp) · 15 unmodified lines | |||
| 97 | if (!xe_pxp_is_enabled(pxp)) | 103 | if (!xe_pxp_is_enabled(pxp)) |
| 98 | return -ENODEV; | 104 | return -ENODEV; |
| 99 | 105 | ||
| 100 | /* if the GSC or HuC FW are in an error state, PXP will never work */ | 106 | /* If the GSC FW is in an error state, PXP will never work */ |
| 101 | if (xe_uc_fw_status_to_error(pxp->gt->uc.huc.fw.status) || | 107 | if (xe_uc_fw_status_to_error(pxp->gt->uc.gsc.fw.status)) |
| 102 | xe_uc_fw_status_to_error(pxp->gt->uc.gsc.fw.status)) | 108 | return -EIO; |
| 109 | |||
| 110 | /* Same for HuC FW, but only if the kernel owns HuC-loading (i.e. pre-NVL) */ | ||
| 111 | if (MEDIA_VER(gt_to_xe(pxp->gt)) < 35 && | ||
| 112 | xe_uc_fw_status_to_error(pxp->gt->uc.huc.fw.status)) | ||
| 103 | return -EIO; | 113 | return -EIO; |
| 104 | 114 | ||
| 105 | guard(xe_pm_runtime)(pxp->xe); | 115 | guard(xe_pm_runtime)(pxp->xe); |
| @@ -361,6 +371,7 @@ static void pxp_fini(void *arg) · 255 unmodified lines | |||
| 361 | int xe_pxp_init(struct xe_device *xe) | 371 | int xe_pxp_init(struct xe_device *xe) |
| 362 | { | 372 | { |
| 363 | struct xe_gt *gt = xe->tiles[0].media_gt; | 373 | struct xe_gt *gt = xe->tiles[0].media_gt; |
| 374 | bool gsc_ok, huc_ok; | ||
| 364 | struct xe_pxp *pxp; | 375 | struct xe_pxp *pxp; |
| 365 | int err; | 376 | int err; |
| 366 | 377 | ||
| @@ -375,10 +386,14 @@ int xe_pxp_init(struct xe_device *xe) · 8 unmodified lines | |||
| 375 | if (!(gt->info.engine_mask & BIT(XE_HW_ENGINE_GSCCS0))) | 386 | if (!(gt->info.engine_mask & BIT(XE_HW_ENGINE_GSCCS0))) |
| 376 | return 0; | 387 | return 0; |
| 377 | 388 | ||
| 378 | /* PXP requires both GSC and HuC firmwares to be available */ | 389 | /* PXP requires GSC FW to be available. Pre-NVL it also requires HuC FW */ |
| 379 | if (!xe_uc_fw_is_loadable(>->uc.gsc.fw) || | 390 | gsc_ok = xe_uc_fw_is_loadable(>->uc.gsc.fw); |
| 380 | !xe_uc_fw_is_loadable(>->uc.huc.fw)) { | 391 | huc_ok = MEDIA_VER(xe) >= 35 || xe_uc_fw_is_loadable(>->uc.huc.fw); |
| 381 | drm_info(&xe->drm, "skipping PXP init due to missing FW dependencies"); | 392 | |
| 393 | if (!gsc_ok || !huc_ok) { | ||
| 394 | drm_info(&xe->drm, "Skipping PXP due to unsatisfied FW deps - GSC=%s, HuC=%s\n", | ||
| 395 | str_yes_no(gsc_ok), | ||
| 396 | MEDIA_VER(xe) >= 35 ? "not needed" : str_yes_no(huc_ok)); | ||
| 382 | return 0; | 397 | return 0; |
| 383 | } | 398 | } |
| 384 | 399 | ||
| @@ -119,6 +119,7 @@ query_engine_cycles(struct xe_device *xe, · 118 unmodified lines | |||
| 119 | struct drm_xe_engine_class_instance *eci; | 119 | struct drm_xe_engine_class_instance *eci; |
| 120 | struct drm_xe_query_engine_cycles resp; | 120 | struct drm_xe_query_engine_cycles resp; |
| 121 | size_t size = sizeof(resp); | 121 | size_t size = sizeof(resp); |
| 122 | enum xe_force_wake_domains fw_domain; | ||
| 122 | __ktime_func_t cpu_clock; | 123 | __ktime_func_t cpu_clock; |
| 123 | struct xe_hw_engine *hwe; | 124 | struct xe_hw_engine *hwe; |
| 124 | struct xe_gt *gt; | 125 | struct xe_gt *gt; |
| @@ -154,8 +155,10 @@ query_engine_cycles(struct xe_device *xe, · 29 unmodified lines | |||
| 154 | if (!hwe) | 155 | if (!hwe) |
| 155 | return -EINVAL; | 156 | return -EINVAL; |
| 156 | 157 | ||
| 157 | xe_with_force_wake(fw_ref, gt_to_fw(gt), XE_FORCEWAKE_ALL) { | 158 | fw_domain = xe_hw_engine_to_fw_domain(hwe); |
| 158 | if (!xe_force_wake_ref_has_domain(fw_ref.domains, XE_FORCEWAKE_ALL)) | 159 | |
| 160 | xe_with_force_wake(fw_ref, gt_to_fw(gt), fw_domain) { | ||
| 161 | if (!xe_force_wake_ref_has_domain(fw_ref.domains, fw_domain)) | ||
| 159 | return -EIO; | 162 | return -EIO; |
| 160 | 163 | ||
| 161 | hwe_read_timestamp(hwe, &resp.engine_cycles, &resp.cpu_timestamp, | 164 | hwe_read_timestamp(hwe, &resp.engine_cycles, &resp.cpu_timestamp, |
| @@ -4,11 +4,15 @@ · 3 unmodified lines | |||
| 4 | */ | 4 | */ |
| 5 | 5 | ||
| 6 | #include "xe_device.h" | 6 | #include "xe_device.h" |
| 7 | #include "xe_drm_ras.h" | ||
| 8 | #include "xe_pm.h" | ||
| 7 | #include "xe_printk.h" | 9 | #include "xe_printk.h" |
| 8 | #include "xe_ras.h" | 10 | #include "xe_ras.h" |
| 9 | #include "xe_ras_types.h" | 11 | #include "xe_ras_types.h" |
| 10 | #include "xe_sysctrl.h" | 12 | #include "xe_sysctrl.h" |
| 11 | #include "xe_sysctrl_event_types.h" | 13 | #include "xe_sysctrl_event_types.h" |
| 14 | #include "xe_sysctrl_mailbox.h" | ||
| 15 | #include "xe_sysctrl_mailbox_types.h" | ||
| 12 | 16 | ||
| 13 | /* Severity of detected errors */ | 17 | /* Severity of detected errors */ |
| 14 | enum xe_ras_severity { | 18 | enum xe_ras_severity { |
| @@ -31,6 +35,17 @@ enum xe_ras_component { · 16 unmodified lines | |||
| 31 | XE_RAS_COMP_MAX | 35 | XE_RAS_COMP_MAX |
| 32 | }; | 36 | }; |
| 33 | 37 | ||
| 38 | /* RAS response status codes */ | ||
| 39 | enum xe_ras_response_status { | ||
| 40 | XE_RAS_STATUS_SUCCESS = 0, | ||
| 41 | XE_RAS_STATUS_INVALID_PARAM, | ||
| 42 | XE_RAS_STATUS_OP_NOT_SUPPORTED, | ||
| 43 | XE_RAS_STATUS_TIMEOUT, | ||
| 44 | XE_RAS_STATUS_HARDWARE_FAILURE, | ||
| 45 | XE_RAS_STATUS_INSUFFICIENT_RESOURCES, | ||
| 46 | XE_RAS_STATUS_MAX | ||
| 47 | }; | ||
| 48 | |||
| 34 | static const char *const xe_ras_severities[] = { | 49 | static const char *const xe_ras_severities[] = { |
| 35 | [XE_RAS_SEV_NOT_SUPPORTED] = "Not Supported", | 50 | [XE_RAS_SEV_NOT_SUPPORTED] = "Not Supported", |
| 36 | [XE_RAS_SEV_CORRECTABLE] = "Correctable Error", | 51 | [XE_RAS_SEV_CORRECTABLE] = "Correctable Error", |
| @@ -50,6 +65,56 @@ static const char *const xe_ras_components[] = { · 13 unmodified lines | |||
| 50 | }; | 65 | }; |
| 51 | static_assert(ARRAY_SIZE(xe_ras_components) == XE_RAS_COMP_MAX); | 66 | static_assert(ARRAY_SIZE(xe_ras_components) == XE_RAS_COMP_MAX); |
| 52 | 67 | ||
| 68 | static u8 drm_to_xe_ras_severity(u8 severity) | ||
| 69 | { | ||
| 70 | switch (severity) { | ||
| 71 | case DRM_XE_RAS_ERR_SEV_CORRECTABLE: | ||
| 72 | return XE_RAS_SEV_CORRECTABLE; | ||
| 73 | case DRM_XE_RAS_ERR_SEV_UNCORRECTABLE: | ||
| 74 | return XE_RAS_SEV_UNCORRECTABLE; | ||
| 75 | default: | ||
| 76 | return XE_RAS_SEV_NOT_SUPPORTED; | ||
| 77 | } | ||
| 78 | } | ||
| 79 | |||
| 80 | static u8 drm_to_xe_ras_component(u8 component) | ||
| 81 | { | ||
| 82 | switch (component) { | ||
| 83 | case DRM_XE_RAS_ERR_COMP_CORE_COMPUTE: | ||
| 84 | return XE_RAS_COMP_CORE_COMPUTE; | ||
| 85 | case DRM_XE_RAS_ERR_COMP_SOC_INTERNAL: | ||
| 86 | return XE_RAS_COMP_SOC_INTERNAL; | ||
| 87 | case DRM_XE_RAS_ERR_COMP_DEVICE_MEMORY: | ||
| 88 | return XE_RAS_COMP_DEVICE_MEMORY; | ||
| 89 | case DRM_XE_RAS_ERR_COMP_PCIE: | ||
| 90 | return XE_RAS_COMP_PCIE; | ||
| 91 | case DRM_XE_RAS_ERR_COMP_FABRIC: | ||
| 92 | return XE_RAS_COMP_FABRIC; | ||
| 93 | default: | ||
| 94 | return XE_RAS_COMP_NOT_SUPPORTED; | ||
| 95 | } | ||
| 96 | } | ||
| 97 | |||
| 98 | static int ras_status_to_errno(u32 status) | ||
| 99 | { | ||
| 100 | switch (status) { | ||
| 101 | case XE_RAS_STATUS_SUCCESS: | ||
| 102 | return 0; | ||
| 103 | case XE_RAS_STATUS_INVALID_PARAM: | ||
| 104 | return -EINVAL; | ||
| 105 | case XE_RAS_STATUS_OP_NOT_SUPPORTED: | ||
| 106 | return -EOPNOTSUPP; | ||
| 107 | case XE_RAS_STATUS_TIMEOUT: | ||
| 108 | return -ETIMEDOUT; | ||
| 109 | case XE_RAS_STATUS_HARDWARE_FAILURE: | ||
| 110 | return -EIO; | ||
| 111 | case XE_RAS_STATUS_INSUFFICIENT_RESOURCES: | ||
| 112 | return -ENOSPC; | ||
| 113 | default: | ||
| 114 | return -EPROTO; | ||
| 115 | } | ||
| 116 | } | ||
| 117 | |||
| 53 | static inline const char *sev_to_str(u8 severity) | 118 | static inline const char *sev_to_str(u8 severity) |
| 54 | { | 119 | { |
| 55 | if (severity >= XE_RAS_SEV_MAX) | 120 | if (severity >= XE_RAS_SEV_MAX) |
| @@ -66,6 +131,68 @@ static inline const char *comp_to_str(u8 component) · 10 unmodified lines | |||
| 66 | return xe_ras_components[component]; | 131 | return xe_ras_components[component]; |
| 67 | } | 132 | } |
| 68 | 133 | ||
| 134 | static struct pci_dev *find_usp_dev(struct pci_dev *pdev) | ||
| 135 | { | ||
| 136 | struct pci_dev *vsp; | ||
| 137 | |||
| 138 | /* | ||
| 139 | * Device Hierarchy: | ||
| 140 | * | ||
| 141 | * Upstream Switch Port (USP) --> Virtual Switch Port (VSP) --> SGunit (GPU endpoint) | ||
| 142 | */ | ||
| 143 | vsp = pci_upstream_bridge(pdev); | ||
| 144 | if (!vsp) | ||
| 145 | return NULL; | ||
| 146 | |||
| 147 | return pci_upstream_bridge(vsp); | ||
| 148 | } | ||
| 149 | |||
| 150 | static void ras_usp_aer_init(struct xe_device *xe) | ||
| 151 | { | ||
| 152 | struct pci_dev *pdev = to_pci_dev(xe->drm.dev); | ||
| 153 | struct pci_dev *usp; | ||
| 154 | u16 aer_cap; | ||
| 155 | u32 status; | ||
| 156 | |||
| 157 | usp = find_usp_dev(pdev); | ||
| 158 | if (!usp) | ||
| 159 | return; | ||
| 160 | |||
| 161 | aer_cap = pci_find_ext_capability(usp, PCI_EXT_CAP_ID_ERR); | ||
| 162 | if (!aer_cap) { | ||
| 163 | dev_warn(&usp->dev, "AER capability unavailable\n"); | ||
| 164 | return; | ||
| 165 | } | ||
| 166 | |||
| 167 | /* | ||
| 168 | * Clear any stale Uncorrectable Internal Error Status event in Uncorrectable Error | ||
| 169 | * Status Register. | ||
| 170 | */ | ||
| 171 | pci_read_config_dword(usp, aer_cap + PCI_ERR_UNCOR_STATUS, &status); | ||
| 172 | if (status & PCI_ERR_UNC_INTN) | ||
| 173 | pci_write_config_dword(usp, aer_cap + PCI_ERR_UNCOR_STATUS, PCI_ERR_UNC_INTN); | ||
| 174 | |||
| 175 | /* | ||
| 176 | * All errors are steered to USP which is a PCIe AER Compliant device. | ||
| 177 | * Downgrade all the errors to non-fatal to prevent PCIe bus driver | ||
| 178 | * from triggering a Secondary Bus Reset (SBR). This allows error | ||
| 179 | * detection, containment and recovery in the driver. | ||
| 180 | * | ||
| 181 | * The Uncorrectable Error Severity Register has the 'Uncorrectable | ||
| 182 | * Internal Error Severity' set to fatal by default. Set this to | ||
| 183 | * non-fatal and unmask the error. | ||
| 184 | */ | ||
| 185 | |||
| 186 | /* Downgrade Uncorrectable Internal Error to non-fatal */ | ||
| 187 | pci_clear_and_set_config_dword(usp, aer_cap + PCI_ERR_UNCOR_SEVER, PCI_ERR_UNC_INTN, 0); | ||
| 188 | |||
| 189 | /* Unmask Uncorrectable Internal Error */ | ||
| 190 | pci_clear_and_set_config_dword(usp, aer_cap + PCI_ERR_UNCOR_MASK, PCI_ERR_UNC_INTN, 0); | ||
| 191 | |||
| 192 | pci_save_state(usp); | ||
| 193 | dev_dbg(&usp->dev, "Uncorrectable Internal Errors downgraded and unmasked\n"); | ||
| 194 | } | ||
| 195 | |||
| 69 | void xe_ras_counter_threshold_crossed(struct xe_device *xe, | 196 | void xe_ras_counter_threshold_crossed(struct xe_device *xe, |
| 70 | struct xe_sysctrl_event_response *response) | 197 | struct xe_sysctrl_event_response *response) |
| 71 | { | 198 | { |
| @@ -91,3 +218,136 @@ void xe_ras_counter_threshold_crossed(struct xe_device *xe, · 19 unmodified lines | |||
| 91 | comp_to_str(component), sev_to_str(severity)); | 218 | comp_to_str(component), sev_to_str(severity)); |
| 92 | } | 219 | } |
| 93 | } | 220 | } |
| 221 | |||
| 222 | static int get_counter(struct xe_device *xe, struct xe_ras_error_class *counter, u32 *value) | ||
| 223 | { | ||
| 224 | struct xe_ras_get_counter_response response = {0}; | ||
| 225 | struct xe_ras_get_counter_request request = {0}; | ||
| 226 | struct xe_sysctrl_mailbox_command command = {0}; | ||
| 227 | struct xe_ras_error_common *common; | ||
| 228 | size_t rlen; | ||
| 229 | int ret; | ||
| 230 | |||
| 231 | request.counter = *counter; | ||
| 232 | |||
| 233 | xe_sysctrl_create_command(&command, XE_SYSCTRL_GROUP_GFSP, XE_SYSCTRL_CMD_GET_COUNTER, | ||
| 234 | &request, sizeof(request), &response, sizeof(response)); | ||
| 235 | |||
| 236 | ret = xe_sysctrl_send_command(&xe->sc, &command, &rlen); | ||
| 237 | if (ret) { | ||
| 238 | xe_err(xe, "sysctrl: failed to get counter %d\n", ret); | ||
| 239 | return ret; | ||
| 240 | } | ||
| 241 | |||
| 242 | if (rlen != sizeof(response)) { | ||
| 243 | xe_err(xe, "sysctrl: unexpected get counter response length %zu (expected %zu)\n", | ||
| 244 | rlen, sizeof(response)); | ||
| 245 | return -EIO; | ||
| 246 | } | ||
| 247 | |||
| 248 | common = &response.counter.common; | ||
| 249 | *value = response.value; | ||
| 250 | |||
| 251 | xe_dbg(xe, "[RAS]: get counter %u for %s %s\n", *value, comp_to_str(common->component), | ||
| 252 | sev_to_str(common->severity)); | ||
| 253 | |||
| 254 | return 0; | ||
| 255 | } | ||
| 256 | |||
| 257 | /** | ||
| 258 | * xe_ras_get_counter() - Get error counter value | ||
| 259 | * @xe: Xe device instance | ||
| 260 | * @severity: Error severity to be queried (&enum drm_xe_ras_error_severity) | ||
| 261 | * @component: Error component to be queried (&enum drm_xe_ras_error_component) | ||
| 262 | * @value: Counter value | ||
| 263 | * | ||
| 264 | * This function retrieves the value of a specific error counter based on | ||
| 265 | * the error severity and component. | ||
| 266 | * | ||
| 267 | * Return: 0 on success, negative error code on failure. | ||
| 268 | */ | ||
| 269 | int xe_ras_get_counter(struct xe_device *xe, u8 severity, u8 component, u32 *value) | ||
| 270 | { | ||
| 271 | struct xe_ras_error_class counter = {0}; | ||
| 272 | |||
| 273 | counter.common.severity = drm_to_xe_ras_severity(severity); | ||
| 274 | counter.common.component = drm_to_xe_ras_component(component); | ||
| 275 | |||
| 276 | guard(xe_pm_runtime)(xe); | ||
| 277 | return get_counter(xe, &counter, value); | ||
| 278 | } | ||
| 279 | |||
| 280 | /** | ||
| 281 | * xe_ras_clear_counter() - Clear error counter value | ||
| 282 | * @xe: Xe device instance | ||
| 283 | * @severity: Error severity to be cleared (&enum drm_xe_ras_error_severity) | ||
| 284 | * @component: Error component to be cleared (&enum drm_xe_ras_error_component) | ||
| 285 | * | ||
| 286 | * This function clears the value of a specific error counter based on | ||
| 287 | * the error severity and component. | ||
| 288 | * | ||
| 289 | * Return: 0 on success, negative error code on failure. | ||
| 290 | */ | ||
| 291 | int xe_ras_clear_counter(struct xe_device *xe, u8 severity, u8 component) | ||
| 292 | { | ||
| 293 | struct xe_ras_clear_counter_response response = {0}; | ||
| 294 | struct xe_ras_clear_counter_request request = {0}; | ||
| 295 | struct xe_sysctrl_mailbox_command command = {0}; | ||
| 296 | struct xe_ras_error_class *counter; | ||
| 297 | size_t rlen; | ||
| 298 | int ret; | ||
| 299 | |||
| 300 | counter = &request.counter; | ||
| 301 | counter->common.severity = drm_to_xe_ras_severity(severity); | ||
| 302 | counter->common.component = drm_to_xe_ras_component(component); | ||
| 303 | |||
| 304 | xe_sysctrl_create_command(&command, XE_SYSCTRL_GROUP_GFSP, XE_SYSCTRL_CMD_CLEAR_COUNTER, | ||
| 305 | &request, sizeof(request), &response, sizeof(response)); | ||
| 306 | |||
| 307 | guard(xe_pm_runtime)(xe); | ||
| 308 | ret = xe_sysctrl_send_command(&xe->sc, &command, &rlen); | ||
| 309 | if (ret) { | ||
| 310 | xe_err(xe, "sysctrl: failed to clear counter %d\n", ret); | ||
| 311 | return ret; | ||
| 312 | } | ||
| 313 | |||
| 314 | if (rlen != sizeof(response)) { | ||
| 315 | xe_err(xe, "sysctrl: unexpected clear counter response length %zu (expected %zu)\n", | ||
| 316 | rlen, sizeof(response)); | ||
| 317 | return -EIO; | ||
| 318 | } | ||
| 319 | |||
| 320 | ret = ras_status_to_errno(response.status); | ||
| 321 | if (ret) { | ||
| 322 | xe_err(xe, "sysctrl: clear counter command failed with status %#x\n", | ||
| 323 | response.status); | ||
| 324 | return ret; | ||
| 325 | } | ||
| 326 | |||
| 327 | counter = &response.counter; | ||
| 328 | |||
| 329 | xe_dbg(xe, "[RAS]: clear counter for %s %s\n", comp_to_str(counter->common.component), | ||
| 330 | sev_to_str(counter->common.severity)); | ||
| 331 | |||
| 332 | return 0; | ||
| 333 | } | ||
| 334 | |||
| 335 | /** | ||
| 336 | * xe_ras_init - Initialize Xe RAS | ||
| 337 | * @xe: xe device instance | ||
| 338 | * | ||
| 339 | * Initialize Xe RAS | ||
| 340 | */ | ||
| 341 | void xe_ras_init(struct xe_device *xe) | ||
| 342 | { | ||
| 343 | if (!xe->info.has_drm_ras) | ||
| 344 | return; | ||
| 345 | |||
| 346 | xe_drm_ras_init(xe); | ||
| 347 | |||
| 348 | if (!xe->info.has_sysctrl) | ||
| 349 | return; | ||
| 350 | |||
| 351 | if (IS_ENABLED(CONFIG_PCIEAER)) | ||
| 352 | ras_usp_aer_init(xe); | ||
| 353 | } | ||
| @@ -6,10 +6,15 @@ · 5 unmodified lines | |||
| 6 | #ifndef _XE_RAS_H_ | 6 | #ifndef _XE_RAS_H_ |
| 7 | #define _XE_RAS_H_ | 7 | #define _XE_RAS_H_ |
| 8 | 8 | ||
| 9 | #include <linux/types.h> | ||
| 10 | |||
| 9 | struct xe_device; | 11 | struct xe_device; |
| 10 | struct xe_sysctrl_event_response; | 12 | struct xe_sysctrl_event_response; |
| 11 | 13 | ||
| 12 | void xe_ras_counter_threshold_crossed(struct xe_device *xe, | 14 | void xe_ras_counter_threshold_crossed(struct xe_device *xe, |
| 13 | struct xe_sysctrl_event_response *response); | 15 | struct xe_sysctrl_event_response *response); |
| 16 | int xe_ras_get_counter(struct xe_device *xe, u8 severity, u8 component, u32 *value); | ||
| 17 | int xe_ras_clear_counter(struct xe_device *xe, u8 severity, u8 component); | ||
| 18 | void xe_ras_init(struct xe_device *xe); | ||
| 14 | 19 | ||
| 15 | #endif | 20 | #endif |
| @@ -70,4 +70,55 @@ struct xe_ras_threshold_crossed { · 69 unmodified lines | |||
| 70 | struct xe_ras_error_class counters[XE_RAS_NUM_COUNTERS]; | 70 | struct xe_ras_error_class counters[XE_RAS_NUM_COUNTERS]; |
| 71 | } __packed; | 71 | } __packed; |
| 72 | 72 | ||
| 73 | /** | ||
| 74 | * struct xe_ras_get_counter_request - Request structure for get counter | ||
| 75 | */ | ||
| 76 | struct xe_ras_get_counter_request { | ||
| 77 | /** @counter: Error counter to be queried */ | ||
| 78 | struct xe_ras_error_class counter; | ||
| 79 | /** @reserved: Reserved for future use */ | ||
| 80 | u32 reserved; | ||
| 81 | } __packed; | ||
| 82 | |||
| 83 | /** | ||
| 84 | * struct xe_ras_get_counter_response - Response structure for get counter | ||
| 85 | */ | ||
| 86 | struct xe_ras_get_counter_response { | ||
| 87 | /** @counter: Error counter that was queried */ | ||
| 88 | struct xe_ras_error_class counter; | ||
| 89 | /** @value: Current counter value */ | ||
| 90 | u32 value; | ||
| 91 | /** @timestamp: Timestamp when counter was last updated */ | ||
| 92 | u64 timestamp; | ||
| 93 | /** @threshold: Threshold value for the counter */ | ||
| 94 | u32 threshold; | ||
| 95 | /** @reserved: Reserved */ | ||
| 96 | u32 reserved[57]; | ||
| 97 | } __packed; | ||
| 98 | |||
| 99 | /** | ||
| 100 | * struct xe_ras_clear_counter_request - Request structure for clear counter | ||
| 101 | */ | ||
| 102 | struct xe_ras_clear_counter_request { | ||
| 103 | /** @counter: Counter class to be cleared */ | ||
| 104 | struct xe_ras_error_class counter; | ||
| 105 | /** @reserved: Reserved for future use */ | ||
| 106 | u32 reserved; | ||
| 107 | } __packed; | ||
| 108 | |||
| 109 | /** | ||
| 110 | * struct xe_ras_clear_counter_response - Response structure for clear counter | ||
| 111 | */ | ||
| 112 | struct xe_ras_clear_counter_response { | ||
| 113 | /** @counter: Counter class that was cleared */ | ||
| 114 | struct xe_ras_error_class counter; | ||
| 115 | /** @reserved: Reserved */ | ||
| 116 | u32 reserved; | ||
| 117 | /** @timestamp: Timestamp when the counter was cleared */ | ||
| 118 | u64 timestamp; | ||
| 119 | /** @status: Status of the clear operation */ | ||
| 120 | u32 status; | ||
| 121 | /** @reserved1: Reserved for future use */ | ||
| 122 | u32 reserved1[3]; | ||
| 123 | } __packed; | ||
| 73 | #endif | 124 | #endif |
| @@ -5,6 +5,8 @@ · 4 unmodified lines | |||
| 5 | 5 | ||
| 6 | #include "xe_reg_whitelist.h" | 6 | #include "xe_reg_whitelist.h" |
| 7 | 7 | ||
| 8 | #include <kunit/visibility.h> | ||
| 9 | |||
| 8 | #include "regs/xe_engine_regs.h" | 10 | #include "regs/xe_engine_regs.h" |
| 9 | #include "regs/xe_gt_regs.h" | 11 | #include "regs/xe_gt_regs.h" |
| 10 | #include "regs/xe_oa_regs.h" | 12 | #include "regs/xe_oa_regs.h" |
| @@ -41,7 +43,7 @@ static bool match_multi_queue_class(const struct xe_device *xe, · 30 unmodified lines | |||
| 41 | return xe_gt_supports_multi_queue(gt, hwe->class); | 43 | return xe_gt_supports_multi_queue(gt, hwe->class); |
| 42 | } | 44 | } |
| 43 | 45 | ||
| 44 | static const struct xe_rtp_table_sr register_whitelist = XE_RTP_TABLE_SR( | 46 | VISIBLE_IF_KUNIT const struct xe_rtp_table_sr register_whitelist = XE_RTP_TABLE_SR( |
| 45 | { XE_RTP_NAME("WaAllowPMDepthAndInvocationCountAccessFromUMD, 1408556865"), | 47 | { XE_RTP_NAME("WaAllowPMDepthAndInvocationCountAccessFromUMD, 1408556865"), |
| 46 | XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1200, 1210), ENGINE_CLASS(RENDER)), | 48 | XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1200, 1210), ENGINE_CLASS(RENDER)), |
| 47 | XE_RTP_ACTIONS(WHITELIST(PS_INVOCATION_COUNT, | 49 | XE_RTP_ACTIONS(WHITELIST(PS_INVOCATION_COUNT, |
| @@ -104,6 +106,7 @@ static const struct xe_rtp_table_sr register_whitelist = XE_RTP_TABLE_SR( · 56 unmodified lines | |||
| 104 | RING_FORCE_TO_NONPRIV_ACCESS_RW)) | 106 | RING_FORCE_TO_NONPRIV_ACCESS_RW)) |
| 105 | }, | 107 | }, |
| 106 | ); | 108 | ); |
| 109 | EXPORT_SYMBOL_IF_KUNIT(register_whitelist); | ||
| 107 | 110 | ||
| 108 | static const struct xe_rtp_table_sr oa_whitelist = XE_RTP_TABLE_SR( | 111 | static const struct xe_rtp_table_sr oa_whitelist = XE_RTP_TABLE_SR( |
| 109 | 112 | ||
| @@ -14,6 +14,10 @@ struct xe_hw_engine; · 13 unmodified lines | |||
| 14 | struct xe_reg_sr; | 14 | struct xe_reg_sr; |
| 15 | struct xe_reg_sr_entry; | 15 | struct xe_reg_sr_entry; |
| 16 | 16 | ||
| 17 | #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) | ||
| 18 | extern const struct xe_rtp_table_sr register_whitelist; | ||
| 19 | #endif | ||
| 20 | |||
| 17 | void xe_reg_whitelist_process_engine(struct xe_hw_engine *hwe); | 21 | void xe_reg_whitelist_process_engine(struct xe_hw_engine *hwe); |
| 18 | 22 | ||
| 19 | void xe_reg_whitelist_oa_regs(struct xe_gt *gt); | 23 | void xe_reg_whitelist_oa_regs(struct xe_gt *gt); |
| @@ -227,17 +227,23 @@ static bool rule_matches(const struct xe_device *xe, · 226 unmodified lines | |||
| 227 | 227 | ||
| 228 | static void rtp_add_sr_entry(const struct xe_rtp_action *action, | 228 | static void rtp_add_sr_entry(const struct xe_rtp_action *action, |
| 229 | struct xe_gt *gt, | 229 | struct xe_gt *gt, |
| 230 | struct xe_hw_engine *hwe, | ||
| 230 | u32 mmio_base, | 231 | u32 mmio_base, |
| 231 | struct xe_reg_sr *sr) | 232 | struct xe_reg_sr *sr) |
| 232 | { | 233 | { |
| 233 | struct xe_reg_sr_entry sr_entry = { | 234 | struct xe_reg_sr_entry sr_entry = { |
| 234 | .reg = action->reg, | 235 | .reg = action->reg, |
| 235 | .clr_bits = action->clr_bits, | 236 | .clr_bits = action->clr_bits, |
| 236 | .set_bits = action->set_bits, | ||
| 237 | .read_mask = action->read_mask, | 237 | .read_mask = action->read_mask, |
| 238 | }; | 238 | }; |
| 239 | 239 | ||
| 240 | if (action->use_func) | ||
| 241 | sr_entry.set_bits = action->set_func(gt, hwe); | ||
| 242 | else | ||
| 243 | sr_entry.set_bits = action->set_bits; | ||
| 244 | |||
| 240 | sr_entry.reg.addr += mmio_base; | 245 | sr_entry.reg.addr += mmio_base; |
| 246 | |||
| 241 | xe_reg_sr_add(sr, &sr_entry, gt); | 247 | xe_reg_sr_add(sr, &sr_entry, gt); |
| 242 | } | 248 | } |
| 243 | 249 | ||
| @@ -259,7 +265,7 @@ static bool rtp_process_one_sr(const struct xe_rtp_entry_sr *entry, · 15 unmodified lines | |||
| 259 | else | 265 | else |
| 260 | mmio_base = 0; | 266 | mmio_base = 0; |
| 261 | 267 | ||
| 262 | rtp_add_sr_entry(action, gt, mmio_base, sr); | 268 | rtp_add_sr_entry(action, gt, hwe, mmio_base, sr); |
| 263 | } | 269 | } |
| 264 | 270 | ||
| 265 | return true; | 271 | return true; |
| @@ -322,6 +322,25 @@ struct xe_reg_sr; · 321 unmodified lines | |||
| 322 | .clr_bits = (mask_bits_), .set_bits = (val_), \ | 322 | .clr_bits = (mask_bits_), .set_bits = (val_), \ |
| 323 | .read_mask = 0, ##__VA_ARGS__ } | 323 | .read_mask = 0, ##__VA_ARGS__ } |
| 324 | 324 | ||
| 325 | /** | ||
| 326 | * XE_RTP_ACTION_FIELD_SET_FUNC: Set a bit range to the value returned by a function | ||
| 327 | * @reg_: Register | ||
| 328 | * @mask_bits_: Mask of bits to be changed in the register, forming a field | ||
| 329 | * @func_: Function that returns value to set in the field denoted by @mask_bits_ | ||
| 330 | * @...: Additional fields to override in the struct xe_rtp_action entry | ||
| 331 | * | ||
| 332 | * This macro works like XE_RTP_ACTION_FIELD_SET(), except that the | ||
| 333 | * field value is evaluated at the time the RTP table is processed. | ||
| 334 | * | ||
| 335 | * @func_ will only be called a single time, when the RTP table is being | ||
| 336 | * processed. After processing, the value in the reg_sr entry is fixed and | ||
| 337 | * will not be re-evaluated. | ||
| 338 | */ | ||
| 339 | #define XE_RTP_ACTION_FIELD_SET_FUNC(reg_, mask_bits_, func_, ...) \ | ||
| 340 | { .reg = XE_RTP_DROP_CAST(reg_), \ | ||
| 341 | .clr_bits = mask_bits_, .set_func = func_, .use_func = 1, \ | ||
| 342 | .read_mask = mask_bits_, ##__VA_ARGS__ } | ||
| 343 | |||
| 325 | /** | 344 | /** |
| 326 | * XE_RTP_ACTION_WHITELIST - Add register to userspace whitelist | 345 | * XE_RTP_ACTION_WHITELIST - Add register to userspace whitelist |
| 327 | * @reg_: Register | 346 | * @reg_: Register |
| @@ -22,20 +22,37 @@ struct xe_gt; · 21 unmodified lines | |||
| 22 | */ | 22 | */ |
| 23 | struct xe_rtp_action { | 23 | struct xe_rtp_action { |
| 24 | /** @reg: Register */ | 24 | /** @reg: Register */ |
| 25 | struct xe_reg reg; | 25 | struct xe_reg reg; |
| 26 | |||
| 26 | /** | 27 | /** |
| 27 | * @clr_bits: bits to clear when updating register. It's always a | 28 | * @clr_bits: bits to clear when updating register. It's always a |
| 28 | * superset of bits being modified | 29 | * superset of bits being modified |
| 29 | */ | 30 | */ |
| 30 | u32 clr_bits; | 31 | u32 clr_bits; |
| 31 | /** @set_bits: bits to set when updating register */ | 32 | |
| 32 | u32 set_bits; | 33 | union { |
| 34 | /** @set_bits: bits to set when updating register */ | ||
| 35 | u32 set_bits; | ||
| 36 | |||
| 37 | /** @set_func: function to provide bits to set when updating register */ | ||
| 38 | u32 (*set_func)(struct xe_gt *gt, | ||
| 39 | struct xe_hw_engine *hwe); | ||
| 40 | }; | ||
| 41 | |||
| 33 | #define XE_RTP_NOCHECK .read_mask = 0 | 42 | #define XE_RTP_NOCHECK .read_mask = 0 |
| 34 | /** @read_mask: mask for bits to consider when reading value back */ | 43 | /** @read_mask: mask for bits to consider when reading value back */ |
| 35 | u32 read_mask; | 44 | u32 read_mask; |
| 45 | |||
| 36 | #define XE_RTP_ACTION_FLAG_ENGINE_BASE BIT(0) | 46 | #define XE_RTP_ACTION_FLAG_ENGINE_BASE BIT(0) |
| 37 | /** @flags: flags to apply on rule evaluation or action */ | 47 | /** @flags: flags to apply on rule evaluation or action */ |
| 38 | u8 flags; | 48 | u8 flags; |
| 49 | |||
| 50 | /** | ||
| 51 | * @use_func: | ||
| 52 | * Internal flag indicating @set_func should be called instead of | ||
| 53 | * using @set_bits. | ||
| 54 | */ | ||
| 55 | u8 use_func:1; | ||
| 39 | }; | 56 | }; |
| 40 | 57 | ||
| 41 | enum { | 58 | enum { |
| @@ -69,6 +86,7 @@ struct xe_rtp_rule { · 27 unmodified lines | |||
| 69 | u8 platform; | 86 | u8 platform; |
| 70 | u8 subplatform; | 87 | u8 subplatform; |
| 71 | }; | 88 | }; |
| 89 | |||
| 72 | /* | 90 | /* |
| 73 | * MATCH_GRAPHICS_VERSION / XE_RTP_MATCH_GRAPHICS_VERSION_RANGE / | 91 | * MATCH_GRAPHICS_VERSION / XE_RTP_MATCH_GRAPHICS_VERSION_RANGE / |
| 74 | * MATCH_MEDIA_VERSION / XE_RTP_MATCH_MEDIA_VERSION_RANGE | 92 | * MATCH_MEDIA_VERSION / XE_RTP_MATCH_MEDIA_VERSION_RANGE |
| @@ -78,15 +96,18 @@ struct xe_rtp_rule { · 3 unmodified lines | |||
| 78 | #define XE_RTP_END_VERSION_UNDEFINED U32_MAX | 96 | #define XE_RTP_END_VERSION_UNDEFINED U32_MAX |
| 79 | u32 ver_end; | 97 | u32 ver_end; |
| 80 | }; | 98 | }; |
| 99 | |||
| 81 | /* MATCH_STEP */ | 100 | /* MATCH_STEP */ |
| 82 | struct { | 101 | struct { |
| 83 | u8 step_start; | 102 | u8 step_start; |
| 84 | u8 step_end; | 103 | u8 step_end; |
| 85 | }; | 104 | }; |
| 105 | |||
| 86 | /* MATCH_ENGINE_CLASS / MATCH_NOT_ENGINE_CLASS */ | 106 | /* MATCH_ENGINE_CLASS / MATCH_NOT_ENGINE_CLASS */ |
| 87 | struct { | 107 | struct { |
| 88 | u8 engine_class; | 108 | u8 engine_class; |
| 89 | }; | 109 | }; |
| 110 | |||
| 90 | /* MATCH_FUNC */ | 111 | /* MATCH_FUNC */ |
| 91 | bool (*match_func)(const struct xe_device *xe, | 112 | bool (*match_func)(const struct xe_device *xe, |
| 92 | const struct xe_gt *gt, | 113 | const struct xe_gt *gt, |
| @@ -110,12 +110,14 @@ __diag_pop(); · 109 unmodified lines | |||
| 110 | 110 | ||
| 111 | /** | 111 | /** |
| 112 | * xe_step_platform_get - Determine platform-level stepping from PCI revid | 112 | * xe_step_platform_get - Determine platform-level stepping from PCI revid |
| 113 | * @xe: Xe device | 113 | * @platform: The Xe platform |
| 114 | * @revid: The PCI revid | ||
| 115 | * @step: Pointer to the step struct to update | ||
| 114 | * | 116 | * |
| 115 | * Convert the PCI revid into a platform-level stepping value and store that | 117 | * Convert the PCI revid into a platform-level stepping value and store that |
| 116 | * in the device info. | 118 | * in @step->platform. |
| 117 | */ | 119 | */ |
| 118 | void xe_step_platform_get(struct xe_device *xe) | 120 | void xe_step_platform_get(enum xe_platform platform, u8 revid, struct xe_step_info *step) |
| 119 | { | 121 | { |
| 120 | /* | 122 | /* |
| 121 | * Not all platforms map PCI revid directly into our symbolic stepping | 123 | * Not all platforms map PCI revid directly into our symbolic stepping |
| @@ -126,18 +128,21 @@ void xe_step_platform_get(struct xe_device *xe) · 4 unmodified lines | |||
| 126 | * checks. | 128 | * checks. |
| 127 | */ | 129 | */ |
| 128 | 130 | ||
| 129 | if (xe->info.platform == XE_NOVALAKE_P) | 131 | if (platform == XE_NOVALAKE_P) |
| 130 | xe->info.step.platform = STEP_A0 + xe->info.revid; | 132 | step->platform = STEP_A0 + revid; |
| 131 | } | 133 | } |
| 132 | 134 | ||
| 133 | /** | 135 | /** |
| 134 | * xe_step_pre_gmdid_get - Determine IP steppings from PCI revid | 136 | * xe_step_pre_gmdid_get - Determine IP steppings from PCI revid |
| 135 | * @xe: Xe device | 137 | * @xe: Xe device |
| 138 | * @step: Pointer to the step struct to update | ||
| 139 | * | ||
| 140 | * Convert the PCI revid into proper IP steppings and update @step->basedie, | ||
| 141 | * @step->graphics and @step->media accordingly. | ||
| 136 | * | 142 | * |
| 137 | * Convert the PCI revid into proper IP steppings. This should only be | 143 | * This should only be used on platforms that do not have GMD_ID support. |
| 138 | * used on platforms that do not have GMD_ID support. | ||
| 139 | */ | 144 | */ |
| 140 | void xe_step_pre_gmdid_get(struct xe_device *xe) | 145 | void xe_step_pre_gmdid_get(struct xe_device *xe, struct xe_step_info *step) |
| 141 | { | 146 | { |
| 142 | const struct xe_step_info *revids = NULL; | 147 | const struct xe_step_info *revids = NULL; |
| 143 | u16 revid = xe->info.revid; | 148 | u16 revid = xe->info.revid; |
| @@ -234,9 +239,9 @@ void xe_step_pre_gmdid_get(struct xe_device *xe) · 90 unmodified lines | |||
| 234 | } | 239 | } |
| 235 | 240 | ||
| 236 | done: | 241 | done: |
| 237 | xe->info.step.graphics = graphics; | 242 | step->graphics = graphics; |
| 238 | xe->info.step.media = media; | 243 | step->media = media; |
| 239 | xe->info.step.basedie = basedie; | 244 | step->basedie = basedie; |
| 240 | } | 245 | } |
| 241 | 246 | ||
| 242 | /** | 247 | /** |
| @@ -244,8 +249,10 @@ done: · 1 unmodified line | |||
| 244 | * @xe: Xe device | 249 | * @xe: Xe device |
| 245 | * @graphics_gmdid_revid: value of graphics GMD_ID register's revid field | 250 | * @graphics_gmdid_revid: value of graphics GMD_ID register's revid field |
| 246 | * @media_gmdid_revid: value of media GMD_ID register's revid field | 251 | * @media_gmdid_revid: value of media GMD_ID register's revid field |
| 252 | * @step: Poninter to the step struct to update. | ||
| 247 | * | 253 | * |
| 248 | * Convert the revid fields of the GMD_ID registers into proper IP steppings. | 254 | * Convert the revid fields of the GMD_ID registers into proper IP steppings |
| 255 | * and update @step->graphics and @step->media accordingly. | ||
| 249 | * | 256 | * |
| 250 | * GMD_ID revid values are currently expected to have consistent meanings on | 257 | * GMD_ID revid values are currently expected to have consistent meanings on |
| 251 | * all platforms: major steppings (A0, B0, etc.) are 4 apart, with minor | 258 | * all platforms: major steppings (A0, B0, etc.) are 4 apart, with minor |
| @@ -253,7 +260,8 @@ done: · 1 unmodified line | |||
| 253 | */ | 260 | */ |
| 254 | void xe_step_gmdid_get(struct xe_device *xe, | 261 | void xe_step_gmdid_get(struct xe_device *xe, |
| 255 | u32 graphics_gmdid_revid, | 262 | u32 graphics_gmdid_revid, |
| 256 | u32 media_gmdid_revid) | 263 | u32 media_gmdid_revid, |
| 264 | struct xe_step_info *step) | ||
| 257 | { | 265 | { |
| 258 | u8 graphics = STEP_A0 + graphics_gmdid_revid; | 266 | u8 graphics = STEP_A0 + graphics_gmdid_revid; |
| 259 | u8 media = STEP_A0 + media_gmdid_revid; | 267 | u8 media = STEP_A0 + media_gmdid_revid; |
| @@ -270,8 +278,8 @@ void xe_step_gmdid_get(struct xe_device *xe, · 10 unmodified lines | |||
| 270 | media_gmdid_revid); | 278 | media_gmdid_revid); |
| 271 | } | 279 | } |
| 272 | 280 | ||
| 273 | xe->info.step.graphics = graphics; | 281 | step->graphics = graphics; |
| 274 | xe->info.step.media = media; | 282 | step->media = media; |
| 275 | } | 283 | } |
| 276 | 284 | ||
| 277 | #define STEP_NAME_CASE(name) \ | 285 | #define STEP_NAME_CASE(name) \ |
| @@ -10,14 +10,16 @@ · 9 unmodified lines | |||
| 10 | 10 | ||
| 11 | #include "xe_step_types.h" | 11 | #include "xe_step_types.h" |
| 12 | 12 | ||
| 13 | enum xe_platform; | ||
| 13 | struct xe_device; | 14 | struct xe_device; |
| 14 | 15 | ||
| 15 | void xe_step_platform_get(struct xe_device *xe); | 16 | void xe_step_platform_get(enum xe_platform platform, u8 revid, struct xe_step_info *step); |
| 16 | 17 | ||
| 17 | void xe_step_pre_gmdid_get(struct xe_device *xe); | 18 | void xe_step_pre_gmdid_get(struct xe_device *xe, struct xe_step_info *step); |
| 18 | void xe_step_gmdid_get(struct xe_device *xe, | 19 | void xe_step_gmdid_get(struct xe_device *xe, |
| 19 | u32 graphics_gmdid_revid, | 20 | u32 graphics_gmdid_revid, |
| 20 | u32 media_gmdid_revid); | 21 | u32 media_gmdid_revid, |
| 22 | struct xe_step_info *step); | ||
| 21 | static inline u32 xe_step_to_gmdid(enum intel_step step) { return step - STEP_A0; } | 23 | static inline u32 xe_step_to_gmdid(enum intel_step step) { return step - STEP_A0; } |
| 22 | 24 | ||
| 23 | const char *xe_step_name(enum intel_step step); | 25 | const char *xe_step_name(enum intel_step step); |
| @@ -293,6 +293,34 @@ static int sysctrl_send_command(struct xe_sysctrl *sc, · 292 unmodified lines | |||
| 293 | return 0; | 293 | return 0; |
| 294 | } | 294 | } |
| 295 | 295 | ||
| 296 | /** | ||
| 297 | * xe_sysctrl_create_command() - Create system controller command | ||
| 298 | * @command: Sysctrl command structure | ||
| 299 | * @group_id: Command group ID | ||
| 300 | * @cmd_id: Command ID | ||
| 301 | * @request: Pointer to request buffer (can be NULL) | ||
| 302 | * @request_len: Size of request buffer | ||
| 303 | * @response: Pointer to response buffer | ||
| 304 | * @response_len: Size of response buffer | ||
| 305 | * | ||
| 306 | * Helper function to create sysctrl command to be sent via %xe_sysctrl_send_command() | ||
| 307 | */ | ||
| 308 | void xe_sysctrl_create_command(struct xe_sysctrl_mailbox_command *command, u8 group_id, u8 cmd_id, | ||
| 309 | void *request, size_t request_len, void *response, | ||
| 310 | size_t response_len) | ||
| 311 | { | ||
| 312 | struct xe_sysctrl_app_msg_hdr header = {0}; | ||
| 313 | |||
| 314 | header.data = FIELD_PREP(APP_HDR_GROUP_ID_MASK, group_id) | | ||
| 315 | FIELD_PREP(APP_HDR_COMMAND_MASK, cmd_id); | ||
| 316 | |||
| 317 | command->header = header; | ||
| 318 | command->data_in = request; | ||
| 319 | command->data_in_len = request_len; | ||
| 320 | command->data_out = response; | ||
| 321 | command->data_out_len = response_len; | ||
| 322 | } | ||
| 323 | |||
| 296 | /** | 324 | /** |
| 297 | * xe_sysctrl_mailbox_init - Initialize System Controller mailbox interface | 325 | * xe_sysctrl_mailbox_init - Initialize System Controller mailbox interface |
| 298 | * @sc: System controller structure | 326 | * @sc: System controller structure |
| @@ -23,6 +23,9 @@ struct xe_sysctrl_mailbox_command; · 22 unmodified lines | |||
| 23 | #define XE_SYSCTRL_APP_HDR_VERSION(hdr) \ | 23 | #define XE_SYSCTRL_APP_HDR_VERSION(hdr) \ |
| 24 | FIELD_GET(APP_HDR_VERSION_MASK, (hdr)->data) | 24 | FIELD_GET(APP_HDR_VERSION_MASK, (hdr)->data) |
| 25 | 25 | ||
| 26 | void xe_sysctrl_create_command(struct xe_sysctrl_mailbox_command *command, u8 group_id, u8 cmd_id, | ||
| 27 | void *request, size_t request_len, void *response, | ||
| 28 | size_t response_len); | ||
| 26 | void xe_sysctrl_mailbox_init(struct xe_sysctrl *sc); | 29 | void xe_sysctrl_mailbox_init(struct xe_sysctrl *sc); |
| 27 | int xe_sysctrl_send_command(struct xe_sysctrl *sc, | 30 | int xe_sysctrl_send_command(struct xe_sysctrl *sc, |
| 28 | struct xe_sysctrl_mailbox_command *cmd, | 31 | struct xe_sysctrl_mailbox_command *cmd, |
| @@ -22,9 +22,13 @@ enum xe_sysctrl_group { · 21 unmodified lines | |||
| 22 | /** | 22 | /** |
| 23 | * enum xe_sysctrl_gfsp_cmd - Commands supported by GFSP group | 23 | * enum xe_sysctrl_gfsp_cmd - Commands supported by GFSP group |
| 24 | * | 24 | * |
| 25 | * @XE_SYSCTRL_CMD_GET_COUNTER: Get error counter value | ||
| 26 | * @XE_SYSCTRL_CMD_CLEAR_COUNTER: Clear error counter value | ||
| 25 | * @XE_SYSCTRL_CMD_GET_PENDING_EVENT: Retrieve pending event | 27 | * @XE_SYSCTRL_CMD_GET_PENDING_EVENT: Retrieve pending event |
| 26 | */ | 28 | */ |
| 27 | enum xe_sysctrl_gfsp_cmd { | 29 | enum xe_sysctrl_gfsp_cmd { |
| 30 | XE_SYSCTRL_CMD_GET_COUNTER = 0x03, | ||
| 31 | XE_SYSCTRL_CMD_CLEAR_COUNTER = 0x04, | ||
| 28 | XE_SYSCTRL_CMD_GET_PENDING_EVENT = 0x07, | 32 | XE_SYSCTRL_CMD_GET_PENDING_EVENT = 0x07, |
| 29 | }; | 33 | }; |
| 30 | 34 | ||
| @@ -157,7 +157,9 @@ int xe_tile_init_early(struct xe_tile *tile, struct xe_device *xe, u8 id) · 156 unmodified lines | |||
| 157 | if (err) | 157 | if (err) |
| 158 | return err; | 158 | return err; |
| 159 | 159 | ||
| 160 | xe_pcode_init(tile); | 160 | err = xe_pcode_init_early(tile); |
| 161 | if (err) | ||
| 162 | return err; | ||
| 161 | 163 | ||
| 162 | return 0; | 164 | return 0; |
| 163 | } | 165 | } |
| @@ -20,7 +20,7 @@ · 19 unmodified lines | |||
| 20 | #undef XE_REG_MCR | 20 | #undef XE_REG_MCR |
| 21 | #define XE_REG_MCR(...) XE_REG(__VA_ARGS__, .mcr = 1) | 21 | #define XE_REG_MCR(...) XE_REG(__VA_ARGS__, .mcr = 1) |
| 22 | 22 | ||
| 23 | static const struct xe_rtp_table_sr gt_tunings = XE_RTP_TABLE_SR( | 23 | VISIBLE_IF_KUNIT const struct xe_rtp_table_sr gt_tunings = XE_RTP_TABLE_SR( |
| 24 | { XE_RTP_NAME("Tuning: Blend Fill Caching Optimization Disable"), | 24 | { XE_RTP_NAME("Tuning: Blend Fill Caching Optimization Disable"), |
| 25 | XE_RTP_RULES(PLATFORM(DG2)), | 25 | XE_RTP_RULES(PLATFORM(DG2)), |
| 26 | XE_RTP_ACTIONS(SET(XEHP_L3SCQREG7, BLEND_FILL_CACHING_OPT_DIS)) | 26 | XE_RTP_ACTIONS(SET(XEHP_L3SCQREG7, BLEND_FILL_CACHING_OPT_DIS)) |
| @@ -101,6 +101,7 @@ static const struct xe_rtp_table_sr gt_tunings = XE_RTP_TABLE_SR( · 74 unmodified lines | |||
| 101 | BANK_HASH_4KB_MODE)) | 101 | BANK_HASH_4KB_MODE)) |
| 102 | }, | 102 | }, |
| 103 | ); | 103 | ); |
| 104 | EXPORT_SYMBOL_IF_KUNIT(gt_tunings); | ||
| 104 | 105 | ||
| 105 | static const struct xe_rtp_table_sr engine_tunings = XE_RTP_TABLE_SR( | 106 | static const struct xe_rtp_table_sr engine_tunings = XE_RTP_TABLE_SR( |
| 106 | { XE_RTP_NAME("Tuning: L3 Hashing Mask"), | 107 | { XE_RTP_NAME("Tuning: L3 Hashing Mask"), |
| @@ -6,6 +6,8 @@ · 5 unmodified lines | |||
| 6 | #ifndef _XE_TUNING_H_ | 6 | #ifndef _XE_TUNING_H_ |
| 7 | #define _XE_TUNING_H_ | 7 | #define _XE_TUNING_H_ |
| 8 | 8 | ||
| 9 | #include <kunit/visibility.h> | ||
| 10 | |||
| 9 | struct drm_printer; | 11 | struct drm_printer; |
| 10 | struct xe_gt; | 12 | struct xe_gt; |
| 11 | struct xe_hw_engine; | 13 | struct xe_hw_engine; |
| @@ -16,4 +18,8 @@ void xe_tuning_process_engine(struct xe_hw_engine *hwe); · 4 unmodified lines | |||
| 16 | void xe_tuning_process_lrc(struct xe_hw_engine *hwe); | 18 | void xe_tuning_process_lrc(struct xe_hw_engine *hwe); |
| 17 | int xe_tuning_dump(struct xe_gt *gt, struct drm_printer *p); | 19 | int xe_tuning_dump(struct xe_gt *gt, struct drm_printer *p); |
| 18 | 20 | ||
| 21 | #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) | ||
| 22 | extern const struct xe_rtp_table_sr gt_tunings; | ||
| 23 | #endif | ||
| 24 | |||
| 19 | #endif | 25 | #endif |
| @@ -332,6 +332,20 @@ static int xe_vm_invalidate_madvise_range(struct xe_vm *vm, u64 start, u64 end) · 331 unmodified lines | |||
| 332 | return err; | 332 | return err; |
| 333 | } | 333 | } |
| 334 | 334 | ||
| 335 | /** | ||
| 336 | * madvise_range_needs_invalidation() - Check whether madvise needs invalidation | ||
| 337 | * @args: madvise ioctl arguments | ||
| 338 | * | ||
| 339 | * Purgeable state updates only touch VMA/BO metadata. PTEs stay valid and are | ||
| 340 | * zapped only if the BO is later purged. | ||
| 341 | * | ||
| 342 | * Return: true when the update needs PTE invalidation. | ||
| 343 | */ | ||
| 344 | static bool madvise_range_needs_invalidation(const struct drm_xe_madvise *args) | ||
| 345 | { | ||
| 346 | return args->type != DRM_XE_VMA_ATTR_PURGEABLE_STATE; | ||
| 347 | } | ||
| 348 | |||
| 335 | static bool madvise_args_are_sane(struct xe_device *xe, const struct drm_xe_madvise *args) | 349 | static bool madvise_args_are_sane(struct xe_device *xe, const struct drm_xe_madvise *args) |
| 336 | { | 350 | { |
| 337 | if (XE_IOCTL_DBG(xe, !args)) | 351 | if (XE_IOCTL_DBG(xe, !args)) |
| @@ -708,8 +722,9 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · 370 unmodified lines | |||
| 708 | madvise_funcs[attr_type](xe, vm, madvise_range.vmas, madvise_range.num_vmas, args, | 722 | madvise_funcs[attr_type](xe, vm, madvise_range.vmas, madvise_range.num_vmas, args, |
| 709 | &details); | 723 | &details); |
| 710 | 724 | ||
| 711 | err = xe_vm_invalidate_madvise_range(vm, madvise_range.addr, | 725 | if (madvise_range_needs_invalidation(args)) |
| 712 | madvise_range.addr + args->range); | 726 | err = xe_vm_invalidate_madvise_range(vm, madvise_range.addr, |
| 727 | madvise_range.addr + args->range); | ||
| 713 | 728 | ||
| 714 | if (madvise_range.has_svm_userptr_vmas) | 729 | if (madvise_range.has_svm_userptr_vmas) |
| 715 | xe_svm_notifier_unlock(vm); | 730 | xe_svm_notifier_unlock(vm); |
| @@ -130,7 +130,7 @@ · 129 unmodified lines | |||
| 130 | __diag_push(); | 130 | __diag_push(); |
| 131 | __diag_ignore_all("-Woverride-init", "Allow field overrides in table"); | 131 | __diag_ignore_all("-Woverride-init", "Allow field overrides in table"); |
| 132 | 132 | ||
| 133 | static const struct xe_rtp_table_sr gt_was = XE_RTP_TABLE_SR( | 133 | VISIBLE_IF_KUNIT const struct xe_rtp_table_sr gt_was = XE_RTP_TABLE_SR( |
| 134 | /* Workarounds applying over a range of IPs */ | 134 | /* Workarounds applying over a range of IPs */ |
| 135 | 135 | ||
| 136 | { XE_RTP_NAME("14011060649"), | 136 | { XE_RTP_NAME("14011060649"), |
| @@ -293,7 +293,7 @@ static const struct xe_rtp_table_sr gt_was = XE_RTP_TABLE_SR( · 156 unmodified lines | |||
| 293 | XE_RTP_ACTIONS(SET(MMIOATSREQLIMIT_GAM_WALK_3D, | 293 | XE_RTP_ACTIONS(SET(MMIOATSREQLIMIT_GAM_WALK_3D, |
| 294 | DIS_ATS_WRONLY_PG)) | 294 | DIS_ATS_WRONLY_PG)) |
| 295 | }, | 295 | }, |
| 296 | { XE_RTP_NAME("14026144927, 16029437861"), | 296 | { XE_RTP_NAME("14026144927, 16029437861, 14026127056"), |
| 297 | XE_RTP_RULES(GRAPHICS_VERSION(3510), GRAPHICS_STEP(A0, B0)), | 297 | XE_RTP_RULES(GRAPHICS_VERSION(3510), GRAPHICS_STEP(A0, B0)), |
| 298 | XE_RTP_ACTIONS(SET(L3SQCREG2, L3_SQ_DISABLE_COAMA_2WAY_COH | | 298 | XE_RTP_ACTIONS(SET(L3SQCREG2, L3_SQ_DISABLE_COAMA_2WAY_COH | |
| 299 | L3_SQ_DISABLE_COAMA)) | 299 | L3_SQ_DISABLE_COAMA)) |
| @@ -307,6 +307,7 @@ static const struct xe_rtp_table_sr gt_was = XE_RTP_TABLE_SR( · 7 unmodified lines | |||
| 307 | XE_RTP_ACTIONS(SET(GUC_INTR_CHICKEN, DISABLE_SIGNALING_ENGINES)) | 307 | XE_RTP_ACTIONS(SET(GUC_INTR_CHICKEN, DISABLE_SIGNALING_ENGINES)) |
| 308 | }, | 308 | }, |
| 309 | ); | 309 | ); |
| 310 | EXPORT_SYMBOL_IF_KUNIT(gt_was); | ||
| 310 | 311 | ||
| 311 | static const struct xe_rtp_table_sr engine_was = XE_RTP_TABLE_SR( | 312 | static const struct xe_rtp_table_sr engine_was = XE_RTP_TABLE_SR( |
| 312 | /* Workarounds applying over a range of IPs */ | 313 | /* Workarounds applying over a range of IPs */ |
| @@ -586,12 +587,12 @@ static const struct xe_rtp_table_sr engine_was = XE_RTP_TABLE_SR( · 273 unmodified lines | |||
| 586 | 587 | ||
| 587 | /* Xe3p_LPG*/ | 588 | /* Xe3p_LPG*/ |
| 588 | 589 | ||
| 589 | { XE_RTP_NAME("22021149932"), | 590 | { XE_RTP_NAME("22021149932, 14026290593"), |
| 590 | XE_RTP_RULES(GRAPHICS_VERSION(3510), GRAPHICS_STEP(A0, B0), | 591 | XE_RTP_RULES(GRAPHICS_VERSION(3510), GRAPHICS_STEP(A0, B0), |
| 591 | FUNC(xe_rtp_match_first_render_or_compute)), | 592 | FUNC(xe_rtp_match_first_render_or_compute)), |
| 592 | XE_RTP_ACTIONS(SET(LSC_CHICKEN_BIT_0_UDW, SAMPLER_LD_LSC_DISABLE)) | 593 | XE_RTP_ACTIONS(SET(LSC_CHICKEN_BIT_0_UDW, SAMPLER_LD_LSC_DISABLE)) |
| 593 | }, | 594 | }, |
| 594 | { XE_RTP_NAME("14025676848"), | 595 | { XE_RTP_NAME("14025676848, 14026270459"), |
| 595 | XE_RTP_RULES(GRAPHICS_VERSION(3510), GRAPHICS_STEP(A0, B0), | 596 | XE_RTP_RULES(GRAPHICS_VERSION(3510), GRAPHICS_STEP(A0, B0), |
| 596 | FUNC(xe_rtp_match_first_render_or_compute)), | 597 | FUNC(xe_rtp_match_first_render_or_compute)), |
| 597 | XE_RTP_ACTIONS(SET(LSC_CHICKEN_BIT_0_UDW, LSCFE_SAME_ADDRESS_ATOMICS_COALESCING_DISABLE)) | 598 | XE_RTP_ACTIONS(SET(LSC_CHICKEN_BIT_0_UDW, LSCFE_SAME_ADDRESS_ATOMICS_COALESCING_DISABLE)) |
| @@ -802,10 +803,11 @@ static const struct xe_rtp_entry oob_was_entries[] = { · 204 unmodified lines | |||
| 802 | 803 | ||
| 803 | static_assert(ARRAY_SIZE(oob_was_entries) == _XE_WA_OOB_COUNT); | 804 | static_assert(ARRAY_SIZE(oob_was_entries) == _XE_WA_OOB_COUNT); |
| 804 | 805 | ||
| 805 | static __maybe_unused const struct xe_rtp_table oob_was = { | 806 | VISIBLE_IF_KUNIT __maybe_unused const struct xe_rtp_table oob_was = { |
| 806 | .entries = oob_was_entries, | 807 | .entries = oob_was_entries, |
| 807 | .n_entries = ARRAY_SIZE(oob_was_entries), | 808 | .n_entries = ARRAY_SIZE(oob_was_entries), |
| 808 | }; | 809 | }; |
| 810 | EXPORT_SYMBOL_IF_KUNIT(oob_was); | ||
| 809 | 811 | ||
| 810 | static const struct xe_rtp_entry device_oob_was_entries[] = { | 812 | static const struct xe_rtp_entry device_oob_was_entries[] = { |
| 811 | #include <generated/xe_device_wa_oob.c> | 813 | #include <generated/xe_device_wa_oob.c> |
| @@ -813,10 +815,11 @@ static const struct xe_rtp_entry device_oob_was_entries[] = { · 1 unmodified line | |||
| 813 | 815 | ||
| 814 | static_assert(ARRAY_SIZE(device_oob_was_entries) == _XE_DEVICE_WA_OOB_COUNT); | 816 | static_assert(ARRAY_SIZE(device_oob_was_entries) == _XE_DEVICE_WA_OOB_COUNT); |
| 815 | 817 | ||
| 816 | static __maybe_unused const struct xe_rtp_table device_oob_was = { | 818 | VISIBLE_IF_KUNIT __maybe_unused const struct xe_rtp_table device_oob_was = { |
| 817 | .entries = device_oob_was_entries, | 819 | .entries = device_oob_was_entries, |
| 818 | .n_entries = ARRAY_SIZE(device_oob_was_entries), | 820 | .n_entries = ARRAY_SIZE(device_oob_was_entries), |
| 819 | }; | 821 | }; |
| 822 | EXPORT_SYMBOL_IF_KUNIT(device_oob_was); | ||
| 820 | 823 | ||
| 821 | __diag_pop(); | 824 | __diag_pop(); |
| 822 | 825 | ||
| @@ -6,6 +6,7 @@ · 5 unmodified lines | |||
| 6 | #ifndef _XE_WA_H_ | 6 | #ifndef _XE_WA_H_ |
| 7 | #define _XE_WA_H_ | 7 | #define _XE_WA_H_ |
| 8 | 8 | ||
| 9 | #include <kunit/visibility.h> | ||
| 9 | #include "xe_assert.h" | 10 | #include "xe_assert.h" |
| 10 | 11 | ||
| 11 | struct drm_printer; | 12 | struct drm_printer; |
| @@ -24,6 +25,12 @@ void xe_wa_apply_tile_workarounds(struct xe_tile *tile); · 12 unmodified lines | |||
| 24 | void xe_wa_device_dump(struct xe_device *xe, struct drm_printer *p); | 25 | void xe_wa_device_dump(struct xe_device *xe, struct drm_printer *p); |
| 25 | int xe_wa_gt_dump(struct xe_gt *gt, struct drm_printer *p); | 26 | int xe_wa_gt_dump(struct xe_gt *gt, struct drm_printer *p); |
| 26 | 27 | ||
| 28 | #if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) | ||
| 29 | extern const struct xe_rtp_table_sr gt_was; | ||
| 30 | extern __maybe_unused const struct xe_rtp_table oob_was; | ||
| 31 | extern __maybe_unused const struct xe_rtp_table device_oob_was; | ||
| 32 | #endif | ||
| 33 | |||
| 27 | /** | 34 | /** |
| 28 | * XE_GT_WA - Out-of-band GT workarounds, to be queried and called as needed. | 35 | * XE_GT_WA - Out-of-band GT workarounds, to be queried and called as needed. |
| 29 | * @gt__: gt instance | 36 | * @gt__: gt instance |
| @@ -65,3 +65,4 @@ · 64 unmodified lines | |||
| 65 | 65 | ||
| 66 | 14025883347 MEDIA_VERSION_RANGE(1301, 3503) | 66 | 14025883347 MEDIA_VERSION_RANGE(1301, 3503) |
| 67 | GRAPHICS_VERSION_RANGE(2004, 3005) | 67 | GRAPHICS_VERSION_RANGE(2004, 3005) |
| 68 | 16029380221 MEDIA_VERSION(3500) | ||
| @@ -893,8 +893,9 @@ · 892 unmodified lines | |||
| 893 | MACRO__(0xD741, ## __VA_ARGS__), \ | 893 | MACRO__(0xD741, ## __VA_ARGS__), \ |
| 894 | MACRO__(0xD742, ## __VA_ARGS__), \ | 894 | MACRO__(0xD742, ## __VA_ARGS__), \ |
| 895 | MACRO__(0xD743, ## __VA_ARGS__), \ | 895 | MACRO__(0xD743, ## __VA_ARGS__), \ |
| 896 | MACRO__(0xD744, ## __VA_ARGS__), \ | 896 | MACRO__(0xD745, ## __VA_ARGS__), \ |
| 897 | MACRO__(0xD745, ## __VA_ARGS__) | 897 | MACRO__(0xD74A, ## __VA_ARGS__), \ |
| 898 | MACRO__(0xD74B, ## __VA_ARGS__) | ||
| 898 | 899 | ||
| 899 | /* CRI */ | 900 | /* CRI */ |
| 900 | #define INTEL_CRI_IDS(MACRO__, ...) \ | 901 | #define INTEL_CRI_IDS(MACRO__, ...) \ |
| @@ -1791,22 +1791,8 @@ static inline int perf_is_paranoid(void) · 1790 unmodified lines | |||
| 1791 | } | 1791 | } |
| 1792 | 1792 | ||
| 1793 | extern int perf_allow_kernel(void); | 1793 | extern int perf_allow_kernel(void); |
| 1794 | 1794 | extern int perf_allow_cpu(void); | |
| 1795 | static inline int perf_allow_cpu(void) | 1795 | extern int perf_allow_tracepoint(void); |
| 1796 | { | ||
| 1797 | if (sysctl_perf_event_paranoid > 0 && !perfmon_capable()) | ||
| 1798 | return -EACCES; | ||
| 1799 | |||
| 1800 | return security_perf_event_open(PERF_SECURITY_CPU); | ||
| 1801 | } | ||
| 1802 | |||
| 1803 | static inline int perf_allow_tracepoint(void) | ||
| 1804 | { | ||
| 1805 | if (sysctl_perf_event_paranoid > -1 && !perfmon_capable()) | ||
| 1806 | return -EPERM; | ||
| 1807 | |||
| 1808 | return security_perf_event_open(PERF_SECURITY_TRACEPOINT); | ||
| 1809 | } | ||
| 1810 | 1796 | ||
| 1811 | extern int perf_exclude_event(struct perf_event *event, struct pt_regs *regs); | 1797 | extern int perf_exclude_event(struct perf_event *event, struct pt_regs *regs); |
| 1812 | 1798 | ||
| @@ -2023,6 +2009,19 @@ perf_event_pause(struct perf_event *event, bool reset) { return 0; } · 210 unmodified lines | |||
| 2023 | static inline int | 2009 | static inline int |
| 2024 | perf_exclude_event(struct perf_event *event, struct pt_regs *regs) { return 0; } | 2010 | perf_exclude_event(struct perf_event *event, struct pt_regs *regs) { return 0; } |
| 2025 | 2011 | ||
| 2012 | static inline int perf_allow_kernel(void) | ||
| 2013 | { | ||
| 2014 | return perfmon_capable() ? 0 : -EACCES; | ||
| 2015 | } | ||
| 2016 | static inline int perf_allow_cpu(void) | ||
| 2017 | { | ||
| 2018 | return perfmon_capable() ? 0 : -EACCES; | ||
| 2019 | } | ||
| 2020 | static inline int perf_allow_tracepoint(void) | ||
| 2021 | { | ||
| 2022 | return perfmon_capable() ? 0 : -EPERM; | ||
| 2023 | } | ||
| 2024 | |||
| 2026 | #endif /* !CONFIG_PERF_EVENTS */ | 2025 | #endif /* !CONFIG_PERF_EVENTS */ |
| 2027 | 2026 | ||
| 2028 | #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL) | 2027 | #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL) |
| @@ -2589,6 +2589,12 @@ enum drm_xe_ras_error_component { · 2588 unmodified lines | |||
| 2589 | DRM_XE_RAS_ERR_COMP_CORE_COMPUTE = 1, | 2589 | DRM_XE_RAS_ERR_COMP_CORE_COMPUTE = 1, |
| 2590 | /** @DRM_XE_RAS_ERR_COMP_SOC_INTERNAL: SoC Internal Error */ | 2590 | /** @DRM_XE_RAS_ERR_COMP_SOC_INTERNAL: SoC Internal Error */ |
| 2591 | DRM_XE_RAS_ERR_COMP_SOC_INTERNAL, | 2591 | DRM_XE_RAS_ERR_COMP_SOC_INTERNAL, |
| 2592 | /** @DRM_XE_RAS_ERR_COMP_DEVICE_MEMORY: Device Memory Error */ | ||
| 2593 | DRM_XE_RAS_ERR_COMP_DEVICE_MEMORY, | ||
| 2594 | /** @DRM_XE_RAS_ERR_COMP_PCIE: PCIe Subsystem Error */ | ||
| 2595 | DRM_XE_RAS_ERR_COMP_PCIE, | ||
| 2596 | /** @DRM_XE_RAS_ERR_COMP_FABRIC: Fabric Subsystem Error */ | ||
| 2597 | DRM_XE_RAS_ERR_COMP_FABRIC, | ||
| 2592 | /** @DRM_XE_RAS_ERR_COMP_MAX: Max Error */ | 2598 | /** @DRM_XE_RAS_ERR_COMP_MAX: Max Error */ |
| 2593 | DRM_XE_RAS_ERR_COMP_MAX /* non-ABI */ | 2599 | DRM_XE_RAS_ERR_COMP_MAX /* non-ABI */ |
| 2594 | }; | 2600 | }; |
| @@ -2606,7 +2612,10 @@ enum drm_xe_ras_error_component { · 11 unmodified lines | |||
| 2606 | */ | 2612 | */ |
| 2607 | #define DRM_XE_RAS_ERROR_COMPONENT_NAMES { \ | 2613 | #define DRM_XE_RAS_ERROR_COMPONENT_NAMES { \ |
| 2608 | [DRM_XE_RAS_ERR_COMP_CORE_COMPUTE] = "core-compute", \ | 2614 | [DRM_XE_RAS_ERR_COMP_CORE_COMPUTE] = "core-compute", \ |
| 2609 | [DRM_XE_RAS_ERR_COMP_SOC_INTERNAL] = "soc-internal" \ | 2615 | [DRM_XE_RAS_ERR_COMP_SOC_INTERNAL] = "soc-internal", \ |
| 2616 | [DRM_XE_RAS_ERR_COMP_DEVICE_MEMORY] = "device-memory", \ | ||
| 2617 | [DRM_XE_RAS_ERR_COMP_PCIE] = "pcie", \ | ||
| 2618 | [DRM_XE_RAS_ERR_COMP_FABRIC] = "fabric", \ | ||
| 2610 | } | 2619 | } |
| 2611 | 2620 | ||
| 2612 | #if defined(__cplusplus) | 2621 | #if defined(__cplusplus) |
| @@ -14769,6 +14769,24 @@ int perf_allow_kernel(void) · 14768 unmodified lines | |||
| 14769 | } | 14769 | } |
| 14770 | EXPORT_SYMBOL_GPL(perf_allow_kernel); | 14770 | EXPORT_SYMBOL_GPL(perf_allow_kernel); |
| 14771 | 14771 | ||
| 14772 | int perf_allow_cpu(void) | ||
| 14773 | { | ||
| 14774 | if (sysctl_perf_event_paranoid > 0 && !perfmon_capable()) | ||
| 14775 | return -EACCES; | ||
| 14776 | |||
| 14777 | return security_perf_event_open(PERF_SECURITY_CPU); | ||
| 14778 | } | ||
| 14779 | EXPORT_SYMBOL_GPL(perf_allow_cpu); | ||
| 14780 | |||
| 14781 | int perf_allow_tracepoint(void) | ||
| 14782 | { | ||
| 14783 | if (sysctl_perf_event_paranoid > -1 && !perfmon_capable()) | ||
| 14784 | return -EPERM; | ||
| 14785 | |||
| 14786 | return security_perf_event_open(PERF_SECURITY_TRACEPOINT); | ||
| 14787 | } | ||
| 14788 | EXPORT_SYMBOL_GPL(perf_allow_tracepoint); | ||
| 14789 | |||
| 14772 | /* | 14790 | /* |
| 14773 | * Inherit an event from parent task to child task. | 14791 | * Inherit an event from parent task to child task. |
| 14774 | * | 14792 | * |