UAPI Changes: - Add additional error components to xe drm_ras (Riana) - Drop 'force_execlist' module parameter (Roper) Cross-subsystem Changes: - Perf events: Export perf_allow_{cpu,tracepoint} to be used by Xe (John) Display Changes: - Skip FORCE_WC and vm_bound check for external dma-bufs (Auld) Driver Changes: - Gate observation streams with perf_allow_cpu (John Hubbard) - Documentation updates and fixes (Michal, Zhan) - Remove unreleased NVL-S GuC (Daniele) - Fix pcode init path (Michal) - RTP fixes and improvements (Gustavo, Violet, Thomas, Roper) - TLB invalidation fixes and improvements (Tilak) - PXP detachment from HuC for newer platforms (Daniele) - Multi-queue fix (Niranjana) - Improve Kconfig.profile help (Rodrigo) - Xe_drm_ras and hw_error updates and fixes (Raag, Riana) - NVL-S updated PCI-IDs and W/a (Gustavo, Nitin) - Fix dma_fence refcound (Wentao) - Madvise: optimize invalidation path (Arvind) - Fix a infinite gt-reset loop in the timeout recovery (Rodrigo) - Fix wa_oob codegen recipe for external module builds (Thomas) - Avoid global forcewake in cycle query path (Xin) - Update TTM device benefical_order (Brost) - Fix buffer overflow in guc capture (Tejas) - Page-table fixes and improvements (Brian, Francois, Auld, Brost) - Removing redundant check (Lu) - General MCR and MMIO clean-up and improvements (Michal) - Don't whitelist OA registers unconditionally (Ashutosh) - SVM error return fix (Brost) - Userptr fix and small related clean-ups (Shuicheng) - Don't attempt to process FAST_REQ or EVENT relays on PF (Michal) - Couple fdinfo improvements (Auld) - Drop manual VF check on i2c (Raag) - Probe info outside of xe_info_init functions (Gustavo) Signed-off-by: Dave Airlie <airlied@redhat.com>
Unified · Split
| @@ -251,6 +251,13 @@ Description: RO. Fan 2 speed in RPM. · 250 unmodified lines | ||
| 251 | 251 | |
| 252 | 252 | Only supported for particular Intel Xe graphics platforms. |
| 253 | 253 | |
| 254 | + On DG2 the driver always shows two fan channels, because the | |
| 255 | + FSC_READ_NUM_FANS command does not work on some cards. OEMs | |
| 256 | + decide how the fans map to tach channels, so two fans can share | |
| 257 | + one tach line. When that happens, the second channel | |
| 258 | + reads 0 RPM even though the fan is spinning. This is normal, not | |
| 259 | + a bug. | |
| 260 | + | |
| 254 | 261 | What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/fan3_input |
| 255 | 262 | Date: March 2025 |
| 256 | 263 | KernelVersion: 6.16 |
| @@ -1,50 +1,71 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0-only |
| 2 | 2 | config DRM_XE_JOB_TIMEOUT_MAX |
| 3 | − int "Default max job timeout (ms)" | |
| 3 | + int "Hard upper limit for job timeout (ms)" | |
| 4 | 4 | default 10000 # milliseconds |
| 5 | 5 | help |
| 6 | − Configures the default max job timeout after which job will | |
| 7 | − be forcefully taken away from scheduler. | |
| 6 | + Absolute upper bound (in milliseconds) for the per-engine-class job | |
| 7 | + timeout. This is the maximum value that can be written to the sysfs | |
| 8 | + job_timeout_ms knob, regardless of privileges. To raise this ceiling, | |
| 9 | + increase this value and rebuild the kernel. | |
| 8 | 10 | config DRM_XE_JOB_TIMEOUT_MIN |
| 9 | − int "Default min job timeout (ms)" | |
| 11 | + int "Hard lower limit for job timeout (ms)" | |
| 10 | 12 | default 1 # milliseconds |
| 11 | 13 | help |
| 12 | − Configures the default min job timeout after which job will | |
| 13 | − be forcefully taken away from scheduler. | |
| 14 | + Absolute lower bound (in milliseconds) for the per-engine-class job | |
| 15 | + timeout. This is the minimum value that can be written to the sysfs | |
| 16 | + job_timeout_ms knob, regardless of privileges. | |
| 17 | + | |
| 18 | + Note: the job timeout default (5000 ms) is hardcoded in the driver | |
| 19 | + and is not configurable here. Use the sysfs job_timeout_ms knob at | |
| 20 | + runtime to change the engine-class default. | |
| 14 | 21 | config DRM_XE_TIMESLICE_MAX |
| 15 | − int "Default max timeslice duration (us)" | |
| 22 | + int "Hard upper limit for timeslice duration (us)" | |
| 16 | 23 | default 10000000 # microseconds |
| 17 | 24 | help |
| 18 | − Configures the default max timeslice duration between multiple | |
| 19 | − contexts by guc scheduling. | |
| 25 | + Absolute upper bound (in microseconds) for the timeslice duration. | |
| 26 | + This caps both the sysfs timeslice_duration_us knob and the value | |
| 27 | + accepted via the DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE UAPI for | |
| 28 | + processes with CAP_SYS_NICE when DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT | |
| 29 | + is enabled. | |
| 20 | 30 | config DRM_XE_TIMESLICE_MIN |
| 21 | − int "Default min timeslice duration (us)" | |
| 31 | + int "Hard lower limit for timeslice duration (us)" | |
| 22 | 32 | default 1 # microseconds |
| 23 | 33 | help |
| 24 | − Configures the default min timeslice duration between multiple | |
| 25 | − contexts by guc scheduling. | |
| 34 | + Absolute lower bound (in microseconds) for the timeslice duration. | |
| 35 | + This caps both the sysfs timeslice_duration_us knob and the value | |
| 36 | + accepted via the DRM_XE_EXEC_QUEUE_SET_PROPERTY_TIMESLICE UAPI for | |
| 37 | + processes with CAP_SYS_NICE when DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT | |
| 38 | + is enabled. | |
| 26 | 39 | config DRM_XE_PREEMPT_TIMEOUT |
| 27 | − int "Preempt timeout (us, jiffy granularity)" | |
| 40 | + int "Default preempt timeout (us, jiffy granularity)" | |
| 28 | 41 | default 640000 # microseconds |
| 29 | 42 | help |
| 30 | − How long to wait (in microseconds) for a preemption event to occur | |
| 31 | − when submitting a new context. If the current context does not hit | |
| 32 | − an arbitration point and yield to HW before the timer expires, the | |
| 33 | − HW will be reset to allow the more important context to execute. | |
| 43 | + Initial per-engine-class preemption timeout (in microseconds). This | |
| 44 | + is the value the driver programs at boot; it can be changed at | |
| 45 | + runtime via the sysfs preempt_timeout_us knob. | |
| 46 | + | |
| 47 | + This is how long the driver waits for the current context to reach | |
| 48 | + an arbitration point and yield the GPU voluntarily when a | |
| 49 | + higher-priority context becomes runnable. If the context does not | |
| 50 | + yield before the timer expires, the HW is reset to allow the | |
| 51 | + higher-priority context to execute. | |
| 52 | + | |
| 53 | + The range userspace may write via sysfs is bounded by | |
| 54 | + DRM_XE_PREEMPT_TIMEOUT_MIN and DRM_XE_PREEMPT_TIMEOUT_MAX. | |
| 34 | 55 | config DRM_XE_PREEMPT_TIMEOUT_MAX |
| 35 | − int "Default max preempt timeout (us)" | |
| 56 | + int "Hard upper limit for preempt timeout (us)" | |
| 36 | 57 | default 10000000 # microseconds |
| 37 | 58 | help |
| 38 | − Configures the default max preempt timeout after which context | |
| 39 | − will be forcefully taken away and higher priority context will | |
| 40 | − run. | |
| 59 | + Absolute upper bound (in microseconds) for the per-engine-class | |
| 60 | + preemption timeout. This is the maximum value that can be written to | |
| 61 | + the sysfs preempt_timeout_us knob, regardless of privileges. | |
| 41 | 62 | config DRM_XE_PREEMPT_TIMEOUT_MIN |
| 42 | − int "Default min preempt timeout (us)" | |
| 63 | + int "Hard lower limit for preempt timeout (us)" | |
| 43 | 64 | default 1 # microseconds |
| 44 | 65 | help |
| 45 | − Configures the default min preempt timeout after which context | |
| 46 | − will be forcefully taken away and higher priority context will | |
| 47 | − run. | |
| 66 | + Absolute lower bound (in microseconds) for the per-engine-class | |
| 67 | + preemption timeout. This is the minimum value that can be written to | |
| 68 | + the sysfs preempt_timeout_us knob, regardless of privileges. | |
| 48 | 69 | config DRM_XE_ENABLE_SCHEDTIMEOUT_LIMIT |
| 49 | 70 | bool "Default configuration of limitation on scheduler timeout" |
| 50 | 71 | default y |
| @@ -101,6 +101,7 @@ xe-y += xe_bb.o \ · 100 unmodified lines | ||
| 101 | 101 | xe_page_reclaim.o \ |
| 102 | 102 | xe_pat.o \ |
| 103 | 103 | xe_pci.o \ |
| 104 | + xe_pci_error.o \ | |
| 104 | 105 | xe_pci_rebar.o \ |
| 105 | 106 | xe_pcode.o \ |
| 106 | 107 | xe_pm.o \ |
| @@ -9,5 +9,6 @@ obj-$(CONFIG_DRM_XE_KUNIT_TEST) += xe_test.o · 8 unmodified lines | ||
| 9 | 9 | xe_test-y = xe_test_mod.o \ |
| 10 | 10 | xe_args_test.o \ |
| 11 | 11 | xe_pci_test.o \ |
| 12 | + xe_rtp_tables_test.o \ | |
| 12 | 13 | xe_rtp_test.o \ |
| 13 | 14 | xe_wa_test.o |
| @@ -311,40 +311,44 @@ const void *xe_pci_id_gen_param(struct kunit *test, const void *prev, char *desc · 310 unmodified lines | ||
| 311 | 311 | } |
| 312 | 312 | EXPORT_SYMBOL_IF_KUNIT(xe_pci_id_gen_param); |
| 313 | 313 | |
| 314 | −static void fake_init_devid(struct xe_device *xe) | |
| 314 | +static int fake_probe_info(struct xe_device *xe, | |
| 315 | + const struct xe_device_desc *desc, | |
| 316 | + struct xe_pci_fake_data *data, | |
| 317 | + struct xe_probed_info *probed_info) | |
| 315 | 318 | { |
| 316 | − /* Nothing to do, just keep zero. */ | |
| 317 | −} | |
| 318 | − | |
| 319 | −static int fake_read_gmdid(struct xe_device *xe, enum xe_gmdid_type type, | |
| 320 | − u32 *ver, u32 *revid) | |
| 321 | −{ | |
| 322 | − struct kunit *test = kunit_get_current_test(); | |
| 323 | − struct xe_pci_fake_data *data = test->priv; | |
| 319 | + probed_info->tile_count = 1 + desc->max_remote_tiles; | |
| 324 | 320 | |
| 325 | − if (type == GMDID_MEDIA) { | |
| 326 | − *ver = data->media_verx100; | |
| 327 | − *revid = xe_step_to_gmdid(data->step.media); | |
| 321 | + if (!data || desc->pre_gmdid_graphics_ip) { | |
| 322 | + probed_info->graphics_ip = desc->pre_gmdid_graphics_ip; | |
| 323 | + probed_info->media_ip = desc->pre_gmdid_media_ip; | |
| 328 | 324 | } else { |
| 329 | − *ver = data->graphics_verx100; | |
| 330 | − *revid = xe_step_to_gmdid(data->step.graphics); | |
| 325 | + probed_info->graphics_ip = find_graphics_ip(data->graphics_verx100); | |
| 326 | + | |
| 327 | + if (data->media_verx100) { | |
| 328 | + probed_info->media_ip = find_media_ip(data->media_verx100); | |
| 329 | + xe_assert(xe, probed_info->media_ip); | |
| 330 | + } | |
| 331 | 331 | } |
| 332 | 332 | |
| 333 | − return 0; | |
| 334 | −} | |
| 333 | + xe_assert(xe, probed_info->graphics_ip); | |
| 334 | + if (!probed_info->graphics_ip) | |
| 335 | + return -ENODEV; | |
| 335 | 336 | |
| 336 | −static void fake_xe_info_probe_tile_count(struct xe_device *xe) | |
| 337 | −{ | |
| 338 | − /* Nothing to do, just use the statically defined value. */ | |
| 337 | + if (data) | |
| 338 | + probed_info->step = data->step; | |
| 339 | + | |
| 340 | + return 0; | |
| 339 | 341 | } |
| 340 | 342 | |
| 341 | 343 | int xe_pci_fake_device_init(struct xe_device *xe) |
| 342 | 344 | { |
| 343 | 345 | struct kunit *test = kunit_get_current_test(); |
| 344 | 346 | struct xe_pci_fake_data *data = test->priv; |
| 347 | + struct xe_probed_info probed_info = {}; | |
| 345 | 348 | const struct pci_device_id *ent = pciidlist; |
| 346 | 349 | const struct xe_device_desc *desc; |
| 347 | 350 | const struct xe_subplatform_desc *subplatform_desc; |
| 351 | + int err; | |
| 348 | 352 | |
| 349 | 353 | if (!data) { |
| 350 | 354 | desc = (const void *)ent->driver_data; |
| @@ -374,13 +378,12 @@ done: · 23 unmodified lines | ||
| 374 | 378 | xe->sriov.__mode = data && data->sriov_mode ? |
| 375 | 379 | data->sriov_mode : XE_SRIOV_MODE_NONE; |
| 376 | 380 | |
| 377 | − kunit_activate_static_stub(test, init_devid, fake_init_devid); | |
| 378 | − kunit_activate_static_stub(test, read_gmdid, fake_read_gmdid); | |
| 379 | − kunit_activate_static_stub(test, xe_info_probe_tile_count, | |
| 380 | − fake_xe_info_probe_tile_count); | |
| 381 | + err = fake_probe_info(xe, desc, data, &probed_info); | |
| 382 | + if (err) | |
| 383 | + return err; | |
| 381 | 384 | |
| 382 | − xe_info_init_early(xe, desc, subplatform_desc); | |
| 383 | − xe_info_init(xe, desc); | |
| 385 | + xe_info_init_early(xe, desc, subplatform_desc, &probed_info); | |
| 386 | + xe_info_init(xe, desc, &probed_info); | |
| 384 | 387 | |
| 385 | 388 | return 0; |
| 386 | 389 | } |
| @@ -0,0 +1,112 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0 | |
| 2 | +/* | |
| 3 | + * Copyright © 2026 Intel Corporation | |
| 4 | + */ | |
| 5 | + | |
| 6 | +#include <kunit/test.h> | |
| 7 | + | |
| 8 | +#include "xe_reg_whitelist.h" | |
| 9 | +#include "xe_rtp_types.h" | |
| 10 | +#include "xe_tuning.h" | |
| 11 | +#include "xe_wa.h" | |
| 12 | + | |
| 13 | +#define RTP_TABLE_PARAM(table) \ | |
| 14 | + static const void *table##_gen_params(struct kunit *test, \ | |
| 15 | + const void *prev, char *desc) \ | |
| 16 | + { \ | |
| 17 | + typeof((table.entries)[0]) *__next = prev ? \ | |
| 18 | + ((typeof(__next))prev) + 1 : (table.entries); \ | |
| 19 | + if (__next - table.entries < table.n_entries) { \ | |
| 20 | + scnprintf(desc, KUNIT_PARAM_DESC_SIZE, #table "/%s", __next->name); \ | |
| 21 | + return __next; \ | |
| 22 | + } \ | |
| 23 | + return NULL; \ | |
| 24 | + } | |
| 25 | + | |
| 26 | +static void xe_rtp_table_gt_test(struct kunit *test) | |
| 27 | +{ | |
| 28 | + const struct xe_rtp_entry_sr *entry = test->param_value; | |
| 29 | + | |
| 30 | + for (int i = 0; i < entry->n_rules; i++) { | |
| 31 | + KUNIT_EXPECT_TRUE(test, | |
| 32 | + entry->rules[i].match_type != XE_RTP_MATCH_ENGINE_CLASS || | |
| 33 | + entry->flags & XE_RTP_ENTRY_FLAG_FOREACH_ENGINE); | |
| 34 | + KUNIT_EXPECT_TRUE(test, | |
| 35 | + entry->rules[i].match_type != XE_RTP_MATCH_NOT_ENGINE_CLASS || | |
| 36 | + entry->flags & XE_RTP_ENTRY_FLAG_FOREACH_ENGINE); | |
| 37 | + } | |
| 38 | +} | |
| 39 | + | |
| 40 | +RTP_TABLE_PARAM(gt_was); | |
| 41 | +RTP_TABLE_PARAM(gt_tunings); | |
| 42 | + | |
| 43 | +static void xe_rtp_table_oob_test(struct kunit *test) | |
| 44 | +{ | |
| 45 | + const struct xe_rtp_entry *entry = test->param_value; | |
| 46 | + | |
| 47 | + for (int i = 0; i < entry->n_rules; i++) { | |
| 48 | + u8 match_type = entry->rules[i].match_type; | |
| 49 | + | |
| 50 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_ENGINE_CLASS); | |
| 51 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_NOT_ENGINE_CLASS); | |
| 52 | + } | |
| 53 | +} | |
| 54 | + | |
| 55 | +RTP_TABLE_PARAM(oob_was); | |
| 56 | + | |
| 57 | +static void xe_rtp_table_dev_oob_test(struct kunit *test) | |
| 58 | +{ | |
| 59 | + const struct xe_rtp_entry *entry = test->param_value; | |
| 60 | + | |
| 61 | + for (int i = 0; i < entry->n_rules; i++) { | |
| 62 | + u8 match_type = entry->rules[i].match_type; | |
| 63 | + | |
| 64 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_ENGINE_CLASS); | |
| 65 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_NOT_ENGINE_CLASS); | |
| 66 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_GRAPHICS_VERSION); | |
| 67 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_GRAPHICS_VERSION_RANGE); | |
| 68 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_GRAPHICS_VERSION_ANY_GT); | |
| 69 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_GRAPHICS_STEP); | |
| 70 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_MEDIA_VERSION); | |
| 71 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_MEDIA_VERSION_RANGE); | |
| 72 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_MEDIA_VERSION_ANY_GT); | |
| 73 | + KUNIT_EXPECT_NE(test, match_type, XE_RTP_MATCH_MEDIA_STEP); | |
| 74 | + } | |
| 75 | +} | |
| 76 | + | |
| 77 | +RTP_TABLE_PARAM(device_oob_was); | |
| 78 | + | |
| 79 | +static void xe_rtp_table_missing_upper_bound_test(struct kunit *test) | |
| 80 | +{ | |
| 81 | + const struct xe_rtp_entry_sr *entry = test->param_value; | |
| 82 | + | |
| 83 | + for (int i = 0; i < entry->n_rules; i++) { | |
| 84 | + u8 match_type = entry->rules[i].match_type; | |
| 85 | + | |
| 86 | + KUNIT_EXPECT_FALSE(test, | |
| 87 | + match_type == XE_RTP_MATCH_GRAPHICS_VERSION_RANGE && | |
| 88 | + entry->rules[i].ver_end == XE_RTP_END_VERSION_UNDEFINED); | |
| 89 | + KUNIT_EXPECT_FALSE(test, | |
| 90 | + match_type == XE_RTP_MATCH_MEDIA_VERSION_RANGE && | |
| 91 | + entry->rules[i].ver_end == XE_RTP_END_VERSION_UNDEFINED); | |
| 92 | + } | |
| 93 | +} | |
| 94 | + | |
| 95 | +RTP_TABLE_PARAM(register_whitelist); | |
| 96 | + | |
| 97 | +static struct kunit_case xe_rtp_table_tests[] = { | |
| 98 | + KUNIT_CASE_PARAM(xe_rtp_table_gt_test, gt_was_gen_params), | |
| 99 | + KUNIT_CASE_PARAM(xe_rtp_table_gt_test, gt_tunings_gen_params), | |
| 100 | + KUNIT_CASE_PARAM(xe_rtp_table_oob_test, oob_was_gen_params), | |
| 101 | + KUNIT_CASE_PARAM(xe_rtp_table_dev_oob_test, device_oob_was_gen_params), | |
| 102 | + KUNIT_CASE_PARAM(xe_rtp_table_missing_upper_bound_test, | |
| 103 | + register_whitelist_gen_params), | |
| 104 | + {} | |
| 105 | +}; | |
| 106 | + | |
| 107 | +static struct kunit_suite xe_rtp_tables_test_suite = { | |
| 108 | + .name = "xe_rtp_tables_test", | |
| 109 | + .test_cases = xe_rtp_table_tests, | |
| 110 | +}; | |
| 111 | + | |
| 112 | +kunit_test_suite(xe_rtp_tables_test_suite); | |
| @@ -280,6 +280,11 @@ static void xe_rtp_rules_tests(struct kunit *test) · 279 unmodified lines | ||
| 280 | 280 | KUNIT_EXPECT_EQ(test, err, param->expected_err); |
| 281 | 281 | } |
| 282 | 282 | |
| 283 | +static u32 bits_2_3_set(struct xe_gt *gt, struct xe_hw_engine *hwe) | |
| 284 | +{ | |
| 285 | + return REG_BIT(2) | REG_BIT(3); | |
| 286 | +} | |
| 287 | + | |
| 283 | 288 | static const struct rtp_to_sr_test_case rtp_to_sr_cases[] = { |
| 284 | 289 | { |
| 285 | 290 | .name = "coalesce-same-reg", |
| @@ -300,6 +305,29 @@ static const struct rtp_to_sr_test_case rtp_to_sr_cases[] = { · 14 unmodified lines | ||
| 300 | 305 | }, |
| 301 | 306 | ), |
| 302 | 307 | }, |
| 308 | + { | |
| 309 | + .name = "coalesce-same-reg-literal-and-func", | |
| 310 | + .expected_reg = REGULAR_REG1, | |
| 311 | + .expected_set_bits = REG_BIT(0) | REG_BIT(1) | REG_BIT(2) | REG_BIT(3), | |
| 312 | + .expected_clr_bits = REG_BIT(0) | REG_BIT(1) | REG_BIT(2) | REG_BIT(3), | |
| 313 | + .expected_active = BIT(0) | BIT(1), | |
| 314 | + .expected_count_sr_entries = 1, | |
| 315 | + /* Different bits on the same register: create a single entry */ | |
| 316 | + .table = XE_RTP_TABLE_SR( | |
| 317 | + { XE_RTP_NAME("basic-1"), | |
| 318 | + XE_RTP_RULES(FUNC(match_yes)), | |
| 319 | + XE_RTP_ACTIONS(FIELD_SET(REGULAR_REG1, | |
| 320 | + REG_BIT(0) | REG_BIT(1), | |
| 321 | + REG_BIT(0) | REG_BIT(1))) | |
| 322 | + }, | |
| 323 | + { XE_RTP_NAME("basic-2"), | |
| 324 | + XE_RTP_RULES(FUNC(match_yes)), | |
| 325 | + XE_RTP_ACTIONS(FIELD_SET_FUNC(REGULAR_REG1, | |
| 326 | + REG_BIT(2) | REG_BIT(3), | |
| 327 | + bits_2_3_set)) | |
| 328 | + }, | |
| 329 | + ), | |
| 330 | + }, | |
| 303 | 331 | { |
| 304 | 332 | .name = "no-match-no-add", |
| 305 | 333 | .expected_reg = REGULAR_REG1, |
| @@ -417,6 +445,30 @@ static const struct rtp_to_sr_test_case rtp_to_sr_cases[] = { · 111 unmodified lines | ||
| 417 | 445 | }, |
| 418 | 446 | ), |
| 419 | 447 | }, |
| 448 | + { | |
| 449 | + .name = "conflict-not-disjoint-literal-and-func", | |
| 450 | + .expected_reg = REGULAR_REG1, | |
| 451 | + .expected_set_bits = REG_BIT(1) | REG_BIT(2), | |
| 452 | + .expected_clr_bits = REG_BIT(1) | REG_BIT(2), | |
| 453 | + .expected_active = BIT(0) | BIT(1), | |
| 454 | + .expected_count_sr_entries = 1, | |
| 455 | + .expected_sr_errors = 1, | |
| 456 | + .table = XE_RTP_TABLE_SR( | |
| 457 | + { XE_RTP_NAME("basic-1"), | |
| 458 | + XE_RTP_RULES(FUNC(match_yes)), | |
| 459 | + XE_RTP_ACTIONS(FIELD_SET(REGULAR_REG1, | |
| 460 | + REG_BIT(1) | REG_BIT(2), | |
| 461 | + REG_BIT(1) | REG_BIT(2))) | |
| 462 | + }, | |
| 463 | + /* drop: bits are not disjoint with previous entries */ | |
| 464 | + { XE_RTP_NAME("basic-2"), | |
| 465 | + XE_RTP_RULES(FUNC(match_yes)), | |
| 466 | + XE_RTP_ACTIONS(FIELD_SET_FUNC(REGULAR_REG1, | |
| 467 | + REG_BIT(2) | REG_BIT(3), | |
| 468 | + bits_2_3_set)) | |
| 469 | + }, | |
| 470 | + ), | |
| 471 | + }, | |
| 420 | 472 | { |
| 421 | 473 | .name = "conflict-reg-type", |
| 422 | 474 | .expected_reg = REGULAR_REG1, |
| @@ -43,9 +43,6 @@ static int xe_wa_test_init(struct kunit *test) · 42 unmodified lines | ||
| 43 | 43 | xe_gt_mmio_init(gt); |
| 44 | 44 | } |
| 45 | 45 | |
| 46 | − if (!param->graphics_verx100) | |
| 47 | − xe->info.step = param->step; | |
| 48 | − | |
| 49 | 46 | /* TODO: init hw engines for engine/LRC WAs */ |
| 50 | 47 | xe->drm.dev = dev; |
| 51 | 48 | test->priv = xe; |
| @@ -117,7 +117,6 @@ static int info(struct seq_file *m, void *data) · 116 unmodified lines | ||
| 117 | 117 | drm_printf(&p, "revid %d\n", xe->info.revid); |
| 118 | 118 | drm_printf(&p, "tile_count %d\n", xe->info.tile_count); |
| 119 | 119 | drm_printf(&p, "vm_max_level %d\n", xe->info.vm_max_level); |
| 120 | − drm_printf(&p, "force_execlist %s\n", str_yes_no(xe->info.force_execlist)); | |
| 121 | 120 | drm_printf(&p, "has_flat_ccs %s\n", str_yes_no(xe->info.has_flat_ccs)); |
| 122 | 121 | drm_printf(&p, "has_usm %s\n", str_yes_no(xe->info.has_usm)); |
| 123 | 122 | drm_printf(&p, "skip_guc_pc %s\n", str_yes_no(xe->info.skip_guc_pc)); |
| @@ -61,6 +61,7 @@ · 60 unmodified lines | ||
| 61 | 61 | #include "xe_psmi.h" |
| 62 | 62 | #include "xe_pxp.h" |
| 63 | 63 | #include "xe_query.h" |
| 64 | +#include "xe_ras.h" | |
| 64 | 65 | #include "xe_shrinker.h" |
| 65 | 66 | #include "xe_soc_remapper.h" |
| 66 | 67 | #include "xe_survivability_mode.h" |
| @@ -738,9 +739,11 @@ static void vf_update_device_info(struct xe_device *xe) · 671 unmodified lines | ||
| 738 | 739 | xe->info.probe_display = 0; |
| 739 | 740 | xe->info.has_heci_cscfi = 0; |
| 740 | 741 | xe->info.has_heci_gscfi = 0; |
| 742 | + xe->info.has_i2c = 0; | |
| 741 | 743 | xe->info.has_late_bind = 0; |
| 742 | 744 | xe->info.skip_guc_pc = 1; |
| 743 | 745 | xe->info.skip_pcode = 1; |
| 746 | + xe->info.has_drm_ras = false; | |
| 744 | 747 | } |
| 745 | 748 | |
| 746 | 749 | static int xe_device_vram_alloc(struct xe_device *xe) |
| @@ -949,6 +952,15 @@ int xe_device_probe(struct xe_device *xe) · 202 unmodified lines | ||
| 949 | 952 | return err; |
| 950 | 953 | } |
| 951 | 954 | |
| 955 | + /* | |
| 956 | + * Wa_16029380221: The affected GT will always use non-coherent | |
| 957 | + * access to page tables, so we must do uncached writes from the | |
| 958 | + * CPU. | |
| 959 | + */ | |
| 960 | + for_each_gt(gt, xe, id) | |
| 961 | + if (XE_GT_WA(gt, 16029380221)) | |
| 962 | + xe->info.has_cached_pt = false; | |
| 963 | + | |
| 952 | 964 | for_each_tile(tile, xe, id) { |
| 953 | 965 | err = xe_ggtt_init_early(tile->mem.ggtt); |
| 954 | 966 | if (err) |
| @@ -989,6 +1001,16 @@ int xe_device_probe(struct xe_device *xe) · 34 unmodified lines | ||
| 989 | 1001 | if (err) |
| 990 | 1002 | return err; |
| 991 | 1003 | |
| 1004 | + err = xe_soc_remapper_init(xe); | |
| 1005 | + if (err) | |
| 1006 | + return err; | |
| 1007 | + | |
| 1008 | + err = xe_sysctrl_init(xe); | |
| 1009 | + if (err) | |
| 1010 | + return err; | |
| 1011 | + | |
| 1012 | + xe_ras_init(xe); | |
| 1013 | + | |
| 992 | 1014 | /* |
| 993 | 1015 | * Now that GT is initialized (TTM in particular), |
| 994 | 1016 | * we can try to init display, and inherit the initial fb. |
| @@ -1029,10 +1051,6 @@ int xe_device_probe(struct xe_device *xe) · 34 unmodified lines | ||
| 1029 | 1051 | |
| 1030 | 1052 | xe_nvm_init(xe); |
| 1031 | 1053 | |
| 1032 | − err = xe_soc_remapper_init(xe); | |
| 1033 | − if (err) | |
| 1034 | − return err; | |
| 1035 | − | |
| 1036 | 1054 | err = xe_heci_gsc_init(xe); |
| 1037 | 1055 | if (err) |
| 1038 | 1056 | return err; |
| @@ -1071,10 +1089,6 @@ int xe_device_probe(struct xe_device *xe) · 32 unmodified lines | ||
| 1071 | 1089 | if (err) |
| 1072 | 1090 | goto err_unregister_display; |
| 1073 | 1091 | |
| 1074 | − err = xe_sysctrl_init(xe); | |
| 1075 | − if (err) | |
| 1076 | − goto err_unregister_display; | |
| 1077 | − | |
| 1078 | 1092 | err = xe_device_sysfs_init(xe); |
| 1079 | 1093 | if (err) |
| 1080 | 1094 | goto err_unregister_display; |
| @@ -116,7 +116,7 @@ static inline struct xe_mmio *xe_root_tile_mmio(struct xe_device *xe) · 115 unmodified lines | ||
| 116 | 116 | |
| 117 | 117 | static inline bool xe_device_uc_enabled(struct xe_device *xe) |
| 118 | 118 | { |
| 119 | − return !xe->info.force_execlist; | |
| 119 | + return true; | |
| 120 | 120 | } |
| 121 | 121 | |
| 122 | 122 | #define for_each_tile(tile__, xe__, id__) \ |
| @@ -181,6 +181,21 @@ static inline bool xe_device_has_mert(const struct xe_device *xe) · 58 unmodified lines | ||
| 181 | 181 | return xe->info.has_mert; |
| 182 | 182 | } |
| 183 | 183 | |
| 184 | +static inline bool xe_device_is_in_reset(struct xe_device *xe) | |
| 185 | +{ | |
| 186 | + return atomic_read(&xe->in_reset); | |
| 187 | +} | |
| 188 | + | |
| 189 | +static inline void xe_device_set_in_reset(struct xe_device *xe) | |
| 190 | +{ | |
| 191 | + atomic_set(&xe->in_reset, 1); | |
| 192 | +} | |
| 193 | + | |
| 194 | +static inline void xe_device_clear_in_reset(struct xe_device *xe) | |
| 195 | +{ | |
| 196 | + atomic_set(&xe->in_reset, 0); | |
| 197 | +} | |
| 198 | + | |
| 184 | 199 | u32 xe_device_ccs_bytes(struct xe_device *xe, u64 size); |
| 185 | 200 | |
| 186 | 201 | void xe_device_snapshot_print(struct xe_device *xe, struct drm_printer *p); |
| @@ -144,8 +144,6 @@ struct xe_device { · 143 unmodified lines | ||
| 144 | 144 | * Keep all flags below alphabetically sorted |
| 145 | 145 | */ |
| 146 | 146 | |
| 147 | − /** @info.force_execlist: Forced execlist submission */ | |
| 148 | − u8 force_execlist:1; | |
| 149 | 147 | /** @info.has_access_counter: Device supports access counter */ |
| 150 | 148 | u8 has_access_counter:1; |
| 151 | 149 | /** @info.has_asid: Has address space ID */ |
| @@ -156,6 +154,8 @@ struct xe_device { · 4 unmodified lines | ||
| 156 | 154 | u8 has_cached_pt:1; |
| 157 | 155 | /** @info.has_device_atomics_on_smem: Supports device atomics on SMEM */ |
| 158 | 156 | u8 has_device_atomics_on_smem:1; |
| 157 | + /** @info.has_drm_ras: Device supports drm_ras (Reliability, Availability, Serviceability) */ | |
| 158 | + u8 has_drm_ras:1; | |
| 159 | 159 | /** @info.has_fan_control: Device supports fan control */ |
| 160 | 160 | u8 has_fan_control:1; |
| 161 | 161 | /** @info.has_flat_ccs: Whether flat CCS metadata is used */ |
| @@ -483,6 +483,9 @@ struct xe_device { · 321 unmodified lines | ||
| 483 | 483 | /** @needs_flr_on_fini: requests function-reset on fini */ |
| 484 | 484 | bool needs_flr_on_fini; |
| 485 | 485 | |
| 486 | + /** @in_reset: Indicates if device is in reset */ | |
| 487 | + atomic_t in_reset; | |
| 488 | + | |
| 486 | 489 | /** @wedged: Struct to control Wedged States and mode */ |
| 487 | 490 | struct { |
| 488 | 491 | /** @wedged.flag: Xe device faced a critical error and is now blocked. */ |
| @@ -495,6 +498,9 @@ struct xe_device { · 6 unmodified lines | ||
| 495 | 498 | bool inconsistent_reset; |
| 496 | 499 | } wedged; |
| 497 | 500 | |
| 501 | + /** @devres_group: devres group */ | |
| 502 | + void *devres_group; | |
| 503 | + | |
| 498 | 504 | /** @bo_device: Struct to control async free of BOs */ |
| 499 | 505 | struct xe_bo_dev { |
| 500 | 506 | /** @bo_device.async_free: Free worker */ |
| @@ -193,7 +193,7 @@ static void bo_meminfo(struct xe_bo *bo, · 192 unmodified lines | ||
| 193 | 193 | if (!dma_resv_test_signaled(bo->ttm.base.resv, |
| 194 | 194 | DMA_RESV_USAGE_BOOKKEEP)) |
| 195 | 195 | stats[mem_type].active += sz; |
| 196 | − else if (mem_type == XE_PL_SYSTEM) | |
| 196 | + else if (mem_type == XE_PL_SYSTEM || xe_bo_madv_is_dontneed(bo)) | |
| 197 | 197 | stats[mem_type].purgeable += sz; |
| 198 | 198 | } |
| 199 | 199 | } |
| @@ -273,8 +273,7 @@ static void show_meminfo(struct drm_printer *p, struct drm_file *file) · 73 unmodified lines | ||
| 273 | 273 | &stats[mem_type], |
| 274 | 274 | DRM_GEM_OBJECT_ACTIVE | |
| 275 | 275 | DRM_GEM_OBJECT_RESIDENT | |
| 276 | − (mem_type != XE_PL_SYSTEM ? 0 : | |
| 277 | − DRM_GEM_OBJECT_PURGEABLE), | |
| 276 | + DRM_GEM_OBJECT_PURGEABLE, | |
| 278 | 277 | xe_mem_type_to_name[mem_type]); |
| 279 | 278 | } |
| 280 | 279 | } |
| @@ -11,27 +11,46 @@ · 10 unmodified lines | ||
| 11 | 11 | |
| 12 | 12 | #include "xe_device_types.h" |
| 13 | 13 | #include "xe_drm_ras.h" |
| 14 | +#include "xe_ras.h" | |
| 14 | 15 | |
| 15 | 16 | static const char * const error_components[] = DRM_XE_RAS_ERROR_COMPONENT_NAMES; |
| 16 | 17 | static const char * const error_severity[] = DRM_XE_RAS_ERROR_SEVERITY_NAMES; |
| 17 | 18 | |
| 18 | −static int hw_query_error_counter(struct xe_drm_ras_counter *info, | |
| 19 | − u32 error_id, const char **name, u32 *val) | |
| 19 | +static int query_error_counter(struct xe_device *xe, | |
| 20 | + enum drm_xe_ras_error_severity severity, | |
| 21 | + u32 error_id, const char **name, u32 *val) | |
| 20 | 22 | { |
| 23 | + struct xe_drm_ras *ras = &xe->ras; | |
| 24 | + struct xe_drm_ras_counter *info = ras->info[severity]; | |
| 25 | + | |
| 21 | 26 | if (!info || !info[error_id].name) |
| 22 | 27 | return -ENOENT; |
| 23 | 28 | |
| 24 | 29 | *name = info[error_id].name; |
| 30 | + | |
| 31 | + /* Fetch counter from system controller if supported */ | |
| 32 | + if (xe->info.has_sysctrl) | |
| 33 | + return xe_ras_get_counter(xe, severity, error_id, val); | |
| 34 | + | |
| 25 | 35 | *val = atomic_read(&info[error_id].counter); |
| 26 | 36 | |
| 27 | 37 | return 0; |
| 28 | 38 | } |
| 29 | 39 | |
| 30 | −static int hw_clear_error_counter(struct xe_drm_ras_counter *info, u32 error_id) | |
| 40 | +static int clear_error_counter(struct xe_device *xe, | |
| 41 | + enum drm_xe_ras_error_severity severity, | |
| 42 | + u32 error_id) | |
| 31 | 43 | { |
| 44 | + struct xe_drm_ras *ras = &xe->ras; | |
| 45 | + struct xe_drm_ras_counter *info = ras->info[severity]; | |
| 46 | + | |
| 32 | 47 | if (!info || !info[error_id].name) |
| 33 | 48 | return -ENOENT; |
| 34 | 49 | |
| 50 | + /* Clear counter from system controller if supported */ | |
| 51 | + if (xe->info.has_sysctrl) | |
| 52 | + return xe_ras_clear_counter(xe, severity, error_id); | |
| 53 | + | |
| 35 | 54 | atomic_set(&info[error_id].counter, 0); |
| 36 | 55 | |
| 37 | 56 | return 0; |
| @@ -41,38 +60,30 @@ static int query_uncorrectable_error_counter(struct drm_ras_node *ep, u32 error_ · 3 unmodified lines | ||
| 41 | 60 | const char **name, u32 *val) |
| 42 | 61 | { |
| 43 | 62 | struct xe_device *xe = ep->priv; |
| 44 | − struct xe_drm_ras *ras = &xe->ras; | |
| 45 | − struct xe_drm_ras_counter *info = ras->info[DRM_XE_RAS_ERR_SEV_UNCORRECTABLE]; | |
| 46 | 63 | |
| 47 | − return hw_query_error_counter(info, error_id, name, val); | |
| 64 | + return query_error_counter(xe, DRM_XE_RAS_ERR_SEV_UNCORRECTABLE, error_id, name, val); | |
| 48 | 65 | } |
| 49 | 66 | |
| 50 | 67 | static int clear_uncorrectable_error_counter(struct drm_ras_node *node, u32 error_id) |
| 51 | 68 | { |
| 52 | 69 | struct xe_device *xe = node->priv; |
| 53 | − struct xe_drm_ras *ras = &xe->ras; | |
| 54 | − struct xe_drm_ras_counter *info = ras->info[DRM_XE_RAS_ERR_SEV_UNCORRECTABLE]; | |
| 55 | 70 | |
| 56 | − return hw_clear_error_counter(info, error_id); | |
| 71 | + return clear_error_counter(xe, DRM_XE_RAS_ERR_SEV_UNCORRECTABLE, error_id); | |
| 57 | 72 | } |
| 58 | 73 | |
| 59 | 74 | static int query_correctable_error_counter(struct drm_ras_node *ep, u32 error_id, |
| 60 | 75 | const char **name, u32 *val) |
| 61 | 76 | { |
| 62 | 77 | struct xe_device *xe = ep->priv; |
| 63 | − struct xe_drm_ras *ras = &xe->ras; | |
| 64 | − struct xe_drm_ras_counter *info = ras->info[DRM_XE_RAS_ERR_SEV_CORRECTABLE]; | |
| 65 | 78 | |
| 66 | − return hw_query_error_counter(info, error_id, name, val); | |
| 79 | + return query_error_counter(xe, DRM_XE_RAS_ERR_SEV_CORRECTABLE, error_id, name, val); | |
| 67 | 80 | } |
| 68 | 81 | |
| 69 | 82 | static int clear_correctable_error_counter(struct drm_ras_node *node, u32 error_id) |
| 70 | 83 | { |
| 71 | 84 | struct xe_device *xe = node->priv; |
| 72 | − struct xe_drm_ras *ras = &xe->ras; | |
| 73 | − struct xe_drm_ras_counter *info = ras->info[DRM_XE_RAS_ERR_SEV_CORRECTABLE]; | |
| 74 | 85 | |
| 75 | − return hw_clear_error_counter(info, error_id); | |
| 86 | + return clear_error_counter(xe, DRM_XE_RAS_ERR_SEV_CORRECTABLE, error_id); | |
| 76 | 87 | } |
| 77 | 88 | |
| 78 | 89 | static struct xe_drm_ras_counter *allocate_and_copy_counters(struct xe_device *xe) |
| @@ -985,9 +985,10 @@ int xe_eu_stall_stream_open(struct drm_device *dev, u64 data, struct drm_file *f · 984 unmodified lines | ||
| 985 | 985 | return -ENODEV; |
| 986 | 986 | } |
| 987 | 987 | |
| 988 | − if (xe_observation_paranoid && !perfmon_capable()) { | |
| 988 | + ret = xe_observation_paranoid_check(); | |
| 989 | + if (ret) { | |
| 989 | 990 | drm_dbg(&xe->drm, "Insufficient privileges for EU stall monitoring\n"); |
| 990 | − return -EACCES; | |
| 991 | + return ret; | |
| 991 | 992 | } |
| 992 | 993 | |
| 993 | 994 | /* Initialize and set default values */ |
| @@ -318,8 +318,6 @@ struct xe_exec_queue_ops { · 317 unmodified lines | ||
| 318 | 318 | void (*resume)(struct xe_exec_queue *q); |
| 319 | 319 | /** @reset_status: check exec queue reset status */ |
| 320 | 320 | bool (*reset_status)(struct xe_exec_queue *q); |
| 321 | − /** @active: check exec queue is active */ | |
| 322 | − bool (*active)(struct xe_exec_queue *q); | |
| 323 | 321 | }; |
| 324 | 322 | |
| 325 | 323 | #endif |
| @@ -458,12 +458,6 @@ static bool execlist_exec_queue_reset_status(struct xe_exec_queue *q) · 457 unmodified lines | ||
| 458 | 458 | return false; |
| 459 | 459 | } |
| 460 | 460 | |
| 461 | −static bool execlist_exec_queue_active(struct xe_exec_queue *q) | |
| 462 | −{ | |
| 463 | − /* NIY */ | |
| 464 | − return false; | |
| 465 | −} | |
| 466 | − | |
| 467 | 461 | static const struct xe_exec_queue_ops execlist_exec_queue_ops = { |
| 468 | 462 | .init = execlist_exec_queue_init, |
| 469 | 463 | .kill = execlist_exec_queue_kill, |
| @@ -476,7 +470,6 @@ static const struct xe_exec_queue_ops execlist_exec_queue_ops = { · 6 unmodified lines | ||
| 476 | 470 | .suspend_wait = execlist_exec_queue_suspend_wait, |
| 477 | 471 | .resume = execlist_exec_queue_resume, |
| 478 | 472 | .reset_status = execlist_exec_queue_reset_status, |
| 479 | − .active = execlist_exec_queue_active, | |
| 480 | 473 | }; |
| 481 | 474 | |
| 482 | 475 | int xe_execlist_init(struct xe_gt *gt) |
| @@ -111,14 +111,14 @@ struct xe_ggtt_pt_ops { · 110 unmodified lines | ||
| 111 | 111 | struct xe_ggtt { |
| 112 | 112 | /** @tile: Back pointer to tile where this GGTT belongs */ |
| 113 | 113 | struct xe_tile *tile; |
| 114 | − /** @start: Start offset of GGTT */ | |
| 114 | + /** @start: Start offset of GGTT */ | |
| 115 | 115 | u64 start; |
| 116 | 116 | /** @size: Total usable size of this GGTT */ |
| 117 | 117 | u64 size; |
| 118 | − | |
| 119 | 118 | /** |
| 120 | − * @flags: Flags for this GGTT | |
| 119 | + * @flags: Flags for this GGTT. | |
| 121 | 120 | * Acceptable flags: |
| 121 | + * | |
| 122 | 122 | * - %XE_GGTT_FLAGS_64K - if PTE size is 64K. Otherwise, regular is 4K. |
| 123 | 123 | * - %XE_GGTT_FLAGS_ONLINE - is GGTT online, protected by ggtt->lock |
| 124 | 124 | * after init |
| @@ -129,7 +129,7 @@ struct xe_ggtt { · 4 unmodified lines | ||
| 129 | 129 | /** @lock: Mutex lock to protect GGTT data */ |
| 130 | 130 | struct mutex lock; |
| 131 | 131 | /** |
| 132 | − * @gsm: The iomem pointer to the actual location of the translation | |
| 132 | + * @gsm: The iomem pointer to the actual location of the translation | |
| 133 | 133 | * table located in the GSM for easy PTE manipulation |
| 134 | 134 | */ |
| 135 | 135 | u64 __iomem *gsm; |
| @@ -917,6 +917,9 @@ static void gt_reset_worker(struct work_struct *w) · 916 unmodified lines | ||
| 917 | 917 | if (xe_device_wedged(gt_to_xe(gt))) |
| 918 | 918 | goto err_pm_put; |
| 919 | 919 | |
| 920 | + if (xe_device_is_in_reset(gt_to_xe(gt))) | |
| 921 | + goto err_pm_put; | |
| 922 | + | |
| 920 | 923 | /* We only support GT resets with GuC submission */ |
| 921 | 924 | if (!xe_device_uc_enabled(gt_to_xe(gt))) |
| 922 | 925 | goto err_pm_put; |
| @@ -977,18 +980,21 @@ err_pm_put: · 54 unmodified lines | ||
| 977 | 980 | |
| 978 | 981 | void xe_gt_reset_async(struct xe_gt *gt) |
| 979 | 982 | { |
| 980 | − xe_gt_info(gt, "trying reset from %ps\n", __builtin_return_address(0)); | |
| 983 | + struct xe_device *xe = gt_to_xe(gt); | |
| 984 | + | |
| 985 | + if (xe_device_is_in_reset(xe)) | |
| 986 | + return; | |
| 981 | 987 | |
| 982 | 988 | /* Don't do a reset while one is already in flight */ |
| 983 | 989 | if (!xe_fault_inject_gt_reset() && xe_uc_reset_prepare(>->uc)) |
| 984 | 990 | return; |
| 985 | 991 | |
| 986 | − xe_gt_info(gt, "reset queued\n"); | |
| 992 | + xe_gt_info(gt, "reset queued from %ps\n", __builtin_return_address(0)); | |
| 987 | 993 | |
| 988 | 994 | /* Pair with put in gt_reset_worker() if work is enqueued */ |
| 989 | − xe_pm_runtime_get_noresume(gt_to_xe(gt)); | |
| 995 | + xe_pm_runtime_get_noresume(xe); | |
| 990 | 996 | if (!queue_work(gt->ordered_wq, >->reset.worker)) |
| 991 | − xe_pm_runtime_put(gt_to_xe(gt)); | |
| 997 | + xe_pm_runtime_put(xe); | |
| 992 | 998 | } |
| 993 | 999 | |
| 994 | 1000 | void xe_gt_suspend_prepare(struct xe_gt *gt) |
| @@ -404,8 +404,7 @@ fallback: · 403 unmodified lines | ||
| 404 | 404 | * Some older platforms don't have tables or don't have complete tables. |
| 405 | 405 | * Newer platforms should always have the required info. |
| 406 | 406 | */ |
| 407 | − if (GRAPHICS_VERx100(gt_to_xe(gt)) >= 2000 && | |
| 408 | − !gt_to_xe(gt)->info.force_execlist) | |
| 407 | + if (GRAPHICS_VERx100(gt_to_xe(gt)) >= 2000) | |
| 409 | 408 | xe_gt_err(gt, "Slice/Subslice counts missing from hwconfig table; using typical fallback values\n"); |
| 410 | 409 | |
| 411 | 410 | if (gt_to_xe(gt)->info.platform == XE_PVC) |
| @@ -507,7 +506,7 @@ void xe_gt_mcr_init_early(struct xe_gt *gt) · 95 unmodified lines | ||
| 507 | 506 | spin_lock_init(>->mcr_lock); |
| 508 | 507 | |
| 509 | 508 | if (gt->info.type == XE_GT_TYPE_MEDIA) { |
| 510 | − drm_WARN_ON(&xe->drm, MEDIA_VER(xe) < 13); | |
| 509 | + xe_gt_WARN_ON(gt, MEDIA_VER(xe) < 13); | |
| 511 | 510 | |
| 512 | 511 | if (MEDIA_VER(xe) >= 30) { |
| 513 | 512 | gt->steering[OADDRM].ranges = xe2lpm_gpmxmt_steering_table; |
| @@ -662,9 +661,9 @@ bool xe_gt_mcr_get_nonterminated_steering(struct xe_gt *gt, · 148 unmodified lines | ||
| 662 | 661 | |
| 663 | 662 | for (int type = 0; type < IMPLICIT_STEERING; type++) { |
| 664 | 663 | if (reg_in_steering_type_ranges(gt, reg, type)) { |
| 665 | − drm_WARN(>_to_xe(gt)->drm, !gt->steering[type].initialized, | |
| 666 | − "Uninitialized usage of MCR register %s/%#x\n", | |
| 667 | − xe_steering_types[type].name, reg.addr); | |
| 664 | + xe_gt_WARN(gt, !gt->steering[type].initialized, | |
| 665 | + "Uninitialized usage of MCR register %s/%#x\n", | |
| 666 | + xe_steering_types[type].name, reg.addr); | |
| 668 | 667 | |
| 669 | 668 | *group = gt->steering[type].group_target; |
| 670 | 669 | *instance = gt->steering[type].instance_target; |
| @@ -679,9 +678,9 @@ bool xe_gt_mcr_get_nonterminated_steering(struct xe_gt *gt, · 8 unmodified lines | ||
| 679 | 678 | * Not found in a steering table and not a register with implicit |
| 680 | 679 | * steering. Just steer to 0/0 as a guess and raise a warning. |
| 681 | 680 | */ |
| 682 | − drm_WARN(>_to_xe(gt)->drm, true, | |
| 683 | − "Did not find MCR register %#x in any MCR steering table\n", | |
| 684 | − reg.addr); | |
| 681 | + xe_gt_WARN(gt, true, | |
| 682 | + "Did not find MCR register %#x in any MCR steering table\n", | |
| 683 | + reg.addr); | |
| 685 | 684 | *group = 0; |
| 686 | 685 | *instance = 0; |
| 687 | 686 | |
| @@ -710,7 +709,7 @@ static void mcr_lock(struct xe_gt *gt) __acquires(>->mcr_lock) · 22 unmodified lines | ||
| 710 | 709 | ret = xe_mmio_wait32(>->mmio, STEER_SEMAPHORE, 0x1, 0x1, 10, NULL, |
| 711 | 710 | true); |
| 712 | 711 | |
| 713 | − drm_WARN_ON_ONCE(&xe->drm, ret == -ETIMEDOUT); | |
| 712 | + xe_gt_WARN_ON_ONCE(gt, ret == -ETIMEDOUT); | |
| 714 | 713 | } |
| 715 | 714 | |
| 716 | 715 | static void mcr_unlock(struct xe_gt *gt) __releases(>->mcr_lock) |
| @@ -1532,7 +1532,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · 1531 unmodified lines | ||
| 1532 | 1532 | * If devcoredump not captured and GuC capture for the job is not ready |
| 1533 | 1533 | * do manual capture first and decide later if we need to use it |
| 1534 | 1534 | */ |
| 1535 | − if (!exec_queue_killed(q) && !xe->devcoredump.captured && | |
| 1535 | + if (!xe_device_is_in_reset(xe) && !exec_queue_killed(q) && !xe->devcoredump.captured && | |
| 1536 | 1536 | !xe_guc_capture_get_matching_and_lock(q)) { |
| 1537 | 1537 | /* take force wake before engine register manual capture */ |
| 1538 | 1538 | CLASS(xe_force_wake, fw_ref)(gt_to_fw(q->gt), XE_FORCEWAKE_ALL); |
| @@ -1554,8 +1554,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · 15 unmodified lines | ||
| 1554 | 1554 | set_exec_queue_banned(q); |
| 1555 | 1555 | |
| 1556 | 1556 | /* Kick job / queue off hardware */ |
| 1557 | − if (!wedged && (exec_queue_enabled(primary) || | |
| 1558 | − exec_queue_pending_disable(primary))) { | |
| 1557 | + if (!xe_device_is_in_reset(xe) && !wedged && | |
| 1558 | + (exec_queue_enabled(primary) || exec_queue_pending_disable(primary))) { | |
| 1559 | 1559 | int ret; |
| 1560 | 1560 | |
| 1561 | 1561 | if (exec_queue_reset(primary)) |
| @@ -1623,7 +1623,8 @@ trigger_reset: · 61 unmodified lines | ||
| 1623 | 1623 | |
| 1624 | 1624 | trace_xe_sched_job_timedout(job); |
| 1625 | 1625 | |
| 1626 | − if (!exec_queue_killed(q)) | |
| 1626 | + /* Do not access device if in reset */ | |
| 1627 | + if (!xe_device_is_in_reset(xe) && !exec_queue_killed(q)) | |
| 1627 | 1628 | xe_devcoredump(q, job, |
| 1628 | 1629 | "Timedout job - seqno=%u, lrc_seqno=%u, guc_id=%d, flags=0x%lx", |
| 1629 | 1630 | xe_sched_job_seqno(job), xe_sched_job_lrc_seqno(job), |
| @@ -2244,14 +2245,6 @@ static bool guc_exec_queue_reset_status(struct xe_exec_queue *q) · 614 unmodified lines | ||
| 2244 | 2245 | return exec_queue_reset(q) || exec_queue_killed_or_banned_or_wedged(q); |
| 2245 | 2246 | } |
| 2246 | 2247 | |
| 2247 | −static bool guc_exec_queue_active(struct xe_exec_queue *q) | |
| 2248 | −{ | |
| 2249 | − struct xe_exec_queue *primary = xe_exec_queue_multi_queue_primary(q); | |
| 2250 | − | |
| 2251 | − return exec_queue_enabled(primary) && | |
| 2252 | − !exec_queue_pending_disable(primary); | |
| 2253 | −} | |
| 2254 | − | |
| 2255 | 2248 | /* |
| 2256 | 2249 | * All of these functions are an abstraction layer which other parts of Xe can |
| 2257 | 2250 | * use to trap into the GuC backend. All of these functions, aside from init, |
| @@ -2271,7 +2264,6 @@ static const struct xe_exec_queue_ops guc_exec_queue_ops = { · 13 unmodified lines | ||
| 2271 | 2264 | .suspend_wait = guc_exec_queue_suspend_wait, |
| 2272 | 2265 | .resume = guc_exec_queue_resume, |
| 2273 | 2266 | .reset_status = guc_exec_queue_reset_status, |
| 2274 | − .active = guc_exec_queue_active, | |
| 2275 | 2267 | }; |
| 2276 | 2268 | |
| 2277 | 2269 | static void guc_exec_queue_stop(struct xe_guc *guc, struct xe_exec_queue *q) |
| @@ -208,9 +208,6 @@ static int send_tlb_inval_asid_ppgtt(struct xe_tlb_inval *tlb_inval, u32 seqno, · 207 unmodified lines | ||
| 208 | 208 | |
| 209 | 209 | lockdep_assert_held(&tlb_inval->seqno_lock); |
| 210 | 210 | |
| 211 | − if (guc_to_xe(guc)->info.force_execlist) | |
| 212 | − return -ECANCELED; | |
| 213 | − | |
| 214 | 211 | return send_tlb_inval_ppgtt(guc, seqno, start, end, asid, |
| 215 | 212 | XE_GUC_TLB_INVAL_PAGE_SELECTIVE, prl_sa); |
| 216 | 213 | } |
| @@ -228,9 +225,6 @@ static int send_tlb_inval_ctx_ppgtt(struct xe_tlb_inval *tlb_inval, u32 seqno, · 11 unmodified lines | ||
| 228 | 225 | |
| 229 | 226 | lockdep_assert_held(&tlb_inval->seqno_lock); |
| 230 | 227 | |
| 231 | − if (xe->info.force_execlist) | |
| 232 | − return -ECANCELED; | |
| 233 | − | |
| 234 | 228 | vm = xe_device_asid_to_vm(xe, asid); |
| 235 | 229 | if (IS_ERR(vm)) |
| 236 | 230 | return PTR_ERR(vm); |
| @@ -337,39 +337,41 @@ static bool xe_rtp_cfeg_wmtp_disabled(const struct xe_device *xe, · 336 unmodified lines | ||
| 337 | 337 | return xe_mmio_read32(&hwe->gt->mmio, XEHP_FUSE4) & CFEG_WMTP_DISABLE; |
| 338 | 338 | } |
| 339 | 339 | |
| 340 | +static u32 blit_cctl_val(struct xe_gt *gt, struct xe_hw_engine *hwe) | |
| 341 | +{ | |
| 342 | + return REG_FIELD_PREP(BLIT_CCTL_DST_MOCS_MASK, gt->mocs.uc_index) | | |
| 343 | + REG_FIELD_PREP(BLIT_CCTL_SRC_MOCS_MASK, gt->mocs.uc_index); | |
| 344 | +} | |
| 345 | + | |
| 346 | +static const struct xe_rtp_table_sr lrc_setup = XE_RTP_TABLE_SR( | |
| 347 | + /* | |
| 348 | + * Some blitter commands do not have a field for MOCS, those | |
| 349 | + * commands will use MOCS index pointed by BLIT_CCTL. | |
| 350 | + * BLIT_CCTL registers are needed to be programmed to un-cached. | |
| 351 | + */ | |
| 352 | + { XE_RTP_NAME("BLIT_CCTL_default_MOCS"), | |
| 353 | + XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1200, 1274), | |
| 354 | + ENGINE_CLASS(COPY)), | |
| 355 | + XE_RTP_ACTIONS(FIELD_SET_FUNC(BLIT_CCTL(0), | |
| 356 | + BLIT_CCTL_DST_MOCS_MASK | | |
| 357 | + BLIT_CCTL_SRC_MOCS_MASK, | |
| 358 | + blit_cctl_val, | |
| 359 | + XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 360 | + }, | |
| 361 | + /* Disable WMTP if HW doesn't support it */ | |
| 362 | + { XE_RTP_NAME("DISABLE_WMTP_ON_UNSUPPORTED_HW"), | |
| 363 | + XE_RTP_RULES(FUNC(xe_rtp_cfeg_wmtp_disabled)), | |
| 364 | + XE_RTP_ACTIONS(FIELD_SET(CS_CHICKEN1(0), | |
| 365 | + PREEMPT_GPGPU_LEVEL_MASK, | |
| 366 | + PREEMPT_GPGPU_THREAD_GROUP_LEVEL)), | |
| 367 | + XE_RTP_ENTRY_FLAG(FOREACH_ENGINE) | |
| 368 | + }, | |
| 369 | +); | |
| 370 | + | |
| 340 | 371 | static void |
| 341 | 372 | hw_engine_setup_default_lrc_state(struct xe_hw_engine *hwe) |
| 342 | 373 | { |
| 343 | − struct xe_gt *gt = hwe->gt; | |
| 344 | − const u8 mocs_write_idx = gt->mocs.uc_index; | |
| 345 | − const u8 mocs_read_idx = gt->mocs.uc_index; | |
| 346 | − u32 blit_cctl_val = REG_FIELD_PREP(BLIT_CCTL_DST_MOCS_MASK, mocs_write_idx) | | |
| 347 | − REG_FIELD_PREP(BLIT_CCTL_SRC_MOCS_MASK, mocs_read_idx); | |
| 348 | 374 | struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe); |
| 349 | − const struct xe_rtp_table_sr lrc_setup = XE_RTP_TABLE_SR( | |
| 350 | − /* | |
| 351 | − * Some blitter commands do not have a field for MOCS, those | |
| 352 | − * commands will use MOCS index pointed by BLIT_CCTL. | |
| 353 | − * BLIT_CCTL registers are needed to be programmed to un-cached. | |
| 354 | − */ | |
| 355 | − { XE_RTP_NAME("BLIT_CCTL_default_MOCS"), | |
| 356 | − XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1200, 1274), | |
| 357 | − ENGINE_CLASS(COPY)), | |
| 358 | − XE_RTP_ACTIONS(FIELD_SET(BLIT_CCTL(0), | |
| 359 | − BLIT_CCTL_DST_MOCS_MASK | | |
| 360 | − BLIT_CCTL_SRC_MOCS_MASK, | |
| 361 | − blit_cctl_val, | |
| 362 | − XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 363 | − }, | |
| 364 | − /* Disable WMTP if HW doesn't support it */ | |
| 365 | − { XE_RTP_NAME("DISABLE_WMTP_ON_UNSUPPORTED_HW"), | |
| 366 | − XE_RTP_RULES(FUNC(xe_rtp_cfeg_wmtp_disabled)), | |
| 367 | − XE_RTP_ACTIONS(FIELD_SET(CS_CHICKEN1(0), | |
| 368 | − PREEMPT_GPGPU_LEVEL_MASK, | |
| 369 | − PREEMPT_GPGPU_THREAD_GROUP_LEVEL)), | |
| 370 | − XE_RTP_ENTRY_FLAG(FOREACH_ENGINE) | |
| 371 | − }, | |
| 372 | − ); | |
| 373 | 375 | |
| 374 | 376 | xe_rtp_process_to_sr(&ctx, &lrc_setup, &hwe->reg_lrc, true); |
| 375 | 377 | } |
| @@ -385,86 +387,92 @@ void xe_hw_engine_setup_reg_lrc(struct xe_hw_engine *hwe) · 9 unmodified lines | ||
| 385 | 387 | xe_tuning_process_lrc(hwe); |
| 386 | 388 | } |
| 387 | 389 | |
| 388 | −static void | |
| 389 | −hw_engine_setup_default_state(struct xe_hw_engine *hwe) | |
| 390 | +/* | |
| 391 | + * RING_CMD_CCTL specifies the default MOCS entry that will be | |
| 392 | + * used by the command streamer when executing commands that | |
| 393 | + * don't have a way to explicitly specify a MOCS setting. | |
| 394 | + * The default should usually reference whichever MOCS entry | |
| 395 | + * corresponds to uncached behavior, although use of a WB cached | |
| 396 | + * entry is recommended by the spec in certain circumstances on | |
| 397 | + * specific platforms. | |
| 398 | + * Bspec: 72161 | |
| 399 | + */ | |
| 400 | +static u32 ring_cmd_cctl_val(struct xe_gt *gt, struct xe_hw_engine *hwe) | |
| 390 | 401 | { |
| 391 | − struct xe_gt *gt = hwe->gt; | |
| 392 | 402 | struct xe_device *xe = gt_to_xe(gt); |
| 403 | + u8 mocs_read_idx = gt->mocs.uc_index; | |
| 404 | + | |
| 405 | + if (hwe->class == XE_ENGINE_CLASS_COMPUTE && IS_DGFX(xe) && | |
| 406 | + (GRAPHICS_VER(xe) >= 20 || xe->info.platform == XE_PVC)) | |
| 407 | + mocs_read_idx = gt->mocs.wb_index; | |
| 408 | + | |
| 409 | + return REG_FIELD_PREP(CMD_CCTL_WRITE_OVERRIDE_MASK, gt->mocs.uc_index) | | |
| 410 | + REG_FIELD_PREP(CMD_CCTL_READ_OVERRIDE_MASK, mocs_read_idx); | |
| 411 | +} | |
| 412 | + | |
| 413 | +static const struct xe_rtp_table_sr engine_sr = XE_RTP_TABLE_SR( | |
| 414 | + { XE_RTP_NAME("RING_CMD_CCTL_default_MOCS"), | |
| 415 | + XE_RTP_RULES(FUNC(xe_rtp_match_always)), | |
| 416 | + XE_RTP_ACTIONS(FIELD_SET_FUNC(RING_CMD_CCTL(0), | |
| 417 | + CMD_CCTL_WRITE_OVERRIDE_MASK | | |
| 418 | + CMD_CCTL_READ_OVERRIDE_MASK, | |
| 419 | + ring_cmd_cctl_val, | |
| 420 | + XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 421 | + }, | |
| 422 | + { XE_RTP_NAME("Disable HW status page updates for interrupts"), | |
| 423 | + XE_RTP_RULES(FUNC(xe_rtp_match_always)), | |
| 424 | + XE_RTP_ACTIONS(SET(RING_HWSTAM(0), ~0x0, | |
| 425 | + XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 426 | + }, | |
| 427 | + { XE_RTP_NAME("Disable engine 'legacy' mode"), | |
| 428 | + XE_RTP_RULES(FUNC(xe_rtp_match_always)), | |
| 429 | + XE_RTP_ACTIONS(SET(GFX_MODE(0), GFX_DISABLE_LEGACY_MODE, | |
| 430 | + XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 431 | + }, | |
| 393 | 432 | /* |
| 394 | − * RING_CMD_CCTL specifies the default MOCS entry that will be | |
| 395 | − * used by the command streamer when executing commands that | |
| 396 | − * don't have a way to explicitly specify a MOCS setting. | |
| 397 | − * The default should usually reference whichever MOCS entry | |
| 398 | − * corresponds to uncached behavior, although use of a WB cached | |
| 399 | − * entry is recommended by the spec in certain circumstances on | |
| 400 | − * specific platforms. | |
| 401 | − * Bspec: 72161 | |
| 433 | + * To allow the GSC engine to go idle on MTL we need to enable | |
| 434 | + * idle messaging and set the hysteresis value (we use 0xA=5us | |
| 435 | + * as recommended in spec). On platforms after MTL this is | |
| 436 | + * enabled by default. | |
| 402 | 437 | */ |
| 403 | − const u8 mocs_write_idx = gt->mocs.uc_index; | |
| 404 | − const u8 mocs_read_idx = hwe->class == XE_ENGINE_CLASS_COMPUTE && IS_DGFX(xe) && | |
| 405 | − (GRAPHICS_VER(xe) >= 20 || xe->info.platform == XE_PVC) ? | |
| 406 | − gt->mocs.wb_index : gt->mocs.uc_index; | |
| 407 | − u32 ring_cmd_cctl_val = REG_FIELD_PREP(CMD_CCTL_WRITE_OVERRIDE_MASK, mocs_write_idx) | | |
| 408 | − REG_FIELD_PREP(CMD_CCTL_READ_OVERRIDE_MASK, mocs_read_idx); | |
| 438 | + { XE_RTP_NAME("MTL GSCCS IDLE MSG enable"), | |
| 439 | + XE_RTP_RULES(MEDIA_VERSION(1300), ENGINE_CLASS(OTHER)), | |
| 440 | + XE_RTP_ACTIONS(CLR(RING_PSMI_CTL(0), | |
| 441 | + IDLE_MSG_DISABLE, | |
| 442 | + XE_RTP_ACTION_FLAG(ENGINE_BASE)), | |
| 443 | + FIELD_SET(RING_PWRCTX_MAXCNT(0), | |
| 444 | + IDLE_WAIT_TIME, | |
| 445 | + 0xA, | |
| 446 | + XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 447 | + }, | |
| 448 | + /* Enable Priority Mem Read */ | |
| 449 | + { XE_RTP_NAME("Priority_Mem_Read"), | |
| 450 | + XE_RTP_RULES(GRAPHICS_VERSION_RANGE(2001, XE_RTP_END_VERSION_UNDEFINED)), | |
| 451 | + XE_RTP_ACTIONS(SET(CSFE_CHICKEN1(0), CS_PRIORITY_MEM_READ, | |
| 452 | + XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 453 | + }, | |
| 454 | + { XE_RTP_NAME("Enable CCS Engine(s)"), | |
| 455 | + XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1255, XE_RTP_END_VERSION_UNDEFINED), | |
| 456 | + FUNC(xe_rtp_match_first_render_or_compute)), | |
| 457 | + XE_RTP_ACTIONS(SET(RCU_MODE, RCU_MODE_CCS_ENABLE)) | |
| 458 | + }, | |
| 459 | + /* Use Fixed slice CCS mode */ | |
| 460 | + { XE_RTP_NAME("RCU_MODE_FIXED_SLICE_CCS_MODE"), | |
| 461 | + XE_RTP_RULES(FUNC(xe_hw_engine_match_fixed_cslice_mode)), | |
| 462 | + XE_RTP_ACTIONS(FIELD_SET(RCU_MODE, RCU_MODE_FIXED_SLICE_CCS_MODE, | |
| 463 | + RCU_MODE_FIXED_SLICE_CCS_MODE)) | |
| 464 | + }, | |
| 465 | + { XE_RTP_NAME("Enable MSI-X interrupt support"), | |
| 466 | + XE_RTP_RULES(FUNC(xe_rtp_match_has_msix)), | |
| 467 | + XE_RTP_ACTIONS(SET(GFX_MODE(0), GFX_MSIX_INTERRUPT_ENABLE, | |
| 468 | + XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 469 | + }, | |
| 470 | +); | |
| 471 | + | |
| 472 | +static void | |
| 473 | +hw_engine_setup_default_state(struct xe_hw_engine *hwe) | |
| 474 | +{ | |
| 409 | 475 | struct xe_rtp_process_ctx ctx = XE_RTP_PROCESS_CTX_INITIALIZER(hwe); |
| 410 | − const struct xe_rtp_table_sr engine_sr = XE_RTP_TABLE_SR( | |
| 411 | − { XE_RTP_NAME("RING_CMD_CCTL_default_MOCS"), | |
| 412 | − XE_RTP_RULES(FUNC(xe_rtp_match_always)), | |
| 413 | − XE_RTP_ACTIONS(FIELD_SET(RING_CMD_CCTL(0), | |
| 414 | − CMD_CCTL_WRITE_OVERRIDE_MASK | | |
| 415 | − CMD_CCTL_READ_OVERRIDE_MASK, | |
| 416 | − ring_cmd_cctl_val, | |
| 417 | − XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 418 | − }, | |
| 419 | − { XE_RTP_NAME("Disable HW status page updates for interrupts"), | |
| 420 | − XE_RTP_RULES(FUNC(xe_rtp_match_always)), | |
| 421 | − XE_RTP_ACTIONS(SET(RING_HWSTAM(0), ~0x0, | |
| 422 | − XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 423 | − }, | |
| 424 | − { XE_RTP_NAME("Disable engine 'legacy' mode"), | |
| 425 | − XE_RTP_RULES(FUNC(xe_rtp_match_always)), | |
| 426 | − XE_RTP_ACTIONS(SET(GFX_MODE(0), GFX_DISABLE_LEGACY_MODE, | |
| 427 | − XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 428 | − }, | |
| 429 | − /* | |
| 430 | − * To allow the GSC engine to go idle on MTL we need to enable | |
| 431 | − * idle messaging and set the hysteresis value (we use 0xA=5us | |
| 432 | − * as recommended in spec). On platforms after MTL this is | |
| 433 | − * enabled by default. | |
| 434 | − */ | |
| 435 | − { XE_RTP_NAME("MTL GSCCS IDLE MSG enable"), | |
| 436 | − XE_RTP_RULES(MEDIA_VERSION(1300), ENGINE_CLASS(OTHER)), | |
| 437 | − XE_RTP_ACTIONS(CLR(RING_PSMI_CTL(0), | |
| 438 | − IDLE_MSG_DISABLE, | |
| 439 | − XE_RTP_ACTION_FLAG(ENGINE_BASE)), | |
| 440 | − FIELD_SET(RING_PWRCTX_MAXCNT(0), | |
| 441 | − IDLE_WAIT_TIME, | |
| 442 | − 0xA, | |
| 443 | − XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 444 | − }, | |
| 445 | − /* Enable Priority Mem Read */ | |
| 446 | − { XE_RTP_NAME("Priority_Mem_Read"), | |
| 447 | − XE_RTP_RULES(GRAPHICS_VERSION_RANGE(2001, XE_RTP_END_VERSION_UNDEFINED)), | |
| 448 | − XE_RTP_ACTIONS(SET(CSFE_CHICKEN1(0), CS_PRIORITY_MEM_READ, | |
| 449 | − XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 450 | − }, | |
| 451 | − { XE_RTP_NAME("Enable CCS Engine(s)"), | |
| 452 | − XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1255, XE_RTP_END_VERSION_UNDEFINED), | |
| 453 | − FUNC(xe_rtp_match_first_render_or_compute)), | |
| 454 | − XE_RTP_ACTIONS(SET(RCU_MODE, RCU_MODE_CCS_ENABLE)) | |
| 455 | − }, | |
| 456 | − /* Use Fixed slice CCS mode */ | |
| 457 | − { XE_RTP_NAME("RCU_MODE_FIXED_SLICE_CCS_MODE"), | |
| 458 | − XE_RTP_RULES(FUNC(xe_hw_engine_match_fixed_cslice_mode)), | |
| 459 | − XE_RTP_ACTIONS(FIELD_SET(RCU_MODE, RCU_MODE_FIXED_SLICE_CCS_MODE, | |
| 460 | − RCU_MODE_FIXED_SLICE_CCS_MODE)) | |
| 461 | − }, | |
| 462 | − { XE_RTP_NAME("Enable MSI-X interrupt support"), | |
| 463 | − XE_RTP_RULES(FUNC(xe_rtp_match_has_msix)), | |
| 464 | − XE_RTP_ACTIONS(SET(GFX_MODE(0), GFX_MSIX_INTERRUPT_ENABLE, | |
| 465 | − XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 466 | − }, | |
| 467 | − ); | |
| 468 | 476 | |
| 469 | 477 | xe_rtp_process_to_sr(&ctx, &engine_sr, &hwe->reg_sr, false); |
| 470 | 478 | } |
| @@ -437,6 +437,16 @@ static void hw_error_source_handler(struct xe_tile *tile, const enum hardware_er · 436 unmodified lines | ||
| 437 | 437 | if (!IS_DGFX(xe)) |
| 438 | 438 | return; |
| 439 | 439 | |
| 440 | + /* | |
| 441 | + * Hardware errors are reported through System Controller on the platforms that | |
| 442 | + * support it, and never routed as direct IRQ to SGUnit. So we should never be | |
| 443 | + * here for those platforms. | |
| 444 | + */ | |
| 445 | + if (xe->info.has_sysctrl) { | |
| 446 | + drm_err_ratelimited(&xe->drm, HW_ERR "Invalid error routing\n"); | |
| 447 | + return; | |
| 448 | + } | |
| 449 | + | |
| 440 | 450 | spin_lock_irqsave(&xe->irq.lock, flags); |
| 441 | 451 | err_src = xe_mmio_read32(&tile->mmio, DEV_ERR_STAT_REG(hw_err)); |
| 442 | 452 | if (!err_src) { |
| @@ -516,14 +526,6 @@ void xe_hw_error_irq_handler(struct xe_tile *tile, const u32 master_ctl) · 73 unmodified lines | ||
| 516 | 526 | } |
| 517 | 527 | } |
| 518 | 528 | |
| 519 | −static int hw_error_info_init(struct xe_device *xe) | |
| 520 | −{ | |
| 521 | − if (xe->info.platform != XE_PVC) | |
| 522 | − return 0; | |
| 523 | − | |
| 524 | − return xe_drm_ras_init(xe); | |
| 525 | −} | |
| 526 | − | |
| 527 | 529 | /* |
| 528 | 530 | * Process hardware errors during boot |
| 529 | 531 | */ |
| @@ -550,16 +552,11 @@ static void process_hw_errors(struct xe_device *xe) · 20 unmodified lines | ||
| 550 | 552 | void xe_hw_error_init(struct xe_device *xe) |
| 551 | 553 | { |
| 552 | 554 | struct xe_tile *tile = xe_device_get_root_tile(xe); |
| 553 | − int ret; | |
| 554 | 555 | |
| 555 | 556 | if (!IS_DGFX(xe) || IS_SRIOV_VF(xe)) |
| 556 | 557 | return; |
| 557 | 558 | |
| 558 | 559 | INIT_WORK(&tile->csc_hw_error_work, csc_hw_error_work); |
| 559 | 560 | |
| 560 | − ret = hw_error_info_init(xe); | |
| 561 | − if (ret) | |
| 562 | − drm_err(&xe->drm, "Failed to initialize XE DRM RAS (%pe)\n", ERR_PTR(ret)); | |
| 563 | − | |
| 564 | 561 | process_hw_errors(xe); |
| 565 | 562 | } |
| @@ -334,9 +334,6 @@ int xe_i2c_probe(struct xe_device *xe) · 333 unmodified lines | ||
| 334 | 334 | if (!xe->info.has_i2c) |
| 335 | 335 | return 0; |
| 336 | 336 | |
| 337 | − if (IS_SRIOV_VF(xe)) | |
| 338 | − return 0; | |
| 339 | − | |
| 340 | 337 | xe_i2c_read_endpoint(xe_root_tile_mmio(xe), &ep); |
| 341 | 338 | if (ep.cookie != XE_I2C_EP_COOKIE_DEVICE) |
| 342 | 339 | return 0; |
| @@ -2618,13 +2618,19 @@ void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot) · 2617 unmodified lines | ||
| 2618 | 2618 | kfree(snapshot); |
| 2619 | 2619 | } |
| 2620 | 2620 | |
| 2621 | +static bool engine_valid_for_utilization(struct xe_gt *gt, struct xe_hw_engine *hwe) | |
| 2622 | +{ | |
| 2623 | + /* The USM-reserved copy engine runs kernel migrate contexts queried here */ | |
| 2624 | + return hwe && (!xe_hw_engine_is_reserved(hwe) || xe_gt_is_usm_hwe(gt, hwe)); | |
| 2625 | +} | |
| 2626 | + | |
| 2621 | 2627 | static struct xe_hw_engine *engine_id_to_hwe(struct xe_gt *gt, u32 engine_id) |
| 2622 | 2628 | { |
| 2623 | 2629 | u16 class = REG_FIELD_GET(ENGINE_CLASS_ID, engine_id); |
| 2624 | 2630 | u16 instance = REG_FIELD_GET(ENGINE_INSTANCE_ID, engine_id); |
| 2625 | 2631 | struct xe_hw_engine *hwe = xe_gt_hw_engine(gt, class, instance, false); |
| 2626 | 2632 | |
| 2627 | − if (xe_gt_WARN_ONCE(gt, !hwe || xe_hw_engine_is_reserved(hwe), | |
| 2633 | + if (xe_gt_WARN_ONCE(gt, !engine_valid_for_utilization(gt, hwe), | |
| 2628 | 2634 | "Unexpected engine class:instance %d:%d for utilization\n", |
| 2629 | 2635 | class, instance)) |
| 2630 | 2636 | return NULL; |
| @@ -11,27 +11,18 @@ · 10 unmodified lines | ||
| 11 | 11 | #include <linux/pci.h> |
| 12 | 12 | |
| 13 | 13 | #include <drm/drm_managed.h> |
| 14 | −#include <drm/drm_print.h> | |
| 15 | 14 | |
| 16 | 15 | #include "regs/xe_bars.h" |
| 17 | 16 | #include "xe_device.h" |
| 18 | 17 | #include "xe_gt_sriov_vf.h" |
| 18 | +#include "xe_printk.h" | |
| 19 | 19 | #include "xe_sriov.h" |
| 20 | +#include "xe_tile_printk.h" | |
| 20 | 21 | #include "xe_trace.h" |
| 21 | 22 | #include "xe_wa.h" |
| 22 | 23 | |
| 23 | 24 | #include "generated/xe_device_wa_oob.h" |
| 24 | 25 | |
| 25 | −static void tiles_fini(void *arg) | |
| 26 | −{ | |
| 27 | − struct xe_device *xe = arg; | |
| 28 | − struct xe_tile *tile; | |
| 29 | − int id; | |
| 30 | − | |
| 31 | − for_each_remote_tile(tile, xe, id) | |
| 32 | − tile->mmio.regs = NULL; | |
| 33 | −} | |
| 34 | − | |
| 35 | 26 | /* |
| 36 | 27 | * On multi-tile devices, partition the BAR space for MMIO on each tile, |
| 37 | 28 | * possibly accounting for register override on the number of tiles available. |
| @@ -56,50 +47,71 @@ static void mmio_multi_tile_setup(struct xe_device *xe, size_t tile_mmio_size) · 18 unmodified lines | ||
| 56 | 47 | struct xe_tile *tile; |
| 57 | 48 | u8 id; |
| 58 | 49 | |
| 59 | − /* | |
| 60 | − * Nothing to be done as tile 0 has already been setup earlier with the | |
| 61 | − * entire BAR mapped - see xe_mmio_probe_early() | |
| 62 | − */ | |
| 63 | − if (xe->info.tile_count == 1) | |
| 64 | − return; | |
| 65 | − | |
| 66 | 50 | for_each_remote_tile(tile, xe, id) |
| 67 | 51 | xe_mmio_init(&tile->mmio, tile, xe->mmio.regs + id * tile_mmio_size, SZ_4M); |
| 68 | 52 | } |
| 69 | 53 | |
| 54 | +/** | |
| 55 | + * xe_mmio_probe_tiles() - Initialize all tiles' MMIO | |
| 56 | + * @xe: the &xe_device | |
| 57 | + * | |
| 58 | + * Initialize the remaining tiles' MMIO instances. | |
| 59 | + * | |
| 60 | + * Return: 0 on success or a negative error code on failure. | |
| 61 | + */ | |
| 70 | 62 | int xe_mmio_probe_tiles(struct xe_device *xe) |
| 71 | 63 | { |
| 72 | 64 | size_t tile_mmio_size = SZ_16M; |
| 73 | 65 | |
| 74 | − mmio_multi_tile_setup(xe, tile_mmio_size); | |
| 66 | + /* | |
| 67 | + * Nothing to be done as tile 0 has already been setup earlier with the | |
| 68 | + * entire BAR mapped - see xe_mmio_probe_early() | |
| 69 | + */ | |
| 70 | + if (xe->info.tile_count == 1) | |
| 71 | + return 0; | |
| 75 | 72 | |
| 76 | − return devm_add_action_or_reset(xe->drm.dev, tiles_fini, xe); | |
| 73 | + if (xe->mmio.size < xe->info.tile_count * tile_mmio_size) { | |
| 74 | + xe_err(xe, "GTTMMADR_BAR is too small for %d tiles: %zu\n", | |
| 75 | + xe->info.tile_count, xe->mmio.size); | |
| 76 | + return -EIO; | |
| 77 | + } | |
| 78 | + | |
| 79 | + mmio_multi_tile_setup(xe, tile_mmio_size); | |
| 80 | + return 0; | |
| 77 | 81 | } |
| 78 | 82 | |
| 79 | 83 | static void mmio_fini(void *arg) |
| 80 | 84 | { |
| 81 | 85 | struct xe_device *xe = arg; |
| 82 | − struct xe_tile *root_tile = xe_device_get_root_tile(xe); | |
| 83 | 86 | |
| 84 | − pci_iounmap(to_pci_dev(xe->drm.dev), xe->mmio.regs); | |
| 85 | 87 | xe->mmio.regs = NULL; |
| 86 | − root_tile->mmio.regs = NULL; | |
| 87 | 88 | } |
| 88 | 89 | |
| 90 | +/** | |
| 91 | + * xe_mmio_probe_early() - Probe and initialize device's MMIO | |
| 92 | + * @xe: the &xe_device | |
| 93 | + * | |
| 94 | + * Map the entire GTTMMADR_BAR and initialize the first tile's MMIO instance. | |
| 95 | + * | |
| 96 | + * The first 16MB of the GTTMMADR_BAR always belongs to the root tile, and | |
| 97 | + * includes: registers (0-4MB), reserved space (4MB-8MB) and GGTT (8MB-16MB). | |
| 98 | + * | |
| 99 | + * Return: 0 on success or a negative error code on failure. | |
| 100 | + */ | |
| 89 | 101 | int xe_mmio_probe_early(struct xe_device *xe) |
| 90 | 102 | { |
| 91 | 103 | struct xe_tile *root_tile = xe_device_get_root_tile(xe); |
| 92 | 104 | struct pci_dev *pdev = to_pci_dev(xe->drm.dev); |
| 93 | 105 | |
| 94 | − /* | |
| 95 | − * Map the entire BAR. | |
| 96 | − * The first 16MB of the BAR, belong to the root tile, and include: | |
| 97 | − * registers (0-4MB), reserved space (4MB-8MB) and GGTT (8MB-16MB). | |
| 98 | − */ | |
| 99 | − xe->mmio.size = pci_resource_len(pdev, GTTMMADR_BAR); | |
| 100 | − xe->mmio.regs = pci_iomap(pdev, GTTMMADR_BAR, 0); | |
| 106 | + xe->mmio.regs = pcim_iomap(pdev, GTTMMADR_BAR, 0); | |
| 101 | 107 | if (!xe->mmio.regs) { |
| 102 | − drm_err(&xe->drm, "failed to map registers\n"); | |
| 108 | + xe_err(xe, "Failed to map GTTMMADR_BAR\n"); | |
| 109 | + return -EIO; | |
| 110 | + } | |
| 111 | + | |
| 112 | + xe->mmio.size = pci_resource_len(pdev, GTTMMADR_BAR); | |
| 113 | + if (xe->mmio.size < SZ_16M) { | |
| 114 | + xe_err(xe, "GTTMMADR_BAR is too small: %zu\n", xe->mmio.size); | |
| 103 | 115 | return -EIO; |
| 104 | 116 | } |
| 105 | 117 | |
| @@ -128,6 +140,11 @@ void xe_mmio_init(struct xe_mmio *mmio, struct xe_tile *tile, void __iomem *ptr, · 22 unmodified lines | ||
| 128 | 140 | mmio->tile = tile; |
| 129 | 141 | } |
| 130 | 142 | |
| 143 | +static bool mmio_available(struct xe_mmio *mmio) | |
| 144 | +{ | |
| 145 | + return !xe_tile_WARN_ON_ONCE(mmio->tile, !mmio->tile->xe->mmio.regs); | |
| 146 | +} | |
| 147 | + | |
| 131 | 148 | static void mmio_flush_pending_writes(struct xe_mmio *mmio) |
| 132 | 149 | { |
| 133 | 150 | #define DUMMY_REG_OFFSET 0x130030 |
| @@ -146,6 +163,9 @@ u8 xe_mmio_read8(struct xe_mmio *mmio, struct xe_reg reg) · 12 unmodified lines | ||
| 146 | 163 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 147 | 164 | u8 val; |
| 148 | 165 | |
| 166 | + if (!mmio_available(mmio)) | |
| 167 | + return 0; | |
| 168 | + | |
| 149 | 169 | mmio_flush_pending_writes(mmio); |
| 150 | 170 | |
| 151 | 171 | val = readb(mmio->regs + addr); |
| @@ -158,6 +178,9 @@ void xe_mmio_write8(struct xe_mmio *mmio, struct xe_reg reg, u8 val) · 6 unmodified lines | ||
| 158 | 178 | { |
| 159 | 179 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 160 | 180 | |
| 181 | + if (!mmio_available(mmio)) | |
| 182 | + return; | |
| 183 | + | |
| 161 | 184 | trace_xe_reg_rw(mmio, true, addr, val, sizeof(val)); |
| 162 | 185 | |
| 163 | 186 | writeb(val, mmio->regs + addr); |
| @@ -168,6 +191,9 @@ u16 xe_mmio_read16(struct xe_mmio *mmio, struct xe_reg reg) · 4 unmodified lines | ||
| 168 | 191 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 169 | 192 | u16 val; |
| 170 | 193 | |
| 194 | + if (!mmio_available(mmio)) | |
| 195 | + return 0; | |
| 196 | + | |
| 171 | 197 | mmio_flush_pending_writes(mmio); |
| 172 | 198 | |
| 173 | 199 | val = readw(mmio->regs + addr); |
| @@ -180,6 +206,9 @@ void xe_mmio_write32(struct xe_mmio *mmio, struct xe_reg reg, u32 val) · 6 unmodified lines | ||
| 180 | 206 | { |
| 181 | 207 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 182 | 208 | |
| 209 | + if (!mmio_available(mmio)) | |
| 210 | + return; | |
| 211 | + | |
| 183 | 212 | trace_xe_reg_rw(mmio, true, addr, val, sizeof(val)); |
| 184 | 213 | |
| 185 | 214 | if (!reg.vf && IS_SRIOV_VF(mmio->tile->xe)) |
| @@ -194,6 +223,9 @@ u32 xe_mmio_read32(struct xe_mmio *mmio, struct xe_reg reg) · 8 unmodified lines | ||
| 194 | 223 | u32 addr = xe_mmio_adjusted_addr(mmio, reg.addr); |
| 195 | 224 | u32 val; |
| 196 | 225 | |
| 226 | + if (!mmio_available(mmio)) | |
| 227 | + return 0; | |
| 228 | + | |
| 197 | 229 | mmio_flush_pending_writes(mmio); |
| 198 | 230 | |
| 199 | 231 | if (!reg.vf && IS_SRIOV_VF(mmio->tile->xe)) |
| @@ -282,8 +314,8 @@ u64 xe_mmio_read64_2x32(struct xe_mmio *mmio, struct xe_reg reg) · 82 unmodified lines | ||
| 282 | 314 | oldudw = udw; |
| 283 | 315 | } |
| 284 | 316 | |
| 285 | − drm_WARN(&mmio->tile->xe->drm, retries == 0, | |
| 286 | − "64-bit read of %#x did not stabilize\n", reg.addr); | |
| 317 | + xe_tile_WARN(mmio->tile, retries == 0, | |
| 318 | + "MMIO: 64-bit read of %#x did not stabilize\n", reg.addr); | |
| 287 | 319 | |
| 288 | 320 | return (u64)udw << 32 | ldw; |
| 289 | 321 | } |
| @@ -36,9 +36,6 @@ module_param_named(svm_notifier_size, xe_modparam.svm_notifier_size, uint, 0600) · 35 unmodified lines | ||
| 36 | 36 | MODULE_PARM_DESC(svm_notifier_size, "Set the svm notifier size in MiB, must be power of 2 " |
| 37 | 37 | "[default=" __stringify(XE_DEFAULT_SVM_NOTIFIER_SIZE) "]"); |
| 38 | 38 | |
| 39 | −module_param_named_unsafe(force_execlist, xe_modparam.force_execlist, bool, 0444); | |
| 40 | −MODULE_PARM_DESC(force_execlist, "Force Execlist submission"); | |
| 41 | − | |
| 42 | 39 | #if IS_ENABLED(CONFIG_DRM_XE_DISPLAY) |
| 43 | 40 | module_param_named(probe_display, xe_modparam.probe_display, bool, 0444); |
| 44 | 41 | MODULE_PARM_DESC(probe_display, "Probe display HW, otherwise it's left untouched " |
| @@ -10,7 +10,6 @@ · 9 unmodified lines | ||
| 10 | 10 | |
| 11 | 11 | /* Module modprobe variables */ |
| 12 | 12 | struct xe_modparam { |
| 13 | − bool force_execlist; | |
| 14 | 13 | bool probe_display; |
| 15 | 14 | int force_vram_bar_size; |
| 16 | 15 | int guc_log_level; |
| @@ -1702,11 +1702,12 @@ static int xe_oa_release(struct inode *inode, struct file *file) · 1701 unmodified lines | ||
| 1702 | 1702 | static int xe_oa_mmap(struct file *file, struct vm_area_struct *vma) |
| 1703 | 1703 | { |
| 1704 | 1704 | struct xe_oa_stream *stream = file->private_data; |
| 1705 | + int ret = xe_observation_paranoid_check(); | |
| 1705 | 1706 | struct xe_bo *bo = stream->oa_buffer.bo; |
| 1706 | 1707 | |
| 1707 | − if (xe_observation_paranoid && !perfmon_capable()) { | |
| 1708 | + if (ret) { | |
| 1708 | 1709 | drm_dbg(&stream->oa->xe->drm, "Insufficient privilege to map OA buffer\n"); |
| 1709 | − return -EACCES; | |
| 1710 | + return ret; | |
| 1710 | 1711 | } |
| 1711 | 1712 | |
| 1712 | 1713 | /* Can mmap the entire OA buffer or nothing (no partial OA buffer mmaps) */ |
| @@ -2080,10 +2081,12 @@ int xe_oa_stream_open_ioctl(struct drm_device *dev, u64 data, struct drm_file *f · 367 unmodified lines | ||
| 2080 | 2081 | privileged_op = true; |
| 2081 | 2082 | } |
| 2082 | 2083 | |
| 2083 | − if (privileged_op && xe_observation_paranoid && !perfmon_capable()) { | |
| 2084 | − drm_dbg(&oa->xe->drm, "Insufficient privileges to open xe OA stream\n"); | |
| 2085 | − ret = -EACCES; | |
| 2086 | − goto err_exec_q; | |
| 2084 | + if (privileged_op) { | |
| 2085 | + ret = xe_observation_paranoid_check(); | |
| 2086 | + if (ret) { | |
| 2087 | + drm_dbg(&oa->xe->drm, "Insufficient privileges to open xe OA stream\n"); | |
| 2088 | + goto err_exec_q; | |
| 2089 | + } | |
| 2087 | 2090 | } |
| 2088 | 2091 | |
| 2089 | 2092 | if (!param.exec_q && !param.sample) { |
| @@ -2365,9 +2368,10 @@ int xe_oa_add_config_ioctl(struct drm_device *dev, u64 data, struct drm_file *fi · 275 unmodified lines | ||
| 2365 | 2368 | return -ENODEV; |
| 2366 | 2369 | } |
| 2367 | 2370 | |
| 2368 | − if (xe_observation_paranoid && !perfmon_capable()) { | |
| 2371 | + err = xe_observation_paranoid_check(); | |
| 2372 | + if (err) { | |
| 2369 | 2373 | drm_dbg(&oa->xe->drm, "Insufficient privileges to add xe OA config\n"); |
| 2370 | − return -EACCES; | |
| 2374 | + return err; | |
| 2371 | 2375 | } |
| 2372 | 2376 | |
| 2373 | 2377 | err = copy_from_user(¶m, u64_to_user_ptr(data), sizeof(param)); |
| @@ -2467,9 +2471,10 @@ int xe_oa_remove_config_ioctl(struct drm_device *dev, u64 data, struct drm_file · 93 unmodified lines | ||
| 2467 | 2471 | return -ENODEV; |
| 2468 | 2472 | } |
| 2469 | 2473 | |
| 2470 | − if (xe_observation_paranoid && !perfmon_capable()) { | |
| 2474 | + ret = xe_observation_paranoid_check(); | |
| 2475 | + if (ret) { | |
| 2471 | 2476 | drm_dbg(&oa->xe->drm, "Insufficient privileges to remove xe OA config\n"); |
| 2472 | − return -EACCES; | |
| 2477 | + return ret; | |
| 2473 | 2478 | } |
| 2474 | 2479 | |
| 2475 | 2480 | ret = get_user(arg, ptr); |
| @@ -4,6 +4,7 @@ · 3 unmodified lines | ||
| 4 | 4 | */ |
| 5 | 5 | |
| 6 | 6 | #include <linux/errno.h> |
| 7 | +#include <linux/perf_event.h> | |
| 7 | 8 | #include <linux/sysctl.h> |
| 8 | 9 | |
| 9 | 10 | #include <uapi/drm/xe_drm.h> |
| @@ -12,9 +13,28 @@ · 2 unmodified lines | ||
| 12 | 13 | #include "xe_oa.h" |
| 13 | 14 | #include "xe_observation.h" |
| 14 | 15 | |
| 15 | −u32 xe_observation_paranoid = true; | |
| 16 | +static u32 xe_observation_paranoid = true; | |
| 16 | 17 | static struct ctl_table_header *sysctl_header; |
| 17 | 18 | |
| 19 | +/** | |
| 20 | + * xe_observation_paranoid_check - Gate access to xe observation streams. | |
| 21 | + * | |
| 22 | + * When the xe-specific observation_paranoid sysctl is enabled (the | |
| 23 | + * default), defer to perf_allow_cpu() so that access is governed by the | |
| 24 | + * same policy as system-wide perf CPU events: kernel.perf_event_paranoid | |
| 25 | + * plus the security_perf_event_open() LSM hook. When the sysctl has been | |
| 26 | + * cleared by a privileged user, observation is open to all callers. | |
| 27 | + * | |
| 28 | + * Return: 0 if access is permitted, a negative errno otherwise. | |
| 29 | + */ | |
| 30 | +int xe_observation_paranoid_check(void) | |
| 31 | +{ | |
| 32 | + if (!xe_observation_paranoid) | |
| 33 | + return 0; | |
| 34 | + | |
| 35 | + return perf_allow_cpu(); | |
| 36 | +} | |
| 37 | + | |
| 18 | 38 | static int xe_oa_ioctl(struct drm_device *dev, struct drm_xe_observation_param *arg, |
| 19 | 39 | struct drm_file *file) |
| 20 | 40 | { |
| @@ -83,11 +103,13 @@ static const struct ctl_table observation_ctl_table[] = { · 62 unmodified lines | ||
| 83 | 103 | }; |
| 84 | 104 | |
| 85 | 105 | /** |
| 86 | − * xe_observation_sysctl_register - Register xe_observation_paranoid sysctl | |
| 106 | + * xe_observation_sysctl_register - Register the observation_paranoid sysctl | |
| 87 | 107 | * |
| 88 | − * Normally only superuser/root can access observation stream | |
| 89 | − * data. However, superuser can set xe_observation_paranoid sysctl to 0 to | |
| 90 | − * allow non-privileged users to also access observation data. | |
| 108 | + * When dev.xe.observation_paranoid is set (the default), access to | |
| 109 | + * observation streams follows the system-wide perf_allow_cpu() policy: | |
| 110 | + * kernel.perf_event_paranoid plus the security_perf_event_open() LSM | |
| 111 | + * hook. A privileged user can clear the sysctl to bypass that gate and | |
| 112 | + * allow unprivileged access to observation data. | |
| 91 | 113 | * |
| 92 | 114 | * Return: always returns 0 |
| 93 | 115 | */ |
| @@ -11,8 +11,7 @@ · 10 unmodified lines | ||
| 11 | 11 | struct drm_device; |
| 12 | 12 | struct drm_file; |
| 13 | 13 | |
| 14 | −extern u32 xe_observation_paranoid; | |
| 15 | − | |
| 14 | +int xe_observation_paranoid_check(void); | |
| 16 | 15 | int xe_observation_ioctl(struct drm_device *dev, void *data, struct drm_file *file); |
| 17 | 16 | int xe_observation_sysctl_register(void); |
| 18 | 17 | void xe_observation_sysctl_unregister(void); |
| @@ -26,6 +26,7 @@ · 25 unmodified lines | ||
| 26 | 26 | #include "xe_guc.h" |
| 27 | 27 | #include "xe_mmio.h" |
| 28 | 28 | #include "xe_module.h" |
| 29 | +#include "xe_pci_error.h" | |
| 29 | 30 | #include "xe_pci_rebar.h" |
| 30 | 31 | #include "xe_pci_sriov.h" |
| 31 | 32 | #include "xe_pci_types.h" |
| @@ -355,6 +356,7 @@ static const __maybe_unused struct xe_device_desc pvc_desc = { · 323 unmodified lines | ||
| 355 | 356 | PLATFORM(PVC), |
| 356 | 357 | .dma_mask_size = 52, |
| 357 | 358 | .has_display = false, |
| 359 | + .has_drm_ras = true, | |
| 358 | 360 | .has_gsc_nvm = 1, |
| 359 | 361 | .has_heci_gscfi = 1, |
| 360 | 362 | .max_gt_per_tile = 1, |
| @@ -457,6 +459,7 @@ static const struct xe_device_desc cri_desc = { · 96 unmodified lines | ||
| 457 | 459 | PLATFORM(CRESCENTISLAND), |
| 458 | 460 | .dma_mask_size = 52, |
| 459 | 461 | .has_display = false, |
| 462 | + .has_drm_ras = true, | |
| 460 | 463 | .has_flat_ccs = false, |
| 461 | 464 | .has_gsc_nvm = 1, |
| 462 | 465 | .has_i2c = true, |
| @@ -599,8 +602,6 @@ static int read_gmdid(struct xe_device *xe, enum xe_gmdid_type type, u32 *ver, u · 136 unmodified lines | ||
| 599 | 602 | struct xe_reg gmdid_reg = GMD_ID; |
| 600 | 603 | u32 val; |
| 601 | 604 | |
| 602 | − KUNIT_STATIC_STUB_REDIRECT(read_gmdid, xe, type, ver, revid); | |
| 603 | − | |
| 604 | 605 | if (IS_SRIOV_VF(xe)) { |
| 605 | 606 | /* |
| 606 | 607 | * To get the value of the GMDID register, VFs must obtain it |
| @@ -726,14 +727,30 @@ static int handle_gmdid(struct xe_device *xe, · 119 unmodified lines | ||
| 726 | 727 | return 0; |
| 727 | 728 | } |
| 728 | 729 | |
| 729 | −static void init_devid(struct xe_device *xe) | |
| 730 | +struct xe_probed_info { | |
| 731 | + u16 devid; | |
| 732 | + u8 revid; | |
| 733 | + u8 tile_count; | |
| 734 | + struct xe_step_info step; | |
| 735 | + const struct xe_ip *graphics_ip; | |
| 736 | + const struct xe_ip *media_ip; | |
| 737 | +}; | |
| 738 | + | |
| 739 | +/* | |
| 740 | + * Probe from the hardware the info required by xe_info_init_early(). | |
| 741 | + */ | |
| 742 | +static int xe_probe_info_early(struct xe_device *xe, | |
| 743 | + const struct xe_device_desc *desc, | |
| 744 | + struct xe_probed_info *probed_info) | |
| 730 | 745 | { |
| 731 | 746 | struct pci_dev *pdev = to_pci_dev(xe->drm.dev); |
| 732 | 747 | |
| 733 | − KUNIT_STATIC_STUB_REDIRECT(init_devid, xe); | |
| 748 | + probed_info->devid = pdev->device; | |
| 749 | + probed_info->revid = pdev->revision; | |
| 734 | 750 | |
| 735 | − xe->info.devid = pdev->device; | |
| 736 | − xe->info.revid = pdev->revision; | |
| 751 | + xe_step_platform_get(desc->platform, probed_info->revid, &probed_info->step); | |
| 752 | + | |
| 753 | + return 0; | |
| 737 | 754 | } |
| 738 | 755 | |
| 739 | 756 | /* |
| @@ -742,17 +759,20 @@ static void init_devid(struct xe_device *xe) · 2 unmodified lines | ||
| 742 | 759 | */ |
| 743 | 760 | static int xe_info_init_early(struct xe_device *xe, |
| 744 | 761 | const struct xe_device_desc *desc, |
| 745 | − const struct xe_subplatform_desc *subplatform_desc) | |
| 762 | + const struct xe_subplatform_desc *subplatform_desc, | |
| 763 | + struct xe_probed_info *probed_info) | |
| 746 | 764 | { |
| 747 | 765 | int err; |
| 748 | 766 | |
| 767 | + xe->info.devid = probed_info->devid; | |
| 768 | + xe->info.revid = probed_info->revid; | |
| 769 | + xe->info.step.platform = probed_info->step.platform; | |
| 770 | + | |
| 749 | 771 | xe->info.platform_name = desc->platform_name; |
| 750 | 772 | xe->info.platform = desc->platform; |
| 751 | 773 | xe->info.subplatform = subplatform_desc ? |
| 752 | 774 | subplatform_desc->subplatform : XE_SUBPLATFORM_NONE; |
| 753 | 775 | |
| 754 | − init_devid(xe); | |
| 755 | − | |
| 756 | 776 | xe->info.dma_mask_size = desc->dma_mask_size; |
| 757 | 777 | xe->info.va_bits = desc->va_bits; |
| 758 | 778 | xe->info.vm_max_level = desc->vm_max_level; |
| @@ -760,6 +780,7 @@ static int xe_info_init_early(struct xe_device *xe, · 1 unmodified line | ||
| 760 | 780 | |
| 761 | 781 | xe->info.is_dgfx = desc->is_dgfx; |
| 762 | 782 | xe->info.has_cached_pt = desc->has_cached_pt; |
| 783 | + xe->info.has_drm_ras = desc->has_drm_ras; | |
| 763 | 784 | xe->info.has_fan_control = desc->has_fan_control; |
| 764 | 785 | /* runtime fusing may force flat_ccs to disabled later */ |
| 765 | 786 | xe->info.has_flat_ccs = desc->has_flat_ccs; |
| @@ -789,14 +810,10 @@ static int xe_info_init_early(struct xe_device *xe, · 23 unmodified lines | ||
| 789 | 810 | xe->info.probe_display = IS_ENABLED(CONFIG_DRM_XE_DISPLAY) && |
| 790 | 811 | xe_modparam.probe_display && |
| 791 | 812 | desc->has_display; |
| 792 | − xe->info.force_execlist = xe_modparam.force_execlist; | |
| 793 | 813 | |
| 794 | 814 | xe_assert(xe, desc->max_gt_per_tile > 0); |
| 795 | 815 | xe_assert(xe, desc->max_gt_per_tile <= XE_MAX_GT_PER_TILE); |
| 796 | 816 | xe->info.max_gt_per_tile = desc->max_gt_per_tile; |
| 797 | − xe->info.tile_count = 1 + desc->max_remote_tiles; | |
| 798 | − | |
| 799 | − xe_step_platform_get(xe); | |
| 800 | 817 | |
| 801 | 818 | err = xe_tile_init_early(xe_device_get_root_tile(xe), xe, 0); |
| 802 | 819 | if (err) |
| @@ -805,22 +822,21 @@ static int xe_info_init_early(struct xe_device *xe, · 2 unmodified lines | ||
| 805 | 822 | return 0; |
| 806 | 823 | } |
| 807 | 824 | |
| 808 | −/* | |
| 809 | − * Possibly override number of tile based on configuration register. | |
| 810 | − */ | |
| 811 | −static void xe_info_probe_tile_count(struct xe_device *xe) | |
| 825 | +static void xe_probe_tile_count(struct xe_device *xe, | |
| 826 | + const struct xe_device_desc *desc, | |
| 827 | + struct xe_probed_info *probed_info) | |
| 812 | 828 | { |
| 813 | 829 | struct xe_mmio *mmio; |
| 814 | 830 | u8 tile_count; |
| 815 | 831 | u32 mtcfg; |
| 816 | 832 | |
| 817 | − KUNIT_STATIC_STUB_REDIRECT(xe_info_probe_tile_count, xe); | |
| 833 | + probed_info->tile_count = 1 + desc->max_remote_tiles; | |
| 818 | 834 | |
| 819 | 835 | /* |
| 820 | 836 | * Probe for tile count only for platforms that support multiple |
| 821 | 837 | * tiles. |
| 822 | 838 | */ |
| 823 | − if (xe->info.tile_count == 1) | |
| 839 | + if (probed_info->tile_count == 1) | |
| 824 | 840 | return; |
| 825 | 841 | |
| 826 | 842 | mmio = xe_root_tile_mmio(xe); |
| @@ -833,10 +849,10 @@ static void xe_info_probe_tile_count(struct xe_device *xe) · 6 unmodified lines | ||
| 833 | 849 | mtcfg = xe_mmio_read32(mmio, XEHP_MTCFG_ADDR); |
| 834 | 850 | tile_count = REG_FIELD_GET(TILE_COUNT, mtcfg) + 1; |
| 835 | 851 | |
| 836 | − if (tile_count < xe->info.tile_count) { | |
| 852 | + if (tile_count < probed_info->tile_count) { | |
| 837 | 853 | drm_info(&xe->drm, "tile_count: %d, reduced_tile_count %d\n", |
| 838 | − xe->info.tile_count, tile_count); | |
| 839 | − xe->info.tile_count = tile_count; | |
| 854 | + probed_info->tile_count, tile_count); | |
| 855 | + probed_info->tile_count = tile_count; | |
| 840 | 856 | } |
| 841 | 857 | } |
| 842 | 858 | |
| @@ -909,25 +925,10 @@ static struct xe_gt *alloc_media_gt(struct xe_tile *tile, · 66 unmodified lines | ||
| 909 | 925 | return gt; |
| 910 | 926 | } |
| 911 | 927 | |
| 912 | −/* | |
| 913 | − * Initialize device info content that does require knowledge about | |
| 914 | − * graphics / media IP version. | |
| 915 | − * Make sure that GT / tile structures allocated by the driver match the data | |
| 916 | − * present in device info. | |
| 917 | − */ | |
| 918 | −static int xe_info_init(struct xe_device *xe, | |
| 919 | − const struct xe_device_desc *desc) | |
| 928 | +static int xe_probe_ips(struct xe_device *xe, | |
| 929 | + const struct xe_device_desc *desc, | |
| 930 | + struct xe_probed_info *probed_info) | |
| 920 | 931 | { |
| 921 | − u32 graphics_gmdid_revid = 0, media_gmdid_revid = 0; | |
| 922 | − const struct xe_ip *graphics_ip; | |
| 923 | − const struct xe_ip *media_ip; | |
| 924 | − const struct xe_graphics_desc *graphics_desc; | |
| 925 | − const struct xe_media_desc *media_desc; | |
| 926 | − struct xe_tile *tile; | |
| 927 | − struct xe_gt *gt; | |
| 928 | − int ret; | |
| 929 | − u8 id; | |
| 930 | − | |
| 931 | 932 | /* |
| 932 | 933 | * If this platform supports GMD_ID, we'll detect the proper IP |
| 933 | 934 | * descriptor to use from hardware registers. |
| @@ -936,17 +937,21 @@ static int xe_info_init(struct xe_device *xe, · 2 unmodified lines | ||
| 936 | 937 | * versions are simply derived from that. |
| 937 | 938 | */ |
| 938 | 939 | if (desc->pre_gmdid_graphics_ip) { |
| 939 | − graphics_ip = desc->pre_gmdid_graphics_ip; | |
| 940 | − media_ip = desc->pre_gmdid_media_ip; | |
| 941 | − xe_step_pre_gmdid_get(xe); | |
| 940 | + probed_info->graphics_ip = desc->pre_gmdid_graphics_ip; | |
| 941 | + probed_info->media_ip = desc->pre_gmdid_media_ip; | |
| 942 | + xe_step_pre_gmdid_get(xe, &probed_info->step); | |
| 942 | 943 | } else { |
| 944 | + int err; | |
| 945 | + u32 graphics_revid, media_revid; | |
| 946 | + | |
| 943 | 947 | xe_assert(xe, !desc->pre_gmdid_media_ip); |
| 944 | − ret = handle_gmdid(xe, &graphics_ip, &media_ip, | |
| 945 | − &graphics_gmdid_revid, &media_gmdid_revid); | |
| 946 | − if (ret) | |
| 947 | − return ret; | |
| 948 | 948 | |
| 949 | − xe_step_gmdid_get(xe, graphics_gmdid_revid, media_gmdid_revid); | |
| 949 | + err = handle_gmdid(xe, &probed_info->graphics_ip, &probed_info->media_ip, | |
| 950 | + &graphics_revid, &media_revid); | |
| 951 | + if (err) | |
| 952 | + return err; | |
| 953 | + | |
| 954 | + xe_step_gmdid_get(xe, graphics_revid, media_revid, &probed_info->step); | |
| 950 | 955 | } |
| 951 | 956 | |
| 952 | 957 | /* |
| @@ -954,9 +959,56 @@ static int xe_info_init(struct xe_device *xe, · 1 unmodified line | ||
| 954 | 959 | * error and we should abort driver load. Failing to detect media |
| 955 | 960 | * IP is non-fatal; we'll just proceed without enabling media support. |
| 956 | 961 | */ |
| 957 | − if (!graphics_ip) | |
| 962 | + if (!probed_info->graphics_ip) | |
| 958 | 963 | return -ENODEV; |
| 959 | 964 | |
| 965 | + return 0; | |
| 966 | +} | |
| 967 | + | |
| 968 | +/* | |
| 969 | + * Probe from the hardware the info required by xe_info_init(). | |
| 970 | + */ | |
| 971 | +static int xe_probe_info(struct xe_device *xe, | |
| 972 | + const struct xe_device_desc *desc, | |
| 973 | + struct xe_probed_info *probed_info) | |
| 974 | +{ | |
| 975 | + int err; | |
| 976 | + | |
| 977 | + xe_probe_tile_count(xe, desc, probed_info); | |
| 978 | + | |
| 979 | + err = xe_probe_ips(xe, desc, probed_info); | |
| 980 | + if (err) | |
| 981 | + return err; | |
| 982 | + | |
| 983 | + return 0; | |
| 984 | +} | |
| 985 | + | |
| 986 | +/* | |
| 987 | + * Initialize device info content that does require knowledge about | |
| 988 | + * graphics / media IP version. | |
| 989 | + * Make sure that GT / tile structures allocated by the driver match the data | |
| 990 | + * present in device info. | |
| 991 | + */ | |
| 992 | +static int xe_info_init(struct xe_device *xe, | |
| 993 | + const struct xe_device_desc *desc, | |
| 994 | + struct xe_probed_info *probed_info) | |
| 995 | +{ | |
| 996 | + const struct xe_ip *graphics_ip; | |
| 997 | + const struct xe_ip *media_ip; | |
| 998 | + const struct xe_graphics_desc *graphics_desc; | |
| 999 | + const struct xe_media_desc *media_desc; | |
| 1000 | + struct xe_tile *tile; | |
| 1001 | + struct xe_gt *gt; | |
| 1002 | + u8 id; | |
| 1003 | + | |
| 1004 | + graphics_ip = probed_info->graphics_ip; | |
| 1005 | + media_ip = probed_info->media_ip; | |
| 1006 | + | |
| 1007 | + xe->info.tile_count = probed_info->tile_count; | |
| 1008 | + xe->info.step.basedie = probed_info->step.basedie; | |
| 1009 | + xe->info.step.graphics = probed_info->step.graphics; | |
| 1010 | + xe->info.step.media = probed_info->step.media; | |
| 1011 | + | |
| 960 | 1012 | xe->info.graphics_verx100 = graphics_ip->verx100; |
| 961 | 1013 | xe->info.graphics_name = graphics_ip->name; |
| 962 | 1014 | graphics_desc = graphics_ip->desc; |
| @@ -988,8 +1040,6 @@ static int xe_info_init(struct xe_device *xe, · 25 unmodified lines | ||
| 988 | 1040 | xe->info.has_soc_remapper_telem = 0; |
| 989 | 1041 | } |
| 990 | 1042 | |
| 991 | − xe_info_probe_tile_count(xe); | |
| 992 | − | |
| 993 | 1043 | for_each_remote_tile(tile, xe, id) { |
| 994 | 1044 | int err; |
| 995 | 1045 | |
| @@ -1072,9 +1122,11 @@ static void xe_pci_remove(struct pci_dev *pdev) · 76 unmodified lines | ||
| 1072 | 1122 | */ |
| 1073 | 1123 | static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) |
| 1074 | 1124 | { |
| 1125 | + struct xe_probed_info probed_info = {}; | |
| 1075 | 1126 | const struct xe_device_desc *desc = (const void *)ent->driver_data; |
| 1076 | 1127 | const struct xe_subplatform_desc *subplatform_desc; |
| 1077 | 1128 | struct xe_device *xe; |
| 1129 | + void *group; | |
| 1078 | 1130 | int err; |
| 1079 | 1131 | |
| 1080 | 1132 | subplatform_desc = find_subplatform(desc, pdev->device); |
| @@ -1102,6 +1154,11 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 21 unmodified lines | ||
| 1102 | 1154 | if (xe_display_driver_probe_defer(pdev)) |
| 1103 | 1155 | return -EPROBE_DEFER; |
| 1104 | 1156 | |
| 1157 | + /* Group all devres so xe_pci_error_slot_reset() can release them as a unit. */ | |
| 1158 | + group = devres_open_group(&pdev->dev, NULL, GFP_KERNEL); | |
| 1159 | + if (!group) | |
| 1160 | + return -ENOMEM; | |
| 1161 | + | |
| 1105 | 1162 | err = pcim_enable_device(pdev); |
| 1106 | 1163 | if (err) |
| 1107 | 1164 | return err; |
| @@ -1110,13 +1167,19 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 2 unmodified lines | ||
| 1110 | 1167 | if (IS_ERR(xe)) |
| 1111 | 1168 | return PTR_ERR(xe); |
| 1112 | 1169 | |
| 1170 | + xe->devres_group = group; | |
| 1171 | + | |
| 1113 | 1172 | pci_set_drvdata(pdev, &xe->drm); |
| 1114 | 1173 | |
| 1115 | 1174 | xe_pm_assert_unbounded_bridge(xe); |
| 1116 | 1175 | |
| 1117 | 1176 | pci_set_master(pdev); |
| 1118 | 1177 | |
| 1119 | − err = xe_info_init_early(xe, desc, subplatform_desc); | |
| 1178 | + err = xe_probe_info_early(xe, desc, &probed_info); | |
| 1179 | + if (err) | |
| 1180 | + return err; | |
| 1181 | + | |
| 1182 | + err = xe_info_init_early(xe, desc, subplatform_desc, &probed_info); | |
| 1120 | 1183 | if (err) |
| 1121 | 1184 | return err; |
| 1122 | 1185 | |
| @@ -1135,7 +1198,11 @@ static int xe_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 12 unmodified lines | ||
| 1135 | 1198 | if (err) |
| 1136 | 1199 | return err; |
| 1137 | 1200 | |
| 1138 | − err = xe_info_init(xe, desc); | |
| 1201 | + err = xe_probe_info(xe, desc, &probed_info); | |
| 1202 | + if (err) | |
| 1203 | + return err; | |
| 1204 | + | |
| 1205 | + err = xe_info_init(xe, desc, &probed_info); | |
| 1139 | 1206 | if (err) |
| 1140 | 1207 | return err; |
| 1141 | 1208 | |
| @@ -1348,6 +1415,7 @@ static struct pci_driver xe_pci_driver = { · 206 unmodified lines | ||
| 1348 | 1415 | .remove = xe_pci_remove, |
| 1349 | 1416 | .shutdown = xe_pci_shutdown, |
| 1350 | 1417 | .sriov_configure = xe_pci_sriov_configure, |
| 1418 | + .err_handler = &xe_pci_error_handlers, | |
| 1351 | 1419 | #ifdef CONFIG_PM_SLEEP |
| 1352 | 1420 | .driver.pm = &xe_pm_ops, |
| 1353 | 1421 | #endif |
| @@ -0,0 +1,122 @@ | ||
| 1 | +// SPDX-License-Identifier: MIT | |
| 2 | +/* | |
| 3 | + * Copyright © 2026 Intel Corporation | |
| 4 | + */ | |
| 5 | + | |
| 6 | +#include <linux/pci.h> | |
| 7 | + | |
| 8 | +#include "xe_device.h" | |
| 9 | +#include "xe_gt.h" | |
| 10 | +#include "xe_pci.h" | |
| 11 | +#include "xe_pm.h" | |
| 12 | +#include "xe_printk.h" | |
| 13 | +#include "xe_survivability_mode.h" | |
| 14 | + | |
| 15 | +static void prepare_device_for_reset(struct pci_dev *pdev) | |
| 16 | +{ | |
| 17 | + struct xe_device *xe = pdev_to_xe_device(pdev); | |
| 18 | + struct xe_gt *gt; | |
| 19 | + u8 id; | |
| 20 | + | |
| 21 | + /* | |
| 22 | + * Wedge the device to prevent userspace access but do not send the uevent. | |
| 23 | + * xe_device_wedged_fini() releases runtime pm if wedged flag is set, so acquire a runtime | |
| 24 | + * pm reference to avoid underflow. | |
| 25 | + */ | |
| 26 | + if (!atomic_xchg(&xe->wedged.flag, 1)) | |
| 27 | + xe_pm_runtime_get_noresume(xe); | |
| 28 | + | |
| 29 | + xe_device_set_in_reset(xe); | |
| 30 | + | |
| 31 | + for_each_gt(gt, xe, id) | |
| 32 | + xe_gt_declare_wedged(gt); | |
| 33 | + | |
| 34 | + pci_disable_device(pdev); | |
| 35 | +} | |
| 36 | + | |
| 37 | +static pci_ers_result_t xe_pci_error_detected(struct pci_dev *pdev, pci_channel_state_t state) | |
| 38 | +{ | |
| 39 | + struct xe_device *xe = pdev_to_xe_device(pdev); | |
| 40 | + | |
| 41 | + xe_info(xe, "PCI error: detected state = %d\n", state); | |
| 42 | + | |
| 43 | + if (state == pci_channel_io_perm_failure) | |
| 44 | + return PCI_ERS_RESULT_DISCONNECT; | |
| 45 | + | |
| 46 | + /* If the device is already wedged or in survivability mode, do not attempt recovery */ | |
| 47 | + if (xe_survivability_mode_is_boot_enabled(xe) || xe_device_wedged(xe)) | |
| 48 | + return PCI_ERS_RESULT_DISCONNECT; | |
| 49 | + | |
| 50 | + switch (state) { | |
| 51 | + case pci_channel_io_normal: | |
| 52 | + return PCI_ERS_RESULT_CAN_RECOVER; | |
| 53 | + case pci_channel_io_frozen: | |
| 54 | + prepare_device_for_reset(pdev); | |
| 55 | + return PCI_ERS_RESULT_NEED_RESET; | |
| 56 | + default: | |
| 57 | + xe_info(xe, "PCI error: unknown state %d\n", state); | |
| 58 | + return PCI_ERS_RESULT_DISCONNECT; | |
| 59 | + } | |
| 60 | +} | |
| 61 | + | |
| 62 | +static pci_ers_result_t xe_pci_error_mmio_enabled(struct pci_dev *pdev) | |
| 63 | +{ | |
| 64 | + struct xe_device *xe = pdev_to_xe_device(pdev); | |
| 65 | + | |
| 66 | + xe_info(xe, "PCI error: MMIO enabled\n"); | |
| 67 | + | |
| 68 | + /* TODO: Query system controller for the type of error and take appropriate action */ | |
| 69 | + return PCI_ERS_RESULT_RECOVERED; | |
| 70 | +} | |
| 71 | + | |
| 72 | +static pci_ers_result_t xe_pci_error_slot_reset(struct pci_dev *pdev) | |
| 73 | +{ | |
| 74 | + const struct pci_device_id *ent = pci_match_id(pdev->driver->id_table, pdev); | |
| 75 | + struct xe_device *xe = pdev_to_xe_device(pdev); | |
| 76 | + | |
| 77 | + xe_info(xe, "PCI error: slot reset\n"); | |
| 78 | + | |
| 79 | + pci_restore_state(pdev); | |
| 80 | + | |
| 81 | + if (pci_enable_device(pdev)) { | |
| 82 | + xe_err(xe, "Cannot re-enable PCI device after reset\n"); | |
| 83 | + return PCI_ERS_RESULT_DISCONNECT; | |
| 84 | + } | |
| 85 | + | |
| 86 | + /* | |
| 87 | + * Secondary Bus Reset causes all VRAM state to be lost along with | |
| 88 | + * hardware state. As an initial step, re-probe the device to | |
| 89 | + * re-initialize the driver and hardware. | |
| 90 | + * TODO: optimize by re-initializing only the hardware state and re-creating | |
| 91 | + * kernel BOs. | |
| 92 | + */ | |
| 93 | + xe_device_clear_in_reset(xe); | |
| 94 | + pdev->driver->remove(pdev); | |
| 95 | + devres_release_group(&pdev->dev, xe->devres_group); | |
| 96 | + | |
| 97 | + if (pdev->driver->probe(pdev, ent)) | |
| 98 | + return PCI_ERS_RESULT_DISCONNECT; | |
| 99 | + | |
| 100 | + xe = pdev_to_xe_device(pdev); | |
| 101 | + | |
| 102 | + /* Wedge the device to prevent I/O operations till the resume callback */ | |
| 103 | + atomic_set(&xe->wedged.flag, 1); | |
| 104 | + | |
| 105 | + return PCI_ERS_RESULT_RECOVERED; | |
| 106 | +} | |
| 107 | + | |
| 108 | +static void xe_pci_error_resume(struct pci_dev *pdev) | |
| 109 | +{ | |
| 110 | + struct xe_device *xe = pdev_to_xe_device(pdev); | |
| 111 | + | |
| 112 | + xe_info(xe, "PCI error: resume\n"); | |
| 113 | + | |
| 114 | + atomic_set(&xe->wedged.flag, 0); | |
| 115 | +} | |
| 116 | + | |
| 117 | +const struct pci_error_handlers xe_pci_error_handlers = { | |
| 118 | + .error_detected = xe_pci_error_detected, | |
| 119 | + .mmio_enabled = xe_pci_error_mmio_enabled, | |
| 120 | + .slot_reset = xe_pci_error_slot_reset, | |
| 121 | + .resume = xe_pci_error_resume, | |
| 122 | +}; | |
| @@ -0,0 +1,13 @@ | ||
| 1 | +/* SPDX-License-Identifier: MIT */ | |
| 2 | +/* | |
| 3 | + * Copyright © 2026 Intel Corporation | |
| 4 | + */ | |
| 5 | + | |
| 6 | +#ifndef _XE_PCI_ERROR_H_ | |
| 7 | +#define _XE_PCI_ERROR_H_ | |
| 8 | + | |
| 9 | +struct pci_error_handlers; | |
| 10 | + | |
| 11 | +extern const struct pci_error_handlers xe_pci_error_handlers; | |
| 12 | + | |
| 13 | +#endif | |
| @@ -40,6 +40,7 @@ struct xe_device_desc { · 39 unmodified lines | ||
| 40 | 40 | |
| 41 | 41 | u8 has_cached_pt:1; |
| 42 | 42 | u8 has_display:1; |
| 43 | + u8 has_drm_ras:1; | |
| 43 | 44 | u8 has_fan_control:1; |
| 44 | 45 | u8 has_flat_ccs:1; |
| 45 | 46 | u8 has_gsc_nvm:1; |
| @@ -323,15 +323,17 @@ int xe_pcode_ready(struct xe_device *xe, bool locked) · 322 unmodified lines | ||
| 323 | 323 | } |
| 324 | 324 | |
| 325 | 325 | /** |
| 326 | − * xe_pcode_init: initialize components of PCODE | |
| 326 | + * xe_pcode_init_early() - Initialize components of PCODE | |
| 327 | 327 | * @tile: tile instance |
| 328 | 328 | * |
| 329 | 329 | * This function initializes the xe_pcode component. |
| 330 | 330 | * To be called once only during probe. |
| 331 | + * | |
| 332 | + * Return: 0 on success or a negative error code on failure. | |
| 331 | 333 | */ |
| 332 | −void xe_pcode_init(struct xe_tile *tile) | |
| 334 | +int xe_pcode_init_early(struct xe_tile *tile) | |
| 333 | 335 | { |
| 334 | − drmm_mutex_init(&tile_to_xe(tile)->drm, &tile->pcode.lock); | |
| 336 | + return drmm_mutex_init(&tile_to_xe(tile)->drm, &tile->pcode.lock); | |
| 335 | 337 | } |
| 336 | 338 | |
| 337 | 339 | /** |
| @@ -12,7 +12,7 @@ struct drm_device; · 11 unmodified lines | ||
| 12 | 12 | struct xe_device; |
| 13 | 13 | struct xe_tile; |
| 14 | 14 | |
| 15 | −void xe_pcode_init(struct xe_tile *tile); | |
| 15 | +int xe_pcode_init_early(struct xe_tile *tile); | |
| 16 | 16 | int xe_pcode_probe_early(struct xe_device *xe); |
| 17 | 17 | int xe_pcode_ready(struct xe_device *xe, bool locked); |
| 18 | 18 | int xe_pcode_init_min_freq_table(struct xe_tile *tile, u32 min_gt_freq, |
| @@ -1026,12 +1026,22 @@ xe_vm_populate_pgtable(struct xe_migrate_pt_update *pt_update, struct xe_tile *t · 1025 unmodified lines | ||
| 1026 | 1026 | u64 *ptr = data; |
| 1027 | 1027 | u32 i; |
| 1028 | 1028 | |
| 1029 | + /* | |
| 1030 | + * @qword_ofs is the absolute entry offset within the page table, while | |
| 1031 | + * @ptes is indexed relative to @update->ofs (its first entry). The GPU | |
| 1032 | + * path (write_pgtable) splits a single update into MAX_PTE_PER_SDI-sized | |
| 1033 | + * chunks, calling this with an advancing @qword_ofs but a fresh @data | |
| 1034 | + * pointer per chunk, so translate back into a @ptes index rather than | |
| 1035 | + * assuming the chunk starts at ptes[0]. | |
| 1036 | + */ | |
| 1029 | 1037 | for (i = 0; i < num_qwords; i++) { |
| 1038 | + u32 idx = qword_ofs - update->ofs + i; | |
| 1039 | + | |
| 1030 | 1040 | if (map) |
| 1031 | 1041 | xe_map_wr(tile_to_xe(tile), map, (qword_ofs + i) * |
| 1032 | − sizeof(u64), u64, ptes[i].pte); | |
| 1042 | + sizeof(u64), u64, ptes[idx].pte); | |
| 1033 | 1043 | else |
| 1034 | − ptr[i] = ptes[i].pte; | |
| 1044 | + ptr[i] = ptes[idx].pte; | |
| 1035 | 1045 | } |
| 1036 | 1046 | } |
| 1037 | 1047 | |
| @@ -2070,6 +2080,9 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile, · 1032 unmodified lines | ||
| 2070 | 2080 | * automatically when the context is re-enabled by the rebind worker, |
| 2071 | 2081 | * or in fault mode it was invalidated on PTE zapping. |
| 2072 | 2082 | * |
| 2083 | + * If rebind, we have to invalidate TLB on context based TLB invalidation | |
| 2084 | + * LR vms, as they cannot be relied on context re-enable. | |
| 2085 | + * | |
| 2073 | 2086 | * If !rebind, and scratch enabled VMs, there is a chance the scratch |
| 2074 | 2087 | * PTE is already cached in the TLB so it needs to be invalidated. |
| 2075 | 2088 | * On !LR VMs this is done in the ring ops preceding a batch, but on |
| @@ -2079,6 +2092,9 @@ static int bind_op_prepare(struct xe_vm *vm, struct xe_tile *tile, · 3 unmodified lines | ||
| 2079 | 2092 | if ((!pt_op->rebind && xe_vm_has_scratch(vm) && |
| 2080 | 2093 | xe_vm_in_lr_mode(vm))) |
| 2081 | 2094 | pt_update_ops->needs_invalidation = true; |
| 2095 | + else if (pt_op->rebind && xe_vm_in_preempt_fence_mode(vm) && | |
| 2096 | + vm->xe->info.has_ctx_tlb_inval) | |
| 2097 | + pt_update_ops->needs_invalidation = true; | |
| 2082 | 2098 | else if (pt_op->rebind && !xe_vm_in_lr_mode(vm)) |
| 2083 | 2099 | /* We bump also if batch_invalidate_tlb is true */ |
| 2084 | 2100 | vm->tlb_flush_seqno++; |
| @@ -59,6 +59,7 @@ bool xe_pxp_is_enabled(const struct xe_pxp *pxp) · 58 unmodified lines | ||
| 59 | 59 | static bool pxp_prerequisites_done(const struct xe_pxp *pxp) |
| 60 | 60 | { |
| 61 | 61 | struct xe_gt *gt = pxp->gt; |
| 62 | + bool huc_ok; | |
| 62 | 63 | bool ready; |
| 63 | 64 | |
| 64 | 65 | CLASS(xe_force_wake, fw_ref)(gt_to_fw(gt), XE_FORCEWAKE_ALL); |
| @@ -73,9 +74,14 @@ static bool pxp_prerequisites_done(const struct xe_pxp *pxp) · 8 unmodified lines | ||
| 73 | 74 | */ |
| 74 | 75 | XE_WARN_ON(!xe_force_wake_ref_has_domain(fw_ref.domains, XE_FORCEWAKE_ALL)); |
| 75 | 76 | |
| 76 | − /* PXP requires both HuC authentication via GSC and GSC proxy initialized */ | |
| 77 | − ready = xe_huc_is_authenticated(>->uc.huc, XE_HUC_AUTH_VIA_GSC) && | |
| 78 | − xe_gsc_proxy_init_done(>->uc.gsc); | |
| 77 | + /* | |
| 78 | + * PXP requires GSC proxy to be initialized. On platforms where the HuC | |
| 79 | + * is loaded by the kernel driver (i.e., pre media 35) PXP also requires | |
| 80 | + * the HuC to be authenticated by GSC. | |
| 81 | + */ | |
| 82 | + huc_ok = MEDIA_VER(gt_to_xe(gt)) >= 35 || | |
| 83 | + xe_huc_is_authenticated(>->uc.huc, XE_HUC_AUTH_VIA_GSC); | |
| 84 | + ready = huc_ok && xe_gsc_proxy_init_done(>->uc.gsc); | |
| 79 | 85 | |
| 80 | 86 | return ready; |
| 81 | 87 | } |
| @@ -97,9 +103,13 @@ int xe_pxp_get_readiness_status(struct xe_pxp *pxp) · 15 unmodified lines | ||
| 97 | 103 | if (!xe_pxp_is_enabled(pxp)) |
| 98 | 104 | return -ENODEV; |
| 99 | 105 | |
| 100 | − /* if the GSC or HuC FW are in an error state, PXP will never work */ | |
| 101 | − if (xe_uc_fw_status_to_error(pxp->gt->uc.huc.fw.status) || | |
| 102 | − xe_uc_fw_status_to_error(pxp->gt->uc.gsc.fw.status)) | |
| 106 | + /* If the GSC FW is in an error state, PXP will never work */ | |
| 107 | + if (xe_uc_fw_status_to_error(pxp->gt->uc.gsc.fw.status)) | |
| 108 | + return -EIO; | |
| 109 | + | |
| 110 | + /* Same for HuC FW, but only if the kernel owns HuC-loading (i.e. pre-NVL) */ | |
| 111 | + if (MEDIA_VER(gt_to_xe(pxp->gt)) < 35 && | |
| 112 | + xe_uc_fw_status_to_error(pxp->gt->uc.huc.fw.status)) | |
| 103 | 113 | return -EIO; |
| 104 | 114 | |
| 105 | 115 | guard(xe_pm_runtime)(pxp->xe); |
| @@ -361,6 +371,7 @@ static void pxp_fini(void *arg) · 255 unmodified lines | ||
| 361 | 371 | int xe_pxp_init(struct xe_device *xe) |
| 362 | 372 | { |
| 363 | 373 | struct xe_gt *gt = xe->tiles[0].media_gt; |
| 374 | + bool gsc_ok, huc_ok; | |
| 364 | 375 | struct xe_pxp *pxp; |
| 365 | 376 | int err; |
| 366 | 377 | |
| @@ -375,10 +386,14 @@ int xe_pxp_init(struct xe_device *xe) · 8 unmodified lines | ||
| 375 | 386 | if (!(gt->info.engine_mask & BIT(XE_HW_ENGINE_GSCCS0))) |
| 376 | 387 | return 0; |
| 377 | 388 | |
| 378 | − /* PXP requires both GSC and HuC firmwares to be available */ | |
| 379 | − if (!xe_uc_fw_is_loadable(>->uc.gsc.fw) || | |
| 380 | − !xe_uc_fw_is_loadable(>->uc.huc.fw)) { | |
| 381 | − drm_info(&xe->drm, "skipping PXP init due to missing FW dependencies"); | |
| 389 | + /* PXP requires GSC FW to be available. Pre-NVL it also requires HuC FW */ | |
| 390 | + gsc_ok = xe_uc_fw_is_loadable(>->uc.gsc.fw); | |
| 391 | + huc_ok = MEDIA_VER(xe) >= 35 || xe_uc_fw_is_loadable(>->uc.huc.fw); | |
| 392 | + | |
| 393 | + if (!gsc_ok || !huc_ok) { | |
| 394 | + drm_info(&xe->drm, "Skipping PXP due to unsatisfied FW deps - GSC=%s, HuC=%s\n", | |
| 395 | + str_yes_no(gsc_ok), | |
| 396 | + MEDIA_VER(xe) >= 35 ? "not needed" : str_yes_no(huc_ok)); | |
| 382 | 397 | return 0; |
| 383 | 398 | } |
| 384 | 399 | |
| @@ -119,6 +119,7 @@ query_engine_cycles(struct xe_device *xe, · 118 unmodified lines | ||
| 119 | 119 | struct drm_xe_engine_class_instance *eci; |
| 120 | 120 | struct drm_xe_query_engine_cycles resp; |
| 121 | 121 | size_t size = sizeof(resp); |
| 122 | + enum xe_force_wake_domains fw_domain; | |
| 122 | 123 | __ktime_func_t cpu_clock; |
| 123 | 124 | struct xe_hw_engine *hwe; |
| 124 | 125 | struct xe_gt *gt; |
| @@ -154,8 +155,10 @@ query_engine_cycles(struct xe_device *xe, · 29 unmodified lines | ||
| 154 | 155 | if (!hwe) |
| 155 | 156 | return -EINVAL; |
| 156 | 157 | |
| 157 | − xe_with_force_wake(fw_ref, gt_to_fw(gt), XE_FORCEWAKE_ALL) { | |
| 158 | − if (!xe_force_wake_ref_has_domain(fw_ref.domains, XE_FORCEWAKE_ALL)) | |
| 158 | + fw_domain = xe_hw_engine_to_fw_domain(hwe); | |
| 159 | + | |
| 160 | + xe_with_force_wake(fw_ref, gt_to_fw(gt), fw_domain) { | |
| 161 | + if (!xe_force_wake_ref_has_domain(fw_ref.domains, fw_domain)) | |
| 159 | 162 | return -EIO; |
| 160 | 163 | |
| 161 | 164 | hwe_read_timestamp(hwe, &resp.engine_cycles, &resp.cpu_timestamp, |
| @@ -4,11 +4,15 @@ · 3 unmodified lines | ||
| 4 | 4 | */ |
| 5 | 5 | |
| 6 | 6 | #include "xe_device.h" |
| 7 | +#include "xe_drm_ras.h" | |
| 8 | +#include "xe_pm.h" | |
| 7 | 9 | #include "xe_printk.h" |
| 8 | 10 | #include "xe_ras.h" |
| 9 | 11 | #include "xe_ras_types.h" |
| 10 | 12 | #include "xe_sysctrl.h" |
| 11 | 13 | #include "xe_sysctrl_event_types.h" |
| 14 | +#include "xe_sysctrl_mailbox.h" | |
| 15 | +#include "xe_sysctrl_mailbox_types.h" | |
| 12 | 16 | |
| 13 | 17 | /* Severity of detected errors */ |
| 14 | 18 | enum xe_ras_severity { |
| @@ -31,6 +35,17 @@ enum xe_ras_component { · 16 unmodified lines | ||
| 31 | 35 | XE_RAS_COMP_MAX |
| 32 | 36 | }; |
| 33 | 37 | |
| 38 | +/* RAS response status codes */ | |
| 39 | +enum xe_ras_response_status { | |
| 40 | + XE_RAS_STATUS_SUCCESS = 0, | |
| 41 | + XE_RAS_STATUS_INVALID_PARAM, | |
| 42 | + XE_RAS_STATUS_OP_NOT_SUPPORTED, | |
| 43 | + XE_RAS_STATUS_TIMEOUT, | |
| 44 | + XE_RAS_STATUS_HARDWARE_FAILURE, | |
| 45 | + XE_RAS_STATUS_INSUFFICIENT_RESOURCES, | |
| 46 | + XE_RAS_STATUS_MAX | |
| 47 | +}; | |
| 48 | + | |
| 34 | 49 | static const char *const xe_ras_severities[] = { |
| 35 | 50 | [XE_RAS_SEV_NOT_SUPPORTED] = "Not Supported", |
| 36 | 51 | [XE_RAS_SEV_CORRECTABLE] = "Correctable Error", |
| @@ -50,6 +65,56 @@ static const char *const xe_ras_components[] = { · 13 unmodified lines | ||
| 50 | 65 | }; |
| 51 | 66 | static_assert(ARRAY_SIZE(xe_ras_components) == XE_RAS_COMP_MAX); |
| 52 | 67 | |
| 68 | +static u8 drm_to_xe_ras_severity(u8 severity) | |
| 69 | +{ | |
| 70 | + switch (severity) { | |
| 71 | + case DRM_XE_RAS_ERR_SEV_CORRECTABLE: | |
| 72 | + return XE_RAS_SEV_CORRECTABLE; | |
| 73 | + case DRM_XE_RAS_ERR_SEV_UNCORRECTABLE: | |
| 74 | + return XE_RAS_SEV_UNCORRECTABLE; | |
| 75 | + default: | |
| 76 | + return XE_RAS_SEV_NOT_SUPPORTED; | |
| 77 | + } | |
| 78 | +} | |
| 79 | + | |
| 80 | +static u8 drm_to_xe_ras_component(u8 component) | |
| 81 | +{ | |
| 82 | + switch (component) { | |
| 83 | + case DRM_XE_RAS_ERR_COMP_CORE_COMPUTE: | |
| 84 | + return XE_RAS_COMP_CORE_COMPUTE; | |
| 85 | + case DRM_XE_RAS_ERR_COMP_SOC_INTERNAL: | |
| 86 | + return XE_RAS_COMP_SOC_INTERNAL; | |
| 87 | + case DRM_XE_RAS_ERR_COMP_DEVICE_MEMORY: | |
| 88 | + return XE_RAS_COMP_DEVICE_MEMORY; | |
| 89 | + case DRM_XE_RAS_ERR_COMP_PCIE: | |
| 90 | + return XE_RAS_COMP_PCIE; | |
| 91 | + case DRM_XE_RAS_ERR_COMP_FABRIC: | |
| 92 | + return XE_RAS_COMP_FABRIC; | |
| 93 | + default: | |
| 94 | + return XE_RAS_COMP_NOT_SUPPORTED; | |
| 95 | + } | |
| 96 | +} | |
| 97 | + | |
| 98 | +static int ras_status_to_errno(u32 status) | |
| 99 | +{ | |
| 100 | + switch (status) { | |
| 101 | + case XE_RAS_STATUS_SUCCESS: | |
| 102 | + return 0; | |
| 103 | + case XE_RAS_STATUS_INVALID_PARAM: | |
| 104 | + return -EINVAL; | |
| 105 | + case XE_RAS_STATUS_OP_NOT_SUPPORTED: | |
| 106 | + return -EOPNOTSUPP; | |
| 107 | + case XE_RAS_STATUS_TIMEOUT: | |
| 108 | + return -ETIMEDOUT; | |
| 109 | + case XE_RAS_STATUS_HARDWARE_FAILURE: | |
| 110 | + return -EIO; | |
| 111 | + case XE_RAS_STATUS_INSUFFICIENT_RESOURCES: | |
| 112 | + return -ENOSPC; | |
| 113 | + default: | |
| 114 | + return -EPROTO; | |
| 115 | + } | |
| 116 | +} | |
| 117 | + | |
| 53 | 118 | static inline const char *sev_to_str(u8 severity) |
| 54 | 119 | { |
| 55 | 120 | if (severity >= XE_RAS_SEV_MAX) |
| @@ -66,6 +131,68 @@ static inline const char *comp_to_str(u8 component) · 10 unmodified lines | ||
| 66 | 131 | return xe_ras_components[component]; |
| 67 | 132 | } |
| 68 | 133 | |
| 134 | +static struct pci_dev *find_usp_dev(struct pci_dev *pdev) | |
| 135 | +{ | |
| 136 | + struct pci_dev *vsp; | |
| 137 | + | |
| 138 | + /* | |
| 139 | + * Device Hierarchy: | |
| 140 | + * | |
| 141 | + * Upstream Switch Port (USP) --> Virtual Switch Port (VSP) --> SGunit (GPU endpoint) | |
| 142 | + */ | |
| 143 | + vsp = pci_upstream_bridge(pdev); | |
| 144 | + if (!vsp) | |
| 145 | + return NULL; | |
| 146 | + | |
| 147 | + return pci_upstream_bridge(vsp); | |
| 148 | +} | |
| 149 | + | |
| 150 | +static void ras_usp_aer_init(struct xe_device *xe) | |
| 151 | +{ | |
| 152 | + struct pci_dev *pdev = to_pci_dev(xe->drm.dev); | |
| 153 | + struct pci_dev *usp; | |
| 154 | + u16 aer_cap; | |
| 155 | + u32 status; | |
| 156 | + | |
| 157 | + usp = find_usp_dev(pdev); | |
| 158 | + if (!usp) | |
| 159 | + return; | |
| 160 | + | |
| 161 | + aer_cap = pci_find_ext_capability(usp, PCI_EXT_CAP_ID_ERR); | |
| 162 | + if (!aer_cap) { | |
| 163 | + dev_warn(&usp->dev, "AER capability unavailable\n"); | |
| 164 | + return; | |
| 165 | + } | |
| 166 | + | |
| 167 | + /* | |
| 168 | + * Clear any stale Uncorrectable Internal Error Status event in Uncorrectable Error | |
| 169 | + * Status Register. | |
| 170 | + */ | |
| 171 | + pci_read_config_dword(usp, aer_cap + PCI_ERR_UNCOR_STATUS, &status); | |
| 172 | + if (status & PCI_ERR_UNC_INTN) | |
| 173 | + pci_write_config_dword(usp, aer_cap + PCI_ERR_UNCOR_STATUS, PCI_ERR_UNC_INTN); | |
| 174 | + | |
| 175 | + /* | |
| 176 | + * All errors are steered to USP which is a PCIe AER Compliant device. | |
| 177 | + * Downgrade all the errors to non-fatal to prevent PCIe bus driver | |
| 178 | + * from triggering a Secondary Bus Reset (SBR). This allows error | |
| 179 | + * detection, containment and recovery in the driver. | |
| 180 | + * | |
| 181 | + * The Uncorrectable Error Severity Register has the 'Uncorrectable | |
| 182 | + * Internal Error Severity' set to fatal by default. Set this to | |
| 183 | + * non-fatal and unmask the error. | |
| 184 | + */ | |
| 185 | + | |
| 186 | + /* Downgrade Uncorrectable Internal Error to non-fatal */ | |
| 187 | + pci_clear_and_set_config_dword(usp, aer_cap + PCI_ERR_UNCOR_SEVER, PCI_ERR_UNC_INTN, 0); | |
| 188 | + | |
| 189 | + /* Unmask Uncorrectable Internal Error */ | |
| 190 | + pci_clear_and_set_config_dword(usp, aer_cap + PCI_ERR_UNCOR_MASK, PCI_ERR_UNC_INTN, 0); | |
| 191 | + | |
| 192 | + pci_save_state(usp); | |
| 193 | + dev_dbg(&usp->dev, "Uncorrectable Internal Errors downgraded and unmasked\n"); | |
| 194 | +} | |
| 195 | + | |
| 69 | 196 | void xe_ras_counter_threshold_crossed(struct xe_device *xe, |
| 70 | 197 | struct xe_sysctrl_event_response *response) |
| 71 | 198 | { |
| @@ -91,3 +218,136 @@ void xe_ras_counter_threshold_crossed(struct xe_device *xe, · 19 unmodified lines | ||
| 91 | 218 | comp_to_str(component), sev_to_str(severity)); |
| 92 | 219 | } |
| 93 | 220 | } |
| 221 | + | |
| 222 | +static int get_counter(struct xe_device *xe, struct xe_ras_error_class *counter, u32 *value) | |
| 223 | +{ | |
| 224 | + struct xe_ras_get_counter_response response = {0}; | |
| 225 | + struct xe_ras_get_counter_request request = {0}; | |
| 226 | + struct xe_sysctrl_mailbox_command command = {0}; | |
| 227 | + struct xe_ras_error_common *common; | |
| 228 | + size_t rlen; | |
| 229 | + int ret; | |
| 230 | + | |
| 231 | + request.counter = *counter; | |
| 232 | + | |
| 233 | + xe_sysctrl_create_command(&command, XE_SYSCTRL_GROUP_GFSP, XE_SYSCTRL_CMD_GET_COUNTER, | |
| 234 | + &request, sizeof(request), &response, sizeof(response)); | |
| 235 | + | |
| 236 | + ret = xe_sysctrl_send_command(&xe->sc, &command, &rlen); | |
| 237 | + if (ret) { | |
| 238 | + xe_err(xe, "sysctrl: failed to get counter %d\n", ret); | |
| 239 | + return ret; | |
| 240 | + } | |
| 241 | + | |
| 242 | + if (rlen != sizeof(response)) { | |
| 243 | + xe_err(xe, "sysctrl: unexpected get counter response length %zu (expected %zu)\n", | |
| 244 | + rlen, sizeof(response)); | |
| 245 | + return -EIO; | |
| 246 | + } | |
| 247 | + | |
| 248 | + common = &response.counter.common; | |
| 249 | + *value = response.value; | |
| 250 | + | |
| 251 | + xe_dbg(xe, "[RAS]: get counter %u for %s %s\n", *value, comp_to_str(common->component), | |
| 252 | + sev_to_str(common->severity)); | |
| 253 | + | |
| 254 | + return 0; | |
| 255 | +} | |
| 256 | + | |
| 257 | +/** | |
| 258 | + * xe_ras_get_counter() - Get error counter value | |
| 259 | + * @xe: Xe device instance | |
| 260 | + * @severity: Error severity to be queried (&enum drm_xe_ras_error_severity) | |
| 261 | + * @component: Error component to be queried (&enum drm_xe_ras_error_component) | |
| 262 | + * @value: Counter value | |
| 263 | + * | |
| 264 | + * This function retrieves the value of a specific error counter based on | |
| 265 | + * the error severity and component. | |
| 266 | + * | |
| 267 | + * Return: 0 on success, negative error code on failure. | |
| 268 | + */ | |
| 269 | +int xe_ras_get_counter(struct xe_device *xe, u8 severity, u8 component, u32 *value) | |
| 270 | +{ | |
| 271 | + struct xe_ras_error_class counter = {0}; | |
| 272 | + | |
| 273 | + counter.common.severity = drm_to_xe_ras_severity(severity); | |
| 274 | + counter.common.component = drm_to_xe_ras_component(component); | |
| 275 | + | |
| 276 | + guard(xe_pm_runtime)(xe); | |
| 277 | + return get_counter(xe, &counter, value); | |
| 278 | +} | |
| 279 | + | |
| 280 | +/** | |
| 281 | + * xe_ras_clear_counter() - Clear error counter value | |
| 282 | + * @xe: Xe device instance | |
| 283 | + * @severity: Error severity to be cleared (&enum drm_xe_ras_error_severity) | |
| 284 | + * @component: Error component to be cleared (&enum drm_xe_ras_error_component) | |
| 285 | + * | |
| 286 | + * This function clears the value of a specific error counter based on | |
| 287 | + * the error severity and component. | |
| 288 | + * | |
| 289 | + * Return: 0 on success, negative error code on failure. | |
| 290 | + */ | |
| 291 | +int xe_ras_clear_counter(struct xe_device *xe, u8 severity, u8 component) | |
| 292 | +{ | |
| 293 | + struct xe_ras_clear_counter_response response = {0}; | |
| 294 | + struct xe_ras_clear_counter_request request = {0}; | |
| 295 | + struct xe_sysctrl_mailbox_command command = {0}; | |
| 296 | + struct xe_ras_error_class *counter; | |
| 297 | + size_t rlen; | |
| 298 | + int ret; | |
| 299 | + | |
| 300 | + counter = &request.counter; | |
| 301 | + counter->common.severity = drm_to_xe_ras_severity(severity); | |
| 302 | + counter->common.component = drm_to_xe_ras_component(component); | |
| 303 | + | |
| 304 | + xe_sysctrl_create_command(&command, XE_SYSCTRL_GROUP_GFSP, XE_SYSCTRL_CMD_CLEAR_COUNTER, | |
| 305 | + &request, sizeof(request), &response, sizeof(response)); | |
| 306 | + | |
| 307 | + guard(xe_pm_runtime)(xe); | |
| 308 | + ret = xe_sysctrl_send_command(&xe->sc, &command, &rlen); | |
| 309 | + if (ret) { | |
| 310 | + xe_err(xe, "sysctrl: failed to clear counter %d\n", ret); | |
| 311 | + return ret; | |
| 312 | + } | |
| 313 | + | |
| 314 | + if (rlen != sizeof(response)) { | |
| 315 | + xe_err(xe, "sysctrl: unexpected clear counter response length %zu (expected %zu)\n", | |
| 316 | + rlen, sizeof(response)); | |
| 317 | + return -EIO; | |
| 318 | + } | |
| 319 | + | |
| 320 | + ret = ras_status_to_errno(response.status); | |
| 321 | + if (ret) { | |
| 322 | + xe_err(xe, "sysctrl: clear counter command failed with status %#x\n", | |
| 323 | + response.status); | |
| 324 | + return ret; | |
| 325 | + } | |
| 326 | + | |
| 327 | + counter = &response.counter; | |
| 328 | + | |
| 329 | + xe_dbg(xe, "[RAS]: clear counter for %s %s\n", comp_to_str(counter->common.component), | |
| 330 | + sev_to_str(counter->common.severity)); | |
| 331 | + | |
| 332 | + return 0; | |
| 333 | +} | |
| 334 | + | |
| 335 | +/** | |
| 336 | + * xe_ras_init - Initialize Xe RAS | |
| 337 | + * @xe: xe device instance | |
| 338 | + * | |
| 339 | + * Initialize Xe RAS | |
| 340 | + */ | |
| 341 | +void xe_ras_init(struct xe_device *xe) | |
| 342 | +{ | |
| 343 | + if (!xe->info.has_drm_ras) | |
| 344 | + return; | |
| 345 | + | |
| 346 | + xe_drm_ras_init(xe); | |
| 347 | + | |
| 348 | + if (!xe->info.has_sysctrl) | |
| 349 | + return; | |
| 350 | + | |
| 351 | + if (IS_ENABLED(CONFIG_PCIEAER)) | |
| 352 | + ras_usp_aer_init(xe); | |
| 353 | +} | |
| @@ -6,10 +6,15 @@ · 5 unmodified lines | ||
| 6 | 6 | #ifndef _XE_RAS_H_ |
| 7 | 7 | #define _XE_RAS_H_ |
| 8 | 8 | |
| 9 | +#include <linux/types.h> | |
| 10 | + | |
| 9 | 11 | struct xe_device; |
| 10 | 12 | struct xe_sysctrl_event_response; |
| 11 | 13 | |
| 12 | 14 | void xe_ras_counter_threshold_crossed(struct xe_device *xe, |
| 13 | 15 | struct xe_sysctrl_event_response *response); |
| 16 | +int xe_ras_get_counter(struct xe_device *xe, u8 severity, u8 component, u32 *value); | |
| 17 | +int xe_ras_clear_counter(struct xe_device *xe, u8 severity, u8 component); | |
| 18 | +void xe_ras_init(struct xe_device *xe); | |
| 14 | 19 | |
| 15 | 20 | #endif |
| @@ -70,4 +70,55 @@ struct xe_ras_threshold_crossed { · 69 unmodified lines | ||
| 70 | 70 | struct xe_ras_error_class counters[XE_RAS_NUM_COUNTERS]; |
| 71 | 71 | } __packed; |
| 72 | 72 | |
| 73 | +/** | |
| 74 | + * struct xe_ras_get_counter_request - Request structure for get counter | |
| 75 | + */ | |
| 76 | +struct xe_ras_get_counter_request { | |
| 77 | + /** @counter: Error counter to be queried */ | |
| 78 | + struct xe_ras_error_class counter; | |
| 79 | + /** @reserved: Reserved for future use */ | |
| 80 | + u32 reserved; | |
| 81 | +} __packed; | |
| 82 | + | |
| 83 | +/** | |
| 84 | + * struct xe_ras_get_counter_response - Response structure for get counter | |
| 85 | + */ | |
| 86 | +struct xe_ras_get_counter_response { | |
| 87 | + /** @counter: Error counter that was queried */ | |
| 88 | + struct xe_ras_error_class counter; | |
| 89 | + /** @value: Current counter value */ | |
| 90 | + u32 value; | |
| 91 | + /** @timestamp: Timestamp when counter was last updated */ | |
| 92 | + u64 timestamp; | |
| 93 | + /** @threshold: Threshold value for the counter */ | |
| 94 | + u32 threshold; | |
| 95 | + /** @reserved: Reserved */ | |
| 96 | + u32 reserved[57]; | |
| 97 | +} __packed; | |
| 98 | + | |
| 99 | +/** | |
| 100 | + * struct xe_ras_clear_counter_request - Request structure for clear counter | |
| 101 | + */ | |
| 102 | +struct xe_ras_clear_counter_request { | |
| 103 | + /** @counter: Counter class to be cleared */ | |
| 104 | + struct xe_ras_error_class counter; | |
| 105 | + /** @reserved: Reserved for future use */ | |
| 106 | + u32 reserved; | |
| 107 | +} __packed; | |
| 108 | + | |
| 109 | +/** | |
| 110 | + * struct xe_ras_clear_counter_response - Response structure for clear counter | |
| 111 | + */ | |
| 112 | +struct xe_ras_clear_counter_response { | |
| 113 | + /** @counter: Counter class that was cleared */ | |
| 114 | + struct xe_ras_error_class counter; | |
| 115 | + /** @reserved: Reserved */ | |
| 116 | + u32 reserved; | |
| 117 | + /** @timestamp: Timestamp when the counter was cleared */ | |
| 118 | + u64 timestamp; | |
| 119 | + /** @status: Status of the clear operation */ | |
| 120 | + u32 status; | |
| 121 | + /** @reserved1: Reserved for future use */ | |
| 122 | + u32 reserved1[3]; | |
| 123 | +} __packed; | |
| 73 | 124 | #endif |
| @@ -5,6 +5,8 @@ · 4 unmodified lines | ||
| 5 | 5 | |
| 6 | 6 | #include "xe_reg_whitelist.h" |
| 7 | 7 | |
| 8 | +#include <kunit/visibility.h> | |
| 9 | + | |
| 8 | 10 | #include "regs/xe_engine_regs.h" |
| 9 | 11 | #include "regs/xe_gt_regs.h" |
| 10 | 12 | #include "regs/xe_oa_regs.h" |
| @@ -41,7 +43,7 @@ static bool match_multi_queue_class(const struct xe_device *xe, · 30 unmodified lines | ||
| 41 | 43 | return xe_gt_supports_multi_queue(gt, hwe->class); |
| 42 | 44 | } |
| 43 | 45 | |
| 44 | −static const struct xe_rtp_table_sr register_whitelist = XE_RTP_TABLE_SR( | |
| 46 | +VISIBLE_IF_KUNIT const struct xe_rtp_table_sr register_whitelist = XE_RTP_TABLE_SR( | |
| 45 | 47 | { XE_RTP_NAME("WaAllowPMDepthAndInvocationCountAccessFromUMD, 1408556865"), |
| 46 | 48 | XE_RTP_RULES(GRAPHICS_VERSION_RANGE(1200, 1210), ENGINE_CLASS(RENDER)), |
| 47 | 49 | XE_RTP_ACTIONS(WHITELIST(PS_INVOCATION_COUNT, |
| @@ -104,6 +106,7 @@ static const struct xe_rtp_table_sr register_whitelist = XE_RTP_TABLE_SR( · 56 unmodified lines | ||
| 104 | 106 | RING_FORCE_TO_NONPRIV_ACCESS_RW)) |
| 105 | 107 | }, |
| 106 | 108 | ); |
| 109 | +EXPORT_SYMBOL_IF_KUNIT(register_whitelist); | |
| 107 | 110 | |
| 108 | 111 | static const struct xe_rtp_table_sr oa_whitelist = XE_RTP_TABLE_SR( |
| 109 | 112 | |
| @@ -14,6 +14,10 @@ struct xe_hw_engine; · 13 unmodified lines | ||
| 14 | 14 | struct xe_reg_sr; |
| 15 | 15 | struct xe_reg_sr_entry; |
| 16 | 16 | |
| 17 | +#if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) | |
| 18 | +extern const struct xe_rtp_table_sr register_whitelist; | |
| 19 | +#endif | |
| 20 | + | |
| 17 | 21 | void xe_reg_whitelist_process_engine(struct xe_hw_engine *hwe); |
| 18 | 22 | |
| 19 | 23 | void xe_reg_whitelist_oa_regs(struct xe_gt *gt); |
| @@ -227,17 +227,23 @@ static bool rule_matches(const struct xe_device *xe, · 226 unmodified lines | ||
| 227 | 227 | |
| 228 | 228 | static void rtp_add_sr_entry(const struct xe_rtp_action *action, |
| 229 | 229 | struct xe_gt *gt, |
| 230 | + struct xe_hw_engine *hwe, | |
| 230 | 231 | u32 mmio_base, |
| 231 | 232 | struct xe_reg_sr *sr) |
| 232 | 233 | { |
| 233 | 234 | struct xe_reg_sr_entry sr_entry = { |
| 234 | 235 | .reg = action->reg, |
| 235 | 236 | .clr_bits = action->clr_bits, |
| 236 | − .set_bits = action->set_bits, | |
| 237 | 237 | .read_mask = action->read_mask, |
| 238 | 238 | }; |
| 239 | 239 | |
| 240 | + if (action->use_func) | |
| 241 | + sr_entry.set_bits = action->set_func(gt, hwe); | |
| 242 | + else | |
| 243 | + sr_entry.set_bits = action->set_bits; | |
| 244 | + | |
| 240 | 245 | sr_entry.reg.addr += mmio_base; |
| 246 | + | |
| 241 | 247 | xe_reg_sr_add(sr, &sr_entry, gt); |
| 242 | 248 | } |
| 243 | 249 | |
| @@ -259,7 +265,7 @@ static bool rtp_process_one_sr(const struct xe_rtp_entry_sr *entry, · 15 unmodified lines | ||
| 259 | 265 | else |
| 260 | 266 | mmio_base = 0; |
| 261 | 267 | |
| 262 | − rtp_add_sr_entry(action, gt, mmio_base, sr); | |
| 268 | + rtp_add_sr_entry(action, gt, hwe, mmio_base, sr); | |
| 263 | 269 | } |
| 264 | 270 | |
| 265 | 271 | return true; |
| @@ -322,6 +322,25 @@ struct xe_reg_sr; · 321 unmodified lines | ||
| 322 | 322 | .clr_bits = (mask_bits_), .set_bits = (val_), \ |
| 323 | 323 | .read_mask = 0, ##__VA_ARGS__ } |
| 324 | 324 | |
| 325 | +/** | |
| 326 | + * XE_RTP_ACTION_FIELD_SET_FUNC: Set a bit range to the value returned by a function | |
| 327 | + * @reg_: Register | |
| 328 | + * @mask_bits_: Mask of bits to be changed in the register, forming a field | |
| 329 | + * @func_: Function that returns value to set in the field denoted by @mask_bits_ | |
| 330 | + * @...: Additional fields to override in the struct xe_rtp_action entry | |
| 331 | + * | |
| 332 | + * This macro works like XE_RTP_ACTION_FIELD_SET(), except that the | |
| 333 | + * field value is evaluated at the time the RTP table is processed. | |
| 334 | + * | |
| 335 | + * @func_ will only be called a single time, when the RTP table is being | |
| 336 | + * processed. After processing, the value in the reg_sr entry is fixed and | |
| 337 | + * will not be re-evaluated. | |
| 338 | + */ | |
| 339 | +#define XE_RTP_ACTION_FIELD_SET_FUNC(reg_, mask_bits_, func_, ...) \ | |
| 340 | + { .reg = XE_RTP_DROP_CAST(reg_), \ | |
| 341 | + .clr_bits = mask_bits_, .set_func = func_, .use_func = 1, \ | |
| 342 | + .read_mask = mask_bits_, ##__VA_ARGS__ } | |
| 343 | + | |
| 325 | 344 | /** |
| 326 | 345 | * XE_RTP_ACTION_WHITELIST - Add register to userspace whitelist |
| 327 | 346 | * @reg_: Register |
| @@ -22,20 +22,37 @@ struct xe_gt; · 21 unmodified lines | ||
| 22 | 22 | */ |
| 23 | 23 | struct xe_rtp_action { |
| 24 | 24 | /** @reg: Register */ |
| 25 | − struct xe_reg reg; | |
| 25 | + struct xe_reg reg; | |
| 26 | + | |
| 26 | 27 | /** |
| 27 | 28 | * @clr_bits: bits to clear when updating register. It's always a |
| 28 | 29 | * superset of bits being modified |
| 29 | 30 | */ |
| 30 | − u32 clr_bits; | |
| 31 | − /** @set_bits: bits to set when updating register */ | |
| 32 | − u32 set_bits; | |
| 31 | + u32 clr_bits; | |
| 32 | + | |
| 33 | + union { | |
| 34 | + /** @set_bits: bits to set when updating register */ | |
| 35 | + u32 set_bits; | |
| 36 | + | |
| 37 | + /** @set_func: function to provide bits to set when updating register */ | |
| 38 | + u32 (*set_func)(struct xe_gt *gt, | |
| 39 | + struct xe_hw_engine *hwe); | |
| 40 | + }; | |
| 41 | + | |
| 33 | 42 | #define XE_RTP_NOCHECK .read_mask = 0 |
| 34 | 43 | /** @read_mask: mask for bits to consider when reading value back */ |
| 35 | − u32 read_mask; | |
| 44 | + u32 read_mask; | |
| 45 | + | |
| 36 | 46 | #define XE_RTP_ACTION_FLAG_ENGINE_BASE BIT(0) |
| 37 | 47 | /** @flags: flags to apply on rule evaluation or action */ |
| 38 | − u8 flags; | |
| 48 | + u8 flags; | |
| 49 | + | |
| 50 | + /** | |
| 51 | + * @use_func: | |
| 52 | + * Internal flag indicating @set_func should be called instead of | |
| 53 | + * using @set_bits. | |
| 54 | + */ | |
| 55 | + u8 use_func:1; | |
| 39 | 56 | }; |
| 40 | 57 | |
| 41 | 58 | enum { |
| @@ -69,6 +86,7 @@ struct xe_rtp_rule { · 27 unmodified lines | ||
| 69 | 86 | u8 platform; |
| 70 | 87 | u8 subplatform; |
| 71 | 88 | }; |
| 89 | + | |
| 72 | 90 | /* |
| 73 | 91 | * MATCH_GRAPHICS_VERSION / XE_RTP_MATCH_GRAPHICS_VERSION_RANGE / |
| 74 | 92 | * MATCH_MEDIA_VERSION / XE_RTP_MATCH_MEDIA_VERSION_RANGE |
| @@ -78,15 +96,18 @@ struct xe_rtp_rule { · 3 unmodified lines | ||
| 78 | 96 | #define XE_RTP_END_VERSION_UNDEFINED U32_MAX |
| 79 | 97 | u32 ver_end; |
| 80 | 98 | }; |
| 99 | + | |
| 81 | 100 | /* MATCH_STEP */ |
| 82 | 101 | struct { |
| 83 | 102 | u8 step_start; |
| 84 | 103 | u8 step_end; |
| 85 | 104 | }; |
| 105 | + | |
| 86 | 106 | /* MATCH_ENGINE_CLASS / MATCH_NOT_ENGINE_CLASS */ |
| 87 | 107 | struct { |
| 88 | 108 | u8 engine_class; |
| 89 | 109 | }; |
| 110 | + | |
| 90 | 111 | /* MATCH_FUNC */ |
| 91 | 112 | bool (*match_func)(const struct xe_device *xe, |
| 92 | 113 | const struct xe_gt *gt, |
| @@ -110,12 +110,14 @@ __diag_pop(); · 109 unmodified lines | ||
| 110 | 110 | |
| 111 | 111 | /** |
| 112 | 112 | * xe_step_platform_get - Determine platform-level stepping from PCI revid |
| 113 | − * @xe: Xe device | |
| 113 | + * @platform: The Xe platform | |
| 114 | + * @revid: The PCI revid | |
| 115 | + * @step: Pointer to the step struct to update | |
| 114 | 116 | * |
| 115 | 117 | * Convert the PCI revid into a platform-level stepping value and store that |
| 116 | − * in the device info. | |
| 118 | + * in @step->platform. | |
| 117 | 119 | */ |
| 118 | −void xe_step_platform_get(struct xe_device *xe) | |
| 120 | +void xe_step_platform_get(enum xe_platform platform, u8 revid, struct xe_step_info *step) | |
| 119 | 121 | { |
| 120 | 122 | /* |
| 121 | 123 | * Not all platforms map PCI revid directly into our symbolic stepping |
| @@ -126,18 +128,21 @@ void xe_step_platform_get(struct xe_device *xe) · 4 unmodified lines | ||
| 126 | 128 | * checks. |
| 127 | 129 | */ |
| 128 | 130 | |
| 129 | − if (xe->info.platform == XE_NOVALAKE_P) | |
| 130 | − xe->info.step.platform = STEP_A0 + xe->info.revid; | |
| 131 | + if (platform == XE_NOVALAKE_P) | |
| 132 | + step->platform = STEP_A0 + revid; | |
| 131 | 133 | } |
| 132 | 134 | |
| 133 | 135 | /** |
| 134 | 136 | * xe_step_pre_gmdid_get - Determine IP steppings from PCI revid |
| 135 | 137 | * @xe: Xe device |
| 138 | + * @step: Pointer to the step struct to update | |
| 139 | + * | |
| 140 | + * Convert the PCI revid into proper IP steppings and update @step->basedie, | |
| 141 | + * @step->graphics and @step->media accordingly. | |
| 136 | 142 | * |
| 137 | − * Convert the PCI revid into proper IP steppings. This should only be | |
| 138 | − * used on platforms that do not have GMD_ID support. | |
| 143 | + * This should only be used on platforms that do not have GMD_ID support. | |
| 139 | 144 | */ |
| 140 | −void xe_step_pre_gmdid_get(struct xe_device *xe) | |
| 145 | +void xe_step_pre_gmdid_get(struct xe_device *xe, struct xe_step_info *step) | |
| 141 | 146 | { |
| 142 | 147 | const struct xe_step_info *revids = NULL; |
| 143 | 148 | u16 revid = xe->info.revid; |
| @@ -234,9 +239,9 @@ void xe_step_pre_gmdid_get(struct xe_device *xe) · 90 unmodified lines | ||
| 234 | 239 | } |
| 235 | 240 | |
| 236 | 241 | done: |
| 237 | − xe->info.step.graphics = graphics; | |
| 238 | − xe->info.step.media = media; | |
| 239 | − xe->info.step.basedie = basedie; | |
| 242 | + step->graphics = graphics; | |
| 243 | + step->media = media; | |
| 244 | + step->basedie = basedie; | |
| 240 | 245 | } |
| 241 | 246 | |
| 242 | 247 | /** |
| @@ -244,8 +249,10 @@ done: · 1 unmodified line | ||
| 244 | 249 | * @xe: Xe device |
| 245 | 250 | * @graphics_gmdid_revid: value of graphics GMD_ID register's revid field |
| 246 | 251 | * @media_gmdid_revid: value of media GMD_ID register's revid field |
| 252 | + * @step: Poninter to the step struct to update. | |
| 247 | 253 | * |
| 248 | − * Convert the revid fields of the GMD_ID registers into proper IP steppings. | |
| 254 | + * Convert the revid fields of the GMD_ID registers into proper IP steppings | |
| 255 | + * and update @step->graphics and @step->media accordingly. | |
| 249 | 256 | * |
| 250 | 257 | * GMD_ID revid values are currently expected to have consistent meanings on |
| 251 | 258 | * all platforms: major steppings (A0, B0, etc.) are 4 apart, with minor |
| @@ -253,7 +260,8 @@ done: · 1 unmodified line | ||
| 253 | 260 | */ |
| 254 | 261 | void xe_step_gmdid_get(struct xe_device *xe, |
| 255 | 262 | u32 graphics_gmdid_revid, |
| 256 | − u32 media_gmdid_revid) | |
| 263 | + u32 media_gmdid_revid, | |
| 264 | + struct xe_step_info *step) | |
| 257 | 265 | { |
| 258 | 266 | u8 graphics = STEP_A0 + graphics_gmdid_revid; |
| 259 | 267 | u8 media = STEP_A0 + media_gmdid_revid; |
| @@ -270,8 +278,8 @@ void xe_step_gmdid_get(struct xe_device *xe, · 10 unmodified lines | ||
| 270 | 278 | media_gmdid_revid); |
| 271 | 279 | } |
| 272 | 280 | |
| 273 | − xe->info.step.graphics = graphics; | |
| 274 | − xe->info.step.media = media; | |
| 281 | + step->graphics = graphics; | |
| 282 | + step->media = media; | |
| 275 | 283 | } |
| 276 | 284 | |
| 277 | 285 | #define STEP_NAME_CASE(name) \ |
| @@ -10,14 +10,16 @@ · 9 unmodified lines | ||
| 10 | 10 | |
| 11 | 11 | #include "xe_step_types.h" |
| 12 | 12 | |
| 13 | +enum xe_platform; | |
| 13 | 14 | struct xe_device; |
| 14 | 15 | |
| 15 | −void xe_step_platform_get(struct xe_device *xe); | |
| 16 | +void xe_step_platform_get(enum xe_platform platform, u8 revid, struct xe_step_info *step); | |
| 16 | 17 | |
| 17 | −void xe_step_pre_gmdid_get(struct xe_device *xe); | |
| 18 | +void xe_step_pre_gmdid_get(struct xe_device *xe, struct xe_step_info *step); | |
| 18 | 19 | void xe_step_gmdid_get(struct xe_device *xe, |
| 19 | 20 | u32 graphics_gmdid_revid, |
| 20 | − u32 media_gmdid_revid); | |
| 21 | + u32 media_gmdid_revid, | |
| 22 | + struct xe_step_info *step); | |
| 21 | 23 | static inline u32 xe_step_to_gmdid(enum intel_step step) { return step - STEP_A0; } |
| 22 | 24 | |
| 23 | 25 | const char *xe_step_name(enum intel_step step); |
| @@ -293,6 +293,34 @@ static int sysctrl_send_command(struct xe_sysctrl *sc, · 292 unmodified lines | ||
| 293 | 293 | return 0; |
| 294 | 294 | } |
| 295 | 295 | |
| 296 | +/** | |
| 297 | + * xe_sysctrl_create_command() - Create system controller command | |
| 298 | + * @command: Sysctrl command structure | |
| 299 | + * @group_id: Command group ID | |
| 300 | + * @cmd_id: Command ID | |
| 301 | + * @request: Pointer to request buffer (can be NULL) | |
| 302 | + * @request_len: Size of request buffer | |
| 303 | + * @response: Pointer to response buffer | |
| 304 | + * @response_len: Size of response buffer | |
| 305 | + * | |
| 306 | + * Helper function to create sysctrl command to be sent via %xe_sysctrl_send_command() | |
| 307 | + */ | |
| 308 | +void xe_sysctrl_create_command(struct xe_sysctrl_mailbox_command *command, u8 group_id, u8 cmd_id, | |
| 309 | + void *request, size_t request_len, void *response, | |
| 310 | + size_t response_len) | |
| 311 | +{ | |
| 312 | + struct xe_sysctrl_app_msg_hdr header = {0}; | |
| 313 | + | |
| 314 | + header.data = FIELD_PREP(APP_HDR_GROUP_ID_MASK, group_id) | | |
| 315 | + FIELD_PREP(APP_HDR_COMMAND_MASK, cmd_id); | |
| 316 | + | |
| 317 | + command->header = header; | |
| 318 | + command->data_in = request; | |
| 319 | + command->data_in_len = request_len; | |
| 320 | + command->data_out = response; | |
| 321 | + command->data_out_len = response_len; | |
| 322 | +} | |
| 323 | + | |
| 296 | 324 | /** |
| 297 | 325 | * xe_sysctrl_mailbox_init - Initialize System Controller mailbox interface |
| 298 | 326 | * @sc: System controller structure |
| @@ -23,6 +23,9 @@ struct xe_sysctrl_mailbox_command; · 22 unmodified lines | ||
| 23 | 23 | #define XE_SYSCTRL_APP_HDR_VERSION(hdr) \ |
| 24 | 24 | FIELD_GET(APP_HDR_VERSION_MASK, (hdr)->data) |
| 25 | 25 | |
| 26 | +void xe_sysctrl_create_command(struct xe_sysctrl_mailbox_command *command, u8 group_id, u8 cmd_id, | |
| 27 | + void *request, size_t request_len, void *response, | |
| 28 | + size_t response_len); | |
| 26 | 29 | void xe_sysctrl_mailbox_init(struct xe_sysctrl *sc); |
| 27 | 30 | int xe_sysctrl_send_command(struct xe_sysctrl *sc, |
| 28 | 31 | struct xe_sysctrl_mailbox_command *cmd, |
| @@ -22,9 +22,13 @@ enum xe_sysctrl_group { · 21 unmodified lines | ||
| 22 | 22 | /** |
| 23 | 23 | * enum xe_sysctrl_gfsp_cmd - Commands supported by GFSP group |
| 24 | 24 | * |
| 25 | + * @XE_SYSCTRL_CMD_GET_COUNTER: Get error counter value | |
| 26 | + * @XE_SYSCTRL_CMD_CLEAR_COUNTER: Clear error counter value | |
| 25 | 27 | * @XE_SYSCTRL_CMD_GET_PENDING_EVENT: Retrieve pending event |
| 26 | 28 | */ |
| 27 | 29 | enum xe_sysctrl_gfsp_cmd { |
| 30 | + XE_SYSCTRL_CMD_GET_COUNTER = 0x03, | |
| 31 | + XE_SYSCTRL_CMD_CLEAR_COUNTER = 0x04, | |
| 28 | 32 | XE_SYSCTRL_CMD_GET_PENDING_EVENT = 0x07, |
| 29 | 33 | }; |
| 30 | 34 | |
| @@ -157,7 +157,9 @@ int xe_tile_init_early(struct xe_tile *tile, struct xe_device *xe, u8 id) · 156 unmodified lines | ||
| 157 | 157 | if (err) |
| 158 | 158 | return err; |
| 159 | 159 | |
| 160 | − xe_pcode_init(tile); | |
| 160 | + err = xe_pcode_init_early(tile); | |
| 161 | + if (err) | |
| 162 | + return err; | |
| 161 | 163 | |
| 162 | 164 | return 0; |
| 163 | 165 | } |
| @@ -20,7 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | #undef XE_REG_MCR |
| 21 | 21 | #define XE_REG_MCR(...) XE_REG(__VA_ARGS__, .mcr = 1) |
| 22 | 22 | |
| 23 | −static const struct xe_rtp_table_sr gt_tunings = XE_RTP_TABLE_SR( | |
| 23 | +VISIBLE_IF_KUNIT const struct xe_rtp_table_sr gt_tunings = XE_RTP_TABLE_SR( | |
| 24 | 24 | { XE_RTP_NAME("Tuning: Blend Fill Caching Optimization Disable"), |
| 25 | 25 | XE_RTP_RULES(PLATFORM(DG2)), |
| 26 | 26 | XE_RTP_ACTIONS(SET(XEHP_L3SCQREG7, BLEND_FILL_CACHING_OPT_DIS)) |
| @@ -101,6 +101,7 @@ static const struct xe_rtp_table_sr gt_tunings = XE_RTP_TABLE_SR( · 74 unmodified lines | ||
| 101 | 101 | BANK_HASH_4KB_MODE)) |
| 102 | 102 | }, |
| 103 | 103 | ); |
| 104 | +EXPORT_SYMBOL_IF_KUNIT(gt_tunings); | |
| 104 | 105 | |
| 105 | 106 | static const struct xe_rtp_table_sr engine_tunings = XE_RTP_TABLE_SR( |
| 106 | 107 | { XE_RTP_NAME("Tuning: L3 Hashing Mask"), |
| @@ -6,6 +6,8 @@ · 5 unmodified lines | ||
| 6 | 6 | #ifndef _XE_TUNING_H_ |
| 7 | 7 | #define _XE_TUNING_H_ |
| 8 | 8 | |
| 9 | +#include <kunit/visibility.h> | |
| 10 | + | |
| 9 | 11 | struct drm_printer; |
| 10 | 12 | struct xe_gt; |
| 11 | 13 | struct xe_hw_engine; |
| @@ -16,4 +18,8 @@ void xe_tuning_process_engine(struct xe_hw_engine *hwe); · 4 unmodified lines | ||
| 16 | 18 | void xe_tuning_process_lrc(struct xe_hw_engine *hwe); |
| 17 | 19 | int xe_tuning_dump(struct xe_gt *gt, struct drm_printer *p); |
| 18 | 20 | |
| 21 | +#if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) | |
| 22 | +extern const struct xe_rtp_table_sr gt_tunings; | |
| 23 | +#endif | |
| 24 | + | |
| 19 | 25 | #endif |
| @@ -332,6 +332,20 @@ static int xe_vm_invalidate_madvise_range(struct xe_vm *vm, u64 start, u64 end) · 331 unmodified lines | ||
| 332 | 332 | return err; |
| 333 | 333 | } |
| 334 | 334 | |
| 335 | +/** | |
| 336 | + * madvise_range_needs_invalidation() - Check whether madvise needs invalidation | |
| 337 | + * @args: madvise ioctl arguments | |
| 338 | + * | |
| 339 | + * Purgeable state updates only touch VMA/BO metadata. PTEs stay valid and are | |
| 340 | + * zapped only if the BO is later purged. | |
| 341 | + * | |
| 342 | + * Return: true when the update needs PTE invalidation. | |
| 343 | + */ | |
| 344 | +static bool madvise_range_needs_invalidation(const struct drm_xe_madvise *args) | |
| 345 | +{ | |
| 346 | + return args->type != DRM_XE_VMA_ATTR_PURGEABLE_STATE; | |
| 347 | +} | |
| 348 | + | |
| 335 | 349 | static bool madvise_args_are_sane(struct xe_device *xe, const struct drm_xe_madvise *args) |
| 336 | 350 | { |
| 337 | 351 | if (XE_IOCTL_DBG(xe, !args)) |
| @@ -708,8 +722,9 @@ int xe_vm_madvise_ioctl(struct drm_device *dev, void *data, struct drm_file *fil · 370 unmodified lines | ||
| 708 | 722 | madvise_funcs[attr_type](xe, vm, madvise_range.vmas, madvise_range.num_vmas, args, |
| 709 | 723 | &details); |
| 710 | 724 | |
| 711 | − err = xe_vm_invalidate_madvise_range(vm, madvise_range.addr, | |
| 712 | − madvise_range.addr + args->range); | |
| 725 | + if (madvise_range_needs_invalidation(args)) | |
| 726 | + err = xe_vm_invalidate_madvise_range(vm, madvise_range.addr, | |
| 727 | + madvise_range.addr + args->range); | |
| 713 | 728 | |
| 714 | 729 | if (madvise_range.has_svm_userptr_vmas) |
| 715 | 730 | xe_svm_notifier_unlock(vm); |
| @@ -130,7 +130,7 @@ · 129 unmodified lines | ||
| 130 | 130 | __diag_push(); |
| 131 | 131 | __diag_ignore_all("-Woverride-init", "Allow field overrides in table"); |
| 132 | 132 | |
| 133 | −static const struct xe_rtp_table_sr gt_was = XE_RTP_TABLE_SR( | |
| 133 | +VISIBLE_IF_KUNIT const struct xe_rtp_table_sr gt_was = XE_RTP_TABLE_SR( | |
| 134 | 134 | /* Workarounds applying over a range of IPs */ |
| 135 | 135 | |
| 136 | 136 | { XE_RTP_NAME("14011060649"), |
| @@ -293,7 +293,7 @@ static const struct xe_rtp_table_sr gt_was = XE_RTP_TABLE_SR( · 156 unmodified lines | ||
| 293 | 293 | XE_RTP_ACTIONS(SET(MMIOATSREQLIMIT_GAM_WALK_3D, |
| 294 | 294 | DIS_ATS_WRONLY_PG)) |
| 295 | 295 | }, |
| 296 | − { XE_RTP_NAME("14026144927, 16029437861"), | |
| 296 | + { XE_RTP_NAME("14026144927, 16029437861, 14026127056"), | |
| 297 | 297 | XE_RTP_RULES(GRAPHICS_VERSION(3510), GRAPHICS_STEP(A0, B0)), |
| 298 | 298 | XE_RTP_ACTIONS(SET(L3SQCREG2, L3_SQ_DISABLE_COAMA_2WAY_COH | |
| 299 | 299 | L3_SQ_DISABLE_COAMA)) |
| @@ -307,6 +307,7 @@ static const struct xe_rtp_table_sr gt_was = XE_RTP_TABLE_SR( · 7 unmodified lines | ||
| 307 | 307 | XE_RTP_ACTIONS(SET(GUC_INTR_CHICKEN, DISABLE_SIGNALING_ENGINES)) |
| 308 | 308 | }, |
| 309 | 309 | ); |
| 310 | +EXPORT_SYMBOL_IF_KUNIT(gt_was); | |
| 310 | 311 | |
| 311 | 312 | static const struct xe_rtp_table_sr engine_was = XE_RTP_TABLE_SR( |
| 312 | 313 | /* Workarounds applying over a range of IPs */ |
| @@ -586,12 +587,12 @@ static const struct xe_rtp_table_sr engine_was = XE_RTP_TABLE_SR( · 273 unmodified lines | ||
| 586 | 587 | |
| 587 | 588 | /* Xe3p_LPG*/ |
| 588 | 589 | |
| 589 | − { XE_RTP_NAME("22021149932"), | |
| 590 | + { XE_RTP_NAME("22021149932, 14026290593"), | |
| 590 | 591 | XE_RTP_RULES(GRAPHICS_VERSION(3510), GRAPHICS_STEP(A0, B0), |
| 591 | 592 | FUNC(xe_rtp_match_first_render_or_compute)), |
| 592 | 593 | XE_RTP_ACTIONS(SET(LSC_CHICKEN_BIT_0_UDW, SAMPLER_LD_LSC_DISABLE)) |
| 593 | 594 | }, |
| 594 | − { XE_RTP_NAME("14025676848"), | |
| 595 | + { XE_RTP_NAME("14025676848, 14026270459"), | |
| 595 | 596 | XE_RTP_RULES(GRAPHICS_VERSION(3510), GRAPHICS_STEP(A0, B0), |
| 596 | 597 | FUNC(xe_rtp_match_first_render_or_compute)), |
| 597 | 598 | XE_RTP_ACTIONS(SET(LSC_CHICKEN_BIT_0_UDW, LSCFE_SAME_ADDRESS_ATOMICS_COALESCING_DISABLE)) |
| @@ -802,10 +803,11 @@ static const struct xe_rtp_entry oob_was_entries[] = { · 204 unmodified lines | ||
| 802 | 803 | |
| 803 | 804 | static_assert(ARRAY_SIZE(oob_was_entries) == _XE_WA_OOB_COUNT); |
| 804 | 805 | |
| 805 | −static __maybe_unused const struct xe_rtp_table oob_was = { | |
| 806 | +VISIBLE_IF_KUNIT __maybe_unused const struct xe_rtp_table oob_was = { | |
| 806 | 807 | .entries = oob_was_entries, |
| 807 | 808 | .n_entries = ARRAY_SIZE(oob_was_entries), |
| 808 | 809 | }; |
| 810 | +EXPORT_SYMBOL_IF_KUNIT(oob_was); | |
| 809 | 811 | |
| 810 | 812 | static const struct xe_rtp_entry device_oob_was_entries[] = { |
| 811 | 813 | #include <generated/xe_device_wa_oob.c> |
| @@ -813,10 +815,11 @@ static const struct xe_rtp_entry device_oob_was_entries[] = { · 1 unmodified line | ||
| 813 | 815 | |
| 814 | 816 | static_assert(ARRAY_SIZE(device_oob_was_entries) == _XE_DEVICE_WA_OOB_COUNT); |
| 815 | 817 | |
| 816 | −static __maybe_unused const struct xe_rtp_table device_oob_was = { | |
| 818 | +VISIBLE_IF_KUNIT __maybe_unused const struct xe_rtp_table device_oob_was = { | |
| 817 | 819 | .entries = device_oob_was_entries, |
| 818 | 820 | .n_entries = ARRAY_SIZE(device_oob_was_entries), |
| 819 | 821 | }; |
| 822 | +EXPORT_SYMBOL_IF_KUNIT(device_oob_was); | |
| 820 | 823 | |
| 821 | 824 | __diag_pop(); |
| 822 | 825 | |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | #ifndef _XE_WA_H_ |
| 7 | 7 | #define _XE_WA_H_ |
| 8 | 8 | |
| 9 | +#include <kunit/visibility.h> | |
| 9 | 10 | #include "xe_assert.h" |
| 10 | 11 | |
| 11 | 12 | struct drm_printer; |
| @@ -24,6 +25,12 @@ void xe_wa_apply_tile_workarounds(struct xe_tile *tile); · 12 unmodified lines | ||
| 24 | 25 | void xe_wa_device_dump(struct xe_device *xe, struct drm_printer *p); |
| 25 | 26 | int xe_wa_gt_dump(struct xe_gt *gt, struct drm_printer *p); |
| 26 | 27 | |
| 28 | +#if IS_ENABLED(CONFIG_DRM_XE_KUNIT_TEST) | |
| 29 | +extern const struct xe_rtp_table_sr gt_was; | |
| 30 | +extern __maybe_unused const struct xe_rtp_table oob_was; | |
| 31 | +extern __maybe_unused const struct xe_rtp_table device_oob_was; | |
| 32 | +#endif | |
| 33 | + | |
| 27 | 34 | /** |
| 28 | 35 | * XE_GT_WA - Out-of-band GT workarounds, to be queried and called as needed. |
| 29 | 36 | * @gt__: gt instance |
| @@ -65,3 +65,4 @@ · 64 unmodified lines | ||
| 65 | 65 | |
| 66 | 66 | 14025883347 MEDIA_VERSION_RANGE(1301, 3503) |
| 67 | 67 | GRAPHICS_VERSION_RANGE(2004, 3005) |
| 68 | +16029380221 MEDIA_VERSION(3500) | |
| @@ -893,8 +893,9 @@ · 892 unmodified lines | ||
| 893 | 893 | MACRO__(0xD741, ## __VA_ARGS__), \ |
| 894 | 894 | MACRO__(0xD742, ## __VA_ARGS__), \ |
| 895 | 895 | MACRO__(0xD743, ## __VA_ARGS__), \ |
| 896 | − MACRO__(0xD744, ## __VA_ARGS__), \ | |
| 897 | − MACRO__(0xD745, ## __VA_ARGS__) | |
| 896 | + MACRO__(0xD745, ## __VA_ARGS__), \ | |
| 897 | + MACRO__(0xD74A, ## __VA_ARGS__), \ | |
| 898 | + MACRO__(0xD74B, ## __VA_ARGS__) | |
| 898 | 899 | |
| 899 | 900 | /* CRI */ |
| 900 | 901 | #define INTEL_CRI_IDS(MACRO__, ...) \ |
| @@ -1791,22 +1791,8 @@ static inline int perf_is_paranoid(void) · 1790 unmodified lines | ||
| 1791 | 1791 | } |
| 1792 | 1792 | |
| 1793 | 1793 | extern int perf_allow_kernel(void); |
| 1794 | − | |
| 1795 | −static inline int perf_allow_cpu(void) | |
| 1796 | −{ | |
| 1797 | − if (sysctl_perf_event_paranoid > 0 && !perfmon_capable()) | |
| 1798 | − return -EACCES; | |
| 1799 | − | |
| 1800 | − return security_perf_event_open(PERF_SECURITY_CPU); | |
| 1801 | −} | |
| 1802 | − | |
| 1803 | −static inline int perf_allow_tracepoint(void) | |
| 1804 | −{ | |
| 1805 | − if (sysctl_perf_event_paranoid > -1 && !perfmon_capable()) | |
| 1806 | − return -EPERM; | |
| 1807 | − | |
| 1808 | − return security_perf_event_open(PERF_SECURITY_TRACEPOINT); | |
| 1809 | −} | |
| 1794 | +extern int perf_allow_cpu(void); | |
| 1795 | +extern int perf_allow_tracepoint(void); | |
| 1810 | 1796 | |
| 1811 | 1797 | extern int perf_exclude_event(struct perf_event *event, struct pt_regs *regs); |
| 1812 | 1798 | |
| @@ -2023,6 +2009,19 @@ perf_event_pause(struct perf_event *event, bool reset) { return 0; } · 210 unmodified lines | ||
| 2023 | 2009 | static inline int |
| 2024 | 2010 | perf_exclude_event(struct perf_event *event, struct pt_regs *regs) { return 0; } |
| 2025 | 2011 | |
| 2012 | +static inline int perf_allow_kernel(void) | |
| 2013 | +{ | |
| 2014 | + return perfmon_capable() ? 0 : -EACCES; | |
| 2015 | +} | |
| 2016 | +static inline int perf_allow_cpu(void) | |
| 2017 | +{ | |
| 2018 | + return perfmon_capable() ? 0 : -EACCES; | |
| 2019 | +} | |
| 2020 | +static inline int perf_allow_tracepoint(void) | |
| 2021 | +{ | |
| 2022 | + return perfmon_capable() ? 0 : -EPERM; | |
| 2023 | +} | |
| 2024 | + | |
| 2026 | 2025 | #endif /* !CONFIG_PERF_EVENTS */ |
| 2027 | 2026 | |
| 2028 | 2027 | #if defined(CONFIG_PERF_EVENTS) && defined(CONFIG_CPU_SUP_INTEL) |
| @@ -2589,6 +2589,12 @@ enum drm_xe_ras_error_component { · 2588 unmodified lines | ||
| 2589 | 2589 | DRM_XE_RAS_ERR_COMP_CORE_COMPUTE = 1, |
| 2590 | 2590 | /** @DRM_XE_RAS_ERR_COMP_SOC_INTERNAL: SoC Internal Error */ |
| 2591 | 2591 | DRM_XE_RAS_ERR_COMP_SOC_INTERNAL, |
| 2592 | + /** @DRM_XE_RAS_ERR_COMP_DEVICE_MEMORY: Device Memory Error */ | |
| 2593 | + DRM_XE_RAS_ERR_COMP_DEVICE_MEMORY, | |
| 2594 | + /** @DRM_XE_RAS_ERR_COMP_PCIE: PCIe Subsystem Error */ | |
| 2595 | + DRM_XE_RAS_ERR_COMP_PCIE, | |
| 2596 | + /** @DRM_XE_RAS_ERR_COMP_FABRIC: Fabric Subsystem Error */ | |
| 2597 | + DRM_XE_RAS_ERR_COMP_FABRIC, | |
| 2592 | 2598 | /** @DRM_XE_RAS_ERR_COMP_MAX: Max Error */ |
| 2593 | 2599 | DRM_XE_RAS_ERR_COMP_MAX /* non-ABI */ |
| 2594 | 2600 | }; |
| @@ -2606,7 +2612,10 @@ enum drm_xe_ras_error_component { · 11 unmodified lines | ||
| 2606 | 2612 | */ |
| 2607 | 2613 | #define DRM_XE_RAS_ERROR_COMPONENT_NAMES { \ |
| 2608 | 2614 | [DRM_XE_RAS_ERR_COMP_CORE_COMPUTE] = "core-compute", \ |
| 2609 | − [DRM_XE_RAS_ERR_COMP_SOC_INTERNAL] = "soc-internal" \ | |
| 2615 | + [DRM_XE_RAS_ERR_COMP_SOC_INTERNAL] = "soc-internal", \ | |
| 2616 | + [DRM_XE_RAS_ERR_COMP_DEVICE_MEMORY] = "device-memory", \ | |
| 2617 | + [DRM_XE_RAS_ERR_COMP_PCIE] = "pcie", \ | |
| 2618 | + [DRM_XE_RAS_ERR_COMP_FABRIC] = "fabric", \ | |
| 2610 | 2619 | } |
| 2611 | 2620 | |
| 2612 | 2621 | #if defined(__cplusplus) |
| @@ -14769,6 +14769,24 @@ int perf_allow_kernel(void) · 14768 unmodified lines | ||
| 14769 | 14769 | } |
| 14770 | 14770 | EXPORT_SYMBOL_GPL(perf_allow_kernel); |
| 14771 | 14771 | |
| 14772 | +int perf_allow_cpu(void) | |
| 14773 | +{ | |
| 14774 | + if (sysctl_perf_event_paranoid > 0 && !perfmon_capable()) | |
| 14775 | + return -EACCES; | |
| 14776 | + | |
| 14777 | + return security_perf_event_open(PERF_SECURITY_CPU); | |
| 14778 | +} | |
| 14779 | +EXPORT_SYMBOL_GPL(perf_allow_cpu); | |
| 14780 | + | |
| 14781 | +int perf_allow_tracepoint(void) | |
| 14782 | +{ | |
| 14783 | + if (sysctl_perf_event_paranoid > -1 && !perfmon_capable()) | |
| 14784 | + return -EPERM; | |
| 14785 | + | |
| 14786 | + return security_perf_event_open(PERF_SECURITY_TRACEPOINT); | |
| 14787 | +} | |
| 14788 | +EXPORT_SYMBOL_GPL(perf_allow_tracepoint); | |
| 14789 | + | |
| 14772 | 14790 | /* |
| 14773 | 14791 | * Inherit an event from parent task to child task. |
| 14774 | 14792 | * |