Merge series from Peng Fan <peng.fan@nxp.com>: This patchset is a pick up of patch 1,2 from [1]. And I also collect Linus's R-b for patch 2. After this patchset, there is only one user of of_gpio.h left in sound driver(pxa2xx-ac97). of_gpio.h is deprecated, update the driver to use GPIO descriptors. Patch 1 is to drop legacy platform data which in-tree no users are using it Patch 2 is to convert to GPIO descriptors Checking the DTS that use the device, all are using GPIOD_ACTIVE_LOW polarity for reset-gpios, so all should work as expected with this patch. [1] https://lore.kernel.org/all/20250408-asoc-gpio-v1-0-c0db9d3fd6e9@nxp.com/
Unified · Split
| @@ -416,6 +416,7 @@ Kenneth W Chen <kenneth.w.chen@intel.com> · 415 unmodified lines | ||
| 416 | 416 | Kenneth Westfield <quic_kwestfie@quicinc.com> <kwestfie@codeaurora.org> |
| 417 | 417 | Kiran Gunda <quic_kgunda@quicinc.com> <kgunda@codeaurora.org> |
| 418 | 418 | Kirill Tkhai <tkhai@ya.ru> <ktkhai@virtuozzo.com> |
| 419 | +Kirill A. Shutemov <kas@kernel.org> <kirill.shutemov@linux.intel.com> | |
| 419 | 420 | Kishon Vijay Abraham I <kishon@kernel.org> <kishon@ti.com> |
| 420 | 421 | Konrad Dybcio <konradybcio@kernel.org> <konrad.dybcio@linaro.org> |
| 421 | 422 | Konrad Dybcio <konradybcio@kernel.org> <konrad.dybcio@somainline.org> |
| @@ -584,6 +584,7 @@ What: /sys/devices/system/cpu/vulnerabilities · 583 unmodified lines | ||
| 584 | 584 | /sys/devices/system/cpu/vulnerabilities/spectre_v1 |
| 585 | 585 | /sys/devices/system/cpu/vulnerabilities/spectre_v2 |
| 586 | 586 | /sys/devices/system/cpu/vulnerabilities/srbds |
| 587 | + /sys/devices/system/cpu/vulnerabilities/tsa | |
| 587 | 588 | /sys/devices/system/cpu/vulnerabilities/tsx_async_abort |
| 588 | 589 | Date: January 2018 |
| 589 | 590 | Contact: Linux kernel mailing list <linux-kernel@vger.kernel.org> |
| @@ -1732,12 +1732,6 @@ The following nested keys are defined. · 1731 unmodified lines | ||
| 1732 | 1732 | numa_hint_faults (npn) |
| 1733 | 1733 | Number of NUMA hinting faults. |
| 1734 | 1734 | |
| 1735 | − numa_task_migrated (npn) | |
| 1736 | − Number of task migration by NUMA balancing. | |
| 1737 | − | |
| 1738 | − numa_task_swapped (npn) | |
| 1739 | − Number of task swap by NUMA balancing. | |
| 1740 | − | |
| 1741 | 1735 | pgdemote_kswapd |
| 1742 | 1736 | Number of pages demoted by kswapd. |
| 1743 | 1737 | |
| @@ -157,9 +157,7 @@ This is achieved by using the otherwise unused and obsolete VERW instruction in · 156 unmodified lines | ||
| 157 | 157 | combination with a microcode update. The microcode clears the affected CPU |
| 158 | 158 | buffers when the VERW instruction is executed. |
| 159 | 159 | |
| 160 | −Kernel reuses the MDS function to invoke the buffer clearing: | |
| 161 | − | |
| 162 | − mds_clear_cpu_buffers() | |
| 160 | +Kernel does the buffer clearing with x86_clear_cpu_buffers(). | |
| 163 | 161 | |
| 164 | 162 | On MDS affected CPUs, the kernel already invokes CPU buffer clear on |
| 165 | 163 | kernel/userspace, hypervisor/guest and C-state (idle) transitions. No |
| @@ -7488,6 +7488,19 @@ · 7487 unmodified lines | ||
| 7488 | 7488 | having this key zero'ed is acceptable. E.g. in testing |
| 7489 | 7489 | scenarios. |
| 7490 | 7490 | |
| 7491 | + tsa= [X86] Control mitigation for Transient Scheduler | |
| 7492 | + Attacks on AMD CPUs. Search the following in your | |
| 7493 | + favourite search engine for more details: | |
| 7494 | + | |
| 7495 | + "Technical guidance for mitigating transient scheduler | |
| 7496 | + attacks". | |
| 7497 | + | |
| 7498 | + off - disable the mitigation | |
| 7499 | + on - enable the mitigation (default) | |
| 7500 | + user - mitigate only user/kernel transitions | |
| 7501 | + vm - mitigate only guest/host transitions | |
| 7502 | + | |
| 7503 | + | |
| 7491 | 7504 | tsc= Disable clocksource stability checks for TSC. |
| 7492 | 7505 | Format: <string> |
| 7493 | 7506 | [x86] reliable: mark tsc clocksource as reliable, this |
| @@ -93,7 +93,7 @@ enters a C-state. · 92 unmodified lines | ||
| 93 | 93 | |
| 94 | 94 | The kernel provides a function to invoke the buffer clearing: |
| 95 | 95 | |
| 96 | − mds_clear_cpu_buffers() | |
| 96 | + x86_clear_cpu_buffers() | |
| 97 | 97 | |
| 98 | 98 | Also macro CLEAR_CPU_BUFFERS can be used in ASM late in exit-to-user path. |
| 99 | 99 | Other than CFLAGS.ZF, this macro doesn't clobber any registers. |
| @@ -185,9 +185,9 @@ Mitigation points · 85 unmodified lines | ||
| 185 | 185 | idle clearing would be a window dressing exercise and is therefore not |
| 186 | 186 | activated. |
| 187 | 187 | |
| 188 | − The invocation is controlled by the static key mds_idle_clear which is | |
| 189 | − switched depending on the chosen mitigation mode and the SMT state of | |
| 190 | − the system. | |
| 188 | + The invocation is controlled by the static key cpu_buf_idle_clear which is | |
| 189 | + switched depending on the chosen mitigation mode and the SMT state of the | |
| 190 | + system. | |
| 191 | 191 | |
| 192 | 192 | The buffer clear is only invoked before entering the C-State to prevent |
| 193 | 193 | that stale data from the idling CPU from spilling to the Hyper-Thread |
| @@ -52,6 +52,9 @@ properties: · 51 unmodified lines | ||
| 52 | 52 | '#clock-cells': |
| 53 | 53 | const: 1 |
| 54 | 54 | |
| 55 | + '#reset-cells': | |
| 56 | + const: 1 | |
| 57 | + | |
| 55 | 58 | required: |
| 56 | 59 | - compatible |
| 57 | 60 | - reg |
| @@ -23,7 +23,7 @@ properties: · 22 unmodified lines | ||
| 23 | 23 | - allwinner,sun20i-d1-emac |
| 24 | 24 | - allwinner,sun50i-h6-emac |
| 25 | 25 | - allwinner,sun50i-h616-emac0 |
| 26 | − - allwinner,sun55i-a523-emac0 | |
| 26 | + - allwinner,sun55i-a523-gmac0 | |
| 27 | 27 | - const: allwinner,sun50i-a64-emac |
| 28 | 28 | |
| 29 | 29 | reg: |
| @@ -7196,6 +7196,10 @@ The valid value for 'flags' is: · 7195 unmodified lines | ||
| 7196 | 7196 | u64 leaf; |
| 7197 | 7197 | u64 r11, r12, r13, r14; |
| 7198 | 7198 | } get_tdvmcall_info; |
| 7199 | + struct { | |
| 7200 | + u64 ret; | |
| 7201 | + u64 vector; | |
| 7202 | + } setup_event_notify; | |
| 7199 | 7203 | }; |
| 7200 | 7204 | } tdx; |
| 7201 | 7205 | |
| @@ -7210,21 +7214,24 @@ number from register R11. The remaining field of the union provide the · 8 unmodified lines | ||
| 7210 | 7214 | inputs and outputs of the TDVMCALL. Currently the following values of |
| 7211 | 7215 | ``nr`` are defined: |
| 7212 | 7216 | |
| 7213 | −* ``TDVMCALL_GET_QUOTE``: the guest has requested to generate a TD-Quote | |
| 7214 | −signed by a service hosting TD-Quoting Enclave operating on the host. | |
| 7215 | −Parameters and return value are in the ``get_quote`` field of the union. | |
| 7216 | −The ``gpa`` field and ``size`` specify the guest physical address | |
| 7217 | −(without the shared bit set) and the size of a shared-memory buffer, in | |
| 7218 | −which the TDX guest passes a TD Report. The ``ret`` field represents | |
| 7219 | −the return value of the GetQuote request. When the request has been | |
| 7220 | −queued successfully, the TDX guest can poll the status field in the | |
| 7221 | −shared-memory area to check whether the Quote generation is completed or | |
| 7222 | −not. When completed, the generated Quote is returned via the same buffer. | |
| 7223 | − | |
| 7224 | −* ``TDVMCALL_GET_TD_VM_CALL_INFO``: the guest has requested the support | |
| 7225 | −status of TDVMCALLs. The output values for the given leaf should be | |
| 7226 | −placed in fields from ``r11`` to ``r14`` of the ``get_tdvmcall_info`` | |
| 7227 | −field of the union. | |
| 7217 | + * ``TDVMCALL_GET_QUOTE``: the guest has requested to generate a TD-Quote | |
| 7218 | + signed by a service hosting TD-Quoting Enclave operating on the host. | |
| 7219 | + Parameters and return value are in the ``get_quote`` field of the union. | |
| 7220 | + The ``gpa`` field and ``size`` specify the guest physical address | |
| 7221 | + (without the shared bit set) and the size of a shared-memory buffer, in | |
| 7222 | + which the TDX guest passes a TD Report. The ``ret`` field represents | |
| 7223 | + the return value of the GetQuote request. When the request has been | |
| 7224 | + queued successfully, the TDX guest can poll the status field in the | |
| 7225 | + shared-memory area to check whether the Quote generation is completed or | |
| 7226 | + not. When completed, the generated Quote is returned via the same buffer. | |
| 7227 | + | |
| 7228 | + * ``TDVMCALL_GET_TD_VM_CALL_INFO``: the guest has requested the support | |
| 7229 | + status of TDVMCALLs. The output values for the given leaf should be | |
| 7230 | + placed in fields from ``r11`` to ``r14`` of the ``get_tdvmcall_info`` | |
| 7231 | + field of the union. | |
| 7232 | + | |
| 7233 | +* ``TDVMCALL_SETUP_EVENT_NOTIFY_INTERRUPT``: the guest has requested to | |
| 7234 | +set up a notification interrupt for vector ``vector``. | |
| 7228 | 7235 | |
| 7229 | 7236 | KVM may add support for more values in the future that may cause a userspace |
| 7230 | 7237 | exit, even without calls to ``KVM_ENABLE_CAP`` or similar. In this case, |
| @@ -79,7 +79,20 @@ to be configured to the TDX guest. · 78 unmodified lines | ||
| 79 | 79 | struct kvm_tdx_capabilities { |
| 80 | 80 | __u64 supported_attrs; |
| 81 | 81 | __u64 supported_xfam; |
| 82 | − __u64 reserved[254]; | |
| 82 | + | |
| 83 | + /* TDG.VP.VMCALL hypercalls executed in kernel and forwarded to | |
| 84 | + * userspace, respectively | |
| 85 | + */ | |
| 86 | + __u64 kernel_tdvmcallinfo_1_r11; | |
| 87 | + __u64 user_tdvmcallinfo_1_r11; | |
| 88 | + | |
| 89 | + /* TDG.VP.VMCALL instruction executions subfunctions executed in kernel | |
| 90 | + * and forwarded to userspace, respectively | |
| 91 | + */ | |
| 92 | + __u64 kernel_tdvmcallinfo_1_r12; | |
| 93 | + __u64 user_tdvmcallinfo_1_r12; | |
| 94 | + | |
| 95 | + __u64 reserved[250]; | |
| 83 | 96 | |
| 84 | 97 | /* Configurable CPUID bits for userspace */ |
| 85 | 98 | struct kvm_cpuid2 cpuid; |
| @@ -4181,6 +4181,7 @@ F: include/linux/cpumask_types.h · 4180 unmodified lines | ||
| 4181 | 4181 | F: include/linux/find.h |
| 4182 | 4182 | F: include/linux/nodemask.h |
| 4183 | 4183 | F: include/linux/nodemask_types.h |
| 4184 | +F: include/uapi/linux/bits.h | |
| 4184 | 4185 | F: include/vdso/bits.h |
| 4185 | 4186 | F: lib/bitmap-str.c |
| 4186 | 4187 | F: lib/bitmap.c |
| @@ -4193,6 +4194,7 @@ F: tools/include/linux/bitfield.h · 6 unmodified lines | ||
| 4193 | 4194 | F: tools/include/linux/bitmap.h |
| 4194 | 4195 | F: tools/include/linux/bits.h |
| 4195 | 4196 | F: tools/include/linux/find.h |
| 4197 | +F: tools/include/uapi/linux/bits.h | |
| 4196 | 4198 | F: tools/include/vdso/bits.h |
| 4197 | 4199 | F: tools/lib/bitmap.c |
| 4198 | 4200 | F: tools/lib/find_bit.c |
| @@ -10504,7 +10506,7 @@ S: Maintained · 6305 unmodified lines | ||
| 10504 | 10506 | F: block/partitions/efi.* |
| 10505 | 10507 | |
| 10506 | 10508 | HABANALABS PCI DRIVER |
| 10507 | −M: Ofir Bitton <obitton@habana.ai> | |
| 10509 | +M: Yaron Avizrat <yaron.avizrat@intel.com> | |
| 10508 | 10510 | L: dri-devel@lists.freedesktop.org |
| 10509 | 10511 | S: Supported |
| 10510 | 10512 | C: irc://irc.oftc.net/dri-devel |
| @@ -16822,8 +16824,8 @@ F: include/dt-bindings/clock/mobileye,eyeq5-clk.h · 6311 unmodified lines | ||
| 16822 | 16824 | MODULE SUPPORT |
| 16823 | 16825 | M: Luis Chamberlain <mcgrof@kernel.org> |
| 16824 | 16826 | M: Petr Pavlu <petr.pavlu@suse.com> |
| 16827 | +M: Daniel Gomez <da.gomez@kernel.org> | |
| 16825 | 16828 | R: Sami Tolvanen <samitolvanen@google.com> |
| 16826 | −R: Daniel Gomez <da.gomez@samsung.com> | |
| 16827 | 16829 | L: linux-modules@vger.kernel.org |
| 16828 | 16830 | L: linux-kernel@vger.kernel.org |
| 16829 | 16831 | S: Maintained |
| @@ -17222,10 +17224,10 @@ F: drivers/rtc/rtc-ntxec.c · 392 unmodified lines | ||
| 17222 | 17224 | F: include/linux/mfd/ntxec.h |
| 17223 | 17225 | |
| 17224 | 17226 | NETRONOME ETHERNET DRIVERS |
| 17225 | −M: Louis Peens <louis.peens@corigine.com> | |
| 17226 | 17227 | R: Jakub Kicinski <kuba@kernel.org> |
| 17228 | +R: Simon Horman <horms@kernel.org> | |
| 17227 | 17229 | L: oss-drivers@corigine.com |
| 17228 | −S: Maintained | |
| 17230 | +S: Odd Fixes | |
| 17229 | 17231 | F: drivers/net/ethernet/netronome/ |
| 17230 | 17232 | |
| 17231 | 17233 | NETWORK BLOCK DEVICE (NBD) |
| @@ -19601,8 +19603,7 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/intel.git · 2369 unmodified lines | ||
| 19601 | 19603 | F: drivers/pinctrl/intel/ |
| 19602 | 19604 | |
| 19603 | 19605 | PIN CONTROLLER - KEEMBAY |
| 19604 | −M: Lakshmi Sowjanya D <lakshmi.sowjanya.d@intel.com> | |
| 19605 | −S: Supported | |
| 19606 | +S: Orphan | |
| 19606 | 19607 | F: drivers/pinctrl/pinctrl-keembay* |
| 19607 | 19608 | |
| 19608 | 19609 | PIN CONTROLLER - MEDIATEK |
| @@ -20155,21 +20156,15 @@ S: Supported · 546 unmodified lines | ||
| 20155 | 20156 | F: Documentation/devicetree/bindings/soc/qcom/qcom,apr* |
| 20156 | 20157 | F: Documentation/devicetree/bindings/sound/qcom,* |
| 20157 | 20158 | F: drivers/soc/qcom/apr.c |
| 20158 | −F: include/dt-bindings/sound/qcom,wcd9335.h | |
| 20159 | −F: include/dt-bindings/sound/qcom,wcd934x.h | |
| 20160 | −F: sound/soc/codecs/lpass-rx-macro.* | |
| 20161 | −F: sound/soc/codecs/lpass-tx-macro.* | |
| 20162 | −F: sound/soc/codecs/lpass-va-macro.c | |
| 20163 | −F: sound/soc/codecs/lpass-wsa-macro.* | |
| 20159 | +F: drivers/soundwire/qcom.c | |
| 20160 | +F: include/dt-bindings/sound/qcom,wcd93* | |
| 20161 | +F: sound/soc/codecs/lpass-*.* | |
| 20164 | 20162 | F: sound/soc/codecs/msm8916-wcd-analog.c |
| 20165 | 20163 | F: sound/soc/codecs/msm8916-wcd-digital.c |
| 20166 | 20164 | F: sound/soc/codecs/wcd-clsh-v2.* |
| 20167 | 20165 | F: sound/soc/codecs/wcd-mbhc-v2.* |
| 20168 | −F: sound/soc/codecs/wcd9335.* | |
| 20169 | −F: sound/soc/codecs/wcd934x.c | |
| 20170 | −F: sound/soc/codecs/wsa881x.c | |
| 20171 | −F: sound/soc/codecs/wsa883x.c | |
| 20172 | −F: sound/soc/codecs/wsa884x.c | |
| 20166 | +F: sound/soc/codecs/wcd93*.* | |
| 20167 | +F: sound/soc/codecs/wsa88*.* | |
| 20173 | 20168 | F: sound/soc/qcom/ |
| 20174 | 20169 | |
| 20175 | 20170 | QCOM EMBEDDED USB DEBUGGER (EUD) |
| @@ -26955,7 +26950,7 @@ F: arch/x86/kernel/stacktrace.c · 6779 unmodified lines | ||
| 26955 | 26950 | F: arch/x86/kernel/unwind_*.c |
| 26956 | 26951 | |
| 26957 | 26952 | X86 TRUST DOMAIN EXTENSIONS (TDX) |
| 26958 | −M: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> | |
| 26953 | +M: Kirill A. Shutemov <kas@kernel.org> | |
| 26959 | 26954 | R: Dave Hansen <dave.hansen@linux.intel.com> |
| 26960 | 26955 | L: x86@kernel.org |
| 26961 | 26956 | L: linux-coco@lists.linux.dev |
| @@ -27324,13 +27319,6 @@ S: Supported · 362 unmodified lines | ||
| 27324 | 27319 | W: http://www.marvell.com |
| 27325 | 27320 | F: drivers/i2c/busses/i2c-xlp9xx.c |
| 27326 | 27321 | |
| 27327 | −XRA1403 GPIO EXPANDER | |
| 27328 | −M: Nandor Han <nandor.han@ge.com> | |
| 27329 | −L: linux-gpio@vger.kernel.org | |
| 27330 | −S: Maintained | |
| 27331 | −F: Documentation/devicetree/bindings/gpio/gpio-xra1403.txt | |
| 27332 | −F: drivers/gpio/gpio-xra1403.c | |
| 27333 | − | |
| 27334 | 27322 | XTENSA XTFPGA PLATFORM SUPPORT |
| 27335 | 27323 | M: Max Filippov <jcmvbkbc@gmail.com> |
| 27336 | 27324 | S: Maintained |
| @@ -2,7 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | VERSION = 6 |
| 3 | 3 | PATCHLEVEL = 16 |
| 4 | 4 | SUBLEVEL = 0 |
| 5 | −EXTRAVERSION = -rc5 | |
| 5 | +EXTRAVERSION = -rc6 | |
| 6 | 6 | NAME = Baby Opossum Posse |
| 7 | 7 | |
| 8 | 8 | # *DOCUMENTATION* |
| @@ -256,6 +256,7 @@ config ARM64 · 255 unmodified lines | ||
| 256 | 256 | select HOTPLUG_SMT if HOTPLUG_CPU |
| 257 | 257 | select IRQ_DOMAIN |
| 258 | 258 | select IRQ_FORCED_THREADING |
| 259 | + select JUMP_LABEL | |
| 259 | 260 | select KASAN_VMALLOC if KASAN |
| 260 | 261 | select LOCK_MM_AND_FIND_VMA |
| 261 | 262 | select MODULES_USE_ELF_RELA |
| @@ -287,17 +287,6 @@ · 286 unmodified lines | ||
| 287 | 287 | .Lskip_fgt2_\@: |
| 288 | 288 | .endm |
| 289 | 289 | |
| 290 | −.macro __init_el2_gcs | |
| 291 | − mrs_s x1, SYS_ID_AA64PFR1_EL1 | |
| 292 | − ubfx x1, x1, #ID_AA64PFR1_EL1_GCS_SHIFT, #4 | |
| 293 | − cbz x1, .Lskip_gcs_\@ | |
| 294 | − | |
| 295 | − /* Ensure GCS is not enabled when we start trying to do BLs */ | |
| 296 | − msr_s SYS_GCSCR_EL1, xzr | |
| 297 | − msr_s SYS_GCSCRE0_EL1, xzr | |
| 298 | −.Lskip_gcs_\@: | |
| 299 | −.endm | |
| 300 | − | |
| 301 | 290 | /** |
| 302 | 291 | * Initialize EL2 registers to sane values. This should be called early on all |
| 303 | 292 | * cores that were booted in EL2. Note that everything gets initialised as |
| @@ -319,7 +308,6 @@ · 15 unmodified lines | ||
| 319 | 308 | __init_el2_cptr |
| 320 | 309 | __init_el2_fgt |
| 321 | 310 | __init_el2_fgt2 |
| 322 | − __init_el2_gcs | |
| 323 | 311 | .endm |
| 324 | 312 | |
| 325 | 313 | #ifndef __KVM_NVHE_HYPERVISOR__ |
| @@ -371,6 +359,13 @@ · 45 unmodified lines | ||
| 371 | 359 | msr_s SYS_MPAMHCR_EL2, xzr // clear TRAP_MPAMIDR_EL1 -> EL2 |
| 372 | 360 | |
| 373 | 361 | .Lskip_mpam_\@: |
| 362 | + check_override id_aa64pfr1, ID_AA64PFR1_EL1_GCS_SHIFT, .Linit_gcs_\@, .Lskip_gcs_\@, x1, x2 | |
| 363 | + | |
| 364 | +.Linit_gcs_\@: | |
| 365 | + msr_s SYS_GCSCR_EL1, xzr | |
| 366 | + msr_s SYS_GCSCRE0_EL1, xzr | |
| 367 | + | |
| 368 | +.Lskip_gcs_\@: | |
| 374 | 369 | check_override id_aa64pfr0, ID_AA64PFR0_EL1_SVE_SHIFT, .Linit_sve_\@, .Lskip_sve_\@, x1, x2 |
| 375 | 370 | |
| 376 | 371 | .Linit_sve_\@: /* SVE register access */ |
| @@ -1480,7 +1480,6 @@ int kvm_vm_ioctl_get_reg_writable_masks(struct kvm *kvm, · 1479 unmodified lines | ||
| 1480 | 1480 | struct reg_mask_range *range); |
| 1481 | 1481 | |
| 1482 | 1482 | /* Guest/host FPSIMD coordination helpers */ |
| 1483 | −int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu); | |
| 1484 | 1483 | void kvm_arch_vcpu_load_fp(struct kvm_vcpu *vcpu); |
| 1485 | 1484 | void kvm_arch_vcpu_ctxflush_fp(struct kvm_vcpu *vcpu); |
| 1486 | 1485 | void kvm_arch_vcpu_ctxsync_fp(struct kvm_vcpu *vcpu); |
| @@ -34,7 +34,7 @@ obj-y := debug-monitors.o entry.o irq.o fpsimd.o \ · 33 unmodified lines | ||
| 34 | 34 | cpufeature.o alternative.o cacheinfo.o \ |
| 35 | 35 | smp.o smp_spin_table.o topology.o smccc-call.o \ |
| 36 | 36 | syscall.o proton-pack.o idle.o patching.o pi/ \ |
| 37 | − rsi.o | |
| 37 | + rsi.o jump_label.o | |
| 38 | 38 | |
| 39 | 39 | obj-$(CONFIG_COMPAT) += sys32.o signal32.o \ |
| 40 | 40 | sys_compat.o |
| @@ -47,7 +47,6 @@ obj-$(CONFIG_PERF_EVENTS) += perf_regs.o perf_callchain.o · 6 unmodified lines | ||
| 47 | 47 | obj-$(CONFIG_HARDLOCKUP_DETECTOR_PERF) += watchdog_hld.o |
| 48 | 48 | obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o |
| 49 | 49 | obj-$(CONFIG_CPU_PM) += sleep.o suspend.o |
| 50 | −obj-$(CONFIG_JUMP_LABEL) += jump_label.o | |
| 51 | 50 | obj-$(CONFIG_KGDB) += kgdb.o |
| 52 | 51 | obj-$(CONFIG_EFI) += efi.o efi-rt-wrapper.o |
| 53 | 52 | obj-$(CONFIG_PCI) += pci.o |
| @@ -3135,6 +3135,13 @@ static bool has_sve_feature(const struct arm64_cpu_capabilities *cap, int scope) · 3134 unmodified lines | ||
| 3135 | 3135 | } |
| 3136 | 3136 | #endif |
| 3137 | 3137 | |
| 3138 | +#ifdef CONFIG_ARM64_SME | |
| 3139 | +static bool has_sme_feature(const struct arm64_cpu_capabilities *cap, int scope) | |
| 3140 | +{ | |
| 3141 | + return system_supports_sme() && has_user_cpuid_feature(cap, scope); | |
| 3142 | +} | |
| 3143 | +#endif | |
| 3144 | + | |
| 3138 | 3145 | static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = { |
| 3139 | 3146 | HWCAP_CAP(ID_AA64ISAR0_EL1, AES, PMULL, CAP_HWCAP, KERNEL_HWCAP_PMULL), |
| 3140 | 3147 | HWCAP_CAP(ID_AA64ISAR0_EL1, AES, AES, CAP_HWCAP, KERNEL_HWCAP_AES), |
| @@ -3223,31 +3230,31 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = { · 82 unmodified lines | ||
| 3223 | 3230 | HWCAP_CAP(ID_AA64ISAR2_EL1, BC, IMP, CAP_HWCAP, KERNEL_HWCAP_HBC), |
| 3224 | 3231 | #ifdef CONFIG_ARM64_SME |
| 3225 | 3232 | HWCAP_CAP(ID_AA64PFR1_EL1, SME, IMP, CAP_HWCAP, KERNEL_HWCAP_SME), |
| 3226 | − HWCAP_CAP(ID_AA64SMFR0_EL1, FA64, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_FA64), | |
| 3227 | − HWCAP_CAP(ID_AA64SMFR0_EL1, LUTv2, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_LUTV2), | |
| 3228 | − HWCAP_CAP(ID_AA64SMFR0_EL1, SMEver, SME2p2, CAP_HWCAP, KERNEL_HWCAP_SME2P2), | |
| 3229 | − HWCAP_CAP(ID_AA64SMFR0_EL1, SMEver, SME2p1, CAP_HWCAP, KERNEL_HWCAP_SME2P1), | |
| 3230 | − HWCAP_CAP(ID_AA64SMFR0_EL1, SMEver, SME2, CAP_HWCAP, KERNEL_HWCAP_SME2), | |
| 3231 | − HWCAP_CAP(ID_AA64SMFR0_EL1, I16I64, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_I16I64), | |
| 3232 | − HWCAP_CAP(ID_AA64SMFR0_EL1, F64F64, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F64F64), | |
| 3233 | − HWCAP_CAP(ID_AA64SMFR0_EL1, I16I32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_I16I32), | |
| 3234 | − HWCAP_CAP(ID_AA64SMFR0_EL1, B16B16, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_B16B16), | |
| 3235 | − HWCAP_CAP(ID_AA64SMFR0_EL1, F16F16, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F16F16), | |
| 3236 | − HWCAP_CAP(ID_AA64SMFR0_EL1, F8F16, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F8F16), | |
| 3237 | − HWCAP_CAP(ID_AA64SMFR0_EL1, F8F32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F8F32), | |
| 3238 | − HWCAP_CAP(ID_AA64SMFR0_EL1, I8I32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_I8I32), | |
| 3239 | − HWCAP_CAP(ID_AA64SMFR0_EL1, F16F32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F16F32), | |
| 3240 | − HWCAP_CAP(ID_AA64SMFR0_EL1, B16F32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_B16F32), | |
| 3241 | − HWCAP_CAP(ID_AA64SMFR0_EL1, BI32I32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_BI32I32), | |
| 3242 | − HWCAP_CAP(ID_AA64SMFR0_EL1, F32F32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F32F32), | |
| 3243 | − HWCAP_CAP(ID_AA64SMFR0_EL1, SF8FMA, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SF8FMA), | |
| 3244 | − HWCAP_CAP(ID_AA64SMFR0_EL1, SF8DP4, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SF8DP4), | |
| 3245 | − HWCAP_CAP(ID_AA64SMFR0_EL1, SF8DP2, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SF8DP2), | |
| 3246 | − HWCAP_CAP(ID_AA64SMFR0_EL1, SBitPerm, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SBITPERM), | |
| 3247 | − HWCAP_CAP(ID_AA64SMFR0_EL1, AES, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_AES), | |
| 3248 | − HWCAP_CAP(ID_AA64SMFR0_EL1, SFEXPA, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SFEXPA), | |
| 3249 | − HWCAP_CAP(ID_AA64SMFR0_EL1, STMOP, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_STMOP), | |
| 3250 | − HWCAP_CAP(ID_AA64SMFR0_EL1, SMOP4, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SMOP4), | |
| 3233 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, FA64, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_FA64), | |
| 3234 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, LUTv2, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_LUTV2), | |
| 3235 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, SMEver, SME2p2, CAP_HWCAP, KERNEL_HWCAP_SME2P2), | |
| 3236 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, SMEver, SME2p1, CAP_HWCAP, KERNEL_HWCAP_SME2P1), | |
| 3237 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, SMEver, SME2, CAP_HWCAP, KERNEL_HWCAP_SME2), | |
| 3238 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, I16I64, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_I16I64), | |
| 3239 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, F64F64, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F64F64), | |
| 3240 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, I16I32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_I16I32), | |
| 3241 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, B16B16, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_B16B16), | |
| 3242 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, F16F16, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F16F16), | |
| 3243 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, F8F16, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F8F16), | |
| 3244 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, F8F32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F8F32), | |
| 3245 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, I8I32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_I8I32), | |
| 3246 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, F16F32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F16F32), | |
| 3247 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, B16F32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_B16F32), | |
| 3248 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, BI32I32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_BI32I32), | |
| 3249 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, F32F32, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_F32F32), | |
| 3250 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, SF8FMA, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SF8FMA), | |
| 3251 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, SF8DP4, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SF8DP4), | |
| 3252 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, SF8DP2, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SF8DP2), | |
| 3253 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, SBitPerm, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SBITPERM), | |
| 3254 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, AES, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_AES), | |
| 3255 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, SFEXPA, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SFEXPA), | |
| 3256 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, STMOP, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_STMOP), | |
| 3257 | + HWCAP_CAP_MATCH_ID(has_sme_feature, ID_AA64SMFR0_EL1, SMOP4, IMP, CAP_HWCAP, KERNEL_HWCAP_SME_SMOP4), | |
| 3251 | 3258 | #endif /* CONFIG_ARM64_SME */ |
| 3252 | 3259 | HWCAP_CAP(ID_AA64FPFR0_EL1, F8CVT, IMP, CAP_HWCAP, KERNEL_HWCAP_F8CVT), |
| 3253 | 3260 | HWCAP_CAP(ID_AA64FPFR0_EL1, F8FMA, IMP, CAP_HWCAP, KERNEL_HWCAP_F8FMA), |
| @@ -15,6 +15,7 @@ · 14 unmodified lines | ||
| 15 | 15 | |
| 16 | 16 | #include <asm/efi.h> |
| 17 | 17 | #include <asm/stacktrace.h> |
| 18 | +#include <asm/vmap_stack.h> | |
| 18 | 19 | |
| 19 | 20 | static bool region_is_misaligned(const efi_memory_desc_t *md) |
| 20 | 21 | { |
| @@ -214,9 +215,13 @@ static int __init arm64_efi_rt_init(void) · 193 unmodified lines | ||
| 214 | 215 | if (!efi_enabled(EFI_RUNTIME_SERVICES)) |
| 215 | 216 | return 0; |
| 216 | 217 | |
| 217 | − p = __vmalloc_node(THREAD_SIZE, THREAD_ALIGN, GFP_KERNEL, | |
| 218 | − NUMA_NO_NODE, &&l); | |
| 219 | −l: if (!p) { | |
| 218 | + if (!IS_ENABLED(CONFIG_VMAP_STACK)) { | |
| 219 | + clear_bit(EFI_RUNTIME_SERVICES, &efi.flags); | |
| 220 | + return -ENOMEM; | |
| 221 | + } | |
| 222 | + | |
| 223 | + p = arch_alloc_vmap_stack(THREAD_SIZE, NUMA_NO_NODE); | |
| 224 | + if (!p) { | |
| 220 | 225 | pr_warn("Failed to allocate EFI runtime stack\n"); |
| 221 | 226 | clear_bit(EFI_RUNTIME_SERVICES, &efi.flags); |
| 222 | 227 | return -ENOMEM; |
| @@ -673,6 +673,11 @@ static void permission_overlay_switch(struct task_struct *next) · 672 unmodified lines | ||
| 673 | 673 | current->thread.por_el0 = read_sysreg_s(SYS_POR_EL0); |
| 674 | 674 | if (current->thread.por_el0 != next->thread.por_el0) { |
| 675 | 675 | write_sysreg_s(next->thread.por_el0, SYS_POR_EL0); |
| 676 | + /* | |
| 677 | + * No ISB required as we can tolerate spurious Overlay faults - | |
| 678 | + * the fault handler will check again based on the new value | |
| 679 | + * of POR_EL0. | |
| 680 | + */ | |
| 676 | 681 | } |
| 677 | 682 | } |
| 678 | 683 | |
| @@ -1143,7 +1143,7 @@ static inline unsigned int num_other_online_cpus(void) · 1142 unmodified lines | ||
| 1143 | 1143 | void smp_send_stop(void) |
| 1144 | 1144 | { |
| 1145 | 1145 | static unsigned long stop_in_progress; |
| 1146 | − cpumask_t mask; | |
| 1146 | + static cpumask_t mask; | |
| 1147 | 1147 | unsigned long timeout; |
| 1148 | 1148 | |
| 1149 | 1149 | /* |
| @@ -825,10 +825,6 @@ int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu) · 824 unmodified lines | ||
| 825 | 825 | if (!kvm_arm_vcpu_is_finalized(vcpu)) |
| 826 | 826 | return -EPERM; |
| 827 | 827 | |
| 828 | − ret = kvm_arch_vcpu_run_map_fp(vcpu); | |
| 829 | − if (ret) | |
| 830 | − return ret; | |
| 831 | − | |
| 832 | 828 | if (likely(vcpu_has_run_once(vcpu))) |
| 833 | 829 | return 0; |
| 834 | 830 | |
| @@ -2129,7 +2125,7 @@ static void cpu_hyp_init(void *discard) · 1294 unmodified lines | ||
| 2129 | 2125 | |
| 2130 | 2126 | static void cpu_hyp_uninit(void *discard) |
| 2131 | 2127 | { |
| 2132 | − if (__this_cpu_read(kvm_hyp_initialized)) { | |
| 2128 | + if (!is_protected_kvm_enabled() && __this_cpu_read(kvm_hyp_initialized)) { | |
| 2133 | 2129 | cpu_hyp_reset(); |
| 2134 | 2130 | __this_cpu_write(kvm_hyp_initialized, 0); |
| 2135 | 2131 | } |
| @@ -2345,8 +2341,13 @@ static void __init teardown_hyp_mode(void) · 209 unmodified lines | ||
| 2345 | 2341 | |
| 2346 | 2342 | free_hyp_pgds(); |
| 2347 | 2343 | for_each_possible_cpu(cpu) { |
| 2344 | + if (per_cpu(kvm_hyp_initialized, cpu)) | |
| 2345 | + continue; | |
| 2346 | + | |
| 2348 | 2347 | free_pages(per_cpu(kvm_arm_hyp_stack_base, cpu), NVHE_STACK_SHIFT - PAGE_SHIFT); |
| 2349 | − free_pages(kvm_nvhe_sym(kvm_arm_hyp_percpu_base)[cpu], nvhe_percpu_order()); | |
| 2348 | + | |
| 2349 | + if (!kvm_nvhe_sym(kvm_arm_hyp_percpu_base)[cpu]) | |
| 2350 | + continue; | |
| 2350 | 2351 | |
| 2351 | 2352 | if (free_sve) { |
| 2352 | 2353 | struct cpu_sve_state *sve_state; |
| @@ -2354,6 +2355,9 @@ static void __init teardown_hyp_mode(void) · 1 unmodified line | ||
| 2354 | 2355 | sve_state = per_cpu_ptr_nvhe_sym(kvm_host_data, cpu)->sve_state; |
| 2355 | 2356 | free_pages((unsigned long) sve_state, pkvm_host_sve_state_order()); |
| 2356 | 2357 | } |
| 2358 | + | |
| 2359 | + free_pages(kvm_nvhe_sym(kvm_arm_hyp_percpu_base)[cpu], nvhe_percpu_order()); | |
| 2360 | + | |
| 2357 | 2361 | } |
| 2358 | 2362 | } |
| 2359 | 2363 | |
| @@ -14,32 +14,6 @@ · 13 unmodified lines | ||
| 14 | 14 | #include <asm/kvm_mmu.h> |
| 15 | 15 | #include <asm/sysreg.h> |
| 16 | 16 | |
| 17 | −/* | |
| 18 | − * Called on entry to KVM_RUN unless this vcpu previously ran at least | |
| 19 | − * once and the most recent prior KVM_RUN for this vcpu was called from | |
| 20 | − * the same task as current (highly likely). | |
| 21 | − * | |
| 22 | − * This is guaranteed to execute before kvm_arch_vcpu_load_fp(vcpu), | |
| 23 | − * such that on entering hyp the relevant parts of current are already | |
| 24 | − * mapped. | |
| 25 | − */ | |
| 26 | −int kvm_arch_vcpu_run_map_fp(struct kvm_vcpu *vcpu) | |
| 27 | −{ | |
| 28 | − struct user_fpsimd_state *fpsimd = ¤t->thread.uw.fpsimd_state; | |
| 29 | − int ret; | |
| 30 | − | |
| 31 | − /* pKVM has its own tracking of the host fpsimd state. */ | |
| 32 | − if (is_protected_kvm_enabled()) | |
| 33 | − return 0; | |
| 34 | − | |
| 35 | − /* Make sure the host task fpsimd state is visible to hyp: */ | |
| 36 | − ret = kvm_share_hyp(fpsimd, fpsimd + 1); | |
| 37 | − if (ret) | |
| 38 | − return ret; | |
| 39 | − | |
| 40 | − return 0; | |
| 41 | −} | |
| 42 | − | |
| 43 | 17 | /* |
| 44 | 18 | * Prepare vcpu for saving the host's FPSIMD state and loading the guest's. |
| 45 | 19 | * The actual loading is done by the FPSIMD access trap taken to hyp. |
| @@ -479,6 +479,7 @@ static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) · 478 unmodified lines | ||
| 479 | 479 | { |
| 480 | 480 | struct kvm_mem_range cur; |
| 481 | 481 | kvm_pte_t pte; |
| 482 | + u64 granule; | |
| 482 | 483 | s8 level; |
| 483 | 484 | int ret; |
| 484 | 485 | |
| @@ -496,18 +497,21 @@ static int host_stage2_adjust_range(u64 addr, struct kvm_mem_range *range) · 11 unmodified lines | ||
| 496 | 497 | return -EPERM; |
| 497 | 498 | } |
| 498 | 499 | |
| 499 | − do { | |
| 500 | − u64 granule = kvm_granule_size(level); | |
| 500 | + for (; level <= KVM_PGTABLE_LAST_LEVEL; level++) { | |
| 501 | + if (!kvm_level_supports_block_mapping(level)) | |
| 502 | + continue; | |
| 503 | + granule = kvm_granule_size(level); | |
| 501 | 504 | cur.start = ALIGN_DOWN(addr, granule); |
| 502 | 505 | cur.end = cur.start + granule; |
| 503 | − level++; | |
| 504 | − } while ((level <= KVM_PGTABLE_LAST_LEVEL) && | |
| 505 | − !(kvm_level_supports_block_mapping(level) && | |
| 506 | − range_included(&cur, range))); | |
| 506 | + if (!range_included(&cur, range)) | |
| 507 | + continue; | |
| 508 | + *range = cur; | |
| 509 | + return 0; | |
| 510 | + } | |
| 507 | 511 | |
| 508 | − *range = cur; | |
| 512 | + WARN_ON(1); | |
| 509 | 513 | |
| 510 | − return 0; | |
| 514 | + return -EINVAL; | |
| 511 | 515 | } |
| 512 | 516 | |
| 513 | 517 | int host_stage2_idmap_locked(phys_addr_t addr, u64 size, |
| @@ -1402,6 +1402,21 @@ static void kvm_map_l1_vncr(struct kvm_vcpu *vcpu) · 1401 unmodified lines | ||
| 1402 | 1402 | } |
| 1403 | 1403 | } |
| 1404 | 1404 | |
| 1405 | +#define has_tgran_2(__r, __sz) \ | |
| 1406 | + ({ \ | |
| 1407 | + u64 _s1, _s2, _mmfr0 = __r; \ | |
| 1408 | + \ | |
| 1409 | + _s2 = SYS_FIELD_GET(ID_AA64MMFR0_EL1, \ | |
| 1410 | + TGRAN##__sz##_2, _mmfr0); \ | |
| 1411 | + \ | |
| 1412 | + _s1 = SYS_FIELD_GET(ID_AA64MMFR0_EL1, \ | |
| 1413 | + TGRAN##__sz, _mmfr0); \ | |
| 1414 | + \ | |
| 1415 | + ((_s2 != ID_AA64MMFR0_EL1_TGRAN##__sz##_2_NI && \ | |
| 1416 | + _s2 != ID_AA64MMFR0_EL1_TGRAN##__sz##_2_TGRAN##__sz) || \ | |
| 1417 | + (_s2 == ID_AA64MMFR0_EL1_TGRAN##__sz##_2_TGRAN##__sz && \ | |
| 1418 | + _s1 != ID_AA64MMFR0_EL1_TGRAN##__sz##_NI)); \ | |
| 1419 | + }) | |
| 1405 | 1420 | /* |
| 1406 | 1421 | * Our emulated CPU doesn't support all the possible features. For the |
| 1407 | 1422 | * sake of simplicity (and probably mental sanity), wipe out a number |
| @@ -1411,6 +1426,8 @@ static void kvm_map_l1_vncr(struct kvm_vcpu *vcpu) · 3 unmodified lines | ||
| 1411 | 1426 | */ |
| 1412 | 1427 | u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) |
| 1413 | 1428 | { |
| 1429 | + u64 orig_val = val; | |
| 1430 | + | |
| 1414 | 1431 | switch (reg) { |
| 1415 | 1432 | case SYS_ID_AA64ISAR0_EL1: |
| 1416 | 1433 | /* Support everything but TME */ |
| @@ -1480,13 +1497,16 @@ u64 limit_nv_id_reg(struct kvm *kvm, u32 reg, u64 val) · 63 unmodified lines | ||
| 1480 | 1497 | */ |
| 1481 | 1498 | switch (PAGE_SIZE) { |
| 1482 | 1499 | case SZ_4K: |
| 1483 | − val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN4_2, IMP); | |
| 1500 | + if (has_tgran_2(orig_val, 4)) | |
| 1501 | + val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN4_2, IMP); | |
| 1484 | 1502 | fallthrough; |
| 1485 | 1503 | case SZ_16K: |
| 1486 | − val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN16_2, IMP); | |
| 1504 | + if (has_tgran_2(orig_val, 16)) | |
| 1505 | + val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN16_2, IMP); | |
| 1487 | 1506 | fallthrough; |
| 1488 | 1507 | case SZ_64K: |
| 1489 | − val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN64_2, IMP); | |
| 1508 | + if (has_tgran_2(orig_val, 64)) | |
| 1509 | + val |= SYS_FIELD_PREP_ENUM(ID_AA64MMFR0_EL1, TGRAN64_2, IMP); | |
| 1490 | 1510 | break; |
| 1491 | 1511 | } |
| 1492 | 1512 | |
| @@ -401,9 +401,7 @@ void vgic_v3_nested_update_mi(struct kvm_vcpu *vcpu) · 400 unmodified lines | ||
| 401 | 401 | { |
| 402 | 402 | bool level; |
| 403 | 403 | |
| 404 | − level = __vcpu_sys_reg(vcpu, ICH_HCR_EL2) & ICH_HCR_EL2_En; | |
| 405 | − if (level) | |
| 406 | − level &= vgic_v3_get_misr(vcpu); | |
| 404 | + level = (__vcpu_sys_reg(vcpu, ICH_HCR_EL2) & ICH_HCR_EL2_En) && vgic_v3_get_misr(vcpu); | |
| 407 | 405 | kvm_vgic_inject_irq(vcpu->kvm, vcpu, |
| 408 | 406 | vcpu->kvm->arch.vgic.mi_intid, level, vcpu); |
| 409 | 407 | } |
| @@ -487,17 +487,29 @@ static void do_bad_area(unsigned long far, unsigned long esr, · 486 unmodified lines | ||
| 487 | 487 | } |
| 488 | 488 | } |
| 489 | 489 | |
| 490 | −static bool fault_from_pkey(unsigned long esr, struct vm_area_struct *vma, | |
| 491 | − unsigned int mm_flags) | |
| 490 | +static bool fault_from_pkey(struct vm_area_struct *vma, unsigned int mm_flags) | |
| 492 | 491 | { |
| 493 | − unsigned long iss2 = ESR_ELx_ISS2(esr); | |
| 494 | − | |
| 495 | 492 | if (!system_supports_poe()) |
| 496 | 493 | return false; |
| 497 | 494 | |
| 498 | − if (esr_fsc_is_permission_fault(esr) && (iss2 & ESR_ELx_Overlay)) | |
| 499 | − return true; | |
| 500 | − | |
| 495 | + /* | |
| 496 | + * We do not check whether an Overlay fault has occurred because we | |
| 497 | + * cannot make a decision based solely on its value: | |
| 498 | + * | |
| 499 | + * - If Overlay is set, a fault did occur due to POE, but it may be | |
| 500 | + * spurious in those cases where we update POR_EL0 without ISB (e.g. | |
| 501 | + * on context-switch). We would then need to manually check POR_EL0 | |
| 502 | + * against vma_pkey(vma), which is exactly what | |
| 503 | + * arch_vma_access_permitted() does. | |
| 504 | + * | |
| 505 | + * - If Overlay is not set, we may still need to report a pkey fault. | |
| 506 | + * This is the case if an access was made within a mapping but with no | |
| 507 | + * page mapped, and POR_EL0 forbids the access (according to | |
| 508 | + * vma_pkey()). Such access will result in a SIGSEGV regardless | |
| 509 | + * because core code checks arch_vma_access_permitted(), but in order | |
| 510 | + * to report the correct error code - SEGV_PKUERR - we must handle | |
| 511 | + * that case here. | |
| 512 | + */ | |
| 501 | 513 | return !arch_vma_access_permitted(vma, |
| 502 | 514 | mm_flags & FAULT_FLAG_WRITE, |
| 503 | 515 | mm_flags & FAULT_FLAG_INSTRUCTION, |
| @@ -635,7 +647,7 @@ static int __kprobes do_page_fault(unsigned long far, unsigned long esr, · 131 unmodified lines | ||
| 635 | 647 | goto bad_area; |
| 636 | 648 | } |
| 637 | 649 | |
| 638 | − if (fault_from_pkey(esr, vma, mm_flags)) { | |
| 650 | + if (fault_from_pkey(vma, mm_flags)) { | |
| 639 | 651 | pkey = vma_pkey(vma); |
| 640 | 652 | vma_end_read(vma); |
| 641 | 653 | fault = 0; |
| @@ -679,7 +691,7 @@ retry: · 37 unmodified lines | ||
| 679 | 691 | goto bad_area; |
| 680 | 692 | } |
| 681 | 693 | |
| 682 | − if (fault_from_pkey(esr, vma, mm_flags)) { | |
| 694 | + if (fault_from_pkey(vma, mm_flags)) { | |
| 683 | 695 | pkey = vma_pkey(vma); |
| 684 | 696 | mmap_read_unlock(mm); |
| 685 | 697 | fault = 0; |
| @@ -518,7 +518,6 @@ alternative_else_nop_endif · 517 unmodified lines | ||
| 518 | 518 | msr REG_PIR_EL1, x0 |
| 519 | 519 | |
| 520 | 520 | orr tcr2, tcr2, TCR2_EL1_PIE |
| 521 | − msr REG_TCR2_EL1, x0 | |
| 522 | 521 | |
| 523 | 522 | .Lskip_indirection: |
| 524 | 523 | |
| @@ -38,6 +38,7 @@ static int s390_sha1_init(struct shash_desc *desc) · 37 unmodified lines | ||
| 38 | 38 | sctx->state[4] = SHA1_H4; |
| 39 | 39 | sctx->count = 0; |
| 40 | 40 | sctx->func = CPACF_KIMD_SHA_1; |
| 41 | + sctx->first_message_part = 0; | |
| 41 | 42 | |
| 42 | 43 | return 0; |
| 43 | 44 | } |
| @@ -60,6 +61,7 @@ static int s390_sha1_import(struct shash_desc *desc, const void *in) · 16 unmodified lines | ||
| 60 | 61 | sctx->count = ictx->count; |
| 61 | 62 | memcpy(sctx->state, ictx->state, sizeof(ictx->state)); |
| 62 | 63 | sctx->func = CPACF_KIMD_SHA_1; |
| 64 | + sctx->first_message_part = 0; | |
| 63 | 65 | return 0; |
| 64 | 66 | } |
| 65 | 67 | |
| @@ -32,6 +32,7 @@ static int sha512_init(struct shash_desc *desc) · 31 unmodified lines | ||
| 32 | 32 | ctx->count = 0; |
| 33 | 33 | ctx->sha512.count_hi = 0; |
| 34 | 34 | ctx->func = CPACF_KIMD_SHA_512; |
| 35 | + ctx->first_message_part = 0; | |
| 35 | 36 | |
| 36 | 37 | return 0; |
| 37 | 38 | } |
| @@ -57,6 +58,7 @@ static int sha512_import(struct shash_desc *desc, const void *in) · 19 unmodified lines | ||
| 57 | 58 | |
| 58 | 59 | memcpy(sctx->state, ictx->state, sizeof(ictx->state)); |
| 59 | 60 | sctx->func = CPACF_KIMD_SHA_512; |
| 61 | + sctx->first_message_part = 0; | |
| 60 | 62 | return 0; |
| 61 | 63 | } |
| 62 | 64 | |
| @@ -97,6 +99,7 @@ static int sha384_init(struct shash_desc *desc) · 34 unmodified lines | ||
| 97 | 99 | ctx->count = 0; |
| 98 | 100 | ctx->sha512.count_hi = 0; |
| 99 | 101 | ctx->func = CPACF_KIMD_SHA_512; |
| 102 | + ctx->first_message_part = 0; | |
| 100 | 103 | |
| 101 | 104 | return 0; |
| 102 | 105 | } |
| @@ -147,7 +147,7 @@ config X86 · 146 unmodified lines | ||
| 147 | 147 | select ARCH_WANTS_DYNAMIC_TASK_STRUCT |
| 148 | 148 | select ARCH_WANTS_NO_INSTR |
| 149 | 149 | select ARCH_WANT_GENERAL_HUGETLB |
| 150 | − select ARCH_WANT_HUGE_PMD_SHARE | |
| 150 | + select ARCH_WANT_HUGE_PMD_SHARE if X86_64 | |
| 151 | 151 | select ARCH_WANT_LD_ORPHAN_WARN |
| 152 | 152 | select ARCH_WANT_OPTIMIZE_DAX_VMEMMAP if X86_64 |
| 153 | 153 | select ARCH_WANT_OPTIMIZE_HUGETLB_VMEMMAP if X86_64 |
| @@ -2695,6 +2695,15 @@ config MITIGATION_ITS · 2541 unmodified lines | ||
| 2695 | 2695 | disabled, mitigation cannot be enabled via cmdline. |
| 2696 | 2696 | See <file:Documentation/admin-guide/hw-vuln/indirect-target-selection.rst> |
| 2697 | 2697 | |
| 2698 | +config MITIGATION_TSA | |
| 2699 | + bool "Mitigate Transient Scheduler Attacks" | |
| 2700 | + depends on CPU_SUP_AMD | |
| 2701 | + default y | |
| 2702 | + help | |
| 2703 | + Enable mitigation for Transient Scheduler Attacks. TSA is a hardware | |
| 2704 | + security vulnerability on AMD CPUs which can lead to forwarding of | |
| 2705 | + invalid info to subsequent instructions and thus can affect their | |
| 2706 | + timing and thereby cause a leakage. | |
| 2698 | 2707 | endif |
| 2699 | 2708 | |
| 2700 | 2709 | config ARCH_HAS_ADD_PAGES |
| @@ -36,20 +36,20 @@ EXPORT_SYMBOL_GPL(write_ibpb); · 35 unmodified lines | ||
| 36 | 36 | |
| 37 | 37 | /* |
| 38 | 38 | * Define the VERW operand that is disguised as entry code so that |
| 39 | − * it can be referenced with KPTI enabled. This ensure VERW can be | |
| 39 | + * it can be referenced with KPTI enabled. This ensures VERW can be | |
| 40 | 40 | * used late in exit-to-user path after page tables are switched. |
| 41 | 41 | */ |
| 42 | 42 | .pushsection .entry.text, "ax" |
| 43 | 43 | |
| 44 | 44 | .align L1_CACHE_BYTES, 0xcc |
| 45 | −SYM_CODE_START_NOALIGN(mds_verw_sel) | |
| 45 | +SYM_CODE_START_NOALIGN(x86_verw_sel) | |
| 46 | 46 | UNWIND_HINT_UNDEFINED |
| 47 | 47 | ANNOTATE_NOENDBR |
| 48 | 48 | .word __KERNEL_DS |
| 49 | 49 | .align L1_CACHE_BYTES, 0xcc |
| 50 | −SYM_CODE_END(mds_verw_sel); | |
| 50 | +SYM_CODE_END(x86_verw_sel); | |
| 51 | 51 | /* For KVM */ |
| 52 | −EXPORT_SYMBOL_GPL(mds_verw_sel); | |
| 52 | +EXPORT_SYMBOL_GPL(x86_verw_sel); | |
| 53 | 53 | |
| 54 | 54 | .popsection |
| 55 | 55 | |
| @@ -456,6 +456,7 @@ · 455 unmodified lines | ||
| 456 | 456 | #define X86_FEATURE_NO_NESTED_DATA_BP (20*32+ 0) /* No Nested Data Breakpoints */ |
| 457 | 457 | #define X86_FEATURE_WRMSR_XX_BASE_NS (20*32+ 1) /* WRMSR to {FS,GS,KERNEL_GS}_BASE is non-serializing */ |
| 458 | 458 | #define X86_FEATURE_LFENCE_RDTSC (20*32+ 2) /* LFENCE always serializing / synchronizes RDTSC */ |
| 459 | +#define X86_FEATURE_VERW_CLEAR (20*32+ 5) /* The memory form of VERW mitigates TSA */ | |
| 459 | 460 | #define X86_FEATURE_NULL_SEL_CLR_BASE (20*32+ 6) /* Null Selector Clears Base */ |
| 460 | 461 | #define X86_FEATURE_AUTOIBRS (20*32+ 8) /* Automatic IBRS */ |
| 461 | 462 | #define X86_FEATURE_NO_SMM_CTL_MSR (20*32+ 9) /* SMM_CTL MSR is not present */ |
| @@ -487,6 +488,9 @@ · 25 unmodified lines | ||
| 487 | 488 | #define X86_FEATURE_PREFER_YMM (21*32+ 8) /* Avoid ZMM registers due to downclocking */ |
| 488 | 489 | #define X86_FEATURE_APX (21*32+ 9) /* Advanced Performance Extensions */ |
| 489 | 490 | #define X86_FEATURE_INDIRECT_THUNK_ITS (21*32+10) /* Use thunk for indirect branches in lower half of cacheline */ |
| 491 | +#define X86_FEATURE_TSA_SQ_NO (21*32+11) /* AMD CPU not vulnerable to TSA-SQ */ | |
| 492 | +#define X86_FEATURE_TSA_L1_NO (21*32+12) /* AMD CPU not vulnerable to TSA-L1 */ | |
| 493 | +#define X86_FEATURE_CLEAR_CPU_BUF_VM (21*32+13) /* Clear CPU buffers using VERW before VMRUN */ | |
| 490 | 494 | |
| 491 | 495 | /* |
| 492 | 496 | * BUG word(s) |
| @@ -542,5 +546,5 @@ · 49 unmodified lines | ||
| 542 | 546 | #define X86_BUG_OLD_MICROCODE X86_BUG( 1*32+ 6) /* "old_microcode" CPU has old microcode, it is surely vulnerable to something */ |
| 543 | 547 | #define X86_BUG_ITS X86_BUG( 1*32+ 7) /* "its" CPU is affected by Indirect Target Selection */ |
| 544 | 548 | #define X86_BUG_ITS_NATIVE_ONLY X86_BUG( 1*32+ 8) /* "its_native_only" CPU is affected by ITS, VMX is not affected */ |
| 545 | − | |
| 549 | +#define X86_BUG_TSA X86_BUG( 1*32+ 9) /* "tsa" CPU is affected by Transient Scheduler Attacks */ | |
| 546 | 550 | #endif /* _ASM_X86_CPUFEATURES_H */ |
| @@ -44,13 +44,13 @@ static __always_inline void native_irq_enable(void) · 43 unmodified lines | ||
| 44 | 44 | |
| 45 | 45 | static __always_inline void native_safe_halt(void) |
| 46 | 46 | { |
| 47 | − mds_idle_clear_cpu_buffers(); | |
| 47 | + x86_idle_clear_cpu_buffers(); | |
| 48 | 48 | asm volatile("sti; hlt": : :"memory"); |
| 49 | 49 | } |
| 50 | 50 | |
| 51 | 51 | static __always_inline void native_halt(void) |
| 52 | 52 | { |
| 53 | − mds_idle_clear_cpu_buffers(); | |
| 53 | + x86_idle_clear_cpu_buffers(); | |
| 54 | 54 | asm volatile("hlt": : :"memory"); |
| 55 | 55 | } |
| 56 | 56 | |
| @@ -700,8 +700,13 @@ struct kvm_vcpu_hv { · 699 unmodified lines | ||
| 700 | 700 | |
| 701 | 701 | struct kvm_vcpu_hv_tlb_flush_fifo tlb_flush_fifo[HV_NR_TLB_FLUSH_FIFOS]; |
| 702 | 702 | |
| 703 | − /* Preallocated buffer for handling hypercalls passing sparse vCPU set */ | |
| 703 | + /* | |
| 704 | + * Preallocated buffers for handling hypercalls that pass sparse vCPU | |
| 705 | + * sets (for high vCPU counts, they're too large to comfortably fit on | |
| 706 | + * the stack). | |
| 707 | + */ | |
| 704 | 708 | u64 sparse_banks[HV_MAX_SPARSE_VCPU_BANKS]; |
| 709 | + DECLARE_BITMAP(vcpu_mask, KVM_MAX_VCPUS); | |
| 705 | 710 | |
| 706 | 711 | struct hv_vp_assist_page vp_assist_page; |
| 707 | 712 | |
| @@ -764,6 +769,7 @@ enum kvm_only_cpuid_leafs { · 56 unmodified lines | ||
| 764 | 769 | CPUID_8000_0022_EAX, |
| 765 | 770 | CPUID_7_2_EDX, |
| 766 | 771 | CPUID_24_0_EBX, |
| 772 | + CPUID_8000_0021_ECX, | |
| 767 | 773 | NR_KVM_CPU_CAPS, |
| 768 | 774 | |
| 769 | 775 | NKVMCAPINTS = NR_KVM_CPU_CAPS - NCAPINTS, |
| @@ -628,6 +628,7 @@ · 627 unmodified lines | ||
| 628 | 628 | #define MSR_AMD64_OSVW_STATUS 0xc0010141 |
| 629 | 629 | #define MSR_AMD_PPIN_CTL 0xc00102f0 |
| 630 | 630 | #define MSR_AMD_PPIN 0xc00102f1 |
| 631 | +#define MSR_AMD64_CPUID_FN_7 0xc0011002 | |
| 631 | 632 | #define MSR_AMD64_CPUID_FN_1 0xc0011004 |
| 632 | 633 | #define MSR_AMD64_LS_CFG 0xc0011020 |
| 633 | 634 | #define MSR_AMD64_DC_CFG 0xc0011022 |
| @@ -43,8 +43,6 @@ static __always_inline void __monitorx(const void *eax, u32 ecx, u32 edx) · 42 unmodified lines | ||
| 43 | 43 | |
| 44 | 44 | static __always_inline void __mwait(u32 eax, u32 ecx) |
| 45 | 45 | { |
| 46 | − mds_idle_clear_cpu_buffers(); | |
| 47 | − | |
| 48 | 46 | /* |
| 49 | 47 | * Use the instruction mnemonic with implicit operands, as the LLVM |
| 50 | 48 | * assembler fails to assemble the mnemonic with explicit operands: |
| @@ -80,7 +78,7 @@ static __always_inline void __mwait(u32 eax, u32 ecx) · 29 unmodified lines | ||
| 80 | 78 | */ |
| 81 | 79 | static __always_inline void __mwaitx(u32 eax, u32 ebx, u32 ecx) |
| 82 | 80 | { |
| 83 | − /* No MDS buffer clear as this is AMD/HYGON only */ | |
| 81 | + /* No need for TSA buffer clearing on AMD */ | |
| 84 | 82 | |
| 85 | 83 | /* "mwaitx %eax, %ebx, %ecx" */ |
| 86 | 84 | asm volatile(".byte 0x0f, 0x01, 0xfb" |
| @@ -98,7 +96,6 @@ static __always_inline void __mwaitx(u32 eax, u32 ebx, u32 ecx) · 11 unmodified lines | ||
| 98 | 96 | */ |
| 99 | 97 | static __always_inline void __sti_mwait(u32 eax, u32 ecx) |
| 100 | 98 | { |
| 101 | − mds_idle_clear_cpu_buffers(); | |
| 102 | 99 | |
| 103 | 100 | asm volatile("sti; mwait" :: "a" (eax), "c" (ecx)); |
| 104 | 101 | } |
| @@ -115,21 +112,29 @@ static __always_inline void __sti_mwait(u32 eax, u32 ecx) · 10 unmodified lines | ||
| 115 | 112 | */ |
| 116 | 113 | static __always_inline void mwait_idle_with_hints(u32 eax, u32 ecx) |
| 117 | 114 | { |
| 115 | + if (need_resched()) | |
| 116 | + return; | |
| 117 | + | |
| 118 | + x86_idle_clear_cpu_buffers(); | |
| 119 | + | |
| 118 | 120 | if (static_cpu_has_bug(X86_BUG_MONITOR) || !current_set_polling_and_test()) { |
| 119 | 121 | const void *addr = ¤t_thread_info()->flags; |
| 120 | 122 | |
| 121 | 123 | alternative_input("", "clflush (%[addr])", X86_BUG_CLFLUSH_MONITOR, [addr] "a" (addr)); |
| 122 | 124 | __monitor(addr, 0, 0); |
| 123 | 125 | |
| 124 | − if (!need_resched()) { | |
| 125 | − if (ecx & 1) { | |
| 126 | − __mwait(eax, ecx); | |
| 127 | − } else { | |
| 128 | − __sti_mwait(eax, ecx); | |
| 129 | − raw_local_irq_disable(); | |
| 130 | − } | |
| 126 | + if (need_resched()) | |
| 127 | + goto out; | |
| 128 | + | |
| 129 | + if (ecx & 1) { | |
| 130 | + __mwait(eax, ecx); | |
| 131 | + } else { | |
| 132 | + __sti_mwait(eax, ecx); | |
| 133 | + raw_local_irq_disable(); | |
| 131 | 134 | } |
| 132 | 135 | } |
| 136 | + | |
| 137 | +out: | |
| 133 | 138 | current_clr_polling(); |
| 134 | 139 | } |
| 135 | 140 | |
| @@ -302,25 +302,31 @@ · 301 unmodified lines | ||
| 302 | 302 | .endm |
| 303 | 303 | |
| 304 | 304 | /* |
| 305 | − * Macro to execute VERW instruction that mitigate transient data sampling | |
| 306 | − * attacks such as MDS. On affected systems a microcode update overloaded VERW | |
| 307 | − * instruction to also clear the CPU buffers. VERW clobbers CFLAGS.ZF. | |
| 308 | − * | |
| 305 | + * Macro to execute VERW insns that mitigate transient data sampling | |
| 306 | + * attacks such as MDS or TSA. On affected systems a microcode update | |
| 307 | + * overloaded VERW insns to also clear the CPU buffers. VERW clobbers | |
| 308 | + * CFLAGS.ZF. | |
| 309 | 309 | * Note: Only the memory operand variant of VERW clears the CPU buffers. |
| 310 | 310 | */ |
| 311 | −.macro CLEAR_CPU_BUFFERS | |
| 311 | +.macro __CLEAR_CPU_BUFFERS feature | |
| 312 | 312 | #ifdef CONFIG_X86_64 |
| 313 | − ALTERNATIVE "", "verw mds_verw_sel(%rip)", X86_FEATURE_CLEAR_CPU_BUF | |
| 313 | + ALTERNATIVE "", "verw x86_verw_sel(%rip)", \feature | |
| 314 | 314 | #else |
| 315 | 315 | /* |
| 316 | 316 | * In 32bit mode, the memory operand must be a %cs reference. The data |
| 317 | 317 | * segments may not be usable (vm86 mode), and the stack segment may not |
| 318 | 318 | * be flat (ESPFIX32). |
| 319 | 319 | */ |
| 320 | − ALTERNATIVE "", "verw %cs:mds_verw_sel", X86_FEATURE_CLEAR_CPU_BUF | |
| 320 | + ALTERNATIVE "", "verw %cs:x86_verw_sel", \feature | |
| 321 | 321 | #endif |
| 322 | 322 | .endm |
| 323 | 323 | |
| 324 | +#define CLEAR_CPU_BUFFERS \ | |
| 325 | + __CLEAR_CPU_BUFFERS X86_FEATURE_CLEAR_CPU_BUF | |
| 326 | + | |
| 327 | +#define VM_CLEAR_CPU_BUFFERS \ | |
| 328 | + __CLEAR_CPU_BUFFERS X86_FEATURE_CLEAR_CPU_BUF_VM | |
| 329 | + | |
| 324 | 330 | #ifdef CONFIG_X86_64 |
| 325 | 331 | .macro CLEAR_BRANCH_HISTORY |
| 326 | 332 | ALTERNATIVE "", "call clear_bhb_loop", X86_FEATURE_CLEAR_BHB_LOOP |
| @@ -567,24 +573,24 @@ DECLARE_STATIC_KEY_FALSE(switch_mm_always_ibpb); · 240 unmodified lines | ||
| 567 | 573 | |
| 568 | 574 | DECLARE_STATIC_KEY_FALSE(switch_vcpu_ibpb); |
| 569 | 575 | |
| 570 | −DECLARE_STATIC_KEY_FALSE(mds_idle_clear); | |
| 576 | +DECLARE_STATIC_KEY_FALSE(cpu_buf_idle_clear); | |
| 571 | 577 | |
| 572 | 578 | DECLARE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush); |
| 573 | 579 | |
| 574 | 580 | DECLARE_STATIC_KEY_FALSE(cpu_buf_vm_clear); |
| 575 | 581 | |
| 576 | −extern u16 mds_verw_sel; | |
| 582 | +extern u16 x86_verw_sel; | |
| 577 | 583 | |
| 578 | 584 | #include <asm/segment.h> |
| 579 | 585 | |
| 580 | 586 | /** |
| 581 | − * mds_clear_cpu_buffers - Mitigation for MDS and TAA vulnerability | |
| 587 | + * x86_clear_cpu_buffers - Buffer clearing support for different x86 CPU vulns | |
| 582 | 588 | * |
| 583 | 589 | * This uses the otherwise unused and obsolete VERW instruction in |
| 584 | 590 | * combination with microcode which triggers a CPU buffer flush when the |
| 585 | 591 | * instruction is executed. |
| 586 | 592 | */ |
| 587 | −static __always_inline void mds_clear_cpu_buffers(void) | |
| 593 | +static __always_inline void x86_clear_cpu_buffers(void) | |
| 588 | 594 | { |
| 589 | 595 | static const u16 ds = __KERNEL_DS; |
| 590 | 596 | |
| @@ -601,14 +607,15 @@ static __always_inline void mds_clear_cpu_buffers(void) · 10 unmodified lines | ||
| 601 | 607 | } |
| 602 | 608 | |
| 603 | 609 | /** |
| 604 | − * mds_idle_clear_cpu_buffers - Mitigation for MDS vulnerability | |
| 610 | + * x86_idle_clear_cpu_buffers - Buffer clearing support in idle for the MDS | |
| 611 | + * and TSA vulnerabilities. | |
| 605 | 612 | * |
| 606 | 613 | * Clear CPU buffers if the corresponding static key is enabled |
| 607 | 614 | */ |
| 608 | −static __always_inline void mds_idle_clear_cpu_buffers(void) | |
| 615 | +static __always_inline void x86_idle_clear_cpu_buffers(void) | |
| 609 | 616 | { |
| 610 | − if (static_branch_likely(&mds_idle_clear)) | |
| 611 | − mds_clear_cpu_buffers(); | |
| 617 | + if (static_branch_likely(&cpu_buf_idle_clear)) | |
| 618 | + x86_clear_cpu_buffers(); | |
| 612 | 619 | } |
| 613 | 620 | |
| 614 | 621 | #endif /* __ASSEMBLER__ */ |
| @@ -72,6 +72,7 @@ · 71 unmodified lines | ||
| 72 | 72 | #define TDVMCALL_MAP_GPA 0x10001 |
| 73 | 73 | #define TDVMCALL_GET_QUOTE 0x10002 |
| 74 | 74 | #define TDVMCALL_REPORT_FATAL_ERROR 0x10003 |
| 75 | +#define TDVMCALL_SETUP_EVENT_NOTIFY_INTERRUPT 0x10004ULL | |
| 75 | 76 | |
| 76 | 77 | /* |
| 77 | 78 | * TDG.VP.VMCALL Status Codes (returned in R10) |
| @@ -965,7 +965,13 @@ struct kvm_tdx_cmd { · 964 unmodified lines | ||
| 965 | 965 | struct kvm_tdx_capabilities { |
| 966 | 966 | __u64 supported_attrs; |
| 967 | 967 | __u64 supported_xfam; |
| 968 | − __u64 reserved[254]; | |
| 968 | + | |
| 969 | + __u64 kernel_tdvmcallinfo_1_r11; | |
| 970 | + __u64 user_tdvmcallinfo_1_r11; | |
| 971 | + __u64 kernel_tdvmcallinfo_1_r12; | |
| 972 | + __u64 user_tdvmcallinfo_1_r12; | |
| 973 | + | |
| 974 | + __u64 reserved[250]; | |
| 969 | 975 | |
| 970 | 976 | /* Configurable CPUID bits for userspace */ |
| 971 | 977 | struct kvm_cpuid2 cpuid; |
| @@ -377,6 +377,47 @@ static void bsp_determine_snp(struct cpuinfo_x86 *c) · 376 unmodified lines | ||
| 377 | 377 | #endif |
| 378 | 378 | } |
| 379 | 379 | |
| 380 | +#define ZEN_MODEL_STEP_UCODE(fam, model, step, ucode) \ | |
| 381 | + X86_MATCH_VFM_STEPS(VFM_MAKE(X86_VENDOR_AMD, fam, model), \ | |
| 382 | + step, step, ucode) | |
| 383 | + | |
| 384 | +static const struct x86_cpu_id amd_tsa_microcode[] = { | |
| 385 | + ZEN_MODEL_STEP_UCODE(0x19, 0x01, 0x1, 0x0a0011d7), | |
| 386 | + ZEN_MODEL_STEP_UCODE(0x19, 0x01, 0x2, 0x0a00123b), | |
| 387 | + ZEN_MODEL_STEP_UCODE(0x19, 0x08, 0x2, 0x0a00820d), | |
| 388 | + ZEN_MODEL_STEP_UCODE(0x19, 0x11, 0x1, 0x0a10114c), | |
| 389 | + ZEN_MODEL_STEP_UCODE(0x19, 0x11, 0x2, 0x0a10124c), | |
| 390 | + ZEN_MODEL_STEP_UCODE(0x19, 0x18, 0x1, 0x0a108109), | |
| 391 | + ZEN_MODEL_STEP_UCODE(0x19, 0x21, 0x0, 0x0a20102e), | |
| 392 | + ZEN_MODEL_STEP_UCODE(0x19, 0x21, 0x2, 0x0a201211), | |
| 393 | + ZEN_MODEL_STEP_UCODE(0x19, 0x44, 0x1, 0x0a404108), | |
| 394 | + ZEN_MODEL_STEP_UCODE(0x19, 0x50, 0x0, 0x0a500012), | |
| 395 | + ZEN_MODEL_STEP_UCODE(0x19, 0x61, 0x2, 0x0a60120a), | |
| 396 | + ZEN_MODEL_STEP_UCODE(0x19, 0x74, 0x1, 0x0a704108), | |
| 397 | + ZEN_MODEL_STEP_UCODE(0x19, 0x75, 0x2, 0x0a705208), | |
| 398 | + ZEN_MODEL_STEP_UCODE(0x19, 0x78, 0x0, 0x0a708008), | |
| 399 | + ZEN_MODEL_STEP_UCODE(0x19, 0x7c, 0x0, 0x0a70c008), | |
| 400 | + ZEN_MODEL_STEP_UCODE(0x19, 0xa0, 0x2, 0x0aa00216), | |
| 401 | + {}, | |
| 402 | +}; | |
| 403 | + | |
| 404 | +static void tsa_init(struct cpuinfo_x86 *c) | |
| 405 | +{ | |
| 406 | + if (cpu_has(c, X86_FEATURE_HYPERVISOR)) | |
| 407 | + return; | |
| 408 | + | |
| 409 | + if (cpu_has(c, X86_FEATURE_ZEN3) || | |
| 410 | + cpu_has(c, X86_FEATURE_ZEN4)) { | |
| 411 | + if (x86_match_min_microcode_rev(amd_tsa_microcode)) | |
| 412 | + setup_force_cpu_cap(X86_FEATURE_VERW_CLEAR); | |
| 413 | + else | |
| 414 | + pr_debug("%s: current revision: 0x%x\n", __func__, c->microcode); | |
| 415 | + } else { | |
| 416 | + setup_force_cpu_cap(X86_FEATURE_TSA_SQ_NO); | |
| 417 | + setup_force_cpu_cap(X86_FEATURE_TSA_L1_NO); | |
| 418 | + } | |
| 419 | +} | |
| 420 | + | |
| 380 | 421 | static void bsp_init_amd(struct cpuinfo_x86 *c) |
| 381 | 422 | { |
| 382 | 423 | if (cpu_has(c, X86_FEATURE_CONSTANT_TSC)) { |
| @@ -489,6 +530,9 @@ static void bsp_init_amd(struct cpuinfo_x86 *c) · 106 unmodified lines | ||
| 489 | 530 | } |
| 490 | 531 | |
| 491 | 532 | bsp_determine_snp(c); |
| 533 | + | |
| 534 | + tsa_init(c); | |
| 535 | + | |
| 492 | 536 | return; |
| 493 | 537 | |
| 494 | 538 | warn: |
| @@ -930,6 +974,16 @@ static void init_amd_zen2(struct cpuinfo_x86 *c) · 435 unmodified lines | ||
| 930 | 974 | init_spectral_chicken(c); |
| 931 | 975 | fix_erratum_1386(c); |
| 932 | 976 | zen2_zenbleed_check(c); |
| 977 | + | |
| 978 | + /* Disable RDSEED on AMD Cyan Skillfish because of an error. */ | |
| 979 | + if (c->x86_model == 0x47 && c->x86_stepping == 0x0) { | |
| 980 | + clear_cpu_cap(c, X86_FEATURE_RDSEED); | |
| 981 | + msr_clear_bit(MSR_AMD64_CPUID_FN_7, 18); | |
| 982 | + pr_emerg("RDSEED is not reliable on this platform; disabling.\n"); | |
| 983 | + } | |
| 984 | + | |
| 985 | + /* Correct misconfigured CPUID on some clients. */ | |
| 986 | + clear_cpu_cap(c, X86_FEATURE_INVLPGB); | |
| 933 | 987 | } |
| 934 | 988 | |
| 935 | 989 | static void init_amd_zen3(struct cpuinfo_x86 *c) |
| @@ -94,6 +94,8 @@ static void __init bhi_apply_mitigation(void); · 93 unmodified lines | ||
| 94 | 94 | static void __init its_select_mitigation(void); |
| 95 | 95 | static void __init its_update_mitigation(void); |
| 96 | 96 | static void __init its_apply_mitigation(void); |
| 97 | +static void __init tsa_select_mitigation(void); | |
| 98 | +static void __init tsa_apply_mitigation(void); | |
| 97 | 99 | |
| 98 | 100 | /* The base value of the SPEC_CTRL MSR without task-specific bits set */ |
| 99 | 101 | u64 x86_spec_ctrl_base; |
| @@ -169,9 +171,9 @@ DEFINE_STATIC_KEY_FALSE(switch_mm_always_ibpb); · 69 unmodified lines | ||
| 169 | 171 | DEFINE_STATIC_KEY_FALSE(switch_vcpu_ibpb); |
| 170 | 172 | EXPORT_SYMBOL_GPL(switch_vcpu_ibpb); |
| 171 | 173 | |
| 172 | −/* Control MDS CPU buffer clear before idling (halt, mwait) */ | |
| 173 | −DEFINE_STATIC_KEY_FALSE(mds_idle_clear); | |
| 174 | −EXPORT_SYMBOL_GPL(mds_idle_clear); | |
| 174 | +/* Control CPU buffer clear before idling (halt, mwait) */ | |
| 175 | +DEFINE_STATIC_KEY_FALSE(cpu_buf_idle_clear); | |
| 176 | +EXPORT_SYMBOL_GPL(cpu_buf_idle_clear); | |
| 175 | 177 | |
| 176 | 178 | /* |
| 177 | 179 | * Controls whether l1d flush based mitigations are enabled, |
| @@ -225,6 +227,7 @@ void __init cpu_select_mitigations(void) · 47 unmodified lines | ||
| 225 | 227 | gds_select_mitigation(); |
| 226 | 228 | its_select_mitigation(); |
| 227 | 229 | bhi_select_mitigation(); |
| 230 | + tsa_select_mitigation(); | |
| 228 | 231 | |
| 229 | 232 | /* |
| 230 | 233 | * After mitigations are selected, some may need to update their |
| @@ -272,6 +275,7 @@ void __init cpu_select_mitigations(void) · 41 unmodified lines | ||
| 272 | 275 | gds_apply_mitigation(); |
| 273 | 276 | its_apply_mitigation(); |
| 274 | 277 | bhi_apply_mitigation(); |
| 278 | + tsa_apply_mitigation(); | |
| 275 | 279 | } |
| 276 | 280 | |
| 277 | 281 | /* |
| @@ -637,7 +641,7 @@ static void __init mmio_apply_mitigation(void) · 359 unmodified lines | ||
| 637 | 641 | * is required irrespective of SMT state. |
| 638 | 642 | */ |
| 639 | 643 | if (!(x86_arch_cap_msr & ARCH_CAP_FBSDP_NO)) |
| 640 | − static_branch_enable(&mds_idle_clear); | |
| 644 | + static_branch_enable(&cpu_buf_idle_clear); | |
| 641 | 645 | |
| 642 | 646 | if (mmio_nosmt || cpu_mitigations_auto_nosmt()) |
| 643 | 647 | cpu_smt_disable(false); |
| @@ -1487,6 +1491,94 @@ static void __init its_apply_mitigation(void) · 843 unmodified lines | ||
| 1487 | 1491 | set_return_thunk(its_return_thunk); |
| 1488 | 1492 | } |
| 1489 | 1493 | |
| 1494 | +#undef pr_fmt | |
| 1495 | +#define pr_fmt(fmt) "Transient Scheduler Attacks: " fmt | |
| 1496 | + | |
| 1497 | +enum tsa_mitigations { | |
| 1498 | + TSA_MITIGATION_NONE, | |
| 1499 | + TSA_MITIGATION_AUTO, | |
| 1500 | + TSA_MITIGATION_UCODE_NEEDED, | |
| 1501 | + TSA_MITIGATION_USER_KERNEL, | |
| 1502 | + TSA_MITIGATION_VM, | |
| 1503 | + TSA_MITIGATION_FULL, | |
| 1504 | +}; | |
| 1505 | + | |
| 1506 | +static const char * const tsa_strings[] = { | |
| 1507 | + [TSA_MITIGATION_NONE] = "Vulnerable", | |
| 1508 | + [TSA_MITIGATION_UCODE_NEEDED] = "Vulnerable: No microcode", | |
| 1509 | + [TSA_MITIGATION_USER_KERNEL] = "Mitigation: Clear CPU buffers: user/kernel boundary", | |
| 1510 | + [TSA_MITIGATION_VM] = "Mitigation: Clear CPU buffers: VM", | |
| 1511 | + [TSA_MITIGATION_FULL] = "Mitigation: Clear CPU buffers", | |
| 1512 | +}; | |
| 1513 | + | |
| 1514 | +static enum tsa_mitigations tsa_mitigation __ro_after_init = | |
| 1515 | + IS_ENABLED(CONFIG_MITIGATION_TSA) ? TSA_MITIGATION_AUTO : TSA_MITIGATION_NONE; | |
| 1516 | + | |
| 1517 | +static int __init tsa_parse_cmdline(char *str) | |
| 1518 | +{ | |
| 1519 | + if (!str) | |
| 1520 | + return -EINVAL; | |
| 1521 | + | |
| 1522 | + if (!strcmp(str, "off")) | |
| 1523 | + tsa_mitigation = TSA_MITIGATION_NONE; | |
| 1524 | + else if (!strcmp(str, "on")) | |
| 1525 | + tsa_mitigation = TSA_MITIGATION_FULL; | |
| 1526 | + else if (!strcmp(str, "user")) | |
| 1527 | + tsa_mitigation = TSA_MITIGATION_USER_KERNEL; | |
| 1528 | + else if (!strcmp(str, "vm")) | |
| 1529 | + tsa_mitigation = TSA_MITIGATION_VM; | |
| 1530 | + else | |
| 1531 | + pr_err("Ignoring unknown tsa=%s option.\n", str); | |
| 1532 | + | |
| 1533 | + return 0; | |
| 1534 | +} | |
| 1535 | +early_param("tsa", tsa_parse_cmdline); | |
| 1536 | + | |
| 1537 | +static void __init tsa_select_mitigation(void) | |
| 1538 | +{ | |
| 1539 | + if (cpu_mitigations_off() || !boot_cpu_has_bug(X86_BUG_TSA)) { | |
| 1540 | + tsa_mitigation = TSA_MITIGATION_NONE; | |
| 1541 | + return; | |
| 1542 | + } | |
| 1543 | + | |
| 1544 | + if (tsa_mitigation == TSA_MITIGATION_NONE) | |
| 1545 | + return; | |
| 1546 | + | |
| 1547 | + if (!boot_cpu_has(X86_FEATURE_VERW_CLEAR)) { | |
| 1548 | + tsa_mitigation = TSA_MITIGATION_UCODE_NEEDED; | |
| 1549 | + goto out; | |
| 1550 | + } | |
| 1551 | + | |
| 1552 | + if (tsa_mitigation == TSA_MITIGATION_AUTO) | |
| 1553 | + tsa_mitigation = TSA_MITIGATION_FULL; | |
| 1554 | + | |
| 1555 | + /* | |
| 1556 | + * No need to set verw_clear_cpu_buf_mitigation_selected - it | |
| 1557 | + * doesn't fit all cases here and it is not needed because this | |
| 1558 | + * is the only VERW-based mitigation on AMD. | |
| 1559 | + */ | |
| 1560 | +out: | |
| 1561 | + pr_info("%s\n", tsa_strings[tsa_mitigation]); | |
| 1562 | +} | |
| 1563 | + | |
| 1564 | +static void __init tsa_apply_mitigation(void) | |
| 1565 | +{ | |
| 1566 | + switch (tsa_mitigation) { | |
| 1567 | + case TSA_MITIGATION_USER_KERNEL: | |
| 1568 | + setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF); | |
| 1569 | + break; | |
| 1570 | + case TSA_MITIGATION_VM: | |
| 1571 | + setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM); | |
| 1572 | + break; | |
| 1573 | + case TSA_MITIGATION_FULL: | |
| 1574 | + setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF); | |
| 1575 | + setup_force_cpu_cap(X86_FEATURE_CLEAR_CPU_BUF_VM); | |
| 1576 | + break; | |
| 1577 | + default: | |
| 1578 | + break; | |
| 1579 | + } | |
| 1580 | +} | |
| 1581 | + | |
| 1490 | 1582 | #undef pr_fmt |
| 1491 | 1583 | #define pr_fmt(fmt) "Spectre V2 : " fmt |
| 1492 | 1584 | |
| @@ -2249,10 +2341,10 @@ static void update_mds_branch_idle(void) · 756 unmodified lines | ||
| 2249 | 2341 | return; |
| 2250 | 2342 | |
| 2251 | 2343 | if (sched_smt_active()) { |
| 2252 | − static_branch_enable(&mds_idle_clear); | |
| 2344 | + static_branch_enable(&cpu_buf_idle_clear); | |
| 2253 | 2345 | } else if (mmio_mitigation == MMIO_MITIGATION_OFF || |
| 2254 | 2346 | (x86_arch_cap_msr & ARCH_CAP_FBSDP_NO)) { |
| 2255 | − static_branch_disable(&mds_idle_clear); | |
| 2347 | + static_branch_disable(&cpu_buf_idle_clear); | |
| 2256 | 2348 | } |
| 2257 | 2349 | } |
| 2258 | 2350 | |
| @@ -2316,6 +2408,25 @@ void cpu_bugs_smt_update(void) · 57 unmodified lines | ||
| 2316 | 2408 | break; |
| 2317 | 2409 | } |
| 2318 | 2410 | |
| 2411 | + switch (tsa_mitigation) { | |
| 2412 | + case TSA_MITIGATION_USER_KERNEL: | |
| 2413 | + case TSA_MITIGATION_VM: | |
| 2414 | + case TSA_MITIGATION_AUTO: | |
| 2415 | + case TSA_MITIGATION_FULL: | |
| 2416 | + /* | |
| 2417 | + * TSA-SQ can potentially lead to info leakage between | |
| 2418 | + * SMT threads. | |
| 2419 | + */ | |
| 2420 | + if (sched_smt_active()) | |
| 2421 | + static_branch_enable(&cpu_buf_idle_clear); | |
| 2422 | + else | |
| 2423 | + static_branch_disable(&cpu_buf_idle_clear); | |
| 2424 | + break; | |
| 2425 | + case TSA_MITIGATION_NONE: | |
| 2426 | + case TSA_MITIGATION_UCODE_NEEDED: | |
| 2427 | + break; | |
| 2428 | + } | |
| 2429 | + | |
| 2319 | 2430 | mutex_unlock(&spec_ctrl_mutex); |
| 2320 | 2431 | } |
| 2321 | 2432 | |
| @@ -3265,6 +3376,11 @@ static ssize_t gds_show_state(char *buf) · 943 unmodified lines | ||
| 3265 | 3376 | return sysfs_emit(buf, "%s\n", gds_strings[gds_mitigation]); |
| 3266 | 3377 | } |
| 3267 | 3378 | |
| 3379 | +static ssize_t tsa_show_state(char *buf) | |
| 3380 | +{ | |
| 3381 | + return sysfs_emit(buf, "%s\n", tsa_strings[tsa_mitigation]); | |
| 3382 | +} | |
| 3383 | + | |
| 3268 | 3384 | static ssize_t cpu_show_common(struct device *dev, struct device_attribute *attr, |
| 3269 | 3385 | char *buf, unsigned int bug) |
| 3270 | 3386 | { |
| @@ -3328,6 +3444,9 @@ static ssize_t cpu_show_common(struct device *dev, struct device_attribute *attr · 57 unmodified lines | ||
| 3328 | 3444 | case X86_BUG_ITS: |
| 3329 | 3445 | return its_show_state(buf); |
| 3330 | 3446 | |
| 3447 | + case X86_BUG_TSA: | |
| 3448 | + return tsa_show_state(buf); | |
| 3449 | + | |
| 3331 | 3450 | default: |
| 3332 | 3451 | break; |
| 3333 | 3452 | } |
| @@ -3414,6 +3533,11 @@ ssize_t cpu_show_indirect_target_selection(struct device *dev, struct device_att · 80 unmodified lines | ||
| 3414 | 3533 | { |
| 3415 | 3534 | return cpu_show_common(dev, attr, buf, X86_BUG_ITS); |
| 3416 | 3535 | } |
| 3536 | + | |
| 3537 | +ssize_t cpu_show_tsa(struct device *dev, struct device_attribute *attr, char *buf) | |
| 3538 | +{ | |
| 3539 | + return cpu_show_common(dev, attr, buf, X86_BUG_TSA); | |
| 3540 | +} | |
| 3417 | 3541 | #endif |
| 3418 | 3542 | |
| 3419 | 3543 | void __warn_thunk(void) |
| @@ -1233,6 +1233,8 @@ static const __initconst struct x86_cpu_id cpu_vuln_whitelist[] = { · 1232 unmodified lines | ||
| 1233 | 1233 | #define ITS BIT(8) |
| 1234 | 1234 | /* CPU is affected by Indirect Target Selection, but guest-host isolation is not affected */ |
| 1235 | 1235 | #define ITS_NATIVE_ONLY BIT(9) |
| 1236 | +/* CPU is affected by Transient Scheduler Attacks */ | |
| 1237 | +#define TSA BIT(10) | |
| 1236 | 1238 | |
| 1237 | 1239 | static const struct x86_cpu_id cpu_vuln_blacklist[] __initconst = { |
| 1238 | 1240 | VULNBL_INTEL_STEPS(INTEL_IVYBRIDGE, X86_STEP_MAX, SRBDS), |
| @@ -1280,7 +1282,7 @@ static const struct x86_cpu_id cpu_vuln_blacklist[] __initconst = { · 41 unmodified lines | ||
| 1280 | 1282 | VULNBL_AMD(0x16, RETBLEED), |
| 1281 | 1283 | VULNBL_AMD(0x17, RETBLEED | SMT_RSB | SRSO), |
| 1282 | 1284 | VULNBL_HYGON(0x18, RETBLEED | SMT_RSB | SRSO), |
| 1283 | − VULNBL_AMD(0x19, SRSO), | |
| 1285 | + VULNBL_AMD(0x19, SRSO | TSA), | |
| 1284 | 1286 | VULNBL_AMD(0x1a, SRSO), |
| 1285 | 1287 | {} |
| 1286 | 1288 | }; |
| @@ -1530,6 +1532,16 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c) · 243 unmodified lines | ||
| 1530 | 1532 | setup_force_cpu_bug(X86_BUG_ITS_NATIVE_ONLY); |
| 1531 | 1533 | } |
| 1532 | 1534 | |
| 1535 | + if (c->x86_vendor == X86_VENDOR_AMD) { | |
| 1536 | + if (!cpu_has(c, X86_FEATURE_TSA_SQ_NO) || | |
| 1537 | + !cpu_has(c, X86_FEATURE_TSA_L1_NO)) { | |
| 1538 | + if (cpu_matches(cpu_vuln_blacklist, TSA) || | |
| 1539 | + /* Enable bug on Zen guests to allow for live migration. */ | |
| 1540 | + (cpu_has(c, X86_FEATURE_HYPERVISOR) && cpu_has(c, X86_FEATURE_ZEN))) | |
| 1541 | + setup_force_cpu_bug(X86_BUG_TSA); | |
| 1542 | + } | |
| 1543 | + } | |
| 1544 | + | |
| 1533 | 1545 | if (cpu_matches(cpu_vuln_whitelist, NO_MELTDOWN)) |
| 1534 | 1546 | return; |
| 1535 | 1547 | |
| @@ -231,6 +231,13 @@ static const struct patch_digest phashes[] = { · 230 unmodified lines | ||
| 231 | 231 | 0x0d,0x5b,0x65,0x34,0x69,0xb2,0x62,0x21, |
| 232 | 232 | } |
| 233 | 233 | }, |
| 234 | + { 0xa0011d7, { | |
| 235 | + 0x35,0x07,0xcd,0x40,0x94,0xbc,0x81,0x6b, | |
| 236 | + 0xfc,0x61,0x56,0x1a,0xe2,0xdb,0x96,0x12, | |
| 237 | + 0x1c,0x1c,0x31,0xb1,0x02,0x6f,0xe5,0xd2, | |
| 238 | + 0xfe,0x1b,0x04,0x03,0x2c,0x8f,0x4c,0x36, | |
| 239 | + } | |
| 240 | + }, | |
| 234 | 241 | { 0xa001223, { |
| 235 | 242 | 0xfb,0x32,0x5f,0xc6,0x83,0x4f,0x8c,0xb8, |
| 236 | 243 | 0xa4,0x05,0xf9,0x71,0x53,0x01,0x16,0xc4, |
| @@ -294,6 +301,13 @@ static const struct patch_digest phashes[] = { · 57 unmodified lines | ||
| 294 | 301 | 0xc0,0xcd,0x33,0xf2,0x8d,0xf9,0xef,0x59, |
| 295 | 302 | } |
| 296 | 303 | }, |
| 304 | + { 0xa00123b, { | |
| 305 | + 0xef,0xa1,0x1e,0x71,0xf1,0xc3,0x2c,0xe2, | |
| 306 | + 0xc3,0xef,0x69,0x41,0x7a,0x54,0xca,0xc3, | |
| 307 | + 0x8f,0x62,0x84,0xee,0xc2,0x39,0xd9,0x28, | |
| 308 | + 0x95,0xa7,0x12,0x49,0x1e,0x30,0x71,0x72, | |
| 309 | + } | |
| 310 | + }, | |
| 297 | 311 | { 0xa00820c, { |
| 298 | 312 | 0xa8,0x0c,0x81,0xc0,0xa6,0x00,0xe7,0xf3, |
| 299 | 313 | 0x5f,0x65,0xd3,0xb9,0x6f,0xea,0x93,0x63, |
| @@ -301,6 +315,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 301 | 315 | 0xe1,0x3b,0x8d,0xb2,0xf8,0x22,0x03,0xe2, |
| 302 | 316 | } |
| 303 | 317 | }, |
| 318 | + { 0xa00820d, { | |
| 319 | + 0xf9,0x2a,0xc0,0xf4,0x9e,0xa4,0x87,0xa4, | |
| 320 | + 0x7d,0x87,0x00,0xfd,0xab,0xda,0x19,0xca, | |
| 321 | + 0x26,0x51,0x32,0xc1,0x57,0x91,0xdf,0xc1, | |
| 322 | + 0x05,0xeb,0x01,0x7c,0x5a,0x95,0x21,0xb7, | |
| 323 | + } | |
| 324 | + }, | |
| 304 | 325 | { 0xa10113e, { |
| 305 | 326 | 0x05,0x3c,0x66,0xd7,0xa9,0x5a,0x33,0x10, |
| 306 | 327 | 0x1b,0xf8,0x9c,0x8f,0xed,0xfc,0xa7,0xa0, |
| @@ -322,6 +343,13 @@ static const struct patch_digest phashes[] = { · 15 unmodified lines | ||
| 322 | 343 | 0xf1,0x5e,0xb0,0xde,0xb4,0x98,0xae,0xc4, |
| 323 | 344 | } |
| 324 | 345 | }, |
| 346 | + { 0xa10114c, { | |
| 347 | + 0x9e,0xb6,0xa2,0xd9,0x87,0x38,0xc5,0x64, | |
| 348 | + 0xd8,0x88,0xfa,0x78,0x98,0xf9,0x6f,0x74, | |
| 349 | + 0x39,0x90,0x1b,0xa5,0xcf,0x5e,0xb4,0x2a, | |
| 350 | + 0x02,0xff,0xd4,0x8c,0x71,0x8b,0xe2,0xc0, | |
| 351 | + } | |
| 352 | + }, | |
| 325 | 353 | { 0xa10123e, { |
| 326 | 354 | 0x03,0xb9,0x2c,0x76,0x48,0x93,0xc9,0x18, |
| 327 | 355 | 0xfb,0x56,0xfd,0xf7,0xe2,0x1d,0xca,0x4d, |
| @@ -343,6 +371,13 @@ static const struct patch_digest phashes[] = { · 15 unmodified lines | ||
| 343 | 371 | 0x1b,0x7d,0x64,0x9d,0x4b,0x53,0x13,0x75, |
| 344 | 372 | } |
| 345 | 373 | }, |
| 374 | + { 0xa10124c, { | |
| 375 | + 0x29,0xea,0xf1,0x2c,0xb2,0xe4,0xef,0x90, | |
| 376 | + 0xa4,0xcd,0x1d,0x86,0x97,0x17,0x61,0x46, | |
| 377 | + 0xfc,0x22,0xcb,0x57,0x75,0x19,0xc8,0xcc, | |
| 378 | + 0x0c,0xf5,0xbc,0xac,0x81,0x9d,0x9a,0xd2, | |
| 379 | + } | |
| 380 | + }, | |
| 346 | 381 | { 0xa108108, { |
| 347 | 382 | 0xed,0xc2,0xec,0xa1,0x15,0xc6,0x65,0xe9, |
| 348 | 383 | 0xd0,0xef,0x39,0xaa,0x7f,0x55,0x06,0xc6, |
| @@ -350,6 +385,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 350 | 385 | 0x28,0x1e,0x9c,0x59,0x69,0x99,0x4d,0x16, |
| 351 | 386 | } |
| 352 | 387 | }, |
| 388 | + { 0xa108109, { | |
| 389 | + 0x85,0xb4,0xbd,0x7c,0x49,0xa7,0xbd,0xfa, | |
| 390 | + 0x49,0x36,0x80,0x81,0xc5,0xb7,0x39,0x1b, | |
| 391 | + 0x9a,0xaa,0x50,0xde,0x9b,0xe9,0x32,0x35, | |
| 392 | + 0x42,0x7e,0x51,0x4f,0x52,0x2c,0x28,0x59, | |
| 393 | + } | |
| 394 | + }, | |
| 353 | 395 | { 0xa20102d, { |
| 354 | 396 | 0xf9,0x6e,0xf2,0x32,0xd3,0x0f,0x5f,0x11, |
| 355 | 397 | 0x59,0xa1,0xfe,0xcc,0xcd,0x9b,0x42,0x89, |
| @@ -357,6 +399,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 357 | 399 | 0x8c,0xe9,0x19,0x3e,0xcc,0x3f,0x7b,0xb4, |
| 358 | 400 | } |
| 359 | 401 | }, |
| 402 | + { 0xa20102e, { | |
| 403 | + 0xbe,0x1f,0x32,0x04,0x0d,0x3c,0x9c,0xdd, | |
| 404 | + 0xe1,0xa4,0xbf,0x76,0x3a,0xec,0xc2,0xf6, | |
| 405 | + 0x11,0x00,0xa7,0xaf,0x0f,0xe5,0x02,0xc5, | |
| 406 | + 0x54,0x3a,0x1f,0x8c,0x16,0xb5,0xff,0xbe, | |
| 407 | + } | |
| 408 | + }, | |
| 360 | 409 | { 0xa201210, { |
| 361 | 410 | 0xe8,0x6d,0x51,0x6a,0x8e,0x72,0xf3,0xfe, |
| 362 | 411 | 0x6e,0x16,0xbc,0x62,0x59,0x40,0x17,0xe9, |
| @@ -364,6 +413,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 364 | 413 | 0xf7,0x55,0xf0,0x13,0xbb,0x22,0xf6,0x41, |
| 365 | 414 | } |
| 366 | 415 | }, |
| 416 | + { 0xa201211, { | |
| 417 | + 0x69,0xa1,0x17,0xec,0xd0,0xf6,0x6c,0x95, | |
| 418 | + 0xe2,0x1e,0xc5,0x59,0x1a,0x52,0x0a,0x27, | |
| 419 | + 0xc4,0xed,0xd5,0x59,0x1f,0xbf,0x00,0xff, | |
| 420 | + 0x08,0x88,0xb5,0xe1,0x12,0xb6,0xcc,0x27, | |
| 421 | + } | |
| 422 | + }, | |
| 367 | 423 | { 0xa404107, { |
| 368 | 424 | 0xbb,0x04,0x4e,0x47,0xdd,0x5e,0x26,0x45, |
| 369 | 425 | 0x1a,0xc9,0x56,0x24,0xa4,0x4c,0x82,0xb0, |
| @@ -371,6 +427,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 371 | 427 | 0x13,0xbc,0xc5,0x25,0xe4,0xc5,0xc3,0x99, |
| 372 | 428 | } |
| 373 | 429 | }, |
| 430 | + { 0xa404108, { | |
| 431 | + 0x69,0x67,0x43,0x06,0xf8,0x0c,0x62,0xdc, | |
| 432 | + 0xa4,0x21,0x30,0x4f,0x0f,0x21,0x2c,0xcb, | |
| 433 | + 0xcc,0x37,0xf1,0x1c,0xc3,0xf8,0x2f,0x19, | |
| 434 | + 0xdf,0x53,0x53,0x46,0xb1,0x15,0xea,0x00, | |
| 435 | + } | |
| 436 | + }, | |
| 374 | 437 | { 0xa500011, { |
| 375 | 438 | 0x23,0x3d,0x70,0x7d,0x03,0xc3,0xc4,0xf4, |
| 376 | 439 | 0x2b,0x82,0xc6,0x05,0xda,0x80,0x0a,0xf1, |
| @@ -378,6 +441,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 378 | 441 | 0x11,0x5e,0x96,0x7e,0x71,0xe9,0xfc,0x74, |
| 379 | 442 | } |
| 380 | 443 | }, |
| 444 | + { 0xa500012, { | |
| 445 | + 0xeb,0x74,0x0d,0x47,0xa1,0x8e,0x09,0xe4, | |
| 446 | + 0x93,0x4c,0xad,0x03,0x32,0x4c,0x38,0x16, | |
| 447 | + 0x10,0x39,0xdd,0x06,0xaa,0xce,0xd6,0x0f, | |
| 448 | + 0x62,0x83,0x9d,0x8e,0x64,0x55,0xbe,0x63, | |
| 449 | + } | |
| 450 | + }, | |
| 381 | 451 | { 0xa601209, { |
| 382 | 452 | 0x66,0x48,0xd4,0x09,0x05,0xcb,0x29,0x32, |
| 383 | 453 | 0x66,0xb7,0x9a,0x76,0xcd,0x11,0xf3,0x30, |
| @@ -385,6 +455,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 385 | 455 | 0xe8,0x73,0xe2,0xd6,0xdb,0xd2,0x77,0x1d, |
| 386 | 456 | } |
| 387 | 457 | }, |
| 458 | + { 0xa60120a, { | |
| 459 | + 0x0c,0x8b,0x3d,0xfd,0x52,0x52,0x85,0x7d, | |
| 460 | + 0x20,0x3a,0xe1,0x7e,0xa4,0x21,0x3b,0x7b, | |
| 461 | + 0x17,0x86,0xae,0xac,0x13,0xb8,0x63,0x9d, | |
| 462 | + 0x06,0x01,0xd0,0xa0,0x51,0x9a,0x91,0x2c, | |
| 463 | + } | |
| 464 | + }, | |
| 388 | 465 | { 0xa704107, { |
| 389 | 466 | 0xf3,0xc6,0x58,0x26,0xee,0xac,0x3f,0xd6, |
| 390 | 467 | 0xce,0xa1,0x72,0x47,0x3b,0xba,0x2b,0x93, |
| @@ -392,6 +469,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 392 | 469 | 0x64,0x39,0x71,0x8c,0xce,0xe7,0x41,0x39, |
| 393 | 470 | } |
| 394 | 471 | }, |
| 472 | + { 0xa704108, { | |
| 473 | + 0xd7,0x55,0x15,0x2b,0xfe,0xc4,0xbc,0x93, | |
| 474 | + 0xec,0x91,0xa0,0xae,0x45,0xb7,0xc3,0x98, | |
| 475 | + 0x4e,0xff,0x61,0x77,0x88,0xc2,0x70,0x49, | |
| 476 | + 0xe0,0x3a,0x1d,0x84,0x38,0x52,0xbf,0x5a, | |
| 477 | + } | |
| 478 | + }, | |
| 395 | 479 | { 0xa705206, { |
| 396 | 480 | 0x8d,0xc0,0x76,0xbd,0x58,0x9f,0x8f,0xa4, |
| 397 | 481 | 0x12,0x9d,0x21,0xfb,0x48,0x21,0xbc,0xe7, |
| @@ -399,6 +483,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 399 | 483 | 0x03,0x35,0xe9,0xbe,0xfb,0x06,0xdf,0xfc, |
| 400 | 484 | } |
| 401 | 485 | }, |
| 486 | + { 0xa705208, { | |
| 487 | + 0x30,0x1d,0x55,0x24,0xbc,0x6b,0x5a,0x19, | |
| 488 | + 0x0c,0x7d,0x1d,0x74,0xaa,0xd1,0xeb,0xd2, | |
| 489 | + 0x16,0x62,0xf7,0x5b,0xe1,0x1f,0x18,0x11, | |
| 490 | + 0x5c,0xf0,0x94,0x90,0x26,0xec,0x69,0xff, | |
| 491 | + } | |
| 492 | + }, | |
| 402 | 493 | { 0xa708007, { |
| 403 | 494 | 0x6b,0x76,0xcc,0x78,0xc5,0x8a,0xa3,0xe3, |
| 404 | 495 | 0x32,0x2d,0x79,0xe4,0xc3,0x80,0xdb,0xb2, |
| @@ -406,6 +497,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 406 | 497 | 0xdf,0x92,0x73,0x84,0x87,0x3c,0x73,0x93, |
| 407 | 498 | } |
| 408 | 499 | }, |
| 500 | + { 0xa708008, { | |
| 501 | + 0x08,0x6e,0xf0,0x22,0x4b,0x8e,0xc4,0x46, | |
| 502 | + 0x58,0x34,0xe6,0x47,0xa2,0x28,0xfd,0xab, | |
| 503 | + 0x22,0x3d,0xdd,0xd8,0x52,0x9e,0x1d,0x16, | |
| 504 | + 0xfa,0x01,0x68,0x14,0x79,0x3e,0xe8,0x6b, | |
| 505 | + } | |
| 506 | + }, | |
| 409 | 507 | { 0xa70c005, { |
| 410 | 508 | 0x88,0x5d,0xfb,0x79,0x64,0xd8,0x46,0x3b, |
| 411 | 509 | 0x4a,0x83,0x8e,0x77,0x7e,0xcf,0xb3,0x0f, |
| @@ -413,6 +511,13 @@ static const struct patch_digest phashes[] = { · 1 unmodified line | ||
| 413 | 511 | 0xee,0x49,0xac,0xe1,0x8b,0x13,0xc5,0x13, |
| 414 | 512 | } |
| 415 | 513 | }, |
| 514 | + { 0xa70c008, { | |
| 515 | + 0x0f,0xdb,0x37,0xa1,0x10,0xaf,0xd4,0x21, | |
| 516 | + 0x94,0x0d,0xa4,0xa2,0xe9,0x86,0x6c,0x0e, | |
| 517 | + 0x85,0x7c,0x36,0x30,0xa3,0x3a,0x78,0x66, | |
| 518 | + 0x18,0x10,0x60,0x0d,0x78,0x3d,0x44,0xd0, | |
| 519 | + } | |
| 520 | + }, | |
| 416 | 521 | { 0xaa00116, { |
| 417 | 522 | 0xe8,0x4c,0x2c,0x88,0xa1,0xac,0x24,0x63, |
| 418 | 523 | 0x65,0xe5,0xaa,0x2d,0x16,0xa9,0xc3,0xf5, |
| @@ -441,4 +546,11 @@ static const struct patch_digest phashes[] = { · 22 unmodified lines | ||
| 441 | 546 | 0x68,0x2f,0x46,0xee,0xfe,0xc6,0x6d,0xef, |
| 442 | 547 | } |
| 443 | 548 | }, |
| 549 | + { 0xaa00216, { | |
| 550 | + 0x79,0xfb,0x5b,0x9f,0xb6,0xe6,0xa8,0xf5, | |
| 551 | + 0x4e,0x7c,0x4f,0x8e,0x1d,0xad,0xd0,0x08, | |
| 552 | + 0xc2,0x43,0x7c,0x8b,0xe6,0xdb,0xd0,0xd2, | |
| 553 | + 0xe8,0x39,0x26,0xc1,0xe5,0x5a,0x48,0xf1, | |
| 554 | + } | |
| 555 | + }, | |
| 444 | 556 | }; |
| @@ -50,6 +50,8 @@ static const struct cpuid_bit cpuid_bits[] = { · 49 unmodified lines | ||
| 50 | 50 | { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, |
| 51 | 51 | { X86_FEATURE_SMBA, CPUID_EBX, 2, 0x80000020, 0 }, |
| 52 | 52 | { X86_FEATURE_BMEC, CPUID_EBX, 3, 0x80000020, 0 }, |
| 53 | + { X86_FEATURE_TSA_SQ_NO, CPUID_ECX, 1, 0x80000021, 0 }, | |
| 54 | + { X86_FEATURE_TSA_L1_NO, CPUID_ECX, 2, 0x80000021, 0 }, | |
| 53 | 55 | { X86_FEATURE_AMD_WORKLOAD_CLASS, CPUID_EAX, 22, 0x80000021, 0 }, |
| 54 | 56 | { X86_FEATURE_PERFMON_V2, CPUID_EAX, 0, 0x80000022, 0 }, |
| 55 | 57 | { X86_FEATURE_AMD_LBR_V2, CPUID_EAX, 1, 0x80000022, 0 }, |
| @@ -907,16 +907,24 @@ static __init bool prefer_mwait_c1_over_halt(void) · 906 unmodified lines | ||
| 907 | 907 | */ |
| 908 | 908 | static __cpuidle void mwait_idle(void) |
| 909 | 909 | { |
| 910 | + if (need_resched()) | |
| 911 | + return; | |
| 912 | + | |
| 913 | + x86_idle_clear_cpu_buffers(); | |
| 914 | + | |
| 910 | 915 | if (!current_set_polling_and_test()) { |
| 911 | 916 | const void *addr = ¤t_thread_info()->flags; |
| 912 | 917 | |
| 913 | 918 | alternative_input("", "clflush (%[addr])", X86_BUG_CLFLUSH_MONITOR, [addr] "a" (addr)); |
| 914 | 919 | __monitor(addr, 0, 0); |
| 915 | − if (!need_resched()) { | |
| 916 | − __sti_mwait(0, 0); | |
| 917 | − raw_local_irq_disable(); | |
| 918 | − } | |
| 920 | + if (need_resched()) | |
| 921 | + goto out; | |
| 922 | + | |
| 923 | + __sti_mwait(0, 0); | |
| 924 | + raw_local_irq_disable(); | |
| 919 | 925 | } |
| 926 | + | |
| 927 | +out: | |
| 920 | 928 | __current_clr_polling(); |
| 921 | 929 | } |
| 922 | 930 | |
| @@ -1165,6 +1165,8 @@ void kvm_set_cpu_caps(void) · 1164 unmodified lines | ||
| 1165 | 1165 | */ |
| 1166 | 1166 | SYNTHESIZED_F(LFENCE_RDTSC), |
| 1167 | 1167 | /* SmmPgCfgLock */ |
| 1168 | + /* 4: Resv */ | |
| 1169 | + SYNTHESIZED_F(VERW_CLEAR), | |
| 1168 | 1170 | F(NULL_SEL_CLR_BASE), |
| 1169 | 1171 | /* UpperAddressIgnore */ |
| 1170 | 1172 | F(AUTOIBRS), |
| @@ -1179,6 +1181,11 @@ void kvm_set_cpu_caps(void) · 8 unmodified lines | ||
| 1179 | 1181 | F(SRSO_USER_KERNEL_NO), |
| 1180 | 1182 | ); |
| 1181 | 1183 | |
| 1184 | + kvm_cpu_cap_init(CPUID_8000_0021_ECX, | |
| 1185 | + SYNTHESIZED_F(TSA_SQ_NO), | |
| 1186 | + SYNTHESIZED_F(TSA_L1_NO), | |
| 1187 | + ); | |
| 1188 | + | |
| 1182 | 1189 | kvm_cpu_cap_init(CPUID_8000_0022_EAX, |
| 1183 | 1190 | F(PERFMON_V2), |
| 1184 | 1191 | ); |
| @@ -1748,8 +1755,9 @@ static inline int __do_cpuid_func(struct kvm_cpuid_array *array, u32 function) · 563 unmodified lines | ||
| 1748 | 1755 | entry->eax = entry->ebx = entry->ecx = entry->edx = 0; |
| 1749 | 1756 | break; |
| 1750 | 1757 | case 0x80000021: |
| 1751 | − entry->ebx = entry->ecx = entry->edx = 0; | |
| 1758 | + entry->ebx = entry->edx = 0; | |
| 1752 | 1759 | cpuid_entry_override(entry, CPUID_8000_0021_EAX); |
| 1760 | + cpuid_entry_override(entry, CPUID_8000_0021_ECX); | |
| 1753 | 1761 | break; |
| 1754 | 1762 | /* AMD Extended Performance Monitoring and Debug */ |
| 1755 | 1763 | case 0x80000022: { |
| @@ -1979,6 +1979,9 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) · 1978 unmodified lines | ||
| 1979 | 1979 | if (entries[i] == KVM_HV_TLB_FLUSHALL_ENTRY) |
| 1980 | 1980 | goto out_flush_all; |
| 1981 | 1981 | |
| 1982 | + if (is_noncanonical_invlpg_address(entries[i], vcpu)) | |
| 1983 | + continue; | |
| 1984 | + | |
| 1982 | 1985 | /* |
| 1983 | 1986 | * Lower 12 bits of 'address' encode the number of additional |
| 1984 | 1987 | * pages to flush. |
| @@ -2001,11 +2004,11 @@ out_flush_all: · 16 unmodified lines | ||
| 2001 | 2004 | static u64 kvm_hv_flush_tlb(struct kvm_vcpu *vcpu, struct kvm_hv_hcall *hc) |
| 2002 | 2005 | { |
| 2003 | 2006 | struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(vcpu); |
| 2007 | + unsigned long *vcpu_mask = hv_vcpu->vcpu_mask; | |
| 2004 | 2008 | u64 *sparse_banks = hv_vcpu->sparse_banks; |
| 2005 | 2009 | struct kvm *kvm = vcpu->kvm; |
| 2006 | 2010 | struct hv_tlb_flush_ex flush_ex; |
| 2007 | 2011 | struct hv_tlb_flush flush; |
| 2008 | − DECLARE_BITMAP(vcpu_mask, KVM_MAX_VCPUS); | |
| 2009 | 2012 | struct kvm_vcpu_hv_tlb_flush_fifo *tlb_flush_fifo; |
| 2010 | 2013 | /* |
| 2011 | 2014 | * Normally, there can be no more than 'KVM_HV_TLB_FLUSH_FIFO_SIZE' |
| @@ -52,6 +52,10 @@ · 51 unmodified lines | ||
| 52 | 52 | /* CPUID level 0x80000022 (EAX) */ |
| 53 | 53 | #define KVM_X86_FEATURE_PERFMON_V2 KVM_X86_FEATURE(CPUID_8000_0022_EAX, 0) |
| 54 | 54 | |
| 55 | +/* CPUID level 0x80000021 (ECX) */ | |
| 56 | +#define KVM_X86_FEATURE_TSA_SQ_NO KVM_X86_FEATURE(CPUID_8000_0021_ECX, 1) | |
| 57 | +#define KVM_X86_FEATURE_TSA_L1_NO KVM_X86_FEATURE(CPUID_8000_0021_ECX, 2) | |
| 58 | + | |
| 55 | 59 | struct cpuid_reg { |
| 56 | 60 | u32 function; |
| 57 | 61 | u32 index; |
| @@ -82,6 +86,7 @@ static const struct cpuid_reg reverse_cpuid[] = { · 24 unmodified lines | ||
| 82 | 86 | [CPUID_8000_0022_EAX] = {0x80000022, 0, CPUID_EAX}, |
| 83 | 87 | [CPUID_7_2_EDX] = { 7, 2, CPUID_EDX}, |
| 84 | 88 | [CPUID_24_0_EBX] = { 0x24, 0, CPUID_EBX}, |
| 89 | + [CPUID_8000_0021_ECX] = {0x80000021, 0, CPUID_ECX}, | |
| 85 | 90 | }; |
| 86 | 91 | |
| 87 | 92 | /* |
| @@ -121,6 +126,8 @@ static __always_inline u32 __feature_translate(int x86_feature) · 33 unmodified lines | ||
| 121 | 126 | KVM_X86_TRANSLATE_FEATURE(PERFMON_V2); |
| 122 | 127 | KVM_X86_TRANSLATE_FEATURE(RRSBA_CTRL); |
| 123 | 128 | KVM_X86_TRANSLATE_FEATURE(BHI_CTRL); |
| 129 | + KVM_X86_TRANSLATE_FEATURE(TSA_SQ_NO); | |
| 130 | + KVM_X86_TRANSLATE_FEATURE(TSA_L1_NO); | |
| 124 | 131 | default: |
| 125 | 132 | return x86_feature; |
| 126 | 133 | } |
| @@ -1971,6 +1971,10 @@ static int sev_check_source_vcpus(struct kvm *dst, struct kvm *src) · 1970 unmodified lines | ||
| 1971 | 1971 | struct kvm_vcpu *src_vcpu; |
| 1972 | 1972 | unsigned long i; |
| 1973 | 1973 | |
| 1974 | + if (src->created_vcpus != atomic_read(&src->online_vcpus) || | |
| 1975 | + dst->created_vcpus != atomic_read(&dst->online_vcpus)) | |
| 1976 | + return -EBUSY; | |
| 1977 | + | |
| 1974 | 1978 | if (!sev_es_guest(src)) |
| 1975 | 1979 | return 0; |
| 1976 | 1980 | |
| @@ -4445,8 +4449,12 @@ static void sev_es_init_vmcb(struct vcpu_svm *svm) · 2468 unmodified lines | ||
| 4445 | 4449 | * the VMSA will be NULL if this vCPU is the destination for intrahost |
| 4446 | 4450 | * migration, and will be copied later. |
| 4447 | 4451 | */ |
| 4448 | − if (svm->sev_es.vmsa && !svm->sev_es.snp_has_guest_vmsa) | |
| 4449 | − svm->vmcb->control.vmsa_pa = __pa(svm->sev_es.vmsa); | |
| 4452 | + if (!svm->sev_es.snp_has_guest_vmsa) { | |
| 4453 | + if (svm->sev_es.vmsa) | |
| 4454 | + svm->vmcb->control.vmsa_pa = __pa(svm->sev_es.vmsa); | |
| 4455 | + else | |
| 4456 | + svm->vmcb->control.vmsa_pa = INVALID_PAGE; | |
| 4457 | + } | |
| 4450 | 4458 | |
| 4451 | 4459 | if (cpu_feature_enabled(X86_FEATURE_ALLOWED_SEV_FEATURES)) |
| 4452 | 4460 | svm->vmcb->control.allowed_sev_features = sev->vmsa_features | |
| @@ -169,6 +169,9 @@ SYM_FUNC_START(__svm_vcpu_run) · 168 unmodified lines | ||
| 169 | 169 | #endif |
| 170 | 170 | mov VCPU_RDI(%_ASM_DI), %_ASM_DI |
| 171 | 171 | |
| 172 | + /* Clobbers EFLAGS.ZF */ | |
| 173 | + VM_CLEAR_CPU_BUFFERS | |
| 174 | + | |
| 172 | 175 | /* Enter guest mode */ |
| 173 | 176 | 3: vmrun %_ASM_AX |
| 174 | 177 | 4: |
| @@ -335,6 +338,9 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) · 160 unmodified lines | ||
| 335 | 338 | mov SVM_current_vmcb(%rdi), %rax |
| 336 | 339 | mov KVM_VMCB_pa(%rax), %rax |
| 337 | 340 | |
| 341 | + /* Clobbers EFLAGS.ZF */ | |
| 342 | + VM_CLEAR_CPU_BUFFERS | |
| 343 | + | |
| 338 | 344 | /* Enter guest mode */ |
| 339 | 345 | 1: vmrun %rax |
| 340 | 346 | 2: |
| @@ -173,6 +173,9 @@ static void td_init_cpuid_entry2(struct kvm_cpuid_entry2 *entry, unsigned char i · 172 unmodified lines | ||
| 173 | 173 | tdx_clear_unsupported_cpuid(entry); |
| 174 | 174 | } |
| 175 | 175 | |
| 176 | +#define TDVMCALLINFO_GET_QUOTE BIT(0) | |
| 177 | +#define TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT BIT(1) | |
| 178 | + | |
| 176 | 179 | static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf, |
| 177 | 180 | struct kvm_tdx_capabilities *caps) |
| 178 | 181 | { |
| @@ -188,6 +191,10 @@ static int init_kvm_tdx_caps(const struct tdx_sys_info_td_conf *td_conf, · 9 unmodified lines | ||
| 188 | 191 | |
| 189 | 192 | caps->cpuid.nent = td_conf->num_cpuid_config; |
| 190 | 193 | |
| 194 | + caps->user_tdvmcallinfo_1_r11 = | |
| 195 | + TDVMCALLINFO_GET_QUOTE | | |
| 196 | + TDVMCALLINFO_SETUP_EVENT_NOTIFY_INTERRUPT; | |
| 197 | + | |
| 191 | 198 | for (i = 0; i < td_conf->num_cpuid_config; i++) |
| 192 | 199 | td_init_cpuid_entry2(&caps->cpuid.entries[i], i); |
| 193 | 200 | |
| @@ -1530,6 +1537,27 @@ static int tdx_get_quote(struct kvm_vcpu *vcpu) · 1336 unmodified lines | ||
| 1530 | 1537 | return 0; |
| 1531 | 1538 | } |
| 1532 | 1539 | |
| 1540 | +static int tdx_setup_event_notify_interrupt(struct kvm_vcpu *vcpu) | |
| 1541 | +{ | |
| 1542 | + struct vcpu_tdx *tdx = to_tdx(vcpu); | |
| 1543 | + u64 vector = tdx->vp_enter_args.r12; | |
| 1544 | + | |
| 1545 | + if (vector < 32 || vector > 255) { | |
| 1546 | + tdvmcall_set_return_code(vcpu, TDVMCALL_STATUS_INVALID_OPERAND); | |
| 1547 | + return 1; | |
| 1548 | + } | |
| 1549 | + | |
| 1550 | + vcpu->run->exit_reason = KVM_EXIT_TDX; | |
| 1551 | + vcpu->run->tdx.flags = 0; | |
| 1552 | + vcpu->run->tdx.nr = TDVMCALL_SETUP_EVENT_NOTIFY_INTERRUPT; | |
| 1553 | + vcpu->run->tdx.setup_event_notify.ret = TDVMCALL_STATUS_SUBFUNC_UNSUPPORTED; | |
| 1554 | + vcpu->run->tdx.setup_event_notify.vector = vector; | |
| 1555 | + | |
| 1556 | + vcpu->arch.complete_userspace_io = tdx_complete_simple; | |
| 1557 | + | |
| 1558 | + return 0; | |
| 1559 | +} | |
| 1560 | + | |
| 1533 | 1561 | static int handle_tdvmcall(struct kvm_vcpu *vcpu) |
| 1534 | 1562 | { |
| 1535 | 1563 | switch (tdvmcall_leaf(vcpu)) { |
| @@ -1541,6 +1569,8 @@ static int handle_tdvmcall(struct kvm_vcpu *vcpu) · 5 unmodified lines | ||
| 1541 | 1569 | return tdx_get_td_vm_call_info(vcpu); |
| 1542 | 1570 | case TDVMCALL_GET_QUOTE: |
| 1543 | 1571 | return tdx_get_quote(vcpu); |
| 1572 | + case TDVMCALL_SETUP_EVENT_NOTIFY_INTERRUPT: | |
| 1573 | + return tdx_setup_event_notify_interrupt(vcpu); | |
| 1544 | 1574 | default: |
| 1545 | 1575 | break; |
| 1546 | 1576 | } |
| @@ -7291,7 +7291,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, · 7290 unmodified lines | ||
| 7291 | 7291 | vmx_l1d_flush(vcpu); |
| 7292 | 7292 | else if (static_branch_unlikely(&cpu_buf_vm_clear) && |
| 7293 | 7293 | kvm_arch_has_assigned_device(vcpu->kvm)) |
| 7294 | − mds_clear_cpu_buffers(); | |
| 7294 | + x86_clear_cpu_buffers(); | |
| 7295 | 7295 | |
| 7296 | 7296 | vmx_disable_fb_clear(vmx); |
| 7297 | 7297 | |
| @@ -3258,9 +3258,11 @@ int kvm_guest_time_update(struct kvm_vcpu *v) · 3257 unmodified lines | ||
| 3258 | 3258 | |
| 3259 | 3259 | /* With all the info we got, fill in the values */ |
| 3260 | 3260 | |
| 3261 | − if (kvm_caps.has_tsc_control) | |
| 3261 | + if (kvm_caps.has_tsc_control) { | |
| 3262 | 3262 | tgt_tsc_khz = kvm_scale_tsc(tgt_tsc_khz, |
| 3263 | 3263 | v->arch.l1_tsc_scaling_ratio); |
| 3264 | + tgt_tsc_khz = tgt_tsc_khz ? : 1; | |
| 3265 | + } | |
| 3264 | 3266 | |
| 3265 | 3267 | if (unlikely(vcpu->hw_tsc_khz != tgt_tsc_khz)) { |
| 3266 | 3268 | kvm_get_time_scale(NSEC_PER_SEC, tgt_tsc_khz * 1000LL, |
| @@ -1971,8 +1971,19 @@ int kvm_xen_setup_evtchn(struct kvm *kvm, · 1970 unmodified lines | ||
| 1971 | 1971 | { |
| 1972 | 1972 | struct kvm_vcpu *vcpu; |
| 1973 | 1973 | |
| 1974 | − if (ue->u.xen_evtchn.port >= max_evtchn_port(kvm)) | |
| 1975 | − return -EINVAL; | |
| 1974 | + /* | |
| 1975 | + * Don't check for the port being within range of max_evtchn_port(). | |
| 1976 | + * Userspace can configure what ever targets it likes; events just won't | |
| 1977 | + * be delivered if/while the target is invalid, just like userspace can | |
| 1978 | + * configure MSIs which target non-existent APICs. | |
| 1979 | + * | |
| 1980 | + * This allow on Live Migration and Live Update, the IRQ routing table | |
| 1981 | + * can be restored *independently* of other things like creating vCPUs, | |
| 1982 | + * without imposing an ordering dependency on userspace. In this | |
| 1983 | + * particular case, the problematic ordering would be with setting the | |
| 1984 | + * Xen 'long mode' flag, which changes max_evtchn_port() to allow 4096 | |
| 1985 | + * instead of 1024 event channels. | |
| 1986 | + */ | |
| 1976 | 1987 | |
| 1977 | 1988 | /* We only support 2 level event channels for now */ |
| 1978 | 1989 | if (ue->u.xen_evtchn.priority != KVM_IRQ_ROUTING_XEN_EVTCHN_PRIO_2LEVEL) |
| @@ -602,6 +602,7 @@ CPU_SHOW_VULN_FALLBACK(reg_file_data_sampling); · 601 unmodified lines | ||
| 602 | 602 | CPU_SHOW_VULN_FALLBACK(ghostwrite); |
| 603 | 603 | CPU_SHOW_VULN_FALLBACK(old_microcode); |
| 604 | 604 | CPU_SHOW_VULN_FALLBACK(indirect_target_selection); |
| 605 | +CPU_SHOW_VULN_FALLBACK(tsa); | |
| 605 | 606 | |
| 606 | 607 | static DEVICE_ATTR(meltdown, 0444, cpu_show_meltdown, NULL); |
| 607 | 608 | static DEVICE_ATTR(spectre_v1, 0444, cpu_show_spectre_v1, NULL); |
| @@ -620,6 +621,7 @@ static DEVICE_ATTR(reg_file_data_sampling, 0444, cpu_show_reg_file_data_sampling · 12 unmodified lines | ||
| 620 | 621 | static DEVICE_ATTR(ghostwrite, 0444, cpu_show_ghostwrite, NULL); |
| 621 | 622 | static DEVICE_ATTR(old_microcode, 0444, cpu_show_old_microcode, NULL); |
| 622 | 623 | static DEVICE_ATTR(indirect_target_selection, 0444, cpu_show_indirect_target_selection, NULL); |
| 624 | +static DEVICE_ATTR(tsa, 0444, cpu_show_tsa, NULL); | |
| 623 | 625 | |
| 624 | 626 | static struct attribute *cpu_root_vulnerabilities_attrs[] = { |
| 625 | 627 | &dev_attr_meltdown.attr, |
| @@ -639,6 +641,7 @@ static struct attribute *cpu_root_vulnerabilities_attrs[] = { · 13 unmodified lines | ||
| 639 | 641 | &dev_attr_ghostwrite.attr, |
| 640 | 642 | &dev_attr_old_microcode.attr, |
| 641 | 643 | &dev_attr_indirect_target_selection.attr, |
| 644 | + &dev_attr_tsa.attr, | |
| 642 | 645 | NULL |
| 643 | 646 | }; |
| 644 | 647 | |
| @@ -1236,8 +1236,8 @@ void dpm_complete(pm_message_t state) · 1235 unmodified lines | ||
| 1236 | 1236 | */ |
| 1237 | 1237 | void dpm_resume_end(pm_message_t state) |
| 1238 | 1238 | { |
| 1239 | − pm_restore_gfp_mask(); | |
| 1240 | 1239 | dpm_resume(state); |
| 1240 | + pm_restore_gfp_mask(); | |
| 1241 | 1241 | dpm_complete(state); |
| 1242 | 1242 | } |
| 1243 | 1243 | EXPORT_SYMBOL_GPL(dpm_resume_end); |
| @@ -2198,9 +2198,7 @@ again: · 2197 unmodified lines | ||
| 2198 | 2198 | goto out; |
| 2199 | 2199 | } |
| 2200 | 2200 | } |
| 2201 | − ret = nbd_start_device(nbd); | |
| 2202 | − if (ret) | |
| 2203 | − goto out; | |
| 2201 | + | |
| 2204 | 2202 | if (info->attrs[NBD_ATTR_BACKEND_IDENTIFIER]) { |
| 2205 | 2203 | nbd->backend = nla_strdup(info->attrs[NBD_ATTR_BACKEND_IDENTIFIER], |
| 2206 | 2204 | GFP_KERNEL); |
| @@ -2216,6 +2214,8 @@ again: · 9 unmodified lines | ||
| 2216 | 2214 | goto out; |
| 2217 | 2215 | } |
| 2218 | 2216 | set_bit(NBD_RT_HAS_BACKEND_FILE, &config->runtime_flags); |
| 2217 | + | |
| 2218 | + ret = nbd_start_device(nbd); | |
| 2219 | 2219 | out: |
| 2220 | 2220 | mutex_unlock(&nbd->config_lock); |
| 2221 | 2221 | if (!ret) { |
| @@ -720,11 +720,6 @@ static const struct pci_device_id agp_amd64_pci_table[] = { · 719 unmodified lines | ||
| 720 | 720 | |
| 721 | 721 | MODULE_DEVICE_TABLE(pci, agp_amd64_pci_table); |
| 722 | 722 | |
| 723 | −static const struct pci_device_id agp_amd64_pci_promisc_table[] = { | |
| 724 | − { PCI_DEVICE_CLASS(0, 0) }, | |
| 725 | − { } | |
| 726 | −}; | |
| 727 | − | |
| 728 | 723 | static DEFINE_SIMPLE_DEV_PM_OPS(agp_amd64_pm_ops, NULL, agp_amd64_resume); |
| 729 | 724 | |
| 730 | 725 | static struct pci_driver agp_amd64_pci_driver = { |
| @@ -739,6 +734,7 @@ static struct pci_driver agp_amd64_pci_driver = { · 8 unmodified lines | ||
| 739 | 734 | /* Not static due to IOMMU code calling it early. */ |
| 740 | 735 | int __init agp_amd64_init(void) |
| 741 | 736 | { |
| 737 | + struct pci_dev *pdev = NULL; | |
| 742 | 738 | int err = 0; |
| 743 | 739 | |
| 744 | 740 | if (agp_off) |
| @@ -767,9 +763,13 @@ int __init agp_amd64_init(void) · 22 unmodified lines | ||
| 767 | 763 | } |
| 768 | 764 | |
| 769 | 765 | /* Look for any AGP bridge */ |
| 770 | − agp_amd64_pci_driver.id_table = agp_amd64_pci_promisc_table; | |
| 771 | − err = driver_attach(&agp_amd64_pci_driver.driver); | |
| 772 | − if (err == 0 && agp_bridges_found == 0) { | |
| 766 | + for_each_pci_dev(pdev) | |
| 767 | + if (pci_find_capability(pdev, PCI_CAP_ID_AGP)) | |
| 768 | + pci_add_dynid(&agp_amd64_pci_driver, | |
| 769 | + pdev->vendor, pdev->device, | |
| 770 | + pdev->subsystem_vendor, | |
| 771 | + pdev->subsystem_device, 0, 0, 0); | |
| 772 | + if (agp_bridges_found == 0) { | |
| 773 | 773 | pci_unregister_driver(&agp_amd64_pci_driver); |
| 774 | 774 | err = -ENODEV; |
| 775 | 775 | } |
| @@ -404,6 +404,7 @@ static int scmi_clocks_probe(struct scmi_device *sdev) · 403 unmodified lines | ||
| 404 | 404 | const struct scmi_handle *handle = sdev->handle; |
| 405 | 405 | struct scmi_protocol_handle *ph; |
| 406 | 406 | const struct clk_ops *scmi_clk_ops_db[SCMI_MAX_CLK_OPS] = {}; |
| 407 | + struct scmi_clk *sclks; | |
| 407 | 408 | |
| 408 | 409 | if (!handle) |
| 409 | 410 | return -ENODEV; |
| @@ -430,18 +431,21 @@ static int scmi_clocks_probe(struct scmi_device *sdev) · 20 unmodified lines | ||
| 430 | 431 | transport_is_atomic = handle->is_transport_atomic(handle, |
| 431 | 432 | &atomic_threshold_us); |
| 432 | 433 | |
| 434 | + sclks = devm_kcalloc(dev, count, sizeof(*sclks), GFP_KERNEL); | |
| 435 | + if (!sclks) | |
| 436 | + return -ENOMEM; | |
| 437 | + | |
| 438 | + for (idx = 0; idx < count; idx++) | |
| 439 | + hws[idx] = &sclks[idx].hw; | |
| 440 | + | |
| 433 | 441 | for (idx = 0; idx < count; idx++) { |
| 434 | − struct scmi_clk *sclk; | |
| 442 | + struct scmi_clk *sclk = &sclks[idx]; | |
| 435 | 443 | const struct clk_ops *scmi_ops; |
| 436 | 444 | |
| 437 | − sclk = devm_kzalloc(dev, sizeof(*sclk), GFP_KERNEL); | |
| 438 | − if (!sclk) | |
| 439 | − return -ENOMEM; | |
| 440 | − | |
| 441 | 445 | sclk->info = scmi_proto_clk_ops->info_get(ph, idx); |
| 442 | 446 | if (!sclk->info) { |
| 443 | 447 | dev_dbg(dev, "invalid clock info for idx %d\n", idx); |
| 444 | − devm_kfree(dev, sclk); | |
| 448 | + hws[idx] = NULL; | |
| 445 | 449 | continue; |
| 446 | 450 | } |
| 447 | 451 | |
| @@ -479,13 +483,11 @@ static int scmi_clocks_probe(struct scmi_device *sdev) · 31 unmodified lines | ||
| 479 | 483 | if (err) { |
| 480 | 484 | dev_err(dev, "failed to register clock %d\n", idx); |
| 481 | 485 | devm_kfree(dev, sclk->parent_data); |
| 482 | − devm_kfree(dev, sclk); | |
| 483 | 486 | hws[idx] = NULL; |
| 484 | 487 | } else { |
| 485 | 488 | dev_dbg(dev, "Registered clock:%s%s\n", |
| 486 | 489 | sclk->info->name, |
| 487 | 490 | scmi_ops->enable ? " (atomic ops)" : ""); |
| 488 | − hws[idx] = &sclk->hw; | |
| 489 | 491 | } |
| 490 | 492 | } |
| 491 | 493 | |
| @@ -219,11 +219,15 @@ static const struct imx95_blk_ctl_dev_data lvds_csr_dev_data = { · 218 unmodified lines | ||
| 219 | 219 | .clk_reg_offset = 0, |
| 220 | 220 | }; |
| 221 | 221 | |
| 222 | +static const char * const disp_engine_parents[] = { | |
| 223 | + "videopll1", "dsi_pll", "ldb_pll_div7" | |
| 224 | +}; | |
| 225 | + | |
| 222 | 226 | static const struct imx95_blk_ctl_clk_dev_data dispmix_csr_clk_dev_data[] = { |
| 223 | 227 | [IMX95_CLK_DISPMIX_ENG0_SEL] = { |
| 224 | 228 | .name = "disp_engine0_sel", |
| 225 | − .parent_names = (const char *[]){"videopll1", "dsi_pll", "ldb_pll_div7", }, | |
| 226 | − .num_parents = 4, | |
| 229 | + .parent_names = disp_engine_parents, | |
| 230 | + .num_parents = ARRAY_SIZE(disp_engine_parents), | |
| 227 | 231 | .reg = 0, |
| 228 | 232 | .bit_idx = 0, |
| 229 | 233 | .bit_width = 2, |
| @@ -232,8 +236,8 @@ static const struct imx95_blk_ctl_clk_dev_data dispmix_csr_clk_dev_data[] = { · 2 unmodified lines | ||
| 232 | 236 | }, |
| 233 | 237 | [IMX95_CLK_DISPMIX_ENG1_SEL] = { |
| 234 | 238 | .name = "disp_engine1_sel", |
| 235 | − .parent_names = (const char *[]){"videopll1", "dsi_pll", "ldb_pll_div7", }, | |
| 236 | − .num_parents = 4, | |
| 239 | + .parent_names = disp_engine_parents, | |
| 240 | + .num_parents = ARRAY_SIZE(disp_engine_parents), | |
| 237 | 241 | .reg = 0, |
| 238 | 242 | .bit_idx = 2, |
| 239 | 243 | .bit_width = 2, |
| @@ -29,14 +29,12 @@ SECTIONS · 28 unmodified lines | ||
| 29 | 29 | . = _etext; |
| 30 | 30 | } |
| 31 | 31 | |
| 32 | −#ifdef CONFIG_EFI_SBAT | |
| 33 | 32 | .sbat : ALIGN(4096) { |
| 34 | 33 | _sbat = .; |
| 35 | 34 | *(.sbat) |
| 36 | 35 | _esbat = ALIGN(4096); |
| 37 | 36 | . = _esbat; |
| 38 | 37 | } |
| 39 | −#endif | |
| 40 | 38 | |
| 41 | 39 | .data : ALIGN(4096) { |
| 42 | 40 | _data = .; |
| @@ -60,6 +58,6 @@ SECTIONS · 17 unmodified lines | ||
| 60 | 58 | PROVIDE(__efistub__gzdata_size = |
| 61 | 59 | ABSOLUTE(__efistub__gzdata_end - __efistub__gzdata_start)); |
| 62 | 60 | |
| 63 | −PROVIDE(__data_rawsize = ABSOLUTE(_edata - _etext)); | |
| 64 | −PROVIDE(__data_size = ABSOLUTE(_end - _etext)); | |
| 61 | +PROVIDE(__data_rawsize = ABSOLUTE(_edata - _data)); | |
| 62 | +PROVIDE(__data_size = ABSOLUTE(_end - _data)); | |
| 65 | 63 | PROVIDE(__sbat_size = ABSOLUTE(_esbat - _sbat)); |
| @@ -708,7 +708,7 @@ struct gpio_desc *of_find_gpio(struct device_node *np, const char *con_id, · 707 unmodified lines | ||
| 708 | 708 | unsigned int idx, unsigned long *flags) |
| 709 | 709 | { |
| 710 | 710 | char propname[32]; /* 32 is max size of property name */ |
| 711 | − enum of_gpio_flags of_flags; | |
| 711 | + enum of_gpio_flags of_flags = 0; | |
| 712 | 712 | const of_find_gpio_quirk *q; |
| 713 | 713 | struct gpio_desc *desc; |
| 714 | 714 | |
| @@ -3297,14 +3297,15 @@ static int gpiod_get_raw_value_commit(const struct gpio_desc *desc) · 3296 unmodified lines | ||
| 3297 | 3297 | static int gpio_chip_get_multiple(struct gpio_chip *gc, |
| 3298 | 3298 | unsigned long *mask, unsigned long *bits) |
| 3299 | 3299 | { |
| 3300 | − int ret; | |
| 3301 | − | |
| 3302 | 3300 | lockdep_assert_held(&gc->gpiodev->srcu); |
| 3303 | 3301 | |
| 3304 | 3302 | if (gc->get_multiple) { |
| 3303 | + int ret; | |
| 3304 | + | |
| 3305 | 3305 | ret = gc->get_multiple(gc, mask, bits); |
| 3306 | 3306 | if (ret > 0) |
| 3307 | 3307 | return -EBADE; |
| 3308 | + return ret; | |
| 3308 | 3309 | } |
| 3309 | 3310 | |
| 3310 | 3311 | if (gc->get) { |
| @@ -862,11 +862,23 @@ EXPORT_SYMBOL_FOR_TESTS_ONLY(drm_framebuffer_free); · 861 unmodified lines | ||
| 862 | 862 | int drm_framebuffer_init(struct drm_device *dev, struct drm_framebuffer *fb, |
| 863 | 863 | const struct drm_framebuffer_funcs *funcs) |
| 864 | 864 | { |
| 865 | + unsigned int i; | |
| 865 | 866 | int ret; |
| 867 | + bool exists; | |
| 866 | 868 | |
| 867 | 869 | if (WARN_ON_ONCE(fb->dev != dev || !fb->format)) |
| 868 | 870 | return -EINVAL; |
| 869 | 871 | |
| 872 | + for (i = 0; i < fb->format->num_planes; i++) { | |
| 873 | + if (drm_WARN_ON_ONCE(dev, fb->internal_flags & DRM_FRAMEBUFFER_HAS_HANDLE_REF(i))) | |
| 874 | + fb->internal_flags &= ~DRM_FRAMEBUFFER_HAS_HANDLE_REF(i); | |
| 875 | + if (fb->obj[i]) { | |
| 876 | + exists = drm_gem_object_handle_get_if_exists_unlocked(fb->obj[i]); | |
| 877 | + if (exists) | |
| 878 | + fb->internal_flags |= DRM_FRAMEBUFFER_HAS_HANDLE_REF(i); | |
| 879 | + } | |
| 880 | + } | |
| 881 | + | |
| 870 | 882 | INIT_LIST_HEAD(&fb->filp_head); |
| 871 | 883 | |
| 872 | 884 | fb->funcs = funcs; |
| @@ -875,7 +887,7 @@ int drm_framebuffer_init(struct drm_device *dev, struct drm_framebuffer *fb, · 2 unmodified lines | ||
| 875 | 887 | ret = __drm_mode_object_add(dev, &fb->base, DRM_MODE_OBJECT_FB, |
| 876 | 888 | false, drm_framebuffer_free); |
| 877 | 889 | if (ret) |
| 878 | − goto out; | |
| 890 | + goto err; | |
| 879 | 891 | |
| 880 | 892 | mutex_lock(&dev->mode_config.fb_lock); |
| 881 | 893 | dev->mode_config.num_fb++; |
| @@ -883,7 +895,16 @@ int drm_framebuffer_init(struct drm_device *dev, struct drm_framebuffer *fb, · 1 unmodified line | ||
| 883 | 895 | mutex_unlock(&dev->mode_config.fb_lock); |
| 884 | 896 | |
| 885 | 897 | drm_mode_object_register(dev, &fb->base); |
| 886 | −out: | |
| 898 | + | |
| 899 | + return 0; | |
| 900 | + | |
| 901 | +err: | |
| 902 | + for (i = 0; i < fb->format->num_planes; i++) { | |
| 903 | + if (fb->internal_flags & DRM_FRAMEBUFFER_HAS_HANDLE_REF(i)) { | |
| 904 | + drm_gem_object_handle_put_unlocked(fb->obj[i]); | |
| 905 | + fb->internal_flags &= ~DRM_FRAMEBUFFER_HAS_HANDLE_REF(i); | |
| 906 | + } | |
| 907 | + } | |
| 887 | 908 | return ret; |
| 888 | 909 | } |
| 889 | 910 | EXPORT_SYMBOL(drm_framebuffer_init); |
| @@ -960,6 +981,12 @@ EXPORT_SYMBOL(drm_framebuffer_unregister_private); · 70 unmodified lines | ||
| 960 | 981 | void drm_framebuffer_cleanup(struct drm_framebuffer *fb) |
| 961 | 982 | { |
| 962 | 983 | struct drm_device *dev = fb->dev; |
| 984 | + unsigned int i; | |
| 985 | + | |
| 986 | + for (i = 0; i < fb->format->num_planes; i++) { | |
| 987 | + if (fb->internal_flags & DRM_FRAMEBUFFER_HAS_HANDLE_REF(i)) | |
| 988 | + drm_gem_object_handle_put_unlocked(fb->obj[i]); | |
| 989 | + } | |
| 963 | 990 | |
| 964 | 991 | mutex_lock(&dev->mode_config.fb_lock); |
| 965 | 992 | list_del(&fb->head); |
| @@ -223,23 +223,34 @@ static void drm_gem_object_handle_get(struct drm_gem_object *obj) · 222 unmodified lines | ||
| 223 | 223 | } |
| 224 | 224 | |
| 225 | 225 | /** |
| 226 | − * drm_gem_object_handle_get_unlocked - acquire reference on user-space handles | |
| 226 | + * drm_gem_object_handle_get_if_exists_unlocked - acquire reference on user-space handle, if any | |
| 227 | 227 | * @obj: GEM object |
| 228 | 228 | * |
| 229 | − * Acquires a reference on the GEM buffer object's handle. Required | |
| 230 | − * to keep the GEM object alive. Call drm_gem_object_handle_put_unlocked() | |
| 231 | − * to release the reference. | |
| 229 | + * Acquires a reference on the GEM buffer object's handle. Required to keep | |
| 230 | + * the GEM object alive. Call drm_gem_object_handle_put_if_exists_unlocked() | |
| 231 | + * to release the reference. Does nothing if the buffer object has no handle. | |
| 232 | + * | |
| 233 | + * Returns: | |
| 234 | + * True if a handle exists, or false otherwise | |
| 232 | 235 | */ |
| 233 | −void drm_gem_object_handle_get_unlocked(struct drm_gem_object *obj) | |
| 236 | +bool drm_gem_object_handle_get_if_exists_unlocked(struct drm_gem_object *obj) | |
| 234 | 237 | { |
| 235 | 238 | struct drm_device *dev = obj->dev; |
| 236 | 239 | |
| 237 | 240 | guard(mutex)(&dev->object_name_lock); |
| 238 | 241 | |
| 239 | − drm_WARN_ON(dev, !obj->handle_count); /* first ref taken in create-tail helper */ | |
| 242 | + /* | |
| 243 | + * First ref taken during GEM object creation, if any. Some | |
| 244 | + * drivers set up internal framebuffers with GEM objects that | |
| 245 | + * do not have a GEM handle. Hence, this counter can be zero. | |
| 246 | + */ | |
| 247 | + if (!obj->handle_count) | |
| 248 | + return false; | |
| 249 | + | |
| 240 | 250 | drm_gem_object_handle_get(obj); |
| 251 | + | |
| 252 | + return true; | |
| 241 | 253 | } |
| 242 | −EXPORT_SYMBOL(drm_gem_object_handle_get_unlocked); | |
| 243 | 254 | |
| 244 | 255 | /** |
| 245 | 256 | * drm_gem_object_handle_free - release resources bound to userspace handles |
| @@ -272,7 +283,7 @@ static void drm_gem_object_exported_dma_buf_free(struct drm_gem_object *obj) · 26 unmodified lines | ||
| 272 | 283 | } |
| 273 | 284 | |
| 274 | 285 | /** |
| 275 | − * drm_gem_object_handle_put_unlocked - releases reference on user-space handles | |
| 286 | + * drm_gem_object_handle_put_unlocked - releases reference on user-space handle | |
| 276 | 287 | * @obj: GEM object |
| 277 | 288 | * |
| 278 | 289 | * Releases a reference on the GEM buffer object's handle. Possibly releases |
| @@ -283,14 +294,14 @@ void drm_gem_object_handle_put_unlocked(struct drm_gem_object *obj) · 4 unmodified lines | ||
| 283 | 294 | struct drm_device *dev = obj->dev; |
| 284 | 295 | bool final = false; |
| 285 | 296 | |
| 286 | − if (WARN_ON(READ_ONCE(obj->handle_count) == 0)) | |
| 297 | + if (drm_WARN_ON(dev, READ_ONCE(obj->handle_count) == 0)) | |
| 287 | 298 | return; |
| 288 | 299 | |
| 289 | 300 | /* |
| 290 | − * Must bump handle count first as this may be the last | |
| 291 | − * ref, in which case the object would disappear before we | |
| 292 | − * checked for a name | |
| 293 | − */ | |
| 301 | + * Must bump handle count first as this may be the last | |
| 302 | + * ref, in which case the object would disappear before | |
| 303 | + * we checked for a name. | |
| 304 | + */ | |
| 294 | 305 | |
| 295 | 306 | mutex_lock(&dev->object_name_lock); |
| 296 | 307 | if (--obj->handle_count == 0) { |
| @@ -303,7 +314,6 @@ void drm_gem_object_handle_put_unlocked(struct drm_gem_object *obj) · 6 unmodified lines | ||
| 303 | 314 | if (final) |
| 304 | 315 | drm_gem_object_put(obj); |
| 305 | 316 | } |
| 306 | −EXPORT_SYMBOL(drm_gem_object_handle_put_unlocked); | |
| 307 | 317 | |
| 308 | 318 | /* |
| 309 | 319 | * Called at device or object close to release the file's |
| @@ -315,6 +325,9 @@ drm_gem_object_release_handle(int id, void *ptr, void *data) · 5 unmodified lines | ||
| 315 | 325 | struct drm_file *file_priv = data; |
| 316 | 326 | struct drm_gem_object *obj = ptr; |
| 317 | 327 | |
| 328 | + if (drm_WARN_ON(obj->dev, !data)) | |
| 329 | + return 0; | |
| 330 | + | |
| 318 | 331 | if (obj->funcs->close) |
| 319 | 332 | obj->funcs->close(obj, file_priv); |
| 320 | 333 | |
| @@ -435,7 +448,7 @@ drm_gem_handle_create_tail(struct drm_file *file_priv, · 114 unmodified lines | ||
| 435 | 448 | idr_preload(GFP_KERNEL); |
| 436 | 449 | spin_lock(&file_priv->table_lock); |
| 437 | 450 | |
| 438 | − ret = idr_alloc(&file_priv->object_idr, obj, 1, 0, GFP_NOWAIT); | |
| 451 | + ret = idr_alloc(&file_priv->object_idr, NULL, 1, 0, GFP_NOWAIT); | |
| 439 | 452 | |
| 440 | 453 | spin_unlock(&file_priv->table_lock); |
| 441 | 454 | idr_preload_end(); |
| @@ -456,6 +469,11 @@ drm_gem_handle_create_tail(struct drm_file *file_priv, · 14 unmodified lines | ||
| 456 | 469 | goto err_revoke; |
| 457 | 470 | } |
| 458 | 471 | |
| 472 | + /* mirrors drm_gem_handle_delete to avoid races */ | |
| 473 | + spin_lock(&file_priv->table_lock); | |
| 474 | + obj = idr_replace(&file_priv->object_idr, obj, handle); | |
| 475 | + WARN_ON(obj != NULL); | |
| 476 | + spin_unlock(&file_priv->table_lock); | |
| 459 | 477 | *handlep = handle; |
| 460 | 478 | return 0; |
| 461 | 479 | |
| @@ -99,7 +99,7 @@ void drm_gem_fb_destroy(struct drm_framebuffer *fb) · 98 unmodified lines | ||
| 99 | 99 | unsigned int i; |
| 100 | 100 | |
| 101 | 101 | for (i = 0; i < fb->format->num_planes; i++) |
| 102 | − drm_gem_object_handle_put_unlocked(fb->obj[i]); | |
| 102 | + drm_gem_object_put(fb->obj[i]); | |
| 103 | 103 | |
| 104 | 104 | drm_framebuffer_cleanup(fb); |
| 105 | 105 | kfree(fb); |
| @@ -182,10 +182,8 @@ int drm_gem_fb_init_with_funcs(struct drm_device *dev, · 76 unmodified lines | ||
| 182 | 182 | if (!objs[i]) { |
| 183 | 183 | drm_dbg_kms(dev, "Failed to lookup GEM object\n"); |
| 184 | 184 | ret = -ENOENT; |
| 185 | − goto err_gem_object_handle_put_unlocked; | |
| 185 | + goto err_gem_object_put; | |
| 186 | 186 | } |
| 187 | − drm_gem_object_handle_get_unlocked(objs[i]); | |
| 188 | − drm_gem_object_put(objs[i]); | |
| 189 | 187 | |
| 190 | 188 | min_size = (height - 1) * mode_cmd->pitches[i] |
| 191 | 189 | + drm_format_info_min_pitch(info, i, width) |
| @@ -195,22 +193,22 @@ int drm_gem_fb_init_with_funcs(struct drm_device *dev, · 3 unmodified lines | ||
| 195 | 193 | drm_dbg_kms(dev, |
| 196 | 194 | "GEM object size (%zu) smaller than minimum size (%u) for plane %d\n", |
| 197 | 195 | objs[i]->size, min_size, i); |
| 198 | − drm_gem_object_handle_put_unlocked(objs[i]); | |
| 196 | + drm_gem_object_put(objs[i]); | |
| 199 | 197 | ret = -EINVAL; |
| 200 | − goto err_gem_object_handle_put_unlocked; | |
| 198 | + goto err_gem_object_put; | |
| 201 | 199 | } |
| 202 | 200 | } |
| 203 | 201 | |
| 204 | 202 | ret = drm_gem_fb_init(dev, fb, mode_cmd, objs, i, funcs); |
| 205 | 203 | if (ret) |
| 206 | − goto err_gem_object_handle_put_unlocked; | |
| 204 | + goto err_gem_object_put; | |
| 207 | 205 | |
| 208 | 206 | return 0; |
| 209 | 207 | |
| 210 | −err_gem_object_handle_put_unlocked: | |
| 208 | +err_gem_object_put: | |
| 211 | 209 | while (i > 0) { |
| 212 | 210 | --i; |
| 213 | − drm_gem_object_handle_put_unlocked(objs[i]); | |
| 211 | + drm_gem_object_put(objs[i]); | |
| 214 | 212 | } |
| 215 | 213 | return ret; |
| 216 | 214 | } |
| @@ -161,7 +161,7 @@ void drm_sysfs_lease_event(struct drm_device *dev); · 160 unmodified lines | ||
| 161 | 161 | |
| 162 | 162 | /* drm_gem.c */ |
| 163 | 163 | int drm_gem_init(struct drm_device *dev); |
| 164 | −void drm_gem_object_handle_get_unlocked(struct drm_gem_object *obj); | |
| 164 | +bool drm_gem_object_handle_get_if_exists_unlocked(struct drm_gem_object *obj); | |
| 165 | 165 | void drm_gem_object_handle_put_unlocked(struct drm_gem_object *obj); |
| 166 | 166 | int drm_gem_handle_create_tail(struct drm_file *file_priv, |
| 167 | 167 | struct drm_gem_object *obj, |
| @@ -27,7 +27,7 @@ · 26 unmodified lines | ||
| 27 | 27 | //! * <https://github.com/erwanvivien/fast_qr> |
| 28 | 28 | //! * <https://github.com/bjguillot/qr> |
| 29 | 29 | |
| 30 | −use kernel::{prelude::*, str::CStr}; | |
| 30 | +use kernel::prelude::*; | |
| 31 | 31 | |
| 32 | 32 | #[derive(Debug, Clone, Copy, PartialEq, Eq, Ord, PartialOrd)] |
| 33 | 33 | struct Version(usize); |
| @@ -1938,7 +1938,7 @@ static int get_init_otp_deassert_fragment_len(struct intel_display *display, · 1937 unmodified lines | ||
| 1938 | 1938 | int index, len; |
| 1939 | 1939 | |
| 1940 | 1940 | if (drm_WARN_ON(display->drm, |
| 1941 | − !data || panel->vbt.dsi.seq_version != 1)) | |
| 1941 | + !data || panel->vbt.dsi.seq_version >= 3)) | |
| 1942 | 1942 | return 0; |
| 1943 | 1943 | |
| 1944 | 1944 | /* index = 1 to skip sequence byte */ |
| @@ -1961,7 +1961,7 @@ static int get_init_otp_deassert_fragment_len(struct intel_display *display, · 16 unmodified lines | ||
| 1961 | 1961 | } |
| 1962 | 1962 | |
| 1963 | 1963 | /* |
| 1964 | − * Some v1 VBT MIPI sequences do the deassert in the init OTP sequence. | |
| 1964 | + * Some v1/v2 VBT MIPI sequences do the deassert in the init OTP sequence. | |
| 1965 | 1965 | * The deassert must be done before calling intel_dsi_device_ready, so for |
| 1966 | 1966 | * these devices we split the init OTP sequence into a deassert sequence and |
| 1967 | 1967 | * the actual init OTP part. |
| @@ -1972,9 +1972,9 @@ static void vlv_fixup_mipi_sequences(struct intel_display *display, · 4 unmodified lines | ||
| 1972 | 1972 | u8 *init_otp; |
| 1973 | 1973 | int len; |
| 1974 | 1974 | |
| 1975 | − /* Limit this to v1 vid-mode sequences */ | |
| 1975 | + /* Limit this to v1/v2 vid-mode sequences */ | |
| 1976 | 1976 | if (panel->vbt.dsi.config->is_cmd_mode || |
| 1977 | − panel->vbt.dsi.seq_version != 1) | |
| 1977 | + panel->vbt.dsi.seq_version >= 3) | |
| 1978 | 1978 | return; |
| 1979 | 1979 | |
| 1980 | 1980 | /* Only do this if there are otp and assert seqs and no deassert seq */ |
| @@ -386,13 +386,13 @@ pvr_power_reset(struct pvr_device *pvr_dev, bool hard_reset) · 385 unmodified lines | ||
| 386 | 386 | if (!err) { |
| 387 | 387 | if (hard_reset) { |
| 388 | 388 | pvr_dev->fw_dev.booted = false; |
| 389 | − WARN_ON(pm_runtime_force_suspend(from_pvr_device(pvr_dev)->dev)); | |
| 389 | + WARN_ON(pvr_power_device_suspend(from_pvr_device(pvr_dev)->dev)); | |
| 390 | 390 | |
| 391 | 391 | err = pvr_fw_hard_reset(pvr_dev); |
| 392 | 392 | if (err) |
| 393 | 393 | goto err_device_lost; |
| 394 | 394 | |
| 395 | − err = pm_runtime_force_resume(from_pvr_device(pvr_dev)->dev); | |
| 395 | + err = pvr_power_device_resume(from_pvr_device(pvr_dev)->dev); | |
| 396 | 396 | pvr_dev->fw_dev.booted = true; |
| 397 | 397 | if (err) |
| 398 | 398 | goto err_device_lost; |
| @@ -314,14 +314,10 @@ nouveau_debugfs_fini(struct nouveau_drm *drm) · 313 unmodified lines | ||
| 314 | 314 | drm->debugfs = NULL; |
| 315 | 315 | } |
| 316 | 316 | |
| 317 | −int | |
| 317 | +void | |
| 318 | 318 | nouveau_module_debugfs_init(void) |
| 319 | 319 | { |
| 320 | 320 | nouveau_debugfs_root = debugfs_create_dir("nouveau", NULL); |
| 321 | − if (IS_ERR(nouveau_debugfs_root)) | |
| 322 | − return PTR_ERR(nouveau_debugfs_root); | |
| 323 | − | |
| 324 | − return 0; | |
| 325 | 321 | } |
| 326 | 322 | |
| 327 | 323 | void |
| @@ -24,7 +24,7 @@ extern void nouveau_debugfs_fini(struct nouveau_drm *); · 23 unmodified lines | ||
| 24 | 24 | |
| 25 | 25 | extern struct dentry *nouveau_debugfs_root; |
| 26 | 26 | |
| 27 | −int nouveau_module_debugfs_init(void); | |
| 27 | +void nouveau_module_debugfs_init(void); | |
| 28 | 28 | void nouveau_module_debugfs_fini(void); |
| 29 | 29 | #else |
| 30 | 30 | static inline void |
| @@ -42,10 +42,9 @@ nouveau_debugfs_fini(struct nouveau_drm *drm) · 11 unmodified lines | ||
| 42 | 42 | { |
| 43 | 43 | } |
| 44 | 44 | |
| 45 | −static inline int | |
| 45 | +static inline void | |
| 46 | 46 | nouveau_module_debugfs_init(void) |
| 47 | 47 | { |
| 48 | − return 0; | |
| 49 | 48 | } |
| 50 | 49 | |
| 51 | 50 | static inline void |
| @@ -1461,9 +1461,7 @@ nouveau_drm_init(void) · 1460 unmodified lines | ||
| 1461 | 1461 | if (!nouveau_modeset) |
| 1462 | 1462 | return 0; |
| 1463 | 1463 | |
| 1464 | − ret = nouveau_module_debugfs_init(); | |
| 1465 | − if (ret) | |
| 1466 | − return ret; | |
| 1464 | + nouveau_module_debugfs_init(); | |
| 1467 | 1465 | |
| 1468 | 1466 | #ifdef CONFIG_NOUVEAU_PLATFORM_DRIVER |
| 1469 | 1467 | platform_driver_register(&nouveau_platform_driver); |
| @@ -719,7 +719,6 @@ r535_gsp_acpi_caps(acpi_handle handle, CAPS_METHOD_DATA *caps) · 718 unmodified lines | ||
| 719 | 719 | union acpi_object argv4 = { |
| 720 | 720 | .buffer.type = ACPI_TYPE_BUFFER, |
| 721 | 721 | .buffer.length = 4, |
| 722 | − .buffer.pointer = kmalloc(argv4.buffer.length, GFP_KERNEL), | |
| 723 | 722 | }, *obj; |
| 724 | 723 | |
| 725 | 724 | caps->status = 0xffff; |
| @@ -727,17 +726,22 @@ r535_gsp_acpi_caps(acpi_handle handle, CAPS_METHOD_DATA *caps) · 1 unmodified line | ||
| 727 | 726 | if (!acpi_check_dsm(handle, &NVOP_DSM_GUID, NVOP_DSM_REV, BIT_ULL(0x1a))) |
| 728 | 727 | return; |
| 729 | 728 | |
| 729 | + argv4.buffer.pointer = kmalloc(argv4.buffer.length, GFP_KERNEL); | |
| 730 | + if (!argv4.buffer.pointer) | |
| 731 | + return; | |
| 732 | + | |
| 730 | 733 | obj = acpi_evaluate_dsm(handle, &NVOP_DSM_GUID, NVOP_DSM_REV, 0x1a, &argv4); |
| 731 | 734 | if (!obj) |
| 732 | − return; | |
| 735 | + goto done; | |
| 733 | 736 | |
| 734 | 737 | if (WARN_ON(obj->type != ACPI_TYPE_BUFFER) || |
| 735 | 738 | WARN_ON(obj->buffer.length != 4)) |
| 736 | − return; | |
| 739 | + goto done; | |
| 737 | 740 | |
| 738 | 741 | caps->status = 0; |
| 739 | 742 | caps->optimusCaps = *(u32 *)obj->buffer.pointer; |
| 740 | 743 | |
| 744 | +done: | |
| 741 | 745 | ACPI_FREE(obj); |
| 742 | 746 | |
| 743 | 747 | kfree(argv4.buffer.pointer); |
| @@ -754,24 +758,28 @@ r535_gsp_acpi_jt(acpi_handle handle, JT_METHOD_DATA *jt) · 10 unmodified lines | ||
| 754 | 758 | union acpi_object argv4 = { |
| 755 | 759 | .buffer.type = ACPI_TYPE_BUFFER, |
| 756 | 760 | .buffer.length = sizeof(caps), |
| 757 | − .buffer.pointer = kmalloc(argv4.buffer.length, GFP_KERNEL), | |
| 758 | 761 | }, *obj; |
| 759 | 762 | |
| 760 | 763 | jt->status = 0xffff; |
| 761 | 764 | |
| 765 | + argv4.buffer.pointer = kmalloc(argv4.buffer.length, GFP_KERNEL); | |
| 766 | + if (!argv4.buffer.pointer) | |
| 767 | + return; | |
| 768 | + | |
| 762 | 769 | obj = acpi_evaluate_dsm(handle, &JT_DSM_GUID, JT_DSM_REV, 0x1, &argv4); |
| 763 | 770 | if (!obj) |
| 764 | − return; | |
| 771 | + goto done; | |
| 765 | 772 | |
| 766 | 773 | if (WARN_ON(obj->type != ACPI_TYPE_BUFFER) || |
| 767 | 774 | WARN_ON(obj->buffer.length != 4)) |
| 768 | − return; | |
| 775 | + goto done; | |
| 769 | 776 | |
| 770 | 777 | jt->status = 0; |
| 771 | 778 | jt->jtCaps = *(u32 *)obj->buffer.pointer; |
| 772 | 779 | jt->jtRevId = (jt->jtCaps & 0xfff00000) >> 20; |
| 773 | 780 | jt->bSBIOSCaps = 0; |
| 774 | 781 | |
| 782 | +done: | |
| 775 | 783 | ACPI_FREE(obj); |
| 776 | 784 | |
| 777 | 785 | kfree(argv4.buffer.pointer); |
| @@ -1744,6 +1752,13 @@ r535_gsp_fini(struct nvkm_gsp *gsp, bool suspend) · 966 unmodified lines | ||
| 1744 | 1752 | nvkm_gsp_sg_free(gsp->subdev.device, &gsp->sr.sgt); |
| 1745 | 1753 | return ret; |
| 1746 | 1754 | } |
| 1755 | + | |
| 1756 | + /* | |
| 1757 | + * TODO: Debug the GSP firmware / RPC handling to find out why | |
| 1758 | + * without this Turing (but none of the other architectures) | |
| 1759 | + * ends up resetting all channels after resume. | |
| 1760 | + */ | |
| 1761 | + msleep(50); | |
| 1747 | 1762 | } |
| 1748 | 1763 | |
| 1749 | 1764 | ret = r535_gsp_rpc_unloading_guest_driver(gsp, suspend); |
| @@ -261,10 +261,8 @@ static int nvdec_load_falcon_firmware(struct nvdec *nvdec) · 260 unmodified lines | ||
| 261 | 261 | |
| 262 | 262 | if (!client->group) { |
| 263 | 263 | virt = dma_alloc_coherent(nvdec->dev, size, &iova, GFP_KERNEL); |
| 264 | − | |
| 265 | − err = dma_mapping_error(nvdec->dev, iova); | |
| 266 | − if (err < 0) | |
| 267 | − return err; | |
| 264 | + if (!virt) | |
| 265 | + return -ENOMEM; | |
| 268 | 266 | } else { |
| 269 | 267 | virt = tegra_drm_alloc(tegra, size, &iova); |
| 270 | 268 | if (IS_ERR(virt)) |
| @@ -171,14 +171,32 @@ static void xe_devcoredump_snapshot_free(struct xe_devcoredump_snapshot *ss) · 170 unmodified lines | ||
| 171 | 171 | |
| 172 | 172 | #define XE_DEVCOREDUMP_CHUNK_MAX (SZ_512M + SZ_1G) |
| 173 | 173 | |
| 174 | +/** | |
| 175 | + * xe_devcoredump_read() - Read data from the Xe device coredump snapshot | |
| 176 | + * @buffer: Destination buffer to copy the coredump data into | |
| 177 | + * @offset: Offset in the coredump data to start reading from | |
| 178 | + * @count: Number of bytes to read | |
| 179 | + * @data: Pointer to the xe_devcoredump structure | |
| 180 | + * @datalen: Length of the data (unused) | |
| 181 | + * | |
| 182 | + * Reads a chunk of the coredump snapshot data into the provided buffer. | |
| 183 | + * If the devcoredump is smaller than 1.5 GB (XE_DEVCOREDUMP_CHUNK_MAX), | |
| 184 | + * it is read directly from a pre-written buffer. For larger devcoredumps, | |
| 185 | + * the pre-written buffer must be periodically repopulated from the snapshot | |
| 186 | + * state due to kmalloc size limitations. | |
| 187 | + * | |
| 188 | + * Return: Number of bytes copied on success, or a negative error code on failure. | |
| 189 | + */ | |
| 174 | 190 | static ssize_t xe_devcoredump_read(char *buffer, loff_t offset, |
| 175 | 191 | size_t count, void *data, size_t datalen) |
| 176 | 192 | { |
| 177 | 193 | struct xe_devcoredump *coredump = data; |
| 178 | 194 | struct xe_devcoredump_snapshot *ss; |
| 179 | − ssize_t byte_copied; | |
| 195 | + ssize_t byte_copied = 0; | |
| 180 | 196 | u32 chunk_offset; |
| 181 | 197 | ssize_t new_chunk_position; |
| 198 | + bool pm_needed = false; | |
| 199 | + int ret = 0; | |
| 182 | 200 | |
| 183 | 201 | if (!coredump) |
| 184 | 202 | return -ENODEV; |
| @@ -188,20 +206,19 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset, · 3 unmodified lines | ||
| 188 | 206 | /* Ensure delayed work is captured before continuing */ |
| 189 | 207 | flush_work(&ss->work); |
| 190 | 208 | |
| 191 | − if (ss->read.size > XE_DEVCOREDUMP_CHUNK_MAX) | |
| 209 | + pm_needed = ss->read.size > XE_DEVCOREDUMP_CHUNK_MAX; | |
| 210 | + if (pm_needed) | |
| 192 | 211 | xe_pm_runtime_get(gt_to_xe(ss->gt)); |
| 193 | 212 | |
| 194 | 213 | mutex_lock(&coredump->lock); |
| 195 | 214 | |
| 196 | 215 | if (!ss->read.buffer) { |
| 197 | − mutex_unlock(&coredump->lock); | |
| 198 | − return -ENODEV; | |
| 216 | + ret = -ENODEV; | |
| 217 | + goto unlock; | |
| 199 | 218 | } |
| 200 | 219 | |
| 201 | − if (offset >= ss->read.size) { | |
| 202 | − mutex_unlock(&coredump->lock); | |
| 203 | − return 0; | |
| 204 | − } | |
| 220 | + if (offset >= ss->read.size) | |
| 221 | + goto unlock; | |
| 205 | 222 | |
| 206 | 223 | new_chunk_position = div_u64_rem(offset, |
| 207 | 224 | XE_DEVCOREDUMP_CHUNK_MAX, |
| @@ -221,12 +238,13 @@ static ssize_t xe_devcoredump_read(char *buffer, loff_t offset, · 13 unmodified lines | ||
| 221 | 238 | ss->read.size - offset; |
| 222 | 239 | memcpy(buffer, ss->read.buffer + chunk_offset, byte_copied); |
| 223 | 240 | |
| 241 | +unlock: | |
| 224 | 242 | mutex_unlock(&coredump->lock); |
| 225 | 243 | |
| 226 | − if (ss->read.size > XE_DEVCOREDUMP_CHUNK_MAX) | |
| 244 | + if (pm_needed) | |
| 227 | 245 | xe_pm_runtime_put(gt_to_xe(ss->gt)); |
| 228 | 246 | |
| 229 | − return byte_copied; | |
| 247 | + return byte_copied ? byte_copied : ret; | |
| 230 | 248 | } |
| 231 | 249 | |
| 232 | 250 | static void xe_devcoredump_free(void *data) |
| @@ -444,6 +444,7 @@ static int xe_alloc_pf_queue(struct xe_gt *gt, struct pf_queue *pf_queue) · 443 unmodified lines | ||
| 444 | 444 | #define PF_MULTIPLIER 8 |
| 445 | 445 | pf_queue->num_dw = |
| 446 | 446 | (num_eus + XE_NUM_HW_ENGINES) * PF_MSG_LEN_DW * PF_MULTIPLIER; |
| 447 | + pf_queue->num_dw = roundup_pow_of_two(pf_queue->num_dw); | |
| 447 | 448 | #undef PF_MULTIPLIER |
| 448 | 449 | |
| 449 | 450 | pf_queue->gt = gt; |
| @@ -78,6 +78,9 @@ static struct xe_lmtt_pt *lmtt_pt_alloc(struct xe_lmtt *lmtt, unsigned int level · 77 unmodified lines | ||
| 78 | 78 | } |
| 79 | 79 | |
| 80 | 80 | lmtt_assert(lmtt, xe_bo_is_vram(bo)); |
| 81 | + lmtt_debug(lmtt, "level=%u addr=%#llx\n", level, (u64)xe_bo_main_addr(bo, XE_PAGE_SIZE)); | |
| 82 | + | |
| 83 | + xe_map_memset(lmtt_to_xe(lmtt), &bo->vmap, 0, 0, bo->size); | |
| 81 | 84 | |
| 82 | 85 | pt->level = level; |
| 83 | 86 | pt->bo = bo; |
| @@ -91,6 +94,9 @@ out: · 7 unmodified lines | ||
| 91 | 94 | |
| 92 | 95 | static void lmtt_pt_free(struct xe_lmtt_pt *pt) |
| 93 | 96 | { |
| 97 | + lmtt_debug(&pt->bo->tile->sriov.pf.lmtt, "level=%u addr=%llx\n", | |
| 98 | + pt->level, (u64)xe_bo_main_addr(pt->bo, XE_PAGE_SIZE)); | |
| 99 | + | |
| 94 | 100 | xe_bo_unpin_map_no_vm(pt->bo); |
| 95 | 101 | kfree(pt); |
| 96 | 102 | } |
| @@ -226,9 +232,14 @@ static void lmtt_write_pte(struct xe_lmtt *lmtt, struct xe_lmtt_pt *pt, · 129 unmodified lines | ||
| 226 | 232 | |
| 227 | 233 | switch (lmtt->ops->lmtt_pte_size(level)) { |
| 228 | 234 | case sizeof(u32): |
| 235 | + lmtt_assert(lmtt, !overflows_type(pte, u32)); | |
| 236 | + lmtt_assert(lmtt, !pte || !iosys_map_rd(&pt->bo->vmap, idx * sizeof(u32), u32)); | |
| 237 | + | |
| 229 | 238 | xe_map_wr(lmtt_to_xe(lmtt), &pt->bo->vmap, idx * sizeof(u32), u32, pte); |
| 230 | 239 | break; |
| 231 | 240 | case sizeof(u64): |
| 241 | + lmtt_assert(lmtt, !pte || !iosys_map_rd(&pt->bo->vmap, idx * sizeof(u64), u64)); | |
| 242 | + | |
| 232 | 243 | xe_map_wr(lmtt_to_xe(lmtt), &pt->bo->vmap, idx * sizeof(u64), u64, pte); |
| 233 | 244 | break; |
| 234 | 245 | default: |
| @@ -863,7 +863,7 @@ struct dma_fence *xe_migrate_copy(struct xe_migrate *m, · 862 unmodified lines | ||
| 863 | 863 | if (src_is_vram && xe_migrate_allow_identity(src_L0, &src_it)) |
| 864 | 864 | xe_res_next(&src_it, src_L0); |
| 865 | 865 | else |
| 866 | − emit_pte(m, bb, src_L0_pt, src_is_vram, copy_system_ccs, | |
| 866 | + emit_pte(m, bb, src_L0_pt, src_is_vram, copy_system_ccs || use_comp_pat, | |
| 867 | 867 | &src_it, src_L0, src); |
| 868 | 868 | |
| 869 | 869 | if (dst_is_vram && xe_migrate_allow_identity(src_L0, &dst_it)) |
| @@ -20,7 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | |
| 21 | 21 | struct xe_modparam xe_modparam = { |
| 22 | 22 | .probe_display = true, |
| 23 | − .guc_log_level = 3, | |
| 23 | + .guc_log_level = IS_ENABLED(CONFIG_DRM_XE_DEBUG) ? 3 : 1, | |
| 24 | 24 | .force_probe = CONFIG_DRM_XE_FORCE_PROBE, |
| 25 | 25 | .wedged_mode = 1, |
| 26 | 26 | .svm_notifier_size = 512, |
| @@ -140,7 +140,6 @@ static const struct xe_graphics_desc graphics_xelpg = { · 139 unmodified lines | ||
| 140 | 140 | .has_asid = 1, \ |
| 141 | 141 | .has_atomic_enable_pte_bit = 1, \ |
| 142 | 142 | .has_flat_ccs = 1, \ |
| 143 | − .has_indirect_ring_state = 1, \ | |
| 144 | 143 | .has_range_tlb_invalidation = 1, \ |
| 145 | 144 | .has_usm = 1, \ |
| 146 | 145 | .has_64bit_timestamp = 1, \ |
| @@ -134,7 +134,7 @@ int xe_pm_suspend(struct xe_device *xe) · 133 unmodified lines | ||
| 134 | 134 | /* FIXME: Super racey... */ |
| 135 | 135 | err = xe_bo_evict_all(xe); |
| 136 | 136 | if (err) |
| 137 | − goto err_pxp; | |
| 137 | + goto err_display; | |
| 138 | 138 | |
| 139 | 139 | for_each_gt(gt, xe, id) { |
| 140 | 140 | err = xe_gt_suspend(gt); |
| @@ -151,7 +151,6 @@ int xe_pm_suspend(struct xe_device *xe) · 10 unmodified lines | ||
| 151 | 151 | |
| 152 | 152 | err_display: |
| 153 | 153 | xe_display_pm_resume(xe); |
| 154 | −err_pxp: | |
| 155 | 154 | xe_pxp_pm_resume(xe->pxp); |
| 156 | 155 | err: |
| 157 | 156 | drm_dbg(&xe->drm, "Device suspend failed %d\n", err); |
| @@ -753,11 +752,13 @@ void xe_pm_assert_unbounded_bridge(struct xe_device *xe) · 595 unmodified lines | ||
| 753 | 752 | } |
| 754 | 753 | |
| 755 | 754 | /** |
| 756 | − * xe_pm_set_vram_threshold - Set a vram threshold for allowing/blocking D3Cold | |
| 755 | + * xe_pm_set_vram_threshold - Set a VRAM threshold for allowing/blocking D3Cold | |
| 757 | 756 | * @xe: xe device instance |
| 758 | − * @threshold: VRAM size in bites for the D3cold threshold | |
| 757 | + * @threshold: VRAM size in MiB for the D3cold threshold | |
| 759 | 758 | * |
| 760 | − * Returns 0 for success, negative error code otherwise. | |
| 759 | + * Return: | |
| 760 | + * * 0 - success | |
| 761 | + * * -EINVAL - invalid argument | |
| 761 | 762 | */ |
| 762 | 763 | int xe_pm_set_vram_threshold(struct xe_device *xe, u32 threshold) |
| 763 | 764 | { |
| @@ -114,10 +114,10 @@ struct fw_blobs_by_type { · 113 unmodified lines | ||
| 114 | 114 | #define XE_GT_TYPE_ANY XE_GT_TYPE_UNINITIALIZED |
| 115 | 115 | |
| 116 | 116 | #define XE_GUC_FIRMWARE_DEFS(fw_def, mmp_ver, major_ver) \ |
| 117 | − fw_def(BATTLEMAGE, GT_TYPE_ANY, major_ver(xe, guc, bmg, 70, 44, 1)) \ | |
| 118 | − fw_def(LUNARLAKE, GT_TYPE_ANY, major_ver(xe, guc, lnl, 70, 44, 1)) \ | |
| 117 | + fw_def(BATTLEMAGE, GT_TYPE_ANY, major_ver(xe, guc, bmg, 70, 45, 2)) \ | |
| 118 | + fw_def(LUNARLAKE, GT_TYPE_ANY, major_ver(xe, guc, lnl, 70, 45, 2)) \ | |
| 119 | 119 | fw_def(METEORLAKE, GT_TYPE_ANY, major_ver(i915, guc, mtl, 70, 44, 1)) \ |
| 120 | − fw_def(DG2, GT_TYPE_ANY, major_ver(i915, guc, dg2, 70, 44, 1)) \ | |
| 120 | + fw_def(DG2, GT_TYPE_ANY, major_ver(i915, guc, dg2, 70, 45, 2)) \ | |
| 121 | 121 | fw_def(DG1, GT_TYPE_ANY, major_ver(i915, guc, dg1, 70, 44, 1)) \ |
| 122 | 122 | fw_def(ALDERLAKE_N, GT_TYPE_ANY, major_ver(i915, guc, tgl, 70, 44, 1)) \ |
| 123 | 123 | fw_def(ALDERLAKE_P, GT_TYPE_ANY, major_ver(i915, guc, adlp, 70, 44, 1)) \ |
| @@ -38,10 +38,10 @@ · 37 unmodified lines | ||
| 38 | 38 | GRAPHICS_VERSION(2004) |
| 39 | 39 | GRAPHICS_VERSION_RANGE(3000, 3001) |
| 40 | 40 | 22019338487 MEDIA_VERSION(2000) |
| 41 | − GRAPHICS_VERSION(2001) | |
| 41 | + GRAPHICS_VERSION(2001), FUNC(xe_rtp_match_not_sriov_vf) | |
| 42 | 42 | MEDIA_VERSION(3000), MEDIA_STEP(A0, B0), FUNC(xe_rtp_match_not_sriov_vf) |
| 43 | 43 | 22019338487_display PLATFORM(LUNARLAKE) |
| 44 | −16023588340 GRAPHICS_VERSION(2001) | |
| 44 | +16023588340 GRAPHICS_VERSION(2001), FUNC(xe_rtp_match_not_sriov_vf) | |
| 45 | 45 | 14019789679 GRAPHICS_VERSION(1255) |
| 46 | 46 | GRAPHICS_VERSION_RANGE(1270, 2004) |
| 47 | 47 | no_media_l3 MEDIA_VERSION(3000) |
| @@ -2366,8 +2366,7 @@ static int bitmap_get_stats(void *data, struct md_bitmap_stats *stats) · 2365 unmodified lines | ||
| 2366 | 2366 | |
| 2367 | 2367 | if (!bitmap) |
| 2368 | 2368 | return -ENOENT; |
| 2369 | − if (!bitmap->mddev->bitmap_info.external && | |
| 2370 | − !bitmap->storage.sb_page) | |
| 2369 | + if (!bitmap->storage.sb_page) | |
| 2371 | 2370 | return -EINVAL; |
| 2372 | 2371 | sb = kmap_local_page(bitmap->storage.sb_page); |
| 2373 | 2372 | stats->sync_size = le64_to_cpu(sb->sync_size); |
| @@ -1399,7 +1399,7 @@ static void raid1_read_request(struct mddev *mddev, struct bio *bio, · 1398 unmodified lines | ||
| 1399 | 1399 | } |
| 1400 | 1400 | read_bio = bio_alloc_clone(mirror->rdev->bdev, bio, gfp, |
| 1401 | 1401 | &mddev->bio_set); |
| 1402 | − | |
| 1402 | + read_bio->bi_opf &= ~REQ_NOWAIT; | |
| 1403 | 1403 | r1_bio->bios[rdisk] = read_bio; |
| 1404 | 1404 | |
| 1405 | 1405 | read_bio->bi_iter.bi_sector = r1_bio->sector + |
| @@ -1649,6 +1649,7 @@ static void raid1_write_request(struct mddev *mddev, struct bio *bio, · 243 unmodified lines | ||
| 1649 | 1649 | wait_for_serialization(rdev, r1_bio); |
| 1650 | 1650 | } |
| 1651 | 1651 | |
| 1652 | + mbio->bi_opf &= ~REQ_NOWAIT; | |
| 1652 | 1653 | r1_bio->bios[i] = mbio; |
| 1653 | 1654 | |
| 1654 | 1655 | mbio->bi_iter.bi_sector = (r1_bio->sector + rdev->data_offset); |
| @@ -3428,6 +3429,7 @@ static int raid1_reshape(struct mddev *mddev) · 1773 unmodified lines | ||
| 3428 | 3429 | /* ok, everything is stopped */ |
| 3429 | 3430 | oldpool = conf->r1bio_pool; |
| 3430 | 3431 | conf->r1bio_pool = newpool; |
| 3432 | + init_waitqueue_head(&conf->r1bio_pool.wait); | |
| 3431 | 3433 | |
| 3432 | 3434 | for (d = d2 = 0; d < conf->raid_disks; d++) { |
| 3433 | 3435 | struct md_rdev *rdev = conf->mirrors[d].rdev; |
| @@ -1182,8 +1182,11 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio, · 1181 unmodified lines | ||
| 1182 | 1182 | } |
| 1183 | 1183 | } |
| 1184 | 1184 | |
| 1185 | − if (!regular_request_wait(mddev, conf, bio, r10_bio->sectors)) | |
| 1185 | + if (!regular_request_wait(mddev, conf, bio, r10_bio->sectors)) { | |
| 1186 | + raid_end_bio_io(r10_bio); | |
| 1186 | 1187 | return; |
| 1188 | + } | |
| 1189 | + | |
| 1187 | 1190 | rdev = read_balance(conf, r10_bio, &max_sectors); |
| 1188 | 1191 | if (!rdev) { |
| 1189 | 1192 | if (err_rdev) { |
| @@ -1221,6 +1224,7 @@ static void raid10_read_request(struct mddev *mddev, struct bio *bio, · 31 unmodified lines | ||
| 1221 | 1224 | r10_bio->master_bio = bio; |
| 1222 | 1225 | } |
| 1223 | 1226 | read_bio = bio_alloc_clone(rdev->bdev, bio, gfp, &mddev->bio_set); |
| 1227 | + read_bio->bi_opf &= ~REQ_NOWAIT; | |
| 1224 | 1228 | |
| 1225 | 1229 | r10_bio->devs[slot].bio = read_bio; |
| 1226 | 1230 | r10_bio->devs[slot].rdev = rdev; |
| @@ -1256,6 +1260,7 @@ static void raid10_write_one_disk(struct mddev *mddev, struct r10bio *r10_bio, · 29 unmodified lines | ||
| 1256 | 1260 | conf->mirrors[devnum].rdev; |
| 1257 | 1261 | |
| 1258 | 1262 | mbio = bio_alloc_clone(rdev->bdev, bio, GFP_NOIO, &mddev->bio_set); |
| 1263 | + mbio->bi_opf &= ~REQ_NOWAIT; | |
| 1259 | 1264 | if (replacement) |
| 1260 | 1265 | r10_bio->devs[n_copy].repl_bio = mbio; |
| 1261 | 1266 | else |
| @@ -1370,8 +1375,11 @@ static void raid10_write_request(struct mddev *mddev, struct bio *bio, · 108 unmodified lines | ||
| 1370 | 1375 | } |
| 1371 | 1376 | |
| 1372 | 1377 | sectors = r10_bio->sectors; |
| 1373 | − if (!regular_request_wait(mddev, conf, bio, sectors)) | |
| 1378 | + if (!regular_request_wait(mddev, conf, bio, sectors)) { | |
| 1379 | + raid_end_bio_io(r10_bio); | |
| 1374 | 1380 | return; |
| 1381 | + } | |
| 1382 | + | |
| 1375 | 1383 | if (test_bit(MD_RECOVERY_RESHAPE, &mddev->recovery) && |
| 1376 | 1384 | (mddev->reshape_backwards |
| 1377 | 1385 | ? (bio->bi_iter.bi_sector < conf->reshape_safe && |
| @@ -665,7 +665,7 @@ static int m_can_handle_lost_msg(struct net_device *dev) · 664 unmodified lines | ||
| 665 | 665 | struct can_frame *frame; |
| 666 | 666 | u32 timestamp = 0; |
| 667 | 667 | |
| 668 | − netdev_err(dev, "msg lost in rxf0\n"); | |
| 668 | + netdev_dbg(dev, "msg lost in rxf0\n"); | |
| 669 | 669 | |
| 670 | 670 | stats->rx_errors++; |
| 671 | 671 | stats->rx_over_errors++; |
| @@ -2984,6 +2984,7 @@ static int airoha_probe(struct platform_device *pdev) · 2983 unmodified lines | ||
| 2984 | 2984 | error_napi_stop: |
| 2985 | 2985 | for (i = 0; i < ARRAY_SIZE(eth->qdma); i++) |
| 2986 | 2986 | airoha_qdma_stop_napi(ð->qdma[i]); |
| 2987 | + airoha_ppe_deinit(eth); | |
| 2987 | 2988 | error_hw_cleanup: |
| 2988 | 2989 | for (i = 0; i < ARRAY_SIZE(eth->qdma); i++) |
| 2989 | 2990 | airoha_hw_cleanup(ð->qdma[i]); |
| @@ -11607,11 +11607,9 @@ static void bnxt_free_irq(struct bnxt *bp) · 11606 unmodified lines | ||
| 11607 | 11607 | |
| 11608 | 11608 | static int bnxt_request_irq(struct bnxt *bp) |
| 11609 | 11609 | { |
| 11610 | + struct cpu_rmap *rmap = NULL; | |
| 11610 | 11611 | int i, j, rc = 0; |
| 11611 | 11612 | unsigned long flags = 0; |
| 11612 | −#ifdef CONFIG_RFS_ACCEL | |
| 11613 | − struct cpu_rmap *rmap; | |
| 11614 | −#endif | |
| 11615 | 11613 | |
| 11616 | 11614 | rc = bnxt_setup_int_mode(bp); |
| 11617 | 11615 | if (rc) { |
| @@ -11632,15 +11630,15 @@ static int bnxt_request_irq(struct bnxt *bp) · 14 unmodified lines | ||
| 11632 | 11630 | int map_idx = bnxt_cp_num_to_irq_num(bp, i); |
| 11633 | 11631 | struct bnxt_irq *irq = &bp->irq_tbl[map_idx]; |
| 11634 | 11632 | |
| 11635 | −#ifdef CONFIG_RFS_ACCEL | |
| 11636 | − if (rmap && bp->bnapi[i]->rx_ring) { | |
| 11633 | + if (IS_ENABLED(CONFIG_RFS_ACCEL) && | |
| 11634 | + rmap && bp->bnapi[i]->rx_ring) { | |
| 11637 | 11635 | rc = irq_cpu_rmap_add(rmap, irq->vector); |
| 11638 | 11636 | if (rc) |
| 11639 | 11637 | netdev_warn(bp->dev, "failed adding irq rmap for ring %d\n", |
| 11640 | 11638 | j); |
| 11641 | 11639 | j++; |
| 11642 | 11640 | } |
| 11643 | −#endif | |
| 11641 | + | |
| 11644 | 11642 | rc = request_irq(irq->vector, irq->handler, flags, irq->name, |
| 11645 | 11643 | bp->bnapi[i]); |
| 11646 | 11644 | if (rc) |
| @@ -368,23 +368,27 @@ static u32 bnxt_get_ctx_coredump(struct bnxt *bp, void *buf, u32 offset, · 367 unmodified lines | ||
| 368 | 368 | if (!ctxm->mem_valid || !seg_id) |
| 369 | 369 | continue; |
| 370 | 370 | |
| 371 | − if (trace) | |
| 371 | + if (trace) { | |
| 372 | 372 | extra_hlen = BNXT_SEG_RCD_LEN; |
| 373 | + if (buf) { | |
| 374 | + u16 trace_type = bnxt_bstore_to_trace[type]; | |
| 375 | + | |
| 376 | + bnxt_fill_drv_seg_record(bp, &record, ctxm, | |
| 377 | + trace_type); | |
| 378 | + } | |
| 379 | + } | |
| 380 | + | |
| 373 | 381 | if (buf) |
| 374 | 382 | data = buf + BNXT_SEG_HDR_LEN + extra_hlen; |
| 383 | + | |
| 375 | 384 | seg_len = bnxt_copy_ctx_mem(bp, ctxm, data, 0) + extra_hlen; |
| 376 | 385 | if (buf) { |
| 377 | 386 | bnxt_fill_coredump_seg_hdr(bp, &seg_hdr, NULL, seg_len, |
| 378 | 387 | 0, 0, 0, comp_id, seg_id); |
| 379 | 388 | memcpy(buf, &seg_hdr, BNXT_SEG_HDR_LEN); |
| 380 | 389 | buf += BNXT_SEG_HDR_LEN; |
| 381 | − if (trace) { | |
| 382 | − u16 trace_type = bnxt_bstore_to_trace[type]; | |
| 383 | − | |
| 384 | − bnxt_fill_drv_seg_record(bp, &record, ctxm, | |
| 385 | − trace_type); | |
| 390 | + if (trace) | |
| 386 | 391 | memcpy(buf, &record, BNXT_SEG_RCD_LEN); |
| 387 | − } | |
| 388 | 392 | buf += seg_len; |
| 389 | 393 | } |
| 390 | 394 | len += BNXT_SEG_HDR_LEN + seg_len; |
| @@ -487,7 +487,9 @@ static int bnxt_ets_validate(struct bnxt *bp, struct ieee_ets *ets, u8 *tc) · 486 unmodified lines | ||
| 487 | 487 | |
| 488 | 488 | if ((ets->tc_tx_bw[i] || ets->tc_tsa[i]) && i > bp->max_tc) |
| 489 | 489 | return -EINVAL; |
| 490 | + } | |
| 490 | 491 | |
| 492 | + for (i = 0; i < max_tc; i++) { | |
| 491 | 493 | switch (ets->tc_tsa[i]) { |
| 492 | 494 | case IEEE_8021QAZ_TSA_STRICT: |
| 493 | 495 | break; |
| @@ -115,7 +115,7 @@ static void __bnxt_xmit_xdp_redirect(struct bnxt *bp, · 114 unmodified lines | ||
| 115 | 115 | tx_buf->action = XDP_REDIRECT; |
| 116 | 116 | tx_buf->xdpf = xdpf; |
| 117 | 117 | dma_unmap_addr_set(tx_buf, mapping, mapping); |
| 118 | − dma_unmap_len_set(tx_buf, len, 0); | |
| 118 | + dma_unmap_len_set(tx_buf, len, len); | |
| 119 | 119 | } |
| 120 | 120 | |
| 121 | 121 | void bnxt_tx_int_xdp(struct bnxt *bp, struct bnxt_napi *bnapi, int budget) |
| @@ -4092,6 +4092,12 @@ static int bcmgenet_probe(struct platform_device *pdev) · 4091 unmodified lines | ||
| 4092 | 4092 | for (i = 0; i <= priv->hw_params->rx_queues; i++) |
| 4093 | 4093 | priv->rx_rings[i].rx_max_coalesced_frames = 1; |
| 4094 | 4094 | |
| 4095 | + /* Initialize u64 stats seq counter for 32bit machines */ | |
| 4096 | + for (i = 0; i <= priv->hw_params->rx_queues; i++) | |
| 4097 | + u64_stats_init(&priv->rx_rings[i].stats64.syncp); | |
| 4098 | + for (i = 0; i <= priv->hw_params->tx_queues; i++) | |
| 4099 | + u64_stats_init(&priv->tx_rings[i].stats64.syncp); | |
| 4100 | + | |
| 4095 | 4101 | /* libphy will determine the link state */ |
| 4096 | 4102 | netif_carrier_off(dev); |
| 4097 | 4103 | |
| @@ -1578,7 +1578,6 @@ napi_del: · 1577 unmodified lines | ||
| 1578 | 1578 | static int nicvf_change_mtu(struct net_device *netdev, int new_mtu) |
| 1579 | 1579 | { |
| 1580 | 1580 | struct nicvf *nic = netdev_priv(netdev); |
| 1581 | − int orig_mtu = netdev->mtu; | |
| 1582 | 1581 | |
| 1583 | 1582 | /* For now just support only the usual MTU sized frames, |
| 1584 | 1583 | * plus some headroom for VLAN, QinQ. |
| @@ -1589,15 +1588,10 @@ static int nicvf_change_mtu(struct net_device *netdev, int new_mtu) · 4 unmodified lines | ||
| 1589 | 1588 | return -EINVAL; |
| 1590 | 1589 | } |
| 1591 | 1590 | |
| 1592 | − WRITE_ONCE(netdev->mtu, new_mtu); | |
| 1593 | − | |
| 1594 | − if (!netif_running(netdev)) | |
| 1595 | − return 0; | |
| 1596 | − | |
| 1597 | − if (nicvf_update_hw_max_frs(nic, new_mtu)) { | |
| 1598 | − netdev->mtu = orig_mtu; | |
| 1591 | + if (netif_running(netdev) && nicvf_update_hw_max_frs(nic, new_mtu)) | |
| 1599 | 1592 | return -EINVAL; |
| 1600 | − } | |
| 1593 | + | |
| 1594 | + WRITE_ONCE(netdev->mtu, new_mtu); | |
| 1601 | 1595 | |
| 1602 | 1596 | return 0; |
| 1603 | 1597 | } |
| @@ -211,7 +211,6 @@ struct ibmvnic_statistics { · 210 unmodified lines | ||
| 211 | 211 | u8 reserved[72]; |
| 212 | 212 | } __packed __aligned(8); |
| 213 | 213 | |
| 214 | −#define NUM_TX_STATS 3 | |
| 215 | 214 | struct ibmvnic_tx_queue_stats { |
| 216 | 215 | u64 batched_packets; |
| 217 | 216 | u64 direct_packets; |
| @@ -219,13 +218,18 @@ struct ibmvnic_tx_queue_stats { · 1 unmodified line | ||
| 219 | 218 | u64 dropped_packets; |
| 220 | 219 | }; |
| 221 | 220 | |
| 222 | −#define NUM_RX_STATS 3 | |
| 221 | +#define NUM_TX_STATS \ | |
| 222 | + (sizeof(struct ibmvnic_tx_queue_stats) / sizeof(u64)) | |
| 223 | + | |
| 223 | 224 | struct ibmvnic_rx_queue_stats { |
| 224 | 225 | u64 packets; |
| 225 | 226 | u64 bytes; |
| 226 | 227 | u64 interrupts; |
| 227 | 228 | }; |
| 228 | 229 | |
| 230 | +#define NUM_RX_STATS \ | |
| 231 | + (sizeof(struct ibmvnic_rx_queue_stats) / sizeof(u64)) | |
| 232 | + | |
| 229 | 233 | struct ibmvnic_acl_buffer { |
| 230 | 234 | __be32 len; |
| 231 | 235 | __be32 version; |
| @@ -18,7 +18,8 @@ enum { · 17 unmodified lines | ||
| 18 | 18 | |
| 19 | 19 | enum { |
| 20 | 20 | MLX5E_TC_PRIO = 0, |
| 21 | − MLX5E_NIC_PRIO | |
| 21 | + MLX5E_PROMISC_PRIO, | |
| 22 | + MLX5E_NIC_PRIO, | |
| 22 | 23 | }; |
| 23 | 24 | |
| 24 | 25 | struct mlx5e_flow_table { |
| @@ -68,9 +69,13 @@ struct mlx5e_l2_table { · 43 unmodified lines | ||
| 68 | 69 | MLX5_HASH_FIELD_SEL_DST_IP |\ |
| 69 | 70 | MLX5_HASH_FIELD_SEL_IPSEC_SPI) |
| 70 | 71 | |
| 71 | −/* NIC prio FTS */ | |
| 72 | +/* NIC promisc FT level */ | |
| 72 | 73 | enum { |
| 73 | 74 | MLX5E_PROMISC_FT_LEVEL, |
| 75 | +}; | |
| 76 | + | |
| 77 | +/* NIC prio FTS */ | |
| 78 | +enum { | |
| 74 | 79 | MLX5E_VLAN_FT_LEVEL, |
| 75 | 80 | MLX5E_L2_FT_LEVEL, |
| 76 | 81 | MLX5E_TTC_FT_LEVEL, |
| @@ -113,7 +113,7 @@ int mlx5e_dim_rx_change(struct mlx5e_rq *rq, bool enable) · 112 unmodified lines | ||
| 113 | 113 | __set_bit(MLX5E_RQ_STATE_DIM, &rq->state); |
| 114 | 114 | } else { |
| 115 | 115 | __clear_bit(MLX5E_RQ_STATE_DIM, &rq->state); |
| 116 | − | |
| 116 | + synchronize_net(); | |
| 117 | 117 | mlx5e_dim_disable(rq->dim); |
| 118 | 118 | rq->dim = NULL; |
| 119 | 119 | } |
| @@ -140,7 +140,7 @@ int mlx5e_dim_tx_change(struct mlx5e_txqsq *sq, bool enable) · 20 unmodified lines | ||
| 140 | 140 | __set_bit(MLX5E_SQ_STATE_DIM, &sq->state); |
| 141 | 141 | } else { |
| 142 | 142 | __clear_bit(MLX5E_SQ_STATE_DIM, &sq->state); |
| 143 | − | |
| 143 | + synchronize_net(); | |
| 144 | 144 | mlx5e_dim_disable(sq->dim); |
| 145 | 145 | sq->dim = NULL; |
| 146 | 146 | } |
| @@ -780,7 +780,7 @@ static int mlx5e_create_promisc_table(struct mlx5e_flow_steering *fs) · 779 unmodified lines | ||
| 780 | 780 | ft_attr.max_fte = MLX5E_PROMISC_TABLE_SIZE; |
| 781 | 781 | ft_attr.autogroup.max_num_groups = 1; |
| 782 | 782 | ft_attr.level = MLX5E_PROMISC_FT_LEVEL; |
| 783 | − ft_attr.prio = MLX5E_NIC_PRIO; | |
| 783 | + ft_attr.prio = MLX5E_PROMISC_PRIO; | |
| 784 | 784 | |
| 785 | 785 | ft->t = mlx5_create_auto_grouped_flow_table(fs->ns, &ft_attr); |
| 786 | 786 | if (IS_ERR(ft->t)) { |
| @@ -1076,6 +1076,7 @@ static int esw_qos_vports_node_update_parent(struct mlx5_esw_sched_node *node, · 1075 unmodified lines | ||
| 1076 | 1076 | return err; |
| 1077 | 1077 | } |
| 1078 | 1078 | esw_qos_node_set_parent(node, parent); |
| 1079 | + node->bw_share = 0; | |
| 1079 | 1080 | |
| 1080 | 1081 | return 0; |
| 1081 | 1082 | } |
| @@ -113,13 +113,16 @@ · 112 unmodified lines | ||
| 113 | 113 | #define ETHTOOL_PRIO_NUM_LEVELS 1 |
| 114 | 114 | #define ETHTOOL_NUM_PRIOS 11 |
| 115 | 115 | #define ETHTOOL_MIN_LEVEL (KERNEL_MIN_LEVEL + ETHTOOL_NUM_PRIOS) |
| 116 | −/* Promiscuous, Vlan, mac, ttc, inner ttc, {UDP/ANY/aRFS/accel/{esp, esp_err}}, IPsec policy, | |
| 116 | +/* Vlan, mac, ttc, inner ttc, {UDP/ANY/aRFS/accel/{esp, esp_err}}, IPsec policy, | |
| 117 | 117 | * {IPsec RoCE MPV,Alias table},IPsec RoCE policy |
| 118 | 118 | */ |
| 119 | −#define KERNEL_NIC_PRIO_NUM_LEVELS 11 | |
| 119 | +#define KERNEL_NIC_PRIO_NUM_LEVELS 10 | |
| 120 | 120 | #define KERNEL_NIC_NUM_PRIOS 1 |
| 121 | −/* One more level for tc */ | |
| 122 | −#define KERNEL_MIN_LEVEL (KERNEL_NIC_PRIO_NUM_LEVELS + 1) | |
| 121 | +/* One more level for tc, and one more for promisc */ | |
| 122 | +#define KERNEL_MIN_LEVEL (KERNEL_NIC_PRIO_NUM_LEVELS + 2) | |
| 123 | + | |
| 124 | +#define KERNEL_NIC_PROMISC_NUM_PRIOS 1 | |
| 125 | +#define KERNEL_NIC_PROMISC_NUM_LEVELS 1 | |
| 123 | 126 | |
| 124 | 127 | #define KERNEL_NIC_TC_NUM_PRIOS 1 |
| 125 | 128 | #define KERNEL_NIC_TC_NUM_LEVELS 3 |
| @@ -187,6 +190,8 @@ static struct init_tree_node { · 61 unmodified lines | ||
| 187 | 190 | ADD_NS(MLX5_FLOW_TABLE_MISS_ACTION_DEF, |
| 188 | 191 | ADD_MULTIPLE_PRIO(KERNEL_NIC_TC_NUM_PRIOS, |
| 189 | 192 | KERNEL_NIC_TC_NUM_LEVELS), |
| 193 | + ADD_MULTIPLE_PRIO(KERNEL_NIC_PROMISC_NUM_PRIOS, | |
| 194 | + KERNEL_NIC_PROMISC_NUM_LEVELS), | |
| 190 | 195 | ADD_MULTIPLE_PRIO(KERNEL_NIC_NUM_PRIOS, |
| 191 | 196 | KERNEL_NIC_PRIO_NUM_LEVELS))), |
| 192 | 197 | ADD_PRIO(0, BY_PASS_MIN_LEVEL, 0, FS_CHAINING_CAPS, |
| @@ -1259,7 +1259,12 @@ static int rtsn_probe(struct platform_device *pdev) · 1258 unmodified lines | ||
| 1259 | 1259 | priv = netdev_priv(ndev); |
| 1260 | 1260 | priv->pdev = pdev; |
| 1261 | 1261 | priv->ndev = ndev; |
| 1262 | + | |
| 1262 | 1263 | priv->ptp_priv = rcar_gen4_ptp_alloc(pdev); |
| 1264 | + if (!priv->ptp_priv) { | |
| 1265 | + ret = -ENOMEM; | |
| 1266 | + goto error_free; | |
| 1267 | + } | |
| 1263 | 1268 | |
| 1264 | 1269 | spin_lock_init(&priv->lock); |
| 1265 | 1270 | platform_set_drvdata(pdev, priv); |
| @@ -364,19 +364,17 @@ static int dwxgmac2_dma_interrupt(struct stmmac_priv *priv, · 363 unmodified lines | ||
| 364 | 364 | } |
| 365 | 365 | |
| 366 | 366 | /* TX/RX NORMAL interrupts */ |
| 367 | − if (likely(intr_status & XGMAC_NIS)) { | |
| 368 | − if (likely(intr_status & XGMAC_RI)) { | |
| 369 | − u64_stats_update_begin(&stats->syncp); | |
| 370 | − u64_stats_inc(&stats->rx_normal_irq_n[chan]); | |
| 371 | − u64_stats_update_end(&stats->syncp); | |
| 372 | − ret |= handle_rx; | |
| 373 | − } | |
| 374 | − if (likely(intr_status & (XGMAC_TI | XGMAC_TBU))) { | |
| 375 | − u64_stats_update_begin(&stats->syncp); | |
| 376 | − u64_stats_inc(&stats->tx_normal_irq_n[chan]); | |
| 377 | − u64_stats_update_end(&stats->syncp); | |
| 378 | − ret |= handle_tx; | |
| 379 | − } | |
| 367 | + if (likely(intr_status & XGMAC_RI)) { | |
| 368 | + u64_stats_update_begin(&stats->syncp); | |
| 369 | + u64_stats_inc(&stats->rx_normal_irq_n[chan]); | |
| 370 | + u64_stats_update_end(&stats->syncp); | |
| 371 | + ret |= handle_rx; | |
| 372 | + } | |
| 373 | + if (likely(intr_status & (XGMAC_TI | XGMAC_TBU))) { | |
| 374 | + u64_stats_update_begin(&stats->syncp); | |
| 375 | + u64_stats_inc(&stats->tx_normal_irq_n[chan]); | |
| 376 | + u64_stats_update_end(&stats->syncp); | |
| 377 | + ret |= handle_tx; | |
| 380 | 378 | } |
| 381 | 379 | |
| 382 | 380 | /* Clear interrupts */ |
| @@ -856,8 +856,6 @@ static struct sk_buff *am65_cpsw_build_skb(void *page_addr, · 855 unmodified lines | ||
| 856 | 856 | { |
| 857 | 857 | struct sk_buff *skb; |
| 858 | 858 | |
| 859 | − len += AM65_CPSW_HEADROOM; | |
| 860 | − | |
| 861 | 859 | skb = build_skb(page_addr, len); |
| 862 | 860 | if (unlikely(!skb)) |
| 863 | 861 | return NULL; |
| @@ -1344,7 +1342,7 @@ static int am65_cpsw_nuss_rx_packets(struct am65_cpsw_rx_flow *flow, · 480 unmodified lines | ||
| 1344 | 1342 | } |
| 1345 | 1343 | |
| 1346 | 1344 | skb = am65_cpsw_build_skb(page_addr, ndev, |
| 1347 | − AM65_CPSW_MAX_PACKET_SIZE, headroom); | |
| 1345 | + PAGE_SIZE, headroom); | |
| 1348 | 1346 | if (unlikely(!skb)) { |
| 1349 | 1347 | new_page = page; |
| 1350 | 1348 | goto requeue; |
| @@ -1309,7 +1309,7 @@ ll_temac_ethtools_set_ringparam(struct net_device *ndev, · 1308 unmodified lines | ||
| 1309 | 1309 | if (ering->rx_pending > RX_BD_NUM_MAX || |
| 1310 | 1310 | ering->rx_mini_pending || |
| 1311 | 1311 | ering->rx_jumbo_pending || |
| 1312 | − ering->rx_pending > TX_BD_NUM_MAX) | |
| 1312 | + ering->tx_pending > TX_BD_NUM_MAX) | |
| 1313 | 1313 | return -EINVAL; |
| 1314 | 1314 | |
| 1315 | 1315 | if (netif_running(ndev)) |
| @@ -332,7 +332,7 @@ static void lan88xx_link_change_notify(struct phy_device *phydev) · 331 unmodified lines | ||
| 332 | 332 | * As workaround, set to 10 before setting to 100 |
| 333 | 333 | * at forced 100 F/H mode. |
| 334 | 334 | */ |
| 335 | − if (!phydev->autoneg && phydev->speed == 100) { | |
| 335 | + if (phydev->state == PHY_NOLINK && !phydev->autoneg && phydev->speed == 100) { | |
| 336 | 336 | /* disable phy interrupt */ |
| 337 | 337 | temp = phy_read(phydev, LAN88XX_INT_MASK); |
| 338 | 338 | temp &= ~LAN88XX_INT_MASK_MDINTPIN_EN_; |
| @@ -488,6 +488,7 @@ static struct phy_driver microchip_phy_driver[] = { · 149 unmodified lines | ||
| 488 | 488 | .config_init = lan88xx_config_init, |
| 489 | 489 | .config_aneg = lan88xx_config_aneg, |
| 490 | 490 | .link_change_notify = lan88xx_link_change_notify, |
| 491 | + .soft_reset = genphy_soft_reset, | |
| 491 | 492 | |
| 492 | 493 | /* Interrupt handling is broken, do not define related |
| 493 | 494 | * functions to force polling. |
| @@ -26,9 +26,6 @@ · 25 unmodified lines | ||
| 26 | 26 | |
| 27 | 27 | #define AT803X_LED_CONTROL 0x18 |
| 28 | 28 | |
| 29 | −#define AT803X_PHY_MMD3_WOL_CTRL 0x8012 | |
| 30 | −#define AT803X_WOL_EN BIT(5) | |
| 31 | − | |
| 32 | 29 | #define AT803X_REG_CHIP_CONFIG 0x1f |
| 33 | 30 | #define AT803X_BT_BX_REG_SEL 0x8000 |
| 34 | 31 | |
| @@ -866,30 +863,6 @@ static int at8031_config_init(struct phy_device *phydev) · 831 unmodified lines | ||
| 866 | 863 | return at803x_config_init(phydev); |
| 867 | 864 | } |
| 868 | 865 | |
| 869 | −static int at8031_set_wol(struct phy_device *phydev, | |
| 870 | − struct ethtool_wolinfo *wol) | |
| 871 | −{ | |
| 872 | − int ret; | |
| 873 | − | |
| 874 | − /* First setup MAC address and enable WOL interrupt */ | |
| 875 | − ret = at803x_set_wol(phydev, wol); | |
| 876 | − if (ret) | |
| 877 | − return ret; | |
| 878 | − | |
| 879 | − if (wol->wolopts & WAKE_MAGIC) | |
| 880 | − /* Enable WOL function for 1588 */ | |
| 881 | − ret = phy_modify_mmd(phydev, MDIO_MMD_PCS, | |
| 882 | − AT803X_PHY_MMD3_WOL_CTRL, | |
| 883 | − 0, AT803X_WOL_EN); | |
| 884 | − else | |
| 885 | − /* Disable WoL function for 1588 */ | |
| 886 | − ret = phy_modify_mmd(phydev, MDIO_MMD_PCS, | |
| 887 | − AT803X_PHY_MMD3_WOL_CTRL, | |
| 888 | − AT803X_WOL_EN, 0); | |
| 889 | − | |
| 890 | − return ret; | |
| 891 | −} | |
| 892 | − | |
| 893 | 866 | static int at8031_config_intr(struct phy_device *phydev) |
| 894 | 867 | { |
| 895 | 868 | struct at803x_priv *priv = phydev->priv; |
| @@ -633,7 +633,7 @@ static struct phy_driver qca808x_driver[] = { · 632 unmodified lines | ||
| 633 | 633 | .handle_interrupt = at803x_handle_interrupt, |
| 634 | 634 | .get_tunable = at803x_get_tunable, |
| 635 | 635 | .set_tunable = at803x_set_tunable, |
| 636 | − .set_wol = at803x_set_wol, | |
| 636 | + .set_wol = at8031_set_wol, | |
| 637 | 637 | .get_wol = at803x_get_wol, |
| 638 | 638 | .get_features = qca808x_get_features, |
| 639 | 639 | .config_aneg = qca808x_config_aneg, |
| @@ -115,6 +115,31 @@ int at803x_set_wol(struct phy_device *phydev, · 114 unmodified lines | ||
| 115 | 115 | } |
| 116 | 116 | EXPORT_SYMBOL_GPL(at803x_set_wol); |
| 117 | 117 | |
| 118 | +int at8031_set_wol(struct phy_device *phydev, | |
| 119 | + struct ethtool_wolinfo *wol) | |
| 120 | +{ | |
| 121 | + int ret; | |
| 122 | + | |
| 123 | + /* First setup MAC address and enable WOL interrupt */ | |
| 124 | + ret = at803x_set_wol(phydev, wol); | |
| 125 | + if (ret) | |
| 126 | + return ret; | |
| 127 | + | |
| 128 | + if (wol->wolopts & WAKE_MAGIC) | |
| 129 | + /* Enable WOL function for 1588 */ | |
| 130 | + ret = phy_modify_mmd(phydev, MDIO_MMD_PCS, | |
| 131 | + AT803X_PHY_MMD3_WOL_CTRL, | |
| 132 | + 0, AT803X_WOL_EN); | |
| 133 | + else | |
| 134 | + /* Disable WoL function for 1588 */ | |
| 135 | + ret = phy_modify_mmd(phydev, MDIO_MMD_PCS, | |
| 136 | + AT803X_PHY_MMD3_WOL_CTRL, | |
| 137 | + AT803X_WOL_EN, 0); | |
| 138 | + | |
| 139 | + return ret; | |
| 140 | +} | |
| 141 | +EXPORT_SYMBOL_GPL(at8031_set_wol); | |
| 142 | + | |
| 118 | 143 | void at803x_get_wol(struct phy_device *phydev, |
| 119 | 144 | struct ethtool_wolinfo *wol) |
| 120 | 145 | { |
| @@ -172,6 +172,9 @@ · 171 unmodified lines | ||
| 172 | 172 | #define AT803X_LOC_MAC_ADDR_16_31_OFFSET 0x804B |
| 173 | 173 | #define AT803X_LOC_MAC_ADDR_32_47_OFFSET 0x804A |
| 174 | 174 | |
| 175 | +#define AT803X_PHY_MMD3_WOL_CTRL 0x8012 | |
| 176 | +#define AT803X_WOL_EN BIT(5) | |
| 177 | + | |
| 175 | 178 | #define AT803X_DEBUG_ADDR 0x1D |
| 176 | 179 | #define AT803X_DEBUG_DATA 0x1E |
| 177 | 180 | |
| @@ -215,6 +218,8 @@ int at803x_debug_reg_mask(struct phy_device *phydev, u16 reg, · 37 unmodified lines | ||
| 215 | 218 | int at803x_debug_reg_write(struct phy_device *phydev, u16 reg, u16 data); |
| 216 | 219 | int at803x_set_wol(struct phy_device *phydev, |
| 217 | 220 | struct ethtool_wolinfo *wol); |
| 221 | +int at8031_set_wol(struct phy_device *phydev, | |
| 222 | + struct ethtool_wolinfo *wol); | |
| 218 | 223 | void at803x_get_wol(struct phy_device *phydev, |
| 219 | 224 | struct ethtool_wolinfo *wol); |
| 220 | 225 | int at803x_ack_interrupt(struct phy_device *phydev); |
| @@ -155,10 +155,29 @@ static int smsc_phy_reset(struct phy_device *phydev) · 154 unmodified lines | ||
| 155 | 155 | |
| 156 | 156 | static int lan87xx_config_aneg(struct phy_device *phydev) |
| 157 | 157 | { |
| 158 | − int rc; | |
| 158 | + u8 mdix_ctrl; | |
| 159 | 159 | int val; |
| 160 | + int rc; | |
| 161 | + | |
| 162 | + /* When auto-negotiation is disabled (forced mode), the PHY's | |
| 163 | + * Auto-MDIX will continue toggling the TX/RX pairs. | |
| 164 | + * | |
| 165 | + * To establish a stable link, we must select a fixed MDI mode. | |
| 166 | + * If the user has not specified a fixed MDI mode (i.e., mdix_ctrl is | |
| 167 | + * 'auto'), we default to ETH_TP_MDI. This choice of a ETH_TP_MDI mode | |
| 168 | + * mirrors the behavior the hardware would exhibit if the AUTOMDIX_EN | |
| 169 | + * strap were configured for a fixed MDI connection. | |
| 170 | + */ | |
| 171 | + if (phydev->autoneg == AUTONEG_DISABLE) { | |
| 172 | + if (phydev->mdix_ctrl == ETH_TP_MDI_AUTO) | |
| 173 | + mdix_ctrl = ETH_TP_MDI; | |
| 174 | + else | |
| 175 | + mdix_ctrl = phydev->mdix_ctrl; | |
| 176 | + } else { | |
| 177 | + mdix_ctrl = phydev->mdix_ctrl; | |
| 178 | + } | |
| 160 | 179 | |
| 161 | − switch (phydev->mdix_ctrl) { | |
| 180 | + switch (mdix_ctrl) { | |
| 162 | 181 | case ETH_TP_MDI: |
| 163 | 182 | val = SPECIAL_CTRL_STS_OVRRD_AMDIX_; |
| 164 | 183 | break; |
| @@ -167,7 +186,8 @@ static int lan87xx_config_aneg(struct phy_device *phydev) · 2 unmodified lines | ||
| 167 | 186 | SPECIAL_CTRL_STS_AMDIX_STATE_; |
| 168 | 187 | break; |
| 169 | 188 | case ETH_TP_MDI_AUTO: |
| 170 | − val = SPECIAL_CTRL_STS_AMDIX_ENABLE_; | |
| 189 | + val = SPECIAL_CTRL_STS_OVRRD_AMDIX_ | | |
| 190 | + SPECIAL_CTRL_STS_AMDIX_ENABLE_; | |
| 171 | 191 | break; |
| 172 | 192 | default: |
| 173 | 193 | return genphy_config_aneg(phydev); |
| @@ -183,7 +203,7 @@ static int lan87xx_config_aneg(struct phy_device *phydev) · 9 unmodified lines | ||
| 183 | 203 | rc |= val; |
| 184 | 204 | phy_write(phydev, SPECIAL_CTRL_STS, rc); |
| 185 | 205 | |
| 186 | − phydev->mdix = phydev->mdix_ctrl; | |
| 206 | + phydev->mdix = mdix_ctrl; | |
| 187 | 207 | return genphy_config_aneg(phydev); |
| 188 | 208 | } |
| 189 | 209 | |
| @@ -261,6 +281,33 @@ int lan87xx_read_status(struct phy_device *phydev) · 71 unmodified lines | ||
| 261 | 281 | } |
| 262 | 282 | EXPORT_SYMBOL_GPL(lan87xx_read_status); |
| 263 | 283 | |
| 284 | +static int lan87xx_phy_config_init(struct phy_device *phydev) | |
| 285 | +{ | |
| 286 | + int rc; | |
| 287 | + | |
| 288 | + /* The LAN87xx PHY's initial MDI-X mode is determined by the AUTOMDIX_EN | |
| 289 | + * hardware strap, but the driver cannot read the strap's status. This | |
| 290 | + * creates an unpredictable initial state. | |
| 291 | + * | |
| 292 | + * To ensure consistent and reliable behavior across all boards, | |
| 293 | + * override the strap configuration on initialization and force the PHY | |
| 294 | + * into a known state with Auto-MDIX enabled, which is the expected | |
| 295 | + * default for modern hardware. | |
| 296 | + */ | |
| 297 | + rc = phy_modify(phydev, SPECIAL_CTRL_STS, | |
| 298 | + SPECIAL_CTRL_STS_OVRRD_AMDIX_ | | |
| 299 | + SPECIAL_CTRL_STS_AMDIX_ENABLE_ | | |
| 300 | + SPECIAL_CTRL_STS_AMDIX_STATE_, | |
| 301 | + SPECIAL_CTRL_STS_OVRRD_AMDIX_ | | |
| 302 | + SPECIAL_CTRL_STS_AMDIX_ENABLE_); | |
| 303 | + if (rc < 0) | |
| 304 | + return rc; | |
| 305 | + | |
| 306 | + phydev->mdix_ctrl = ETH_TP_MDI_AUTO; | |
| 307 | + | |
| 308 | + return smsc_phy_config_init(phydev); | |
| 309 | +} | |
| 310 | + | |
| 264 | 311 | static int lan874x_phy_config_init(struct phy_device *phydev) |
| 265 | 312 | { |
| 266 | 313 | u16 val; |
| @@ -695,7 +742,7 @@ static struct phy_driver smsc_phy_driver[] = { · 428 unmodified lines | ||
| 695 | 742 | |
| 696 | 743 | /* basic functions */ |
| 697 | 744 | .read_status = lan87xx_read_status, |
| 698 | − .config_init = smsc_phy_config_init, | |
| 745 | + .config_init = lan87xx_phy_config_init, | |
| 699 | 746 | .soft_reset = smsc_phy_reset, |
| 700 | 747 | .config_aneg = lan87xx_config_aneg, |
| 701 | 748 | |
| @@ -459,7 +459,9 @@ mwifiex_process_mgmt_packet(struct mwifiex_private *priv, · 458 unmodified lines | ||
| 459 | 459 | "auth: receive authentication from %pM\n", |
| 460 | 460 | ieee_hdr->addr3); |
| 461 | 461 | } else { |
| 462 | − if (!priv->wdev.connected) | |
| 462 | + if (!priv->wdev.connected || | |
| 463 | + !ether_addr_equal(ieee_hdr->addr3, | |
| 464 | + priv->curr_bss_params.bss_descriptor.mac_address)) | |
| 463 | 465 | return 0; |
| 464 | 466 | |
| 465 | 467 | if (ieee80211_is_deauth(ieee_hdr->frame_control)) { |
| @@ -1224,6 +1224,16 @@ static inline int mt76_wed_dma_setup(struct mt76_dev *dev, struct mt76_queue *q, · 1223 unmodified lines | ||
| 1224 | 1224 | #define mt76_dereference(p, dev) \ |
| 1225 | 1225 | rcu_dereference_protected(p, lockdep_is_held(&(dev)->mutex)) |
| 1226 | 1226 | |
| 1227 | +static inline struct mt76_wcid * | |
| 1228 | +__mt76_wcid_ptr(struct mt76_dev *dev, u16 idx) | |
| 1229 | +{ | |
| 1230 | + if (idx >= ARRAY_SIZE(dev->wcid)) | |
| 1231 | + return NULL; | |
| 1232 | + return rcu_dereference(dev->wcid[idx]); | |
| 1233 | +} | |
| 1234 | + | |
| 1235 | +#define mt76_wcid_ptr(dev, idx) __mt76_wcid_ptr(&(dev)->mt76, idx) | |
| 1236 | + | |
| 1227 | 1237 | struct mt76_dev *mt76_alloc_device(struct device *pdev, unsigned int size, |
| 1228 | 1238 | const struct ieee80211_ops *ops, |
| 1229 | 1239 | const struct mt76_driver_ops *drv_ops); |
| @@ -44,7 +44,7 @@ mt7603_rx_loopback_skb(struct mt7603_dev *dev, struct sk_buff *skb) · 43 unmodified lines | ||
| 44 | 44 | if (idx >= MT7603_WTBL_STA - 1) |
| 45 | 45 | goto free; |
| 46 | 46 | |
| 47 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 47 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 48 | 48 | if (!wcid) |
| 49 | 49 | goto free; |
| 50 | 50 | |
| @@ -487,10 +487,7 @@ mt7603_rx_get_wcid(struct mt7603_dev *dev, u8 idx, bool unicast) · 486 unmodified lines | ||
| 487 | 487 | struct mt7603_sta *sta; |
| 488 | 488 | struct mt76_wcid *wcid; |
| 489 | 489 | |
| 490 | − if (idx >= MT7603_WTBL_SIZE) | |
| 491 | − return NULL; | |
| 492 | − | |
| 493 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 490 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 494 | 491 | if (unicast || !wcid) |
| 495 | 492 | return wcid; |
| 496 | 493 | |
| @@ -1266,12 +1263,9 @@ void mt7603_mac_add_txs(struct mt7603_dev *dev, void *data) · 769 unmodified lines | ||
| 1266 | 1263 | if (pid == MT_PACKET_ID_NO_ACK) |
| 1267 | 1264 | return; |
| 1268 | 1265 | |
| 1269 | − if (wcidx >= MT7603_WTBL_SIZE) | |
| 1270 | − return; | |
| 1271 | − | |
| 1272 | 1266 | rcu_read_lock(); |
| 1273 | 1267 | |
| 1274 | − wcid = rcu_dereference(dev->mt76.wcid[wcidx]); | |
| 1268 | + wcid = mt76_wcid_ptr(dev, wcidx); | |
| 1275 | 1269 | if (!wcid) |
| 1276 | 1270 | goto out; |
| 1277 | 1271 | |
| @@ -90,10 +90,7 @@ static struct mt76_wcid *mt7615_rx_get_wcid(struct mt7615_dev *dev, · 89 unmodified lines | ||
| 90 | 90 | struct mt7615_sta *sta; |
| 91 | 91 | struct mt76_wcid *wcid; |
| 92 | 92 | |
| 93 | − if (idx >= MT7615_WTBL_SIZE) | |
| 94 | − return NULL; | |
| 95 | − | |
| 96 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 93 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 97 | 94 | if (unicast || !wcid) |
| 98 | 95 | return wcid; |
| 99 | 96 | |
| @@ -1504,7 +1501,7 @@ static void mt7615_mac_add_txs(struct mt7615_dev *dev, void *data) · 1404 unmodified lines | ||
| 1504 | 1501 | |
| 1505 | 1502 | rcu_read_lock(); |
| 1506 | 1503 | |
| 1507 | − wcid = rcu_dereference(dev->mt76.wcid[wcidx]); | |
| 1504 | + wcid = mt76_wcid_ptr(dev, wcidx); | |
| 1508 | 1505 | if (!wcid) |
| 1509 | 1506 | goto out; |
| 1510 | 1507 | |
| @@ -1172,7 +1172,7 @@ void mt76_connac2_txwi_free(struct mt76_dev *dev, struct mt76_txwi_cache *t, · 1171 unmodified lines | ||
| 1172 | 1172 | wcid_idx = wcid->idx; |
| 1173 | 1173 | } else { |
| 1174 | 1174 | wcid_idx = le32_get_bits(txwi[1], MT_TXD1_WLAN_IDX); |
| 1175 | − wcid = rcu_dereference(dev->wcid[wcid_idx]); | |
| 1175 | + wcid = __mt76_wcid_ptr(dev, wcid_idx); | |
| 1176 | 1176 | |
| 1177 | 1177 | if (wcid && wcid->sta) { |
| 1178 | 1178 | sta = container_of((void *)wcid, struct ieee80211_sta, |
| @@ -287,7 +287,7 @@ __mt76_connac_mcu_alloc_sta_req(struct mt76_dev *dev, struct mt76_vif_link *mvif · 286 unmodified lines | ||
| 287 | 287 | |
| 288 | 288 | mt76_connac_mcu_get_wlan_idx(dev, wcid, &hdr.wlan_idx_lo, |
| 289 | 289 | &hdr.wlan_idx_hi); |
| 290 | − skb = mt76_mcu_msg_alloc(dev, NULL, len); | |
| 290 | + skb = __mt76_mcu_msg_alloc(dev, NULL, len, len, GFP_ATOMIC); | |
| 291 | 291 | if (!skb) |
| 292 | 292 | return ERR_PTR(-ENOMEM); |
| 293 | 293 | |
| @@ -1740,8 +1740,8 @@ int mt76_connac_mcu_hw_scan(struct mt76_phy *phy, struct ieee80211_vif *vif, · 1446 unmodified lines | ||
| 1740 | 1740 | if (!sreq->ssids[i].ssid_len) |
| 1741 | 1741 | continue; |
| 1742 | 1742 | |
| 1743 | − req->ssids[i].ssid_len = cpu_to_le32(sreq->ssids[i].ssid_len); | |
| 1744 | − memcpy(req->ssids[i].ssid, sreq->ssids[i].ssid, | |
| 1743 | + req->ssids[n_ssids].ssid_len = cpu_to_le32(sreq->ssids[i].ssid_len); | |
| 1744 | + memcpy(req->ssids[n_ssids].ssid, sreq->ssids[i].ssid, | |
| 1745 | 1745 | sreq->ssids[i].ssid_len); |
| 1746 | 1746 | n_ssids++; |
| 1747 | 1747 | } |
| @@ -262,10 +262,7 @@ mt76x02_rx_get_sta(struct mt76_dev *dev, u8 idx) · 261 unmodified lines | ||
| 262 | 262 | { |
| 263 | 263 | struct mt76_wcid *wcid; |
| 264 | 264 | |
| 265 | − if (idx >= MT76x02_N_WCIDS) | |
| 266 | − return NULL; | |
| 267 | − | |
| 268 | − wcid = rcu_dereference(dev->wcid[idx]); | |
| 265 | + wcid = __mt76_wcid_ptr(dev, idx); | |
| 269 | 266 | if (!wcid) |
| 270 | 267 | return NULL; |
| 271 | 268 | |
| @@ -564,9 +564,7 @@ void mt76x02_send_tx_status(struct mt76x02_dev *dev, · 563 unmodified lines | ||
| 564 | 564 | |
| 565 | 565 | rcu_read_lock(); |
| 566 | 566 | |
| 567 | − if (stat->wcid < MT76x02_N_WCIDS) | |
| 568 | − wcid = rcu_dereference(dev->mt76.wcid[stat->wcid]); | |
| 569 | − | |
| 567 | + wcid = mt76_wcid_ptr(dev, stat->wcid); | |
| 570 | 568 | if (wcid && wcid->sta) { |
| 571 | 569 | void *priv; |
| 572 | 570 | |
| @@ -56,10 +56,7 @@ static struct mt76_wcid *mt7915_rx_get_wcid(struct mt7915_dev *dev, · 55 unmodified lines | ||
| 56 | 56 | struct mt7915_sta *sta; |
| 57 | 57 | struct mt76_wcid *wcid; |
| 58 | 58 | |
| 59 | − if (idx >= ARRAY_SIZE(dev->mt76.wcid)) | |
| 60 | − return NULL; | |
| 61 | − | |
| 62 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 59 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 63 | 60 | if (unicast || !wcid) |
| 64 | 61 | return wcid; |
| 65 | 62 | |
| @@ -917,7 +914,7 @@ mt7915_mac_tx_free(struct mt7915_dev *dev, void *data, int len) · 851 unmodified lines | ||
| 917 | 914 | u16 idx; |
| 918 | 915 | |
| 919 | 916 | idx = FIELD_GET(MT_TX_FREE_WLAN_ID, info); |
| 920 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 917 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 921 | 918 | sta = wcid_to_sta(wcid); |
| 922 | 919 | if (!sta) |
| 923 | 920 | continue; |
| @@ -1013,12 +1010,9 @@ static void mt7915_mac_add_txs(struct mt7915_dev *dev, void *data) · 89 unmodified lines | ||
| 1013 | 1010 | if (pid < MT_PACKET_ID_WED) |
| 1014 | 1011 | return; |
| 1015 | 1012 | |
| 1016 | − if (wcidx >= mt7915_wtbl_size(dev)) | |
| 1017 | − return; | |
| 1018 | − | |
| 1019 | 1013 | rcu_read_lock(); |
| 1020 | 1014 | |
| 1021 | − wcid = rcu_dereference(dev->mt76.wcid[wcidx]); | |
| 1015 | + wcid = mt76_wcid_ptr(dev, wcidx); | |
| 1022 | 1016 | if (!wcid) |
| 1023 | 1017 | goto out; |
| 1024 | 1018 | |
| @@ -3986,7 +3986,7 @@ int mt7915_mcu_wed_wa_tx_stats(struct mt7915_dev *dev, u16 wlan_idx) · 3985 unmodified lines | ||
| 3986 | 3986 | |
| 3987 | 3987 | rcu_read_lock(); |
| 3988 | 3988 | |
| 3989 | − wcid = rcu_dereference(dev->mt76.wcid[wlan_idx]); | |
| 3989 | + wcid = mt76_wcid_ptr(dev, wlan_idx); | |
| 3990 | 3990 | if (wcid) |
| 3991 | 3991 | wcid->stats.tx_packets += le32_to_cpu(res->tx_packets); |
| 3992 | 3992 | else |
| @@ -587,12 +587,9 @@ static void mt7915_mmio_wed_update_rx_stats(struct mtk_wed_device *wed, · 586 unmodified lines | ||
| 587 | 587 | |
| 588 | 588 | dev = container_of(wed, struct mt7915_dev, mt76.mmio.wed); |
| 589 | 589 | |
| 590 | − if (idx >= mt7915_wtbl_size(dev)) | |
| 591 | − return; | |
| 592 | − | |
| 593 | 590 | rcu_read_lock(); |
| 594 | 591 | |
| 595 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 592 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 596 | 593 | if (wcid) { |
| 597 | 594 | wcid->stats.rx_bytes += le32_to_cpu(stats->rx_byte_cnt); |
| 598 | 595 | wcid->stats.rx_packets += le32_to_cpu(stats->rx_pkt_cnt); |
| @@ -465,7 +465,7 @@ void mt7921_mac_add_txs(struct mt792x_dev *dev, void *data) · 464 unmodified lines | ||
| 465 | 465 | |
| 466 | 466 | rcu_read_lock(); |
| 467 | 467 | |
| 468 | − wcid = rcu_dereference(dev->mt76.wcid[wcidx]); | |
| 468 | + wcid = mt76_wcid_ptr(dev, wcidx); | |
| 469 | 469 | if (!wcid) |
| 470 | 470 | goto out; |
| 471 | 471 | |
| @@ -516,7 +516,7 @@ static void mt7921_mac_tx_free(struct mt792x_dev *dev, void *data, int len) · 44 unmodified lines | ||
| 516 | 516 | |
| 517 | 517 | count++; |
| 518 | 518 | idx = FIELD_GET(MT_TX_FREE_WLAN_ID, info); |
| 519 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 519 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 520 | 520 | sta = wcid_to_sta(wcid); |
| 521 | 521 | if (!sta) |
| 522 | 522 | continue; |
| @@ -816,7 +816,7 @@ void mt7921_usb_sdio_tx_complete_skb(struct mt76_dev *mdev, · 293 unmodified lines | ||
| 816 | 816 | u16 idx; |
| 817 | 817 | |
| 818 | 818 | idx = le32_get_bits(txwi[1], MT_TXD1_WLAN_IDX); |
| 819 | − wcid = rcu_dereference(mdev->wcid[idx]); | |
| 819 | + wcid = __mt76_wcid_ptr(mdev, idx); | |
| 820 | 820 | sta = wcid_to_sta(wcid); |
| 821 | 821 | |
| 822 | 822 | if (sta && likely(e->skb->protocol != cpu_to_be16(ETH_P_PAE))) |
| @@ -1180,6 +1180,9 @@ static void mt7921_sta_set_decap_offload(struct ieee80211_hw *hw, · 1179 unmodified lines | ||
| 1180 | 1180 | struct mt792x_sta *msta = (struct mt792x_sta *)sta->drv_priv; |
| 1181 | 1181 | struct mt792x_dev *dev = mt792x_hw_dev(hw); |
| 1182 | 1182 | |
| 1183 | + if (!msta->deflink.wcid.sta) | |
| 1184 | + return; | |
| 1185 | + | |
| 1183 | 1186 | mt792x_mutex_acquire(dev); |
| 1184 | 1187 | |
| 1185 | 1188 | if (enabled) |
| @@ -52,6 +52,8 @@ static int mt7925_thermal_init(struct mt792x_phy *phy) · 51 unmodified lines | ||
| 52 | 52 | |
| 53 | 53 | name = devm_kasprintf(&wiphy->dev, GFP_KERNEL, "mt7925_%s", |
| 54 | 54 | wiphy_name(wiphy)); |
| 55 | + if (!name) | |
| 56 | + return -ENOMEM; | |
| 55 | 57 | |
| 56 | 58 | hwmon = devm_hwmon_device_register_with_groups(&wiphy->dev, name, phy, |
| 57 | 59 | mt7925_hwmon_groups); |
| @@ -1040,7 +1040,7 @@ void mt7925_mac_add_txs(struct mt792x_dev *dev, void *data) · 1039 unmodified lines | ||
| 1040 | 1040 | |
| 1041 | 1041 | rcu_read_lock(); |
| 1042 | 1042 | |
| 1043 | − wcid = rcu_dereference(dev->mt76.wcid[wcidx]); | |
| 1043 | + wcid = mt76_wcid_ptr(dev, wcidx); | |
| 1044 | 1044 | if (!wcid) |
| 1045 | 1045 | goto out; |
| 1046 | 1046 | |
| @@ -1122,7 +1122,7 @@ mt7925_mac_tx_free(struct mt792x_dev *dev, void *data, int len) · 75 unmodified lines | ||
| 1122 | 1122 | u16 idx; |
| 1123 | 1123 | |
| 1124 | 1124 | idx = FIELD_GET(MT_TXFREE_INFO_WLAN_ID, info); |
| 1125 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 1125 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 1126 | 1126 | sta = wcid_to_sta(wcid); |
| 1127 | 1127 | if (!sta) |
| 1128 | 1128 | continue; |
| @@ -1445,7 +1445,7 @@ void mt7925_usb_sdio_tx_complete_skb(struct mt76_dev *mdev, · 316 unmodified lines | ||
| 1445 | 1445 | u16 idx; |
| 1446 | 1446 | |
| 1447 | 1447 | idx = le32_get_bits(txwi[1], MT_TXD1_WLAN_IDX); |
| 1448 | − wcid = rcu_dereference(mdev->wcid[idx]); | |
| 1448 | + wcid = __mt76_wcid_ptr(mdev, idx); | |
| 1449 | 1449 | sta = wcid_to_sta(wcid); |
| 1450 | 1450 | |
| 1451 | 1451 | if (sta && likely(e->skb->protocol != cpu_to_be16(ETH_P_PAE))) |
| @@ -1481,7 +1481,7 @@ mt7925_start_sched_scan(struct ieee80211_hw *hw, struct ieee80211_vif *vif, · 1480 unmodified lines | ||
| 1481 | 1481 | |
| 1482 | 1482 | mt792x_mutex_acquire(dev); |
| 1483 | 1483 | |
| 1484 | − err = mt7925_mcu_sched_scan_req(mphy, vif, req); | |
| 1484 | + err = mt7925_mcu_sched_scan_req(mphy, vif, req, ies); | |
| 1485 | 1485 | if (err < 0) |
| 1486 | 1486 | goto out; |
| 1487 | 1487 | |
| @@ -1603,6 +1603,9 @@ static void mt7925_sta_set_decap_offload(struct ieee80211_hw *hw, · 115 unmodified lines | ||
| 1603 | 1603 | unsigned long valid = mvif->valid_links; |
| 1604 | 1604 | u8 i; |
| 1605 | 1605 | |
| 1606 | + if (!msta->vif) | |
| 1607 | + return; | |
| 1608 | + | |
| 1606 | 1609 | mt792x_mutex_acquire(dev); |
| 1607 | 1610 | |
| 1608 | 1611 | valid = ieee80211_vif_is_mld(vif) ? mvif->valid_links : BIT(0); |
| @@ -1617,6 +1620,9 @@ static void mt7925_sta_set_decap_offload(struct ieee80211_hw *hw, · 8 unmodified lines | ||
| 1617 | 1620 | else |
| 1618 | 1621 | clear_bit(MT_WCID_FLAG_HDR_TRANS, &mlink->wcid.flags); |
| 1619 | 1622 | |
| 1623 | + if (!mlink->wcid.sta) | |
| 1624 | + continue; | |
| 1625 | + | |
| 1620 | 1626 | mt7925_mcu_wtbl_update_hdr_trans(dev, vif, sta, i); |
| 1621 | 1627 | } |
| 1622 | 1628 | |
| @@ -164,6 +164,7 @@ mt7925_connac_mcu_set_wow_ctrl(struct mt76_phy *phy, struct ieee80211_vif *vif, · 163 unmodified lines | ||
| 164 | 164 | bool suspend, struct cfg80211_wowlan *wowlan) |
| 165 | 165 | { |
| 166 | 166 | struct mt76_vif_link *mvif = (struct mt76_vif_link *)vif->drv_priv; |
| 167 | + struct ieee80211_scan_ies ies = {}; | |
| 167 | 168 | struct mt76_dev *dev = phy->dev; |
| 168 | 169 | struct { |
| 169 | 170 | struct { |
| @@ -194,7 +195,7 @@ mt7925_connac_mcu_set_wow_ctrl(struct mt76_phy *phy, struct ieee80211_vif *vif, · 24 unmodified lines | ||
| 194 | 195 | req.wow_ctrl_tlv.trigger |= (UNI_WOW_DETECT_TYPE_DISCONNECT | |
| 195 | 196 | UNI_WOW_DETECT_TYPE_BCN_LOST); |
| 196 | 197 | if (wowlan->nd_config) { |
| 197 | − mt7925_mcu_sched_scan_req(phy, vif, wowlan->nd_config); | |
| 198 | + mt7925_mcu_sched_scan_req(phy, vif, wowlan->nd_config, &ies); | |
| 198 | 199 | req.wow_ctrl_tlv.trigger |= UNI_WOW_DETECT_TYPE_SCH_SCAN_HIT; |
| 199 | 200 | mt7925_mcu_sched_scan_enable(phy, vif, suspend); |
| 200 | 201 | } |
| @@ -2818,6 +2819,54 @@ int mt7925_mcu_set_dbdc(struct mt76_phy *phy, bool enable) · 2617 unmodified lines | ||
| 2818 | 2819 | return err; |
| 2819 | 2820 | } |
| 2820 | 2821 | |
| 2822 | +static void | |
| 2823 | +mt7925_mcu_build_scan_ie_tlv(struct mt76_dev *mdev, | |
| 2824 | + struct sk_buff *skb, | |
| 2825 | + struct ieee80211_scan_ies *scan_ies) | |
| 2826 | +{ | |
| 2827 | + u32 max_len = sizeof(struct scan_ie_tlv) + MT76_CONNAC_SCAN_IE_LEN; | |
| 2828 | + struct scan_ie_tlv *ie; | |
| 2829 | + enum nl80211_band i; | |
| 2830 | + struct tlv *tlv; | |
| 2831 | + const u8 *ies; | |
| 2832 | + u16 ies_len; | |
| 2833 | + | |
| 2834 | + for (i = 0; i <= NL80211_BAND_6GHZ; i++) { | |
| 2835 | + if (i == NL80211_BAND_60GHZ) | |
| 2836 | + continue; | |
| 2837 | + | |
| 2838 | + ies = scan_ies->ies[i]; | |
| 2839 | + ies_len = scan_ies->len[i]; | |
| 2840 | + | |
| 2841 | + if (!ies || !ies_len) | |
| 2842 | + continue; | |
| 2843 | + | |
| 2844 | + if (ies_len > max_len) | |
| 2845 | + return; | |
| 2846 | + | |
| 2847 | + tlv = mt76_connac_mcu_add_tlv(skb, UNI_SCAN_IE, | |
| 2848 | + sizeof(*ie) + ies_len); | |
| 2849 | + ie = (struct scan_ie_tlv *)tlv; | |
| 2850 | + | |
| 2851 | + memcpy(ie->ies, ies, ies_len); | |
| 2852 | + ie->ies_len = cpu_to_le16(ies_len); | |
| 2853 | + | |
| 2854 | + switch (i) { | |
| 2855 | + case NL80211_BAND_2GHZ: | |
| 2856 | + ie->band = 1; | |
| 2857 | + break; | |
| 2858 | + case NL80211_BAND_6GHZ: | |
| 2859 | + ie->band = 3; | |
| 2860 | + break; | |
| 2861 | + default: | |
| 2862 | + ie->band = 2; | |
| 2863 | + break; | |
| 2864 | + } | |
| 2865 | + | |
| 2866 | + max_len -= (sizeof(*ie) + ies_len); | |
| 2867 | + } | |
| 2868 | +} | |
| 2869 | + | |
| 2821 | 2870 | int mt7925_mcu_hw_scan(struct mt76_phy *phy, struct ieee80211_vif *vif, |
| 2822 | 2871 | struct ieee80211_scan_request *scan_req) |
| 2823 | 2872 | { |
| @@ -2843,7 +2892,8 @@ int mt7925_mcu_hw_scan(struct mt76_phy *phy, struct ieee80211_vif *vif, · 19 unmodified lines | ||
| 2843 | 2892 | |
| 2844 | 2893 | max_len = sizeof(*hdr) + sizeof(*req) + sizeof(*ssid) + |
| 2845 | 2894 | sizeof(*bssid) * MT7925_RNR_SCAN_MAX_BSSIDS + |
| 2846 | − sizeof(*chan_info) + sizeof(*misc) + sizeof(*ie); | |
| 2895 | + sizeof(*chan_info) + sizeof(*misc) + sizeof(*ie) + | |
| 2896 | + MT76_CONNAC_SCAN_IE_LEN; | |
| 2847 | 2897 | |
| 2848 | 2898 | skb = mt76_mcu_msg_alloc(mdev, NULL, max_len); |
| 2849 | 2899 | if (!skb) |
| @@ -2869,8 +2919,8 @@ int mt7925_mcu_hw_scan(struct mt76_phy *phy, struct ieee80211_vif *vif, · 19 unmodified lines | ||
| 2869 | 2919 | if (i > MT7925_RNR_SCAN_MAX_BSSIDS) |
| 2870 | 2920 | break; |
| 2871 | 2921 | |
| 2872 | − ssid->ssids[i].ssid_len = cpu_to_le32(sreq->ssids[i].ssid_len); | |
| 2873 | − memcpy(ssid->ssids[i].ssid, sreq->ssids[i].ssid, | |
| 2922 | + ssid->ssids[n_ssids].ssid_len = cpu_to_le32(sreq->ssids[i].ssid_len); | |
| 2923 | + memcpy(ssid->ssids[n_ssids].ssid, sreq->ssids[i].ssid, | |
| 2874 | 2924 | sreq->ssids[i].ssid_len); |
| 2875 | 2925 | n_ssids++; |
| 2876 | 2926 | } |
| @@ -2925,13 +2975,6 @@ int mt7925_mcu_hw_scan(struct mt76_phy *phy, struct ieee80211_vif *vif, · 48 unmodified lines | ||
| 2925 | 2975 | } |
| 2926 | 2976 | chan_info->channel_type = sreq->n_channels ? 4 : 0; |
| 2927 | 2977 | |
| 2928 | − tlv = mt76_connac_mcu_add_tlv(skb, UNI_SCAN_IE, sizeof(*ie)); | |
| 2929 | − ie = (struct scan_ie_tlv *)tlv; | |
| 2930 | − if (sreq->ie_len > 0) { | |
| 2931 | − memcpy(ie->ies, sreq->ie, sreq->ie_len); | |
| 2932 | − ie->ies_len = cpu_to_le16(sreq->ie_len); | |
| 2933 | − } | |
| 2934 | − | |
| 2935 | 2978 | req->scan_func |= SCAN_FUNC_SPLIT_SCAN; |
| 2936 | 2979 | |
| 2937 | 2980 | tlv = mt76_connac_mcu_add_tlv(skb, UNI_SCAN_MISC, sizeof(*misc)); |
| @@ -2942,6 +2985,9 @@ int mt7925_mcu_hw_scan(struct mt76_phy *phy, struct ieee80211_vif *vif, · 4 unmodified lines | ||
| 2942 | 2985 | req->scan_func |= SCAN_FUNC_RANDOM_MAC; |
| 2943 | 2986 | } |
| 2944 | 2987 | |
| 2988 | + /* Append scan probe IEs as the last tlv */ | |
| 2989 | + mt7925_mcu_build_scan_ie_tlv(mdev, skb, &scan_req->ies); | |
| 2990 | + | |
| 2945 | 2991 | err = mt76_mcu_skb_send_msg(mdev, skb, MCU_UNI_CMD(SCAN_REQ), |
| 2946 | 2992 | true); |
| 2947 | 2993 | if (err < 0) |
| @@ -2953,7 +2999,8 @@ EXPORT_SYMBOL_GPL(mt7925_mcu_hw_scan); · 5 unmodified lines | ||
| 2953 | 2999 | |
| 2954 | 3000 | int mt7925_mcu_sched_scan_req(struct mt76_phy *phy, |
| 2955 | 3001 | struct ieee80211_vif *vif, |
| 2956 | − struct cfg80211_sched_scan_request *sreq) | |
| 3002 | + struct cfg80211_sched_scan_request *sreq, | |
| 3003 | + struct ieee80211_scan_ies *ies) | |
| 2957 | 3004 | { |
| 2958 | 3005 | struct mt76_vif_link *mvif = (struct mt76_vif_link *)vif->drv_priv; |
| 2959 | 3006 | struct ieee80211_channel **scan_list = sreq->channels; |
| @@ -3041,12 +3088,8 @@ int mt7925_mcu_sched_scan_req(struct mt76_phy *phy, · 81 unmodified lines | ||
| 3041 | 3088 | } |
| 3042 | 3089 | chan_info->channel_type = sreq->n_channels ? 4 : 0; |
| 3043 | 3090 | |
| 3044 | − tlv = mt76_connac_mcu_add_tlv(skb, UNI_SCAN_IE, sizeof(*ie)); | |
| 3045 | − ie = (struct scan_ie_tlv *)tlv; | |
| 3046 | − if (sreq->ie_len > 0) { | |
| 3047 | − memcpy(ie->ies, sreq->ie, sreq->ie_len); | |
| 3048 | − ie->ies_len = cpu_to_le16(sreq->ie_len); | |
| 3049 | − } | |
| 3091 | + /* Append scan probe IEs as the last tlv */ | |
| 3092 | + mt7925_mcu_build_scan_ie_tlv(mdev, skb, ies); | |
| 3050 | 3093 | |
| 3051 | 3094 | return mt76_mcu_skb_send_msg(mdev, skb, MCU_UNI_CMD(SCAN_REQ), |
| 3052 | 3095 | true); |
| @@ -269,7 +269,7 @@ struct scan_ie_tlv { · 268 unmodified lines | ||
| 269 | 269 | __le16 ies_len; |
| 270 | 270 | u8 band; |
| 271 | 271 | u8 pad; |
| 272 | − u8 ies[MT76_CONNAC_SCAN_IE_LEN]; | |
| 272 | + u8 ies[]; | |
| 273 | 273 | }; |
| 274 | 274 | |
| 275 | 275 | struct scan_misc_tlv { |
| @@ -673,7 +673,8 @@ int mt7925_mcu_cancel_hw_scan(struct mt76_phy *phy, · 397 unmodified lines | ||
| 673 | 673 | struct ieee80211_vif *vif); |
| 674 | 674 | int mt7925_mcu_sched_scan_req(struct mt76_phy *phy, |
| 675 | 675 | struct ieee80211_vif *vif, |
| 676 | − struct cfg80211_sched_scan_request *sreq); | |
| 676 | + struct cfg80211_sched_scan_request *sreq, | |
| 677 | + struct ieee80211_scan_ies *ies); | |
| 677 | 678 | int mt7925_mcu_sched_scan_enable(struct mt76_phy *phy, |
| 678 | 679 | struct ieee80211_vif *vif, |
| 679 | 680 | bool enable); |
| @@ -58,7 +58,7 @@ · 57 unmodified lines | ||
| 58 | 58 | |
| 59 | 59 | #define MT_INT_TX_DONE_MCU (MT_INT_TX_DONE_MCU_WM | \ |
| 60 | 60 | MT_INT_TX_DONE_FWDL) |
| 61 | −#define MT_INT_TX_DONE_ALL (MT_INT_TX_DONE_MCU_WM | \ | |
| 61 | +#define MT_INT_TX_DONE_ALL (MT_INT_TX_DONE_MCU | \ | |
| 62 | 62 | MT_INT_TX_DONE_BAND0 | \ |
| 63 | 63 | GENMASK(18, 4)) |
| 64 | 64 | |
| @@ -28,7 +28,7 @@ static const struct ieee80211_iface_combination if_comb[] = { · 27 unmodified lines | ||
| 28 | 28 | }, |
| 29 | 29 | }; |
| 30 | 30 | |
| 31 | −static const struct ieee80211_iface_limit if_limits_chanctx[] = { | |
| 31 | +static const struct ieee80211_iface_limit if_limits_chanctx_mcc[] = { | |
| 32 | 32 | { |
| 33 | 33 | .max = 2, |
| 34 | 34 | .types = BIT(NL80211_IFTYPE_STATION) | |
| @@ -36,8 +36,23 @@ static const struct ieee80211_iface_limit if_limits_chanctx[] = { · 1 unmodified line | ||
| 36 | 36 | }, |
| 37 | 37 | { |
| 38 | 38 | .max = 1, |
| 39 | − .types = BIT(NL80211_IFTYPE_AP) | | |
| 40 | − BIT(NL80211_IFTYPE_P2P_GO) | |
| 39 | + .types = BIT(NL80211_IFTYPE_P2P_GO) | |
| 40 | + }, | |
| 41 | + { | |
| 42 | + .max = 1, | |
| 43 | + .types = BIT(NL80211_IFTYPE_P2P_DEVICE) | |
| 44 | + } | |
| 45 | +}; | |
| 46 | + | |
| 47 | +static const struct ieee80211_iface_limit if_limits_chanctx_scc[] = { | |
| 48 | + { | |
| 49 | + .max = 2, | |
| 50 | + .types = BIT(NL80211_IFTYPE_STATION) | | |
| 51 | + BIT(NL80211_IFTYPE_P2P_CLIENT) | |
| 52 | + }, | |
| 53 | + { | |
| 54 | + .max = 1, | |
| 55 | + .types = BIT(NL80211_IFTYPE_AP) | |
| 41 | 56 | }, |
| 42 | 57 | { |
| 43 | 58 | .max = 1, |
| @@ -47,11 +62,18 @@ static const struct ieee80211_iface_limit if_limits_chanctx[] = { · 3 unmodified lines | ||
| 47 | 62 | |
| 48 | 63 | static const struct ieee80211_iface_combination if_comb_chanctx[] = { |
| 49 | 64 | { |
| 50 | − .limits = if_limits_chanctx, | |
| 51 | − .n_limits = ARRAY_SIZE(if_limits_chanctx), | |
| 65 | + .limits = if_limits_chanctx_mcc, | |
| 66 | + .n_limits = ARRAY_SIZE(if_limits_chanctx_mcc), | |
| 52 | 67 | .max_interfaces = 3, |
| 53 | 68 | .num_different_channels = 2, |
| 54 | 69 | .beacon_int_infra_match = false, |
| 70 | + }, | |
| 71 | + { | |
| 72 | + .limits = if_limits_chanctx_scc, | |
| 73 | + .n_limits = ARRAY_SIZE(if_limits_chanctx_scc), | |
| 74 | + .max_interfaces = 3, | |
| 75 | + .num_different_channels = 1, | |
| 76 | + .beacon_int_infra_match = false, | |
| 55 | 77 | } |
| 56 | 78 | }; |
| 57 | 79 | |
| @@ -142,10 +142,7 @@ struct mt76_wcid *mt792x_rx_get_wcid(struct mt792x_dev *dev, u16 idx, · 141 unmodified lines | ||
| 142 | 142 | struct mt792x_sta *sta; |
| 143 | 143 | struct mt76_wcid *wcid; |
| 144 | 144 | |
| 145 | − if (idx >= ARRAY_SIZE(dev->mt76.wcid)) | |
| 146 | − return NULL; | |
| 147 | − | |
| 148 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 145 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 149 | 146 | if (unicast || !wcid) |
| 150 | 147 | return wcid; |
| 151 | 148 | |
| @@ -61,10 +61,7 @@ static struct mt76_wcid *mt7996_rx_get_wcid(struct mt7996_dev *dev, · 60 unmodified lines | ||
| 61 | 61 | struct mt76_wcid *wcid; |
| 62 | 62 | int i; |
| 63 | 63 | |
| 64 | − if (idx >= ARRAY_SIZE(dev->mt76.wcid)) | |
| 65 | − return NULL; | |
| 66 | − | |
| 67 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 64 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 68 | 65 | if (!wcid) |
| 69 | 66 | return NULL; |
| 70 | 67 | |
| @@ -1249,7 +1246,7 @@ mt7996_mac_tx_free(struct mt7996_dev *dev, void *data, int len) · 1178 unmodified lines | ||
| 1249 | 1246 | u16 idx; |
| 1250 | 1247 | |
| 1251 | 1248 | idx = FIELD_GET(MT_TXFREE_INFO_WLAN_ID, info); |
| 1252 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 1249 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 1253 | 1250 | sta = wcid_to_sta(wcid); |
| 1254 | 1251 | if (!sta) |
| 1255 | 1252 | goto next; |
| @@ -1471,12 +1468,9 @@ static void mt7996_mac_add_txs(struct mt7996_dev *dev, void *data) · 215 unmodified lines | ||
| 1471 | 1468 | if (pid < MT_PACKET_ID_NO_SKB) |
| 1472 | 1469 | return; |
| 1473 | 1470 | |
| 1474 | − if (wcidx >= mt7996_wtbl_size(dev)) | |
| 1475 | − return; | |
| 1476 | − | |
| 1477 | 1471 | rcu_read_lock(); |
| 1478 | 1472 | |
| 1479 | − wcid = rcu_dereference(dev->mt76.wcid[wcidx]); | |
| 1473 | + wcid = mt76_wcid_ptr(dev, wcidx); | |
| 1480 | 1474 | if (!wcid) |
| 1481 | 1475 | goto out; |
| 1482 | 1476 | |
| @@ -2353,20 +2347,12 @@ void mt7996_mac_update_stats(struct mt7996_phy *phy) · 870 unmodified lines | ||
| 2353 | 2347 | void mt7996_mac_sta_rc_work(struct work_struct *work) |
| 2354 | 2348 | { |
| 2355 | 2349 | struct mt7996_dev *dev = container_of(work, struct mt7996_dev, rc_work); |
| 2356 | − struct ieee80211_bss_conf *link_conf; | |
| 2357 | − struct ieee80211_link_sta *link_sta; | |
| 2358 | 2350 | struct mt7996_sta_link *msta_link; |
| 2359 | − struct mt7996_vif_link *link; | |
| 2360 | − struct mt76_vif_link *mlink; | |
| 2361 | − struct ieee80211_sta *sta; | |
| 2362 | 2351 | struct ieee80211_vif *vif; |
| 2363 | − struct mt7996_sta *msta; | |
| 2364 | 2352 | struct mt7996_vif *mvif; |
| 2365 | 2353 | LIST_HEAD(list); |
| 2366 | 2354 | u32 changed; |
| 2367 | − u8 link_id; | |
| 2368 | 2355 | |
| 2369 | − rcu_read_lock(); | |
| 2370 | 2356 | spin_lock_bh(&dev->mt76.sta_poll_lock); |
| 2371 | 2357 | list_splice_init(&dev->sta_rc_list, &list); |
| 2372 | 2358 | |
| @@ -2377,46 +2363,28 @@ void mt7996_mac_sta_rc_work(struct work_struct *work) · 4 unmodified lines | ||
| 2377 | 2363 | |
| 2378 | 2364 | changed = msta_link->changed; |
| 2379 | 2365 | msta_link->changed = 0; |
| 2380 | − | |
| 2381 | − sta = wcid_to_sta(&msta_link->wcid); | |
| 2382 | − link_id = msta_link->wcid.link_id; | |
| 2383 | − msta = msta_link->sta; | |
| 2384 | − mvif = msta->vif; | |
| 2385 | − vif = container_of((void *)mvif, struct ieee80211_vif, drv_priv); | |
| 2386 | − | |
| 2387 | − mlink = rcu_dereference(mvif->mt76.link[link_id]); | |
| 2388 | − if (!mlink) | |
| 2389 | − continue; | |
| 2390 | − | |
| 2391 | − link_sta = rcu_dereference(sta->link[link_id]); | |
| 2392 | − if (!link_sta) | |
| 2393 | − continue; | |
| 2394 | − | |
| 2395 | − link_conf = rcu_dereference(vif->link_conf[link_id]); | |
| 2396 | − if (!link_conf) | |
| 2397 | − continue; | |
| 2366 | + mvif = msta_link->sta->vif; | |
| 2367 | + vif = container_of((void *)mvif, struct ieee80211_vif, | |
| 2368 | + drv_priv); | |
| 2398 | 2369 | |
| 2399 | 2370 | spin_unlock_bh(&dev->mt76.sta_poll_lock); |
| 2400 | 2371 | |
| 2401 | − link = (struct mt7996_vif_link *)mlink; | |
| 2402 | − | |
| 2403 | 2372 | if (changed & (IEEE80211_RC_SUPP_RATES_CHANGED | |
| 2404 | 2373 | IEEE80211_RC_NSS_CHANGED | |
| 2405 | 2374 | IEEE80211_RC_BW_CHANGED)) |
| 2406 | − mt7996_mcu_add_rate_ctrl(dev, vif, link_conf, | |
| 2407 | − link_sta, link, msta_link, | |
| 2375 | + mt7996_mcu_add_rate_ctrl(dev, msta_link->sta, vif, | |
| 2376 | + msta_link->wcid.link_id, | |
| 2408 | 2377 | true); |
| 2409 | 2378 | |
| 2410 | 2379 | if (changed & IEEE80211_RC_SMPS_CHANGED) |
| 2411 | − mt7996_mcu_set_fixed_field(dev, link_sta, link, | |
| 2412 | − msta_link, NULL, | |
| 2380 | + mt7996_mcu_set_fixed_field(dev, msta_link->sta, NULL, | |
| 2381 | + msta_link->wcid.link_id, | |
| 2413 | 2382 | RATE_PARAM_MMPS_UPDATE); |
| 2414 | 2383 | |
| 2415 | 2384 | spin_lock_bh(&dev->mt76.sta_poll_lock); |
| 2416 | 2385 | } |
| 2417 | 2386 | |
| 2418 | 2387 | spin_unlock_bh(&dev->mt76.sta_poll_lock); |
| 2419 | − rcu_read_unlock(); | |
| 2420 | 2388 | } |
| 2421 | 2389 | |
| 2422 | 2390 | void mt7996_mac_work(struct work_struct *work) |
| @@ -1112,9 +1112,8 @@ mt7996_mac_sta_event(struct mt7996_dev *dev, struct ieee80211_vif *vif, · 1111 unmodified lines | ||
| 1112 | 1112 | if (err) |
| 1113 | 1113 | return err; |
| 1114 | 1114 | |
| 1115 | − err = mt7996_mcu_add_rate_ctrl(dev, vif, link_conf, | |
| 1116 | − link_sta, link, | |
| 1117 | − msta_link, false); | |
| 1115 | + err = mt7996_mcu_add_rate_ctrl(dev, msta_link->sta, vif, | |
| 1116 | + link_id, false); | |
| 1118 | 1117 | if (err) |
| 1119 | 1118 | return err; |
| 1120 | 1119 | |
| @@ -555,7 +555,7 @@ mt7996_mcu_rx_all_sta_info_event(struct mt7996_dev *dev, struct sk_buff *skb) · 554 unmodified lines | ||
| 555 | 555 | switch (le16_to_cpu(res->tag)) { |
| 556 | 556 | case UNI_ALL_STA_TXRX_RATE: |
| 557 | 557 | wlan_idx = le16_to_cpu(res->rate[i].wlan_idx); |
| 558 | − wcid = rcu_dereference(dev->mt76.wcid[wlan_idx]); | |
| 558 | + wcid = mt76_wcid_ptr(dev, wlan_idx); | |
| 559 | 559 | |
| 560 | 560 | if (!wcid) |
| 561 | 561 | break; |
| @@ -565,7 +565,7 @@ mt7996_mcu_rx_all_sta_info_event(struct mt7996_dev *dev, struct sk_buff *skb) · 3 unmodified lines | ||
| 565 | 565 | break; |
| 566 | 566 | case UNI_ALL_STA_TXRX_ADM_STAT: |
| 567 | 567 | wlan_idx = le16_to_cpu(res->adm_stat[i].wlan_idx); |
| 568 | − wcid = rcu_dereference(dev->mt76.wcid[wlan_idx]); | |
| 568 | + wcid = mt76_wcid_ptr(dev, wlan_idx); | |
| 569 | 569 | |
| 570 | 570 | if (!wcid) |
| 571 | 571 | break; |
| @@ -579,7 +579,7 @@ mt7996_mcu_rx_all_sta_info_event(struct mt7996_dev *dev, struct sk_buff *skb) · 7 unmodified lines | ||
| 579 | 579 | break; |
| 580 | 580 | case UNI_ALL_STA_TXRX_MSDU_COUNT: |
| 581 | 581 | wlan_idx = le16_to_cpu(res->msdu_cnt[i].wlan_idx); |
| 582 | − wcid = rcu_dereference(dev->mt76.wcid[wlan_idx]); | |
| 582 | + wcid = mt76_wcid_ptr(dev, wlan_idx); | |
| 583 | 583 | |
| 584 | 584 | if (!wcid) |
| 585 | 585 | break; |
| @@ -676,10 +676,7 @@ mt7996_mcu_wed_rro_event(struct mt7996_dev *dev, struct sk_buff *skb) · 90 unmodified lines | ||
| 676 | 676 | |
| 677 | 677 | e = (void *)skb->data; |
| 678 | 678 | idx = le16_to_cpu(e->wlan_id); |
| 679 | − if (idx >= ARRAY_SIZE(dev->mt76.wcid)) | |
| 680 | − break; | |
| 681 | − | |
| 682 | − wcid = rcu_dereference(dev->mt76.wcid[idx]); | |
| 679 | + wcid = mt76_wcid_ptr(dev, idx); | |
| 683 | 680 | if (!wcid || !wcid->sta) |
| 684 | 681 | break; |
| 685 | 682 | |
| @@ -1905,22 +1902,35 @@ int mt7996_mcu_set_fixed_rate_ctrl(struct mt7996_dev *dev, · 1219 unmodified lines | ||
| 1905 | 1902 | MCU_WM_UNI_CMD(RA), true); |
| 1906 | 1903 | } |
| 1907 | 1904 | |
| 1908 | −int mt7996_mcu_set_fixed_field(struct mt7996_dev *dev, | |
| 1909 | − struct ieee80211_link_sta *link_sta, | |
| 1910 | − struct mt7996_vif_link *link, | |
| 1911 | − struct mt7996_sta_link *msta_link, | |
| 1912 | − void *data, u32 field) | |
| 1905 | +int mt7996_mcu_set_fixed_field(struct mt7996_dev *dev, struct mt7996_sta *msta, | |
| 1906 | + void *data, u8 link_id, u32 field) | |
| 1913 | 1907 | { |
| 1914 | − struct sta_phy_uni *phy = data; | |
| 1908 | + struct mt7996_vif *mvif = msta->vif; | |
| 1909 | + struct mt7996_sta_link *msta_link; | |
| 1915 | 1910 | struct sta_rec_ra_fixed_uni *ra; |
| 1911 | + struct sta_phy_uni *phy = data; | |
| 1912 | + struct mt76_vif_link *mlink; | |
| 1916 | 1913 | struct sk_buff *skb; |
| 1914 | + int err = -ENODEV; | |
| 1917 | 1915 | struct tlv *tlv; |
| 1918 | 1916 | |
| 1919 | − skb = __mt76_connac_mcu_alloc_sta_req(&dev->mt76, &link->mt76, | |
| 1917 | + rcu_read_lock(); | |
| 1918 | + | |
| 1919 | + mlink = rcu_dereference(mvif->mt76.link[link_id]); | |
| 1920 | + if (!mlink) | |
| 1921 | + goto error_unlock; | |
| 1922 | + | |
| 1923 | + msta_link = rcu_dereference(msta->link[link_id]); | |
| 1924 | + if (!msta_link) | |
| 1925 | + goto error_unlock; | |
| 1926 | + | |
| 1927 | + skb = __mt76_connac_mcu_alloc_sta_req(&dev->mt76, mlink, | |
| 1920 | 1928 | &msta_link->wcid, |
| 1921 | 1929 | MT7996_STA_UPDATE_MAX_SIZE); |
| 1922 | − if (IS_ERR(skb)) | |
| 1923 | − return PTR_ERR(skb); | |
| 1930 | + if (IS_ERR(skb)) { | |
| 1931 | + err = PTR_ERR(skb); | |
| 1932 | + goto error_unlock; | |
| 1933 | + } | |
| 1924 | 1934 | |
| 1925 | 1935 | tlv = mt76_connac_mcu_add_tlv(skb, STA_REC_RA_UPDATE, sizeof(*ra)); |
| 1926 | 1936 | ra = (struct sta_rec_ra_fixed_uni *)tlv; |
| @@ -1935,106 +1945,149 @@ int mt7996_mcu_set_fixed_field(struct mt7996_dev *dev, · 8 unmodified lines | ||
| 1935 | 1945 | if (phy) |
| 1936 | 1946 | ra->phy = *phy; |
| 1937 | 1947 | break; |
| 1938 | − case RATE_PARAM_MMPS_UPDATE: | |
| 1948 | + case RATE_PARAM_MMPS_UPDATE: { | |
| 1949 | + struct ieee80211_sta *sta = wcid_to_sta(&msta_link->wcid); | |
| 1950 | + struct ieee80211_link_sta *link_sta; | |
| 1951 | + | |
| 1952 | + link_sta = rcu_dereference(sta->link[link_id]); | |
| 1953 | + if (!link_sta) { | |
| 1954 | + dev_kfree_skb(skb); | |
| 1955 | + goto error_unlock; | |
| 1956 | + } | |
| 1957 | + | |
| 1939 | 1958 | ra->mmps_mode = mt7996_mcu_get_mmps_mode(link_sta->smps_mode); |
| 1940 | 1959 | break; |
| 1960 | + } | |
| 1941 | 1961 | default: |
| 1942 | 1962 | break; |
| 1943 | 1963 | } |
| 1944 | 1964 | ra->field = cpu_to_le32(field); |
| 1945 | 1965 | |
| 1966 | + rcu_read_unlock(); | |
| 1967 | + | |
| 1946 | 1968 | return mt76_mcu_skb_send_msg(&dev->mt76, skb, |
| 1947 | 1969 | MCU_WMWA_UNI_CMD(STA_REC_UPDATE), true); |
| 1970 | +error_unlock: | |
| 1971 | + rcu_read_unlock(); | |
| 1972 | + | |
| 1973 | + return err; | |
| 1948 | 1974 | } |
| 1949 | 1975 | |
| 1950 | 1976 | static int |
| 1951 | −mt7996_mcu_add_rate_ctrl_fixed(struct mt7996_dev *dev, | |
| 1952 | − struct ieee80211_link_sta *link_sta, | |
| 1953 | − struct mt7996_vif_link *link, | |
| 1954 | − struct mt7996_sta_link *msta_link) | |
| 1977 | +mt7996_mcu_add_rate_ctrl_fixed(struct mt7996_dev *dev, struct mt7996_sta *msta, | |
| 1978 | + struct ieee80211_vif *vif, u8 link_id) | |
| 1955 | 1979 | { |
| 1956 | − struct cfg80211_chan_def *chandef = &link->phy->mt76->chandef; | |
| 1957 | − struct cfg80211_bitrate_mask *mask = &link->bitrate_mask; | |
| 1958 | − enum nl80211_band band = chandef->chan->band; | |
| 1980 | + struct ieee80211_link_sta *link_sta; | |
| 1981 | + struct cfg80211_bitrate_mask mask; | |
| 1982 | + struct mt7996_sta_link *msta_link; | |
| 1983 | + struct mt7996_vif_link *link; | |
| 1959 | 1984 | struct sta_phy_uni phy = {}; |
| 1960 | − int ret, nrates = 0; | |
| 1985 | + struct ieee80211_sta *sta; | |
| 1986 | + int ret, nrates = 0, idx; | |
| 1987 | + enum nl80211_band band; | |
| 1988 | + bool has_he; | |
| 1961 | 1989 | |
| 1962 | 1990 | #define __sta_phy_bitrate_mask_check(_mcs, _gi, _ht, _he) \ |
| 1963 | 1991 | do { \ |
| 1964 | − u8 i, gi = mask->control[band]._gi; \ | |
| 1992 | + u8 i, gi = mask.control[band]._gi; \ | |
| 1965 | 1993 | gi = (_he) ? gi : gi == NL80211_TXRATE_FORCE_SGI; \ |
| 1966 | 1994 | phy.sgi = gi; \ |
| 1967 | − phy.he_ltf = mask->control[band].he_ltf; \ | |
| 1968 | − for (i = 0; i < ARRAY_SIZE(mask->control[band]._mcs); i++) { \ | |
| 1969 | − if (!mask->control[band]._mcs[i]) \ | |
| 1995 | + phy.he_ltf = mask.control[band].he_ltf; \ | |
| 1996 | + for (i = 0; i < ARRAY_SIZE(mask.control[band]._mcs); i++) { \ | |
| 1997 | + if (!mask.control[band]._mcs[i]) \ | |
| 1970 | 1998 | continue; \ |
| 1971 | − nrates += hweight16(mask->control[band]._mcs[i]); \ | |
| 1972 | − phy.mcs = ffs(mask->control[band]._mcs[i]) - 1; \ | |
| 1999 | + nrates += hweight16(mask.control[band]._mcs[i]); \ | |
| 2000 | + phy.mcs = ffs(mask.control[band]._mcs[i]) - 1; \ | |
| 1973 | 2001 | if (_ht) \ |
| 1974 | 2002 | phy.mcs += 8 * i; \ |
| 1975 | 2003 | } \ |
| 1976 | 2004 | } while (0) |
| 1977 | 2005 | |
| 1978 | − if (link_sta->he_cap.has_he) { | |
| 2006 | + rcu_read_lock(); | |
| 2007 | + | |
| 2008 | + link = mt7996_vif_link(dev, vif, link_id); | |
| 2009 | + if (!link) | |
| 2010 | + goto error_unlock; | |
| 2011 | + | |
| 2012 | + msta_link = rcu_dereference(msta->link[link_id]); | |
| 2013 | + if (!msta_link) | |
| 2014 | + goto error_unlock; | |
| 2015 | + | |
| 2016 | + sta = wcid_to_sta(&msta_link->wcid); | |
| 2017 | + link_sta = rcu_dereference(sta->link[link_id]); | |
| 2018 | + if (!link_sta) | |
| 2019 | + goto error_unlock; | |
| 2020 | + | |
| 2021 | + band = link->phy->mt76->chandef.chan->band; | |
| 2022 | + has_he = link_sta->he_cap.has_he; | |
| 2023 | + mask = link->bitrate_mask; | |
| 2024 | + idx = msta_link->wcid.idx; | |
| 2025 | + | |
| 2026 | + if (has_he) { | |
| 1979 | 2027 | __sta_phy_bitrate_mask_check(he_mcs, he_gi, 0, 1); |
| 1980 | 2028 | } else if (link_sta->vht_cap.vht_supported) { |
| 1981 | 2029 | __sta_phy_bitrate_mask_check(vht_mcs, gi, 0, 0); |
| 1982 | 2030 | } else if (link_sta->ht_cap.ht_supported) { |
| 1983 | 2031 | __sta_phy_bitrate_mask_check(ht_mcs, gi, 1, 0); |
| 1984 | 2032 | } else { |
| 1985 | − nrates = hweight32(mask->control[band].legacy); | |
| 1986 | − phy.mcs = ffs(mask->control[band].legacy) - 1; | |
| 2033 | + nrates = hweight32(mask.control[band].legacy); | |
| 2034 | + phy.mcs = ffs(mask.control[band].legacy) - 1; | |
| 1987 | 2035 | } |
| 2036 | + | |
| 2037 | + rcu_read_unlock(); | |
| 2038 | + | |
| 1988 | 2039 | #undef __sta_phy_bitrate_mask_check |
| 1989 | 2040 | |
| 1990 | 2041 | /* fall back to auto rate control */ |
| 1991 | − if (mask->control[band].gi == NL80211_TXRATE_DEFAULT_GI && | |
| 1992 | − mask->control[band].he_gi == GENMASK(7, 0) && | |
| 1993 | − mask->control[band].he_ltf == GENMASK(7, 0) && | |
| 2042 | + if (mask.control[band].gi == NL80211_TXRATE_DEFAULT_GI && | |
| 2043 | + mask.control[band].he_gi == GENMASK(7, 0) && | |
| 2044 | + mask.control[band].he_ltf == GENMASK(7, 0) && | |
| 1994 | 2045 | nrates != 1) |
| 1995 | 2046 | return 0; |
| 1996 | 2047 | |
| 1997 | 2048 | /* fixed single rate */ |
| 1998 | 2049 | if (nrates == 1) { |
| 1999 | − ret = mt7996_mcu_set_fixed_field(dev, link_sta, link, | |
| 2000 | − msta_link, &phy, | |
| 2050 | + ret = mt7996_mcu_set_fixed_field(dev, msta, &phy, link_id, | |
| 2001 | 2051 | RATE_PARAM_FIXED_MCS); |
| 2002 | 2052 | if (ret) |
| 2003 | 2053 | return ret; |
| 2004 | 2054 | } |
| 2005 | 2055 | |
| 2006 | 2056 | /* fixed GI */ |
| 2007 | − if (mask->control[band].gi != NL80211_TXRATE_DEFAULT_GI || | |
| 2008 | − mask->control[band].he_gi != GENMASK(7, 0)) { | |
| 2057 | + if (mask.control[band].gi != NL80211_TXRATE_DEFAULT_GI || | |
| 2058 | + mask.control[band].he_gi != GENMASK(7, 0)) { | |
| 2009 | 2059 | u32 addr; |
| 2010 | 2060 | |
| 2011 | 2061 | /* firmware updates only TXCMD but doesn't take WTBL into |
| 2012 | 2062 | * account, so driver should update here to reflect the |
| 2013 | 2063 | * actual txrate hardware sends out. |
| 2014 | 2064 | */ |
| 2015 | − addr = mt7996_mac_wtbl_lmac_addr(dev, msta_link->wcid.idx, 7); | |
| 2016 | − if (link_sta->he_cap.has_he) | |
| 2065 | + addr = mt7996_mac_wtbl_lmac_addr(dev, idx, 7); | |
| 2066 | + if (has_he) | |
| 2017 | 2067 | mt76_rmw_field(dev, addr, GENMASK(31, 24), phy.sgi); |
| 2018 | 2068 | else |
| 2019 | 2069 | mt76_rmw_field(dev, addr, GENMASK(15, 12), phy.sgi); |
| 2020 | 2070 | |
| 2021 | − ret = mt7996_mcu_set_fixed_field(dev, link_sta, link, | |
| 2022 | − msta_link, &phy, | |
| 2071 | + ret = mt7996_mcu_set_fixed_field(dev, msta, &phy, link_id, | |
| 2023 | 2072 | RATE_PARAM_FIXED_GI); |
| 2024 | 2073 | if (ret) |
| 2025 | 2074 | return ret; |
| 2026 | 2075 | } |
| 2027 | 2076 | |
| 2028 | 2077 | /* fixed HE_LTF */ |
| 2029 | − if (mask->control[band].he_ltf != GENMASK(7, 0)) { | |
| 2030 | − ret = mt7996_mcu_set_fixed_field(dev, link_sta, link, | |
| 2031 | − msta_link, &phy, | |
| 2078 | + if (mask.control[band].he_ltf != GENMASK(7, 0)) { | |
| 2079 | + ret = mt7996_mcu_set_fixed_field(dev, msta, &phy, link_id, | |
| 2032 | 2080 | RATE_PARAM_FIXED_HE_LTF); |
| 2033 | 2081 | if (ret) |
| 2034 | 2082 | return ret; |
| 2035 | 2083 | } |
| 2036 | 2084 | |
| 2037 | 2085 | return 0; |
| 2086 | + | |
| 2087 | +error_unlock: | |
| 2088 | + rcu_read_unlock(); | |
| 2089 | + | |
| 2090 | + return -ENODEV; | |
| 2038 | 2091 | } |
| 2039 | 2092 | |
| 2040 | 2093 | static void |
| @@ -2145,21 +2198,44 @@ mt7996_mcu_sta_rate_ctrl_tlv(struct sk_buff *skb, struct mt7996_dev *dev, · 104 unmodified lines | ||
| 2145 | 2198 | memset(ra->rx_rcpi, INIT_RCPI, sizeof(ra->rx_rcpi)); |
| 2146 | 2199 | } |
| 2147 | 2200 | |
| 2148 | −int mt7996_mcu_add_rate_ctrl(struct mt7996_dev *dev, | |
| 2149 | − struct ieee80211_vif *vif, | |
| 2150 | − struct ieee80211_bss_conf *link_conf, | |
| 2151 | − struct ieee80211_link_sta *link_sta, | |
| 2152 | − struct mt7996_vif_link *link, | |
| 2153 | − struct mt7996_sta_link *msta_link, bool changed) | |
| 2201 | +int mt7996_mcu_add_rate_ctrl(struct mt7996_dev *dev, struct mt7996_sta *msta, | |
| 2202 | + struct ieee80211_vif *vif, u8 link_id, | |
| 2203 | + bool changed) | |
| 2154 | 2204 | { |
| 2205 | + struct ieee80211_bss_conf *link_conf; | |
| 2206 | + struct ieee80211_link_sta *link_sta; | |
| 2207 | + struct mt7996_sta_link *msta_link; | |
| 2208 | + struct mt7996_vif_link *link; | |
| 2209 | + struct ieee80211_sta *sta; | |
| 2155 | 2210 | struct sk_buff *skb; |
| 2156 | − int ret; | |
| 2211 | + int ret = -ENODEV; | |
| 2212 | + | |
| 2213 | + rcu_read_lock(); | |
| 2214 | + | |
| 2215 | + link = mt7996_vif_link(dev, vif, link_id); | |
| 2216 | + if (!link) | |
| 2217 | + goto error_unlock; | |
| 2218 | + | |
| 2219 | + msta_link = rcu_dereference(msta->link[link_id]); | |
| 2220 | + if (!msta_link) | |
| 2221 | + goto error_unlock; | |
| 2222 | + | |
| 2223 | + sta = wcid_to_sta(&msta_link->wcid); | |
| 2224 | + link_sta = rcu_dereference(sta->link[link_id]); | |
| 2225 | + if (!link_sta) | |
| 2226 | + goto error_unlock; | |
| 2227 | + | |
| 2228 | + link_conf = rcu_dereference(vif->link_conf[link_id]); | |
| 2229 | + if (!link_conf) | |
| 2230 | + goto error_unlock; | |
| 2157 | 2231 | |
| 2158 | 2232 | skb = __mt76_connac_mcu_alloc_sta_req(&dev->mt76, &link->mt76, |
| 2159 | 2233 | &msta_link->wcid, |
| 2160 | 2234 | MT7996_STA_UPDATE_MAX_SIZE); |
| 2161 | − if (IS_ERR(skb)) | |
| 2162 | − return PTR_ERR(skb); | |
| 2235 | + if (IS_ERR(skb)) { | |
| 2236 | + ret = PTR_ERR(skb); | |
| 2237 | + goto error_unlock; | |
| 2238 | + } | |
| 2163 | 2239 | |
| 2164 | 2240 | /* firmware rc algorithm refers to sta_rec_he for HE control. |
| 2165 | 2241 | * once dev->rc_work changes the settings driver should also |
| @@ -2173,12 +2249,19 @@ int mt7996_mcu_add_rate_ctrl(struct mt7996_dev *dev, · 7 unmodified lines | ||
| 2173 | 2249 | */ |
| 2174 | 2250 | mt7996_mcu_sta_rate_ctrl_tlv(skb, dev, vif, link_conf, link_sta, link); |
| 2175 | 2251 | |
| 2252 | + rcu_read_unlock(); | |
| 2253 | + | |
| 2176 | 2254 | ret = mt76_mcu_skb_send_msg(&dev->mt76, skb, |
| 2177 | 2255 | MCU_WMWA_UNI_CMD(STA_REC_UPDATE), true); |
| 2178 | 2256 | if (ret) |
| 2179 | 2257 | return ret; |
| 2180 | 2258 | |
| 2181 | − return mt7996_mcu_add_rate_ctrl_fixed(dev, link_sta, link, msta_link); | |
| 2259 | + return mt7996_mcu_add_rate_ctrl_fixed(dev, msta, vif, link_id); | |
| 2260 | + | |
| 2261 | +error_unlock: | |
| 2262 | + rcu_read_unlock(); | |
| 2263 | + | |
| 2264 | + return ret; | |
| 2182 | 2265 | } |
| 2183 | 2266 | |
| 2184 | 2267 | static int |
| @@ -620,23 +620,17 @@ int mt7996_mcu_beacon_inband_discov(struct mt7996_dev *dev, · 619 unmodified lines | ||
| 620 | 620 | int mt7996_mcu_add_obss_spr(struct mt7996_phy *phy, |
| 621 | 621 | struct mt7996_vif_link *link, |
| 622 | 622 | struct ieee80211_he_obss_pd *he_obss_pd); |
| 623 | −int mt7996_mcu_add_rate_ctrl(struct mt7996_dev *dev, | |
| 624 | − struct ieee80211_vif *vif, | |
| 625 | − struct ieee80211_bss_conf *link_conf, | |
| 626 | − struct ieee80211_link_sta *link_sta, | |
| 627 | − struct mt7996_vif_link *link, | |
| 628 | − struct mt7996_sta_link *msta_link, bool changed); | |
| 623 | +int mt7996_mcu_add_rate_ctrl(struct mt7996_dev *dev, struct mt7996_sta *msta, | |
| 624 | + struct ieee80211_vif *vif, u8 link_id, | |
| 625 | + bool changed); | |
| 629 | 626 | int mt7996_set_channel(struct mt76_phy *mphy); |
| 630 | 627 | int mt7996_mcu_set_chan_info(struct mt7996_phy *phy, u16 tag); |
| 631 | 628 | int mt7996_mcu_set_tx(struct mt7996_dev *dev, struct ieee80211_vif *vif, |
| 632 | 629 | struct ieee80211_bss_conf *link_conf); |
| 633 | 630 | int mt7996_mcu_set_fixed_rate_ctrl(struct mt7996_dev *dev, |
| 634 | 631 | void *data, u16 version); |
| 635 | −int mt7996_mcu_set_fixed_field(struct mt7996_dev *dev, | |
| 636 | − struct ieee80211_link_sta *link_sta, | |
| 637 | − struct mt7996_vif_link *link, | |
| 638 | − struct mt7996_sta_link *msta_link, | |
| 639 | − void *data, u32 field); | |
| 632 | +int mt7996_mcu_set_fixed_field(struct mt7996_dev *dev, struct mt7996_sta *msta, | |
| 633 | + void *data, u8 link_id, u32 field); | |
| 640 | 634 | int mt7996_mcu_set_eeprom(struct mt7996_dev *dev); |
| 641 | 635 | int mt7996_mcu_get_eeprom(struct mt7996_dev *dev, u32 offset, u8 *buf, u32 buf_len); |
| 642 | 636 | int mt7996_mcu_get_eeprom_free_block(struct mt7996_dev *dev, u8 *block_num); |
| @@ -64,7 +64,7 @@ mt76_tx_status_unlock(struct mt76_dev *dev, struct sk_buff_head *list) · 63 unmodified lines | ||
| 64 | 64 | struct mt76_tx_cb *cb = mt76_tx_skb_cb(skb); |
| 65 | 65 | struct mt76_wcid *wcid; |
| 66 | 66 | |
| 67 | − wcid = rcu_dereference(dev->wcid[cb->wcid]); | |
| 67 | + wcid = __mt76_wcid_ptr(dev, cb->wcid); | |
| 68 | 68 | if (wcid) { |
| 69 | 69 | status.sta = wcid_to_sta(wcid); |
| 70 | 70 | if (status.sta && (wcid->rate.flags || wcid->rate.legacy)) { |
| @@ -251,9 +251,7 @@ void __mt76_tx_complete_skb(struct mt76_dev *dev, u16 wcid_idx, struct sk_buff * · 180 unmodified lines | ||
| 251 | 251 | |
| 252 | 252 | rcu_read_lock(); |
| 253 | 253 | |
| 254 | − if (wcid_idx < ARRAY_SIZE(dev->wcid)) | |
| 255 | − wcid = rcu_dereference(dev->wcid[wcid_idx]); | |
| 256 | − | |
| 254 | + wcid = __mt76_wcid_ptr(dev, wcid_idx); | |
| 257 | 255 | mt76_tx_check_non_aql(dev, wcid, skb); |
| 258 | 256 | |
| 259 | 257 | #ifdef CONFIG_NL80211_TESTMODE |
| @@ -538,7 +536,7 @@ mt76_txq_schedule_list(struct mt76_phy *phy, enum mt76_txq_id qid) · 278 unmodified lines | ||
| 538 | 536 | break; |
| 539 | 537 | |
| 540 | 538 | mtxq = (struct mt76_txq *)txq->drv_priv; |
| 541 | − wcid = rcu_dereference(dev->wcid[mtxq->wcid]); | |
| 539 | + wcid = __mt76_wcid_ptr(dev, mtxq->wcid); | |
| 542 | 540 | if (!wcid || test_bit(MT_WCID_FLAG_PS, &wcid->flags)) |
| 543 | 541 | continue; |
| 544 | 542 | |
| @@ -617,7 +615,8 @@ mt76_txq_schedule_pending_wcid(struct mt76_phy *phy, struct mt76_wcid *wcid, · 72 unmodified lines | ||
| 617 | 615 | if ((dev->drv->drv_flags & MT_DRV_HW_MGMT_TXQ) && |
| 618 | 616 | !(info->flags & IEEE80211_TX_CTL_HW_80211_ENCAP) && |
| 619 | 617 | !ieee80211_is_data(hdr->frame_control) && |
| 620 | − !ieee80211_is_bufferable_mmpdu(skb)) | |
| 618 | + (!ieee80211_is_bufferable_mmpdu(skb) || | |
| 619 | + ieee80211_is_deauth(hdr->frame_control))) | |
| 621 | 620 | qid = MT_TXQ_PSD; |
| 622 | 621 | |
| 623 | 622 | q = phy->q_tx[qid]; |
| @@ -83,7 +83,7 @@ int mt76_get_min_avg_rssi(struct mt76_dev *dev, u8 phy_idx) · 82 unmodified lines | ||
| 83 | 83 | if (!(mask & 1)) |
| 84 | 84 | continue; |
| 85 | 85 | |
| 86 | − wcid = rcu_dereference(dev->wcid[j]); | |
| 86 | + wcid = __mt76_wcid_ptr(dev, j); | |
| 87 | 87 | if (!wcid || wcid->phy_idx != phy_idx) |
| 88 | 88 | continue; |
| 89 | 89 | |
| @@ -108,7 +108,7 @@ exit_free_device: · 107 unmodified lines | ||
| 108 | 108 | } |
| 109 | 109 | EXPORT_SYMBOL_GPL(rt2x00soc_probe); |
| 110 | 110 | |
| 111 | −int rt2x00soc_remove(struct platform_device *pdev) | |
| 111 | +void rt2x00soc_remove(struct platform_device *pdev) | |
| 112 | 112 | { |
| 113 | 113 | struct ieee80211_hw *hw = platform_get_drvdata(pdev); |
| 114 | 114 | struct rt2x00_dev *rt2x00dev = hw->priv; |
| @@ -119,8 +119,6 @@ int rt2x00soc_remove(struct platform_device *pdev) · 4 unmodified lines | ||
| 119 | 119 | rt2x00lib_remove_dev(rt2x00dev); |
| 120 | 120 | rt2x00soc_free_reg(rt2x00dev); |
| 121 | 121 | ieee80211_free_hw(hw); |
| 122 | − | |
| 123 | − return 0; | |
| 124 | 122 | } |
| 125 | 123 | EXPORT_SYMBOL_GPL(rt2x00soc_remove); |
| 126 | 124 | |
| @@ -17,7 +17,7 @@ · 16 unmodified lines | ||
| 17 | 17 | * SoC driver handlers. |
| 18 | 18 | */ |
| 19 | 19 | int rt2x00soc_probe(struct platform_device *pdev, const struct rt2x00_ops *ops); |
| 20 | −int rt2x00soc_remove(struct platform_device *pdev); | |
| 20 | +void rt2x00soc_remove(struct platform_device *pdev); | |
| 21 | 21 | #ifdef CONFIG_PM |
| 22 | 22 | int rt2x00soc_suspend(struct platform_device *pdev, pm_message_t state); |
| 23 | 23 | int rt2x00soc_resume(struct platform_device *pdev); |
| @@ -583,7 +583,11 @@ void zd_mac_tx_to_dev(struct sk_buff *skb, int error) · 582 unmodified lines | ||
| 583 | 583 | |
| 584 | 584 | skb_queue_tail(q, skb); |
| 585 | 585 | while (skb_queue_len(q) > ZD_MAC_MAX_ACK_WAITERS) { |
| 586 | − zd_mac_tx_status(hw, skb_dequeue(q), | |
| 586 | + skb = skb_dequeue(q); | |
| 587 | + if (!skb) | |
| 588 | + break; | |
| 589 | + | |
| 590 | + zd_mac_tx_status(hw, skb, | |
| 587 | 591 | mac->ack_pending ? mac->ack_signal : 0, |
| 588 | 592 | NULL); |
| 589 | 593 | mac->ack_pending = 0; |
| @@ -64,13 +64,13 @@ int pci_host_common_init(struct platform_device *pdev, · 63 unmodified lines | ||
| 64 | 64 | |
| 65 | 65 | of_pci_check_probe_only(); |
| 66 | 66 | |
| 67 | + platform_set_drvdata(pdev, bridge); | |
| 68 | + | |
| 67 | 69 | /* Parse and map our Configuration Space windows */ |
| 68 | 70 | cfg = gen_pci_init(dev, bridge, ops); |
| 69 | 71 | if (IS_ERR(cfg)) |
| 70 | 72 | return PTR_ERR(cfg); |
| 71 | 73 | |
| 72 | − platform_set_drvdata(pdev, bridge); | |
| 73 | − | |
| 74 | 74 | bridge->sysdata = cfg; |
| 75 | 75 | bridge->ops = (struct pci_ops *)&ops->pci_ops; |
| 76 | 76 | bridge->enable_device = ops->enable_device; |
| @@ -187,6 +187,7 @@ struct apple_pcie { · 186 unmodified lines | ||
| 187 | 187 | const struct hw_info *hw; |
| 188 | 188 | unsigned long *bitmap; |
| 189 | 189 | struct list_head ports; |
| 190 | + struct list_head entry; | |
| 190 | 191 | struct completion event; |
| 191 | 192 | struct irq_fwspec fwspec; |
| 192 | 193 | u32 nvecs; |
| @@ -205,6 +206,9 @@ struct apple_pcie_port { · 12 unmodified lines | ||
| 205 | 206 | int idx; |
| 206 | 207 | }; |
| 207 | 208 | |
| 209 | +static LIST_HEAD(pcie_list); | |
| 210 | +static DEFINE_MUTEX(pcie_list_lock); | |
| 211 | + | |
| 208 | 212 | static void rmw_set(u32 set, void __iomem *addr) |
| 209 | 213 | { |
| 210 | 214 | writel_relaxed(readl_relaxed(addr) | set, addr); |
| @@ -720,13 +724,45 @@ static int apple_msi_init(struct apple_pcie *pcie) · 509 unmodified lines | ||
| 720 | 724 | return 0; |
| 721 | 725 | } |
| 722 | 726 | |
| 727 | +static void apple_pcie_register(struct apple_pcie *pcie) | |
| 728 | +{ | |
| 729 | + guard(mutex)(&pcie_list_lock); | |
| 730 | + | |
| 731 | + list_add_tail(&pcie->entry, &pcie_list); | |
| 732 | +} | |
| 733 | + | |
| 734 | +static void apple_pcie_unregister(struct apple_pcie *pcie) | |
| 735 | +{ | |
| 736 | + guard(mutex)(&pcie_list_lock); | |
| 737 | + | |
| 738 | + list_del(&pcie->entry); | |
| 739 | +} | |
| 740 | + | |
| 741 | +static struct apple_pcie *apple_pcie_lookup(struct device *dev) | |
| 742 | +{ | |
| 743 | + struct apple_pcie *pcie; | |
| 744 | + | |
| 745 | + guard(mutex)(&pcie_list_lock); | |
| 746 | + | |
| 747 | + list_for_each_entry(pcie, &pcie_list, entry) { | |
| 748 | + if (pcie->dev == dev) | |
| 749 | + return pcie; | |
| 750 | + } | |
| 751 | + | |
| 752 | + return NULL; | |
| 753 | +} | |
| 754 | + | |
| 723 | 755 | static struct apple_pcie_port *apple_pcie_get_port(struct pci_dev *pdev) |
| 724 | 756 | { |
| 725 | 757 | struct pci_config_window *cfg = pdev->sysdata; |
| 726 | − struct apple_pcie *pcie = cfg->priv; | |
| 758 | + struct apple_pcie *pcie; | |
| 727 | 759 | struct pci_dev *port_pdev; |
| 728 | 760 | struct apple_pcie_port *port; |
| 729 | 761 | |
| 762 | + pcie = apple_pcie_lookup(cfg->parent); | |
| 763 | + if (WARN_ON(!pcie)) | |
| 764 | + return NULL; | |
| 765 | + | |
| 730 | 766 | /* Find the root port this device is on */ |
| 731 | 767 | port_pdev = pcie_find_root_port(pdev); |
| 732 | 768 | |
| @@ -806,10 +842,14 @@ static void apple_pcie_disable_device(struct pci_host_bridge *bridge, struct pci · 73 unmodified lines | ||
| 806 | 842 | |
| 807 | 843 | static int apple_pcie_init(struct pci_config_window *cfg) |
| 808 | 844 | { |
| 809 | − struct apple_pcie *pcie = cfg->priv; | |
| 810 | 845 | struct device *dev = cfg->parent; |
| 846 | + struct apple_pcie *pcie; | |
| 811 | 847 | int ret; |
| 812 | 848 | |
| 849 | + pcie = apple_pcie_lookup(dev); | |
| 850 | + if (WARN_ON(!pcie)) | |
| 851 | + return -ENOENT; | |
| 852 | + | |
| 813 | 853 | for_each_available_child_of_node_scoped(dev->of_node, of_port) { |
| 814 | 854 | ret = apple_pcie_setup_port(pcie, of_port); |
| 815 | 855 | if (ret) { |
| @@ -852,13 +892,18 @@ static int apple_pcie_probe(struct platform_device *pdev) · 36 unmodified lines | ||
| 852 | 892 | |
| 853 | 893 | mutex_init(&pcie->lock); |
| 854 | 894 | INIT_LIST_HEAD(&pcie->ports); |
| 855 | − dev_set_drvdata(dev, pcie); | |
| 856 | 895 | |
| 857 | 896 | ret = apple_msi_init(pcie); |
| 858 | 897 | if (ret) |
| 859 | 898 | return ret; |
| 860 | 899 | |
| 861 | − return pci_host_common_init(pdev, &apple_pcie_cfg_ecam_ops); | |
| 900 | + apple_pcie_register(pcie); | |
| 901 | + | |
| 902 | + ret = pci_host_common_init(pdev, &apple_pcie_cfg_ecam_ops); | |
| 903 | + if (ret) | |
| 904 | + apple_pcie_unregister(pcie); | |
| 905 | + | |
| 906 | + return ret; | |
| 862 | 907 | } |
| 863 | 908 | |
| 864 | 909 | static const struct of_device_id apple_pcie_of_match[] = { |
| @@ -84,8 +84,6 @@ struct pci_config_window *pci_ecam_create(struct device *dev, · 83 unmodified lines | ||
| 84 | 84 | goto err_exit_iomap; |
| 85 | 85 | } |
| 86 | 86 | |
| 87 | − cfg->priv = dev_get_drvdata(dev); | |
| 88 | − | |
| 89 | 87 | if (ops->init) { |
| 90 | 88 | err = ops->init(cfg); |
| 91 | 89 | if (err) |
| @@ -934,10 +934,12 @@ int pci_msix_write_tph_tag(struct pci_dev *pdev, unsigned int index, u16 tag) · 933 unmodified lines | ||
| 934 | 934 | if (!pdev->msix_enabled) |
| 935 | 935 | return -ENXIO; |
| 936 | 936 | |
| 937 | − guard(msi_descs_lock)(&pdev->dev); | |
| 938 | 937 | virq = msi_get_virq(&pdev->dev, index); |
| 939 | 938 | if (!virq) |
| 940 | 939 | return -ENXIO; |
| 940 | + | |
| 941 | + guard(msi_descs_lock)(&pdev->dev); | |
| 942 | + | |
| 941 | 943 | /* |
| 942 | 944 | * This is a horrible hack, but short of implementing a PCI |
| 943 | 945 | * specific interrupt chip callback and a huge pile of |
| @@ -1074,7 +1074,10 @@ static int ma35_pinctrl_probe_dt(struct platform_device *pdev, struct ma35_pinct · 1073 unmodified lines | ||
| 1074 | 1074 | u32 idx = 0; |
| 1075 | 1075 | int ret; |
| 1076 | 1076 | |
| 1077 | − for_each_gpiochip_node(dev, child) { | |
| 1077 | + device_for_each_child_node(dev, child) { | |
| 1078 | + if (fwnode_property_present(child, "gpio-controller")) | |
| 1079 | + continue; | |
| 1080 | + | |
| 1078 | 1081 | npctl->nfunctions++; |
| 1079 | 1082 | npctl->ngroups += of_get_child_count(to_of_node(child)); |
| 1080 | 1083 | } |
| @@ -1092,7 +1095,10 @@ static int ma35_pinctrl_probe_dt(struct platform_device *pdev, struct ma35_pinct · 11 unmodified lines | ||
| 1092 | 1095 | if (!npctl->groups) |
| 1093 | 1096 | return -ENOMEM; |
| 1094 | 1097 | |
| 1095 | − for_each_gpiochip_node(dev, child) { | |
| 1098 | + device_for_each_child_node(dev, child) { | |
| 1099 | + if (fwnode_property_present(child, "gpio-controller")) | |
| 1100 | + continue; | |
| 1101 | + | |
| 1096 | 1102 | ret = ma35_pinctrl_parse_functions(child, npctl, idx++); |
| 1097 | 1103 | if (ret) { |
| 1098 | 1104 | fwnode_handle_put(child); |
| @@ -979,6 +979,17 @@ static int amd_gpio_suspend_hibernate_common(struct device *dev, bool is_suspend · 978 unmodified lines | ||
| 979 | 979 | pin, is_suspend ? "suspend" : "hibernate"); |
| 980 | 980 | } |
| 981 | 981 | |
| 982 | + /* | |
| 983 | + * debounce enabled over suspend has shown issues with a GPIO | |
| 984 | + * being unable to wake the system, as we're only interested in | |
| 985 | + * the actual wakeup event, clear it. | |
| 986 | + */ | |
| 987 | + if (gpio_dev->saved_regs[i] & (DB_CNTRl_MASK << DB_CNTRL_OFF)) { | |
| 988 | + amd_gpio_set_debounce(gpio_dev, pin, 0); | |
| 989 | + pm_pr_dbg("Clearing debounce for GPIO #%d during %s.\n", | |
| 990 | + pin, is_suspend ? "suspend" : "hibernate"); | |
| 991 | + } | |
| 992 | + | |
| 982 | 993 | raw_spin_unlock_irqrestore(&gpio_dev->lock, flags); |
| 983 | 994 | } |
| 984 | 995 | |
| @@ -784,7 +784,7 @@ static int aw9523_init_gpiochip(struct aw9523 *awi, unsigned int npins) · 783 unmodified lines | ||
| 784 | 784 | gc->set_config = gpiochip_generic_config; |
| 785 | 785 | gc->parent = dev; |
| 786 | 786 | gc->owner = THIS_MODULE; |
| 787 | − gc->can_sleep = false; | |
| 787 | + gc->can_sleep = true; | |
| 788 | 788 | |
| 789 | 789 | return 0; |
| 790 | 790 | } |
| @@ -1038,6 +1038,25 @@ static bool msm_gpio_needs_dual_edge_parent_workaround(struct irq_data *d, · 1037 unmodified lines | ||
| 1038 | 1038 | test_bit(d->hwirq, pctrl->skip_wake_irqs); |
| 1039 | 1039 | } |
| 1040 | 1040 | |
| 1041 | +static void msm_gpio_irq_init_valid_mask(struct gpio_chip *gc, | |
| 1042 | + unsigned long *valid_mask, | |
| 1043 | + unsigned int ngpios) | |
| 1044 | +{ | |
| 1045 | + struct msm_pinctrl *pctrl = gpiochip_get_data(gc); | |
| 1046 | + const struct msm_pingroup *g; | |
| 1047 | + int i; | |
| 1048 | + | |
| 1049 | + bitmap_fill(valid_mask, ngpios); | |
| 1050 | + | |
| 1051 | + for (i = 0; i < ngpios; i++) { | |
| 1052 | + g = &pctrl->soc->groups[i]; | |
| 1053 | + | |
| 1054 | + if (g->intr_detection_width != 1 && | |
| 1055 | + g->intr_detection_width != 2) | |
| 1056 | + clear_bit(i, valid_mask); | |
| 1057 | + } | |
| 1058 | +} | |
| 1059 | + | |
| 1041 | 1060 | static int msm_gpio_irq_set_type(struct irq_data *d, unsigned int type) |
| 1042 | 1061 | { |
| 1043 | 1062 | struct gpio_chip *gc = irq_data_get_irq_chip_data(d); |
| @@ -1441,6 +1460,7 @@ static int msm_gpio_init(struct msm_pinctrl *pctrl) · 397 unmodified lines | ||
| 1441 | 1460 | girq->default_type = IRQ_TYPE_NONE; |
| 1442 | 1461 | girq->handler = handle_bad_irq; |
| 1443 | 1462 | girq->parents[0] = pctrl->irq; |
| 1463 | + girq->init_valid_mask = msm_gpio_irq_init_valid_mask; | |
| 1444 | 1464 | |
| 1445 | 1465 | ret = devm_gpiochip_add_data(pctrl->dev, &pctrl->chip, pctrl); |
| 1446 | 1466 | if (ret) { |
| @@ -596,7 +596,7 @@ static bool pwm_state_valid(const struct pwm_state *state) · 595 unmodified lines | ||
| 596 | 596 | * and supposed to be ignored. So also ignore any strange values and |
| 597 | 597 | * consider the state ok. |
| 598 | 598 | */ |
| 599 | − if (state->enabled) | |
| 599 | + if (!state->enabled) | |
| 600 | 600 | return true; |
| 601 | 601 | |
| 602 | 602 | if (!state->period) |
| @@ -130,8 +130,10 @@ static int pwm_mediatek_config(struct pwm_chip *chip, struct pwm_device *pwm, · 129 unmodified lines | ||
| 130 | 130 | return ret; |
| 131 | 131 | |
| 132 | 132 | clk_rate = clk_get_rate(pc->clk_pwms[pwm->hwpwm]); |
| 133 | − if (!clk_rate) | |
| 134 | − return -EINVAL; | |
| 133 | + if (!clk_rate) { | |
| 134 | + ret = -EINVAL; | |
| 135 | + goto out; | |
| 136 | + } | |
| 135 | 137 | |
| 136 | 138 | /* Make sure we use the bus clock and not the 26MHz clock */ |
| 137 | 139 | if (pc->soc->has_ck_26m_sel) |
| @@ -150,9 +152,9 @@ static int pwm_mediatek_config(struct pwm_chip *chip, struct pwm_device *pwm, · 12 unmodified lines | ||
| 150 | 152 | } |
| 151 | 153 | |
| 152 | 154 | if (clkdiv > PWM_CLK_DIV_MAX) { |
| 153 | − pwm_mediatek_clk_disable(chip, pwm); | |
| 154 | 155 | dev_err(pwmchip_parent(chip), "period of %d ns not supported\n", period_ns); |
| 155 | − return -EINVAL; | |
| 156 | + ret = -EINVAL; | |
| 157 | + goto out; | |
| 156 | 158 | } |
| 157 | 159 | |
| 158 | 160 | if (pc->soc->pwm45_fixup && pwm->hwpwm > 2) { |
| @@ -169,9 +171,10 @@ static int pwm_mediatek_config(struct pwm_chip *chip, struct pwm_device *pwm, · 10 unmodified lines | ||
| 169 | 171 | pwm_mediatek_writel(pc, pwm->hwpwm, reg_width, cnt_period); |
| 170 | 172 | pwm_mediatek_writel(pc, pwm->hwpwm, reg_thres, cnt_duty); |
| 171 | 173 | |
| 174 | +out: | |
| 172 | 175 | pwm_mediatek_clk_disable(chip, pwm); |
| 173 | 176 | |
| 174 | − return 0; | |
| 177 | + return ret; | |
| 175 | 178 | } |
| 176 | 179 | |
| 177 | 180 | static int pwm_mediatek_enable(struct pwm_chip *chip, struct pwm_device *pwm) |
| @@ -85,7 +85,7 @@ void bch2_btree_node_to_freelist(struct bch_fs *c, struct btree *b) · 84 unmodified lines | ||
| 85 | 85 | six_unlock_intent(&b->c.lock); |
| 86 | 86 | } |
| 87 | 87 | |
| 88 | −static void __btree_node_data_free(struct btree_cache *bc, struct btree *b) | |
| 88 | +void __btree_node_data_free(struct btree *b) | |
| 89 | 89 | { |
| 90 | 90 | BUG_ON(!list_empty(&b->list)); |
| 91 | 91 | BUG_ON(btree_node_hashed(b)); |
| @@ -112,16 +112,17 @@ static void __btree_node_data_free(struct btree_cache *bc, struct btree *b) · 20 unmodified lines | ||
| 112 | 112 | munmap(b->aux_data, btree_aux_data_bytes(b)); |
| 113 | 113 | #endif |
| 114 | 114 | b->aux_data = NULL; |
| 115 | − | |
| 116 | − btree_node_to_freedlist(bc, b); | |
| 117 | 115 | } |
| 118 | 116 | |
| 119 | 117 | static void btree_node_data_free(struct btree_cache *bc, struct btree *b) |
| 120 | 118 | { |
| 121 | 119 | BUG_ON(list_empty(&b->list)); |
| 122 | 120 | list_del_init(&b->list); |
| 121 | + | |
| 122 | + __btree_node_data_free(b); | |
| 123 | + | |
| 123 | 124 | --bc->nr_freeable; |
| 124 | − __btree_node_data_free(bc, b); | |
| 125 | + btree_node_to_freedlist(bc, b); | |
| 125 | 126 | } |
| 126 | 127 | |
| 127 | 128 | static int bch2_btree_cache_cmp_fn(struct rhashtable_compare_arg *arg, |
| @@ -185,10 +186,7 @@ static struct btree *__btree_node_mem_alloc(struct bch_fs *c, gfp_t gfp) · 57 unmodified lines | ||
| 185 | 186 | |
| 186 | 187 | struct btree *__bch2_btree_node_mem_alloc(struct bch_fs *c) |
| 187 | 188 | { |
| 188 | − struct btree_cache *bc = &c->btree_cache; | |
| 189 | − struct btree *b; | |
| 190 | − | |
| 191 | − b = __btree_node_mem_alloc(c, GFP_KERNEL); | |
| 189 | + struct btree *b = __btree_node_mem_alloc(c, GFP_KERNEL); | |
| 192 | 190 | if (!b) |
| 193 | 191 | return NULL; |
| 194 | 192 | |
| @@ -198,8 +196,6 @@ struct btree *__bch2_btree_node_mem_alloc(struct bch_fs *c) · 3 unmodified lines | ||
| 198 | 196 | } |
| 199 | 197 | |
| 200 | 198 | bch2_btree_lock_init(&b->c, 0, GFP_KERNEL); |
| 201 | − | |
| 202 | − __bch2_btree_node_to_freelist(bc, b); | |
| 203 | 199 | return b; |
| 204 | 200 | } |
| 205 | 201 | |
| @@ -524,7 +520,8 @@ restart: · 318 unmodified lines | ||
| 524 | 520 | --touched;; |
| 525 | 521 | } else if (!btree_node_reclaim(c, b)) { |
| 526 | 522 | __bch2_btree_node_hash_remove(bc, b); |
| 527 | − __btree_node_data_free(bc, b); | |
| 523 | + __btree_node_data_free(b); | |
| 524 | + btree_node_to_freedlist(bc, b); | |
| 528 | 525 | |
| 529 | 526 | freed++; |
| 530 | 527 | bc->nr_freed++; |
| @@ -652,9 +649,12 @@ int bch2_fs_btree_cache_init(struct bch_fs *c) · 121 unmodified lines | ||
| 652 | 649 | |
| 653 | 650 | bch2_recalc_btree_reserve(c); |
| 654 | 651 | |
| 655 | − for (i = 0; i < bc->nr_reserve; i++) | |
| 656 | − if (!__bch2_btree_node_mem_alloc(c)) | |
| 652 | + for (i = 0; i < bc->nr_reserve; i++) { | |
| 653 | + struct btree *b = __bch2_btree_node_mem_alloc(c); | |
| 654 | + if (!b) | |
| 657 | 655 | goto err; |
| 656 | + __bch2_btree_node_to_freelist(bc, b); | |
| 657 | + } | |
| 658 | 658 | |
| 659 | 659 | list_splice_init(&bc->live[0].list, &bc->freeable); |
| 660 | 660 | |
| @@ -30,6 +30,7 @@ void bch2_btree_node_update_key_early(struct btree_trans *, enum btree_id, unsig · 29 unmodified lines | ||
| 30 | 30 | void bch2_btree_cache_cannibalize_unlock(struct btree_trans *); |
| 31 | 31 | int bch2_btree_cache_cannibalize_lock(struct btree_trans *, struct closure *); |
| 32 | 32 | |
| 33 | +void __btree_node_data_free(struct btree *); | |
| 33 | 34 | struct btree *__bch2_btree_node_mem_alloc(struct bch_fs *); |
| 34 | 35 | struct btree *bch2_btree_node_mem_alloc(struct btree_trans *, bool); |
| 35 | 36 | |
| @@ -568,9 +568,9 @@ static int __btree_err(int ret, · 567 unmodified lines | ||
| 568 | 568 | bch2_mark_btree_validate_failure(failed, ca->dev_idx); |
| 569 | 569 | |
| 570 | 570 | struct extent_ptr_decoded pick; |
| 571 | − have_retry = !bch2_bkey_pick_read_device(c, | |
| 571 | + have_retry = bch2_bkey_pick_read_device(c, | |
| 572 | 572 | bkey_i_to_s_c(&b->key), |
| 573 | − failed, &pick, -1); | |
| 573 | + failed, &pick, -1) == 1; | |
| 574 | 574 | } |
| 575 | 575 | |
| 576 | 576 | if (!have_retry && ret == -BCH_ERR_btree_node_read_err_want_retry) |
| @@ -615,7 +615,6 @@ static int __btree_err(int ret, · 38 unmodified lines | ||
| 615 | 615 | goto out; |
| 616 | 616 | case -BCH_ERR_btree_node_read_err_bad_node: |
| 617 | 617 | prt_str(&out, ", "); |
| 618 | − ret = __bch2_topology_error(c, &out); | |
| 619 | 618 | break; |
| 620 | 619 | } |
| 621 | 620 | |
| @@ -644,7 +643,6 @@ static int __btree_err(int ret, · 22 unmodified lines | ||
| 644 | 643 | goto out; |
| 645 | 644 | case -BCH_ERR_btree_node_read_err_bad_node: |
| 646 | 645 | prt_str(&out, ", "); |
| 647 | − ret = __bch2_topology_error(c, &out); | |
| 648 | 646 | break; |
| 649 | 647 | } |
| 650 | 648 | print: |
| @@ -1408,7 +1406,7 @@ static void btree_node_read_work(struct work_struct *work) · 757 unmodified lines | ||
| 1408 | 1406 | ret = bch2_bkey_pick_read_device(c, |
| 1409 | 1407 | bkey_i_to_s_c(&b->key), |
| 1410 | 1408 | &failed, &rb->pick, -1); |
| 1411 | − if (ret) { | |
| 1409 | + if (ret <= 0) { | |
| 1412 | 1410 | set_btree_node_read_error(b); |
| 1413 | 1411 | break; |
| 1414 | 1412 | } |
| @@ -75,39 +75,6 @@ static inline u64 bkey_journal_seq(struct bkey_s_c k) · 74 unmodified lines | ||
| 75 | 75 | } |
| 76 | 76 | } |
| 77 | 77 | |
| 78 | −static bool found_btree_node_is_readable(struct btree_trans *trans, | |
| 79 | − struct found_btree_node *f) | |
| 80 | −{ | |
| 81 | − struct { __BKEY_PADDED(k, BKEY_BTREE_PTR_VAL_U64s_MAX); } tmp; | |
| 82 | − | |
| 83 | − found_btree_node_to_key(&tmp.k, f); | |
| 84 | − | |
| 85 | − struct btree *b = bch2_btree_node_get_noiter(trans, &tmp.k, f->btree_id, f->level, false); | |
| 86 | − bool ret = !IS_ERR_OR_NULL(b); | |
| 87 | − if (!ret) | |
| 88 | − return ret; | |
| 89 | − | |
| 90 | − f->sectors_written = b->written; | |
| 91 | − f->journal_seq = le64_to_cpu(b->data->keys.journal_seq); | |
| 92 | − | |
| 93 | − struct bkey_s_c k; | |
| 94 | − struct bkey unpacked; | |
| 95 | − struct btree_node_iter iter; | |
| 96 | − for_each_btree_node_key_unpack(b, k, &iter, &unpacked) | |
| 97 | − f->journal_seq = max(f->journal_seq, bkey_journal_seq(k)); | |
| 98 | − | |
| 99 | − six_unlock_read(&b->c.lock); | |
| 100 | − | |
| 101 | − /* | |
| 102 | − * We might update this node's range; if that happens, we need the node | |
| 103 | − * to be re-read so the read path can trim keys that are no longer in | |
| 104 | − * this node | |
| 105 | − */ | |
| 106 | − if (b != btree_node_root(trans->c, b)) | |
| 107 | − bch2_btree_node_evict(trans, &tmp.k); | |
| 108 | − return ret; | |
| 109 | −} | |
| 110 | − | |
| 111 | 78 | static int found_btree_node_cmp_cookie(const void *_l, const void *_r) |
| 112 | 79 | { |
| 113 | 80 | const struct found_btree_node *l = _l; |
| @@ -159,17 +126,17 @@ static const struct min_heap_callbacks found_btree_node_heap_cbs = { · 45 unmodified lines | ||
| 159 | 126 | }; |
| 160 | 127 | |
| 161 | 128 | static void try_read_btree_node(struct find_btree_nodes *f, struct bch_dev *ca, |
| 162 | − struct bio *bio, struct btree_node *bn, u64 offset) | |
| 129 | + struct btree *b, struct bio *bio, u64 offset) | |
| 163 | 130 | { |
| 164 | 131 | struct bch_fs *c = container_of(f, struct bch_fs, found_btree_nodes); |
| 132 | + struct btree_node *bn = b->data; | |
| 165 | 133 | |
| 166 | 134 | bio_reset(bio, ca->disk_sb.bdev, REQ_OP_READ); |
| 167 | 135 | bio->bi_iter.bi_sector = offset; |
| 168 | − bch2_bio_map(bio, bn, PAGE_SIZE); | |
| 136 | + bch2_bio_map(bio, b->data, c->opts.block_size); | |
| 169 | 137 | |
| 170 | 138 | u64 submit_time = local_clock(); |
| 171 | 139 | submit_bio_wait(bio); |
| 172 | − | |
| 173 | 140 | bch2_account_io_completion(ca, BCH_MEMBER_ERROR_read, submit_time, !bio->bi_status); |
| 174 | 141 | |
| 175 | 142 | if (bio->bi_status) { |
| @@ -201,6 +168,14 @@ static void try_read_btree_node(struct find_btree_nodes *f, struct bch_dev *ca, · 25 unmodified lines | ||
| 201 | 168 | if (BTREE_NODE_ID(bn) >= BTREE_ID_NR_MAX) |
| 202 | 169 | return; |
| 203 | 170 | |
| 171 | + bio_reset(bio, ca->disk_sb.bdev, REQ_OP_READ); | |
| 172 | + bio->bi_iter.bi_sector = offset; | |
| 173 | + bch2_bio_map(bio, b->data, c->opts.btree_node_size); | |
| 174 | + | |
| 175 | + submit_time = local_clock(); | |
| 176 | + submit_bio_wait(bio); | |
| 177 | + bch2_account_io_completion(ca, BCH_MEMBER_ERROR_read, submit_time, !bio->bi_status); | |
| 178 | + | |
| 204 | 179 | rcu_read_lock(); |
| 205 | 180 | struct found_btree_node n = { |
| 206 | 181 | .btree_id = BTREE_NODE_ID(bn), |
| @@ -217,7 +192,20 @@ static void try_read_btree_node(struct find_btree_nodes *f, struct bch_dev *ca, · 10 unmodified lines | ||
| 217 | 192 | }; |
| 218 | 193 | rcu_read_unlock(); |
| 219 | 194 | |
| 220 | − if (bch2_trans_run(c, found_btree_node_is_readable(trans, &n))) { | |
| 195 | + found_btree_node_to_key(&b->key, &n); | |
| 196 | + | |
| 197 | + CLASS(printbuf, buf)(); | |
| 198 | + if (!bch2_btree_node_read_done(c, ca, b, NULL, &buf)) { | |
| 199 | + /* read_done will swap out b->data for another buffer */ | |
| 200 | + bn = b->data; | |
| 201 | + /* | |
| 202 | + * Grab journal_seq here because we want the max journal_seq of | |
| 203 | + * any bset; read_done sorts down to a single set and picks the | |
| 204 | + * max journal_seq | |
| 205 | + */ | |
| 206 | + n.journal_seq = le64_to_cpu(bn->keys.journal_seq), | |
| 207 | + n.sectors_written = b->written; | |
| 208 | + | |
| 221 | 209 | mutex_lock(&f->lock); |
| 222 | 210 | if (BSET_BIG_ENDIAN(&bn->keys) != CPU_BIG_ENDIAN) { |
| 223 | 211 | bch_err(c, "try_read_btree_node() can't handle endian conversion"); |
| @@ -237,12 +225,20 @@ static int read_btree_nodes_worker(void *p) · 13 unmodified lines | ||
| 237 | 225 | struct find_btree_nodes_worker *w = p; |
| 238 | 226 | struct bch_fs *c = container_of(w->f, struct bch_fs, found_btree_nodes); |
| 239 | 227 | struct bch_dev *ca = w->ca; |
| 240 | − void *buf = (void *) __get_free_page(GFP_KERNEL); | |
| 241 | − struct bio *bio = bio_alloc(NULL, 1, 0, GFP_KERNEL); | |
| 242 | 228 | unsigned long last_print = jiffies; |
| 229 | + struct btree *b = NULL; | |
| 230 | + struct bio *bio = NULL; | |
| 231 | + | |
| 232 | + b = __bch2_btree_node_mem_alloc(c); | |
| 233 | + if (!b) { | |
| 234 | + bch_err(c, "read_btree_nodes_worker: error allocating buf"); | |
| 235 | + w->f->ret = -ENOMEM; | |
| 236 | + goto err; | |
| 237 | + } | |
| 243 | 238 | |
| 244 | − if (!buf || !bio) { | |
| 245 | − bch_err(c, "read_btree_nodes_worker: error allocating bio/buf"); | |
| 239 | + bio = bio_alloc(NULL, buf_pages(b->data, c->opts.btree_node_size), 0, GFP_KERNEL); | |
| 240 | + if (!bio) { | |
| 241 | + bch_err(c, "read_btree_nodes_worker: error allocating bio"); | |
| 246 | 242 | w->f->ret = -ENOMEM; |
| 247 | 243 | goto err; |
| 248 | 244 | } |
| @@ -266,11 +262,13 @@ static int read_btree_nodes_worker(void *p) · 17 unmodified lines | ||
| 266 | 262 | !bch2_dev_btree_bitmap_marked_sectors(ca, sector, btree_sectors(c))) |
| 267 | 263 | continue; |
| 268 | 264 | |
| 269 | − try_read_btree_node(w->f, ca, bio, buf, sector); | |
| 265 | + try_read_btree_node(w->f, ca, b, bio, sector); | |
| 270 | 266 | } |
| 271 | 267 | err: |
| 268 | + if (b) | |
| 269 | + __btree_node_data_free(b); | |
| 270 | + kfree(b); | |
| 272 | 271 | bio_put(bio); |
| 273 | − free_page((unsigned long) buf); | |
| 274 | 272 | enumerated_ref_put(&ca->io_ref[READ], BCH_DEV_READ_REF_btree_node_scan); |
| 275 | 273 | closure_put(w->cl); |
| 276 | 274 | kfree(w); |
| @@ -153,8 +153,6 @@ void __bch2_btree_verify(struct bch_fs *c, struct btree *b) · 152 unmodified lines | ||
| 153 | 153 | c->verify_data = __bch2_btree_node_mem_alloc(c); |
| 154 | 154 | if (!c->verify_data) |
| 155 | 155 | goto out; |
| 156 | − | |
| 157 | − list_del_init(&c->verify_data->list); | |
| 158 | 156 | } |
| 159 | 157 | |
| 160 | 158 | BUG_ON(b->nsets != 1); |
| @@ -586,6 +584,8 @@ static ssize_t bch2_btree_transactions_read(struct file *file, char __user *buf, · 425 unmodified lines | ||
| 586 | 584 | i->ubuf = buf; |
| 587 | 585 | i->size = size; |
| 588 | 586 | i->ret = 0; |
| 587 | + | |
| 588 | + int srcu_idx = srcu_read_lock(&c->btree_trans_barrier); | |
| 589 | 589 | restart: |
| 590 | 590 | seqmutex_lock(&c->btree_trans_lock); |
| 591 | 591 | list_sort(&c->btree_trans_list, list_ptr_order_cmp); |
| @@ -599,6 +599,11 @@ restart: · 7 unmodified lines | ||
| 599 | 599 | if (!closure_get_not_zero(&trans->ref)) |
| 600 | 600 | continue; |
| 601 | 601 | |
| 602 | + if (!trans->srcu_held) { | |
| 603 | + closure_put(&trans->ref); | |
| 604 | + continue; | |
| 605 | + } | |
| 606 | + | |
| 602 | 607 | u32 seq = seqmutex_unlock(&c->btree_trans_lock); |
| 603 | 608 | |
| 604 | 609 | bch2_btree_trans_to_text(&i->buf, trans); |
| @@ -620,6 +625,8 @@ restart: · 15 unmodified lines | ||
| 620 | 625 | } |
| 621 | 626 | seqmutex_unlock(&c->btree_trans_lock); |
| 622 | 627 | unlocked: |
| 628 | + srcu_read_unlock(&c->btree_trans_barrier, srcu_idx); | |
| 629 | + | |
| 623 | 630 | if (i->buf.allocation_failure) |
| 624 | 631 | ret = -ENOMEM; |
| 625 | 632 | |
| @@ -282,7 +282,6 @@ · 281 unmodified lines | ||
| 282 | 282 | x(EIO, sb_not_downgraded) \ |
| 283 | 283 | x(EIO, btree_node_write_all_failed) \ |
| 284 | 284 | x(EIO, btree_node_read_error) \ |
| 285 | − x(EIO, btree_node_read_validate_error) \ | |
| 286 | 285 | x(EIO, btree_need_topology_repair) \ |
| 287 | 286 | x(EIO, bucket_ref_update) \ |
| 288 | 287 | x(EIO, trigger_alloc) \ |
| @@ -103,7 +103,7 @@ int __bch2_topology_error(struct bch_fs *c, struct printbuf *out) · 102 unmodified lines | ||
| 103 | 103 | return bch_err_throw(c, btree_need_topology_repair); |
| 104 | 104 | } else { |
| 105 | 105 | return bch2_run_explicit_recovery_pass(c, out, BCH_RECOVERY_PASS_check_topology, 0) ?: |
| 106 | − bch_err_throw(c, btree_node_read_validate_error); | |
| 106 | + bch_err_throw(c, btree_need_topology_repair); | |
| 107 | 107 | } |
| 108 | 108 | } |
| 109 | 109 | |
| @@ -633,7 +633,9 @@ err: · 523 unmodified lines | ||
| 633 | 633 | * log_fsck_err()s: that would require us to track for every error type |
| 634 | 634 | * which recovery pass corrects it, to get the fsck exit status correct: |
| 635 | 635 | */ |
| 636 | − if (bch2_err_matches(ret, BCH_ERR_fsck_fix)) { | |
| 636 | + if (bch2_err_matches(ret, BCH_ERR_transaction_restart)) { | |
| 637 | + /* nothing */ | |
| 638 | + } else if (bch2_err_matches(ret, BCH_ERR_fsck_fix)) { | |
| 637 | 639 | set_bit(BCH_FS_errors_fixed, &c->flags); |
| 638 | 640 | } else { |
| 639 | 641 | set_bit(BCH_FS_errors_not_fixed, &c->flags); |
| @@ -50,19 +50,17 @@ void bch2_io_failures_to_text(struct printbuf *out, · 49 unmodified lines | ||
| 50 | 50 | struct bch_io_failures *failed) |
| 51 | 51 | { |
| 52 | 52 | static const char * const error_types[] = { |
| 53 | − "io", "checksum", "ec reconstruct", NULL | |
| 53 | + "btree validate", "io", "checksum", "ec reconstruct", NULL | |
| 54 | 54 | }; |
| 55 | 55 | |
| 56 | 56 | for (struct bch_dev_io_failures *f = failed->devs; |
| 57 | 57 | f < failed->devs + failed->nr; |
| 58 | 58 | f++) { |
| 59 | 59 | unsigned errflags = |
| 60 | − ((!!f->failed_io) << 0) | | |
| 61 | − ((!!f->failed_csum_nr) << 1) | | |
| 62 | − ((!!f->failed_ec) << 2); | |
| 63 | − | |
| 64 | − if (!errflags) | |
| 65 | − continue; | |
| 60 | + ((!!f->failed_btree_validate) << 0) | | |
| 61 | + ((!!f->failed_io) << 1) | | |
| 62 | + ((!!f->failed_csum_nr) << 2) | | |
| 63 | + ((!!f->failed_ec) << 3); | |
| 66 | 64 | |
| 67 | 65 | bch2_printbuf_make_room(out, 1024); |
| 68 | 66 | out->atomic++; |
| @@ -77,7 +75,9 @@ void bch2_io_failures_to_text(struct printbuf *out, · 8 unmodified lines | ||
| 77 | 75 | |
| 78 | 76 | prt_char(out, ' '); |
| 79 | 77 | |
| 80 | − if (is_power_of_2(errflags)) { | |
| 78 | + if (!errflags) { | |
| 79 | + prt_str(out, "no error - confused"); | |
| 80 | + } else if (is_power_of_2(errflags)) { | |
| 81 | 81 | prt_bitflags(out, error_types, errflags); |
| 82 | 82 | prt_str(out, " error"); |
| 83 | 83 | } else { |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include "fs.h" |
| 13 | 13 | #include "fsck.h" |
| 14 | 14 | #include "inode.h" |
| 15 | +#include "io_misc.h" | |
| 15 | 16 | #include "keylist.h" |
| 16 | 17 | #include "namei.h" |
| 17 | 18 | #include "recovery_passes.h" |
| @@ -1919,33 +1920,11 @@ static int check_extent(struct btree_trans *trans, struct btree_iter *iter, · 1901 unmodified lines | ||
| 1919 | 1920 | "extent type past end of inode %llu:%u, i_size %llu\n%s", |
| 1920 | 1921 | i->inode.bi_inum, i->inode.bi_snapshot, i->inode.bi_size, |
| 1921 | 1922 | (bch2_bkey_val_to_text(&buf, c, k), buf.buf))) { |
| 1922 | − struct bkey_i *whiteout = bch2_trans_kmalloc(trans, sizeof(*whiteout)); | |
| 1923 | − ret = PTR_ERR_OR_ZERO(whiteout); | |
| 1924 | − if (ret) | |
| 1925 | − goto err; | |
| 1926 | − | |
| 1927 | − bkey_init(&whiteout->k); | |
| 1928 | − whiteout->k.p = SPOS(k.k->p.inode, | |
| 1929 | − last_block, | |
| 1930 | − i->inode.bi_snapshot); | |
| 1931 | − bch2_key_resize(&whiteout->k, | |
| 1932 | − min(KEY_SIZE_MAX & (~0 << c->block_bits), | |
| 1933 | − U64_MAX - whiteout->k.p.offset)); | |
| 1934 | − | |
| 1935 | − | |
| 1936 | − /* | |
| 1937 | − * Need a normal (not BTREE_ITER_all_snapshots) | |
| 1938 | − * iterator, if we're deleting in a different | |
| 1939 | − * snapshot and need to emit a whiteout | |
| 1940 | − */ | |
| 1941 | − struct btree_iter iter2; | |
| 1942 | − bch2_trans_iter_init(trans, &iter2, BTREE_ID_extents, | |
| 1943 | − bkey_start_pos(&whiteout->k), | |
| 1944 | − BTREE_ITER_intent); | |
| 1945 | − ret = bch2_btree_iter_traverse(trans, &iter2) ?: | |
| 1946 | − bch2_trans_update(trans, &iter2, whiteout, | |
| 1947 | − BTREE_UPDATE_internal_snapshot_node); | |
| 1948 | − bch2_trans_iter_exit(trans, &iter2); | |
| 1923 | + ret = bch2_fpunch_snapshot(trans, | |
| 1924 | + SPOS(i->inode.bi_inum, | |
| 1925 | + last_block, | |
| 1926 | + i->inode.bi_snapshot), | |
| 1927 | + POS(i->inode.bi_inum, U64_MAX)); | |
| 1949 | 1928 | if (ret) |
| 1950 | 1929 | goto err; |
| 1951 | 1930 | |
| @@ -135,6 +135,33 @@ err_noprint: · 134 unmodified lines | ||
| 135 | 135 | return ret; |
| 136 | 136 | } |
| 137 | 137 | |
| 138 | +/* For fsck */ | |
| 139 | +int bch2_fpunch_snapshot(struct btree_trans *trans, struct bpos start, struct bpos end) | |
| 140 | +{ | |
| 141 | + u32 restart_count = trans->restart_count; | |
| 142 | + struct bch_fs *c = trans->c; | |
| 143 | + struct disk_reservation disk_res = bch2_disk_reservation_init(c, 0); | |
| 144 | + unsigned max_sectors = KEY_SIZE_MAX & (~0 << c->block_bits); | |
| 145 | + struct bkey_i delete; | |
| 146 | + | |
| 147 | + int ret = for_each_btree_key_max_commit(trans, iter, BTREE_ID_extents, | |
| 148 | + start, end, 0, k, | |
| 149 | + &disk_res, NULL, BCH_TRANS_COMMIT_no_enospc, ({ | |
| 150 | + bkey_init(&delete.k); | |
| 151 | + delete.k.p = iter.pos; | |
| 152 | + | |
| 153 | + /* create the biggest key we can */ | |
| 154 | + bch2_key_resize(&delete.k, max_sectors); | |
| 155 | + bch2_cut_back(end, &delete); | |
| 156 | + | |
| 157 | + bch2_extent_trim_atomic(trans, &iter, &delete) ?: | |
| 158 | + bch2_trans_update(trans, &iter, &delete, 0); | |
| 159 | + })); | |
| 160 | + | |
| 161 | + bch2_disk_reservation_put(c, &disk_res); | |
| 162 | + return ret ?: trans_was_restarted(trans, restart_count); | |
| 163 | +} | |
| 164 | + | |
| 138 | 165 | /* |
| 139 | 166 | * Returns -BCH_ERR_transacton_restart if we had to drop locks: |
| 140 | 167 | */ |
| @@ -5,6 +5,8 @@ · 4 unmodified lines | ||
| 5 | 5 | int bch2_extent_fallocate(struct btree_trans *, subvol_inum, struct btree_iter *, |
| 6 | 6 | u64, struct bch_io_opts, s64 *, |
| 7 | 7 | struct write_point_specifier); |
| 8 | + | |
| 9 | +int bch2_fpunch_snapshot(struct btree_trans *, struct bpos, struct bpos); | |
| 8 | 10 | int bch2_fpunch_at(struct btree_trans *, struct btree_iter *, |
| 9 | 11 | subvol_inum, u64, s64 *); |
| 10 | 12 | int bch2_fpunch(struct bch_fs *c, subvol_inum, u64, u64, s64 *); |
| @@ -169,6 +169,12 @@ static struct journal_space __journal_space_available(struct journal *j, unsigne · 168 unmodified lines | ||
| 169 | 169 | if (nr_devs < nr_devs_want) |
| 170 | 170 | return (struct journal_space) { 0, 0 }; |
| 171 | 171 | |
| 172 | + /* | |
| 173 | + * It's possible for bucket size to be misaligned w.r.t. the filesystem | |
| 174 | + * block size: | |
| 175 | + */ | |
| 176 | + min_bucket_size = round_down(min_bucket_size, block_sectors(c)); | |
| 177 | + | |
| 172 | 178 | /* |
| 173 | 179 | * We sorted largest to smallest, and we want the smallest out of the |
| 174 | 180 | * @nr_devs_want largest devices: |
| @@ -273,24 +273,35 @@ static int bch2_journal_replay_key(struct btree_trans *trans, · 272 unmodified lines | ||
| 273 | 273 | goto out; |
| 274 | 274 | |
| 275 | 275 | struct btree_path *path = btree_iter_path(trans, &iter); |
| 276 | − if (unlikely(!btree_path_node(path, k->level) && | |
| 277 | − !k->allocated)) { | |
| 276 | + if (unlikely(!btree_path_node(path, k->level))) { | |
| 278 | 277 | struct bch_fs *c = trans->c; |
| 279 | 278 | |
| 279 | + CLASS(printbuf, buf)(); | |
| 280 | + prt_str(&buf, "btree="); | |
| 281 | + bch2_btree_id_to_text(&buf, k->btree_id); | |
| 282 | + prt_printf(&buf, " level=%u ", k->level); | |
| 283 | + bch2_bkey_val_to_text(&buf, c, bkey_i_to_s_c(k->k)); | |
| 284 | + | |
| 280 | 285 | if (!(c->recovery.passes_complete & (BIT_ULL(BCH_RECOVERY_PASS_scan_for_btree_nodes)| |
| 281 | 286 | BIT_ULL(BCH_RECOVERY_PASS_check_topology)))) { |
| 282 | − bch_err(c, "have key in journal replay for btree depth that does not exist, confused"); | |
| 287 | + bch_err(c, "have key in journal replay for btree depth that does not exist, confused\n%s", | |
| 288 | + buf.buf); | |
| 283 | 289 | ret = -EINVAL; |
| 284 | 290 | } |
| 285 | −#if 0 | |
| 291 | + | |
| 292 | + if (!k->allocated) { | |
| 293 | + bch_notice(c, "dropping key in journal replay for depth that does not exist because we're recovering from scan\n%s", | |
| 294 | + buf.buf); | |
| 295 | + k->overwritten = true; | |
| 296 | + goto out; | |
| 297 | + } | |
| 298 | + | |
| 286 | 299 | bch2_trans_iter_exit(trans, &iter); |
| 287 | 300 | bch2_trans_node_iter_init(trans, &iter, k->btree_id, k->k->k.p, |
| 288 | 301 | BTREE_MAX_DEPTH, 0, iter_flags); |
| 289 | 302 | ret = bch2_btree_iter_traverse(trans, &iter) ?: |
| 290 | 303 | bch2_btree_increase_depth(trans, iter.path, 0) ?: |
| 291 | 304 | -BCH_ERR_transaction_restart_nested; |
| 292 | −#endif | |
| 293 | − k->overwritten = true; | |
| 294 | 305 | goto out; |
| 295 | 306 | } |
| 296 | 307 | |
| @@ -360,7 +360,7 @@ int __bch2_run_explicit_recovery_pass(struct bch_fs *c, · 359 unmodified lines | ||
| 360 | 360 | !(r->passes_complete & BIT_ULL(pass)); |
| 361 | 361 | bool ratelimit = flags & RUN_RECOVERY_PASS_ratelimit; |
| 362 | 362 | |
| 363 | − if (!(in_recovery && (flags & RUN_RECOVERY_PASS_nopersistent))) { | |
| 363 | + if (!(flags & RUN_RECOVERY_PASS_nopersistent)) { | |
| 364 | 364 | struct bch_sb_field_ext *ext = bch2_sb_field_get(c->disk_sb.sb, ext); |
| 365 | 365 | __set_bit_le64(bch2_recovery_pass_to_stable(pass), ext->recovery_passes_required); |
| 366 | 366 | } |
| @@ -214,9 +214,11 @@ int erofs_map_dev(struct super_block *sb, struct erofs_map_dev *map) · 213 unmodified lines | ||
| 214 | 214 | |
| 215 | 215 | /* |
| 216 | 216 | * bit 30: I/O error occurred on this folio |
| 217 | + * bit 29: CPU has dirty data in D-cache (needs aliasing handling); | |
| 217 | 218 | * bit 0 - 29: remaining parts to complete this folio |
| 218 | 219 | */ |
| 219 | −#define EROFS_ONLINEFOLIO_EIO (1 << 30) | |
| 220 | +#define EROFS_ONLINEFOLIO_EIO 30 | |
| 221 | +#define EROFS_ONLINEFOLIO_DIRTY 29 | |
| 220 | 222 | |
| 221 | 223 | void erofs_onlinefolio_init(struct folio *folio) |
| 222 | 224 | { |
| @@ -233,19 +235,23 @@ void erofs_onlinefolio_split(struct folio *folio) · 10 unmodified lines | ||
| 233 | 235 | atomic_inc((atomic_t *)&folio->private); |
| 234 | 236 | } |
| 235 | 237 | |
| 236 | −void erofs_onlinefolio_end(struct folio *folio, int err) | |
| 238 | +void erofs_onlinefolio_end(struct folio *folio, int err, bool dirty) | |
| 237 | 239 | { |
| 238 | 240 | int orig, v; |
| 239 | 241 | |
| 240 | 242 | do { |
| 241 | 243 | orig = atomic_read((atomic_t *)&folio->private); |
| 242 | − v = (orig - 1) | (err ? EROFS_ONLINEFOLIO_EIO : 0); | |
| 244 | + DBG_BUGON(orig <= 0); | |
| 245 | + v = dirty << EROFS_ONLINEFOLIO_DIRTY; | |
| 246 | + v |= (orig - 1) | (!!err << EROFS_ONLINEFOLIO_EIO); | |
| 243 | 247 | } while (atomic_cmpxchg((atomic_t *)&folio->private, orig, v) != orig); |
| 244 | 248 | |
| 245 | − if (v & ~EROFS_ONLINEFOLIO_EIO) | |
| 249 | + if (v & (BIT(EROFS_ONLINEFOLIO_DIRTY) - 1)) | |
| 246 | 250 | return; |
| 247 | 251 | folio->private = 0; |
| 248 | − folio_end_read(folio, !(v & EROFS_ONLINEFOLIO_EIO)); | |
| 252 | + if (v & BIT(EROFS_ONLINEFOLIO_DIRTY)) | |
| 253 | + flush_dcache_folio(folio); | |
| 254 | + folio_end_read(folio, !(v & BIT(EROFS_ONLINEFOLIO_EIO))); | |
| 249 | 255 | } |
| 250 | 256 | |
| 251 | 257 | static int erofs_iomap_begin(struct inode *inode, loff_t offset, loff_t length, |
| @@ -351,11 +357,16 @@ int erofs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, · 99 unmodified lines | ||
| 351 | 357 | */ |
| 352 | 358 | static int erofs_read_folio(struct file *file, struct folio *folio) |
| 353 | 359 | { |
| 360 | + trace_erofs_read_folio(folio, true); | |
| 361 | + | |
| 354 | 362 | return iomap_read_folio(folio, &erofs_iomap_ops); |
| 355 | 363 | } |
| 356 | 364 | |
| 357 | 365 | static void erofs_readahead(struct readahead_control *rac) |
| 358 | 366 | { |
| 367 | + trace_erofs_readahead(rac->mapping->host, readahead_index(rac), | |
| 368 | + readahead_count(rac), true); | |
| 369 | + | |
| 359 | 370 | return iomap_readahead(rac, &erofs_iomap_ops); |
| 360 | 371 | } |
| 361 | 372 | |
| @@ -301,13 +301,11 @@ static int z_erofs_transform_plain(struct z_erofs_decompress_req *rq, · 300 unmodified lines | ||
| 301 | 301 | cur = min(cur, rq->outputsize); |
| 302 | 302 | if (cur && rq->out[0]) { |
| 303 | 303 | kin = kmap_local_page(rq->in[nrpages_in - 1]); |
| 304 | − if (rq->out[0] == rq->in[nrpages_in - 1]) { | |
| 304 | + if (rq->out[0] == rq->in[nrpages_in - 1]) | |
| 305 | 305 | memmove(kin + rq->pageofs_out, kin + pi, cur); |
| 306 | − flush_dcache_page(rq->out[0]); | |
| 307 | − } else { | |
| 306 | + else | |
| 308 | 307 | memcpy_to_page(rq->out[0], rq->pageofs_out, |
| 309 | 308 | kin + pi, cur); |
| 310 | − } | |
| 311 | 309 | kunmap_local(kin); |
| 312 | 310 | } |
| 313 | 311 | rq->outputsize -= cur; |
| @@ -325,14 +323,12 @@ static int z_erofs_transform_plain(struct z_erofs_decompress_req *rq, · 11 unmodified lines | ||
| 325 | 323 | po = (rq->pageofs_out + cur + pi) & ~PAGE_MASK; |
| 326 | 324 | DBG_BUGON(no >= nrpages_out); |
| 327 | 325 | cnt = min(insz - pi, PAGE_SIZE - po); |
| 328 | − if (rq->out[no] == rq->in[ni]) { | |
| 326 | + if (rq->out[no] == rq->in[ni]) | |
| 329 | 327 | memmove(kin + po, |
| 330 | 328 | kin + rq->pageofs_in + pi, cnt); |
| 331 | − flush_dcache_page(rq->out[no]); | |
| 332 | − } else if (rq->out[no]) { | |
| 329 | + else if (rq->out[no]) | |
| 333 | 330 | memcpy_to_page(rq->out[no], po, |
| 334 | 331 | kin + rq->pageofs_in + pi, cnt); |
| 335 | − } | |
| 336 | 332 | pi += cnt; |
| 337 | 333 | } while (pi < insz); |
| 338 | 334 | kunmap_local(kin); |
| @@ -58,6 +58,11 @@ static int erofs_readdir(struct file *f, struct dir_context *ctx) · 57 unmodified lines | ||
| 58 | 58 | struct erofs_dirent *de; |
| 59 | 59 | unsigned int nameoff, maxsize; |
| 60 | 60 | |
| 61 | + if (fatal_signal_pending(current)) { | |
| 62 | + err = -ERESTARTSYS; | |
| 63 | + break; | |
| 64 | + } | |
| 65 | + | |
| 61 | 66 | de = erofs_bread(&buf, dbstart, true); |
| 62 | 67 | if (IS_ERR(de)) { |
| 63 | 68 | erofs_err(sb, "failed to readdir of logical block %llu of nid %llu", |
| @@ -88,6 +93,7 @@ static int erofs_readdir(struct file *f, struct dir_context *ctx) · 24 unmodified lines | ||
| 88 | 93 | break; |
| 89 | 94 | ctx->pos = dbstart + maxsize; |
| 90 | 95 | ofs = 0; |
| 96 | + cond_resched(); | |
| 91 | 97 | } |
| 92 | 98 | erofs_put_metabuf(&buf); |
| 93 | 99 | if (EROFS_I(dir)->dot_omitted && ctx->pos == dir->i_size) { |
| @@ -38,7 +38,7 @@ static void erofs_fileio_ki_complete(struct kiocb *iocb, long ret) · 37 unmodified lines | ||
| 38 | 38 | } else { |
| 39 | 39 | bio_for_each_folio_all(fi, &rq->bio) { |
| 40 | 40 | DBG_BUGON(folio_test_uptodate(fi.folio)); |
| 41 | − erofs_onlinefolio_end(fi.folio, ret); | |
| 41 | + erofs_onlinefolio_end(fi.folio, ret, false); | |
| 42 | 42 | } |
| 43 | 43 | } |
| 44 | 44 | bio_uninit(&rq->bio); |
| @@ -96,8 +96,6 @@ static int erofs_fileio_scan_folio(struct erofs_fileio *io, struct folio *folio) · 51 unmodified lines | ||
| 96 | 96 | struct erofs_map_blocks *map = &io->map; |
| 97 | 97 | unsigned int cur = 0, end = folio_size(folio), len, attached = 0; |
| 98 | 98 | loff_t pos = folio_pos(folio), ofs; |
| 99 | − struct iov_iter iter; | |
| 100 | − struct bio_vec bv; | |
| 101 | 99 | int err = 0; |
| 102 | 100 | |
| 103 | 101 | erofs_onlinefolio_init(folio); |
| @@ -122,13 +120,7 @@ static int erofs_fileio_scan_folio(struct erofs_fileio *io, struct folio *folio) · 18 unmodified lines | ||
| 122 | 120 | err = PTR_ERR(src); |
| 123 | 121 | break; |
| 124 | 122 | } |
| 125 | − bvec_set_folio(&bv, folio, len, cur); | |
| 126 | − iov_iter_bvec(&iter, ITER_DEST, &bv, 1, len); | |
| 127 | − if (copy_to_iter(src, len, &iter) != len) { | |
| 128 | − erofs_put_metabuf(&buf); | |
| 129 | − err = -EIO; | |
| 130 | − break; | |
| 131 | − } | |
| 123 | + memcpy_to_folio(folio, cur, src, len); | |
| 132 | 124 | erofs_put_metabuf(&buf); |
| 133 | 125 | } else if (!(map->m_flags & EROFS_MAP_MAPPED)) { |
| 134 | 126 | folio_zero_segment(folio, cur, cur + len); |
| @@ -162,7 +154,7 @@ io_retry: · 27 unmodified lines | ||
| 162 | 154 | } |
| 163 | 155 | cur += len; |
| 164 | 156 | } |
| 165 | − erofs_onlinefolio_end(folio, err); | |
| 157 | + erofs_onlinefolio_end(folio, err, false); | |
| 166 | 158 | return err; |
| 167 | 159 | } |
| 168 | 160 | |
| @@ -315,10 +315,12 @@ static inline struct folio *erofs_grab_folio_nowait(struct address_space *as, · 314 unmodified lines | ||
| 315 | 315 | /* The length of extent is full */ |
| 316 | 316 | #define EROFS_MAP_FULL_MAPPED 0x0008 |
| 317 | 317 | /* Located in the special packed inode */ |
| 318 | −#define EROFS_MAP_FRAGMENT 0x0010 | |
| 318 | +#define __EROFS_MAP_FRAGMENT 0x0010 | |
| 319 | 319 | /* The extent refers to partial decompressed data */ |
| 320 | 320 | #define EROFS_MAP_PARTIAL_REF 0x0020 |
| 321 | 321 | |
| 322 | +#define EROFS_MAP_FRAGMENT (EROFS_MAP_MAPPED | __EROFS_MAP_FRAGMENT) | |
| 323 | + | |
| 322 | 324 | struct erofs_map_blocks { |
| 323 | 325 | struct erofs_buf buf; |
| 324 | 326 | |
| @@ -390,7 +392,7 @@ int erofs_fiemap(struct inode *inode, struct fiemap_extent_info *fieinfo, · 65 unmodified lines | ||
| 390 | 392 | int erofs_map_blocks(struct inode *inode, struct erofs_map_blocks *map); |
| 391 | 393 | void erofs_onlinefolio_init(struct folio *folio); |
| 392 | 394 | void erofs_onlinefolio_split(struct folio *folio); |
| 393 | −void erofs_onlinefolio_end(struct folio *folio, int err); | |
| 395 | +void erofs_onlinefolio_end(struct folio *folio, int err, bool dirty); | |
| 394 | 396 | struct inode *erofs_iget(struct super_block *sb, erofs_nid_t nid); |
| 395 | 397 | int erofs_getattr(struct mnt_idmap *idmap, const struct path *path, |
| 396 | 398 | struct kstat *stat, u32 request_mask, |
| @@ -1034,7 +1034,7 @@ static int z_erofs_scan_folio(struct z_erofs_frontend *f, · 1033 unmodified lines | ||
| 1034 | 1034 | if (!(map->m_flags & EROFS_MAP_MAPPED)) { |
| 1035 | 1035 | folio_zero_segment(folio, cur, end); |
| 1036 | 1036 | tight = false; |
| 1037 | − } else if (map->m_flags & EROFS_MAP_FRAGMENT) { | |
| 1037 | + } else if (map->m_flags & __EROFS_MAP_FRAGMENT) { | |
| 1038 | 1038 | erofs_off_t fpos = offset + cur - map->m_la; |
| 1039 | 1039 | |
| 1040 | 1040 | err = z_erofs_read_fragment(inode->i_sb, folio, cur, |
| @@ -1091,7 +1091,7 @@ static int z_erofs_scan_folio(struct z_erofs_frontend *f, · 50 unmodified lines | ||
| 1091 | 1091 | tight = (bs == PAGE_SIZE); |
| 1092 | 1092 | } |
| 1093 | 1093 | } while ((end = cur) > 0); |
| 1094 | − erofs_onlinefolio_end(folio, err); | |
| 1094 | + erofs_onlinefolio_end(folio, err, false); | |
| 1095 | 1095 | return err; |
| 1096 | 1096 | } |
| 1097 | 1097 | |
| @@ -1196,7 +1196,7 @@ static void z_erofs_fill_other_copies(struct z_erofs_backend *be, int err) · 98 unmodified lines | ||
| 1196 | 1196 | cur += len; |
| 1197 | 1197 | } |
| 1198 | 1198 | kunmap_local(dst); |
| 1199 | − erofs_onlinefolio_end(page_folio(bvi->bvec.page), err); | |
| 1199 | + erofs_onlinefolio_end(page_folio(bvi->bvec.page), err, true); | |
| 1200 | 1200 | list_del(p); |
| 1201 | 1201 | kfree(bvi); |
| 1202 | 1202 | } |
| @@ -1355,7 +1355,7 @@ static int z_erofs_decompress_pcluster(struct z_erofs_backend *be, int err) · 152 unmodified lines | ||
| 1355 | 1355 | |
| 1356 | 1356 | DBG_BUGON(z_erofs_page_is_invalidated(page)); |
| 1357 | 1357 | if (!z_erofs_is_shortlived_page(page)) { |
| 1358 | − erofs_onlinefolio_end(page_folio(page), err); | |
| 1358 | + erofs_onlinefolio_end(page_folio(page), err, true); | |
| 1359 | 1359 | continue; |
| 1360 | 1360 | } |
| 1361 | 1361 | if (pcl->algorithmformat != Z_EROFS_COMPRESSION_LZ4) { |
| @@ -413,8 +413,7 @@ static int z_erofs_map_blocks_fo(struct inode *inode, · 412 unmodified lines | ||
| 413 | 413 | !vi->z_tailextent_headlcn) { |
| 414 | 414 | map->m_la = 0; |
| 415 | 415 | map->m_llen = inode->i_size; |
| 416 | − map->m_flags = EROFS_MAP_MAPPED | | |
| 417 | − EROFS_MAP_FULL_MAPPED | EROFS_MAP_FRAGMENT; | |
| 416 | + map->m_flags = EROFS_MAP_FRAGMENT; | |
| 418 | 417 | return 0; |
| 419 | 418 | } |
| 420 | 419 | initial_lcn = ofs >> lclusterbits; |
| @@ -489,7 +488,7 @@ static int z_erofs_map_blocks_fo(struct inode *inode, · 68 unmodified lines | ||
| 489 | 488 | goto unmap_out; |
| 490 | 489 | } |
| 491 | 490 | } else if (fragment && m.lcn == vi->z_tailextent_headlcn) { |
| 492 | − map->m_flags |= EROFS_MAP_FRAGMENT; | |
| 491 | + map->m_flags = EROFS_MAP_FRAGMENT; | |
| 493 | 492 | } else { |
| 494 | 493 | map->m_pa = erofs_pos(sb, m.pblk); |
| 495 | 494 | err = z_erofs_get_extent_compressedlen(&m, initial_lcn); |
| @@ -617,7 +616,7 @@ static int z_erofs_map_blocks_ext(struct inode *inode, · 121 unmodified lines | ||
| 617 | 616 | if (lstart < lend) { |
| 618 | 617 | map->m_la = lstart; |
| 619 | 618 | if (last && (vi->z_advise & Z_EROFS_ADVISE_FRAGMENT_PCLUSTER)) { |
| 620 | − map->m_flags |= EROFS_MAP_MAPPED | EROFS_MAP_FRAGMENT; | |
| 619 | + map->m_flags = EROFS_MAP_FRAGMENT; | |
| 621 | 620 | vi->z_fragmentoff = map->m_plen; |
| 622 | 621 | if (recsz > offsetof(struct z_erofs_extent, pstart_lo)) |
| 623 | 622 | vi->z_fragmentoff |= map->m_pa << 32; |
| @@ -797,7 +796,7 @@ static int z_erofs_iomap_begin_report(struct inode *inode, loff_t offset, · 173 unmodified lines | ||
| 797 | 796 | iomap->length = map.m_llen; |
| 798 | 797 | if (map.m_flags & EROFS_MAP_MAPPED) { |
| 799 | 798 | iomap->type = IOMAP_MAPPED; |
| 800 | − iomap->addr = map.m_flags & EROFS_MAP_FRAGMENT ? | |
| 799 | + iomap->addr = map.m_flags & __EROFS_MAP_FRAGMENT ? | |
| 801 | 800 | IOMAP_NULL_ADDR : map.m_pa; |
| 802 | 801 | } else { |
| 803 | 802 | iomap->type = IOMAP_HOLE; |
| @@ -137,7 +137,13 @@ struct epitem { · 136 unmodified lines | ||
| 137 | 137 | }; |
| 138 | 138 | |
| 139 | 139 | /* List header used to link this structure to the eventpoll ready list */ |
| 140 | − struct llist_node rdllink; | |
| 140 | + struct list_head rdllink; | |
| 141 | + | |
| 142 | + /* | |
| 143 | + * Works together "struct eventpoll"->ovflist in keeping the | |
| 144 | + * single linked chain of items. | |
| 145 | + */ | |
| 146 | + struct epitem *next; | |
| 141 | 147 | |
| 142 | 148 | /* The file descriptor information this item refers to */ |
| 143 | 149 | struct epoll_filefd ffd; |
| @@ -185,15 +191,22 @@ struct eventpoll { · 41 unmodified lines | ||
| 185 | 191 | /* Wait queue used by file->poll() */ |
| 186 | 192 | wait_queue_head_t poll_wait; |
| 187 | 193 | |
| 188 | − /* | |
| 189 | − * List of ready file descriptors. Adding to this list is lockless. Items can be removed | |
| 190 | − * only with eventpoll::mtx | |
| 191 | − */ | |
| 192 | − struct llist_head rdllist; | |
| 194 | + /* List of ready file descriptors */ | |
| 195 | + struct list_head rdllist; | |
| 196 | + | |
| 197 | + /* Lock which protects rdllist and ovflist */ | |
| 198 | + rwlock_t lock; | |
| 193 | 199 | |
| 194 | 200 | /* RB tree root used to store monitored fd structs */ |
| 195 | 201 | struct rb_root_cached rbr; |
| 196 | 202 | |
| 203 | + /* | |
| 204 | + * This is a single linked list that chains all the "struct epitem" that | |
| 205 | + * happened while transferring ready events to userspace w/out | |
| 206 | + * holding ->lock. | |
| 207 | + */ | |
| 208 | + struct epitem *ovflist; | |
| 209 | + | |
| 197 | 210 | /* wakeup_source used when ep_send_events or __ep_eventpoll_poll is running */ |
| 198 | 211 | struct wakeup_source *ws; |
| 199 | 212 | |
| @@ -348,14 +361,10 @@ static inline int ep_cmp_ffd(struct epoll_filefd *p1, · 148 unmodified lines | ||
| 348 | 361 | (p1->file < p2->file ? -1 : p1->fd - p2->fd)); |
| 349 | 362 | } |
| 350 | 363 | |
| 351 | −/* | |
| 352 | − * Add the item to its container eventpoll's rdllist; do nothing if the item is already on rdllist. | |
| 353 | − */ | |
| 354 | −static void epitem_ready(struct epitem *epi) | |
| 364 | +/* Tells us if the item is currently linked */ | |
| 365 | +static inline int ep_is_linked(struct epitem *epi) | |
| 355 | 366 | { |
| 356 | − if (&epi->rdllink == cmpxchg(&epi->rdllink.next, &epi->rdllink, NULL)) | |
| 357 | − llist_add(&epi->rdllink, &epi->ep->rdllist); | |
| 358 | − | |
| 367 | + return !list_empty(&epi->rdllink); | |
| 359 | 368 | } |
| 360 | 369 | |
| 361 | 370 | static inline struct eppoll_entry *ep_pwq_from_wait(wait_queue_entry_t *p) |
| @@ -374,26 +383,13 @@ static inline struct epitem *ep_item_from_wait(wait_queue_entry_t *p) · 12 unmodified lines | ||
| 374 | 383 | * |
| 375 | 384 | * @ep: Pointer to the eventpoll context. |
| 376 | 385 | * |
| 377 | − * Return: true if ready events might be available, false otherwise. | |
| 386 | + * Return: a value different than %zero if ready events are available, | |
| 387 | + * or %zero otherwise. | |
| 378 | 388 | */ |
| 379 | −static inline bool ep_events_available(struct eventpoll *ep) | |
| 389 | +static inline int ep_events_available(struct eventpoll *ep) | |
| 380 | 390 | { |
| 381 | − bool available; | |
| 382 | − int locked; | |
| 383 | − | |
| 384 | − locked = mutex_trylock(&ep->mtx); | |
| 385 | − if (!locked) { | |
| 386 | − /* | |
| 387 | − * The lock held and someone might have removed all items while inspecting it. The | |
| 388 | − * llist_empty() check in this case is futile. Assume that something is enqueued and | |
| 389 | − * let ep_try_send_events() figure it out. | |
| 390 | − */ | |
| 391 | − return true; | |
| 392 | − } | |
| 393 | − | |
| 394 | − available = !llist_empty(&ep->rdllist); | |
| 395 | − mutex_unlock(&ep->mtx); | |
| 396 | − return available; | |
| 391 | + return !list_empty_careful(&ep->rdllist) || | |
| 392 | + READ_ONCE(ep->ovflist) != EP_UNACTIVE_PTR; | |
| 397 | 393 | } |
| 398 | 394 | |
| 399 | 395 | #ifdef CONFIG_NET_RX_BUSY_POLL |
| @@ -728,6 +724,77 @@ static inline void ep_pm_stay_awake_rcu(struct epitem *epi) · 328 unmodified lines | ||
| 728 | 724 | rcu_read_unlock(); |
| 729 | 725 | } |
| 730 | 726 | |
| 727 | + | |
| 728 | +/* | |
| 729 | + * ep->mutex needs to be held because we could be hit by | |
| 730 | + * eventpoll_release_file() and epoll_ctl(). | |
| 731 | + */ | |
| 732 | +static void ep_start_scan(struct eventpoll *ep, struct list_head *txlist) | |
| 733 | +{ | |
| 734 | + /* | |
| 735 | + * Steal the ready list, and re-init the original one to the | |
| 736 | + * empty list. Also, set ep->ovflist to NULL so that events | |
| 737 | + * happening while looping w/out locks, are not lost. We cannot | |
| 738 | + * have the poll callback to queue directly on ep->rdllist, | |
| 739 | + * because we want the "sproc" callback to be able to do it | |
| 740 | + * in a lockless way. | |
| 741 | + */ | |
| 742 | + lockdep_assert_irqs_enabled(); | |
| 743 | + write_lock_irq(&ep->lock); | |
| 744 | + list_splice_init(&ep->rdllist, txlist); | |
| 745 | + WRITE_ONCE(ep->ovflist, NULL); | |
| 746 | + write_unlock_irq(&ep->lock); | |
| 747 | +} | |
| 748 | + | |
| 749 | +static void ep_done_scan(struct eventpoll *ep, | |
| 750 | + struct list_head *txlist) | |
| 751 | +{ | |
| 752 | + struct epitem *epi, *nepi; | |
| 753 | + | |
| 754 | + write_lock_irq(&ep->lock); | |
| 755 | + /* | |
| 756 | + * During the time we spent inside the "sproc" callback, some | |
| 757 | + * other events might have been queued by the poll callback. | |
| 758 | + * We re-insert them inside the main ready-list here. | |
| 759 | + */ | |
| 760 | + for (nepi = READ_ONCE(ep->ovflist); (epi = nepi) != NULL; | |
| 761 | + nepi = epi->next, epi->next = EP_UNACTIVE_PTR) { | |
| 762 | + /* | |
| 763 | + * We need to check if the item is already in the list. | |
| 764 | + * During the "sproc" callback execution time, items are | |
| 765 | + * queued into ->ovflist but the "txlist" might already | |
| 766 | + * contain them, and the list_splice() below takes care of them. | |
| 767 | + */ | |
| 768 | + if (!ep_is_linked(epi)) { | |
| 769 | + /* | |
| 770 | + * ->ovflist is LIFO, so we have to reverse it in order | |
| 771 | + * to keep in FIFO. | |
| 772 | + */ | |
| 773 | + list_add(&epi->rdllink, &ep->rdllist); | |
| 774 | + ep_pm_stay_awake(epi); | |
| 775 | + } | |
| 776 | + } | |
| 777 | + /* | |
| 778 | + * We need to set back ep->ovflist to EP_UNACTIVE_PTR, so that after | |
| 779 | + * releasing the lock, events will be queued in the normal way inside | |
| 780 | + * ep->rdllist. | |
| 781 | + */ | |
| 782 | + WRITE_ONCE(ep->ovflist, EP_UNACTIVE_PTR); | |
| 783 | + | |
| 784 | + /* | |
| 785 | + * Quickly re-inject items left on "txlist". | |
| 786 | + */ | |
| 787 | + list_splice(txlist, &ep->rdllist); | |
| 788 | + __pm_relax(ep->ws); | |
| 789 | + | |
| 790 | + if (!list_empty(&ep->rdllist)) { | |
| 791 | + if (waitqueue_active(&ep->wq)) | |
| 792 | + wake_up(&ep->wq); | |
| 793 | + } | |
| 794 | + | |
| 795 | + write_unlock_irq(&ep->lock); | |
| 796 | +} | |
| 797 | + | |
| 731 | 798 | static void ep_get(struct eventpoll *ep) |
| 732 | 799 | { |
| 733 | 800 | refcount_inc(&ep->refcount); |
| @@ -765,12 +832,10 @@ static void ep_free(struct eventpoll *ep) · 31 unmodified lines | ||
| 765 | 832 | static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force) |
| 766 | 833 | { |
| 767 | 834 | struct file *file = epi->ffd.file; |
| 768 | − struct llist_node *put_back_last; | |
| 769 | 835 | struct epitems_head *to_free; |
| 770 | 836 | struct hlist_head *head; |
| 771 | − LLIST_HEAD(put_back); | |
| 772 | 837 | |
| 773 | − lockdep_assert_held(&ep->mtx); | |
| 838 | + lockdep_assert_irqs_enabled(); | |
| 774 | 839 | |
| 775 | 840 | /* |
| 776 | 841 | * Removes poll wait queue hooks. |
| @@ -802,20 +867,10 @@ static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force) · 25 unmodified lines | ||
| 802 | 867 | |
| 803 | 868 | rb_erase_cached(&epi->rbn, &ep->rbr); |
| 804 | 869 | |
| 805 | − if (llist_on_list(&epi->rdllink)) { | |
| 806 | − put_back_last = NULL; | |
| 807 | − while (true) { | |
| 808 | − struct llist_node *n = llist_del_first(&ep->rdllist); | |
| 809 | − | |
| 810 | − if (&epi->rdllink == n || WARN_ON(!n)) | |
| 811 | − break; | |
| 812 | − if (!put_back_last) | |
| 813 | − put_back_last = n; | |
| 814 | − __llist_add(n, &put_back); | |
| 815 | − } | |
| 816 | − if (put_back_last) | |
| 817 | − llist_add_batch(put_back.first, put_back_last, &ep->rdllist); | |
| 818 | − } | |
| 870 | + write_lock_irq(&ep->lock); | |
| 871 | + if (ep_is_linked(epi)) | |
| 872 | + list_del_init(&epi->rdllink); | |
| 873 | + write_unlock_irq(&ep->lock); | |
| 819 | 874 | |
| 820 | 875 | wakeup_source_unregister(ep_wakeup_source(epi)); |
| 821 | 876 | /* |
| @@ -828,7 +883,7 @@ static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force) · 6 unmodified lines | ||
| 828 | 883 | kfree_rcu(epi, rcu); |
| 829 | 884 | |
| 830 | 885 | percpu_counter_dec(&ep->user->epoll_watches); |
| 831 | − return ep_refcount_dec_and_test(ep); | |
| 886 | + return true; | |
| 832 | 887 | } |
| 833 | 888 | |
| 834 | 889 | /* |
| @@ -836,14 +891,14 @@ static bool __ep_remove(struct eventpoll *ep, struct epitem *epi, bool force) · 1 unmodified line | ||
| 836 | 891 | */ |
| 837 | 892 | static void ep_remove_safe(struct eventpoll *ep, struct epitem *epi) |
| 838 | 893 | { |
| 839 | − WARN_ON_ONCE(__ep_remove(ep, epi, false)); | |
| 894 | + if (__ep_remove(ep, epi, false)) | |
| 895 | + WARN_ON_ONCE(ep_refcount_dec_and_test(ep)); | |
| 840 | 896 | } |
| 841 | 897 | |
| 842 | 898 | static void ep_clear_and_put(struct eventpoll *ep) |
| 843 | 899 | { |
| 844 | 900 | struct rb_node *rbp, *next; |
| 845 | 901 | struct epitem *epi; |
| 846 | − bool dispose; | |
| 847 | 902 | |
| 848 | 903 | /* We need to release all tasks waiting for these file */ |
| 849 | 904 | if (waitqueue_active(&ep->poll_wait)) |
| @@ -876,10 +931,8 @@ static void ep_clear_and_put(struct eventpoll *ep) · 26 unmodified lines | ||
| 876 | 931 | cond_resched(); |
| 877 | 932 | } |
| 878 | 933 | |
| 879 | − dispose = ep_refcount_dec_and_test(ep); | |
| 880 | 934 | mutex_unlock(&ep->mtx); |
| 881 | − | |
| 882 | − if (dispose) | |
| 935 | + if (ep_refcount_dec_and_test(ep)) | |
| 883 | 936 | ep_free(ep); |
| 884 | 937 | } |
| 885 | 938 | |
| @@ -919,9 +972,8 @@ static __poll_t ep_item_poll(const struct epitem *epi, poll_table *pt, int depth · 33 unmodified lines | ||
| 919 | 972 | static __poll_t __ep_eventpoll_poll(struct file *file, poll_table *wait, int depth) |
| 920 | 973 | { |
| 921 | 974 | struct eventpoll *ep = file->private_data; |
| 922 | − struct wakeup_source *ws; | |
| 923 | − struct llist_node *n; | |
| 924 | − struct epitem *epi; | |
| 975 | + LIST_HEAD(txlist); | |
| 976 | + struct epitem *epi, *tmp; | |
| 925 | 977 | poll_table pt; |
| 926 | 978 | __poll_t res = 0; |
| 927 | 979 | |
| @@ -935,39 +987,22 @@ static __poll_t __ep_eventpoll_poll(struct file *file, poll_table *wait, int dep · 7 unmodified lines | ||
| 935 | 987 | * the ready list. |
| 936 | 988 | */ |
| 937 | 989 | mutex_lock_nested(&ep->mtx, depth); |
| 938 | − while (true) { | |
| 939 | − n = llist_del_first_init(&ep->rdllist); | |
| 940 | − if (!n) | |
| 941 | − break; | |
| 942 | − | |
| 943 | − epi = llist_entry(n, struct epitem, rdllink); | |
| 944 | − | |
| 990 | + ep_start_scan(ep, &txlist); | |
| 991 | + list_for_each_entry_safe(epi, tmp, &txlist, rdllink) { | |
| 945 | 992 | if (ep_item_poll(epi, &pt, depth + 1)) { |
| 946 | 993 | res = EPOLLIN | EPOLLRDNORM; |
| 947 | − epitem_ready(epi); | |
| 948 | 994 | break; |
| 949 | 995 | } else { |
| 950 | 996 | /* |
| 951 | − * We need to activate ep before deactivating epi, to prevent autosuspend | |
| 952 | − * just in case epi becomes active after ep_item_poll() above. | |
| 953 | − * | |
| 954 | − * This is similar to ep_send_events(). | |
| 997 | + * Item has been dropped into the ready list by the poll | |
| 998 | + * callback, but it's not actually ready, as far as | |
| 999 | + * caller requested events goes. We can remove it here. | |
| 955 | 1000 | */ |
| 956 | − ws = ep_wakeup_source(epi); | |
| 957 | − if (ws) { | |
| 958 | − if (ws->active) | |
| 959 | − __pm_stay_awake(ep->ws); | |
| 960 | − __pm_relax(ws); | |
| 961 | − } | |
| 962 | 1001 | __pm_relax(ep_wakeup_source(epi)); |
| 963 | − | |
| 964 | − /* Just in case epi becomes active right before __pm_relax() */ | |
| 965 | − if (unlikely(ep_item_poll(epi, &pt, depth + 1))) | |
| 966 | − ep_pm_stay_awake(epi); | |
| 967 | − | |
| 968 | − __pm_relax(ep->ws); | |
| 1002 | + list_del_init(&epi->rdllink); | |
| 969 | 1003 | } |
| 970 | 1004 | } |
| 1005 | + ep_done_scan(ep, &txlist); | |
| 971 | 1006 | mutex_unlock(&ep->mtx); |
| 972 | 1007 | return res; |
| 973 | 1008 | } |
| @@ -1100,7 +1135,7 @@ again: · 126 unmodified lines | ||
| 1100 | 1135 | dispose = __ep_remove(ep, epi, true); |
| 1101 | 1136 | mutex_unlock(&ep->mtx); |
| 1102 | 1137 | |
| 1103 | − if (dispose) | |
| 1138 | + if (dispose && ep_refcount_dec_and_test(ep)) | |
| 1104 | 1139 | ep_free(ep); |
| 1105 | 1140 | goto again; |
| 1106 | 1141 | } |
| @@ -1116,10 +1151,12 @@ static int ep_alloc(struct eventpoll **pep) · 9 unmodified lines | ||
| 1116 | 1151 | return -ENOMEM; |
| 1117 | 1152 | |
| 1118 | 1153 | mutex_init(&ep->mtx); |
| 1154 | + rwlock_init(&ep->lock); | |
| 1119 | 1155 | init_waitqueue_head(&ep->wq); |
| 1120 | 1156 | init_waitqueue_head(&ep->poll_wait); |
| 1121 | − init_llist_head(&ep->rdllist); | |
| 1157 | + INIT_LIST_HEAD(&ep->rdllist); | |
| 1122 | 1158 | ep->rbr = RB_ROOT_CACHED; |
| 1159 | + ep->ovflist = EP_UNACTIVE_PTR; | |
| 1123 | 1160 | ep->user = get_current_user(); |
| 1124 | 1161 | refcount_set(&ep->refcount, 1); |
| 1125 | 1162 | |
| @@ -1201,11 +1238,94 @@ struct file *get_epoll_tfile_raw_ptr(struct file *file, int tfd, · 75 unmodified lines | ||
| 1201 | 1238 | } |
| 1202 | 1239 | #endif /* CONFIG_KCMP */ |
| 1203 | 1240 | |
| 1241 | +/* | |
| 1242 | + * Adds a new entry to the tail of the list in a lockless way, i.e. | |
| 1243 | + * multiple CPUs are allowed to call this function concurrently. | |
| 1244 | + * | |
| 1245 | + * Beware: it is necessary to prevent any other modifications of the | |
| 1246 | + * existing list until all changes are completed, in other words | |
| 1247 | + * concurrent list_add_tail_lockless() calls should be protected | |
| 1248 | + * with a read lock, where write lock acts as a barrier which | |
| 1249 | + * makes sure all list_add_tail_lockless() calls are fully | |
| 1250 | + * completed. | |
| 1251 | + * | |
| 1252 | + * Also an element can be locklessly added to the list only in one | |
| 1253 | + * direction i.e. either to the tail or to the head, otherwise | |
| 1254 | + * concurrent access will corrupt the list. | |
| 1255 | + * | |
| 1256 | + * Return: %false if element has been already added to the list, %true | |
| 1257 | + * otherwise. | |
| 1258 | + */ | |
| 1259 | +static inline bool list_add_tail_lockless(struct list_head *new, | |
| 1260 | + struct list_head *head) | |
| 1261 | +{ | |
| 1262 | + struct list_head *prev; | |
| 1263 | + | |
| 1264 | + /* | |
| 1265 | + * This is simple 'new->next = head' operation, but cmpxchg() | |
| 1266 | + * is used in order to detect that same element has been just | |
| 1267 | + * added to the list from another CPU: the winner observes | |
| 1268 | + * new->next == new. | |
| 1269 | + */ | |
| 1270 | + if (!try_cmpxchg(&new->next, &new, head)) | |
| 1271 | + return false; | |
| 1272 | + | |
| 1273 | + /* | |
| 1274 | + * Initially ->next of a new element must be updated with the head | |
| 1275 | + * (we are inserting to the tail) and only then pointers are atomically | |
| 1276 | + * exchanged. XCHG guarantees memory ordering, thus ->next should be | |
| 1277 | + * updated before pointers are actually swapped and pointers are | |
| 1278 | + * swapped before prev->next is updated. | |
| 1279 | + */ | |
| 1280 | + | |
| 1281 | + prev = xchg(&head->prev, new); | |
| 1282 | + | |
| 1283 | + /* | |
| 1284 | + * It is safe to modify prev->next and new->prev, because a new element | |
| 1285 | + * is added only to the tail and new->next is updated before XCHG. | |
| 1286 | + */ | |
| 1287 | + | |
| 1288 | + prev->next = new; | |
| 1289 | + new->prev = prev; | |
| 1290 | + | |
| 1291 | + return true; | |
| 1292 | +} | |
| 1293 | + | |
| 1294 | +/* | |
| 1295 | + * Chains a new epi entry to the tail of the ep->ovflist in a lockless way, | |
| 1296 | + * i.e. multiple CPUs are allowed to call this function concurrently. | |
| 1297 | + * | |
| 1298 | + * Return: %false if epi element has been already chained, %true otherwise. | |
| 1299 | + */ | |
| 1300 | +static inline bool chain_epi_lockless(struct epitem *epi) | |
| 1301 | +{ | |
| 1302 | + struct eventpoll *ep = epi->ep; | |
| 1303 | + | |
| 1304 | + /* Fast preliminary check */ | |
| 1305 | + if (epi->next != EP_UNACTIVE_PTR) | |
| 1306 | + return false; | |
| 1307 | + | |
| 1308 | + /* Check that the same epi has not been just chained from another CPU */ | |
| 1309 | + if (cmpxchg(&epi->next, EP_UNACTIVE_PTR, NULL) != EP_UNACTIVE_PTR) | |
| 1310 | + return false; | |
| 1311 | + | |
| 1312 | + /* Atomically exchange tail */ | |
| 1313 | + epi->next = xchg(&ep->ovflist, epi); | |
| 1314 | + | |
| 1315 | + return true; | |
| 1316 | +} | |
| 1317 | + | |
| 1204 | 1318 | /* |
| 1205 | 1319 | * This is the callback that is passed to the wait queue wakeup |
| 1206 | 1320 | * mechanism. It is called by the stored file descriptors when they |
| 1207 | 1321 | * have events to report. |
| 1208 | 1322 | * |
| 1323 | + * This callback takes a read lock in order not to contend with concurrent | |
| 1324 | + * events from another file descriptor, thus all modifications to ->rdllist | |
| 1325 | + * or ->ovflist are lockless. Read lock is paired with the write lock from | |
| 1326 | + * ep_start/done_scan(), which stops all list modifications and guarantees | |
| 1327 | + * that lists state is seen correctly. | |
| 1328 | + * | |
| 1209 | 1329 | * Another thing worth to mention is that ep_poll_callback() can be called |
| 1210 | 1330 | * concurrently for the same @epi from different CPUs if poll table was inited |
| 1211 | 1331 | * with several wait queues entries. Plural wakeup from different CPUs of a |
| @@ -1215,11 +1335,15 @@ struct file *get_epoll_tfile_raw_ptr(struct file *file, int tfd, · 3 unmodified lines | ||
| 1215 | 1335 | */ |
| 1216 | 1336 | static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, void *key) |
| 1217 | 1337 | { |
| 1338 | + int pwake = 0; | |
| 1218 | 1339 | struct epitem *epi = ep_item_from_wait(wait); |
| 1219 | 1340 | struct eventpoll *ep = epi->ep; |
| 1220 | 1341 | __poll_t pollflags = key_to_poll(key); |
| 1342 | + unsigned long flags; | |
| 1221 | 1343 | int ewake = 0; |
| 1222 | 1344 | |
| 1345 | + read_lock_irqsave(&ep->lock, flags); | |
| 1346 | + | |
| 1223 | 1347 | ep_set_busy_poll_napi_id(epi); |
| 1224 | 1348 | |
| 1225 | 1349 | /* |
| @@ -1229,7 +1353,7 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v · 3 unmodified lines | ||
| 1229 | 1353 | * until the next EPOLL_CTL_MOD will be issued. |
| 1230 | 1354 | */ |
| 1231 | 1355 | if (!(epi->event.events & ~EP_PRIVATE_BITS)) |
| 1232 | − goto out; | |
| 1356 | + goto out_unlock; | |
| 1233 | 1357 | |
| 1234 | 1358 | /* |
| 1235 | 1359 | * Check the events coming with the callback. At this stage, not |
| @@ -1238,10 +1362,22 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v · 2 unmodified lines | ||
| 1238 | 1362 | * test for "key" != NULL before the event match test. |
| 1239 | 1363 | */ |
| 1240 | 1364 | if (pollflags && !(pollflags & epi->event.events)) |
| 1241 | − goto out; | |
| 1365 | + goto out_unlock; | |
| 1242 | 1366 | |
| 1243 | − ep_pm_stay_awake_rcu(epi); | |
| 1244 | − epitem_ready(epi); | |
| 1367 | + /* | |
| 1368 | + * If we are transferring events to userspace, we can hold no locks | |
| 1369 | + * (because we're accessing user memory, and because of linux f_op->poll() | |
| 1370 | + * semantics). All the events that happen during that period of time are | |
| 1371 | + * chained in ep->ovflist and requeued later on. | |
| 1372 | + */ | |
| 1373 | + if (READ_ONCE(ep->ovflist) != EP_UNACTIVE_PTR) { | |
| 1374 | + if (chain_epi_lockless(epi)) | |
| 1375 | + ep_pm_stay_awake_rcu(epi); | |
| 1376 | + } else if (!ep_is_linked(epi)) { | |
| 1377 | + /* In the usual case, add event to ready list. */ | |
| 1378 | + if (list_add_tail_lockless(&epi->rdllink, &ep->rdllist)) | |
| 1379 | + ep_pm_stay_awake_rcu(epi); | |
| 1380 | + } | |
| 1245 | 1381 | |
| 1246 | 1382 | /* |
| 1247 | 1383 | * Wake up ( if active ) both the eventpoll wait list and the ->poll() |
| @@ -1270,9 +1406,15 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v · 22 unmodified lines | ||
| 1270 | 1406 | wake_up(&ep->wq); |
| 1271 | 1407 | } |
| 1272 | 1408 | if (waitqueue_active(&ep->poll_wait)) |
| 1409 | + pwake++; | |
| 1410 | + | |
| 1411 | +out_unlock: | |
| 1412 | + read_unlock_irqrestore(&ep->lock, flags); | |
| 1413 | + | |
| 1414 | + /* We have to call this outside the lock */ | |
| 1415 | + if (pwake) | |
| 1273 | 1416 | ep_poll_safewake(ep, epi, pollflags & EPOLL_URING_WAKE); |
| 1274 | 1417 | |
| 1275 | −out: | |
| 1276 | 1418 | if (!(epi->event.events & EPOLLEXCLUSIVE)) |
| 1277 | 1419 | ewake = 1; |
| 1278 | 1420 | |
| @@ -1517,6 +1659,8 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, · 238 unmodified lines | ||
| 1517 | 1659 | if (is_file_epoll(tfile)) |
| 1518 | 1660 | tep = tfile->private_data; |
| 1519 | 1661 | |
| 1662 | + lockdep_assert_irqs_enabled(); | |
| 1663 | + | |
| 1520 | 1664 | if (unlikely(percpu_counter_compare(&ep->user->epoll_watches, |
| 1521 | 1665 | max_user_watches) >= 0)) |
| 1522 | 1666 | return -ENOSPC; |
| @@ -1528,10 +1672,11 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, · 5 unmodified lines | ||
| 1528 | 1672 | } |
| 1529 | 1673 | |
| 1530 | 1674 | /* Item initialization follow here ... */ |
| 1531 | − init_llist_node(&epi->rdllink); | |
| 1675 | + INIT_LIST_HEAD(&epi->rdllink); | |
| 1532 | 1676 | epi->ep = ep; |
| 1533 | 1677 | ep_set_ffd(&epi->ffd, tfile, fd); |
| 1534 | 1678 | epi->event = *event; |
| 1679 | + epi->next = EP_UNACTIVE_PTR; | |
| 1535 | 1680 | |
| 1536 | 1681 | if (tep) |
| 1537 | 1682 | mutex_lock_nested(&tep->mtx, 1); |
| @@ -1598,13 +1743,16 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, · 60 unmodified lines | ||
| 1598 | 1743 | return -ENOMEM; |
| 1599 | 1744 | } |
| 1600 | 1745 | |
| 1746 | + /* We have to drop the new item inside our item list to keep track of it */ | |
| 1747 | + write_lock_irq(&ep->lock); | |
| 1748 | + | |
| 1601 | 1749 | /* record NAPI ID of new item if present */ |
| 1602 | 1750 | ep_set_busy_poll_napi_id(epi); |
| 1603 | 1751 | |
| 1604 | 1752 | /* If the file is already "ready" we drop it inside the ready list */ |
| 1605 | − if (revents) { | |
| 1753 | + if (revents && !ep_is_linked(epi)) { | |
| 1754 | + list_add_tail(&epi->rdllink, &ep->rdllist); | |
| 1606 | 1755 | ep_pm_stay_awake(epi); |
| 1607 | − epitem_ready(epi); | |
| 1608 | 1756 | |
| 1609 | 1757 | /* Notify waiting tasks that events are available */ |
| 1610 | 1758 | if (waitqueue_active(&ep->wq)) |
| @@ -1613,6 +1761,8 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, · 2 unmodified lines | ||
| 1613 | 1761 | pwake++; |
| 1614 | 1762 | } |
| 1615 | 1763 | |
| 1764 | + write_unlock_irq(&ep->lock); | |
| 1765 | + | |
| 1616 | 1766 | /* We have to call this outside the lock */ |
| 1617 | 1767 | if (pwake) |
| 1618 | 1768 | ep_poll_safewake(ep, NULL, 0); |
| @@ -1627,8 +1777,11 @@ static int ep_insert(struct eventpoll *ep, const struct epoll_event *event, · 8 unmodified lines | ||
| 1627 | 1777 | static int ep_modify(struct eventpoll *ep, struct epitem *epi, |
| 1628 | 1778 | const struct epoll_event *event) |
| 1629 | 1779 | { |
| 1780 | + int pwake = 0; | |
| 1630 | 1781 | poll_table pt; |
| 1631 | 1782 | |
| 1783 | + lockdep_assert_irqs_enabled(); | |
| 1784 | + | |
| 1632 | 1785 | init_poll_funcptr(&pt, NULL); |
| 1633 | 1786 | |
| 1634 | 1787 | /* |
| @@ -1672,16 +1825,24 @@ static int ep_modify(struct eventpoll *ep, struct epitem *epi, · 37 unmodified lines | ||
| 1672 | 1825 | * list, push it inside. |
| 1673 | 1826 | */ |
| 1674 | 1827 | if (ep_item_poll(epi, &pt, 1)) { |
| 1675 | − ep_pm_stay_awake(epi); | |
| 1676 | − epitem_ready(epi); | |
| 1828 | + write_lock_irq(&ep->lock); | |
| 1829 | + if (!ep_is_linked(epi)) { | |
| 1830 | + list_add_tail(&epi->rdllink, &ep->rdllist); | |
| 1831 | + ep_pm_stay_awake(epi); | |
| 1677 | 1832 | |
| 1678 | − /* Notify waiting tasks that events are available */ | |
| 1679 | − if (waitqueue_active(&ep->wq)) | |
| 1680 | − wake_up(&ep->wq); | |
| 1681 | − if (waitqueue_active(&ep->poll_wait)) | |
| 1682 | − ep_poll_safewake(ep, NULL, 0); | |
| 1833 | + /* Notify waiting tasks that events are available */ | |
| 1834 | + if (waitqueue_active(&ep->wq)) | |
| 1835 | + wake_up(&ep->wq); | |
| 1836 | + if (waitqueue_active(&ep->poll_wait)) | |
| 1837 | + pwake++; | |
| 1838 | + } | |
| 1839 | + write_unlock_irq(&ep->lock); | |
| 1683 | 1840 | } |
| 1684 | 1841 | |
| 1842 | + /* We have to call this outside the lock */ | |
| 1843 | + if (pwake) | |
| 1844 | + ep_poll_safewake(ep, NULL, 0); | |
| 1845 | + | |
| 1685 | 1846 | return 0; |
| 1686 | 1847 | } |
| 1687 | 1848 | |
| @@ -1689,7 +1850,7 @@ static int ep_send_events(struct eventpoll *ep, · 1 unmodified line | ||
| 1689 | 1850 | struct epoll_event __user *events, int maxevents) |
| 1690 | 1851 | { |
| 1691 | 1852 | struct epitem *epi, *tmp; |
| 1692 | − LLIST_HEAD(txlist); | |
| 1853 | + LIST_HEAD(txlist); | |
| 1693 | 1854 | poll_table pt; |
| 1694 | 1855 | int res = 0; |
| 1695 | 1856 | |
| @@ -1704,18 +1865,19 @@ static int ep_send_events(struct eventpoll *ep, · 8 unmodified lines | ||
| 1704 | 1865 | init_poll_funcptr(&pt, NULL); |
| 1705 | 1866 | |
| 1706 | 1867 | mutex_lock(&ep->mtx); |
| 1868 | + ep_start_scan(ep, &txlist); | |
| 1707 | 1869 | |
| 1708 | − while (res < maxevents) { | |
| 1870 | + /* | |
| 1871 | + * We can loop without lock because we are passed a task private list. | |
| 1872 | + * Items cannot vanish during the loop we are holding ep->mtx. | |
| 1873 | + */ | |
| 1874 | + list_for_each_entry_safe(epi, tmp, &txlist, rdllink) { | |
| 1709 | 1875 | struct wakeup_source *ws; |
| 1710 | − struct llist_node *n; | |
| 1711 | 1876 | __poll_t revents; |
| 1712 | 1877 | |
| 1713 | − n = llist_del_first(&ep->rdllist); | |
| 1714 | − if (!n) | |
| 1878 | + if (res >= maxevents) | |
| 1715 | 1879 | break; |
| 1716 | 1880 | |
| 1717 | − epi = llist_entry(n, struct epitem, rdllink); | |
| 1718 | − | |
| 1719 | 1881 | /* |
| 1720 | 1882 | * Activate ep->ws before deactivating epi->ws to prevent |
| 1721 | 1883 | * triggering auto-suspend here (in case we reactive epi->ws |
| @@ -1732,30 +1894,21 @@ static int ep_send_events(struct eventpoll *ep, · 10 unmodified lines | ||
| 1732 | 1894 | __pm_relax(ws); |
| 1733 | 1895 | } |
| 1734 | 1896 | |
| 1897 | + list_del_init(&epi->rdllink); | |
| 1898 | + | |
| 1735 | 1899 | /* |
| 1736 | 1900 | * If the event mask intersect the caller-requested one, |
| 1737 | 1901 | * deliver the event to userspace. Again, we are holding ep->mtx, |
| 1738 | 1902 | * so no operations coming from userspace can change the item. |
| 1739 | 1903 | */ |
| 1740 | 1904 | revents = ep_item_poll(epi, &pt, 1); |
| 1741 | − if (!revents) { | |
| 1742 | − init_llist_node(n); | |
| 1743 | − | |
| 1744 | − /* | |
| 1745 | − * Just in case epi becomes ready after ep_item_poll() above, but before | |
| 1746 | − * init_llist_node(). Make sure to add it to the ready list, otherwise an | |
| 1747 | − * event may be lost. | |
| 1748 | − */ | |
| 1749 | − if (unlikely(ep_item_poll(epi, &pt, 1))) { | |
| 1750 | − ep_pm_stay_awake(epi); | |
| 1751 | − epitem_ready(epi); | |
| 1752 | − } | |
| 1905 | + if (!revents) | |
| 1753 | 1906 | continue; |
| 1754 | − } | |
| 1755 | 1907 | |
| 1756 | 1908 | events = epoll_put_uevent(revents, epi->event.data, events); |
| 1757 | 1909 | if (!events) { |
| 1758 | − llist_add(&epi->rdllink, &ep->rdllist); | |
| 1910 | + list_add(&epi->rdllink, &txlist); | |
| 1911 | + ep_pm_stay_awake(epi); | |
| 1759 | 1912 | if (!res) |
| 1760 | 1913 | res = -EFAULT; |
| 1761 | 1914 | break; |
| @@ -1763,31 +1916,25 @@ static int ep_send_events(struct eventpoll *ep, · 1 unmodified line | ||
| 1763 | 1916 | res++; |
| 1764 | 1917 | if (epi->event.events & EPOLLONESHOT) |
| 1765 | 1918 | epi->event.events &= EP_PRIVATE_BITS; |
| 1766 | − __llist_add(n, &txlist); | |
| 1767 | − } | |
| 1768 | − | |
| 1769 | − llist_for_each_entry_safe(epi, tmp, txlist.first, rdllink) { | |
| 1770 | − init_llist_node(&epi->rdllink); | |
| 1771 | − | |
| 1772 | − if (!(epi->event.events & EPOLLET)) { | |
| 1919 | + else if (!(epi->event.events & EPOLLET)) { | |
| 1773 | 1920 | /* |
| 1774 | − * If this file has been added with Level Trigger mode, we need to insert | |
| 1775 | − * back inside the ready list, so that the next call to epoll_wait() will | |
| 1776 | − * check again the events availability. | |
| 1921 | + * If this file has been added with Level | |
| 1922 | + * Trigger mode, we need to insert back inside | |
| 1923 | + * the ready list, so that the next call to | |
| 1924 | + * epoll_wait() will check again the events | |
| 1925 | + * availability. At this point, no one can insert | |
| 1926 | + * into ep->rdllist besides us. The epoll_ctl() | |
| 1927 | + * callers are locked out by | |
| 1928 | + * ep_send_events() holding "mtx" and the | |
| 1929 | + * poll callback will queue them in ep->ovflist. | |
| 1777 | 1930 | */ |
| 1931 | + list_add_tail(&epi->rdllink, &ep->rdllist); | |
| 1778 | 1932 | ep_pm_stay_awake(epi); |
| 1779 | − epitem_ready(epi); | |
| 1780 | 1933 | } |
| 1781 | 1934 | } |
| 1782 | − | |
| 1783 | − __pm_relax(ep->ws); | |
| 1935 | + ep_done_scan(ep, &txlist); | |
| 1784 | 1936 | mutex_unlock(&ep->mtx); |
| 1785 | 1937 | |
| 1786 | − if (!llist_empty(&ep->rdllist)) { | |
| 1787 | − if (waitqueue_active(&ep->wq)) | |
| 1788 | − wake_up(&ep->wq); | |
| 1789 | − } | |
| 1790 | − | |
| 1791 | 1938 | return res; |
| 1792 | 1939 | } |
| 1793 | 1940 | |
| @@ -1880,6 +2027,8 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, · 86 unmodified lines | ||
| 1880 | 2027 | wait_queue_entry_t wait; |
| 1881 | 2028 | ktime_t expires, *to = NULL; |
| 1882 | 2029 | |
| 2030 | + lockdep_assert_irqs_enabled(); | |
| 2031 | + | |
| 1883 | 2032 | if (timeout && (timeout->tv_sec | timeout->tv_nsec)) { |
| 1884 | 2033 | slack = select_estimate_accuracy(timeout); |
| 1885 | 2034 | to = &expires; |
| @@ -1939,15 +2088,54 @@ static int ep_poll(struct eventpoll *ep, struct epoll_event __user *events, · 53 unmodified lines | ||
| 1939 | 2088 | init_wait(&wait); |
| 1940 | 2089 | wait.func = ep_autoremove_wake_function; |
| 1941 | 2090 | |
| 1942 | − prepare_to_wait_exclusive(&ep->wq, &wait, TASK_INTERRUPTIBLE); | |
| 2091 | + write_lock_irq(&ep->lock); | |
| 2092 | + /* | |
| 2093 | + * Barrierless variant, waitqueue_active() is called under | |
| 2094 | + * the same lock on wakeup ep_poll_callback() side, so it | |
| 2095 | + * is safe to avoid an explicit barrier. | |
| 2096 | + */ | |
| 2097 | + __set_current_state(TASK_INTERRUPTIBLE); | |
| 2098 | + | |
| 2099 | + /* | |
| 2100 | + * Do the final check under the lock. ep_start/done_scan() | |
| 2101 | + * plays with two lists (->rdllist and ->ovflist) and there | |
| 2102 | + * is always a race when both lists are empty for short | |
| 2103 | + * period of time although events are pending, so lock is | |
| 2104 | + * important. | |
| 2105 | + */ | |
| 2106 | + eavail = ep_events_available(ep); | |
| 2107 | + if (!eavail) | |
| 2108 | + __add_wait_queue_exclusive(&ep->wq, &wait); | |
| 2109 | + | |
| 2110 | + write_unlock_irq(&ep->lock); | |
| 1943 | 2111 | |
| 1944 | − if (!ep_events_available(ep)) | |
| 2112 | + if (!eavail) | |
| 1945 | 2113 | timed_out = !ep_schedule_timeout(to) || |
| 1946 | 2114 | !schedule_hrtimeout_range(to, slack, |
| 1947 | 2115 | HRTIMER_MODE_ABS); |
| 2116 | + __set_current_state(TASK_RUNNING); | |
| 1948 | 2117 | |
| 1949 | − finish_wait(&ep->wq, &wait); | |
| 1950 | − eavail = ep_events_available(ep); | |
| 2118 | + /* | |
| 2119 | + * We were woken up, thus go and try to harvest some events. | |
| 2120 | + * If timed out and still on the wait queue, recheck eavail | |
| 2121 | + * carefully under lock, below. | |
| 2122 | + */ | |
| 2123 | + eavail = 1; | |
| 2124 | + | |
| 2125 | + if (!list_empty_careful(&wait.entry)) { | |
| 2126 | + write_lock_irq(&ep->lock); | |
| 2127 | + /* | |
| 2128 | + * If the thread timed out and is not on the wait queue, | |
| 2129 | + * it means that the thread was woken up after its | |
| 2130 | + * timeout expired before it could reacquire the lock. | |
| 2131 | + * Thus, when wait.entry is empty, it needs to harvest | |
| 2132 | + * events. | |
| 2133 | + */ | |
| 2134 | + if (timed_out) | |
| 2135 | + eavail = list_empty(&wait.entry); | |
| 2136 | + __remove_wait_queue(&ep->wq, &wait); | |
| 2137 | + write_unlock_irq(&ep->lock); | |
| 2138 | + } | |
| 1951 | 2139 | } |
| 1952 | 2140 | } |
| 1953 | 2141 | |
| @@ -36,9 +36,9 @@ void task_mem(struct seq_file *m, struct mm_struct *mm) · 35 unmodified lines | ||
| 36 | 36 | unsigned long text, lib, swap, anon, file, shmem; |
| 37 | 37 | unsigned long hiwater_vm, total_vm, hiwater_rss, total_rss; |
| 38 | 38 | |
| 39 | − anon = get_mm_counter(mm, MM_ANONPAGES); | |
| 40 | − file = get_mm_counter(mm, MM_FILEPAGES); | |
| 41 | − shmem = get_mm_counter(mm, MM_SHMEMPAGES); | |
| 39 | + anon = get_mm_counter_sum(mm, MM_ANONPAGES); | |
| 40 | + file = get_mm_counter_sum(mm, MM_FILEPAGES); | |
| 41 | + shmem = get_mm_counter_sum(mm, MM_SHMEMPAGES); | |
| 42 | 42 | |
| 43 | 43 | /* |
| 44 | 44 | * Note: to minimize their overhead, mm maintains hiwater_vm and |
| @@ -59,7 +59,7 @@ void task_mem(struct seq_file *m, struct mm_struct *mm) · 14 unmodified lines | ||
| 59 | 59 | text = min(text, mm->exec_vm << PAGE_SHIFT); |
| 60 | 60 | lib = (mm->exec_vm << PAGE_SHIFT) - text; |
| 61 | 61 | |
| 62 | − swap = get_mm_counter(mm, MM_SWAPENTS); | |
| 62 | + swap = get_mm_counter_sum(mm, MM_SWAPENTS); | |
| 63 | 63 | SEQ_PUT_DEC("VmPeak:\t", hiwater_vm); |
| 64 | 64 | SEQ_PUT_DEC(" kB\nVmSize:\t", total_vm); |
| 65 | 65 | SEQ_PUT_DEC(" kB\nVmLck:\t", mm->locked_vm); |
| @@ -92,12 +92,12 @@ unsigned long task_statm(struct mm_struct *mm, · 26 unmodified lines | ||
| 92 | 92 | unsigned long *shared, unsigned long *text, |
| 93 | 93 | unsigned long *data, unsigned long *resident) |
| 94 | 94 | { |
| 95 | − *shared = get_mm_counter(mm, MM_FILEPAGES) + | |
| 96 | − get_mm_counter(mm, MM_SHMEMPAGES); | |
| 95 | + *shared = get_mm_counter_sum(mm, MM_FILEPAGES) + | |
| 96 | + get_mm_counter_sum(mm, MM_SHMEMPAGES); | |
| 97 | 97 | *text = (PAGE_ALIGN(mm->end_code) - (mm->start_code & PAGE_MASK)) |
| 98 | 98 | >> PAGE_SHIFT; |
| 99 | 99 | *data = mm->data_vm + mm->stack_vm; |
| 100 | − *resident = *shared + get_mm_counter(mm, MM_ANONPAGES); | |
| 100 | + *resident = *shared + get_mm_counter_sum(mm, MM_ANONPAGES); | |
| 101 | 101 | return mm->total_vm; |
| 102 | 102 | } |
| 103 | 103 | |
| @@ -8573,11 +8573,6 @@ static void smb20_oplock_break_ack(struct ksmbd_work *work) · 8572 unmodified lines | ||
| 8573 | 8573 | goto err_out; |
| 8574 | 8574 | } |
| 8575 | 8575 | |
| 8576 | − opinfo->op_state = OPLOCK_STATE_NONE; | |
| 8577 | − wake_up_interruptible_all(&opinfo->oplock_q); | |
| 8578 | − opinfo_put(opinfo); | |
| 8579 | − ksmbd_fd_put(work, fp); | |
| 8580 | − | |
| 8581 | 8576 | rsp->StructureSize = cpu_to_le16(24); |
| 8582 | 8577 | rsp->OplockLevel = rsp_oplevel; |
| 8583 | 8578 | rsp->Reserved = 0; |
| @@ -8585,16 +8580,15 @@ static void smb20_oplock_break_ack(struct ksmbd_work *work) · 1 unmodified line | ||
| 8585 | 8580 | rsp->VolatileFid = volatile_id; |
| 8586 | 8581 | rsp->PersistentFid = persistent_id; |
| 8587 | 8582 | ret = ksmbd_iov_pin_rsp(work, rsp, sizeof(struct smb2_oplock_break)); |
| 8588 | − if (!ret) | |
| 8589 | − return; | |
| 8590 | − | |
| 8583 | + if (ret) { | |
| 8591 | 8584 | err_out: |
| 8585 | + smb2_set_err_rsp(work); | |
| 8586 | + } | |
| 8587 | + | |
| 8592 | 8588 | opinfo->op_state = OPLOCK_STATE_NONE; |
| 8593 | 8589 | wake_up_interruptible_all(&opinfo->oplock_q); |
| 8594 | − | |
| 8595 | 8590 | opinfo_put(opinfo); |
| 8596 | 8591 | ksmbd_fd_put(work, fp); |
| 8597 | − smb2_set_err_rsp(work); | |
| 8598 | 8592 | } |
| 8599 | 8593 | |
| 8600 | 8594 | static int check_lease_state(struct lease *lease, __le32 req_state) |
| @@ -8724,11 +8718,6 @@ static void smb21_lease_break_ack(struct ksmbd_work *work) · 123 unmodified lines | ||
| 8724 | 8718 | } |
| 8725 | 8719 | |
| 8726 | 8720 | lease_state = lease->state; |
| 8727 | − opinfo->op_state = OPLOCK_STATE_NONE; | |
| 8728 | − wake_up_interruptible_all(&opinfo->oplock_q); | |
| 8729 | − atomic_dec(&opinfo->breaking_cnt); | |
| 8730 | − wake_up_interruptible_all(&opinfo->oplock_brk); | |
| 8731 | − opinfo_put(opinfo); | |
| 8732 | 8721 | |
| 8733 | 8722 | rsp->StructureSize = cpu_to_le16(36); |
| 8734 | 8723 | rsp->Reserved = 0; |
| @@ -8737,16 +8726,16 @@ static void smb21_lease_break_ack(struct ksmbd_work *work) · 2 unmodified lines | ||
| 8737 | 8726 | rsp->LeaseState = lease_state; |
| 8738 | 8727 | rsp->LeaseDuration = 0; |
| 8739 | 8728 | ret = ksmbd_iov_pin_rsp(work, rsp, sizeof(struct smb2_lease_ack)); |
| 8740 | − if (!ret) | |
| 8741 | − return; | |
| 8742 | − | |
| 8729 | + if (ret) { | |
| 8743 | 8730 | err_out: |
| 8731 | + smb2_set_err_rsp(work); | |
| 8732 | + } | |
| 8733 | + | |
| 8734 | + opinfo->op_state = OPLOCK_STATE_NONE; | |
| 8744 | 8735 | wake_up_interruptible_all(&opinfo->oplock_q); |
| 8745 | 8736 | atomic_dec(&opinfo->breaking_cnt); |
| 8746 | 8737 | wake_up_interruptible_all(&opinfo->oplock_brk); |
| 8747 | − | |
| 8748 | 8738 | opinfo_put(opinfo); |
| 8749 | − smb2_set_err_rsp(work); | |
| 8750 | 8739 | } |
| 8751 | 8740 | |
| 8752 | 8741 | /** |
| @@ -433,7 +433,8 @@ static void free_transport(struct smb_direct_transport *t) · 432 unmodified lines | ||
| 433 | 433 | if (t->qp) { |
| 434 | 434 | ib_drain_qp(t->qp); |
| 435 | 435 | ib_mr_pool_destroy(t->qp, &t->qp->rdma_mrs); |
| 436 | − ib_destroy_qp(t->qp); | |
| 436 | + t->qp = NULL; | |
| 437 | + rdma_destroy_qp(t->cm_id); | |
| 437 | 438 | } |
| 438 | 439 | |
| 439 | 440 | ksmbd_debug(RDMA, "drain the reassembly queue\n"); |
| @@ -1940,8 +1941,8 @@ static int smb_direct_create_qpair(struct smb_direct_transport *t, · 1500 unmodified lines | ||
| 1940 | 1941 | return 0; |
| 1941 | 1942 | err: |
| 1942 | 1943 | if (t->qp) { |
| 1943 | − ib_destroy_qp(t->qp); | |
| 1944 | 1944 | t->qp = NULL; |
| 1945 | + rdma_destroy_qp(t->cm_id); | |
| 1945 | 1946 | } |
| 1946 | 1947 | if (t->recv_cq) { |
| 1947 | 1948 | ib_destroy_cq(t->recv_cq); |
| @@ -1282,6 +1282,7 @@ out1: · 1281 unmodified lines | ||
| 1282 | 1282 | |
| 1283 | 1283 | err = ksmbd_vfs_lock_parent(parent_path->dentry, path->dentry); |
| 1284 | 1284 | if (err) { |
| 1285 | + mnt_drop_write(parent_path->mnt); | |
| 1285 | 1286 | path_put(path); |
| 1286 | 1287 | path_put(parent_path); |
| 1287 | 1288 | } |
| @@ -300,6 +300,9 @@ struct drm_file { · 299 unmodified lines | ||
| 300 | 300 | * |
| 301 | 301 | * Mapping of mm object handles to object pointers. Used by the GEM |
| 302 | 302 | * subsystem. Protected by @table_lock. |
| 303 | + * | |
| 304 | + * Note that allocated entries might be NULL as a transient state when | |
| 305 | + * creating or deleting a handle. | |
| 303 | 306 | */ |
| 304 | 307 | struct idr object_idr; |
| 305 | 308 | |
| @@ -23,6 +23,7 @@ · 22 unmodified lines | ||
| 23 | 23 | #ifndef __DRM_FRAMEBUFFER_H__ |
| 24 | 24 | #define __DRM_FRAMEBUFFER_H__ |
| 25 | 25 | |
| 26 | +#include <linux/bits.h> | |
| 26 | 27 | #include <linux/ctype.h> |
| 27 | 28 | #include <linux/list.h> |
| 28 | 29 | #include <linux/sched.h> |
| @@ -100,6 +101,8 @@ struct drm_framebuffer_funcs { · 71 unmodified lines | ||
| 100 | 101 | unsigned num_clips); |
| 101 | 102 | }; |
| 102 | 103 | |
| 104 | +#define DRM_FRAMEBUFFER_HAS_HANDLE_REF(_i) BIT(0u + (_i)) | |
| 105 | + | |
| 103 | 106 | /** |
| 104 | 107 | * struct drm_framebuffer - frame buffer object |
| 105 | 108 | * |
| @@ -188,6 +191,10 @@ struct drm_framebuffer { · 82 unmodified lines | ||
| 188 | 191 | * DRM_MODE_FB_MODIFIERS. |
| 189 | 192 | */ |
| 190 | 193 | int flags; |
| 194 | + /** | |
| 195 | + * @internal_flags: Framebuffer flags like DRM_FRAMEBUFFER_HAS_HANDLE_REF. | |
| 196 | + */ | |
| 197 | + unsigned int internal_flags; | |
| 191 | 198 | /** |
| 192 | 199 | * @filp_head: Placed on &drm_file.fbs, protected by &drm_file.fbs_lock. |
| 193 | 200 | */ |
| @@ -269,11 +269,16 @@ static inline dev_t disk_devt(struct gendisk *disk) · 268 unmodified lines | ||
| 269 | 269 | return MKDEV(disk->major, disk->first_minor); |
| 270 | 270 | } |
| 271 | 271 | |
| 272 | +#ifdef CONFIG_TRANSPARENT_HUGEPAGE | |
| 272 | 273 | /* |
| 273 | 274 | * We should strive for 1 << (PAGE_SHIFT + MAX_PAGECACHE_ORDER) |
| 274 | 275 | * however we constrain this to what we can validate and test. |
| 275 | 276 | */ |
| 276 | 277 | #define BLK_MAX_BLOCK_SIZE SZ_64K |
| 278 | +#else | |
| 279 | +#define BLK_MAX_BLOCK_SIZE PAGE_SIZE | |
| 280 | +#endif | |
| 281 | + | |
| 277 | 282 | |
| 278 | 283 | /* blk_validate_limits() validates bsize, so drivers don't usually need to */ |
| 279 | 284 | static inline int blk_validate_block_size(unsigned long bsize) |
| @@ -82,6 +82,7 @@ extern ssize_t cpu_show_old_microcode(struct device *dev, · 81 unmodified lines | ||
| 82 | 82 | struct device_attribute *attr, char *buf); |
| 83 | 83 | extern ssize_t cpu_show_indirect_target_selection(struct device *dev, |
| 84 | 84 | struct device_attribute *attr, char *buf); |
| 85 | +extern ssize_t cpu_show_tsa(struct device *dev, struct device_attribute *attr, char *buf); | |
| 85 | 86 | |
| 86 | 87 | extern __printf(4, 5) |
| 87 | 88 | struct device *cpu_device_create(struct device *parent, void *drvdata, |
| @@ -662,18 +662,6 @@ static inline bool ieee80211_s1g_has_cssid(__le16 fc) · 661 unmodified lines | ||
| 662 | 662 | (fc & cpu_to_le16(IEEE80211_S1G_BCN_CSSID)); |
| 663 | 663 | } |
| 664 | 664 | |
| 665 | −/** | |
| 666 | − * ieee80211_is_s1g_short_beacon - check if frame is an S1G short beacon | |
| 667 | − * @fc: frame control bytes in little-endian byteorder | |
| 668 | − * Return: whether or not the frame is an S1G short beacon, | |
| 669 | − * i.e. it is an S1G beacon with 'next TBTT' flag set | |
| 670 | − */ | |
| 671 | −static inline bool ieee80211_is_s1g_short_beacon(__le16 fc) | |
| 672 | −{ | |
| 673 | − return ieee80211_is_s1g_beacon(fc) && | |
| 674 | − (fc & cpu_to_le16(IEEE80211_S1G_BCN_NEXT_TBTT)); | |
| 675 | −} | |
| 676 | − | |
| 677 | 665 | /** |
| 678 | 666 | * ieee80211_is_atim - check if IEEE80211_FTYPE_MGMT && IEEE80211_STYPE_ATIM |
| 679 | 667 | * @fc: frame control bytes in little-endian byteorder |
| @@ -4901,6 +4889,39 @@ static inline bool ieee80211_is_ftm(struct sk_buff *skb) · 4221 unmodified lines | ||
| 4901 | 4889 | return false; |
| 4902 | 4890 | } |
| 4903 | 4891 | |
| 4892 | +/** | |
| 4893 | + * ieee80211_is_s1g_short_beacon - check if frame is an S1G short beacon | |
| 4894 | + * @fc: frame control bytes in little-endian byteorder | |
| 4895 | + * @variable: pointer to the beacon frame elements | |
| 4896 | + * @variable_len: length of the frame elements | |
| 4897 | + * Return: whether or not the frame is an S1G short beacon. As per | |
| 4898 | + * IEEE80211-2024 11.1.3.10.1, The S1G beacon compatibility element shall | |
| 4899 | + * always be present as the first element in beacon frames generated at a | |
| 4900 | + * TBTT (Target Beacon Transmission Time), so any frame not containing | |
| 4901 | + * this element must have been generated at a TSBTT (Target Short Beacon | |
| 4902 | + * Transmission Time) that is not a TBTT. Additionally, short beacons are | |
| 4903 | + * prohibited from containing the S1G beacon compatibility element as per | |
| 4904 | + * IEEE80211-2024 9.3.4.3 Table 9-76, so if we have an S1G beacon with | |
| 4905 | + * either no elements or the first element is not the beacon compatibility | |
| 4906 | + * element, we have a short beacon. | |
| 4907 | + */ | |
| 4908 | +static inline bool ieee80211_is_s1g_short_beacon(__le16 fc, const u8 *variable, | |
| 4909 | + size_t variable_len) | |
| 4910 | +{ | |
| 4911 | + if (!ieee80211_is_s1g_beacon(fc)) | |
| 4912 | + return false; | |
| 4913 | + | |
| 4914 | + /* | |
| 4915 | + * If the frame does not contain at least 1 element (this is perfectly | |
| 4916 | + * valid in a short beacon) and is an S1G beacon, we have a short | |
| 4917 | + * beacon. | |
| 4918 | + */ | |
| 4919 | + if (variable_len < 2) | |
| 4920 | + return true; | |
| 4921 | + | |
| 4922 | + return variable[0] != WLAN_EID_S1G_BCN_COMPAT; | |
| 4923 | +} | |
| 4924 | + | |
| 4904 | 4925 | struct element { |
| 4905 | 4926 | u8 id; |
| 4906 | 4927 | u8 datalen; |
| @@ -698,6 +698,8 @@ struct io_kiocb { · 697 unmodified lines | ||
| 698 | 698 | struct hlist_node hash_node; |
| 699 | 699 | /* For IOPOLL setup queues, with hybrid polling */ |
| 700 | 700 | u64 iopoll_start; |
| 701 | + /* for private io_kiocb freeing */ | |
| 702 | + struct rcu_head rcu_head; | |
| 701 | 703 | }; |
| 702 | 704 | /* internal polling, see IORING_FEAT_FAST_POLL */ |
| 703 | 705 | struct async_poll *apoll; |
| @@ -17,6 +17,7 @@ · 16 unmodified lines | ||
| 17 | 17 | |
| 18 | 18 | #define MATCH_PLATFORM_MSI BIT(DOMAIN_BUS_PLATFORM_MSI) |
| 19 | 19 | |
| 20 | +struct msi_domain_info; | |
| 20 | 21 | int msi_lib_irq_domain_select(struct irq_domain *d, struct irq_fwspec *fwspec, |
| 21 | 22 | enum irq_domain_bus_token bus_token); |
| 22 | 23 | |
| @@ -2568,6 +2568,11 @@ static inline unsigned long get_mm_counter(struct mm_struct *mm, int member) · 2567 unmodified lines | ||
| 2568 | 2568 | return percpu_counter_read_positive(&mm->rss_stat[member]); |
| 2569 | 2569 | } |
| 2570 | 2570 | |
| 2571 | +static inline unsigned long get_mm_counter_sum(struct mm_struct *mm, int member) | |
| 2572 | +{ | |
| 2573 | + return percpu_counter_sum_positive(&mm->rss_stat[member]); | |
| 2574 | +} | |
| 2575 | + | |
| 2571 | 2576 | void mm_trace_rss_stat(struct mm_struct *mm, int member); |
| 2572 | 2577 | |
| 2573 | 2578 | static inline void add_mm_counter(struct mm_struct *mm, int member, long value) |
| @@ -594,6 +594,7 @@ struct sev_data_snp_addr { · 593 unmodified lines | ||
| 594 | 594 | * @imi_en: launch flow is launching an IMI (Incoming Migration Image) for the |
| 595 | 595 | * purpose of guest-assisted migration. |
| 596 | 596 | * @rsvd: reserved |
| 597 | + * @desired_tsc_khz: hypervisor desired mean TSC freq in kHz of the guest | |
| 597 | 598 | * @gosvw: guest OS-visible workarounds, as defined by hypervisor |
| 598 | 599 | */ |
| 599 | 600 | struct sev_data_snp_launch_start { |
| @@ -603,6 +604,7 @@ struct sev_data_snp_launch_start { · 3 unmodified lines | ||
| 603 | 604 | u32 ma_en:1; /* In */ |
| 604 | 605 | u32 imi_en:1; /* In */ |
| 605 | 606 | u32 rsvd:30; |
| 607 | + u32 desired_tsc_khz; /* In */ | |
| 606 | 608 | u8 gosvw[16]; /* In */ |
| 607 | 609 | } __packed; |
| 608 | 610 | |
| @@ -548,10 +548,6 @@ struct sched_statistics { · 547 unmodified lines | ||
| 548 | 548 | u64 nr_failed_migrations_running; |
| 549 | 549 | u64 nr_failed_migrations_hot; |
| 550 | 550 | u64 nr_forced_migrations; |
| 551 | −#ifdef CONFIG_NUMA_BALANCING | |
| 552 | − u64 numa_task_migrated; | |
| 553 | − u64 numa_task_swapped; | |
| 554 | −#endif | |
| 555 | 551 | |
| 556 | 552 | u64 nr_wakeups; |
| 557 | 553 | u64 nr_wakeups_sync; |
| @@ -66,8 +66,6 @@ enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT, · 65 unmodified lines | ||
| 66 | 66 | NUMA_HINT_FAULTS, |
| 67 | 67 | NUMA_HINT_FAULTS_LOCAL, |
| 68 | 68 | NUMA_PAGE_MIGRATE, |
| 69 | − NUMA_TASK_MIGRATE, | |
| 70 | − NUMA_TASK_SWAP, | |
| 71 | 69 | #endif |
| 72 | 70 | #ifdef CONFIG_MIGRATION |
| 73 | 71 | PGMIGRATE_SUCCESS, PGMIGRATE_FAIL, |
| @@ -243,8 +243,8 @@ int __vsock_dgram_recvmsg(struct socket *sock, struct msghdr *msg, · 242 unmodified lines | ||
| 243 | 243 | int vsock_dgram_recvmsg(struct socket *sock, struct msghdr *msg, |
| 244 | 244 | size_t len, int flags); |
| 245 | 245 | |
| 246 | −#ifdef CONFIG_BPF_SYSCALL | |
| 247 | 246 | extern struct proto vsock_proto; |
| 247 | +#ifdef CONFIG_BPF_SYSCALL | |
| 248 | 248 | int vsock_bpf_update_proto(struct sock *sk, struct sk_psock *psock, bool restore); |
| 249 | 249 | void __init vsock_bpf_build_proto(void); |
| 250 | 250 | #else |
| @@ -1350,8 +1350,7 @@ hci_conn_hash_lookup_big_state(struct hci_dev *hdev, __u8 handle, __u16 state) · 1349 unmodified lines | ||
| 1350 | 1350 | rcu_read_lock(); |
| 1351 | 1351 | |
| 1352 | 1352 | list_for_each_entry_rcu(c, &h->list, list) { |
| 1353 | − if (c->type != BIS_LINK || bacmp(&c->dst, BDADDR_ANY) || | |
| 1354 | − c->state != state) | |
| 1353 | + if (c->type != BIS_LINK || c->state != state) | |
| 1355 | 1354 | continue; |
| 1356 | 1355 | |
| 1357 | 1356 | if (handle == c->iso_qos.bcast.big) { |
| @@ -370,7 +370,7 @@ static inline __be16 __nf_flow_pppoe_proto(const struct sk_buff *skb) · 369 unmodified lines | ||
| 370 | 370 | |
| 371 | 371 | static inline bool nf_flow_pppoe_proto(struct sk_buff *skb, __be16 *inner_proto) |
| 372 | 372 | { |
| 373 | − if (!pskb_may_pull(skb, PPPOE_SES_HLEN)) | |
| 373 | + if (!pskb_may_pull(skb, ETH_HLEN + PPPOE_SES_HLEN)) | |
| 374 | 374 | return false; |
| 375 | 375 | |
| 376 | 376 | *inner_proto = __nf_flow_pppoe_proto(skb); |
| @@ -114,7 +114,6 @@ struct qdisc_rate_table *qdisc_get_rtab(struct tc_ratespec *r, · 113 unmodified lines | ||
| 114 | 114 | struct netlink_ext_ack *extack); |
| 115 | 115 | void qdisc_put_rtab(struct qdisc_rate_table *tab); |
| 116 | 116 | void qdisc_put_stab(struct qdisc_size_table *tab); |
| 117 | −void qdisc_warn_nonwc(const char *txt, struct Qdisc *qdisc); | |
| 118 | 117 | bool sch_direct_xmit(struct sk_buff *skb, struct Qdisc *q, |
| 119 | 118 | struct net_device *dev, struct netdev_queue *txq, |
| 120 | 119 | spinlock_t *root_lock, bool validate); |
| @@ -290,4 +289,28 @@ static inline bool tc_qdisc_stats_dump(struct Qdisc *sch, · 169 unmodified lines | ||
| 290 | 289 | return true; |
| 291 | 290 | } |
| 292 | 291 | |
| 292 | +static inline void qdisc_warn_nonwc(const char *txt, struct Qdisc *qdisc) | |
| 293 | +{ | |
| 294 | + if (!(qdisc->flags & TCQ_F_WARN_NONWC)) { | |
| 295 | + pr_warn("%s: %s qdisc %X: is non-work-conserving?\n", | |
| 296 | + txt, qdisc->ops->id, qdisc->handle >> 16); | |
| 297 | + qdisc->flags |= TCQ_F_WARN_NONWC; | |
| 298 | + } | |
| 299 | +} | |
| 300 | + | |
| 301 | +static inline unsigned int qdisc_peek_len(struct Qdisc *sch) | |
| 302 | +{ | |
| 303 | + struct sk_buff *skb; | |
| 304 | + unsigned int len; | |
| 305 | + | |
| 306 | + skb = sch->ops->peek(sch); | |
| 307 | + if (unlikely(skb == NULL)) { | |
| 308 | + qdisc_warn_nonwc("qdisc_peek_len", sch); | |
| 309 | + return 0; | |
| 310 | + } | |
| 311 | + len = qdisc_pkt_len(skb); | |
| 312 | + | |
| 313 | + return len; | |
| 314 | +} | |
| 315 | + | |
| 293 | 316 | #endif |
| @@ -40,13 +40,4 @@ · 39 unmodified lines | ||
| 40 | 40 | struct aic32x4_setup_data { |
| 41 | 41 | unsigned int gpio_func[5]; |
| 42 | 42 | }; |
| 43 | − | |
| 44 | −struct aic32x4_pdata { | |
| 45 | − struct aic32x4_setup_data *setup; | |
| 46 | − u32 power_cfg; | |
| 47 | − u32 micpga_routing; | |
| 48 | − bool swapdacs; | |
| 49 | − int rstn_gpio; | |
| 50 | −}; | |
| 51 | − | |
| 52 | 43 | #endif |
| @@ -4,9 +4,9 @@ · 3 unmodified lines | ||
| 4 | 4 | #ifndef _UAPI_LINUX_BITS_H |
| 5 | 5 | #define _UAPI_LINUX_BITS_H |
| 6 | 6 | |
| 7 | −#define __GENMASK(h, l) (((~_UL(0)) << (l)) & (~_UL(0) >> (BITS_PER_LONG - 1 - (h)))) | |
| 7 | +#define __GENMASK(h, l) (((~_UL(0)) << (l)) & (~_UL(0) >> (__BITS_PER_LONG - 1 - (h)))) | |
| 8 | 8 | |
| 9 | −#define __GENMASK_ULL(h, l) (((~_ULL(0)) << (l)) & (~_ULL(0) >> (BITS_PER_LONG_LONG - 1 - (h)))) | |
| 9 | +#define __GENMASK_ULL(h, l) (((~_ULL(0)) << (l)) & (~_ULL(0) >> (__BITS_PER_LONG_LONG - 1 - (h)))) | |
| 10 | 10 | |
| 11 | 11 | #define __GENMASK_U128(h, l) \ |
| 12 | 12 | ((_BIT128((h)) << 1) - (_BIT128(l))) |
| @@ -467,6 +467,10 @@ struct kvm_run { · 466 unmodified lines | ||
| 467 | 467 | __u64 leaf; |
| 468 | 468 | __u64 r11, r12, r13, r14; |
| 469 | 469 | } get_tdvmcall_info; |
| 470 | + struct { | |
| 471 | + __u64 ret; | |
| 472 | + __u64 vector; | |
| 473 | + } setup_event_notify; | |
| 470 | 474 | }; |
| 471 | 475 | } tdx; |
| 472 | 476 | /* Fix the size of the union. */ |
| @@ -1666,12 +1666,11 @@ static void io_iopoll_req_issued(struct io_kiocb *req, unsigned int issue_flags) · 1665 unmodified lines | ||
| 1666 | 1666 | |
| 1667 | 1667 | io_req_flags_t io_file_get_flags(struct file *file) |
| 1668 | 1668 | { |
| 1669 | − struct inode *inode = file_inode(file); | |
| 1670 | 1669 | io_req_flags_t res = 0; |
| 1671 | 1670 | |
| 1672 | 1671 | BUILD_BUG_ON(REQ_F_ISREG_BIT != REQ_F_SUPPORT_NOWAIT_BIT + 1); |
| 1673 | 1672 | |
| 1674 | − if (S_ISREG(inode->i_mode) && !(inode->i_flags & S_ANON_INODE)) | |
| 1673 | + if (S_ISREG(file_inode(file)->i_mode)) | |
| 1675 | 1674 | res |= REQ_F_ISREG; |
| 1676 | 1675 | if ((file->f_flags & O_NONBLOCK) || (file->f_mode & FMODE_NOWAIT)) |
| 1677 | 1676 | res |= REQ_F_SUPPORT_NOWAIT; |
| @@ -82,7 +82,7 @@ static void io_msg_tw_complete(struct io_kiocb *req, io_tw_token_t tw) · 81 unmodified lines | ||
| 82 | 82 | spin_unlock(&ctx->msg_lock); |
| 83 | 83 | } |
| 84 | 84 | if (req) |
| 85 | − kmem_cache_free(req_cachep, req); | |
| 85 | + kfree_rcu(req, rcu_head); | |
| 86 | 86 | percpu_ref_put(&ctx->refs); |
| 87 | 87 | } |
| 88 | 88 | |
| @@ -90,7 +90,7 @@ static int io_msg_remote_post(struct io_ring_ctx *ctx, struct io_kiocb *req, · 1 unmodified line | ||
| 90 | 90 | int res, u32 cflags, u64 user_data) |
| 91 | 91 | { |
| 92 | 92 | if (!READ_ONCE(ctx->submitter_task)) { |
| 93 | − kmem_cache_free(req_cachep, req); | |
| 93 | + kfree_rcu(req, rcu_head); | |
| 94 | 94 | return -EOWNERDEAD; |
| 95 | 95 | } |
| 96 | 96 | req->opcode = IORING_OP_NOP; |
| @@ -863,10 +863,7 @@ static int io_pp_zc_init(struct page_pool *pp) · 862 unmodified lines | ||
| 863 | 863 | static void io_pp_zc_destroy(struct page_pool *pp) |
| 864 | 864 | { |
| 865 | 865 | struct io_zcrx_ifq *ifq = io_pp_to_ifq(pp); |
| 866 | − struct io_zcrx_area *area = ifq->area; | |
| 867 | 866 | |
| 868 | − if (WARN_ON_ONCE(area->free_count != area->nia.num_niovs)) | |
| 869 | − return; | |
| 870 | 867 | percpu_ref_put(&ifq->ctx->refs); |
| 871 | 868 | } |
| 872 | 869 | |
| @@ -222,7 +222,10 @@ void __init dma_contiguous_reserve(phys_addr_t limit) · 221 unmodified lines | ||
| 222 | 222 | if (size_cmdline != -1) { |
| 223 | 223 | selected_size = size_cmdline; |
| 224 | 224 | selected_base = base_cmdline; |
| 225 | − selected_limit = min_not_zero(limit_cmdline, limit); | |
| 225 | + | |
| 226 | + /* Hornor the user setup dma address limit */ | |
| 227 | + selected_limit = limit_cmdline ?: limit; | |
| 228 | + | |
| 226 | 229 | if (base_cmdline + size_cmdline == limit_cmdline) |
| 227 | 230 | fixed = true; |
| 228 | 231 | } else { |
| @@ -7204,18 +7204,18 @@ void perf_event_wakeup(struct perf_event *event) · 7203 unmodified lines | ||
| 7204 | 7204 | static void perf_sigtrap(struct perf_event *event) |
| 7205 | 7205 | { |
| 7206 | 7206 | /* |
| 7207 | − * We'd expect this to only occur if the irq_work is delayed and either | |
| 7208 | − * ctx->task or current has changed in the meantime. This can be the | |
| 7209 | − * case on architectures that do not implement arch_irq_work_raise(). | |
| 7207 | + * Both perf_pending_task() and perf_pending_irq() can race with the | |
| 7208 | + * task exiting. | |
| 7210 | 7209 | */ |
| 7211 | − if (WARN_ON_ONCE(event->ctx->task != current)) | |
| 7210 | + if (current->flags & PF_EXITING) | |
| 7212 | 7211 | return; |
| 7213 | 7212 | |
| 7214 | 7213 | /* |
| 7215 | − * Both perf_pending_task() and perf_pending_irq() can race with the | |
| 7216 | − * task exiting. | |
| 7214 | + * We'd expect this to only occur if the irq_work is delayed and either | |
| 7215 | + * ctx->task or current has changed in the meantime. This can be the | |
| 7216 | + * case on architectures that do not implement arch_irq_work_raise(). | |
| 7217 | 7217 | */ |
| 7218 | − if (current->flags & PF_EXITING) | |
| 7218 | + if (WARN_ON_ONCE(event->ctx->task != current)) | |
| 7219 | 7219 | return; |
| 7220 | 7220 | |
| 7221 | 7221 | send_sig_perf((void __user *)event->pending_addr, |
| @@ -1573,8 +1573,14 @@ static int apply_relocations(struct module *mod, const struct load_info *info) · 1572 unmodified lines | ||
| 1573 | 1573 | if (infosec >= info->hdr->e_shnum) |
| 1574 | 1574 | continue; |
| 1575 | 1575 | |
| 1576 | − /* Don't bother with non-allocated sections */ | |
| 1577 | − if (!(info->sechdrs[infosec].sh_flags & SHF_ALLOC)) | |
| 1576 | + /* | |
| 1577 | + * Don't bother with non-allocated sections. | |
| 1578 | + * An exception is the percpu section, which has separate allocations | |
| 1579 | + * for individual CPUs. We relocate the percpu section in the initial | |
| 1580 | + * ELF template and subsequently copy it to the per-CPU destinations. | |
| 1581 | + */ | |
| 1582 | + if (!(info->sechdrs[infosec].sh_flags & SHF_ALLOC) && | |
| 1583 | + (!infosec || infosec != info->index.pcpu)) | |
| 1578 | 1584 | continue; |
| 1579 | 1585 | |
| 1580 | 1586 | if (info->sechdrs[i].sh_flags & SHF_RELA_LIVEPATCH) |
| @@ -2696,9 +2702,8 @@ static int find_module_sections(struct module *mod, struct load_info *info) · 1115 unmodified lines | ||
| 2696 | 2702 | |
| 2697 | 2703 | static int move_module(struct module *mod, struct load_info *info) |
| 2698 | 2704 | { |
| 2699 | − int i; | |
| 2700 | − enum mod_mem_type t = 0; | |
| 2701 | − int ret = -ENOMEM; | |
| 2705 | + int i, ret; | |
| 2706 | + enum mod_mem_type t = MOD_MEM_NUM_TYPES; | |
| 2702 | 2707 | bool codetag_section_found = false; |
| 2703 | 2708 | |
| 2704 | 2709 | for_each_mod_mem_type(type) { |
| @@ -2776,7 +2781,7 @@ static int move_module(struct module *mod, struct load_info *info) · 71 unmodified lines | ||
| 2776 | 2781 | return 0; |
| 2777 | 2782 | out_err: |
| 2778 | 2783 | module_memory_restore_rox(mod); |
| 2779 | − for (t--; t >= 0; t--) | |
| 2784 | + while (t--) | |
| 2780 | 2785 | module_memory_free(mod, t); |
| 2781 | 2786 | if (codetag_section_found) |
| 2782 | 2787 | codetag_free_module_sections(mod); |
| @@ -3362,10 +3362,6 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu) · 3361 unmodified lines | ||
| 3362 | 3362 | #ifdef CONFIG_NUMA_BALANCING |
| 3363 | 3363 | static void __migrate_swap_task(struct task_struct *p, int cpu) |
| 3364 | 3364 | { |
| 3365 | − __schedstat_inc(p->stats.numa_task_swapped); | |
| 3366 | − count_vm_numa_event(NUMA_TASK_SWAP); | |
| 3367 | − count_memcg_event_mm(p->mm, NUMA_TASK_SWAP); | |
| 3368 | − | |
| 3369 | 3365 | if (task_on_rq_queued(p)) { |
| 3370 | 3366 | struct rq *src_rq, *dst_rq; |
| 3371 | 3367 | struct rq_flags srf, drf; |
| @@ -7939,9 +7935,8 @@ int migrate_task_to(struct task_struct *p, int target_cpu) · 4567 unmodified lines | ||
| 7939 | 7935 | if (!cpumask_test_cpu(target_cpu, p->cpus_ptr)) |
| 7940 | 7936 | return -EINVAL; |
| 7941 | 7937 | |
| 7942 | − __schedstat_inc(p->stats.numa_task_migrated); | |
| 7943 | − count_vm_numa_event(NUMA_TASK_MIGRATE); | |
| 7944 | − count_memcg_event_mm(p->mm, NUMA_TASK_MIGRATE); | |
| 7938 | + /* TODO: This is not properly updating schedstats */ | |
| 7939 | + | |
| 7945 | 7940 | trace_sched_move_numa(p, curr_cpu, target_cpu); |
| 7946 | 7941 | return stop_one_cpu(curr_cpu, migration_cpu_stop, &arg); |
| 7947 | 7942 | } |
| @@ -1210,10 +1210,6 @@ void proc_sched_show_task(struct task_struct *p, struct pid_namespace *ns, · 1209 unmodified lines | ||
| 1210 | 1210 | P_SCHEDSTAT(nr_failed_migrations_running); |
| 1211 | 1211 | P_SCHEDSTAT(nr_failed_migrations_hot); |
| 1212 | 1212 | P_SCHEDSTAT(nr_forced_migrations); |
| 1213 | −#ifdef CONFIG_NUMA_BALANCING | |
| 1214 | − P_SCHEDSTAT(numa_task_migrated); | |
| 1215 | − P_SCHEDSTAT(numa_task_swapped); | |
| 1216 | −#endif | |
| 1217 | 1213 | P_SCHEDSTAT(nr_wakeups); |
| 1218 | 1214 | P_SCHEDSTAT(nr_wakeups_sync); |
| 1219 | 1215 | P_SCHEDSTAT(nr_wakeups_migrate); |
| @@ -135,6 +135,9 @@ size_t alloc_tag_top_users(struct codetag_bytes *tags, size_t count, bool can_sl · 134 unmodified lines | ||
| 135 | 135 | struct codetag_bytes n; |
| 136 | 136 | unsigned int i, nr = 0; |
| 137 | 137 | |
| 138 | + if (IS_ERR_OR_NULL(alloc_tag_cttype)) | |
| 139 | + return 0; | |
| 140 | + | |
| 138 | 141 | if (can_sleep) |
| 139 | 142 | codetag_lock_module_list(alloc_tag_cttype, true); |
| 140 | 143 | else if (!codetag_trylock_module_list(alloc_tag_cttype)) |
| @@ -5319,6 +5319,7 @@ static void mt_destroy_walk(struct maple_enode *enode, struct maple_tree *mt, · 5318 unmodified lines | ||
| 5319 | 5319 | struct maple_enode *start; |
| 5320 | 5320 | |
| 5321 | 5321 | if (mte_is_leaf(enode)) { |
| 5322 | + mte_set_node_dead(enode); | |
| 5322 | 5323 | node->type = mte_node_type(enode); |
| 5323 | 5324 | goto free_leaf; |
| 5324 | 5325 | } |
| @@ -1449,6 +1449,7 @@ static unsigned long damon_get_intervals_score(struct damon_ctx *c) · 1448 unmodified lines | ||
| 1449 | 1449 | } |
| 1450 | 1450 | } |
| 1451 | 1451 | target_access_events = max_access_events * goal_bp / 10000; |
| 1452 | + target_access_events = target_access_events ? : 1; | |
| 1452 | 1453 | return access_events * 10000 / target_access_events; |
| 1453 | 1454 | } |
| 1454 | 1455 | |
| @@ -2355,9 +2356,8 @@ static void kdamond_usleep(unsigned long usecs) · 900 unmodified lines | ||
| 2355 | 2356 | * |
| 2356 | 2357 | * If there is a &struct damon_call_control request that registered via |
| 2357 | 2358 | * &damon_call() on @ctx, do or cancel the invocation of the function depending |
| 2358 | − * on @cancel. @cancel is set when the kdamond is deactivated by DAMOS | |
| 2359 | − * watermarks, or the kdamond is already out of the main loop and therefore | |
| 2360 | − * will be terminated. | |
| 2359 | + * on @cancel. @cancel is set when the kdamond is already out of the main loop | |
| 2360 | + * and therefore will be terminated. | |
| 2361 | 2361 | */ |
| 2362 | 2362 | static void kdamond_call(struct damon_ctx *ctx, bool cancel) |
| 2363 | 2363 | { |
| @@ -2405,7 +2405,7 @@ static int kdamond_wait_activation(struct damon_ctx *ctx) · 41 unmodified lines | ||
| 2405 | 2405 | if (ctx->callback.after_wmarks_check && |
| 2406 | 2406 | ctx->callback.after_wmarks_check(ctx)) |
| 2407 | 2407 | break; |
| 2408 | − kdamond_call(ctx, true); | |
| 2408 | + kdamond_call(ctx, false); | |
| 2409 | 2409 | damos_walk_cancel(ctx); |
| 2410 | 2410 | } |
| 2411 | 2411 | return -EBUSY; |
| @@ -2340,12 +2340,15 @@ struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, · 2339 unmodified lines | ||
| 2340 | 2340 | struct folio *folio; |
| 2341 | 2341 | |
| 2342 | 2342 | spin_lock_irq(&hugetlb_lock); |
| 2343 | + if (!h->resv_huge_pages) { | |
| 2344 | + spin_unlock_irq(&hugetlb_lock); | |
| 2345 | + return NULL; | |
| 2346 | + } | |
| 2347 | + | |
| 2343 | 2348 | folio = dequeue_hugetlb_folio_nodemask(h, gfp_mask, preferred_nid, |
| 2344 | 2349 | nmask); |
| 2345 | − if (folio) { | |
| 2346 | − VM_BUG_ON(!h->resv_huge_pages); | |
| 2350 | + if (folio) | |
| 2347 | 2351 | h->resv_huge_pages--; |
| 2348 | − } | |
| 2349 | 2352 | |
| 2350 | 2353 | spin_unlock_irq(&hugetlb_lock); |
| 2351 | 2354 | return folio; |
| @@ -370,36 +370,6 @@ static inline bool init_task_stack_addr(const void *addr) · 369 unmodified lines | ||
| 370 | 370 | sizeof(init_thread_union.stack)); |
| 371 | 371 | } |
| 372 | 372 | |
| 373 | −/* | |
| 374 | − * This function is invoked with report_lock (a raw_spinlock) held. A | |
| 375 | − * PREEMPT_RT kernel cannot call find_vm_area() as it will acquire a sleeping | |
| 376 | − * rt_spinlock. | |
| 377 | − * | |
| 378 | − * For !RT kernel, the PROVE_RAW_LOCK_NESTING config option will print a | |
| 379 | − * lockdep warning for this raw_spinlock -> spinlock dependency. This config | |
| 380 | − * option is enabled by default to ensure better test coverage to expose this | |
| 381 | − * kind of RT kernel problem. This lockdep splat, however, can be suppressed | |
| 382 | − * by using DEFINE_WAIT_OVERRIDE_MAP() if it serves a useful purpose and the | |
| 383 | − * invalid PREEMPT_RT case has been taken care of. | |
| 384 | − */ | |
| 385 | −static inline struct vm_struct *kasan_find_vm_area(void *addr) | |
| 386 | −{ | |
| 387 | − static DEFINE_WAIT_OVERRIDE_MAP(vmalloc_map, LD_WAIT_SLEEP); | |
| 388 | − struct vm_struct *va; | |
| 389 | − | |
| 390 | − if (IS_ENABLED(CONFIG_PREEMPT_RT)) | |
| 391 | − return NULL; | |
| 392 | − | |
| 393 | − /* | |
| 394 | − * Suppress lockdep warning and fetch vmalloc area of the | |
| 395 | − * offending address. | |
| 396 | − */ | |
| 397 | − lock_map_acquire_try(&vmalloc_map); | |
| 398 | − va = find_vm_area(addr); | |
| 399 | − lock_map_release(&vmalloc_map); | |
| 400 | − return va; | |
| 401 | −} | |
| 402 | − | |
| 403 | 373 | static void print_address_description(void *addr, u8 tag, |
| 404 | 374 | struct kasan_report_info *info) |
| 405 | 375 | { |
| @@ -429,19 +399,8 @@ static void print_address_description(void *addr, u8 tag, · 23 unmodified lines | ||
| 429 | 399 | } |
| 430 | 400 | |
| 431 | 401 | if (is_vmalloc_addr(addr)) { |
| 432 | − struct vm_struct *va = kasan_find_vm_area(addr); | |
| 433 | − | |
| 434 | − if (va) { | |
| 435 | − pr_err("The buggy address belongs to the virtual mapping at\n" | |
| 436 | − " [%px, %px) created by:\n" | |
| 437 | − " %pS\n", | |
| 438 | − va->addr, va->addr + va->size, va->caller); | |
| 439 | − pr_err("\n"); | |
| 440 | − | |
| 441 | − page = vmalloc_to_page(addr); | |
| 442 | − } else { | |
| 443 | − pr_err("The buggy address %px belongs to a vmalloc virtual mapping\n", addr); | |
| 444 | − } | |
| 402 | + pr_err("The buggy address %px belongs to a vmalloc virtual mapping\n", addr); | |
| 403 | + page = vmalloc_to_page(addr); | |
| 445 | 404 | } |
| 446 | 405 | |
| 447 | 406 | if (page) { |
| @@ -474,8 +474,6 @@ static const unsigned int memcg_vm_event_stat[] = { · 473 unmodified lines | ||
| 474 | 474 | NUMA_PAGE_MIGRATE, |
| 475 | 475 | NUMA_PTE_UPDATES, |
| 476 | 476 | NUMA_HINT_FAULTS, |
| 477 | − NUMA_TASK_MIGRATE, | |
| 478 | − NUMA_TASK_SWAP, | |
| 479 | 477 | #endif |
| 480 | 478 | }; |
| 481 | 479 | |
| @@ -2399,6 +2399,7 @@ set_status: · 2398 unmodified lines | ||
| 2399 | 2399 | |
| 2400 | 2400 | static int get_compat_pages_array(const void __user *chunk_pages[], |
| 2401 | 2401 | const void __user * __user *pages, |
| 2402 | + unsigned long chunk_offset, | |
| 2402 | 2403 | unsigned long chunk_nr) |
| 2403 | 2404 | { |
| 2404 | 2405 | compat_uptr_t __user *pages32 = (compat_uptr_t __user *)pages; |
| @@ -2406,7 +2407,7 @@ static int get_compat_pages_array(const void __user *chunk_pages[], · 1 unmodified line | ||
| 2406 | 2407 | int i; |
| 2407 | 2408 | |
| 2408 | 2409 | for (i = 0; i < chunk_nr; i++) { |
| 2409 | − if (get_user(p, pages32 + i)) | |
| 2410 | + if (get_user(p, pages32 + chunk_offset + i)) | |
| 2410 | 2411 | return -EFAULT; |
| 2411 | 2412 | chunk_pages[i] = compat_ptr(p); |
| 2412 | 2413 | } |
| @@ -2425,27 +2426,28 @@ static int do_pages_stat(struct mm_struct *mm, unsigned long nr_pages, · 12 unmodified lines | ||
| 2425 | 2426 | #define DO_PAGES_STAT_CHUNK_NR 16UL |
| 2426 | 2427 | const void __user *chunk_pages[DO_PAGES_STAT_CHUNK_NR]; |
| 2427 | 2428 | int chunk_status[DO_PAGES_STAT_CHUNK_NR]; |
| 2429 | + unsigned long chunk_offset = 0; | |
| 2428 | 2430 | |
| 2429 | 2431 | while (nr_pages) { |
| 2430 | 2432 | unsigned long chunk_nr = min(nr_pages, DO_PAGES_STAT_CHUNK_NR); |
| 2431 | 2433 | |
| 2432 | 2434 | if (in_compat_syscall()) { |
| 2433 | 2435 | if (get_compat_pages_array(chunk_pages, pages, |
| 2434 | − chunk_nr)) | |
| 2436 | + chunk_offset, chunk_nr)) | |
| 2435 | 2437 | break; |
| 2436 | 2438 | } else { |
| 2437 | − if (copy_from_user(chunk_pages, pages, | |
| 2439 | + if (copy_from_user(chunk_pages, pages + chunk_offset, | |
| 2438 | 2440 | chunk_nr * sizeof(*chunk_pages))) |
| 2439 | 2441 | break; |
| 2440 | 2442 | } |
| 2441 | 2443 | |
| 2442 | 2444 | do_pages_stat_array(mm, chunk_nr, chunk_pages, chunk_status); |
| 2443 | 2445 | |
| 2444 | − if (copy_to_user(status, chunk_status, chunk_nr * sizeof(*status))) | |
| 2446 | + if (copy_to_user(status + chunk_offset, chunk_status, | |
| 2447 | + chunk_nr * sizeof(*status))) | |
| 2445 | 2448 | break; |
| 2446 | 2449 | |
| 2447 | − pages += chunk_nr; | |
| 2448 | − status += chunk_nr; | |
| 2450 | + chunk_offset += chunk_nr; | |
| 2449 | 2451 | nr_pages -= chunk_nr; |
| 2450 | 2452 | } |
| 2451 | 2453 | return nr_pages ? -EFAULT : 0; |
| @@ -1845,23 +1845,32 @@ void folio_remove_rmap_pud(struct folio *folio, struct page *page, · 1844 unmodified lines | ||
| 1845 | 1845 | #endif |
| 1846 | 1846 | } |
| 1847 | 1847 | |
| 1848 | −/* We support batch unmapping of PTEs for lazyfree large folios */ | |
| 1849 | −static inline bool can_batch_unmap_folio_ptes(unsigned long addr, | |
| 1850 | − struct folio *folio, pte_t *ptep) | |
| 1848 | +static inline unsigned int folio_unmap_pte_batch(struct folio *folio, | |
| 1849 | + struct page_vma_mapped_walk *pvmw, | |
| 1850 | + enum ttu_flags flags, pte_t pte) | |
| 1851 | 1851 | { |
| 1852 | 1852 | const fpb_t fpb_flags = FPB_IGNORE_DIRTY | FPB_IGNORE_SOFT_DIRTY; |
| 1853 | − int max_nr = folio_nr_pages(folio); | |
| 1854 | − pte_t pte = ptep_get(ptep); | |
| 1853 | + unsigned long end_addr, addr = pvmw->address; | |
| 1854 | + struct vm_area_struct *vma = pvmw->vma; | |
| 1855 | + unsigned int max_nr; | |
| 1856 | + | |
| 1857 | + if (flags & TTU_HWPOISON) | |
| 1858 | + return 1; | |
| 1859 | + if (!folio_test_large(folio)) | |
| 1860 | + return 1; | |
| 1855 | 1861 | |
| 1862 | + /* We may only batch within a single VMA and a single page table. */ | |
| 1863 | + end_addr = pmd_addr_end(addr, vma->vm_end); | |
| 1864 | + max_nr = (end_addr - addr) >> PAGE_SHIFT; | |
| 1865 | + | |
| 1866 | + /* We only support lazyfree batching for now ... */ | |
| 1856 | 1867 | if (!folio_test_anon(folio) || folio_test_swapbacked(folio)) |
| 1857 | − return false; | |
| 1868 | + return 1; | |
| 1858 | 1869 | if (pte_unused(pte)) |
| 1859 | − return false; | |
| 1860 | − if (pte_pfn(pte) != folio_pfn(folio)) | |
| 1861 | − return false; | |
| 1870 | + return 1; | |
| 1862 | 1871 | |
| 1863 | − return folio_pte_batch(folio, addr, ptep, pte, max_nr, fpb_flags, NULL, | |
| 1864 | − NULL, NULL) == max_nr; | |
| 1872 | + return folio_pte_batch(folio, addr, pvmw->pte, pte, max_nr, fpb_flags, | |
| 1873 | + NULL, NULL, NULL); | |
| 1865 | 1874 | } |
| 1866 | 1875 | |
| 1867 | 1876 | /* |
| @@ -2024,9 +2033,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, · 156 unmodified lines | ||
| 2024 | 2033 | if (pte_dirty(pteval)) |
| 2025 | 2034 | folio_mark_dirty(folio); |
| 2026 | 2035 | } else if (likely(pte_present(pteval))) { |
| 2027 | − if (folio_test_large(folio) && !(flags & TTU_HWPOISON) && | |
| 2028 | − can_batch_unmap_folio_ptes(address, folio, pvmw.pte)) | |
| 2029 | − nr_pages = folio_nr_pages(folio); | |
| 2036 | + nr_pages = folio_unmap_pte_batch(folio, &pvmw, flags, pteval); | |
| 2030 | 2037 | end_addr = address + nr_pages * PAGE_SIZE; |
| 2031 | 2038 | flush_cache_range(vma, address, end_addr); |
| 2032 | 2039 | |
| @@ -2206,13 +2213,16 @@ discard: · 173 unmodified lines | ||
| 2206 | 2213 | hugetlb_remove_rmap(folio); |
| 2207 | 2214 | } else { |
| 2208 | 2215 | folio_remove_rmap_ptes(folio, subpage, nr_pages, vma); |
| 2209 | − folio_ref_sub(folio, nr_pages - 1); | |
| 2210 | 2216 | } |
| 2211 | 2217 | if (vma->vm_flags & VM_LOCKED) |
| 2212 | 2218 | mlock_drain_local(); |
| 2213 | − folio_put(folio); | |
| 2214 | − /* We have already batched the entire folio */ | |
| 2215 | − if (nr_pages > 1) | |
| 2219 | + folio_put_refs(folio, nr_pages); | |
| 2220 | + | |
| 2221 | + /* | |
| 2222 | + * If we are sure that we batched the entire folio and cleared | |
| 2223 | + * all PTEs, we can just optimize and stop right here. | |
| 2224 | + */ | |
| 2225 | + if (nr_pages == folio_nr_pages(folio)) | |
| 2216 | 2226 | goto walk_done; |
| 2217 | 2227 | continue; |
| 2218 | 2228 | walk_abort: |
| @@ -514,6 +514,7 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, · 513 unmodified lines | ||
| 514 | 514 | unsigned long end, pgprot_t prot, struct page **pages, int *nr, |
| 515 | 515 | pgtbl_mod_mask *mask) |
| 516 | 516 | { |
| 517 | + int err = 0; | |
| 517 | 518 | pte_t *pte; |
| 518 | 519 | |
| 519 | 520 | /* |
| @@ -530,12 +531,18 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, · 10 unmodified lines | ||
| 530 | 531 | do { |
| 531 | 532 | struct page *page = pages[*nr]; |
| 532 | 533 | |
| 533 | − if (WARN_ON(!pte_none(ptep_get(pte)))) | |
| 534 | − return -EBUSY; | |
| 535 | − if (WARN_ON(!page)) | |
| 536 | − return -ENOMEM; | |
| 537 | − if (WARN_ON(!pfn_valid(page_to_pfn(page)))) | |
| 538 | − return -EINVAL; | |
| 534 | + if (WARN_ON(!pte_none(ptep_get(pte)))) { | |
| 535 | + err = -EBUSY; | |
| 536 | + break; | |
| 537 | + } | |
| 538 | + if (WARN_ON(!page)) { | |
| 539 | + err = -ENOMEM; | |
| 540 | + break; | |
| 541 | + } | |
| 542 | + if (WARN_ON(!pfn_valid(page_to_pfn(page)))) { | |
| 543 | + err = -EINVAL; | |
| 544 | + break; | |
| 545 | + } | |
| 539 | 546 | |
| 540 | 547 | set_pte_at(&init_mm, addr, pte, mk_pte(page, prot)); |
| 541 | 548 | (*nr)++; |
| @@ -543,7 +550,8 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, · 1 unmodified line | ||
| 543 | 550 | |
| 544 | 551 | arch_leave_lazy_mmu_mode(); |
| 545 | 552 | *mask |= PGTBL_PTE_MODIFIED; |
| 546 | − return 0; | |
| 553 | + | |
| 554 | + return err; | |
| 547 | 555 | } |
| 548 | 556 | |
| 549 | 557 | static int vmap_pages_pmd_range(pud_t *pud, unsigned long addr, |
| @@ -1346,8 +1346,6 @@ const char * const vmstat_text[] = { · 1345 unmodified lines | ||
| 1346 | 1346 | "numa_hint_faults", |
| 1347 | 1347 | "numa_hint_faults_local", |
| 1348 | 1348 | "numa_pages_migrated", |
| 1349 | − "numa_task_migrated", | |
| 1350 | − "numa_task_swapped", | |
| 1351 | 1349 | #endif |
| 1352 | 1350 | #ifdef CONFIG_MIGRATION |
| 1353 | 1351 | "pgmigrate_success", |
| @@ -576,6 +576,7 @@ static int atrtr_create(struct rtentry *r, struct net_device *devhint) · 575 unmodified lines | ||
| 576 | 576 | |
| 577 | 577 | /* Fill in the routing entry */ |
| 578 | 578 | rt->target = ta->sat_addr; |
| 579 | + dev_put(rt->dev); /* Release old device */ | |
| 579 | 580 | dev_hold(devhint); |
| 580 | 581 | rt->dev = devhint; |
| 581 | 582 | rt->flags = r->rt_flags; |
| @@ -45,7 +45,8 @@ · 44 unmodified lines | ||
| 45 | 45 | #include <net/atmclip.h> |
| 46 | 46 | |
| 47 | 47 | static struct net_device *clip_devs; |
| 48 | −static struct atm_vcc *atmarpd; | |
| 48 | +static struct atm_vcc __rcu *atmarpd; | |
| 49 | +static DEFINE_MUTEX(atmarpd_lock); | |
| 49 | 50 | static struct timer_list idle_timer; |
| 50 | 51 | static const struct neigh_ops clip_neigh_ops; |
| 51 | 52 | |
| @@ -53,24 +54,35 @@ static int to_atmarpd(enum atmarp_ctrl_type type, int itf, __be32 ip) · 1 unmodified line | ||
| 53 | 54 | { |
| 54 | 55 | struct sock *sk; |
| 55 | 56 | struct atmarp_ctrl *ctrl; |
| 57 | + struct atm_vcc *vcc; | |
| 56 | 58 | struct sk_buff *skb; |
| 59 | + int err = 0; | |
| 57 | 60 | |
| 58 | 61 | pr_debug("(%d)\n", type); |
| 59 | − if (!atmarpd) | |
| 60 | − return -EUNATCH; | |
| 62 | + | |
| 63 | + rcu_read_lock(); | |
| 64 | + vcc = rcu_dereference(atmarpd); | |
| 65 | + if (!vcc) { | |
| 66 | + err = -EUNATCH; | |
| 67 | + goto unlock; | |
| 68 | + } | |
| 61 | 69 | skb = alloc_skb(sizeof(struct atmarp_ctrl), GFP_ATOMIC); |
| 62 | − if (!skb) | |
| 63 | − return -ENOMEM; | |
| 70 | + if (!skb) { | |
| 71 | + err = -ENOMEM; | |
| 72 | + goto unlock; | |
| 73 | + } | |
| 64 | 74 | ctrl = skb_put(skb, sizeof(struct atmarp_ctrl)); |
| 65 | 75 | ctrl->type = type; |
| 66 | 76 | ctrl->itf_num = itf; |
| 67 | 77 | ctrl->ip = ip; |
| 68 | − atm_force_charge(atmarpd, skb->truesize); | |
| 78 | + atm_force_charge(vcc, skb->truesize); | |
| 69 | 79 | |
| 70 | − sk = sk_atm(atmarpd); | |
| 80 | + sk = sk_atm(vcc); | |
| 71 | 81 | skb_queue_tail(&sk->sk_receive_queue, skb); |
| 72 | 82 | sk->sk_data_ready(sk); |
| 73 | − return 0; | |
| 83 | +unlock: | |
| 84 | + rcu_read_unlock(); | |
| 85 | + return err; | |
| 74 | 86 | } |
| 75 | 87 | |
| 76 | 88 | static void link_vcc(struct clip_vcc *clip_vcc, struct atmarp_entry *entry) |
| @@ -417,6 +429,8 @@ static int clip_mkip(struct atm_vcc *vcc, int timeout) · 340 unmodified lines | ||
| 417 | 429 | |
| 418 | 430 | if (!vcc->push) |
| 419 | 431 | return -EBADFD; |
| 432 | + if (vcc->user_back) | |
| 433 | + return -EINVAL; | |
| 420 | 434 | clip_vcc = kmalloc(sizeof(struct clip_vcc), GFP_KERNEL); |
| 421 | 435 | if (!clip_vcc) |
| 422 | 436 | return -ENOMEM; |
| @@ -607,17 +621,27 @@ static void atmarpd_close(struct atm_vcc *vcc) · 184 unmodified lines | ||
| 607 | 621 | { |
| 608 | 622 | pr_debug("\n"); |
| 609 | 623 | |
| 610 | − rtnl_lock(); | |
| 611 | − atmarpd = NULL; | |
| 624 | + mutex_lock(&atmarpd_lock); | |
| 625 | + RCU_INIT_POINTER(atmarpd, NULL); | |
| 626 | + mutex_unlock(&atmarpd_lock); | |
| 627 | + | |
| 628 | + synchronize_rcu(); | |
| 612 | 629 | skb_queue_purge(&sk_atm(vcc)->sk_receive_queue); |
| 613 | − rtnl_unlock(); | |
| 614 | 630 | |
| 615 | 631 | pr_debug("(done)\n"); |
| 616 | 632 | module_put(THIS_MODULE); |
| 617 | 633 | } |
| 618 | 634 | |
| 635 | +static int atmarpd_send(struct atm_vcc *vcc, struct sk_buff *skb) | |
| 636 | +{ | |
| 637 | + atm_return_tx(vcc, skb); | |
| 638 | + dev_kfree_skb_any(skb); | |
| 639 | + return 0; | |
| 640 | +} | |
| 641 | + | |
| 619 | 642 | static const struct atmdev_ops atmarpd_dev_ops = { |
| 620 | − .close = atmarpd_close | |
| 643 | + .close = atmarpd_close, | |
| 644 | + .send = atmarpd_send | |
| 621 | 645 | }; |
| 622 | 646 | |
| 623 | 647 | |
| @@ -631,15 +655,18 @@ static struct atm_dev atmarpd_dev = { · 7 unmodified lines | ||
| 631 | 655 | |
| 632 | 656 | static int atm_init_atmarp(struct atm_vcc *vcc) |
| 633 | 657 | { |
| 634 | − rtnl_lock(); | |
| 658 | + if (vcc->push == clip_push) | |
| 659 | + return -EINVAL; | |
| 660 | + | |
| 661 | + mutex_lock(&atmarpd_lock); | |
| 635 | 662 | if (atmarpd) { |
| 636 | − rtnl_unlock(); | |
| 663 | + mutex_unlock(&atmarpd_lock); | |
| 637 | 664 | return -EADDRINUSE; |
| 638 | 665 | } |
| 639 | 666 | |
| 640 | 667 | mod_timer(&idle_timer, jiffies + CLIP_CHECK_INTERVAL * HZ); |
| 641 | 668 | |
| 642 | − atmarpd = vcc; | |
| 669 | + rcu_assign_pointer(atmarpd, vcc); | |
| 643 | 670 | set_bit(ATM_VF_META, &vcc->flags); |
| 644 | 671 | set_bit(ATM_VF_READY, &vcc->flags); |
| 645 | 672 | /* allow replies and avoid getting closed if signaling dies */ |
| @@ -648,13 +675,14 @@ static int atm_init_atmarp(struct atm_vcc *vcc) · 2 unmodified lines | ||
| 648 | 675 | vcc->push = NULL; |
| 649 | 676 | vcc->pop = NULL; /* crash */ |
| 650 | 677 | vcc->push_oam = NULL; /* crash */ |
| 651 | − rtnl_unlock(); | |
| 678 | + mutex_unlock(&atmarpd_lock); | |
| 652 | 679 | return 0; |
| 653 | 680 | } |
| 654 | 681 | |
| 655 | 682 | static int clip_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg) |
| 656 | 683 | { |
| 657 | 684 | struct atm_vcc *vcc = ATM_SD(sock); |
| 685 | + struct sock *sk = sock->sk; | |
| 658 | 686 | int err = 0; |
| 659 | 687 | |
| 660 | 688 | switch (cmd) { |
| @@ -675,14 +703,18 @@ static int clip_ioctl(struct socket *sock, unsigned int cmd, unsigned long arg) · 14 unmodified lines | ||
| 675 | 703 | err = clip_create(arg); |
| 676 | 704 | break; |
| 677 | 705 | case ATMARPD_CTRL: |
| 706 | + lock_sock(sk); | |
| 678 | 707 | err = atm_init_atmarp(vcc); |
| 679 | 708 | if (!err) { |
| 680 | 709 | sock->state = SS_CONNECTED; |
| 681 | 710 | __module_get(THIS_MODULE); |
| 682 | 711 | } |
| 712 | + release_sock(sk); | |
| 683 | 713 | break; |
| 684 | 714 | case ATMARP_MKIP: |
| 715 | + lock_sock(sk); | |
| 685 | 716 | err = clip_mkip(vcc, arg); |
| 717 | + release_sock(sk); | |
| 686 | 718 | break; |
| 687 | 719 | case ATMARP_SETENTRY: |
| 688 | 720 | err = clip_setentry(vcc, (__force __be32)arg); |
| @@ -6966,7 +6966,10 @@ static void hci_le_big_sync_established_evt(struct hci_dev *hdev, void *data, · 6965 unmodified lines | ||
| 6966 | 6966 | bis->iso_qos.bcast.in.sdu = le16_to_cpu(ev->max_pdu); |
| 6967 | 6967 | |
| 6968 | 6968 | if (!ev->status) { |
| 6969 | + bis->state = BT_CONNECTED; | |
| 6969 | 6970 | set_bit(HCI_CONN_BIG_SYNC, &bis->flags); |
| 6971 | + hci_debugfs_create_conn(bis); | |
| 6972 | + hci_conn_add_sysfs(bis); | |
| 6970 | 6973 | hci_iso_setup_path(bis); |
| 6971 | 6974 | } |
| 6972 | 6975 | } |
| @@ -1345,7 +1345,7 @@ int hci_setup_ext_adv_instance_sync(struct hci_dev *hdev, u8 instance) · 1344 unmodified lines | ||
| 1345 | 1345 | * Command Disallowed error, so we must first disable the |
| 1346 | 1346 | * instance if it is active. |
| 1347 | 1347 | */ |
| 1348 | − if (adv && !adv->pending) { | |
| 1348 | + if (adv) { | |
| 1349 | 1349 | err = hci_disable_ext_adv_instance_sync(hdev, instance); |
| 1350 | 1350 | if (err) |
| 1351 | 1351 | return err; |
| @@ -5493,7 +5493,7 @@ static int hci_disconnect_sync(struct hci_dev *hdev, struct hci_conn *conn, · 4141 unmodified lines | ||
| 5493 | 5493 | { |
| 5494 | 5494 | struct hci_cp_disconnect cp; |
| 5495 | 5495 | |
| 5496 | − if (test_bit(HCI_CONN_BIG_CREATED, &conn->flags)) { | |
| 5496 | + if (conn->type == BIS_LINK) { | |
| 5497 | 5497 | /* This is a BIS connection, hci_conn_del will |
| 5498 | 5498 | * do the necessary cleanup. |
| 5499 | 5499 | */ |
| @@ -1176,7 +1176,7 @@ restart: · 1175 unmodified lines | ||
| 1176 | 1176 | goto do_error; |
| 1177 | 1177 | |
| 1178 | 1178 | while (msg_data_left(msg)) { |
| 1179 | − ssize_t copy = 0; | |
| 1179 | + int copy = 0; | |
| 1180 | 1180 | |
| 1181 | 1181 | skb = tcp_write_queue_tail(sk); |
| 1182 | 1182 | if (skb) |
| @@ -5181,7 +5181,9 @@ end: · 5180 unmodified lines | ||
| 5181 | 5181 | skb_condense(skb); |
| 5182 | 5182 | skb_set_owner_r(skb, sk); |
| 5183 | 5183 | } |
| 5184 | − tcp_rcvbuf_grow(sk); | |
| 5184 | + /* do not grow rcvbuf for not-yet-accepted or orphaned sockets. */ | |
| 5185 | + if (sk->sk_socket) | |
| 5186 | + tcp_rcvbuf_grow(sk); | |
| 5185 | 5187 | } |
| 5186 | 5188 | |
| 5187 | 5189 | static int __must_check tcp_queue_rcv(struct sock *sk, struct sk_buff *skb, |
| @@ -3525,11 +3525,9 @@ static void addrconf_gre_config(struct net_device *dev) · 3524 unmodified lines | ||
| 3525 | 3525 | |
| 3526 | 3526 | ASSERT_RTNL(); |
| 3527 | 3527 | |
| 3528 | − idev = ipv6_find_idev(dev); | |
| 3529 | − if (IS_ERR(idev)) { | |
| 3530 | − pr_debug("%s: add_dev failed\n", __func__); | |
| 3528 | + idev = addrconf_add_dev(dev); | |
| 3529 | + if (IS_ERR(idev)) | |
| 3531 | 3530 | return; |
| 3532 | − } | |
| 3533 | 3531 | |
| 3534 | 3532 | /* Generate the IPv6 link-local address using addrconf_addr_gen(), |
| 3535 | 3533 | * unless we have an IPv4 GRE device not bound to an IP address and |
| @@ -3543,9 +3541,6 @@ static void addrconf_gre_config(struct net_device *dev) · 7 unmodified lines | ||
| 3543 | 3541 | } |
| 3544 | 3542 | |
| 3545 | 3543 | add_v4_addrs(idev); |
| 3546 | − | |
| 3547 | − if (dev->flags & IFF_POINTOPOINT) | |
| 3548 | − addrconf_add_mroute(dev); | |
| 3549 | 3544 | } |
| 3550 | 3545 | #endif |
| 3551 | 3546 | |
| @@ -1959,6 +1959,20 @@ static int sta_link_apply_parameters(struct ieee80211_local *local, · 1958 unmodified lines | ||
| 1959 | 1959 | ieee80211_sta_init_nss(link_sta); |
| 1960 | 1960 | |
| 1961 | 1961 | if (params->opmode_notif_used) { |
| 1962 | + enum nl80211_chan_width width = link->conf->chanreq.oper.width; | |
| 1963 | + | |
| 1964 | + switch (width) { | |
| 1965 | + case NL80211_CHAN_WIDTH_20: | |
| 1966 | + case NL80211_CHAN_WIDTH_40: | |
| 1967 | + case NL80211_CHAN_WIDTH_80: | |
| 1968 | + case NL80211_CHAN_WIDTH_160: | |
| 1969 | + case NL80211_CHAN_WIDTH_80P80: | |
| 1970 | + case NL80211_CHAN_WIDTH_320: /* not VHT, allowed for HE/EHT */ | |
| 1971 | + break; | |
| 1972 | + default: | |
| 1973 | + return -EINVAL; | |
| 1974 | + } | |
| 1975 | + | |
| 1962 | 1976 | /* returned value is only needed for rc update, but the |
| 1963 | 1977 | * rc isn't initialized here yet, so ignore it |
| 1964 | 1978 | */ |
| @@ -1150,6 +1150,8 @@ static void ieee80211_sdata_init(struct ieee80211_local *local, · 1149 unmodified lines | ||
| 1150 | 1150 | { |
| 1151 | 1151 | sdata->local = local; |
| 1152 | 1152 | |
| 1153 | + INIT_LIST_HEAD(&sdata->key_list); | |
| 1154 | + | |
| 1153 | 1155 | /* |
| 1154 | 1156 | * Initialize the default link, so we can use link_id 0 for non-MLD, |
| 1155 | 1157 | * and that continues to work for non-MLD-aware drivers that use just |
| @@ -2210,8 +2212,6 @@ int ieee80211_if_add(struct ieee80211_local *local, const char *name, · 1054 unmodified lines | ||
| 2210 | 2212 | |
| 2211 | 2213 | ieee80211_init_frag_cache(&sdata->frags); |
| 2212 | 2214 | |
| 2213 | − INIT_LIST_HEAD(&sdata->key_list); | |
| 2214 | − | |
| 2215 | 2215 | wiphy_delayed_work_init(&sdata->dec_tailroom_needed_wk, |
| 2216 | 2216 | ieee80211_delayed_tailroom_dec); |
| 2217 | 2217 | |
| @@ -3934,6 +3934,9 @@ static void ieee80211_set_disassoc(struct ieee80211_sub_if_data *sdata, · 3933 unmodified lines | ||
| 3934 | 3934 | |
| 3935 | 3935 | lockdep_assert_wiphy(local->hw.wiphy); |
| 3936 | 3936 | |
| 3937 | + if (frame_buf) | |
| 3938 | + memset(frame_buf, 0, IEEE80211_DEAUTH_FRAME_LEN); | |
| 3939 | + | |
| 3937 | 3940 | if (WARN_ON(!ap_sta)) |
| 3938 | 3941 | return; |
| 3939 | 3942 | |
| @@ -7195,6 +7198,7 @@ static void ieee80211_rx_mgmt_beacon(struct ieee80211_link_data *link, · 3255 unmodified lines | ||
| 7195 | 7198 | struct ieee80211_bss_conf *bss_conf = link->conf; |
| 7196 | 7199 | struct ieee80211_vif_cfg *vif_cfg = &sdata->vif.cfg; |
| 7197 | 7200 | struct ieee80211_mgmt *mgmt = (void *) hdr; |
| 7201 | + struct ieee80211_ext *ext = NULL; | |
| 7198 | 7202 | size_t baselen; |
| 7199 | 7203 | struct ieee802_11_elems *elems; |
| 7200 | 7204 | struct ieee80211_local *local = sdata->local; |
| @@ -7220,7 +7224,7 @@ static void ieee80211_rx_mgmt_beacon(struct ieee80211_link_data *link, · 19 unmodified lines | ||
| 7220 | 7224 | /* Process beacon from the current BSS */ |
| 7221 | 7225 | bssid = ieee80211_get_bssid(hdr, len, sdata->vif.type); |
| 7222 | 7226 | if (ieee80211_is_s1g_beacon(mgmt->frame_control)) { |
| 7223 | − struct ieee80211_ext *ext = (void *) mgmt; | |
| 7227 | + ext = (void *)mgmt; | |
| 7224 | 7228 | variable = ext->u.s1g_beacon.variable + |
| 7225 | 7229 | ieee80211_s1g_optional_len(ext->frame_control); |
| 7226 | 7230 | } |
| @@ -7407,7 +7411,9 @@ static void ieee80211_rx_mgmt_beacon(struct ieee80211_link_data *link, · 180 unmodified lines | ||
| 7407 | 7411 | } |
| 7408 | 7412 | |
| 7409 | 7413 | if ((ncrc == link->u.mgd.beacon_crc && link->u.mgd.beacon_crc_valid) || |
| 7410 | − ieee80211_is_s1g_short_beacon(mgmt->frame_control)) | |
| 7414 | + (ext && ieee80211_is_s1g_short_beacon(ext->frame_control, | |
| 7415 | + parse_params.start, | |
| 7416 | + parse_params.len))) | |
| 7411 | 7417 | goto free; |
| 7412 | 7418 | link->u.mgd.beacon_crc = ncrc; |
| 7413 | 7419 | link->u.mgd.beacon_crc_valid = true; |
| @@ -10699,8 +10705,8 @@ static void ieee80211_ml_epcs(struct ieee80211_sub_if_data *sdata, · 3285 unmodified lines | ||
| 10699 | 10705 | */ |
| 10700 | 10706 | for_each_mle_subelement(sub, (const u8 *)elems->ml_epcs, |
| 10701 | 10707 | elems->ml_epcs_len) { |
| 10708 | + struct ieee802_11_elems *link_elems __free(kfree) = NULL; | |
| 10702 | 10709 | struct ieee80211_link_data *link; |
| 10703 | − struct ieee802_11_elems *link_elems __free(kfree); | |
| 10704 | 10710 | u8 *pos = (void *)sub->data; |
| 10705 | 10711 | u16 control; |
| 10706 | 10712 | ssize_t len; |
| @@ -758,7 +758,6 @@ static size_t ieee802_11_find_bssid_profile(const u8 *start, size_t len, · 757 unmodified lines | ||
| 758 | 758 | { |
| 759 | 759 | const struct element *elem, *sub; |
| 760 | 760 | size_t profile_len = 0; |
| 761 | − bool found = false; | |
| 762 | 761 | |
| 763 | 762 | if (!bss || !bss->transmitted_bss) |
| 764 | 763 | return profile_len; |
| @@ -809,15 +808,14 @@ static size_t ieee802_11_find_bssid_profile(const u8 *start, size_t len, · 44 unmodified lines | ||
| 809 | 808 | index[2], |
| 810 | 809 | new_bssid); |
| 811 | 810 | if (ether_addr_equal(new_bssid, bss->bssid)) { |
| 812 | − found = true; | |
| 813 | 811 | elems->bssid_index_len = index[1]; |
| 814 | 812 | elems->bssid_index = (void *)&index[2]; |
| 815 | − break; | |
| 813 | + return profile_len; | |
| 816 | 814 | } |
| 817 | 815 | } |
| 818 | 816 | } |
| 819 | 817 | |
| 820 | − return found ? profile_len : 0; | |
| 818 | + return 0; | |
| 821 | 819 | } |
| 822 | 820 | |
| 823 | 821 | static void |
| @@ -2144,11 +2144,6 @@ int ieee80211_reconfig(struct ieee80211_local *local) · 2143 unmodified lines | ||
| 2144 | 2144 | cfg80211_sched_scan_stopped_locked(local->hw.wiphy, 0); |
| 2145 | 2145 | |
| 2146 | 2146 | wake_up: |
| 2147 | − | |
| 2148 | − if (local->virt_monitors > 0 && | |
| 2149 | − local->virt_monitors == local->open_count) | |
| 2150 | − ieee80211_add_virtual_monitor(local); | |
| 2151 | − | |
| 2152 | 2147 | /* |
| 2153 | 2148 | * Clear the WLAN_STA_BLOCK_BA flag so new aggregation |
| 2154 | 2149 | * sessions can be established after a resume. |
| @@ -2202,6 +2197,10 @@ int ieee80211_reconfig(struct ieee80211_local *local) · 47 unmodified lines | ||
| 2202 | 2197 | } |
| 2203 | 2198 | } |
| 2204 | 2199 | |
| 2200 | + if (local->virt_monitors > 0 && | |
| 2201 | + local->virt_monitors == local->open_count) | |
| 2202 | + ieee80211_add_virtual_monitor(local); | |
| 2203 | + | |
| 2205 | 2204 | if (!suspended) |
| 2206 | 2205 | return 0; |
| 2207 | 2206 | |
| @@ -387,7 +387,6 @@ static void netlink_skb_set_owner_r(struct sk_buff *skb, struct sock *sk) · 386 unmodified lines | ||
| 387 | 387 | WARN_ON(skb->sk != NULL); |
| 388 | 388 | skb->sk = sk; |
| 389 | 389 | skb->destructor = netlink_skb_destructor; |
| 390 | − atomic_add(skb->truesize, &sk->sk_rmem_alloc); | |
| 391 | 390 | sk_mem_charge(sk, skb->truesize); |
| 392 | 391 | } |
| 393 | 392 | |
| @@ -1212,41 +1211,48 @@ struct sk_buff *netlink_alloc_large_skb(unsigned int size, int broadcast) · 818 unmodified lines | ||
| 1212 | 1211 | int netlink_attachskb(struct sock *sk, struct sk_buff *skb, |
| 1213 | 1212 | long *timeo, struct sock *ssk) |
| 1214 | 1213 | { |
| 1214 | + DECLARE_WAITQUEUE(wait, current); | |
| 1215 | 1215 | struct netlink_sock *nlk; |
| 1216 | + unsigned int rmem; | |
| 1216 | 1217 | |
| 1217 | 1218 | nlk = nlk_sk(sk); |
| 1219 | + rmem = atomic_add_return(skb->truesize, &sk->sk_rmem_alloc); | |
| 1218 | 1220 | |
| 1219 | − if ((atomic_read(&sk->sk_rmem_alloc) > sk->sk_rcvbuf || | |
| 1220 | − test_bit(NETLINK_S_CONGESTED, &nlk->state))) { | |
| 1221 | − DECLARE_WAITQUEUE(wait, current); | |
| 1222 | − if (!*timeo) { | |
| 1223 | − if (!ssk || netlink_is_kernel(ssk)) | |
| 1224 | − netlink_overrun(sk); | |
| 1225 | − sock_put(sk); | |
| 1226 | − kfree_skb(skb); | |
| 1227 | − return -EAGAIN; | |
| 1228 | − } | |
| 1229 | − | |
| 1230 | − __set_current_state(TASK_INTERRUPTIBLE); | |
| 1231 | − add_wait_queue(&nlk->wait, &wait); | |
| 1221 | + if ((rmem == skb->truesize || rmem < READ_ONCE(sk->sk_rcvbuf)) && | |
| 1222 | + !test_bit(NETLINK_S_CONGESTED, &nlk->state)) { | |
| 1223 | + netlink_skb_set_owner_r(skb, sk); | |
| 1224 | + return 0; | |
| 1225 | + } | |
| 1232 | 1226 | |
| 1233 | − if ((atomic_read(&sk->sk_rmem_alloc) > sk->sk_rcvbuf || | |
| 1234 | − test_bit(NETLINK_S_CONGESTED, &nlk->state)) && | |
| 1235 | − !sock_flag(sk, SOCK_DEAD)) | |
| 1236 | − *timeo = schedule_timeout(*timeo); | |
| 1227 | + atomic_sub(skb->truesize, &sk->sk_rmem_alloc); | |
| 1237 | 1228 | |
| 1238 | − __set_current_state(TASK_RUNNING); | |
| 1239 | − remove_wait_queue(&nlk->wait, &wait); | |
| 1229 | + if (!*timeo) { | |
| 1230 | + if (!ssk || netlink_is_kernel(ssk)) | |
| 1231 | + netlink_overrun(sk); | |
| 1240 | 1232 | sock_put(sk); |
| 1233 | + kfree_skb(skb); | |
| 1234 | + return -EAGAIN; | |
| 1235 | + } | |
| 1241 | 1236 | |
| 1242 | − if (signal_pending(current)) { | |
| 1243 | − kfree_skb(skb); | |
| 1244 | − return sock_intr_errno(*timeo); | |
| 1245 | − } | |
| 1246 | − return 1; | |
| 1237 | + __set_current_state(TASK_INTERRUPTIBLE); | |
| 1238 | + add_wait_queue(&nlk->wait, &wait); | |
| 1239 | + rmem = atomic_read(&sk->sk_rmem_alloc); | |
| 1240 | + | |
| 1241 | + if (((rmem && rmem + skb->truesize > READ_ONCE(sk->sk_rcvbuf)) || | |
| 1242 | + test_bit(NETLINK_S_CONGESTED, &nlk->state)) && | |
| 1243 | + !sock_flag(sk, SOCK_DEAD)) | |
| 1244 | + *timeo = schedule_timeout(*timeo); | |
| 1245 | + | |
| 1246 | + __set_current_state(TASK_RUNNING); | |
| 1247 | + remove_wait_queue(&nlk->wait, &wait); | |
| 1248 | + sock_put(sk); | |
| 1249 | + | |
| 1250 | + if (signal_pending(current)) { | |
| 1251 | + kfree_skb(skb); | |
| 1252 | + return sock_intr_errno(*timeo); | |
| 1247 | 1253 | } |
| 1248 | − netlink_skb_set_owner_r(skb, sk); | |
| 1249 | − return 0; | |
| 1254 | + | |
| 1255 | + return 1; | |
| 1250 | 1256 | } |
| 1251 | 1257 | |
| 1252 | 1258 | static int __netlink_sendskb(struct sock *sk, struct sk_buff *skb) |
| @@ -1307,6 +1313,7 @@ static int netlink_unicast_kernel(struct sock *sk, struct sk_buff *skb, · 54 unmodified lines | ||
| 1307 | 1313 | ret = -ECONNREFUSED; |
| 1308 | 1314 | if (nlk->netlink_rcv != NULL) { |
| 1309 | 1315 | ret = skb->len; |
| 1316 | + atomic_add(skb->truesize, &sk->sk_rmem_alloc); | |
| 1310 | 1317 | netlink_skb_set_owner_r(skb, sk); |
| 1311 | 1318 | NETLINK_CB(skb).sk = ssk; |
| 1312 | 1319 | netlink_deliver_tap_kernel(sk, ssk, skb); |
| @@ -1383,13 +1390,19 @@ EXPORT_SYMBOL_GPL(netlink_strict_get_check); · 70 unmodified lines | ||
| 1383 | 1390 | static int netlink_broadcast_deliver(struct sock *sk, struct sk_buff *skb) |
| 1384 | 1391 | { |
| 1385 | 1392 | struct netlink_sock *nlk = nlk_sk(sk); |
| 1393 | + unsigned int rmem, rcvbuf; | |
| 1386 | 1394 | |
| 1387 | − if (atomic_read(&sk->sk_rmem_alloc) <= sk->sk_rcvbuf && | |
| 1395 | + rmem = atomic_add_return(skb->truesize, &sk->sk_rmem_alloc); | |
| 1396 | + rcvbuf = READ_ONCE(sk->sk_rcvbuf); | |
| 1397 | + | |
| 1398 | + if ((rmem == skb->truesize || rmem <= rcvbuf) && | |
| 1388 | 1399 | !test_bit(NETLINK_S_CONGESTED, &nlk->state)) { |
| 1389 | 1400 | netlink_skb_set_owner_r(skb, sk); |
| 1390 | 1401 | __netlink_sendskb(sk, skb); |
| 1391 | − return atomic_read(&sk->sk_rmem_alloc) > (sk->sk_rcvbuf >> 1); | |
| 1402 | + return rmem > (rcvbuf >> 1); | |
| 1392 | 1403 | } |
| 1404 | + | |
| 1405 | + atomic_sub(skb->truesize, &sk->sk_rmem_alloc); | |
| 1393 | 1406 | return -1; |
| 1394 | 1407 | } |
| 1395 | 1408 | |
| @@ -2245,6 +2258,7 @@ static int netlink_dump(struct sock *sk, bool lock_taken) · 849 unmodified lines | ||
| 2245 | 2258 | struct netlink_ext_ack extack = {}; |
| 2246 | 2259 | struct netlink_callback *cb; |
| 2247 | 2260 | struct sk_buff *skb = NULL; |
| 2261 | + unsigned int rmem, rcvbuf; | |
| 2248 | 2262 | size_t max_recvmsg_len; |
| 2249 | 2263 | struct module *module; |
| 2250 | 2264 | int err = -ENOBUFS; |
| @@ -2258,9 +2272,6 @@ static int netlink_dump(struct sock *sk, bool lock_taken) · 7 unmodified lines | ||
| 2258 | 2272 | goto errout_skb; |
| 2259 | 2273 | } |
| 2260 | 2274 | |
| 2261 | − if (atomic_read(&sk->sk_rmem_alloc) >= sk->sk_rcvbuf) | |
| 2262 | − goto errout_skb; | |
| 2263 | − | |
| 2264 | 2275 | /* NLMSG_GOODSIZE is small to avoid high order allocations being |
| 2265 | 2276 | * required, but it makes sense to _attempt_ a 32KiB allocation |
| 2266 | 2277 | * to reduce number of system calls on dump operations, if user |
| @@ -2283,6 +2294,13 @@ static int netlink_dump(struct sock *sk, bool lock_taken) · 16 unmodified lines | ||
| 2283 | 2294 | if (!skb) |
| 2284 | 2295 | goto errout_skb; |
| 2285 | 2296 | |
| 2297 | + rcvbuf = READ_ONCE(sk->sk_rcvbuf); | |
| 2298 | + rmem = atomic_add_return(skb->truesize, &sk->sk_rmem_alloc); | |
| 2299 | + if (rmem != skb->truesize && rmem >= rcvbuf) { | |
| 2300 | + atomic_sub(skb->truesize, &sk->sk_rmem_alloc); | |
| 2301 | + goto errout_skb; | |
| 2302 | + } | |
| 2303 | + | |
| 2286 | 2304 | /* Trim skb to allocated size. User is expected to provide buffer as |
| 2287 | 2305 | * large as max(min_dump_alloc, 32KiB (max_recvmsg_len capped at |
| 2288 | 2306 | * netlink_recvmsg())). dump will pack as many smaller messages as |
| @@ -361,12 +361,15 @@ struct rxrpc_local { · 360 unmodified lines | ||
| 361 | 361 | struct list_head new_client_calls; /* Newly created client calls need connection */ |
| 362 | 362 | spinlock_t client_call_lock; /* Lock for ->new_client_calls */ |
| 363 | 363 | struct sockaddr_rxrpc srx; /* local address */ |
| 364 | − /* Provide a kvec table sufficiently large to manage either a DATA | |
| 365 | − * packet with a maximum set of jumbo subpackets or a PING ACK padded | |
| 366 | − * out to 64K with zeropages for PMTUD. | |
| 367 | − */ | |
| 368 | − struct kvec kvec[1 + RXRPC_MAX_NR_JUMBO > 3 + 16 ? | |
| 369 | − 1 + RXRPC_MAX_NR_JUMBO : 3 + 16]; | |
| 364 | + union { | |
| 365 | + /* Provide a kvec table sufficiently large to manage either a | |
| 366 | + * DATA packet with a maximum set of jumbo subpackets or a PING | |
| 367 | + * ACK padded out to 64K with zeropages for PMTUD. | |
| 368 | + */ | |
| 369 | + struct kvec kvec[1 + RXRPC_MAX_NR_JUMBO > 3 + 16 ? | |
| 370 | + 1 + RXRPC_MAX_NR_JUMBO : 3 + 16]; | |
| 371 | + struct bio_vec bvec[3 + 16]; | |
| 372 | + }; | |
| 370 | 373 | }; |
| 371 | 374 | |
| 372 | 375 | /* |
| @@ -149,6 +149,7 @@ static int rxrpc_service_prealloc_one(struct rxrpc_sock *rx, · 148 unmodified lines | ||
| 149 | 149 | |
| 150 | 150 | id_in_use: |
| 151 | 151 | write_unlock(&rx->call_lock); |
| 152 | + rxrpc_prefail_call(call, RXRPC_CALL_LOCAL_ERROR, -EBADSLT); | |
| 152 | 153 | rxrpc_cleanup_call(call); |
| 153 | 154 | _leave(" = -EBADSLT"); |
| 154 | 155 | return -EBADSLT; |
| @@ -254,6 +255,9 @@ static struct rxrpc_call *rxrpc_alloc_incoming_call(struct rxrpc_sock *rx, · 99 unmodified lines | ||
| 254 | 255 | unsigned short call_tail, conn_tail, peer_tail; |
| 255 | 256 | unsigned short call_count, conn_count; |
| 256 | 257 | |
| 258 | + if (!b) | |
| 259 | + return NULL; | |
| 260 | + | |
| 257 | 261 | /* #calls >= #conns >= #peers must hold true. */ |
| 258 | 262 | call_head = smp_load_acquire(&b->call_backlog_head); |
| 259 | 263 | call_tail = b->call_backlog_tail; |
| @@ -924,7 +924,7 @@ void rxrpc_send_response(struct rxrpc_connection *conn, struct sk_buff *response · 923 unmodified lines | ||
| 924 | 924 | { |
| 925 | 925 | struct rxrpc_skb_priv *sp = rxrpc_skb(response); |
| 926 | 926 | struct scatterlist sg[16]; |
| 927 | − struct bio_vec bvec[16]; | |
| 927 | + struct bio_vec *bvec = conn->local->bvec; | |
| 928 | 928 | struct msghdr msg; |
| 929 | 929 | size_t len = sp->resp.len; |
| 930 | 930 | __be32 wserial; |
| @@ -938,6 +938,9 @@ void rxrpc_send_response(struct rxrpc_connection *conn, struct sk_buff *response · 7 unmodified lines | ||
| 938 | 938 | if (ret < 0) |
| 939 | 939 | goto fail; |
| 940 | 940 | nr_sg = ret; |
| 941 | + ret = -EIO; | |
| 942 | + if (WARN_ON_ONCE(nr_sg > ARRAY_SIZE(conn->local->bvec))) | |
| 943 | + goto fail; | |
| 941 | 944 | |
| 942 | 945 | for (int i = 0; i < nr_sg; i++) |
| 943 | 946 | bvec_set_page(&bvec[i], sg_page(&sg[i]), sg[i].length, sg[i].offset); |
| @@ -336,17 +336,22 @@ out: · 335 unmodified lines | ||
| 336 | 336 | return q; |
| 337 | 337 | } |
| 338 | 338 | |
| 339 | −static struct Qdisc *qdisc_leaf(struct Qdisc *p, u32 classid) | |
| 339 | +static struct Qdisc *qdisc_leaf(struct Qdisc *p, u32 classid, | |
| 340 | + struct netlink_ext_ack *extack) | |
| 340 | 341 | { |
| 341 | 342 | unsigned long cl; |
| 342 | 343 | const struct Qdisc_class_ops *cops = p->ops->cl_ops; |
| 343 | 344 | |
| 344 | − if (cops == NULL) | |
| 345 | − return NULL; | |
| 345 | + if (cops == NULL) { | |
| 346 | + NL_SET_ERR_MSG(extack, "Parent qdisc is not classful"); | |
| 347 | + return ERR_PTR(-EOPNOTSUPP); | |
| 348 | + } | |
| 346 | 349 | cl = cops->find(p, classid); |
| 347 | 350 | |
| 348 | − if (cl == 0) | |
| 349 | − return NULL; | |
| 351 | + if (cl == 0) { | |
| 352 | + NL_SET_ERR_MSG(extack, "Specified class not found"); | |
| 353 | + return ERR_PTR(-ENOENT); | |
| 354 | + } | |
| 350 | 355 | return cops->leaf(p, cl); |
| 351 | 356 | } |
| 352 | 357 | |
| @@ -596,16 +601,6 @@ out: · 243 unmodified lines | ||
| 596 | 601 | qdisc_skb_cb(skb)->pkt_len = pkt_len; |
| 597 | 602 | } |
| 598 | 603 | |
| 599 | −void qdisc_warn_nonwc(const char *txt, struct Qdisc *qdisc) | |
| 600 | −{ | |
| 601 | − if (!(qdisc->flags & TCQ_F_WARN_NONWC)) { | |
| 602 | − pr_warn("%s: %s qdisc %X: is non-work-conserving?\n", | |
| 603 | − txt, qdisc->ops->id, qdisc->handle >> 16); | |
| 604 | − qdisc->flags |= TCQ_F_WARN_NONWC; | |
| 605 | − } | |
| 606 | −} | |
| 607 | −EXPORT_SYMBOL(qdisc_warn_nonwc); | |
| 608 | − | |
| 609 | 604 | static enum hrtimer_restart qdisc_watchdog(struct hrtimer *timer) |
| 610 | 605 | { |
| 611 | 606 | struct qdisc_watchdog *wd = container_of(timer, struct qdisc_watchdog, |
| @@ -1490,7 +1485,7 @@ static int __tc_get_qdisc(struct sk_buff *skb, struct nlmsghdr *n, · 878 unmodified lines | ||
| 1490 | 1485 | NL_SET_ERR_MSG(extack, "Failed to find qdisc with specified classid"); |
| 1491 | 1486 | return -ENOENT; |
| 1492 | 1487 | } |
| 1493 | − q = qdisc_leaf(p, clid); | |
| 1488 | + q = qdisc_leaf(p, clid, extack); | |
| 1494 | 1489 | } else if (dev_ingress_queue(dev)) { |
| 1495 | 1490 | q = rtnl_dereference(dev_ingress_queue(dev)->qdisc_sleeping); |
| 1496 | 1491 | } |
| @@ -1501,6 +1496,8 @@ static int __tc_get_qdisc(struct sk_buff *skb, struct nlmsghdr *n, · 4 unmodified lines | ||
| 1501 | 1496 | NL_SET_ERR_MSG(extack, "Cannot find specified qdisc on specified device"); |
| 1502 | 1497 | return -ENOENT; |
| 1503 | 1498 | } |
| 1499 | + if (IS_ERR(q)) | |
| 1500 | + return PTR_ERR(q); | |
| 1504 | 1501 | |
| 1505 | 1502 | if (tcm->tcm_handle && q->handle != tcm->tcm_handle) { |
| 1506 | 1503 | NL_SET_ERR_MSG(extack, "Invalid handle"); |
| @@ -1602,7 +1599,9 @@ static int __tc_modify_qdisc(struct sk_buff *skb, struct nlmsghdr *n, · 95 unmodified lines | ||
| 1602 | 1599 | NL_SET_ERR_MSG(extack, "Failed to find specified qdisc"); |
| 1603 | 1600 | return -ENOENT; |
| 1604 | 1601 | } |
| 1605 | − q = qdisc_leaf(p, clid); | |
| 1602 | + q = qdisc_leaf(p, clid, extack); | |
| 1603 | + if (IS_ERR(q)) | |
| 1604 | + return PTR_ERR(q); | |
| 1606 | 1605 | } else if (dev_ingress_queue_create(dev)) { |
| 1607 | 1606 | q = rtnl_dereference(dev_ingress_queue(dev)->qdisc_sleeping); |
| 1608 | 1607 | } |
| @@ -835,22 +835,6 @@ update_vf(struct hfsc_class *cl, unsigned int len, u64 cur_time) · 834 unmodified lines | ||
| 835 | 835 | } |
| 836 | 836 | } |
| 837 | 837 | |
| 838 | −static unsigned int | |
| 839 | −qdisc_peek_len(struct Qdisc *sch) | |
| 840 | −{ | |
| 841 | − struct sk_buff *skb; | |
| 842 | − unsigned int len; | |
| 843 | − | |
| 844 | − skb = sch->ops->peek(sch); | |
| 845 | − if (unlikely(skb == NULL)) { | |
| 846 | − qdisc_warn_nonwc("qdisc_peek_len", sch); | |
| 847 | − return 0; | |
| 848 | − } | |
| 849 | − len = qdisc_pkt_len(skb); | |
| 850 | − | |
| 851 | − return len; | |
| 852 | −} | |
| 853 | − | |
| 854 | 838 | static void |
| 855 | 839 | hfsc_adjust_levels(struct hfsc_class *cl) |
| 856 | 840 | { |
| @@ -989,7 +989,7 @@ static struct sk_buff *agg_dequeue(struct qfq_aggregate *agg, · 988 unmodified lines | ||
| 989 | 989 | |
| 990 | 990 | if (cl->qdisc->q.qlen == 0) /* no more packets, remove from list */ |
| 991 | 991 | list_del_init(&cl->alist); |
| 992 | − else if (cl->deficit < qdisc_pkt_len(cl->qdisc->ops->peek(cl->qdisc))) { | |
| 992 | + else if (cl->deficit < qdisc_peek_len(cl->qdisc)) { | |
| 993 | 993 | cl->deficit += agg->lmax; |
| 994 | 994 | list_move_tail(&cl->alist, &agg->active); |
| 995 | 995 | } |
| @@ -704,8 +704,10 @@ static void tipc_topsrv_stop(struct net *net) · 703 unmodified lines | ||
| 704 | 704 | for (id = 0; srv->idr_in_use; id++) { |
| 705 | 705 | con = idr_find(&srv->conn_idr, id); |
| 706 | 706 | if (con) { |
| 707 | + conn_get(con); | |
| 707 | 708 | spin_unlock_bh(&srv->idr_lock); |
| 708 | 709 | tipc_conn_close(con); |
| 710 | + conn_put(con); | |
| 709 | 711 | spin_lock_bh(&srv->idr_lock); |
| 710 | 712 | } |
| 711 | 713 | } |
| @@ -407,6 +407,8 @@ EXPORT_SYMBOL_GPL(vsock_enqueue_accept); · 406 unmodified lines | ||
| 407 | 407 | |
| 408 | 408 | static bool vsock_use_local_transport(unsigned int remote_cid) |
| 409 | 409 | { |
| 410 | + lockdep_assert_held(&vsock_register_mutex); | |
| 411 | + | |
| 410 | 412 | if (!transport_local) |
| 411 | 413 | return false; |
| 412 | 414 | |
| @@ -464,6 +466,8 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk) · 51 unmodified lines | ||
| 464 | 466 | |
| 465 | 467 | remote_flags = vsk->remote_addr.svm_flags; |
| 466 | 468 | |
| 469 | + mutex_lock(&vsock_register_mutex); | |
| 470 | + | |
| 467 | 471 | switch (sk->sk_type) { |
| 468 | 472 | case SOCK_DGRAM: |
| 469 | 473 | new_transport = transport_dgram; |
| @@ -479,12 +483,15 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk) · 9 unmodified lines | ||
| 479 | 483 | new_transport = transport_h2g; |
| 480 | 484 | break; |
| 481 | 485 | default: |
| 482 | − return -ESOCKTNOSUPPORT; | |
| 486 | + ret = -ESOCKTNOSUPPORT; | |
| 487 | + goto err; | |
| 483 | 488 | } |
| 484 | 489 | |
| 485 | 490 | if (vsk->transport) { |
| 486 | − if (vsk->transport == new_transport) | |
| 487 | − return 0; | |
| 491 | + if (vsk->transport == new_transport) { | |
| 492 | + ret = 0; | |
| 493 | + goto err; | |
| 494 | + } | |
| 488 | 495 | |
| 489 | 496 | /* transport->release() must be called with sock lock acquired. |
| 490 | 497 | * This path can only be taken during vsock_connect(), where we |
| @@ -508,8 +515,16 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk) · 17 unmodified lines | ||
| 508 | 515 | /* We increase the module refcnt to prevent the transport unloading |
| 509 | 516 | * while there are open sockets assigned to it. |
| 510 | 517 | */ |
| 511 | − if (!new_transport || !try_module_get(new_transport->module)) | |
| 512 | − return -ENODEV; | |
| 518 | + if (!new_transport || !try_module_get(new_transport->module)) { | |
| 519 | + ret = -ENODEV; | |
| 520 | + goto err; | |
| 521 | + } | |
| 522 | + | |
| 523 | + /* It's safe to release the mutex after a successful try_module_get(). | |
| 524 | + * Whichever transport `new_transport` points at, it won't go away until | |
| 525 | + * the last module_put() below or in vsock_deassign_transport(). | |
| 526 | + */ | |
| 527 | + mutex_unlock(&vsock_register_mutex); | |
| 513 | 528 | |
| 514 | 529 | if (sk->sk_type == SOCK_SEQPACKET) { |
| 515 | 530 | if (!new_transport->seqpacket_allow || |
| @@ -528,12 +543,31 @@ int vsock_assign_transport(struct vsock_sock *vsk, struct vsock_sock *psk) · 12 unmodified lines | ||
| 528 | 543 | vsk->transport = new_transport; |
| 529 | 544 | |
| 530 | 545 | return 0; |
| 546 | +err: | |
| 547 | + mutex_unlock(&vsock_register_mutex); | |
| 548 | + return ret; | |
| 531 | 549 | } |
| 532 | 550 | EXPORT_SYMBOL_GPL(vsock_assign_transport); |
| 533 | 551 | |
| 552 | +/* | |
| 553 | + * Provide safe access to static transport_{h2g,g2h,dgram,local} callbacks. | |
| 554 | + * Otherwise we may race with module removal. Do not use on `vsk->transport`. | |
| 555 | + */ | |
| 556 | +static u32 vsock_registered_transport_cid(const struct vsock_transport **transport) | |
| 557 | +{ | |
| 558 | + u32 cid = VMADDR_CID_ANY; | |
| 559 | + | |
| 560 | + mutex_lock(&vsock_register_mutex); | |
| 561 | + if (*transport) | |
| 562 | + cid = (*transport)->get_local_cid(); | |
| 563 | + mutex_unlock(&vsock_register_mutex); | |
| 564 | + | |
| 565 | + return cid; | |
| 566 | +} | |
| 567 | + | |
| 534 | 568 | bool vsock_find_cid(unsigned int cid) |
| 535 | 569 | { |
| 536 | − if (transport_g2h && cid == transport_g2h->get_local_cid()) | |
| 570 | + if (cid == vsock_registered_transport_cid(&transport_g2h)) | |
| 537 | 571 | return true; |
| 538 | 572 | |
| 539 | 573 | if (transport_h2g && cid == VMADDR_CID_HOST) |
| @@ -2536,18 +2570,19 @@ static long vsock_dev_do_ioctl(struct file *filp, · 1996 unmodified lines | ||
| 2536 | 2570 | unsigned int cmd, void __user *ptr) |
| 2537 | 2571 | { |
| 2538 | 2572 | u32 __user *p = ptr; |
| 2539 | − u32 cid = VMADDR_CID_ANY; | |
| 2540 | 2573 | int retval = 0; |
| 2574 | + u32 cid; | |
| 2541 | 2575 | |
| 2542 | 2576 | switch (cmd) { |
| 2543 | 2577 | case IOCTL_VM_SOCKETS_GET_LOCAL_CID: |
| 2544 | 2578 | /* To be compatible with the VMCI behavior, we prioritize the |
| 2545 | 2579 | * guest CID instead of well-know host CID (VMADDR_CID_HOST). |
| 2546 | 2580 | */ |
| 2547 | − if (transport_g2h) | |
| 2548 | − cid = transport_g2h->get_local_cid(); | |
| 2549 | − else if (transport_h2g) | |
| 2550 | − cid = transport_h2g->get_local_cid(); | |
| 2581 | + cid = vsock_registered_transport_cid(&transport_g2h); | |
| 2582 | + if (cid == VMADDR_CID_ANY) | |
| 2583 | + cid = vsock_registered_transport_cid(&transport_h2g); | |
| 2584 | + if (cid == VMADDR_CID_ANY) | |
| 2585 | + cid = vsock_registered_transport_cid(&transport_local); | |
| 2551 | 2586 | |
| 2552 | 2587 | if (put_user(cid, p) != 0) |
| 2553 | 2588 | retval = -EFAULT; |
| @@ -229,6 +229,7 @@ static int validate_beacon_head(const struct nlattr *attr, · 228 unmodified lines | ||
| 229 | 229 | unsigned int len = nla_len(attr); |
| 230 | 230 | const struct element *elem; |
| 231 | 231 | const struct ieee80211_mgmt *mgmt = (void *)data; |
| 232 | + const struct ieee80211_ext *ext; | |
| 232 | 233 | unsigned int fixedlen, hdrlen; |
| 233 | 234 | bool s1g_bcn; |
| 234 | 235 | |
| @@ -237,8 +238,10 @@ static int validate_beacon_head(const struct nlattr *attr, · 2 unmodified lines | ||
| 237 | 238 | |
| 238 | 239 | s1g_bcn = ieee80211_is_s1g_beacon(mgmt->frame_control); |
| 239 | 240 | if (s1g_bcn) { |
| 240 | − fixedlen = offsetof(struct ieee80211_ext, | |
| 241 | − u.s1g_beacon.variable); | |
| 241 | + ext = (struct ieee80211_ext *)mgmt; | |
| 242 | + fixedlen = | |
| 243 | + offsetof(struct ieee80211_ext, u.s1g_beacon.variable) + | |
| 244 | + ieee80211_s1g_optional_len(ext->frame_control); | |
| 242 | 245 | hdrlen = offsetof(struct ieee80211_ext, u.s1g_beacon); |
| 243 | 246 | } else { |
| 244 | 247 | fixedlen = offsetof(struct ieee80211_mgmt, |
| @@ -820,6 +820,52 @@ bool ieee80211_is_valid_amsdu(struct sk_buff *skb, u8 mesh_hdr) · 819 unmodified lines | ||
| 820 | 820 | } |
| 821 | 821 | EXPORT_SYMBOL(ieee80211_is_valid_amsdu); |
| 822 | 822 | |
| 823 | + | |
| 824 | +/* | |
| 825 | + * Detects if an MSDU frame was maliciously converted into an A-MSDU | |
| 826 | + * frame by an adversary. This is done by parsing the received frame | |
| 827 | + * as if it were a regular MSDU, even though the A-MSDU flag is set. | |
| 828 | + * | |
| 829 | + * For non-mesh interfaces, detection involves checking whether the | |
| 830 | + * payload, when interpreted as an MSDU, begins with a valid RFC1042 | |
| 831 | + * header. This is done by comparing the A-MSDU subheader's destination | |
| 832 | + * address to the start of the RFC1042 header. | |
| 833 | + * | |
| 834 | + * For mesh interfaces, the MSDU includes a 6-byte Mesh Control field | |
| 835 | + * and an optional variable-length Mesh Address Extension field before | |
| 836 | + * the RFC1042 header. The position of the RFC1042 header must therefore | |
| 837 | + * be calculated based on the mesh header length. | |
| 838 | + * | |
| 839 | + * Since this function intentionally parses an A-MSDU frame as an MSDU, | |
| 840 | + * it only assumes that the A-MSDU subframe header is present, and | |
| 841 | + * beyond this it performs its own bounds checks under the assumption | |
| 842 | + * that the frame is instead parsed as a non-aggregated MSDU. | |
| 843 | + */ | |
| 844 | +static bool | |
| 845 | +is_amsdu_aggregation_attack(struct ethhdr *eth, struct sk_buff *skb, | |
| 846 | + enum nl80211_iftype iftype) | |
| 847 | +{ | |
| 848 | + int offset; | |
| 849 | + | |
| 850 | + /* Non-mesh case can be directly compared */ | |
| 851 | + if (iftype != NL80211_IFTYPE_MESH_POINT) | |
| 852 | + return ether_addr_equal(eth->h_dest, rfc1042_header); | |
| 853 | + | |
| 854 | + offset = __ieee80211_get_mesh_hdrlen(eth->h_dest[0]); | |
| 855 | + if (offset == 6) { | |
| 856 | + /* Mesh case with empty address extension field */ | |
| 857 | + return ether_addr_equal(eth->h_source, rfc1042_header); | |
| 858 | + } else if (offset + ETH_ALEN <= skb->len) { | |
| 859 | + /* Mesh case with non-empty address extension field */ | |
| 860 | + u8 temp[ETH_ALEN]; | |
| 861 | + | |
| 862 | + skb_copy_bits(skb, offset, temp, ETH_ALEN); | |
| 863 | + return ether_addr_equal(temp, rfc1042_header); | |
| 864 | + } | |
| 865 | + | |
| 866 | + return false; | |
| 867 | +} | |
| 868 | + | |
| 823 | 869 | void ieee80211_amsdu_to_8023s(struct sk_buff *skb, struct sk_buff_head *list, |
| 824 | 870 | const u8 *addr, enum nl80211_iftype iftype, |
| 825 | 871 | const unsigned int extra_headroom, |
| @@ -861,8 +907,10 @@ void ieee80211_amsdu_to_8023s(struct sk_buff *skb, struct sk_buff_head *list, · 35 unmodified lines | ||
| 861 | 907 | /* the last MSDU has no padding */ |
| 862 | 908 | if (subframe_len > remaining) |
| 863 | 909 | goto purge; |
| 864 | − /* mitigate A-MSDU aggregation injection attacks */ | |
| 865 | − if (ether_addr_equal(hdr.eth.h_dest, rfc1042_header)) | |
| 910 | + /* mitigate A-MSDU aggregation injection attacks, to be | |
| 911 | + * checked when processing first subframe (offset == 0). | |
| 912 | + */ | |
| 913 | + if (offset == 0 && is_amsdu_aggregation_attack(&hdr.eth, skb, iftype)) | |
| 866 | 914 | goto purge; |
| 867 | 915 | |
| 868 | 916 | offset += sizeof(struct ethhdr); |
| @@ -66,7 +66,7 @@ impl<T: drm::Driver> Device<T> { · 65 unmodified lines | ||
| 66 | 66 | open: Some(drm::File::<T::File>::open_callback), |
| 67 | 67 | postclose: Some(drm::File::<T::File>::postclose_callback), |
| 68 | 68 | unload: None, |
| 69 | − release: None, | |
| 69 | + release: Some(Self::release), | |
| 70 | 70 | master_set: None, |
| 71 | 71 | master_drop: None, |
| 72 | 72 | debugfs_init: None, |
| @@ -162,6 +162,16 @@ impl<T: drm::Driver> Device<T> { · 89 unmodified lines | ||
| 162 | 162 | // SAFETY: `ptr` is valid by the safety requirements of this function. |
| 163 | 163 | unsafe { &*ptr.cast() } |
| 164 | 164 | } |
| 165 | + | |
| 166 | + extern "C" fn release(ptr: *mut bindings::drm_device) { | |
| 167 | + // SAFETY: `ptr` is a valid pointer to a `struct drm_device` and embedded in `Self`. | |
| 168 | + let this = unsafe { Self::from_drm_device(ptr) }; | |
| 169 | + | |
| 170 | + // SAFETY: | |
| 171 | + // - When `release` runs it is guaranteed that there is no further access to `this`. | |
| 172 | + // - `this` is valid for dropping. | |
| 173 | + unsafe { core::ptr::drop_in_place(this) }; | |
| 174 | + } | |
| 165 | 175 | } |
| 166 | 176 | |
| 167 | 177 | impl<T: drm::Driver> Deref for Device<T> { |
| @@ -10,7 +10,6 @@ use crate::{ · 9 unmodified lines | ||
| 10 | 10 | drm, |
| 11 | 11 | error::{to_result, Result}, |
| 12 | 12 | prelude::*, |
| 13 | − str::CStr, | |
| 14 | 13 | types::ARef, |
| 15 | 14 | }; |
| 16 | 15 | use macros::vtable; |
| @@ -164,8 +164,12 @@ static int damon_sample_mtier_enable_store( · 163 unmodified lines | ||
| 164 | 164 | if (enable == enabled) |
| 165 | 165 | return 0; |
| 166 | 166 | |
| 167 | − if (enable) | |
| 168 | − return damon_sample_mtier_start(); | |
| 167 | + if (enable) { | |
| 168 | + err = damon_sample_mtier_start(); | |
| 169 | + if (err) | |
| 170 | + enable = false; | |
| 171 | + return err; | |
| 172 | + } | |
| 169 | 173 | damon_sample_mtier_stop(); |
| 170 | 174 | return 0; |
| 171 | 175 | } |
| @@ -122,8 +122,12 @@ static int damon_sample_prcl_enable_store( · 121 unmodified lines | ||
| 122 | 122 | if (enable == enabled) |
| 123 | 123 | return 0; |
| 124 | 124 | |
| 125 | − if (enable) | |
| 126 | − return damon_sample_prcl_start(); | |
| 125 | + if (enable) { | |
| 126 | + err = damon_sample_prcl_start(); | |
| 127 | + if (err) | |
| 128 | + enable = false; | |
| 129 | + return err; | |
| 130 | + } | |
| 127 | 131 | damon_sample_prcl_stop(); |
| 128 | 132 | return 0; |
| 129 | 133 | } |
| @@ -102,8 +102,12 @@ static int damon_sample_wsse_enable_store( · 101 unmodified lines | ||
| 102 | 102 | if (enable == enabled) |
| 103 | 103 | return 0; |
| 104 | 104 | |
| 105 | − if (enable) | |
| 106 | − return damon_sample_wsse_start(); | |
| 105 | + if (enable) { | |
| 106 | + err = damon_sample_wsse_start(); | |
| 107 | + if (err) | |
| 108 | + enable = false; | |
| 109 | + return err; | |
| 110 | + } | |
| 107 | 111 | damon_sample_wsse_stop(); |
| 108 | 112 | return 0; |
| 109 | 113 | } |
| @@ -20,6 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | #include <linux/of_fdt.h> |
| 21 | 21 | #include <linux/page_ext.h> |
| 22 | 22 | #include <linux/radix-tree.h> |
| 23 | +#include <linux/maple_tree.h> | |
| 23 | 24 | #include <linux/slab.h> |
| 24 | 25 | #include <linux/threads.h> |
| 25 | 26 | #include <linux/vmalloc.h> |
| @@ -93,6 +94,12 @@ LX_GDBPARSED(RADIX_TREE_MAP_SIZE) · 67 unmodified lines | ||
| 93 | 94 | LX_GDBPARSED(RADIX_TREE_MAP_SHIFT) |
| 94 | 95 | LX_GDBPARSED(RADIX_TREE_MAP_MASK) |
| 95 | 96 | |
| 97 | +/* linux/maple_tree.h */ | |
| 98 | +LX_VALUE(MAPLE_NODE_SLOTS) | |
| 99 | +LX_VALUE(MAPLE_RANGE64_SLOTS) | |
| 100 | +LX_VALUE(MAPLE_ARANGE64_SLOTS) | |
| 101 | +LX_GDBPARSED(MAPLE_NODE_MASK) | |
| 102 | + | |
| 96 | 103 | /* linux/vmalloc.h */ |
| 97 | 104 | LX_VALUE(VM_IOREMAP) |
| 98 | 105 | LX_VALUE(VM_ALLOC) |
| @@ -7,7 +7,7 @@ import gdb · 6 unmodified lines | ||
| 7 | 7 | from linux import constants |
| 8 | 8 | from linux import cpus |
| 9 | 9 | from linux import utils |
| 10 | −from linux import radixtree | |
| 10 | +from linux import mapletree | |
| 11 | 11 | |
| 12 | 12 | irq_desc_type = utils.CachedType("struct irq_desc") |
| 13 | 13 | |
| @@ -23,12 +23,12 @@ def irqd_is_level(desc): · 9 unmodified lines | ||
| 23 | 23 | def show_irq_desc(prec, irq): |
| 24 | 24 | text = "" |
| 25 | 25 | |
| 26 | − desc = radixtree.lookup(gdb.parse_and_eval("&irq_desc_tree"), irq) | |
| 26 | + desc = mapletree.mtree_load(gdb.parse_and_eval("&sparse_irqs"), irq) | |
| 27 | 27 | if desc is None: |
| 28 | 28 | return text |
| 29 | 29 | |
| 30 | − desc = desc.cast(irq_desc_type.get_type()) | |
| 31 | − if desc is None: | |
| 30 | + desc = desc.cast(irq_desc_type.get_type().pointer()) | |
| 31 | + if desc == 0: | |
| 32 | 32 | return text |
| 33 | 33 | |
| 34 | 34 | if irq_settings_is_hidden(desc): |
| @@ -110,7 +110,7 @@ def x86_show_mce(prec, var, pfx, desc): · 75 unmodified lines | ||
| 110 | 110 | pvar = gdb.parse_and_eval(var) |
| 111 | 111 | text = "%*s: " % (prec, pfx) |
| 112 | 112 | for cpu in cpus.each_online_cpu(): |
| 113 | − text += "%10u " % (cpus.per_cpu(pvar, cpu)) | |
| 113 | + text += "%10u " % (cpus.per_cpu(pvar, cpu).dereference()) | |
| 114 | 114 | text += " %s\n" % (desc) |
| 115 | 115 | return text |
| 116 | 116 | |
| @@ -142,7 +142,7 @@ def x86_show_interupts(prec): · 25 unmodified lines | ||
| 142 | 142 | |
| 143 | 143 | if constants.LX_CONFIG_X86_MCE: |
| 144 | 144 | text += x86_show_mce(prec, "&mce_exception_count", "MCE", "Machine check exceptions") |
| 145 | − text == x86_show_mce(prec, "&mce_poll_count", "MCP", "Machine check polls") | |
| 145 | + text += x86_show_mce(prec, "&mce_poll_count", "MCP", "Machine check polls") | |
| 146 | 146 | |
| 147 | 147 | text += show_irq_err_count(prec) |
| 148 | 148 | |
| @@ -221,8 +221,8 @@ class LxInterruptList(gdb.Command): · 72 unmodified lines | ||
| 221 | 221 | gdb.write("CPU%-8d" % cpu) |
| 222 | 222 | gdb.write("\n") |
| 223 | 223 | |
| 224 | − if utils.gdb_eval_or_none("&irq_desc_tree") is None: | |
| 225 | − return | |
| 224 | + if utils.gdb_eval_or_none("&sparse_irqs") is None: | |
| 225 | + raise gdb.GdbError("Unable to find the sparse IRQ tree, is CONFIG_SPARSE_IRQ enabled?") | |
| 226 | 226 | |
| 227 | 227 | for irq in range(nr_irqs): |
| 228 | 228 | gdb.write(show_irq_desc(prec, irq)) |
| @@ -0,0 +1,252 @@ | ||
| 1 | +# SPDX-License-Identifier: GPL-2.0 | |
| 2 | +# | |
| 3 | +# Maple tree helpers | |
| 4 | +# | |
| 5 | +# Copyright (c) 2025 Broadcom | |
| 6 | +# | |
| 7 | +# Authors: | |
| 8 | +# Florian Fainelli <florian.fainelli@broadcom.com> | |
| 9 | + | |
| 10 | +import gdb | |
| 11 | + | |
| 12 | +from linux import utils | |
| 13 | +from linux import constants | |
| 14 | +from linux import xarray | |
| 15 | + | |
| 16 | +maple_tree_root_type = utils.CachedType("struct maple_tree") | |
| 17 | +maple_node_type = utils.CachedType("struct maple_node") | |
| 18 | +maple_enode_type = utils.CachedType("void") | |
| 19 | + | |
| 20 | +maple_dense = 0 | |
| 21 | +maple_leaf_64 = 1 | |
| 22 | +maple_range_64 = 2 | |
| 23 | +maple_arange_64 = 3 | |
| 24 | + | |
| 25 | +class Mas(object): | |
| 26 | + ma_active = 0 | |
| 27 | + ma_start = 1 | |
| 28 | + ma_root = 2 | |
| 29 | + ma_none = 3 | |
| 30 | + ma_pause = 4 | |
| 31 | + ma_overflow = 5 | |
| 32 | + ma_underflow = 6 | |
| 33 | + ma_error = 7 | |
| 34 | + | |
| 35 | + def __init__(self, mt, first, end): | |
| 36 | + if mt.type == maple_tree_root_type.get_type().pointer(): | |
| 37 | + self.tree = mt.dereference() | |
| 38 | + elif mt.type != maple_tree_root_type.get_type(): | |
| 39 | + raise gdb.GdbError("must be {} not {}" | |
| 40 | + .format(maple_tree_root_type.get_type().pointer(), mt.type)) | |
| 41 | + self.tree = mt | |
| 42 | + self.index = first | |
| 43 | + self.last = end | |
| 44 | + self.node = None | |
| 45 | + self.status = self.ma_start | |
| 46 | + self.min = 0 | |
| 47 | + self.max = -1 | |
| 48 | + | |
| 49 | + def is_start(self): | |
| 50 | + # mas_is_start() | |
| 51 | + return self.status == self.ma_start | |
| 52 | + | |
| 53 | + def is_ptr(self): | |
| 54 | + # mas_is_ptr() | |
| 55 | + return self.status == self.ma_root | |
| 56 | + | |
| 57 | + def is_none(self): | |
| 58 | + # mas_is_none() | |
| 59 | + return self.status == self.ma_none | |
| 60 | + | |
| 61 | + def root(self): | |
| 62 | + # mas_root() | |
| 63 | + return self.tree['ma_root'].cast(maple_enode_type.get_type().pointer()) | |
| 64 | + | |
| 65 | + def start(self): | |
| 66 | + # mas_start() | |
| 67 | + if self.is_start() is False: | |
| 68 | + return None | |
| 69 | + | |
| 70 | + self.min = 0 | |
| 71 | + self.max = ~0 | |
| 72 | + | |
| 73 | + while True: | |
| 74 | + self.depth = 0 | |
| 75 | + root = self.root() | |
| 76 | + if xarray.xa_is_node(root): | |
| 77 | + self.depth = 0 | |
| 78 | + self.status = self.ma_active | |
| 79 | + self.node = mte_safe_root(root) | |
| 80 | + self.offset = 0 | |
| 81 | + if mte_dead_node(self.node) is True: | |
| 82 | + continue | |
| 83 | + | |
| 84 | + return None | |
| 85 | + | |
| 86 | + self.node = None | |
| 87 | + # Empty tree | |
| 88 | + if root is None: | |
| 89 | + self.status = self.ma_none | |
| 90 | + self.offset = constants.LX_MAPLE_NODE_SLOTS | |
| 91 | + return None | |
| 92 | + | |
| 93 | + # Single entry tree | |
| 94 | + self.status = self.ma_root | |
| 95 | + self.offset = constants.LX_MAPLE_NODE_SLOTS | |
| 96 | + | |
| 97 | + if self.index != 0: | |
| 98 | + return None | |
| 99 | + | |
| 100 | + return root | |
| 101 | + | |
| 102 | + return None | |
| 103 | + | |
| 104 | + def reset(self): | |
| 105 | + # mas_reset() | |
| 106 | + self.status = self.ma_start | |
| 107 | + self.node = None | |
| 108 | + | |
| 109 | +def mte_safe_root(node): | |
| 110 | + if node.type != maple_enode_type.get_type().pointer(): | |
| 111 | + raise gdb.GdbError("{} must be {} not {}" | |
| 112 | + .format(mte_safe_root.__name__, maple_enode_type.get_type().pointer(), node.type)) | |
| 113 | + ulong_type = utils.get_ulong_type() | |
| 114 | + indirect_ptr = node.cast(ulong_type) & ~0x2 | |
| 115 | + val = indirect_ptr.cast(maple_enode_type.get_type().pointer()) | |
| 116 | + return val | |
| 117 | + | |
| 118 | +def mte_node_type(entry): | |
| 119 | + ulong_type = utils.get_ulong_type() | |
| 120 | + val = None | |
| 121 | + if entry.type == maple_enode_type.get_type().pointer(): | |
| 122 | + val = entry.cast(ulong_type) | |
| 123 | + elif entry.type == ulong_type: | |
| 124 | + val = entry | |
| 125 | + else: | |
| 126 | + raise gdb.GdbError("{} must be {} not {}" | |
| 127 | + .format(mte_node_type.__name__, maple_enode_type.get_type().pointer(), entry.type)) | |
| 128 | + return (val >> 0x3) & 0xf | |
| 129 | + | |
| 130 | +def ma_dead_node(node): | |
| 131 | + if node.type != maple_node_type.get_type().pointer(): | |
| 132 | + raise gdb.GdbError("{} must be {} not {}" | |
| 133 | + .format(ma_dead_node.__name__, maple_node_type.get_type().pointer(), node.type)) | |
| 134 | + ulong_type = utils.get_ulong_type() | |
| 135 | + parent = node['parent'] | |
| 136 | + indirect_ptr = node['parent'].cast(ulong_type) & ~constants.LX_MAPLE_NODE_MASK | |
| 137 | + return indirect_ptr == node | |
| 138 | + | |
| 139 | +def mte_to_node(enode): | |
| 140 | + ulong_type = utils.get_ulong_type() | |
| 141 | + if enode.type == maple_enode_type.get_type().pointer(): | |
| 142 | + indirect_ptr = enode.cast(ulong_type) | |
| 143 | + elif enode.type == ulong_type: | |
| 144 | + indirect_ptr = enode | |
| 145 | + else: | |
| 146 | + raise gdb.GdbError("{} must be {} not {}" | |
| 147 | + .format(mte_to_node.__name__, maple_enode_type.get_type().pointer(), enode.type)) | |
| 148 | + indirect_ptr = indirect_ptr & ~constants.LX_MAPLE_NODE_MASK | |
| 149 | + return indirect_ptr.cast(maple_node_type.get_type().pointer()) | |
| 150 | + | |
| 151 | +def mte_dead_node(enode): | |
| 152 | + if enode.type != maple_enode_type.get_type().pointer(): | |
| 153 | + raise gdb.GdbError("{} must be {} not {}" | |
| 154 | + .format(mte_dead_node.__name__, maple_enode_type.get_type().pointer(), enode.type)) | |
| 155 | + node = mte_to_node(enode) | |
| 156 | + return ma_dead_node(node) | |
| 157 | + | |
| 158 | +def ma_is_leaf(tp): | |
| 159 | + result = tp < maple_range_64 | |
| 160 | + return tp < maple_range_64 | |
| 161 | + | |
| 162 | +def mt_pivots(t): | |
| 163 | + if t == maple_dense: | |
| 164 | + return 0 | |
| 165 | + elif t == maple_leaf_64 or t == maple_range_64: | |
| 166 | + return constants.LX_MAPLE_RANGE64_SLOTS - 1 | |
| 167 | + elif t == maple_arange_64: | |
| 168 | + return constants.LX_MAPLE_ARANGE64_SLOTS - 1 | |
| 169 | + | |
| 170 | +def ma_pivots(node, t): | |
| 171 | + if node.type != maple_node_type.get_type().pointer(): | |
| 172 | + raise gdb.GdbError("{}: must be {} not {}" | |
| 173 | + .format(ma_pivots.__name__, maple_node_type.get_type().pointer(), node.type)) | |
| 174 | + if t == maple_arange_64: | |
| 175 | + return node['ma64']['pivot'] | |
| 176 | + elif t == maple_leaf_64 or t == maple_range_64: | |
| 177 | + return node['mr64']['pivot'] | |
| 178 | + else: | |
| 179 | + return None | |
| 180 | + | |
| 181 | +def ma_slots(node, tp): | |
| 182 | + if node.type != maple_node_type.get_type().pointer(): | |
| 183 | + raise gdb.GdbError("{}: must be {} not {}" | |
| 184 | + .format(ma_slots.__name__, maple_node_type.get_type().pointer(), node.type)) | |
| 185 | + if tp == maple_arange_64: | |
| 186 | + return node['ma64']['slot'] | |
| 187 | + elif tp == maple_range_64 or tp == maple_leaf_64: | |
| 188 | + return node['mr64']['slot'] | |
| 189 | + elif tp == maple_dense: | |
| 190 | + return node['slot'] | |
| 191 | + else: | |
| 192 | + return None | |
| 193 | + | |
| 194 | +def mt_slot(mt, slots, offset): | |
| 195 | + ulong_type = utils.get_ulong_type() | |
| 196 | + return slots[offset].cast(ulong_type) | |
| 197 | + | |
| 198 | +def mtree_lookup_walk(mas): | |
| 199 | + ulong_type = utils.get_ulong_type() | |
| 200 | + n = mas.node | |
| 201 | + | |
| 202 | + while True: | |
| 203 | + node = mte_to_node(n) | |
| 204 | + tp = mte_node_type(n) | |
| 205 | + pivots = ma_pivots(node, tp) | |
| 206 | + end = mt_pivots(tp) | |
| 207 | + offset = 0 | |
| 208 | + while True: | |
| 209 | + if pivots[offset] >= mas.index: | |
| 210 | + break | |
| 211 | + if offset >= end: | |
| 212 | + break | |
| 213 | + offset += 1 | |
| 214 | + | |
| 215 | + slots = ma_slots(node, tp) | |
| 216 | + n = mt_slot(mas.tree, slots, offset) | |
| 217 | + if ma_dead_node(node) is True: | |
| 218 | + mas.reset() | |
| 219 | + return None | |
| 220 | + break | |
| 221 | + | |
| 222 | + if ma_is_leaf(tp) is True: | |
| 223 | + break | |
| 224 | + | |
| 225 | + return n | |
| 226 | + | |
| 227 | +def mtree_load(mt, index): | |
| 228 | + ulong_type = utils.get_ulong_type() | |
| 229 | + # MT_STATE(...) | |
| 230 | + mas = Mas(mt, index, index) | |
| 231 | + entry = None | |
| 232 | + | |
| 233 | + while True: | |
| 234 | + entry = mas.start() | |
| 235 | + if mas.is_none(): | |
| 236 | + return None | |
| 237 | + | |
| 238 | + if mas.is_ptr(): | |
| 239 | + if index != 0: | |
| 240 | + entry = None | |
| 241 | + return entry | |
| 242 | + | |
| 243 | + entry = mtree_lookup_walk(mas) | |
| 244 | + if entry is None and mas.is_start(): | |
| 245 | + continue | |
| 246 | + else: | |
| 247 | + break | |
| 248 | + | |
| 249 | + if xarray.xa_is_zero(entry): | |
| 250 | + return None | |
| 251 | + | |
| 252 | + return entry | |
| @@ -22,7 +22,7 @@ def dentry_name(d): · 21 unmodified lines | ||
| 22 | 22 | if parent == d or parent == 0: |
| 23 | 23 | return "" |
| 24 | 24 | p = dentry_name(d['d_parent']) + "/" |
| 25 | − return p + d['d_shortname']['string'].string() | |
| 25 | + return p + d['d_name']['name'].string() | |
| 26 | 26 | |
| 27 | 27 | class DentryName(gdb.Function): |
| 28 | 28 | """Return string of the full path of a dentry. |
| @@ -0,0 +1,28 @@ | ||
| 1 | +# SPDX-License-Identifier: GPL-2.0 | |
| 2 | +# | |
| 3 | +# Xarray helpers | |
| 4 | +# | |
| 5 | +# Copyright (c) 2025 Broadcom | |
| 6 | +# | |
| 7 | +# Authors: | |
| 8 | +# Florian Fainelli <florian.fainelli@broadcom.com> | |
| 9 | + | |
| 10 | +import gdb | |
| 11 | + | |
| 12 | +from linux import utils | |
| 13 | +from linux import constants | |
| 14 | + | |
| 15 | +def xa_is_internal(entry): | |
| 16 | + ulong_type = utils.get_ulong_type() | |
| 17 | + return ((entry.cast(ulong_type) & 3) == 2) | |
| 18 | + | |
| 19 | +def xa_mk_internal(v): | |
| 20 | + return ((v << 2) | 2) | |
| 21 | + | |
| 22 | +def xa_is_zero(entry): | |
| 23 | + ulong_type = utils.get_ulong_type() | |
| 24 | + return entry.cast(ulong_type) == xa_mk_internal(257) | |
| 25 | + | |
| 26 | +def xa_is_node(entry): | |
| 27 | + ulong_type = utils.get_ulong_type() | |
| 28 | + return xa_is_internal(entry) and (entry.cast(ulong_type) > 4096) | |
| @@ -98,7 +98,7 @@ static int snd_card_ad1816a_pnp(int dev, struct pnp_card_link *card, · 97 unmodified lines | ||
| 98 | 98 | pdev = pnp_request_card_device(card, id->devs[1].id, NULL); |
| 99 | 99 | if (pdev == NULL) { |
| 100 | 100 | mpu_port[dev] = -1; |
| 101 | − dev_warn(&pdev->dev, "MPU401 device busy, skipping.\n"); | |
| 101 | + pr_warn("MPU401 device busy, skipping.\n"); | |
| 102 | 102 | return 0; |
| 103 | 103 | } |
| 104 | 104 | |
| @@ -4551,7 +4551,9 @@ HDA_CODEC_ENTRY(0x10de002e, "Tegra186 HDMI/DP1", patch_tegra_hdmi), · 4550 unmodified lines | ||
| 4551 | 4551 | HDA_CODEC_ENTRY(0x10de002f, "Tegra194 HDMI/DP2", patch_tegra_hdmi), |
| 4552 | 4552 | HDA_CODEC_ENTRY(0x10de0030, "Tegra194 HDMI/DP3", patch_tegra_hdmi), |
| 4553 | 4553 | HDA_CODEC_ENTRY(0x10de0031, "Tegra234 HDMI/DP", patch_tegra234_hdmi), |
| 4554 | +HDA_CODEC_ENTRY(0x10de0033, "SoC 33 HDMI/DP", patch_tegra234_hdmi), | |
| 4554 | 4555 | HDA_CODEC_ENTRY(0x10de0034, "Tegra264 HDMI/DP", patch_tegra234_hdmi), |
| 4556 | +HDA_CODEC_ENTRY(0x10de0035, "SoC 35 HDMI/DP", patch_tegra234_hdmi), | |
| 4555 | 4557 | HDA_CODEC_ENTRY(0x10de0040, "GPU 40 HDMI/DP", patch_nvhdmi), |
| 4556 | 4558 | HDA_CODEC_ENTRY(0x10de0041, "GPU 41 HDMI/DP", patch_nvhdmi), |
| 4557 | 4559 | HDA_CODEC_ENTRY(0x10de0042, "GPU 42 HDMI/DP", patch_nvhdmi), |
| @@ -4590,15 +4592,32 @@ HDA_CODEC_ENTRY(0x10de0097, "GPU 97 HDMI/DP", patch_nvhdmi), · 32 unmodified lines | ||
| 4590 | 4592 | HDA_CODEC_ENTRY(0x10de0098, "GPU 98 HDMI/DP", patch_nvhdmi), |
| 4591 | 4593 | HDA_CODEC_ENTRY(0x10de0099, "GPU 99 HDMI/DP", patch_nvhdmi), |
| 4592 | 4594 | HDA_CODEC_ENTRY(0x10de009a, "GPU 9a HDMI/DP", patch_nvhdmi), |
| 4595 | +HDA_CODEC_ENTRY(0x10de009b, "GPU 9b HDMI/DP", patch_nvhdmi), | |
| 4596 | +HDA_CODEC_ENTRY(0x10de009c, "GPU 9c HDMI/DP", patch_nvhdmi), | |
| 4593 | 4597 | HDA_CODEC_ENTRY(0x10de009d, "GPU 9d HDMI/DP", patch_nvhdmi), |
| 4594 | 4598 | HDA_CODEC_ENTRY(0x10de009e, "GPU 9e HDMI/DP", patch_nvhdmi), |
| 4595 | 4599 | HDA_CODEC_ENTRY(0x10de009f, "GPU 9f HDMI/DP", patch_nvhdmi), |
| 4596 | 4600 | HDA_CODEC_ENTRY(0x10de00a0, "GPU a0 HDMI/DP", patch_nvhdmi), |
| 4601 | +HDA_CODEC_ENTRY(0x10de00a1, "GPU a1 HDMI/DP", patch_nvhdmi), | |
| 4597 | 4602 | HDA_CODEC_ENTRY(0x10de00a3, "GPU a3 HDMI/DP", patch_nvhdmi), |
| 4598 | 4603 | HDA_CODEC_ENTRY(0x10de00a4, "GPU a4 HDMI/DP", patch_nvhdmi), |
| 4599 | 4604 | HDA_CODEC_ENTRY(0x10de00a5, "GPU a5 HDMI/DP", patch_nvhdmi), |
| 4600 | 4605 | HDA_CODEC_ENTRY(0x10de00a6, "GPU a6 HDMI/DP", patch_nvhdmi), |
| 4601 | 4606 | HDA_CODEC_ENTRY(0x10de00a7, "GPU a7 HDMI/DP", patch_nvhdmi), |
| 4607 | +HDA_CODEC_ENTRY(0x10de00a8, "GPU a8 HDMI/DP", patch_nvhdmi), | |
| 4608 | +HDA_CODEC_ENTRY(0x10de00a9, "GPU a9 HDMI/DP", patch_nvhdmi), | |
| 4609 | +HDA_CODEC_ENTRY(0x10de00aa, "GPU aa HDMI/DP", patch_nvhdmi), | |
| 4610 | +HDA_CODEC_ENTRY(0x10de00ab, "GPU ab HDMI/DP", patch_nvhdmi), | |
| 4611 | +HDA_CODEC_ENTRY(0x10de00ad, "GPU ad HDMI/DP", patch_nvhdmi), | |
| 4612 | +HDA_CODEC_ENTRY(0x10de00ae, "GPU ae HDMI/DP", patch_nvhdmi), | |
| 4613 | +HDA_CODEC_ENTRY(0x10de00af, "GPU af HDMI/DP", patch_nvhdmi), | |
| 4614 | +HDA_CODEC_ENTRY(0x10de00b0, "GPU b0 HDMI/DP", patch_nvhdmi), | |
| 4615 | +HDA_CODEC_ENTRY(0x10de00b1, "GPU b1 HDMI/DP", patch_nvhdmi), | |
| 4616 | +HDA_CODEC_ENTRY(0x10de00c0, "GPU c0 HDMI/DP", patch_nvhdmi), | |
| 4617 | +HDA_CODEC_ENTRY(0x10de00c1, "GPU c1 HDMI/DP", patch_nvhdmi), | |
| 4618 | +HDA_CODEC_ENTRY(0x10de00c3, "GPU c3 HDMI/DP", patch_nvhdmi), | |
| 4619 | +HDA_CODEC_ENTRY(0x10de00c4, "GPU c4 HDMI/DP", patch_nvhdmi), | |
| 4620 | +HDA_CODEC_ENTRY(0x10de00c5, "GPU c5 HDMI/DP", patch_nvhdmi), | |
| 4602 | 4621 | HDA_CODEC_ENTRY(0x10de8001, "MCP73 HDMI", patch_nvhdmi_2ch), |
| 4603 | 4622 | HDA_CODEC_ENTRY(0x10de8067, "MCP67/68 HDMI", patch_nvhdmi_2ch), |
| 4604 | 4623 | HDA_CODEC_ENTRY(0x67663d82, "Arise 82 HDMI/DP", patch_gf_hdmi), |
| @@ -10881,6 +10881,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = { · 10880 unmodified lines | ||
| 10881 | 10881 | SND_PCI_QUIRK(0x103c, 0x8ce0, "HP SnowWhite", ALC287_FIXUP_CS35L41_I2C_2_HP_GPIO_LED), |
| 10882 | 10882 | SND_PCI_QUIRK(0x103c, 0x8cf5, "HP ZBook Studio 16", ALC245_FIXUP_CS35L41_SPI_4_HP_GPIO_LED), |
| 10883 | 10883 | SND_PCI_QUIRK(0x103c, 0x8d01, "HP ZBook Power 14 G12", ALC285_FIXUP_HP_GPIO_LED), |
| 10884 | + SND_PCI_QUIRK(0x103c, 0x8d07, "HP Victus 15-fb2xxx (MB 8D07)", ALC245_FIXUP_HP_MUTE_LED_COEFBIT), | |
| 10884 | 10885 | SND_PCI_QUIRK(0x103c, 0x8d18, "HP EliteStudio 8 AIO", ALC274_FIXUP_HP_AIO_BIND_DACS), |
| 10885 | 10886 | SND_PCI_QUIRK(0x103c, 0x8d84, "HP EliteBook X G1i", ALC285_FIXUP_HP_GPIO_LED), |
| 10886 | 10887 | SND_PCI_QUIRK(0x103c, 0x8d85, "HP EliteBook 14 G12", ALC285_FIXUP_HP_GPIO_LED), |
| @@ -11040,6 +11041,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = { · 153 unmodified lines | ||
| 11040 | 11041 | SND_PCI_QUIRK(0x1043, 0x1e63, "ASUS H7606W", ALC285_FIXUP_ASUS_GU605_SPI_SPEAKER2_TO_DAC1), |
| 11041 | 11042 | SND_PCI_QUIRK(0x1043, 0x1e83, "ASUS GA605W", ALC285_FIXUP_ASUS_GU605_SPI_SPEAKER2_TO_DAC1), |
| 11042 | 11043 | SND_PCI_QUIRK(0x1043, 0x1e8e, "ASUS Zephyrus G15", ALC289_FIXUP_ASUS_GA401), |
| 11044 | + SND_PCI_QUIRK(0x1043, 0x1e93, "ASUS ExpertBook B9403CVAR", ALC294_FIXUP_ASUS_HPE), | |
| 11043 | 11045 | SND_PCI_QUIRK(0x1043, 0x1eb3, "ASUS Ally RCLA72", ALC287_FIXUP_TAS2781_I2C), |
| 11044 | 11046 | SND_PCI_QUIRK(0x1043, 0x1ed3, "ASUS HN7306W", ALC287_FIXUP_CS35L41_I2C_2), |
| 11045 | 11047 | SND_PCI_QUIRK(0x1043, 0x1ee2, "ASUS UM6702RA/RC", ALC287_FIXUP_CS35L41_I2C_2), |
| @@ -11424,6 +11426,7 @@ static const struct hda_quirk alc269_fixup_tbl[] = { · 378 unmodified lines | ||
| 11424 | 11426 | SND_PCI_QUIRK(0x2782, 0x0228, "Infinix ZERO BOOK 13", ALC269VB_FIXUP_INFINIX_ZERO_BOOK_13), |
| 11425 | 11427 | SND_PCI_QUIRK(0x2782, 0x0232, "CHUWI CoreBook XPro", ALC269VB_FIXUP_CHUWI_COREBOOK_XPRO), |
| 11426 | 11428 | SND_PCI_QUIRK(0x2782, 0x1407, "Positivo P15X", ALC269_FIXUP_POSITIVO_P15X_HEADSET_MIC), |
| 11429 | + SND_PCI_QUIRK(0x2782, 0x1409, "Positivo K116J", ALC269_FIXUP_POSITIVO_P15X_HEADSET_MIC), | |
| 11427 | 11430 | SND_PCI_QUIRK(0x2782, 0x1701, "Infinix Y4 Max", ALC269VC_FIXUP_INFINIX_Y4_MAX), |
| 11428 | 11431 | SND_PCI_QUIRK(0x2782, 0x1705, "MEDION E15433", ALC269VC_FIXUP_INFINIX_Y4_MAX), |
| 11429 | 11432 | SND_PCI_QUIRK(0x2782, 0x1707, "Vaio VJFE-ADL", ALC298_FIXUP_SPK_VOLUME), |
| @@ -44,7 +44,7 @@ static void tas2781_apply_calib(struct tasdevice_priv *p) · 43 unmodified lines | ||
| 44 | 44 | TASDEVICE_REG(0, 0x13, 0x70), |
| 45 | 45 | TASDEVICE_REG(0, 0x18, 0x7c), |
| 46 | 46 | }; |
| 47 | − unsigned int crc, oft; | |
| 47 | + unsigned int crc, oft, node_num; | |
| 48 | 48 | unsigned char *buf; |
| 49 | 49 | int i, j, k, l; |
| 50 | 50 | |
| @@ -80,8 +80,9 @@ static void tas2781_apply_calib(struct tasdevice_priv *p) · 29 unmodified lines | ||
| 80 | 80 | dev_err(p->dev, "%s: CRC error\n", __func__); |
| 81 | 81 | return; |
| 82 | 82 | } |
| 83 | + node_num = tmp_val[1]; | |
| 83 | 84 | |
| 84 | − for (j = 0, k = 0; j < tmp_val[1]; j++) { | |
| 85 | + for (j = 0, k = 0; j < node_num; j++) { | |
| 85 | 86 | oft = j * 6 + 3; |
| 86 | 87 | if (tmp_val[oft] == TASDEV_UEFI_CALI_REG_ADDR_FLG) { |
| 87 | 88 | for (i = 0; i < TASDEV_CALIB_N; i++) { |
| @@ -99,8 +100,9 @@ static void tas2781_apply_calib(struct tasdevice_priv *p) · 11 unmodified lines | ||
| 99 | 100 | } |
| 100 | 101 | |
| 101 | 102 | data[l] = k; |
| 103 | + oft++; | |
| 102 | 104 | for (i = 0; i < TASDEV_CALIB_N * 4; i++) |
| 103 | − data[l + i] = data[4 * oft + i]; | |
| 105 | + data[l + i + 1] = data[4 * oft + i]; | |
| 104 | 106 | k++; |
| 105 | 107 | } |
| 106 | 108 | } |
| @@ -980,7 +980,7 @@ int cs35l56_hw_init(struct cs35l56_base *cs35l56_base) · 979 unmodified lines | ||
| 980 | 980 | break; |
| 981 | 981 | default: |
| 982 | 982 | dev_err(cs35l56_base->dev, "Unknown device %x\n", devid); |
| 983 | − return ret; | |
| 983 | + return -ENODEV; | |
| 984 | 984 | } |
| 985 | 985 | |
| 986 | 986 | cs35l56_base->type = devid & 0xFF; |
| @@ -9,27 +9,26 @@ · 8 unmodified lines | ||
| 9 | 9 | * Based on sound/soc/codecs/wm8974 and TI driver for kernel 2.6.27. |
| 10 | 10 | */ |
| 11 | 11 | |
| 12 | −#include <linux/module.h> | |
| 13 | −#include <linux/moduleparam.h> | |
| 14 | −#include <linux/init.h> | |
| 15 | −#include <linux/delay.h> | |
| 16 | −#include <linux/pm.h> | |
| 17 | −#include <linux/gpio.h> | |
| 18 | −#include <linux/of_gpio.h> | |
| 19 | 12 | #include <linux/cdev.h> |
| 20 | −#include <linux/slab.h> | |
| 21 | 13 | #include <linux/clk.h> |
| 14 | +#include <linux/delay.h> | |
| 15 | +#include <linux/gpio/consumer.h> | |
| 16 | +#include <linux/init.h> | |
| 17 | +#include <linux/module.h> | |
| 18 | +#include <linux/moduleparam.h> | |
| 22 | 19 | #include <linux/of_clk.h> |
| 20 | +#include <linux/pm.h> | |
| 23 | 21 | #include <linux/regulator/consumer.h> |
| 22 | +#include <linux/slab.h> | |
| 24 | 23 | |
| 25 | −#include <sound/tlv320aic32x4.h> | |
| 26 | 24 | #include <sound/core.h> |
| 25 | +#include <sound/initval.h> | |
| 27 | 26 | #include <sound/pcm.h> |
| 28 | 27 | #include <sound/pcm_params.h> |
| 29 | 28 | #include <sound/soc.h> |
| 30 | 29 | #include <sound/soc-dapm.h> |
| 31 | −#include <sound/initval.h> | |
| 32 | 30 | #include <sound/tlv.h> |
| 31 | +#include <sound/tlv320aic32x4.h> | |
| 33 | 32 | |
| 34 | 33 | #include "tlv320aic32x4.h" |
| 35 | 34 | |
| @@ -38,7 +37,7 @@ struct aic32x4_priv { · 2 unmodified lines | ||
| 38 | 37 | u32 power_cfg; |
| 39 | 38 | u32 micpga_routing; |
| 40 | 39 | bool swapdacs; |
| 41 | − int rstn_gpio; | |
| 40 | + struct gpio_desc *rstn_gpio; | |
| 42 | 41 | const char *mclk_name; |
| 43 | 42 | |
| 44 | 43 | struct regulator *supply_ldo; |
| @@ -1236,7 +1235,14 @@ static int aic32x4_parse_dt(struct aic32x4_priv *aic32x4, · 1191 unmodified lines | ||
| 1236 | 1235 | |
| 1237 | 1236 | aic32x4->swapdacs = false; |
| 1238 | 1237 | aic32x4->micpga_routing = 0; |
| 1239 | − aic32x4->rstn_gpio = of_get_named_gpio(np, "reset-gpios", 0); | |
| 1238 | + /* Assert reset using GPIOD_OUT_HIGH, because reset is GPIO_ACTIVE_LOW */ | |
| 1239 | + aic32x4->rstn_gpio = devm_gpiod_get_optional(aic32x4->dev, "reset", GPIOD_OUT_HIGH); | |
| 1240 | + if (IS_ERR(aic32x4->rstn_gpio)) { | |
| 1241 | + return dev_err_probe(aic32x4->dev, PTR_ERR(aic32x4->rstn_gpio), | |
| 1242 | + "Failed to get reset gpio\n"); | |
| 1243 | + } else { | |
| 1244 | + gpiod_set_consumer_name(aic32x4->rstn_gpio, "tlv320aic32x4_rstn"); | |
| 1245 | + } | |
| 1240 | 1246 | |
| 1241 | 1247 | if (of_property_read_u32_array(np, "aic32x4-gpio-func", |
| 1242 | 1248 | aic32x4_setup->gpio_func, 5) >= 0) |
| @@ -1346,7 +1352,6 @@ int aic32x4_probe(struct device *dev, struct regmap *regmap, · 103 unmodified lines | ||
| 1346 | 1352 | enum aic32x4_type type) |
| 1347 | 1353 | { |
| 1348 | 1354 | struct aic32x4_priv *aic32x4; |
| 1349 | − struct aic32x4_pdata *pdata = dev->platform_data; | |
| 1350 | 1355 | struct device_node *np = dev->of_node; |
| 1351 | 1356 | int ret; |
| 1352 | 1357 | |
| @@ -1363,13 +1368,7 @@ int aic32x4_probe(struct device *dev, struct regmap *regmap, · 10 unmodified lines | ||
| 1363 | 1368 | |
| 1364 | 1369 | dev_set_drvdata(dev, aic32x4); |
| 1365 | 1370 | |
| 1366 | − if (pdata) { | |
| 1367 | − aic32x4->power_cfg = pdata->power_cfg; | |
| 1368 | − aic32x4->swapdacs = pdata->swapdacs; | |
| 1369 | − aic32x4->micpga_routing = pdata->micpga_routing; | |
| 1370 | − aic32x4->rstn_gpio = pdata->rstn_gpio; | |
| 1371 | − aic32x4->mclk_name = "mclk"; | |
| 1372 | − } else if (np) { | |
| 1371 | + if (np) { | |
| 1373 | 1372 | ret = aic32x4_parse_dt(aic32x4, np); |
| 1374 | 1373 | if (ret) { |
| 1375 | 1374 | dev_err(dev, "Failed to parse DT node\n"); |
| @@ -1379,26 +1378,20 @@ int aic32x4_probe(struct device *dev, struct regmap *regmap, · 3 unmodified lines | ||
| 1379 | 1378 | aic32x4->power_cfg = 0; |
| 1380 | 1379 | aic32x4->swapdacs = false; |
| 1381 | 1380 | aic32x4->micpga_routing = 0; |
| 1382 | − aic32x4->rstn_gpio = -1; | |
| 1381 | + aic32x4->rstn_gpio = NULL; | |
| 1383 | 1382 | aic32x4->mclk_name = "mclk"; |
| 1384 | 1383 | } |
| 1385 | 1384 | |
| 1386 | − if (gpio_is_valid(aic32x4->rstn_gpio)) { | |
| 1387 | − ret = devm_gpio_request_one(dev, aic32x4->rstn_gpio, | |
| 1388 | − GPIOF_OUT_INIT_LOW, "tlv320aic32x4 rstn"); | |
| 1389 | − if (ret != 0) | |
| 1390 | − return ret; | |
| 1391 | − } | |
| 1392 | − | |
| 1393 | 1385 | ret = aic32x4_setup_regulators(dev, aic32x4); |
| 1394 | 1386 | if (ret) { |
| 1395 | 1387 | dev_err(dev, "Failed to setup regulators\n"); |
| 1396 | 1388 | return ret; |
| 1397 | 1389 | } |
| 1398 | 1390 | |
| 1399 | − if (gpio_is_valid(aic32x4->rstn_gpio)) { | |
| 1391 | + if (!aic32x4->rstn_gpio) { | |
| 1400 | 1392 | ndelay(10); |
| 1401 | − gpio_set_value_cansleep(aic32x4->rstn_gpio, 1); | |
| 1393 | + /* deassert reset */ | |
| 1394 | + gpiod_set_value_cansleep(aic32x4->rstn_gpio, 0); | |
| 1402 | 1395 | mdelay(1); |
| 1403 | 1396 | } |
| 1404 | 1397 | |
| @@ -517,7 +517,8 @@ static int fsl_asrc_config_pair(struct fsl_asrc_pair *pair, bool use_ideal_rate) · 516 unmodified lines | ||
| 517 | 517 | regmap_update_bits(asrc->regmap, REG_ASRCTR, |
| 518 | 518 | ASRCTR_ATSi_MASK(index), ASRCTR_ATS(index)); |
| 519 | 519 | regmap_update_bits(asrc->regmap, REG_ASRCTR, |
| 520 | − ASRCTR_USRi_MASK(index), 0); | |
| 520 | + ASRCTR_IDRi_MASK(index) | ASRCTR_USRi_MASK(index), | |
| 521 | + ASRCTR_USR(index)); | |
| 521 | 522 | |
| 522 | 523 | /* Set the input and output clock sources */ |
| 523 | 524 | regmap_update_bits(asrc->regmap, REG_ASRCSR, |
| @@ -803,13 +803,15 @@ static void fsl_sai_config_disable(struct fsl_sai *sai, int dir) · 802 unmodified lines | ||
| 803 | 803 | * anymore. Add software reset to fix this issue. |
| 804 | 804 | * This is a hardware bug, and will be fix in the |
| 805 | 805 | * next sai version. |
| 806 | + * | |
| 807 | + * In consumer mode, this can happen even after a | |
| 808 | + * single open/close, especially if both tx and rx | |
| 809 | + * are running concurrently. | |
| 806 | 810 | */ |
| 807 | − if (!sai->is_consumer_mode[tx]) { | |
| 808 | − /* Software Reset */ | |
| 809 | − regmap_write(sai->regmap, FSL_SAI_xCSR(tx, ofs), FSL_SAI_CSR_SR); | |
| 810 | − /* Clear SR bit to finish the reset */ | |
| 811 | − regmap_write(sai->regmap, FSL_SAI_xCSR(tx, ofs), 0); | |
| 812 | − } | |
| 811 | + /* Software Reset */ | |
| 812 | + regmap_write(sai->regmap, FSL_SAI_xCSR(tx, ofs), FSL_SAI_CSR_SR); | |
| 813 | + /* Clear SR bit to finish the reset */ | |
| 814 | + regmap_write(sai->regmap, FSL_SAI_xCSR(tx, ofs), 0); | |
| 813 | 815 | } |
| 814 | 816 | |
| 815 | 817 | static int fsl_sai_trigger(struct snd_pcm_substream *substream, int cmd, |
| @@ -42,6 +42,7 @@ config SND_SOC_INTEL_SOF_NUVOTON_COMMON · 41 unmodified lines | ||
| 42 | 42 | tristate |
| 43 | 43 | |
| 44 | 44 | config SND_SOC_INTEL_SOF_BOARD_HELPERS |
| 45 | + select SND_SOC_ACPI_INTEL_MATCH | |
| 45 | 46 | tristate |
| 46 | 47 | |
| 47 | 48 | if SND_SOC_INTEL_CATPT |
| @@ -783,6 +783,9 @@ static const struct dmi_system_id sof_sdw_quirk_table[] = { · 782 unmodified lines | ||
| 783 | 783 | static const struct snd_pci_quirk sof_sdw_ssid_quirk_table[] = { |
| 784 | 784 | SND_PCI_QUIRK(0x1043, 0x1e13, "ASUS Zenbook S14", SOC_SDW_CODEC_MIC), |
| 785 | 785 | SND_PCI_QUIRK(0x1043, 0x1f43, "ASUS Zenbook S16", SOC_SDW_CODEC_MIC), |
| 786 | + SND_PCI_QUIRK(0x17aa, 0x2347, "Lenovo P16", SOC_SDW_CODEC_MIC), | |
| 787 | + SND_PCI_QUIRK(0x17aa, 0x2348, "Lenovo P16", SOC_SDW_CODEC_MIC), | |
| 788 | + SND_PCI_QUIRK(0x17aa, 0x2349, "Lenovo P1", SOC_SDW_CODEC_MIC), | |
| 786 | 789 | {} |
| 787 | 790 | }; |
| 788 | 791 | |
| @@ -468,17 +468,17 @@ struct snd_soc_acpi_mach snd_soc_acpi_intel_arl_sdw_machines[] = { · 467 unmodified lines | ||
| 468 | 468 | .get_function_tplg_files = sof_sdw_get_tplg_files, |
| 469 | 469 | }, |
| 470 | 470 | { |
| 471 | − .link_mask = BIT(2), | |
| 472 | − .links = arl_cs42l43_l2, | |
| 471 | + .link_mask = BIT(2) | BIT(3), | |
| 472 | + .links = arl_cs42l43_l2_cs35l56_l3, | |
| 473 | 473 | .drv_name = "sof_sdw", |
| 474 | − .sof_tplg_filename = "sof-arl-cs42l43-l2.tplg", | |
| 474 | + .sof_tplg_filename = "sof-arl-cs42l43-l2-cs35l56-l3.tplg", | |
| 475 | 475 | .get_function_tplg_files = sof_sdw_get_tplg_files, |
| 476 | 476 | }, |
| 477 | 477 | { |
| 478 | − .link_mask = BIT(2) | BIT(3), | |
| 479 | − .links = arl_cs42l43_l2_cs35l56_l3, | |
| 478 | + .link_mask = BIT(2), | |
| 479 | + .links = arl_cs42l43_l2, | |
| 480 | 480 | .drv_name = "sof_sdw", |
| 481 | − .sof_tplg_filename = "sof-arl-cs42l43-l2-cs35l56-l3.tplg", | |
| 481 | + .sof_tplg_filename = "sof-arl-cs42l43-l2.tplg", | |
| 482 | 482 | .get_function_tplg_files = sof_sdw_get_tplg_files, |
| 483 | 483 | }, |
| 484 | 484 | { |
| @@ -310,16 +310,14 @@ static bool focusrite_valid_sample_rate(struct snd_usb_audio *chip, · 309 unmodified lines | ||
| 310 | 310 | struct audioformat *fp, |
| 311 | 311 | unsigned int rate) |
| 312 | 312 | { |
| 313 | − struct usb_interface *iface; | |
| 314 | 313 | struct usb_host_interface *alts; |
| 315 | 314 | unsigned char *fmt; |
| 316 | 315 | unsigned int max_rate; |
| 317 | 316 | |
| 318 | − iface = usb_ifnum_to_if(chip->dev, fp->iface); | |
| 319 | − if (!iface) | |
| 317 | + alts = snd_usb_get_host_interface(chip, fp->iface, fp->altsetting); | |
| 318 | + if (!alts) | |
| 320 | 319 | return true; |
| 321 | 320 | |
| 322 | − alts = &iface->altsetting[fp->altset_idx]; | |
| 323 | 321 | fmt = snd_usb_find_csint_desc(alts->extra, alts->extralen, |
| 324 | 322 | NULL, UAC_FORMAT_TYPE); |
| 325 | 323 | if (!fmt) |
| @@ -328,20 +326,20 @@ static bool focusrite_valid_sample_rate(struct snd_usb_audio *chip, · 2 unmodified lines | ||
| 328 | 326 | if (fmt[0] == 10) { /* bLength */ |
| 329 | 327 | max_rate = combine_quad(&fmt[6]); |
| 330 | 328 | |
| 331 | − /* Validate max rate */ | |
| 332 | − if (max_rate != 48000 && | |
| 333 | − max_rate != 96000 && | |
| 334 | − max_rate != 192000 && | |
| 335 | − max_rate != 384000) { | |
| 336 | − | |
| 329 | + switch (max_rate) { | |
| 330 | + case 48000: | |
| 331 | + return (rate == 44100 || rate == 48000); | |
| 332 | + case 96000: | |
| 333 | + return (rate == 88200 || rate == 96000); | |
| 334 | + case 192000: | |
| 335 | + return (rate == 176400 || rate == 192000); | |
| 336 | + default: | |
| 337 | 337 | usb_audio_info(chip, |
| 338 | 338 | "%u:%d : unexpected max rate: %u\n", |
| 339 | 339 | fp->iface, fp->altsetting, max_rate); |
| 340 | 340 | |
| 341 | 341 | return true; |
| 342 | 342 | } |
| 343 | − | |
| 344 | − return rate <= max_rate; | |
| 345 | 343 | } |
| 346 | 344 | |
| 347 | 345 | return true; |
| @@ -628,6 +628,7 @@ · 627 unmodified lines | ||
| 628 | 628 | #define MSR_AMD64_OSVW_STATUS 0xc0010141 |
| 629 | 629 | #define MSR_AMD_PPIN_CTL 0xc00102f0 |
| 630 | 630 | #define MSR_AMD_PPIN 0xc00102f1 |
| 631 | +#define MSR_AMD64_CPUID_FN_7 0xc0011002 | |
| 631 | 632 | #define MSR_AMD64_CPUID_FN_1 0xc0011004 |
| 632 | 633 | #define MSR_AMD64_LS_CFG 0xc0011020 |
| 633 | 634 | #define MSR_AMD64_DC_CFG 0xc0011022 |
| @@ -18,6 +18,7 @@ static inline const char *kallsyms_lookup(unsigned long addr, · 17 unmodified lines | ||
| 18 | 18 | return NULL; |
| 19 | 19 | } |
| 20 | 20 | |
| 21 | +#ifdef HAVE_BACKTRACE_SUPPORT | |
| 21 | 22 | #include <execinfo.h> |
| 22 | 23 | #include <stdlib.h> |
| 23 | 24 | static inline void print_ip_sym(const char *loglvl, unsigned long ip) |
| @@ -30,5 +31,8 @@ static inline void print_ip_sym(const char *loglvl, unsigned long ip) · 6 unmodified lines | ||
| 30 | 31 | |
| 31 | 32 | free(name); |
| 32 | 33 | } |
| 34 | +#else | |
| 35 | +static inline void print_ip_sym(const char *loglvl, unsigned long ip) {} | |
| 36 | +#endif | |
| 33 | 37 | |
| 34 | 38 | #endif |
| @@ -4,9 +4,9 @@ · 3 unmodified lines | ||
| 4 | 4 | #ifndef _UAPI_LINUX_BITS_H |
| 5 | 5 | #define _UAPI_LINUX_BITS_H |
| 6 | 6 | |
| 7 | −#define __GENMASK(h, l) (((~_UL(0)) << (l)) & (~_UL(0) >> (BITS_PER_LONG - 1 - (h)))) | |
| 7 | +#define __GENMASK(h, l) (((~_UL(0)) << (l)) & (~_UL(0) >> (__BITS_PER_LONG - 1 - (h)))) | |
| 8 | 8 | |
| 9 | −#define __GENMASK_ULL(h, l) (((~_ULL(0)) << (l)) & (~_ULL(0) >> (BITS_PER_LONG_LONG - 1 - (h)))) | |
| 9 | +#define __GENMASK_ULL(h, l) (((~_ULL(0)) << (l)) & (~_ULL(0) >> (__BITS_PER_LONG_LONG - 1 - (h)))) | |
| 10 | 10 | |
| 11 | 11 | #define __GENMASK_U128(h, l) \ |
| 12 | 12 | ((_BIT128((h)) << 1) - (_BIT128(l))) |
| @@ -74,6 +74,7 @@ int main(int argc, char *argv[]) · 73 unmodified lines | ||
| 74 | 74 | int testcase; |
| 75 | 75 | char test[80]; |
| 76 | 76 | |
| 77 | + TEST_REQUIRE(this_cpu_has(X86_FEATURE_MWAIT)); | |
| 77 | 78 | TEST_REQUIRE(kvm_has_cap(KVM_CAP_DISABLE_QUIRKS2)); |
| 78 | 79 | |
| 79 | 80 | ksft_print_header(); |
| @@ -24,7 +24,10 @@ setup_basenet() · 23 unmodified lines | ||
| 24 | 24 | ip -netns "${NS0}" address add dev lo 2001:db8::10/64 nodad |
| 25 | 25 | } |
| 26 | 26 | |
| 27 | −# Check if network device has an IPv6 link-local address assigned. | |
| 27 | +# Check the IPv6 configuration of a network device. | |
| 28 | +# | |
| 29 | +# We currently check the generation of the link-local IPv6 address and the | |
| 30 | +# creation of the ff00::/8 multicast route. | |
| 28 | 31 | # |
| 29 | 32 | # Parameters: |
| 30 | 33 | # |
| @@ -35,7 +38,7 @@ setup_basenet() · 4 unmodified lines | ||
| 35 | 38 | # a link-local address) |
| 36 | 39 | # * $4: The user visible name for the scenario being tested |
| 37 | 40 | # |
| 38 | −check_ipv6_ll_addr() | |
| 41 | +check_ipv6_device_config() | |
| 39 | 42 | { |
| 40 | 43 | local DEV="$1" |
| 41 | 44 | local EXTRA_MATCH="$2" |
| @@ -45,7 +48,11 @@ check_ipv6_ll_addr() · 3 unmodified lines | ||
| 45 | 48 | RET=0 |
| 46 | 49 | set +e |
| 47 | 50 | ip -netns "${NS0}" -6 address show dev "${DEV}" scope link | grep "fe80::" | grep -q "${EXTRA_MATCH}" |
| 48 | − check_err_fail "${XRET}" $? "" | |
| 51 | + check_err_fail "${XRET}" $? "IPv6 link-local address generation" | |
| 52 | + | |
| 53 | + ip -netns "${NS0}" -6 route show table local type multicast ff00::/8 proto kernel | grep -q "${DEV}" | |
| 54 | + check_err_fail 0 $? "IPv6 multicast route creation" | |
| 55 | + | |
| 49 | 56 | log_test "${MSG}" |
| 50 | 57 | set -e |
| 51 | 58 | } |
| @@ -102,20 +109,20 @@ test_gre_device() · 50 unmodified lines | ||
| 102 | 109 | ;; |
| 103 | 110 | esac |
| 104 | 111 | |
| 105 | − # Check that IPv6 link-local address is generated when device goes up | |
| 112 | + # Check the IPv6 device configuration when it goes up | |
| 106 | 113 | ip netns exec "${NS0}" sysctl -qw net.ipv6.conf.gretest.addr_gen_mode="${ADDR_GEN_MODE}" |
| 107 | 114 | ip -netns "${NS0}" link set dev gretest up |
| 108 | − check_ipv6_ll_addr gretest "${MATCH_REGEXP}" "${XRET}" "config: ${MSG}" | |
| 115 | + check_ipv6_device_config gretest "${MATCH_REGEXP}" "${XRET}" "config: ${MSG}" | |
| 109 | 116 | |
| 110 | 117 | # Now disable link-local address generation |
| 111 | 118 | ip -netns "${NS0}" link set dev gretest down |
| 112 | 119 | ip netns exec "${NS0}" sysctl -qw net.ipv6.conf.gretest.addr_gen_mode=1 |
| 113 | 120 | ip -netns "${NS0}" link set dev gretest up |
| 114 | 121 | |
| 115 | − # Check that link-local address generation works when re-enabled while | |
| 116 | − # the device is already up | |
| 122 | + # Check the IPv6 device configuration when link-local address | |
| 123 | + # generation is re-enabled while the device is already up | |
| 117 | 124 | ip netns exec "${NS0}" sysctl -qw net.ipv6.conf.gretest.addr_gen_mode="${ADDR_GEN_MODE}" |
| 118 | − check_ipv6_ll_addr gretest "${MATCH_REGEXP}" "${XRET}" "update: ${MSG}" | |
| 125 | + check_ipv6_device_config gretest "${MATCH_REGEXP}" "${XRET}" "update: ${MSG}" | |
| 119 | 126 | |
| 120 | 127 | ip -netns "${NS0}" link del dev gretest |
| 121 | 128 | } |
| @@ -126,7 +133,7 @@ test_gre4() · 4 unmodified lines | ||
| 126 | 133 | local MODE |
| 127 | 134 | |
| 128 | 135 | for GRE_TYPE in "gre" "gretap"; do |
| 129 | − printf "\n####\nTesting IPv6 link-local address generation on ${GRE_TYPE} devices\n####\n\n" | |
| 136 | + printf "\n####\nTesting IPv6 configuration of ${GRE_TYPE} devices\n####\n\n" | |
| 130 | 137 | |
| 131 | 138 | for MODE in "eui64" "none" "stable-privacy" "random"; do |
| 132 | 139 | test_gre_device "${GRE_TYPE}" 192.0.2.10 192.0.2.11 "${MODE}" |
| @@ -142,7 +149,7 @@ test_gre6() · 9 unmodified lines | ||
| 142 | 149 | local MODE |
| 143 | 150 | |
| 144 | 151 | for GRE_TYPE in "ip6gre" "ip6gretap"; do |
| 145 | − printf "\n####\nTesting IPv6 link-local address generation on ${GRE_TYPE} devices\n####\n\n" | |
| 152 | + printf "\n####\nTesting IPv6 configuration of ${GRE_TYPE} devices\n####\n\n" | |
| 146 | 153 | |
| 147 | 154 | for MODE in "eui64" "none" "stable-privacy" "random"; do |
| 148 | 155 | test_gre_device "${GRE_TYPE}" 2001:db8::10 2001:db8::11 "${MODE}" |
| @@ -312,7 +312,7 @@ log_test_result() · 311 unmodified lines | ||
| 312 | 312 | local test_name=$1; shift |
| 313 | 313 | local opt_str=$1; shift |
| 314 | 314 | local result=$1; shift |
| 315 | − local retmsg=$1; shift | |
| 315 | + local retmsg=$1 | |
| 316 | 316 | |
| 317 | 317 | printf "TEST: %-60s [%s]\n" "$test_name $opt_str" "$result" |
| 318 | 318 | if [[ $retmsg ]]; then |
| @@ -0,0 +1,53 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0 | |
| 2 | + | |
| 3 | +--mss=1000 | |
| 4 | + | |
| 5 | +`./defaults.sh | |
| 6 | +sysctl -q net.ipv4.tcp_rmem="4096 131072 $((32*1024*1024))"` | |
| 7 | + | |
| 8 | +// Test that a not-yet-accepted socket does not change | |
| 9 | +// its initial sk_rcvbuf (tcp_rmem[1]) when receiving ooo packets. | |
| 10 | + | |
| 11 | + +0 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3 | |
| 12 | + +0 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0 | |
| 13 | + +0 bind(3, ..., ...) = 0 | |
| 14 | + +0 listen(3, 1) = 0 | |
| 15 | + | |
| 16 | + +0 < S 0:0(0) win 65535 <mss 1000,nop,nop,sackOK,nop,wscale 7> | |
| 17 | + +0 > S. 0:0(0) ack 1 <mss 1460,nop,nop,sackOK,nop,wscale 10> | |
| 18 | + +.1 < . 1:1(0) ack 1 win 257 | |
| 19 | + +0 < . 2001:41001(39000) ack 1 win 257 | |
| 20 | + +0 > . 1:1(0) ack 1 <nop,nop,sack 2001:41001> | |
| 21 | + +0 < . 41001:101001(60000) ack 1 win 257 | |
| 22 | + +0 > . 1:1(0) ack 1 <nop,nop,sack 2001:101001> | |
| 23 | + +0 < . 1:1001(1000) ack 1 win 257 | |
| 24 | + +0 > . 1:1(0) ack 1001 <nop,nop,sack 2001:101001> | |
| 25 | + +0 < . 1001:2001(1000) ack 1 win 257 | |
| 26 | + +0 > . 1:1(0) ack 101001 | |
| 27 | + | |
| 28 | + +0 accept(3, ..., ...) = 4 | |
| 29 | + | |
| 30 | + +0 %{ assert SK_MEMINFO_RCVBUF == 131072, SK_MEMINFO_RCVBUF }% | |
| 31 | + | |
| 32 | + +0 close(4) = 0 | |
| 33 | + +0 close(3) = 0 | |
| 34 | + | |
| 35 | +// Test that ooo packets for accepted sockets do increase sk_rcvbuf | |
| 36 | + +0 socket(..., SOCK_STREAM, IPPROTO_TCP) = 3 | |
| 37 | + +0 setsockopt(3, SOL_SOCKET, SO_REUSEADDR, [1], 4) = 0 | |
| 38 | + +0 bind(3, ..., ...) = 0 | |
| 39 | + +0 listen(3, 1) = 0 | |
| 40 | + | |
| 41 | + +0 < S 0:0(0) win 65535 <mss 1000,nop,nop,sackOK,nop,wscale 7> | |
| 42 | + +0 > S. 0:0(0) ack 1 <mss 1460,nop,nop,sackOK,nop,wscale 10> | |
| 43 | + +.1 < . 1:1(0) ack 1 win 257 | |
| 44 | + | |
| 45 | + +0 accept(3, ..., ...) = 4 | |
| 46 | + | |
| 47 | + +0 < . 2001:41001(39000) ack 1 win 257 | |
| 48 | + +0 > . 1:1(0) ack 1 <nop,nop,sack 2001:41001> | |
| 49 | + +0 < . 41001:101001(60000) ack 1 win 257 | |
| 50 | + +0 > . 1:1(0) ack 1 <nop,nop,sack 2001:101001> | |
| 51 | + | |
| 52 | + +0 %{ assert SK_MEMINFO_RCVBUF > 131072, SK_MEMINFO_RCVBUF }% | |
| 53 | + | |
| @@ -635,5 +635,42 @@ · 634 unmodified lines | ||
| 635 | 635 | "$TC qdisc del dev $DUMMY handle 1:0 root", |
| 636 | 636 | "$IP addr del 10.10.10.10/24 dev $DUMMY || true" |
| 637 | 637 | ] |
| 638 | + }, | |
| 639 | + { | |
| 640 | + "id": "d74b", | |
| 641 | + "name": "Test use-after-free with DRR/NETEM/BLACKHOLE chain", | |
| 642 | + "category": [ | |
| 643 | + "qdisc", | |
| 644 | + "hfsc", | |
| 645 | + "drr", | |
| 646 | + "netem", | |
| 647 | + "blackhole" | |
| 648 | + ], | |
| 649 | + "plugins": { | |
| 650 | + "requires": [ | |
| 651 | + "nsPlugin", | |
| 652 | + "scapyPlugin" | |
| 653 | + ] | |
| 654 | + }, | |
| 655 | + "setup": [ | |
| 656 | + "$IP link set dev $DUMMY up || true", | |
| 657 | + "$IP addr add 10.10.11.10/24 dev $DUMMY || true", | |
| 658 | + "$TC qdisc add dev $DUMMY root handle 1: drr", | |
| 659 | + "$TC filter add dev $DUMMY parent 1: basic classid 1:1", | |
| 660 | + "$TC class add dev $DUMMY parent 1: classid 1:1 drr", | |
| 661 | + "$TC qdisc add dev $DUMMY parent 1:1 handle 2: hfsc def 1", | |
| 662 | + "$TC class add dev $DUMMY parent 2: classid 2:1 hfsc rt m1 8 d 1 m2 0", | |
| 663 | + "$TC qdisc add dev $DUMMY parent 2:1 handle 3: netem", | |
| 664 | + "$TC qdisc add dev $DUMMY parent 3:1 handle 4: blackhole", | |
| 665 | + "ping -c1 -W0.01 -I $DUMMY 10.10.11.11 || true", | |
| 666 | + "$TC class del dev $DUMMY classid 1:1" | |
| 667 | + ], | |
| 668 | + "cmdUnderTest": "ping -c1 -W0.01 -I $DUMMY 10.10.11.11", | |
| 669 | + "expExitCode": "1", | |
| 670 | + "verifyCmd": "$TC -j class ls dev $DUMMY classid 1:1", | |
| 671 | + "matchJSON": [], | |
| 672 | + "teardown": [ | |
| 673 | + "$TC qdisc del dev $DUMMY root handle 1: drr" | |
| 674 | + ] | |
| 638 | 675 | } |
| 639 | 676 | ] |
| @@ -2572,6 +2572,8 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, · 2571 unmodified lines | ||
| 2572 | 2572 | r = xa_reserve(&kvm->mem_attr_array, i, GFP_KERNEL_ACCOUNT); |
| 2573 | 2573 | if (r) |
| 2574 | 2574 | goto out_unlock; |
| 2575 | + | |
| 2576 | + cond_resched(); | |
| 2575 | 2577 | } |
| 2576 | 2578 | |
| 2577 | 2579 | kvm_handle_gfn_range(kvm, &pre_set_range); |
| @@ -2580,6 +2582,7 @@ static int kvm_vm_set_mem_attributes(struct kvm *kvm, gfn_t start, gfn_t end, · 2 unmodified lines | ||
| 2580 | 2582 | r = xa_err(xa_store(&kvm->mem_attr_array, i, entry, |
| 2581 | 2583 | GFP_KERNEL_ACCOUNT)); |
| 2582 | 2584 | KVM_BUG_ON(r, kvm); |
| 2585 | + cond_resched(); | |
| 2583 | 2586 | } |
| 2584 | 2587 | |
| 2585 | 2588 | kvm_handle_gfn_range(kvm, &post_set_range); |