This is done primarily to get a docs build fix merged via another tree so that "make htmldocs" stops failing.
Unified · Split
| @@ -166,6 +166,7 @@ Daniel Borkmann <daniel@iogearbox.net> <dborkman@redhat.com> · 165 unmodified lines | ||
| 166 | 166 | Daniel Borkmann <daniel@iogearbox.net> <dxchgb@gmail.com> |
| 167 | 167 | David Brownell <david-b@pacbell.net> |
| 168 | 168 | David Collins <quic_collinsd@quicinc.com> <collinsd@codeaurora.org> |
| 169 | +David Heidelberg <david@ixit.cz> <d.okias@gmail.com> | |
| 169 | 170 | David Rheinsberg <david@readahead.eu> <dh.herrmann@gmail.com> |
| 170 | 171 | David Rheinsberg <david@readahead.eu> <dh.herrmann@googlemail.com> |
| 171 | 172 | David Rheinsberg <david@readahead.eu> <david.rheinsberg@gmail.com> |
| @@ -353,6 +354,8 @@ Kenneth Westfield <quic_kwestfie@quicinc.com> <kwestfie@codeaurora.org> · 181 unmodified lines | ||
| 353 | 354 | Kiran Gunda <quic_kgunda@quicinc.com> <kgunda@codeaurora.org> |
| 354 | 355 | Kirill Tkhai <tkhai@ya.ru> <ktkhai@virtuozzo.com> |
| 355 | 356 | Kishon Vijay Abraham I <kishon@kernel.org> <kishon@ti.com> |
| 357 | +Konrad Dybcio <konradybcio@kernel.org> <konrad.dybcio@linaro.org> | |
| 358 | +Konrad Dybcio <konradybcio@kernel.org> <konrad.dybcio@somainline.org> | |
| 356 | 359 | Konstantin Khlebnikov <koct9i@gmail.com> <khlebnikov@yandex-team.ru> |
| 357 | 360 | Konstantin Khlebnikov <koct9i@gmail.com> <k.khlebnikov@samsung.com> |
| 358 | 361 | Koushik <raghavendra.koushik@neterion.com> |
| @@ -613,6 +616,7 @@ Simon Kelley <simon@thekelleys.org.uk> · 254 unmodified lines | ||
| 613 | 616 | Sricharan Ramabadhran <quic_srichara@quicinc.com> <sricharan@codeaurora.org> |
| 614 | 617 | Srinivas Ramana <quic_sramana@quicinc.com> <sramana@codeaurora.org> |
| 615 | 618 | Sriram R <quic_srirrama@quicinc.com> <srirrama@codeaurora.org> |
| 619 | +Sriram Yagnaraman <sriram.yagnaraman@ericsson.com> <sriram.yagnaraman@est.tech> | |
| 616 | 620 | Stanislav Fomichev <sdf@fomichev.me> <sdf@google.com> |
| 617 | 621 | Stefan Wahren <wahrenst@gmx.net> <stefan.wahren@i2se.com> |
| 618 | 622 | Stéphane Witzmann <stephane.witzmann@ubpmes.univ-bpclermont.fr> |
| @@ -32,9 +32,9 @@ Description: (RW) The front button on the Turris Omnia router can be · 31 unmodified lines | ||
| 32 | 32 | interrupt. |
| 33 | 33 | |
| 34 | 34 | This file switches between these two modes: |
| 35 | − - "mcu" makes the button press event be handled by the MCU to | |
| 36 | − change the LEDs panel intensity. | |
| 37 | − - "cpu" makes the button press event be handled by the CPU. | |
| 35 | + - ``mcu`` makes the button press event be handled by the MCU to | |
| 36 | + change the LEDs panel intensity. | |
| 37 | + - ``cpu`` makes the button press event be handled by the CPU. | |
| 38 | 38 | |
| 39 | 39 | Format: %s. |
| 40 | 40 | |
| @@ -562,7 +562,8 @@ Description: Control Symmetric Multi Threading (SMT) · 561 unmodified lines | ||
| 562 | 562 | ================ ========================================= |
| 563 | 563 | |
| 564 | 564 | If control status is "forceoff" or "notsupported" writes |
| 565 | − are rejected. | |
| 565 | + are rejected. Note that enabling SMT on PowerPC skips | |
| 566 | + offline cores. | |
| 566 | 567 | |
| 567 | 568 | What: /sys/devices/system/cpu/cpuX/power/energy_perf_bias |
| 568 | 569 | Date: March 2019 |
| @@ -742,7 +742,7 @@ SecurityFlags Flags which control security negotiation and · 741 unmodified lines | ||
| 742 | 742 | may use NTLMSSP 0x00080 |
| 743 | 743 | must use NTLMSSP 0x80080 |
| 744 | 744 | seal (packet encryption) 0x00040 |
| 745 | − must seal (not implemented yet) 0x40040 | |
| 745 | + must seal 0x40040 | |
| 746 | 746 | |
| 747 | 747 | cifsFYI If set to non-zero value, additional debug information |
| 748 | 748 | will be logged to the system error log. This field |
| @@ -4798,11 +4798,9 @@ · 4797 unmodified lines | ||
| 4798 | 4798 | |
| 4799 | 4799 | profile= [KNL] Enable kernel profiling via /proc/profile |
| 4800 | 4800 | Format: [<profiletype>,]<number> |
| 4801 | − Param: <profiletype>: "schedule", "sleep", or "kvm" | |
| 4801 | + Param: <profiletype>: "schedule" or "kvm" | |
| 4802 | 4802 | [defaults to kernel profiling] |
| 4803 | 4803 | Param: "schedule" - profile schedule points. |
| 4804 | − Param: "sleep" - profile D-state sleeping (millisecs). | |
| 4805 | − Requires CONFIG_SCHEDSTATS | |
| 4806 | 4804 | Param: "kvm" - profile VM exits. |
| 4807 | 4805 | Param: <number> - step/bucket size as a power of 2 for |
| 4808 | 4806 | statistical time based profiling. |
| @@ -122,10 +122,18 @@ stable kernels. · 121 unmodified lines | ||
| 122 | 122 | +----------------+-----------------+-----------------+-----------------------------+ |
| 123 | 123 | | ARM | Cortex-A76 | #1490853 | N/A | |
| 124 | 124 | +----------------+-----------------+-----------------+-----------------------------+ |
| 125 | +| ARM | Cortex-A76 | #3324349 | ARM64_ERRATUM_3194386 | | |
| 126 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 125 | 127 | | ARM | Cortex-A77 | #1491015 | N/A | |
| 126 | 128 | +----------------+-----------------+-----------------+-----------------------------+ |
| 127 | 129 | | ARM | Cortex-A77 | #1508412 | ARM64_ERRATUM_1508412 | |
| 128 | 130 | +----------------+-----------------+-----------------+-----------------------------+ |
| 131 | +| ARM | Cortex-A77 | #3324348 | ARM64_ERRATUM_3194386 | | |
| 132 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 133 | +| ARM | Cortex-A78 | #3324344 | ARM64_ERRATUM_3194386 | | |
| 134 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 135 | +| ARM | Cortex-A78C | #3324346,3324347| ARM64_ERRATUM_3194386 | | |
| 136 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 129 | 137 | | ARM | Cortex-A710 | #2119858 | ARM64_ERRATUM_2119858 | |
| 130 | 138 | +----------------+-----------------+-----------------+-----------------------------+ |
| 131 | 139 | | ARM | Cortex-A710 | #2054223 | ARM64_ERRATUM_2054223 | |
| @@ -138,8 +146,14 @@ stable kernels. · 6 unmodified lines | ||
| 138 | 146 | +----------------+-----------------+-----------------+-----------------------------+ |
| 139 | 147 | | ARM | Cortex-A720 | #3456091 | ARM64_ERRATUM_3194386 | |
| 140 | 148 | +----------------+-----------------+-----------------+-----------------------------+ |
| 149 | +| ARM | Cortex-A725 | #3456106 | ARM64_ERRATUM_3194386 | | |
| 150 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 141 | 151 | | ARM | Cortex-X1 | #1502854 | N/A | |
| 142 | 152 | +----------------+-----------------+-----------------+-----------------------------+ |
| 153 | +| ARM | Cortex-X1 | #3324344 | ARM64_ERRATUM_3194386 | | |
| 154 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 155 | +| ARM | Cortex-X1C | #3324346 | ARM64_ERRATUM_3194386 | | |
| 156 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 143 | 157 | | ARM | Cortex-X2 | #2119858 | ARM64_ERRATUM_2119858 | |
| 144 | 158 | +----------------+-----------------+-----------------+-----------------------------+ |
| 145 | 159 | | ARM | Cortex-X2 | #2224489 | ARM64_ERRATUM_2224489 | |
| @@ -160,6 +174,8 @@ stable kernels. · 14 unmodified lines | ||
| 160 | 174 | +----------------+-----------------+-----------------+-----------------------------+ |
| 161 | 175 | | ARM | Neoverse-N1 | #1542419 | ARM64_ERRATUM_1542419 | |
| 162 | 176 | +----------------+-----------------+-----------------+-----------------------------+ |
| 177 | +| ARM | Neoverse-N1 | #3324349 | ARM64_ERRATUM_3194386 | | |
| 178 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 163 | 179 | | ARM | Neoverse-N2 | #2139208 | ARM64_ERRATUM_2139208 | |
| 164 | 180 | +----------------+-----------------+-----------------+-----------------------------+ |
| 165 | 181 | | ARM | Neoverse-N2 | #2067961 | ARM64_ERRATUM_2067961 | |
| @@ -170,6 +186,8 @@ stable kernels. · 4 unmodified lines | ||
| 170 | 186 | +----------------+-----------------+-----------------+-----------------------------+ |
| 171 | 187 | | ARM | Neoverse-V1 | #1619801 | N/A | |
| 172 | 188 | +----------------+-----------------+-----------------+-----------------------------+ |
| 189 | +| ARM | Neoverse-V1 | #3324341 | ARM64_ERRATUM_3194386 | | |
| 190 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 173 | 191 | | ARM | Neoverse-V2 | #3324336 | ARM64_ERRATUM_3194386 | |
| 174 | 192 | +----------------+-----------------+-----------------+-----------------------------+ |
| 175 | 193 | | ARM | Neoverse-V3 | #3312417 | ARM64_ERRATUM_3194386 | |
| @@ -239,25 +239,33 @@ The following keys are defined: · 238 unmodified lines | ||
| 239 | 239 | ratified in commit 98918c844281 ("Merge pull request #1217 from |
| 240 | 240 | riscv/zawrs") of riscv-isa-manual. |
| 241 | 241 | |
| 242 | −* :c:macro:`RISCV_HWPROBE_KEY_CPUPERF_0`: A bitmask that contains performance | |
| 243 | − information about the selected set of processors. | |
| 242 | +* :c:macro:`RISCV_HWPROBE_KEY_CPUPERF_0`: Deprecated. Returns similar values to | |
| 243 | + :c:macro:`RISCV_HWPROBE_KEY_MISALIGNED_SCALAR_PERF`, but the key was | |
| 244 | + mistakenly classified as a bitmask rather than a value. | |
| 244 | 245 | |
| 245 | − * :c:macro:`RISCV_HWPROBE_MISALIGNED_UNKNOWN`: The performance of misaligned | |
| 246 | − accesses is unknown. | |
| 246 | +* :c:macro:`RISCV_HWPROBE_KEY_MISALIGNED_SCALAR_PERF`: An enum value describing | |
| 247 | + the performance of misaligned scalar native word accesses on the selected set | |
| 248 | + of processors. | |
| 247 | 249 | |
| 248 | − * :c:macro:`RISCV_HWPROBE_MISALIGNED_EMULATED`: Misaligned accesses are | |
| 249 | − emulated via software, either in or below the kernel. These accesses are | |
| 250 | − always extremely slow. | |
| 250 | + * :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN`: The performance of | |
| 251 | + misaligned scalar accesses is unknown. | |
| 251 | 252 | |
| 252 | − * :c:macro:`RISCV_HWPROBE_MISALIGNED_SLOW`: Misaligned accesses are slower | |
| 253 | − than equivalent byte accesses. Misaligned accesses may be supported | |
| 254 | − directly in hardware, or trapped and emulated by software. | |
| 253 | + * :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED`: Misaligned scalar | |
| 254 | + accesses are emulated via software, either in or below the kernel. These | |
| 255 | + accesses are always extremely slow. | |
| 255 | 256 | |
| 256 | − * :c:macro:`RISCV_HWPROBE_MISALIGNED_FAST`: Misaligned accesses are faster | |
| 257 | − than equivalent byte accesses. | |
| 257 | + * :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_SLOW`: Misaligned scalar native | |
| 258 | + word sized accesses are slower than the equivalent quantity of byte | |
| 259 | + accesses. Misaligned accesses may be supported directly in hardware, or | |
| 260 | + trapped and emulated by software. | |
| 258 | 261 | |
| 259 | − * :c:macro:`RISCV_HWPROBE_MISALIGNED_UNSUPPORTED`: Misaligned accesses are | |
| 260 | − not supported at all and will generate a misaligned address fault. | |
| 262 | + * :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_FAST`: Misaligned scalar native | |
| 263 | + word sized accesses are faster than the equivalent quantity of byte | |
| 264 | + accesses. | |
| 265 | + | |
| 266 | + * :c:macro:`RISCV_HWPROBE_MISALIGNED_SCALAR_UNSUPPORTED`: Misaligned scalar | |
| 267 | + accesses are not supported at all and will generate a misaligned address | |
| 268 | + fault. | |
| 261 | 269 | |
| 262 | 270 | * :c:macro:`RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE`: An unsigned int which |
| 263 | 271 | represents the size of the Zicboz block in bytes. |
| @@ -260,7 +260,7 @@ Some users depend on strict execution ordering where only one work item · 259 unmodified lines | ||
| 260 | 260 | is in flight at any given time and the work items are processed in |
| 261 | 261 | queueing order. While the combination of ``@max_active`` of 1 and |
| 262 | 262 | ``WQ_UNBOUND`` used to achieve this behavior, this is no longer the |
| 263 | −case. Use ``alloc_ordered_queue()`` instead. | |
| 263 | +case. Use alloc_ordered_workqueue() instead. | |
| 264 | 264 | |
| 265 | 265 | |
| 266 | 266 | Example Execution Scenarios |
| @@ -35,6 +35,9 @@ properties: · 34 unmodified lines | ||
| 35 | 35 | ports-implemented: |
| 36 | 36 | const: 1 |
| 37 | 37 | |
| 38 | + power-domains: | |
| 39 | + maxItems: 1 | |
| 40 | + | |
| 38 | 41 | sata-port@0: |
| 39 | 42 | $ref: /schemas/ata/snps,dwc-ahci-common.yaml#/$defs/dwc-ahci-port |
| 40 | 43 | |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Display Clock & Reset Controller on SM6350 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@somainline.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm display clock control module provides the clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Global Clock & Reset Controller on MSM8994 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@somainline.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm global clock control module provides the clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Global Clock & Reset Controller on SM6125 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@somainline.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm global clock control module provides the clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Global Clock & Reset Controller on SM6350 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@somainline.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm global clock control module provides the clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Graphics Clock & Reset Controller on SM6115 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm graphics clock control module provides clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Graphics Clock & Reset Controller on SM6125 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm graphics clock control module provides clocks and power domains on |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Camera Clock & Reset Controller on SM6350 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm camera clock control module provides the clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Display Clock & Reset Controller on SM6375 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm display clock control module provides the clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Global Clock & Reset Controller on SM6375 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@somainline.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm global clock control module provides the clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Graphics Clock & Reset Controller on SM6375 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm graphics clock control module provides clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm SM8350 Video Clock & Reset Controller |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm video clock control module provides the clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Graphics Clock & Reset Controller on SM8450 |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm graphics clock control module provides the clocks, resets and power |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm SM6375 Display MDSS |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: |
| 13 | 13 | SM6375 MSM Mobile Display Subsystem (MDSS), which encapsulates sub-blocks |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: ASUS Z00T TM5P5 NT35596 5.5" 1080×1920 LCD Panel |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konradybcio@gmail.com> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: |+ |
| 13 | 13 | This panel seems to only be found in the Asus Z00T |
| @@ -17,9 +17,12 @@ properties: · 16 unmodified lines | ||
| 17 | 17 | oneOf: |
| 18 | 18 | # Samsung 13.3" FHD (1920x1080 pixels) eDP AMOLED panel |
| 19 | 19 | - const: samsung,atna33xc20 |
| 20 | − # Samsung 14.5" WQXGA+ (2880x1800 pixels) eDP AMOLED panel | |
| 21 | 20 | - items: |
| 22 | − - const: samsung,atna45af01 | |
| 21 | + - enum: | |
| 22 | + # Samsung 14.5" WQXGA+ (2880x1800 pixels) eDP AMOLED panel | |
| 23 | + - samsung,atna45af01 | |
| 24 | + # Samsung 14.5" 3K (2944x1840 pixels) eDP AMOLED panel | |
| 25 | + - samsung,atna45dc02 | |
| 23 | 26 | - const: samsung,atna33xc20 |
| 24 | 27 | |
| 25 | 28 | enable-gpios: true |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Sony TD4353 JDI 5 / 5.7" 2160x1080 MIPI-DSI Panel |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@somainline.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | The Sony TD4353 JDI is a 5 (XZ2c) / 5.7 (XZ2) inch 2160x1080 |
| @@ -28,6 +28,7 @@ properties: · 27 unmodified lines | ||
| 28 | 28 | - anvo,anv32e61w |
| 29 | 29 | - atmel,at25256B |
| 30 | 30 | - fujitsu,mb85rs1mt |
| 31 | + - fujitsu,mb85rs256 | |
| 31 | 32 | - fujitsu,mb85rs64 |
| 32 | 33 | - microchip,at25160bn |
| 33 | 34 | - microchip,25lc040 |
| @@ -42,6 +42,7 @@ properties: · 41 unmodified lines | ||
| 42 | 42 | - focaltech,ft5426 |
| 43 | 43 | - focaltech,ft5452 |
| 44 | 44 | - focaltech,ft6236 |
| 45 | + - focaltech,ft8201 | |
| 45 | 46 | - focaltech,ft8719 |
| 46 | 47 | |
| 47 | 48 | reg: |
| @@ -8,7 +8,7 @@ title: Qualcomm RPMh Network-On-Chip Interconnect on SC7280 · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | 10 | - Bjorn Andersson <andersson@kernel.org> |
| 11 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 11 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 12 | 12 | |
| 13 | 13 | description: | |
| 14 | 14 | RPMh interconnect providers support system bandwidth requirements through |
| @@ -8,7 +8,7 @@ title: Qualcomm RPMh Network-On-Chip Interconnect on SC8280XP · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | 10 | - Bjorn Andersson <andersson@kernel.org> |
| 11 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 11 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 12 | 12 | |
| 13 | 13 | description: | |
| 14 | 14 | RPMh interconnect providers support system bandwidth requirements through |
| @@ -8,7 +8,7 @@ title: Qualcomm RPMh Network-On-Chip Interconnect on SM8450 · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | 10 | - Bjorn Andersson <andersson@kernel.org> |
| 11 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 11 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 12 | 12 | |
| 13 | 13 | description: | |
| 14 | 14 | RPMh interconnect providers support system bandwidth requirements through |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Technologies legacy IOMMU implementations |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | Qualcomm "B" family devices which are not compatible with arm-smmu have |
| @@ -38,6 +38,10 @@ properties: · 37 unmodified lines | ||
| 38 | 38 | |
| 39 | 39 | managed: true |
| 40 | 40 | |
| 41 | + phys: | |
| 42 | + description: A reference to the SerDes lane(s) | |
| 43 | + maxItems: 1 | |
| 44 | + | |
| 41 | 45 | required: |
| 42 | 46 | - reg |
| 43 | 47 | |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Technologies, Inc. MDM9607 TLMM block |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@somainline.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: |
| 13 | 13 | Top Level Mode Multiplexer pin controller in Qualcomm MDM9607 SoC. |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Technologies, Inc. SM6350 TLMM block |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@somainline.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: |
| 13 | 13 | Top Level Mode Multiplexer pin controller in Qualcomm SM6350 SoC. |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Technologies, Inc. SM6375 TLMM block |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@somainline.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: |
| 13 | 13 | Top Level Mode Multiplexer pin controller in Qualcomm SM6375 SoC. |
| @@ -8,7 +8,7 @@ title: Qualcomm Resource Power Manager (RPM) Processor/Subsystem · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | 10 | - Bjorn Andersson <andersson@kernel.org> |
| 11 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 11 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 12 | 12 | - Stephan Gerhold <stephan@gerhold.net> |
| 13 | 13 | |
| 14 | 14 | description: | |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: Qualcomm Technologies, Inc. (QTI) RPM Master Stats |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 10 | + - Konrad Dybcio <konradybcio@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | description: | |
| 13 | 13 | The Qualcomm RPM (Resource Power Manager) architecture includes a concept |
| @@ -199,10 +199,11 @@ additionalProperties: false · 198 unmodified lines | ||
| 199 | 199 | |
| 200 | 200 | examples: |
| 201 | 201 | - | |
| 202 | + #include <dt-bindings/gpio/gpio.h> | |
| 202 | 203 | codec@1,0{ |
| 203 | 204 | compatible = "slim217,250"; |
| 204 | 205 | reg = <1 0>; |
| 205 | − reset-gpios = <&tlmm 64 0>; | |
| 206 | + reset-gpios = <&tlmm 64 GPIO_ACTIVE_LOW>; | |
| 206 | 207 | slim-ifc-dev = <&wcd9340_ifd>; |
| 207 | 208 | #sound-dai-cells = <1>; |
| 208 | 209 | interrupt-parent = <&tlmm>; |
| @@ -42,7 +42,7 @@ examples: · 41 unmodified lines | ||
| 42 | 42 | pinctrl-names = "default", "sleep"; |
| 43 | 43 | pinctrl-0 = <&wcd_reset_n>; |
| 44 | 44 | pinctrl-1 = <&wcd_reset_n_sleep>; |
| 45 | − reset-gpios = <&tlmm 83 GPIO_ACTIVE_HIGH>; | |
| 45 | + reset-gpios = <&tlmm 83 GPIO_ACTIVE_LOW>; | |
| 46 | 46 | vdd-buck-supply = <&vreg_l17b_1p8>; |
| 47 | 47 | vdd-rxtx-supply = <&vreg_l18b_1p8>; |
| 48 | 48 | vdd-px-supply = <&vreg_l18b_1p8>; |
| @@ -34,9 +34,10 @@ unevaluatedProperties: false · 33 unmodified lines | ||
| 34 | 34 | |
| 35 | 35 | examples: |
| 36 | 36 | - | |
| 37 | + #include <dt-bindings/gpio/gpio.h> | |
| 37 | 38 | codec { |
| 38 | 39 | compatible = "qcom,wcd9380-codec"; |
| 39 | − reset-gpios = <&tlmm 32 0>; | |
| 40 | + reset-gpios = <&tlmm 32 GPIO_ACTIVE_LOW>; | |
| 40 | 41 | #sound-dai-cells = <1>; |
| 41 | 42 | qcom,tx-device = <&wcd938x_tx>; |
| 42 | 43 | qcom,rx-device = <&wcd938x_rx>; |
| @@ -52,10 +52,10 @@ unevaluatedProperties: false · 51 unmodified lines | ||
| 52 | 52 | |
| 53 | 53 | examples: |
| 54 | 54 | - | |
| 55 | − #include <dt-bindings/interrupt-controller/irq.h> | |
| 55 | + #include <dt-bindings/gpio/gpio.h> | |
| 56 | 56 | codec { |
| 57 | 57 | compatible = "qcom,wcd9390-codec"; |
| 58 | − reset-gpios = <&tlmm 32 IRQ_TYPE_NONE>; | |
| 58 | + reset-gpios = <&tlmm 32 GPIO_ACTIVE_LOW>; | |
| 59 | 59 | #sound-dai-cells = <1>; |
| 60 | 60 | qcom,tx-device = <&wcd939x_tx>; |
| 61 | 61 | qcom,rx-device = <&wcd939x_rx>; |
| @@ -10,7 +10,7 @@ maintainers: · 9 unmodified lines | ||
| 10 | 10 | - Fabio Estevam <festevam@gmail.com> |
| 11 | 11 | |
| 12 | 12 | allOf: |
| 13 | − - $ref: usb-hcd.yaml# | |
| 13 | + - $ref: usb-device.yaml# | |
| 14 | 14 | |
| 15 | 15 | properties: |
| 16 | 16 | compatible: |
| @@ -18,6 +18,7 @@ properties: · 1 unmodified line | ||
| 18 | 18 | - usb424,2412 |
| 19 | 19 | - usb424,2417 |
| 20 | 20 | - usb424,2514 |
| 21 | + - usb424,2517 | |
| 21 | 22 | |
| 22 | 23 | reg: true |
| 23 | 24 | |
| @@ -35,6 +36,13 @@ required: · 11 unmodified lines | ||
| 35 | 36 | - compatible |
| 36 | 37 | - reg |
| 37 | 38 | |
| 39 | +patternProperties: | |
| 40 | + "^.*@[0-9a-f]{1,2}$": | |
| 41 | + description: The hard wired USB devices | |
| 42 | + type: object | |
| 43 | + $ref: /schemas/usb/usb-device.yaml | |
| 44 | + additionalProperties: true | |
| 45 | + | |
| 38 | 46 | unevaluatedProperties: false |
| 39 | 47 | |
| 40 | 48 | examples: |
| @@ -4,8 +4,6 @@ Generic Thermal Sysfs driver How To · 3 unmodified lines | ||
| 4 | 4 | |
| 5 | 5 | Written by Sujith Thomas <sujith.thomas@intel.com>, Zhang Rui <rui.zhang@intel.com> |
| 6 | 6 | |
| 7 | −Updated: 2 January 2008 | |
| 8 | − | |
| 9 | 7 | Copyright (c) 2008 Intel Corporation |
| 10 | 8 | |
| 11 | 9 | |
| @@ -38,23 +36,23 @@ temperature) and throttle appropriate devices. · 26 unmodified lines | ||
| 38 | 36 | |
| 39 | 37 | :: |
| 40 | 38 | |
| 41 | − struct thermal_zone_device | |
| 42 | − *thermal_zone_device_register(char *type, | |
| 43 | − int trips, int mask, void *devdata, | |
| 44 | − struct thermal_zone_device_ops *ops, | |
| 45 | − const struct thermal_zone_params *tzp, | |
| 46 | − int passive_delay, int polling_delay)) | |
| 39 | + struct thermal_zone_device * | |
| 40 | + thermal_zone_device_register_with_trips(const char *type, | |
| 41 | + const struct thermal_trip *trips, | |
| 42 | + int num_trips, void *devdata, | |
| 43 | + const struct thermal_zone_device_ops *ops, | |
| 44 | + const struct thermal_zone_params *tzp, | |
| 45 | + unsigned int passive_delay, | |
| 46 | + unsigned int polling_delay) | |
| 47 | 47 | |
| 48 | − This interface function adds a new thermal zone device (sensor) to | |
| 48 | + This interface function adds a new thermal zone device (sensor) to the | |
| 49 | 49 | /sys/class/thermal folder as `thermal_zone[0-*]`. It tries to bind all the |
| 50 | − thermal cooling devices registered at the same time. | |
| 50 | + thermal cooling devices registered to it at the same time. | |
| 51 | 51 | |
| 52 | 52 | type: |
| 53 | 53 | the thermal zone type. |
| 54 | 54 | trips: |
| 55 | − the total number of trip points this thermal zone supports. | |
| 56 | − mask: | |
| 57 | − Bit string: If 'n'th bit is set, then trip point 'n' is writable. | |
| 55 | + the table of trip points for this thermal zone. | |
| 58 | 56 | devdata: |
| 59 | 57 | device private data |
| 60 | 58 | ops: |
| @@ -67,32 +65,29 @@ temperature) and throttle appropriate devices. · 6 unmodified lines | ||
| 67 | 65 | .get_temp: |
| 68 | 66 | get the current temperature of the thermal zone. |
| 69 | 67 | .set_trips: |
| 70 | − set the trip points window. Whenever the current temperature | |
| 71 | − is updated, the trip points immediately below and above the | |
| 72 | − current temperature are found. | |
| 73 | − .get_mode: | |
| 74 | − get the current mode (enabled/disabled) of the thermal zone. | |
| 75 | − | |
| 76 | − - "enabled" means the kernel thermal management is | |
| 77 | − enabled. | |
| 78 | − - "disabled" will prevent kernel thermal driver action | |
| 79 | − upon trip points so that user applications can take | |
| 80 | − charge of thermal management. | |
| 81 | − .set_mode: | |
| 82 | − set the mode (enabled/disabled) of the thermal zone. | |
| 83 | − .get_trip_type: | |
| 84 | − get the type of certain trip point. | |
| 85 | − .get_trip_temp: | |
| 86 | − get the temperature above which the certain trip point | |
| 87 | − will be fired. | |
| 68 | + set the trip points window. Whenever the current temperature | |
| 69 | + is updated, the trip points immediately below and above the | |
| 70 | + current temperature are found. | |
| 71 | + .change_mode: | |
| 72 | + change the mode (enabled/disabled) of the thermal zone. | |
| 73 | + .set_trip_temp: | |
| 74 | + set the temperature of a given trip point. | |
| 75 | + .get_crit_temp: | |
| 76 | + get the critical temperature for this thermal zone. | |
| 88 | 77 | .set_emul_temp: |
| 89 | − set the emulation temperature which helps in debugging | |
| 90 | − different threshold temperature points. | |
| 78 | + set the emulation temperature which helps in debugging | |
| 79 | + different threshold temperature points. | |
| 80 | + .get_trend: | |
| 81 | + get the trend of most recent zone temperature changes. | |
| 82 | + .hot: | |
| 83 | + hot trip point crossing handler. | |
| 84 | + .critical: | |
| 85 | + critical trip point crossing handler. | |
| 91 | 86 | tzp: |
| 92 | 87 | thermal zone platform parameters. |
| 93 | 88 | passive_delay: |
| 94 | − number of milliseconds to wait between polls when | |
| 95 | − performing passive cooling. | |
| 89 | + number of milliseconds to wait between polls when performing passive | |
| 90 | + cooling. | |
| 96 | 91 | polling_delay: |
| 97 | 92 | number of milliseconds to wait between polls when checking |
| 98 | 93 | whether trip points have been crossed (0 for interrupt driven systems). |
| @@ -318,10 +318,10 @@ where the columns are: · 317 unmodified lines | ||
| 318 | 318 | Debugging |
| 319 | 319 | ========= |
| 320 | 320 | |
| 321 | −If CONFIG_FSCACHE_DEBUG is enabled, the FS-Cache facility can have runtime | |
| 322 | −debugging enabled by adjusting the value in:: | |
| 321 | +If CONFIG_NETFS_DEBUG is enabled, the FS-Cache facility and NETFS support can | |
| 322 | +have runtime debugging enabled by adjusting the value in:: | |
| 323 | 323 | |
| 324 | − /sys/module/fscache/parameters/debug | |
| 324 | + /sys/module/netfs/parameters/debug | |
| 325 | 325 | |
| 326 | 326 | This is a bitmask of debugging streams to enable: |
| 327 | 327 | |
| @@ -343,6 +343,6 @@ This is a bitmask of debugging streams to enable: · 15 unmodified lines | ||
| 343 | 343 | The appropriate set of values should be OR'd together and the result written to |
| 344 | 344 | the control file. For example:: |
| 345 | 345 | |
| 346 | − echo $((1|8|512)) >/sys/module/fscache/parameters/debug | |
| 346 | + echo $((1|8|512)) >/sys/module/netfs/parameters/debug | |
| 347 | 347 | |
| 348 | 348 | will turn on all function entry debugging. |
| @@ -75,7 +75,7 @@ Here are the main features of EROFS: · 74 unmodified lines | ||
| 75 | 75 | |
| 76 | 76 | - Support merging tail-end data into a special inode as fragments. |
| 77 | 77 | |
| 78 | − - Support large folios for uncompressed files. | |
| 78 | + - Support large folios to make use of THPs (Transparent Hugepages); | |
| 79 | 79 | |
| 80 | 80 | - Support direct I/O on uncompressed files to avoid double caching for loop |
| 81 | 81 | devices; |
| @@ -13,7 +13,7 @@ KSMBD architecture · 12 unmodified lines | ||
| 13 | 13 | The subset of performance related operations belong in kernelspace and |
| 14 | 14 | the other subset which belong to operations which are not really related with |
| 15 | 15 | performance in userspace. So, DCE/RPC management that has historically resulted |
| 16 | −into number of buffer overflow issues and dangerous security bugs and user | |
| 16 | +into a number of buffer overflow issues and dangerous security bugs and user | |
| 17 | 17 | account management are implemented in user space as ksmbd.mountd. |
| 18 | 18 | File operations that are related with performance (open/read/write/close etc.) |
| 19 | 19 | in kernel space (ksmbd). This also allows for easier integration with VFS |
| @@ -24,8 +24,8 @@ ksmbd (kernel daemon) · 4 unmodified lines | ||
| 24 | 24 | |
| 25 | 25 | When the server daemon is started, It starts up a forker thread |
| 26 | 26 | (ksmbd/interface name) at initialization time and open a dedicated port 445 |
| 27 | −for listening to SMB requests. Whenever new clients make request, Forker | |
| 28 | −thread will accept the client connection and fork a new thread for dedicated | |
| 27 | +for listening to SMB requests. Whenever new clients make a request, the Forker | |
| 28 | +thread will accept the client connection and fork a new thread for a dedicated | |
| 29 | 29 | communication channel between the client and the server. It allows for parallel |
| 30 | 30 | processing of SMB requests(commands) from clients as well as allowing for new |
| 31 | 31 | clients to make new connections. Each instance is named ksmbd/1~n(port number) |
| @@ -34,12 +34,12 @@ thread can decide to pass through the commands to the user space (ksmbd.mountd), · 2 unmodified lines | ||
| 34 | 34 | currently DCE/RPC commands are identified to be handled through the user space. |
| 35 | 35 | To further utilize the linux kernel, it has been chosen to process the commands |
| 36 | 36 | as workitems and to be executed in the handlers of the ksmbd-io kworker threads. |
| 37 | −It allows for multiplexing of the handlers as the kernel take care of initiating | |
| 37 | +It allows for multiplexing of the handlers as the kernel takes care of initiating | |
| 38 | 38 | extra worker threads if the load is increased and vice versa, if the load is |
| 39 | −decreased it destroys the extra worker threads. So, after connection is | |
| 40 | −established with client. Dedicated ksmbd/1..n(port number) takes complete | |
| 39 | +decreased it destroys the extra worker threads. So, after the connection is | |
| 40 | +established with the client. Dedicated ksmbd/1..n(port number) takes complete | |
| 41 | 41 | ownership of receiving/parsing of SMB commands. Each received command is worked |
| 42 | −in parallel i.e., There can be multiple clients commands which are worked in | |
| 42 | +in parallel i.e., there can be multiple client commands which are worked in | |
| 43 | 43 | parallel. After receiving each command a separated kernel workitem is prepared |
| 44 | 44 | for each command which is further queued to be handled by ksmbd-io kworkers. |
| 45 | 45 | So, each SMB workitem is queued to the kworkers. This allows the benefit of load |
| @@ -49,9 +49,9 @@ performance by handling client commands in parallel. · 3 unmodified lines | ||
| 49 | 49 | ksmbd.mountd (user space daemon) |
| 50 | 50 | -------------------------------- |
| 51 | 51 | |
| 52 | −ksmbd.mountd is userspace process to, transfer user account and password that | |
| 52 | +ksmbd.mountd is a userspace process to, transfer the user account and password that | |
| 53 | 53 | are registered using ksmbd.adduser (part of utils for user space). Further it |
| 54 | −allows sharing information parameters that parsed from smb.conf to ksmbd in | |
| 54 | +allows sharing information parameters that are parsed from smb.conf to ksmbd in | |
| 55 | 55 | kernel. For the execution part it has a daemon which is continuously running |
| 56 | 56 | and connected to the kernel interface using netlink socket, it waits for the |
| 57 | 57 | requests (dcerpc and share/user info). It handles RPC calls (at a minimum few |
| @@ -124,7 +124,7 @@ How to run · 66 unmodified lines | ||
| 124 | 124 | 1. Download ksmbd-tools(https://github.com/cifsd-team/ksmbd-tools/releases) and |
| 125 | 125 | compile them. |
| 126 | 126 | |
| 127 | − - Refer README(https://github.com/cifsd-team/ksmbd-tools/blob/master/README.md) | |
| 127 | + - Refer to README(https://github.com/cifsd-team/ksmbd-tools/blob/master/README.md) | |
| 128 | 128 | to know how to use ksmbd.mountd/adduser/addshare/control utils |
| 129 | 129 | |
| 130 | 130 | $ ./autogen.sh |
| @@ -133,7 +133,7 @@ How to run · 2 unmodified lines | ||
| 133 | 133 | |
| 134 | 134 | 2. Create /usr/local/etc/ksmbd/ksmbd.conf file, add SMB share in ksmbd.conf file. |
| 135 | 135 | |
| 136 | − - Refer ksmbd.conf.example in ksmbd-utils, See ksmbd.conf manpage | |
| 136 | + - Refer to ksmbd.conf.example in ksmbd-utils, See ksmbd.conf manpage | |
| 137 | 137 | for details to configure shares. |
| 138 | 138 | |
| 139 | 139 | $ man ksmbd.conf |
| @@ -145,7 +145,7 @@ How to run · 5 unmodified lines | ||
| 145 | 145 | $ man ksmbd.adduser |
| 146 | 146 | $ sudo ksmbd.adduser -a <Enter USERNAME for SMB share access> |
| 147 | 147 | |
| 148 | −4. Insert ksmbd.ko module after build your kernel. No need to load module | |
| 148 | +4. Insert the ksmbd.ko module after you build your kernel. No need to load the module | |
| 149 | 149 | if ksmbd is built into the kernel. |
| 150 | 150 | |
| 151 | 151 | - Set ksmbd in menuconfig(e.g. $ make menuconfig) |
| @@ -175,7 +175,7 @@ Each layer · 23 unmodified lines | ||
| 175 | 175 | 1. Enable all component prints |
| 176 | 176 | # sudo ksmbd.control -d "all" |
| 177 | 177 | |
| 178 | −2. Enable one of components (smb, auth, vfs, oplock, ipc, conn, rdma) | |
| 178 | +2. Enable one of the components (smb, auth, vfs, oplock, ipc, conn, rdma) | |
| 179 | 179 | # sudo ksmbd.control -d "smb" |
| 180 | 180 | |
| 181 | 181 | 3. Show what prints are enabled. |
| @@ -126,7 +126,7 @@ Ccache · 125 unmodified lines | ||
| 126 | 126 | |
| 127 | 127 | ``ccache`` can be used with ``clang`` to improve subsequent builds, (though |
| 128 | 128 | KBUILD_BUILD_TIMESTAMP_ should be set to a deterministic value between builds |
| 129 | −in order to avoid 100% cache misses, see Reproducible_builds_ for more info): | |
| 129 | +in order to avoid 100% cache misses, see Reproducible_builds_ for more info):: | |
| 130 | 130 | |
| 131 | 131 | KBUILD_BUILD_TIMESTAMP='' make LLVM=1 CC="ccache clang" |
| 132 | 132 | |
| @@ -1753,6 +1753,7 @@ operations: · 1752 unmodified lines | ||
| 1753 | 1753 | request: |
| 1754 | 1754 | attributes: |
| 1755 | 1755 | - header |
| 1756 | + - context | |
| 1756 | 1757 | reply: |
| 1757 | 1758 | attributes: |
| 1758 | 1759 | - header |
| @@ -1761,7 +1762,6 @@ operations: · 2 unmodified lines | ||
| 1761 | 1762 | - indir |
| 1762 | 1763 | - hkey |
| 1763 | 1764 | - input_xfrm |
| 1764 | − dump: *rss-get-op | |
| 1765 | 1765 | - |
| 1766 | 1766 | name: plca-get-cfg |
| 1767 | 1767 | doc: Get PLCA params. |
| @@ -1875,6 +1875,7 @@ Kernel response contents: · 1874 unmodified lines | ||
| 1875 | 1875 | |
| 1876 | 1876 | ===================================== ====== ========================== |
| 1877 | 1877 | ``ETHTOOL_A_RSS_HEADER`` nested reply header |
| 1878 | + ``ETHTOOL_A_RSS_CONTEXT`` u32 context number | |
| 1878 | 1879 | ``ETHTOOL_A_RSS_HFUNC`` u32 RSS hash func |
| 1879 | 1880 | ``ETHTOOL_A_RSS_INDIR`` binary Indir table bytes |
| 1880 | 1881 | ``ETHTOOL_A_RSS_HKEY`` binary Hash key bytes |
| @@ -629,18 +629,6 @@ The preferred style for long (multi-line) comments is: · 628 unmodified lines | ||
| 629 | 629 | * with beginning and ending almost-blank lines. |
| 630 | 630 | */ |
| 631 | 631 | |
| 632 | −For files in net/ and drivers/net/ the preferred style for long (multi-line) | |
| 633 | −comments is a little different. | |
| 634 | − | |
| 635 | −.. code-block:: c | |
| 636 | − | |
| 637 | − /* The preferred comment style for files in net/ and drivers/net | |
| 638 | − * looks like this. | |
| 639 | − * | |
| 640 | − * It is nearly the same as the generally preferred comment style, | |
| 641 | − * but there is no initial almost-blank line. | |
| 642 | − */ | |
| 643 | − | |
| 644 | 632 | It's also important to comment data, whether they are basic types or derived |
| 645 | 633 | types. To this end, use just one data declaration per line (no commas for |
| 646 | 634 | multiple data declarations). This leaves you room for a small comment on each |
| @@ -13,9 +13,9 @@ kernel. · 12 unmodified lines | ||
| 13 | 13 | Hardware issues like Meltdown, Spectre, L1TF etc. must be treated |
| 14 | 14 | differently because they usually affect all Operating Systems ("OS") and |
| 15 | 15 | therefore need coordination across different OS vendors, distributions, |
| 16 | −hardware vendors and other parties. For some of the issues, software | |
| 17 | −mitigations can depend on microcode or firmware updates, which need further | |
| 18 | −coordination. | |
| 16 | +silicon vendors, hardware integrators, and other parties. For some of the | |
| 17 | +issues, software mitigations can depend on microcode or firmware updates, | |
| 18 | +which need further coordination. | |
| 19 | 19 | |
| 20 | 20 | .. _Contact: |
| 21 | 21 | |
| @@ -32,8 +32,8 @@ Linux kernel security team (:ref:`Documentation/admin-guide/ · 10 unmodified lines | ||
| 32 | 32 | <securitybugs>`) instead. |
| 33 | 33 | |
| 34 | 34 | The team can be contacted by email at <hardware-security@kernel.org>. This |
| 35 | −is a private list of security officers who will help you to coordinate a | |
| 36 | −fix according to our documented process. | |
| 35 | +is a private list of security officers who will help you coordinate a fix | |
| 36 | +according to our documented process. | |
| 37 | 37 | |
| 38 | 38 | The list is encrypted and email to the list can be sent by either PGP or |
| 39 | 39 | S/MIME encrypted and must be signed with the reporter's PGP key or S/MIME |
| @@ -43,7 +43,7 @@ the following URLs: · 3 unmodified lines | ||
| 43 | 43 | - PGP: https://www.kernel.org/static/files/hardware-security.asc |
| 44 | 44 | - S/MIME: https://www.kernel.org/static/files/hardware-security.crt |
| 45 | 45 | |
| 46 | −While hardware security issues are often handled by the affected hardware | |
| 46 | +While hardware security issues are often handled by the affected silicon | |
| 47 | 47 | vendor, we welcome contact from researchers or individuals who have |
| 48 | 48 | identified a potential hardware flaw. |
| 49 | 49 | |
| @@ -65,7 +65,7 @@ of Linux Foundation's IT operations personnel technically have the · 15 unmodified lines | ||
| 65 | 65 | ability to access the embargoed information, but are obliged to |
| 66 | 66 | confidentiality by their employment contract. Linux Foundation IT |
| 67 | 67 | personnel are also responsible for operating and managing the rest of |
| 68 | −kernel.org infrastructure. | |
| 68 | +kernel.org's infrastructure. | |
| 69 | 69 | |
| 70 | 70 | The Linux Foundation's current director of IT Project infrastructure is |
| 71 | 71 | Konstantin Ryabitsev. |
| @@ -85,7 +85,7 @@ Memorandum of Understanding · 13 unmodified lines | ||
| 85 | 85 | |
| 86 | 86 | The Linux kernel community has a deep understanding of the requirement to |
| 87 | 87 | keep hardware security issues under embargo for coordination between |
| 88 | −different OS vendors, distributors, hardware vendors and other parties. | |
| 88 | +different OS vendors, distributors, silicon vendors, and other parties. | |
| 89 | 89 | |
| 90 | 90 | The Linux kernel community has successfully handled hardware security |
| 91 | 91 | issues in the past and has the necessary mechanisms in place to allow |
| @@ -103,11 +103,11 @@ the issue in the best technical way. · 11 unmodified lines | ||
| 103 | 103 | All involved developers pledge to adhere to the embargo rules and to keep |
| 104 | 104 | the received information confidential. Violation of the pledge will lead to |
| 105 | 105 | immediate exclusion from the current issue and removal from all related |
| 106 | −mailing-lists. In addition, the hardware security team will also exclude | |
| 106 | +mailing lists. In addition, the hardware security team will also exclude | |
| 107 | 107 | the offender from future issues. The impact of this consequence is a highly |
| 108 | 108 | effective deterrent in our community. In case a violation happens the |
| 109 | 109 | hardware security team will inform the involved parties immediately. If you |
| 110 | −or anyone becomes aware of a potential violation, please report it | |
| 110 | +or anyone else becomes aware of a potential violation, please report it | |
| 111 | 111 | immediately to the Hardware security officers. |
| 112 | 112 | |
| 113 | 113 | |
| @@ -124,14 +124,16 @@ method for these types of issues. · 10 unmodified lines | ||
| 124 | 124 | Start of Disclosure |
| 125 | 125 | """"""""""""""""""" |
| 126 | 126 | |
| 127 | −Disclosure starts by contacting the Linux kernel hardware security team by | |
| 128 | −email. This initial contact should contain a description of the problem and | |
| 129 | −a list of any known affected hardware. If your organization builds or | |
| 130 | −distributes the affected hardware, we encourage you to also consider what | |
| 131 | −other hardware could be affected. | |
| 127 | +Disclosure starts by emailing the Linux kernel hardware security team per | |
| 128 | +the Contact section above. This initial contact should contain a | |
| 129 | +description of the problem and a list of any known affected silicon. If | |
| 130 | +your organization builds or distributes the affected hardware, we encourage | |
| 131 | +you to also consider what other hardware could be affected. The disclosing | |
| 132 | +party is responsible for contacting the affected silicon vendors in a | |
| 133 | +timely manner. | |
| 132 | 134 | |
| 133 | 135 | The hardware security team will provide an incident-specific encrypted |
| 134 | −mailing-list which will be used for initial discussion with the reporter, | |
| 136 | +mailing list which will be used for initial discussion with the reporter, | |
| 135 | 137 | further disclosure, and coordination of fixes. |
| 136 | 138 | |
| 137 | 139 | The hardware security team will provide the disclosing party a list of |
| @@ -158,8 +160,8 @@ This serves several purposes: · 20 unmodified lines | ||
| 158 | 160 | - The disclosed entities can be contacted to name experts who should |
| 159 | 161 | participate in the mitigation development. |
| 160 | 162 | |
| 161 | − - If an expert which is required to handle an issue is employed by an | |
| 162 | − listed entity or member of an listed entity, then the response teams can | |
| 163 | + - If an expert who is required to handle an issue is employed by a listed | |
| 164 | + entity or member of an listed entity, then the response teams can | |
| 163 | 165 | request the disclosure of that expert from that entity. This ensures |
| 164 | 166 | that the expert is also part of the entity's response team. |
| 165 | 167 | |
| @@ -169,8 +171,8 @@ Disclosure · 3 unmodified lines | ||
| 169 | 171 | The disclosing party provides detailed information to the initial response |
| 170 | 172 | team via the specific encrypted mailing-list. |
| 171 | 173 | |
| 172 | −From our experience the technical documentation of these issues is usually | |
| 173 | −a sufficient starting point and further technical clarification is best | |
| 174 | +From our experience, the technical documentation of these issues is usually | |
| 175 | +a sufficient starting point, and further technical clarification is best | |
| 174 | 176 | done via email. |
| 175 | 177 | |
| 176 | 178 | Mitigation development |
| @@ -179,57 +181,93 @@ Mitigation development · 2 unmodified lines | ||
| 179 | 181 | The initial response team sets up an encrypted mailing-list or repurposes |
| 180 | 182 | an existing one if appropriate. |
| 181 | 183 | |
| 182 | −Using a mailing-list is close to the normal Linux development process and | |
| 183 | −has been successfully used in developing mitigations for various hardware | |
| 184 | +Using a mailing list is close to the normal Linux development process and | |
| 185 | +has been successfully used to develop mitigations for various hardware | |
| 184 | 186 | security issues in the past. |
| 185 | 187 | |
| 186 | −The mailing-list operates in the same way as normal Linux development. | |
| 187 | −Patches are posted, discussed and reviewed and if agreed on applied to a | |
| 188 | −non-public git repository which is only accessible to the participating | |
| 188 | +The mailing list operates in the same way as normal Linux development. | |
| 189 | +Patches are posted, discussed, and reviewed and if agreed upon, applied to | |
| 190 | +a non-public git repository which is only accessible to the participating | |
| 189 | 191 | developers via a secure connection. The repository contains the main |
| 190 | 192 | development branch against the mainline kernel and backport branches for |
| 191 | 193 | stable kernel versions as necessary. |
| 192 | 194 | |
| 193 | 195 | The initial response team will identify further experts from the Linux |
| 194 | −kernel developer community as needed. Bringing in experts can happen at any | |
| 195 | −time of the development process and needs to be handled in a timely manner. | |
| 196 | +kernel developer community as needed. Any involved party can suggest | |
| 197 | +further experts to be included, each of which will be subject to the same | |
| 198 | +requirements outlined above. | |
| 196 | 199 | |
| 197 | −If an expert is employed by or member of an entity on the disclosure list | |
| 200 | +Bringing in experts can happen at any time in the development process and | |
| 201 | +needs to be handled in a timely manner. | |
| 202 | + | |
| 203 | +If an expert is employed by or a member of an entity on the disclosure list | |
| 198 | 204 | provided by the disclosing party, then participation will be requested from |
| 199 | 205 | the relevant entity. |
| 200 | 206 | |
| 201 | −If not, then the disclosing party will be informed about the experts | |
| 207 | +If not, then the disclosing party will be informed about the experts' | |
| 202 | 208 | participation. The experts are covered by the Memorandum of Understanding |
| 203 | −and the disclosing party is requested to acknowledge the participation. In | |
| 204 | −case that the disclosing party has a compelling reason to object, then this | |
| 205 | −objection has to be raised within five work days and resolved with the | |
| 206 | −incident team immediately. If the disclosing party does not react within | |
| 207 | −five work days this is taken as silent acknowledgement. | |
| 209 | +and the disclosing party is requested to acknowledge their participation. | |
| 210 | +In the case where the disclosing party has a compelling reason to object, | |
| 211 | +any objection must to be raised within five working days and resolved with | |
| 212 | +the incident team immediately. If the disclosing party does not react | |
| 213 | +within five working days this is taken as silent acknowledgment. | |
| 208 | 214 | |
| 209 | −After acknowledgement or resolution of an objection the expert is disclosed | |
| 210 | −by the incident team and brought into the development process. | |
| 215 | +After the incident team acknowledges or resolves an objection, the expert | |
| 216 | +is disclosed and brought into the development process. | |
| 211 | 217 | |
| 212 | 218 | List participants may not communicate about the issue outside of the |
| 213 | 219 | private mailing list. List participants may not use any shared resources |
| 214 | 220 | (e.g. employer build farms, CI systems, etc) when working on patches. |
| 215 | 221 | |
| 222 | +Early access | |
| 223 | +"""""""""""" | |
| 224 | + | |
| 225 | +The patches discussed and developed on the list can neither be distributed | |
| 226 | +to any individual who is not a member of the response team nor to any other | |
| 227 | +organization. | |
| 228 | + | |
| 229 | +To allow the affected silicon vendors to work with their internal teams and | |
| 230 | +industry partners on testing, validation, and logistics, the following | |
| 231 | +exception is provided: | |
| 232 | + | |
| 233 | + Designated representatives of the affected silicon vendors are | |
| 234 | + allowed to hand over the patches at any time to the silicon | |
| 235 | + vendor’s response team. The representative must notify the kernel | |
| 236 | + response team about the handover. The affected silicon vendor must | |
| 237 | + have and maintain their own documented security process for any | |
| 238 | + patches shared with their response team that is consistent with | |
| 239 | + this policy. | |
| 240 | + | |
| 241 | + The silicon vendor’s response team can distribute these patches to | |
| 242 | + their industry partners and to their internal teams under the | |
| 243 | + silicon vendor’s documented security process. Feedback from the | |
| 244 | + industry partners goes back to the silicon vendor and is | |
| 245 | + communicated by the silicon vendor to the kernel response team. | |
| 246 | + | |
| 247 | + The handover to the silicon vendor’s response team removes any | |
| 248 | + responsibility or liability from the kernel response team regarding | |
| 249 | + premature disclosure, which happens due to the involvement of the | |
| 250 | + silicon vendor’s internal teams or industry partners. The silicon | |
| 251 | + vendor guarantees this release of liability by agreeing to this | |
| 252 | + process. | |
| 216 | 253 | |
| 217 | 254 | Coordinated release |
| 218 | 255 | """"""""""""""""""" |
| 219 | 256 | |
| 220 | −The involved parties will negotiate the date and time where the embargo | |
| 221 | −ends. At that point the prepared mitigations are integrated into the | |
| 222 | −relevant kernel trees and published. There is no pre-notification process: | |
| 223 | −fixes are published in public and available to everyone at the same time. | |
| 257 | +The involved parties will negotiate the date and time when the embargo | |
| 258 | +ends. At that point, the prepared mitigations are published into the | |
| 259 | +relevant kernel trees. There is no pre-notification process: the | |
| 260 | +mitigations are published in public and available to everyone at the same | |
| 261 | +time. | |
| 224 | 262 | |
| 225 | 263 | While we understand that hardware security issues need coordinated embargo |
| 226 | −time, the embargo time should be constrained to the minimum time which is | |
| 227 | −required for all involved parties to develop, test and prepare the | |
| 264 | +time, the embargo time should be constrained to the minimum time that is | |
| 265 | +required for all involved parties to develop, test, and prepare their | |
| 228 | 266 | mitigations. Extending embargo time artificially to meet conference talk |
| 229 | −dates or other non-technical reasons is creating more work and burden for | |
| 230 | −the involved developers and response teams as the patches need to be kept | |
| 231 | −up to date in order to follow the ongoing upstream kernel development, | |
| 232 | −which might create conflicting changes. | |
| 267 | +dates or other non-technical reasons creates more work and burden for the | |
| 268 | +involved developers and response teams as the patches need to be kept up to | |
| 269 | +date in order to follow the ongoing upstream kernel development, which | |
| 270 | +might create conflicting changes. | |
| 233 | 271 | |
| 234 | 272 | CVE assignment |
| 235 | 273 | """""""""""""" |
| @@ -275,34 +313,35 @@ an involved disclosed party. The current ambassadors list: · 39 unmodified lines | ||
| 275 | 313 | |
| 276 | 314 | If you want your organization to be added to the ambassadors list, please |
| 277 | 315 | contact the hardware security team. The nominated ambassador has to |
| 278 | −understand and support our process fully and is ideally well connected in | |
| 316 | +understand and support our process fully and is ideally well-connected in | |
| 279 | 317 | the Linux kernel community. |
| 280 | 318 | |
| 281 | 319 | Encrypted mailing-lists |
| 282 | 320 | ----------------------- |
| 283 | 321 | |
| 284 | −We use encrypted mailing-lists for communication. The operating principle | |
| 322 | +We use encrypted mailing lists for communication. The operating principle | |
| 285 | 323 | of these lists is that email sent to the list is encrypted either with the |
| 286 | −list's PGP key or with the list's S/MIME certificate. The mailing-list | |
| 324 | +list's PGP key or with the list's S/MIME certificate. The mailing list | |
| 287 | 325 | software decrypts the email and re-encrypts it individually for each |
| 288 | 326 | subscriber with the subscriber's PGP key or S/MIME certificate. Details |
| 289 | −about the mailing-list software and the setup which is used to ensure the | |
| 327 | +about the mailing list software and the setup that is used to ensure the | |
| 290 | 328 | security of the lists and protection of the data can be found here: |
| 291 | 329 | https://korg.wiki.kernel.org/userdoc/remail. |
| 292 | 330 | |
| 293 | 331 | List keys |
| 294 | 332 | ^^^^^^^^^ |
| 295 | 333 | |
| 296 | −For initial contact see :ref:`Contact`. For incident specific mailing-lists | |
| 297 | −the key and S/MIME certificate are conveyed to the subscribers by email | |
| 298 | −sent from the specific list. | |
| 334 | +For initial contact see the :ref:`Contact` section above. For incident | |
| 335 | +specific mailing lists, the key and S/MIME certificate are conveyed to the | |
| 336 | +subscribers by email sent from the specific list. | |
| 299 | 337 | |
| 300 | −Subscription to incident specific lists | |
| 338 | +Subscription to incident-specific lists | |
| 301 | 339 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 302 | 340 | |
| 303 | −Subscription is handled by the response teams. Disclosed parties who want | |
| 304 | −to participate in the communication send a list of potential subscribers to | |
| 305 | −the response team so the response team can validate subscription requests. | |
| 341 | +Subscription to incident-specific lists is handled by the response teams. | |
| 342 | +Disclosed parties who want to participate in the communication send a list | |
| 343 | +of potential experts to the response team so the response team can validate | |
| 344 | +subscription requests. | |
| 306 | 345 | |
| 307 | 346 | Each subscriber needs to send a subscription request to the response team |
| 308 | 347 | by email. The email must be signed with the subscriber's PGP key or S/MIME |
| @@ -355,23 +355,6 @@ just do it. As a result, a sequence of smaller series gets merged quicker and · 354 unmodified lines | ||
| 355 | 355 | with better review coverage. Re-posting large series also increases the mailing |
| 356 | 356 | list traffic. |
| 357 | 357 | |
| 358 | −Multi-line comments | |
| 359 | −~~~~~~~~~~~~~~~~~~~ | |
| 360 | − | |
| 361 | −Comment style convention is slightly different for networking and most of | |
| 362 | −the tree. Instead of this:: | |
| 363 | − | |
| 364 | − /* | |
| 365 | − * foobar blah blah blah | |
| 366 | − * another line of text | |
| 367 | − */ | |
| 368 | − | |
| 369 | −it is requested that you make it look like this:: | |
| 370 | − | |
| 371 | − /* foobar blah blah blah | |
| 372 | − * another line of text | |
| 373 | − */ | |
| 374 | − | |
| 375 | 358 | Local variable ordering ("reverse xmas tree", "RCS") |
| 376 | 359 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| 377 | 360 | |
| @@ -21,9 +21,9 @@ are often referred to as greyscale formats. · 20 unmodified lines | ||
| 21 | 21 | |
| 22 | 22 | .. raw:: latex |
| 23 | 23 | |
| 24 | − \scriptsize | |
| 24 | + \tiny | |
| 25 | 25 | |
| 26 | −.. tabularcolumns:: |p{3.6cm}|p{3.0cm}|p{1.3cm}|p{2.6cm}|p{1.3cm}|p{1.3cm}|p{1.3cm}| | |
| 26 | +.. tabularcolumns:: |p{3.6cm}|p{2.4cm}|p{1.3cm}|p{1.3cm}|p{1.3cm}|p{1.3cm}|p{1.3cm}|p{1.3cm}|p{1.3cm}| | |
| 27 | 27 | |
| 28 | 28 | .. flat-table:: Luma-Only Image Formats |
| 29 | 29 | :header-rows: 1 |
| @@ -2592,7 +2592,7 @@ Specifically: · 2591 unmodified lines | ||
| 2592 | 2592 | 0x6030 0000 0010 004a SPSR_ABT 64 spsr[KVM_SPSR_ABT] |
| 2593 | 2593 | 0x6030 0000 0010 004c SPSR_UND 64 spsr[KVM_SPSR_UND] |
| 2594 | 2594 | 0x6030 0000 0010 004e SPSR_IRQ 64 spsr[KVM_SPSR_IRQ] |
| 2595 | − 0x6060 0000 0010 0050 SPSR_FIQ 64 spsr[KVM_SPSR_FIQ] | |
| 2595 | + 0x6030 0000 0010 0050 SPSR_FIQ 64 spsr[KVM_SPSR_FIQ] | |
| 2596 | 2596 | 0x6040 0000 0010 0054 V0 128 fp_regs.vregs[0] [1]_ |
| 2597 | 2597 | 0x6040 0000 0010 0058 V1 128 fp_regs.vregs[1] [1]_ |
| 2598 | 2598 | ... |
| @@ -6368,7 +6368,7 @@ a single guest_memfd file, but the bound ranges must not overlap). · 3769 unmodified lines | ||
| 6368 | 6368 | See KVM_SET_USER_MEMORY_REGION2 for additional details. |
| 6369 | 6369 | |
| 6370 | 6370 | 4.143 KVM_PRE_FAULT_MEMORY |
| 6371 | −------------------------ | |
| 6371 | +--------------------------- | |
| 6372 | 6372 | |
| 6373 | 6373 | :Capability: KVM_CAP_PRE_FAULT_MEMORY |
| 6374 | 6374 | :Architectures: none |
| @@ -6405,6 +6405,12 @@ for the current vCPU state. KVM maps memory as if the vCPU generated a · 30 unmodified lines | ||
| 6405 | 6405 | stage-2 read page fault, e.g. faults in memory as needed, but doesn't break |
| 6406 | 6406 | CoW. However, KVM does not mark any newly created stage-2 PTE as Accessed. |
| 6407 | 6407 | |
| 6408 | +In the case of confidential VM types where there is an initial set up of | |
| 6409 | +private guest memory before the guest is 'finalized'/measured, this ioctl | |
| 6410 | +should only be issued after completing all the necessary setup to put the | |
| 6411 | +guest into a 'finalized' state so that the above semantics can be reliably | |
| 6412 | +ensured. | |
| 6413 | + | |
| 6408 | 6414 | In some cases, multiple vCPUs might share the page tables. In this |
| 6409 | 6415 | case, the ioctl can be called in parallel. |
| 6410 | 6416 | |
| @@ -130,12 +130,12 @@ data using the `bmfdec <https://github.com/pali/bmfdec>`_ utility: · 129 unmodified lines | ||
| 130 | 130 | |
| 131 | 131 | Due to a peculiarity in how Windows handles the ``CreateByteField()`` ACPI operator (errors only |
| 132 | 132 | happen when a invalid byte field is ultimately accessed), all methods require a 32 byte input |
| 133 | −buffer, even if the Binay MOF says otherwise. | |
| 133 | +buffer, even if the Binary MOF says otherwise. | |
| 134 | 134 | |
| 135 | 135 | The input buffer contains a single byte to select the subfeature to be accessed and 31 bytes of |
| 136 | 136 | input data, the meaning of which depends on the subfeature being accessed. |
| 137 | 137 | |
| 138 | −The output buffer contains a singe byte which signals success or failure (``0x00`` on failure) | |
| 138 | +The output buffer contains a single byte which signals success or failure (``0x00`` on failure) | |
| 139 | 139 | and 31 bytes of output data, the meaning if which depends on the subfeature being accessed. |
| 140 | 140 | |
| 141 | 141 | WMI method Get_EC() |
| @@ -147,7 +147,7 @@ data contains a flag byte and a 28 byte controller firmware version string. · 5 unmodified lines | ||
| 147 | 147 | The first 4 bits of the flag byte contain the minor version of the embedded controller interface, |
| 148 | 148 | with the next 2 bits containing the major version of the embedded controller interface. |
| 149 | 149 | |
| 150 | −The 7th bit signals if the embedded controller page chaged (exact meaning is unknown), and the | |
| 150 | +The 7th bit signals if the embedded controller page changed (exact meaning is unknown), and the | |
| 151 | 151 | last bit signals if the platform is a Tigerlake platform. |
| 152 | 152 | |
| 153 | 153 | The MSI software seems to only use this interface when the last bit is set. |
| @@ -1880,6 +1880,10 @@ F: Documentation/devicetree/bindings/iommu/arm,smmu* · 1879 unmodified lines | ||
| 1880 | 1880 | F: drivers/iommu/arm/ |
| 1881 | 1881 | F: drivers/iommu/io-pgtable-arm* |
| 1882 | 1882 | |
| 1883 | +ARM SMMU SVA SUPPORT | |
| 1884 | +R: Jean-Philippe Brucker <jean-philippe@linaro.org> | |
| 1885 | +F: drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | |
| 1886 | + | |
| 1883 | 1887 | ARM SUB-ARCHITECTURES |
| 1884 | 1888 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 1885 | 1889 | S: Maintained |
| @@ -2535,8 +2539,7 @@ L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) · 649 unmodified lines | ||
| 2535 | 2539 | S: Supported |
| 2536 | 2540 | W: http://www.linux4sam.org |
| 2537 | 2541 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/at91/linux.git |
| 2538 | −F: arch/arm/boot/dts/microchip/at91* | |
| 2539 | −F: arch/arm/boot/dts/microchip/sama* | |
| 2542 | +F: arch/arm/boot/dts/microchip/ | |
| 2540 | 2543 | F: arch/arm/include/debug/at91.S |
| 2541 | 2544 | F: arch/arm/mach-at91/ |
| 2542 | 2545 | F: drivers/memory/atmel* |
| @@ -2745,7 +2748,7 @@ F: include/linux/soc/qcom/ · 202 unmodified lines | ||
| 2745 | 2748 | |
| 2746 | 2749 | ARM/QUALCOMM SUPPORT |
| 2747 | 2750 | M: Bjorn Andersson <andersson@kernel.org> |
| 2748 | −M: Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 2751 | +M: Konrad Dybcio <konradybcio@kernel.org> | |
| 2749 | 2752 | L: linux-arm-msm@vger.kernel.org |
| 2750 | 2753 | S: Maintained |
| 2751 | 2754 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/qcom/linux.git |
| @@ -3504,7 +3507,9 @@ S: Maintained · 752 unmodified lines | ||
| 3504 | 3507 | W: http://linux-atm.sourceforge.net |
| 3505 | 3508 | F: drivers/atm/ |
| 3506 | 3509 | F: include/linux/atm* |
| 3510 | +F: include/linux/sonet.h | |
| 3507 | 3511 | F: include/uapi/linux/atm* |
| 3512 | +F: include/uapi/linux/sonet.h | |
| 3508 | 3513 | |
| 3509 | 3514 | ATMEL MACB ETHERNET DRIVER |
| 3510 | 3515 | M: Nicolas Ferre <nicolas.ferre@microchip.com> |
| @@ -5306,7 +5311,7 @@ F: drivers/media/cec/i2c/ch7322.c · 1795 unmodified lines | ||
| 5306 | 5311 | CIRRUS LOGIC AUDIO CODEC DRIVERS |
| 5307 | 5312 | M: David Rhodes <david.rhodes@cirrus.com> |
| 5308 | 5313 | M: Richard Fitzgerald <rf@opensource.cirrus.com> |
| 5309 | −L: alsa-devel@alsa-project.org (moderated for non-subscribers) | |
| 5314 | +L: linux-sound@vger.kernel.org | |
| 5310 | 5315 | L: patches@opensource.cirrus.com |
| 5311 | 5316 | S: Maintained |
| 5312 | 5317 | F: Documentation/devicetree/bindings/sound/cirrus,cs* |
| @@ -5375,7 +5380,7 @@ F: sound/soc/codecs/lochnagar-sc.c · 62 unmodified lines | ||
| 5375 | 5380 | CIRRUS LOGIC MADERA CODEC DRIVERS |
| 5376 | 5381 | M: Charles Keepax <ckeepax@opensource.cirrus.com> |
| 5377 | 5382 | M: Richard Fitzgerald <rf@opensource.cirrus.com> |
| 5378 | −L: alsa-devel@alsa-project.org (moderated for non-subscribers) | |
| 5383 | +L: linux-sound@vger.kernel.org | |
| 5379 | 5384 | L: patches@opensource.cirrus.com |
| 5380 | 5385 | S: Supported |
| 5381 | 5386 | W: https://github.com/CirrusLogic/linux-drivers/wiki |
| @@ -7108,7 +7113,7 @@ F: drivers/gpu/drm/tiny/panel-mipi-dbi.c · 1726 unmodified lines | ||
| 7108 | 7113 | DRM DRIVER for Qualcomm Adreno GPUs |
| 7109 | 7114 | M: Rob Clark <robdclark@gmail.com> |
| 7110 | 7115 | R: Sean Paul <sean@poorly.run> |
| 7111 | −R: Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 7116 | +R: Konrad Dybcio <konradybcio@kernel.org> | |
| 7112 | 7117 | L: linux-arm-msm@vger.kernel.org |
| 7113 | 7118 | L: dri-devel@lists.freedesktop.org |
| 7114 | 7119 | L: freedreno@lists.freedesktop.org |
| @@ -10175,7 +10180,7 @@ F: Documentation/devicetree/bindings/infiniband/hisilicon-hns-roce.txt · 3060 unmodified lines | ||
| 10175 | 10180 | F: drivers/infiniband/hw/hns/ |
| 10176 | 10181 | |
| 10177 | 10182 | HISILICON SAS Controller |
| 10178 | −M: Xiang Chen <chenxiang66@hisilicon.com> | |
| 10183 | +M: Yihang Li <liyihang9@huawei.com> | |
| 10179 | 10184 | S: Supported |
| 10180 | 10185 | W: http://www.hisilicon.com |
| 10181 | 10186 | F: Documentation/devicetree/bindings/scsi/hisilicon-sas.txt |
| @@ -11995,7 +12000,7 @@ F: fs/jfs/ · 1813 unmodified lines | ||
| 11995 | 12000 | JME NETWORK DRIVER |
| 11996 | 12001 | M: Guo-Fu Tseng <cooldavid@cooldavid.org> |
| 11997 | 12002 | L: netdev@vger.kernel.org |
| 11998 | −S: Maintained | |
| 12003 | +S: Odd Fixes | |
| 11999 | 12004 | F: drivers/net/ethernet/jme.* |
| 12000 | 12005 | |
| 12001 | 12006 | JOURNALLING FLASH FILE SYSTEM V2 (JFFS2) |
| @@ -12167,7 +12172,7 @@ KERNEL NFSD, SUNRPC, AND LOCKD SERVERS · 165 unmodified lines | ||
| 12167 | 12172 | M: Chuck Lever <chuck.lever@oracle.com> |
| 12168 | 12173 | M: Jeff Layton <jlayton@kernel.org> |
| 12169 | 12174 | R: Neil Brown <neilb@suse.de> |
| 12170 | −R: Olga Kornievskaia <kolga@netapp.com> | |
| 12175 | +R: Olga Kornievskaia <okorniev@redhat.com> | |
| 12171 | 12176 | R: Dai Ngo <Dai.Ngo@oracle.com> |
| 12172 | 12177 | R: Tom Talpey <tom@talpey.com> |
| 12173 | 12178 | L: linux-nfs@vger.kernel.org |
| @@ -13326,14 +13331,16 @@ F: Documentation/devicetree/bindings/i2c/i2c-mux-ltc4306.txt · 1152 unmodified lines | ||
| 13326 | 13331 | F: drivers/i2c/muxes/i2c-mux-ltc4306.c |
| 13327 | 13332 | |
| 13328 | 13333 | LTP (Linux Test Project) |
| 13334 | +M: Andrea Cervesato <andrea.cervesato@suse.com> | |
| 13329 | 13335 | M: Cyril Hrubis <chrubis@suse.cz> |
| 13330 | 13336 | M: Jan Stancek <jstancek@redhat.com> |
| 13331 | 13337 | M: Petr Vorel <pvorel@suse.cz> |
| 13332 | 13338 | M: Li Wang <liwang@redhat.com> |
| 13333 | 13339 | M: Yang Xu <xuyang2018.jy@fujitsu.com> |
| 13340 | +M: Xiao Yang <yangx.jy@fujitsu.com> | |
| 13334 | 13341 | L: ltp@lists.linux.it (subscribers-only) |
| 13335 | 13342 | S: Maintained |
| 13336 | −W: http://linux-test-project.github.io/ | |
| 13343 | +W: https://linux-test-project.readthedocs.io/ | |
| 13337 | 13344 | T: git https://github.com/linux-test-project/ltp.git |
| 13338 | 13345 | |
| 13339 | 13346 | LTR390 AMBIENT/UV LIGHT SENSOR DRIVER |
| @@ -13541,7 +13548,7 @@ MARVELL GIGABIT ETHERNET DRIVERS (skge/sky2) · 201 unmodified lines | ||
| 13541 | 13548 | M: Mirko Lindner <mlindner@marvell.com> |
| 13542 | 13549 | M: Stephen Hemminger <stephen@networkplumber.org> |
| 13543 | 13550 | L: netdev@vger.kernel.org |
| 13544 | −S: Maintained | |
| 13551 | +S: Odd fixes | |
| 13545 | 13552 | F: drivers/net/ethernet/marvell/sk* |
| 13546 | 13553 | |
| 13547 | 13554 | MARVELL LIBERTAS WIRELESS DRIVER |
| @@ -15877,15 +15884,19 @@ F: drivers/net/ · 2329 unmodified lines | ||
| 15877 | 15884 | F: include/dt-bindings/net/ |
| 15878 | 15885 | F: include/linux/cn_proc.h |
| 15879 | 15886 | F: include/linux/etherdevice.h |
| 15887 | +F: include/linux/ethtool_netlink.h | |
| 15880 | 15888 | F: include/linux/fcdevice.h |
| 15881 | 15889 | F: include/linux/fddidevice.h |
| 15882 | 15890 | F: include/linux/hippidevice.h |
| 15883 | 15891 | F: include/linux/if_* |
| 15884 | 15892 | F: include/linux/inetdevice.h |
| 15885 | −F: include/linux/netdevice.h | |
| 15893 | +F: include/linux/netdev* | |
| 15894 | +F: include/linux/platform_data/wiznet.h | |
| 15886 | 15895 | F: include/uapi/linux/cn_proc.h |
| 15896 | +F: include/uapi/linux/ethtool_netlink.h | |
| 15887 | 15897 | F: include/uapi/linux/if_* |
| 15888 | −F: include/uapi/linux/netdevice.h | |
| 15898 | +F: include/uapi/linux/netdev* | |
| 15899 | +F: tools/testing/selftests/drivers/net/ | |
| 15889 | 15900 | X: drivers/net/wireless/ |
| 15890 | 15901 | |
| 15891 | 15902 | NETWORKING DRIVERS (WIRELESS) |
| @@ -15936,13 +15947,28 @@ F: include/linux/framer/framer-provider.h · 44 unmodified lines | ||
| 15936 | 15947 | F: include/linux/framer/framer.h |
| 15937 | 15948 | F: include/linux/in.h |
| 15938 | 15949 | F: include/linux/indirect_call_wrapper.h |
| 15950 | +F: include/linux/inet.h | |
| 15951 | +F: include/linux/inet_diag.h | |
| 15939 | 15952 | F: include/linux/net.h |
| 15940 | −F: include/linux/netdevice.h | |
| 15953 | +F: include/linux/netdev* | |
| 15954 | +F: include/linux/netlink.h | |
| 15955 | +F: include/linux/netpoll.h | |
| 15956 | +F: include/linux/rtnetlink.h | |
| 15957 | +F: include/linux/seq_file_net.h | |
| 15958 | +F: include/linux/skbuff* | |
| 15941 | 15959 | F: include/net/ |
| 15960 | +F: include/uapi/linux/genetlink.h | |
| 15961 | +F: include/uapi/linux/hsr_netlink.h | |
| 15942 | 15962 | F: include/uapi/linux/in.h |
| 15963 | +F: include/uapi/linux/inet_diag.h | |
| 15964 | +F: include/uapi/linux/nbd-netlink.h | |
| 15943 | 15965 | F: include/uapi/linux/net.h |
| 15944 | 15966 | F: include/uapi/linux/net_namespace.h |
| 15945 | −F: include/uapi/linux/netdevice.h | |
| 15967 | +F: include/uapi/linux/netconf.h | |
| 15968 | +F: include/uapi/linux/netdev* | |
| 15969 | +F: include/uapi/linux/netlink.h | |
| 15970 | +F: include/uapi/linux/netlink_diag.h | |
| 15971 | +F: include/uapi/linux/rtnetlink.h | |
| 15946 | 15972 | F: lib/net_utils.c |
| 15947 | 15973 | F: lib/random32.c |
| 15948 | 15974 | F: net/ |
| @@ -17414,6 +17440,7 @@ M: Roy Zang <roy.zang@nxp.com> · 1465 unmodified lines | ||
| 17414 | 17440 | L: linuxppc-dev@lists.ozlabs.org |
| 17415 | 17441 | L: linux-pci@vger.kernel.org |
| 17416 | 17442 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 17443 | +L: imx@lists.linux.dev | |
| 17417 | 17444 | S: Maintained |
| 17418 | 17445 | F: drivers/pci/controller/dwc/*layerscape* |
| 17419 | 17446 | |
| @@ -17440,6 +17467,7 @@ M: Richard Zhu <hongxing.zhu@nxp.com> · 20 unmodified lines | ||
| 17440 | 17467 | M: Lucas Stach <l.stach@pengutronix.de> |
| 17441 | 17468 | L: linux-pci@vger.kernel.org |
| 17442 | 17469 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 17470 | +L: imx@lists.linux.dev | |
| 17443 | 17471 | S: Maintained |
| 17444 | 17472 | F: Documentation/devicetree/bindings/pci/fsl,imx6q-pcie-common.yaml |
| 17445 | 17473 | F: Documentation/devicetree/bindings/pci/fsl,imx6q-pcie-ep.yaml |
| @@ -17618,6 +17646,7 @@ F: drivers/pci/controller/pci-xgene-msi.c · 172 unmodified lines | ||
| 17618 | 17646 | PCI NATIVE HOST BRIDGE AND ENDPOINT DRIVERS |
| 17619 | 17647 | M: Lorenzo Pieralisi <lpieralisi@kernel.org> |
| 17620 | 17648 | M: Krzysztof Wilczyński <kw@linux.com> |
| 17649 | +R: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org> | |
| 17621 | 17650 | R: Rob Herring <robh@kernel.org> |
| 17622 | 17651 | L: linux-pci@vger.kernel.org |
| 17623 | 17652 | S: Supported |
| @@ -18524,7 +18553,6 @@ F: drivers/crypto/intel/qat/ · 900 unmodified lines | ||
| 18524 | 18553 | |
| 18525 | 18554 | QCOM AUDIO (ASoC) DRIVERS |
| 18526 | 18555 | M: Srinivas Kandagatla <srinivas.kandagatla@linaro.org> |
| 18527 | −M: Banajit Goswami <bgoswami@quicinc.com> | |
| 18528 | 18556 | L: alsa-devel@alsa-project.org (moderated for non-subscribers) |
| 18529 | 18557 | L: linux-arm-msm@vger.kernel.org |
| 18530 | 18558 | S: Supported |
| @@ -18558,7 +18586,7 @@ F: drivers/usb/misc/qcom_eud.c · 27 unmodified lines | ||
| 18558 | 18586 | QCOM IPA DRIVER |
| 18559 | 18587 | M: Alex Elder <elder@kernel.org> |
| 18560 | 18588 | L: netdev@vger.kernel.org |
| 18561 | −S: Supported | |
| 18589 | +S: Maintained | |
| 18562 | 18590 | F: drivers/net/ipa/ |
| 18563 | 18591 | |
| 18564 | 18592 | QEMU MACHINE EMULATOR AND VIRTUALIZER SUPPORT |
| @@ -18773,7 +18801,7 @@ F: include/uapi/drm/qaic_accel.h · 208 unmodified lines | ||
| 18773 | 18801 | |
| 18774 | 18802 | QUALCOMM CORE POWER REDUCTION (CPR) AVS DRIVER |
| 18775 | 18803 | M: Bjorn Andersson <andersson@kernel.org> |
| 18776 | −M: Konrad Dybcio <konrad.dybcio@linaro.org> | |
| 18804 | +M: Konrad Dybcio <konradybcio@kernel.org> | |
| 18777 | 18805 | L: linux-pm@vger.kernel.org |
| 18778 | 18806 | L: linux-arm-msm@vger.kernel.org |
| 18779 | 18807 | S: Maintained |
| @@ -20353,6 +20381,7 @@ F: Documentation/devicetree/bindings/scsi/ · 1573 unmodified lines | ||
| 20353 | 20381 | F: drivers/scsi/ |
| 20354 | 20382 | F: drivers/ufs/ |
| 20355 | 20383 | F: include/scsi/ |
| 20384 | +F: include/uapi/scsi/ | |
| 20356 | 20385 | |
| 20357 | 20386 | SCSI TAPE DRIVER |
| 20358 | 20387 | M: Kai Mäkisara <Kai.Makisara@kolumbus.fi> |
| @@ -21053,6 +21082,7 @@ SOCKET TIMESTAMPING · 694 unmodified lines | ||
| 21053 | 21082 | M: Willem de Bruijn <willemdebruijn.kernel@gmail.com> |
| 21054 | 21083 | S: Maintained |
| 21055 | 21084 | F: Documentation/networking/timestamping.rst |
| 21085 | +F: include/linux/net_tstamp.h | |
| 21056 | 21086 | F: include/uapi/linux/net_tstamp.h |
| 21057 | 21087 | F: tools/testing/selftests/net/so_txtime.c |
| 21058 | 21088 | |
| @@ -23820,10 +23850,8 @@ F: drivers/media/usb/uvc/ · 2761 unmodified lines | ||
| 23820 | 23850 | F: include/uapi/linux/uvcvideo.h |
| 23821 | 23851 | |
| 23822 | 23852 | USB WEBCAM GADGET |
| 23823 | −M: Laurent Pinchart <laurent.pinchart@ideasonboard.com> | |
| 23824 | −M: Daniel Scally <dan.scally@ideasonboard.com> | |
| 23825 | 23853 | L: linux-usb@vger.kernel.org |
| 23826 | −S: Maintained | |
| 23854 | +S: Orphan | |
| 23827 | 23855 | F: drivers/usb/gadget/function/*uvc* |
| 23828 | 23856 | F: drivers/usb/gadget/legacy/webcam.c |
| 23829 | 23857 | F: include/uapi/linux/usb/g_uvc.h |
| @@ -2,7 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | VERSION = 6 |
| 3 | 3 | PATCHLEVEL = 11 |
| 4 | 4 | SUBLEVEL = 0 |
| 5 | −EXTRAVERSION = -rc1 | |
| 5 | +EXTRAVERSION = -rc6 | |
| 6 | 6 | NAME = Baby Opossum Posse |
| 7 | 7 | |
| 8 | 8 | # *DOCUMENTATION* |
| @@ -1963,7 +1963,7 @@ tags TAGS cscope gtags: FORCE · 1954 unmodified lines | ||
| 1963 | 1963 | # Protocol). |
| 1964 | 1964 | PHONY += rust-analyzer |
| 1965 | 1965 | rust-analyzer: |
| 1966 | − $(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh | |
| 1966 | + +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh | |
| 1967 | 1967 | $(Q)$(MAKE) $(build)=rust $@ |
| 1968 | 1968 | |
| 1969 | 1969 | # Script to generate missing namespace dependencies |
| @@ -1980,7 +1980,7 @@ nsdeps: modules · 10 unmodified lines | ||
| 1980 | 1980 | quiet_cmd_gen_compile_commands = GEN $@ |
| 1981 | 1981 | cmd_gen_compile_commands = $(PYTHON3) $< -a $(AR) -o $@ $(filter-out $<, $(real-prereqs)) |
| 1982 | 1982 | |
| 1983 | −$(extmod_prefix)compile_commands.json: scripts/clang-tools/gen_compile_commands.py \ | |
| 1983 | +$(extmod_prefix)compile_commands.json: $(srctree)/scripts/clang-tools/gen_compile_commands.py \ | |
| 1984 | 1984 | $(if $(KBUILD_EXTMOD),, vmlinux.a $(KBUILD_VMLINUX_LIBS)) \ |
| 1985 | 1985 | $(if $(CONFIG_MODULES), $(MODORDER)) FORCE |
| 1986 | 1986 | $(call if_changed,gen_compile_commands) |
| @@ -534,8 +534,10 @@ extern inline void writeq(u64 b, volatile void __iomem *addr) · 533 unmodified lines | ||
| 534 | 534 | |
| 535 | 535 | #define ioread16be(p) swab16(ioread16(p)) |
| 536 | 536 | #define ioread32be(p) swab32(ioread32(p)) |
| 537 | +#define ioread64be(p) swab64(ioread64(p)) | |
| 537 | 538 | #define iowrite16be(v,p) iowrite16(swab16(v), (p)) |
| 538 | 539 | #define iowrite32be(v,p) iowrite32(swab32(v), (p)) |
| 540 | +#define iowrite64be(v,p) iowrite64(swab64(v), (p)) | |
| 539 | 541 | |
| 540 | 542 | #define inb_p inb |
| 541 | 543 | #define inw_p inw |
| @@ -634,8 +636,6 @@ extern void outsl (unsigned long port, const void *src, unsigned long count); · 92 unmodified lines | ||
| 634 | 636 | */ |
| 635 | 637 | #define ioread64 ioread64 |
| 636 | 638 | #define iowrite64 iowrite64 |
| 637 | −#define ioread64be ioread64be | |
| 638 | −#define iowrite64be iowrite64be | |
| 639 | 639 | #define ioread8_rep ioread8_rep |
| 640 | 640 | #define ioread16_rep ioread16_rep |
| 641 | 641 | #define ioread32_rep ioread32_rep |
| @@ -87,6 +87,7 @@ config ARM · 86 unmodified lines | ||
| 87 | 87 | select HAVE_ARCH_PFN_VALID |
| 88 | 88 | select HAVE_ARCH_SECCOMP |
| 89 | 89 | select HAVE_ARCH_SECCOMP_FILTER if AEABI && !OABI_COMPAT |
| 90 | + select HAVE_ARCH_STACKLEAK | |
| 90 | 91 | select HAVE_ARCH_THREAD_STRUCT_WHITELIST |
| 91 | 92 | select HAVE_ARCH_TRACEHOOK |
| 92 | 93 | select HAVE_ARCH_TRANSPARENT_HUGEPAGE if ARM_LPAE |
| @@ -116,6 +117,7 @@ config ARM · 23 unmodified lines | ||
| 116 | 117 | select HAVE_KERNEL_XZ |
| 117 | 118 | select HAVE_KPROBES if !XIP_KERNEL && !CPU_ENDIAN_BE32 && !CPU_V7M |
| 118 | 119 | select HAVE_KRETPROBES if HAVE_KPROBES |
| 120 | + select HAVE_LD_DEAD_CODE_DATA_ELIMINATION | |
| 119 | 121 | select HAVE_MOD_ARCH_SPECIFIC |
| 120 | 122 | select HAVE_NMI |
| 121 | 123 | select HAVE_OPTPROBES if !THUMB2_KERNEL |
| @@ -736,7 +738,7 @@ config ARM_ERRATA_764319 · 614 unmodified lines | ||
| 736 | 738 | bool "ARM errata: Read to DBGPRSR and DBGOSLSR may generate Undefined instruction" |
| 737 | 739 | depends on CPU_V7 |
| 738 | 740 | help |
| 739 | − This option enables the workaround for the 764319 Cortex A-9 erratum. | |
| 741 | + This option enables the workaround for the 764319 Cortex-A9 erratum. | |
| 740 | 742 | CP14 read accesses to the DBGPRSR and DBGOSLSR registers generate an |
| 741 | 743 | unexpected Undefined Instruction exception when the DBGSWENABLE |
| 742 | 744 | external pin is set to 0, even when the CP14 accesses are performed |
| @@ -9,6 +9,7 @@ OBJS = · 8 unmodified lines | ||
| 9 | 9 | |
| 10 | 10 | HEAD = head.o |
| 11 | 11 | OBJS += misc.o decompress.o |
| 12 | +CFLAGS_decompress.o += $(DISABLE_STACKLEAK_PLUGIN) | |
| 12 | 13 | ifeq ($(CONFIG_DEBUG_UNCOMPRESS),y) |
| 13 | 14 | OBJS += debug.o |
| 14 | 15 | AFLAGS_head.o += -DDEBUG |
| @@ -125,7 +125,7 @@ SECTIONS · 124 unmodified lines | ||
| 125 | 125 | |
| 126 | 126 | . = BSS_START; |
| 127 | 127 | __bss_start = .; |
| 128 | − .bss : { *(.bss) } | |
| 128 | + .bss : { *(.bss .bss.*) } | |
| 129 | 129 | _end = .; |
| 130 | 130 | |
| 131 | 131 | . = ALIGN(8); /* the stack must be 64-bit aligned */ |
| @@ -157,7 +157,7 @@ · 156 unmodified lines | ||
| 157 | 157 | clocks = <&xtal24mhz>; |
| 158 | 158 | }; |
| 159 | 159 | |
| 160 | − pclk: clock-24000000 { | |
| 160 | + pclk: clock-pclk { | |
| 161 | 161 | #clock-cells = <0>; |
| 162 | 162 | compatible = "fixed-factor-clock"; |
| 163 | 163 | clock-div = <1>; |
| @@ -274,24 +274,24 @@ · 273 unmodified lines | ||
| 274 | 274 | |
| 275 | 275 | led@0 { |
| 276 | 276 | chan-name = "R"; |
| 277 | − led-cur = /bits/ 8 <0x20>; | |
| 278 | − max-cur = /bits/ 8 <0x60>; | |
| 277 | + led-cur = /bits/ 8 <0x6e>; | |
| 278 | + max-cur = /bits/ 8 <0xc8>; | |
| 279 | 279 | reg = <0>; |
| 280 | 280 | color = <LED_COLOR_ID_RED>; |
| 281 | 281 | }; |
| 282 | 282 | |
| 283 | 283 | led@1 { |
| 284 | 284 | chan-name = "G"; |
| 285 | − led-cur = /bits/ 8 <0x20>; | |
| 286 | − max-cur = /bits/ 8 <0x60>; | |
| 285 | + led-cur = /bits/ 8 <0xbe>; | |
| 286 | + max-cur = /bits/ 8 <0xc8>; | |
| 287 | 287 | reg = <1>; |
| 288 | 288 | color = <LED_COLOR_ID_GREEN>; |
| 289 | 289 | }; |
| 290 | 290 | |
| 291 | 291 | led@2 { |
| 292 | 292 | chan-name = "B"; |
| 293 | − led-cur = /bits/ 8 <0x20>; | |
| 294 | − max-cur = /bits/ 8 <0x60>; | |
| 293 | + led-cur = /bits/ 8 <0xbe>; | |
| 294 | + max-cur = /bits/ 8 <0xc8>; | |
| 295 | 295 | reg = <2>; |
| 296 | 296 | color = <LED_COLOR_ID_BLUE>; |
| 297 | 297 | }; |
| @@ -781,7 +781,7 @@ · 780 unmodified lines | ||
| 781 | 781 | |
| 782 | 782 | mount-matrix = "-1", "0", "0", |
| 783 | 783 | "0", "1", "0", |
| 784 | − "0", "0", "1"; | |
| 784 | + "0", "0", "-1"; | |
| 785 | 785 | }; |
| 786 | 786 | |
| 787 | 787 | cam1: camera@3e { |
| @@ -26,6 +26,13 @@ struct stackframe { · 25 unmodified lines | ||
| 26 | 26 | #endif |
| 27 | 27 | }; |
| 28 | 28 | |
| 29 | +static inline bool on_thread_stack(void) | |
| 30 | +{ | |
| 31 | + unsigned long delta = current_stack_pointer ^ (unsigned long)current->stack; | |
| 32 | + | |
| 33 | + return delta < THREAD_SIZE; | |
| 34 | +} | |
| 35 | + | |
| 29 | 36 | static __always_inline |
| 30 | 37 | void arm_get_current_stackframe(struct pt_regs *regs, struct stackframe *frame) |
| 31 | 38 | { |
| @@ -42,7 +42,7 @@ · 41 unmodified lines | ||
| 42 | 42 | #define PROC_INFO \ |
| 43 | 43 | . = ALIGN(4); \ |
| 44 | 44 | __proc_info_begin = .; \ |
| 45 | − *(.proc.info.init) \ | |
| 45 | + KEEP(*(.proc.info.init)) \ | |
| 46 | 46 | __proc_info_end = .; |
| 47 | 47 | |
| 48 | 48 | #define IDMAP_TEXT \ |
| @@ -1065,6 +1065,7 @@ vector_addrexcptn: · 1064 unmodified lines | ||
| 1065 | 1065 | .globl vector_fiq |
| 1066 | 1066 | |
| 1067 | 1067 | .section .vectors, "ax", %progbits |
| 1068 | + .reloc .text, R_ARM_NONE, . | |
| 1068 | 1069 | W(b) vector_rst |
| 1069 | 1070 | W(b) vector_und |
| 1070 | 1071 | ARM( .reloc ., R_ARM_LDR_PC_G0, .L__vector_swi ) |
| @@ -1078,6 +1079,7 @@ THUMB( .reloc ., R_ARM_THM_PC12, .L__vector_swi ) · 7 unmodified lines | ||
| 1078 | 1079 | |
| 1079 | 1080 | #ifdef CONFIG_HARDEN_BRANCH_HISTORY |
| 1080 | 1081 | .section .vectors.bhb.loop8, "ax", %progbits |
| 1082 | + .reloc .text, R_ARM_NONE, . | |
| 1081 | 1083 | W(b) vector_rst |
| 1082 | 1084 | W(b) vector_bhb_loop8_und |
| 1083 | 1085 | ARM( .reloc ., R_ARM_LDR_PC_G0, .L__vector_bhb_loop8_swi ) |
| @@ -1090,6 +1092,7 @@ THUMB( .reloc ., R_ARM_THM_PC12, .L__vector_bhb_loop8_swi ) · 6 unmodified lines | ||
| 1090 | 1092 | W(b) vector_bhb_loop8_fiq |
| 1091 | 1093 | |
| 1092 | 1094 | .section .vectors.bhb.bpiall, "ax", %progbits |
| 1095 | + .reloc .text, R_ARM_NONE, . | |
| 1093 | 1096 | W(b) vector_rst |
| 1094 | 1097 | W(b) vector_bhb_bpiall_und |
| 1095 | 1098 | ARM( .reloc ., R_ARM_LDR_PC_G0, .L__vector_bhb_bpiall_swi ) |
| @@ -119,6 +119,9 @@ no_work_pending: · 118 unmodified lines | ||
| 119 | 119 | |
| 120 | 120 | ct_user_enter save = 0 |
| 121 | 121 | |
| 122 | +#ifdef CONFIG_GCC_PLUGIN_STACKLEAK | |
| 123 | + bl stackleak_erase_on_task_stack | |
| 124 | +#endif | |
| 122 | 125 | restore_user_regs fast = 0, offset = 0 |
| 123 | 126 | ENDPROC(ret_to_user_from_irq) |
| 124 | 127 | ENDPROC(ret_to_user) |
| @@ -395,11 +395,6 @@ apply_relocate(Elf32_Shdr *sechdrs, const char *strtab, unsigned int symindex, · 394 unmodified lines | ||
| 395 | 395 | return 0; |
| 396 | 396 | } |
| 397 | 397 | |
| 398 | −struct mod_unwind_map { | |
| 399 | − const Elf_Shdr *unw_sec; | |
| 400 | − const Elf_Shdr *txt_sec; | |
| 401 | −}; | |
| 402 | − | |
| 403 | 398 | static const Elf_Shdr *find_mod_section(const Elf32_Ehdr *hdr, |
| 404 | 399 | const Elf_Shdr *sechdrs, const char *name) |
| 405 | 400 | { |
| @@ -85,8 +85,7 @@ static bool · 84 unmodified lines | ||
| 85 | 85 | callchain_trace(void *data, unsigned long pc) |
| 86 | 86 | { |
| 87 | 87 | struct perf_callchain_entry_ctx *entry = data; |
| 88 | − perf_callchain_store(entry, pc); | |
| 89 | − return true; | |
| 88 | + return perf_callchain_store(entry, pc) == 0; | |
| 90 | 89 | } |
| 91 | 90 | |
| 92 | 91 | void |
| @@ -63,7 +63,7 @@ SECTIONS · 62 unmodified lines | ||
| 63 | 63 | . = ALIGN(4); |
| 64 | 64 | __ex_table : AT(ADDR(__ex_table) - LOAD_OFFSET) { |
| 65 | 65 | __start___ex_table = .; |
| 66 | − ARM_MMU_KEEP(*(__ex_table)) | |
| 66 | + ARM_MMU_KEEP(KEEP(*(__ex_table))) | |
| 67 | 67 | __stop___ex_table = .; |
| 68 | 68 | } |
| 69 | 69 | |
| @@ -83,7 +83,7 @@ SECTIONS · 13 unmodified lines | ||
| 83 | 83 | } |
| 84 | 84 | .init.arch.info : { |
| 85 | 85 | __arch_info_begin = .; |
| 86 | − *(.arch.info.init) | |
| 86 | + KEEP(*(.arch.info.init)) | |
| 87 | 87 | __arch_info_end = .; |
| 88 | 88 | } |
| 89 | 89 | .init.tagtable : { |
| @@ -74,7 +74,7 @@ SECTIONS · 73 unmodified lines | ||
| 74 | 74 | . = ALIGN(4); |
| 75 | 75 | __ex_table : AT(ADDR(__ex_table) - LOAD_OFFSET) { |
| 76 | 76 | __start___ex_table = .; |
| 77 | − ARM_MMU_KEEP(*(__ex_table)) | |
| 77 | + ARM_MMU_KEEP(KEEP(*(__ex_table))) | |
| 78 | 78 | __stop___ex_table = .; |
| 79 | 79 | } |
| 80 | 80 | |
| @@ -99,7 +99,7 @@ SECTIONS · 18 unmodified lines | ||
| 99 | 99 | } |
| 100 | 100 | .init.arch.info : { |
| 101 | 101 | __arch_info_begin = .; |
| 102 | − *(.arch.info.init) | |
| 102 | + KEEP(*(.arch.info.init)) | |
| 103 | 103 | __arch_info_end = .; |
| 104 | 104 | } |
| 105 | 105 | .init.tagtable : { |
| @@ -116,7 +116,7 @@ SECTIONS · 10 unmodified lines | ||
| 116 | 116 | #endif |
| 117 | 117 | .init.pv_table : { |
| 118 | 118 | __pv_table_begin = .; |
| 119 | − *(.pv_table) | |
| 119 | + KEEP(*(.pv_table)) | |
| 120 | 120 | __pv_table_end = .; |
| 121 | 121 | } |
| 122 | 122 | |
| @@ -29,7 +29,7 @@ int alpine_cpu_wakeup(unsigned int phys_cpu, uint32_t phys_resume_addr) · 28 unmodified lines | ||
| 29 | 29 | /* |
| 30 | 30 | * Set CPU resume address - |
| 31 | 31 | * secure firmware running on boot will jump to this address |
| 32 | − * after setting proper CPU mode, and initialiing e.g. secure | |
| 32 | + * after setting proper CPU mode, and initializing e.g. secure | |
| 33 | 33 | * regs (the same mode all CPUs are booted to - usually HYP) |
| 34 | 34 | */ |
| 35 | 35 | writel(phys_resume_addr, |
| @@ -21,6 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | #include <linux/mtd/mtd.h> |
| 22 | 22 | #include <linux/mtd/partitions.h> |
| 23 | 23 | #include <linux/gpio/machine.h> |
| 24 | +#include <linux/gpio/property.h> | |
| 24 | 25 | #include <linux/gpio.h> |
| 25 | 26 | #include <linux/err.h> |
| 26 | 27 | #include <linux/clk.h> |
| @@ -40,6 +41,7 @@ · 13 unmodified lines | ||
| 40 | 41 | #include <linux/platform_data/mmc-pxamci.h> |
| 41 | 42 | #include "udc.h" |
| 42 | 43 | #include "gumstix.h" |
| 44 | +#include "devices.h" | |
| 43 | 45 | |
| 44 | 46 | #include "generic.h" |
| 45 | 47 | |
| @@ -99,8 +101,8 @@ static void __init gumstix_mmc_init(void) · 53 unmodified lines | ||
| 99 | 101 | } |
| 100 | 102 | #endif |
| 101 | 103 | |
| 102 | −#ifdef CONFIG_USB_PXA25X | |
| 103 | −static const struct property_entry spitz_mci_props[] __initconst = { | |
| 104 | +#if IS_ENABLED(CONFIG_USB_PXA25X) | |
| 105 | +static const struct property_entry gumstix_vbus_props[] __initconst = { | |
| 104 | 106 | PROPERTY_ENTRY_GPIO("vbus-gpios", &pxa2xx_gpiochip_node, |
| 105 | 107 | GPIO_GUMSTIX_USB_GPIOn, GPIO_ACTIVE_HIGH), |
| 106 | 108 | PROPERTY_ENTRY_GPIO("pullup-gpios", &pxa2xx_gpiochip_node, |
| @@ -109,8 +111,9 @@ static const struct property_entry spitz_mci_props[] __initconst = { · 2 unmodified lines | ||
| 109 | 111 | }; |
| 110 | 112 | |
| 111 | 113 | static const struct platform_device_info gumstix_gpio_vbus_info __initconst = { |
| 112 | − .name = "gpio-vbus", | |
| 113 | − .id = PLATFORM_DEVID_NONE, | |
| 114 | + .name = "gpio-vbus", | |
| 115 | + .id = PLATFORM_DEVID_NONE, | |
| 116 | + .properties = gumstix_vbus_props, | |
| 114 | 117 | }; |
| 115 | 118 | |
| 116 | 119 | static void __init gumstix_udc_init(void) |
| @@ -1109,7 +1109,7 @@ void ecard_remove_driver(struct ecard_driver *drv) · 1108 unmodified lines | ||
| 1109 | 1109 | driver_unregister(&drv->drv); |
| 1110 | 1110 | } |
| 1111 | 1111 | |
| 1112 | −static int ecard_match(struct device *_dev, struct device_driver *_drv) | |
| 1112 | +static int ecard_match(struct device *_dev, const struct device_driver *_drv) | |
| 1113 | 1113 | { |
| 1114 | 1114 | struct expansion_card *ec = ECARD_DEV(_dev); |
| 1115 | 1115 | struct ecard_driver *drv = ECARD_DRV(_drv); |
| @@ -17,7 +17,7 @@ void cpu_arm7tdmi_proc_init(void); · 16 unmodified lines | ||
| 17 | 17 | __ADDRESSABLE(cpu_arm7tdmi_proc_init); |
| 18 | 18 | void cpu_arm7tdmi_proc_fin(void); |
| 19 | 19 | __ADDRESSABLE(cpu_arm7tdmi_proc_fin); |
| 20 | −void cpu_arm7tdmi_reset(void); | |
| 20 | +void cpu_arm7tdmi_reset(unsigned long addr, bool hvc); | |
| 21 | 21 | __ADDRESSABLE(cpu_arm7tdmi_reset); |
| 22 | 22 | int cpu_arm7tdmi_do_idle(void); |
| 23 | 23 | __ADDRESSABLE(cpu_arm7tdmi_do_idle); |
| @@ -32,7 +32,7 @@ void cpu_arm720_proc_init(void); · 8 unmodified lines | ||
| 32 | 32 | __ADDRESSABLE(cpu_arm720_proc_init); |
| 33 | 33 | void cpu_arm720_proc_fin(void); |
| 34 | 34 | __ADDRESSABLE(cpu_arm720_proc_fin); |
| 35 | −void cpu_arm720_reset(void); | |
| 35 | +void cpu_arm720_reset(unsigned long addr, bool hvc); | |
| 36 | 36 | __ADDRESSABLE(cpu_arm720_reset); |
| 37 | 37 | int cpu_arm720_do_idle(void); |
| 38 | 38 | __ADDRESSABLE(cpu_arm720_do_idle); |
| @@ -49,7 +49,7 @@ void cpu_arm740_proc_init(void); · 10 unmodified lines | ||
| 49 | 49 | __ADDRESSABLE(cpu_arm740_proc_init); |
| 50 | 50 | void cpu_arm740_proc_fin(void); |
| 51 | 51 | __ADDRESSABLE(cpu_arm740_proc_fin); |
| 52 | −void cpu_arm740_reset(void); | |
| 52 | +void cpu_arm740_reset(unsigned long addr, bool hvc); | |
| 53 | 53 | __ADDRESSABLE(cpu_arm740_reset); |
| 54 | 54 | int cpu_arm740_do_idle(void); |
| 55 | 55 | __ADDRESSABLE(cpu_arm740_do_idle); |
| @@ -64,7 +64,7 @@ void cpu_arm9tdmi_proc_init(void); · 8 unmodified lines | ||
| 64 | 64 | __ADDRESSABLE(cpu_arm9tdmi_proc_init); |
| 65 | 65 | void cpu_arm9tdmi_proc_fin(void); |
| 66 | 66 | __ADDRESSABLE(cpu_arm9tdmi_proc_fin); |
| 67 | −void cpu_arm9tdmi_reset(void); | |
| 67 | +void cpu_arm9tdmi_reset(unsigned long addr, bool hvc); | |
| 68 | 68 | __ADDRESSABLE(cpu_arm9tdmi_reset); |
| 69 | 69 | int cpu_arm9tdmi_do_idle(void); |
| 70 | 70 | __ADDRESSABLE(cpu_arm9tdmi_do_idle); |
| @@ -79,7 +79,7 @@ void cpu_arm920_proc_init(void); · 8 unmodified lines | ||
| 79 | 79 | __ADDRESSABLE(cpu_arm920_proc_init); |
| 80 | 80 | void cpu_arm920_proc_fin(void); |
| 81 | 81 | __ADDRESSABLE(cpu_arm920_proc_fin); |
| 82 | −void cpu_arm920_reset(void); | |
| 82 | +void cpu_arm920_reset(unsigned long addr, bool hvc); | |
| 83 | 83 | __ADDRESSABLE(cpu_arm920_reset); |
| 84 | 84 | int cpu_arm920_do_idle(void); |
| 85 | 85 | __ADDRESSABLE(cpu_arm920_do_idle); |
| @@ -102,7 +102,7 @@ void cpu_arm922_proc_init(void); · 16 unmodified lines | ||
| 102 | 102 | __ADDRESSABLE(cpu_arm922_proc_init); |
| 103 | 103 | void cpu_arm922_proc_fin(void); |
| 104 | 104 | __ADDRESSABLE(cpu_arm922_proc_fin); |
| 105 | −void cpu_arm922_reset(void); | |
| 105 | +void cpu_arm922_reset(unsigned long addr, bool hvc); | |
| 106 | 106 | __ADDRESSABLE(cpu_arm922_reset); |
| 107 | 107 | int cpu_arm922_do_idle(void); |
| 108 | 108 | __ADDRESSABLE(cpu_arm922_do_idle); |
| @@ -119,7 +119,7 @@ void cpu_arm925_proc_init(void); · 10 unmodified lines | ||
| 119 | 119 | __ADDRESSABLE(cpu_arm925_proc_init); |
| 120 | 120 | void cpu_arm925_proc_fin(void); |
| 121 | 121 | __ADDRESSABLE(cpu_arm925_proc_fin); |
| 122 | −void cpu_arm925_reset(void); | |
| 122 | +void cpu_arm925_reset(unsigned long addr, bool hvc); | |
| 123 | 123 | __ADDRESSABLE(cpu_arm925_reset); |
| 124 | 124 | int cpu_arm925_do_idle(void); |
| 125 | 125 | __ADDRESSABLE(cpu_arm925_do_idle); |
| @@ -159,7 +159,7 @@ void cpu_arm940_proc_init(void); · 33 unmodified lines | ||
| 159 | 159 | __ADDRESSABLE(cpu_arm940_proc_init); |
| 160 | 160 | void cpu_arm940_proc_fin(void); |
| 161 | 161 | __ADDRESSABLE(cpu_arm940_proc_fin); |
| 162 | −void cpu_arm940_reset(void); | |
| 162 | +void cpu_arm940_reset(unsigned long addr, bool hvc); | |
| 163 | 163 | __ADDRESSABLE(cpu_arm940_reset); |
| 164 | 164 | int cpu_arm940_do_idle(void); |
| 165 | 165 | __ADDRESSABLE(cpu_arm940_do_idle); |
| @@ -174,7 +174,7 @@ void cpu_arm946_proc_init(void); · 8 unmodified lines | ||
| 174 | 174 | __ADDRESSABLE(cpu_arm946_proc_init); |
| 175 | 175 | void cpu_arm946_proc_fin(void); |
| 176 | 176 | __ADDRESSABLE(cpu_arm946_proc_fin); |
| 177 | −void cpu_arm946_reset(void); | |
| 177 | +void cpu_arm946_reset(unsigned long addr, bool hvc); | |
| 178 | 178 | __ADDRESSABLE(cpu_arm946_reset); |
| 179 | 179 | int cpu_arm946_do_idle(void); |
| 180 | 180 | __ADDRESSABLE(cpu_arm946_do_idle); |
| @@ -429,7 +429,7 @@ void cpu_v7_proc_init(void); · 248 unmodified lines | ||
| 429 | 429 | __ADDRESSABLE(cpu_v7_proc_init); |
| 430 | 430 | void cpu_v7_proc_fin(void); |
| 431 | 431 | __ADDRESSABLE(cpu_v7_proc_fin); |
| 432 | −void cpu_v7_reset(void); | |
| 432 | +void cpu_v7_reset(unsigned long addr, bool hvc); | |
| 433 | 433 | __ADDRESSABLE(cpu_v7_reset); |
| 434 | 434 | int cpu_v7_do_idle(void); |
| 435 | 435 | __ADDRESSABLE(cpu_v7_do_idle); |
| @@ -1069,18 +1069,28 @@ config ARM64_ERRATUM_3117295 · 1068 unmodified lines | ||
| 1069 | 1069 | If unsure, say Y. |
| 1070 | 1070 | |
| 1071 | 1071 | config ARM64_ERRATUM_3194386 |
| 1072 | − bool "Cortex-{A720,X4,X925}/Neoverse-V3: workaround for MSR SSBS not self-synchronizing" | |
| 1072 | + bool "Cortex-*/Neoverse-*: workaround for MSR SSBS not self-synchronizing" | |
| 1073 | 1073 | default y |
| 1074 | 1074 | help |
| 1075 | 1075 | This option adds the workaround for the following errata: |
| 1076 | 1076 | |
| 1077 | + * ARM Cortex-A76 erratum 3324349 | |
| 1078 | + * ARM Cortex-A77 erratum 3324348 | |
| 1079 | + * ARM Cortex-A78 erratum 3324344 | |
| 1080 | + * ARM Cortex-A78C erratum 3324346 | |
| 1081 | + * ARM Cortex-A78C erratum 3324347 | |
| 1077 | 1082 | * ARM Cortex-A710 erratam 3324338 |
| 1078 | 1083 | * ARM Cortex-A720 erratum 3456091 |
| 1084 | + * ARM Cortex-A725 erratum 3456106 | |
| 1085 | + * ARM Cortex-X1 erratum 3324344 | |
| 1086 | + * ARM Cortex-X1C erratum 3324346 | |
| 1079 | 1087 | * ARM Cortex-X2 erratum 3324338 |
| 1080 | 1088 | * ARM Cortex-X3 erratum 3324335 |
| 1081 | 1089 | * ARM Cortex-X4 erratum 3194386 |
| 1082 | 1090 | * ARM Cortex-X925 erratum 3324334 |
| 1091 | + * ARM Neoverse-N1 erratum 3324349 | |
| 1083 | 1092 | * ARM Neoverse N2 erratum 3324339 |
| 1093 | + * ARM Neoverse-V1 erratum 3324341 | |
| 1084 | 1094 | * ARM Neoverse V2 erratum 3324336 |
| 1085 | 1095 | * ARM Neoverse-V3 erratum 3312417 |
| 1086 | 1096 | |
| @@ -1088,11 +1098,11 @@ config ARM64_ERRATUM_3194386 · 1 unmodified line | ||
| 1088 | 1098 | subsequent speculative instructions, which may permit unexepected |
| 1089 | 1099 | speculative store bypassing. |
| 1090 | 1100 | |
| 1091 | − Work around this problem by placing a speculation barrier after | |
| 1092 | − kernel changes to SSBS. The presence of the SSBS special-purpose | |
| 1093 | − register is hidden from hwcaps and EL0 reads of ID_AA64PFR1_EL1, such | |
| 1094 | − that userspace will use the PR_SPEC_STORE_BYPASS prctl to change | |
| 1095 | − SSBS. | |
| 1101 | + Work around this problem by placing a Speculation Barrier (SB) or | |
| 1102 | + Instruction Synchronization Barrier (ISB) after kernel changes to | |
| 1103 | + SSBS. The presence of the SSBS special-purpose register is hidden | |
| 1104 | + from hwcaps and EL0 reads of ID_AA64PFR1_EL1, such that userspace | |
| 1105 | + will use the PR_SPEC_STORE_BYPASS prctl to change SSBS. | |
| 1096 | 1106 | |
| 1097 | 1107 | If unsure, say Y. |
| 1098 | 1108 | |
| @@ -175,7 +175,7 @@ · 174 unmodified lines | ||
| 175 | 175 | }; |
| 176 | 176 | }; |
| 177 | 177 | |
| 178 | − core-cluster-thermal { | |
| 178 | + cluster-thermal { | |
| 179 | 179 | polling-delay-passive = <1000>; |
| 180 | 180 | polling-delay = <5000>; |
| 181 | 181 | thermal-sensors = <&tmu 1>; |
| @@ -214,7 +214,7 @@ · 213 unmodified lines | ||
| 214 | 214 | }; |
| 215 | 215 | }; |
| 216 | 216 | |
| 217 | − core-cluster-thermal { | |
| 217 | + cluster-thermal { | |
| 218 | 218 | polling-delay-passive = <1000>; |
| 219 | 219 | polling-delay = <5000>; |
| 220 | 220 | thermal-sensors = <&tmu 3>; |
| @@ -182,7 +182,7 @@ · 181 unmodified lines | ||
| 182 | 182 | }; |
| 183 | 183 | }; |
| 184 | 184 | |
| 185 | − core-cluster-thermal { | |
| 185 | + cluster-thermal { | |
| 186 | 186 | polling-delay-passive = <1000>; |
| 187 | 187 | polling-delay = <5000>; |
| 188 | 188 | thermal-sensors = <&tmu 3>; |
| @@ -131,7 +131,7 @@ · 130 unmodified lines | ||
| 131 | 131 | }; |
| 132 | 132 | |
| 133 | 133 | thermal-zones { |
| 134 | − core-cluster-thermal { | |
| 134 | + cluster-thermal { | |
| 135 | 135 | polling-delay-passive = <1000>; |
| 136 | 136 | polling-delay = <5000>; |
| 137 | 137 | thermal-sensors = <&tmu 0>; |
| @@ -122,7 +122,7 @@ · 121 unmodified lines | ||
| 122 | 122 | }; |
| 123 | 123 | }; |
| 124 | 124 | |
| 125 | − core-cluster1-thermal { | |
| 125 | + cluster1-thermal { | |
| 126 | 126 | polling-delay-passive = <1000>; |
| 127 | 127 | polling-delay = <5000>; |
| 128 | 128 | thermal-sensors = <&tmu 4>; |
| @@ -151,7 +151,7 @@ · 22 unmodified lines | ||
| 151 | 151 | }; |
| 152 | 152 | }; |
| 153 | 153 | |
| 154 | − core-cluster2-thermal { | |
| 154 | + cluster2-thermal { | |
| 155 | 155 | polling-delay-passive = <1000>; |
| 156 | 156 | polling-delay = <5000>; |
| 157 | 157 | thermal-sensors = <&tmu 5>; |
| @@ -180,7 +180,7 @@ · 22 unmodified lines | ||
| 180 | 180 | }; |
| 181 | 181 | }; |
| 182 | 182 | |
| 183 | − core-cluster3-thermal { | |
| 183 | + cluster3-thermal { | |
| 184 | 184 | polling-delay-passive = <1000>; |
| 185 | 185 | polling-delay = <5000>; |
| 186 | 186 | thermal-sensors = <&tmu 6>; |
| @@ -209,7 +209,7 @@ · 22 unmodified lines | ||
| 209 | 209 | }; |
| 210 | 210 | }; |
| 211 | 211 | |
| 212 | − core-cluster4-thermal { | |
| 212 | + cluster4-thermal { | |
| 213 | 213 | polling-delay-passive = <1000>; |
| 214 | 214 | polling-delay = <5000>; |
| 215 | 215 | thermal-sensors = <&tmu 7>; |
| @@ -492,7 +492,7 @@ · 491 unmodified lines | ||
| 492 | 492 | }; |
| 493 | 493 | }; |
| 494 | 494 | |
| 495 | − ddr-cluster5-thermal { | |
| 495 | + ddr-ctrl5-thermal { | |
| 496 | 496 | polling-delay-passive = <1000>; |
| 497 | 497 | polling-delay = <5000>; |
| 498 | 498 | thermal-sensors = <&tmu 1>; |
| @@ -21,7 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | |
| 22 | 22 | &gpio3 { |
| 23 | 23 | pinctrl-names = "default"; |
| 24 | − pinctrcl-0 = <&pinctrl_gpio3_hog>; | |
| 24 | + pinctrl-0 = <&pinctrl_gpio3_hog>; | |
| 25 | 25 | |
| 26 | 26 | uart4_rs485_en { |
| 27 | 27 | gpio-hog; |
| @@ -22,7 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | |
| 23 | 23 | &gpio3 { |
| 24 | 24 | pinctrl-names = "default"; |
| 25 | − pinctrcl-0 = <&pinctrl_gpio3_hog>; | |
| 25 | + pinctrl-0 = <&pinctrl_gpio3_hog>; | |
| 26 | 26 | |
| 27 | 27 | uart4_rs485_en { |
| 28 | 28 | gpio-hog; |
| @@ -211,13 +211,12 @@ · 210 unmodified lines | ||
| 211 | 211 | |
| 212 | 212 | simple-audio-card,cpu { |
| 213 | 213 | sound-dai = <&sai3>; |
| 214 | + frame-master; | |
| 215 | + bitclock-master; | |
| 214 | 216 | }; |
| 215 | 217 | |
| 216 | 218 | simple-audio-card,codec { |
| 217 | 219 | sound-dai = <&wm8962>; |
| 218 | − clocks = <&clk IMX8MP_CLK_IPP_DO_CLKO1>; | |
| 219 | − frame-master; | |
| 220 | − bitclock-master; | |
| 221 | 220 | }; |
| 222 | 221 | }; |
| 223 | 222 | }; |
| @@ -507,10 +506,9 @@ · 283 unmodified lines | ||
| 507 | 506 | &sai3 { |
| 508 | 507 | pinctrl-names = "default"; |
| 509 | 508 | pinctrl-0 = <&pinctrl_sai3>; |
| 510 | − assigned-clocks = <&clk IMX8MP_CLK_SAI3>, | |
| 511 | − <&clk IMX8MP_AUDIO_PLL2> ; | |
| 512 | − assigned-clock-parents = <&clk IMX8MP_AUDIO_PLL2_OUT>; | |
| 513 | − assigned-clock-rates = <12288000>, <361267200>; | |
| 509 | + assigned-clocks = <&clk IMX8MP_CLK_SAI3>; | |
| 510 | + assigned-clock-parents = <&clk IMX8MP_AUDIO_PLL1_OUT>; | |
| 511 | + assigned-clock-rates = <12288000>; | |
| 514 | 512 | fsl,sai-mclk-direction-output; |
| 515 | 513 | status = "okay"; |
| 516 | 514 | }; |
| @@ -499,7 +499,7 @@ · 498 unmodified lines | ||
| 499 | 499 | pinctrl-0 = <&pinctrl_usdhc2_hs>, <&pinctrl_usdhc2_gpio>; |
| 500 | 500 | pinctrl-1 = <&pinctrl_usdhc2_uhs>, <&pinctrl_usdhc2_gpio>; |
| 501 | 501 | pinctrl-2 = <&pinctrl_usdhc2_uhs>, <&pinctrl_usdhc2_gpio>; |
| 502 | − cd-gpios = <&gpio3 00 GPIO_ACTIVE_LOW>; | |
| 502 | + cd-gpios = <&gpio3 0 GPIO_ACTIVE_LOW>; | |
| 503 | 503 | vmmc-supply = <®_usdhc2_vmmc>; |
| 504 | 504 | bus-width = <4>; |
| 505 | 505 | no-sdio; |
| @@ -19,7 +19,7 @@ · 18 unmodified lines | ||
| 19 | 19 | linux,cma { |
| 20 | 20 | compatible = "shared-dma-pool"; |
| 21 | 21 | reusable; |
| 22 | − alloc-ranges = <0 0x60000000 0 0x40000000>; | |
| 22 | + alloc-ranges = <0 0x80000000 0 0x40000000>; | |
| 23 | 23 | size = <0 0x10000000>; |
| 24 | 24 | linux,cma-default; |
| 25 | 25 | }; |
| @@ -156,6 +156,7 @@ · 130 unmodified lines | ||
| 156 | 156 | &wdog3 { |
| 157 | 157 | pinctrl-names = "default"; |
| 158 | 158 | pinctrl-0 = <&pinctrl_wdog>; |
| 159 | + fsl,ext-reset-output; | |
| 159 | 160 | status = "okay"; |
| 160 | 161 | }; |
| 161 | 162 | |
| @@ -1105,7 +1105,7 @@ · 1104 unmodified lines | ||
| 1105 | 1105 | <&clk IMX93_CLK_SYS_PLL_PFD0_DIV2>; |
| 1106 | 1106 | assigned-clock-rates = <100000000>, <250000000>; |
| 1107 | 1107 | intf_mode = <&wakeupmix_gpr 0x28>; |
| 1108 | − snps,clk-csr = <0>; | |
| 1108 | + snps,clk-csr = <6>; | |
| 1109 | 1109 | nvmem-cells = <ð_mac2>; |
| 1110 | 1110 | nvmem-cell-names = "mac-address"; |
| 1111 | 1111 | status = "disabled"; |
| @@ -27,7 +27,7 @@ · 26 unmodified lines | ||
| 27 | 27 | reg = <0x0>; |
| 28 | 28 | enable-method = "psci"; |
| 29 | 29 | #cooling-cells = <2>; |
| 30 | − power-domains = <&scmi_devpd IMX95_PERF_A55>; | |
| 30 | + power-domains = <&scmi_perf IMX95_PERF_A55>; | |
| 31 | 31 | power-domain-names = "perf"; |
| 32 | 32 | i-cache-size = <32768>; |
| 33 | 33 | i-cache-line-size = <64>; |
| @@ -44,7 +44,7 @@ · 10 unmodified lines | ||
| 44 | 44 | reg = <0x100>; |
| 45 | 45 | enable-method = "psci"; |
| 46 | 46 | #cooling-cells = <2>; |
| 47 | − power-domains = <&scmi_devpd IMX95_PERF_A55>; | |
| 47 | + power-domains = <&scmi_perf IMX95_PERF_A55>; | |
| 48 | 48 | power-domain-names = "perf"; |
| 49 | 49 | i-cache-size = <32768>; |
| 50 | 50 | i-cache-line-size = <64>; |
| @@ -61,7 +61,7 @@ · 10 unmodified lines | ||
| 61 | 61 | reg = <0x200>; |
| 62 | 62 | enable-method = "psci"; |
| 63 | 63 | #cooling-cells = <2>; |
| 64 | − power-domains = <&scmi_devpd IMX95_PERF_A55>; | |
| 64 | + power-domains = <&scmi_perf IMX95_PERF_A55>; | |
| 65 | 65 | power-domain-names = "perf"; |
| 66 | 66 | i-cache-size = <32768>; |
| 67 | 67 | i-cache-line-size = <64>; |
| @@ -78,7 +78,7 @@ · 10 unmodified lines | ||
| 78 | 78 | reg = <0x300>; |
| 79 | 79 | enable-method = "psci"; |
| 80 | 80 | #cooling-cells = <2>; |
| 81 | − power-domains = <&scmi_devpd IMX95_PERF_A55>; | |
| 81 | + power-domains = <&scmi_perf IMX95_PERF_A55>; | |
| 82 | 82 | power-domain-names = "perf"; |
| 83 | 83 | i-cache-size = <32768>; |
| 84 | 84 | i-cache-line-size = <64>; |
| @@ -93,7 +93,7 @@ · 8 unmodified lines | ||
| 93 | 93 | device_type = "cpu"; |
| 94 | 94 | compatible = "arm,cortex-a55"; |
| 95 | 95 | reg = <0x400>; |
| 96 | − power-domains = <&scmi_devpd IMX95_PERF_A55>; | |
| 96 | + power-domains = <&scmi_perf IMX95_PERF_A55>; | |
| 97 | 97 | power-domain-names = "perf"; |
| 98 | 98 | enable-method = "psci"; |
| 99 | 99 | #cooling-cells = <2>; |
| @@ -110,7 +110,7 @@ · 10 unmodified lines | ||
| 110 | 110 | device_type = "cpu"; |
| 111 | 111 | compatible = "arm,cortex-a55"; |
| 112 | 112 | reg = <0x500>; |
| 113 | − power-domains = <&scmi_devpd IMX95_PERF_A55>; | |
| 113 | + power-domains = <&scmi_perf IMX95_PERF_A55>; | |
| 114 | 114 | power-domain-names = "perf"; |
| 115 | 115 | enable-method = "psci"; |
| 116 | 116 | #cooling-cells = <2>; |
| @@ -187,7 +187,7 @@ · 70 unmodified lines | ||
| 187 | 187 | compatible = "cache"; |
| 188 | 188 | cache-size = <524288>; |
| 189 | 189 | cache-line-size = <64>; |
| 190 | − cache-sets = <1024>; | |
| 190 | + cache-sets = <512>; | |
| 191 | 191 | cache-level = <3>; |
| 192 | 192 | cache-unified; |
| 193 | 193 | }; |
| @@ -320,8 +320,8 @@ · 319 unmodified lines | ||
| 320 | 320 | reg = <0x08af8800 0x400>; |
| 321 | 321 | |
| 322 | 322 | interrupts = <GIC_SPI 62 IRQ_TYPE_LEVEL_HIGH>, |
| 323 | − <GIC_SPI 53 IRQ_TYPE_EDGE_BOTH>, | |
| 324 | − <GIC_SPI 52 IRQ_TYPE_EDGE_BOTH>; | |
| 323 | + <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>, | |
| 324 | + <GIC_SPI 52 IRQ_TYPE_LEVEL_HIGH>; | |
| 325 | 325 | interrupt-names = "pwr_event", |
| 326 | 326 | "dp_hs_phy_irq", |
| 327 | 327 | "dm_hs_phy_irq"; |
| @@ -278,6 +278,13 @@ · 277 unmodified lines | ||
| 278 | 278 | vdd-l3-supply = <&vreg_s1f_0p7>; |
| 279 | 279 | vdd-s1-supply = <&vph_pwr>; |
| 280 | 280 | vdd-s2-supply = <&vph_pwr>; |
| 281 | + | |
| 282 | + vreg_l3i_0p8: ldo3 { | |
| 283 | + regulator-name = "vreg_l3i_0p8"; | |
| 284 | + regulator-min-microvolt = <880000>; | |
| 285 | + regulator-max-microvolt = <920000>; | |
| 286 | + regulator-initial-mode = <RPMH_REGULATOR_MODE_HPM>; | |
| 287 | + }; | |
| 281 | 288 | }; |
| 282 | 289 | |
| 283 | 290 | regulators-7 { |
| @@ -423,11 +430,17 @@ · 139 unmodified lines | ||
| 423 | 430 | }; |
| 424 | 431 | |
| 425 | 432 | &pcie4 { |
| 433 | + perst-gpios = <&tlmm 146 GPIO_ACTIVE_LOW>; | |
| 434 | + wake-gpios = <&tlmm 148 GPIO_ACTIVE_LOW>; | |
| 435 | + | |
| 436 | + pinctrl-0 = <&pcie4_default>; | |
| 437 | + pinctrl-names = "default"; | |
| 438 | + | |
| 426 | 439 | status = "okay"; |
| 427 | 440 | }; |
| 428 | 441 | |
| 429 | 442 | &pcie4_phy { |
| 430 | − vdda-phy-supply = <&vreg_l3j_0p8>; | |
| 443 | + vdda-phy-supply = <&vreg_l3i_0p8>; | |
| 431 | 444 | vdda-pll-supply = <&vreg_l3e_1p2>; |
| 432 | 445 | |
| 433 | 446 | status = "okay"; |
| @@ -517,7 +530,30 @@ · 83 unmodified lines | ||
| 517 | 530 | bias-disable; |
| 518 | 531 | }; |
| 519 | 532 | |
| 520 | − pcie6a_default: pcie2a-default-state { | |
| 533 | + pcie4_default: pcie4-default-state { | |
| 534 | + clkreq-n-pins { | |
| 535 | + pins = "gpio147"; | |
| 536 | + function = "pcie4_clk"; | |
| 537 | + drive-strength = <2>; | |
| 538 | + bias-pull-up; | |
| 539 | + }; | |
| 540 | + | |
| 541 | + perst-n-pins { | |
| 542 | + pins = "gpio146"; | |
| 543 | + function = "gpio"; | |
| 544 | + drive-strength = <2>; | |
| 545 | + bias-disable; | |
| 546 | + }; | |
| 547 | + | |
| 548 | + wake-n-pins { | |
| 549 | + pins = "gpio148"; | |
| 550 | + function = "gpio"; | |
| 551 | + drive-strength = <2>; | |
| 552 | + bias-pull-up; | |
| 553 | + }; | |
| 554 | + }; | |
| 555 | + | |
| 556 | + pcie6a_default: pcie6a-default-state { | |
| 521 | 557 | clkreq-n-pins { |
| 522 | 558 | pins = "gpio153"; |
| 523 | 559 | function = "pcie6a_clk"; |
| @@ -529,7 +565,7 @@ · 5 unmodified lines | ||
| 529 | 565 | pins = "gpio152"; |
| 530 | 566 | function = "gpio"; |
| 531 | 567 | drive-strength = <2>; |
| 532 | − bias-pull-down; | |
| 568 | + bias-disable; | |
| 533 | 569 | }; |
| 534 | 570 | |
| 535 | 571 | wake-n-pins { |
| @@ -268,7 +268,6 @@ · 267 unmodified lines | ||
| 268 | 268 | pinctrl-0 = <&edp_reg_en>; |
| 269 | 269 | pinctrl-names = "default"; |
| 270 | 270 | |
| 271 | − regulator-always-on; | |
| 272 | 271 | regulator-boot-on; |
| 273 | 272 | }; |
| 274 | 273 | |
| @@ -637,6 +636,14 @@ · 362 unmodified lines | ||
| 637 | 636 | }; |
| 638 | 637 | }; |
| 639 | 638 | |
| 639 | +&gpu { | |
| 640 | + status = "okay"; | |
| 641 | + | |
| 642 | + zap-shader { | |
| 643 | + firmware-name = "qcom/x1e80100/gen70500_zap.mbn"; | |
| 644 | + }; | |
| 645 | +}; | |
| 646 | + | |
| 640 | 647 | &i2c0 { |
| 641 | 648 | clock-frequency = <400000>; |
| 642 | 649 | |
| @@ -724,9 +731,13 @@ · 81 unmodified lines | ||
| 724 | 731 | |
| 725 | 732 | aux-bus { |
| 726 | 733 | panel { |
| 727 | − compatible = "edp-panel"; | |
| 734 | + compatible = "samsung,atna45af01", "samsung,atna33xc20"; | |
| 735 | + enable-gpios = <&pmc8380_3_gpios 4 GPIO_ACTIVE_HIGH>; | |
| 728 | 736 | power-supply = <&vreg_edp_3p3>; |
| 729 | 737 | |
| 738 | + pinctrl-0 = <&edp_bl_en>; | |
| 739 | + pinctrl-names = "default"; | |
| 740 | + | |
| 730 | 741 | port { |
| 731 | 742 | edp_panel_in: endpoint { |
| 732 | 743 | remote-endpoint = <&mdss_dp3_out>; |
| @@ -756,11 +767,17 @@ · 23 unmodified lines | ||
| 756 | 767 | }; |
| 757 | 768 | |
| 758 | 769 | &pcie4 { |
| 770 | + perst-gpios = <&tlmm 146 GPIO_ACTIVE_LOW>; | |
| 771 | + wake-gpios = <&tlmm 148 GPIO_ACTIVE_LOW>; | |
| 772 | + | |
| 773 | + pinctrl-0 = <&pcie4_default>; | |
| 774 | + pinctrl-names = "default"; | |
| 775 | + | |
| 759 | 776 | status = "okay"; |
| 760 | 777 | }; |
| 761 | 778 | |
| 762 | 779 | &pcie4_phy { |
| 763 | − vdda-phy-supply = <&vreg_l3j_0p8>; | |
| 780 | + vdda-phy-supply = <&vreg_l3i_0p8>; | |
| 764 | 781 | vdda-pll-supply = <&vreg_l3e_1p2>; |
| 765 | 782 | |
| 766 | 783 | status = "okay"; |
| @@ -785,6 +802,16 @@ · 18 unmodified lines | ||
| 785 | 802 | status = "okay"; |
| 786 | 803 | }; |
| 787 | 804 | |
| 805 | +&pmc8380_3_gpios { | |
| 806 | + edp_bl_en: edp-bl-en-state { | |
| 807 | + pins = "gpio4"; | |
| 808 | + function = "normal"; | |
| 809 | + power-source = <1>; /* 1.8V */ | |
| 810 | + input-disable; | |
| 811 | + output-enable; | |
| 812 | + }; | |
| 813 | +}; | |
| 814 | + | |
| 788 | 815 | &qupv3_0 { |
| 789 | 816 | status = "okay"; |
| 790 | 817 | }; |
| @@ -931,7 +958,30 @@ · 140 unmodified lines | ||
| 931 | 958 | bias-disable; |
| 932 | 959 | }; |
| 933 | 960 | |
| 934 | − pcie6a_default: pcie2a-default-state { | |
| 961 | + pcie4_default: pcie4-default-state { | |
| 962 | + clkreq-n-pins { | |
| 963 | + pins = "gpio147"; | |
| 964 | + function = "pcie4_clk"; | |
| 965 | + drive-strength = <2>; | |
| 966 | + bias-pull-up; | |
| 967 | + }; | |
| 968 | + | |
| 969 | + perst-n-pins { | |
| 970 | + pins = "gpio146"; | |
| 971 | + function = "gpio"; | |
| 972 | + drive-strength = <2>; | |
| 973 | + bias-disable; | |
| 974 | + }; | |
| 975 | + | |
| 976 | + wake-n-pins { | |
| 977 | + pins = "gpio148"; | |
| 978 | + function = "gpio"; | |
| 979 | + drive-strength = <2>; | |
| 980 | + bias-pull-up; | |
| 981 | + }; | |
| 982 | + }; | |
| 983 | + | |
| 984 | + pcie6a_default: pcie6a-default-state { | |
| 935 | 985 | clkreq-n-pins { |
| 936 | 986 | pins = "gpio153"; |
| 937 | 987 | function = "pcie6a_clk"; |
| @@ -943,15 +993,15 @@ · 5 unmodified lines | ||
| 943 | 993 | pins = "gpio152"; |
| 944 | 994 | function = "gpio"; |
| 945 | 995 | drive-strength = <2>; |
| 946 | − bias-pull-down; | |
| 996 | + bias-disable; | |
| 947 | 997 | }; |
| 948 | 998 | |
| 949 | 999 | wake-n-pins { |
| 950 | − pins = "gpio154"; | |
| 951 | − function = "gpio"; | |
| 952 | − drive-strength = <2>; | |
| 953 | − bias-pull-up; | |
| 954 | − }; | |
| 1000 | + pins = "gpio154"; | |
| 1001 | + function = "gpio"; | |
| 1002 | + drive-strength = <2>; | |
| 1003 | + bias-pull-up; | |
| 1004 | + }; | |
| 955 | 1005 | }; |
| 956 | 1006 | |
| 957 | 1007 | tpad_default: tpad-default-state { |
| @@ -625,16 +625,31 @@ · 624 unmodified lines | ||
| 625 | 625 | }; |
| 626 | 626 | |
| 627 | 627 | &pcie4 { |
| 628 | + perst-gpios = <&tlmm 146 GPIO_ACTIVE_LOW>; | |
| 629 | + wake-gpios = <&tlmm 148 GPIO_ACTIVE_LOW>; | |
| 630 | + | |
| 631 | + pinctrl-0 = <&pcie4_default>; | |
| 632 | + pinctrl-names = "default"; | |
| 633 | + | |
| 628 | 634 | status = "okay"; |
| 629 | 635 | }; |
| 630 | 636 | |
| 631 | 637 | &pcie4_phy { |
| 632 | − vdda-phy-supply = <&vreg_l3j_0p8>; | |
| 638 | + vdda-phy-supply = <&vreg_l3i_0p8>; | |
| 633 | 639 | vdda-pll-supply = <&vreg_l3e_1p2>; |
| 634 | 640 | |
| 635 | 641 | status = "okay"; |
| 636 | 642 | }; |
| 637 | 643 | |
| 644 | +&pcie4_port0 { | |
| 645 | + wifi@0 { | |
| 646 | + compatible = "pci17cb,1107"; | |
| 647 | + reg = <0x10000 0x0 0x0 0x0 0x0>; | |
| 648 | + | |
| 649 | + qcom,ath12k-calibration-variant = "LES790"; | |
| 650 | + }; | |
| 651 | +}; | |
| 652 | + | |
| 638 | 653 | &pcie6a { |
| 639 | 654 | perst-gpios = <&tlmm 152 GPIO_ACTIVE_LOW>; |
| 640 | 655 | wake-gpios = <&tlmm 154 GPIO_ACTIVE_LOW>; |
| @@ -782,7 +797,30 @@ · 141 unmodified lines | ||
| 782 | 797 | bias-disable; |
| 783 | 798 | }; |
| 784 | 799 | |
| 785 | − pcie6a_default: pcie2a-default-state { | |
| 800 | + pcie4_default: pcie4-default-state { | |
| 801 | + clkreq-n-pins { | |
| 802 | + pins = "gpio147"; | |
| 803 | + function = "pcie4_clk"; | |
| 804 | + drive-strength = <2>; | |
| 805 | + bias-pull-up; | |
| 806 | + }; | |
| 807 | + | |
| 808 | + perst-n-pins { | |
| 809 | + pins = "gpio146"; | |
| 810 | + function = "gpio"; | |
| 811 | + drive-strength = <2>; | |
| 812 | + bias-disable; | |
| 813 | + }; | |
| 814 | + | |
| 815 | + wake-n-pins { | |
| 816 | + pins = "gpio148"; | |
| 817 | + function = "gpio"; | |
| 818 | + drive-strength = <2>; | |
| 819 | + bias-pull-up; | |
| 820 | + }; | |
| 821 | + }; | |
| 822 | + | |
| 823 | + pcie6a_default: pcie6a-default-state { | |
| 786 | 824 | clkreq-n-pins { |
| 787 | 825 | pins = "gpio153"; |
| 788 | 826 | function = "pcie6a_clk"; |
| @@ -794,15 +832,15 @@ · 5 unmodified lines | ||
| 794 | 832 | pins = "gpio152"; |
| 795 | 833 | function = "gpio"; |
| 796 | 834 | drive-strength = <2>; |
| 797 | − bias-pull-down; | |
| 835 | + bias-disable; | |
| 798 | 836 | }; |
| 799 | 837 | |
| 800 | 838 | wake-n-pins { |
| 801 | − pins = "gpio154"; | |
| 802 | − function = "gpio"; | |
| 803 | − drive-strength = <2>; | |
| 804 | − bias-pull-up; | |
| 805 | − }; | |
| 839 | + pins = "gpio154"; | |
| 840 | + function = "gpio"; | |
| 841 | + drive-strength = <2>; | |
| 842 | + bias-pull-up; | |
| 843 | + }; | |
| 806 | 844 | }; |
| 807 | 845 | |
| 808 | 846 | tpad_default: tpad-default-state { |
| @@ -606,6 +606,14 @@ · 605 unmodified lines | ||
| 606 | 606 | }; |
| 607 | 607 | }; |
| 608 | 608 | |
| 609 | +&gpu { | |
| 610 | + status = "okay"; | |
| 611 | + | |
| 612 | + zap-shader { | |
| 613 | + firmware-name = "qcom/x1e80100/gen70500_zap.mbn"; | |
| 614 | + }; | |
| 615 | +}; | |
| 616 | + | |
| 609 | 617 | &lpass_tlmm { |
| 610 | 618 | spkr_01_sd_n_active: spkr-01-sd-n-active-state { |
| 611 | 619 | pins = "gpio12"; |
| @@ -660,11 +668,17 @@ · 48 unmodified lines | ||
| 660 | 668 | }; |
| 661 | 669 | |
| 662 | 670 | &pcie4 { |
| 671 | + perst-gpios = <&tlmm 146 GPIO_ACTIVE_LOW>; | |
| 672 | + wake-gpios = <&tlmm 148 GPIO_ACTIVE_LOW>; | |
| 673 | + | |
| 674 | + pinctrl-0 = <&pcie4_default>; | |
| 675 | + pinctrl-names = "default"; | |
| 676 | + | |
| 663 | 677 | status = "okay"; |
| 664 | 678 | }; |
| 665 | 679 | |
| 666 | 680 | &pcie4_phy { |
| 667 | − vdda-phy-supply = <&vreg_l3j_0p8>; | |
| 681 | + vdda-phy-supply = <&vreg_l3i_0p8>; | |
| 668 | 682 | vdda-pll-supply = <&vreg_l3e_1p2>; |
| 669 | 683 | |
| 670 | 684 | status = "okay"; |
| @@ -804,7 +818,30 @@ · 133 unmodified lines | ||
| 804 | 818 | bias-disable; |
| 805 | 819 | }; |
| 806 | 820 | |
| 807 | − pcie6a_default: pcie2a-default-state { | |
| 821 | + pcie4_default: pcie4-default-state { | |
| 822 | + clkreq-n-pins { | |
| 823 | + pins = "gpio147"; | |
| 824 | + function = "pcie4_clk"; | |
| 825 | + drive-strength = <2>; | |
| 826 | + bias-pull-up; | |
| 827 | + }; | |
| 828 | + | |
| 829 | + perst-n-pins { | |
| 830 | + pins = "gpio146"; | |
| 831 | + function = "gpio"; | |
| 832 | + drive-strength = <2>; | |
| 833 | + bias-disable; | |
| 834 | + }; | |
| 835 | + | |
| 836 | + wake-n-pins { | |
| 837 | + pins = "gpio148"; | |
| 838 | + function = "gpio"; | |
| 839 | + drive-strength = <2>; | |
| 840 | + bias-pull-up; | |
| 841 | + }; | |
| 842 | + }; | |
| 843 | + | |
| 844 | + pcie6a_default: pcie6a-default-state { | |
| 808 | 845 | clkreq-n-pins { |
| 809 | 846 | pins = "gpio153"; |
| 810 | 847 | function = "pcie6a_clk"; |
| @@ -816,15 +853,15 @@ · 5 unmodified lines | ||
| 816 | 853 | pins = "gpio152"; |
| 817 | 854 | function = "gpio"; |
| 818 | 855 | drive-strength = <2>; |
| 819 | − bias-pull-down; | |
| 856 | + bias-disable; | |
| 820 | 857 | }; |
| 821 | 858 | |
| 822 | 859 | wake-n-pins { |
| 823 | − pins = "gpio154"; | |
| 824 | − function = "gpio"; | |
| 825 | − drive-strength = <2>; | |
| 826 | − bias-pull-up; | |
| 827 | − }; | |
| 860 | + pins = "gpio154"; | |
| 861 | + function = "gpio"; | |
| 862 | + drive-strength = <2>; | |
| 863 | + bias-pull-up; | |
| 864 | + }; | |
| 828 | 865 | }; |
| 829 | 866 | |
| 830 | 867 | wcd_default: wcd-reset-n-active-state { |
| @@ -2901,7 +2901,7 @@ · 2900 unmodified lines | ||
| 2901 | 2901 | |
| 2902 | 2902 | dma-coherent; |
| 2903 | 2903 | |
| 2904 | − linux,pci-domain = <7>; | |
| 2904 | + linux,pci-domain = <6>; | |
| 2905 | 2905 | num-lanes = <2>; |
| 2906 | 2906 | |
| 2907 | 2907 | interrupts = <GIC_SPI 773 IRQ_TYPE_LEVEL_HIGH>, |
| @@ -2959,6 +2959,7 @@ · 51 unmodified lines | ||
| 2959 | 2959 | "link_down"; |
| 2960 | 2960 | |
| 2961 | 2961 | power-domains = <&gcc GCC_PCIE_6A_GDSC>; |
| 2962 | + required-opps = <&rpmhpd_opp_nom>; | |
| 2962 | 2963 | |
| 2963 | 2964 | phys = <&pcie6a_phy>; |
| 2964 | 2965 | phy-names = "pciephy"; |
| @@ -3022,7 +3023,7 @@ · 57 unmodified lines | ||
| 3022 | 3023 | |
| 3023 | 3024 | dma-coherent; |
| 3024 | 3025 | |
| 3025 | − linux,pci-domain = <5>; | |
| 3026 | + linux,pci-domain = <4>; | |
| 3026 | 3027 | num-lanes = <2>; |
| 3027 | 3028 | |
| 3028 | 3029 | interrupts = <GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>, |
| @@ -3080,11 +3081,22 @@ · 51 unmodified lines | ||
| 3080 | 3081 | "link_down"; |
| 3081 | 3082 | |
| 3082 | 3083 | power-domains = <&gcc GCC_PCIE_4_GDSC>; |
| 3084 | + required-opps = <&rpmhpd_opp_nom>; | |
| 3083 | 3085 | |
| 3084 | 3086 | phys = <&pcie4_phy>; |
| 3085 | 3087 | phy-names = "pciephy"; |
| 3086 | 3088 | |
| 3087 | 3089 | status = "disabled"; |
| 3090 | + | |
| 3091 | + pcie4_port0: pcie@0 { | |
| 3092 | + device_type = "pci"; | |
| 3093 | + reg = <0x0 0x0 0x0 0x0 0x0>; | |
| 3094 | + bus-range = <0x01 0xff>; | |
| 3095 | + | |
| 3096 | + #address-cells = <3>; | |
| 3097 | + #size-cells = <2>; | |
| 3098 | + ranges; | |
| 3099 | + }; | |
| 3088 | 3100 | }; |
| 3089 | 3101 | |
| 3090 | 3102 | pcie4_phy: phy@1c0e000 { |
| @@ -3155,9 +3167,10 @@ · 64 unmodified lines | ||
| 3155 | 3167 | interconnects = <&gem_noc MASTER_GFX3D 0 &mc_virt SLAVE_EBI1 0>; |
| 3156 | 3168 | interconnect-names = "gfx-mem"; |
| 3157 | 3169 | |
| 3170 | + status = "disabled"; | |
| 3171 | + | |
| 3158 | 3172 | zap-shader { |
| 3159 | 3173 | memory-region = <&gpu_microcode_mem>; |
| 3160 | − firmware-name = "qcom/gen70500_zap.mbn"; | |
| 3161 | 3174 | }; |
| 3162 | 3175 | |
| 3163 | 3176 | gpu_opp_table: opp-table { |
| @@ -3288,7 +3301,7 @@ · 124 unmodified lines | ||
| 3288 | 3301 | reg = <0x0 0x03da0000 0x0 0x40000>; |
| 3289 | 3302 | #iommu-cells = <2>; |
| 3290 | 3303 | #global-interrupts = <1>; |
| 3291 | − interrupts = <GIC_SPI 673 IRQ_TYPE_LEVEL_HIGH>, | |
| 3304 | + interrupts = <GIC_SPI 674 IRQ_TYPE_LEVEL_HIGH>, | |
| 3292 | 3305 | <GIC_SPI 678 IRQ_TYPE_LEVEL_HIGH>, |
| 3293 | 3306 | <GIC_SPI 679 IRQ_TYPE_LEVEL_HIGH>, |
| 3294 | 3307 | <GIC_SPI 680 IRQ_TYPE_LEVEL_HIGH>, |
| @@ -43,15 +43,6 @@ · 42 unmodified lines | ||
| 43 | 43 | sound-dai = <&mcasp0>; |
| 44 | 44 | }; |
| 45 | 45 | }; |
| 46 | − | |
| 47 | − reg_usb_hub: regulator-usb-hub { | |
| 48 | − compatible = "regulator-fixed"; | |
| 49 | − enable-active-high; | |
| 50 | − /* Verdin CTRL_SLEEP_MOCI# (SODIMM 256) */ | |
| 51 | − gpio = <&main_gpio0 31 GPIO_ACTIVE_HIGH>; | |
| 52 | − regulator-boot-on; | |
| 53 | − regulator-name = "HUB_PWR_EN"; | |
| 54 | − }; | |
| 55 | 46 | }; |
| 56 | 47 | |
| 57 | 48 | /* Verdin ETHs */ |
| @@ -193,11 +184,6 @@ · 135 unmodified lines | ||
| 193 | 184 | status = "okay"; |
| 194 | 185 | }; |
| 195 | 186 | |
| 196 | −/* Do not force CTRL_SLEEP_MOCI# always enabled */ | |
| 197 | −®_force_sleep_moci { | |
| 198 | − status = "disabled"; | |
| 199 | −}; | |
| 200 | − | |
| 201 | 187 | /* Verdin SD_1 */ |
| 202 | 188 | &sdhci1 { |
| 203 | 189 | status = "okay"; |
| @@ -218,15 +204,7 @@ · 14 unmodified lines | ||
| 218 | 204 | }; |
| 219 | 205 | |
| 220 | 206 | &usb1 { |
| 221 | − #address-cells = <1>; | |
| 222 | − #size-cells = <0>; | |
| 223 | 207 | status = "okay"; |
| 224 | − | |
| 225 | − usb-hub@1 { | |
| 226 | − compatible = "usb424,2744"; | |
| 227 | − reg = <1>; | |
| 228 | − vdd-supply = <®_usb_hub>; | |
| 229 | − }; | |
| 230 | 208 | }; |
| 231 | 209 | |
| 232 | 210 | /* Verdin CTRL_WAKE1_MICO# */ |
| @@ -138,12 +138,6 @@ · 137 unmodified lines | ||
| 138 | 138 | vin-supply = <®_1v8>; |
| 139 | 139 | }; |
| 140 | 140 | |
| 141 | − /* | |
| 142 | − * By default we enable CTRL_SLEEP_MOCI#, this is required to have | |
| 143 | − * peripherals on the carrier board powered. | |
| 144 | − * If more granularity or power saving is required this can be disabled | |
| 145 | − * in the carrier board device tree files. | |
| 146 | − */ | |
| 147 | 141 | reg_force_sleep_moci: regulator-force-sleep-moci { |
| 148 | 142 | compatible = "regulator-fixed"; |
| 149 | 143 | enable-active-high; |
| @@ -146,6 +146,8 @@ · 145 unmodified lines | ||
| 146 | 146 | power-domains = <&k3_pds 79 TI_SCI_PD_EXCLUSIVE>; |
| 147 | 147 | clocks = <&k3_clks 79 0>; |
| 148 | 148 | clock-names = "gpio"; |
| 149 | + gpio-ranges = <&mcu_pmx0 0 0 21>, <&mcu_pmx0 21 23 1>, | |
| 150 | + <&mcu_pmx0 22 32 2>; | |
| 149 | 151 | }; |
| 150 | 152 | |
| 151 | 153 | mcu_rti0: watchdog@4880000 { |
| @@ -45,7 +45,8 @@ · 44 unmodified lines | ||
| 45 | 45 | &main_pmx0 { |
| 46 | 46 | pinctrl-single,gpio-range = |
| 47 | 47 | <&main_pmx0_range 0 32 PIN_GPIO_RANGE_IOPAD>, |
| 48 | − <&main_pmx0_range 33 92 PIN_GPIO_RANGE_IOPAD>, | |
| 48 | + <&main_pmx0_range 33 38 PIN_GPIO_RANGE_IOPAD>, | |
| 49 | + <&main_pmx0_range 72 22 PIN_GPIO_RANGE_IOPAD>, | |
| 49 | 50 | <&main_pmx0_range 137 5 PIN_GPIO_RANGE_IOPAD>, |
| 50 | 51 | <&main_pmx0_range 143 3 PIN_GPIO_RANGE_IOPAD>, |
| 51 | 52 | <&main_pmx0_range 149 2 PIN_GPIO_RANGE_IOPAD>; |
| @@ -193,7 +193,8 @@ · 192 unmodified lines | ||
| 193 | 193 | &main_pmx0 { |
| 194 | 194 | pinctrl-single,gpio-range = |
| 195 | 195 | <&main_pmx0_range 0 32 PIN_GPIO_RANGE_IOPAD>, |
| 196 | − <&main_pmx0_range 33 55 PIN_GPIO_RANGE_IOPAD>, | |
| 196 | + <&main_pmx0_range 33 38 PIN_GPIO_RANGE_IOPAD>, | |
| 197 | + <&main_pmx0_range 72 17 PIN_GPIO_RANGE_IOPAD>, | |
| 197 | 198 | <&main_pmx0_range 101 25 PIN_GPIO_RANGE_IOPAD>, |
| 198 | 199 | <&main_pmx0_range 137 5 PIN_GPIO_RANGE_IOPAD>, |
| 199 | 200 | <&main_pmx0_range 143 3 PIN_GPIO_RANGE_IOPAD>, |
| @@ -1262,6 +1262,14 @@ · 1261 unmodified lines | ||
| 1262 | 1262 | &serdes0 { |
| 1263 | 1263 | status = "okay"; |
| 1264 | 1264 | |
| 1265 | + serdes0_pcie1_link: phy@0 { | |
| 1266 | + reg = <0>; | |
| 1267 | + cdns,num-lanes = <2>; | |
| 1268 | + #phy-cells = <0>; | |
| 1269 | + cdns,phy-type = <PHY_TYPE_PCIE>; | |
| 1270 | + resets = <&serdes_wiz0 1>, <&serdes_wiz0 2>; | |
| 1271 | + }; | |
| 1272 | + | |
| 1265 | 1273 | serdes0_usb_link: phy@3 { |
| 1266 | 1274 | reg = <3>; |
| 1267 | 1275 | cdns,num-lanes = <1>; |
| @@ -1386,23 +1394,6 @@ · 118 unmodified lines | ||
| 1386 | 1394 | phys = <&transceiver3>; |
| 1387 | 1395 | }; |
| 1388 | 1396 | |
| 1389 | −&serdes0 { | |
| 1390 | − status = "okay"; | |
| 1391 | − | |
| 1392 | − serdes0_pcie1_link: phy@0 { | |
| 1393 | − reg = <0>; | |
| 1394 | − cdns,num-lanes = <4>; | |
| 1395 | − #phy-cells = <0>; | |
| 1396 | − cdns,phy-type = <PHY_TYPE_PCIE>; | |
| 1397 | − resets = <&serdes_wiz0 1>, <&serdes_wiz0 2>, | |
| 1398 | − <&serdes_wiz0 3>, <&serdes_wiz0 4>; | |
| 1399 | − }; | |
| 1400 | −}; | |
| 1401 | − | |
| 1402 | −&serdes_wiz0 { | |
| 1403 | − status = "okay"; | |
| 1404 | −}; | |
| 1405 | − | |
| 1406 | 1397 | &pcie1_rc { |
| 1407 | 1398 | status = "okay"; |
| 1408 | 1399 | num-lanes = <2>; |
| @@ -2755,7 +2755,7 @@ · 2754 unmodified lines | ||
| 2755 | 2755 | interrupts = <GIC_SPI 550 IRQ_TYPE_LEVEL_HIGH>, |
| 2756 | 2756 | <GIC_SPI 551 IRQ_TYPE_LEVEL_HIGH>; |
| 2757 | 2757 | interrupt-names = "tx", "rx"; |
| 2758 | − dmas = <&main_udmap 0xc500>, <&main_udmap 0x4500>; | |
| 2758 | + dmas = <&main_udmap 0xc403>, <&main_udmap 0x4403>; | |
| 2759 | 2759 | dma-names = "tx", "rx"; |
| 2760 | 2760 | clocks = <&k3_clks 268 0>; |
| 2761 | 2761 | clock-names = "fck"; |
| @@ -2773,7 +2773,7 @@ · 11 unmodified lines | ||
| 2773 | 2773 | interrupts = <GIC_SPI 552 IRQ_TYPE_LEVEL_HIGH>, |
| 2774 | 2774 | <GIC_SPI 553 IRQ_TYPE_LEVEL_HIGH>; |
| 2775 | 2775 | interrupt-names = "tx", "rx"; |
| 2776 | − dmas = <&main_udmap 0xc501>, <&main_udmap 0x4501>; | |
| 2776 | + dmas = <&main_udmap 0xc404>, <&main_udmap 0x4404>; | |
| 2777 | 2777 | dma-names = "tx", "rx"; |
| 2778 | 2778 | clocks = <&k3_clks 269 0>; |
| 2779 | 2779 | clock-names = "fck"; |
| @@ -887,6 +887,7 @@ CONFIG_DRM_PANEL_KHADAS_TS050=m · 886 unmodified lines | ||
| 887 | 887 | CONFIG_DRM_PANEL_MANTIX_MLAF057WE51=m |
| 888 | 888 | CONFIG_DRM_PANEL_NOVATEK_NT36672E=m |
| 889 | 889 | CONFIG_DRM_PANEL_RAYDIUM_RM67191=m |
| 890 | +CONFIG_DRM_PANEL_SAMSUNG_ATNA33XC20=m | |
| 890 | 891 | CONFIG_DRM_PANEL_SITRONIX_ST7703=m |
| 891 | 892 | CONFIG_DRM_PANEL_TRULY_NT35597_WQXGA=m |
| 892 | 893 | CONFIG_DRM_PANEL_VISIONOX_VTDR6130=m |
| @@ -86,12 +86,14 @@ · 85 unmodified lines | ||
| 86 | 86 | #define ARM_CPU_PART_CORTEX_X2 0xD48 |
| 87 | 87 | #define ARM_CPU_PART_NEOVERSE_N2 0xD49 |
| 88 | 88 | #define ARM_CPU_PART_CORTEX_A78C 0xD4B |
| 89 | +#define ARM_CPU_PART_CORTEX_X1C 0xD4C | |
| 89 | 90 | #define ARM_CPU_PART_CORTEX_X3 0xD4E |
| 90 | 91 | #define ARM_CPU_PART_NEOVERSE_V2 0xD4F |
| 91 | 92 | #define ARM_CPU_PART_CORTEX_A720 0xD81 |
| 92 | 93 | #define ARM_CPU_PART_CORTEX_X4 0xD82 |
| 93 | 94 | #define ARM_CPU_PART_NEOVERSE_V3 0xD84 |
| 94 | 95 | #define ARM_CPU_PART_CORTEX_X925 0xD85 |
| 96 | +#define ARM_CPU_PART_CORTEX_A725 0xD87 | |
| 95 | 97 | |
| 96 | 98 | #define APM_CPU_PART_XGENE 0x000 |
| 97 | 99 | #define APM_CPU_VAR_POTENZA 0x00 |
| @@ -165,12 +167,14 @@ · 67 unmodified lines | ||
| 165 | 167 | #define MIDR_CORTEX_X2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X2) |
| 166 | 168 | #define MIDR_NEOVERSE_N2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_N2) |
| 167 | 169 | #define MIDR_CORTEX_A78C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A78C) |
| 170 | +#define MIDR_CORTEX_X1C MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X1C) | |
| 168 | 171 | #define MIDR_CORTEX_X3 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X3) |
| 169 | 172 | #define MIDR_NEOVERSE_V2 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V2) |
| 170 | 173 | #define MIDR_CORTEX_A720 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A720) |
| 171 | 174 | #define MIDR_CORTEX_X4 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X4) |
| 172 | 175 | #define MIDR_NEOVERSE_V3 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_NEOVERSE_V3) |
| 173 | 176 | #define MIDR_CORTEX_X925 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_X925) |
| 177 | +#define MIDR_CORTEX_A725 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A725) | |
| 174 | 178 | #define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX) |
| 175 | 179 | #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX) |
| 176 | 180 | #define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX) |
| @@ -13,6 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <linux/types.h> |
| 14 | 14 | #include <asm/insn.h> |
| 15 | 15 | |
| 16 | +#define HAVE_JUMP_LABEL_BATCH | |
| 16 | 17 | #define JUMP_LABEL_NOP_SIZE AARCH64_INSN_SIZE |
| 17 | 18 | |
| 18 | 19 | #define JUMP_TABLE_ENTRY(key, label) \ |
| @@ -104,7 +104,7 @@ alternative_else_nop_endif · 103 unmodified lines | ||
| 104 | 104 | |
| 105 | 105 | #define __ptrauth_save_key(ctxt, key) \ |
| 106 | 106 | do { \ |
| 107 | − u64 __val; \ | |
| 107 | + u64 __val; \ | |
| 108 | 108 | __val = read_sysreg_s(SYS_ ## key ## KEYLO_EL1); \ |
| 109 | 109 | ctxt_sys_reg(ctxt, key ## KEYLO_EL1) = __val; \ |
| 110 | 110 | __val = read_sysreg_s(SYS_ ## key ## KEYHI_EL1); \ |
| @@ -188,7 +188,7 @@ static inline void __user *__uaccess_mask_ptr(const void __user *ptr) · 187 unmodified lines | ||
| 188 | 188 | #define __get_mem_asm(load, reg, x, addr, label, type) \ |
| 189 | 189 | asm_goto_output( \ |
| 190 | 190 | "1: " load " " reg "0, [%1]\n" \ |
| 191 | − _ASM_EXTABLE_##type##ACCESS_ERR(1b, %l2, %w0) \ | |
| 191 | + _ASM_EXTABLE_##type##ACCESS(1b, %l2) \ | |
| 192 | 192 | : "=r" (x) \ |
| 193 | 193 | : "r" (addr) : : label) |
| 194 | 194 | #else |
| @@ -1,6 +1,6 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0 |
| 2 | 2 | |
| 3 | 3 | syscall_abis_32 += |
| 4 | −syscall_abis_64 += renameat newstat rlimit memfd_secret | |
| 4 | +syscall_abis_64 += renameat rlimit memfd_secret | |
| 5 | 5 | |
| 6 | 6 | syscalltbl = arch/arm64/tools/syscall_%.tbl |
| @@ -27,7 +27,7 @@ · 26 unmodified lines | ||
| 27 | 27 | |
| 28 | 28 | #include <asm/numa.h> |
| 29 | 29 | |
| 30 | −static int acpi_early_node_map[NR_CPUS] __initdata = { NUMA_NO_NODE }; | |
| 30 | +static int acpi_early_node_map[NR_CPUS] __initdata = { [0 ... NR_CPUS - 1] = NUMA_NO_NODE }; | |
| 31 | 31 | |
| 32 | 32 | int __init acpi_numa_get_nid(unsigned int cpu) |
| 33 | 33 | { |
| @@ -434,15 +434,24 @@ static const struct midr_range erratum_spec_unpriv_load_list[] = { · 433 unmodified lines | ||
| 434 | 434 | |
| 435 | 435 | #ifdef CONFIG_ARM64_ERRATUM_3194386 |
| 436 | 436 | static const struct midr_range erratum_spec_ssbs_list[] = { |
| 437 | + MIDR_ALL_VERSIONS(MIDR_CORTEX_A76), | |
| 438 | + MIDR_ALL_VERSIONS(MIDR_CORTEX_A77), | |
| 439 | + MIDR_ALL_VERSIONS(MIDR_CORTEX_A78), | |
| 440 | + MIDR_ALL_VERSIONS(MIDR_CORTEX_A78C), | |
| 437 | 441 | MIDR_ALL_VERSIONS(MIDR_CORTEX_A710), |
| 438 | 442 | MIDR_ALL_VERSIONS(MIDR_CORTEX_A720), |
| 443 | + MIDR_ALL_VERSIONS(MIDR_CORTEX_A725), | |
| 444 | + MIDR_ALL_VERSIONS(MIDR_CORTEX_X1), | |
| 445 | + MIDR_ALL_VERSIONS(MIDR_CORTEX_X1C), | |
| 439 | 446 | MIDR_ALL_VERSIONS(MIDR_CORTEX_X2), |
| 440 | 447 | MIDR_ALL_VERSIONS(MIDR_CORTEX_X3), |
| 441 | 448 | MIDR_ALL_VERSIONS(MIDR_CORTEX_X4), |
| 442 | 449 | MIDR_ALL_VERSIONS(MIDR_CORTEX_X925), |
| 450 | + MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N1), | |
| 443 | 451 | MIDR_ALL_VERSIONS(MIDR_NEOVERSE_N2), |
| 444 | − MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V3), | |
| 452 | + MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V1), | |
| 445 | 453 | MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V2), |
| 454 | + MIDR_ALL_VERSIONS(MIDR_NEOVERSE_V3), | |
| 446 | 455 | {} |
| 447 | 456 | }; |
| 448 | 457 | #endif |
| @@ -7,11 +7,12 @@ · 6 unmodified lines | ||
| 7 | 7 | */ |
| 8 | 8 | #include <linux/kernel.h> |
| 9 | 9 | #include <linux/jump_label.h> |
| 10 | +#include <linux/smp.h> | |
| 10 | 11 | #include <asm/insn.h> |
| 11 | 12 | #include <asm/patching.h> |
| 12 | 13 | |
| 13 | −void arch_jump_label_transform(struct jump_entry *entry, | |
| 14 | − enum jump_label_type type) | |
| 14 | +bool arch_jump_label_transform_queue(struct jump_entry *entry, | |
| 15 | + enum jump_label_type type) | |
| 15 | 16 | { |
| 16 | 17 | void *addr = (void *)jump_entry_code(entry); |
| 17 | 18 | u32 insn; |
| @@ -25,4 +26,10 @@ void arch_jump_label_transform(struct jump_entry *entry, · 7 unmodified lines | ||
| 25 | 26 | } |
| 26 | 27 | |
| 27 | 28 | aarch64_insn_patch_text_nosync(addr, insn); |
| 29 | + return true; | |
| 30 | +} | |
| 31 | + | |
| 32 | +void arch_jump_label_transform_apply(void) | |
| 33 | +{ | |
| 34 | + kick_all_cpus_sync(); | |
| 28 | 35 | } |
| @@ -355,9 +355,6 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p) · 354 unmodified lines | ||
| 355 | 355 | smp_init_cpus(); |
| 356 | 356 | smp_build_mpidr_hash(); |
| 357 | 357 | |
| 358 | − /* Init percpu seeds for random tags after cpus are set up. */ | |
| 359 | − kasan_init_sw_tags(); | |
| 360 | − | |
| 361 | 358 | #ifdef CONFIG_ARM64_SW_TTBR0_PAN |
| 362 | 359 | /* |
| 363 | 360 | * Make sure init_thread_info.ttbr0 always generates translation |
| @@ -467,6 +467,8 @@ void __init smp_prepare_boot_cpu(void) · 466 unmodified lines | ||
| 467 | 467 | init_gic_priority_masking(); |
| 468 | 468 | |
| 469 | 469 | kasan_init_hw_tags(); |
| 470 | + /* Init percpu seeds for random tags after cpus are set up. */ | |
| 471 | + kasan_init_sw_tags(); | |
| 470 | 472 | } |
| 471 | 473 | |
| 472 | 474 | /* |
| @@ -19,6 +19,7 @@ if VIRTUALIZATION · 18 unmodified lines | ||
| 19 | 19 | |
| 20 | 20 | menuconfig KVM |
| 21 | 21 | bool "Kernel-based Virtual Machine (KVM) support" |
| 22 | + depends on AS_HAS_ARMV8_4 | |
| 22 | 23 | select KVM_COMMON |
| 23 | 24 | select KVM_GENERIC_HARDWARE_ENABLING |
| 24 | 25 | select KVM_GENERIC_MMU_NOTIFIER |
| @@ -10,6 +10,9 @@ include $(srctree)/virt/kvm/Makefile.kvm · 9 unmodified lines | ||
| 10 | 10 | obj-$(CONFIG_KVM) += kvm.o |
| 11 | 11 | obj-$(CONFIG_KVM) += hyp/ |
| 12 | 12 | |
| 13 | +CFLAGS_sys_regs.o += -Wno-override-init | |
| 14 | +CFLAGS_handle_exit.o += -Wno-override-init | |
| 15 | + | |
| 13 | 16 | kvm-y += arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.o \ |
| 14 | 17 | inject_fault.o va_layout.o handle_exit.o \ |
| 15 | 18 | guest.o debug.o reset.o sys_regs.o stacktrace.o \ |
| @@ -164,6 +164,7 @@ static int kvm_arm_default_max_vcpus(void) · 163 unmodified lines | ||
| 164 | 164 | /** |
| 165 | 165 | * kvm_arch_init_vm - initializes a VM data structure |
| 166 | 166 | * @kvm: pointer to the KVM struct |
| 167 | + * @type: kvm device type | |
| 167 | 168 | */ |
| 168 | 169 | int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) |
| 169 | 170 | { |
| @@ -521,10 +522,10 @@ void kvm_arch_vcpu_unblocking(struct kvm_vcpu *vcpu) · 351 unmodified lines | ||
| 521 | 522 | |
| 522 | 523 | static void vcpu_set_pauth_traps(struct kvm_vcpu *vcpu) |
| 523 | 524 | { |
| 524 | − if (vcpu_has_ptrauth(vcpu)) { | |
| 525 | + if (vcpu_has_ptrauth(vcpu) && !is_protected_kvm_enabled()) { | |
| 525 | 526 | /* |
| 526 | − * Either we're running running an L2 guest, and the API/APK | |
| 527 | − * bits come from L1's HCR_EL2, or API/APK are both set. | |
| 527 | + * Either we're running an L2 guest, and the API/APK bits come | |
| 528 | + * from L1's HCR_EL2, or API/APK are both set. | |
| 528 | 529 | */ |
| 529 | 530 | if (unlikely(vcpu_has_nv(vcpu) && !is_hyp_ctxt(vcpu))) { |
| 530 | 531 | u64 val; |
| @@ -541,16 +542,10 @@ static void vcpu_set_pauth_traps(struct kvm_vcpu *vcpu) · 10 unmodified lines | ||
| 541 | 542 | * Save the host keys if there is any chance for the guest |
| 542 | 543 | * to use pauth, as the entry code will reload the guest |
| 543 | 544 | * keys in that case. |
| 544 | − * Protected mode is the exception to that rule, as the | |
| 545 | − * entry into the EL2 code eagerly switch back and forth | |
| 546 | − * between host and hyp keys (and kvm_hyp_ctxt is out of | |
| 547 | − * reach anyway). | |
| 548 | 545 | */ |
| 549 | − if (is_protected_kvm_enabled()) | |
| 550 | − return; | |
| 551 | − | |
| 552 | 546 | if (vcpu->arch.hcr_el2 & (HCR_API | HCR_APK)) { |
| 553 | 547 | struct kvm_cpu_context *ctxt; |
| 548 | + | |
| 554 | 549 | ctxt = this_cpu_ptr_hyp_sym(kvm_hyp_ctxt); |
| 555 | 550 | ptrauth_save_keys(ctxt); |
| 556 | 551 | } |
| @@ -27,7 +27,6 @@ · 26 unmodified lines | ||
| 27 | 27 | #include <asm/kvm_hyp.h> |
| 28 | 28 | #include <asm/kvm_mmu.h> |
| 29 | 29 | #include <asm/kvm_nested.h> |
| 30 | −#include <asm/kvm_ptrauth.h> | |
| 31 | 30 | #include <asm/fpsimd.h> |
| 32 | 31 | #include <asm/debug-monitors.h> |
| 33 | 32 | #include <asm/processor.h> |
| @@ -20,6 +20,8 @@ HOST_EXTRACFLAGS += -I$(objtree)/include · 19 unmodified lines | ||
| 20 | 20 | lib-objs := clear_page.o copy_page.o memcpy.o memset.o |
| 21 | 21 | lib-objs := $(addprefix ../../../lib/, $(lib-objs)) |
| 22 | 22 | |
| 23 | +CFLAGS_switch.nvhe.o += -Wno-override-init | |
| 24 | + | |
| 23 | 25 | hyp-obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o host.o \ |
| 24 | 26 | hyp-main.o hyp-smp.o psci-relay.o early_alloc.o page_alloc.o \ |
| 25 | 27 | cache.o setup.o mm.o mem_protect.o sys_regs.o pkvm.o stacktrace.o ffa.o |
| @@ -173,9 +173,8 @@ static void __pmu_switch_to_host(struct kvm_vcpu *vcpu) · 172 unmodified lines | ||
| 173 | 173 | static bool kvm_handle_pvm_sys64(struct kvm_vcpu *vcpu, u64 *exit_code) |
| 174 | 174 | { |
| 175 | 175 | /* |
| 176 | − * Make sure we handle the exit for workarounds and ptrauth | |
| 177 | − * before the pKVM handling, as the latter could decide to | |
| 178 | − * UNDEF. | |
| 176 | + * Make sure we handle the exit for workarounds before the pKVM | |
| 177 | + * handling, as the latter could decide to UNDEF. | |
| 179 | 178 | */ |
| 180 | 179 | return (kvm_hyp_handle_sysreg(vcpu, exit_code) || |
| 181 | 180 | kvm_handle_pvm_sysreg(vcpu, exit_code)); |
| @@ -6,6 +6,8 @@ · 5 unmodified lines | ||
| 6 | 6 | asflags-y := -D__KVM_VHE_HYPERVISOR__ |
| 7 | 7 | ccflags-y := -D__KVM_VHE_HYPERVISOR__ |
| 8 | 8 | |
| 9 | +CFLAGS_switch.o += -Wno-override-init | |
| 10 | + | |
| 9 | 11 | obj-y := timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o |
| 10 | 12 | obj-y += ../vgic-v3-sr.o ../aarch32.o ../vgic-v2-cpuif-proxy.o ../entry.o \ |
| 11 | 13 | ../fpsimd.o ../hyp-entry.o ../exception.o |
| @@ -1540,8 +1540,15 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, · 1539 unmodified lines | ||
| 1540 | 1540 | vma_pagesize = min(vma_pagesize, (long)max_map_size); |
| 1541 | 1541 | } |
| 1542 | 1542 | |
| 1543 | − if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE) | |
| 1543 | + /* | |
| 1544 | + * Both the canonical IPA and fault IPA must be hugepage-aligned to | |
| 1545 | + * ensure we find the right PFN and lay down the mapping in the right | |
| 1546 | + * place. | |
| 1547 | + */ | |
| 1548 | + if (vma_pagesize == PMD_SIZE || vma_pagesize == PUD_SIZE) { | |
| 1544 | 1549 | fault_ipa &= ~(vma_pagesize - 1); |
| 1550 | + ipa &= ~(vma_pagesize - 1); | |
| 1551 | + } | |
| 1545 | 1552 | |
| 1546 | 1553 | gfn = ipa >> PAGE_SHIFT; |
| 1547 | 1554 | mte_allowed = kvm_vma_mte_allowed(vma); |
| @@ -786,7 +786,7 @@ void kvm_arch_flush_shadow_all(struct kvm *kvm) · 785 unmodified lines | ||
| 786 | 786 | if (!WARN_ON(atomic_read(&mmu->refcnt))) |
| 787 | 787 | kvm_free_stage2_pgd(mmu); |
| 788 | 788 | } |
| 789 | − kfree(kvm->arch.nested_mmus); | |
| 789 | + kvfree(kvm->arch.nested_mmus); | |
| 790 | 790 | kvm->arch.nested_mmus = NULL; |
| 791 | 791 | kvm->arch.nested_mmus_size = 0; |
| 792 | 792 | kvm_uninit_stage2_mmu(kvm); |
| @@ -33,6 +33,7 @@ · 32 unmodified lines | ||
| 33 | 33 | #include <trace/events/kvm.h> |
| 34 | 34 | |
| 35 | 35 | #include "sys_regs.h" |
| 36 | +#include "vgic/vgic.h" | |
| 36 | 37 | |
| 37 | 38 | #include "trace.h" |
| 38 | 39 | |
| @@ -435,6 +436,11 @@ static bool access_gic_sgi(struct kvm_vcpu *vcpu, · 396 unmodified lines | ||
| 435 | 436 | { |
| 436 | 437 | bool g1; |
| 437 | 438 | |
| 439 | + if (!kvm_has_gicv3(vcpu->kvm)) { | |
| 440 | + kvm_inject_undefined(vcpu); | |
| 441 | + return false; | |
| 442 | + } | |
| 443 | + | |
| 438 | 444 | if (!p->is_write) |
| 439 | 445 | return read_from_write_only(vcpu, p, r); |
| 440 | 446 | |
| @@ -45,7 +45,8 @@ static void iter_next(struct kvm *kvm, struct vgic_state_iter *iter) · 44 unmodified lines | ||
| 45 | 45 | * Let the xarray drive the iterator after the last SPI, as the iterator |
| 46 | 46 | * has exhausted the sequentially-allocated INTID space. |
| 47 | 47 | */ |
| 48 | − if (iter->intid >= (iter->nr_spis + VGIC_NR_PRIVATE_IRQS - 1)) { | |
| 48 | + if (iter->intid >= (iter->nr_spis + VGIC_NR_PRIVATE_IRQS - 1) && | |
| 49 | + iter->nr_lpis) { | |
| 49 | 50 | if (iter->lpi_idx < iter->nr_lpis) |
| 50 | 51 | xa_find_after(&dist->lpi_xa, &iter->intid, |
| 51 | 52 | VGIC_LPI_MAX_INTID, |
| @@ -84,7 +85,7 @@ static void iter_unmark_lpis(struct kvm *kvm) · 32 unmodified lines | ||
| 84 | 85 | struct vgic_irq *irq; |
| 85 | 86 | unsigned long intid; |
| 86 | 87 | |
| 87 | − xa_for_each(&dist->lpi_xa, intid, irq) { | |
| 88 | + xa_for_each_marked(&dist->lpi_xa, intid, irq, LPI_XA_MARK_DEBUG_ITER) { | |
| 88 | 89 | xa_clear_mark(&dist->lpi_xa, intid, LPI_XA_MARK_DEBUG_ITER); |
| 89 | 90 | vgic_put_irq(kvm, irq); |
| 90 | 91 | } |
| @@ -112,7 +113,7 @@ static bool end_of_vgic(struct vgic_state_iter *iter) · 21 unmodified lines | ||
| 112 | 113 | return iter->dist_id > 0 && |
| 113 | 114 | iter->vcpu_id == iter->nr_cpus && |
| 114 | 115 | iter->intid >= (iter->nr_spis + VGIC_NR_PRIVATE_IRQS) && |
| 115 | − iter->lpi_idx > iter->nr_lpis; | |
| 116 | + (!iter->nr_lpis || iter->lpi_idx > iter->nr_lpis); | |
| 116 | 117 | } |
| 117 | 118 | |
| 118 | 119 | static void *vgic_debug_start(struct seq_file *s, loff_t *pos) |
| @@ -417,10 +417,8 @@ static void __kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu) · 416 unmodified lines | ||
| 417 | 417 | kfree(vgic_cpu->private_irqs); |
| 418 | 418 | vgic_cpu->private_irqs = NULL; |
| 419 | 419 | |
| 420 | − if (vcpu->kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) { | |
| 421 | − vgic_unregister_redist_iodev(vcpu); | |
| 420 | + if (vcpu->kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) | |
| 422 | 421 | vgic_cpu->rd_iodev.base_addr = VGIC_ADDR_UNDEF; |
| 423 | − } | |
| 424 | 422 | } |
| 425 | 423 | |
| 426 | 424 | void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu) |
| @@ -438,17 +436,21 @@ void kvm_vgic_destroy(struct kvm *kvm) · 11 unmodified lines | ||
| 438 | 436 | unsigned long i; |
| 439 | 437 | |
| 440 | 438 | mutex_lock(&kvm->slots_lock); |
| 439 | + mutex_lock(&kvm->arch.config_lock); | |
| 441 | 440 | |
| 442 | 441 | vgic_debug_destroy(kvm); |
| 443 | 442 | |
| 444 | 443 | kvm_for_each_vcpu(i, vcpu, kvm) |
| 445 | 444 | __kvm_vgic_vcpu_destroy(vcpu); |
| 446 | 445 | |
| 447 | − mutex_lock(&kvm->arch.config_lock); | |
| 448 | − | |
| 449 | 446 | kvm_vgic_dist_destroy(kvm); |
| 450 | 447 | |
| 451 | 448 | mutex_unlock(&kvm->arch.config_lock); |
| 449 | + | |
| 450 | + if (kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3) | |
| 451 | + kvm_for_each_vcpu(i, vcpu, kvm) | |
| 452 | + vgic_unregister_redist_iodev(vcpu); | |
| 453 | + | |
| 452 | 454 | mutex_unlock(&kvm->slots_lock); |
| 453 | 455 | } |
| 454 | 456 | |
| @@ -9,7 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | #include <kvm/arm_vgic.h> |
| 10 | 10 | #include "vgic.h" |
| 11 | 11 | |
| 12 | −/** | |
| 12 | +/* | |
| 13 | 13 | * vgic_irqfd_set_irq: inject the IRQ corresponding to the |
| 14 | 14 | * irqchip routing entry |
| 15 | 15 | * |
| @@ -75,7 +75,8 @@ static void kvm_populate_msi(struct kvm_kernel_irq_routing_entry *e, · 59 unmodified lines | ||
| 75 | 75 | msi->flags = e->msi.flags; |
| 76 | 76 | msi->devid = e->msi.devid; |
| 77 | 77 | } |
| 78 | −/** | |
| 78 | + | |
| 79 | +/* | |
| 79 | 80 | * kvm_set_msi: inject the MSI corresponding to the |
| 80 | 81 | * MSI routing entry |
| 81 | 82 | * |
| @@ -98,7 +99,7 @@ int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e, · 16 unmodified lines | ||
| 98 | 99 | return vgic_its_inject_msi(kvm, &msi); |
| 99 | 100 | } |
| 100 | 101 | |
| 101 | −/** | |
| 102 | +/* | |
| 102 | 103 | * kvm_arch_set_irq_inatomic: fast-path for irqfd injection |
| 103 | 104 | */ |
| 104 | 105 | int kvm_arch_set_irq_inatomic(struct kvm_kernel_irq_routing_entry *e, |
| @@ -2040,6 +2040,7 @@ typedef int (*entry_fn_t)(struct vgic_its *its, u32 id, void *entry, · 2039 unmodified lines | ||
| 2040 | 2040 | * @start_id: the ID of the first entry in the table |
| 2041 | 2041 | * (non zero for 2d level tables) |
| 2042 | 2042 | * @fn: function to apply on each entry |
| 2043 | + * @opaque: pointer to opaque data | |
| 2043 | 2044 | * |
| 2044 | 2045 | * Return: < 0 on error, 0 if last element was identified, 1 otherwise |
| 2045 | 2046 | * (the last element may not be found on second level tables) |
| @@ -2079,7 +2080,7 @@ static int scan_its_table(struct vgic_its *its, gpa_t base, int size, u32 esz, · 33 unmodified lines | ||
| 2079 | 2080 | return 1; |
| 2080 | 2081 | } |
| 2081 | 2082 | |
| 2082 | −/** | |
| 2083 | +/* | |
| 2083 | 2084 | * vgic_its_save_ite - Save an interrupt translation entry at @gpa |
| 2084 | 2085 | */ |
| 2085 | 2086 | static int vgic_its_save_ite(struct vgic_its *its, struct its_device *dev, |
| @@ -2099,6 +2100,8 @@ static int vgic_its_save_ite(struct vgic_its *its, struct its_device *dev, · 13 unmodified lines | ||
| 2099 | 2100 | |
| 2100 | 2101 | /** |
| 2101 | 2102 | * vgic_its_restore_ite - restore an interrupt translation entry |
| 2103 | + * | |
| 2104 | + * @its: its handle | |
| 2102 | 2105 | * @event_id: id used for indexing |
| 2103 | 2106 | * @ptr: pointer to the ITE entry |
| 2104 | 2107 | * @opaque: pointer to the its_device |
| @@ -2231,6 +2234,7 @@ static int vgic_its_restore_itt(struct vgic_its *its, struct its_device *dev) · 126 unmodified lines | ||
| 2231 | 2234 | * @its: ITS handle |
| 2232 | 2235 | * @dev: ITS device |
| 2233 | 2236 | * @ptr: GPA |
| 2237 | + * @dte_esz: device table entry size | |
| 2234 | 2238 | */ |
| 2235 | 2239 | static int vgic_its_save_dte(struct vgic_its *its, struct its_device *dev, |
| 2236 | 2240 | gpa_t ptr, int dte_esz) |
| @@ -2313,7 +2317,7 @@ static int vgic_its_device_cmp(void *priv, const struct list_head *a, · 76 unmodified lines | ||
| 2313 | 2317 | return 1; |
| 2314 | 2318 | } |
| 2315 | 2319 | |
| 2316 | −/** | |
| 2320 | +/* | |
| 2317 | 2321 | * vgic_its_save_device_tables - Save the device table and all ITT |
| 2318 | 2322 | * into guest RAM |
| 2319 | 2323 | * |
| @@ -2386,7 +2390,7 @@ static int handle_l1_dte(struct vgic_its *its, u32 id, void *addr, · 66 unmodified lines | ||
| 2386 | 2390 | return ret; |
| 2387 | 2391 | } |
| 2388 | 2392 | |
| 2389 | −/** | |
| 2393 | +/* | |
| 2390 | 2394 | * vgic_its_restore_device_tables - Restore the device table and all ITT |
| 2391 | 2395 | * from guest RAM to internal data structs |
| 2392 | 2396 | */ |
| @@ -2478,7 +2482,7 @@ static int vgic_its_restore_cte(struct vgic_its *its, gpa_t gpa, int esz) · 85 unmodified lines | ||
| 2478 | 2482 | return 1; |
| 2479 | 2483 | } |
| 2480 | 2484 | |
| 2481 | −/** | |
| 2485 | +/* | |
| 2482 | 2486 | * vgic_its_save_collection_table - Save the collection table into |
| 2483 | 2487 | * guest RAM |
| 2484 | 2488 | */ |
| @@ -2518,7 +2522,7 @@ static int vgic_its_save_collection_table(struct vgic_its *its) · 33 unmodified lines | ||
| 2518 | 2522 | return ret; |
| 2519 | 2523 | } |
| 2520 | 2524 | |
| 2521 | −/** | |
| 2525 | +/* | |
| 2522 | 2526 | * vgic_its_restore_collection_table - reads the collection table |
| 2523 | 2527 | * in guest memory and restores the ITS internal state. Requires the |
| 2524 | 2528 | * BASER registers to be restored before. |
| @@ -2556,7 +2560,7 @@ static int vgic_its_restore_collection_table(struct vgic_its *its) · 31 unmodified lines | ||
| 2556 | 2560 | return ret; |
| 2557 | 2561 | } |
| 2558 | 2562 | |
| 2559 | −/** | |
| 2563 | +/* | |
| 2560 | 2564 | * vgic_its_save_tables_v0 - Save the ITS tables into guest ARM |
| 2561 | 2565 | * according to v0 ABI |
| 2562 | 2566 | */ |
| @@ -2571,7 +2575,7 @@ static int vgic_its_save_tables_v0(struct vgic_its *its) · 8 unmodified lines | ||
| 2571 | 2575 | return vgic_its_save_collection_table(its); |
| 2572 | 2576 | } |
| 2573 | 2577 | |
| 2574 | −/** | |
| 2578 | +/* | |
| 2575 | 2579 | * vgic_its_restore_tables_v0 - Restore the ITS tables from guest RAM |
| 2576 | 2580 | * to internal data structs according to V0 ABI |
| 2577 | 2581 | * |
| @@ -370,7 +370,7 @@ static void map_all_vpes(struct kvm *kvm) · 369 unmodified lines | ||
| 370 | 370 | dist->its_vm.vpes[i]->irq)); |
| 371 | 371 | } |
| 372 | 372 | |
| 373 | −/** | |
| 373 | +/* | |
| 374 | 374 | * vgic_v3_save_pending_tables - Save the pending tables into guest RAM |
| 375 | 375 | * kvm lock and all vcpu lock must be held |
| 376 | 376 | */ |
| @@ -36,6 +36,11 @@ struct vgic_global kvm_vgic_global_state __ro_after_init = { · 35 unmodified lines | ||
| 36 | 36 | * we have to disable IRQs before taking this lock and everything lower |
| 37 | 37 | * than it. |
| 38 | 38 | * |
| 39 | + * The config_lock has additional ordering requirements: | |
| 40 | + * kvm->slots_lock | |
| 41 | + * kvm->srcu | |
| 42 | + * kvm->arch.config_lock | |
| 43 | + * | |
| 39 | 44 | * If you need to take multiple locks, always take the upper lock first, |
| 40 | 45 | * then the lower ones, e.g. first take the its_lock, then the irq_lock. |
| 41 | 46 | * If you are already holding a lock and need to take a higher one, you |
| @@ -313,7 +318,7 @@ static bool vgic_validate_injection(struct vgic_irq *irq, bool level, void *owne · 271 unmodified lines | ||
| 313 | 318 | * with all locks dropped. |
| 314 | 319 | */ |
| 315 | 320 | bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq, |
| 316 | − unsigned long flags) | |
| 321 | + unsigned long flags) __releases(&irq->irq_lock) | |
| 317 | 322 | { |
| 318 | 323 | struct kvm_vcpu *vcpu; |
| 319 | 324 | |
| @@ -186,7 +186,7 @@ bool vgic_get_phys_line_level(struct vgic_irq *irq); · 185 unmodified lines | ||
| 186 | 186 | void vgic_irq_set_phys_pending(struct vgic_irq *irq, bool pending); |
| 187 | 187 | void vgic_irq_set_phys_active(struct vgic_irq *irq, bool active); |
| 188 | 188 | bool vgic_queue_irq_unlock(struct kvm *kvm, struct vgic_irq *irq, |
| 189 | − unsigned long flags); | |
| 189 | + unsigned long flags) __releases(&irq->irq_lock); | |
| 190 | 190 | void vgic_kick_vcpus(struct kvm *kvm); |
| 191 | 191 | void vgic_irq_handle_resampling(struct vgic_irq *irq, |
| 192 | 192 | bool lr_deactivated, bool lr_pending); |
| @@ -346,4 +346,11 @@ void vgic_v4_configure_vsgis(struct kvm *kvm); · 153 unmodified lines | ||
| 346 | 346 | void vgic_v4_get_vlpi_state(struct vgic_irq *irq, bool *val); |
| 347 | 347 | int vgic_v4_request_vpe_irq(struct kvm_vcpu *vcpu, int irq); |
| 348 | 348 | |
| 349 | +static inline bool kvm_has_gicv3(struct kvm *kvm) | |
| 350 | +{ | |
| 351 | + return (static_branch_unlikely(&kvm_vgic_global_state.gicv3_cpuif) && | |
| 352 | + irqchip_in_kernel(kvm) && | |
| 353 | + kvm->arch.vgic.vgic_model == KVM_DEV_TYPE_ARM_VGIC_V3); | |
| 354 | +} | |
| 355 | + | |
| 349 | 356 | #endif |
| @@ -1,11 +0,0 @@ | ||
| 1 | −/* SPDX-License-Identifier: GPL-2.0 */ | |
| 2 | −/* | |
| 3 | − * Copyright (C) 2020-2022 Loongson Technology Corporation Limited | |
| 4 | − */ | |
| 5 | −#ifndef _LOONGARCH_DMA_DIRECT_H | |
| 6 | −#define _LOONGARCH_DMA_DIRECT_H | |
| 7 | − | |
| 8 | −dma_addr_t phys_to_dma(struct device *dev, phys_addr_t paddr); | |
| 9 | −phys_addr_t dma_to_phys(struct device *dev, dma_addr_t daddr); | |
| 10 | − | |
| 11 | −#endif /* _LOONGARCH_DMA_DIRECT_H */ | |
| @@ -34,7 +34,7 @@ static inline pte_t huge_ptep_get_and_clear(struct mm_struct *mm, · 33 unmodified lines | ||
| 34 | 34 | unsigned long addr, pte_t *ptep) |
| 35 | 35 | { |
| 36 | 36 | pte_t clear; |
| 37 | − pte_t pte = *ptep; | |
| 37 | + pte_t pte = ptep_get(ptep); | |
| 38 | 38 | |
| 39 | 39 | pte_val(clear) = (unsigned long)invalid_pte_table; |
| 40 | 40 | set_pte_at(mm, addr, ptep, clear); |
| @@ -65,7 +65,7 @@ static inline int huge_ptep_set_access_flags(struct vm_area_struct *vma, · 24 unmodified lines | ||
| 65 | 65 | pte_t *ptep, pte_t pte, |
| 66 | 66 | int dirty) |
| 67 | 67 | { |
| 68 | − int changed = !pte_same(*ptep, pte); | |
| 68 | + int changed = !pte_same(ptep_get(ptep), pte); | |
| 69 | 69 | |
| 70 | 70 | if (changed) { |
| 71 | 71 | set_pte_at(vma->vm_mm, addr, ptep, pte); |
| @@ -9,6 +9,8 @@ · 8 unmodified lines | ||
| 9 | 9 | |
| 10 | 10 | extern atomic_t irq_err_count; |
| 11 | 11 | |
| 12 | +#define ARCH_IRQ_INIT_FLAGS IRQ_NOPROBE | |
| 13 | + | |
| 12 | 14 | /* |
| 13 | 15 | * interrupt-retrigger: NOP for now. This may not be appropriate for all |
| 14 | 16 | * machines, we'll see ... |
| @@ -53,13 +53,13 @@ static inline bool kfence_protect_page(unsigned long addr, bool protect) · 52 unmodified lines | ||
| 53 | 53 | { |
| 54 | 54 | pte_t *pte = virt_to_kpte(addr); |
| 55 | 55 | |
| 56 | − if (WARN_ON(!pte) || pte_none(*pte)) | |
| 56 | + if (WARN_ON(!pte) || pte_none(ptep_get(pte))) | |
| 57 | 57 | return false; |
| 58 | 58 | |
| 59 | 59 | if (protect) |
| 60 | − set_pte(pte, __pte(pte_val(*pte) & ~(_PAGE_VALID | _PAGE_PRESENT))); | |
| 60 | + set_pte(pte, __pte(pte_val(ptep_get(pte)) & ~(_PAGE_VALID | _PAGE_PRESENT))); | |
| 61 | 61 | else |
| 62 | − set_pte(pte, __pte(pte_val(*pte) | (_PAGE_VALID | _PAGE_PRESENT))); | |
| 62 | + set_pte(pte, __pte(pte_val(ptep_get(pte)) | (_PAGE_VALID | _PAGE_PRESENT))); | |
| 63 | 63 | |
| 64 | 64 | preempt_disable(); |
| 65 | 65 | local_flush_tlb_one(addr); |
| @@ -26,8 +26,6 @@ · 25 unmodified lines | ||
| 26 | 26 | |
| 27 | 27 | #define KVM_MAX_VCPUS 256 |
| 28 | 28 | #define KVM_MAX_CPUCFG_REGS 21 |
| 29 | −/* memory slots that does not exposed to userspace */ | |
| 30 | −#define KVM_PRIVATE_MEM_SLOTS 0 | |
| 31 | 29 | |
| 32 | 30 | #define KVM_HALT_POLL_NS_DEFAULT 500000 |
| 33 | 31 | #define KVM_REQ_TLB_FLUSH_GPA KVM_ARCH_REQ(0) |
| @@ -39,9 +39,9 @@ struct kvm_steal_time { · 38 unmodified lines | ||
| 39 | 39 | * Hypercall interface for KVM hypervisor |
| 40 | 40 | * |
| 41 | 41 | * a0: function identifier |
| 42 | − * a1-a6: args | |
| 42 | + * a1-a5: args | |
| 43 | 43 | * Return value will be placed in a0. |
| 44 | − * Up to 6 arguments are passed in a1, a2, a3, a4, a5, a6. | |
| 44 | + * Up to 5 arguments are passed in a1, a2, a3, a4, a5. | |
| 45 | 45 | */ |
| 46 | 46 | static __always_inline long kvm_hypercall0(u64 fid) |
| 47 | 47 | { |
| @@ -76,7 +76,6 @@ static inline void kvm_restore_lasx(struct loongarch_fpu *fpu) { } · 75 unmodified lines | ||
| 76 | 76 | #endif |
| 77 | 77 | |
| 78 | 78 | void kvm_init_timer(struct kvm_vcpu *vcpu, unsigned long hz); |
| 79 | −void kvm_reset_timer(struct kvm_vcpu *vcpu); | |
| 80 | 79 | void kvm_save_timer(struct kvm_vcpu *vcpu); |
| 81 | 80 | void kvm_restore_timer(struct kvm_vcpu *vcpu); |
| 82 | 81 | |
| @@ -106,6 +106,9 @@ extern unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)]; · 105 unmodified lines | ||
| 106 | 106 | #define KFENCE_AREA_START (VMEMMAP_END + 1) |
| 107 | 107 | #define KFENCE_AREA_END (KFENCE_AREA_START + KFENCE_AREA_SIZE - 1) |
| 108 | 108 | |
| 109 | +#define ptep_get(ptep) READ_ONCE(*(ptep)) | |
| 110 | +#define pmdp_get(pmdp) READ_ONCE(*(pmdp)) | |
| 111 | + | |
| 109 | 112 | #define pte_ERROR(e) \ |
| 110 | 113 | pr_err("%s:%d: bad pte %016lx.\n", __FILE__, __LINE__, pte_val(e)) |
| 111 | 114 | #ifndef __PAGETABLE_PMD_FOLDED |
| @@ -147,11 +150,6 @@ static inline int p4d_present(p4d_t p4d) · 35 unmodified lines | ||
| 147 | 150 | return p4d_val(p4d) != (unsigned long)invalid_pud_table; |
| 148 | 151 | } |
| 149 | 152 | |
| 150 | −static inline void p4d_clear(p4d_t *p4dp) | |
| 151 | −{ | |
| 152 | − p4d_val(*p4dp) = (unsigned long)invalid_pud_table; | |
| 153 | −} | |
| 154 | − | |
| 155 | 153 | static inline pud_t *p4d_pgtable(p4d_t p4d) |
| 156 | 154 | { |
| 157 | 155 | return (pud_t *)p4d_val(p4d); |
| @@ -159,7 +157,12 @@ static inline pud_t *p4d_pgtable(p4d_t p4d) · 1 unmodified line | ||
| 159 | 157 | |
| 160 | 158 | static inline void set_p4d(p4d_t *p4d, p4d_t p4dval) |
| 161 | 159 | { |
| 162 | − *p4d = p4dval; | |
| 160 | + WRITE_ONCE(*p4d, p4dval); | |
| 161 | +} | |
| 162 | + | |
| 163 | +static inline void p4d_clear(p4d_t *p4dp) | |
| 164 | +{ | |
| 165 | + set_p4d(p4dp, __p4d((unsigned long)invalid_pud_table)); | |
| 163 | 166 | } |
| 164 | 167 | |
| 165 | 168 | #define p4d_phys(p4d) PHYSADDR(p4d_val(p4d)) |
| @@ -193,17 +196,20 @@ static inline int pud_present(pud_t pud) · 27 unmodified lines | ||
| 193 | 196 | return pud_val(pud) != (unsigned long)invalid_pmd_table; |
| 194 | 197 | } |
| 195 | 198 | |
| 196 | −static inline void pud_clear(pud_t *pudp) | |
| 199 | +static inline pmd_t *pud_pgtable(pud_t pud) | |
| 197 | 200 | { |
| 198 | − pud_val(*pudp) = ((unsigned long)invalid_pmd_table); | |
| 201 | + return (pmd_t *)pud_val(pud); | |
| 199 | 202 | } |
| 200 | 203 | |
| 201 | −static inline pmd_t *pud_pgtable(pud_t pud) | |
| 204 | +static inline void set_pud(pud_t *pud, pud_t pudval) | |
| 202 | 205 | { |
| 203 | − return (pmd_t *)pud_val(pud); | |
| 206 | + WRITE_ONCE(*pud, pudval); | |
| 204 | 207 | } |
| 205 | 208 | |
| 206 | −#define set_pud(pudptr, pudval) do { *(pudptr) = (pudval); } while (0) | |
| 209 | +static inline void pud_clear(pud_t *pudp) | |
| 210 | +{ | |
| 211 | + set_pud(pudp, __pud((unsigned long)invalid_pmd_table)); | |
| 212 | +} | |
| 207 | 213 | |
| 208 | 214 | #define pud_phys(pud) PHYSADDR(pud_val(pud)) |
| 209 | 215 | #define pud_page(pud) (pfn_to_page(pud_phys(pud) >> PAGE_SHIFT)) |
| @@ -231,12 +237,15 @@ static inline int pmd_present(pmd_t pmd) · 21 unmodified lines | ||
| 231 | 237 | return pmd_val(pmd) != (unsigned long)invalid_pte_table; |
| 232 | 238 | } |
| 233 | 239 | |
| 234 | −static inline void pmd_clear(pmd_t *pmdp) | |
| 240 | +static inline void set_pmd(pmd_t *pmd, pmd_t pmdval) | |
| 235 | 241 | { |
| 236 | − pmd_val(*pmdp) = ((unsigned long)invalid_pte_table); | |
| 242 | + WRITE_ONCE(*pmd, pmdval); | |
| 237 | 243 | } |
| 238 | 244 | |
| 239 | −#define set_pmd(pmdptr, pmdval) do { *(pmdptr) = (pmdval); } while (0) | |
| 245 | +static inline void pmd_clear(pmd_t *pmdp) | |
| 246 | +{ | |
| 247 | + set_pmd(pmdp, __pmd((unsigned long)invalid_pte_table)); | |
| 248 | +} | |
| 240 | 249 | |
| 241 | 250 | #define pmd_phys(pmd) PHYSADDR(pmd_val(pmd)) |
| 242 | 251 | |
| @@ -314,7 +323,8 @@ extern void paging_init(void); · 71 unmodified lines | ||
| 314 | 323 | |
| 315 | 324 | static inline void set_pte(pte_t *ptep, pte_t pteval) |
| 316 | 325 | { |
| 317 | − *ptep = pteval; | |
| 326 | + WRITE_ONCE(*ptep, pteval); | |
| 327 | + | |
| 318 | 328 | if (pte_val(pteval) & _PAGE_GLOBAL) { |
| 319 | 329 | pte_t *buddy = ptep_buddy(ptep); |
| 320 | 330 | /* |
| @@ -341,8 +351,8 @@ static inline void set_pte(pte_t *ptep, pte_t pteval) · 20 unmodified lines | ||
| 341 | 351 | : [buddy] "+m" (buddy->pte), [tmp] "=&r" (tmp) |
| 342 | 352 | : [global] "r" (page_global)); |
| 343 | 353 | #else /* !CONFIG_SMP */ |
| 344 | − if (pte_none(*buddy)) | |
| 345 | − pte_val(*buddy) = pte_val(*buddy) | _PAGE_GLOBAL; | |
| 354 | + if (pte_none(ptep_get(buddy))) | |
| 355 | + WRITE_ONCE(*buddy, __pte(pte_val(ptep_get(buddy)) | _PAGE_GLOBAL)); | |
| 346 | 356 | #endif /* CONFIG_SMP */ |
| 347 | 357 | } |
| 348 | 358 | } |
| @@ -350,7 +360,7 @@ static inline void set_pte(pte_t *ptep, pte_t pteval) · 1 unmodified line | ||
| 350 | 360 | static inline void pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep) |
| 351 | 361 | { |
| 352 | 362 | /* Preserve global status for the pair */ |
| 353 | − if (pte_val(*ptep_buddy(ptep)) & _PAGE_GLOBAL) | |
| 363 | + if (pte_val(ptep_get(ptep_buddy(ptep))) & _PAGE_GLOBAL) | |
| 354 | 364 | set_pte(ptep, __pte(_PAGE_GLOBAL)); |
| 355 | 365 | else |
| 356 | 366 | set_pte(ptep, __pte(0)); |
| @@ -603,7 +613,7 @@ static inline pmd_t pmd_mkinvalid(pmd_t pmd) · 246 unmodified lines | ||
| 603 | 613 | static inline pmd_t pmdp_huge_get_and_clear(struct mm_struct *mm, |
| 604 | 614 | unsigned long address, pmd_t *pmdp) |
| 605 | 615 | { |
| 606 | − pmd_t old = *pmdp; | |
| 616 | + pmd_t old = pmdp_get(pmdp); | |
| 607 | 617 | |
| 608 | 618 | pmd_clear(pmdp); |
| 609 | 619 | |
| @@ -1,3 +1,4 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0 |
| 2 | 2 | |
| 3 | −syscall_abis_64 += newstat | |
| 3 | +# No special ABIs on loongarch so far | |
| 4 | +syscall_abis_64 += | |
| @@ -66,6 +66,12 @@ void __init efi_runtime_init(void) · 65 unmodified lines | ||
| 66 | 66 | set_bit(EFI_RUNTIME_SERVICES, &efi.flags); |
| 67 | 67 | } |
| 68 | 68 | |
| 69 | +bool efi_poweroff_required(void) | |
| 70 | +{ | |
| 71 | + return efi_enabled(EFI_RUNTIME_SERVICES) && | |
| 72 | + (acpi_gbl_reduced_hardware || acpi_no_s5); | |
| 73 | +} | |
| 74 | + | |
| 69 | 75 | unsigned long __initdata screen_info_table = EFI_INVALID_TABLE_ADDR; |
| 70 | 76 | |
| 71 | 77 | #if defined(CONFIG_SYSFB) || defined(CONFIG_EFI_EARLYCON) |
| @@ -530,6 +530,10 @@ SYM_FUNC_END(_restore_lasx_context) · 529 unmodified lines | ||
| 530 | 530 | |
| 531 | 531 | #ifdef CONFIG_CPU_HAS_LBT |
| 532 | 532 | STACK_FRAME_NON_STANDARD _restore_fp |
| 533 | +#ifdef CONFIG_CPU_HAS_LSX | |
| 533 | 534 | STACK_FRAME_NON_STANDARD _restore_lsx |
| 535 | +#endif | |
| 536 | +#ifdef CONFIG_CPU_HAS_LASX | |
| 534 | 537 | STACK_FRAME_NON_STANDARD _restore_lasx |
| 535 | 538 | #endif |
| 539 | +#endif | |
| @@ -102,9 +102,6 @@ void __init init_IRQ(void) · 101 unmodified lines | ||
| 102 | 102 | mp_ops.init_ipi(); |
| 103 | 103 | #endif |
| 104 | 104 | |
| 105 | − for (i = 0; i < NR_IRQS; i++) | |
| 106 | − irq_set_noprobe(i); | |
| 107 | − | |
| 108 | 105 | for_each_possible_cpu(i) { |
| 109 | 106 | page = alloc_pages_node(cpu_to_node(i), GFP_KERNEL, order); |
| 110 | 107 | |
| @@ -714,19 +714,19 @@ static int host_pfn_mapping_level(struct kvm *kvm, gfn_t gfn, · 713 unmodified lines | ||
| 714 | 714 | * value) and then p*d_offset() walks into the target huge page instead |
| 715 | 715 | * of the old page table (sees the new value). |
| 716 | 716 | */ |
| 717 | − pgd = READ_ONCE(*pgd_offset(kvm->mm, hva)); | |
| 717 | + pgd = pgdp_get(pgd_offset(kvm->mm, hva)); | |
| 718 | 718 | if (pgd_none(pgd)) |
| 719 | 719 | goto out; |
| 720 | 720 | |
| 721 | − p4d = READ_ONCE(*p4d_offset(&pgd, hva)); | |
| 721 | + p4d = p4dp_get(p4d_offset(&pgd, hva)); | |
| 722 | 722 | if (p4d_none(p4d) || !p4d_present(p4d)) |
| 723 | 723 | goto out; |
| 724 | 724 | |
| 725 | − pud = READ_ONCE(*pud_offset(&p4d, hva)); | |
| 725 | + pud = pudp_get(pud_offset(&p4d, hva)); | |
| 726 | 726 | if (pud_none(pud) || !pud_present(pud)) |
| 727 | 727 | goto out; |
| 728 | 728 | |
| 729 | − pmd = READ_ONCE(*pmd_offset(&pud, hva)); | |
| 729 | + pmd = pmdp_get(pmd_offset(&pud, hva)); | |
| 730 | 730 | if (pmd_none(pmd) || !pmd_present(pmd)) |
| 731 | 731 | goto out; |
| 732 | 732 | |
| @@ -277,6 +277,10 @@ SYM_DATA(kvm_enter_guest_size, .quad kvm_enter_guest_end - kvm_enter_guest) · 276 unmodified lines | ||
| 277 | 277 | |
| 278 | 278 | #ifdef CONFIG_CPU_HAS_LBT |
| 279 | 279 | STACK_FRAME_NON_STANDARD kvm_restore_fpu |
| 280 | +#ifdef CONFIG_CPU_HAS_LSX | |
| 280 | 281 | STACK_FRAME_NON_STANDARD kvm_restore_lsx |
| 282 | +#endif | |
| 283 | +#ifdef CONFIG_CPU_HAS_LASX | |
| 281 | 284 | STACK_FRAME_NON_STANDARD kvm_restore_lasx |
| 282 | 285 | #endif |
| 286 | +#endif | |
| @@ -188,10 +188,3 @@ void kvm_save_timer(struct kvm_vcpu *vcpu) · 187 unmodified lines | ||
| 188 | 188 | kvm_save_hw_gcsr(csr, LOONGARCH_CSR_ESTAT); |
| 189 | 189 | preempt_enable(); |
| 190 | 190 | } |
| 191 | − | |
| 192 | −void kvm_reset_timer(struct kvm_vcpu *vcpu) | |
| 193 | −{ | |
| 194 | − write_gcsr_timercfg(0); | |
| 195 | − kvm_write_sw_gcsr(vcpu->arch.csr, LOONGARCH_CSR_TCFG, 0); | |
| 196 | − hrtimer_cancel(&vcpu->arch.swtimer); | |
| 197 | −} | |
| @@ -647,7 +647,7 @@ static int kvm_set_one_reg(struct kvm_vcpu *vcpu, · 646 unmodified lines | ||
| 647 | 647 | vcpu->kvm->arch.time_offset = (signed long)(v - drdtime()); |
| 648 | 648 | break; |
| 649 | 649 | case KVM_REG_LOONGARCH_VCPU_RESET: |
| 650 | − kvm_reset_timer(vcpu); | |
| 650 | + vcpu->arch.st.guest_addr = 0; | |
| 651 | 651 | memset(&vcpu->arch.irq_pending, 0, sizeof(vcpu->arch.irq_pending)); |
| 652 | 652 | memset(&vcpu->arch.irq_clear, 0, sizeof(vcpu->arch.irq_clear)); |
| 653 | 653 | break; |
| @@ -39,11 +39,11 @@ pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr, · 38 unmodified lines | ||
| 39 | 39 | pmd_t *pmd = NULL; |
| 40 | 40 | |
| 41 | 41 | pgd = pgd_offset(mm, addr); |
| 42 | − if (pgd_present(*pgd)) { | |
| 42 | + if (pgd_present(pgdp_get(pgd))) { | |
| 43 | 43 | p4d = p4d_offset(pgd, addr); |
| 44 | − if (p4d_present(*p4d)) { | |
| 44 | + if (p4d_present(p4dp_get(p4d))) { | |
| 45 | 45 | pud = pud_offset(p4d, addr); |
| 46 | − if (pud_present(*pud)) | |
| 46 | + if (pud_present(pudp_get(pud))) | |
| 47 | 47 | pmd = pmd_offset(pud, addr); |
| 48 | 48 | } |
| 49 | 49 | } |
| @@ -141,7 +141,7 @@ void __meminit vmemmap_set_pmd(pmd_t *pmd, void *p, int node, · 140 unmodified lines | ||
| 141 | 141 | int __meminit vmemmap_check_pmd(pmd_t *pmd, int node, |
| 142 | 142 | unsigned long addr, unsigned long next) |
| 143 | 143 | { |
| 144 | − int huge = pmd_val(*pmd) & _PAGE_HUGE; | |
| 144 | + int huge = pmd_val(pmdp_get(pmd)) & _PAGE_HUGE; | |
| 145 | 145 | |
| 146 | 146 | if (huge) |
| 147 | 147 | vmemmap_verify((pte_t *)pmd, node, addr, next); |
| @@ -173,7 +173,7 @@ pte_t * __init populate_kernel_pte(unsigned long addr) · 25 unmodified lines | ||
| 173 | 173 | pud_t *pud; |
| 174 | 174 | pmd_t *pmd; |
| 175 | 175 | |
| 176 | − if (p4d_none(*p4d)) { | |
| 176 | + if (p4d_none(p4dp_get(p4d))) { | |
| 177 | 177 | pud = memblock_alloc(PAGE_SIZE, PAGE_SIZE); |
| 178 | 178 | if (!pud) |
| 179 | 179 | panic("%s: Failed to allocate memory\n", __func__); |
| @@ -184,7 +184,7 @@ pte_t * __init populate_kernel_pte(unsigned long addr) · 4 unmodified lines | ||
| 184 | 184 | } |
| 185 | 185 | |
| 186 | 186 | pud = pud_offset(p4d, addr); |
| 187 | − if (pud_none(*pud)) { | |
| 187 | + if (pud_none(pudp_get(pud))) { | |
| 188 | 188 | pmd = memblock_alloc(PAGE_SIZE, PAGE_SIZE); |
| 189 | 189 | if (!pmd) |
| 190 | 190 | panic("%s: Failed to allocate memory\n", __func__); |
| @@ -195,7 +195,7 @@ pte_t * __init populate_kernel_pte(unsigned long addr) · 4 unmodified lines | ||
| 195 | 195 | } |
| 196 | 196 | |
| 197 | 197 | pmd = pmd_offset(pud, addr); |
| 198 | − if (!pmd_present(*pmd)) { | |
| 198 | + if (!pmd_present(pmdp_get(pmd))) { | |
| 199 | 199 | pte_t *pte; |
| 200 | 200 | |
| 201 | 201 | pte = memblock_alloc(PAGE_SIZE, PAGE_SIZE); |
| @@ -216,7 +216,7 @@ void __init __set_fixmap(enum fixed_addresses idx, · 14 unmodified lines | ||
| 216 | 216 | BUG_ON(idx <= FIX_HOLE || idx >= __end_of_fixed_addresses); |
| 217 | 217 | |
| 218 | 218 | ptep = populate_kernel_pte(addr); |
| 219 | − if (!pte_none(*ptep)) { | |
| 219 | + if (!pte_none(ptep_get(ptep))) { | |
| 220 | 220 | pte_ERROR(*ptep); |
| 221 | 221 | return; |
| 222 | 222 | } |
| @@ -105,7 +105,7 @@ static phys_addr_t __init kasan_alloc_zeroed_page(int node) · 104 unmodified lines | ||
| 105 | 105 | |
| 106 | 106 | static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, bool early) |
| 107 | 107 | { |
| 108 | − if (__pmd_none(early, READ_ONCE(*pmdp))) { | |
| 108 | + if (__pmd_none(early, pmdp_get(pmdp))) { | |
| 109 | 109 | phys_addr_t pte_phys = early ? |
| 110 | 110 | __pa_symbol(kasan_early_shadow_pte) : kasan_alloc_zeroed_page(node); |
| 111 | 111 | if (!early) |
| @@ -118,7 +118,7 @@ static pte_t *__init kasan_pte_offset(pmd_t *pmdp, unsigned long addr, int node, · 6 unmodified lines | ||
| 118 | 118 | |
| 119 | 119 | static pmd_t *__init kasan_pmd_offset(pud_t *pudp, unsigned long addr, int node, bool early) |
| 120 | 120 | { |
| 121 | − if (__pud_none(early, READ_ONCE(*pudp))) { | |
| 121 | + if (__pud_none(early, pudp_get(pudp))) { | |
| 122 | 122 | phys_addr_t pmd_phys = early ? |
| 123 | 123 | __pa_symbol(kasan_early_shadow_pmd) : kasan_alloc_zeroed_page(node); |
| 124 | 124 | if (!early) |
| @@ -131,7 +131,7 @@ static pmd_t *__init kasan_pmd_offset(pud_t *pudp, unsigned long addr, int node, · 6 unmodified lines | ||
| 131 | 131 | |
| 132 | 132 | static pud_t *__init kasan_pud_offset(p4d_t *p4dp, unsigned long addr, int node, bool early) |
| 133 | 133 | { |
| 134 | − if (__p4d_none(early, READ_ONCE(*p4dp))) { | |
| 134 | + if (__p4d_none(early, p4dp_get(p4dp))) { | |
| 135 | 135 | phys_addr_t pud_phys = early ? |
| 136 | 136 | __pa_symbol(kasan_early_shadow_pud) : kasan_alloc_zeroed_page(node); |
| 137 | 137 | if (!early) |
| @@ -154,7 +154,7 @@ static void __init kasan_pte_populate(pmd_t *pmdp, unsigned long addr, · 16 unmodified lines | ||
| 154 | 154 | : kasan_alloc_zeroed_page(node); |
| 155 | 155 | next = addr + PAGE_SIZE; |
| 156 | 156 | set_pte(ptep, pfn_pte(__phys_to_pfn(page_phys), PAGE_KERNEL)); |
| 157 | − } while (ptep++, addr = next, addr != end && __pte_none(early, READ_ONCE(*ptep))); | |
| 157 | + } while (ptep++, addr = next, addr != end && __pte_none(early, ptep_get(ptep))); | |
| 158 | 158 | } |
| 159 | 159 | |
| 160 | 160 | static void __init kasan_pmd_populate(pud_t *pudp, unsigned long addr, |
| @@ -166,7 +166,7 @@ static void __init kasan_pmd_populate(pud_t *pudp, unsigned long addr, · 5 unmodified lines | ||
| 166 | 166 | do { |
| 167 | 167 | next = pmd_addr_end(addr, end); |
| 168 | 168 | kasan_pte_populate(pmdp, addr, next, node, early); |
| 169 | − } while (pmdp++, addr = next, addr != end && __pmd_none(early, READ_ONCE(*pmdp))); | |
| 169 | + } while (pmdp++, addr = next, addr != end && __pmd_none(early, pmdp_get(pmdp))); | |
| 170 | 170 | } |
| 171 | 171 | |
| 172 | 172 | static void __init kasan_pud_populate(p4d_t *p4dp, unsigned long addr, |
| @@ -128,7 +128,7 @@ pmd_t mk_pmd(struct page *page, pgprot_t prot) · 127 unmodified lines | ||
| 128 | 128 | void set_pmd_at(struct mm_struct *mm, unsigned long addr, |
| 129 | 129 | pmd_t *pmdp, pmd_t pmd) |
| 130 | 130 | { |
| 131 | − *pmdp = pmd; | |
| 131 | + WRITE_ONCE(*pmdp, pmd); | |
| 132 | 132 | flush_tlb_all(); |
| 133 | 133 | } |
| 134 | 134 | |
| @@ -193,11 +193,6 @@ asmlinkage void __init mmu_init(void) · 192 unmodified lines | ||
| 193 | 193 | { |
| 194 | 194 | unsigned int kstart, ksize; |
| 195 | 195 | |
| 196 | − if (!memblock.reserved.cnt) { | |
| 197 | − pr_emerg("Error memory count\n"); | |
| 198 | − machine_restart(NULL); | |
| 199 | − } | |
| 200 | − | |
| 201 | 196 | if ((u32) memblock.memory.regions[0].size < 0x400000) { |
| 202 | 197 | pr_emerg("Memory must be greater than 4MB\n"); |
| 203 | 198 | machine_restart(NULL); |
| @@ -303,13 +303,6 @@ int r4k_clockevent_init(void) · 302 unmodified lines | ||
| 303 | 303 | if (!c0_compare_int_usable()) |
| 304 | 304 | return -ENXIO; |
| 305 | 305 | |
| 306 | − /* | |
| 307 | − * With vectored interrupts things are getting platform specific. | |
| 308 | − * get_c0_compare_int is a hook to allow a platform to return the | |
| 309 | − * interrupt number of its liking. | |
| 310 | − */ | |
| 311 | − irq = get_c0_compare_int(); | |
| 312 | − | |
| 313 | 306 | cd = &per_cpu(mips_clockevent_device, cpu); |
| 314 | 307 | |
| 315 | 308 | cd->name = "MIPS"; |
| @@ -320,7 +313,6 @@ int r4k_clockevent_init(void) · 4 unmodified lines | ||
| 320 | 313 | min_delta = calculate_min_delta(); |
| 321 | 314 | |
| 322 | 315 | cd->rating = 300; |
| 323 | − cd->irq = irq; | |
| 324 | 316 | cd->cpumask = cpumask_of(cpu); |
| 325 | 317 | cd->set_next_event = mips_next_event; |
| 326 | 318 | cd->event_handler = mips_event_handler; |
| @@ -332,6 +324,13 @@ int r4k_clockevent_init(void) · 5 unmodified lines | ||
| 332 | 324 | |
| 333 | 325 | cp0_timer_irq_installed = 1; |
| 334 | 326 | |
| 327 | + /* | |
| 328 | + * With vectored interrupts things are getting platform specific. | |
| 329 | + * get_c0_compare_int is a hook to allow a platform to return the | |
| 330 | + * interrupt number of its liking. | |
| 331 | + */ | |
| 332 | + irq = get_c0_compare_int(); | |
| 333 | + | |
| 335 | 334 | if (request_irq(irq, c0_compare_interrupt, flags, "timer", |
| 336 | 335 | c0_compare_interrupt)) |
| 337 | 336 | pr_err("Failed to request irq %d (timer)\n", irq); |
| @@ -1724,12 +1724,16 @@ static inline void cpu_probe_loongson(struct cpuinfo_mips *c, unsigned int cpu) · 1723 unmodified lines | ||
| 1724 | 1724 | c->ases |= (MIPS_ASE_LOONGSON_MMI | MIPS_ASE_LOONGSON_CAM | |
| 1725 | 1725 | MIPS_ASE_LOONGSON_EXT | MIPS_ASE_LOONGSON_EXT2); |
| 1726 | 1726 | c->ases &= ~MIPS_ASE_VZ; /* VZ of Loongson-3A2000/3000 is incomplete */ |
| 1727 | + change_c0_config6(LOONGSON_CONF6_EXTIMER | LOONGSON_CONF6_INTIMER, | |
| 1728 | + LOONGSON_CONF6_INTIMER); | |
| 1727 | 1729 | break; |
| 1728 | 1730 | case PRID_IMP_LOONGSON_64G: |
| 1729 | 1731 | __cpu_name[cpu] = "ICT Loongson-3"; |
| 1730 | 1732 | set_elf_platform(cpu, "loongson3a"); |
| 1731 | 1733 | set_isa(c, MIPS_CPU_ISA_M64R2); |
| 1732 | 1734 | decode_cpucfg(c); |
| 1735 | + change_c0_config6(LOONGSON_CONF6_EXTIMER | LOONGSON_CONF6_INTIMER, | |
| 1736 | + LOONGSON_CONF6_INTIMER); | |
| 1733 | 1737 | break; |
| 1734 | 1738 | default: |
| 1735 | 1739 | panic("Unknown Loongson Processor ID!"); |
| @@ -21,9 +21,7 @@ static struct clocksource clocksource_mips = { · 20 unmodified lines | ||
| 21 | 21 | .name = "MIPS", |
| 22 | 22 | .read = c0_hpt_read, |
| 23 | 23 | .mask = CLOCKSOURCE_MASK(32), |
| 24 | − .flags = CLOCK_SOURCE_IS_CONTINUOUS | | |
| 25 | − CLOCK_SOURCE_MUST_VERIFY | | |
| 26 | − CLOCK_SOURCE_VERIFY_PERCPU, | |
| 24 | + .flags = CLOCK_SOURCE_IS_CONTINUOUS, | |
| 27 | 25 | }; |
| 28 | 26 | |
| 29 | 27 | static u64 __maybe_unused notrace r4k_read_sched_clock(void) |
| @@ -111,7 +111,7 @@ void gio_device_unregister(struct gio_device *giodev) · 110 unmodified lines | ||
| 111 | 111 | } |
| 112 | 112 | EXPORT_SYMBOL_GPL(gio_device_unregister); |
| 113 | 113 | |
| 114 | −static int gio_bus_match(struct device *dev, struct device_driver *drv) | |
| 114 | +static int gio_bus_match(struct device *dev, const struct device_driver *drv) | |
| 115 | 115 | { |
| 116 | 116 | struct gio_device *gio_dev = to_gio_device(dev); |
| 117 | 117 | struct gio_driver *gio_drv = to_gio_driver(drv); |
| @@ -20,6 +20,7 @@ config PARISC · 19 unmodified lines | ||
| 20 | 20 | select ARCH_SUPPORTS_HUGETLBFS if PA20 |
| 21 | 21 | select ARCH_SUPPORTS_MEMORY_FAILURE |
| 22 | 22 | select ARCH_STACKWALK |
| 23 | + select ARCH_HAS_CACHE_LINE_SIZE | |
| 23 | 24 | select ARCH_HAS_DEBUG_VM_PGTABLE |
| 24 | 25 | select HAVE_RELIABLE_STACKTRACE |
| 25 | 26 | select DMA_OPS |
| @@ -20,7 +20,16 @@ · 19 unmodified lines | ||
| 20 | 20 | |
| 21 | 21 | #define SMP_CACHE_BYTES L1_CACHE_BYTES |
| 22 | 22 | |
| 23 | −#define ARCH_DMA_MINALIGN L1_CACHE_BYTES | |
| 23 | +#ifdef CONFIG_PA20 | |
| 24 | +#define ARCH_DMA_MINALIGN 128 | |
| 25 | +#else | |
| 26 | +#define ARCH_DMA_MINALIGN 32 | |
| 27 | +#endif | |
| 28 | +#define ARCH_KMALLOC_MINALIGN 16 /* ldcw requires 16-byte alignment */ | |
| 29 | + | |
| 30 | +#define arch_slab_minalign() ((unsigned)dcache_stride) | |
| 31 | +#define cache_line_size() dcache_stride | |
| 32 | +#define dma_get_cache_alignment cache_line_size | |
| 24 | 33 | |
| 25 | 34 | #define __read_mostly __section(".data..read_mostly") |
| 26 | 35 | |
| @@ -114,7 +114,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) · 113 unmodified lines | ||
| 114 | 114 | jit_data->header = |
| 115 | 115 | bpf_jit_binary_alloc(prog_size + extable_size, |
| 116 | 116 | &jit_data->image, |
| 117 | − sizeof(u32), | |
| 117 | + sizeof(long), | |
| 118 | 118 | bpf_fill_ill_insns); |
| 119 | 119 | if (!jit_data->header) { |
| 120 | 120 | prog = orig_prog; |
| @@ -145,6 +145,7 @@ static inline int cpu_to_coregroup_id(int cpu) · 144 unmodified lines | ||
| 145 | 145 | |
| 146 | 146 | #ifdef CONFIG_HOTPLUG_SMT |
| 147 | 147 | #include <linux/cpu_smt.h> |
| 148 | +#include <linux/cpumask.h> | |
| 148 | 149 | #include <asm/cputhreads.h> |
| 149 | 150 | |
| 150 | 151 | static inline bool topology_is_primary_thread(unsigned int cpu) |
| @@ -156,6 +157,18 @@ static inline bool topology_smt_thread_allowed(unsigned int cpu) · 5 unmodified lines | ||
| 156 | 157 | { |
| 157 | 158 | return cpu_thread_in_core(cpu) < cpu_smt_num_threads; |
| 158 | 159 | } |
| 160 | + | |
| 161 | +#define topology_is_core_online topology_is_core_online | |
| 162 | +static inline bool topology_is_core_online(unsigned int cpu) | |
| 163 | +{ | |
| 164 | + int i, first_cpu = cpu_first_thread_sibling(cpu); | |
| 165 | + | |
| 166 | + for (i = first_cpu; i < first_cpu + threads_per_core; ++i) { | |
| 167 | + if (cpu_online(i)) | |
| 168 | + return true; | |
| 169 | + } | |
| 170 | + return false; | |
| 171 | +} | |
| 159 | 172 | #endif |
| 160 | 173 | |
| 161 | 174 | #endif /* __KERNEL__ */ |
| @@ -959,6 +959,7 @@ void __init setup_arch(char **cmdline_p) · 958 unmodified lines | ||
| 959 | 959 | mem_topology_setup(); |
| 960 | 960 | /* Set max_mapnr before paging_init() */ |
| 961 | 961 | set_max_mapnr(max_pfn); |
| 962 | + high_memory = (void *)__va(max_low_pfn * PAGE_SIZE); | |
| 962 | 963 | |
| 963 | 964 | /* |
| 964 | 965 | * Release secondary cpus out of their spinloops at 0x60 now that |
| @@ -73,7 +73,7 @@ void setup_kup(void) · 72 unmodified lines | ||
| 73 | 73 | |
| 74 | 74 | #define CTOR(shift) static void ctor_##shift(void *addr) \ |
| 75 | 75 | { \ |
| 76 | − memset(addr, 0, sizeof(void *) << (shift)); \ | |
| 76 | + memset(addr, 0, sizeof(pgd_t) << (shift)); \ | |
| 77 | 77 | } |
| 78 | 78 | |
| 79 | 79 | CTOR(0); CTOR(1); CTOR(2); CTOR(3); CTOR(4); CTOR(5); CTOR(6); CTOR(7); |
| @@ -117,7 +117,7 @@ EXPORT_SYMBOL_GPL(pgtable_cache); /* used by kvm_hv module */ · 37 unmodified lines | ||
| 117 | 117 | void pgtable_cache_add(unsigned int shift) |
| 118 | 118 | { |
| 119 | 119 | char *name; |
| 120 | − unsigned long table_size = sizeof(void *) << shift; | |
| 120 | + unsigned long table_size = sizeof(pgd_t) << shift; | |
| 121 | 121 | unsigned long align = table_size; |
| 122 | 122 | |
| 123 | 123 | /* When batching pgtable pointers for RCU freeing, we store |
| @@ -290,8 +290,6 @@ void __init mem_init(void) · 289 unmodified lines | ||
| 290 | 290 | swiotlb_init(ppc_swiotlb_enable, ppc_swiotlb_flags); |
| 291 | 291 | #endif |
| 292 | 292 | |
| 293 | − high_memory = (void *) __va(max_low_pfn * PAGE_SIZE); | |
| 294 | − | |
| 295 | 293 | kasan_late_init(); |
| 296 | 294 | |
| 297 | 295 | memblock_free_all(); |
| @@ -8,7 +8,7 @@ · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | #include <uapi/asm/hwprobe.h> |
| 10 | 10 | |
| 11 | −#define RISCV_HWPROBE_MAX_KEY 8 | |
| 11 | +#define RISCV_HWPROBE_MAX_KEY 9 | |
| 12 | 12 | |
| 13 | 13 | static inline bool riscv_hwprobe_key_is_valid(__s64 key) |
| 14 | 14 | { |
| @@ -82,6 +82,12 @@ struct riscv_hwprobe { · 81 unmodified lines | ||
| 82 | 82 | #define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 |
| 83 | 83 | #define RISCV_HWPROBE_KEY_HIGHEST_VIRT_ADDRESS 7 |
| 84 | 84 | #define RISCV_HWPROBE_KEY_TIME_CSR_FREQ 8 |
| 85 | +#define RISCV_HWPROBE_KEY_MISALIGNED_SCALAR_PERF 9 | |
| 86 | +#define RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN 0 | |
| 87 | +#define RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED 1 | |
| 88 | +#define RISCV_HWPROBE_MISALIGNED_SCALAR_SLOW 2 | |
| 89 | +#define RISCV_HWPROBE_MISALIGNED_SCALAR_FAST 3 | |
| 90 | +#define RISCV_HWPROBE_MISALIGNED_SCALAR_UNSUPPORTED 4 | |
| 85 | 91 | /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */ |
| 86 | 92 | |
| 87 | 93 | /* Flags */ |
| @@ -1,4 +1,4 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0 |
| 2 | 2 | |
| 3 | 3 | syscall_abis_32 += riscv memfd_secret |
| 4 | −syscall_abis_64 += riscv newstat rlimit memfd_secret | |
| 4 | +syscall_abis_64 += riscv rlimit memfd_secret | |
| @@ -28,7 +28,7 @@ · 27 unmodified lines | ||
| 28 | 28 | |
| 29 | 29 | #include <asm/numa.h> |
| 30 | 30 | |
| 31 | −static int acpi_early_node_map[NR_CPUS] __initdata = { NUMA_NO_NODE }; | |
| 31 | +static int acpi_early_node_map[NR_CPUS] __initdata = { [0 ... NR_CPUS - 1] = NUMA_NO_NODE }; | |
| 32 | 32 | |
| 33 | 33 | int __init acpi_numa_get_nid(unsigned int cpu) |
| 34 | 34 | { |
| @@ -432,28 +432,26 @@ static void __init riscv_resolve_isa(unsigned long *source_isa, · 431 unmodified lines | ||
| 432 | 432 | bitmap_copy(prev_resolved_isa, resolved_isa, RISCV_ISA_EXT_MAX); |
| 433 | 433 | for_each_set_bit(bit, source_isa, RISCV_ISA_EXT_MAX) { |
| 434 | 434 | ext = riscv_get_isa_ext_data(bit); |
| 435 | − if (!ext) | |
| 436 | − continue; | |
| 437 | 435 | |
| 438 | − if (ext->validate) { | |
| 436 | + if (ext && ext->validate) { | |
| 439 | 437 | ret = ext->validate(ext, resolved_isa); |
| 440 | 438 | if (ret == -EPROBE_DEFER) { |
| 441 | 439 | loop = true; |
| 442 | 440 | continue; |
| 443 | 441 | } else if (ret) { |
| 444 | 442 | /* Disable the extension entirely */ |
| 445 | − clear_bit(ext->id, source_isa); | |
| 443 | + clear_bit(bit, source_isa); | |
| 446 | 444 | continue; |
| 447 | 445 | } |
| 448 | 446 | } |
| 449 | 447 | |
| 450 | − set_bit(ext->id, resolved_isa); | |
| 448 | + set_bit(bit, resolved_isa); | |
| 451 | 449 | /* No need to keep it in source isa now that it is enabled */ |
| 452 | − clear_bit(ext->id, source_isa); | |
| 450 | + clear_bit(bit, source_isa); | |
| 453 | 451 | |
| 454 | 452 | /* Single letter extensions get set in hwcap */ |
| 455 | − if (ext->id < RISCV_ISA_EXT_BASE) | |
| 456 | − *this_hwcap |= isa2hwcap[ext->id]; | |
| 453 | + if (bit < RISCV_ISA_EXT_BASE) | |
| 454 | + *this_hwcap |= isa2hwcap[bit]; | |
| 457 | 455 | } |
| 458 | 456 | } while (loop && memcmp(prev_resolved_isa, resolved_isa, sizeof(prev_resolved_isa))); |
| 459 | 457 | } |
| @@ -205,6 +205,8 @@ int patch_text_set_nosync(void *addr, u8 c, size_t len) · 204 unmodified lines | ||
| 205 | 205 | int ret; |
| 206 | 206 | |
| 207 | 207 | ret = patch_insn_set(addr, c, len); |
| 208 | + if (!ret) | |
| 209 | + flush_icache_range((uintptr_t)addr, (uintptr_t)addr + len); | |
| 208 | 210 | |
| 209 | 211 | return ret; |
| 210 | 212 | } |
| @@ -239,6 +241,8 @@ int patch_text_nosync(void *addr, const void *insns, size_t len) · 28 unmodified lines | ||
| 239 | 241 | int ret; |
| 240 | 242 | |
| 241 | 243 | ret = patch_insn_write(addr, insns, len); |
| 244 | + if (!ret) | |
| 245 | + flush_icache_range((uintptr_t)addr, (uintptr_t)addr + len); | |
| 242 | 246 | |
| 243 | 247 | return ret; |
| 244 | 248 | } |
| @@ -71,7 +71,7 @@ void __init sbi_ipi_init(void) · 70 unmodified lines | ||
| 71 | 71 | * the masking/unmasking of virtual IPIs is done |
| 72 | 72 | * via generic IPI-Mux |
| 73 | 73 | */ |
| 74 | − cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, | |
| 74 | + cpuhp_setup_state(CPUHP_AP_IRQ_RISCV_SBI_IPI_STARTING, | |
| 75 | 75 | "irqchip/sbi-ipi:starting", |
| 76 | 76 | sbi_ipi_starting_cpu, NULL); |
| 77 | 77 | |
| @@ -178,13 +178,13 @@ static u64 hwprobe_misaligned(const struct cpumask *cpus) · 177 unmodified lines | ||
| 178 | 178 | perf = this_perf; |
| 179 | 179 | |
| 180 | 180 | if (perf != this_perf) { |
| 181 | − perf = RISCV_HWPROBE_MISALIGNED_UNKNOWN; | |
| 181 | + perf = RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN; | |
| 182 | 182 | break; |
| 183 | 183 | } |
| 184 | 184 | } |
| 185 | 185 | |
| 186 | 186 | if (perf == -1ULL) |
| 187 | − return RISCV_HWPROBE_MISALIGNED_UNKNOWN; | |
| 187 | + return RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN; | |
| 188 | 188 | |
| 189 | 189 | return perf; |
| 190 | 190 | } |
| @@ -192,12 +192,12 @@ static u64 hwprobe_misaligned(const struct cpumask *cpus) · 1 unmodified line | ||
| 192 | 192 | static u64 hwprobe_misaligned(const struct cpumask *cpus) |
| 193 | 193 | { |
| 194 | 194 | if (IS_ENABLED(CONFIG_RISCV_EFFICIENT_UNALIGNED_ACCESS)) |
| 195 | − return RISCV_HWPROBE_MISALIGNED_FAST; | |
| 195 | + return RISCV_HWPROBE_MISALIGNED_SCALAR_FAST; | |
| 196 | 196 | |
| 197 | 197 | if (IS_ENABLED(CONFIG_RISCV_EMULATED_UNALIGNED_ACCESS) && unaligned_ctl_available()) |
| 198 | − return RISCV_HWPROBE_MISALIGNED_EMULATED; | |
| 198 | + return RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED; | |
| 199 | 199 | |
| 200 | − return RISCV_HWPROBE_MISALIGNED_SLOW; | |
| 200 | + return RISCV_HWPROBE_MISALIGNED_SCALAR_SLOW; | |
| 201 | 201 | } |
| 202 | 202 | #endif |
| 203 | 203 | |
| @@ -225,6 +225,7 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pair, · 21 unmodified lines | ||
| 225 | 225 | break; |
| 226 | 226 | |
| 227 | 227 | case RISCV_HWPROBE_KEY_CPUPERF_0: |
| 228 | + case RISCV_HWPROBE_KEY_MISALIGNED_SCALAR_PERF: | |
| 228 | 229 | pair->value = hwprobe_misaligned(cpus); |
| 229 | 230 | break; |
| 230 | 231 | |
| @@ -319,6 +319,7 @@ void do_trap_ecall_u(struct pt_regs *regs) · 318 unmodified lines | ||
| 319 | 319 | |
| 320 | 320 | regs->epc += 4; |
| 321 | 321 | regs->orig_a0 = regs->a0; |
| 322 | + regs->a0 = -ENOSYS; | |
| 322 | 323 | |
| 323 | 324 | riscv_v_vstate_discard(regs); |
| 324 | 325 | |
| @@ -328,8 +329,7 @@ void do_trap_ecall_u(struct pt_regs *regs) · 3 unmodified lines | ||
| 328 | 329 | |
| 329 | 330 | if (syscall >= 0 && syscall < NR_syscalls) |
| 330 | 331 | syscall_handler(regs, syscall); |
| 331 | − else if (syscall != -1) | |
| 332 | − regs->a0 = -ENOSYS; | |
| 332 | + | |
| 333 | 333 | /* |
| 334 | 334 | * Ultimately, this value will get limited by KSTACK_OFFSET_MAX(), |
| 335 | 335 | * so the maximum stack offset is 1k bytes (10 bits). |
| @@ -338,7 +338,7 @@ int handle_misaligned_load(struct pt_regs *regs) · 337 unmodified lines | ||
| 338 | 338 | perf_sw_event(PERF_COUNT_SW_ALIGNMENT_FAULTS, 1, regs, addr); |
| 339 | 339 | |
| 340 | 340 | #ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS |
| 341 | − *this_cpu_ptr(&misaligned_access_speed) = RISCV_HWPROBE_MISALIGNED_EMULATED; | |
| 341 | + *this_cpu_ptr(&misaligned_access_speed) = RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED; | |
| 342 | 342 | #endif |
| 343 | 343 | |
| 344 | 344 | if (!unaligned_enabled) |
| @@ -532,13 +532,13 @@ static bool check_unaligned_access_emulated(int cpu) · 187 unmodified lines | ||
| 532 | 532 | unsigned long tmp_var, tmp_val; |
| 533 | 533 | bool misaligned_emu_detected; |
| 534 | 534 | |
| 535 | − *mas_ptr = RISCV_HWPROBE_MISALIGNED_UNKNOWN; | |
| 535 | + *mas_ptr = RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN; | |
| 536 | 536 | |
| 537 | 537 | __asm__ __volatile__ ( |
| 538 | 538 | " "REG_L" %[tmp], 1(%[ptr])\n" |
| 539 | 539 | : [tmp] "=r" (tmp_val) : [ptr] "r" (&tmp_var) : "memory"); |
| 540 | 540 | |
| 541 | − misaligned_emu_detected = (*mas_ptr == RISCV_HWPROBE_MISALIGNED_EMULATED); | |
| 541 | + misaligned_emu_detected = (*mas_ptr == RISCV_HWPROBE_MISALIGNED_SCALAR_EMULATED); | |
| 542 | 542 | /* |
| 543 | 543 | * If unaligned_ctl is already set, this means that we detected that all |
| 544 | 544 | * CPUS uses emulated misaligned access at boot time. If that changed |
| @@ -34,9 +34,9 @@ static int check_unaligned_access(void *param) · 33 unmodified lines | ||
| 34 | 34 | struct page *page = param; |
| 35 | 35 | void *dst; |
| 36 | 36 | void *src; |
| 37 | − long speed = RISCV_HWPROBE_MISALIGNED_SLOW; | |
| 37 | + long speed = RISCV_HWPROBE_MISALIGNED_SCALAR_SLOW; | |
| 38 | 38 | |
| 39 | − if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_UNKNOWN) | |
| 39 | + if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN) | |
| 40 | 40 | return 0; |
| 41 | 41 | |
| 42 | 42 | /* Make an unaligned destination buffer. */ |
| @@ -95,14 +95,14 @@ static int check_unaligned_access(void *param) · 52 unmodified lines | ||
| 95 | 95 | } |
| 96 | 96 | |
| 97 | 97 | if (word_cycles < byte_cycles) |
| 98 | − speed = RISCV_HWPROBE_MISALIGNED_FAST; | |
| 98 | + speed = RISCV_HWPROBE_MISALIGNED_SCALAR_FAST; | |
| 99 | 99 | |
| 100 | 100 | ratio = div_u64((byte_cycles * 100), word_cycles); |
| 101 | 101 | pr_info("cpu%d: Ratio of byte access time to unaligned word access is %d.%02d, unaligned accesses are %s\n", |
| 102 | 102 | cpu, |
| 103 | 103 | ratio / 100, |
| 104 | 104 | ratio % 100, |
| 105 | − (speed == RISCV_HWPROBE_MISALIGNED_FAST) ? "fast" : "slow"); | |
| 105 | + (speed == RISCV_HWPROBE_MISALIGNED_SCALAR_FAST) ? "fast" : "slow"); | |
| 106 | 106 | |
| 107 | 107 | per_cpu(misaligned_access_speed, cpu) = speed; |
| 108 | 108 | |
| @@ -110,7 +110,7 @@ static int check_unaligned_access(void *param) · 1 unmodified line | ||
| 110 | 110 | * Set the value of fast_misaligned_access of a CPU. These operations |
| 111 | 111 | * are atomic to avoid race conditions. |
| 112 | 112 | */ |
| 113 | − if (speed == RISCV_HWPROBE_MISALIGNED_FAST) | |
| 113 | + if (speed == RISCV_HWPROBE_MISALIGNED_SCALAR_FAST) | |
| 114 | 114 | cpumask_set_cpu(cpu, &fast_misaligned_access); |
| 115 | 115 | else |
| 116 | 116 | cpumask_clear_cpu(cpu, &fast_misaligned_access); |
| @@ -188,7 +188,7 @@ static int riscv_online_cpu(unsigned int cpu) · 71 unmodified lines | ||
| 188 | 188 | static struct page *buf; |
| 189 | 189 | |
| 190 | 190 | /* We are already set since the last check */ |
| 191 | − if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_UNKNOWN) | |
| 191 | + if (per_cpu(misaligned_access_speed, cpu) != RISCV_HWPROBE_MISALIGNED_SCALAR_UNKNOWN) | |
| 192 | 192 | goto exit; |
| 193 | 193 | |
| 194 | 194 | buf = alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER); |
| @@ -38,7 +38,7 @@ bool __riscv_isa_vendor_extension_available(int cpu, unsigned long vendor, unsig · 37 unmodified lines | ||
| 38 | 38 | #ifdef CONFIG_RISCV_ISA_VENDOR_EXT_ANDES |
| 39 | 39 | case ANDES_VENDOR_ID: |
| 40 | 40 | bmap = &riscv_isa_vendor_ext_list_andes.all_harts_isa_bitmap; |
| 41 | − cpu_bmap = &riscv_isa_vendor_ext_list_andes.per_hart_isa_bitmap[cpu]; | |
| 41 | + cpu_bmap = riscv_isa_vendor_ext_list_andes.per_hart_isa_bitmap; | |
| 42 | 42 | break; |
| 43 | 43 | #endif |
| 44 | 44 | default: |
| @@ -61,26 +61,27 @@ static inline void no_context(struct pt_regs *regs, unsigned long addr) · 60 unmodified lines | ||
| 61 | 61 | |
| 62 | 62 | static inline void mm_fault_error(struct pt_regs *regs, unsigned long addr, vm_fault_t fault) |
| 63 | 63 | { |
| 64 | + if (!user_mode(regs)) { | |
| 65 | + no_context(regs, addr); | |
| 66 | + return; | |
| 67 | + } | |
| 68 | + | |
| 64 | 69 | if (fault & VM_FAULT_OOM) { |
| 65 | 70 | /* |
| 66 | 71 | * We ran out of memory, call the OOM killer, and return the userspace |
| 67 | 72 | * (which will retry the fault, or kill us if we got oom-killed). |
| 68 | 73 | */ |
| 69 | − if (!user_mode(regs)) { | |
| 70 | − no_context(regs, addr); | |
| 71 | − return; | |
| 72 | − } | |
| 73 | 74 | pagefault_out_of_memory(); |
| 74 | 75 | return; |
| 75 | 76 | } else if (fault & (VM_FAULT_SIGBUS | VM_FAULT_HWPOISON | VM_FAULT_HWPOISON_LARGE)) { |
| 76 | 77 | /* Kernel mode? Handle exceptions or die */ |
| 77 | − if (!user_mode(regs)) { | |
| 78 | − no_context(regs, addr); | |
| 79 | − return; | |
| 80 | − } | |
| 81 | 78 | do_trap(regs, SIGBUS, BUS_ADRERR, addr); |
| 82 | 79 | return; |
| 80 | + } else if (fault & VM_FAULT_SIGSEGV) { | |
| 81 | + do_trap(regs, SIGSEGV, SEGV_MAPERR, addr); | |
| 82 | + return; | |
| 83 | 83 | } |
| 84 | + | |
| 84 | 85 | BUG(); |
| 85 | 86 | } |
| 86 | 87 | |
| @@ -234,8 +234,6 @@ static void __init setup_bootmem(void) · 233 unmodified lines | ||
| 234 | 234 | */ |
| 235 | 235 | memblock_reserve(vmlinux_start, vmlinux_end - vmlinux_start); |
| 236 | 236 | |
| 237 | − phys_ram_end = memblock_end_of_DRAM(); | |
| 238 | − | |
| 239 | 237 | /* |
| 240 | 238 | * Make sure we align the start of the memory on a PMD boundary so that |
| 241 | 239 | * at worst, we map the linear mapping with PMD mappings. |
| @@ -250,6 +248,16 @@ static void __init setup_bootmem(void) · 8 unmodified lines | ||
| 250 | 248 | if (IS_ENABLED(CONFIG_64BIT) && IS_ENABLED(CONFIG_MMU)) |
| 251 | 249 | kernel_map.va_pa_offset = PAGE_OFFSET - phys_ram_base; |
| 252 | 250 | |
| 251 | + /* | |
| 252 | + * The size of the linear page mapping may restrict the amount of | |
| 253 | + * usable RAM. | |
| 254 | + */ | |
| 255 | + if (IS_ENABLED(CONFIG_64BIT)) { | |
| 256 | + max_mapped_addr = __pa(PAGE_OFFSET) + KERN_VIRT_SIZE; | |
| 257 | + memblock_cap_memory_range(phys_ram_base, | |
| 258 | + max_mapped_addr - phys_ram_base); | |
| 259 | + } | |
| 260 | + | |
| 253 | 261 | /* |
| 254 | 262 | * Reserve physical address space that would be mapped to virtual |
| 255 | 263 | * addresses greater than (void *)(-PAGE_SIZE) because: |
| @@ -266,6 +274,7 @@ static void __init setup_bootmem(void) · 10 unmodified lines | ||
| 266 | 274 | memblock_reserve(max_mapped_addr, (phys_addr_t)-max_mapped_addr); |
| 267 | 275 | } |
| 268 | 276 | |
| 277 | + phys_ram_end = memblock_end_of_DRAM(); | |
| 269 | 278 | min_low_pfn = PFN_UP(phys_ram_base); |
| 270 | 279 | max_low_pfn = max_pfn = PFN_DOWN(phys_ram_end); |
| 271 | 280 | high_memory = (void *)(__va(PFN_PHYS(max_low_pfn))); |
| @@ -918,7 +927,7 @@ static void __init create_kernel_page_table(pgd_t *pgdir, · 646 unmodified lines | ||
| 918 | 927 | PMD_SIZE, PAGE_KERNEL_EXEC); |
| 919 | 928 | |
| 920 | 929 | /* Map the data in RAM */ |
| 921 | − end_va = kernel_map.virt_addr + XIP_OFFSET + kernel_map.size; | |
| 930 | + end_va = kernel_map.virt_addr + kernel_map.size; | |
| 922 | 931 | for (va = kernel_map.virt_addr + XIP_OFFSET; va < end_va; va += PMD_SIZE) |
| 923 | 932 | create_pgd_mapping(pgdir, va, |
| 924 | 933 | kernel_map.phys_addr + (va - (kernel_map.virt_addr + XIP_OFFSET)), |
| @@ -1087,7 +1096,7 @@ asmlinkage void __init setup_vm(uintptr_t dtb_pa) · 162 unmodified lines | ||
| 1087 | 1096 | |
| 1088 | 1097 | phys_ram_base = CONFIG_PHYS_RAM_BASE; |
| 1089 | 1098 | kernel_map.phys_addr = (uintptr_t)CONFIG_PHYS_RAM_BASE; |
| 1090 | − kernel_map.size = (uintptr_t)(&_end) - (uintptr_t)(&_sdata); | |
| 1099 | + kernel_map.size = (uintptr_t)(&_end) - (uintptr_t)(&_start); | |
| 1091 | 1100 | |
| 1092 | 1101 | kernel_map.va_kernel_xip_pa_offset = kernel_map.virt_addr - kernel_map.xiprom; |
| 1093 | 1102 | #else |
| @@ -1284,8 +1293,6 @@ static void __init create_linear_mapping_page_table(void) · 190 unmodified lines | ||
| 1284 | 1293 | if (start <= __pa(PAGE_OFFSET) && |
| 1285 | 1294 | __pa(PAGE_OFFSET) < end) |
| 1286 | 1295 | start = __pa(PAGE_OFFSET); |
| 1287 | − if (end >= __pa(PAGE_OFFSET) + memory_limit) | |
| 1288 | − end = __pa(PAGE_OFFSET) + memory_limit; | |
| 1289 | 1296 | |
| 1290 | 1297 | create_linear_mapping_range(start, end, 0, NULL); |
| 1291 | 1298 | } |
| @@ -7,6 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | * Author: Li Zhengyu (lizhengyu3@huawei.com) |
| 8 | 8 | * |
| 9 | 9 | */ |
| 10 | +#include <asm/asm.h> | |
| 10 | 11 | #include <linux/linkage.h> |
| 11 | 12 | |
| 12 | 13 | .text |
| @@ -34,6 +35,7 @@ SYM_CODE_END(purgatory_start) · 21 unmodified lines | ||
| 34 | 35 | |
| 35 | 36 | .data |
| 36 | 37 | |
| 38 | +.align LGREG | |
| 37 | 39 | SYM_DATA(riscv_kernel_entry, .quad 0) |
| 38 | 40 | |
| 39 | 41 | .end |
| @@ -604,6 +604,19 @@ config RANDOMIZE_BASE · 603 unmodified lines | ||
| 604 | 604 | as a security feature that deters exploit attempts relying on |
| 605 | 605 | knowledge of the location of kernel internals. |
| 606 | 606 | |
| 607 | +config RANDOMIZE_IDENTITY_BASE | |
| 608 | + bool "Randomize the address of the identity mapping base" | |
| 609 | + depends on RANDOMIZE_BASE | |
| 610 | + default DEBUG_VM | |
| 611 | + help | |
| 612 | + The identity mapping base address is pinned to zero by default. | |
| 613 | + Allow randomization of that base to expose otherwise missed | |
| 614 | + notion of physical and virtual addresses of data structures. | |
| 615 | + That does not have any impact on the base address at which the | |
| 616 | + kernel image is loaded. | |
| 617 | + | |
| 618 | + If unsure, say N | |
| 619 | + | |
| 607 | 620 | config KERNEL_IMAGE_BASE |
| 608 | 621 | hex "Kernel image base address" |
| 609 | 622 | range 0x100000 0x1FFFFFE0000000 if !KASAN |
| @@ -162,7 +162,7 @@ static void kaslr_adjust_relocs(unsigned long min_addr, unsigned long max_addr, · 161 unmodified lines | ||
| 162 | 162 | loc = (long)*reloc + phys_offset; |
| 163 | 163 | if (loc < min_addr || loc > max_addr) |
| 164 | 164 | error("64-bit relocation outside of kernel!\n"); |
| 165 | − *(u64 *)loc += offset - __START_KERNEL; | |
| 165 | + *(u64 *)loc += offset; | |
| 166 | 166 | } |
| 167 | 167 | } |
| 168 | 168 | |
| @@ -177,7 +177,7 @@ static void kaslr_adjust_got(unsigned long offset) · 8 unmodified lines | ||
| 177 | 177 | */ |
| 178 | 178 | for (entry = (u64 *)vmlinux.got_start; entry < (u64 *)vmlinux.got_end; entry++) { |
| 179 | 179 | if (*entry) |
| 180 | − *entry += offset - __START_KERNEL; | |
| 180 | + *entry += offset; | |
| 181 | 181 | } |
| 182 | 182 | } |
| 183 | 183 | |
| @@ -252,7 +252,7 @@ static unsigned long setup_kernel_memory_layout(unsigned long kernel_size) · 68 unmodified lines | ||
| 252 | 252 | vmemmap_size = SECTION_ALIGN_UP(pages) * sizeof(struct page); |
| 253 | 253 | |
| 254 | 254 | /* choose kernel address space layout: 4 or 3 levels. */ |
| 255 | − BUILD_BUG_ON(!IS_ALIGNED(__START_KERNEL, THREAD_SIZE)); | |
| 255 | + BUILD_BUG_ON(!IS_ALIGNED(TEXT_OFFSET, THREAD_SIZE)); | |
| 256 | 256 | BUILD_BUG_ON(!IS_ALIGNED(__NO_KASLR_START_KERNEL, THREAD_SIZE)); |
| 257 | 257 | BUILD_BUG_ON(__NO_KASLR_END_KERNEL > _REGION1_SIZE); |
| 258 | 258 | vsize = get_vmem_size(ident_map_size, vmemmap_size, vmalloc_size, _REGION3_SIZE); |
| @@ -341,7 +341,8 @@ static unsigned long setup_kernel_memory_layout(unsigned long kernel_size) · 82 unmodified lines | ||
| 341 | 341 | BUILD_BUG_ON(MAX_DCSS_ADDR > (1UL << MAX_PHYSMEM_BITS)); |
| 342 | 342 | max_mappable = max(ident_map_size, MAX_DCSS_ADDR); |
| 343 | 343 | max_mappable = min(max_mappable, vmemmap_start); |
| 344 | − __identity_base = round_down(vmemmap_start - max_mappable, rte_size); | |
| 344 | + if (IS_ENABLED(CONFIG_RANDOMIZE_IDENTITY_BASE)) | |
| 345 | + __identity_base = round_down(vmemmap_start - max_mappable, rte_size); | |
| 345 | 346 | |
| 346 | 347 | return asce_limit; |
| 347 | 348 | } |
| @@ -388,31 +389,25 @@ static void kaslr_adjust_vmlinux_info(long offset) · 40 unmodified lines | ||
| 388 | 389 | #endif |
| 389 | 390 | } |
| 390 | 391 | |
| 391 | −static void fixup_vmlinux_info(void) | |
| 392 | −{ | |
| 393 | − vmlinux.entry -= __START_KERNEL; | |
| 394 | − kaslr_adjust_vmlinux_info(-__START_KERNEL); | |
| 395 | −} | |
| 396 | − | |
| 397 | 392 | void startup_kernel(void) |
| 398 | 393 | { |
| 399 | − unsigned long kernel_size = vmlinux.image_size + vmlinux.bss_size; | |
| 400 | − unsigned long nokaslr_offset_phys, kaslr_large_page_offset; | |
| 401 | − unsigned long amode31_lma = 0; | |
| 394 | + unsigned long vmlinux_size = vmlinux.image_size + vmlinux.bss_size; | |
| 395 | + unsigned long nokaslr_text_lma, text_lma = 0, amode31_lma = 0; | |
| 396 | + unsigned long kernel_size = TEXT_OFFSET + vmlinux_size; | |
| 397 | + unsigned long kaslr_large_page_offset; | |
| 402 | 398 | unsigned long max_physmem_end; |
| 403 | 399 | unsigned long asce_limit; |
| 404 | 400 | unsigned long safe_addr; |
| 405 | 401 | psw_t psw; |
| 406 | 402 | |
| 407 | − fixup_vmlinux_info(); | |
| 408 | 403 | setup_lpp(); |
| 409 | 404 | |
| 410 | 405 | /* |
| 411 | 406 | * Non-randomized kernel physical start address must be _SEGMENT_SIZE |
| 412 | 407 | * aligned (see blow). |
| 413 | 408 | */ |
| 414 | − nokaslr_offset_phys = ALIGN(mem_safe_offset(), _SEGMENT_SIZE); | |
| 415 | − safe_addr = PAGE_ALIGN(nokaslr_offset_phys + kernel_size); | |
| 409 | + nokaslr_text_lma = ALIGN(mem_safe_offset(), _SEGMENT_SIZE); | |
| 410 | + safe_addr = PAGE_ALIGN(nokaslr_text_lma + vmlinux_size); | |
| 416 | 411 | |
| 417 | 412 | /* |
| 418 | 413 | * Reserve decompressor memory together with decompression heap, |
| @@ -456,16 +451,27 @@ void startup_kernel(void) · 37 unmodified lines | ||
| 456 | 451 | */ |
| 457 | 452 | kaslr_large_page_offset = __kaslr_offset & ~_SEGMENT_MASK; |
| 458 | 453 | if (kaslr_enabled()) { |
| 459 | − unsigned long end = ident_map_size - kaslr_large_page_offset; | |
| 454 | + unsigned long size = vmlinux_size + kaslr_large_page_offset; | |
| 460 | 455 | |
| 461 | − __kaslr_offset_phys = randomize_within_range(kernel_size, _SEGMENT_SIZE, 0, end); | |
| 456 | + text_lma = randomize_within_range(size, _SEGMENT_SIZE, TEXT_OFFSET, ident_map_size); | |
| 462 | 457 | } |
| 463 | − if (!__kaslr_offset_phys) | |
| 464 | − __kaslr_offset_phys = nokaslr_offset_phys; | |
| 465 | − __kaslr_offset_phys |= kaslr_large_page_offset; | |
| 458 | + if (!text_lma) | |
| 459 | + text_lma = nokaslr_text_lma; | |
| 460 | + text_lma |= kaslr_large_page_offset; | |
| 461 | + | |
| 462 | + /* | |
| 463 | + * [__kaslr_offset_phys..__kaslr_offset_phys + TEXT_OFFSET] region is | |
| 464 | + * never accessed via the kernel image mapping as per the linker script: | |
| 465 | + * | |
| 466 | + * . = TEXT_OFFSET; | |
| 467 | + * | |
| 468 | + * Therefore, this region could be used for something else and does | |
| 469 | + * not need to be reserved. See how it is skipped in setup_vmem(). | |
| 470 | + */ | |
| 471 | + __kaslr_offset_phys = text_lma - TEXT_OFFSET; | |
| 466 | 472 | kaslr_adjust_vmlinux_info(__kaslr_offset_phys); |
| 467 | − physmem_reserve(RR_VMLINUX, __kaslr_offset_phys, kernel_size); | |
| 468 | − deploy_kernel((void *)__kaslr_offset_phys); | |
| 473 | + physmem_reserve(RR_VMLINUX, text_lma, vmlinux_size); | |
| 474 | + deploy_kernel((void *)text_lma); | |
| 469 | 475 | |
| 470 | 476 | /* vmlinux decompression is done, shrink reserved low memory */ |
| 471 | 477 | physmem_reserve(RR_DECOMPRESSOR, 0, (unsigned long)_decompressor_end); |
| @@ -488,7 +494,7 @@ void startup_kernel(void) · 16 unmodified lines | ||
| 488 | 494 | amode31_lma = randomize_within_range(vmlinux.amode31_size, PAGE_SIZE, amode31_min, SZ_2G); |
| 489 | 495 | } |
| 490 | 496 | if (!amode31_lma) |
| 491 | − amode31_lma = __kaslr_offset_phys - vmlinux.amode31_size; | |
| 497 | + amode31_lma = text_lma - vmlinux.amode31_size; | |
| 492 | 498 | physmem_reserve(RR_AMODE31, amode31_lma, vmlinux.amode31_size); |
| 493 | 499 | |
| 494 | 500 | /* |
| @@ -504,8 +510,8 @@ void startup_kernel(void) · 9 unmodified lines | ||
| 504 | 510 | * - copy_bootdata() must follow setup_vmem() to propagate changes |
| 505 | 511 | * to bootdata made by setup_vmem() |
| 506 | 512 | */ |
| 507 | − clear_bss_section(__kaslr_offset_phys); | |
| 508 | − kaslr_adjust_relocs(__kaslr_offset_phys, __kaslr_offset_phys + vmlinux.image_size, | |
| 513 | + clear_bss_section(text_lma); | |
| 514 | + kaslr_adjust_relocs(text_lma, text_lma + vmlinux.image_size, | |
| 509 | 515 | __kaslr_offset, __kaslr_offset_phys); |
| 510 | 516 | kaslr_adjust_got(__kaslr_offset); |
| 511 | 517 | setup_vmem(__kaslr_offset, __kaslr_offset + kernel_size, asce_limit); |
| @@ -90,7 +90,7 @@ static void kasan_populate_shadow(unsigned long kernel_start, unsigned long kern · 89 unmodified lines | ||
| 90 | 90 | } |
| 91 | 91 | memgap_start = end; |
| 92 | 92 | } |
| 93 | − kasan_populate(kernel_start, kernel_end, POPULATE_KASAN_MAP_SHADOW); | |
| 93 | + kasan_populate(kernel_start + TEXT_OFFSET, kernel_end, POPULATE_KASAN_MAP_SHADOW); | |
| 94 | 94 | kasan_populate(0, (unsigned long)__identity_va(0), POPULATE_KASAN_ZERO_SHADOW); |
| 95 | 95 | kasan_populate(AMODE31_START, AMODE31_END, POPULATE_KASAN_ZERO_SHADOW); |
| 96 | 96 | if (IS_ENABLED(CONFIG_KASAN_VMALLOC)) { |
| @@ -475,7 +475,17 @@ void setup_vmem(unsigned long kernel_start, unsigned long kernel_end, unsigned l · 378 unmodified lines | ||
| 475 | 475 | (unsigned long)__identity_va(end), |
| 476 | 476 | POPULATE_IDENTITY); |
| 477 | 477 | } |
| 478 | − pgtable_populate(kernel_start, kernel_end, POPULATE_KERNEL); | |
| 478 | + | |
| 479 | + /* | |
| 480 | + * [kernel_start..kernel_start + TEXT_OFFSET] region is never | |
| 481 | + * accessed as per the linker script: | |
| 482 | + * | |
| 483 | + * . = TEXT_OFFSET; | |
| 484 | + * | |
| 485 | + * Therefore, skip mapping TEXT_OFFSET bytes to prevent access to | |
| 486 | + * [__kaslr_offset_phys..__kaslr_offset_phys + TEXT_OFFSET] region. | |
| 487 | + */ | |
| 488 | + pgtable_populate(kernel_start + TEXT_OFFSET, kernel_end, POPULATE_KERNEL); | |
| 479 | 489 | pgtable_populate(AMODE31_START, AMODE31_END, POPULATE_DIRECT); |
| 480 | 490 | pgtable_populate(__abs_lowcore, __abs_lowcore + sizeof(struct lowcore), |
| 481 | 491 | POPULATE_ABS_LOWCORE); |
| @@ -109,7 +109,12 @@ SECTIONS · 108 unmodified lines | ||
| 109 | 109 | #ifdef CONFIG_KERNEL_UNCOMPRESSED |
| 110 | 110 | . = ALIGN(PAGE_SIZE); |
| 111 | 111 | . += AMODE31_SIZE; /* .amode31 section */ |
| 112 | − . = ALIGN(1 << 20); /* _SEGMENT_SIZE */ | |
| 112 | + | |
| 113 | + /* | |
| 114 | + * Make sure the location counter is not less than TEXT_OFFSET. | |
| 115 | + * _SEGMENT_SIZE is not available, use ALIGN(1 << 20) instead. | |
| 116 | + */ | |
| 117 | + . = MAX(TEXT_OFFSET, ALIGN(1 << 20)); | |
| 113 | 118 | #else |
| 114 | 119 | . = ALIGN(8); |
| 115 | 120 | #endif |
| @@ -279,8 +279,9 @@ static inline unsigned long virt_to_pfn(const void *kaddr) · 278 unmodified lines | ||
| 279 | 279 | #define AMODE31_SIZE (3 * PAGE_SIZE) |
| 280 | 280 | |
| 281 | 281 | #define KERNEL_IMAGE_SIZE (512 * 1024 * 1024) |
| 282 | −#define __START_KERNEL 0x100000 | |
| 283 | 282 | #define __NO_KASLR_START_KERNEL CONFIG_KERNEL_IMAGE_BASE |
| 284 | 283 | #define __NO_KASLR_END_KERNEL (__NO_KASLR_START_KERNEL + KERNEL_IMAGE_SIZE) |
| 285 | 284 | |
| 285 | +#define TEXT_OFFSET 0x100000 | |
| 286 | + | |
| 286 | 287 | #endif /* _S390_PAGE_H */ |
| @@ -441,7 +441,10 @@ static inline int share(unsigned long addr, u16 cmd) · 440 unmodified lines | ||
| 441 | 441 | |
| 442 | 442 | if (!uv_call(0, (u64)&uvcb)) |
| 443 | 443 | return 0; |
| 444 | − return -EINVAL; | |
| 444 | + pr_err("%s UVC failed (rc: 0x%x, rrc: 0x%x), possible hypervisor bug.\n", | |
| 445 | + uvcb.header.cmd == UVC_CMD_SET_SHARED_ACCESS ? "Share" : "Unshare", | |
| 446 | + uvcb.header.rc, uvcb.header.rrc); | |
| 447 | + panic("System security cannot be guaranteed unless the system panics now.\n"); | |
| 445 | 448 | } |
| 446 | 449 | |
| 447 | 450 | /* |
Empty file deleted.
| @@ -113,7 +113,7 @@ void load_fpu_state(struct fpu *state, int flags) · 112 unmodified lines | ||
| 113 | 113 | int mask; |
| 114 | 114 | |
| 115 | 115 | if (flags & KERNEL_FPC) |
| 116 | − fpu_lfpc(&state->fpc); | |
| 116 | + fpu_lfpc_safe(&state->fpc); | |
| 117 | 117 | if (!cpu_has_vx()) { |
| 118 | 118 | if (flags & KERNEL_VXR_V0V7) |
| 119 | 119 | load_fp_regs_vx(state->vxrs); |
| @@ -734,7 +734,23 @@ static void __init memblock_add_physmem_info(void) · 733 unmodified lines | ||
| 734 | 734 | } |
| 735 | 735 | |
| 736 | 736 | /* |
| 737 | − * Reserve memory used for lowcore/command line/kernel image. | |
| 737 | + * Reserve memory used for lowcore. | |
| 738 | + */ | |
| 739 | +static void __init reserve_lowcore(void) | |
| 740 | +{ | |
| 741 | + void *lowcore_start = get_lowcore(); | |
| 742 | + void *lowcore_end = lowcore_start + sizeof(struct lowcore); | |
| 743 | + void *start, *end; | |
| 744 | + | |
| 745 | + if ((void *)__identity_base < lowcore_end) { | |
| 746 | + start = max(lowcore_start, (void *)__identity_base); | |
| 747 | + end = min(lowcore_end, (void *)(__identity_base + ident_map_size)); | |
| 748 | + memblock_reserve(__pa(start), __pa(end)); | |
| 749 | + } | |
| 750 | +} | |
| 751 | + | |
| 752 | +/* | |
| 753 | + * Reserve memory used for absolute lowcore/command line/kernel image. | |
| 738 | 754 | */ |
| 739 | 755 | static void __init reserve_kernel(void) |
| 740 | 756 | { |
| @@ -918,6 +934,7 @@ void __init setup_arch(char **cmdline_p) · 177 unmodified lines | ||
| 918 | 934 | |
| 919 | 935 | /* Do some memory reservations *before* memory is added to memblock */ |
| 920 | 936 | reserve_pgtables(); |
| 937 | + reserve_lowcore(); | |
| 921 | 938 | reserve_kernel(); |
| 922 | 939 | reserve_initrd(); |
| 923 | 940 | reserve_certificate_list(); |
| @@ -39,7 +39,7 @@ PHDRS { · 38 unmodified lines | ||
| 39 | 39 | |
| 40 | 40 | SECTIONS |
| 41 | 41 | { |
| 42 | − . = __START_KERNEL; | |
| 42 | + . = TEXT_OFFSET; | |
| 43 | 43 | .text : { |
| 44 | 44 | _stext = .; /* Start of text section */ |
| 45 | 45 | _text = .; /* Text and read-only data */ |
| @@ -59,14 +59,6 @@ SECTIONS · 13 unmodified lines | ||
| 59 | 59 | } :text = 0x0700 |
| 60 | 60 | |
| 61 | 61 | RO_DATA(PAGE_SIZE) |
| 62 | − .data.rel.ro : { | |
| 63 | − *(.data.rel.ro .data.rel.ro.*) | |
| 64 | − } | |
| 65 | − .got : { | |
| 66 | − __got_start = .; | |
| 67 | − *(.got) | |
| 68 | − __got_end = .; | |
| 69 | − } | |
| 70 | 62 | |
| 71 | 63 | . = ALIGN(PAGE_SIZE); |
| 72 | 64 | _sdata = .; /* Start of data section */ |
| @@ -80,6 +72,15 @@ SECTIONS · 7 unmodified lines | ||
| 80 | 72 | . = ALIGN(PAGE_SIZE); |
| 81 | 73 | __end_ro_after_init = .; |
| 82 | 74 | |
| 75 | + .data.rel.ro : { | |
| 76 | + *(.data.rel.ro .data.rel.ro.*) | |
| 77 | + } | |
| 78 | + .got : { | |
| 79 | + __got_start = .; | |
| 80 | + *(.got) | |
| 81 | + __got_end = .; | |
| 82 | + } | |
| 83 | + | |
| 83 | 84 | RW_DATA(0x100, PAGE_SIZE, THREAD_SIZE) |
| 84 | 85 | .data.rel : { |
| 85 | 86 | *(.data.rel*) |
| @@ -267,7 +267,12 @@ static inline unsigned long kvm_s390_get_gfn_end(struct kvm_memslots *slots) · 266 unmodified lines | ||
| 267 | 267 | |
| 268 | 268 | static inline u32 kvm_s390_get_gisa_desc(struct kvm *kvm) |
| 269 | 269 | { |
| 270 | − u32 gd = virt_to_phys(kvm->arch.gisa_int.origin); | |
| 270 | + u32 gd; | |
| 271 | + | |
| 272 | + if (!kvm->arch.gisa_int.origin) | |
| 273 | + return 0; | |
| 274 | + | |
| 275 | + gd = virt_to_phys(kvm->arch.gisa_int.origin); | |
| 271 | 276 | |
| 272 | 277 | if (gd && sclp.has_gisaf) |
| 273 | 278 | gd |= GISA_FORMAT1; |
| @@ -3,6 +3,7 @@ · 2 unmodified lines | ||
| 3 | 3 | #include <linux/ptdump.h> |
| 4 | 4 | #include <linux/seq_file.h> |
| 5 | 5 | #include <linux/debugfs.h> |
| 6 | +#include <linux/sort.h> | |
| 6 | 7 | #include <linux/mm.h> |
| 7 | 8 | #include <linux/kfence.h> |
| 8 | 9 | #include <linux/kasan.h> |
| @@ -15,13 +16,15 @@ · 6 unmodified lines | ||
| 15 | 16 | static unsigned long max_addr; |
| 16 | 17 | |
| 17 | 18 | struct addr_marker { |
| 19 | + int is_start; | |
| 18 | 20 | unsigned long start_address; |
| 19 | 21 | const char *name; |
| 20 | 22 | }; |
| 21 | 23 | |
| 22 | 24 | enum address_markers_idx { |
| 23 | − IDENTITY_BEFORE_NR = 0, | |
| 24 | − IDENTITY_BEFORE_END_NR, | |
| 25 | + KVA_NR = 0, | |
| 26 | + LOWCORE_START_NR, | |
| 27 | + LOWCORE_END_NR, | |
| 25 | 28 | AMODE31_START_NR, |
| 26 | 29 | AMODE31_END_NR, |
| 27 | 30 | KERNEL_START_NR, |
| @@ -30,8 +33,8 @@ enum address_markers_idx { · 2 unmodified lines | ||
| 30 | 33 | KFENCE_START_NR, |
| 31 | 34 | KFENCE_END_NR, |
| 32 | 35 | #endif |
| 33 | − IDENTITY_AFTER_NR, | |
| 34 | − IDENTITY_AFTER_END_NR, | |
| 36 | + IDENTITY_START_NR, | |
| 37 | + IDENTITY_END_NR, | |
| 35 | 38 | VMEMMAP_NR, |
| 36 | 39 | VMEMMAP_END_NR, |
| 37 | 40 | VMALLOC_NR, |
| @@ -59,43 +62,44 @@ enum address_markers_idx { · 21 unmodified lines | ||
| 59 | 62 | }; |
| 60 | 63 | |
| 61 | 64 | static struct addr_marker address_markers[] = { |
| 62 | − [IDENTITY_BEFORE_NR] = {0, "Identity Mapping Start"}, | |
| 63 | − [IDENTITY_BEFORE_END_NR] = {(unsigned long)_stext, "Identity Mapping End"}, | |
| 64 | − [AMODE31_START_NR] = {0, "Amode31 Area Start"}, | |
| 65 | − [AMODE31_END_NR] = {0, "Amode31 Area End"}, | |
| 66 | − [KERNEL_START_NR] = {(unsigned long)_stext, "Kernel Image Start"}, | |
| 67 | − [KERNEL_END_NR] = {(unsigned long)_end, "Kernel Image End"}, | |
| 65 | + [KVA_NR] = {0, 0, "Kernel Virtual Address Space"}, | |
| 66 | + [LOWCORE_START_NR] = {1, 0, "Lowcore Start"}, | |
| 67 | + [LOWCORE_END_NR] = {0, 0, "Lowcore End"}, | |
| 68 | + [IDENTITY_START_NR] = {1, 0, "Identity Mapping Start"}, | |
| 69 | + [IDENTITY_END_NR] = {0, 0, "Identity Mapping End"}, | |
| 70 | + [AMODE31_START_NR] = {1, 0, "Amode31 Area Start"}, | |
| 71 | + [AMODE31_END_NR] = {0, 0, "Amode31 Area End"}, | |
| 72 | + [KERNEL_START_NR] = {1, (unsigned long)_stext, "Kernel Image Start"}, | |
| 73 | + [KERNEL_END_NR] = {0, (unsigned long)_end, "Kernel Image End"}, | |
| 68 | 74 | #ifdef CONFIG_KFENCE |
| 69 | − [KFENCE_START_NR] = {0, "KFence Pool Start"}, | |
| 70 | − [KFENCE_END_NR] = {0, "KFence Pool End"}, | |
| 75 | + [KFENCE_START_NR] = {1, 0, "KFence Pool Start"}, | |
| 76 | + [KFENCE_END_NR] = {0, 0, "KFence Pool End"}, | |
| 71 | 77 | #endif |
| 72 | − [IDENTITY_AFTER_NR] = {(unsigned long)_end, "Identity Mapping Start"}, | |
| 73 | − [IDENTITY_AFTER_END_NR] = {0, "Identity Mapping End"}, | |
| 74 | − [VMEMMAP_NR] = {0, "vmemmap Area Start"}, | |
| 75 | − [VMEMMAP_END_NR] = {0, "vmemmap Area End"}, | |
| 76 | − [VMALLOC_NR] = {0, "vmalloc Area Start"}, | |
| 77 | − [VMALLOC_END_NR] = {0, "vmalloc Area End"}, | |
| 78 | + [VMEMMAP_NR] = {1, 0, "vmemmap Area Start"}, | |
| 79 | + [VMEMMAP_END_NR] = {0, 0, "vmemmap Area End"}, | |
| 80 | + [VMALLOC_NR] = {1, 0, "vmalloc Area Start"}, | |
| 81 | + [VMALLOC_END_NR] = {0, 0, "vmalloc Area End"}, | |
| 78 | 82 | #ifdef CONFIG_KMSAN |
| 79 | − [KMSAN_VMALLOC_SHADOW_START_NR] = {0, "Kmsan vmalloc Shadow Start"}, | |
| 80 | − [KMSAN_VMALLOC_SHADOW_END_NR] = {0, "Kmsan vmalloc Shadow End"}, | |
| 81 | − [KMSAN_VMALLOC_ORIGIN_START_NR] = {0, "Kmsan vmalloc Origins Start"}, | |
| 82 | − [KMSAN_VMALLOC_ORIGIN_END_NR] = {0, "Kmsan vmalloc Origins End"}, | |
| 83 | − [KMSAN_MODULES_SHADOW_START_NR] = {0, "Kmsan Modules Shadow Start"}, | |
| 84 | − [KMSAN_MODULES_SHADOW_END_NR] = {0, "Kmsan Modules Shadow End"}, | |
| 85 | − [KMSAN_MODULES_ORIGIN_START_NR] = {0, "Kmsan Modules Origins Start"}, | |
| 86 | − [KMSAN_MODULES_ORIGIN_END_NR] = {0, "Kmsan Modules Origins End"}, | |
| 83 | + [KMSAN_VMALLOC_SHADOW_START_NR] = {1, 0, "Kmsan vmalloc Shadow Start"}, | |
| 84 | + [KMSAN_VMALLOC_SHADOW_END_NR] = {0, 0, "Kmsan vmalloc Shadow End"}, | |
| 85 | + [KMSAN_VMALLOC_ORIGIN_START_NR] = {1, 0, "Kmsan vmalloc Origins Start"}, | |
| 86 | + [KMSAN_VMALLOC_ORIGIN_END_NR] = {0, 0, "Kmsan vmalloc Origins End"}, | |
| 87 | + [KMSAN_MODULES_SHADOW_START_NR] = {1, 0, "Kmsan Modules Shadow Start"}, | |
| 88 | + [KMSAN_MODULES_SHADOW_END_NR] = {0, 0, "Kmsan Modules Shadow End"}, | |
| 89 | + [KMSAN_MODULES_ORIGIN_START_NR] = {1, 0, "Kmsan Modules Origins Start"}, | |
| 90 | + [KMSAN_MODULES_ORIGIN_END_NR] = {0, 0, "Kmsan Modules Origins End"}, | |
| 87 | 91 | #endif |
| 88 | − [MODULES_NR] = {0, "Modules Area Start"}, | |
| 89 | − [MODULES_END_NR] = {0, "Modules Area End"}, | |
| 90 | − [ABS_LOWCORE_NR] = {0, "Lowcore Area Start"}, | |
| 91 | − [ABS_LOWCORE_END_NR] = {0, "Lowcore Area End"}, | |
| 92 | − [MEMCPY_REAL_NR] = {0, "Real Memory Copy Area Start"}, | |
| 93 | − [MEMCPY_REAL_END_NR] = {0, "Real Memory Copy Area End"}, | |
| 92 | + [MODULES_NR] = {1, 0, "Modules Area Start"}, | |
| 93 | + [MODULES_END_NR] = {0, 0, "Modules Area End"}, | |
| 94 | + [ABS_LOWCORE_NR] = {1, 0, "Lowcore Area Start"}, | |
| 95 | + [ABS_LOWCORE_END_NR] = {0, 0, "Lowcore Area End"}, | |
| 96 | + [MEMCPY_REAL_NR] = {1, 0, "Real Memory Copy Area Start"}, | |
| 97 | + [MEMCPY_REAL_END_NR] = {0, 0, "Real Memory Copy Area End"}, | |
| 94 | 98 | #ifdef CONFIG_KASAN |
| 95 | − [KASAN_SHADOW_START_NR] = {KASAN_SHADOW_START, "Kasan Shadow Start"}, | |
| 96 | − [KASAN_SHADOW_END_NR] = {KASAN_SHADOW_END, "Kasan Shadow End"}, | |
| 99 | + [KASAN_SHADOW_START_NR] = {1, KASAN_SHADOW_START, "Kasan Shadow Start"}, | |
| 100 | + [KASAN_SHADOW_END_NR] = {0, KASAN_SHADOW_END, "Kasan Shadow End"}, | |
| 97 | 101 | #endif |
| 98 | − { -1, NULL } | |
| 102 | + {1, -1UL, NULL} | |
| 99 | 103 | }; |
| 100 | 104 | |
| 101 | 105 | struct pg_state { |
| @@ -163,6 +167,19 @@ static void note_prot_wx(struct pg_state *st, unsigned long addr) · 61 unmodified lines | ||
| 163 | 167 | st->wx_pages += (addr - st->start_address) / PAGE_SIZE; |
| 164 | 168 | } |
| 165 | 169 | |
| 170 | +static void note_page_update_state(struct pg_state *st, unsigned long addr, unsigned int prot, int level) | |
| 171 | +{ | |
| 172 | + struct seq_file *m = st->seq; | |
| 173 | + | |
| 174 | + while (addr >= st->marker[1].start_address) { | |
| 175 | + st->marker++; | |
| 176 | + pt_dump_seq_printf(m, "---[ %s ]---\n", st->marker->name); | |
| 177 | + } | |
| 178 | + st->start_address = addr; | |
| 179 | + st->current_prot = prot; | |
| 180 | + st->level = level; | |
| 181 | +} | |
| 182 | + | |
| 166 | 183 | static void note_page(struct ptdump_state *pt_st, unsigned long addr, int level, u64 val) |
| 167 | 184 | { |
| 168 | 185 | int width = sizeof(unsigned long) * 2; |
| @@ -186,9 +203,7 @@ static void note_page(struct ptdump_state *pt_st, unsigned long addr, int level, · 17 unmodified lines | ||
| 186 | 203 | addr = max_addr; |
| 187 | 204 | if (st->level == -1) { |
| 188 | 205 | pt_dump_seq_printf(m, "---[ %s ]---\n", st->marker->name); |
| 189 | − st->start_address = addr; | |
| 190 | − st->current_prot = prot; | |
| 191 | − st->level = level; | |
| 206 | + note_page_update_state(st, addr, prot, level); | |
| 192 | 207 | } else if (prot != st->current_prot || level != st->level || |
| 193 | 208 | addr >= st->marker[1].start_address) { |
| 194 | 209 | note_prot_wx(st, addr); |
| @@ -202,13 +217,7 @@ static void note_page(struct ptdump_state *pt_st, unsigned long addr, int level, · 7 unmodified lines | ||
| 202 | 217 | } |
| 203 | 218 | pt_dump_seq_printf(m, "%9lu%c ", delta, *unit); |
| 204 | 219 | print_prot(m, st->current_prot, st->level); |
| 205 | − while (addr >= st->marker[1].start_address) { | |
| 206 | − st->marker++; | |
| 207 | − pt_dump_seq_printf(m, "---[ %s ]---\n", st->marker->name); | |
| 208 | − } | |
| 209 | − st->start_address = addr; | |
| 210 | − st->current_prot = prot; | |
| 211 | − st->level = level; | |
| 220 | + note_page_update_state(st, addr, prot, level); | |
| 212 | 221 | } |
| 213 | 222 | } |
| 214 | 223 | |
| @@ -280,22 +289,25 @@ static int ptdump_show(struct seq_file *m, void *v) · 65 unmodified lines | ||
| 280 | 289 | DEFINE_SHOW_ATTRIBUTE(ptdump); |
| 281 | 290 | #endif /* CONFIG_PTDUMP_DEBUGFS */ |
| 282 | 291 | |
| 283 | −/* | |
| 284 | − * Heapsort from lib/sort.c is not a stable sorting algorithm, do a simple | |
| 285 | − * insertion sort to preserve the original order of markers with the same | |
| 286 | − * start address. | |
| 287 | − */ | |
| 288 | −static void sort_address_markers(void) | |
| 292 | +static int ptdump_cmp(const void *a, const void *b) | |
| 289 | 293 | { |
| 290 | − struct addr_marker tmp; | |
| 291 | − int i, j; | |
| 294 | + const struct addr_marker *ama = a; | |
| 295 | + const struct addr_marker *amb = b; | |
| 292 | 296 | |
| 293 | − for (i = 1; i < ARRAY_SIZE(address_markers) - 1; i++) { | |
| 294 | − tmp = address_markers[i]; | |
| 295 | − for (j = i - 1; j >= 0 && address_markers[j].start_address > tmp.start_address; j--) | |
| 296 | − address_markers[j + 1] = address_markers[j]; | |
| 297 | − address_markers[j + 1] = tmp; | |
| 298 | − } | |
| 297 | + if (ama->start_address > amb->start_address) | |
| 298 | + return 1; | |
| 299 | + if (ama->start_address < amb->start_address) | |
| 300 | + return -1; | |
| 301 | + /* | |
| 302 | + * If the start addresses of two markers are identical consider the | |
| 303 | + * marker which defines the start of an area higher than the one which | |
| 304 | + * defines the end of an area. This keeps pairs of markers sorted. | |
| 305 | + */ | |
| 306 | + if (ama->is_start) | |
| 307 | + return 1; | |
| 308 | + if (amb->is_start) | |
| 309 | + return -1; | |
| 310 | + return 0; | |
| 299 | 311 | } |
| 300 | 312 | |
| 301 | 313 | static int pt_dump_init(void) |
| @@ -303,6 +315,8 @@ static int pt_dump_init(void) · 1 unmodified line | ||
| 303 | 315 | #ifdef CONFIG_KFENCE |
| 304 | 316 | unsigned long kfence_start = (unsigned long)__kfence_pool; |
| 305 | 317 | #endif |
| 318 | + unsigned long lowcore = (unsigned long)get_lowcore(); | |
| 319 | + | |
| 306 | 320 | /* |
| 307 | 321 | * Figure out the maximum virtual address being accessible with the |
| 308 | 322 | * kernel ASCE. We need this to keep the page table walker functions |
| @@ -310,7 +324,10 @@ static int pt_dump_init(void) · 1 unmodified line | ||
| 310 | 324 | */ |
| 311 | 325 | max_addr = (get_lowcore()->kernel_asce.val & _REGION_ENTRY_TYPE_MASK) >> 2; |
| 312 | 326 | max_addr = 1UL << (max_addr * 11 + 31); |
| 313 | − address_markers[IDENTITY_AFTER_END_NR].start_address = ident_map_size; | |
| 327 | + address_markers[LOWCORE_START_NR].start_address = lowcore; | |
| 328 | + address_markers[LOWCORE_END_NR].start_address = lowcore + sizeof(struct lowcore); | |
| 329 | + address_markers[IDENTITY_START_NR].start_address = __identity_base; | |
| 330 | + address_markers[IDENTITY_END_NR].start_address = __identity_base + ident_map_size; | |
| 314 | 331 | address_markers[AMODE31_START_NR].start_address = (unsigned long)__samode31; |
| 315 | 332 | address_markers[AMODE31_END_NR].start_address = (unsigned long)__eamode31; |
| 316 | 333 | address_markers[MODULES_NR].start_address = MODULES_VADDR; |
| @@ -337,7 +354,8 @@ static int pt_dump_init(void) · 20 unmodified lines | ||
| 337 | 354 | address_markers[KMSAN_MODULES_ORIGIN_START_NR].start_address = KMSAN_MODULES_ORIGIN_START; |
| 338 | 355 | address_markers[KMSAN_MODULES_ORIGIN_END_NR].start_address = KMSAN_MODULES_ORIGIN_END; |
| 339 | 356 | #endif |
| 340 | − sort_address_markers(); | |
| 357 | + sort(address_markers, ARRAY_SIZE(address_markers) - 1, | |
| 358 | + sizeof(address_markers[0]), ptdump_cmp, NULL); | |
| 341 | 359 | #ifdef CONFIG_PTDUMP_DEBUGFS |
| 342 | 360 | debugfs_create_file("kernel_page_tables", 0400, NULL, NULL, &ptdump_fops); |
| 343 | 361 | #endif /* CONFIG_PTDUMP_DEBUGFS */ |
| @@ -108,6 +108,8 @@ void mark_rodata_ro(void) · 107 unmodified lines | ||
| 108 | 108 | { |
| 109 | 109 | unsigned long size = __end_ro_after_init - __start_ro_after_init; |
| 110 | 110 | |
| 111 | + if (MACHINE_HAS_NX) | |
| 112 | + system_ctl_set_bit(0, CR0_INSTRUCTION_EXEC_PROTECTION_BIT); | |
| 111 | 113 | __set_memory_ro(__start_ro_after_init, __end_ro_after_init); |
| 112 | 114 | pr_info("Write protected read-only-after-init data: %luk\n", size >> 10); |
| 113 | 115 | } |
| @@ -170,13 +172,6 @@ void __init mem_init(void) · 56 unmodified lines | ||
| 170 | 172 | setup_zero_pages(); /* Setup zeroed pages. */ |
| 171 | 173 | } |
| 172 | 174 | |
| 173 | −void free_initmem(void) | |
| 174 | −{ | |
| 175 | − set_memory_rwnx((unsigned long)_sinittext, | |
| 176 | − (unsigned long)(_einittext - _sinittext) >> PAGE_SHIFT); | |
| 177 | − free_initmem_default(POISON_FREE_INITMEM); | |
| 178 | −} | |
| 179 | − | |
| 180 | 175 | unsigned long memory_block_size_bytes(void) |
| 181 | 176 | { |
| 182 | 177 | /* |
| @@ -661,7 +661,6 @@ void __init vmem_map_init(void) · 660 unmodified lines | ||
| 661 | 661 | { |
| 662 | 662 | __set_memory_rox(_stext, _etext); |
| 663 | 663 | __set_memory_ro(_etext, __end_rodata); |
| 664 | − __set_memory_rox(_sinittext, _einittext); | |
| 665 | 664 | __set_memory_rox(__stext_amode31, __etext_amode31); |
| 666 | 665 | /* |
| 667 | 666 | * If the BEAR-enhancement facility is not installed the first |
| @@ -670,16 +669,8 @@ void __init vmem_map_init(void) · 2 unmodified lines | ||
| 670 | 669 | */ |
| 671 | 670 | if (!static_key_enabled(&cpu_has_bear)) |
| 672 | 671 | set_memory_x(0, 1); |
| 673 | − if (debug_pagealloc_enabled()) { | |
| 674 | − /* | |
| 675 | − * Use RELOC_HIDE() as long as __va(0) translates to NULL, | |
| 676 | − * since performing pointer arithmetic on a NULL pointer | |
| 677 | − * has undefined behavior and generates compiler warnings. | |
| 678 | − */ | |
| 679 | − __set_memory_4k(__va(0), RELOC_HIDE(__va(0), ident_map_size)); | |
| 680 | − } | |
| 681 | − if (MACHINE_HAS_NX) | |
| 682 | − system_ctl_set_bit(0, CR0_INSTRUCTION_EXEC_PROTECTION_BIT); | |
| 672 | + if (debug_pagealloc_enabled()) | |
| 673 | + __set_memory_4k(__va(0), __va(0) + ident_map_size); | |
| 683 | 674 | pr_info("Write protected kernel read-only data: %luk\n", |
| 684 | 675 | (unsigned long)(__end_rodata - _stext) >> 10); |
| 685 | 676 | } |
| @@ -280,7 +280,7 @@ static int do_reloc(struct section *sec, Elf_Rel *rel) · 279 unmodified lines | ||
| 280 | 280 | case R_390_GOTOFF64: |
| 281 | 281 | break; |
| 282 | 282 | case R_390_64: |
| 283 | − add_reloc(&relocs64, offset - ehdr.e_entry); | |
| 283 | + add_reloc(&relocs64, offset); | |
| 284 | 284 | break; |
| 285 | 285 | default: |
| 286 | 286 | die("Unsupported relocation type: %d\n", r_type); |
| @@ -71,7 +71,9 @@ static struct mconsole_command *mconsole_parse(struct mc_request *req) · 70 unmodified lines | ||
| 71 | 71 | return NULL; |
| 72 | 72 | } |
| 73 | 73 | |
| 74 | +#ifndef MIN | |
| 74 | 75 | #define MIN(a,b) ((a)<(b) ? (a):(b)) |
| 76 | +#endif | |
| 75 | 77 | |
| 76 | 78 | #define STRINGX(x) #x |
| 77 | 79 | #define STRING(x) STRINGX(x) |
| @@ -163,7 +163,7 @@ struct sev_config { · 162 unmodified lines | ||
| 163 | 163 | */ |
| 164 | 164 | use_cas : 1, |
| 165 | 165 | |
| 166 | − __reserved : 62; | |
| 166 | + __reserved : 61; | |
| 167 | 167 | }; |
| 168 | 168 | |
| 169 | 169 | static struct sev_config sev_cfg __read_mostly; |
| @@ -344,6 +344,7 @@ · 343 unmodified lines | ||
| 344 | 344 | 332 common statx sys_statx |
| 345 | 345 | 333 common io_pgetevents sys_io_pgetevents |
| 346 | 346 | 334 common rseq sys_rseq |
| 347 | +335 common uretprobe sys_uretprobe | |
| 347 | 348 | # don't use numbers 387 through 423, add new calls after the last |
| 348 | 349 | # 'common' entry |
| 349 | 350 | 424 common pidfd_send_signal sys_pidfd_send_signal |
| @@ -385,7 +386,6 @@ · 35 unmodified lines | ||
| 385 | 386 | 460 common lsm_set_self_attr sys_lsm_set_self_attr |
| 386 | 387 | 461 common lsm_list_modules sys_lsm_list_modules |
| 387 | 388 | 462 common mseal sys_mseal |
| 388 | −467 common uretprobe sys_uretprobe | |
| 389 | 389 | |
| 390 | 390 | # |
| 391 | 391 | # Due to a historical design error, certain syscalls are numbered differently |
| @@ -1520,20 +1520,23 @@ static void x86_pmu_start(struct perf_event *event, int flags) · 1519 unmodified lines | ||
| 1520 | 1520 | void perf_event_print_debug(void) |
| 1521 | 1521 | { |
| 1522 | 1522 | u64 ctrl, status, overflow, pmc_ctrl, pmc_count, prev_left, fixed; |
| 1523 | + unsigned long *cntr_mask, *fixed_cntr_mask; | |
| 1524 | + struct event_constraint *pebs_constraints; | |
| 1525 | + struct cpu_hw_events *cpuc; | |
| 1523 | 1526 | u64 pebs, debugctl; |
| 1524 | − int cpu = smp_processor_id(); | |
| 1525 | − struct cpu_hw_events *cpuc = &per_cpu(cpu_hw_events, cpu); | |
| 1526 | − unsigned long *cntr_mask = hybrid(cpuc->pmu, cntr_mask); | |
| 1527 | − unsigned long *fixed_cntr_mask = hybrid(cpuc->pmu, fixed_cntr_mask); | |
| 1528 | − struct event_constraint *pebs_constraints = hybrid(cpuc->pmu, pebs_constraints); | |
| 1529 | − unsigned long flags; | |
| 1530 | − int idx; | |
| 1527 | + int cpu, idx; | |
| 1528 | + | |
| 1529 | + guard(irqsave)(); | |
| 1530 | + | |
| 1531 | + cpu = smp_processor_id(); | |
| 1532 | + cpuc = &per_cpu(cpu_hw_events, cpu); | |
| 1533 | + cntr_mask = hybrid(cpuc->pmu, cntr_mask); | |
| 1534 | + fixed_cntr_mask = hybrid(cpuc->pmu, fixed_cntr_mask); | |
| 1535 | + pebs_constraints = hybrid(cpuc->pmu, pebs_constraints); | |
| 1531 | 1536 | |
| 1532 | 1537 | if (!*(u64 *)cntr_mask) |
| 1533 | 1538 | return; |
| 1534 | 1539 | |
| 1535 | − local_irq_save(flags); | |
| 1536 | − | |
| 1537 | 1540 | if (x86_pmu.version >= 2) { |
| 1538 | 1541 | rdmsrl(MSR_CORE_PERF_GLOBAL_CTRL, ctrl); |
| 1539 | 1542 | rdmsrl(MSR_CORE_PERF_GLOBAL_STATUS, status); |
| @@ -1577,7 +1580,6 @@ void perf_event_print_debug(void) · 37 unmodified lines | ||
| 1577 | 1580 | pr_info("CPU#%d: fixed-PMC%d count: %016llx\n", |
| 1578 | 1581 | cpu, idx, pmc_count); |
| 1579 | 1582 | } |
| 1580 | − local_irq_restore(flags); | |
| 1581 | 1583 | } |
| 1582 | 1584 | |
| 1583 | 1585 | void x86_pmu_stop(struct perf_event *event, int flags) |
| @@ -64,7 +64,7 @@ · 63 unmodified lines | ||
| 64 | 64 | * perf code: 0x00 |
| 65 | 65 | * Available model: SNB,IVB,HSW,BDW,SKL,KNL,GLM,CNL, |
| 66 | 66 | * KBL,CML,ICL,ICX,TGL,TNT,RKL,ADL, |
| 67 | − * RPL,SPR,MTL,ARL,LNL | |
| 67 | + * RPL,SPR,MTL,ARL,LNL,SRF | |
| 68 | 68 | * Scope: Package (physical package) |
| 69 | 69 | * MSR_PKG_C3_RESIDENCY: Package C3 Residency Counter. |
| 70 | 70 | * perf code: 0x01 |
| @@ -693,7 +693,8 @@ static const struct cstate_model srf_cstates __initconst = { · 622 unmodified lines | ||
| 693 | 693 | .core_events = BIT(PERF_CSTATE_CORE_C1_RES) | |
| 694 | 694 | BIT(PERF_CSTATE_CORE_C6_RES), |
| 695 | 695 | |
| 696 | − .pkg_events = BIT(PERF_CSTATE_PKG_C6_RES), | |
| 696 | + .pkg_events = BIT(PERF_CSTATE_PKG_C2_RES) | | |
| 697 | + BIT(PERF_CSTATE_PKG_C6_RES), | |
| 697 | 698 | |
| 698 | 699 | .module_events = BIT(PERF_CSTATE_MODULE_C6_RES), |
| 699 | 700 | }; |
| @@ -2,6 +2,10 @@ · 1 unmodified line | ||
| 2 | 2 | #ifndef _ASM_X86_CMDLINE_H |
| 3 | 3 | #define _ASM_X86_CMDLINE_H |
| 4 | 4 | |
| 5 | +#include <asm/setup.h> | |
| 6 | + | |
| 7 | +extern char builtin_cmdline[COMMAND_LINE_SIZE]; | |
| 8 | + | |
| 5 | 9 | int cmdline_find_option_bool(const char *cmdline_ptr, const char *option); |
| 6 | 10 | int cmdline_find_option(const char *cmdline_ptr, const char *option, |
| 7 | 11 | char *buffer, int bufsize); |
| @@ -1305,6 +1305,7 @@ struct kvm_arch { · 1304 unmodified lines | ||
| 1305 | 1305 | u8 vm_type; |
| 1306 | 1306 | bool has_private_mem; |
| 1307 | 1307 | bool has_protected_state; |
| 1308 | + bool pre_fault_allowed; | |
| 1308 | 1309 | struct hlist_head mmu_page_hash[KVM_NUM_MMU_PAGES]; |
| 1309 | 1310 | struct list_head active_mmu_pages; |
| 1310 | 1311 | struct list_head zapped_obsolete_pages; |
| @@ -2191,6 +2192,8 @@ void kvm_configure_mmu(bool enable_tdp, int tdp_forced_root_level, · 880 unmodified lines | ||
| 2191 | 2192 | #define kvm_arch_has_private_mem(kvm) false |
| 2192 | 2193 | #endif |
| 2193 | 2194 | |
| 2195 | +#define kvm_arch_has_readonly_mem(kvm) (!(kvm)->arch.has_protected_state) | |
| 2196 | + | |
| 2194 | 2197 | static inline u16 kvm_read_ldt(void) |
| 2195 | 2198 | { |
| 2196 | 2199 | u16 ldt; |
| @@ -66,13 +66,15 @@ static inline bool vcpu_is_preempted(long cpu) · 65 unmodified lines | ||
| 66 | 66 | |
| 67 | 67 | #ifdef CONFIG_PARAVIRT |
| 68 | 68 | /* |
| 69 | − * virt_spin_lock_key - enables (by default) the virt_spin_lock() hijack. | |
| 69 | + * virt_spin_lock_key - disables by default the virt_spin_lock() hijack. | |
| 70 | 70 | * |
| 71 | − * Native (and PV wanting native due to vCPU pinning) should disable this key. | |
| 72 | − * It is done in this backwards fashion to only have a single direction change, | |
| 73 | − * which removes ordering between native_pv_spin_init() and HV setup. | |
| 71 | + * Native (and PV wanting native due to vCPU pinning) should keep this key | |
| 72 | + * disabled. Native does not touch the key. | |
| 73 | + * | |
| 74 | + * When in a guest then native_pv_lock_init() enables the key first and | |
| 75 | + * KVM/XEN might conditionally disable it later in the boot process again. | |
| 74 | 76 | */ |
| 75 | −DECLARE_STATIC_KEY_TRUE(virt_spin_lock_key); | |
| 77 | +DECLARE_STATIC_KEY_FALSE(virt_spin_lock_key); | |
| 76 | 78 | |
| 77 | 79 | /* |
| 78 | 80 | * Shortcut for the queued_spin_lock_slowpath() function that allows |
| @@ -19,7 +19,7 @@ · 18 unmodified lines | ||
| 19 | 19 | static u64 acpi_mp_wake_mailbox_paddr __ro_after_init; |
| 20 | 20 | |
| 21 | 21 | /* Virtual address of the Multiprocessor Wakeup Structure mailbox */ |
| 22 | −static struct acpi_madt_multiproc_wakeup_mailbox *acpi_mp_wake_mailbox __ro_after_init; | |
| 22 | +static struct acpi_madt_multiproc_wakeup_mailbox *acpi_mp_wake_mailbox; | |
| 23 | 23 | |
| 24 | 24 | static u64 acpi_mp_pgd __ro_after_init; |
| 25 | 25 | static u64 acpi_mp_reset_vector_paddr __ro_after_init; |
| @@ -462,7 +462,7 @@ static void bsp_init_amd(struct cpuinfo_x86 *c) · 461 unmodified lines | ||
| 462 | 462 | switch (c->x86_model) { |
| 463 | 463 | case 0x00 ... 0x2f: |
| 464 | 464 | case 0x40 ... 0x4f: |
| 465 | − case 0x70 ... 0x7f: | |
| 465 | + case 0x60 ... 0x7f: | |
| 466 | 466 | setup_force_cpu_cap(X86_FEATURE_ZEN5); |
| 467 | 467 | break; |
| 468 | 468 | default: |
| @@ -306,7 +306,7 @@ static void freq_invariance_enable(void) · 305 unmodified lines | ||
| 306 | 306 | WARN_ON_ONCE(1); |
| 307 | 307 | return; |
| 308 | 308 | } |
| 309 | − static_branch_enable(&arch_scale_freq_key); | |
| 309 | + static_branch_enable_cpuslocked(&arch_scale_freq_key); | |
| 310 | 310 | register_freq_invariance_syscore_ops(); |
| 311 | 311 | pr_info("Estimated ratio of average max frequency by base frequency (times 1024): %llu\n", arch_max_freq_ratio); |
| 312 | 312 | } |
| @@ -323,8 +323,10 @@ static void __init bp_init_freq_invariance(void) · 10 unmodified lines | ||
| 323 | 323 | if (boot_cpu_data.x86_vendor != X86_VENDOR_INTEL) |
| 324 | 324 | return; |
| 325 | 325 | |
| 326 | − if (intel_set_max_freq_ratio()) | |
| 326 | + if (intel_set_max_freq_ratio()) { | |
| 327 | + guard(cpus_read_lock)(); | |
| 327 | 328 | freq_invariance_enable(); |
| 329 | + } | |
| 328 | 330 | } |
| 329 | 331 | |
| 330 | 332 | static void disable_freq_invariance_workfn(struct work_struct *work) |
| @@ -609,7 +609,7 @@ void mtrr_save_state(void) · 608 unmodified lines | ||
| 609 | 609 | { |
| 610 | 610 | int first_cpu; |
| 611 | 611 | |
| 612 | − if (!mtrr_enabled()) | |
| 612 | + if (!mtrr_enabled() || !mtrr_state.have_fixed) | |
| 613 | 613 | return; |
| 614 | 614 | |
| 615 | 615 | first_cpu = cpumask_first(cpu_online_mask); |
| @@ -51,13 +51,12 @@ DEFINE_ASM_FUNC(pv_native_irq_enable, "sti", .noinstr.text); · 50 unmodified lines | ||
| 51 | 51 | DEFINE_ASM_FUNC(pv_native_read_cr2, "mov %cr2, %rax", .noinstr.text); |
| 52 | 52 | #endif |
| 53 | 53 | |
| 54 | −DEFINE_STATIC_KEY_TRUE(virt_spin_lock_key); | |
| 54 | +DEFINE_STATIC_KEY_FALSE(virt_spin_lock_key); | |
| 55 | 55 | |
| 56 | 56 | void __init native_pv_lock_init(void) |
| 57 | 57 | { |
| 58 | − if (IS_ENABLED(CONFIG_PARAVIRT_SPINLOCKS) && | |
| 59 | − !boot_cpu_has(X86_FEATURE_HYPERVISOR)) | |
| 60 | − static_branch_disable(&virt_spin_lock_key); | |
| 58 | + if (boot_cpu_has(X86_FEATURE_HYPERVISOR)) | |
| 59 | + static_branch_enable(&virt_spin_lock_key); | |
| 61 | 60 | } |
| 62 | 61 | |
| 63 | 62 | static void native_tlb_remove_table(struct mmu_gather *tlb, void *table) |
| @@ -164,7 +164,7 @@ unsigned long saved_video_mode; · 163 unmodified lines | ||
| 164 | 164 | |
| 165 | 165 | static char __initdata command_line[COMMAND_LINE_SIZE]; |
| 166 | 166 | #ifdef CONFIG_CMDLINE_BOOL |
| 167 | −static char __initdata builtin_cmdline[COMMAND_LINE_SIZE] = CONFIG_CMDLINE; | |
| 167 | +char builtin_cmdline[COMMAND_LINE_SIZE] = CONFIG_CMDLINE; | |
| 168 | 168 | bool builtin_cmdline_added __ro_after_init; |
| 169 | 169 | #endif |
| 170 | 170 | |
| @@ -141,8 +141,8 @@ config KVM_AMD_SEV · 140 unmodified lines | ||
| 141 | 141 | depends on CRYPTO_DEV_SP_PSP && !(KVM_AMD=y && CRYPTO_DEV_CCP_DD=m) |
| 142 | 142 | select ARCH_HAS_CC_PLATFORM |
| 143 | 143 | select KVM_GENERIC_PRIVATE_MEM |
| 144 | − select HAVE_KVM_GMEM_PREPARE | |
| 145 | − select HAVE_KVM_GMEM_INVALIDATE | |
| 144 | + select HAVE_KVM_ARCH_GMEM_PREPARE | |
| 145 | + select HAVE_KVM_ARCH_GMEM_INVALIDATE | |
| 146 | 146 | help |
| 147 | 147 | Provides support for launching Encrypted VMs (SEV) and Encrypted VMs |
| 148 | 148 | with Encrypted State (SEV-ES) on AMD processors. |
| @@ -286,7 +286,6 @@ static inline int kvm_hv_hypercall(struct kvm_vcpu *vcpu) · 285 unmodified lines | ||
| 286 | 286 | return HV_STATUS_ACCESS_DENIED; |
| 287 | 287 | } |
| 288 | 288 | static inline void kvm_hv_vcpu_purge_flush_tlb(struct kvm_vcpu *vcpu) {} |
| 289 | −static inline void kvm_hv_free_pa_page(struct kvm *kvm) {} | |
| 290 | 289 | static inline bool kvm_hv_synic_has_vector(struct kvm_vcpu *vcpu, int vector) |
| 291 | 290 | { |
| 292 | 291 | return false; |
| @@ -351,10 +351,8 @@ static void kvm_recalculate_logical_map(struct kvm_apic_map *new, · 350 unmodified lines | ||
| 351 | 351 | * reversing the LDR calculation to get cluster of APICs, i.e. no |
| 352 | 352 | * additional work is required. |
| 353 | 353 | */ |
| 354 | − if (apic_x2apic_mode(apic)) { | |
| 355 | − WARN_ON_ONCE(ldr != kvm_apic_calc_x2apic_ldr(kvm_x2apic_id(apic))); | |
| 354 | + if (apic_x2apic_mode(apic)) | |
| 356 | 355 | return; |
| 357 | − } | |
| 358 | 356 | |
| 359 | 357 | if (WARN_ON_ONCE(!kvm_apic_map_get_logical_dest(new, ldr, |
| 360 | 358 | &cluster, &mask))) { |
| @@ -1743,7 +1741,7 @@ static void limit_periodic_timer_frequency(struct kvm_lapic *apic) · 1382 unmodified lines | ||
| 1743 | 1741 | s64 min_period = min_timer_period_us * 1000LL; |
| 1744 | 1742 | |
| 1745 | 1743 | if (apic->lapic_timer.period < min_period) { |
| 1746 | − pr_info_ratelimited( | |
| 1744 | + pr_info_once( | |
| 1747 | 1745 | "vcpu %i: requested %lld ns " |
| 1748 | 1746 | "lapic timer period limited to %lld ns\n", |
| 1749 | 1747 | apic->vcpu->vcpu_id, |
| @@ -2966,18 +2964,28 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu, · 1216 unmodified lines | ||
| 2966 | 2964 | struct kvm_lapic_state *s, bool set) |
| 2967 | 2965 | { |
| 2968 | 2966 | if (apic_x2apic_mode(vcpu->arch.apic)) { |
| 2967 | + u32 x2apic_id = kvm_x2apic_id(vcpu->arch.apic); | |
| 2969 | 2968 | u32 *id = (u32 *)(s->regs + APIC_ID); |
| 2970 | 2969 | u32 *ldr = (u32 *)(s->regs + APIC_LDR); |
| 2971 | 2970 | u64 icr; |
| 2972 | 2971 | |
| 2973 | 2972 | if (vcpu->kvm->arch.x2apic_format) { |
| 2974 | − if (*id != vcpu->vcpu_id) | |
| 2973 | + if (*id != x2apic_id) | |
| 2975 | 2974 | return -EINVAL; |
| 2976 | 2975 | } else { |
| 2976 | + /* | |
| 2977 | + * Ignore the userspace value when setting APIC state. | |
| 2978 | + * KVM's model is that the x2APIC ID is readonly, e.g. | |
| 2979 | + * KVM only supports delivering interrupts to KVM's | |
| 2980 | + * version of the x2APIC ID. However, for backwards | |
| 2981 | + * compatibility, don't reject attempts to set a | |
| 2982 | + * mismatched ID for userspace that hasn't opted into | |
| 2983 | + * x2apic_format. | |
| 2984 | + */ | |
| 2977 | 2985 | if (set) |
| 2978 | − *id >>= 24; | |
| 2986 | + *id = x2apic_id; | |
| 2979 | 2987 | else |
| 2980 | − *id <<= 24; | |
| 2988 | + *id = x2apic_id << 24; | |
| 2981 | 2989 | } |
| 2982 | 2990 | |
| 2983 | 2991 | /* |
| @@ -2986,7 +2994,7 @@ static int kvm_apic_state_fixup(struct kvm_vcpu *vcpu, · 2 unmodified lines | ||
| 2986 | 2994 | * split to ICR+ICR2 in userspace for backwards compatibility. |
| 2987 | 2995 | */ |
| 2988 | 2996 | if (set) { |
| 2989 | − *ldr = kvm_apic_calc_x2apic_ldr(*id); | |
| 2997 | + *ldr = kvm_apic_calc_x2apic_ldr(x2apic_id); | |
| 2990 | 2998 | |
| 2991 | 2999 | icr = __kvm_lapic_get_reg(s->regs, APIC_ICR) | |
| 2992 | 3000 | (u64)__kvm_lapic_get_reg(s->regs, APIC_ICR2) << 32; |
| @@ -4335,7 +4335,7 @@ static u8 kvm_max_private_mapping_level(struct kvm *kvm, kvm_pfn_t pfn, · 4334 unmodified lines | ||
| 4335 | 4335 | if (req_max_level) |
| 4336 | 4336 | max_level = min(max_level, req_max_level); |
| 4337 | 4337 | |
| 4338 | − return req_max_level; | |
| 4338 | + return max_level; | |
| 4339 | 4339 | } |
| 4340 | 4340 | |
| 4341 | 4341 | static int kvm_faultin_pfn_private(struct kvm_vcpu *vcpu, |
| @@ -4743,6 +4743,9 @@ long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu, · 401 unmodified lines | ||
| 4743 | 4743 | u64 end; |
| 4744 | 4744 | int r; |
| 4745 | 4745 | |
| 4746 | + if (!vcpu->kvm->arch.pre_fault_allowed) | |
| 4747 | + return -EOPNOTSUPP; | |
| 4748 | + | |
| 4746 | 4749 | /* |
| 4747 | 4750 | * reload is efficient when called repeatedly, so we can do it on |
| 4748 | 4751 | * every iteration. |
| @@ -7510,7 +7513,7 @@ static bool hugepage_has_attrs(struct kvm *kvm, struct kvm_memory_slot *slot, · 2761 unmodified lines | ||
| 7510 | 7513 | const unsigned long end = start + KVM_PAGES_PER_HPAGE(level); |
| 7511 | 7514 | |
| 7512 | 7515 | if (level == PG_LEVEL_2M) |
| 7513 | − return kvm_range_has_memory_attributes(kvm, start, end, attrs); | |
| 7516 | + return kvm_range_has_memory_attributes(kvm, start, end, ~0, attrs); | |
| 7514 | 7517 | |
| 7515 | 7518 | for (gfn = start; gfn < end; gfn += KVM_PAGES_PER_HPAGE(level - 1)) { |
| 7516 | 7519 | if (hugepage_test_mixed(slot, gfn, level - 1) || |
| @@ -2276,30 +2276,24 @@ static int sev_gmem_post_populate(struct kvm *kvm, gfn_t gfn_start, kvm_pfn_t pf · 2275 unmodified lines | ||
| 2276 | 2276 | |
| 2277 | 2277 | for (gfn = gfn_start, i = 0; gfn < gfn_start + npages; gfn++, i++) { |
| 2278 | 2278 | struct sev_data_snp_launch_update fw_args = {0}; |
| 2279 | − bool assigned; | |
| 2279 | + bool assigned = false; | |
| 2280 | 2280 | int level; |
| 2281 | 2281 | |
| 2282 | − if (!kvm_mem_is_private(kvm, gfn)) { | |
| 2283 | − pr_debug("%s: Failed to ensure GFN 0x%llx has private memory attribute set\n", | |
| 2284 | − __func__, gfn); | |
| 2285 | − ret = -EINVAL; | |
| 2286 | − goto err; | |
| 2287 | − } | |
| 2288 | − | |
| 2289 | 2282 | ret = snp_lookup_rmpentry((u64)pfn + i, &assigned, &level); |
| 2290 | 2283 | if (ret || assigned) { |
| 2291 | 2284 | pr_debug("%s: Failed to ensure GFN 0x%llx RMP entry is initial shared state, ret: %d assigned: %d\n", |
| 2292 | 2285 | __func__, gfn, ret, assigned); |
| 2293 | − ret = -EINVAL; | |
| 2286 | + ret = ret ? -EINVAL : -EEXIST; | |
| 2294 | 2287 | goto err; |
| 2295 | 2288 | } |
| 2296 | 2289 | |
| 2297 | 2290 | if (src) { |
| 2298 | 2291 | void *vaddr = kmap_local_pfn(pfn + i); |
| 2299 | 2292 | |
| 2300 | − ret = copy_from_user(vaddr, src + i * PAGE_SIZE, PAGE_SIZE); | |
| 2301 | − if (ret) | |
| 2293 | + if (copy_from_user(vaddr, src + i * PAGE_SIZE, PAGE_SIZE)) { | |
| 2294 | + ret = -EFAULT; | |
| 2302 | 2295 | goto err; |
| 2296 | + } | |
| 2303 | 2297 | kunmap_local(vaddr); |
| 2304 | 2298 | } |
| 2305 | 2299 | |
| @@ -2549,6 +2543,14 @@ static int snp_launch_finish(struct kvm *kvm, struct kvm_sev_cmd *argp) · 243 unmodified lines | ||
| 2549 | 2543 | data->gctx_paddr = __psp_pa(sev->snp_context); |
| 2550 | 2544 | ret = sev_issue_cmd(kvm, SEV_CMD_SNP_LAUNCH_FINISH, data, &argp->error); |
| 2551 | 2545 | |
| 2546 | + /* | |
| 2547 | + * Now that there will be no more SNP_LAUNCH_UPDATE ioctls, private pages | |
| 2548 | + * can be given to the guest simply by marking the RMP entry as private. | |
| 2549 | + * This can happen on first access and also with KVM_PRE_FAULT_MEMORY. | |
| 2550 | + */ | |
| 2551 | + if (!ret) | |
| 2552 | + kvm->arch.pre_fault_allowed = true; | |
| 2553 | + | |
| 2552 | 2554 | kfree(id_auth); |
| 2553 | 2555 | |
| 2554 | 2556 | e_free_id_block: |
| @@ -4949,6 +4949,7 @@ static int svm_vm_init(struct kvm *kvm) · 4948 unmodified lines | ||
| 4949 | 4949 | to_kvm_sev_info(kvm)->need_init = true; |
| 4950 | 4950 | |
| 4951 | 4951 | kvm->arch.has_private_mem = (type == KVM_X86_SNP_VM); |
| 4952 | + kvm->arch.pre_fault_allowed = !kvm->arch.has_private_mem; | |
| 4952 | 4953 | } |
| 4953 | 4954 | |
| 4954 | 4955 | if (!pause_filter_count || !pause_filter_thresh) |
| @@ -427,8 +427,7 @@ static void kvm_user_return_msr_cpu_online(void) · 426 unmodified lines | ||
| 427 | 427 | |
| 428 | 428 | int kvm_set_user_return_msr(unsigned slot, u64 value, u64 mask) |
| 429 | 429 | { |
| 430 | − unsigned int cpu = smp_processor_id(); | |
| 431 | − struct kvm_user_return_msrs *msrs = per_cpu_ptr(user_return_msrs, cpu); | |
| 430 | + struct kvm_user_return_msrs *msrs = this_cpu_ptr(user_return_msrs); | |
| 432 | 431 | int err; |
| 433 | 432 | |
| 434 | 433 | value = (value & mask) | (msrs->values[slot].host & ~mask); |
| @@ -450,8 +449,7 @@ EXPORT_SYMBOL_GPL(kvm_set_user_return_msr); · 15 unmodified lines | ||
| 450 | 449 | |
| 451 | 450 | static void drop_user_return_notifiers(void) |
| 452 | 451 | { |
| 453 | − unsigned int cpu = smp_processor_id(); | |
| 454 | − struct kvm_user_return_msrs *msrs = per_cpu_ptr(user_return_msrs, cpu); | |
| 452 | + struct kvm_user_return_msrs *msrs = this_cpu_ptr(user_return_msrs); | |
| 455 | 453 | |
| 456 | 454 | if (msrs->registered) |
| 457 | 455 | kvm_on_user_return(&msrs->urn); |
| @@ -12646,6 +12644,9 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) · 12188 unmodified lines | ||
| 12646 | 12644 | kvm->arch.vm_type = type; |
| 12647 | 12645 | kvm->arch.has_private_mem = |
| 12648 | 12646 | (type == KVM_X86_SW_PROTECTED_VM); |
| 12647 | + /* Decided by the vendor code for other VM types. */ | |
| 12648 | + kvm->arch.pre_fault_allowed = | |
| 12649 | + type == KVM_X86_DEFAULT_VM || type == KVM_X86_SW_PROTECTED_VM; | |
| 12649 | 12650 | |
| 12650 | 12651 | ret = kvm_page_track_init(kvm); |
| 12651 | 12652 | if (ret) |
| @@ -13641,19 +13642,14 @@ bool kvm_arch_no_poll(struct kvm_vcpu *vcpu) · 989 unmodified lines | ||
| 13641 | 13642 | } |
| 13642 | 13643 | EXPORT_SYMBOL_GPL(kvm_arch_no_poll); |
| 13643 | 13644 | |
| 13644 | −#ifdef CONFIG_HAVE_KVM_GMEM_PREPARE | |
| 13645 | −bool kvm_arch_gmem_prepare_needed(struct kvm *kvm) | |
| 13646 | −{ | |
| 13647 | − return kvm->arch.vm_type == KVM_X86_SNP_VM; | |
| 13648 | −} | |
| 13649 | − | |
| 13645 | +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_PREPARE | |
| 13650 | 13646 | int kvm_arch_gmem_prepare(struct kvm *kvm, gfn_t gfn, kvm_pfn_t pfn, int max_order) |
| 13651 | 13647 | { |
| 13652 | 13648 | return kvm_x86_call(gmem_prepare)(kvm, pfn, gfn, max_order); |
| 13653 | 13649 | } |
| 13654 | 13650 | #endif |
| 13655 | 13651 | |
| 13656 | −#ifdef CONFIG_HAVE_KVM_GMEM_INVALIDATE | |
| 13652 | +#ifdef CONFIG_HAVE_KVM_ARCH_GMEM_INVALIDATE | |
| 13657 | 13653 | void kvm_arch_gmem_invalidate(kvm_pfn_t start, kvm_pfn_t end) |
| 13658 | 13654 | { |
| 13659 | 13655 | kvm_x86_call(gmem_invalidate)(start, end); |
| @@ -207,18 +207,29 @@ __cmdline_find_option(const char *cmdline, int max_cmdline_size, · 206 unmodified lines | ||
| 207 | 207 | |
| 208 | 208 | int cmdline_find_option_bool(const char *cmdline, const char *option) |
| 209 | 209 | { |
| 210 | − if (IS_ENABLED(CONFIG_CMDLINE_BOOL)) | |
| 211 | − WARN_ON_ONCE(!builtin_cmdline_added); | |
| 210 | + int ret; | |
| 212 | 211 | |
| 213 | − return __cmdline_find_option_bool(cmdline, COMMAND_LINE_SIZE, option); | |
| 212 | + ret = __cmdline_find_option_bool(cmdline, COMMAND_LINE_SIZE, option); | |
| 213 | + if (ret > 0) | |
| 214 | + return ret; | |
| 215 | + | |
| 216 | + if (IS_ENABLED(CONFIG_CMDLINE_BOOL) && !builtin_cmdline_added) | |
| 217 | + return __cmdline_find_option_bool(builtin_cmdline, COMMAND_LINE_SIZE, option); | |
| 218 | + | |
| 219 | + return ret; | |
| 214 | 220 | } |
| 215 | 221 | |
| 216 | 222 | int cmdline_find_option(const char *cmdline, const char *option, char *buffer, |
| 217 | 223 | int bufsize) |
| 218 | 224 | { |
| 219 | − if (IS_ENABLED(CONFIG_CMDLINE_BOOL)) | |
| 220 | − WARN_ON_ONCE(!builtin_cmdline_added); | |
| 225 | + int ret; | |
| 226 | + | |
| 227 | + ret = __cmdline_find_option(cmdline, COMMAND_LINE_SIZE, option, buffer, bufsize); | |
| 228 | + if (ret > 0) | |
| 229 | + return ret; | |
| 230 | + | |
| 231 | + if (IS_ENABLED(CONFIG_CMDLINE_BOOL) && !builtin_cmdline_added) | |
| 232 | + return __cmdline_find_option(builtin_cmdline, COMMAND_LINE_SIZE, option, buffer, bufsize); | |
| 221 | 233 | |
| 222 | − return __cmdline_find_option(cmdline, COMMAND_LINE_SIZE, option, | |
| 223 | − buffer, bufsize); | |
| 234 | + return ret; | |
| 224 | 235 | } |
| @@ -88,12 +88,14 @@ SYM_FUNC_END(__get_user_4) · 87 unmodified lines | ||
| 88 | 88 | EXPORT_SYMBOL(__get_user_4) |
| 89 | 89 | |
| 90 | 90 | SYM_FUNC_START(__get_user_8) |
| 91 | +#ifndef CONFIG_X86_64 | |
| 92 | + xor %ecx,%ecx | |
| 93 | +#endif | |
| 91 | 94 | check_range size=8 |
| 92 | 95 | ASM_STAC |
| 93 | 96 | #ifdef CONFIG_X86_64 |
| 94 | 97 | UACCESS movq (%_ASM_AX),%rdx |
| 95 | 98 | #else |
| 96 | − xor %ecx,%ecx | |
| 97 | 99 | UACCESS movl (%_ASM_AX),%edx |
| 98 | 100 | UACCESS movl 4(%_ASM_AX),%ecx |
| 99 | 101 | #endif |
| @@ -241,7 +241,7 @@ static pmd_t *pti_user_pagetable_walk_pmd(unsigned long address) · 240 unmodified lines | ||
| 241 | 241 | * |
| 242 | 242 | * Returns a pointer to a PTE on success, or NULL on failure. |
| 243 | 243 | */ |
| 244 | −static pte_t *pti_user_pagetable_walk_pte(unsigned long address) | |
| 244 | +static pte_t *pti_user_pagetable_walk_pte(unsigned long address, bool late_text) | |
| 245 | 245 | { |
| 246 | 246 | gfp_t gfp = (GFP_KERNEL | __GFP_NOTRACK | __GFP_ZERO); |
| 247 | 247 | pmd_t *pmd; |
| @@ -251,10 +251,15 @@ static pte_t *pti_user_pagetable_walk_pte(unsigned long address) · 3 unmodified lines | ||
| 251 | 251 | if (!pmd) |
| 252 | 252 | return NULL; |
| 253 | 253 | |
| 254 | − /* We can't do anything sensible if we hit a large mapping. */ | |
| 254 | + /* Large PMD mapping found */ | |
| 255 | 255 | if (pmd_leaf(*pmd)) { |
| 256 | − WARN_ON(1); | |
| 257 | − return NULL; | |
| 256 | + /* Clear the PMD if we hit a large mapping from the first round */ | |
| 257 | + if (late_text) { | |
| 258 | + set_pmd(pmd, __pmd(0)); | |
| 259 | + } else { | |
| 260 | + WARN_ON_ONCE(1); | |
| 261 | + return NULL; | |
| 262 | + } | |
| 258 | 263 | } |
| 259 | 264 | |
| 260 | 265 | if (pmd_none(*pmd)) { |
| @@ -283,7 +288,7 @@ static void __init pti_setup_vsyscall(void) · 22 unmodified lines | ||
| 283 | 288 | if (!pte || WARN_ON(level != PG_LEVEL_4K) || pte_none(*pte)) |
| 284 | 289 | return; |
| 285 | 290 | |
| 286 | − target_pte = pti_user_pagetable_walk_pte(VSYSCALL_ADDR); | |
| 291 | + target_pte = pti_user_pagetable_walk_pte(VSYSCALL_ADDR, false); | |
| 287 | 292 | if (WARN_ON(!target_pte)) |
| 288 | 293 | return; |
| 289 | 294 | |
| @@ -301,7 +306,7 @@ enum pti_clone_level { · 11 unmodified lines | ||
| 301 | 306 | |
| 302 | 307 | static void |
| 303 | 308 | pti_clone_pgtable(unsigned long start, unsigned long end, |
| 304 | − enum pti_clone_level level) | |
| 309 | + enum pti_clone_level level, bool late_text) | |
| 305 | 310 | { |
| 306 | 311 | unsigned long addr; |
| 307 | 312 | |
| @@ -374,14 +379,14 @@ pti_clone_pgtable(unsigned long start, unsigned long end, · 66 unmodified lines | ||
| 374 | 379 | */ |
| 375 | 380 | *target_pmd = *pmd; |
| 376 | 381 | |
| 377 | − addr += PMD_SIZE; | |
| 382 | + addr = round_up(addr + 1, PMD_SIZE); | |
| 378 | 383 | |
| 379 | 384 | } else if (level == PTI_CLONE_PTE) { |
| 380 | 385 | |
| 381 | 386 | /* Walk the page-table down to the pte level */ |
| 382 | 387 | pte = pte_offset_kernel(pmd, addr); |
| 383 | 388 | if (pte_none(*pte)) { |
| 384 | − addr += PAGE_SIZE; | |
| 389 | + addr = round_up(addr + 1, PAGE_SIZE); | |
| 385 | 390 | continue; |
| 386 | 391 | } |
| 387 | 392 | |
| @@ -390,7 +395,7 @@ pti_clone_pgtable(unsigned long start, unsigned long end, · 2 unmodified lines | ||
| 390 | 395 | return; |
| 391 | 396 | |
| 392 | 397 | /* Allocate PTE in the user page-table */ |
| 393 | − target_pte = pti_user_pagetable_walk_pte(addr); | |
| 398 | + target_pte = pti_user_pagetable_walk_pte(addr, late_text); | |
| 394 | 399 | if (WARN_ON(!target_pte)) |
| 395 | 400 | return; |
| 396 | 401 | |
| @@ -401,7 +406,7 @@ pti_clone_pgtable(unsigned long start, unsigned long end, · 4 unmodified lines | ||
| 401 | 406 | /* Clone the PTE */ |
| 402 | 407 | *target_pte = *pte; |
| 403 | 408 | |
| 404 | − addr += PAGE_SIZE; | |
| 409 | + addr = round_up(addr + 1, PAGE_SIZE); | |
| 405 | 410 | |
| 406 | 411 | } else { |
| 407 | 412 | BUG(); |
| @@ -452,7 +457,7 @@ static void __init pti_clone_user_shared(void) · 44 unmodified lines | ||
| 452 | 457 | phys_addr_t pa = per_cpu_ptr_to_phys((void *)va); |
| 453 | 458 | pte_t *target_pte; |
| 454 | 459 | |
| 455 | − target_pte = pti_user_pagetable_walk_pte(va); | |
| 460 | + target_pte = pti_user_pagetable_walk_pte(va, false); | |
| 456 | 461 | if (WARN_ON(!target_pte)) |
| 457 | 462 | return; |
| 458 | 463 | |
| @@ -475,7 +480,7 @@ static void __init pti_clone_user_shared(void) · 16 unmodified lines | ||
| 475 | 480 | start = CPU_ENTRY_AREA_BASE; |
| 476 | 481 | end = start + (PAGE_SIZE * CPU_ENTRY_AREA_PAGES); |
| 477 | 482 | |
| 478 | − pti_clone_pgtable(start, end, PTI_CLONE_PMD); | |
| 483 | + pti_clone_pgtable(start, end, PTI_CLONE_PMD, false); | |
| 479 | 484 | } |
| 480 | 485 | #endif /* CONFIG_X86_64 */ |
| 481 | 486 | |
| @@ -492,11 +497,11 @@ static void __init pti_setup_espfix64(void) · 10 unmodified lines | ||
| 492 | 497 | /* |
| 493 | 498 | * Clone the populated PMDs of the entry text and force it RO. |
| 494 | 499 | */ |
| 495 | −static void pti_clone_entry_text(void) | |
| 500 | +static void pti_clone_entry_text(bool late) | |
| 496 | 501 | { |
| 497 | 502 | pti_clone_pgtable((unsigned long) __entry_text_start, |
| 498 | 503 | (unsigned long) __entry_text_end, |
| 499 | − PTI_CLONE_PMD); | |
| 504 | + PTI_LEVEL_KERNEL_IMAGE, late); | |
| 500 | 505 | } |
| 501 | 506 | |
| 502 | 507 | /* |
| @@ -571,7 +576,7 @@ static void pti_clone_kernel_text(void) · 68 unmodified lines | ||
| 571 | 576 | * pti_set_kernel_image_nonglobal() did to clear the |
| 572 | 577 | * global bit. |
| 573 | 578 | */ |
| 574 | − pti_clone_pgtable(start, end_clone, PTI_LEVEL_KERNEL_IMAGE); | |
| 579 | + pti_clone_pgtable(start, end_clone, PTI_LEVEL_KERNEL_IMAGE, false); | |
| 575 | 580 | |
| 576 | 581 | /* |
| 577 | 582 | * pti_clone_pgtable() will set the global bit in any PMDs |
| @@ -638,8 +643,15 @@ void __init pti_init(void) · 60 unmodified lines | ||
| 638 | 643 | |
| 639 | 644 | /* Undo all global bits from the init pagetables in head_64.S: */ |
| 640 | 645 | pti_set_kernel_image_nonglobal(); |
| 646 | + | |
| 641 | 647 | /* Replace some of the global bits just for shared entry text: */ |
| 642 | − pti_clone_entry_text(); | |
| 648 | + /* | |
| 649 | + * This is very early in boot. Device and Late initcalls can do | |
| 650 | + * modprobe before free_initmem() and mark_readonly(). This | |
| 651 | + * pti_clone_entry_text() allows those user-mode-helpers to function, | |
| 652 | + * but notably the text is still RW. | |
| 653 | + */ | |
| 654 | + pti_clone_entry_text(false); | |
| 643 | 655 | pti_setup_espfix64(); |
| 644 | 656 | pti_setup_vsyscall(); |
| 645 | 657 | } |
| @@ -656,10 +668,11 @@ void pti_finalize(void) · 10 unmodified lines | ||
| 656 | 668 | if (!boot_cpu_has(X86_FEATURE_PTI)) |
| 657 | 669 | return; |
| 658 | 670 | /* |
| 659 | − * We need to clone everything (again) that maps parts of the | |
| 660 | − * kernel image. | |
| 671 | + * This is after free_initmem() (all initcalls are done) and we've done | |
| 672 | + * mark_readonly(). Text is now NX which might've split some PMDs | |
| 673 | + * relative to the early clone. | |
| 661 | 674 | */ |
| 662 | − pti_clone_entry_text(); | |
| 675 | + pti_clone_entry_text(true); | |
| 663 | 676 | pti_clone_kernel_text(); |
| 664 | 677 | |
| 665 | 678 | debug_checkwx_user(); |
| @@ -111,13 +111,20 @@ static sector_t bio_write_zeroes_limit(struct block_device *bdev) · 110 unmodified lines | ||
| 111 | 111 | (UINT_MAX >> SECTOR_SHIFT) & ~bs_mask); |
| 112 | 112 | } |
| 113 | 113 | |
| 114 | +/* | |
| 115 | + * There is no reliable way for the SCSI subsystem to determine whether a | |
| 116 | + * device supports a WRITE SAME operation without actually performing a write | |
| 117 | + * to media. As a result, write_zeroes is enabled by default and will be | |
| 118 | + * disabled if a zeroing operation subsequently fails. This means that this | |
| 119 | + * queue limit is likely to change at runtime. | |
| 120 | + */ | |
| 114 | 121 | static void __blkdev_issue_write_zeroes(struct block_device *bdev, |
| 115 | 122 | sector_t sector, sector_t nr_sects, gfp_t gfp_mask, |
| 116 | − struct bio **biop, unsigned flags) | |
| 123 | + struct bio **biop, unsigned flags, sector_t limit) | |
| 117 | 124 | { |
| 125 | + | |
| 118 | 126 | while (nr_sects) { |
| 119 | − unsigned int len = min_t(sector_t, nr_sects, | |
| 120 | − bio_write_zeroes_limit(bdev)); | |
| 127 | + unsigned int len = min(nr_sects, limit); | |
| 121 | 128 | struct bio *bio; |
| 122 | 129 | |
| 123 | 130 | if ((flags & BLKDEV_ZERO_KILLABLE) && |
| @@ -141,12 +148,14 @@ static void __blkdev_issue_write_zeroes(struct block_device *bdev, · 17 unmodified lines | ||
| 141 | 148 | static int blkdev_issue_write_zeroes(struct block_device *bdev, sector_t sector, |
| 142 | 149 | sector_t nr_sects, gfp_t gfp, unsigned flags) |
| 143 | 150 | { |
| 151 | + sector_t limit = bio_write_zeroes_limit(bdev); | |
| 144 | 152 | struct bio *bio = NULL; |
| 145 | 153 | struct blk_plug plug; |
| 146 | 154 | int ret = 0; |
| 147 | 155 | |
| 148 | 156 | blk_start_plug(&plug); |
| 149 | − __blkdev_issue_write_zeroes(bdev, sector, nr_sects, gfp, &bio, flags); | |
| 157 | + __blkdev_issue_write_zeroes(bdev, sector, nr_sects, gfp, &bio, | |
| 158 | + flags, limit); | |
| 150 | 159 | if (bio) { |
| 151 | 160 | if ((flags & BLKDEV_ZERO_KILLABLE) && |
| 152 | 161 | fatal_signal_pending(current)) { |
| @@ -265,12 +274,14 @@ int __blkdev_issue_zeroout(struct block_device *bdev, sector_t sector, · 112 unmodified lines | ||
| 265 | 274 | sector_t nr_sects, gfp_t gfp_mask, struct bio **biop, |
| 266 | 275 | unsigned flags) |
| 267 | 276 | { |
| 277 | + sector_t limit = bio_write_zeroes_limit(bdev); | |
| 278 | + | |
| 268 | 279 | if (bdev_read_only(bdev)) |
| 269 | 280 | return -EPERM; |
| 270 | 281 | |
| 271 | − if (bdev_write_zeroes_sectors(bdev)) { | |
| 282 | + if (limit) { | |
| 272 | 283 | __blkdev_issue_write_zeroes(bdev, sector, nr_sects, |
| 273 | − gfp_mask, biop, flags); | |
| 284 | + gfp_mask, biop, flags, limit); | |
| 274 | 285 | } else { |
| 275 | 286 | if (flags & BLKDEV_ZERO_NOFALLBACK) |
| 276 | 287 | return -EOPNOTSUPP; |
| @@ -38,6 +38,7 @@ static void blk_mq_update_wake_batch(struct blk_mq_tags *tags, · 37 unmodified lines | ||
| 38 | 38 | void __blk_mq_tag_busy(struct blk_mq_hw_ctx *hctx) |
| 39 | 39 | { |
| 40 | 40 | unsigned int users; |
| 41 | + unsigned long flags; | |
| 41 | 42 | struct blk_mq_tags *tags = hctx->tags; |
| 42 | 43 | |
| 43 | 44 | /* |
| @@ -56,11 +57,11 @@ void __blk_mq_tag_busy(struct blk_mq_hw_ctx *hctx) · 12 unmodified lines | ||
| 56 | 57 | return; |
| 57 | 58 | } |
| 58 | 59 | |
| 59 | − spin_lock_irq(&tags->lock); | |
| 60 | + spin_lock_irqsave(&tags->lock, flags); | |
| 60 | 61 | users = tags->active_queues + 1; |
| 61 | 62 | WRITE_ONCE(tags->active_queues, users); |
| 62 | 63 | blk_mq_update_wake_batch(tags, users); |
| 63 | − spin_unlock_irq(&tags->lock); | |
| 64 | + spin_unlock_irqrestore(&tags->lock, flags); | |
| 64 | 65 | } |
| 65 | 66 | |
| 66 | 67 | /* |
| @@ -31,14 +31,6 @@ static struct workqueue_struct *kthrotld_workqueue; · 30 unmodified lines | ||
| 31 | 31 | |
| 32 | 32 | #define rb_entry_tg(node) rb_entry((node), struct throtl_grp, rb_node) |
| 33 | 33 | |
| 34 | −/* We measure latency for request size from <= 4k to >= 1M */ | |
| 35 | −#define LATENCY_BUCKET_SIZE 9 | |
| 36 | − | |
| 37 | −struct latency_bucket { | |
| 38 | − unsigned long total_latency; /* ns / 1024 */ | |
| 39 | − int samples; | |
| 40 | −}; | |
| 41 | − | |
| 42 | 34 | struct throtl_data |
| 43 | 35 | { |
| 44 | 36 | /* service tree for active throtl groups */ |
| @@ -116,9 +108,6 @@ static unsigned int tg_iops_limit(struct throtl_grp *tg, int rw) · 71 unmodified lines | ||
| 116 | 108 | return tg->iops[rw]; |
| 117 | 109 | } |
| 118 | 110 | |
| 119 | −#define request_bucket_index(sectors) \ | |
| 120 | − clamp_t(int, order_base_2(sectors) - 3, 0, LATENCY_BUCKET_SIZE - 1) | |
| 121 | − | |
| 122 | 111 | /** |
| 123 | 112 | * throtl_log - log debug message via blktrace |
| 124 | 113 | * @sq: the service_queue being reported |
| @@ -10,7 +10,6 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <stdio.h> |
| 11 | 11 | #include <libgen.h> |
| 12 | 12 | #include <string.h> |
| 13 | −#include <linux/version.h> | |
| 14 | 13 | #include <ctype.h> |
| 15 | 14 | #include "utils.h" |
| 16 | 15 | |
| @@ -10,7 +10,6 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <stdio.h> |
| 11 | 11 | #include <libgen.h> |
| 12 | 12 | #include <string.h> |
| 13 | −#include <linux/version.h> | |
| 14 | 13 | #include <ctype.h> |
| 15 | 14 | #include "utils.h" |
| 16 | 15 | |
| @@ -188,13 +188,9 @@ acpi_ev_detach_region(union acpi_operand_object *region_obj, · 187 unmodified lines | ||
| 188 | 188 | u8 acpi_ns_is_locked); |
| 189 | 189 | |
| 190 | 190 | void |
| 191 | −acpi_ev_execute_reg_methods(struct acpi_namespace_node *node, | |
| 191 | +acpi_ev_execute_reg_methods(struct acpi_namespace_node *node, u32 max_depth, | |
| 192 | 192 | acpi_adr_space_type space_id, u32 function); |
| 193 | 193 | |
| 194 | −void | |
| 195 | −acpi_ev_execute_orphan_reg_method(struct acpi_namespace_node *node, | |
| 196 | − acpi_adr_space_type space_id); | |
| 197 | − | |
| 198 | 194 | acpi_status |
| 199 | 195 | acpi_ev_execute_reg_method(union acpi_operand_object *region_obj, u32 function); |
| 200 | 196 | |
| @@ -20,6 +20,10 @@ extern u8 acpi_gbl_default_address_spaces[]; · 19 unmodified lines | ||
| 20 | 20 | |
| 21 | 21 | /* Local prototypes */ |
| 22 | 22 | |
| 23 | +static void | |
| 24 | +acpi_ev_execute_orphan_reg_method(struct acpi_namespace_node *device_node, | |
| 25 | + acpi_adr_space_type space_id); | |
| 26 | + | |
| 23 | 27 | static acpi_status |
| 24 | 28 | acpi_ev_reg_run(acpi_handle obj_handle, |
| 25 | 29 | u32 level, void *context, void **return_value); |
| @@ -61,6 +65,7 @@ acpi_status acpi_ev_initialize_op_regions(void) · 35 unmodified lines | ||
| 61 | 65 | acpi_gbl_default_address_spaces |
| 62 | 66 | [i])) { |
| 63 | 67 | acpi_ev_execute_reg_methods(acpi_gbl_root_node, |
| 68 | + ACPI_UINT32_MAX, | |
| 64 | 69 | acpi_gbl_default_address_spaces |
| 65 | 70 | [i], ACPI_REG_CONNECT); |
| 66 | 71 | } |
| @@ -668,6 +673,7 @@ cleanup1: · 601 unmodified lines | ||
| 668 | 673 | * FUNCTION: acpi_ev_execute_reg_methods |
| 669 | 674 | * |
| 670 | 675 | * PARAMETERS: node - Namespace node for the device |
| 676 | + * max_depth - Depth to which search for _REG | |
| 671 | 677 | * space_id - The address space ID |
| 672 | 678 | * function - Passed to _REG: On (1) or Off (0) |
| 673 | 679 | * |
| @@ -679,7 +685,7 @@ cleanup1: · 5 unmodified lines | ||
| 679 | 685 | ******************************************************************************/ |
| 680 | 686 | |
| 681 | 687 | void |
| 682 | −acpi_ev_execute_reg_methods(struct acpi_namespace_node *node, | |
| 688 | +acpi_ev_execute_reg_methods(struct acpi_namespace_node *node, u32 max_depth, | |
| 683 | 689 | acpi_adr_space_type space_id, u32 function) |
| 684 | 690 | { |
| 685 | 691 | struct acpi_reg_walk_info info; |
| @@ -713,7 +719,7 @@ acpi_ev_execute_reg_methods(struct acpi_namespace_node *node, · 27 unmodified lines | ||
| 713 | 719 | * regions and _REG methods. (i.e. handlers must be installed for all |
| 714 | 720 | * regions of this Space ID before we can run any _REG methods) |
| 715 | 721 | */ |
| 716 | − (void)acpi_ns_walk_namespace(ACPI_TYPE_ANY, node, ACPI_UINT32_MAX, | |
| 722 | + (void)acpi_ns_walk_namespace(ACPI_TYPE_ANY, node, max_depth, | |
| 717 | 723 | ACPI_NS_WALK_UNLOCK, acpi_ev_reg_run, NULL, |
| 718 | 724 | &info, NULL); |
| 719 | 725 | |
| @@ -814,7 +820,7 @@ acpi_ev_reg_run(acpi_handle obj_handle, · 94 unmodified lines | ||
| 814 | 820 | * |
| 815 | 821 | ******************************************************************************/ |
| 816 | 822 | |
| 817 | −void | |
| 823 | +static void | |
| 818 | 824 | acpi_ev_execute_orphan_reg_method(struct acpi_namespace_node *device_node, |
| 819 | 825 | acpi_adr_space_type space_id) |
| 820 | 826 | { |
| @@ -85,7 +85,8 @@ acpi_install_address_space_handler_internal(acpi_handle device, · 84 unmodified lines | ||
| 85 | 85 | /* Run all _REG methods for this address space */ |
| 86 | 86 | |
| 87 | 87 | if (run_reg) { |
| 88 | − acpi_ev_execute_reg_methods(node, space_id, ACPI_REG_CONNECT); | |
| 88 | + acpi_ev_execute_reg_methods(node, ACPI_UINT32_MAX, space_id, | |
| 89 | + ACPI_REG_CONNECT); | |
| 89 | 90 | } |
| 90 | 91 | |
| 91 | 92 | unlock_and_exit: |
| @@ -263,6 +264,7 @@ ACPI_EXPORT_SYMBOL(acpi_remove_address_space_handler) · 171 unmodified lines | ||
| 263 | 264 | * FUNCTION: acpi_execute_reg_methods |
| 264 | 265 | * |
| 265 | 266 | * PARAMETERS: device - Handle for the device |
| 267 | + * max_depth - Depth to which search for _REG | |
| 266 | 268 | * space_id - The address space ID |
| 267 | 269 | * |
| 268 | 270 | * RETURN: Status |
| @@ -271,7 +273,8 @@ ACPI_EXPORT_SYMBOL(acpi_remove_address_space_handler) · 2 unmodified lines | ||
| 271 | 273 | * |
| 272 | 274 | ******************************************************************************/ |
| 273 | 275 | acpi_status |
| 274 | −acpi_execute_reg_methods(acpi_handle device, acpi_adr_space_type space_id) | |
| 276 | +acpi_execute_reg_methods(acpi_handle device, u32 max_depth, | |
| 277 | + acpi_adr_space_type space_id) | |
| 275 | 278 | { |
| 276 | 279 | struct acpi_namespace_node *node; |
| 277 | 280 | acpi_status status; |
| @@ -296,7 +299,8 @@ acpi_execute_reg_methods(acpi_handle device, acpi_adr_space_type space_id) · 18 unmodified lines | ||
| 296 | 299 | |
| 297 | 300 | /* Run all _REG methods for this address space */ |
| 298 | 301 | |
| 299 | − acpi_ev_execute_reg_methods(node, space_id, ACPI_REG_CONNECT); | |
| 302 | + acpi_ev_execute_reg_methods(node, max_depth, space_id, | |
| 303 | + ACPI_REG_CONNECT); | |
| 300 | 304 | } else { |
| 301 | 305 | status = AE_BAD_PARAMETER; |
| 302 | 306 | } |
| @@ -306,57 +310,3 @@ acpi_execute_reg_methods(acpi_handle device, acpi_adr_space_type space_id) · 3 unmodified lines | ||
| 306 | 310 | } |
| 307 | 311 | |
| 308 | 312 | ACPI_EXPORT_SYMBOL(acpi_execute_reg_methods) |
| 309 | − | |
| 310 | −/******************************************************************************* | |
| 311 | − * | |
| 312 | − * FUNCTION: acpi_execute_orphan_reg_method | |
| 313 | − * | |
| 314 | − * PARAMETERS: device - Handle for the device | |
| 315 | − * space_id - The address space ID | |
| 316 | − * | |
| 317 | − * RETURN: Status | |
| 318 | − * | |
| 319 | − * DESCRIPTION: Execute an "orphan" _REG method that appears under an ACPI | |
| 320 | − * device. This is a _REG method that has no corresponding region | |
| 321 | − * within the device's scope. | |
| 322 | − * | |
| 323 | − ******************************************************************************/ | |
| 324 | −acpi_status | |
| 325 | −acpi_execute_orphan_reg_method(acpi_handle device, acpi_adr_space_type space_id) | |
| 326 | −{ | |
| 327 | − struct acpi_namespace_node *node; | |
| 328 | − acpi_status status; | |
| 329 | − | |
| 330 | − ACPI_FUNCTION_TRACE(acpi_execute_orphan_reg_method); | |
| 331 | − | |
| 332 | − /* Parameter validation */ | |
| 333 | − | |
| 334 | − if (!device) { | |
| 335 | − return_ACPI_STATUS(AE_BAD_PARAMETER); | |
| 336 | − } | |
| 337 | − | |
| 338 | − status = acpi_ut_acquire_mutex(ACPI_MTX_NAMESPACE); | |
| 339 | − if (ACPI_FAILURE(status)) { | |
| 340 | − return_ACPI_STATUS(status); | |
| 341 | − } | |
| 342 | − | |
| 343 | − /* Convert and validate the device handle */ | |
| 344 | − | |
| 345 | − node = acpi_ns_validate_handle(device); | |
| 346 | − if (node) { | |
| 347 | − | |
| 348 | − /* | |
| 349 | − * If an "orphan" _REG method is present in the device's scope | |
| 350 | − * for the given address space ID, run it. | |
| 351 | − */ | |
| 352 | − | |
| 353 | − acpi_ev_execute_orphan_reg_method(node, space_id); | |
| 354 | − } else { | |
| 355 | − status = AE_BAD_PARAMETER; | |
| 356 | − } | |
| 357 | − | |
| 358 | − (void)acpi_ut_release_mutex(ACPI_MTX_NAMESPACE); | |
| 359 | − return_ACPI_STATUS(status); | |
| 360 | −} | |
| 361 | − | |
| 362 | −ACPI_EXPORT_SYMBOL(acpi_execute_orphan_reg_method) | |
| @@ -1487,12 +1487,13 @@ static bool install_gpio_irq_event_handler(struct acpi_ec *ec) · 1486 unmodified lines | ||
| 1487 | 1487 | static int ec_install_handlers(struct acpi_ec *ec, struct acpi_device *device, |
| 1488 | 1488 | bool call_reg) |
| 1489 | 1489 | { |
| 1490 | − acpi_handle scope_handle = ec == first_ec ? ACPI_ROOT_OBJECT : ec->handle; | |
| 1491 | 1490 | acpi_status status; |
| 1492 | 1491 | |
| 1493 | 1492 | acpi_ec_start(ec, false); |
| 1494 | 1493 | |
| 1495 | 1494 | if (!test_bit(EC_FLAGS_EC_HANDLER_INSTALLED, &ec->flags)) { |
| 1495 | + acpi_handle scope_handle = ec == first_ec ? ACPI_ROOT_OBJECT : ec->handle; | |
| 1496 | + | |
| 1496 | 1497 | acpi_ec_enter_noirq(ec); |
| 1497 | 1498 | status = acpi_install_address_space_handler_no_reg(scope_handle, |
| 1498 | 1499 | ACPI_ADR_SPACE_EC, |
| @@ -1506,10 +1507,7 @@ static int ec_install_handlers(struct acpi_ec *ec, struct acpi_device *device, · 7 unmodified lines | ||
| 1506 | 1507 | } |
| 1507 | 1508 | |
| 1508 | 1509 | if (call_reg && !test_bit(EC_FLAGS_EC_REG_CALLED, &ec->flags)) { |
| 1509 | − acpi_execute_reg_methods(scope_handle, ACPI_ADR_SPACE_EC); | |
| 1510 | − if (scope_handle != ec->handle) | |
| 1511 | − acpi_execute_orphan_reg_method(ec->handle, ACPI_ADR_SPACE_EC); | |
| 1512 | − | |
| 1510 | + acpi_execute_reg_methods(ec->handle, ACPI_UINT32_MAX, ACPI_ADR_SPACE_EC); | |
| 1513 | 1511 | set_bit(EC_FLAGS_EC_REG_CALLED, &ec->flags); |
| 1514 | 1512 | } |
| 1515 | 1513 | |
| @@ -1724,6 +1722,12 @@ static void acpi_ec_remove(struct acpi_device *device) · 208 unmodified lines | ||
| 1724 | 1722 | } |
| 1725 | 1723 | } |
| 1726 | 1724 | |
| 1725 | +void acpi_ec_register_opregions(struct acpi_device *adev) | |
| 1726 | +{ | |
| 1727 | + if (first_ec && first_ec->handle != adev->handle) | |
| 1728 | + acpi_execute_reg_methods(adev->handle, 1, ACPI_ADR_SPACE_EC); | |
| 1729 | +} | |
| 1730 | + | |
| 1727 | 1731 | static acpi_status |
| 1728 | 1732 | ec_parse_io_ports(struct acpi_resource *resource, void *context) |
| 1729 | 1733 | { |
| @@ -223,6 +223,7 @@ int acpi_ec_add_query_handler(struct acpi_ec *ec, u8 query_bit, · 222 unmodified lines | ||
| 223 | 223 | acpi_handle handle, acpi_ec_query_func func, |
| 224 | 224 | void *data); |
| 225 | 225 | void acpi_ec_remove_query_handler(struct acpi_ec *ec, u8 query_bit); |
| 226 | +void acpi_ec_register_opregions(struct acpi_device *adev); | |
| 226 | 227 | |
| 227 | 228 | #ifdef CONFIG_PM_SLEEP |
| 228 | 229 | void acpi_ec_flush_work(void); |
| @@ -2273,6 +2273,8 @@ static int acpi_bus_attach(struct acpi_device *device, void *first_pass) · 2272 unmodified lines | ||
| 2273 | 2273 | if (device->handler) |
| 2274 | 2274 | goto ok; |
| 2275 | 2275 | |
| 2276 | + acpi_ec_register_opregions(device); | |
| 2277 | + | |
| 2276 | 2278 | if (!device->flags.initialized) { |
| 2277 | 2279 | device->flags.power_manageable = |
| 2278 | 2280 | device->power.states[ACPI_STATE_D0].flags.valid; |
| @@ -54,6 +54,8 @@ static void acpi_video_parse_cmdline(void) · 53 unmodified lines | ||
| 54 | 54 | acpi_backlight_cmdline = acpi_backlight_nvidia_wmi_ec; |
| 55 | 55 | if (!strcmp("apple_gmux", acpi_video_backlight_string)) |
| 56 | 56 | acpi_backlight_cmdline = acpi_backlight_apple_gmux; |
| 57 | + if (!strcmp("dell_uart", acpi_video_backlight_string)) | |
| 58 | + acpi_backlight_cmdline = acpi_backlight_dell_uart; | |
| 57 | 59 | if (!strcmp("none", acpi_video_backlight_string)) |
| 58 | 60 | acpi_backlight_cmdline = acpi_backlight_none; |
| 59 | 61 | } |
| @@ -821,6 +823,21 @@ static const struct dmi_system_id video_detect_dmi_table[] = { · 761 unmodified lines | ||
| 821 | 823 | }, |
| 822 | 824 | }, |
| 823 | 825 | |
| 826 | + /* | |
| 827 | + * Dell AIO (All in Ones) which advertise an UART attached backlight | |
| 828 | + * controller board in their ACPI tables (and may even have one), but | |
| 829 | + * which need native backlight control nevertheless. | |
| 830 | + */ | |
| 831 | + { | |
| 832 | + /* https://bugzilla.redhat.com/show_bug.cgi?id=2303936 */ | |
| 833 | + .callback = video_detect_force_native, | |
| 834 | + /* Dell OptiPlex 7760 AIO */ | |
| 835 | + .matches = { | |
| 836 | + DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), | |
| 837 | + DMI_MATCH(DMI_PRODUCT_NAME, "OptiPlex 7760 AIO"), | |
| 838 | + }, | |
| 839 | + }, | |
| 840 | + | |
| 824 | 841 | /* |
| 825 | 842 | * Models which have nvidia-ec-wmi support, but should not use it. |
| 826 | 843 | * Note this indicates a likely firmware bug on these models and should |
| @@ -918,6 +935,7 @@ enum acpi_backlight_type __acpi_video_get_backlight_type(bool native, bool *auto · 91 unmodified lines | ||
| 918 | 935 | static DEFINE_MUTEX(init_mutex); |
| 919 | 936 | static bool nvidia_wmi_ec_present; |
| 920 | 937 | static bool apple_gmux_present; |
| 938 | + static bool dell_uart_present; | |
| 921 | 939 | static bool native_available; |
| 922 | 940 | static bool init_done; |
| 923 | 941 | static long video_caps; |
| @@ -932,6 +950,7 @@ enum acpi_backlight_type __acpi_video_get_backlight_type(bool native, bool *auto · 8 unmodified lines | ||
| 932 | 950 | &video_caps, NULL); |
| 933 | 951 | nvidia_wmi_ec_present = nvidia_wmi_ec_supported(); |
| 934 | 952 | apple_gmux_present = apple_gmux_detect(NULL, NULL); |
| 953 | + dell_uart_present = acpi_dev_present("DELL0501", NULL, -1); | |
| 935 | 954 | init_done = true; |
| 936 | 955 | } |
| 937 | 956 | if (native) |
| @@ -962,6 +981,9 @@ enum acpi_backlight_type __acpi_video_get_backlight_type(bool native, bool *auto · 24 unmodified lines | ||
| 962 | 981 | if (apple_gmux_present) |
| 963 | 982 | return acpi_backlight_apple_gmux; |
| 964 | 983 | |
| 984 | + if (dell_uart_present) | |
| 985 | + return acpi_backlight_dell_uart; | |
| 986 | + | |
| 965 | 987 | /* Use ACPI video if available, except when native should be preferred. */ |
| 966 | 988 | if ((video_caps & ACPI_VIDEO_BACKLIGHT) && |
| 967 | 989 | !(native_available && prefer_native_over_acpi_video())) |
| @@ -1044,13 +1044,13 @@ static struct binder_ref *binder_get_ref_olocked(struct binder_proc *proc, · 1043 unmodified lines | ||
| 1044 | 1044 | } |
| 1045 | 1045 | |
| 1046 | 1046 | /* Find the smallest unused descriptor the "slow way" */ |
| 1047 | −static u32 slow_desc_lookup_olocked(struct binder_proc *proc) | |
| 1047 | +static u32 slow_desc_lookup_olocked(struct binder_proc *proc, u32 offset) | |
| 1048 | 1048 | { |
| 1049 | 1049 | struct binder_ref *ref; |
| 1050 | 1050 | struct rb_node *n; |
| 1051 | 1051 | u32 desc; |
| 1052 | 1052 | |
| 1053 | − desc = 1; | |
| 1053 | + desc = offset; | |
| 1054 | 1054 | for (n = rb_first(&proc->refs_by_desc); n; n = rb_next(n)) { |
| 1055 | 1055 | ref = rb_entry(n, struct binder_ref, rb_node_desc); |
| 1056 | 1056 | if (ref->data.desc > desc) |
| @@ -1071,21 +1071,18 @@ static int get_ref_desc_olocked(struct binder_proc *proc, · 14 unmodified lines | ||
| 1071 | 1071 | u32 *desc) |
| 1072 | 1072 | { |
| 1073 | 1073 | struct dbitmap *dmap = &proc->dmap; |
| 1074 | + unsigned int nbits, offset; | |
| 1074 | 1075 | unsigned long *new, bit; |
| 1075 | − unsigned int nbits; | |
| 1076 | 1076 | |
| 1077 | 1077 | /* 0 is reserved for the context manager */ |
| 1078 | − if (node == proc->context->binder_context_mgr_node) { | |
| 1079 | − *desc = 0; | |
| 1080 | − return 0; | |
| 1081 | − } | |
| 1078 | + offset = (node == proc->context->binder_context_mgr_node) ? 0 : 1; | |
| 1082 | 1079 | |
| 1083 | 1080 | if (!dbitmap_enabled(dmap)) { |
| 1084 | − *desc = slow_desc_lookup_olocked(proc); | |
| 1081 | + *desc = slow_desc_lookup_olocked(proc, offset); | |
| 1085 | 1082 | return 0; |
| 1086 | 1083 | } |
| 1087 | 1084 | |
| 1088 | − if (dbitmap_acquire_first_zero_bit(dmap, &bit) == 0) { | |
| 1085 | + if (dbitmap_acquire_next_zero_bit(dmap, offset, &bit) == 0) { | |
| 1089 | 1086 | *desc = bit; |
| 1090 | 1087 | return 0; |
| 1091 | 1088 | } |
| @@ -939,9 +939,9 @@ void binder_alloc_deferred_release(struct binder_alloc *alloc) · 938 unmodified lines | ||
| 939 | 939 | __free_page(alloc->pages[i].page_ptr); |
| 940 | 940 | page_count++; |
| 941 | 941 | } |
| 942 | − kvfree(alloc->pages); | |
| 943 | 942 | } |
| 944 | 943 | spin_unlock(&alloc->lock); |
| 944 | + kvfree(alloc->pages); | |
| 945 | 945 | if (alloc->mm) |
| 946 | 946 | mmdrop(alloc->mm); |
| 947 | 947 | |
| @@ -6,8 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | * |
| 7 | 7 | * Used by the binder driver to optimize the allocation of the smallest |
| 8 | 8 | * available descriptor ID. Each bit in the bitmap represents the state |
| 9 | − * of an ID, with the exception of BIT(0) which is used exclusively to | |
| 10 | − * reference binder's context manager. | |
| 9 | + * of an ID. | |
| 11 | 10 | * |
| 12 | 11 | * A dbitmap can grow or shrink as needed. This part has been designed |
| 13 | 12 | * considering that users might need to briefly release their locks in |
| @@ -58,11 +57,7 @@ static inline unsigned int dbitmap_shrink_nbits(struct dbitmap *dmap) · 44 unmodified lines | ||
| 58 | 57 | if (bit < (dmap->nbits >> 2)) |
| 59 | 58 | return dmap->nbits >> 1; |
| 60 | 59 | |
| 61 | − /* | |
| 62 | − * Note that find_last_bit() returns dmap->nbits when no bits | |
| 63 | − * are set. While this is technically not possible here since | |
| 64 | − * BIT(0) is always set, this check is left for extra safety. | |
| 65 | − */ | |
| 60 | + /* find_last_bit() returns dmap->nbits when no bits are set. */ | |
| 66 | 61 | if (bit == dmap->nbits) |
| 67 | 62 | return NBITS_MIN; |
| 68 | 63 | |
| @@ -132,16 +127,17 @@ dbitmap_grow(struct dbitmap *dmap, unsigned long *new, unsigned int nbits) · 63 unmodified lines | ||
| 132 | 127 | } |
| 133 | 128 | |
| 134 | 129 | /* |
| 135 | − * Finds and sets the first zero bit in the bitmap. Upon success @bit | |
| 130 | + * Finds and sets the next zero bit in the bitmap. Upon success @bit | |
| 136 | 131 | * is populated with the index and 0 is returned. Otherwise, -ENOSPC |
| 137 | 132 | * is returned to indicate that a dbitmap_grow() is needed. |
| 138 | 133 | */ |
| 139 | 134 | static inline int |
| 140 | −dbitmap_acquire_first_zero_bit(struct dbitmap *dmap, unsigned long *bit) | |
| 135 | +dbitmap_acquire_next_zero_bit(struct dbitmap *dmap, unsigned long offset, | |
| 136 | + unsigned long *bit) | |
| 141 | 137 | { |
| 142 | 138 | unsigned long n; |
| 143 | 139 | |
| 144 | − n = find_first_zero_bit(dmap->map, dmap->nbits); | |
| 140 | + n = find_next_zero_bit(dmap->map, dmap->nbits, offset); | |
| 145 | 141 | if (n == dmap->nbits) |
| 146 | 142 | return -ENOSPC; |
| 147 | 143 | |
| @@ -154,9 +150,7 @@ dbitmap_acquire_first_zero_bit(struct dbitmap *dmap, unsigned long *bit) · 6 unmodified lines | ||
| 154 | 150 | static inline void |
| 155 | 151 | dbitmap_clear_bit(struct dbitmap *dmap, unsigned long bit) |
| 156 | 152 | { |
| 157 | − /* BIT(0) should always set for the context manager */ | |
| 158 | − if (bit) | |
| 159 | − clear_bit(bit, dmap->map); | |
| 153 | + clear_bit(bit, dmap->map); | |
| 160 | 154 | } |
| 161 | 155 | |
| 162 | 156 | static inline int dbitmap_init(struct dbitmap *dmap) |
| @@ -168,8 +162,6 @@ static inline int dbitmap_init(struct dbitmap *dmap) · 5 unmodified lines | ||
| 168 | 162 | } |
| 169 | 163 | |
| 170 | 164 | dmap->nbits = NBITS_MIN; |
| 171 | − /* BIT(0) is reserved for the context manager */ | |
| 172 | − set_bit(0, dmap->map); | |
| 173 | 165 | |
| 174 | 166 | return 0; |
| 175 | 167 | } |
| @@ -951,8 +951,19 @@ static void ata_gen_passthru_sense(struct ata_queued_cmd *qc) · 950 unmodified lines | ||
| 951 | 951 | &sense_key, &asc, &ascq); |
| 952 | 952 | ata_scsi_set_sense(qc->dev, cmd, sense_key, asc, ascq); |
| 953 | 953 | } else { |
| 954 | − /* ATA PASS-THROUGH INFORMATION AVAILABLE */ | |
| 955 | − ata_scsi_set_sense(qc->dev, cmd, RECOVERED_ERROR, 0, 0x1D); | |
| 954 | + /* | |
| 955 | + * ATA PASS-THROUGH INFORMATION AVAILABLE | |
| 956 | + * | |
| 957 | + * Note: we are supposed to call ata_scsi_set_sense(), which | |
| 958 | + * respects the D_SENSE bit, instead of unconditionally | |
| 959 | + * generating the sense data in descriptor format. However, | |
| 960 | + * because hdparm, hddtemp, and udisks incorrectly assume sense | |
| 961 | + * data in descriptor format, without even looking at the | |
| 962 | + * RESPONSE CODE field in the returned sense data (to see which | |
| 963 | + * format the returned sense data is in), we are stuck with | |
| 964 | + * being bug compatible with older kernels. | |
| 965 | + */ | |
| 966 | + scsi_build_sense(cmd, 1, RECOVERED_ERROR, 0, 0x1D); | |
| 956 | 967 | } |
| 957 | 968 | } |
| 958 | 969 | |
| @@ -208,6 +208,19 @@ static const char* macio_ata_names[] = { · 207 unmodified lines | ||
| 208 | 208 | /* Don't let a DMA segment go all the way to 64K */ |
| 209 | 209 | #define MAX_DBDMA_SEG 0xff00 |
| 210 | 210 | |
| 211 | +#ifdef CONFIG_PAGE_SIZE_64KB | |
| 212 | +/* | |
| 213 | + * The SCSI core requires the segment size to cover at least a page, so | |
| 214 | + * for 64K page size kernels it must be at least 64K. However the | |
| 215 | + * hardware can't handle 64K, so pata_macio_qc_prep() will split large | |
| 216 | + * requests. To handle the split requests the tablesize must be halved. | |
| 217 | + */ | |
| 218 | +#define PATA_MACIO_MAX_SEGMENT_SIZE SZ_64K | |
| 219 | +#define PATA_MACIO_SG_TABLESIZE (MAX_DCMDS / 2) | |
| 220 | +#else | |
| 221 | +#define PATA_MACIO_MAX_SEGMENT_SIZE MAX_DBDMA_SEG | |
| 222 | +#define PATA_MACIO_SG_TABLESIZE MAX_DCMDS | |
| 223 | +#endif | |
| 211 | 224 | |
| 212 | 225 | /* |
| 213 | 226 | * Wait 1s for disk to answer on IDE bus after a hard reset |
| @@ -541,7 +554,8 @@ static enum ata_completion_errors pata_macio_qc_prep(struct ata_queued_cmd *qc) · 327 unmodified lines | ||
| 541 | 554 | |
| 542 | 555 | while (sg_len) { |
| 543 | 556 | /* table overflow should never happen */ |
| 544 | − BUG_ON (pi++ >= MAX_DCMDS); | |
| 557 | + if (WARN_ON_ONCE(pi >= MAX_DCMDS)) | |
| 558 | + return AC_ERR_SYSTEM; | |
| 545 | 559 | |
| 546 | 560 | len = (sg_len < MAX_DBDMA_SEG) ? sg_len : MAX_DBDMA_SEG; |
| 547 | 561 | table->command = cpu_to_le16(write ? OUTPUT_MORE: INPUT_MORE); |
| @@ -553,11 +567,13 @@ static enum ata_completion_errors pata_macio_qc_prep(struct ata_queued_cmd *qc) · 5 unmodified lines | ||
| 553 | 567 | addr += len; |
| 554 | 568 | sg_len -= len; |
| 555 | 569 | ++table; |
| 570 | + ++pi; | |
| 556 | 571 | } |
| 557 | 572 | } |
| 558 | 573 | |
| 559 | 574 | /* Should never happen according to Tejun */ |
| 560 | − BUG_ON(!pi); | |
| 575 | + if (WARN_ON_ONCE(!pi)) | |
| 576 | + return AC_ERR_SYSTEM; | |
| 561 | 577 | |
| 562 | 578 | /* Convert the last command to an input/output */ |
| 563 | 579 | table--; |
| @@ -912,16 +928,10 @@ static int pata_macio_do_resume(struct pata_macio_priv *priv) · 348 unmodified lines | ||
| 912 | 928 | |
| 913 | 929 | static const struct scsi_host_template pata_macio_sht = { |
| 914 | 930 | __ATA_BASE_SHT(DRV_NAME), |
| 915 | − .sg_tablesize = MAX_DCMDS, | |
| 931 | + .sg_tablesize = PATA_MACIO_SG_TABLESIZE, | |
| 916 | 932 | /* We may not need that strict one */ |
| 917 | 933 | .dma_boundary = ATA_DMA_BOUNDARY, |
| 918 | − /* | |
| 919 | − * The SCSI core requires the segment size to cover at least a page, so | |
| 920 | − * for 64K page size kernels this must be at least 64K. However the | |
| 921 | − * hardware can't handle 64K, so pata_macio_qc_prep() will split large | |
| 922 | − * requests. | |
| 923 | − */ | |
| 924 | − .max_segment_size = SZ_64K, | |
| 934 | + .max_segment_size = PATA_MACIO_MAX_SEGMENT_SIZE, | |
| 925 | 935 | .device_configure = pata_macio_device_configure, |
| 926 | 936 | .sdev_groups = ata_common_sdev_groups, |
| 927 | 937 | .can_queue = ATA_DEF_QUEUE, |
| @@ -1118,8 +1118,8 @@ dequeue_rx(struct idt77252_dev *card, struct rsq_entry *rsqe) · 1117 unmodified lines | ||
| 1118 | 1118 | rpp->len += skb->len; |
| 1119 | 1119 | |
| 1120 | 1120 | if (stat & SAR_RSQE_EPDU) { |
| 1121 | + unsigned int len, truesize; | |
| 1121 | 1122 | unsigned char *l1l2; |
| 1122 | − unsigned int len; | |
| 1123 | 1123 | |
| 1124 | 1124 | l1l2 = (unsigned char *) ((unsigned long) skb->data + skb->len - 6); |
| 1125 | 1125 | |
| @@ -1189,14 +1189,15 @@ dequeue_rx(struct idt77252_dev *card, struct rsq_entry *rsqe) · 63 unmodified lines | ||
| 1189 | 1189 | ATM_SKB(skb)->vcc = vcc; |
| 1190 | 1190 | __net_timestamp(skb); |
| 1191 | 1191 | |
| 1192 | + truesize = skb->truesize; | |
| 1192 | 1193 | vcc->push(vcc, skb); |
| 1193 | 1194 | atomic_inc(&vcc->stats->rx); |
| 1194 | 1195 | |
| 1195 | − if (skb->truesize > SAR_FB_SIZE_3) | |
| 1196 | + if (truesize > SAR_FB_SIZE_3) | |
| 1196 | 1197 | add_rx_skb(card, 3, SAR_FB_SIZE_3, 1); |
| 1197 | − else if (skb->truesize > SAR_FB_SIZE_2) | |
| 1198 | + else if (truesize > SAR_FB_SIZE_2) | |
| 1198 | 1199 | add_rx_skb(card, 2, SAR_FB_SIZE_2, 1); |
| 1199 | − else if (skb->truesize > SAR_FB_SIZE_1) | |
| 1200 | + else if (truesize > SAR_FB_SIZE_1) | |
| 1200 | 1201 | add_rx_skb(card, 1, SAR_FB_SIZE_1, 1); |
| 1201 | 1202 | else |
| 1202 | 1203 | add_rx_skb(card, 0, SAR_FB_SIZE_0, 1); |
| @@ -25,6 +25,7 @@ · 24 unmodified lines | ||
| 25 | 25 | #include <linux/mutex.h> |
| 26 | 26 | #include <linux/pm_runtime.h> |
| 27 | 27 | #include <linux/netdevice.h> |
| 28 | +#include <linux/rcupdate.h> | |
| 28 | 29 | #include <linux/sched/signal.h> |
| 29 | 30 | #include <linux/sched/mm.h> |
| 30 | 31 | #include <linux/string_helpers.h> |
| @@ -2640,6 +2641,7 @@ static const char *dev_uevent_name(const struct kobject *kobj) · 2609 unmodified lines | ||
| 2640 | 2641 | static int dev_uevent(const struct kobject *kobj, struct kobj_uevent_env *env) |
| 2641 | 2642 | { |
| 2642 | 2643 | const struct device *dev = kobj_to_dev(kobj); |
| 2644 | + struct device_driver *driver; | |
| 2643 | 2645 | int retval = 0; |
| 2644 | 2646 | |
| 2645 | 2647 | /* add device node properties if present */ |
| @@ -2668,8 +2670,12 @@ static int dev_uevent(const struct kobject *kobj, struct kobj_uevent_env *env) · 22 unmodified lines | ||
| 2668 | 2670 | if (dev->type && dev->type->name) |
| 2669 | 2671 | add_uevent_var(env, "DEVTYPE=%s", dev->type->name); |
| 2670 | 2672 | |
| 2671 | − if (dev->driver) | |
| 2672 | − add_uevent_var(env, "DRIVER=%s", dev->driver->name); | |
| 2673 | + /* Synchronize with module_remove_driver() */ | |
| 2674 | + rcu_read_lock(); | |
| 2675 | + driver = READ_ONCE(dev->driver); | |
| 2676 | + if (driver) | |
| 2677 | + add_uevent_var(env, "DRIVER=%s", driver->name); | |
| 2678 | + rcu_read_unlock(); | |
| 2673 | 2679 | |
| 2674 | 2680 | /* Add common DT information about the device */ |
| 2675 | 2681 | of_device_uevent(dev, env); |
| @@ -2739,11 +2745,8 @@ static ssize_t uevent_show(struct device *dev, struct device_attribute *attr, · 63 unmodified lines | ||
| 2739 | 2745 | if (!env) |
| 2740 | 2746 | return -ENOMEM; |
| 2741 | 2747 | |
| 2742 | − /* Synchronize with really_probe() */ | |
| 2743 | − device_lock(dev); | |
| 2744 | 2748 | /* let the kset specific function add its keys */ |
| 2745 | 2749 | retval = kset->uevent_ops->uevent(&dev->kobj, env); |
| 2746 | − device_unlock(dev); | |
| 2747 | 2750 | if (retval) |
| 2748 | 2751 | goto out; |
| 2749 | 2752 | |
| @@ -7,6 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | #include <linux/errno.h> |
| 8 | 8 | #include <linux/slab.h> |
| 9 | 9 | #include <linux/string.h> |
| 10 | +#include <linux/rcupdate.h> | |
| 10 | 11 | #include "base.h" |
| 11 | 12 | |
| 12 | 13 | static char *make_driver_name(const struct device_driver *drv) |
| @@ -97,6 +98,9 @@ void module_remove_driver(const struct device_driver *drv) · 84 unmodified lines | ||
| 97 | 98 | if (!drv) |
| 98 | 99 | return; |
| 99 | 100 | |
| 101 | + /* Synchronize with dev_uevent() */ | |
| 102 | + synchronize_rcu(); | |
| 103 | + | |
| 100 | 104 | sysfs_remove_link(&drv->p->kobj, "module"); |
| 101 | 105 | |
| 102 | 106 | if (drv->owner) |
| @@ -413,6 +413,7 @@ config BT_ATH3K · 412 unmodified lines | ||
| 413 | 413 | config BT_MTKSDIO |
| 414 | 414 | tristate "MediaTek HCI SDIO driver" |
| 415 | 415 | depends on MMC |
| 416 | + depends on USB || !BT_HCIBTUSB_MTK | |
| 416 | 417 | select BT_MTK |
| 417 | 418 | help |
| 418 | 419 | MediaTek Bluetooth HCI SDIO driver. |
| @@ -425,6 +426,7 @@ config BT_MTKSDIO · 6 unmodified lines | ||
| 425 | 426 | config BT_MTKUART |
| 426 | 427 | tristate "MediaTek HCI UART driver" |
| 427 | 428 | depends on SERIAL_DEV_BUS |
| 429 | + depends on USB || !BT_HCIBTUSB_MTK | |
| 428 | 430 | select BT_MTK |
| 429 | 431 | help |
| 430 | 432 | MediaTek Bluetooth HCI UART driver. |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <linux/acpi.h> |
| 13 | 13 | #include <acpi/acpi_bus.h> |
| 14 | 14 | #include <asm/unaligned.h> |
| 15 | +#include <linux/efi.h> | |
| 15 | 16 | |
| 16 | 17 | #include <net/bluetooth/bluetooth.h> |
| 17 | 18 | #include <net/bluetooth/hci_core.h> |
| @@ -26,6 +27,8 @@ · 8 unmodified lines | ||
| 26 | 27 | #define ECDSA_OFFSET 644 |
| 27 | 28 | #define ECDSA_HEADER_LEN 320 |
| 28 | 29 | |
| 30 | +#define BTINTEL_EFI_DSBR L"UefiCnvCommonDSBR" | |
| 31 | + | |
| 29 | 32 | enum { |
| 30 | 33 | DSM_SET_WDISABLE2_DELAY = 1, |
| 31 | 34 | DSM_SET_RESET_METHOD = 3, |
| @@ -2616,6 +2619,120 @@ static u8 btintel_classify_pkt_type(struct hci_dev *hdev, struct sk_buff *skb) · 2584 unmodified lines | ||
| 2616 | 2619 | return hci_skb_pkt_type(skb); |
| 2617 | 2620 | } |
| 2618 | 2621 | |
| 2622 | +/* | |
| 2623 | + * UefiCnvCommonDSBR UEFI variable provides information from the OEM platforms | |
| 2624 | + * if they have replaced the BRI (Bluetooth Radio Interface) resistor to | |
| 2625 | + * overcome the potential STEP errors on their designs. Based on the | |
| 2626 | + * configauration, bluetooth firmware shall adjust the BRI response line drive | |
| 2627 | + * strength. The below structure represents DSBR data. | |
| 2628 | + * struct { | |
| 2629 | + * u8 header; | |
| 2630 | + * u32 dsbr; | |
| 2631 | + * } __packed; | |
| 2632 | + * | |
| 2633 | + * header - defines revision number of the structure | |
| 2634 | + * dsbr - defines drive strength BRI response | |
| 2635 | + * bit0 | |
| 2636 | + * 0 - instructs bluetooth firmware to use default values | |
| 2637 | + * 1 - instructs bluetooth firmware to override default values | |
| 2638 | + * bit3:1 | |
| 2639 | + * Reserved | |
| 2640 | + * bit7:4 | |
| 2641 | + * DSBR override values (only if bit0 is set. Default value is 0xF | |
| 2642 | + * bit31:7 | |
| 2643 | + * Reserved | |
| 2644 | + * Expected values for dsbr field: | |
| 2645 | + * 1. 0xF1 - indicates that the resistor on board is 33 Ohm | |
| 2646 | + * 2. 0x00 or 0xB1 - indicates that the resistor on board is 10 Ohm | |
| 2647 | + * 3. Non existing UEFI variable or invalid (none of the above) - indicates | |
| 2648 | + * that the resistor on board is 10 Ohm | |
| 2649 | + * Even if uefi variable is not present, driver shall send 0xfc0a command to | |
| 2650 | + * firmware to use default values. | |
| 2651 | + * | |
| 2652 | + */ | |
| 2653 | +static int btintel_uefi_get_dsbr(u32 *dsbr_var) | |
| 2654 | +{ | |
| 2655 | + struct btintel_dsbr { | |
| 2656 | + u8 header; | |
| 2657 | + u32 dsbr; | |
| 2658 | + } __packed data; | |
| 2659 | + | |
| 2660 | + efi_status_t status; | |
| 2661 | + unsigned long data_size = 0; | |
| 2662 | + efi_guid_t guid = EFI_GUID(0xe65d8884, 0xd4af, 0x4b20, 0x8d, 0x03, | |
| 2663 | + 0x77, 0x2e, 0xcc, 0x3d, 0xa5, 0x31); | |
| 2664 | + | |
| 2665 | + if (!IS_ENABLED(CONFIG_EFI)) | |
| 2666 | + return -EOPNOTSUPP; | |
| 2667 | + | |
| 2668 | + if (!efi_rt_services_supported(EFI_RT_SUPPORTED_GET_VARIABLE)) | |
| 2669 | + return -EOPNOTSUPP; | |
| 2670 | + | |
| 2671 | + status = efi.get_variable(BTINTEL_EFI_DSBR, &guid, NULL, &data_size, | |
| 2672 | + NULL); | |
| 2673 | + | |
| 2674 | + if (status != EFI_BUFFER_TOO_SMALL || !data_size) | |
| 2675 | + return -EIO; | |
| 2676 | + | |
| 2677 | + status = efi.get_variable(BTINTEL_EFI_DSBR, &guid, NULL, &data_size, | |
| 2678 | + &data); | |
| 2679 | + | |
| 2680 | + if (status != EFI_SUCCESS) | |
| 2681 | + return -ENXIO; | |
| 2682 | + | |
| 2683 | + *dsbr_var = data.dsbr; | |
| 2684 | + return 0; | |
| 2685 | +} | |
| 2686 | + | |
| 2687 | +static int btintel_set_dsbr(struct hci_dev *hdev, struct intel_version_tlv *ver) | |
| 2688 | +{ | |
| 2689 | + struct btintel_dsbr_cmd { | |
| 2690 | + u8 enable; | |
| 2691 | + u8 dsbr; | |
| 2692 | + } __packed; | |
| 2693 | + | |
| 2694 | + struct btintel_dsbr_cmd cmd; | |
| 2695 | + struct sk_buff *skb; | |
| 2696 | + u8 status; | |
| 2697 | + u32 dsbr; | |
| 2698 | + bool apply_dsbr; | |
| 2699 | + int err; | |
| 2700 | + | |
| 2701 | + /* DSBR command needs to be sent for BlazarI + B0 step product after | |
| 2702 | + * downloading IML image. | |
| 2703 | + */ | |
| 2704 | + apply_dsbr = (ver->img_type == BTINTEL_IMG_IML && | |
| 2705 | + ((ver->cnvi_top & 0xfff) == BTINTEL_CNVI_BLAZARI) && | |
| 2706 | + INTEL_CNVX_TOP_STEP(ver->cnvi_top) == 0x01); | |
| 2707 | + | |
| 2708 | + if (!apply_dsbr) | |
| 2709 | + return 0; | |
| 2710 | + | |
| 2711 | + dsbr = 0; | |
| 2712 | + err = btintel_uefi_get_dsbr(&dsbr); | |
| 2713 | + if (err < 0) | |
| 2714 | + bt_dev_dbg(hdev, "Error reading efi: %ls (%d)", | |
| 2715 | + BTINTEL_EFI_DSBR, err); | |
| 2716 | + | |
| 2717 | + cmd.enable = dsbr & BIT(0); | |
| 2718 | + cmd.dsbr = dsbr >> 4 & 0xF; | |
| 2719 | + | |
| 2720 | + bt_dev_info(hdev, "dsbr: enable: 0x%2.2x value: 0x%2.2x", cmd.enable, | |
| 2721 | + cmd.dsbr); | |
| 2722 | + | |
| 2723 | + skb = __hci_cmd_sync(hdev, 0xfc0a, sizeof(cmd), &cmd, HCI_CMD_TIMEOUT); | |
| 2724 | + if (IS_ERR(skb)) | |
| 2725 | + return -bt_to_errno(PTR_ERR(skb)); | |
| 2726 | + | |
| 2727 | + status = skb->data[0]; | |
| 2728 | + kfree_skb(skb); | |
| 2729 | + | |
| 2730 | + if (status) | |
| 2731 | + return -bt_to_errno(status); | |
| 2732 | + | |
| 2733 | + return 0; | |
| 2734 | +} | |
| 2735 | + | |
| 2619 | 2736 | int btintel_bootloader_setup_tlv(struct hci_dev *hdev, |
| 2620 | 2737 | struct intel_version_tlv *ver) |
| 2621 | 2738 | { |
| @@ -2650,6 +2767,13 @@ int btintel_bootloader_setup_tlv(struct hci_dev *hdev, · 28 unmodified lines | ||
| 2650 | 2767 | if (err) |
| 2651 | 2768 | return err; |
| 2652 | 2769 | |
| 2770 | + /* set drive strength of BRI response */ | |
| 2771 | + err = btintel_set_dsbr(hdev, ver); | |
| 2772 | + if (err) { | |
| 2773 | + bt_dev_err(hdev, "Failed to send dsbr command (%d)", err); | |
| 2774 | + return err; | |
| 2775 | + } | |
| 2776 | + | |
| 2653 | 2777 | /* If image type returned is BTINTEL_IMG_IML, then controller supports |
| 2654 | 2778 | * intermediate loader image |
| 2655 | 2779 | */ |
| @@ -2945,9 +3069,6 @@ static int btintel_setup_combined(struct hci_dev *hdev) · 289 unmodified lines | ||
| 2945 | 3069 | INTEL_ROM_LEGACY_NO_WBS_SUPPORT)) |
| 2946 | 3070 | set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, |
| 2947 | 3071 | &hdev->quirks); |
| 2948 | − if (ver.hw_variant == 0x08 && ver.fw_variant == 0x22) | |
| 2949 | − set_bit(HCI_QUIRK_VALID_LE_STATES, | |
| 2950 | − &hdev->quirks); | |
| 2951 | 3072 | |
| 2952 | 3073 | err = btintel_legacy_rom_setup(hdev, &ver); |
| 2953 | 3074 | break; |
| @@ -2956,7 +3077,6 @@ static int btintel_setup_combined(struct hci_dev *hdev) · 2 unmodified lines | ||
| 2956 | 3077 | case 0x12: /* ThP */ |
| 2957 | 3078 | case 0x13: /* HrP */ |
| 2958 | 3079 | case 0x14: /* CcP */ |
| 2959 | − set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); | |
| 2960 | 3080 | fallthrough; |
| 2961 | 3081 | case 0x0c: /* WsP */ |
| 2962 | 3082 | /* Apply the device specific HCI quirks |
| @@ -3048,9 +3168,6 @@ static int btintel_setup_combined(struct hci_dev *hdev) · 85 unmodified lines | ||
| 3048 | 3168 | /* These variants don't seem to support LE Coded PHY */ |
| 3049 | 3169 | set_bit(HCI_QUIRK_BROKEN_LE_CODED, &hdev->quirks); |
| 3050 | 3170 | |
| 3051 | − /* Set Valid LE States quirk */ | |
| 3052 | − set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); | |
| 3053 | − | |
| 3054 | 3171 | /* Setup MSFT Extension support */ |
| 3055 | 3172 | btintel_set_msft_opcode(hdev, ver.hw_variant); |
| 3056 | 3173 | |
| @@ -3076,15 +3193,15 @@ static int btintel_setup_combined(struct hci_dev *hdev) · 19 unmodified lines | ||
| 3076 | 3193 | */ |
| 3077 | 3194 | set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks); |
| 3078 | 3195 | |
| 3079 | − /* Apply LE States quirk from solar onwards */ | |
| 3080 | − set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); | |
| 3081 | − | |
| 3082 | 3196 | /* Setup MSFT Extension support */ |
| 3083 | 3197 | btintel_set_msft_opcode(hdev, |
| 3084 | 3198 | INTEL_HW_VARIANT(ver_tlv.cnvi_bt)); |
| 3085 | 3199 | btintel_set_dsm_reset_method(hdev, &ver_tlv); |
| 3086 | 3200 | |
| 3087 | 3201 | err = btintel_bootloader_setup_tlv(hdev, &ver_tlv); |
| 3202 | + if (err) | |
| 3203 | + goto exit_error; | |
| 3204 | + | |
| 3088 | 3205 | btintel_register_devcoredump_support(hdev); |
| 3089 | 3206 | btintel_print_fseq_info(hdev); |
| 3090 | 3207 | break; |
| @@ -1180,9 +1180,6 @@ static int btintel_pcie_setup(struct hci_dev *hdev) · 1179 unmodified lines | ||
| 1180 | 1180 | */ |
| 1181 | 1181 | set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks); |
| 1182 | 1182 | |
| 1183 | − /* Apply LE States quirk from solar onwards */ | |
| 1184 | − set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); | |
| 1185 | − | |
| 1186 | 1183 | /* Setup MSFT Extension support */ |
| 1187 | 1184 | btintel_set_msft_opcode(hdev, |
| 1188 | 1185 | INTEL_HW_VARIANT(ver_tlv.cnvi_bt)); |
| @@ -437,6 +437,7 @@ int btmtk_process_coredump(struct hci_dev *hdev, struct sk_buff *skb) · 436 unmodified lines | ||
| 437 | 437 | } |
| 438 | 438 | EXPORT_SYMBOL_GPL(btmtk_process_coredump); |
| 439 | 439 | |
| 440 | +#if IS_ENABLED(CONFIG_BT_HCIBTUSB_MTK) | |
| 440 | 441 | static void btmtk_usb_wmt_recv(struct urb *urb) |
| 441 | 442 | { |
| 442 | 443 | struct hci_dev *hdev = urb->context; |
| @@ -1262,7 +1263,8 @@ int btmtk_usb_suspend(struct hci_dev *hdev) · 819 unmodified lines | ||
| 1262 | 1263 | struct btmtk_data *btmtk_data = hci_get_priv(hdev); |
| 1263 | 1264 | |
| 1264 | 1265 | /* Stop urb anchor for iso data transmission */ |
| 1265 | − usb_kill_anchored_urbs(&btmtk_data->isopkt_anchor); | |
| 1266 | + if (test_bit(BTMTK_ISOPKT_RUNNING, &btmtk_data->flags)) | |
| 1267 | + usb_kill_anchored_urbs(&btmtk_data->isopkt_anchor); | |
| 1266 | 1268 | |
| 1267 | 1269 | return 0; |
| 1268 | 1270 | } |
| @@ -1487,6 +1489,7 @@ int btmtk_usb_shutdown(struct hci_dev *hdev) · 218 unmodified lines | ||
| 1487 | 1489 | return 0; |
| 1488 | 1490 | } |
| 1489 | 1491 | EXPORT_SYMBOL_GPL(btmtk_usb_shutdown); |
| 1492 | +#endif | |
| 1490 | 1493 | |
| 1491 | 1494 | MODULE_AUTHOR("Sean Wang <sean.wang@mediatek.com>"); |
| 1492 | 1495 | MODULE_AUTHOR("Mark Chen <mark-yw.chen@mediatek.com>"); |
| @@ -1148,9 +1148,6 @@ static int btmtksdio_setup(struct hci_dev *hdev) · 1147 unmodified lines | ||
| 1148 | 1148 | } |
| 1149 | 1149 | } |
| 1150 | 1150 | |
| 1151 | − /* Valid LE States quirk for MediaTek 7921 */ | |
| 1152 | − set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); | |
| 1153 | − | |
| 1154 | 1151 | break; |
| 1155 | 1152 | case 0x7663: |
| 1156 | 1153 | case 0x7668: |
| @@ -449,6 +449,23 @@ static bool ps_wakeup(struct btnxpuart_dev *nxpdev) · 448 unmodified lines | ||
| 449 | 449 | return false; |
| 450 | 450 | } |
| 451 | 451 | |
| 452 | +static void ps_cleanup(struct btnxpuart_dev *nxpdev) | |
| 453 | +{ | |
| 454 | + struct ps_data *psdata = &nxpdev->psdata; | |
| 455 | + u8 ps_state; | |
| 456 | + | |
| 457 | + mutex_lock(&psdata->ps_lock); | |
| 458 | + ps_state = psdata->ps_state; | |
| 459 | + mutex_unlock(&psdata->ps_lock); | |
| 460 | + | |
| 461 | + if (ps_state != PS_STATE_AWAKE) | |
| 462 | + ps_control(psdata->hdev, PS_STATE_AWAKE); | |
| 463 | + | |
| 464 | + ps_cancel_timer(nxpdev); | |
| 465 | + cancel_work_sync(&psdata->work); | |
| 466 | + mutex_destroy(&psdata->ps_lock); | |
| 467 | +} | |
| 468 | + | |
| 452 | 469 | static int send_ps_cmd(struct hci_dev *hdev, void *data) |
| 453 | 470 | { |
| 454 | 471 | struct btnxpuart_dev *nxpdev = hci_get_drvdata(hdev); |
| @@ -1363,7 +1380,6 @@ static int btnxpuart_close(struct hci_dev *hdev) · 908 unmodified lines | ||
| 1363 | 1380 | { |
| 1364 | 1381 | struct btnxpuart_dev *nxpdev = hci_get_drvdata(hdev); |
| 1365 | 1382 | |
| 1366 | − ps_wakeup(nxpdev); | |
| 1367 | 1383 | serdev_device_close(nxpdev->serdev); |
| 1368 | 1384 | skb_queue_purge(&nxpdev->txq); |
| 1369 | 1385 | if (!IS_ERR_OR_NULL(nxpdev->rx_skb)) { |
| @@ -1516,8 +1532,8 @@ static void nxp_serdev_remove(struct serdev_device *serdev) · 146 unmodified lines | ||
| 1516 | 1532 | nxpdev->new_baudrate = nxpdev->fw_init_baudrate; |
| 1517 | 1533 | nxp_set_baudrate_cmd(hdev, NULL); |
| 1518 | 1534 | } |
| 1519 | − ps_cancel_timer(nxpdev); | |
| 1520 | 1535 | } |
| 1536 | + ps_cleanup(nxpdev); | |
| 1521 | 1537 | hci_unregister_dev(hdev); |
| 1522 | 1538 | hci_free_dev(hdev); |
| 1523 | 1539 | } |
| @@ -1287,7 +1287,6 @@ void btrtl_set_quirks(struct hci_dev *hdev, struct btrtl_device_info *btrtl_dev) · 1286 unmodified lines | ||
| 1287 | 1287 | case CHIP_ID_8852C: |
| 1288 | 1288 | case CHIP_ID_8851B: |
| 1289 | 1289 | case CHIP_ID_8852BT: |
| 1290 | − set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); | |
| 1291 | 1290 | set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks); |
| 1292 | 1291 | |
| 1293 | 1292 | /* RTL8852C needs to transmit mSBC data continuously without |
| @@ -3956,8 +3956,8 @@ static int btusb_probe(struct usb_interface *intf, · 3955 unmodified lines | ||
| 3956 | 3956 | if (id->driver_info & BTUSB_WIDEBAND_SPEECH) |
| 3957 | 3957 | set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks); |
| 3958 | 3958 | |
| 3959 | − if (id->driver_info & BTUSB_VALID_LE_STATES) | |
| 3960 | − set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); | |
| 3959 | + if (!(id->driver_info & BTUSB_VALID_LE_STATES)) | |
| 3960 | + set_bit(HCI_QUIRK_BROKEN_LE_STATES, &hdev->quirks); | |
| 3961 | 3961 | |
| 3962 | 3962 | if (id->driver_info & BTUSB_DIGIANSWER) { |
| 3963 | 3963 | data->cmdreq_type = USB_TYPE_VENDOR; |
| @@ -2160,7 +2160,7 @@ static void qca_power_shutdown(struct hci_uart *hu) · 2159 unmodified lines | ||
| 2160 | 2160 | qcadev = serdev_device_get_drvdata(hu->serdev); |
| 2161 | 2161 | power = qcadev->bt_power; |
| 2162 | 2162 | |
| 2163 | − if (power->pwrseq) { | |
| 2163 | + if (power && power->pwrseq) { | |
| 2164 | 2164 | pwrseq_power_off(power->pwrseq); |
| 2165 | 2165 | set_bit(QCA_BT_OFF, &qca->flags); |
| 2166 | 2166 | return; |
| @@ -2187,10 +2187,6 @@ static void qca_power_shutdown(struct hci_uart *hu) · 20 unmodified lines | ||
| 2187 | 2187 | } |
| 2188 | 2188 | break; |
| 2189 | 2189 | |
| 2190 | − case QCA_QCA6390: | |
| 2191 | − pwrseq_power_off(qcadev->bt_power->pwrseq); | |
| 2192 | − break; | |
| 2193 | − | |
| 2194 | 2190 | default: |
| 2195 | 2191 | gpiod_set_value_cansleep(qcadev->bt_en, 0); |
| 2196 | 2192 | } |
| @@ -2416,11 +2412,14 @@ static int qca_serdev_probe(struct serdev_device *serdev) · 219 unmodified lines | ||
| 2416 | 2412 | break; |
| 2417 | 2413 | |
| 2418 | 2414 | case QCA_QCA6390: |
| 2419 | − qcadev->bt_power->pwrseq = devm_pwrseq_get(&serdev->dev, | |
| 2420 | − "bluetooth"); | |
| 2421 | − if (IS_ERR(qcadev->bt_power->pwrseq)) | |
| 2422 | − return PTR_ERR(qcadev->bt_power->pwrseq); | |
| 2423 | − break; | |
| 2415 | + if (dev_of_node(&serdev->dev)) { | |
| 2416 | + qcadev->bt_power->pwrseq = devm_pwrseq_get(&serdev->dev, | |
| 2417 | + "bluetooth"); | |
| 2418 | + if (IS_ERR(qcadev->bt_power->pwrseq)) | |
| 2419 | + return PTR_ERR(qcadev->bt_power->pwrseq); | |
| 2420 | + break; | |
| 2421 | + } | |
| 2422 | + fallthrough; | |
| 2424 | 2423 | |
| 2425 | 2424 | default: |
| 2426 | 2425 | qcadev->bt_en = devm_gpiod_get_optional(&serdev->dev, "enable", |
| @@ -2475,8 +2474,8 @@ static int qca_serdev_probe(struct serdev_device *serdev) · 48 unmodified lines | ||
| 2475 | 2474 | set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, |
| 2476 | 2475 | &hdev->quirks); |
| 2477 | 2476 | |
| 2478 | − if (data->capabilities & QCA_CAP_VALID_LE_STATES) | |
| 2479 | − set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); | |
| 2477 | + if (!(data->capabilities & QCA_CAP_VALID_LE_STATES)) | |
| 2478 | + set_bit(HCI_QUIRK_BROKEN_LE_STATES, &hdev->quirks); | |
| 2480 | 2479 | } |
| 2481 | 2480 | |
| 2482 | 2481 | return 0; |
| @@ -425,8 +425,6 @@ static int __vhci_create_device(struct vhci_data *data, __u8 opcode) · 424 unmodified lines | ||
| 425 | 425 | if (opcode & 0x80) |
| 426 | 426 | set_bit(HCI_QUIRK_RAW_DEVICE, &hdev->quirks); |
| 427 | 427 | |
| 428 | − set_bit(HCI_QUIRK_VALID_LE_STATES, &hdev->quirks); | |
| 429 | − | |
| 430 | 428 | if (hci_register_dev(hdev) < 0) { |
| 431 | 429 | BT_ERR("Can't register HCI device"); |
| 432 | 430 | hci_free_dev(hdev); |
| @@ -18,6 +18,7 @@ config STARFIVE_STARLINK_CACHE · 17 unmodified lines | ||
| 18 | 18 | bool "StarFive StarLink Cache controller" |
| 19 | 19 | depends on RISCV |
| 20 | 20 | depends on ARCH_STARFIVE |
| 21 | + depends on 64BIT | |
| 21 | 22 | select RISCV_DMA_NONCOHERENT |
| 22 | 23 | select RISCV_NONSTANDARD_CACHE_OPS |
| 23 | 24 | help |
| @@ -421,4 +421,5 @@ static void __exit ds1620_exit(void) · 420 unmodified lines | ||
| 421 | 421 | module_init(ds1620_init); |
| 422 | 422 | module_exit(ds1620_exit); |
| 423 | 423 | |
| 424 | +MODULE_DESCRIPTION("Dallas Semiconductor DS1620 thermometer driver"); | |
| 424 | 425 | MODULE_LICENSE("GPL"); |
| @@ -241,6 +241,7 @@ static void __exit nwbutton_exit (void) · 240 unmodified lines | ||
| 241 | 241 | |
| 242 | 242 | |
| 243 | 243 | MODULE_AUTHOR("Alex Holden"); |
| 244 | +MODULE_DESCRIPTION("NetWinder button driver"); | |
| 244 | 245 | MODULE_LICENSE("GPL"); |
| 245 | 246 | |
| 246 | 247 | module_init(nwbutton_init); |
| @@ -618,6 +618,7 @@ static void __exit nwflash_exit(void) · 617 unmodified lines | ||
| 618 | 618 | iounmap((void *)FLASH_BASE); |
| 619 | 619 | } |
| 620 | 620 | |
| 621 | +MODULE_DESCRIPTION("NetWinder flash memory driver"); | |
| 621 | 622 | MODULE_LICENSE("GPL"); |
| 622 | 623 | |
| 623 | 624 | module_param(flashdebug, bool, 0644); |
| @@ -698,6 +698,10 @@ static int tpm_ibmvtpm_probe(struct vio_dev *vio_dev, · 697 unmodified lines | ||
| 698 | 698 | rc = tpm2_get_cc_attrs_tbl(chip); |
| 699 | 699 | if (rc) |
| 700 | 700 | goto init_irq_cleanup; |
| 701 | + | |
| 702 | + rc = tpm2_sessions_init(chip); | |
| 703 | + if (rc) | |
| 704 | + goto init_irq_cleanup; | |
| 701 | 705 | } |
| 702 | 706 | |
| 703 | 707 | return tpm_chip_register(chip); |
| @@ -50,6 +50,7 @@ MODULE_LICENSE("GPL v2"); · 49 unmodified lines | ||
| 50 | 50 | static const char xillyname[] = "xillyusb"; |
| 51 | 51 | |
| 52 | 52 | static unsigned int fifo_buf_order; |
| 53 | +static struct workqueue_struct *wakeup_wq; | |
| 53 | 54 | |
| 54 | 55 | #define USB_VENDOR_ID_XILINX 0x03fd |
| 55 | 56 | #define USB_VENDOR_ID_ALTERA 0x09fb |
| @@ -569,10 +570,6 @@ static void cleanup_dev(struct kref *kref) · 513 unmodified lines | ||
| 569 | 570 | * errors if executed. The mechanism relies on that xdev->error is assigned |
| 570 | 571 | * a non-zero value by report_io_error() prior to queueing wakeup_all(), |
| 571 | 572 | * which prevents bulk_in_work() from calling process_bulk_in(). |
| 572 | − * | |
| 573 | − * The fact that wakeup_all() and bulk_in_work() are queued on the same | |
| 574 | − * workqueue makes their concurrent execution very unlikely, however the | |
| 575 | − * kernel's API doesn't seem to ensure this strictly. | |
| 576 | 573 | */ |
| 577 | 574 | |
| 578 | 575 | static void wakeup_all(struct work_struct *work) |
| @@ -627,7 +624,7 @@ static void report_io_error(struct xillyusb_dev *xdev, · 48 unmodified lines | ||
| 627 | 624 | |
| 628 | 625 | if (do_once) { |
| 629 | 626 | kref_get(&xdev->kref); /* xdev is used by work item */ |
| 630 | − queue_work(xdev->workq, &xdev->wakeup_workitem); | |
| 627 | + queue_work(wakeup_wq, &xdev->wakeup_workitem); | |
| 631 | 628 | } |
| 632 | 629 | } |
| 633 | 630 | |
| @@ -1906,6 +1903,13 @@ static const struct file_operations xillyusb_fops = { · 1272 unmodified lines | ||
| 1906 | 1903 | |
| 1907 | 1904 | static int xillyusb_setup_base_eps(struct xillyusb_dev *xdev) |
| 1908 | 1905 | { |
| 1906 | + struct usb_device *udev = xdev->udev; | |
| 1907 | + | |
| 1908 | + /* Verify that device has the two fundamental bulk in/out endpoints */ | |
| 1909 | + if (usb_pipe_type_check(udev, usb_sndbulkpipe(udev, MSG_EP_NUM)) || | |
| 1910 | + usb_pipe_type_check(udev, usb_rcvbulkpipe(udev, IN_EP_NUM))) | |
| 1911 | + return -ENODEV; | |
| 1912 | + | |
| 1909 | 1913 | xdev->msg_ep = endpoint_alloc(xdev, MSG_EP_NUM | USB_DIR_OUT, |
| 1910 | 1914 | bulk_out_work, 1, 2); |
| 1911 | 1915 | if (!xdev->msg_ep) |
| @@ -1935,14 +1939,15 @@ static int setup_channels(struct xillyusb_dev *xdev, · 23 unmodified lines | ||
| 1935 | 1939 | __le16 *chandesc, |
| 1936 | 1940 | int num_channels) |
| 1937 | 1941 | { |
| 1938 | − struct xillyusb_channel *chan; | |
| 1942 | + struct usb_device *udev = xdev->udev; | |
| 1943 | + struct xillyusb_channel *chan, *new_channels; | |
| 1939 | 1944 | int i; |
| 1940 | 1945 | |
| 1941 | 1946 | chan = kcalloc(num_channels, sizeof(*chan), GFP_KERNEL); |
| 1942 | 1947 | if (!chan) |
| 1943 | 1948 | return -ENOMEM; |
| 1944 | 1949 | |
| 1945 | − xdev->channels = chan; | |
| 1950 | + new_channels = chan; | |
| 1946 | 1951 | |
| 1947 | 1952 | for (i = 0; i < num_channels; i++, chan++) { |
| 1948 | 1953 | unsigned int in_desc = le16_to_cpu(*chandesc++); |
| @@ -1971,6 +1976,15 @@ static int setup_channels(struct xillyusb_dev *xdev, · 22 unmodified lines | ||
| 1971 | 1976 | */ |
| 1972 | 1977 | |
| 1973 | 1978 | if ((out_desc & 0x80) && i < 14) { /* Entry is valid */ |
| 1979 | + if (usb_pipe_type_check(udev, | |
| 1980 | + usb_sndbulkpipe(udev, i + 2))) { | |
| 1981 | + dev_err(xdev->dev, | |
| 1982 | + "Missing BULK OUT endpoint %d\n", | |
| 1983 | + i + 2); | |
| 1984 | + kfree(new_channels); | |
| 1985 | + return -ENODEV; | |
| 1986 | + } | |
| 1987 | + | |
| 1974 | 1988 | chan->writable = 1; |
| 1975 | 1989 | chan->out_synchronous = !!(out_desc & 0x40); |
| 1976 | 1990 | chan->out_seekable = !!(out_desc & 0x20); |
| @@ -1980,6 +1994,7 @@ static int setup_channels(struct xillyusb_dev *xdev, · 3 unmodified lines | ||
| 1980 | 1994 | } |
| 1981 | 1995 | } |
| 1982 | 1996 | |
| 1997 | + xdev->channels = new_channels; | |
| 1983 | 1998 | return 0; |
| 1984 | 1999 | } |
| 1985 | 2000 | |
| @@ -2096,9 +2111,11 @@ static int xillyusb_discovery(struct usb_interface *interface) · 110 unmodified lines | ||
| 2096 | 2111 | * just after responding with the IDT, there is no reason for any |
| 2097 | 2112 | * work item to be running now. To be sure that xdev->channels |
| 2098 | 2113 | * is updated on anything that might run in parallel, flush the |
| 2099 | − * workqueue, which rarely does anything. | |
| 2114 | + * device's workqueue and the wakeup work item. This rarely | |
| 2115 | + * does anything. | |
| 2100 | 2116 | */ |
| 2101 | 2117 | flush_workqueue(xdev->workq); |
| 2118 | + flush_work(&xdev->wakeup_workitem); | |
| 2102 | 2119 | |
| 2103 | 2120 | xdev->num_channels = num_channels; |
| 2104 | 2121 | |
| @@ -2258,6 +2275,10 @@ static int __init xillyusb_init(void) · 153 unmodified lines | ||
| 2258 | 2275 | { |
| 2259 | 2276 | int rc = 0; |
| 2260 | 2277 | |
| 2278 | + wakeup_wq = alloc_workqueue(xillyname, 0, 0); | |
| 2279 | + if (!wakeup_wq) | |
| 2280 | + return -ENOMEM; | |
| 2281 | + | |
| 2261 | 2282 | if (LOG2_INITIAL_FIFO_BUF_SIZE > PAGE_SHIFT) |
| 2262 | 2283 | fifo_buf_order = LOG2_INITIAL_FIFO_BUF_SIZE - PAGE_SHIFT; |
| 2263 | 2284 | else |
| @@ -2265,12 +2286,17 @@ static int __init xillyusb_init(void) · 1 unmodified line | ||
| 2265 | 2286 | |
| 2266 | 2287 | rc = usb_register(&xillyusb_driver); |
| 2267 | 2288 | |
| 2289 | + if (rc) | |
| 2290 | + destroy_workqueue(wakeup_wq); | |
| 2291 | + | |
| 2268 | 2292 | return rc; |
| 2269 | 2293 | } |
| 2270 | 2294 | |
| 2271 | 2295 | static void __exit xillyusb_exit(void) |
| 2272 | 2296 | { |
| 2273 | 2297 | usb_deregister(&xillyusb_driver); |
| 2298 | + | |
| 2299 | + destroy_workqueue(wakeup_wq); | |
| 2274 | 2300 | } |
| 2275 | 2301 | |
| 2276 | 2302 | module_init(xillyusb_init); |
| @@ -738,7 +738,7 @@ static struct ccu_div vp_axi_clk = { · 737 unmodified lines | ||
| 738 | 738 | .hw.init = CLK_HW_INIT_PARENTS_HW("vp-axi", |
| 739 | 739 | video_pll_clk_parent, |
| 740 | 740 | &ccu_div_ops, |
| 741 | − 0), | |
| 741 | + CLK_IGNORE_UNUSED), | |
| 742 | 742 | }, |
| 743 | 743 | }; |
| 744 | 744 | |
| @@ -160,14 +160,17 @@ static void amd_pstate_ut_check_perf(u32 index) · 159 unmodified lines | ||
| 160 | 160 | lowest_perf = AMD_CPPC_LOWEST_PERF(cap1); |
| 161 | 161 | } |
| 162 | 162 | |
| 163 | − if ((highest_perf != READ_ONCE(cpudata->highest_perf)) || | |
| 164 | − (nominal_perf != READ_ONCE(cpudata->nominal_perf)) || | |
| 163 | + if (highest_perf != READ_ONCE(cpudata->highest_perf) && !cpudata->hw_prefcore) { | |
| 164 | + pr_err("%s cpu%d highest=%d %d highest perf doesn't match\n", | |
| 165 | + __func__, cpu, highest_perf, cpudata->highest_perf); | |
| 166 | + goto skip_test; | |
| 167 | + } | |
| 168 | + if ((nominal_perf != READ_ONCE(cpudata->nominal_perf)) || | |
| 165 | 169 | (lowest_nonlinear_perf != READ_ONCE(cpudata->lowest_nonlinear_perf)) || |
| 166 | 170 | (lowest_perf != READ_ONCE(cpudata->lowest_perf))) { |
| 167 | 171 | amd_pstate_ut_cases[index].result = AMD_PSTATE_UT_RESULT_FAIL; |
| 168 | − pr_err("%s cpu%d highest=%d %d nominal=%d %d lowest_nonlinear=%d %d lowest=%d %d, they should be equal!\n", | |
| 169 | − __func__, cpu, highest_perf, cpudata->highest_perf, | |
| 170 | − nominal_perf, cpudata->nominal_perf, | |
| 172 | + pr_err("%s cpu%d nominal=%d %d lowest_nonlinear=%d %d lowest=%d %d, they should be equal!\n", | |
| 173 | + __func__, cpu, nominal_perf, cpudata->nominal_perf, | |
| 171 | 174 | lowest_nonlinear_perf, cpudata->lowest_nonlinear_perf, |
| 172 | 175 | lowest_perf, cpudata->lowest_perf); |
| 173 | 176 | goto skip_test; |
| @@ -321,7 +321,7 @@ static inline int pstate_enable(bool enable) · 320 unmodified lines | ||
| 321 | 321 | return 0; |
| 322 | 322 | |
| 323 | 323 | for_each_present_cpu(cpu) { |
| 324 | − unsigned long logical_id = topology_logical_die_id(cpu); | |
| 324 | + unsigned long logical_id = topology_logical_package_id(cpu); | |
| 325 | 325 | |
| 326 | 326 | if (test_bit(logical_id, &logical_proc_id_mask)) |
| 327 | 327 | continue; |
| @@ -692,7 +692,7 @@ static int amd_pstate_cpu_boost_update(struct cpufreq_policy *policy, bool on) · 364 unmodified lines | ||
| 692 | 692 | struct amd_cpudata *cpudata = policy->driver_data; |
| 693 | 693 | struct cppc_perf_ctrls perf_ctrls; |
| 694 | 694 | u32 highest_perf, nominal_perf, nominal_freq, max_freq; |
| 695 | − int ret; | |
| 695 | + int ret = 0; | |
| 696 | 696 | |
| 697 | 697 | highest_perf = READ_ONCE(cpudata->highest_perf); |
| 698 | 698 | nominal_perf = READ_ONCE(cpudata->nominal_perf); |
| @@ -3405,6 +3405,7 @@ static const struct x86_cpu_id intel_epp_default[] = { · 3404 unmodified lines | ||
| 3405 | 3405 | */ |
| 3406 | 3406 | X86_MATCH_VFM(INTEL_ALDERLAKE_L, HWP_SET_DEF_BALANCE_PERF_EPP(102)), |
| 3407 | 3407 | X86_MATCH_VFM(INTEL_SAPPHIRERAPIDS_X, HWP_SET_DEF_BALANCE_PERF_EPP(32)), |
| 3408 | + X86_MATCH_VFM(INTEL_EMERALDRAPIDS_X, HWP_SET_DEF_BALANCE_PERF_EPP(32)), | |
| 3408 | 3409 | X86_MATCH_VFM(INTEL_METEORLAKE_L, HWP_SET_EPP_VALUES(HWP_EPP_POWERSAVE, |
| 3409 | 3410 | 179, 64, 16)), |
| 3410 | 3411 | X86_MATCH_VFM(INTEL_ARROWLAKE, HWP_SET_EPP_VALUES(HWP_EPP_POWERSAVE, |
| @@ -834,11 +834,13 @@ static void cxl_disable_rch_root_ints(struct cxl_dport *dport) · 833 unmodified lines | ||
| 834 | 834 | void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport) |
| 835 | 835 | { |
| 836 | 836 | struct device *dport_dev = dport->dport_dev; |
| 837 | − struct pci_host_bridge *host_bridge; | |
| 838 | 837 | |
| 839 | − host_bridge = to_pci_host_bridge(dport_dev); | |
| 840 | − if (host_bridge->native_aer) | |
| 841 | − dport->rcrb.aer_cap = cxl_rcrb_to_aer(dport_dev, dport->rcrb.base); | |
| 838 | + if (dport->rch) { | |
| 839 | + struct pci_host_bridge *host_bridge = to_pci_host_bridge(dport_dev); | |
| 840 | + | |
| 841 | + if (host_bridge->native_aer) | |
| 842 | + dport->rcrb.aer_cap = cxl_rcrb_to_aer(dport_dev, dport->rcrb.base); | |
| 843 | + } | |
| 842 | 844 | |
| 843 | 845 | dport->reg_map.host = host; |
| 844 | 846 | cxl_dport_map_regs(dport); |
| @@ -17,8 +17,8 @@ enum dw_hdma_control { · 16 unmodified lines | ||
| 17 | 17 | DW_HDMA_V0_CB = BIT(0), |
| 18 | 18 | DW_HDMA_V0_TCB = BIT(1), |
| 19 | 19 | DW_HDMA_V0_LLP = BIT(2), |
| 20 | − DW_HDMA_V0_LIE = BIT(3), | |
| 21 | − DW_HDMA_V0_RIE = BIT(4), | |
| 20 | + DW_HDMA_V0_LWIE = BIT(3), | |
| 21 | + DW_HDMA_V0_RWIE = BIT(4), | |
| 22 | 22 | DW_HDMA_V0_CCS = BIT(8), |
| 23 | 23 | DW_HDMA_V0_LLE = BIT(9), |
| 24 | 24 | }; |
| @@ -195,25 +195,14 @@ static void dw_hdma_v0_write_ll_link(struct dw_edma_chunk *chunk, · 170 unmodified lines | ||
| 195 | 195 | static void dw_hdma_v0_core_write_chunk(struct dw_edma_chunk *chunk) |
| 196 | 196 | { |
| 197 | 197 | struct dw_edma_burst *child; |
| 198 | − struct dw_edma_chan *chan = chunk->chan; | |
| 199 | 198 | u32 control = 0, i = 0; |
| 200 | − int j; | |
| 201 | 199 | |
| 202 | 200 | if (chunk->cb) |
| 203 | 201 | control = DW_HDMA_V0_CB; |
| 204 | 202 | |
| 205 | − j = chunk->bursts_alloc; | |
| 206 | − list_for_each_entry(child, &chunk->burst->list, list) { | |
| 207 | − j--; | |
| 208 | − if (!j) { | |
| 209 | − control |= DW_HDMA_V0_LIE; | |
| 210 | − if (!(chan->dw->chip->flags & DW_EDMA_CHIP_LOCAL)) | |
| 211 | − control |= DW_HDMA_V0_RIE; | |
| 212 | − } | |
| 213 | − | |
| 203 | + list_for_each_entry(child, &chunk->burst->list, list) | |
| 214 | 204 | dw_hdma_v0_write_ll_data(chunk, i++, control, child->sz, |
| 215 | 205 | child->sar, child->dar); |
| 216 | − } | |
| 217 | 206 | |
| 218 | 207 | control = DW_HDMA_V0_LLP | DW_HDMA_V0_TCB; |
| 219 | 208 | if (!chunk->cb) |
| @@ -247,10 +236,11 @@ static void dw_hdma_v0_core_start(struct dw_edma_chunk *chunk, bool first) · 27 unmodified lines | ||
| 247 | 236 | if (first) { |
| 248 | 237 | /* Enable engine */ |
| 249 | 238 | SET_CH_32(dw, chan->dir, chan->id, ch_en, BIT(0)); |
| 250 | − /* Interrupt enable&unmask - done, abort */ | |
| 251 | − tmp = GET_CH_32(dw, chan->dir, chan->id, int_setup) | | |
| 252 | − HDMA_V0_STOP_INT_MASK | HDMA_V0_ABORT_INT_MASK | | |
| 253 | − HDMA_V0_LOCAL_STOP_INT_EN | HDMA_V0_LOCAL_ABORT_INT_EN; | |
| 239 | + /* Interrupt unmask - stop, abort */ | |
| 240 | + tmp = GET_CH_32(dw, chan->dir, chan->id, int_setup); | |
| 241 | + tmp &= ~(HDMA_V0_STOP_INT_MASK | HDMA_V0_ABORT_INT_MASK); | |
| 242 | + /* Interrupt enable - stop, abort */ | |
| 243 | + tmp |= HDMA_V0_LOCAL_STOP_INT_EN | HDMA_V0_LOCAL_ABORT_INT_EN; | |
| 254 | 244 | if (!(dw->chip->flags & DW_EDMA_CHIP_LOCAL)) |
| 255 | 245 | tmp |= HDMA_V0_REMOTE_STOP_INT_EN | HDMA_V0_REMOTE_ABORT_INT_EN; |
| 256 | 246 | SET_CH_32(dw, chan->dir, chan->id, int_setup, tmp); |
| @@ -16,6 +16,7 @@ · 15 unmodified lines | ||
| 16 | 16 | #include <linux/init.h> |
| 17 | 17 | #include <linux/interrupt.h> |
| 18 | 18 | #include <linux/io.h> |
| 19 | +#include <linux/log2.h> | |
| 19 | 20 | #include <linux/mm.h> |
| 20 | 21 | #include <linux/module.h> |
| 21 | 22 | #include <linux/slab.h> |
| @@ -621,12 +622,10 @@ dwc_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl, · 599 unmodified lines | ||
| 621 | 622 | struct dw_desc *prev; |
| 622 | 623 | struct dw_desc *first; |
| 623 | 624 | u32 ctllo, ctlhi; |
| 624 | − u8 m_master = dwc->dws.m_master; | |
| 625 | − u8 lms = DWC_LLP_LMS(m_master); | |
| 625 | + u8 lms = DWC_LLP_LMS(dwc->dws.m_master); | |
| 626 | 626 | dma_addr_t reg; |
| 627 | 627 | unsigned int reg_width; |
| 628 | 628 | unsigned int mem_width; |
| 629 | − unsigned int data_width = dw->pdata->data_width[m_master]; | |
| 630 | 629 | unsigned int i; |
| 631 | 630 | struct scatterlist *sg; |
| 632 | 631 | size_t total_len = 0; |
| @@ -660,7 +659,7 @@ dwc_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl, · 27 unmodified lines | ||
| 660 | 659 | mem = sg_dma_address(sg); |
| 661 | 660 | len = sg_dma_len(sg); |
| 662 | 661 | |
| 663 | − mem_width = __ffs(data_width | mem | len); | |
| 662 | + mem_width = __ffs(sconfig->src_addr_width | mem | len); | |
| 664 | 663 | |
| 665 | 664 | slave_sg_todev_fill_desc: |
| 666 | 665 | desc = dwc_desc_get(dwc); |
| @@ -720,7 +719,7 @@ slave_sg_fromdev_fill_desc: · 53 unmodified lines | ||
| 720 | 719 | lli_write(desc, sar, reg); |
| 721 | 720 | lli_write(desc, dar, mem); |
| 722 | 721 | lli_write(desc, ctlhi, ctlhi); |
| 723 | − mem_width = __ffs(data_width | mem); | |
| 722 | + mem_width = __ffs(sconfig->dst_addr_width | mem); | |
| 724 | 723 | lli_write(desc, ctllo, ctllo | DWC_CTLL_DST_WIDTH(mem_width)); |
| 725 | 724 | desc->len = dlen; |
| 726 | 725 | |
| @@ -780,20 +779,108 @@ bool dw_dma_filter(struct dma_chan *chan, void *param) · 53 unmodified lines | ||
| 780 | 779 | } |
| 781 | 780 | EXPORT_SYMBOL_GPL(dw_dma_filter); |
| 782 | 781 | |
| 783 | −static int dwc_config(struct dma_chan *chan, struct dma_slave_config *sconfig) | |
| 782 | +static int dwc_verify_maxburst(struct dma_chan *chan) | |
| 784 | 783 | { |
| 785 | 784 | struct dw_dma_chan *dwc = to_dw_dma_chan(chan); |
| 786 | − struct dw_dma *dw = to_dw_dma(chan->device); | |
| 787 | 785 | |
| 788 | − memcpy(&dwc->dma_sconfig, sconfig, sizeof(*sconfig)); | |
| 786 | + dwc->dma_sconfig.src_maxburst = | |
| 787 | + clamp(dwc->dma_sconfig.src_maxburst, 1U, dwc->max_burst); | |
| 788 | + dwc->dma_sconfig.dst_maxburst = | |
| 789 | + clamp(dwc->dma_sconfig.dst_maxburst, 1U, dwc->max_burst); | |
| 789 | 790 | |
| 790 | 791 | dwc->dma_sconfig.src_maxburst = |
| 791 | − clamp(dwc->dma_sconfig.src_maxburst, 0U, dwc->max_burst); | |
| 792 | + rounddown_pow_of_two(dwc->dma_sconfig.src_maxburst); | |
| 792 | 793 | dwc->dma_sconfig.dst_maxburst = |
| 793 | − clamp(dwc->dma_sconfig.dst_maxburst, 0U, dwc->max_burst); | |
| 794 | + rounddown_pow_of_two(dwc->dma_sconfig.dst_maxburst); | |
| 794 | 795 | |
| 795 | − dw->encode_maxburst(dwc, &dwc->dma_sconfig.src_maxburst); | |
| 796 | − dw->encode_maxburst(dwc, &dwc->dma_sconfig.dst_maxburst); | |
| 796 | + return 0; | |
| 797 | +} | |
| 798 | + | |
| 799 | +static int dwc_verify_p_buswidth(struct dma_chan *chan) | |
| 800 | +{ | |
| 801 | + struct dw_dma_chan *dwc = to_dw_dma_chan(chan); | |
| 802 | + struct dw_dma *dw = to_dw_dma(chan->device); | |
| 803 | + u32 reg_width, max_width; | |
| 804 | + | |
| 805 | + if (dwc->dma_sconfig.direction == DMA_MEM_TO_DEV) | |
| 806 | + reg_width = dwc->dma_sconfig.dst_addr_width; | |
| 807 | + else if (dwc->dma_sconfig.direction == DMA_DEV_TO_MEM) | |
| 808 | + reg_width = dwc->dma_sconfig.src_addr_width; | |
| 809 | + else /* DMA_MEM_TO_MEM */ | |
| 810 | + return 0; | |
| 811 | + | |
| 812 | + max_width = dw->pdata->data_width[dwc->dws.p_master]; | |
| 813 | + | |
| 814 | + /* Fall-back to 1-byte transfer width if undefined */ | |
| 815 | + if (reg_width == DMA_SLAVE_BUSWIDTH_UNDEFINED) | |
| 816 | + reg_width = DMA_SLAVE_BUSWIDTH_1_BYTE; | |
| 817 | + else if (!is_power_of_2(reg_width) || reg_width > max_width) | |
| 818 | + return -EINVAL; | |
| 819 | + else /* bus width is valid */ | |
| 820 | + return 0; | |
| 821 | + | |
| 822 | + /* Update undefined addr width value */ | |
| 823 | + if (dwc->dma_sconfig.direction == DMA_MEM_TO_DEV) | |
| 824 | + dwc->dma_sconfig.dst_addr_width = reg_width; | |
| 825 | + else /* DMA_DEV_TO_MEM */ | |
| 826 | + dwc->dma_sconfig.src_addr_width = reg_width; | |
| 827 | + | |
| 828 | + return 0; | |
| 829 | +} | |
| 830 | + | |
| 831 | +static int dwc_verify_m_buswidth(struct dma_chan *chan) | |
| 832 | +{ | |
| 833 | + struct dw_dma_chan *dwc = to_dw_dma_chan(chan); | |
| 834 | + struct dw_dma *dw = to_dw_dma(chan->device); | |
| 835 | + u32 reg_width, reg_burst, mem_width; | |
| 836 | + | |
| 837 | + mem_width = dw->pdata->data_width[dwc->dws.m_master]; | |
| 838 | + | |
| 839 | + /* | |
| 840 | + * It's possible to have a data portion locked in the DMA FIFO in case | |
| 841 | + * of the channel suspension. Subsequent channel disabling will cause | |
| 842 | + * that data silent loss. In order to prevent that maintain the src and | |
| 843 | + * dst transfer widths coherency by means of the relation: | |
| 844 | + * (CTLx.SRC_TR_WIDTH * CTLx.SRC_MSIZE >= CTLx.DST_TR_WIDTH) | |
| 845 | + * Look for the details in the commit message that brings this change. | |
| 846 | + * | |
| 847 | + * Note the DMA configs utilized in the calculations below must have | |
| 848 | + * been verified to have correct values by this method call. | |
| 849 | + */ | |
| 850 | + if (dwc->dma_sconfig.direction == DMA_MEM_TO_DEV) { | |
| 851 | + reg_width = dwc->dma_sconfig.dst_addr_width; | |
| 852 | + if (mem_width < reg_width) | |
| 853 | + return -EINVAL; | |
| 854 | + | |
| 855 | + dwc->dma_sconfig.src_addr_width = mem_width; | |
| 856 | + } else if (dwc->dma_sconfig.direction == DMA_DEV_TO_MEM) { | |
| 857 | + reg_width = dwc->dma_sconfig.src_addr_width; | |
| 858 | + reg_burst = dwc->dma_sconfig.src_maxburst; | |
| 859 | + | |
| 860 | + dwc->dma_sconfig.dst_addr_width = min(mem_width, reg_width * reg_burst); | |
| 861 | + } | |
| 862 | + | |
| 863 | + return 0; | |
| 864 | +} | |
| 865 | + | |
| 866 | +static int dwc_config(struct dma_chan *chan, struct dma_slave_config *sconfig) | |
| 867 | +{ | |
| 868 | + struct dw_dma_chan *dwc = to_dw_dma_chan(chan); | |
| 869 | + int ret; | |
| 870 | + | |
| 871 | + memcpy(&dwc->dma_sconfig, sconfig, sizeof(*sconfig)); | |
| 872 | + | |
| 873 | + ret = dwc_verify_maxburst(chan); | |
| 874 | + if (ret) | |
| 875 | + return ret; | |
| 876 | + | |
| 877 | + ret = dwc_verify_p_buswidth(chan); | |
| 878 | + if (ret) | |
| 879 | + return ret; | |
| 880 | + | |
| 881 | + ret = dwc_verify_m_buswidth(chan); | |
| 882 | + if (ret) | |
| 883 | + return ret; | |
| 797 | 884 | |
| 798 | 885 | return 0; |
| 799 | 886 | } |
| @@ -1068,7 +1155,7 @@ int do_dma_probe(struct dw_dma_chip *chip) · 268 unmodified lines | ||
| 1068 | 1155 | bool autocfg = false; |
| 1069 | 1156 | unsigned int dw_params; |
| 1070 | 1157 | unsigned int i; |
| 1071 | − int err; | |
| 1158 | + int ret; | |
| 1072 | 1159 | |
| 1073 | 1160 | dw->pdata = devm_kzalloc(chip->dev, sizeof(*dw->pdata), GFP_KERNEL); |
| 1074 | 1161 | if (!dw->pdata) |
| @@ -1084,7 +1171,7 @@ int do_dma_probe(struct dw_dma_chip *chip) · 9 unmodified lines | ||
| 1084 | 1171 | |
| 1085 | 1172 | autocfg = dw_params >> DW_PARAMS_EN & 1; |
| 1086 | 1173 | if (!autocfg) { |
| 1087 | − err = -EINVAL; | |
| 1174 | + ret = -EINVAL; | |
| 1088 | 1175 | goto err_pdata; |
| 1089 | 1176 | } |
| 1090 | 1177 | |
| @@ -1104,7 +1191,7 @@ int do_dma_probe(struct dw_dma_chip *chip) · 13 unmodified lines | ||
| 1104 | 1191 | pdata->chan_allocation_order = CHAN_ALLOCATION_ASCENDING; |
| 1105 | 1192 | pdata->chan_priority = CHAN_PRIORITY_ASCENDING; |
| 1106 | 1193 | } else if (chip->pdata->nr_channels > DW_DMA_MAX_NR_CHANNELS) { |
| 1107 | − err = -EINVAL; | |
| 1194 | + ret = -EINVAL; | |
| 1108 | 1195 | goto err_pdata; |
| 1109 | 1196 | } else { |
| 1110 | 1197 | memcpy(dw->pdata, chip->pdata, sizeof(*dw->pdata)); |
| @@ -1116,7 +1203,7 @@ int do_dma_probe(struct dw_dma_chip *chip) · 5 unmodified lines | ||
| 1116 | 1203 | dw->chan = devm_kcalloc(chip->dev, pdata->nr_channels, sizeof(*dw->chan), |
| 1117 | 1204 | GFP_KERNEL); |
| 1118 | 1205 | if (!dw->chan) { |
| 1119 | − err = -ENOMEM; | |
| 1206 | + ret = -ENOMEM; | |
| 1120 | 1207 | goto err_pdata; |
| 1121 | 1208 | } |
| 1122 | 1209 | |
| @@ -1134,15 +1221,15 @@ int do_dma_probe(struct dw_dma_chip *chip) · 11 unmodified lines | ||
| 1134 | 1221 | sizeof(struct dw_desc), 4, 0); |
| 1135 | 1222 | if (!dw->desc_pool) { |
| 1136 | 1223 | dev_err(chip->dev, "No memory for descriptors dma pool\n"); |
| 1137 | − err = -ENOMEM; | |
| 1224 | + ret = -ENOMEM; | |
| 1138 | 1225 | goto err_pdata; |
| 1139 | 1226 | } |
| 1140 | 1227 | |
| 1141 | 1228 | tasklet_setup(&dw->tasklet, dw_dma_tasklet); |
| 1142 | 1229 | |
| 1143 | − err = request_irq(chip->irq, dw_dma_interrupt, IRQF_SHARED, | |
| 1230 | + ret = request_irq(chip->irq, dw_dma_interrupt, IRQF_SHARED, | |
| 1144 | 1231 | dw->name, dw); |
| 1145 | − if (err) | |
| 1232 | + if (ret) | |
| 1146 | 1233 | goto err_pdata; |
| 1147 | 1234 | |
| 1148 | 1235 | INIT_LIST_HEAD(&dw->dma.channels); |
| @@ -1254,8 +1341,8 @@ int do_dma_probe(struct dw_dma_chip *chip) · 105 unmodified lines | ||
| 1254 | 1341 | */ |
| 1255 | 1342 | dma_set_max_seg_size(dw->dma.dev, dw->chan[0].block_size); |
| 1256 | 1343 | |
| 1257 | − err = dma_async_device_register(&dw->dma); | |
| 1258 | − if (err) | |
| 1344 | + ret = dma_async_device_register(&dw->dma); | |
| 1345 | + if (ret) | |
| 1259 | 1346 | goto err_dma_register; |
| 1260 | 1347 | |
| 1261 | 1348 | dev_info(chip->dev, "DesignWare DMA Controller, %d channels\n", |
| @@ -1269,7 +1356,7 @@ err_dma_register: · 7 unmodified lines | ||
| 1269 | 1356 | free_irq(chip->irq, dw); |
| 1270 | 1357 | err_pdata: |
| 1271 | 1358 | pm_runtime_put_sync_suspend(chip->dev); |
| 1272 | − return err; | |
| 1359 | + return ret; | |
| 1273 | 1360 | } |
| 1274 | 1361 | |
| 1275 | 1362 | int do_dma_remove(struct dw_dma_chip *chip) |
| @@ -64,30 +64,39 @@ static size_t dw_dma_block2bytes(struct dw_dma_chan *dwc, u32 block, u32 width) · 63 unmodified lines | ||
| 64 | 64 | return DWC_CTLH_BLOCK_TS(block) << width; |
| 65 | 65 | } |
| 66 | 66 | |
| 67 | +static inline u8 dw_dma_encode_maxburst(u32 maxburst) | |
| 68 | +{ | |
| 69 | + /* | |
| 70 | + * Fix burst size according to dw_dmac. We need to convert them as: | |
| 71 | + * 1 -> 0, 4 -> 1, 8 -> 2, 16 -> 3. | |
| 72 | + */ | |
| 73 | + return maxburst > 1 ? fls(maxburst) - 2 : 0; | |
| 74 | +} | |
| 75 | + | |
| 67 | 76 | static u32 dw_dma_prepare_ctllo(struct dw_dma_chan *dwc) |
| 68 | 77 | { |
| 69 | 78 | struct dma_slave_config *sconfig = &dwc->dma_sconfig; |
| 70 | − u8 smsize = (dwc->direction == DMA_DEV_TO_MEM) ? sconfig->src_maxburst : 0; | |
| 71 | − u8 dmsize = (dwc->direction == DMA_MEM_TO_DEV) ? sconfig->dst_maxburst : 0; | |
| 72 | − u8 p_master = dwc->dws.p_master; | |
| 73 | − u8 m_master = dwc->dws.m_master; | |
| 74 | − u8 dms = (dwc->direction == DMA_MEM_TO_DEV) ? p_master : m_master; | |
| 75 | − u8 sms = (dwc->direction == DMA_DEV_TO_MEM) ? p_master : m_master; | |
| 79 | + u8 smsize = 0, dmsize = 0; | |
| 80 | + u8 sms, dms; | |
| 81 | + | |
| 82 | + if (dwc->direction == DMA_MEM_TO_DEV) { | |
| 83 | + sms = dwc->dws.m_master; | |
| 84 | + dms = dwc->dws.p_master; | |
| 85 | + dmsize = dw_dma_encode_maxburst(sconfig->dst_maxburst); | |
| 86 | + } else if (dwc->direction == DMA_DEV_TO_MEM) { | |
| 87 | + sms = dwc->dws.p_master; | |
| 88 | + dms = dwc->dws.m_master; | |
| 89 | + smsize = dw_dma_encode_maxburst(sconfig->src_maxburst); | |
| 90 | + } else /* DMA_MEM_TO_MEM */ { | |
| 91 | + sms = dwc->dws.m_master; | |
| 92 | + dms = dwc->dws.m_master; | |
| 93 | + } | |
| 76 | 94 | |
| 77 | 95 | return DWC_CTLL_LLP_D_EN | DWC_CTLL_LLP_S_EN | |
| 78 | 96 | DWC_CTLL_DST_MSIZE(dmsize) | DWC_CTLL_SRC_MSIZE(smsize) | |
| 79 | 97 | DWC_CTLL_DMS(dms) | DWC_CTLL_SMS(sms); |
| 80 | 98 | } |
| 81 | 99 | |
| 82 | −static void dw_dma_encode_maxburst(struct dw_dma_chan *dwc, u32 *maxburst) | |
| 83 | −{ | |
| 84 | − /* | |
| 85 | − * Fix burst size according to dw_dmac. We need to convert them as: | |
| 86 | − * 1 -> 0, 4 -> 1, 8 -> 2, 16 -> 3. | |
| 87 | − */ | |
| 88 | − *maxburst = *maxburst > 1 ? fls(*maxburst) - 2 : 0; | |
| 89 | −} | |
| 90 | − | |
| 91 | 100 | static void dw_dma_set_device_name(struct dw_dma *dw, int id) |
| 92 | 101 | { |
| 93 | 102 | snprintf(dw->name, sizeof(dw->name), "dw:dmac%d", id); |
| @@ -116,7 +125,6 @@ int dw_dma_probe(struct dw_dma_chip *chip) · 22 unmodified lines | ||
| 116 | 125 | dw->suspend_chan = dw_dma_suspend_chan; |
| 117 | 126 | dw->resume_chan = dw_dma_resume_chan; |
| 118 | 127 | dw->prepare_ctllo = dw_dma_prepare_ctllo; |
| 119 | − dw->encode_maxburst = dw_dma_encode_maxburst; | |
| 120 | 128 | dw->bytes2block = dw_dma_bytes2block; |
| 121 | 129 | dw->block2bytes = dw_dma_block2bytes; |
| 122 | 130 | |
| @@ -199,21 +199,25 @@ static size_t idma32_block2bytes(struct dw_dma_chan *dwc, u32 block, u32 width) · 198 unmodified lines | ||
| 199 | 199 | return IDMA32C_CTLH_BLOCK_TS(block); |
| 200 | 200 | } |
| 201 | 201 | |
| 202 | +static inline u8 idma32_encode_maxburst(u32 maxburst) | |
| 203 | +{ | |
| 204 | + return maxburst > 1 ? fls(maxburst) - 1 : 0; | |
| 205 | +} | |
| 206 | + | |
| 202 | 207 | static u32 idma32_prepare_ctllo(struct dw_dma_chan *dwc) |
| 203 | 208 | { |
| 204 | 209 | struct dma_slave_config *sconfig = &dwc->dma_sconfig; |
| 205 | − u8 smsize = (dwc->direction == DMA_DEV_TO_MEM) ? sconfig->src_maxburst : 0; | |
| 206 | − u8 dmsize = (dwc->direction == DMA_MEM_TO_DEV) ? sconfig->dst_maxburst : 0; | |
| 210 | + u8 smsize = 0, dmsize = 0; | |
| 211 | + | |
| 212 | + if (dwc->direction == DMA_MEM_TO_DEV) | |
| 213 | + dmsize = idma32_encode_maxburst(sconfig->dst_maxburst); | |
| 214 | + else if (dwc->direction == DMA_DEV_TO_MEM) | |
| 215 | + smsize = idma32_encode_maxburst(sconfig->src_maxburst); | |
| 207 | 216 | |
| 208 | 217 | return DWC_CTLL_LLP_D_EN | DWC_CTLL_LLP_S_EN | |
| 209 | 218 | DWC_CTLL_DST_MSIZE(dmsize) | DWC_CTLL_SRC_MSIZE(smsize); |
| 210 | 219 | } |
| 211 | 220 | |
| 212 | −static void idma32_encode_maxburst(struct dw_dma_chan *dwc, u32 *maxburst) | |
| 213 | −{ | |
| 214 | − *maxburst = *maxburst > 1 ? fls(*maxburst) - 1 : 0; | |
| 215 | −} | |
| 216 | − | |
| 217 | 221 | static void idma32_set_device_name(struct dw_dma *dw, int id) |
| 218 | 222 | { |
| 219 | 223 | snprintf(dw->name, sizeof(dw->name), "idma32:dmac%d", id); |
| @@ -270,7 +274,6 @@ int idma32_dma_probe(struct dw_dma_chip *chip) · 50 unmodified lines | ||
| 270 | 274 | dw->suspend_chan = idma32_suspend_chan; |
| 271 | 275 | dw->resume_chan = idma32_resume_chan; |
| 272 | 276 | dw->prepare_ctllo = idma32_prepare_ctllo; |
| 273 | − dw->encode_maxburst = idma32_encode_maxburst; | |
| 274 | 277 | dw->bytes2block = idma32_bytes2block; |
| 275 | 278 | dw->block2bytes = idma32_block2bytes; |
| 276 | 279 | |
| @@ -29,7 +29,7 @@ static int dw_probe(struct platform_device *pdev) · 28 unmodified lines | ||
| 29 | 29 | struct dw_dma_chip_pdata *data; |
| 30 | 30 | struct dw_dma_chip *chip; |
| 31 | 31 | struct device *dev = &pdev->dev; |
| 32 | − int err; | |
| 32 | + int ret; | |
| 33 | 33 | |
| 34 | 34 | match = device_get_match_data(dev); |
| 35 | 35 | if (!match) |
| @@ -51,9 +51,9 @@ static int dw_probe(struct platform_device *pdev) · 15 unmodified lines | ||
| 51 | 51 | if (IS_ERR(chip->regs)) |
| 52 | 52 | return PTR_ERR(chip->regs); |
| 53 | 53 | |
| 54 | − err = dma_coerce_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 55 | − if (err) | |
| 56 | − return err; | |
| 54 | + ret = dma_coerce_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32)); | |
| 55 | + if (ret) | |
| 56 | + return ret; | |
| 57 | 57 | |
| 58 | 58 | if (!data->pdata) |
| 59 | 59 | data->pdata = dev_get_platdata(dev); |
| @@ -69,14 +69,14 @@ static int dw_probe(struct platform_device *pdev) · 9 unmodified lines | ||
| 69 | 69 | chip->clk = devm_clk_get_optional(chip->dev, "hclk"); |
| 70 | 70 | if (IS_ERR(chip->clk)) |
| 71 | 71 | return PTR_ERR(chip->clk); |
| 72 | − err = clk_prepare_enable(chip->clk); | |
| 73 | − if (err) | |
| 74 | − return err; | |
| 72 | + ret = clk_prepare_enable(chip->clk); | |
| 73 | + if (ret) | |
| 74 | + return ret; | |
| 75 | 75 | |
| 76 | 76 | pm_runtime_enable(&pdev->dev); |
| 77 | 77 | |
| 78 | − err = data->probe(chip); | |
| 79 | − if (err) | |
| 78 | + ret = data->probe(chip); | |
| 79 | + if (ret) | |
| 80 | 80 | goto err_dw_dma_probe; |
| 81 | 81 | |
| 82 | 82 | platform_set_drvdata(pdev, data); |
| @@ -90,7 +90,7 @@ static int dw_probe(struct platform_device *pdev) · 7 unmodified lines | ||
| 90 | 90 | err_dw_dma_probe: |
| 91 | 91 | pm_runtime_disable(&pdev->dev); |
| 92 | 92 | clk_disable_unprepare(chip->clk); |
| 93 | − return err; | |
| 93 | + return ret; | |
| 94 | 94 | } |
| 95 | 95 | |
| 96 | 96 | static void dw_remove(struct platform_device *pdev) |
| @@ -327,7 +327,6 @@ struct dw_dma { · 326 unmodified lines | ||
| 327 | 327 | void (*suspend_chan)(struct dw_dma_chan *dwc, bool drain); |
| 328 | 328 | void (*resume_chan)(struct dw_dma_chan *dwc, bool drain); |
| 329 | 329 | u32 (*prepare_ctllo)(struct dw_dma_chan *dwc); |
| 330 | − void (*encode_maxburst)(struct dw_dma_chan *dwc, u32 *maxburst); | |
| 331 | 330 | u32 (*bytes2block)(struct dw_dma_chan *dwc, size_t bytes, |
| 332 | 331 | unsigned int width, size_t *len); |
| 333 | 332 | size_t (*block2bytes)(struct dw_dma_chan *dwc, u32 block, u32 width); |
| @@ -403,6 +403,7 @@ static struct stm32_dma3_swdesc *stm32_dma3_chan_desc_alloc(struct stm32_dma3_ch · 402 unmodified lines | ||
| 403 | 403 | swdesc = kzalloc(struct_size(swdesc, lli, count), GFP_NOWAIT); |
| 404 | 404 | if (!swdesc) |
| 405 | 405 | return NULL; |
| 406 | + swdesc->lli_size = count; | |
| 406 | 407 | |
| 407 | 408 | for (i = 0; i < count; i++) { |
| 408 | 409 | swdesc->lli[i].hwdesc = dma_pool_zalloc(chan->lli_pool, GFP_NOWAIT, |
| @@ -410,7 +411,6 @@ static struct stm32_dma3_swdesc *stm32_dma3_chan_desc_alloc(struct stm32_dma3_ch · 1 unmodified line | ||
| 410 | 411 | if (!swdesc->lli[i].hwdesc) |
| 411 | 412 | goto err_pool_free; |
| 412 | 413 | } |
| 413 | − swdesc->lli_size = count; | |
| 414 | 414 | swdesc->ccr = 0; |
| 415 | 415 | |
| 416 | 416 | /* Set LL base address */ |
| @@ -1186,10 +1186,10 @@ static struct dma_async_tx_descriptor *omap_dma_prep_dma_cyclic( · 1185 unmodified lines | ||
| 1186 | 1186 | d->dev_addr = dev_addr; |
| 1187 | 1187 | d->fi = burst; |
| 1188 | 1188 | d->es = es; |
| 1189 | + d->sglen = 1; | |
| 1189 | 1190 | d->sg[0].addr = buf_addr; |
| 1190 | 1191 | d->sg[0].en = period_len / es_bytes[es]; |
| 1191 | 1192 | d->sg[0].fn = buf_len / period_len; |
| 1192 | − d->sglen = 1; | |
| 1193 | 1193 | |
| 1194 | 1194 | d->ccr = c->ccr; |
| 1195 | 1195 | if (dir == DMA_DEV_TO_MEM) |
| @@ -1258,10 +1258,10 @@ static struct dma_async_tx_descriptor *omap_dma_prep_dma_memcpy( · 62 unmodified lines | ||
| 1258 | 1258 | d->dev_addr = src; |
| 1259 | 1259 | d->fi = 0; |
| 1260 | 1260 | d->es = data_type; |
| 1261 | + d->sglen = 1; | |
| 1261 | 1262 | d->sg[0].en = len / BIT(data_type); |
| 1262 | 1263 | d->sg[0].fn = 1; |
| 1263 | 1264 | d->sg[0].addr = dest; |
| 1264 | − d->sglen = 1; | |
| 1265 | 1265 | d->ccr = c->ccr; |
| 1266 | 1266 | d->ccr |= CCR_DST_AMODE_POSTINC | CCR_SRC_AMODE_POSTINC; |
| 1267 | 1267 | |
| @@ -1309,6 +1309,7 @@ static struct dma_async_tx_descriptor *omap_dma_prep_dma_interleaved( · 41 unmodified lines | ||
| 1309 | 1309 | if (data_type > CSDP_DATA_TYPE_32) |
| 1310 | 1310 | data_type = CSDP_DATA_TYPE_32; |
| 1311 | 1311 | |
| 1312 | + d->sglen = 1; | |
| 1312 | 1313 | sg = &d->sg[0]; |
| 1313 | 1314 | d->dir = DMA_MEM_TO_MEM; |
| 1314 | 1315 | d->dev_addr = xt->src_start; |
| @@ -1316,7 +1317,6 @@ static struct dma_async_tx_descriptor *omap_dma_prep_dma_interleaved( · 1 unmodified line | ||
| 1316 | 1317 | sg->en = xt->sgl[0].size / BIT(data_type); |
| 1317 | 1318 | sg->fn = xt->numf; |
| 1318 | 1319 | sg->addr = xt->dst_start; |
| 1319 | − d->sglen = 1; | |
| 1320 | 1320 | d->ccr = c->ccr; |
| 1321 | 1321 | |
| 1322 | 1322 | src_icg = dmaengine_get_src_icg(xt, &xt->sgl[0]); |
| @@ -45,7 +45,6 @@ · 44 unmodified lines | ||
| 45 | 45 | #define I10NM_NUM_CHANNELS MAX(I10NM_NUM_DDR_CHANNELS, I10NM_NUM_HBM_CHANNELS) |
| 46 | 46 | #define I10NM_NUM_DIMMS MAX(I10NM_NUM_DDR_DIMMS, I10NM_NUM_HBM_DIMMS) |
| 47 | 47 | |
| 48 | −#define MAX(a, b) ((a) > (b) ? (a) : (b)) | |
| 49 | 48 | #define NUM_IMC MAX(SKX_NUM_IMC, I10NM_NUM_IMC) |
| 50 | 49 | #define NUM_CHANNELS MAX(SKX_NUM_CHANNELS, I10NM_NUM_CHANNELS) |
| 51 | 50 | #define NUM_DIMMS MAX(SKX_NUM_DIMMS, I10NM_NUM_DIMMS) |
| @@ -27,7 +27,8 @@ cflags-$(CONFIG_ARM64) += -fpie $(DISABLE_STACKLEAK_PLUGIN) \ · 26 unmodified lines | ||
| 27 | 27 | cflags-$(CONFIG_ARM) += -DEFI_HAVE_STRLEN -DEFI_HAVE_STRNLEN \ |
| 28 | 28 | -DEFI_HAVE_MEMCHR -DEFI_HAVE_STRRCHR \ |
| 29 | 29 | -DEFI_HAVE_STRCMP -fno-builtin -fpic \ |
| 30 | − $(call cc-option,-mno-single-pic-base) | |
| 30 | + $(call cc-option,-mno-single-pic-base) \ | |
| 31 | + $(DISABLE_STACKLEAK_PLUGIN) | |
| 31 | 32 | cflags-$(CONFIG_RISCV) += -fpic -DNO_ALTERNATIVE -mno-relax \ |
| 32 | 33 | $(DISABLE_STACKLEAK_PLUGIN) |
| 33 | 34 | cflags-$(CONFIG_LOONGARCH) += -fpie |
| @@ -57,6 +58,10 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) · 23 unmodified lines | ||
| 57 | 58 | # disable LTO |
| 58 | 59 | KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS)) |
| 59 | 60 | |
| 61 | +# The .data section would be renamed to .data.efistub, therefore, remove | |
| 62 | +# `-fdata-sections` flag from KBUILD_CFLAGS_KERNEL | |
| 63 | +KBUILD_CFLAGS_KERNEL := $(filter-out -fdata-sections, $(KBUILD_CFLAGS_KERNEL)) | |
| 64 | + | |
| 60 | 65 | lib-y := efi-stub-helper.o gop.o secureboot.o tpm.o \ |
| 61 | 66 | file.o mem.o random.o randomalloc.o pci.o \ |
| 62 | 67 | skip_spaces.o lib-cmdline.o lib-ctype.o \ |
| @@ -166,7 +166,7 @@ static enum fw_upload_err mpfs_auto_update_poll_complete(struct fw_upload *fw_up · 165 unmodified lines | ||
| 166 | 166 | */ |
| 167 | 167 | ret = wait_for_completion_timeout(&priv->programming_complete, |
| 168 | 168 | msecs_to_jiffies(AUTO_UPDATE_TIMEOUT_MS)); |
| 169 | − if (ret) | |
| 169 | + if (!ret) | |
| 170 | 170 | return FW_UPLOAD_ERR_TIMEOUT; |
| 171 | 171 | |
| 172 | 172 | return FW_UPLOAD_ERR_NONE; |
| @@ -73,7 +73,7 @@ int scm_get_wq_ctx(u32 *wq_ctx, u32 *flags, u32 *more_pending) · 72 unmodified lines | ||
| 73 | 73 | struct arm_smccc_res get_wq_res; |
| 74 | 74 | struct arm_smccc_args get_wq_ctx = {0}; |
| 75 | 75 | |
| 76 | − get_wq_ctx.args[0] = ARM_SMCCC_CALL_VAL(ARM_SMCCC_STD_CALL, | |
| 76 | + get_wq_ctx.args[0] = ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, | |
| 77 | 77 | ARM_SMCCC_SMC_64, ARM_SMCCC_OWNER_SIP, |
| 78 | 78 | SCM_SMC_FNID(QCOM_SCM_SVC_WAITQ, QCOM_SCM_WAITQ_GET_WQ_CTX)); |
| 79 | 79 | |
| @@ -40,7 +40,6 @@ struct qcom_tzmem_pool { · 39 unmodified lines | ||
| 40 | 40 | }; |
| 41 | 41 | |
| 42 | 42 | struct qcom_tzmem_chunk { |
| 43 | − phys_addr_t paddr; | |
| 44 | 43 | size_t size; |
| 45 | 44 | struct qcom_tzmem_pool *owner; |
| 46 | 45 | }; |
| @@ -78,6 +77,7 @@ static bool qcom_tzmem_using_shm_bridge; · 31 unmodified lines | ||
| 78 | 77 | /* List of machines that are known to not support SHM bridge correctly. */ |
| 79 | 78 | static const char *const qcom_tzmem_blacklist[] = { |
| 80 | 79 | "qcom,sc8180x", |
| 80 | + "qcom,sdm670", /* failure in GPU firmware loading */ | |
| 81 | 81 | "qcom,sdm845", /* reset in rmtfs memory assignment */ |
| 82 | 82 | "qcom,sm8150", /* reset in rmtfs memory assignment */ |
| 83 | 83 | NULL |
| @@ -385,7 +385,6 @@ again: · 301 unmodified lines | ||
| 385 | 385 | return NULL; |
| 386 | 386 | } |
| 387 | 387 | |
| 388 | − chunk->paddr = gen_pool_virt_to_phys(pool->genpool, vaddr); | |
| 389 | 388 | chunk->size = size; |
| 390 | 389 | chunk->owner = pool; |
| 391 | 390 | |
| @@ -431,25 +430,37 @@ void qcom_tzmem_free(void *vaddr) · 39 unmodified lines | ||
| 431 | 430 | EXPORT_SYMBOL_GPL(qcom_tzmem_free); |
| 432 | 431 | |
| 433 | 432 | /** |
| 434 | − * qcom_tzmem_to_phys() - Map the virtual address of a TZ buffer to physical. | |
| 435 | − * @vaddr: Virtual address of the buffer allocated from a TZ memory pool. | |
| 433 | + * qcom_tzmem_to_phys() - Map the virtual address of TZ memory to physical. | |
| 434 | + * @vaddr: Virtual address of memory allocated from a TZ memory pool. | |
| 436 | 435 | * |
| 437 | − * Can be used in any context. The address must have been returned by a call | |
| 438 | − * to qcom_tzmem_alloc(). | |
| 436 | + * Can be used in any context. The address must point to memory allocated | |
| 437 | + * using qcom_tzmem_alloc(). | |
| 439 | 438 | * |
| 440 | − * Returns: Physical address of the buffer. | |
| 439 | + * Returns: | |
| 440 | + * Physical address mapped from the virtual or 0 if the mapping failed. | |
| 441 | 441 | */ |
| 442 | 442 | phys_addr_t qcom_tzmem_to_phys(void *vaddr) |
| 443 | 443 | { |
| 444 | 444 | struct qcom_tzmem_chunk *chunk; |
| 445 | + struct radix_tree_iter iter; | |
| 446 | + void __rcu **slot; | |
| 447 | + phys_addr_t ret; | |
| 445 | 448 | |
| 446 | 449 | guard(spinlock_irqsave)(&qcom_tzmem_chunks_lock); |
| 447 | 450 | |
| 448 | − chunk = radix_tree_lookup(&qcom_tzmem_chunks, (unsigned long)vaddr); | |
| 449 | − if (!chunk) | |
| 450 | − return 0; | |
| 451 | + radix_tree_for_each_slot(slot, &qcom_tzmem_chunks, &iter, 0) { | |
| 452 | + chunk = radix_tree_deref_slot_protected(slot, | |
| 453 | + &qcom_tzmem_chunks_lock); | |
| 451 | 454 | |
| 452 | − return chunk->paddr; | |
| 455 | + ret = gen_pool_virt_to_phys(chunk->owner->genpool, | |
| 456 | + (unsigned long)vaddr); | |
| 457 | + if (ret == -1) | |
| 458 | + continue; | |
| 459 | + | |
| 460 | + return ret; | |
| 461 | + } | |
| 462 | + | |
| 463 | + return 0; | |
| 453 | 464 | } |
| 454 | 465 | EXPORT_SYMBOL_GPL(qcom_tzmem_to_phys); |
| 455 | 466 | |
| @@ -39,6 +39,8 @@ static struct platform_device *pd; · 38 unmodified lines | ||
| 39 | 39 | static DEFINE_MUTEX(disable_lock); |
| 40 | 40 | static bool disabled; |
| 41 | 41 | |
| 42 | +static struct device *sysfb_parent_dev(const struct screen_info *si); | |
| 43 | + | |
| 42 | 44 | static bool sysfb_unregister(void) |
| 43 | 45 | { |
| 44 | 46 | if (IS_ERR_OR_NULL(pd)) |
| @@ -52,6 +54,7 @@ static bool sysfb_unregister(void) · 7 unmodified lines | ||
| 52 | 54 | |
| 53 | 55 | /** |
| 54 | 56 | * sysfb_disable() - disable the Generic System Framebuffers support |
| 57 | + * @dev: the device to check if non-NULL | |
| 55 | 58 | * |
| 56 | 59 | * This disables the registration of system framebuffer devices that match the |
| 57 | 60 | * generic drivers that make use of the system framebuffer set up by firmware. |
| @@ -61,17 +64,21 @@ static bool sysfb_unregister(void) · 3 unmodified lines | ||
| 61 | 64 | * Context: The function can sleep. A @disable_lock mutex is acquired to serialize |
| 62 | 65 | * against sysfb_init(), that registers a system framebuffer device. |
| 63 | 66 | */ |
| 64 | −void sysfb_disable(void) | |
| 67 | +void sysfb_disable(struct device *dev) | |
| 65 | 68 | { |
| 69 | + struct screen_info *si = &screen_info; | |
| 70 | + | |
| 66 | 71 | mutex_lock(&disable_lock); |
| 67 | − sysfb_unregister(); | |
| 68 | − disabled = true; | |
| 72 | + if (!dev || dev == sysfb_parent_dev(si)) { | |
| 73 | + sysfb_unregister(); | |
| 74 | + disabled = true; | |
| 75 | + } | |
| 69 | 76 | mutex_unlock(&disable_lock); |
| 70 | 77 | } |
| 71 | 78 | EXPORT_SYMBOL_GPL(sysfb_disable); |
| 72 | 79 | |
| 73 | 80 | #if defined(CONFIG_PCI) |
| 74 | −static __init bool sysfb_pci_dev_is_enabled(struct pci_dev *pdev) | |
| 81 | +static bool sysfb_pci_dev_is_enabled(struct pci_dev *pdev) | |
| 75 | 82 | { |
| 76 | 83 | /* |
| 77 | 84 | * TODO: Try to integrate this code into the PCI subsystem |
| @@ -87,13 +94,13 @@ static __init bool sysfb_pci_dev_is_enabled(struct pci_dev *pdev) · 9 unmodified lines | ||
| 87 | 94 | return true; |
| 88 | 95 | } |
| 89 | 96 | #else |
| 90 | −static __init bool sysfb_pci_dev_is_enabled(struct pci_dev *pdev) | |
| 97 | +static bool sysfb_pci_dev_is_enabled(struct pci_dev *pdev) | |
| 91 | 98 | { |
| 92 | 99 | return false; |
| 93 | 100 | } |
| 94 | 101 | #endif |
| 95 | 102 | |
| 96 | −static __init struct device *sysfb_parent_dev(const struct screen_info *si) | |
| 103 | +static struct device *sysfb_parent_dev(const struct screen_info *si) | |
| 97 | 104 | { |
| 98 | 105 | struct pci_dev *pdev; |
| 99 | 106 | |
| @@ -1444,5 +1444,6 @@ static void fsi_exit(void) · 1443 unmodified lines | ||
| 1444 | 1444 | } |
| 1445 | 1445 | module_exit(fsi_exit); |
| 1446 | 1446 | module_param(discard_errors, int, 0664); |
| 1447 | +MODULE_DESCRIPTION("FSI core driver"); | |
| 1447 | 1448 | MODULE_LICENSE("GPL"); |
| 1448 | 1449 | MODULE_PARM_DESC(discard_errors, "Don't invoke error handling on bus accesses"); |
| @@ -670,4 +670,5 @@ static struct platform_driver fsi_master_aspeed_driver = { · 669 unmodified lines | ||
| 670 | 670 | }; |
| 671 | 671 | |
| 672 | 672 | module_platform_driver(fsi_master_aspeed_driver); |
| 673 | +MODULE_DESCRIPTION("FSI master driver for AST2600"); | |
| 673 | 674 | MODULE_LICENSE("GPL"); |
| @@ -1,7 +1,7 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | // Copyright 2018 IBM Corp |
| 3 | 3 | /* |
| 4 | − * A FSI master controller, using a simple GPIO bit-banging interface | |
| 4 | + * A FSI master based on Aspeed ColdFire coprocessor | |
| 5 | 5 | */ |
| 6 | 6 | |
| 7 | 7 | #include <linux/crc4.h> |
| @@ -1438,5 +1438,6 @@ static struct platform_driver fsi_master_acf = { · 1430 unmodified lines | ||
| 1438 | 1438 | }; |
| 1439 | 1439 | |
| 1440 | 1440 | module_platform_driver(fsi_master_acf); |
| 1441 | +MODULE_DESCRIPTION("A FSI master based on Aspeed ColdFire coprocessor"); | |
| 1441 | 1442 | MODULE_LICENSE("GPL"); |
| 1442 | 1443 | MODULE_FIRMWARE(FW_FILE_NAME); |
| @@ -892,4 +892,5 @@ static struct platform_driver fsi_master_gpio_driver = { · 891 unmodified lines | ||
| 892 | 892 | }; |
| 893 | 893 | |
| 894 | 894 | module_platform_driver(fsi_master_gpio_driver); |
| 895 | +MODULE_DESCRIPTION("A FSI master controller, using a simple GPIO bit-banging interface"); | |
| 895 | 896 | MODULE_LICENSE("GPL"); |
| @@ -295,4 +295,5 @@ static struct fsi_driver hub_master_driver = { · 294 unmodified lines | ||
| 295 | 295 | }; |
| 296 | 296 | |
| 297 | 297 | module_fsi_driver(hub_master_driver); |
| 298 | +MODULE_DESCRIPTION("FSI hub master driver"); | |
| 298 | 299 | MODULE_LICENSE("GPL"); |
| @@ -625,4 +625,5 @@ static void scom_exit(void) · 624 unmodified lines | ||
| 625 | 625 | |
| 626 | 626 | module_init(scom_init); |
| 627 | 627 | module_exit(scom_exit); |
| 628 | +MODULE_DESCRIPTION("SCOM FSI Client device driver"); | |
| 628 | 629 | MODULE_LICENSE("GPL"); |
| @@ -39,6 +39,8 @@ · 38 unmodified lines | ||
| 39 | 39 | #define MLXBF_GPIO_CAUSE_OR_EVTEN0 0x14 |
| 40 | 40 | #define MLXBF_GPIO_CAUSE_OR_CLRCAUSE 0x18 |
| 41 | 41 | |
| 42 | +#define MLXBF_GPIO_CLR_ALL_INTS GENMASK(31, 0) | |
| 43 | + | |
| 42 | 44 | struct mlxbf3_gpio_context { |
| 43 | 45 | struct gpio_chip gc; |
| 44 | 46 | |
| @@ -82,6 +84,8 @@ static void mlxbf3_gpio_irq_disable(struct irq_data *irqd) · 37 unmodified lines | ||
| 82 | 84 | val = readl(gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_EVTEN0); |
| 83 | 85 | val &= ~BIT(offset); |
| 84 | 86 | writel(val, gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_EVTEN0); |
| 87 | + | |
| 88 | + writel(BIT(offset), gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_CLRCAUSE); | |
| 85 | 89 | raw_spin_unlock_irqrestore(&gs->gc.bgpio_lock, flags); |
| 86 | 90 | |
| 87 | 91 | gpiochip_disable_irq(gc, offset); |
| @@ -253,6 +257,15 @@ static int mlxbf3_gpio_probe(struct platform_device *pdev) · 165 unmodified lines | ||
| 253 | 257 | return 0; |
| 254 | 258 | } |
| 255 | 259 | |
| 260 | +static void mlxbf3_gpio_shutdown(struct platform_device *pdev) | |
| 261 | +{ | |
| 262 | + struct mlxbf3_gpio_context *gs = platform_get_drvdata(pdev); | |
| 263 | + | |
| 264 | + /* Disable and clear all interrupts */ | |
| 265 | + writel(0, gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_EVTEN0); | |
| 266 | + writel(MLXBF_GPIO_CLR_ALL_INTS, gs->gpio_cause_io + MLXBF_GPIO_CAUSE_OR_CLRCAUSE); | |
| 267 | +} | |
| 268 | + | |
| 256 | 269 | static const struct acpi_device_id mlxbf3_gpio_acpi_match[] = { |
| 257 | 270 | { "MLNXBF33", 0 }, |
| 258 | 271 | {} |
| @@ -265,6 +278,7 @@ static struct platform_driver mlxbf3_gpio_driver = { · 6 unmodified lines | ||
| 265 | 278 | .acpi_match_table = mlxbf3_gpio_acpi_match, |
| 266 | 279 | }, |
| 267 | 280 | .probe = mlxbf3_gpio_probe, |
| 281 | + .shutdown = mlxbf3_gpio_shutdown, | |
| 268 | 282 | }; |
| 269 | 283 | module_platform_driver(mlxbf3_gpio_driver); |
| 270 | 284 | |
| @@ -268,6 +268,7 @@ config DRM_EXEC · 267 unmodified lines | ||
| 268 | 268 | config DRM_GPUVM |
| 269 | 269 | tristate |
| 270 | 270 | depends on DRM |
| 271 | + select DRM_EXEC | |
| 271 | 272 | help |
| 272 | 273 | GPU-VM representation providing helpers to manage a GPUs virtual |
| 273 | 274 | address space |
| @@ -1057,6 +1057,9 @@ static int amdgpu_cs_patch_ibs(struct amdgpu_cs_parser *p, · 1056 unmodified lines | ||
| 1057 | 1057 | r = amdgpu_ring_parse_cs(ring, p, job, ib); |
| 1058 | 1058 | if (r) |
| 1059 | 1059 | return r; |
| 1060 | + | |
| 1061 | + if (ib->sa_bo) | |
| 1062 | + ib->gpu_addr = amdgpu_sa_bo_gpu_addr(ib->sa_bo); | |
| 1060 | 1063 | } else { |
| 1061 | 1064 | ib->ptr = (uint32_t *)kptr; |
| 1062 | 1065 | r = amdgpu_ring_patch_cs_in_place(ring, p, job, ib); |
| @@ -1778,7 +1781,7 @@ int amdgpu_cs_find_mapping(struct amdgpu_cs_parser *parser, · 715 unmodified lines | ||
| 1778 | 1781 | struct ttm_operation_ctx ctx = { false, false }; |
| 1779 | 1782 | struct amdgpu_vm *vm = &fpriv->vm; |
| 1780 | 1783 | struct amdgpu_bo_va_mapping *mapping; |
| 1781 | − int r; | |
| 1784 | + int i, r; | |
| 1782 | 1785 | |
| 1783 | 1786 | addr /= AMDGPU_GPU_PAGE_SIZE; |
| 1784 | 1787 | |
| @@ -1793,13 +1796,13 @@ int amdgpu_cs_find_mapping(struct amdgpu_cs_parser *parser, · 8 unmodified lines | ||
| 1793 | 1796 | if (dma_resv_locking_ctx((*bo)->tbo.base.resv) != &parser->exec.ticket) |
| 1794 | 1797 | return -EINVAL; |
| 1795 | 1798 | |
| 1796 | − if (!((*bo)->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS)) { | |
| 1797 | − (*bo)->flags |= AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS; | |
| 1798 | − amdgpu_bo_placement_from_domain(*bo, (*bo)->allowed_domains); | |
| 1799 | − r = ttm_bo_validate(&(*bo)->tbo, &(*bo)->placement, &ctx); | |
| 1800 | − if (r) | |
| 1801 | − return r; | |
| 1802 | − } | |
| 1799 | + (*bo)->flags |= AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS; | |
| 1800 | + amdgpu_bo_placement_from_domain(*bo, (*bo)->allowed_domains); | |
| 1801 | + for (i = 0; i < (*bo)->placement.num_placement; i++) | |
| 1802 | + (*bo)->placements[i].flags |= TTM_PL_FLAG_CONTIGUOUS; | |
| 1803 | + r = ttm_bo_validate(&(*bo)->tbo, &(*bo)->placement, &ctx); | |
| 1804 | + if (r) | |
| 1805 | + return r; | |
| 1803 | 1806 | |
| 1804 | 1807 | return amdgpu_ttm_alloc_gart(&(*bo)->tbo); |
| 1805 | 1808 | } |
| @@ -685,16 +685,24 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, · 684 unmodified lines | ||
| 685 | 685 | |
| 686 | 686 | switch (args->in.op) { |
| 687 | 687 | case AMDGPU_CTX_OP_ALLOC_CTX: |
| 688 | + if (args->in.flags) | |
| 689 | + return -EINVAL; | |
| 688 | 690 | r = amdgpu_ctx_alloc(adev, fpriv, filp, priority, &id); |
| 689 | 691 | args->out.alloc.ctx_id = id; |
| 690 | 692 | break; |
| 691 | 693 | case AMDGPU_CTX_OP_FREE_CTX: |
| 694 | + if (args->in.flags) | |
| 695 | + return -EINVAL; | |
| 692 | 696 | r = amdgpu_ctx_free(fpriv, id); |
| 693 | 697 | break; |
| 694 | 698 | case AMDGPU_CTX_OP_QUERY_STATE: |
| 699 | + if (args->in.flags) | |
| 700 | + return -EINVAL; | |
| 695 | 701 | r = amdgpu_ctx_query(adev, fpriv, id, &args->out); |
| 696 | 702 | break; |
| 697 | 703 | case AMDGPU_CTX_OP_QUERY_STATE2: |
| 704 | + if (args->in.flags) | |
| 705 | + return -EINVAL; | |
| 698 | 706 | r = amdgpu_ctx_query2(adev, fpriv, id, &args->out); |
| 699 | 707 | break; |
| 700 | 708 | case AMDGPU_CTX_OP_GET_STABLE_PSTATE: |
| @@ -278,7 +278,7 @@ static int amdgpu_discovery_read_binary_from_mem(struct amdgpu_device *adev, · 277 unmodified lines | ||
| 278 | 278 | msg = RREG32(mmMP0_SMN_C2PMSG_33); |
| 279 | 279 | if (msg & 0x80000000) |
| 280 | 280 | break; |
| 281 | − usleep_range(1000, 1100); | |
| 281 | + msleep(1); | |
| 282 | 282 | } |
| 283 | 283 | } |
| 284 | 284 | |
| @@ -1500,6 +1500,7 @@ union gc_info { · 1215 unmodified lines | ||
| 1500 | 1500 | struct gc_info_v1_0 v1; |
| 1501 | 1501 | struct gc_info_v1_1 v1_1; |
| 1502 | 1502 | struct gc_info_v1_2 v1_2; |
| 1503 | + struct gc_info_v1_3 v1_3; | |
| 1503 | 1504 | struct gc_info_v2_0 v2; |
| 1504 | 1505 | struct gc_info_v2_1 v2_1; |
| 1505 | 1506 | }; |
| @@ -1558,6 +1559,16 @@ static int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev) · 52 unmodified lines | ||
| 1558 | 1559 | adev->gfx.config.gc_gl1c_size_per_instance = le32_to_cpu(gc_info->v1_2.gc_gl1c_size_per_instance); |
| 1559 | 1560 | adev->gfx.config.gc_gl2c_per_gpu = le32_to_cpu(gc_info->v1_2.gc_gl2c_per_gpu); |
| 1560 | 1561 | } |
| 1562 | + if (le16_to_cpu(gc_info->v1.header.version_minor) >= 3) { | |
| 1563 | + adev->gfx.config.gc_tcp_size_per_cu = le32_to_cpu(gc_info->v1_3.gc_tcp_size_per_cu); | |
| 1564 | + adev->gfx.config.gc_tcp_cache_line_size = le32_to_cpu(gc_info->v1_3.gc_tcp_cache_line_size); | |
| 1565 | + adev->gfx.config.gc_instruction_cache_size_per_sqc = le32_to_cpu(gc_info->v1_3.gc_instruction_cache_size_per_sqc); | |
| 1566 | + adev->gfx.config.gc_instruction_cache_line_size = le32_to_cpu(gc_info->v1_3.gc_instruction_cache_line_size); | |
| 1567 | + adev->gfx.config.gc_scalar_data_cache_size_per_sqc = le32_to_cpu(gc_info->v1_3.gc_scalar_data_cache_size_per_sqc); | |
| 1568 | + adev->gfx.config.gc_scalar_data_cache_line_size = le32_to_cpu(gc_info->v1_3.gc_scalar_data_cache_line_size); | |
| 1569 | + adev->gfx.config.gc_tcc_size = le32_to_cpu(gc_info->v1_3.gc_tcc_size); | |
| 1570 | + adev->gfx.config.gc_tcc_cache_line_size = le32_to_cpu(gc_info->v1_3.gc_tcc_cache_line_size); | |
| 1571 | + } | |
| 1561 | 1572 | break; |
| 1562 | 1573 | case 2: |
| 1563 | 1574 | adev->gfx.config.max_shader_engines = le32_to_cpu(gc_info->v2.gc_num_se); |
| @@ -509,6 +509,16 @@ int amdgpu_gfx_disable_kcq(struct amdgpu_device *adev, int xcc_id) · 508 unmodified lines | ||
| 509 | 509 | int i, r = 0; |
| 510 | 510 | int j; |
| 511 | 511 | |
| 512 | + if (adev->enable_mes) { | |
| 513 | + for (i = 0; i < adev->gfx.num_compute_rings; i++) { | |
| 514 | + j = i + xcc_id * adev->gfx.num_compute_rings; | |
| 515 | + amdgpu_mes_unmap_legacy_queue(adev, | |
| 516 | + &adev->gfx.compute_ring[j], | |
| 517 | + RESET_QUEUES, 0, 0); | |
| 518 | + } | |
| 519 | + return 0; | |
| 520 | + } | |
| 521 | + | |
| 512 | 522 | if (!kiq->pmf || !kiq->pmf->kiq_unmap_queues) |
| 513 | 523 | return -EINVAL; |
| 514 | 524 | |
| @@ -551,6 +561,18 @@ int amdgpu_gfx_disable_kgq(struct amdgpu_device *adev, int xcc_id) · 36 unmodified lines | ||
| 551 | 561 | int i, r = 0; |
| 552 | 562 | int j; |
| 553 | 563 | |
| 564 | + if (adev->enable_mes) { | |
| 565 | + if (amdgpu_gfx_is_master_xcc(adev, xcc_id)) { | |
| 566 | + for (i = 0; i < adev->gfx.num_gfx_rings; i++) { | |
| 567 | + j = i + xcc_id * adev->gfx.num_gfx_rings; | |
| 568 | + amdgpu_mes_unmap_legacy_queue(adev, | |
| 569 | + &adev->gfx.gfx_ring[j], | |
| 570 | + PREEMPT_QUEUES, 0, 0); | |
| 571 | + } | |
| 572 | + } | |
| 573 | + return 0; | |
| 574 | + } | |
| 575 | + | |
| 554 | 576 | if (!kiq->pmf || !kiq->pmf->kiq_unmap_queues) |
| 555 | 577 | return -EINVAL; |
| 556 | 578 | |
| @@ -995,7 +1017,7 @@ uint32_t amdgpu_kiq_rreg(struct amdgpu_device *adev, uint32_t reg, uint32_t xcc_ · 438 unmodified lines | ||
| 995 | 1017 | if (amdgpu_device_skip_hw_access(adev)) |
| 996 | 1018 | return 0; |
| 997 | 1019 | |
| 998 | − if (adev->mes.ring.sched.ready) | |
| 1020 | + if (adev->mes.ring[0].sched.ready) | |
| 999 | 1021 | return amdgpu_mes_rreg(adev, reg); |
| 1000 | 1022 | |
| 1001 | 1023 | BUG_ON(!ring->funcs->emit_rreg); |
| @@ -1065,7 +1087,7 @@ void amdgpu_kiq_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v, uint3 · 63 unmodified lines | ||
| 1065 | 1087 | if (amdgpu_device_skip_hw_access(adev)) |
| 1066 | 1088 | return; |
| 1067 | 1089 | |
| 1068 | − if (adev->mes.ring.sched.ready) { | |
| 1090 | + if (adev->mes.ring[0].sched.ready) { | |
| 1069 | 1091 | amdgpu_mes_wreg(adev, reg, v); |
| 1070 | 1092 | return; |
| 1071 | 1093 | } |
| @@ -240,6 +240,12 @@ struct amdgpu_gfx_config { · 239 unmodified lines | ||
| 240 | 240 | uint32_t gc_tcp_size_per_cu; |
| 241 | 241 | uint32_t gc_num_cu_per_sqc; |
| 242 | 242 | uint32_t gc_tcc_size; |
| 243 | + uint32_t gc_tcp_cache_line_size; | |
| 244 | + uint32_t gc_instruction_cache_size_per_sqc; | |
| 245 | + uint32_t gc_instruction_cache_line_size; | |
| 246 | + uint32_t gc_scalar_data_cache_size_per_sqc; | |
| 247 | + uint32_t gc_scalar_data_cache_line_size; | |
| 248 | + uint32_t gc_tcc_cache_line_size; | |
| 243 | 249 | }; |
| 244 | 250 | |
| 245 | 251 | struct amdgpu_cu_info { |
| @@ -589,7 +589,8 @@ int amdgpu_gmc_allocate_vm_inv_eng(struct amdgpu_device *adev) · 588 unmodified lines | ||
| 589 | 589 | ring = adev->rings[i]; |
| 590 | 590 | vmhub = ring->vm_hub; |
| 591 | 591 | |
| 592 | − if (ring == &adev->mes.ring || | |
| 592 | + if (ring == &adev->mes.ring[0] || | |
| 593 | + ring == &adev->mes.ring[1] || | |
| 593 | 594 | ring == &adev->umsch_mm.ring) |
| 594 | 595 | continue; |
| 595 | 596 | |
| @@ -761,7 +762,7 @@ void amdgpu_gmc_fw_reg_write_reg_wait(struct amdgpu_device *adev, · 165 unmodified lines | ||
| 761 | 762 | unsigned long flags; |
| 762 | 763 | uint32_t seq; |
| 763 | 764 | |
| 764 | − if (adev->mes.ring.sched.ready) { | |
| 765 | + if (adev->mes.ring[0].sched.ready) { | |
| 765 | 766 | amdgpu_mes_reg_write_reg_wait(adev, reg0, reg1, |
| 766 | 767 | ref, mask); |
| 767 | 768 | return; |
| @@ -156,6 +156,8 @@ struct amdgpu_gmc_funcs { · 155 unmodified lines | ||
| 156 | 156 | uint64_t addr, uint64_t *flags); |
| 157 | 157 | /* get the amount of memory used by the vbios for pre-OS console */ |
| 158 | 158 | unsigned int (*get_vbios_fb_size)(struct amdgpu_device *adev); |
| 159 | + /* get the DCC buffer alignment */ | |
| 160 | + unsigned int (*get_dcc_alignment)(struct amdgpu_device *adev); | |
| 159 | 161 | |
| 160 | 162 | enum amdgpu_memory_partition (*query_mem_partition_mode)( |
| 161 | 163 | struct amdgpu_device *adev); |
| @@ -363,6 +365,10 @@ struct amdgpu_gmc { · 201 unmodified lines | ||
| 363 | 365 | (adev)->gmc.gmc_funcs->override_vm_pte_flags \ |
| 364 | 366 | ((adev), (vm), (addr), (pte_flags)) |
| 365 | 367 | #define amdgpu_gmc_get_vbios_fb_size(adev) (adev)->gmc.gmc_funcs->get_vbios_fb_size((adev)) |
| 368 | +#define amdgpu_gmc_get_dcc_alignment(adev) ({ \ | |
| 369 | + typeof(adev) _adev = (adev); \ | |
| 370 | + _adev->gmc.gmc_funcs->get_dcc_alignment(_adev); \ | |
| 371 | +}) | |
| 366 | 372 | |
| 367 | 373 | /** |
| 368 | 374 | * amdgpu_gmc_vram_full_visible - Check if full VRAM is visible through the BAR |
| @@ -264,9 +264,8 @@ amdgpu_job_prepare_job(struct drm_sched_job *sched_job, · 263 unmodified lines | ||
| 264 | 264 | struct dma_fence *fence = NULL; |
| 265 | 265 | int r; |
| 266 | 266 | |
| 267 | − /* Ignore soft recovered fences here */ | |
| 268 | 267 | r = drm_sched_entity_error(s_entity); |
| 269 | − if (r && r != -ENODATA) | |
| 268 | + if (r) | |
| 270 | 269 | goto error; |
| 271 | 270 | |
| 272 | 271 | if (!fence && job->gang_submit) |
| @@ -103,7 +103,7 @@ static int amdgpu_mes_event_log_init(struct amdgpu_device *adev) · 102 unmodified lines | ||
| 103 | 103 | if (!amdgpu_mes_log_enable) |
| 104 | 104 | return 0; |
| 105 | 105 | |
| 106 | − r = amdgpu_bo_create_kernel(adev, AMDGPU_MES_LOG_BUFFER_SIZE, PAGE_SIZE, | |
| 106 | + r = amdgpu_bo_create_kernel(adev, adev->mes.event_log_size, PAGE_SIZE, | |
| 107 | 107 | AMDGPU_GEM_DOMAIN_GTT, |
| 108 | 108 | &adev->mes.event_log_gpu_obj, |
| 109 | 109 | &adev->mes.event_log_gpu_addr, |
| @@ -113,7 +113,7 @@ static int amdgpu_mes_event_log_init(struct amdgpu_device *adev) · 3 unmodified lines | ||
| 113 | 113 | return r; |
| 114 | 114 | } |
| 115 | 115 | |
| 116 | − memset(adev->mes.event_log_cpu_addr, 0, PAGE_SIZE); | |
| 116 | + memset(adev->mes.event_log_cpu_addr, 0, adev->mes.event_log_size); | |
| 117 | 117 | |
| 118 | 118 | return 0; |
| 119 | 119 | |
| @@ -135,9 +135,11 @@ int amdgpu_mes_init(struct amdgpu_device *adev) · 15 unmodified lines | ||
| 135 | 135 | idr_init(&adev->mes.queue_id_idr); |
| 136 | 136 | ida_init(&adev->mes.doorbell_ida); |
| 137 | 137 | spin_lock_init(&adev->mes.queue_id_lock); |
| 138 | − spin_lock_init(&adev->mes.ring_lock); | |
| 139 | 138 | mutex_init(&adev->mes.mutex_hidden); |
| 140 | 139 | |
| 140 | + for (i = 0; i < AMDGPU_MAX_MES_PIPES; i++) | |
| 141 | + spin_lock_init(&adev->mes.ring_lock[i]); | |
| 142 | + | |
| 141 | 143 | adev->mes.total_max_queue = AMDGPU_FENCE_MES_QUEUE_ID_MASK; |
| 142 | 144 | adev->mes.vmid_mask_mmhub = 0xffffff00; |
| 143 | 145 | adev->mes.vmid_mask_gfxhub = 0xffffff00; |
| @@ -163,36 +165,38 @@ int amdgpu_mes_init(struct amdgpu_device *adev) · 19 unmodified lines | ||
| 163 | 165 | adev->mes.sdma_hqd_mask[i] = 0xfc; |
| 164 | 166 | } |
| 165 | 167 | |
| 166 | − r = amdgpu_device_wb_get(adev, &adev->mes.sch_ctx_offs); | |
| 167 | − if (r) { | |
| 168 | − dev_err(adev->dev, | |
| 169 | − "(%d) ring trail_fence_offs wb alloc failed\n", r); | |
| 170 | − goto error_ids; | |
| 171 | − } | |
| 172 | − adev->mes.sch_ctx_gpu_addr = | |
| 173 | − adev->wb.gpu_addr + (adev->mes.sch_ctx_offs * 4); | |
| 174 | − adev->mes.sch_ctx_ptr = | |
| 175 | − (uint64_t *)&adev->wb.wb[adev->mes.sch_ctx_offs]; | |
| 168 | + for (i = 0; i < AMDGPU_MAX_MES_PIPES; i++) { | |
| 169 | + r = amdgpu_device_wb_get(adev, &adev->mes.sch_ctx_offs[i]); | |
| 170 | + if (r) { | |
| 171 | + dev_err(adev->dev, | |
| 172 | + "(%d) ring trail_fence_offs wb alloc failed\n", | |
| 173 | + r); | |
| 174 | + goto error; | |
| 175 | + } | |
| 176 | + adev->mes.sch_ctx_gpu_addr[i] = | |
| 177 | + adev->wb.gpu_addr + (adev->mes.sch_ctx_offs[i] * 4); | |
| 178 | + adev->mes.sch_ctx_ptr[i] = | |
| 179 | + (uint64_t *)&adev->wb.wb[adev->mes.sch_ctx_offs[i]]; | |
| 176 | 180 | |
| 177 | − r = amdgpu_device_wb_get(adev, &adev->mes.query_status_fence_offs); | |
| 178 | − if (r) { | |
| 179 | − amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs); | |
| 180 | − dev_err(adev->dev, | |
| 181 | − "(%d) query_status_fence_offs wb alloc failed\n", r); | |
| 182 | − goto error_ids; | |
| 181 | + r = amdgpu_device_wb_get(adev, | |
| 182 | + &adev->mes.query_status_fence_offs[i]); | |
| 183 | + if (r) { | |
| 184 | + dev_err(adev->dev, | |
| 185 | + "(%d) query_status_fence_offs wb alloc failed\n", | |
| 186 | + r); | |
| 187 | + goto error; | |
| 188 | + } | |
| 189 | + adev->mes.query_status_fence_gpu_addr[i] = adev->wb.gpu_addr + | |
| 190 | + (adev->mes.query_status_fence_offs[i] * 4); | |
| 191 | + adev->mes.query_status_fence_ptr[i] = | |
| 192 | + (uint64_t *)&adev->wb.wb[adev->mes.query_status_fence_offs[i]]; | |
| 183 | 193 | } |
| 184 | − adev->mes.query_status_fence_gpu_addr = | |
| 185 | − adev->wb.gpu_addr + (adev->mes.query_status_fence_offs * 4); | |
| 186 | − adev->mes.query_status_fence_ptr = | |
| 187 | − (uint64_t *)&adev->wb.wb[adev->mes.query_status_fence_offs]; | |
| 188 | 194 | |
| 189 | 195 | r = amdgpu_device_wb_get(adev, &adev->mes.read_val_offs); |
| 190 | 196 | if (r) { |
| 191 | − amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs); | |
| 192 | − amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs); | |
| 193 | 197 | dev_err(adev->dev, |
| 194 | 198 | "(%d) read_val_offs alloc failed\n", r); |
| 195 | − goto error_ids; | |
| 199 | + goto error; | |
| 196 | 200 | } |
| 197 | 201 | adev->mes.read_val_gpu_addr = |
| 198 | 202 | adev->wb.gpu_addr + (adev->mes.read_val_offs * 4); |
| @@ -212,10 +216,16 @@ int amdgpu_mes_init(struct amdgpu_device *adev) · 13 unmodified lines | ||
| 212 | 216 | error_doorbell: |
| 213 | 217 | amdgpu_mes_doorbell_free(adev); |
| 214 | 218 | error: |
| 215 | − amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs); | |
| 216 | − amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs); | |
| 217 | − amdgpu_device_wb_free(adev, adev->mes.read_val_offs); | |
| 218 | −error_ids: | |
| 219 | + for (i = 0; i < AMDGPU_MAX_MES_PIPES; i++) { | |
| 220 | + if (adev->mes.sch_ctx_ptr[i]) | |
| 221 | + amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs[i]); | |
| 222 | + if (adev->mes.query_status_fence_ptr[i]) | |
| 223 | + amdgpu_device_wb_free(adev, | |
| 224 | + adev->mes.query_status_fence_offs[i]); | |
| 225 | + } | |
| 226 | + if (adev->mes.read_val_ptr) | |
| 227 | + amdgpu_device_wb_free(adev, adev->mes.read_val_offs); | |
| 228 | + | |
| 219 | 229 | idr_destroy(&adev->mes.pasid_idr); |
| 220 | 230 | idr_destroy(&adev->mes.gang_id_idr); |
| 221 | 231 | idr_destroy(&adev->mes.queue_id_idr); |
| @@ -226,13 +236,22 @@ error_ids: · 4 unmodified lines | ||
| 226 | 236 | |
| 227 | 237 | void amdgpu_mes_fini(struct amdgpu_device *adev) |
| 228 | 238 | { |
| 239 | + int i; | |
| 240 | + | |
| 229 | 241 | amdgpu_bo_free_kernel(&adev->mes.event_log_gpu_obj, |
| 230 | 242 | &adev->mes.event_log_gpu_addr, |
| 231 | 243 | &adev->mes.event_log_cpu_addr); |
| 232 | 244 | |
| 233 | − amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs); | |
| 234 | − amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs); | |
| 235 | − amdgpu_device_wb_free(adev, adev->mes.read_val_offs); | |
| 245 | + for (i = 0; i < AMDGPU_MAX_MES_PIPES; i++) { | |
| 246 | + if (adev->mes.sch_ctx_ptr[i]) | |
| 247 | + amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs[i]); | |
| 248 | + if (adev->mes.query_status_fence_ptr[i]) | |
| 249 | + amdgpu_device_wb_free(adev, | |
| 250 | + adev->mes.query_status_fence_offs[i]); | |
| 251 | + } | |
| 252 | + if (adev->mes.read_val_ptr) | |
| 253 | + amdgpu_device_wb_free(adev, adev->mes.read_val_offs); | |
| 254 | + | |
| 236 | 255 | amdgpu_mes_doorbell_free(adev); |
| 237 | 256 | |
| 238 | 257 | idr_destroy(&adev->mes.pasid_idr); |
| @@ -1499,7 +1518,7 @@ int amdgpu_mes_init_microcode(struct amdgpu_device *adev, int pipe) · 1260 unmodified lines | ||
| 1499 | 1518 | |
| 1500 | 1519 | amdgpu_ucode_ip_version_decode(adev, GC_HWIP, ucode_prefix, |
| 1501 | 1520 | sizeof(ucode_prefix)); |
| 1502 | − if (adev->enable_uni_mes && pipe == AMDGPU_MES_SCHED_PIPE) { | |
| 1521 | + if (adev->enable_uni_mes) { | |
| 1503 | 1522 | snprintf(fw_name, sizeof(fw_name), |
| 1504 | 1523 | "amdgpu/%s_uni_mes.bin", ucode_prefix); |
| 1505 | 1524 | } else if (amdgpu_ip_version(adev, GC_HWIP, 0) >= IP_VERSION(11, 0, 0) && |
| @@ -1573,7 +1592,7 @@ static int amdgpu_debugfs_mes_event_log_show(struct seq_file *m, void *unused) · 67 unmodified lines | ||
| 1573 | 1592 | uint32_t *mem = (uint32_t *)(adev->mes.event_log_cpu_addr); |
| 1574 | 1593 | |
| 1575 | 1594 | seq_hex_dump(m, "", DUMP_PREFIX_OFFSET, 32, 4, |
| 1576 | − mem, AMDGPU_MES_LOG_BUFFER_SIZE, false); | |
| 1595 | + mem, adev->mes.event_log_size, false); | |
| 1577 | 1596 | |
| 1578 | 1597 | return 0; |
| 1579 | 1598 | } |
| @@ -52,7 +52,6 @@ enum amdgpu_mes_priority_level { · 51 unmodified lines | ||
| 52 | 52 | |
| 53 | 53 | #define AMDGPU_MES_PROC_CTX_SIZE 0x1000 /* one page area */ |
| 54 | 54 | #define AMDGPU_MES_GANG_CTX_SIZE 0x1000 /* one page area */ |
| 55 | −#define AMDGPU_MES_LOG_BUFFER_SIZE 0x4000 /* Maximu log buffer size for MES */ | |
| 56 | 55 | |
| 57 | 56 | struct amdgpu_mes_funcs; |
| 58 | 57 | |
| @@ -83,8 +82,8 @@ struct amdgpu_mes { · 24 unmodified lines | ||
| 83 | 82 | uint64_t default_process_quantum; |
| 84 | 83 | uint64_t default_gang_quantum; |
| 85 | 84 | |
| 86 | − struct amdgpu_ring ring; | |
| 87 | − spinlock_t ring_lock; | |
| 85 | + struct amdgpu_ring ring[AMDGPU_MAX_MES_PIPES]; | |
| 86 | + spinlock_t ring_lock[AMDGPU_MAX_MES_PIPES]; | |
| 88 | 87 | |
| 89 | 88 | const struct firmware *fw[AMDGPU_MAX_MES_PIPES]; |
| 90 | 89 | |
| @@ -113,12 +112,12 @@ struct amdgpu_mes { · 22 unmodified lines | ||
| 113 | 112 | uint32_t gfx_hqd_mask[AMDGPU_MES_MAX_GFX_PIPES]; |
| 114 | 113 | uint32_t sdma_hqd_mask[AMDGPU_MES_MAX_SDMA_PIPES]; |
| 115 | 114 | uint32_t aggregated_doorbells[AMDGPU_MES_PRIORITY_NUM_LEVELS]; |
| 116 | − uint32_t sch_ctx_offs; | |
| 117 | − uint64_t sch_ctx_gpu_addr; | |
| 118 | − uint64_t *sch_ctx_ptr; | |
| 119 | − uint32_t query_status_fence_offs; | |
| 120 | − uint64_t query_status_fence_gpu_addr; | |
| 121 | − uint64_t *query_status_fence_ptr; | |
| 115 | + uint32_t sch_ctx_offs[AMDGPU_MAX_MES_PIPES]; | |
| 116 | + uint64_t sch_ctx_gpu_addr[AMDGPU_MAX_MES_PIPES]; | |
| 117 | + uint64_t *sch_ctx_ptr[AMDGPU_MAX_MES_PIPES]; | |
| 118 | + uint32_t query_status_fence_offs[AMDGPU_MAX_MES_PIPES]; | |
| 119 | + uint64_t query_status_fence_gpu_addr[AMDGPU_MAX_MES_PIPES]; | |
| 120 | + uint64_t *query_status_fence_ptr[AMDGPU_MAX_MES_PIPES]; | |
| 122 | 121 | uint32_t read_val_offs; |
| 123 | 122 | uint64_t read_val_gpu_addr; |
| 124 | 123 | uint32_t *read_val_ptr; |
| @@ -135,8 +134,9 @@ struct amdgpu_mes { · 10 unmodified lines | ||
| 135 | 134 | unsigned long *doorbell_bitmap; |
| 136 | 135 | |
| 137 | 136 | /* MES event log buffer */ |
| 138 | − struct amdgpu_bo *event_log_gpu_obj; | |
| 139 | − uint64_t event_log_gpu_addr; | |
| 137 | + uint32_t event_log_size; | |
| 138 | + struct amdgpu_bo *event_log_gpu_obj; | |
| 139 | + uint64_t event_log_gpu_addr; | |
| 140 | 140 | void *event_log_cpu_addr; |
| 141 | 141 | |
| 142 | 142 | /* ip specific functions */ |
| @@ -166,6 +166,9 @@ static ssize_t ta_if_load_debugfs_write(struct file *fp, const char *buf, size_t · 165 unmodified lines | ||
| 166 | 166 | if (ret) |
| 167 | 167 | return -EFAULT; |
| 168 | 168 | |
| 169 | + if (ta_bin_len > PSP_1_MEG) | |
| 170 | + return -EINVAL; | |
| 171 | + | |
| 169 | 172 | copy_pos += sizeof(uint32_t); |
| 170 | 173 | |
| 171 | 174 | ta_bin = kzalloc(ta_bin_len, GFP_KERNEL); |
| @@ -212,6 +212,8 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, · 211 unmodified lines | ||
| 212 | 212 | */ |
| 213 | 213 | if (ring->funcs->type == AMDGPU_RING_TYPE_KIQ) |
| 214 | 214 | sched_hw_submission = max(sched_hw_submission, 256); |
| 215 | + if (ring->funcs->type == AMDGPU_RING_TYPE_MES) | |
| 216 | + sched_hw_submission = 8; | |
| 215 | 217 | else if (ring == &adev->sdma.instance[0].page) |
| 216 | 218 | sched_hw_submission = 256; |
| 217 | 219 | |
| @@ -461,8 +461,11 @@ struct amdgpu_vcn5_fw_shared { · 460 unmodified lines | ||
| 461 | 461 | struct amdgpu_fw_shared_unified_queue_struct sq; |
| 462 | 462 | uint8_t pad1[8]; |
| 463 | 463 | struct amdgpu_fw_shared_fw_logging fw_log; |
| 464 | + uint8_t pad2[20]; | |
| 464 | 465 | struct amdgpu_fw_shared_rb_setup rb_setup; |
| 465 | − uint8_t pad2[4]; | |
| 466 | + struct amdgpu_fw_shared_smu_interface_info smu_dpm_interface; | |
| 467 | + struct amdgpu_fw_shared_drm_key_wa drm_key_wa; | |
| 468 | + uint8_t pad3[9]; | |
| 466 | 469 | }; |
| 467 | 470 | |
| 468 | 471 | #define VCN_BLOCK_ENCODE_DISABLE_MASK 0x80 |
| @@ -858,7 +858,7 @@ void amdgpu_virt_post_reset(struct amdgpu_device *adev) · 857 unmodified lines | ||
| 858 | 858 | adev->gfx.is_poweron = false; |
| 859 | 859 | } |
| 860 | 860 | |
| 861 | − adev->mes.ring.sched.ready = false; | |
| 861 | + adev->mes.ring[0].sched.ready = false; | |
| 862 | 862 | } |
| 863 | 863 | |
| 864 | 864 | bool amdgpu_virt_fw_load_skip_check(struct amdgpu_device *adev, uint32_t ucode_id) |
| @@ -456,6 +456,7 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, · 455 unmodified lines | ||
| 456 | 456 | u64 vis_usage = 0, max_bytes, min_block_size; |
| 457 | 457 | struct amdgpu_vram_mgr_resource *vres; |
| 458 | 458 | u64 size, remaining_size, lpfn, fpfn; |
| 459 | + unsigned int adjust_dcc_size = 0; | |
| 459 | 460 | struct drm_buddy *mm = &mgr->mm; |
| 460 | 461 | struct drm_buddy_block *block; |
| 461 | 462 | unsigned long pages_per_block; |
| @@ -511,7 +512,19 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, · 49 unmodified lines | ||
| 511 | 512 | /* Allocate blocks in desired range */ |
| 512 | 513 | vres->flags |= DRM_BUDDY_RANGE_ALLOCATION; |
| 513 | 514 | |
| 515 | + if (bo->flags & AMDGPU_GEM_CREATE_GFX12_DCC && | |
| 516 | + adev->gmc.gmc_funcs->get_dcc_alignment) | |
| 517 | + adjust_dcc_size = amdgpu_gmc_get_dcc_alignment(adev); | |
| 518 | + | |
| 514 | 519 | remaining_size = (u64)vres->base.size; |
| 520 | + if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS && adjust_dcc_size) { | |
| 521 | + unsigned int dcc_size; | |
| 522 | + | |
| 523 | + dcc_size = roundup_pow_of_two(vres->base.size + adjust_dcc_size); | |
| 524 | + remaining_size = (u64)dcc_size; | |
| 525 | + | |
| 526 | + vres->flags |= DRM_BUDDY_TRIM_DISABLE; | |
| 527 | + } | |
| 515 | 528 | |
| 516 | 529 | mutex_lock(&mgr->lock); |
| 517 | 530 | while (remaining_size) { |
| @@ -521,8 +534,11 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, · 3 unmodified lines | ||
| 521 | 534 | min_block_size = mgr->default_page_size; |
| 522 | 535 | |
| 523 | 536 | size = remaining_size; |
| 524 | − if ((size >= (u64)pages_per_block << PAGE_SHIFT) && | |
| 525 | − !(size & (((u64)pages_per_block << PAGE_SHIFT) - 1))) | |
| 537 | + | |
| 538 | + if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS && adjust_dcc_size) | |
| 539 | + min_block_size = size; | |
| 540 | + else if ((size >= (u64)pages_per_block << PAGE_SHIFT) && | |
| 541 | + !(size & (((u64)pages_per_block << PAGE_SHIFT) - 1))) | |
| 526 | 542 | min_block_size = (u64)pages_per_block << PAGE_SHIFT; |
| 527 | 543 | |
| 528 | 544 | BUG_ON(min_block_size < mm->chunk_size); |
| @@ -553,6 +569,22 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, · 24 unmodified lines | ||
| 553 | 569 | } |
| 554 | 570 | mutex_unlock(&mgr->lock); |
| 555 | 571 | |
| 572 | + if (bo->flags & AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS && adjust_dcc_size) { | |
| 573 | + struct drm_buddy_block *dcc_block; | |
| 574 | + unsigned long dcc_start; | |
| 575 | + u64 trim_start; | |
| 576 | + | |
| 577 | + dcc_block = amdgpu_vram_mgr_first_block(&vres->blocks); | |
| 578 | + /* Adjust the start address for DCC buffers only */ | |
| 579 | + dcc_start = | |
| 580 | + roundup((unsigned long)amdgpu_vram_mgr_block_start(dcc_block), | |
| 581 | + adjust_dcc_size); | |
| 582 | + trim_start = (u64)dcc_start; | |
| 583 | + drm_buddy_block_trim(mm, &trim_start, | |
| 584 | + (u64)vres->base.size, | |
| 585 | + &vres->blocks); | |
| 586 | + } | |
| 587 | + | |
| 556 | 588 | vres->base.start = 0; |
| 557 | 589 | size = max_t(u64, amdgpu_vram_mgr_blocks_size(&vres->blocks), |
| 558 | 590 | vres->base.size); |
| @@ -4116,6 +4116,7 @@ static void gfx_v10_0_check_gfxoff_flag(struct amdgpu_device *adev) · 4115 unmodified lines | ||
| 4116 | 4116 | |
| 4117 | 4117 | static int gfx_v10_0_init_microcode(struct amdgpu_device *adev) |
| 4118 | 4118 | { |
| 4119 | + char fw_name[53]; | |
| 4119 | 4120 | char ucode_prefix[30]; |
| 4120 | 4121 | const char *wks = ""; |
| 4121 | 4122 | int err; |
| @@ -4149,8 +4150,8 @@ static int gfx_v10_0_init_microcode(struct amdgpu_device *adev) · 27 unmodified lines | ||
| 4149 | 4150 | amdgpu_gfx_cp_init_microcode(adev, AMDGPU_UCODE_ID_CP_CE); |
| 4150 | 4151 | |
| 4151 | 4152 | if (!amdgpu_sriov_vf(adev)) { |
| 4152 | − err = amdgpu_ucode_request(adev, &adev->gfx.rlc_fw, | |
| 4153 | − "amdgpu/%s_rlc.bin", ucode_prefix); | |
| 4153 | + snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_rlc.bin", ucode_prefix); | |
| 4154 | + err = request_firmware(&adev->gfx.rlc_fw, fw_name, adev->dev); | |
| 4154 | 4155 | if (err) |
| 4155 | 4156 | goto out; |
| 4156 | 4157 | |
| @@ -202,6 +202,12 @@ static const struct amdgpu_hwip_reg_entry gc_gfx_queue_reg_list_12[] = { · 201 unmodified lines | ||
| 202 | 202 | SOC15_REG_ENTRY_STR(GC, 0, regCP_IB1_BUFSZ) |
| 203 | 203 | }; |
| 204 | 204 | |
| 205 | +static const struct soc15_reg_golden golden_settings_gc_12_0[] = { | |
| 206 | + SOC15_REG_GOLDEN_VALUE(GC, 0, regDB_MEM_CONFIG, 0x0000000f, 0x0000000f), | |
| 207 | + SOC15_REG_GOLDEN_VALUE(GC, 0, regCB_HW_CONTROL_1, 0x03000000, 0x03000000), | |
| 208 | + SOC15_REG_GOLDEN_VALUE(GC, 0, regGL2C_CTRL5, 0x00000070, 0x00000020) | |
| 209 | +}; | |
| 210 | + | |
| 205 | 211 | #define DEFAULT_SH_MEM_CONFIG \ |
| 206 | 212 | ((SH_MEM_ADDRESS_MODE_64 << SH_MEM_CONFIG__ADDRESS_MODE__SHIFT) | \ |
| 207 | 213 | (SH_MEM_ALIGNMENT_MODE_UNALIGNED << SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT) | \ |
| @@ -2999,7 +3005,7 @@ static int gfx_v12_0_compute_mqd_init(struct amdgpu_device *adev, void *m, · 2791 unmodified lines | ||
| 2999 | 3005 | (order_base_2(prop->queue_size / 4) - 1)); |
| 3000 | 3006 | tmp = REG_SET_FIELD(tmp, CP_HQD_PQ_CONTROL, RPTR_BLOCK_SIZE, |
| 3001 | 3007 | (order_base_2(AMDGPU_GPU_PAGE_SIZE / 4) - 1)); |
| 3002 | − tmp = REG_SET_FIELD(tmp, CP_HQD_PQ_CONTROL, UNORD_DISPATCH, 0); | |
| 3008 | + tmp = REG_SET_FIELD(tmp, CP_HQD_PQ_CONTROL, UNORD_DISPATCH, 1); | |
| 3003 | 3009 | tmp = REG_SET_FIELD(tmp, CP_HQD_PQ_CONTROL, TUNNEL_DISPATCH, 0); |
| 3004 | 3010 | tmp = REG_SET_FIELD(tmp, CP_HQD_PQ_CONTROL, PRIV_STATE, 1); |
| 3005 | 3011 | tmp = REG_SET_FIELD(tmp, CP_HQD_PQ_CONTROL, KMD_QUEUE, 1); |
| @@ -3432,6 +3438,24 @@ static void gfx_v12_0_disable_gpa_mode(struct amdgpu_device *adev) · 426 unmodified lines | ||
| 3432 | 3438 | WREG32_SOC15(GC, 0, regCPG_PSP_DEBUG, data); |
| 3433 | 3439 | } |
| 3434 | 3440 | |
| 3441 | +static void gfx_v12_0_init_golden_registers(struct amdgpu_device *adev) | |
| 3442 | +{ | |
| 3443 | + if (amdgpu_sriov_vf(adev)) | |
| 3444 | + return; | |
| 3445 | + | |
| 3446 | + switch (amdgpu_ip_version(adev, GC_HWIP, 0)) { | |
| 3447 | + case IP_VERSION(12, 0, 0): | |
| 3448 | + case IP_VERSION(12, 0, 1): | |
| 3449 | + if (adev->rev_id == 0) | |
| 3450 | + soc15_program_register_sequence(adev, | |
| 3451 | + golden_settings_gc_12_0, | |
| 3452 | + (const u32)ARRAY_SIZE(golden_settings_gc_12_0)); | |
| 3453 | + break; | |
| 3454 | + default: | |
| 3455 | + break; | |
| 3456 | + } | |
| 3457 | +} | |
| 3458 | + | |
| 3435 | 3459 | static int gfx_v12_0_hw_init(void *handle) |
| 3436 | 3460 | { |
| 3437 | 3461 | int r; |
| @@ -3472,6 +3496,9 @@ static int gfx_v12_0_hw_init(void *handle) · 34 unmodified lines | ||
| 3472 | 3496 | } |
| 3473 | 3497 | } |
| 3474 | 3498 | |
| 3499 | + if (!amdgpu_emu_mode) | |
| 3500 | + gfx_v12_0_init_golden_registers(adev); | |
| 3501 | + | |
| 3475 | 3502 | adev->gfx.is_poweron = true; |
| 3476 | 3503 | |
| 3477 | 3504 | if (get_gb_addr_config(adev)) |
| @@ -3519,33 +3546,9 @@ static int gfx_v12_0_hw_init(void *handle) · 41 unmodified lines | ||
| 3519 | 3546 | return r; |
| 3520 | 3547 | } |
| 3521 | 3548 | |
| 3522 | −static int gfx_v12_0_kiq_disable_kgq(struct amdgpu_device *adev) | |
| 3523 | −{ | |
| 3524 | − struct amdgpu_kiq *kiq = &adev->gfx.kiq[0]; | |
| 3525 | − struct amdgpu_ring *kiq_ring = &kiq->ring; | |
| 3526 | − int i, r = 0; | |
| 3527 | − | |
| 3528 | − if (!kiq->pmf || !kiq->pmf->kiq_unmap_queues) | |
| 3529 | − return -EINVAL; | |
| 3530 | − | |
| 3531 | − if (amdgpu_ring_alloc(kiq_ring, kiq->pmf->unmap_queues_size * | |
| 3532 | − adev->gfx.num_gfx_rings)) | |
| 3533 | − return -ENOMEM; | |
| 3534 | − | |
| 3535 | − for (i = 0; i < adev->gfx.num_gfx_rings; i++) | |
| 3536 | − kiq->pmf->kiq_unmap_queues(kiq_ring, &adev->gfx.gfx_ring[i], | |
| 3537 | − PREEMPT_QUEUES, 0, 0); | |
| 3538 | − | |
| 3539 | − if (adev->gfx.kiq[0].ring.sched.ready) | |
| 3540 | − r = amdgpu_ring_test_helper(kiq_ring); | |
| 3541 | − | |
| 3542 | − return r; | |
| 3543 | −} | |
| 3544 | − | |
| 3545 | 3549 | static int gfx_v12_0_hw_fini(void *handle) |
| 3546 | 3550 | { |
| 3547 | 3551 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 3548 | − int r; | |
| 3549 | 3552 | uint32_t tmp; |
| 3550 | 3553 | |
| 3551 | 3554 | amdgpu_irq_put(adev, &adev->gfx.priv_reg_irq, 0); |
| @@ -3553,8 +3556,7 @@ static int gfx_v12_0_hw_fini(void *handle) · 1 unmodified line | ||
| 3553 | 3556 | |
| 3554 | 3557 | if (!adev->no_hw_access) { |
| 3555 | 3558 | if (amdgpu_async_gfx_ring) { |
| 3556 | − r = gfx_v12_0_kiq_disable_kgq(adev); | |
| 3557 | − if (r) | |
| 3559 | + if (amdgpu_gfx_disable_kgq(adev, 0)) | |
| 3558 | 3560 | DRM_ERROR("KGQ disable failed\n"); |
| 3559 | 3561 | } |
| 3560 | 3562 | |
| @@ -231,7 +231,7 @@ static void gmc_v11_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 230 unmodified lines | ||
| 231 | 231 | /* This is necessary for SRIOV as well as for GFXOFF to function |
| 232 | 232 | * properly under bare metal |
| 233 | 233 | */ |
| 234 | − if ((adev->gfx.kiq[0].ring.sched.ready || adev->mes.ring.sched.ready) && | |
| 234 | + if ((adev->gfx.kiq[0].ring.sched.ready || adev->mes.ring[0].sched.ready) && | |
| 235 | 235 | (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev))) { |
| 236 | 236 | amdgpu_gmc_fw_reg_write_reg_wait(adev, req, ack, inv_req, |
| 237 | 237 | 1 << vmid, GET_INST(GC, 0)); |
| @@ -299,7 +299,7 @@ static void gmc_v12_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 298 unmodified lines | ||
| 299 | 299 | /* This is necessary for SRIOV as well as for GFXOFF to function |
| 300 | 300 | * properly under bare metal |
| 301 | 301 | */ |
| 302 | − if ((adev->gfx.kiq[0].ring.sched.ready || adev->mes.ring.sched.ready) && | |
| 302 | + if ((adev->gfx.kiq[0].ring.sched.ready || adev->mes.ring[0].sched.ready) && | |
| 303 | 303 | (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev))) { |
| 304 | 304 | struct amdgpu_vmhub *hub = &adev->vmhub[vmhub]; |
| 305 | 305 | const unsigned eng = 17; |
| @@ -542,6 +542,23 @@ static unsigned gmc_v12_0_get_vbios_fb_size(struct amdgpu_device *adev) · 236 unmodified lines | ||
| 542 | 542 | return 0; |
| 543 | 543 | } |
| 544 | 544 | |
| 545 | +static unsigned int gmc_v12_0_get_dcc_alignment(struct amdgpu_device *adev) | |
| 546 | +{ | |
| 547 | + unsigned int max_tex_channel_caches, alignment; | |
| 548 | + | |
| 549 | + if (amdgpu_ip_version(adev, GC_HWIP, 0) != IP_VERSION(12, 0, 0) && | |
| 550 | + amdgpu_ip_version(adev, GC_HWIP, 0) != IP_VERSION(12, 0, 1)) | |
| 551 | + return 0; | |
| 552 | + | |
| 553 | + max_tex_channel_caches = adev->gfx.config.max_texture_channel_caches; | |
| 554 | + if (is_power_of_2(max_tex_channel_caches)) | |
| 555 | + alignment = (unsigned int)(max_tex_channel_caches / SZ_4); | |
| 556 | + else | |
| 557 | + alignment = roundup_pow_of_two(max_tex_channel_caches); | |
| 558 | + | |
| 559 | + return (unsigned int)(alignment * max_tex_channel_caches * SZ_1K); | |
| 560 | +} | |
| 561 | + | |
| 545 | 562 | static const struct amdgpu_gmc_funcs gmc_v12_0_gmc_funcs = { |
| 546 | 563 | .flush_gpu_tlb = gmc_v12_0_flush_gpu_tlb, |
| 547 | 564 | .flush_gpu_tlb_pasid = gmc_v12_0_flush_gpu_tlb_pasid, |
| @@ -551,6 +568,7 @@ static const struct amdgpu_gmc_funcs gmc_v12_0_gmc_funcs = { · 3 unmodified lines | ||
| 551 | 568 | .get_vm_pde = gmc_v12_0_get_vm_pde, |
| 552 | 569 | .get_vm_pte = gmc_v12_0_get_vm_pte, |
| 553 | 570 | .get_vbios_fb_size = gmc_v12_0_get_vbios_fb_size, |
| 571 | + .get_dcc_alignment = gmc_v12_0_get_dcc_alignment, | |
| 554 | 572 | }; |
| 555 | 573 | |
| 556 | 574 | static void gmc_v12_0_set_gmc_funcs(struct amdgpu_device *adev) |
| @@ -538,11 +538,11 @@ void jpeg_v2_0_dec_ring_emit_ib(struct amdgpu_ring *ring, · 537 unmodified lines | ||
| 538 | 538 | |
| 539 | 539 | amdgpu_ring_write(ring, PACKETJ(mmUVD_LMI_JRBC_IB_VMID_INTERNAL_OFFSET, |
| 540 | 540 | 0, 0, PACKETJ_TYPE0)); |
| 541 | − amdgpu_ring_write(ring, (vmid | (vmid << 4))); | |
| 541 | + amdgpu_ring_write(ring, (vmid | (vmid << 4) | (vmid << 8))); | |
| 542 | 542 | |
| 543 | 543 | amdgpu_ring_write(ring, PACKETJ(mmUVD_LMI_JPEG_VMID_INTERNAL_OFFSET, |
| 544 | 544 | 0, 0, PACKETJ_TYPE0)); |
| 545 | − amdgpu_ring_write(ring, (vmid | (vmid << 4))); | |
| 545 | + amdgpu_ring_write(ring, (vmid | (vmid << 4) | (vmid << 8))); | |
| 546 | 546 | |
| 547 | 547 | amdgpu_ring_write(ring, PACKETJ(mmUVD_LMI_JRBC_IB_64BIT_BAR_LOW_INTERNAL_OFFSET, |
| 548 | 548 | 0, 0, PACKETJ_TYPE0)); |
| @@ -23,6 +23,7 @@ · 22 unmodified lines | ||
| 23 | 23 | |
| 24 | 24 | #include "amdgpu.h" |
| 25 | 25 | #include "amdgpu_jpeg.h" |
| 26 | +#include "amdgpu_cs.h" | |
| 26 | 27 | #include "soc15.h" |
| 27 | 28 | #include "soc15d.h" |
| 28 | 29 | #include "jpeg_v4_0_3.h" |
| @@ -782,11 +783,15 @@ void jpeg_v4_0_3_dec_ring_emit_ib(struct amdgpu_ring *ring, · 753 unmodified lines | ||
| 782 | 783 | |
| 783 | 784 | amdgpu_ring_write(ring, PACKETJ(regUVD_LMI_JRBC_IB_VMID_INTERNAL_OFFSET, |
| 784 | 785 | 0, 0, PACKETJ_TYPE0)); |
| 785 | − amdgpu_ring_write(ring, (vmid | (vmid << 4))); | |
| 786 | + | |
| 787 | + if (ring->funcs->parse_cs) | |
| 788 | + amdgpu_ring_write(ring, 0); | |
| 789 | + else | |
| 790 | + amdgpu_ring_write(ring, (vmid | (vmid << 4) | (vmid << 8))); | |
| 786 | 791 | |
| 787 | 792 | amdgpu_ring_write(ring, PACKETJ(regUVD_LMI_JPEG_VMID_INTERNAL_OFFSET, |
| 788 | 793 | 0, 0, PACKETJ_TYPE0)); |
| 789 | − amdgpu_ring_write(ring, (vmid | (vmid << 4))); | |
| 794 | + amdgpu_ring_write(ring, (vmid | (vmid << 4) | (vmid << 8))); | |
| 790 | 795 | |
| 791 | 796 | amdgpu_ring_write(ring, PACKETJ(regUVD_LMI_JRBC_IB_64BIT_BAR_LOW_INTERNAL_OFFSET, |
| 792 | 797 | 0, 0, PACKETJ_TYPE0)); |
| @@ -1084,6 +1089,7 @@ static const struct amdgpu_ring_funcs jpeg_v4_0_3_dec_ring_vm_funcs = { · 291 unmodified lines | ||
| 1084 | 1089 | .get_rptr = jpeg_v4_0_3_dec_ring_get_rptr, |
| 1085 | 1090 | .get_wptr = jpeg_v4_0_3_dec_ring_get_wptr, |
| 1086 | 1091 | .set_wptr = jpeg_v4_0_3_dec_ring_set_wptr, |
| 1092 | + .parse_cs = jpeg_v4_0_3_dec_ring_parse_cs, | |
| 1087 | 1093 | .emit_frame_size = |
| 1088 | 1094 | SOC15_FLUSH_GPU_TLB_NUM_WREG * 6 + |
| 1089 | 1095 | SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 8 + |
| @@ -1248,3 +1254,56 @@ static void jpeg_v4_0_3_set_ras_funcs(struct amdgpu_device *adev) · 158 unmodified lines | ||
| 1248 | 1254 | { |
| 1249 | 1255 | adev->jpeg.ras = &jpeg_v4_0_3_ras; |
| 1250 | 1256 | } |
| 1257 | + | |
| 1258 | +/** | |
| 1259 | + * jpeg_v4_0_3_dec_ring_parse_cs - command submission parser | |
| 1260 | + * | |
| 1261 | + * @parser: Command submission parser context | |
| 1262 | + * @job: the job to parse | |
| 1263 | + * @ib: the IB to parse | |
| 1264 | + * | |
| 1265 | + * Parse the command stream, return -EINVAL for invalid packet, | |
| 1266 | + * 0 otherwise | |
| 1267 | + */ | |
| 1268 | +int jpeg_v4_0_3_dec_ring_parse_cs(struct amdgpu_cs_parser *parser, | |
| 1269 | + struct amdgpu_job *job, | |
| 1270 | + struct amdgpu_ib *ib) | |
| 1271 | +{ | |
| 1272 | + uint32_t i, reg, res, cond, type; | |
| 1273 | + struct amdgpu_device *adev = parser->adev; | |
| 1274 | + | |
| 1275 | + for (i = 0; i < ib->length_dw ; i += 2) { | |
| 1276 | + reg = CP_PACKETJ_GET_REG(ib->ptr[i]); | |
| 1277 | + res = CP_PACKETJ_GET_RES(ib->ptr[i]); | |
| 1278 | + cond = CP_PACKETJ_GET_COND(ib->ptr[i]); | |
| 1279 | + type = CP_PACKETJ_GET_TYPE(ib->ptr[i]); | |
| 1280 | + | |
| 1281 | + if (res) /* only support 0 at the moment */ | |
| 1282 | + return -EINVAL; | |
| 1283 | + | |
| 1284 | + switch (type) { | |
| 1285 | + case PACKETJ_TYPE0: | |
| 1286 | + if (cond != PACKETJ_CONDITION_CHECK0 || reg < JPEG_REG_RANGE_START || reg > JPEG_REG_RANGE_END) { | |
| 1287 | + dev_err(adev->dev, "Invalid packet [0x%08x]!\n", ib->ptr[i]); | |
| 1288 | + return -EINVAL; | |
| 1289 | + } | |
| 1290 | + break; | |
| 1291 | + case PACKETJ_TYPE3: | |
| 1292 | + if (cond != PACKETJ_CONDITION_CHECK3 || reg < JPEG_REG_RANGE_START || reg > JPEG_REG_RANGE_END) { | |
| 1293 | + dev_err(adev->dev, "Invalid packet [0x%08x]!\n", ib->ptr[i]); | |
| 1294 | + return -EINVAL; | |
| 1295 | + } | |
| 1296 | + break; | |
| 1297 | + case PACKETJ_TYPE6: | |
| 1298 | + if (ib->ptr[i] == CP_PACKETJ_NOP) | |
| 1299 | + continue; | |
| 1300 | + dev_err(adev->dev, "Invalid packet [0x%08x]!\n", ib->ptr[i]); | |
| 1301 | + return -EINVAL; | |
| 1302 | + default: | |
| 1303 | + dev_err(adev->dev, "Unknown packet type %d !\n", type); | |
| 1304 | + return -EINVAL; | |
| 1305 | + } | |
| 1306 | + } | |
| 1307 | + | |
| 1308 | + return 0; | |
| 1309 | +} | |
| @@ -46,6 +46,9 @@ · 45 unmodified lines | ||
| 46 | 46 | |
| 47 | 47 | #define JRBC_DEC_EXTERNAL_REG_WRITE_ADDR 0x18000 |
| 48 | 48 | |
| 49 | +#define JPEG_REG_RANGE_START 0x4000 | |
| 50 | +#define JPEG_REG_RANGE_END 0x41c2 | |
| 51 | + | |
| 49 | 52 | extern const struct amdgpu_ip_block_version jpeg_v4_0_3_ip_block; |
| 50 | 53 | |
| 51 | 54 | void jpeg_v4_0_3_dec_ring_emit_ib(struct amdgpu_ring *ring, |
| @@ -62,5 +65,7 @@ void jpeg_v4_0_3_dec_ring_insert_end(struct amdgpu_ring *ring); · 10 unmodified lines | ||
| 62 | 65 | void jpeg_v4_0_3_dec_ring_emit_wreg(struct amdgpu_ring *ring, uint32_t reg, uint32_t val); |
| 63 | 66 | void jpeg_v4_0_3_dec_ring_emit_reg_wait(struct amdgpu_ring *ring, uint32_t reg, |
| 64 | 67 | uint32_t val, uint32_t mask); |
| 65 | − | |
| 68 | +int jpeg_v4_0_3_dec_ring_parse_cs(struct amdgpu_cs_parser *parser, | |
| 69 | + struct amdgpu_job *job, | |
| 70 | + struct amdgpu_ib *ib); | |
| 66 | 71 | #endif /* __JPEG_V4_0_3_H__ */ |
| @@ -646,6 +646,7 @@ static const struct amdgpu_ring_funcs jpeg_v5_0_0_dec_ring_vm_funcs = { · 645 unmodified lines | ||
| 646 | 646 | .get_rptr = jpeg_v5_0_0_dec_ring_get_rptr, |
| 647 | 647 | .get_wptr = jpeg_v5_0_0_dec_ring_get_wptr, |
| 648 | 648 | .set_wptr = jpeg_v5_0_0_dec_ring_set_wptr, |
| 649 | + .parse_cs = jpeg_v4_0_3_dec_ring_parse_cs, | |
| 649 | 650 | .emit_frame_size = |
| 650 | 651 | SOC15_FLUSH_GPU_TLB_NUM_WREG * 6 + |
| 651 | 652 | SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 8 + |
| @@ -162,13 +162,13 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, · 161 unmodified lines | ||
| 162 | 162 | union MESAPI__QUERY_MES_STATUS mes_status_pkt; |
| 163 | 163 | signed long timeout = 3000000; /* 3000 ms */ |
| 164 | 164 | struct amdgpu_device *adev = mes->adev; |
| 165 | − struct amdgpu_ring *ring = &mes->ring; | |
| 165 | + struct amdgpu_ring *ring = &mes->ring[0]; | |
| 166 | 166 | struct MES_API_STATUS *api_status; |
| 167 | 167 | union MESAPI__MISC *x_pkt = pkt; |
| 168 | 168 | const char *op_str, *misc_op_str; |
| 169 | 169 | unsigned long flags; |
| 170 | 170 | u64 status_gpu_addr; |
| 171 | − u32 status_offset; | |
| 171 | + u32 seq, status_offset; | |
| 172 | 172 | u64 *status_ptr; |
| 173 | 173 | signed long r; |
| 174 | 174 | int ret; |
| @@ -191,11 +191,18 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, · 16 unmodified lines | ||
| 191 | 191 | status_ptr = (u64 *)&adev->wb.wb[status_offset]; |
| 192 | 192 | *status_ptr = 0; |
| 193 | 193 | |
| 194 | − spin_lock_irqsave(&mes->ring_lock, flags); | |
| 194 | + spin_lock_irqsave(&mes->ring_lock[0], flags); | |
| 195 | 195 | r = amdgpu_ring_alloc(ring, (size + sizeof(mes_status_pkt)) / 4); |
| 196 | 196 | if (r) |
| 197 | 197 | goto error_unlock_free; |
| 198 | 198 | |
| 199 | + seq = ++ring->fence_drv.sync_seq; | |
| 200 | + r = amdgpu_fence_wait_polling(ring, | |
| 201 | + seq - ring->fence_drv.num_fences_mask, | |
| 202 | + timeout); | |
| 203 | + if (r < 1) | |
| 204 | + goto error_undo; | |
| 205 | + | |
| 199 | 206 | api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off); |
| 200 | 207 | api_status->api_completion_fence_addr = status_gpu_addr; |
| 201 | 208 | api_status->api_completion_fence_value = 1; |
| @@ -208,14 +215,13 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, · 6 unmodified lines | ||
| 208 | 215 | mes_status_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS; |
| 209 | 216 | mes_status_pkt.api_status.api_completion_fence_addr = |
| 210 | 217 | ring->fence_drv.gpu_addr; |
| 211 | − mes_status_pkt.api_status.api_completion_fence_value = | |
| 212 | − ++ring->fence_drv.sync_seq; | |
| 218 | + mes_status_pkt.api_status.api_completion_fence_value = seq; | |
| 213 | 219 | |
| 214 | 220 | amdgpu_ring_write_multiple(ring, &mes_status_pkt, |
| 215 | 221 | sizeof(mes_status_pkt) / 4); |
| 216 | 222 | |
| 217 | 223 | amdgpu_ring_commit(ring); |
| 218 | − spin_unlock_irqrestore(&mes->ring_lock, flags); | |
| 224 | + spin_unlock_irqrestore(&mes->ring_lock[0], flags); | |
| 219 | 225 | |
| 220 | 226 | op_str = mes_v11_0_get_op_string(x_pkt); |
| 221 | 227 | misc_op_str = mes_v11_0_get_misc_op_string(x_pkt); |
| @@ -229,7 +235,7 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, · 7 unmodified lines | ||
| 229 | 235 | dev_dbg(adev->dev, "MES msg=%d was emitted\n", |
| 230 | 236 | x_pkt->header.opcode); |
| 231 | 237 | |
| 232 | − r = amdgpu_fence_wait_polling(ring, ring->fence_drv.sync_seq, timeout); | |
| 238 | + r = amdgpu_fence_wait_polling(ring, seq, timeout); | |
| 233 | 239 | if (r < 1 || !*status_ptr) { |
| 234 | 240 | |
| 235 | 241 | if (misc_op_str) |
| @@ -252,8 +258,12 @@ static int mes_v11_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, · 16 unmodified lines | ||
| 252 | 258 | amdgpu_device_wb_free(adev, status_offset); |
| 253 | 259 | return 0; |
| 254 | 260 | |
| 261 | +error_undo: | |
| 262 | + dev_err(adev->dev, "MES ring buffer is full.\n"); | |
| 263 | + amdgpu_ring_undo(ring); | |
| 264 | + | |
| 255 | 265 | error_unlock_free: |
| 256 | − spin_unlock_irqrestore(&mes->ring_lock, flags); | |
| 266 | + spin_unlock_irqrestore(&mes->ring_lock[0], flags); | |
| 257 | 267 | |
| 258 | 268 | error_wb_free: |
| 259 | 269 | amdgpu_device_wb_free(adev, status_offset); |
| @@ -512,9 +522,9 @@ static int mes_v11_0_set_hw_resources(struct amdgpu_mes *mes) · 252 unmodified lines | ||
| 512 | 522 | mes_set_hw_res_pkt.vmid_mask_gfxhub = mes->vmid_mask_gfxhub; |
| 513 | 523 | mes_set_hw_res_pkt.gds_size = adev->gds.gds_size; |
| 514 | 524 | mes_set_hw_res_pkt.paging_vmid = 0; |
| 515 | − mes_set_hw_res_pkt.g_sch_ctx_gpu_mc_ptr = mes->sch_ctx_gpu_addr; | |
| 525 | + mes_set_hw_res_pkt.g_sch_ctx_gpu_mc_ptr = mes->sch_ctx_gpu_addr[0]; | |
| 516 | 526 | mes_set_hw_res_pkt.query_status_fence_gpu_mc_ptr = |
| 517 | − mes->query_status_fence_gpu_addr; | |
| 527 | + mes->query_status_fence_gpu_addr[0]; | |
| 518 | 528 | |
| 519 | 529 | for (i = 0; i < MAX_COMPUTE_PIPES; i++) |
| 520 | 530 | mes_set_hw_res_pkt.compute_hqd_mask[i] = |
| @@ -1015,7 +1025,7 @@ static int mes_v11_0_kiq_enable_queue(struct amdgpu_device *adev) · 494 unmodified lines | ||
| 1015 | 1025 | return r; |
| 1016 | 1026 | } |
| 1017 | 1027 | |
| 1018 | − kiq->pmf->kiq_map_queues(kiq_ring, &adev->mes.ring); | |
| 1028 | + kiq->pmf->kiq_map_queues(kiq_ring, &adev->mes.ring[0]); | |
| 1019 | 1029 | |
| 1020 | 1030 | return amdgpu_ring_test_helper(kiq_ring); |
| 1021 | 1031 | } |
| @@ -1029,7 +1039,7 @@ static int mes_v11_0_queue_init(struct amdgpu_device *adev, · 7 unmodified lines | ||
| 1029 | 1039 | if (pipe == AMDGPU_MES_KIQ_PIPE) |
| 1030 | 1040 | ring = &adev->gfx.kiq[0].ring; |
| 1031 | 1041 | else if (pipe == AMDGPU_MES_SCHED_PIPE) |
| 1032 | − ring = &adev->mes.ring; | |
| 1042 | + ring = &adev->mes.ring[0]; | |
| 1033 | 1043 | else |
| 1034 | 1044 | BUG(); |
| 1035 | 1045 | |
| @@ -1071,7 +1081,7 @@ static int mes_v11_0_ring_init(struct amdgpu_device *adev) · 35 unmodified lines | ||
| 1071 | 1081 | { |
| 1072 | 1082 | struct amdgpu_ring *ring; |
| 1073 | 1083 | |
| 1074 | − ring = &adev->mes.ring; | |
| 1084 | + ring = &adev->mes.ring[0]; | |
| 1075 | 1085 | |
| 1076 | 1086 | ring->funcs = &mes_v11_0_ring_funcs; |
| 1077 | 1087 | |
| @@ -1124,7 +1134,7 @@ static int mes_v11_0_mqd_sw_init(struct amdgpu_device *adev, · 46 unmodified lines | ||
| 1124 | 1134 | if (pipe == AMDGPU_MES_KIQ_PIPE) |
| 1125 | 1135 | ring = &adev->gfx.kiq[0].ring; |
| 1126 | 1136 | else if (pipe == AMDGPU_MES_SCHED_PIPE) |
| 1127 | − ring = &adev->mes.ring; | |
| 1137 | + ring = &adev->mes.ring[0]; | |
| 1128 | 1138 | else |
| 1129 | 1139 | BUG(); |
| 1130 | 1140 | |
| @@ -1163,6 +1173,8 @@ static int mes_v11_0_sw_init(void *handle) · 32 unmodified lines | ||
| 1163 | 1173 | adev->mes.kiq_hw_init = &mes_v11_0_kiq_hw_init; |
| 1164 | 1174 | adev->mes.kiq_hw_fini = &mes_v11_0_kiq_hw_fini; |
| 1165 | 1175 | |
| 1176 | + adev->mes.event_log_size = AMDGPU_MES_LOG_BUFFER_SIZE; | |
| 1177 | + | |
| 1166 | 1178 | r = amdgpu_mes_init(adev); |
| 1167 | 1179 | if (r) |
| 1168 | 1180 | return r; |
| @@ -1198,9 +1210,6 @@ static int mes_v11_0_sw_fini(void *handle) · 29 unmodified lines | ||
| 1198 | 1210 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1199 | 1211 | int pipe; |
| 1200 | 1212 | |
| 1201 | − amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs); | |
| 1202 | − amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs); | |
| 1203 | − | |
| 1204 | 1213 | for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) { |
| 1205 | 1214 | kfree(adev->mes.mqd_backup[pipe]); |
| 1206 | 1215 | |
| @@ -1214,12 +1223,12 @@ static int mes_v11_0_sw_fini(void *handle) · 7 unmodified lines | ||
| 1214 | 1223 | &adev->gfx.kiq[0].ring.mqd_gpu_addr, |
| 1215 | 1224 | &adev->gfx.kiq[0].ring.mqd_ptr); |
| 1216 | 1225 | |
| 1217 | − amdgpu_bo_free_kernel(&adev->mes.ring.mqd_obj, | |
| 1218 | − &adev->mes.ring.mqd_gpu_addr, | |
| 1219 | − &adev->mes.ring.mqd_ptr); | |
| 1226 | + amdgpu_bo_free_kernel(&adev->mes.ring[0].mqd_obj, | |
| 1227 | + &adev->mes.ring[0].mqd_gpu_addr, | |
| 1228 | + &adev->mes.ring[0].mqd_ptr); | |
| 1220 | 1229 | |
| 1221 | 1230 | amdgpu_ring_fini(&adev->gfx.kiq[0].ring); |
| 1222 | − amdgpu_ring_fini(&adev->mes.ring); | |
| 1231 | + amdgpu_ring_fini(&adev->mes.ring[0]); | |
| 1223 | 1232 | |
| 1224 | 1233 | if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) { |
| 1225 | 1234 | mes_v11_0_free_ucode_buffers(adev, AMDGPU_MES_KIQ_PIPE); |
| @@ -1330,9 +1339,9 @@ failure: · 104 unmodified lines | ||
| 1330 | 1339 | |
| 1331 | 1340 | static int mes_v11_0_kiq_hw_fini(struct amdgpu_device *adev) |
| 1332 | 1341 | { |
| 1333 | − if (adev->mes.ring.sched.ready) { | |
| 1334 | − mes_v11_0_kiq_dequeue(&adev->mes.ring); | |
| 1335 | − adev->mes.ring.sched.ready = false; | |
| 1342 | + if (adev->mes.ring[0].sched.ready) { | |
| 1343 | + mes_v11_0_kiq_dequeue(&adev->mes.ring[0]); | |
| 1344 | + adev->mes.ring[0].sched.ready = false; | |
| 1336 | 1345 | } |
| 1337 | 1346 | |
| 1338 | 1347 | if (amdgpu_sriov_vf(adev)) { |
| @@ -1350,7 +1359,7 @@ static int mes_v11_0_hw_init(void *handle) · 11 unmodified lines | ||
| 1350 | 1359 | int r; |
| 1351 | 1360 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1352 | 1361 | |
| 1353 | − if (adev->mes.ring.sched.ready) | |
| 1362 | + if (adev->mes.ring[0].sched.ready) | |
| 1354 | 1363 | goto out; |
| 1355 | 1364 | |
| 1356 | 1365 | if (!adev->enable_mes_kiq) { |
| @@ -1395,7 +1404,7 @@ out: · 38 unmodified lines | ||
| 1395 | 1404 | * with MES enabled. |
| 1396 | 1405 | */ |
| 1397 | 1406 | adev->gfx.kiq[0].ring.sched.ready = false; |
| 1398 | − adev->mes.ring.sched.ready = true; | |
| 1407 | + adev->mes.ring[0].sched.ready = true; | |
| 1399 | 1408 | |
| 1400 | 1409 | return 0; |
| 1401 | 1410 | |
| @@ -142,19 +142,20 @@ static const char *mes_v12_0_get_misc_op_string(union MESAPI__MISC *x_pkt) · 141 unmodified lines | ||
| 142 | 142 | } |
| 143 | 143 | |
| 144 | 144 | static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, |
| 145 | − void *pkt, int size, | |
| 146 | − int api_status_off) | |
| 145 | + int pipe, void *pkt, int size, | |
| 146 | + int api_status_off) | |
| 147 | 147 | { |
| 148 | 148 | union MESAPI__QUERY_MES_STATUS mes_status_pkt; |
| 149 | 149 | signed long timeout = 3000000; /* 3000 ms */ |
| 150 | 150 | struct amdgpu_device *adev = mes->adev; |
| 151 | − struct amdgpu_ring *ring = &mes->ring; | |
| 151 | + struct amdgpu_ring *ring = &mes->ring[pipe]; | |
| 152 | + spinlock_t *ring_lock = &mes->ring_lock[pipe]; | |
| 152 | 153 | struct MES_API_STATUS *api_status; |
| 153 | 154 | union MESAPI__MISC *x_pkt = pkt; |
| 154 | 155 | const char *op_str, *misc_op_str; |
| 155 | 156 | unsigned long flags; |
| 156 | 157 | u64 status_gpu_addr; |
| 157 | − u32 status_offset; | |
| 158 | + u32 seq, status_offset; | |
| 158 | 159 | u64 *status_ptr; |
| 159 | 160 | signed long r; |
| 160 | 161 | int ret; |
| @@ -177,11 +178,18 @@ static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, · 16 unmodified lines | ||
| 177 | 178 | status_ptr = (u64 *)&adev->wb.wb[status_offset]; |
| 178 | 179 | *status_ptr = 0; |
| 179 | 180 | |
| 180 | − spin_lock_irqsave(&mes->ring_lock, flags); | |
| 181 | + spin_lock_irqsave(ring_lock, flags); | |
| 181 | 182 | r = amdgpu_ring_alloc(ring, (size + sizeof(mes_status_pkt)) / 4); |
| 182 | 183 | if (r) |
| 183 | 184 | goto error_unlock_free; |
| 184 | 185 | |
| 186 | + seq = ++ring->fence_drv.sync_seq; | |
| 187 | + r = amdgpu_fence_wait_polling(ring, | |
| 188 | + seq - ring->fence_drv.num_fences_mask, | |
| 189 | + timeout); | |
| 190 | + if (r < 1) | |
| 191 | + goto error_undo; | |
| 192 | + | |
| 185 | 193 | api_status = (struct MES_API_STATUS *)((char *)pkt + api_status_off); |
| 186 | 194 | api_status->api_completion_fence_addr = status_gpu_addr; |
| 187 | 195 | api_status->api_completion_fence_value = 1; |
| @@ -194,39 +202,39 @@ static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, · 6 unmodified lines | ||
| 194 | 202 | mes_status_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS; |
| 195 | 203 | mes_status_pkt.api_status.api_completion_fence_addr = |
| 196 | 204 | ring->fence_drv.gpu_addr; |
| 197 | − mes_status_pkt.api_status.api_completion_fence_value = | |
| 198 | − ++ring->fence_drv.sync_seq; | |
| 205 | + mes_status_pkt.api_status.api_completion_fence_value = seq; | |
| 199 | 206 | |
| 200 | 207 | amdgpu_ring_write_multiple(ring, &mes_status_pkt, |
| 201 | 208 | sizeof(mes_status_pkt) / 4); |
| 202 | 209 | |
| 203 | 210 | amdgpu_ring_commit(ring); |
| 204 | − spin_unlock_irqrestore(&mes->ring_lock, flags); | |
| 211 | + spin_unlock_irqrestore(ring_lock, flags); | |
| 205 | 212 | |
| 206 | 213 | op_str = mes_v12_0_get_op_string(x_pkt); |
| 207 | 214 | misc_op_str = mes_v12_0_get_misc_op_string(x_pkt); |
| 208 | 215 | |
| 209 | 216 | if (misc_op_str) |
| 210 | − dev_dbg(adev->dev, "MES msg=%s (%s) was emitted\n", op_str, | |
| 211 | − misc_op_str); | |
| 217 | + dev_dbg(adev->dev, "MES(%d) msg=%s (%s) was emitted\n", | |
| 218 | + pipe, op_str, misc_op_str); | |
| 212 | 219 | else if (op_str) |
| 213 | − dev_dbg(adev->dev, "MES msg=%s was emitted\n", op_str); | |
| 220 | + dev_dbg(adev->dev, "MES(%d) msg=%s was emitted\n", | |
| 221 | + pipe, op_str); | |
| 214 | 222 | else |
| 215 | − dev_dbg(adev->dev, "MES msg=%d was emitted\n", | |
| 216 | − x_pkt->header.opcode); | |
| 223 | + dev_dbg(adev->dev, "MES(%d) msg=%d was emitted\n", | |
| 224 | + pipe, x_pkt->header.opcode); | |
| 217 | 225 | |
| 218 | − r = amdgpu_fence_wait_polling(ring, ring->fence_drv.sync_seq, timeout); | |
| 226 | + r = amdgpu_fence_wait_polling(ring, seq, timeout); | |
| 219 | 227 | if (r < 1 || !*status_ptr) { |
| 220 | 228 | |
| 221 | 229 | if (misc_op_str) |
| 222 | − dev_err(adev->dev, "MES failed to respond to msg=%s (%s)\n", | |
| 223 | − op_str, misc_op_str); | |
| 230 | + dev_err(adev->dev, "MES(%d) failed to respond to msg=%s (%s)\n", | |
| 231 | + pipe, op_str, misc_op_str); | |
| 224 | 232 | else if (op_str) |
| 225 | − dev_err(adev->dev, "MES failed to respond to msg=%s\n", | |
| 226 | − op_str); | |
| 233 | + dev_err(adev->dev, "MES(%d) failed to respond to msg=%s\n", | |
| 234 | + pipe, op_str); | |
| 227 | 235 | else |
| 228 | − dev_err(adev->dev, "MES failed to respond to msg=%d\n", | |
| 229 | − x_pkt->header.opcode); | |
| 236 | + dev_err(adev->dev, "MES(%d) failed to respond to msg=%d\n", | |
| 237 | + pipe, x_pkt->header.opcode); | |
| 230 | 238 | |
| 231 | 239 | while (halt_if_hws_hang) |
| 232 | 240 | schedule(); |
| @@ -238,8 +246,12 @@ static int mes_v12_0_submit_pkt_and_poll_completion(struct amdgpu_mes *mes, · 5 unmodified lines | ||
| 238 | 246 | amdgpu_device_wb_free(adev, status_offset); |
| 239 | 247 | return 0; |
| 240 | 248 | |
| 249 | +error_undo: | |
| 250 | + dev_err(adev->dev, "MES ring buffer is full.\n"); | |
| 251 | + amdgpu_ring_undo(ring); | |
| 252 | + | |
| 241 | 253 | error_unlock_free: |
| 242 | − spin_unlock_irqrestore(&mes->ring_lock, flags); | |
| 254 | + spin_unlock_irqrestore(ring_lock, flags); | |
| 243 | 255 | |
| 244 | 256 | error_wb_free: |
| 245 | 257 | amdgpu_device_wb_free(adev, status_offset); |
| @@ -254,6 +266,8 @@ static int convert_to_mes_queue_type(int queue_type) · 8 unmodified lines | ||
| 254 | 266 | return MES_QUEUE_TYPE_COMPUTE; |
| 255 | 267 | else if (queue_type == AMDGPU_RING_TYPE_SDMA) |
| 256 | 268 | return MES_QUEUE_TYPE_SDMA; |
| 269 | + else if (queue_type == AMDGPU_RING_TYPE_MES) | |
| 270 | + return MES_QUEUE_TYPE_SCHQ; | |
| 257 | 271 | else |
| 258 | 272 | BUG(); |
| 259 | 273 | return -1; |
| @@ -311,6 +325,7 @@ static int mes_v12_0_add_hw_queue(struct amdgpu_mes *mes, · 51 unmodified lines | ||
| 311 | 325 | mes_add_queue_pkt.gds_size = input->queue_size; |
| 312 | 326 | |
| 313 | 327 | return mes_v12_0_submit_pkt_and_poll_completion(mes, |
| 328 | + AMDGPU_MES_SCHED_PIPE, | |
| 314 | 329 | &mes_add_queue_pkt, sizeof(mes_add_queue_pkt), |
| 315 | 330 | offsetof(union MESAPI__ADD_QUEUE, api_status)); |
| 316 | 331 | } |
| @@ -330,6 +345,7 @@ static int mes_v12_0_remove_hw_queue(struct amdgpu_mes *mes, · 13 unmodified lines | ||
| 330 | 345 | mes_remove_queue_pkt.gang_context_addr = input->gang_context_addr; |
| 331 | 346 | |
| 332 | 347 | return mes_v12_0_submit_pkt_and_poll_completion(mes, |
| 348 | + AMDGPU_MES_SCHED_PIPE, | |
| 333 | 349 | &mes_remove_queue_pkt, sizeof(mes_remove_queue_pkt), |
| 334 | 350 | offsetof(union MESAPI__REMOVE_QUEUE, api_status)); |
| 335 | 351 | } |
| @@ -338,6 +354,7 @@ static int mes_v12_0_map_legacy_queue(struct amdgpu_mes *mes, · 2 unmodified lines | ||
| 338 | 354 | struct mes_map_legacy_queue_input *input) |
| 339 | 355 | { |
| 340 | 356 | union MESAPI__ADD_QUEUE mes_add_queue_pkt; |
| 357 | + int pipe; | |
| 341 | 358 | |
| 342 | 359 | memset(&mes_add_queue_pkt, 0, sizeof(mes_add_queue_pkt)); |
| 343 | 360 | |
| @@ -354,7 +371,12 @@ static int mes_v12_0_map_legacy_queue(struct amdgpu_mes *mes, · 10 unmodified lines | ||
| 354 | 371 | convert_to_mes_queue_type(input->queue_type); |
| 355 | 372 | mes_add_queue_pkt.map_legacy_kq = 1; |
| 356 | 373 | |
| 357 | − return mes_v12_0_submit_pkt_and_poll_completion(mes, | |
| 374 | + if (mes->adev->enable_uni_mes) | |
| 375 | + pipe = AMDGPU_MES_KIQ_PIPE; | |
| 376 | + else | |
| 377 | + pipe = AMDGPU_MES_SCHED_PIPE; | |
| 378 | + | |
| 379 | + return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe, | |
| 358 | 380 | &mes_add_queue_pkt, sizeof(mes_add_queue_pkt), |
| 359 | 381 | offsetof(union MESAPI__ADD_QUEUE, api_status)); |
| 360 | 382 | } |
| @@ -363,6 +385,7 @@ static int mes_v12_0_unmap_legacy_queue(struct amdgpu_mes *mes, · 2 unmodified lines | ||
| 363 | 385 | struct mes_unmap_legacy_queue_input *input) |
| 364 | 386 | { |
| 365 | 387 | union MESAPI__REMOVE_QUEUE mes_remove_queue_pkt; |
| 388 | + int pipe; | |
| 366 | 389 | |
| 367 | 390 | memset(&mes_remove_queue_pkt, 0, sizeof(mes_remove_queue_pkt)); |
| 368 | 391 | |
| @@ -387,7 +410,12 @@ static int mes_v12_0_unmap_legacy_queue(struct amdgpu_mes *mes, · 18 unmodified lines | ||
| 387 | 410 | convert_to_mes_queue_type(input->queue_type); |
| 388 | 411 | } |
| 389 | 412 | |
| 390 | − return mes_v12_0_submit_pkt_and_poll_completion(mes, | |
| 413 | + if (mes->adev->enable_uni_mes) | |
| 414 | + pipe = AMDGPU_MES_KIQ_PIPE; | |
| 415 | + else | |
| 416 | + pipe = AMDGPU_MES_SCHED_PIPE; | |
| 417 | + | |
| 418 | + return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe, | |
| 391 | 419 | &mes_remove_queue_pkt, sizeof(mes_remove_queue_pkt), |
| 392 | 420 | offsetof(union MESAPI__REMOVE_QUEUE, api_status)); |
| 393 | 421 | } |
| @@ -404,7 +432,7 @@ static int mes_v12_0_resume_gang(struct amdgpu_mes *mes, · 10 unmodified lines | ||
| 404 | 432 | return 0; |
| 405 | 433 | } |
| 406 | 434 | |
| 407 | −static int mes_v12_0_query_sched_status(struct amdgpu_mes *mes) | |
| 435 | +static int mes_v12_0_query_sched_status(struct amdgpu_mes *mes, int pipe) | |
| 408 | 436 | { |
| 409 | 437 | union MESAPI__QUERY_MES_STATUS mes_status_pkt; |
| 410 | 438 | |
| @@ -414,7 +442,7 @@ static int mes_v12_0_query_sched_status(struct amdgpu_mes *mes) · 3 unmodified lines | ||
| 414 | 442 | mes_status_pkt.header.opcode = MES_SCH_API_QUERY_SCHEDULER_STATUS; |
| 415 | 443 | mes_status_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS; |
| 416 | 444 | |
| 417 | − return mes_v12_0_submit_pkt_and_poll_completion(mes, | |
| 445 | + return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe, | |
| 418 | 446 | &mes_status_pkt, sizeof(mes_status_pkt), |
| 419 | 447 | offsetof(union MESAPI__QUERY_MES_STATUS, api_status)); |
| 420 | 448 | } |
| @@ -423,6 +451,7 @@ static int mes_v12_0_misc_op(struct amdgpu_mes *mes, · 2 unmodified lines | ||
| 423 | 451 | struct mes_misc_op_input *input) |
| 424 | 452 | { |
| 425 | 453 | union MESAPI__MISC misc_pkt; |
| 454 | + int pipe; | |
| 426 | 455 | |
| 427 | 456 | memset(&misc_pkt, 0, sizeof(misc_pkt)); |
| 428 | 457 | |
| @@ -475,12 +504,17 @@ static int mes_v12_0_misc_op(struct amdgpu_mes *mes, · 46 unmodified lines | ||
| 475 | 504 | return -EINVAL; |
| 476 | 505 | } |
| 477 | 506 | |
| 478 | − return mes_v12_0_submit_pkt_and_poll_completion(mes, | |
| 507 | + if (mes->adev->enable_uni_mes) | |
| 508 | + pipe = AMDGPU_MES_KIQ_PIPE; | |
| 509 | + else | |
| 510 | + pipe = AMDGPU_MES_SCHED_PIPE; | |
| 511 | + | |
| 512 | + return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe, | |
| 479 | 513 | &misc_pkt, sizeof(misc_pkt), |
| 480 | 514 | offsetof(union MESAPI__MISC, api_status)); |
| 481 | 515 | } |
| 482 | 516 | |
| 483 | −static int mes_v12_0_set_hw_resources_1(struct amdgpu_mes *mes) | |
| 517 | +static int mes_v12_0_set_hw_resources_1(struct amdgpu_mes *mes, int pipe) | |
| 484 | 518 | { |
| 485 | 519 | union MESAPI_SET_HW_RESOURCES_1 mes_set_hw_res_1_pkt; |
| 486 | 520 | |
| @@ -491,12 +525,12 @@ static int mes_v12_0_set_hw_resources_1(struct amdgpu_mes *mes) · 4 unmodified lines | ||
| 491 | 525 | mes_set_hw_res_1_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS; |
| 492 | 526 | mes_set_hw_res_1_pkt.mes_kiq_unmap_timeout = 100; |
| 493 | 527 | |
| 494 | − return mes_v12_0_submit_pkt_and_poll_completion(mes, | |
| 528 | + return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe, | |
| 495 | 529 | &mes_set_hw_res_1_pkt, sizeof(mes_set_hw_res_1_pkt), |
| 496 | 530 | offsetof(union MESAPI_SET_HW_RESOURCES_1, api_status)); |
| 497 | 531 | } |
| 498 | 532 | |
| 499 | −static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes) | |
| 533 | +static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes, int pipe) | |
| 500 | 534 | { |
| 501 | 535 | int i; |
| 502 | 536 | struct amdgpu_device *adev = mes->adev; |
| @@ -508,27 +542,33 @@ static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes) · 5 unmodified lines | ||
| 508 | 542 | mes_set_hw_res_pkt.header.opcode = MES_SCH_API_SET_HW_RSRC; |
| 509 | 543 | mes_set_hw_res_pkt.header.dwsize = API_FRAME_SIZE_IN_DWORDS; |
| 510 | 544 | |
| 511 | − mes_set_hw_res_pkt.vmid_mask_mmhub = mes->vmid_mask_mmhub; | |
| 512 | − mes_set_hw_res_pkt.vmid_mask_gfxhub = mes->vmid_mask_gfxhub; | |
| 513 | − mes_set_hw_res_pkt.gds_size = adev->gds.gds_size; | |
| 514 | − mes_set_hw_res_pkt.paging_vmid = 0; | |
| 515 | − mes_set_hw_res_pkt.g_sch_ctx_gpu_mc_ptr = mes->sch_ctx_gpu_addr; | |
| 516 | − mes_set_hw_res_pkt.query_status_fence_gpu_mc_ptr = | |
| 517 | − mes->query_status_fence_gpu_addr; | |
| 518 | − | |
| 519 | − for (i = 0; i < MAX_COMPUTE_PIPES; i++) | |
| 520 | − mes_set_hw_res_pkt.compute_hqd_mask[i] = | |
| 521 | − mes->compute_hqd_mask[i]; | |
| 522 | − | |
| 523 | − for (i = 0; i < MAX_GFX_PIPES; i++) | |
| 524 | − mes_set_hw_res_pkt.gfx_hqd_mask[i] = mes->gfx_hqd_mask[i]; | |
| 525 | − | |
| 526 | − for (i = 0; i < MAX_SDMA_PIPES; i++) | |
| 527 | − mes_set_hw_res_pkt.sdma_hqd_mask[i] = mes->sdma_hqd_mask[i]; | |
| 545 | + if (pipe == AMDGPU_MES_SCHED_PIPE) { | |
| 546 | + mes_set_hw_res_pkt.vmid_mask_mmhub = mes->vmid_mask_mmhub; | |
| 547 | + mes_set_hw_res_pkt.vmid_mask_gfxhub = mes->vmid_mask_gfxhub; | |
| 548 | + mes_set_hw_res_pkt.gds_size = adev->gds.gds_size; | |
| 549 | + mes_set_hw_res_pkt.paging_vmid = 0; | |
| 550 | + | |
| 551 | + for (i = 0; i < MAX_COMPUTE_PIPES; i++) | |
| 552 | + mes_set_hw_res_pkt.compute_hqd_mask[i] = | |
| 553 | + mes->compute_hqd_mask[i]; | |
| 554 | + | |
| 555 | + for (i = 0; i < MAX_GFX_PIPES; i++) | |
| 556 | + mes_set_hw_res_pkt.gfx_hqd_mask[i] = | |
| 557 | + mes->gfx_hqd_mask[i]; | |
| 558 | + | |
| 559 | + for (i = 0; i < MAX_SDMA_PIPES; i++) | |
| 560 | + mes_set_hw_res_pkt.sdma_hqd_mask[i] = | |
| 561 | + mes->sdma_hqd_mask[i]; | |
| 562 | + | |
| 563 | + for (i = 0; i < AMD_PRIORITY_NUM_LEVELS; i++) | |
| 564 | + mes_set_hw_res_pkt.aggregated_doorbells[i] = | |
| 565 | + mes->aggregated_doorbells[i]; | |
| 566 | + } | |
| 528 | 567 | |
| 529 | − for (i = 0; i < AMD_PRIORITY_NUM_LEVELS; i++) | |
| 530 | − mes_set_hw_res_pkt.aggregated_doorbells[i] = | |
| 531 | − mes->aggregated_doorbells[i]; | |
| 568 | + mes_set_hw_res_pkt.g_sch_ctx_gpu_mc_ptr = | |
| 569 | + mes->sch_ctx_gpu_addr[pipe]; | |
| 570 | + mes_set_hw_res_pkt.query_status_fence_gpu_mc_ptr = | |
| 571 | + mes->query_status_fence_gpu_addr[pipe]; | |
| 532 | 572 | |
| 533 | 573 | for (i = 0; i < 5; i++) { |
| 534 | 574 | mes_set_hw_res_pkt.gc_base[i] = adev->reg_offset[GC_HWIP][0][i]; |
| @@ -551,10 +591,12 @@ static int mes_v12_0_set_hw_resources(struct amdgpu_mes *mes) · 16 unmodified lines | ||
| 551 | 591 | mes_set_hw_res_pkt.oversubscription_timer = 50; |
| 552 | 592 | mes_set_hw_res_pkt.unmapped_doorbell_handling = 1; |
| 553 | 593 | |
| 554 | − mes_set_hw_res_pkt.enable_mes_event_int_logging = 0; | |
| 555 | − mes_set_hw_res_pkt.event_intr_history_gpu_mc_ptr = mes->event_log_gpu_addr; | |
| 594 | + if (amdgpu_mes_log_enable) { | |
| 595 | + mes_set_hw_res_pkt.enable_mes_event_int_logging = 1; | |
| 596 | + mes_set_hw_res_pkt.event_intr_history_gpu_mc_ptr = mes->event_log_gpu_addr; | |
| 597 | + } | |
| 556 | 598 | |
| 557 | − return mes_v12_0_submit_pkt_and_poll_completion(mes, | |
| 599 | + return mes_v12_0_submit_pkt_and_poll_completion(mes, pipe, | |
| 558 | 600 | &mes_set_hw_res_pkt, sizeof(mes_set_hw_res_pkt), |
| 559 | 601 | offsetof(union MESAPI_SET_HW_RESOURCES, api_status)); |
| 560 | 602 | } |
| @@ -732,16 +774,11 @@ static void mes_v12_0_enable(struct amdgpu_device *adev, bool enable) · 171 unmodified lines | ||
| 732 | 774 | if (enable) { |
| 733 | 775 | data = RREG32_SOC15(GC, 0, regCP_MES_CNTL); |
| 734 | 776 | data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE0_RESET, 1); |
| 735 | − data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_RESET, | |
| 736 | − (!adev->enable_uni_mes && adev->enable_mes_kiq) ? 1 : 0); | |
| 777 | + data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_RESET, 1); | |
| 737 | 778 | WREG32_SOC15(GC, 0, regCP_MES_CNTL, data); |
| 738 | 779 | |
| 739 | 780 | mutex_lock(&adev->srbm_mutex); |
| 740 | 781 | for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) { |
| 741 | − if ((!adev->enable_mes_kiq || adev->enable_uni_mes) && | |
| 742 | − pipe == AMDGPU_MES_KIQ_PIPE) | |
| 743 | − continue; | |
| 744 | − | |
| 745 | 782 | soc21_grbm_select(adev, 3, pipe, 0, 0); |
| 746 | 783 | |
| 747 | 784 | ucode_addr = adev->mes.uc_start_addr[pipe] >> 2; |
| @@ -755,8 +792,7 @@ static void mes_v12_0_enable(struct amdgpu_device *adev, bool enable) · 7 unmodified lines | ||
| 755 | 792 | |
| 756 | 793 | /* unhalt MES and activate pipe0 */ |
| 757 | 794 | data = REG_SET_FIELD(0, CP_MES_CNTL, MES_PIPE0_ACTIVE, 1); |
| 758 | − data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_ACTIVE, | |
| 759 | − (!adev->enable_uni_mes && adev->enable_mes_kiq) ? 1 : 0); | |
| 795 | + data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_ACTIVE, 1); | |
| 760 | 796 | WREG32_SOC15(GC, 0, regCP_MES_CNTL, data); |
| 761 | 797 | |
| 762 | 798 | if (amdgpu_emu_mode) |
| @@ -772,8 +808,7 @@ static void mes_v12_0_enable(struct amdgpu_device *adev, bool enable) · 9 unmodified lines | ||
| 772 | 808 | data = REG_SET_FIELD(data, CP_MES_CNTL, |
| 773 | 809 | MES_INVALIDATE_ICACHE, 1); |
| 774 | 810 | data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE0_RESET, 1); |
| 775 | − data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_RESET, | |
| 776 | − (!adev->enable_uni_mes && adev->enable_mes_kiq) ? 1 : 0); | |
| 811 | + data = REG_SET_FIELD(data, CP_MES_CNTL, MES_PIPE1_RESET, 1); | |
| 777 | 812 | data = REG_SET_FIELD(data, CP_MES_CNTL, MES_HALT, 1); |
| 778 | 813 | WREG32_SOC15(GC, 0, regCP_MES_CNTL, data); |
| 779 | 814 | } |
| @@ -788,10 +823,6 @@ static void mes_v12_0_set_ucode_start_addr(struct amdgpu_device *adev) · 8 unmodified lines | ||
| 788 | 823 | |
| 789 | 824 | mutex_lock(&adev->srbm_mutex); |
| 790 | 825 | for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) { |
| 791 | − if ((!adev->enable_mes_kiq || adev->enable_uni_mes) && | |
| 792 | − pipe == AMDGPU_MES_KIQ_PIPE) | |
| 793 | − continue; | |
| 794 | − | |
| 795 | 826 | /* me=3, queue=0 */ |
| 796 | 827 | soc21_grbm_select(adev, 3, pipe, 0, 0); |
| 797 | 828 | |
| @@ -1083,7 +1114,7 @@ static int mes_v12_0_kiq_enable_queue(struct amdgpu_device *adev) · 285 unmodified lines | ||
| 1083 | 1114 | return r; |
| 1084 | 1115 | } |
| 1085 | 1116 | |
| 1086 | − kiq->pmf->kiq_map_queues(kiq_ring, &adev->mes.ring); | |
| 1117 | + kiq->pmf->kiq_map_queues(kiq_ring, &adev->mes.ring[0]); | |
| 1087 | 1118 | |
| 1088 | 1119 | r = amdgpu_ring_test_ring(kiq_ring); |
| 1089 | 1120 | if (r) { |
| @@ -1099,14 +1130,12 @@ static int mes_v12_0_queue_init(struct amdgpu_device *adev, · 9 unmodified lines | ||
| 1099 | 1130 | struct amdgpu_ring *ring; |
| 1100 | 1131 | int r; |
| 1101 | 1132 | |
| 1102 | − if (pipe == AMDGPU_MES_KIQ_PIPE) | |
| 1133 | + if (!adev->enable_uni_mes && pipe == AMDGPU_MES_KIQ_PIPE) | |
| 1103 | 1134 | ring = &adev->gfx.kiq[0].ring; |
| 1104 | − else if (pipe == AMDGPU_MES_SCHED_PIPE) | |
| 1105 | − ring = &adev->mes.ring; | |
| 1106 | 1135 | else |
| 1107 | − BUG(); | |
| 1136 | + ring = &adev->mes.ring[pipe]; | |
| 1108 | 1137 | |
| 1109 | − if ((pipe == AMDGPU_MES_SCHED_PIPE) && | |
| 1138 | + if ((adev->enable_uni_mes || pipe == AMDGPU_MES_SCHED_PIPE) && | |
| 1110 | 1139 | (amdgpu_in_reset(adev) || adev->in_suspend)) { |
| 1111 | 1140 | *(ring->wptr_cpu_addr) = 0; |
| 1112 | 1141 | *(ring->rptr_cpu_addr) = 0; |
| @@ -1118,13 +1147,12 @@ static int mes_v12_0_queue_init(struct amdgpu_device *adev, · 5 unmodified lines | ||
| 1118 | 1147 | return r; |
| 1119 | 1148 | |
| 1120 | 1149 | if (pipe == AMDGPU_MES_SCHED_PIPE) { |
| 1121 | − if (adev->enable_uni_mes) { | |
| 1122 | − mes_v12_0_queue_init_register(ring); | |
| 1123 | − } else { | |
| 1150 | + if (adev->enable_uni_mes) | |
| 1151 | + r = amdgpu_mes_map_legacy_queue(adev, ring); | |
| 1152 | + else | |
| 1124 | 1153 | r = mes_v12_0_kiq_enable_queue(adev); |
| 1125 | − if (r) | |
| 1126 | − return r; | |
| 1127 | − } | |
| 1154 | + if (r) | |
| 1155 | + return r; | |
| 1128 | 1156 | } else { |
| 1129 | 1157 | mes_v12_0_queue_init_register(ring); |
| 1130 | 1158 | } |
| @@ -1144,25 +1172,29 @@ static int mes_v12_0_queue_init(struct amdgpu_device *adev, · 13 unmodified lines | ||
| 1144 | 1172 | return 0; |
| 1145 | 1173 | } |
| 1146 | 1174 | |
| 1147 | −static int mes_v12_0_ring_init(struct amdgpu_device *adev) | |
| 1175 | +static int mes_v12_0_ring_init(struct amdgpu_device *adev, int pipe) | |
| 1148 | 1176 | { |
| 1149 | 1177 | struct amdgpu_ring *ring; |
| 1150 | 1178 | |
| 1151 | − ring = &adev->mes.ring; | |
| 1179 | + ring = &adev->mes.ring[pipe]; | |
| 1152 | 1180 | |
| 1153 | 1181 | ring->funcs = &mes_v12_0_ring_funcs; |
| 1154 | 1182 | |
| 1155 | 1183 | ring->me = 3; |
| 1156 | − ring->pipe = 0; | |
| 1184 | + ring->pipe = pipe; | |
| 1157 | 1185 | ring->queue = 0; |
| 1158 | 1186 | |
| 1159 | 1187 | ring->ring_obj = NULL; |
| 1160 | 1188 | ring->use_doorbell = true; |
| 1161 | − ring->doorbell_index = adev->doorbell_index.mes_ring0 << 1; | |
| 1162 | − ring->eop_gpu_addr = adev->mes.eop_gpu_addr[AMDGPU_MES_SCHED_PIPE]; | |
| 1189 | + ring->eop_gpu_addr = adev->mes.eop_gpu_addr[pipe]; | |
| 1163 | 1190 | ring->no_scheduler = true; |
| 1164 | 1191 | sprintf(ring->name, "mes_%d.%d.%d", ring->me, ring->pipe, ring->queue); |
| 1165 | 1192 | |
| 1193 | + if (pipe == AMDGPU_MES_SCHED_PIPE) | |
| 1194 | + ring->doorbell_index = adev->doorbell_index.mes_ring0 << 1; | |
| 1195 | + else | |
| 1196 | + ring->doorbell_index = adev->doorbell_index.mes_ring1 << 1; | |
| 1197 | + | |
| 1166 | 1198 | return amdgpu_ring_init(adev, ring, 1024, NULL, 0, |
| 1167 | 1199 | AMDGPU_RING_PRIO_DEFAULT, NULL); |
| 1168 | 1200 | } |
| @@ -1176,7 +1208,7 @@ static int mes_v12_0_kiq_ring_init(struct amdgpu_device *adev) · 7 unmodified lines | ||
| 1176 | 1208 | ring = &adev->gfx.kiq[0].ring; |
| 1177 | 1209 | |
| 1178 | 1210 | ring->me = 3; |
| 1179 | − ring->pipe = adev->enable_uni_mes ? 0 : 1; | |
| 1211 | + ring->pipe = 1; | |
| 1180 | 1212 | ring->queue = 0; |
| 1181 | 1213 | |
| 1182 | 1214 | ring->adev = NULL; |
| @@ -1198,12 +1230,10 @@ static int mes_v12_0_mqd_sw_init(struct amdgpu_device *adev, · 15 unmodified lines | ||
| 1198 | 1230 | int r, mqd_size = sizeof(struct v12_compute_mqd); |
| 1199 | 1231 | struct amdgpu_ring *ring; |
| 1200 | 1232 | |
| 1201 | − if (pipe == AMDGPU_MES_KIQ_PIPE) | |
| 1233 | + if (!adev->enable_uni_mes && pipe == AMDGPU_MES_KIQ_PIPE) | |
| 1202 | 1234 | ring = &adev->gfx.kiq[0].ring; |
| 1203 | − else if (pipe == AMDGPU_MES_SCHED_PIPE) | |
| 1204 | − ring = &adev->mes.ring; | |
| 1205 | 1235 | else |
| 1206 | − BUG(); | |
| 1236 | + ring = &adev->mes.ring[pipe]; | |
| 1207 | 1237 | |
| 1208 | 1238 | if (ring->mqd_obj) |
| 1209 | 1239 | return 0; |
| @@ -1237,14 +1267,13 @@ static int mes_v12_0_sw_init(void *handle) · 27 unmodified lines | ||
| 1237 | 1267 | adev->mes.kiq_hw_init = &mes_v12_0_kiq_hw_init; |
| 1238 | 1268 | adev->mes.kiq_hw_fini = &mes_v12_0_kiq_hw_fini; |
| 1239 | 1269 | |
| 1270 | + adev->mes.event_log_size = AMDGPU_MES_LOG_BUFFER_SIZE; | |
| 1271 | + | |
| 1240 | 1272 | r = amdgpu_mes_init(adev); |
| 1241 | 1273 | if (r) |
| 1242 | 1274 | return r; |
| 1243 | 1275 | |
| 1244 | 1276 | for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) { |
| 1245 | − if (!adev->enable_mes_kiq && pipe == AMDGPU_MES_KIQ_PIPE) | |
| 1246 | − continue; | |
| 1247 | − | |
| 1248 | 1277 | r = mes_v12_0_allocate_eop_buf(adev, pipe); |
| 1249 | 1278 | if (r) |
| 1250 | 1279 | return r; |
| @@ -1252,18 +1281,15 @@ static int mes_v12_0_sw_init(void *handle) · 1 unmodified line | ||
| 1252 | 1281 | r = mes_v12_0_mqd_sw_init(adev, pipe); |
| 1253 | 1282 | if (r) |
| 1254 | 1283 | return r; |
| 1255 | − } | |
| 1256 | 1284 | |
| 1257 | − if (adev->enable_mes_kiq) { | |
| 1258 | − r = mes_v12_0_kiq_ring_init(adev); | |
| 1285 | + if (!adev->enable_uni_mes && pipe == AMDGPU_MES_KIQ_PIPE) | |
| 1286 | + r = mes_v12_0_kiq_ring_init(adev); | |
| 1287 | + else | |
| 1288 | + r = mes_v12_0_ring_init(adev, pipe); | |
| 1259 | 1289 | if (r) |
| 1260 | 1290 | return r; |
| 1261 | 1291 | } |
| 1262 | 1292 | |
| 1263 | − r = mes_v12_0_ring_init(adev); | |
| 1264 | − if (r) | |
| 1265 | − return r; | |
| 1266 | − | |
| 1267 | 1293 | return 0; |
| 1268 | 1294 | } |
| 1269 | 1295 | |
| @@ -1272,9 +1298,6 @@ static int mes_v12_0_sw_fini(void *handle) · 2 unmodified lines | ||
| 1272 | 1298 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1273 | 1299 | int pipe; |
| 1274 | 1300 | |
| 1275 | − amdgpu_device_wb_free(adev, adev->mes.sch_ctx_offs); | |
| 1276 | − amdgpu_device_wb_free(adev, adev->mes.query_status_fence_offs); | |
| 1277 | − | |
| 1278 | 1301 | for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) { |
| 1279 | 1302 | kfree(adev->mes.mqd_backup[pipe]); |
| 1280 | 1303 | |
| @@ -1282,18 +1305,21 @@ static int mes_v12_0_sw_fini(void *handle) · 1 unmodified line | ||
| 1282 | 1305 | &adev->mes.eop_gpu_addr[pipe], |
| 1283 | 1306 | NULL); |
| 1284 | 1307 | amdgpu_ucode_release(&adev->mes.fw[pipe]); |
| 1285 | − } | |
| 1286 | 1308 | |
| 1287 | − amdgpu_bo_free_kernel(&adev->gfx.kiq[0].ring.mqd_obj, | |
| 1288 | − &adev->gfx.kiq[0].ring.mqd_gpu_addr, | |
| 1289 | − &adev->gfx.kiq[0].ring.mqd_ptr); | |
| 1290 | − | |
| 1291 | − amdgpu_bo_free_kernel(&adev->mes.ring.mqd_obj, | |
| 1292 | − &adev->mes.ring.mqd_gpu_addr, | |
| 1293 | − &adev->mes.ring.mqd_ptr); | |
| 1309 | + if (adev->enable_uni_mes || pipe == AMDGPU_MES_SCHED_PIPE) { | |
| 1310 | + amdgpu_bo_free_kernel(&adev->mes.ring[pipe].mqd_obj, | |
| 1311 | + &adev->mes.ring[pipe].mqd_gpu_addr, | |
| 1312 | + &adev->mes.ring[pipe].mqd_ptr); | |
| 1313 | + amdgpu_ring_fini(&adev->mes.ring[pipe]); | |
| 1314 | + } | |
| 1315 | + } | |
| 1294 | 1316 | |
| 1295 | − amdgpu_ring_fini(&adev->gfx.kiq[0].ring); | |
| 1296 | − amdgpu_ring_fini(&adev->mes.ring); | |
| 1317 | + if (!adev->enable_uni_mes) { | |
| 1318 | + amdgpu_bo_free_kernel(&adev->gfx.kiq[0].ring.mqd_obj, | |
| 1319 | + &adev->gfx.kiq[0].ring.mqd_gpu_addr, | |
| 1320 | + &adev->gfx.kiq[0].ring.mqd_ptr); | |
| 1321 | + amdgpu_ring_fini(&adev->gfx.kiq[0].ring); | |
| 1322 | + } | |
| 1297 | 1323 | |
| 1298 | 1324 | if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) { |
| 1299 | 1325 | mes_v12_0_free_ucode_buffers(adev, AMDGPU_MES_KIQ_PIPE); |
| @@ -1337,7 +1363,7 @@ static void mes_v12_0_kiq_dequeue_sched(struct amdgpu_device *adev) · 37 unmodified lines | ||
| 1337 | 1363 | soc21_grbm_select(adev, 0, 0, 0, 0); |
| 1338 | 1364 | mutex_unlock(&adev->srbm_mutex); |
| 1339 | 1365 | |
| 1340 | − adev->mes.ring.sched.ready = false; | |
| 1366 | + adev->mes.ring[0].sched.ready = false; | |
| 1341 | 1367 | } |
| 1342 | 1368 | |
| 1343 | 1369 | static void mes_v12_0_kiq_setting(struct amdgpu_ring *ring) |
| @@ -1358,10 +1384,10 @@ static int mes_v12_0_kiq_hw_init(struct amdgpu_device *adev) · 14 unmodified lines | ||
| 1358 | 1384 | { |
| 1359 | 1385 | int r = 0; |
| 1360 | 1386 | |
| 1361 | − mes_v12_0_kiq_setting(&adev->gfx.kiq[0].ring); | |
| 1362 | − | |
| 1363 | 1387 | if (adev->enable_uni_mes) |
| 1364 | − return mes_v12_0_hw_init(adev); | |
| 1388 | + mes_v12_0_kiq_setting(&adev->mes.ring[AMDGPU_MES_KIQ_PIPE]); | |
| 1389 | + else | |
| 1390 | + mes_v12_0_kiq_setting(&adev->gfx.kiq[0].ring); | |
| 1365 | 1391 | |
| 1366 | 1392 | if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) { |
| 1367 | 1393 | |
| @@ -1388,6 +1414,14 @@ static int mes_v12_0_kiq_hw_init(struct amdgpu_device *adev) · 20 unmodified lines | ||
| 1388 | 1414 | if (r) |
| 1389 | 1415 | goto failure; |
| 1390 | 1416 | |
| 1417 | + if (adev->enable_uni_mes) { | |
| 1418 | + r = mes_v12_0_set_hw_resources(&adev->mes, AMDGPU_MES_KIQ_PIPE); | |
| 1419 | + if (r) | |
| 1420 | + goto failure; | |
| 1421 | + | |
| 1422 | + mes_v12_0_set_hw_resources_1(&adev->mes, AMDGPU_MES_KIQ_PIPE); | |
| 1423 | + } | |
| 1424 | + | |
| 1391 | 1425 | r = mes_v12_0_hw_init(adev); |
| 1392 | 1426 | if (r) |
| 1393 | 1427 | goto failure; |
| @@ -1401,9 +1435,15 @@ failure: · 7 unmodified lines | ||
| 1401 | 1435 | |
| 1402 | 1436 | static int mes_v12_0_kiq_hw_fini(struct amdgpu_device *adev) |
| 1403 | 1437 | { |
| 1404 | − if (adev->mes.ring.sched.ready) { | |
| 1405 | − mes_v12_0_kiq_dequeue_sched(adev); | |
| 1406 | − adev->mes.ring.sched.ready = false; | |
| 1438 | + if (adev->mes.ring[0].sched.ready) { | |
| 1439 | + if (adev->enable_uni_mes) | |
| 1440 | + amdgpu_mes_unmap_legacy_queue(adev, | |
| 1441 | + &adev->mes.ring[AMDGPU_MES_SCHED_PIPE], | |
| 1442 | + RESET_QUEUES, 0, 0); | |
| 1443 | + else | |
| 1444 | + mes_v12_0_kiq_dequeue_sched(adev); | |
| 1445 | + | |
| 1446 | + adev->mes.ring[0].sched.ready = false; | |
| 1407 | 1447 | } |
| 1408 | 1448 | |
| 1409 | 1449 | mes_v12_0_enable(adev, false); |
| @@ -1416,10 +1456,10 @@ static int mes_v12_0_hw_init(void *handle) · 6 unmodified lines | ||
| 1416 | 1456 | int r; |
| 1417 | 1457 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1418 | 1458 | |
| 1419 | − if (adev->mes.ring.sched.ready) | |
| 1459 | + if (adev->mes.ring[0].sched.ready) | |
| 1420 | 1460 | goto out; |
| 1421 | 1461 | |
| 1422 | − if (!adev->enable_mes_kiq || adev->enable_uni_mes) { | |
| 1462 | + if (!adev->enable_mes_kiq) { | |
| 1423 | 1463 | if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT) { |
| 1424 | 1464 | r = mes_v12_0_load_microcode(adev, |
| 1425 | 1465 | AMDGPU_MES_SCHED_PIPE, true); |
| @@ -1439,23 +1479,23 @@ static int mes_v12_0_hw_init(void *handle) · 13 unmodified lines | ||
| 1439 | 1479 | mes_v12_0_enable(adev, true); |
| 1440 | 1480 | } |
| 1441 | 1481 | |
| 1482 | + /* Enable the MES to handle doorbell ring on unmapped queue */ | |
| 1483 | + mes_v12_0_enable_unmapped_doorbell_handling(&adev->mes, true); | |
| 1484 | + | |
| 1442 | 1485 | r = mes_v12_0_queue_init(adev, AMDGPU_MES_SCHED_PIPE); |
| 1443 | 1486 | if (r) |
| 1444 | 1487 | goto failure; |
| 1445 | 1488 | |
| 1446 | − r = mes_v12_0_set_hw_resources(&adev->mes); | |
| 1489 | + r = mes_v12_0_set_hw_resources(&adev->mes, AMDGPU_MES_SCHED_PIPE); | |
| 1447 | 1490 | if (r) |
| 1448 | 1491 | goto failure; |
| 1449 | 1492 | |
| 1450 | 1493 | if (adev->enable_uni_mes) |
| 1451 | − mes_v12_0_set_hw_resources_1(&adev->mes); | |
| 1494 | + mes_v12_0_set_hw_resources_1(&adev->mes, AMDGPU_MES_SCHED_PIPE); | |
| 1452 | 1495 | |
| 1453 | 1496 | mes_v12_0_init_aggregated_doorbell(&adev->mes); |
| 1454 | 1497 | |
| 1455 | − /* Enable the MES to handle doorbell ring on unmapped queue */ | |
| 1456 | − mes_v12_0_enable_unmapped_doorbell_handling(&adev->mes, true); | |
| 1457 | − | |
| 1458 | − r = mes_v12_0_query_sched_status(&adev->mes); | |
| 1498 | + r = mes_v12_0_query_sched_status(&adev->mes, AMDGPU_MES_SCHED_PIPE); | |
| 1459 | 1499 | if (r) { |
| 1460 | 1500 | DRM_ERROR("MES is busy\n"); |
| 1461 | 1501 | goto failure; |
| @@ -1468,7 +1508,7 @@ out: · 6 unmodified lines | ||
| 1468 | 1508 | * with MES enabled. |
| 1469 | 1509 | */ |
| 1470 | 1510 | adev->gfx.kiq[0].ring.sched.ready = false; |
| 1471 | − adev->mes.ring.sched.ready = true; | |
| 1511 | + adev->mes.ring[0].sched.ready = true; | |
| 1472 | 1512 | |
| 1473 | 1513 | return 0; |
| 1474 | 1514 | |
| @@ -1511,17 +1551,7 @@ static int mes_v12_0_early_init(void *handle) · 36 unmodified lines | ||
| 1511 | 1551 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1512 | 1552 | int pipe, r; |
| 1513 | 1553 | |
| 1514 | − if (adev->enable_uni_mes) { | |
| 1515 | − r = amdgpu_mes_init_microcode(adev, AMDGPU_MES_SCHED_PIPE); | |
| 1516 | − if (!r) | |
| 1517 | − return 0; | |
| 1518 | − | |
| 1519 | − adev->enable_uni_mes = false; | |
| 1520 | − } | |
| 1521 | − | |
| 1522 | 1554 | for (pipe = 0; pipe < AMDGPU_MAX_MES_PIPES; pipe++) { |
| 1523 | − if (!adev->enable_mes_kiq && pipe == AMDGPU_MES_KIQ_PIPE) | |
| 1524 | − continue; | |
| 1525 | 1555 | r = amdgpu_mes_init_microcode(adev, pipe); |
| 1526 | 1556 | if (r) |
| 1527 | 1557 | return r; |
| @@ -80,7 +80,8 @@ static uint32_t mmhub_v4_1_0_get_invalidate_req(unsigned int vmid, · 79 unmodified lines | ||
| 80 | 80 | /* invalidate using legacy mode on vmid*/ |
| 81 | 81 | req = REG_SET_FIELD(req, MMVM_INVALIDATE_ENG0_REQ, |
| 82 | 82 | PER_VMID_INVALIDATE_REQ, 1 << vmid); |
| 83 | − req = REG_SET_FIELD(req, MMVM_INVALIDATE_ENG0_REQ, FLUSH_TYPE, flush_type); | |
| 83 | + /* Only use legacy inv on mmhub side */ | |
| 84 | + req = REG_SET_FIELD(req, MMVM_INVALIDATE_ENG0_REQ, FLUSH_TYPE, 0); | |
| 84 | 85 | req = REG_SET_FIELD(req, MMVM_INVALIDATE_ENG0_REQ, INVALIDATE_L2_PTES, 1); |
| 85 | 86 | req = REG_SET_FIELD(req, MMVM_INVALIDATE_ENG0_REQ, INVALIDATE_L2_PDE0, 1); |
| 86 | 87 | req = REG_SET_FIELD(req, MMVM_INVALIDATE_ENG0_REQ, INVALIDATE_L2_PDE1, 1); |
| @@ -176,14 +176,16 @@ static void sdma_v5_2_ring_set_wptr(struct amdgpu_ring *ring) · 175 unmodified lines | ||
| 176 | 176 | DRM_DEBUG("calling WDOORBELL64(0x%08x, 0x%016llx)\n", |
| 177 | 177 | ring->doorbell_index, ring->wptr << 2); |
| 178 | 178 | WDOORBELL64(ring->doorbell_index, ring->wptr << 2); |
| 179 | − /* SDMA seems to miss doorbells sometimes when powergating kicks in. | |
| 180 | − * Updating the wptr directly will wake it. This is only safe because | |
| 181 | − * we disallow gfxoff in begin_use() and then allow it again in end_use(). | |
| 182 | − */ | |
| 183 | − WREG32(sdma_v5_2_get_reg_offset(adev, ring->me, mmSDMA0_GFX_RB_WPTR), | |
| 184 | − lower_32_bits(ring->wptr << 2)); | |
| 185 | − WREG32(sdma_v5_2_get_reg_offset(adev, ring->me, mmSDMA0_GFX_RB_WPTR_HI), | |
| 186 | − upper_32_bits(ring->wptr << 2)); | |
| 179 | + if (amdgpu_ip_version(adev, SDMA0_HWIP, 0) == IP_VERSION(5, 2, 1)) { | |
| 180 | + /* SDMA seems to miss doorbells sometimes when powergating kicks in. | |
| 181 | + * Updating the wptr directly will wake it. This is only safe because | |
| 182 | + * we disallow gfxoff in begin_use() and then allow it again in end_use(). | |
| 183 | + */ | |
| 184 | + WREG32(sdma_v5_2_get_reg_offset(adev, ring->me, mmSDMA0_GFX_RB_WPTR), | |
| 185 | + lower_32_bits(ring->wptr << 2)); | |
| 186 | + WREG32(sdma_v5_2_get_reg_offset(adev, ring->me, mmSDMA0_GFX_RB_WPTR_HI), | |
| 187 | + upper_32_bits(ring->wptr << 2)); | |
| 188 | + } | |
| 187 | 189 | } else { |
| 188 | 190 | DRM_DEBUG("Not using doorbell -- " |
| 189 | 191 | "mmSDMA%i_GFX_RB_WPTR == 0x%08x " |
| @@ -1575,8 +1575,7 @@ static void sdma_v7_0_emit_copy_buffer(struct amdgpu_ib *ib, · 1574 unmodified lines | ||
| 1575 | 1575 | ib->ptr[ib->length_dw++] = SDMA_PKT_COPY_LINEAR_HEADER_OP(SDMA_OP_COPY) | |
| 1576 | 1576 | SDMA_PKT_COPY_LINEAR_HEADER_SUB_OP(SDMA_SUBOP_COPY_LINEAR) | |
| 1577 | 1577 | SDMA_PKT_COPY_LINEAR_HEADER_TMZ((copy_flags & AMDGPU_COPY_FLAGS_TMZ) ? 1 : 0) | |
| 1578 | − SDMA_PKT_COPY_LINEAR_HEADER_CPV((copy_flags & | |
| 1579 | − (AMDGPU_COPY_FLAGS_READ_DECOMPRESSED | AMDGPU_COPY_FLAGS_WRITE_COMPRESSED)) ? 1 : 0); | |
| 1578 | + SDMA_PKT_COPY_LINEAR_HEADER_CPV(1); | |
| 1580 | 1579 | |
| 1581 | 1580 | ib->ptr[ib->length_dw++] = byte_count - 1; |
| 1582 | 1581 | ib->ptr[ib->length_dw++] = 0; /* src/dst endian swap */ |
| @@ -1590,6 +1589,8 @@ static void sdma_v7_0_emit_copy_buffer(struct amdgpu_ib *ib, · 7 unmodified lines | ||
| 1590 | 1589 | ((copy_flags & AMDGPU_COPY_FLAGS_READ_DECOMPRESSED) ? SDMA_DCC_READ_CM(2) : 0) | |
| 1591 | 1590 | ((copy_flags & AMDGPU_COPY_FLAGS_WRITE_COMPRESSED) ? SDMA_DCC_WRITE_CM(1) : 0) | |
| 1592 | 1591 | SDMA_DCC_MAX_COM(max_com) | SDMA_DCC_MAX_UCOM(1); |
| 1592 | + else | |
| 1593 | + ib->ptr[ib->length_dw++] = 0; | |
| 1593 | 1594 | } |
| 1594 | 1595 | |
| 1595 | 1596 | /** |
| @@ -1616,7 +1617,7 @@ static void sdma_v7_0_emit_fill_buffer(struct amdgpu_ib *ib, · 20 unmodified lines | ||
| 1616 | 1617 | |
| 1617 | 1618 | static const struct amdgpu_buffer_funcs sdma_v7_0_buffer_funcs = { |
| 1618 | 1619 | .copy_max_bytes = 0x400000, |
| 1619 | − .copy_num_dw = 7, | |
| 1620 | + .copy_num_dw = 8, | |
| 1620 | 1621 | .emit_copy_buffer = sdma_v7_0_emit_copy_buffer, |
| 1621 | 1622 | .fill_max_bytes = 0x400000, |
| 1622 | 1623 | .fill_num_dw = 5, |
| @@ -76,6 +76,12 @@ · 75 unmodified lines | ||
| 76 | 76 | ((cond & 0xF) << 24) | \ |
| 77 | 77 | ((type & 0xF) << 28)) |
| 78 | 78 | |
| 79 | +#define CP_PACKETJ_NOP 0x60000000 | |
| 80 | +#define CP_PACKETJ_GET_REG(x) ((x) & 0x3FFFF) | |
| 81 | +#define CP_PACKETJ_GET_RES(x) (((x) >> 18) & 0x3F) | |
| 82 | +#define CP_PACKETJ_GET_COND(x) (((x) >> 24) & 0xF) | |
| 83 | +#define CP_PACKETJ_GET_TYPE(x) (((x) >> 28) & 0xF) | |
| 84 | + | |
| 79 | 85 | /* Packet 3 types */ |
| 80 | 86 | #define PACKET3_NOP 0x10 |
| 81 | 87 | #define PACKET3_SET_BASE 0x11 |
| @@ -406,6 +406,7 @@ static int soc24_common_early_init(void *handle) · 405 unmodified lines | ||
| 406 | 406 | AMD_CG_SUPPORT_ATHUB_MGCG | |
| 407 | 407 | AMD_CG_SUPPORT_ATHUB_LS | |
| 408 | 408 | AMD_CG_SUPPORT_MC_MGCG | |
| 409 | + AMD_CG_SUPPORT_HDP_SD | | |
| 409 | 410 | AMD_CG_SUPPORT_MC_LS; |
| 410 | 411 | adev->pg_flags = AMD_PG_SUPPORT_VCN | |
| 411 | 412 | AMD_PG_SUPPORT_JPEG | |
| @@ -424,6 +425,7 @@ static int soc24_common_early_init(void *handle) · 12 unmodified lines | ||
| 424 | 425 | AMD_CG_SUPPORT_ATHUB_MGCG | |
| 425 | 426 | AMD_CG_SUPPORT_ATHUB_LS | |
| 426 | 427 | AMD_CG_SUPPORT_MC_MGCG | |
| 428 | + AMD_CG_SUPPORT_HDP_SD | | |
| 427 | 429 | AMD_CG_SUPPORT_MC_LS; |
| 428 | 430 | |
| 429 | 431 | adev->pg_flags = AMD_PG_SUPPORT_VCN | |
| @@ -187,6 +187,7 @@ static void update_mqd(struct mqd_manager *mm, void *mqd, · 186 unmodified lines | ||
| 187 | 187 | m->cp_hqd_pq_control = 5 << CP_HQD_PQ_CONTROL__RPTR_BLOCK_SIZE__SHIFT; |
| 188 | 188 | m->cp_hqd_pq_control |= |
| 189 | 189 | ffs(q->queue_size / sizeof(unsigned int)) - 1 - 1; |
| 190 | + m->cp_hqd_pq_control |= CP_HQD_PQ_CONTROL__UNORD_DISPATCH_MASK; | |
| 190 | 191 | pr_debug("cp_hqd_pq_control 0x%x\n", m->cp_hqd_pq_control); |
| 191 | 192 | |
| 192 | 193 | m->cp_hqd_pq_base_lo = lower_32_bits((uint64_t)q->queue_address >> 8); |
| @@ -2893,6 +2893,9 @@ static int dm_suspend(void *handle) · 2892 unmodified lines | ||
| 2893 | 2893 | |
| 2894 | 2894 | hpd_rx_irq_work_suspend(dm); |
| 2895 | 2895 | |
| 2896 | + if (adev->dm.dc->caps.ips_support) | |
| 2897 | + dc_allow_idle_optimizations(adev->dm.dc, true); | |
| 2898 | + | |
| 2896 | 2899 | dc_set_power_state(dm->dc, DC_ACPI_CM_POWER_STATE_D3); |
| 2897 | 2900 | dc_dmub_srv_set_power_state(dm->dc->ctx->dmub_srv, DC_ACPI_CM_POWER_STATE_D3); |
| 2898 | 2901 | |
| @@ -804,12 +804,25 @@ struct dsc_mst_fairness_params { · 803 unmodified lines | ||
| 804 | 804 | }; |
| 805 | 805 | |
| 806 | 806 | #if defined(CONFIG_DRM_AMD_DC_FP) |
| 807 | −static int kbps_to_peak_pbn(int kbps) | |
| 807 | +static uint16_t get_fec_overhead_multiplier(struct dc_link *dc_link) | |
| 808 | +{ | |
| 809 | + u8 link_coding_cap; | |
| 810 | + uint16_t fec_overhead_multiplier_x1000 = PBN_FEC_OVERHEAD_MULTIPLIER_8B_10B; | |
| 811 | + | |
| 812 | + link_coding_cap = dc_link_dp_mst_decide_link_encoding_format(dc_link); | |
| 813 | + if (link_coding_cap == DP_128b_132b_ENCODING) | |
| 814 | + fec_overhead_multiplier_x1000 = PBN_FEC_OVERHEAD_MULTIPLIER_128B_132B; | |
| 815 | + | |
| 816 | + return fec_overhead_multiplier_x1000; | |
| 817 | +} | |
| 818 | + | |
| 819 | +static int kbps_to_peak_pbn(int kbps, uint16_t fec_overhead_multiplier_x1000) | |
| 808 | 820 | { |
| 809 | 821 | u64 peak_kbps = kbps; |
| 810 | 822 | |
| 811 | 823 | peak_kbps *= 1006; |
| 812 | − peak_kbps = div_u64(peak_kbps, 1000); | |
| 824 | + peak_kbps *= fec_overhead_multiplier_x1000; | |
| 825 | + peak_kbps = div_u64(peak_kbps, 1000 * 1000); | |
| 813 | 826 | return (int) DIV64_U64_ROUND_UP(peak_kbps * 64, (54 * 8 * 1000)); |
| 814 | 827 | } |
| 815 | 828 | |
| @@ -910,11 +923,12 @@ static int increase_dsc_bpp(struct drm_atomic_state *state, · 94 unmodified lines | ||
| 910 | 923 | int link_timeslots_used; |
| 911 | 924 | int fair_pbn_alloc; |
| 912 | 925 | int ret = 0; |
| 926 | + uint16_t fec_overhead_multiplier_x1000 = get_fec_overhead_multiplier(dc_link); | |
| 913 | 927 | |
| 914 | 928 | for (i = 0; i < count; i++) { |
| 915 | 929 | if (vars[i + k].dsc_enabled) { |
| 916 | 930 | initial_slack[i] = |
| 917 | − kbps_to_peak_pbn(params[i].bw_range.max_kbps) - vars[i + k].pbn; | |
| 931 | + kbps_to_peak_pbn(params[i].bw_range.max_kbps, fec_overhead_multiplier_x1000) - vars[i + k].pbn; | |
| 918 | 932 | bpp_increased[i] = false; |
| 919 | 933 | remaining_to_increase += 1; |
| 920 | 934 | } else { |
| @@ -1010,6 +1024,7 @@ static int try_disable_dsc(struct drm_atomic_state *state, · 89 unmodified lines | ||
| 1010 | 1024 | int next_index; |
| 1011 | 1025 | int remaining_to_try = 0; |
| 1012 | 1026 | int ret; |
| 1027 | + uint16_t fec_overhead_multiplier_x1000 = get_fec_overhead_multiplier(dc_link); | |
| 1013 | 1028 | |
| 1014 | 1029 | for (i = 0; i < count; i++) { |
| 1015 | 1030 | if (vars[i + k].dsc_enabled |
| @@ -1039,7 +1054,7 @@ static int try_disable_dsc(struct drm_atomic_state *state, · 23 unmodified lines | ||
| 1039 | 1054 | if (next_index == -1) |
| 1040 | 1055 | break; |
| 1041 | 1056 | |
| 1042 | − vars[next_index].pbn = kbps_to_peak_pbn(params[next_index].bw_range.stream_kbps); | |
| 1057 | + vars[next_index].pbn = kbps_to_peak_pbn(params[next_index].bw_range.stream_kbps, fec_overhead_multiplier_x1000); | |
| 1043 | 1058 | ret = drm_dp_atomic_find_time_slots(state, |
| 1044 | 1059 | params[next_index].port->mgr, |
| 1045 | 1060 | params[next_index].port, |
| @@ -1052,8 +1067,7 @@ static int try_disable_dsc(struct drm_atomic_state *state, · 6 unmodified lines | ||
| 1052 | 1067 | vars[next_index].dsc_enabled = false; |
| 1053 | 1068 | vars[next_index].bpp_x16 = 0; |
| 1054 | 1069 | } else { |
| 1055 | − vars[next_index].pbn = kbps_to_peak_pbn( | |
| 1056 | − params[next_index].bw_range.max_kbps); | |
| 1070 | + vars[next_index].pbn = kbps_to_peak_pbn(params[next_index].bw_range.stream_kbps, fec_overhead_multiplier_x1000); | |
| 1057 | 1071 | ret = drm_dp_atomic_find_time_slots(state, |
| 1058 | 1072 | params[next_index].port->mgr, |
| 1059 | 1073 | params[next_index].port, |
| @@ -1082,6 +1096,7 @@ static int compute_mst_dsc_configs_for_link(struct drm_atomic_state *state, · 22 unmodified lines | ||
| 1082 | 1096 | int count = 0; |
| 1083 | 1097 | int i, k, ret; |
| 1084 | 1098 | bool debugfs_overwrite = false; |
| 1099 | + uint16_t fec_overhead_multiplier_x1000 = get_fec_overhead_multiplier(dc_link); | |
| 1085 | 1100 | |
| 1086 | 1101 | memset(params, 0, sizeof(params)); |
| 1087 | 1102 | |
| @@ -1146,7 +1161,7 @@ static int compute_mst_dsc_configs_for_link(struct drm_atomic_state *state, · 58 unmodified lines | ||
| 1146 | 1161 | /* Try no compression */ |
| 1147 | 1162 | for (i = 0; i < count; i++) { |
| 1148 | 1163 | vars[i + k].aconnector = params[i].aconnector; |
| 1149 | − vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.stream_kbps); | |
| 1164 | + vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.stream_kbps, fec_overhead_multiplier_x1000); | |
| 1150 | 1165 | vars[i + k].dsc_enabled = false; |
| 1151 | 1166 | vars[i + k].bpp_x16 = 0; |
| 1152 | 1167 | ret = drm_dp_atomic_find_time_slots(state, params[i].port->mgr, params[i].port, |
| @@ -1165,7 +1180,7 @@ static int compute_mst_dsc_configs_for_link(struct drm_atomic_state *state, · 12 unmodified lines | ||
| 1165 | 1180 | /* Try max compression */ |
| 1166 | 1181 | for (i = 0; i < count; i++) { |
| 1167 | 1182 | if (params[i].compression_possible && params[i].clock_force_enable != DSC_CLK_FORCE_DISABLE) { |
| 1168 | − vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.min_kbps); | |
| 1183 | + vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.min_kbps, fec_overhead_multiplier_x1000); | |
| 1169 | 1184 | vars[i + k].dsc_enabled = true; |
| 1170 | 1185 | vars[i + k].bpp_x16 = params[i].bw_range.min_target_bpp_x16; |
| 1171 | 1186 | ret = drm_dp_atomic_find_time_slots(state, params[i].port->mgr, |
| @@ -1173,7 +1188,7 @@ static int compute_mst_dsc_configs_for_link(struct drm_atomic_state *state, · 1 unmodified line | ||
| 1173 | 1188 | if (ret < 0) |
| 1174 | 1189 | return ret; |
| 1175 | 1190 | } else { |
| 1176 | − vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.stream_kbps); | |
| 1191 | + vars[i + k].pbn = kbps_to_peak_pbn(params[i].bw_range.stream_kbps, fec_overhead_multiplier_x1000); | |
| 1177 | 1192 | vars[i + k].dsc_enabled = false; |
| 1178 | 1193 | vars[i + k].bpp_x16 = 0; |
| 1179 | 1194 | ret = drm_dp_atomic_find_time_slots(state, params[i].port->mgr, |
| @@ -1270,6 +1285,9 @@ static bool is_dsc_need_re_compute( · 90 unmodified lines | ||
| 1270 | 1285 | } |
| 1271 | 1286 | } |
| 1272 | 1287 | |
| 1288 | + if (new_stream_on_link_num == 0) | |
| 1289 | + return false; | |
| 1290 | + | |
| 1273 | 1291 | /* check current_state if there stream on link but it is not in |
| 1274 | 1292 | * new request state |
| 1275 | 1293 | */ |
| @@ -46,6 +46,9 @@ · 45 unmodified lines | ||
| 46 | 46 | #define SYNAPTICS_CASCADED_HUB_ID 0x5A |
| 47 | 47 | #define IS_SYNAPTICS_CASCADED_PANAMERA(devName, data) ((IS_SYNAPTICS_PANAMERA(devName) && ((int)data[2] == SYNAPTICS_CASCADED_HUB_ID)) ? 1 : 0) |
| 48 | 48 | |
| 49 | +#define PBN_FEC_OVERHEAD_MULTIPLIER_8B_10B 1031 | |
| 50 | +#define PBN_FEC_OVERHEAD_MULTIPLIER_128B_132B 1000 | |
| 51 | + | |
| 49 | 52 | enum mst_msg_ready_type { |
| 50 | 53 | NONE_MSG_RDY_EVENT = 0, |
| 51 | 54 | DOWN_REP_MSG_RDY_EVENT = 1, |
| @@ -28,6 +28,7 @@ · 27 unmodified lines | ||
| 28 | 28 | #include <drm/drm_blend.h> |
| 29 | 29 | #include <drm/drm_gem_atomic_helper.h> |
| 30 | 30 | #include <drm/drm_plane_helper.h> |
| 31 | +#include <drm/drm_gem_framebuffer_helper.h> | |
| 31 | 32 | #include <drm/drm_fourcc.h> |
| 32 | 33 | |
| 33 | 34 | #include "amdgpu.h" |
| @@ -935,10 +936,14 @@ static int amdgpu_dm_plane_helper_prepare_fb(struct drm_plane *plane, · 901 unmodified lines | ||
| 935 | 936 | } |
| 936 | 937 | |
| 937 | 938 | afb = to_amdgpu_framebuffer(new_state->fb); |
| 938 | − obj = new_state->fb->obj[0]; | |
| 939 | + obj = drm_gem_fb_get_obj(new_state->fb, 0); | |
| 940 | + if (!obj) { | |
| 941 | + DRM_ERROR("Failed to get obj from framebuffer\n"); | |
| 942 | + return -EINVAL; | |
| 943 | + } | |
| 944 | + | |
| 939 | 945 | rbo = gem_to_amdgpu_bo(obj); |
| 940 | 946 | adev = amdgpu_ttm_adev(rbo->tbo.bdev); |
| 941 | − | |
| 942 | 947 | r = amdgpu_bo_reserve(rbo, true); |
| 943 | 948 | if (r) { |
| 944 | 949 | dev_err(adev->dev, "fail to reserve bo (%d)\n", r); |
| @@ -35,8 +35,10 @@ · 34 unmodified lines | ||
| 35 | 35 | #include "dc_stream_priv.h" |
| 36 | 36 | |
| 37 | 37 | #define DC_LOGGER dc->ctx->logger |
| 38 | +#ifndef MIN | |
| 38 | 39 | #define MIN(X, Y) ((X) < (Y) ? (X) : (Y)) |
| 39 | 40 | #define MAX(x, y) ((x > y) ? x : y) |
| 41 | +#endif | |
| 40 | 42 | |
| 41 | 43 | /******************************************************************************* |
| 42 | 44 | * Private functions |
| @@ -185,8 +185,7 @@ static bool dmub_replay_copy_settings(struct dmub_replay *dmub, · 184 unmodified lines | ||
| 185 | 185 | else |
| 186 | 186 | copy_settings_data->flags.bitfields.force_wakeup_by_tps3 = 0; |
| 187 | 187 | |
| 188 | − | |
| 189 | − dm_execute_dmub_cmd(dc, &cmd, DM_DMUB_WAIT_TYPE_WAIT); | |
| 188 | + dc_wake_and_execute_dmub_cmd(dc, &cmd, DM_DMUB_WAIT_TYPE_WAIT); | |
| 190 | 189 | |
| 191 | 190 | return true; |
| 192 | 191 | } |
| @@ -83,6 +83,8 @@ CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn31/display_rq_dlg_calc_31.o := $(dml_rcfla · 82 unmodified lines | ||
| 83 | 83 | CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn32/display_mode_vba_32.o := $(dml_rcflags) |
| 84 | 84 | CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn32/display_rq_dlg_calc_32.o := $(dml_rcflags) |
| 85 | 85 | CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn32/display_mode_vba_util_32.o := $(dml_rcflags) |
| 86 | +CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn314/display_mode_vba_314.o := $(dml_rcflags) | |
| 87 | +CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn314/display_rq_dlg_calc_314.o := $(dml_rcflags) | |
| 86 | 88 | CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dcn301/dcn301_fpu.o := $(dml_rcflags) |
| 87 | 89 | CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/display_mode_lib.o := $(dml_rcflags) |
| 88 | 90 | CFLAGS_REMOVE_$(AMDDALPATH)/dc/dml/dsc/rc_calc_fpu.o := $(dml_rcflags) |
| @@ -1402,6 +1402,8 @@ void dcn10_init_pipes(struct dc *dc, struct dc_state *context) · 1401 unmodified lines | ||
| 1402 | 1402 | if (hubbub && hubp) { |
| 1403 | 1403 | if (hubbub->funcs->program_det_size) |
| 1404 | 1404 | hubbub->funcs->program_det_size(hubbub, hubp->inst, 0); |
| 1405 | + if (hubbub->funcs->program_det_segments) | |
| 1406 | + hubbub->funcs->program_det_segments(hubbub, hubp->inst, 0); | |
| 1405 | 1407 | } |
| 1406 | 1408 | } |
| 1407 | 1409 | |
| @@ -3587,7 +3589,7 @@ void dcn10_set_cursor_position(struct pipe_ctx *pipe_ctx) · 2179 unmodified lines | ||
| 3587 | 3589 | (int)hubp->curs_attr.width || pos_cpy.x |
| 3588 | 3590 | <= (int)hubp->curs_attr.width + |
| 3589 | 3591 | pipe_ctx->plane_state->src_rect.x) { |
| 3590 | − pos_cpy.x = temp_x + viewport_width; | |
| 3592 | + pos_cpy.x = 2 * viewport_width - temp_x; | |
| 3591 | 3593 | } |
| 3592 | 3594 | } |
| 3593 | 3595 | } else { |
| @@ -3680,7 +3682,7 @@ void dcn10_set_cursor_position(struct pipe_ctx *pipe_ctx) · 86 unmodified lines | ||
| 3680 | 3682 | (int)hubp->curs_attr.width || pos_cpy.x |
| 3681 | 3683 | <= (int)hubp->curs_attr.width + |
| 3682 | 3684 | pipe_ctx->plane_state->src_rect.x) { |
| 3683 | − pos_cpy.x = 2 * viewport_width - temp_x; | |
| 3685 | + pos_cpy.x = temp_x + viewport_width; | |
| 3684 | 3686 | } |
| 3685 | 3687 | } |
| 3686 | 3688 | } else { |
| @@ -771,6 +771,8 @@ void dcn35_init_pipes(struct dc *dc, struct dc_state *context) · 770 unmodified lines | ||
| 771 | 771 | if (hubbub && hubp) { |
| 772 | 772 | if (hubbub->funcs->program_det_size) |
| 773 | 773 | hubbub->funcs->program_det_size(hubbub, hubp->inst, 0); |
| 774 | + if (hubbub->funcs->program_det_segments) | |
| 775 | + hubbub->funcs->program_det_segments(hubbub, hubp->inst, 0); | |
| 774 | 776 | } |
| 775 | 777 | } |
| 776 | 778 | |
| @@ -1778,6 +1778,9 @@ static bool dcn321_resource_construct( · 1777 unmodified lines | ||
| 1778 | 1778 | dc->caps.color.mpc.ogam_rom_caps.hlg = 0; |
| 1779 | 1779 | dc->caps.color.mpc.ocsc = 1; |
| 1780 | 1780 | |
| 1781 | + /* Use pipe context based otg sync logic */ | |
| 1782 | + dc->config.use_pipe_ctx_sync_logic = true; | |
| 1783 | + | |
| 1781 | 1784 | dc->config.dc_mode_clk_limit_support = true; |
| 1782 | 1785 | dc->config.enable_windowed_mpo_odm = true; |
| 1783 | 1786 | /* read VBIOS LTTPR caps */ |
| @@ -723,6 +723,7 @@ static const struct dc_debug_options debug_defaults_drv = { · 722 unmodified lines | ||
| 723 | 723 | .min_prefetch_in_strobe_ns = 60000, // 60us |
| 724 | 724 | .disable_unbounded_requesting = false, |
| 725 | 725 | .enable_legacy_fast_update = false, |
| 726 | + .dcc_meta_propagation_delay_us = 10, | |
| 726 | 727 | .fams2_config = { |
| 727 | 728 | .bits = { |
| 728 | 729 | .enable = true, |
| @@ -138,7 +138,9 @@ void dcn401_prepare_mcache_programming(struct dc *dc, struct dc_state *context); · 137 unmodified lines | ||
| 138 | 138 | SRI_ARR(DCHUBP_MALL_CONFIG, HUBP, id), \ |
| 139 | 139 | SRI_ARR(DCHUBP_VMPG_CONFIG, HUBP, id), \ |
| 140 | 140 | SRI_ARR(UCLK_PSTATE_FORCE, HUBPREQ, id), \ |
| 141 | − HUBP_3DLUT_FL_REG_LIST_DCN401(id) | |
| 141 | + HUBP_3DLUT_FL_REG_LIST_DCN401(id), \ | |
| 142 | + SRI_ARR(DCSURF_VIEWPORT_MCACHE_SPLIT_COORDINATE, HUBP, id), \ | |
| 143 | + SRI_ARR(DCHUBP_MCACHEID_CONFIG, HUBP, id) | |
| 142 | 144 | |
| 143 | 145 | /* ABM */ |
| 144 | 146 | #define ABM_DCN401_REG_LIST_RI(id) \ |
| @@ -25,7 +25,9 @@ · 24 unmodified lines | ||
| 25 | 25 | |
| 26 | 26 | #include "hdcp.h" |
| 27 | 27 | |
| 28 | +#ifndef MIN | |
| 28 | 29 | #define MIN(a, b) ((a) < (b) ? (a) : (b)) |
| 30 | +#endif | |
| 29 | 31 | #define HDCP_I2C_ADDR 0x3a /* 0x74 >> 1*/ |
| 30 | 32 | #define KSV_READ_SIZE 0xf /* 0x6803b - 0x6802c */ |
| 31 | 33 | #define HDCP_MAX_AUX_TRANSACTION_SIZE 16 |
| @@ -258,6 +258,48 @@ struct gc_info_v1_2 { · 257 unmodified lines | ||
| 258 | 258 | uint32_t gc_gl2c_per_gpu; |
| 259 | 259 | }; |
| 260 | 260 | |
| 261 | +struct gc_info_v1_3 { | |
| 262 | + struct gpu_info_header header; | |
| 263 | + uint32_t gc_num_se; | |
| 264 | + uint32_t gc_num_wgp0_per_sa; | |
| 265 | + uint32_t gc_num_wgp1_per_sa; | |
| 266 | + uint32_t gc_num_rb_per_se; | |
| 267 | + uint32_t gc_num_gl2c; | |
| 268 | + uint32_t gc_num_gprs; | |
| 269 | + uint32_t gc_num_max_gs_thds; | |
| 270 | + uint32_t gc_gs_table_depth; | |
| 271 | + uint32_t gc_gsprim_buff_depth; | |
| 272 | + uint32_t gc_parameter_cache_depth; | |
| 273 | + uint32_t gc_double_offchip_lds_buffer; | |
| 274 | + uint32_t gc_wave_size; | |
| 275 | + uint32_t gc_max_waves_per_simd; | |
| 276 | + uint32_t gc_max_scratch_slots_per_cu; | |
| 277 | + uint32_t gc_lds_size; | |
| 278 | + uint32_t gc_num_sc_per_se; | |
| 279 | + uint32_t gc_num_sa_per_se; | |
| 280 | + uint32_t gc_num_packer_per_sc; | |
| 281 | + uint32_t gc_num_gl2a; | |
| 282 | + uint32_t gc_num_tcp_per_sa; | |
| 283 | + uint32_t gc_num_sdp_interface; | |
| 284 | + uint32_t gc_num_tcps; | |
| 285 | + uint32_t gc_num_tcp_per_wpg; | |
| 286 | + uint32_t gc_tcp_l1_size; | |
| 287 | + uint32_t gc_num_sqc_per_wgp; | |
| 288 | + uint32_t gc_l1_instruction_cache_size_per_sqc; | |
| 289 | + uint32_t gc_l1_data_cache_size_per_sqc; | |
| 290 | + uint32_t gc_gl1c_per_sa; | |
| 291 | + uint32_t gc_gl1c_size_per_instance; | |
| 292 | + uint32_t gc_gl2c_per_gpu; | |
| 293 | + uint32_t gc_tcp_size_per_cu; | |
| 294 | + uint32_t gc_tcp_cache_line_size; | |
| 295 | + uint32_t gc_instruction_cache_size_per_sqc; | |
| 296 | + uint32_t gc_instruction_cache_line_size; | |
| 297 | + uint32_t gc_scalar_data_cache_size_per_sqc; | |
| 298 | + uint32_t gc_scalar_data_cache_line_size; | |
| 299 | + uint32_t gc_tcc_size; | |
| 300 | + uint32_t gc_tcc_cache_line_size; | |
| 301 | +}; | |
| 302 | + | |
| 261 | 303 | struct gc_info_v2_0 { |
| 262 | 304 | struct gpu_info_header header; |
| 263 | 305 | |
| @@ -28,6 +28,9 @@ · 27 unmodified lines | ||
| 28 | 28 | |
| 29 | 29 | #define MES_API_VERSION 1 |
| 30 | 30 | |
| 31 | +/* Maximum log buffer size for MES. Needs to be updated if MES expands MES_EVT_INTR_HIST_LOG */ | |
| 32 | +#define AMDGPU_MES_LOG_BUFFER_SIZE 0x4000 | |
| 33 | + | |
| 31 | 34 | /* Driver submits one API(cmd) as a single Frame and this command size is same |
| 32 | 35 | * for all API to ease the debugging and parsing of ring buffer. |
| 33 | 36 | */ |
| @@ -28,6 +28,9 @@ · 27 unmodified lines | ||
| 28 | 28 | |
| 29 | 29 | #define MES_API_VERSION 0x14 |
| 30 | 30 | |
| 31 | +/* Maximum log buffer size for MES. Needs to be updated if MES expands MES_EVT_INTR_HIST_LOG_12 */ | |
| 32 | +#define AMDGPU_MES_LOG_BUFFER_SIZE 0xC000 | |
| 33 | + | |
| 31 | 34 | /* Driver submits one API(cmd) as a single Frame and this command size is same for all API |
| 32 | 35 | * to ease the debugging and parsing of ring buffer. |
| 33 | 36 | */ |
| @@ -94,6 +97,7 @@ enum MES_QUEUE_TYPE { · 60 unmodified lines | ||
| 94 | 97 | MES_QUEUE_TYPE_SDMA, |
| 95 | 98 | |
| 96 | 99 | MES_QUEUE_TYPE_MAX, |
| 100 | + MES_QUEUE_TYPE_SCHQ = MES_QUEUE_TYPE_MAX, | |
| 97 | 101 | }; |
| 98 | 102 | |
| 99 | 103 | struct MES_API_STATUS { |
| @@ -239,8 +243,12 @@ union MESAPI_SET_HW_RESOURCES { · 139 unmodified lines | ||
| 239 | 243 | uint32_t send_write_data : 1; |
| 240 | 244 | uint32_t os_tdr_timeout_override : 1; |
| 241 | 245 | uint32_t use_rs64mem_for_proc_gang_ctx : 1; |
| 246 | + uint32_t halt_on_misaligned_access : 1; | |
| 247 | + uint32_t use_add_queue_unmap_flag_addr : 1; | |
| 248 | + uint32_t enable_mes_sch_stb_log : 1; | |
| 249 | + uint32_t limit_single_process : 1; | |
| 242 | 250 | uint32_t unmapped_doorbell_handling: 2; |
| 243 | − uint32_t reserved : 15; | |
| 251 | + uint32_t reserved : 11; | |
| 244 | 252 | }; |
| 245 | 253 | uint32_t uint32_all; |
| 246 | 254 | }; |
| @@ -618,7 +618,8 @@ int amdgpu_pm_load_smu_firmware(struct amdgpu_device *adev, uint32_t *smu_versio · 617 unmodified lines | ||
| 618 | 618 | const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs; |
| 619 | 619 | int r = 0; |
| 620 | 620 | |
| 621 | − if (!pp_funcs || !pp_funcs->load_firmware || adev->flags & AMD_IS_APU) | |
| 621 | + if (!pp_funcs || !pp_funcs->load_firmware || | |
| 622 | + (is_support_sw_smu(adev) && (adev->flags & AMD_IS_APU))) | |
| 622 | 623 | return 0; |
| 623 | 624 | |
| 624 | 625 | mutex_lock(&adev->pm.mutex); |
| @@ -22,12 +22,18 @@ · 21 unmodified lines | ||
| 22 | 22 | */ |
| 23 | 23 | #include <asm/div64.h> |
| 24 | 24 | |
| 25 | −#define SHIFT_AMOUNT 16 /* We multiply all original integers with 2^SHIFT_AMOUNT to get the fInt representation */ | |
| 25 | +enum ppevvmath_constants { | |
| 26 | + /* We multiply all original integers with 2^SHIFT_AMOUNT to get the fInt representation */ | |
| 27 | + SHIFT_AMOUNT = 16, | |
| 26 | 28 | |
| 27 | −#define PRECISION 5 /* Change this value to change the number of decimal places in the final output - 5 is a good default */ | |
| 29 | + /* Change this value to change the number of decimal places in the final output - 5 is a good default */ | |
| 30 | + PRECISION = 5, | |
| 28 | 31 | |
| 29 | −#define SHIFTED_2 (2 << SHIFT_AMOUNT) | |
| 30 | −#define MAX (1 << (SHIFT_AMOUNT - 1)) - 1 /* 32767 - Might change in the future */ | |
| 32 | + SHIFTED_2 = (2 << SHIFT_AMOUNT), | |
| 33 | + | |
| 34 | + /* 32767 - Might change in the future */ | |
| 35 | + MAX = (1 << (SHIFT_AMOUNT - 1)) - 1, | |
| 36 | +}; | |
| 31 | 37 | |
| 32 | 38 | /* ------------------------------------------------------------------------------- |
| 33 | 39 | * NEW TYPE - fINT |
| @@ -2224,8 +2224,9 @@ static int smu_bump_power_profile_mode(struct smu_context *smu, · 2223 unmodified lines | ||
| 2224 | 2224 | } |
| 2225 | 2225 | |
| 2226 | 2226 | static int smu_adjust_power_state_dynamic(struct smu_context *smu, |
| 2227 | − enum amd_dpm_forced_level level, | |
| 2228 | − bool skip_display_settings) | |
| 2227 | + enum amd_dpm_forced_level level, | |
| 2228 | + bool skip_display_settings, | |
| 2229 | + bool force_update) | |
| 2229 | 2230 | { |
| 2230 | 2231 | int ret = 0; |
| 2231 | 2232 | int index = 0; |
| @@ -2254,7 +2255,7 @@ static int smu_adjust_power_state_dynamic(struct smu_context *smu, · 22 unmodified lines | ||
| 2254 | 2255 | } |
| 2255 | 2256 | } |
| 2256 | 2257 | |
| 2257 | − if (smu_dpm_ctx->dpm_level != level) { | |
| 2258 | + if (force_update || smu_dpm_ctx->dpm_level != level) { | |
| 2258 | 2259 | ret = smu_asic_set_performance_level(smu, level); |
| 2259 | 2260 | if (ret) { |
| 2260 | 2261 | dev_err(smu->adev->dev, "Failed to set performance level!"); |
| @@ -2265,13 +2266,12 @@ static int smu_adjust_power_state_dynamic(struct smu_context *smu, · 4 unmodified lines | ||
| 2265 | 2266 | smu_dpm_ctx->dpm_level = level; |
| 2266 | 2267 | } |
| 2267 | 2268 | |
| 2268 | − if (smu_dpm_ctx->dpm_level != AMD_DPM_FORCED_LEVEL_MANUAL && | |
| 2269 | − smu_dpm_ctx->dpm_level != AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM) { | |
| 2269 | + if (smu_dpm_ctx->dpm_level != AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM) { | |
| 2270 | 2270 | index = fls(smu->workload_mask); |
| 2271 | 2271 | index = index > 0 && index <= WORKLOAD_POLICY_MAX ? index - 1 : 0; |
| 2272 | 2272 | workload[0] = smu->workload_setting[index]; |
| 2273 | 2273 | |
| 2274 | − if (smu->power_profile_mode != workload[0]) | |
| 2274 | + if (force_update || smu->power_profile_mode != workload[0]) | |
| 2275 | 2275 | smu_bump_power_profile_mode(smu, workload, 0); |
| 2276 | 2276 | } |
| 2277 | 2277 | |
| @@ -2292,11 +2292,13 @@ static int smu_handle_task(struct smu_context *smu, · 14 unmodified lines | ||
| 2292 | 2292 | ret = smu_pre_display_config_changed(smu); |
| 2293 | 2293 | if (ret) |
| 2294 | 2294 | return ret; |
| 2295 | − ret = smu_adjust_power_state_dynamic(smu, level, false); | |
| 2295 | + ret = smu_adjust_power_state_dynamic(smu, level, false, false); | |
| 2296 | 2296 | break; |
| 2297 | 2297 | case AMD_PP_TASK_COMPLETE_INIT: |
| 2298 | + ret = smu_adjust_power_state_dynamic(smu, level, true, true); | |
| 2299 | + break; | |
| 2298 | 2300 | case AMD_PP_TASK_READJUST_POWER_STATE: |
| 2299 | − ret = smu_adjust_power_state_dynamic(smu, level, true); | |
| 2301 | + ret = smu_adjust_power_state_dynamic(smu, level, true, false); | |
| 2300 | 2302 | break; |
| 2301 | 2303 | default: |
| 2302 | 2304 | break; |
| @@ -2343,8 +2345,7 @@ static int smu_switch_power_profile(void *handle, · 40 unmodified lines | ||
| 2343 | 2345 | workload[0] = smu->workload_setting[index]; |
| 2344 | 2346 | } |
| 2345 | 2347 | |
| 2346 | − if (smu_dpm_ctx->dpm_level != AMD_DPM_FORCED_LEVEL_MANUAL && | |
| 2347 | − smu_dpm_ctx->dpm_level != AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM) | |
| 2348 | + if (smu_dpm_ctx->dpm_level != AMD_DPM_FORCED_LEVEL_PERF_DETERMINISM) | |
| 2348 | 2349 | smu_bump_power_profile_mode(smu, workload, 0); |
| 2349 | 2350 | |
| 2350 | 2351 | return 0; |
| @@ -92,7 +92,6 @@ · 91 unmodified lines | ||
| 92 | 92 | |
| 93 | 93 | //Resets |
| 94 | 94 | #define PPSMC_MSG_PrepareMp1ForUnload 0x2E |
| 95 | −#define PPSMC_MSG_Mode1Reset 0x2F | |
| 96 | 95 | |
| 97 | 96 | //Set SystemVirtual DramAddrHigh |
| 98 | 97 | #define PPSMC_MSG_SetSystemVirtualDramAddrHigh 0x30 |
| @@ -119,11 +118,12 @@ · 20 unmodified lines | ||
| 119 | 118 | |
| 120 | 119 | //STB to dram log |
| 121 | 120 | #define PPSMC_MSG_DumpSTBtoDram 0x3D |
| 122 | −#define PPSMC_MSG_STBtoDramLogSetDramAddrHigh 0x3E | |
| 123 | −#define PPSMC_MSG_STBtoDramLogSetDramAddrLow 0x3F | |
| 121 | +#define PPSMC_MSG_STBtoDramLogSetDramAddress 0x3E | |
| 122 | +#define PPSMC_MSG_DummyUndefined 0x3F | |
| 124 | 123 | #define PPSMC_MSG_STBtoDramLogSetDramSize 0x40 |
| 125 | 124 | #define PPSMC_MSG_SetOBMTraceBufferLogging 0x41 |
| 126 | 125 | |
| 126 | +#define PPSMC_MSG_UseProfilingMode 0x42 | |
| 127 | 127 | #define PPSMC_MSG_AllowGfxDcs 0x43 |
| 128 | 128 | #define PPSMC_MSG_DisallowGfxDcs 0x44 |
| 129 | 129 | #define PPSMC_MSG_EnableAudioStutterWA 0x45 |
| @@ -135,6 +135,16 @@ · 5 unmodified lines | ||
| 135 | 135 | #define PPSMC_MSG_SetBadMemoryPagesRetiredFlagsPerChannel 0x4B |
| 136 | 136 | #define PPSMC_MSG_SetPriorityDeltaGain 0x4C |
| 137 | 137 | #define PPSMC_MSG_AllowIHHostInterrupt 0x4D |
| 138 | +#define PPSMC_MSG_EnableShadowDpm 0x4E | |
| 138 | 139 | #define PPSMC_MSG_Mode3Reset 0x4F |
| 139 | −#define PPSMC_Message_Count 0x50 | |
| 140 | +#define PPSMC_MSG_SetDriverDramAddr 0x50 | |
| 141 | +#define PPSMC_MSG_SetToolsDramAddr 0x51 | |
| 142 | +#define PPSMC_MSG_TransferTableSmu2DramWithAddr 0x52 | |
| 143 | +#define PPSMC_MSG_TransferTableDram2SmuWithAddr 0x53 | |
| 144 | +#define PPSMC_MSG_GetAllRunningSmuFeatures 0x54 | |
| 145 | +#define PPSMC_MSG_GetSvi3Voltage 0x55 | |
| 146 | +#define PPSMC_MSG_UpdatePolicy 0x56 | |
| 147 | +#define PPSMC_MSG_ExtPwrConnSupport 0x57 | |
| 148 | +#define PPSMC_MSG_PreloadSwPstateForUclkOverDrive 0x58 | |
| 149 | +#define PPSMC_Message_Count 0x59 | |
| 140 | 150 | #endif |
| @@ -27,7 +27,8 @@ · 26 unmodified lines | ||
| 27 | 27 | |
| 28 | 28 | #pragma pack(push, 1) |
| 29 | 29 | |
| 30 | −#define SMU_14_0_2_TABLE_FORMAT_REVISION 3 | |
| 30 | +#define SMU_14_0_2_TABLE_FORMAT_REVISION 23 | |
| 31 | +#define SMU_14_0_2_CUSTOM_TABLE_FORMAT_REVISION 1 | |
| 31 | 32 | |
| 32 | 33 | // POWERPLAYTABLE::ulPlatformCaps |
| 33 | 34 | #define SMU_14_0_2_PP_PLATFORM_CAP_POWERPLAY 0x1 // This cap indicates whether CCC need to show Powerplay page. |
| @@ -43,6 +44,7 @@ · 9 unmodified lines | ||
| 43 | 44 | #define SMU_14_0_2_PP_THERMALCONTROLLER_NONE 0 |
| 44 | 45 | |
| 45 | 46 | #define SMU_14_0_2_PP_OVERDRIVE_VERSION 0x1 // TODO: FIX OverDrive Version TBD |
| 47 | +#define SMU_14_0_2_PP_CUSTOM_OVERDRIVE_VERSION 0x1 | |
| 46 | 48 | #define SMU_14_0_2_PP_POWERSAVINGCLOCK_VERSION 0x01 // Power Saving Clock Table Version 1.00 |
| 47 | 49 | |
| 48 | 50 | enum SMU_14_0_2_OD_SW_FEATURE_CAP |
| @@ -107,6 +109,7 @@ enum SMU_14_0_2_PWRMODE_SETTING · 58 unmodified lines | ||
| 107 | 109 | SMU_14_0_2_PMSETTING_ACOUSTIC_LIMIT_RPM_BALANCE, |
| 108 | 110 | SMU_14_0_2_PMSETTING_ACOUSTIC_LIMIT_RPM_TURBO, |
| 109 | 111 | SMU_14_0_2_PMSETTING_ACOUSTIC_LIMIT_RPM_RAGE, |
| 112 | + SMU_14_0_2_PMSETTING_COUNT | |
| 110 | 113 | }; |
| 111 | 114 | #define SMU_14_0_2_MAX_PMSETTING 32 // Maximum Number of PowerMode Settings |
| 112 | 115 | |
| @@ -127,17 +130,24 @@ struct smu_14_0_2_overdrive_table · 14 unmodified lines | ||
| 127 | 130 | int16_t pm_setting[SMU_14_0_2_MAX_PMSETTING]; // Optimized power mode feature settings |
| 128 | 131 | }; |
| 129 | 132 | |
| 133 | +enum smu_14_0_3_pptable_source { | |
| 134 | + PPTABLE_SOURCE_IFWI = 0, | |
| 135 | + PPTABLE_SOURCE_DRIVER_HARDCODED = 1, | |
| 136 | + PPTABLE_SOURCE_PPGEN_REGISTRY = 2, | |
| 137 | + PPTABLE_SOURCE_MAX = PPTABLE_SOURCE_PPGEN_REGISTRY, | |
| 138 | +}; | |
| 139 | + | |
| 130 | 140 | struct smu_14_0_2_powerplay_table |
| 131 | 141 | { |
| 132 | 142 | struct atom_common_table_header header; // header.format_revision = 3 (HAS TO MATCH SMU_14_0_2_TABLE_FORMAT_REVISION), header.content_revision = ? structuresize is calculated by PPGen. |
| 133 | 143 | uint8_t table_revision; // PPGen use only: table_revision = 3 |
| 134 | − uint8_t padding; // Padding 1 byte to align table_size offset to 6 bytes (pmfw_start_offset, for PMFW to know the starting offset of PPTable_t). | |
| 144 | + uint8_t pptable_source; // PPGen UI dropdown box | |
| 135 | 145 | uint16_t pmfw_pptable_start_offset; // The start offset of the pmfw portion. i.e. start of PPTable_t (start of SkuTable_t) |
| 136 | 146 | uint16_t pmfw_pptable_size; // The total size of pmfw_pptable, i.e PPTable_t. |
| 137 | − uint16_t pmfw_pfe_table_start_offset; // The start offset of the PFE_Settings_t within pmfw_pptable. | |
| 138 | − uint16_t pmfw_pfe_table_size; // The size of PFE_Settings_t. | |
| 139 | − uint16_t pmfw_board_table_start_offset; // The start offset of the BoardTable_t within pmfw_pptable. | |
| 140 | − uint16_t pmfw_board_table_size; // The size of BoardTable_t. | |
| 147 | + uint16_t pmfw_sku_table_start_offset; // DO NOT CHANGE ORDER; The absolute start offset of the SkuTable_t (within smu_14_0_3_powerplay_table). | |
| 148 | + uint16_t pmfw_sku_table_size; // DO NOT CHANGE ORDER; The size of SkuTable_t. | |
| 149 | + uint16_t pmfw_board_table_start_offset; // The start offset of the BoardTable_t | |
| 150 | + uint16_t pmfw_board_table_size; // The size of BoardTable_t. | |
| 141 | 151 | uint16_t pmfw_custom_sku_table_start_offset; // The start offset of the CustomSkuTable_t within pmfw_pptable. |
| 142 | 152 | uint16_t pmfw_custom_sku_table_size; // The size of the CustomSkuTable_t. |
| 143 | 153 | uint32_t golden_pp_id; // PPGen use only: PP Table ID on the Golden Data Base |
| @@ -159,6 +169,36 @@ struct smu_14_0_2_powerplay_table · 15 unmodified lines | ||
| 159 | 169 | PPTable_t smc_pptable; // PPTable_t in driver_if.h -- as requested by PMFW, this offset should start at a 32-byte boundary, and the table_size above should remain at offset=6 bytes |
| 160 | 170 | }; |
| 161 | 171 | |
| 172 | +enum SMU_14_0_2_CUSTOM_OD_SW_FEATURE_CAP { | |
| 173 | + SMU_14_0_2_CUSTOM_ODCAP_POWER_MODE = 0, | |
| 174 | + SMU_14_0_2_CUSTOM_ODCAP_COUNT | |
| 175 | +}; | |
| 176 | + | |
| 177 | +enum SMU_14_0_2_CUSTOM_OD_FEATURE_SETTING_ID { | |
| 178 | + SMU_14_0_2_CUSTOM_ODSETTING_POWER_MODE = 0, | |
| 179 | + SMU_14_0_2_CUSTOM_ODSETTING_COUNT, | |
| 180 | +}; | |
| 181 | + | |
| 182 | +struct smu_14_0_2_custom_overdrive_table { | |
| 183 | + uint8_t revision; | |
| 184 | + uint8_t reserve[3]; | |
| 185 | + uint8_t cap[SMU_14_0_2_CUSTOM_ODCAP_COUNT]; | |
| 186 | + int32_t max[SMU_14_0_2_CUSTOM_ODSETTING_COUNT]; | |
| 187 | + int32_t min[SMU_14_0_2_CUSTOM_ODSETTING_COUNT]; | |
| 188 | + int16_t pm_setting[SMU_14_0_2_PMSETTING_COUNT]; | |
| 189 | +}; | |
| 190 | + | |
| 191 | +struct smu_14_0_3_custom_powerplay_table { | |
| 192 | + uint8_t custom_table_revision; | |
| 193 | + uint16_t custom_table_size; | |
| 194 | + uint16_t custom_sku_table_offset; | |
| 195 | + uint32_t custom_platform_caps; | |
| 196 | + uint16_t software_shutdown_temp; | |
| 197 | + struct smu_14_0_2_custom_overdrive_table custom_overdrive_table; | |
| 198 | + uint32_t reserve[8]; | |
| 199 | + CustomSkuTable_t custom_sku_table_pmfw; | |
| 200 | +}; | |
| 201 | + | |
| 162 | 202 | #pragma pack(pop) |
| 163 | 203 | |
| 164 | 204 | #endif |
| @@ -121,6 +121,7 @@ struct mca_ras_info { · 120 unmodified lines | ||
| 121 | 121 | |
| 122 | 122 | #define P2S_TABLE_ID_A 0x50325341 |
| 123 | 123 | #define P2S_TABLE_ID_X 0x50325358 |
| 124 | +#define P2S_TABLE_ID_3 0x50325303 | |
| 124 | 125 | |
| 125 | 126 | // clang-format off |
| 126 | 127 | static const struct cmn2asic_msg_mapping smu_v13_0_6_message_map[SMU_MSG_MAX_COUNT] = { |
| @@ -271,14 +272,18 @@ static int smu_v13_0_6_init_microcode(struct smu_context *smu) · 144 unmodified lines | ||
| 271 | 272 | struct amdgpu_device *adev = smu->adev; |
| 272 | 273 | uint32_t p2s_table_id = P2S_TABLE_ID_A; |
| 273 | 274 | int ret = 0, i, p2stable_count; |
| 275 | + int var = (adev->pdev->device & 0xF); | |
| 274 | 276 | char ucode_prefix[15]; |
| 275 | 277 | |
| 276 | 278 | /* No need to load P2S tables in IOV mode */ |
| 277 | 279 | if (amdgpu_sriov_vf(adev)) |
| 278 | 280 | return 0; |
| 279 | 281 | |
| 280 | − if (!(adev->flags & AMD_IS_APU)) | |
| 282 | + if (!(adev->flags & AMD_IS_APU)) { | |
| 281 | 283 | p2s_table_id = P2S_TABLE_ID_X; |
| 284 | + if (var == 0x5) | |
| 285 | + p2s_table_id = P2S_TABLE_ID_3; | |
| 286 | + } | |
| 282 | 287 | |
| 283 | 288 | amdgpu_ucode_ip_version_decode(adev, MP1_HWIP, ucode_prefix, |
| 284 | 289 | sizeof(ucode_prefix)); |
| @@ -2378,7 +2378,7 @@ static int smu_v13_0_7_get_power_profile_mode(struct smu_context *smu, char *buf · 2377 unmodified lines | ||
| 2378 | 2378 | |
| 2379 | 2379 | size += sysfs_emit_at(buf, size, " "); |
| 2380 | 2380 | for (i = 0; i <= PP_SMC_POWER_PROFILE_WINDOW3D; i++) |
| 2381 | − size += sysfs_emit_at(buf, size, "%-14s%s", amdgpu_pp_profile_name[i], | |
| 2381 | + size += sysfs_emit_at(buf, size, "%d %-14s%s", i, amdgpu_pp_profile_name[i], | |
| 2382 | 2382 | (i == smu->power_profile_mode) ? "* " : " "); |
| 2383 | 2383 | |
| 2384 | 2384 | size += sysfs_emit_at(buf, size, "\n"); |
| @@ -2408,7 +2408,7 @@ static int smu_v13_0_7_get_power_profile_mode(struct smu_context *smu, char *buf · 23 unmodified lines | ||
| 2408 | 2408 | do { \ |
| 2409 | 2409 | size += sysfs_emit_at(buf, size, "%-30s", #field); \ |
| 2410 | 2410 | for (j = 0; j <= PP_SMC_POWER_PROFILE_WINDOW3D; j++) \ |
| 2411 | − size += sysfs_emit_at(buf, size, "%-16d", activity_monitor_external[j].DpmActivityMonitorCoeffInt.field); \ | |
| 2411 | + size += sysfs_emit_at(buf, size, "%-18d", activity_monitor_external[j].DpmActivityMonitorCoeffInt.field); \ | |
| 2412 | 2412 | size += sysfs_emit_at(buf, size, "\n"); \ |
| 2413 | 2413 | } while (0) |
| 2414 | 2414 | |
| @@ -66,6 +66,7 @@ · 65 unmodified lines | ||
| 66 | 66 | |
| 67 | 67 | #define MP0_MP1_DATA_REGION_SIZE_COMBOPPTABLE 0x4000 |
| 68 | 68 | #define DEBUGSMC_MSG_Mode1Reset 2 |
| 69 | +#define LINK_SPEED_MAX 3 | |
| 69 | 70 | |
| 70 | 71 | static struct cmn2asic_msg_mapping smu_v14_0_2_message_map[SMU_MSG_MAX_COUNT] = { |
| 71 | 72 | MSG_MAP(TestMessage, PPSMC_MSG_TestMessage, 1), |
| @@ -114,7 +115,6 @@ static struct cmn2asic_msg_mapping smu_v14_0_2_message_map[SMU_MSG_MAX_COUNT] = · 42 unmodified lines | ||
| 114 | 115 | MSG_MAP(SetMGpuFanBoostLimitRpm, PPSMC_MSG_SetMGpuFanBoostLimitRpm, 0), |
| 115 | 116 | MSG_MAP(GetPptLimit, PPSMC_MSG_GetPptLimit, 0), |
| 116 | 117 | MSG_MAP(NotifyPowerSource, PPSMC_MSG_NotifyPowerSource, 0), |
| 117 | − MSG_MAP(Mode1Reset, PPSMC_MSG_Mode1Reset, 0), | |
| 118 | 118 | MSG_MAP(PrepareMp1ForUnload, PPSMC_MSG_PrepareMp1ForUnload, 0), |
| 119 | 119 | MSG_MAP(DFCstateControl, PPSMC_MSG_SetExternalClientDfCstateAllow, 0), |
| 120 | 120 | MSG_MAP(ArmD3, PPSMC_MSG_ArmD3, 0), |
| @@ -221,7 +221,6 @@ static struct cmn2asic_mapping smu_v14_0_2_workload_map[PP_SMC_POWER_PROFILE_COU · 100 unmodified lines | ||
| 221 | 221 | WORKLOAD_MAP(PP_SMC_POWER_PROFILE_WINDOW3D, WORKLOAD_PPLIB_WINDOW_3D_BIT), |
| 222 | 222 | }; |
| 223 | 223 | |
| 224 | −#if 0 | |
| 225 | 224 | static const uint8_t smu_v14_0_2_throttler_map[] = { |
| 226 | 225 | [THROTTLER_PPT0_BIT] = (SMU_THROTTLER_PPT0_BIT), |
| 227 | 226 | [THROTTLER_PPT1_BIT] = (SMU_THROTTLER_PPT1_BIT), |
| @@ -241,7 +240,6 @@ static const uint8_t smu_v14_0_2_throttler_map[] = { · 13 unmodified lines | ||
| 241 | 240 | [THROTTLER_GFX_APCC_PLUS_BIT] = (SMU_THROTTLER_APCC_BIT), |
| 242 | 241 | [THROTTLER_FIT_BIT] = (SMU_THROTTLER_FIT_BIT), |
| 243 | 242 | }; |
| 244 | −#endif | |
| 245 | 243 | |
| 246 | 244 | static int |
| 247 | 245 | smu_v14_0_2_get_allowed_feature_mask(struct smu_context *smu, |
| @@ -1825,48 +1823,86 @@ static void smu_v14_0_2_set_smu_mailbox_registers(struct smu_context *smu) · 1577 unmodified lines | ||
| 1825 | 1823 | smu->debug_resp_reg = SOC15_REG_OFFSET(MP1, 0, regMP1_SMN_C2PMSG_54); |
| 1826 | 1824 | } |
| 1827 | 1825 | |
| 1828 | −static int smu_v14_0_2_smu_send_bad_mem_page_num(struct smu_context *smu, | |
| 1829 | − uint32_t size) | |
| 1826 | +static ssize_t smu_v14_0_2_get_gpu_metrics(struct smu_context *smu, | |
| 1827 | + void **table) | |
| 1830 | 1828 | { |
| 1829 | + struct smu_table_context *smu_table = &smu->smu_table; | |
| 1830 | + struct gpu_metrics_v1_3 *gpu_metrics = | |
| 1831 | + (struct gpu_metrics_v1_3 *)smu_table->gpu_metrics_table; | |
| 1832 | + SmuMetricsExternal_t metrics_ext; | |
| 1833 | + SmuMetrics_t *metrics = &metrics_ext.SmuMetrics; | |
| 1831 | 1834 | int ret = 0; |
| 1832 | 1835 | |
| 1833 | − /* message SMU to update the bad page number on SMUBUS */ | |
| 1834 | − ret = smu_cmn_send_smc_msg_with_param(smu, | |
| 1835 | − SMU_MSG_SetNumBadMemoryPagesRetired, | |
| 1836 | − size, NULL); | |
| 1836 | + ret = smu_cmn_get_metrics_table(smu, | |
| 1837 | + &metrics_ext, | |
| 1838 | + true); | |
| 1837 | 1839 | if (ret) |
| 1838 | − dev_err(smu->adev->dev, | |
| 1839 | − "[%s] failed to message SMU to update bad memory pages number\n", | |
| 1840 | − __func__); | |
| 1840 | + return ret; | |
| 1841 | 1841 | |
| 1842 | − return ret; | |
| 1843 | −} | |
| 1842 | + smu_cmn_init_soft_gpu_metrics(gpu_metrics, 1, 3); | |
| 1844 | 1843 | |
| 1845 | −static int smu_v14_0_2_send_bad_mem_channel_flag(struct smu_context *smu, | |
| 1846 | − uint32_t size) | |
| 1847 | −{ | |
| 1848 | − int ret = 0; | |
| 1844 | + gpu_metrics->temperature_edge = metrics->AvgTemperature[TEMP_EDGE]; | |
| 1845 | + gpu_metrics->temperature_hotspot = metrics->AvgTemperature[TEMP_HOTSPOT]; | |
| 1846 | + gpu_metrics->temperature_mem = metrics->AvgTemperature[TEMP_MEM]; | |
| 1847 | + gpu_metrics->temperature_vrgfx = metrics->AvgTemperature[TEMP_VR_GFX]; | |
| 1848 | + gpu_metrics->temperature_vrsoc = metrics->AvgTemperature[TEMP_VR_SOC]; | |
| 1849 | + gpu_metrics->temperature_vrmem = max(metrics->AvgTemperature[TEMP_VR_MEM0], | |
| 1850 | + metrics->AvgTemperature[TEMP_VR_MEM1]); | |
| 1849 | 1851 | |
| 1850 | − /* message SMU to update the bad channel info on SMUBUS */ | |
| 1851 | − ret = smu_cmn_send_smc_msg_with_param(smu, | |
| 1852 | − SMU_MSG_SetBadMemoryPagesRetiredFlagsPerChannel, | |
| 1853 | − size, NULL); | |
| 1854 | − if (ret) | |
| 1855 | − dev_err(smu->adev->dev, | |
| 1856 | − "[%s] failed to message SMU to update bad memory pages channel info\n", | |
| 1857 | − __func__); | |
| 1852 | + gpu_metrics->average_gfx_activity = metrics->AverageGfxActivity; | |
| 1853 | + gpu_metrics->average_umc_activity = metrics->AverageUclkActivity; | |
| 1854 | + gpu_metrics->average_mm_activity = max(metrics->Vcn0ActivityPercentage, | |
| 1855 | + metrics->Vcn1ActivityPercentage); | |
| 1858 | 1856 | |
| 1859 | − return ret; | |
| 1860 | −} | |
| 1857 | + gpu_metrics->average_socket_power = metrics->AverageSocketPower; | |
| 1858 | + gpu_metrics->energy_accumulator = metrics->EnergyAccumulator; | |
| 1861 | 1859 | |
| 1862 | −static ssize_t smu_v14_0_2_get_ecc_info(struct smu_context *smu, | |
| 1863 | − void *table) | |
| 1864 | −{ | |
| 1865 | − int ret = 0; | |
| 1860 | + if (metrics->AverageGfxActivity <= SMU_14_0_2_BUSY_THRESHOLD) | |
| 1861 | + gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPostDs; | |
| 1862 | + else | |
| 1863 | + gpu_metrics->average_gfxclk_frequency = metrics->AverageGfxclkFrequencyPreDs; | |
| 1866 | 1864 | |
| 1867 | − // TODO | |
| 1865 | + if (metrics->AverageUclkActivity <= SMU_14_0_2_BUSY_THRESHOLD) | |
| 1866 | + gpu_metrics->average_uclk_frequency = metrics->AverageMemclkFrequencyPostDs; | |
| 1867 | + else | |
| 1868 | + gpu_metrics->average_uclk_frequency = metrics->AverageMemclkFrequencyPreDs; | |
| 1869 | + | |
| 1870 | + gpu_metrics->average_vclk0_frequency = metrics->AverageVclk0Frequency; | |
| 1871 | + gpu_metrics->average_dclk0_frequency = metrics->AverageDclk0Frequency; | |
| 1872 | + gpu_metrics->average_vclk1_frequency = metrics->AverageVclk1Frequency; | |
| 1873 | + gpu_metrics->average_dclk1_frequency = metrics->AverageDclk1Frequency; | |
| 1874 | + | |
| 1875 | + gpu_metrics->current_gfxclk = gpu_metrics->average_gfxclk_frequency; | |
| 1876 | + gpu_metrics->current_socclk = metrics->CurrClock[PPCLK_SOCCLK]; | |
| 1877 | + gpu_metrics->current_uclk = metrics->CurrClock[PPCLK_UCLK]; | |
| 1878 | + gpu_metrics->current_vclk0 = metrics->CurrClock[PPCLK_VCLK_0]; | |
| 1879 | + gpu_metrics->current_dclk0 = metrics->CurrClock[PPCLK_DCLK_0]; | |
| 1880 | + gpu_metrics->current_vclk1 = metrics->CurrClock[PPCLK_VCLK_0]; | |
| 1881 | + gpu_metrics->current_dclk1 = metrics->CurrClock[PPCLK_DCLK_0]; | |
| 1882 | + | |
| 1883 | + gpu_metrics->throttle_status = | |
| 1884 | + smu_v14_0_2_get_throttler_status(metrics); | |
| 1885 | + gpu_metrics->indep_throttle_status = | |
| 1886 | + smu_cmn_get_indep_throttler_status(gpu_metrics->throttle_status, | |
| 1887 | + smu_v14_0_2_throttler_map); | |
| 1888 | + | |
| 1889 | + gpu_metrics->current_fan_speed = metrics->AvgFanRpm; | |
| 1890 | + | |
| 1891 | + gpu_metrics->pcie_link_width = metrics->PcieWidth; | |
| 1892 | + if ((metrics->PcieRate - 1) > LINK_SPEED_MAX) | |
| 1893 | + gpu_metrics->pcie_link_speed = pcie_gen_to_speed(1); | |
| 1894 | + else | |
| 1895 | + gpu_metrics->pcie_link_speed = pcie_gen_to_speed(metrics->PcieRate); | |
| 1868 | 1896 | |
| 1869 | − return ret; | |
| 1897 | + gpu_metrics->system_clock_counter = ktime_get_boottime_ns(); | |
| 1898 | + | |
| 1899 | + gpu_metrics->voltage_gfx = metrics->AvgVoltage[SVI_PLANE_VDD_GFX]; | |
| 1900 | + gpu_metrics->voltage_soc = metrics->AvgVoltage[SVI_PLANE_VDD_SOC]; | |
| 1901 | + gpu_metrics->voltage_mem = metrics->AvgVoltage[SVI_PLANE_VDDIO_MEM]; | |
| 1902 | + | |
| 1903 | + *table = (void *)gpu_metrics; | |
| 1904 | + | |
| 1905 | + return sizeof(struct gpu_metrics_v1_3); | |
| 1870 | 1906 | } |
| 1871 | 1907 | |
| 1872 | 1908 | static const struct pptable_funcs smu_v14_0_2_ppt_funcs = { |
| @@ -1905,6 +1941,7 @@ static const struct pptable_funcs smu_v14_0_2_ppt_funcs = { · 32 unmodified lines | ||
| 1905 | 1941 | .enable_thermal_alert = smu_v14_0_enable_thermal_alert, |
| 1906 | 1942 | .disable_thermal_alert = smu_v14_0_disable_thermal_alert, |
| 1907 | 1943 | .notify_memory_pool_location = smu_v14_0_notify_memory_pool_location, |
| 1944 | + .get_gpu_metrics = smu_v14_0_2_get_gpu_metrics, | |
| 1908 | 1945 | .set_soft_freq_limited_range = smu_v14_0_set_soft_freq_limited_range, |
| 1909 | 1946 | .init_pptable_microcode = smu_v14_0_init_pptable_microcode, |
| 1910 | 1947 | .populate_umd_state_clk = smu_v14_0_2_populate_umd_state_clk, |
| @@ -1933,12 +1970,9 @@ static const struct pptable_funcs smu_v14_0_2_ppt_funcs = { · 22 unmodified lines | ||
| 1933 | 1970 | .enable_gfx_features = smu_v14_0_2_enable_gfx_features, |
| 1934 | 1971 | .set_mp1_state = smu_v14_0_2_set_mp1_state, |
| 1935 | 1972 | .set_df_cstate = smu_v14_0_2_set_df_cstate, |
| 1936 | − .send_hbm_bad_pages_num = smu_v14_0_2_smu_send_bad_mem_page_num, | |
| 1937 | − .send_hbm_bad_channel_flag = smu_v14_0_2_send_bad_mem_channel_flag, | |
| 1938 | 1973 | #if 0 |
| 1939 | 1974 | .gpo_control = smu_v14_0_gpo_control, |
| 1940 | 1975 | #endif |
| 1941 | − .get_ecc_info = smu_v14_0_2_get_ecc_info, | |
| 1942 | 1976 | }; |
| 1943 | 1977 | |
| 1944 | 1978 | void smu_v14_0_2_set_ppt_funcs(struct smu_context *smu) |
| @@ -794,7 +794,7 @@ static const char *smu_get_feature_name(struct smu_context *smu, · 793 unmodified lines | ||
| 794 | 794 | size_t smu_cmn_get_pp_feature_mask(struct smu_context *smu, |
| 795 | 795 | char *buf) |
| 796 | 796 | { |
| 797 | − int8_t sort_feature[max(SMU_FEATURE_COUNT, SMU_FEATURE_MAX)]; | |
| 797 | + int8_t sort_feature[MAX(SMU_FEATURE_COUNT, SMU_FEATURE_MAX)]; | |
| 798 | 798 | uint64_t feature_mask; |
| 799 | 799 | int i, feature_index; |
| 800 | 800 | uint32_t count = 0; |
| @@ -158,7 +158,14 @@ void ast_dp_launch(struct drm_device *dev) · 157 unmodified lines | ||
| 158 | 158 | ASTDP_HOST_EDID_READ_DONE); |
| 159 | 159 | } |
| 160 | 160 | |
| 161 | +bool ast_dp_power_is_on(struct ast_device *ast) | |
| 162 | +{ | |
| 163 | + u8 vgacre3; | |
| 164 | + | |
| 165 | + vgacre3 = ast_get_index_reg(ast, AST_IO_VGACRI, 0xe3); | |
| 161 | 166 | |
| 167 | + return !(vgacre3 & AST_DP_PHY_SLEEP); | |
| 168 | +} | |
| 162 | 169 | |
| 163 | 170 | void ast_dp_power_on_off(struct drm_device *dev, bool on) |
| 164 | 171 | { |
| @@ -391,6 +391,11 @@ static int ast_drm_freeze(struct drm_device *dev) · 390 unmodified lines | ||
| 391 | 391 | |
| 392 | 392 | static int ast_drm_thaw(struct drm_device *dev) |
| 393 | 393 | { |
| 394 | + struct ast_device *ast = to_ast_device(dev); | |
| 395 | + | |
| 396 | + ast_enable_vga(ast->ioregs); | |
| 397 | + ast_open_key(ast->ioregs); | |
| 398 | + ast_enable_mmio(dev->dev, ast->ioregs); | |
| 394 | 399 | ast_post_gpu(dev); |
| 395 | 400 | |
| 396 | 401 | return drm_mode_config_helper_resume(dev); |
| @@ -472,6 +472,7 @@ void ast_init_3rdtx(struct drm_device *dev); · 471 unmodified lines | ||
| 472 | 472 | bool ast_astdp_is_connected(struct ast_device *ast); |
| 473 | 473 | int ast_astdp_read_edid(struct drm_device *dev, u8 *ediddata); |
| 474 | 474 | void ast_dp_launch(struct drm_device *dev); |
| 475 | +bool ast_dp_power_is_on(struct ast_device *ast); | |
| 475 | 476 | void ast_dp_power_on_off(struct drm_device *dev, bool no); |
| 476 | 477 | void ast_dp_set_on_off(struct drm_device *dev, bool no); |
| 477 | 478 | void ast_dp_set_mode(struct drm_crtc *crtc, struct ast_vbios_mode_info *vbios_mode); |
| @@ -28,6 +28,7 @@ · 27 unmodified lines | ||
| 28 | 28 | * Authors: Dave Airlie <airlied@redhat.com> |
| 29 | 29 | */ |
| 30 | 30 | |
| 31 | +#include <linux/delay.h> | |
| 31 | 32 | #include <linux/export.h> |
| 32 | 33 | #include <linux/pci.h> |
| 33 | 34 | |
| @@ -1687,11 +1688,35 @@ static int ast_astdp_connector_helper_detect_ctx(struct drm_connector *connector · 1653 unmodified lines | ||
| 1687 | 1688 | struct drm_modeset_acquire_ctx *ctx, |
| 1688 | 1689 | bool force) |
| 1689 | 1690 | { |
| 1691 | + struct drm_device *dev = connector->dev; | |
| 1690 | 1692 | struct ast_device *ast = to_ast_device(connector->dev); |
| 1693 | + enum drm_connector_status status = connector_status_disconnected; | |
| 1694 | + struct drm_connector_state *connector_state = connector->state; | |
| 1695 | + bool is_active = false; | |
| 1696 | + | |
| 1697 | + mutex_lock(&ast->modeset_lock); | |
| 1698 | + | |
| 1699 | + if (connector_state && connector_state->crtc) { | |
| 1700 | + struct drm_crtc_state *crtc_state = connector_state->crtc->state; | |
| 1701 | + | |
| 1702 | + if (crtc_state && crtc_state->active) | |
| 1703 | + is_active = true; | |
| 1704 | + } | |
| 1705 | + | |
| 1706 | + if (!is_active && !ast_dp_power_is_on(ast)) { | |
| 1707 | + ast_dp_power_on_off(dev, true); | |
| 1708 | + msleep(50); | |
| 1709 | + } | |
| 1691 | 1710 | |
| 1692 | 1711 | if (ast_astdp_is_connected(ast)) |
| 1693 | − return connector_status_connected; | |
| 1694 | − return connector_status_disconnected; | |
| 1712 | + status = connector_status_connected; | |
| 1713 | + | |
| 1714 | + if (!is_active && status == connector_status_disconnected) | |
| 1715 | + ast_dp_power_on_off(dev, false); | |
| 1716 | + | |
| 1717 | + mutex_unlock(&ast->modeset_lock); | |
| 1718 | + | |
| 1719 | + return status; | |
| 1695 | 1720 | } |
| 1696 | 1721 | |
| 1697 | 1722 | static const struct drm_connector_helper_funcs ast_astdp_connector_helper_funcs = { |
| @@ -1070,21 +1070,17 @@ int drm_atomic_set_property(struct drm_atomic_state *state, · 1069 unmodified lines | ||
| 1070 | 1070 | break; |
| 1071 | 1071 | } |
| 1072 | 1072 | |
| 1073 | − if (async_flip && prop != config->prop_fb_id) { | |
| 1073 | + if (async_flip && | |
| 1074 | + (plane_state->plane->type != DRM_PLANE_TYPE_PRIMARY || | |
| 1075 | + (prop != config->prop_fb_id && | |
| 1076 | + prop != config->prop_in_fence_fd && | |
| 1077 | + prop != config->prop_fb_damage_clips))) { | |
| 1074 | 1078 | ret = drm_atomic_plane_get_property(plane, plane_state, |
| 1075 | 1079 | prop, &old_val); |
| 1076 | 1080 | ret = drm_atomic_check_prop_changes(ret, old_val, prop_value, prop); |
| 1077 | 1081 | break; |
| 1078 | 1082 | } |
| 1079 | 1083 | |
| 1080 | − if (async_flip && plane_state->plane->type != DRM_PLANE_TYPE_PRIMARY) { | |
| 1081 | − drm_dbg_atomic(prop->dev, | |
| 1082 | − "[OBJECT:%d] Only primary planes can be changed during async flip\n", | |
| 1083 | − obj->id); | |
| 1084 | − ret = -EINVAL; | |
| 1085 | − break; | |
| 1086 | − } | |
| 1087 | − | |
| 1088 | 1084 | ret = drm_atomic_plane_set_property(plane, |
| 1089 | 1085 | plane_state, file_priv, |
| 1090 | 1086 | prop, prop_value); |
| @@ -443,10 +443,8 @@ struct drm_connector *drm_bridge_connector_init(struct drm_device *drm, · 442 unmodified lines | ||
| 443 | 443 | panel_bridge = bridge; |
| 444 | 444 | } |
| 445 | 445 | |
| 446 | − if (connector_type == DRM_MODE_CONNECTOR_Unknown) { | |
| 447 | − kfree(bridge_connector); | |
| 446 | + if (connector_type == DRM_MODE_CONNECTOR_Unknown) | |
| 448 | 447 | return ERR_PTR(-EINVAL); |
| 449 | − } | |
| 450 | 448 | |
| 451 | 449 | if (bridge_connector->bridge_hdmi) |
| 452 | 450 | ret = drmm_connector_hdmi_init(drm, connector, |
| @@ -461,10 +459,8 @@ struct drm_connector *drm_bridge_connector_init(struct drm_device *drm, · 8 unmodified lines | ||
| 461 | 459 | ret = drmm_connector_init(drm, connector, |
| 462 | 460 | &drm_bridge_connector_funcs, |
| 463 | 461 | connector_type, ddc); |
| 464 | − if (ret) { | |
| 465 | − kfree(bridge_connector); | |
| 462 | + if (ret) | |
| 466 | 463 | return ERR_PTR(ret); |
| 467 | − } | |
| 468 | 464 | |
| 469 | 465 | drm_connector_helper_add(connector, &drm_bridge_connector_helper_funcs); |
| 470 | 466 | |
| @@ -851,6 +851,7 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, · 850 unmodified lines | ||
| 851 | 851 | * drm_buddy_block_trim - free unused pages |
| 852 | 852 | * |
| 853 | 853 | * @mm: DRM buddy manager |
| 854 | + * @start: start address to begin the trimming. | |
| 854 | 855 | * @new_size: original size requested |
| 855 | 856 | * @blocks: Input and output list of allocated blocks. |
| 856 | 857 | * MUST contain single block as input to be trimmed. |
| @@ -866,11 +867,13 @@ static int __alloc_contig_try_harder(struct drm_buddy *mm, · 9 unmodified lines | ||
| 866 | 867 | * 0 on success, error code on failure. |
| 867 | 868 | */ |
| 868 | 869 | int drm_buddy_block_trim(struct drm_buddy *mm, |
| 870 | + u64 *start, | |
| 869 | 871 | u64 new_size, |
| 870 | 872 | struct list_head *blocks) |
| 871 | 873 | { |
| 872 | 874 | struct drm_buddy_block *parent; |
| 873 | 875 | struct drm_buddy_block *block; |
| 876 | + u64 block_start, block_end; | |
| 874 | 877 | LIST_HEAD(dfs); |
| 875 | 878 | u64 new_start; |
| 876 | 879 | int err; |
| @@ -882,6 +885,9 @@ int drm_buddy_block_trim(struct drm_buddy *mm, · 5 unmodified lines | ||
| 882 | 885 | struct drm_buddy_block, |
| 883 | 886 | link); |
| 884 | 887 | |
| 888 | + block_start = drm_buddy_block_offset(block); | |
| 889 | + block_end = block_start + drm_buddy_block_size(mm, block); | |
| 890 | + | |
| 885 | 891 | if (WARN_ON(!drm_buddy_block_is_allocated(block))) |
| 886 | 892 | return -EINVAL; |
| 887 | 893 | |
| @@ -894,6 +900,20 @@ int drm_buddy_block_trim(struct drm_buddy *mm, · 6 unmodified lines | ||
| 894 | 900 | if (new_size == drm_buddy_block_size(mm, block)) |
| 895 | 901 | return 0; |
| 896 | 902 | |
| 903 | + new_start = block_start; | |
| 904 | + if (start) { | |
| 905 | + new_start = *start; | |
| 906 | + | |
| 907 | + if (new_start < block_start) | |
| 908 | + return -EINVAL; | |
| 909 | + | |
| 910 | + if (!IS_ALIGNED(new_start, mm->chunk_size)) | |
| 911 | + return -EINVAL; | |
| 912 | + | |
| 913 | + if (range_overflows(new_start, new_size, block_end)) | |
| 914 | + return -EINVAL; | |
| 915 | + } | |
| 916 | + | |
| 897 | 917 | list_del(&block->link); |
| 898 | 918 | mark_free(mm, block); |
| 899 | 919 | mm->avail += drm_buddy_block_size(mm, block); |
| @@ -904,7 +924,6 @@ int drm_buddy_block_trim(struct drm_buddy *mm, · 4 unmodified lines | ||
| 904 | 924 | parent = block->parent; |
| 905 | 925 | block->parent = NULL; |
| 906 | 926 | |
| 907 | − new_start = drm_buddy_block_offset(block); | |
| 908 | 927 | list_add(&block->tmp_link, &dfs); |
| 909 | 928 | err = __alloc_range(mm, &dfs, new_start, new_size, blocks, NULL); |
| 910 | 929 | if (err) { |
| @@ -1066,7 +1085,8 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, · 155 unmodified lines | ||
| 1066 | 1085 | } while (1); |
| 1067 | 1086 | |
| 1068 | 1087 | /* Trim the allocated block to the required size */ |
| 1069 | − if (original_size != size) { | |
| 1088 | + if (!(flags & DRM_BUDDY_TRIM_DISABLE) && | |
| 1089 | + original_size != size) { | |
| 1070 | 1090 | struct list_head *trim_list; |
| 1071 | 1091 | LIST_HEAD(temp); |
| 1072 | 1092 | u64 trim_size; |
| @@ -1083,6 +1103,7 @@ int drm_buddy_alloc_blocks(struct drm_buddy *mm, · 10 unmodified lines | ||
| 1083 | 1103 | } |
| 1084 | 1104 | |
| 1085 | 1105 | drm_buddy_block_trim(mm, |
| 1106 | + NULL, | |
| 1086 | 1107 | trim_size, |
| 1087 | 1108 | trim_list); |
| 1088 | 1109 | |
| @@ -355,7 +355,7 @@ int drm_client_buffer_vmap_local(struct drm_client_buffer *buffer, · 354 unmodified lines | ||
| 355 | 355 | |
| 356 | 356 | err_drm_gem_vmap_unlocked: |
| 357 | 357 | drm_gem_unlock(gem); |
| 358 | − return 0; | |
| 358 | + return ret; | |
| 359 | 359 | } |
| 360 | 360 | EXPORT_SYMBOL(drm_client_buffer_vmap_local); |
| 361 | 361 | |
| @@ -880,6 +880,11 @@ int drm_client_modeset_probe(struct drm_client_dev *client, unsigned int width, · 879 unmodified lines | ||
| 880 | 880 | |
| 881 | 881 | kfree(modeset->mode); |
| 882 | 882 | modeset->mode = drm_mode_duplicate(dev, mode); |
| 883 | + if (!modeset->mode) { | |
| 884 | + ret = -ENOMEM; | |
| 885 | + break; | |
| 886 | + } | |
| 887 | + | |
| 883 | 888 | drm_connector_get(connector); |
| 884 | 889 | modeset->connectors[modeset->num_connectors++] = connector; |
| 885 | 890 | modeset->x = offset->x; |
| @@ -624,6 +624,17 @@ static void drm_fb_helper_add_damage_clip(struct drm_fb_helper *helper, u32 x, u · 623 unmodified lines | ||
| 624 | 624 | static void drm_fb_helper_damage(struct drm_fb_helper *helper, u32 x, u32 y, |
| 625 | 625 | u32 width, u32 height) |
| 626 | 626 | { |
| 627 | + /* | |
| 628 | + * This function may be invoked by panic() to flush the frame | |
| 629 | + * buffer, where all CPUs except the panic CPU are stopped. | |
| 630 | + * During the following schedule_work(), the panic CPU needs | |
| 631 | + * the worker_pool lock, which might be held by a stopped CPU, | |
| 632 | + * causing schedule_work() and panic() to block. Return early on | |
| 633 | + * oops_in_progress to prevent this blocking. | |
| 634 | + */ | |
| 635 | + if (oops_in_progress) | |
| 636 | + return; | |
| 637 | + | |
| 627 | 638 | drm_fb_helper_add_damage_clip(helper, x, y, width, height); |
| 628 | 639 | |
| 629 | 640 | schedule_work(&helper->damage_work); |
| @@ -208,6 +208,18 @@ static const struct dmi_system_id orientation_data[] = { · 207 unmodified lines | ||
| 208 | 208 | DMI_MATCH(DMI_BOARD_NAME, "KUN"), |
| 209 | 209 | }, |
| 210 | 210 | .driver_data = (void *)&lcd1600x2560_rightside_up, |
| 211 | + }, { /* AYN Loki Max */ | |
| 212 | + .matches = { | |
| 213 | + DMI_EXACT_MATCH(DMI_SYS_VENDOR, "ayn"), | |
| 214 | + DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "Loki Max"), | |
| 215 | + }, | |
| 216 | + .driver_data = (void *)&lcd1080x1920_leftside_up, | |
| 217 | + }, { /* AYN Loki Zero */ | |
| 218 | + .matches = { | |
| 219 | + DMI_EXACT_MATCH(DMI_SYS_VENDOR, "ayn"), | |
| 220 | + DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "Loki Zero"), | |
| 221 | + }, | |
| 222 | + .driver_data = (void *)&lcd1080x1920_leftside_up, | |
| 211 | 223 | }, { /* Chuwi HiBook (CWI514) */ |
| 212 | 224 | .matches = { |
| 213 | 225 | DMI_MATCH(DMI_BOARD_VENDOR, "Hampoo"), |
| @@ -414,6 +426,12 @@ static const struct dmi_system_id orientation_data[] = { · 200 unmodified lines | ||
| 414 | 426 | DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "ONE XPLAYER"), |
| 415 | 427 | }, |
| 416 | 428 | .driver_data = (void *)&lcd1600x2560_leftside_up, |
| 429 | + }, { /* OrangePi Neo */ | |
| 430 | + .matches = { | |
| 431 | + DMI_EXACT_MATCH(DMI_SYS_VENDOR, "OrangePi"), | |
| 432 | + DMI_EXACT_MATCH(DMI_PRODUCT_NAME, "NEO-01"), | |
| 433 | + }, | |
| 434 | + .driver_data = (void *)&lcd1200x1920_rightside_up, | |
| 417 | 435 | }, { /* Samsung GalaxyBook 10.6 */ |
| 418 | 436 | .matches = { |
| 419 | 437 | DMI_EXACT_MATCH(DMI_SYS_VENDOR, "SAMSUNG ELECTRONICS CO., LTD."), |
| @@ -1449,6 +1449,9 @@ bxt_setup_backlight(struct intel_connector *connector, enum pipe unused) · 1448 unmodified lines | ||
| 1449 | 1449 | |
| 1450 | 1450 | static int cnp_num_backlight_controllers(struct drm_i915_private *i915) |
| 1451 | 1451 | { |
| 1452 | + if (INTEL_PCH_TYPE(i915) >= PCH_MTL) | |
| 1453 | + return 2; | |
| 1454 | + | |
| 1452 | 1455 | if (INTEL_PCH_TYPE(i915) >= PCH_DG1) |
| 1453 | 1456 | return 1; |
| 1454 | 1457 | |
| @@ -5935,6 +5935,18 @@ intel_dp_detect(struct drm_connector *connector, · 5934 unmodified lines | ||
| 5935 | 5935 | else |
| 5936 | 5936 | status = connector_status_disconnected; |
| 5937 | 5937 | |
| 5938 | + if (status != connector_status_disconnected && | |
| 5939 | + !intel_dp_mst_verify_dpcd_state(intel_dp)) | |
| 5940 | + /* | |
| 5941 | + * This requires retrying detection for instance to re-enable | |
| 5942 | + * the MST mode that got reset via a long HPD pulse. The retry | |
| 5943 | + * will happen either via the hotplug handler's retry logic, | |
| 5944 | + * ensured by setting the connector here to SST/disconnected, | |
| 5945 | + * or via a userspace connector probing in response to the | |
| 5946 | + * hotplug uevent sent when removing the MST connectors. | |
| 5947 | + */ | |
| 5948 | + status = connector_status_disconnected; | |
| 5949 | + | |
| 5938 | 5950 | if (status == connector_status_disconnected) { |
| 5939 | 5951 | memset(&intel_dp->compliance, 0, sizeof(intel_dp->compliance)); |
| 5940 | 5952 | memset(intel_connector->dp.dsc_dpcd, 0, sizeof(intel_connector->dp.dsc_dpcd)); |
| @@ -39,7 +39,9 @@ static u32 transcoder_to_stream_enc_status(enum transcoder cpu_transcoder) · 38 unmodified lines | ||
| 39 | 39 | static void intel_dp_hdcp_wait_for_cp_irq(struct intel_connector *connector, |
| 40 | 40 | int timeout) |
| 41 | 41 | { |
| 42 | − struct intel_hdcp *hdcp = &connector->hdcp; | |
| 42 | + struct intel_digital_port *dig_port = intel_attached_dig_port(connector); | |
| 43 | + struct intel_dp *dp = &dig_port->dp; | |
| 44 | + struct intel_hdcp *hdcp = &dp->attached_connector->hdcp; | |
| 43 | 45 | long ret; |
| 44 | 46 | |
| 45 | 47 | #define C (hdcp->cp_irq_count_cached != atomic_read(&hdcp->cp_irq_count)) |
| @@ -1998,3 +1998,43 @@ bool intel_dp_mst_crtc_needs_modeset(struct intel_atomic_state *state, · 1997 unmodified lines | ||
| 1998 | 1998 | |
| 1999 | 1999 | return false; |
| 2000 | 2000 | } |
| 2001 | + | |
| 2002 | +/* | |
| 2003 | + * intel_dp_mst_verify_dpcd_state - verify the MST SW enabled state wrt. the DPCD | |
| 2004 | + * @intel_dp: DP port object | |
| 2005 | + * | |
| 2006 | + * Verify if @intel_dp's MST enabled SW state matches the corresponding DPCD | |
| 2007 | + * state. A long HPD pulse - not long enough to be detected as a disconnected | |
| 2008 | + * state - could've reset the DPCD state, which requires tearing | |
| 2009 | + * down/recreating the MST topology. | |
| 2010 | + * | |
| 2011 | + * Returns %true if the SW MST enabled and DPCD states match, %false | |
| 2012 | + * otherwise. | |
| 2013 | + */ | |
| 2014 | +bool intel_dp_mst_verify_dpcd_state(struct intel_dp *intel_dp) | |
| 2015 | +{ | |
| 2016 | + struct intel_display *display = to_intel_display(intel_dp); | |
| 2017 | + struct intel_connector *connector = intel_dp->attached_connector; | |
| 2018 | + struct intel_digital_port *dig_port = dp_to_dig_port(intel_dp); | |
| 2019 | + struct intel_encoder *encoder = &dig_port->base; | |
| 2020 | + int ret; | |
| 2021 | + u8 val; | |
| 2022 | + | |
| 2023 | + if (!intel_dp->is_mst) | |
| 2024 | + return true; | |
| 2025 | + | |
| 2026 | + ret = drm_dp_dpcd_readb(intel_dp->mst_mgr.aux, DP_MSTM_CTRL, &val); | |
| 2027 | + | |
| 2028 | + /* Adjust the expected register value for SST + SideBand. */ | |
| 2029 | + if (ret < 0 || val != (DP_MST_EN | DP_UP_REQ_EN | DP_UPSTREAM_IS_SRC)) { | |
| 2030 | + drm_dbg_kms(display->drm, | |
| 2031 | + "[CONNECTOR:%d:%s][ENCODER:%d:%s] MST mode got reset, removing topology (ret=%d, ctrl=0x%02x)\n", | |
| 2032 | + connector->base.base.id, connector->base.name, | |
| 2033 | + encoder->base.base.id, encoder->base.name, | |
| 2034 | + ret, val); | |
| 2035 | + | |
| 2036 | + return false; | |
| 2037 | + } | |
| 2038 | + | |
| 2039 | + return true; | |
| 2040 | +} | |
| @@ -27,5 +27,6 @@ int intel_dp_mst_atomic_check_link(struct intel_atomic_state *state, · 26 unmodified lines | ||
| 27 | 27 | struct intel_link_bw_limits *limits); |
| 28 | 28 | bool intel_dp_mst_crtc_needs_modeset(struct intel_atomic_state *state, |
| 29 | 29 | struct intel_crtc *crtc); |
| 30 | +bool intel_dp_mst_verify_dpcd_state(struct intel_dp *intel_dp); | |
| 30 | 31 | |
| 31 | 32 | #endif /* __INTEL_DP_MST_H__ */ |
| @@ -1658,7 +1658,7 @@ static void skl_wrpll_params_populate(struct skl_wrpll_params *params, · 1657 unmodified lines | ||
| 1658 | 1658 | } |
| 1659 | 1659 | |
| 1660 | 1660 | static int |
| 1661 | −skl_ddi_calculate_wrpll(int clock /* in Hz */, | |
| 1661 | +skl_ddi_calculate_wrpll(int clock, | |
| 1662 | 1662 | int ref_clock, |
| 1663 | 1663 | struct skl_wrpll_params *wrpll_params) |
| 1664 | 1664 | { |
| @@ -1683,7 +1683,7 @@ skl_ddi_calculate_wrpll(int clock /* in Hz */, · 18 unmodified lines | ||
| 1683 | 1683 | }; |
| 1684 | 1684 | unsigned int dco, d, i; |
| 1685 | 1685 | unsigned int p0, p1, p2; |
| 1686 | − u64 afe_clock = clock * 5; /* AFE Clock is 5x Pixel clock */ | |
| 1686 | + u64 afe_clock = (u64)clock * 1000 * 5; /* AFE Clock is 5x Pixel clock, in Hz */ | |
| 1687 | 1687 | |
| 1688 | 1688 | for (d = 0; d < ARRAY_SIZE(dividers); d++) { |
| 1689 | 1689 | for (dco = 0; dco < ARRAY_SIZE(dco_central_freq); dco++) { |
| @@ -1808,7 +1808,7 @@ static int skl_ddi_hdmi_pll_dividers(struct intel_crtc_state *crtc_state) · 118 unmodified lines | ||
| 1808 | 1808 | struct skl_wrpll_params wrpll_params = {}; |
| 1809 | 1809 | int ret; |
| 1810 | 1810 | |
| 1811 | − ret = skl_ddi_calculate_wrpll(crtc_state->port_clock * 1000, | |
| 1811 | + ret = skl_ddi_calculate_wrpll(crtc_state->port_clock, | |
| 1812 | 1812 | i915->display.dpll.ref_clks.nssc, &wrpll_params); |
| 1813 | 1813 | if (ret) |
| 1814 | 1814 | return ret; |
| @@ -251,7 +251,7 @@ · 250 unmodified lines | ||
| 251 | 251 | #define HDCP2_STREAM_STATUS(dev_priv, trans, port) \ |
| 252 | 252 | (TRANS_HDCP(dev_priv) ? \ |
| 253 | 253 | TRANS_HDCP2_STREAM_STATUS(trans) : \ |
| 254 | − PIPE_HDCP2_STREAM_STATUS(pipe)) | |
| 254 | + PIPE_HDCP2_STREAM_STATUS(port)) | |
| 255 | 255 | |
| 256 | 256 | #define _PORTA_HDCP2_AUTH_STREAM 0x66F00 |
| 257 | 257 | #define _PORTB_HDCP2_AUTH_STREAM 0x66F04 |
| @@ -351,6 +351,9 @@ static int intel_num_pps(struct drm_i915_private *i915) · 350 unmodified lines | ||
| 351 | 351 | if (IS_GEMINILAKE(i915) || IS_BROXTON(i915)) |
| 352 | 352 | return 2; |
| 353 | 353 | |
| 354 | + if (INTEL_PCH_TYPE(i915) >= PCH_MTL) | |
| 355 | + return 2; | |
| 356 | + | |
| 354 | 357 | if (INTEL_PCH_TYPE(i915) >= PCH_DG1) |
| 355 | 358 | return 1; |
| 356 | 359 | |
| @@ -1870,7 +1870,6 @@ static const struct dmi_system_id vlv_dsi_dmi_quirk_table[] = { · 1869 unmodified lines | ||
| 1870 | 1870 | /* Lenovo Yoga Tab 3 Pro YT3-X90F */ |
| 1871 | 1871 | .matches = { |
| 1872 | 1872 | DMI_MATCH(DMI_SYS_VENDOR, "Intel Corporation"), |
| 1873 | − DMI_MATCH(DMI_PRODUCT_NAME, "CHERRYVIEW D1 PLATFORM"), | |
| 1874 | 1873 | DMI_MATCH(DMI_PRODUCT_VERSION, "Blade3-10A-001"), |
| 1875 | 1874 | }, |
| 1876 | 1875 | .driver_data = (void *)vlv_dsi_lenovo_yoga_tab3_backlight_fixup, |
| @@ -290,6 +290,41 @@ out: · 289 unmodified lines | ||
| 290 | 290 | return i915_error_to_vmf_fault(err); |
| 291 | 291 | } |
| 292 | 292 | |
| 293 | +static void set_address_limits(struct vm_area_struct *area, | |
| 294 | + struct i915_vma *vma, | |
| 295 | + unsigned long obj_offset, | |
| 296 | + unsigned long *start_vaddr, | |
| 297 | + unsigned long *end_vaddr) | |
| 298 | +{ | |
| 299 | + unsigned long vm_start, vm_end, vma_size; /* user's memory parameters */ | |
| 300 | + long start, end; /* memory boundaries */ | |
| 301 | + | |
| 302 | + /* | |
| 303 | + * Let's move into the ">> PAGE_SHIFT" | |
| 304 | + * domain to be sure not to lose bits | |
| 305 | + */ | |
| 306 | + vm_start = area->vm_start >> PAGE_SHIFT; | |
| 307 | + vm_end = area->vm_end >> PAGE_SHIFT; | |
| 308 | + vma_size = vma->size >> PAGE_SHIFT; | |
| 309 | + | |
| 310 | + /* | |
| 311 | + * Calculate the memory boundaries by considering the offset | |
| 312 | + * provided by the user during memory mapping and the offset | |
| 313 | + * provided for the partial mapping. | |
| 314 | + */ | |
| 315 | + start = vm_start; | |
| 316 | + start -= obj_offset; | |
| 317 | + start += vma->gtt_view.partial.offset; | |
| 318 | + end = start + vma_size; | |
| 319 | + | |
| 320 | + start = max_t(long, start, vm_start); | |
| 321 | + end = min_t(long, end, vm_end); | |
| 322 | + | |
| 323 | + /* Let's move back into the "<< PAGE_SHIFT" domain */ | |
| 324 | + *start_vaddr = (unsigned long)start << PAGE_SHIFT; | |
| 325 | + *end_vaddr = (unsigned long)end << PAGE_SHIFT; | |
| 326 | +} | |
| 327 | + | |
| 293 | 328 | static vm_fault_t vm_fault_gtt(struct vm_fault *vmf) |
| 294 | 329 | { |
| 295 | 330 | #define MIN_CHUNK_PAGES (SZ_1M >> PAGE_SHIFT) |
| @@ -302,14 +337,18 @@ static vm_fault_t vm_fault_gtt(struct vm_fault *vmf) · 6 unmodified lines | ||
| 302 | 337 | struct i915_ggtt *ggtt = to_gt(i915)->ggtt; |
| 303 | 338 | bool write = area->vm_flags & VM_WRITE; |
| 304 | 339 | struct i915_gem_ww_ctx ww; |
| 340 | + unsigned long obj_offset; | |
| 341 | + unsigned long start, end; /* memory boundaries */ | |
| 305 | 342 | intel_wakeref_t wakeref; |
| 306 | 343 | struct i915_vma *vma; |
| 307 | 344 | pgoff_t page_offset; |
| 345 | + unsigned long pfn; | |
| 308 | 346 | int srcu; |
| 309 | 347 | int ret; |
| 310 | 348 | |
| 311 | − /* We don't use vmf->pgoff since that has the fake offset */ | |
| 349 | + obj_offset = area->vm_pgoff - drm_vma_node_start(&mmo->vma_node); | |
| 312 | 350 | page_offset = (vmf->address - area->vm_start) >> PAGE_SHIFT; |
| 351 | + page_offset += obj_offset; | |
| 313 | 352 | |
| 314 | 353 | trace_i915_gem_object_fault(obj, page_offset, true, write); |
| 315 | 354 | |
| @@ -402,12 +441,14 @@ retry: · 86 unmodified lines | ||
| 402 | 441 | if (ret) |
| 403 | 442 | goto err_unpin; |
| 404 | 443 | |
| 444 | + set_address_limits(area, vma, obj_offset, &start, &end); | |
| 445 | + | |
| 446 | + pfn = (ggtt->gmadr.start + i915_ggtt_offset(vma)) >> PAGE_SHIFT; | |
| 447 | + pfn += (start - area->vm_start) >> PAGE_SHIFT; | |
| 448 | + pfn += obj_offset - vma->gtt_view.partial.offset; | |
| 449 | + | |
| 405 | 450 | /* Finally, remap it using the new GTT offset */ |
| 406 | − ret = remap_io_mapping(area, | |
| 407 | − area->vm_start + (vma->gtt_view.partial.offset << PAGE_SHIFT), | |
| 408 | − (ggtt->gmadr.start + i915_ggtt_offset(vma)) >> PAGE_SHIFT, | |
| 409 | − min_t(u64, vma->size, area->vm_end - area->vm_start), | |
| 410 | − &ggtt->iomap); | |
| 451 | + ret = remap_io_mapping(area, start, pfn, end - start, &ggtt->iomap); | |
| 411 | 452 | if (ret) |
| 412 | 453 | goto err_fence; |
| 413 | 454 | |
| @@ -1084,6 +1125,8 @@ int i915_gem_fb_mmap(struct drm_i915_gem_object *obj, struct vm_area_struct *vma · 670 unmodified lines | ||
| 1084 | 1125 | mmo = mmap_offset_attach(obj, mmap_type, NULL); |
| 1085 | 1126 | if (IS_ERR(mmo)) |
| 1086 | 1127 | return PTR_ERR(mmo); |
| 1128 | + | |
| 1129 | + vma->vm_pgoff += drm_vma_node_start(&mmo->vma_node); | |
| 1087 | 1130 | } |
| 1088 | 1131 | |
| 1089 | 1132 | /* |
| @@ -165,7 +165,6 @@ i915_ttm_placement_from_obj(const struct drm_i915_gem_object *obj, · 164 unmodified lines | ||
| 165 | 165 | i915_ttm_place_from_region(num_allowed ? obj->mm.placements[0] : |
| 166 | 166 | obj->mm.region, &places[0], obj->bo_offset, |
| 167 | 167 | obj->base.size, flags); |
| 168 | − places[0].flags |= TTM_PL_FLAG_DESIRED; | |
| 169 | 168 | |
| 170 | 169 | /* Cache this on object? */ |
| 171 | 170 | for (i = 0; i < num_allowed; ++i) { |
| @@ -779,13 +778,16 @@ static int __i915_ttm_get_pages(struct drm_i915_gem_object *obj, · 607 unmodified lines | ||
| 779 | 778 | .interruptible = true, |
| 780 | 779 | .no_wait_gpu = false, |
| 781 | 780 | }; |
| 782 | − int real_num_busy; | |
| 781 | + struct ttm_placement initial_placement; | |
| 782 | + struct ttm_place initial_place; | |
| 783 | 783 | int ret; |
| 784 | 784 | |
| 785 | 785 | /* First try only the requested placement. No eviction. */ |
| 786 | − real_num_busy = placement->num_placement; | |
| 787 | − placement->num_placement = 1; | |
| 788 | − ret = ttm_bo_validate(bo, placement, &ctx); | |
| 786 | + initial_placement.num_placement = 1; | |
| 787 | + memcpy(&initial_place, placement->placement, sizeof(struct ttm_place)); | |
| 788 | + initial_place.flags |= TTM_PL_FLAG_DESIRED; | |
| 789 | + initial_placement.placement = &initial_place; | |
| 790 | + ret = ttm_bo_validate(bo, &initial_placement, &ctx); | |
| 789 | 791 | if (ret) { |
| 790 | 792 | ret = i915_ttm_err_to_gem(ret); |
| 791 | 793 | /* |
| @@ -800,7 +802,6 @@ static int __i915_ttm_get_pages(struct drm_i915_gem_object *obj, · 8 unmodified lines | ||
| 800 | 802 | * If the initial attempt fails, allow all accepted placements, |
| 801 | 803 | * evicting if necessary. |
| 802 | 804 | */ |
| 803 | − placement->num_placement = real_num_busy; | |
| 804 | 805 | ret = ttm_bo_validate(bo, placement, &ctx); |
| 805 | 806 | if (ret) |
| 806 | 807 | return i915_ttm_err_to_gem(ret); |
| @@ -212,6 +212,37 @@ int intel_gsc_fw_get_binary_info(struct intel_uc_fw *gsc_fw, const void *data, s · 211 unmodified lines | ||
| 212 | 212 | } |
| 213 | 213 | } |
| 214 | 214 | |
| 215 | + if (IS_ARROWLAKE(gt->i915)) { | |
| 216 | + bool too_old = false; | |
| 217 | + | |
| 218 | + /* | |
| 219 | + * ARL requires a newer firmware than MTL did (102.0.10.1878) but the | |
| 220 | + * firmware is actually common. So, need to do an explicit version check | |
| 221 | + * here rather than using a separate table entry. And if the older | |
| 222 | + * MTL-only version is found, then just don't use GSC rather than aborting | |
| 223 | + * the driver load. | |
| 224 | + */ | |
| 225 | + if (gsc->release.major < 102) { | |
| 226 | + too_old = true; | |
| 227 | + } else if (gsc->release.major == 102) { | |
| 228 | + if (gsc->release.minor == 0) { | |
| 229 | + if (gsc->release.patch < 10) { | |
| 230 | + too_old = true; | |
| 231 | + } else if (gsc->release.patch == 10) { | |
| 232 | + if (gsc->release.build < 1878) | |
| 233 | + too_old = true; | |
| 234 | + } | |
| 235 | + } | |
| 236 | + } | |
| 237 | + | |
| 238 | + if (too_old) { | |
| 239 | + gt_info(gt, "GSC firmware too old for ARL, got %d.%d.%d.%d but need at least 102.0.10.1878", | |
| 240 | + gsc->release.major, gsc->release.minor, | |
| 241 | + gsc->release.patch, gsc->release.build); | |
| 242 | + return -EINVAL; | |
| 243 | + } | |
| 244 | + } | |
| 245 | + | |
| 215 | 246 | return 0; |
| 216 | 247 | } |
| 217 | 248 | |
| @@ -698,12 +698,18 @@ static int check_gsc_manifest(struct intel_gt *gt, · 697 unmodified lines | ||
| 698 | 698 | const struct firmware *fw, |
| 699 | 699 | struct intel_uc_fw *uc_fw) |
| 700 | 700 | { |
| 701 | + int ret; | |
| 702 | + | |
| 701 | 703 | switch (uc_fw->type) { |
| 702 | 704 | case INTEL_UC_FW_TYPE_HUC: |
| 703 | − intel_huc_fw_get_binary_info(uc_fw, fw->data, fw->size); | |
| 705 | + ret = intel_huc_fw_get_binary_info(uc_fw, fw->data, fw->size); | |
| 706 | + if (ret) | |
| 707 | + return ret; | |
| 704 | 708 | break; |
| 705 | 709 | case INTEL_UC_FW_TYPE_GSC: |
| 706 | − intel_gsc_fw_get_binary_info(uc_fw, fw->data, fw->size); | |
| 710 | + ret = intel_gsc_fw_get_binary_info(uc_fw, fw->data, fw->size); | |
| 711 | + if (ret) | |
| 712 | + return ret; | |
| 707 | 713 | break; |
| 708 | 714 | default: |
| 709 | 715 | MISSING_CASE(uc_fw->type); |
| @@ -546,6 +546,8 @@ IS_SUBPLATFORM(const struct drm_i915_private *i915, · 545 unmodified lines | ||
| 546 | 546 | #define IS_LUNARLAKE(i915) (0 && i915) |
| 547 | 547 | #define IS_BATTLEMAGE(i915) (0 && i915) |
| 548 | 548 | |
| 549 | +#define IS_ARROWLAKE(i915) \ | |
| 550 | + IS_SUBPLATFORM(i915, INTEL_METEORLAKE, INTEL_SUBPLATFORM_ARL) | |
| 549 | 551 | #define IS_DG2_G10(i915) \ |
| 550 | 552 | IS_SUBPLATFORM(i915, INTEL_DG2, INTEL_SUBPLATFORM_G10) |
| 551 | 553 | #define IS_DG2_G11(i915) \ |
| @@ -2748,26 +2748,6 @@ oa_configure_all_contexts(struct i915_perf_stream *stream, · 2747 unmodified lines | ||
| 2748 | 2748 | return 0; |
| 2749 | 2749 | } |
| 2750 | 2750 | |
| 2751 | −static int | |
| 2752 | −gen12_configure_all_contexts(struct i915_perf_stream *stream, | |
| 2753 | − const struct i915_oa_config *oa_config, | |
| 2754 | − struct i915_active *active) | |
| 2755 | −{ | |
| 2756 | − struct flex regs[] = { | |
| 2757 | − { | |
| 2758 | − GEN8_R_PWR_CLK_STATE(RENDER_RING_BASE), | |
| 2759 | − CTX_R_PWR_CLK_STATE, | |
| 2760 | − }, | |
| 2761 | − }; | |
| 2762 | − | |
| 2763 | − if (stream->engine->class != RENDER_CLASS) | |
| 2764 | − return 0; | |
| 2765 | − | |
| 2766 | − return oa_configure_all_contexts(stream, | |
| 2767 | − regs, ARRAY_SIZE(regs), | |
| 2768 | − active); | |
| 2769 | −} | |
| 2770 | − | |
| 2771 | 2751 | static int |
| 2772 | 2752 | lrc_configure_all_contexts(struct i915_perf_stream *stream, |
| 2773 | 2753 | const struct i915_oa_config *oa_config, |
| @@ -2874,7 +2854,6 @@ gen12_enable_metric_set(struct i915_perf_stream *stream, · 100 unmodified lines | ||
| 2874 | 2854 | { |
| 2875 | 2855 | struct drm_i915_private *i915 = stream->perf->i915; |
| 2876 | 2856 | struct intel_uncore *uncore = stream->uncore; |
| 2877 | − struct i915_oa_config *oa_config = stream->oa_config; | |
| 2878 | 2857 | bool periodic = stream->periodic; |
| 2879 | 2858 | u32 period_exponent = stream->period_exponent; |
| 2880 | 2859 | u32 sqcnt1; |
| @@ -2918,15 +2897,6 @@ gen12_enable_metric_set(struct i915_perf_stream *stream, · 37 unmodified lines | ||
| 2918 | 2897 | |
| 2919 | 2898 | intel_uncore_rmw(uncore, GEN12_SQCNT1, 0, sqcnt1); |
| 2920 | 2899 | |
| 2921 | − /* | |
| 2922 | − * Update all contexts prior writing the mux configurations as we need | |
| 2923 | − * to make sure all slices/subslices are ON before writing to NOA | |
| 2924 | − * registers. | |
| 2925 | − */ | |
| 2926 | − ret = gen12_configure_all_contexts(stream, oa_config, active); | |
| 2927 | − if (ret) | |
| 2928 | − return ret; | |
| 2929 | − | |
| 2930 | 2900 | /* |
| 2931 | 2901 | * For Gen12, performance counters are context |
| 2932 | 2902 | * saved/restored. Only enable it for the context that |
| @@ -2980,9 +2950,6 @@ static void gen12_disable_metric_set(struct i915_perf_stream *stream) · 47 unmodified lines | ||
| 2980 | 2950 | _MASKED_BIT_DISABLE(GEN12_DISABLE_DOP_GATING)); |
| 2981 | 2951 | } |
| 2982 | 2952 | |
| 2983 | − /* Reset all contexts' slices/subslices configurations. */ | |
| 2984 | − gen12_configure_all_contexts(stream, NULL, NULL); | |
| 2985 | − | |
| 2986 | 2953 | /* disable the context save/restore or OAR counters */ |
| 2987 | 2954 | if (stream->ctx) |
| 2988 | 2955 | gen12_configure_oar_context(stream, NULL); |
| @@ -203,6 +203,10 @@ static const u16 subplatform_g12_ids[] = { · 202 unmodified lines | ||
| 203 | 203 | INTEL_DG2_G12_IDS(ID), |
| 204 | 204 | }; |
| 205 | 205 | |
| 206 | +static const u16 subplatform_arl_ids[] = { | |
| 207 | + INTEL_ARL_IDS(ID), | |
| 208 | +}; | |
| 209 | + | |
| 206 | 210 | static bool find_devid(u16 id, const u16 *p, unsigned int num) |
| 207 | 211 | { |
| 208 | 212 | for (; num; num--, p++) { |
| @@ -260,6 +264,9 @@ static void intel_device_info_subplatform_init(struct drm_i915_private *i915) · 51 unmodified lines | ||
| 260 | 264 | } else if (find_devid(devid, subplatform_g12_ids, |
| 261 | 265 | ARRAY_SIZE(subplatform_g12_ids))) { |
| 262 | 266 | mask = BIT(INTEL_SUBPLATFORM_G12); |
| 267 | + } else if (find_devid(devid, subplatform_arl_ids, | |
| 268 | + ARRAY_SIZE(subplatform_arl_ids))) { | |
| 269 | + mask = BIT(INTEL_SUBPLATFORM_ARL); | |
| 263 | 270 | } |
| 264 | 271 | |
| 265 | 272 | GEM_BUG_ON(mask & ~INTEL_SUBPLATFORM_MASK); |
| @@ -127,6 +127,9 @@ enum intel_platform { · 126 unmodified lines | ||
| 127 | 127 | #define INTEL_SUBPLATFORM_N 1 |
| 128 | 128 | #define INTEL_SUBPLATFORM_RPLU 2 |
| 129 | 129 | |
| 130 | +/* MTL */ | |
| 131 | +#define INTEL_SUBPLATFORM_ARL 0 | |
| 132 | + | |
| 130 | 133 | enum intel_ppgtt_type { |
| 131 | 134 | INTEL_PPGTT_NONE = I915_GEM_PPGTT_NONE, |
| 132 | 135 | INTEL_PPGTT_ALIASING = I915_GEM_PPGTT_ALIASING, |
| @@ -539,8 +539,8 @@ static int mtk_drm_kms_init(struct drm_device *drm) · 538 unmodified lines | ||
| 539 | 539 | } |
| 540 | 540 | |
| 541 | 541 | /* IGT will check if the cursor size is configured */ |
| 542 | − drm->mode_config.cursor_width = drm->mode_config.max_width; | |
| 543 | − drm->mode_config.cursor_height = drm->mode_config.max_height; | |
| 542 | + drm->mode_config.cursor_width = 512; | |
| 543 | + drm->mode_config.cursor_height = 512; | |
| 544 | 544 | |
| 545 | 545 | /* Use OVL device for all DMA memory allocations */ |
| 546 | 546 | crtc = drm_crtc_from_index(drm, 0); |
| @@ -99,7 +99,7 @@ static int zap_shader_load_mdt(struct msm_gpu *gpu, const char *fwname, · 98 unmodified lines | ||
| 99 | 99 | * was a bad idea, and is only provided for backwards |
| 100 | 100 | * compatibility for older targets. |
| 101 | 101 | */ |
| 102 | − return -ENODEV; | |
| 102 | + return -ENOENT; | |
| 103 | 103 | } |
| 104 | 104 | |
| 105 | 105 | if (IS_ERR(fw)) { |
| @@ -1171,8 +1171,6 @@ static void dpu_encoder_virt_atomic_mode_set(struct drm_encoder *drm_enc, · 1170 unmodified lines | ||
| 1171 | 1171 | |
| 1172 | 1172 | cstate->num_mixers = num_lm; |
| 1173 | 1173 | |
| 1174 | − dpu_enc->connector = conn_state->connector; | |
| 1175 | − | |
| 1176 | 1174 | for (i = 0; i < dpu_enc->num_phys_encs; i++) { |
| 1177 | 1175 | struct dpu_encoder_phys *phys = dpu_enc->phys_encs[i]; |
| 1178 | 1176 | |
| @@ -1270,6 +1268,8 @@ static void dpu_encoder_virt_atomic_enable(struct drm_encoder *drm_enc, · 91 unmodified lines | ||
| 1270 | 1268 | |
| 1271 | 1269 | dpu_enc->commit_done_timedout = false; |
| 1272 | 1270 | |
| 1271 | + dpu_enc->connector = drm_atomic_get_new_connector_for_encoder(state, drm_enc); | |
| 1272 | + | |
| 1273 | 1273 | cur_mode = &dpu_enc->base.crtc->state->adjusted_mode; |
| 1274 | 1274 | |
| 1275 | 1275 | dpu_enc->wide_bus_en = dpu_encoder_is_widebus_enabled(drm_enc); |
| @@ -308,8 +308,8 @@ static const u32 wb2_formats_rgb_yuv[] = { · 307 unmodified lines | ||
| 308 | 308 | { \ |
| 309 | 309 | .maxdwnscale = SSPP_UNITY_SCALE, \ |
| 310 | 310 | .maxupscale = SSPP_UNITY_SCALE, \ |
| 311 | − .format_list = plane_formats_yuv, \ | |
| 312 | − .num_formats = ARRAY_SIZE(plane_formats_yuv), \ | |
| 311 | + .format_list = plane_formats, \ | |
| 312 | + .num_formats = ARRAY_SIZE(plane_formats), \ | |
| 313 | 313 | .virt_format_list = plane_formats, \ |
| 314 | 314 | .virt_num_formats = ARRAY_SIZE(plane_formats), \ |
| 315 | 315 | } |
| @@ -31,24 +31,14 @@ · 30 unmodified lines | ||
| 31 | 31 | * @fmt: Pointer to format string |
| 32 | 32 | */ |
| 33 | 33 | #define DPU_DEBUG(fmt, ...) \ |
| 34 | − do { \ | |
| 35 | − if (drm_debug_enabled(DRM_UT_KMS)) \ | |
| 36 | − DRM_DEBUG(fmt, ##__VA_ARGS__); \ | |
| 37 | − else \ | |
| 38 | − pr_debug(fmt, ##__VA_ARGS__); \ | |
| 39 | − } while (0) | |
| 34 | + DRM_DEBUG_DRIVER(fmt, ##__VA_ARGS__) | |
| 40 | 35 | |
| 41 | 36 | /** |
| 42 | 37 | * DPU_DEBUG_DRIVER - macro for hardware driver logging |
| 43 | 38 | * @fmt: Pointer to format string |
| 44 | 39 | */ |
| 45 | 40 | #define DPU_DEBUG_DRIVER(fmt, ...) \ |
| 46 | − do { \ | |
| 47 | − if (drm_debug_enabled(DRM_UT_DRIVER)) \ | |
| 48 | − DRM_ERROR(fmt, ##__VA_ARGS__); \ | |
| 49 | − else \ | |
| 50 | − pr_debug(fmt, ##__VA_ARGS__); \ | |
| 51 | − } while (0) | |
| 41 | + DRM_DEBUG_DRIVER(fmt, ##__VA_ARGS__) | |
| 52 | 42 | |
| 53 | 43 | #define DPU_ERROR(fmt, ...) pr_err("[dpu error]" fmt, ##__VA_ARGS__) |
| 54 | 44 | #define DPU_ERROR_RATELIMITED(fmt, ...) pr_err_ratelimited("[dpu error]" fmt, ##__VA_ARGS__) |
| @@ -681,6 +681,9 @@ static int dpu_plane_prepare_fb(struct drm_plane *plane, · 680 unmodified lines | ||
| 681 | 681 | new_state->fb, &layout); |
| 682 | 682 | if (ret) { |
| 683 | 683 | DPU_ERROR_PLANE(pdpu, "failed to get format layout, %d\n", ret); |
| 684 | + if (pstate->aspace) | |
| 685 | + msm_framebuffer_cleanup(new_state->fb, pstate->aspace, | |
| 686 | + pstate->needs_dirtyfb); | |
| 684 | 687 | return ret; |
| 685 | 688 | } |
| 686 | 689 | |
| @@ -744,10 +747,9 @@ static int dpu_plane_atomic_check_pipe(struct dpu_plane *pdpu, · 57 unmodified lines | ||
| 744 | 747 | min_src_size = MSM_FORMAT_IS_YUV(fmt) ? 2 : 1; |
| 745 | 748 | |
| 746 | 749 | if (MSM_FORMAT_IS_YUV(fmt) && |
| 747 | − (!pipe->sspp->cap->sblk->scaler_blk.len || | |
| 748 | − !pipe->sspp->cap->sblk->csc_blk.len)) { | |
| 750 | + !pipe->sspp->cap->sblk->csc_blk.len) { | |
| 749 | 751 | DPU_DEBUG_PLANE(pdpu, |
| 750 | − "plane doesn't have scaler/csc for yuv\n"); | |
| 752 | + "plane doesn't have csc for yuv\n"); | |
| 751 | 753 | return -EINVAL; |
| 752 | 754 | } |
| 753 | 755 | |
| @@ -864,6 +866,10 @@ static int dpu_plane_atomic_check(struct drm_plane *plane, · 110 unmodified lines | ||
| 864 | 866 | |
| 865 | 867 | max_linewidth = pdpu->catalog->caps->max_linewidth; |
| 866 | 868 | |
| 869 | + drm_rect_rotate(&pipe_cfg->src_rect, | |
| 870 | + new_plane_state->fb->width, new_plane_state->fb->height, | |
| 871 | + new_plane_state->rotation); | |
| 872 | + | |
| 867 | 873 | if ((drm_rect_width(&pipe_cfg->src_rect) > max_linewidth) || |
| 868 | 874 | _dpu_plane_calc_clk(&crtc_state->adjusted_mode, pipe_cfg) > max_mdp_clk_rate) { |
| 869 | 875 | /* |
| @@ -913,6 +919,14 @@ static int dpu_plane_atomic_check(struct drm_plane *plane, · 43 unmodified lines | ||
| 913 | 919 | r_pipe_cfg->dst_rect.x1 = pipe_cfg->dst_rect.x2; |
| 914 | 920 | } |
| 915 | 921 | |
| 922 | + drm_rect_rotate_inv(&pipe_cfg->src_rect, | |
| 923 | + new_plane_state->fb->width, new_plane_state->fb->height, | |
| 924 | + new_plane_state->rotation); | |
| 925 | + if (r_pipe->sspp) | |
| 926 | + drm_rect_rotate_inv(&r_pipe_cfg->src_rect, | |
| 927 | + new_plane_state->fb->width, new_plane_state->fb->height, | |
| 928 | + new_plane_state->rotation); | |
| 929 | + | |
| 916 | 930 | ret = dpu_plane_atomic_check_pipe(pdpu, pipe, pipe_cfg, fmt, &crtc_state->adjusted_mode); |
| 917 | 931 | if (ret) |
| 918 | 932 | return ret; |
| @@ -1286,6 +1286,8 @@ static int dp_ctrl_link_train(struct dp_ctrl_private *ctrl, · 1285 unmodified lines | ||
| 1286 | 1286 | link_info.rate = ctrl->link->link_params.rate; |
| 1287 | 1287 | link_info.capabilities = DP_LINK_CAP_ENHANCED_FRAMING; |
| 1288 | 1288 | |
| 1289 | + dp_link_reset_phy_params_vx_px(ctrl->link); | |
| 1290 | + | |
| 1289 | 1291 | dp_aux_link_configure(ctrl->aux, &link_info); |
| 1290 | 1292 | |
| 1291 | 1293 | if (drm_dp_max_downspread(dpcd)) |
| @@ -90,22 +90,22 @@ static int dp_panel_read_dpcd(struct dp_panel *dp_panel) · 89 unmodified lines | ||
| 90 | 90 | static u32 dp_panel_get_supported_bpp(struct dp_panel *dp_panel, |
| 91 | 91 | u32 mode_edid_bpp, u32 mode_pclk_khz) |
| 92 | 92 | { |
| 93 | − struct dp_link_info *link_info; | |
| 93 | + const struct dp_link_info *link_info; | |
| 94 | 94 | const u32 max_supported_bpp = 30, min_supported_bpp = 18; |
| 95 | − u32 bpp = 0, data_rate_khz = 0; | |
| 95 | + u32 bpp, data_rate_khz; | |
| 96 | 96 | |
| 97 | − bpp = min_t(u32, mode_edid_bpp, max_supported_bpp); | |
| 97 | + bpp = min(mode_edid_bpp, max_supported_bpp); | |
| 98 | 98 | |
| 99 | 99 | link_info = &dp_panel->link_info; |
| 100 | 100 | data_rate_khz = link_info->num_lanes * link_info->rate * 8; |
| 101 | 101 | |
| 102 | − while (bpp > min_supported_bpp) { | |
| 102 | + do { | |
| 103 | 103 | if (mode_pclk_khz * bpp <= data_rate_khz) |
| 104 | − break; | |
| 104 | + return bpp; | |
| 105 | 105 | bpp -= 6; |
| 106 | − } | |
| 106 | + } while (bpp > min_supported_bpp); | |
| 107 | 107 | |
| 108 | − return bpp; | |
| 108 | + return min_supported_bpp; | |
| 109 | 109 | } |
| 110 | 110 | |
| 111 | 111 | int dp_panel_read_sink_caps(struct dp_panel *dp_panel, |
| @@ -423,8 +423,9 @@ int dp_panel_init_panel_info(struct dp_panel *dp_panel) · 311 unmodified lines | ||
| 423 | 423 | drm_mode->clock); |
| 424 | 424 | drm_dbg_dp(panel->drm_dev, "bpp = %d\n", dp_panel->dp_mode.bpp); |
| 425 | 425 | |
| 426 | − dp_panel->dp_mode.bpp = max_t(u32, 18, | |
| 427 | − min_t(u32, dp_panel->dp_mode.bpp, 30)); | |
| 426 | + dp_panel->dp_mode.bpp = dp_panel_get_mode_bpp(dp_panel, dp_panel->dp_mode.bpp, | |
| 427 | + dp_panel->dp_mode.drm_mode.clock); | |
| 428 | + | |
| 428 | 429 | drm_dbg_dp(panel->drm_dev, "updated bpp = %d\n", |
| 429 | 430 | dp_panel->dp_mode.bpp); |
| 430 | 431 | |
| @@ -577,7 +577,7 @@ static const struct msm_mdss_data sc7180_data = { · 576 unmodified lines | ||
| 577 | 577 | .ubwc_enc_version = UBWC_2_0, |
| 578 | 578 | .ubwc_dec_version = UBWC_2_0, |
| 579 | 579 | .ubwc_static = 0x1e, |
| 580 | − .highest_bank_bit = 0x3, | |
| 580 | + .highest_bank_bit = 0x1, | |
| 581 | 581 | .reg_bus_bw = 76800, |
| 582 | 582 | }; |
| 583 | 583 | |
| @@ -898,7 +898,7 @@ nouveau_bo_move_m2mf(struct ttm_buffer_object *bo, int evict, · 897 unmodified lines | ||
| 898 | 898 | * Without this the operation can timeout and we'll fallback to a |
| 899 | 899 | * software copy, which might take several minutes to finish. |
| 900 | 900 | */ |
| 901 | − nouveau_fence_wait(fence, false); | |
| 901 | + nouveau_fence_wait(fence, false, false); | |
| 902 | 902 | ret = ttm_bo_move_accel_cleanup(bo, &fence->base, evict, false, |
| 903 | 903 | new_reg); |
| 904 | 904 | nouveau_fence_unref(&fence); |
| @@ -72,7 +72,7 @@ nouveau_channel_idle(struct nouveau_channel *chan) · 71 unmodified lines | ||
| 72 | 72 | |
| 73 | 73 | ret = nouveau_fence_new(&fence, chan); |
| 74 | 74 | if (!ret) { |
| 75 | − ret = nouveau_fence_wait(fence, false); | |
| 75 | + ret = nouveau_fence_wait(fence, false, false); | |
| 76 | 76 | nouveau_fence_unref(&fence); |
| 77 | 77 | } |
| 78 | 78 | |
| @@ -128,7 +128,7 @@ static void nouveau_dmem_page_free(struct page *page) · 127 unmodified lines | ||
| 128 | 128 | static void nouveau_dmem_fence_done(struct nouveau_fence **fence) |
| 129 | 129 | { |
| 130 | 130 | if (fence) { |
| 131 | − nouveau_fence_wait(*fence, false); | |
| 131 | + nouveau_fence_wait(*fence, true, false); | |
| 132 | 132 | nouveau_fence_unref(fence); |
| 133 | 133 | } else { |
| 134 | 134 | /* |
| @@ -311,11 +311,39 @@ nouveau_fence_wait_legacy(struct dma_fence *f, bool intr, long wait) · 310 unmodified lines | ||
| 311 | 311 | return timeout - t; |
| 312 | 312 | } |
| 313 | 313 | |
| 314 | +static int | |
| 315 | +nouveau_fence_wait_busy(struct nouveau_fence *fence, bool intr) | |
| 316 | +{ | |
| 317 | + int ret = 0; | |
| 318 | + | |
| 319 | + while (!nouveau_fence_done(fence)) { | |
| 320 | + if (time_after_eq(jiffies, fence->timeout)) { | |
| 321 | + ret = -EBUSY; | |
| 322 | + break; | |
| 323 | + } | |
| 324 | + | |
| 325 | + __set_current_state(intr ? | |
| 326 | + TASK_INTERRUPTIBLE : | |
| 327 | + TASK_UNINTERRUPTIBLE); | |
| 328 | + | |
| 329 | + if (intr && signal_pending(current)) { | |
| 330 | + ret = -ERESTARTSYS; | |
| 331 | + break; | |
| 332 | + } | |
| 333 | + } | |
| 334 | + | |
| 335 | + __set_current_state(TASK_RUNNING); | |
| 336 | + return ret; | |
| 337 | +} | |
| 338 | + | |
| 314 | 339 | int |
| 315 | −nouveau_fence_wait(struct nouveau_fence *fence, bool intr) | |
| 340 | +nouveau_fence_wait(struct nouveau_fence *fence, bool lazy, bool intr) | |
| 316 | 341 | { |
| 317 | 342 | long ret; |
| 318 | 343 | |
| 344 | + if (!lazy) | |
| 345 | + return nouveau_fence_wait_busy(fence, intr); | |
| 346 | + | |
| 319 | 347 | ret = dma_fence_wait_timeout(&fence->base, intr, 15 * HZ); |
| 320 | 348 | if (ret < 0) |
| 321 | 349 | return ret; |
| @@ -23,7 +23,7 @@ void nouveau_fence_unref(struct nouveau_fence **); · 22 unmodified lines | ||
| 23 | 23 | |
| 24 | 24 | int nouveau_fence_emit(struct nouveau_fence *); |
| 25 | 25 | bool nouveau_fence_done(struct nouveau_fence *); |
| 26 | −int nouveau_fence_wait(struct nouveau_fence *, bool intr); | |
| 26 | +int nouveau_fence_wait(struct nouveau_fence *, bool lazy, bool intr); | |
| 27 | 27 | int nouveau_fence_sync(struct nouveau_bo *, struct nouveau_channel *, bool exclusive, bool intr); |
| 28 | 28 | |
| 29 | 29 | struct nouveau_fence_chan { |
| @@ -928,7 +928,7 @@ revalidate: · 927 unmodified lines | ||
| 928 | 928 | } |
| 929 | 929 | |
| 930 | 930 | if (sync) { |
| 931 | − if (!(ret = nouveau_fence_wait(fence, false))) { | |
| 931 | + if (!(ret = nouveau_fence_wait(fence, false, false))) { | |
| 932 | 932 | if ((ret = dma_fence_get_status(&fence->base)) == 1) |
| 933 | 933 | ret = 0; |
| 934 | 934 | } |
| @@ -64,7 +64,8 @@ struct drm_gem_object *nouveau_gem_prime_import_sg_table(struct drm_device *dev, · 63 unmodified lines | ||
| 64 | 64 | * to the caller, instead of a normal nouveau_bo ttm reference. */ |
| 65 | 65 | ret = drm_gem_object_init(dev, &nvbo->bo.base, size); |
| 66 | 66 | if (ret) { |
| 67 | − nouveau_bo_ref(NULL, &nvbo); | |
| 67 | + drm_gem_object_release(&nvbo->bo.base); | |
| 68 | + kfree(nvbo); | |
| 68 | 69 | obj = ERR_PTR(-ENOMEM); |
| 69 | 70 | goto unlock; |
| 70 | 71 | } |
| @@ -1803,6 +1803,7 @@ nouveau_uvmm_bo_validate(struct drm_gpuvm_bo *vm_bo, struct drm_exec *exec) · 1802 unmodified lines | ||
| 1803 | 1803 | { |
| 1804 | 1804 | struct nouveau_bo *nvbo = nouveau_gem_object(vm_bo->obj); |
| 1805 | 1805 | |
| 1806 | + nouveau_bo_placement_set(nvbo, nvbo->valid_domains, 0); | |
| 1806 | 1807 | return nouveau_bo_validate(nvbo, true, false); |
| 1807 | 1808 | } |
| 1808 | 1809 | |
| @@ -205,7 +205,8 @@ nvkm_firmware_dtor(struct nvkm_firmware *fw) · 204 unmodified lines | ||
| 205 | 205 | break; |
| 206 | 206 | case NVKM_FIRMWARE_IMG_DMA: |
| 207 | 207 | nvkm_memory_unref(&memory); |
| 208 | − dma_free_coherent(fw->device->dev, sg_dma_len(&fw->mem.sgl), fw->img, fw->phys); | |
| 208 | + dma_free_noncoherent(fw->device->dev, sg_dma_len(&fw->mem.sgl), | |
| 209 | + fw->img, fw->phys, DMA_TO_DEVICE); | |
| 209 | 210 | break; |
| 210 | 211 | case NVKM_FIRMWARE_IMG_SGT: |
| 211 | 212 | nvkm_memory_unref(&memory); |
| @@ -236,10 +237,12 @@ nvkm_firmware_ctor(const struct nvkm_firmware_func *func, const char *name, · 24 unmodified lines | ||
| 236 | 237 | break; |
| 237 | 238 | case NVKM_FIRMWARE_IMG_DMA: { |
| 238 | 239 | dma_addr_t addr; |
| 239 | − | |
| 240 | 240 | len = ALIGN(fw->len, PAGE_SIZE); |
| 241 | 241 | |
| 242 | − fw->img = dma_alloc_coherent(fw->device->dev, len, &addr, GFP_KERNEL); | |
| 242 | + fw->img = dma_alloc_noncoherent(fw->device->dev, | |
| 243 | + len, &addr, | |
| 244 | + DMA_TO_DEVICE, | |
| 245 | + GFP_KERNEL); | |
| 243 | 246 | if (fw->img) { |
| 244 | 247 | memcpy(fw->img, src, fw->len); |
| 245 | 248 | fw->phys = addr; |
| @@ -89,6 +89,12 @@ nvkm_falcon_fw_boot(struct nvkm_falcon_fw *fw, struct nvkm_subdev *user, · 88 unmodified lines | ||
| 89 | 89 | nvkm_falcon_fw_dtor_sigs(fw); |
| 90 | 90 | } |
| 91 | 91 | |
| 92 | + /* after last write to the img, sync dma mappings */ | |
| 93 | + dma_sync_single_for_device(fw->fw.device->dev, | |
| 94 | + fw->fw.phys, | |
| 95 | + sg_dma_len(&fw->fw.mem.sgl), | |
| 96 | + DMA_TO_DEVICE); | |
| 97 | + | |
| 92 | 98 | FLCNFW_DBG(fw, "resetting"); |
| 93 | 99 | fw->func->reset(fw); |
| 94 | 100 | |
| @@ -1,6 +1,7 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0-only |
| 2 | 2 | config DRM_OMAP |
| 3 | 3 | tristate "OMAP DRM" |
| 4 | + depends on MMU | |
| 4 | 5 | depends on DRM && OF |
| 5 | 6 | depends on ARCH_OMAP2PLUS || (COMPILE_TEST && PAGE_SIZE_LESS_THAN_64KB) |
| 6 | 7 | select DRM_KMS_HELPER |
| @@ -33,8 +33,10 @@ · 32 unmodified lines | ||
| 33 | 33 | #include "evergreen_reg_safe.h" |
| 34 | 34 | #include "cayman_reg_safe.h" |
| 35 | 35 | |
| 36 | +#ifndef MIN | |
| 36 | 37 | #define MAX(a, b) (((a) > (b)) ? (a) : (b)) |
| 37 | 38 | #define MIN(a, b) (((a) < (b)) ? (a) : (b)) |
| 39 | +#endif | |
| 38 | 40 | |
| 39 | 41 | #define REG_SAFE_BM_SIZE ARRAY_SIZE(evergreen_reg_safe_bm) |
| 40 | 42 | |
| @@ -279,7 +279,6 @@ static int inno_hdmi_upload_frame(struct drm_connector *connector, · 278 unmodified lines | ||
| 279 | 279 | const u8 *buffer, size_t len) |
| 280 | 280 | { |
| 281 | 281 | struct inno_hdmi *hdmi = connector_to_inno_hdmi(connector); |
| 282 | − u8 packed_frame[HDMI_MAXIMUM_INFO_FRAME_SIZE]; | |
| 283 | 282 | ssize_t i; |
| 284 | 283 | |
| 285 | 284 | if (type != HDMI_INFOFRAME_TYPE_AVI) { |
| @@ -291,8 +290,7 @@ static int inno_hdmi_upload_frame(struct drm_connector *connector, · 5 unmodified lines | ||
| 291 | 290 | inno_hdmi_disable_frame(connector, type); |
| 292 | 291 | |
| 293 | 292 | for (i = 0; i < len; i++) |
| 294 | − hdmi_writeb(hdmi, HDMI_CONTROL_PACKET_ADDR + i, | |
| 295 | − packed_frame[i]); | |
| 293 | + hdmi_writeb(hdmi, HDMI_CONTROL_PACKET_ADDR + i, buffer[i]); | |
| 296 | 294 | |
| 297 | 295 | return 0; |
| 298 | 296 | } |
| @@ -102,6 +102,17 @@ static void drm_gem_shmem_test_obj_create_private(struct kunit *test) · 101 unmodified lines | ||
| 102 | 102 | |
| 103 | 103 | sg_init_one(sgt->sgl, buf, TEST_SIZE); |
| 104 | 104 | |
| 105 | + /* | |
| 106 | + * Set the DMA mask to 64-bits and map the sgtables | |
| 107 | + * otherwise drm_gem_shmem_free will cause a warning | |
| 108 | + * on debug kernels. | |
| 109 | + */ | |
| 110 | + ret = dma_set_mask(drm_dev->dev, DMA_BIT_MASK(64)); | |
| 111 | + KUNIT_ASSERT_EQ(test, ret, 0); | |
| 112 | + | |
| 113 | + ret = dma_map_sgtable(drm_dev->dev, sgt, DMA_BIDIRECTIONAL, 0); | |
| 114 | + KUNIT_ASSERT_EQ(test, ret, 0); | |
| 115 | + | |
| 105 | 116 | /* Init a mock DMA-BUF */ |
| 106 | 117 | buf_mock.size = TEST_SIZE; |
| 107 | 118 | attach_mock.dmabuf = &buf_mock; |
| @@ -565,6 +565,10 @@ void v3d_mmu_insert_ptes(struct v3d_bo *bo); · 564 unmodified lines | ||
| 565 | 565 | void v3d_mmu_remove_ptes(struct v3d_bo *bo); |
| 566 | 566 | |
| 567 | 567 | /* v3d_sched.c */ |
| 568 | +void v3d_timestamp_query_info_free(struct v3d_timestamp_query_info *query_info, | |
| 569 | + unsigned int count); | |
| 570 | +void v3d_performance_query_info_free(struct v3d_performance_query_info *query_info, | |
| 571 | + unsigned int count); | |
| 568 | 572 | void v3d_job_update_stats(struct v3d_job *job, enum v3d_queue queue); |
| 569 | 573 | int v3d_sched_init(struct v3d_dev *v3d); |
| 570 | 574 | void v3d_sched_fini(struct v3d_dev *v3d); |
| @@ -73,24 +73,44 @@ v3d_sched_job_free(struct drm_sched_job *sched_job) · 72 unmodified lines | ||
| 73 | 73 | v3d_job_cleanup(job); |
| 74 | 74 | } |
| 75 | 75 | |
| 76 | +void | |
| 77 | +v3d_timestamp_query_info_free(struct v3d_timestamp_query_info *query_info, | |
| 78 | + unsigned int count) | |
| 79 | +{ | |
| 80 | + if (query_info->queries) { | |
| 81 | + unsigned int i; | |
| 82 | + | |
| 83 | + for (i = 0; i < count; i++) | |
| 84 | + drm_syncobj_put(query_info->queries[i].syncobj); | |
| 85 | + | |
| 86 | + kvfree(query_info->queries); | |
| 87 | + } | |
| 88 | +} | |
| 89 | + | |
| 90 | +void | |
| 91 | +v3d_performance_query_info_free(struct v3d_performance_query_info *query_info, | |
| 92 | + unsigned int count) | |
| 93 | +{ | |
| 94 | + if (query_info->queries) { | |
| 95 | + unsigned int i; | |
| 96 | + | |
| 97 | + for (i = 0; i < count; i++) | |
| 98 | + drm_syncobj_put(query_info->queries[i].syncobj); | |
| 99 | + | |
| 100 | + kvfree(query_info->queries); | |
| 101 | + } | |
| 102 | +} | |
| 103 | + | |
| 76 | 104 | static void |
| 77 | 105 | v3d_cpu_job_free(struct drm_sched_job *sched_job) |
| 78 | 106 | { |
| 79 | 107 | struct v3d_cpu_job *job = to_cpu_job(sched_job); |
| 80 | − struct v3d_timestamp_query_info *timestamp_query = &job->timestamp_query; | |
| 81 | − struct v3d_performance_query_info *performance_query = &job->performance_query; | |
| 82 | 108 | |
| 83 | − if (timestamp_query->queries) { | |
| 84 | − for (int i = 0; i < timestamp_query->count; i++) | |
| 85 | − drm_syncobj_put(timestamp_query->queries[i].syncobj); | |
| 86 | − kvfree(timestamp_query->queries); | |
| 87 | − } | |
| 109 | + v3d_timestamp_query_info_free(&job->timestamp_query, | |
| 110 | + job->timestamp_query.count); | |
| 88 | 111 | |
| 89 | − if (performance_query->queries) { | |
| 90 | − for (int i = 0; i < performance_query->count; i++) | |
| 91 | − drm_syncobj_put(performance_query->queries[i].syncobj); | |
| 92 | − kvfree(performance_query->queries); | |
| 93 | − } | |
| 112 | + v3d_performance_query_info_free(&job->performance_query, | |
| 113 | + job->performance_query.count); | |
| 94 | 114 | |
| 95 | 115 | v3d_job_cleanup(&job->base); |
| 96 | 116 | } |
| @@ -114,6 +134,8 @@ v3d_job_start_stats(struct v3d_job *job, enum v3d_queue queue) · 17 unmodified lines | ||
| 114 | 134 | struct v3d_stats *local_stats = &file->stats[queue]; |
| 115 | 135 | u64 now = local_clock(); |
| 116 | 136 | |
| 137 | + preempt_disable(); | |
| 138 | + | |
| 117 | 139 | write_seqcount_begin(&local_stats->lock); |
| 118 | 140 | local_stats->start_ns = now; |
| 119 | 141 | write_seqcount_end(&local_stats->lock); |
| @@ -121,6 +143,8 @@ v3d_job_start_stats(struct v3d_job *job, enum v3d_queue queue) · 1 unmodified line | ||
| 121 | 143 | write_seqcount_begin(&global_stats->lock); |
| 122 | 144 | global_stats->start_ns = now; |
| 123 | 145 | write_seqcount_end(&global_stats->lock); |
| 146 | + | |
| 147 | + preempt_enable(); | |
| 124 | 148 | } |
| 125 | 149 | |
| 126 | 150 | static void |
| @@ -142,8 +166,10 @@ v3d_job_update_stats(struct v3d_job *job, enum v3d_queue queue) · 15 unmodified lines | ||
| 142 | 166 | struct v3d_stats *local_stats = &file->stats[queue]; |
| 143 | 167 | u64 now = local_clock(); |
| 144 | 168 | |
| 169 | + preempt_disable(); | |
| 145 | 170 | v3d_stats_update(local_stats, now); |
| 146 | 171 | v3d_stats_update(global_stats, now); |
| 172 | + preempt_enable(); | |
| 147 | 173 | } |
| 148 | 174 | |
| 149 | 175 | static struct dma_fence *v3d_bin_job_run(struct drm_sched_job *sched_job) |
| @@ -295,7 +321,7 @@ v3d_csd_job_run(struct drm_sched_job *sched_job) · 145 unmodified lines | ||
| 295 | 321 | struct v3d_dev *v3d = job->base.v3d; |
| 296 | 322 | struct drm_device *dev = &v3d->drm; |
| 297 | 323 | struct dma_fence *fence; |
| 298 | − int i, csd_cfg0_reg, csd_cfg_reg_count; | |
| 324 | + int i, csd_cfg0_reg; | |
| 299 | 325 | |
| 300 | 326 | v3d->csd_job = job; |
| 301 | 327 | |
| @@ -315,9 +341,17 @@ v3d_csd_job_run(struct drm_sched_job *sched_job) · 13 unmodified lines | ||
| 315 | 341 | v3d_switch_perfmon(v3d, &job->base); |
| 316 | 342 | |
| 317 | 343 | csd_cfg0_reg = V3D_CSD_QUEUED_CFG0(v3d->ver); |
| 318 | − csd_cfg_reg_count = v3d->ver < 71 ? 6 : 7; | |
| 319 | − for (i = 1; i <= csd_cfg_reg_count; i++) | |
| 344 | + for (i = 1; i <= 6; i++) | |
| 320 | 345 | V3D_CORE_WRITE(0, csd_cfg0_reg + 4 * i, job->args.cfg[i]); |
| 346 | + | |
| 347 | + /* Although V3D 7.1 has an eighth configuration register, we are not | |
| 348 | + * using it. Therefore, make sure it remains unused. | |
| 349 | + * | |
| 350 | + * XXX: Set the CFG7 register | |
| 351 | + */ | |
| 352 | + if (v3d->ver >= 71) | |
| 353 | + V3D_CORE_WRITE(0, V3D_V7_CSD_QUEUED_CFG7, 0); | |
| 354 | + | |
| 321 | 355 | /* CFG0 write kicks off the job. */ |
| 322 | 356 | V3D_CORE_WRITE(0, csd_cfg0_reg, job->args.cfg[0]); |
| 323 | 357 | |
| @@ -452,6 +452,8 @@ v3d_get_cpu_timestamp_query_params(struct drm_file *file_priv, · 451 unmodified lines | ||
| 452 | 452 | { |
| 453 | 453 | u32 __user *offsets, *syncs; |
| 454 | 454 | struct drm_v3d_timestamp_query timestamp; |
| 455 | + unsigned int i; | |
| 456 | + int err; | |
| 455 | 457 | |
| 456 | 458 | if (!job) { |
| 457 | 459 | DRM_DEBUG("CPU job extension was attached to a GPU job.\n"); |
| @@ -480,26 +482,34 @@ v3d_get_cpu_timestamp_query_params(struct drm_file *file_priv, · 22 unmodified lines | ||
| 480 | 482 | offsets = u64_to_user_ptr(timestamp.offsets); |
| 481 | 483 | syncs = u64_to_user_ptr(timestamp.syncs); |
| 482 | 484 | |
| 483 | − for (int i = 0; i < timestamp.count; i++) { | |
| 485 | + for (i = 0; i < timestamp.count; i++) { | |
| 484 | 486 | u32 offset, sync; |
| 485 | 487 | |
| 486 | 488 | if (copy_from_user(&offset, offsets++, sizeof(offset))) { |
| 487 | − kvfree(job->timestamp_query.queries); | |
| 488 | − return -EFAULT; | |
| 489 | + err = -EFAULT; | |
| 490 | + goto error; | |
| 489 | 491 | } |
| 490 | 492 | |
| 491 | 493 | job->timestamp_query.queries[i].offset = offset; |
| 492 | 494 | |
| 493 | 495 | if (copy_from_user(&sync, syncs++, sizeof(sync))) { |
| 494 | − kvfree(job->timestamp_query.queries); | |
| 495 | − return -EFAULT; | |
| 496 | + err = -EFAULT; | |
| 497 | + goto error; | |
| 496 | 498 | } |
| 497 | 499 | |
| 498 | 500 | job->timestamp_query.queries[i].syncobj = drm_syncobj_find(file_priv, sync); |
| 501 | + if (!job->timestamp_query.queries[i].syncobj) { | |
| 502 | + err = -ENOENT; | |
| 503 | + goto error; | |
| 504 | + } | |
| 499 | 505 | } |
| 500 | 506 | job->timestamp_query.count = timestamp.count; |
| 501 | 507 | |
| 502 | 508 | return 0; |
| 509 | + | |
| 510 | +error: | |
| 511 | + v3d_timestamp_query_info_free(&job->timestamp_query, i); | |
| 512 | + return err; | |
| 503 | 513 | } |
| 504 | 514 | |
| 505 | 515 | static int |
| @@ -509,6 +519,8 @@ v3d_get_cpu_reset_timestamp_params(struct drm_file *file_priv, · 3 unmodified lines | ||
| 509 | 519 | { |
| 510 | 520 | u32 __user *syncs; |
| 511 | 521 | struct drm_v3d_reset_timestamp_query reset; |
| 522 | + unsigned int i; | |
| 523 | + int err; | |
| 512 | 524 | |
| 513 | 525 | if (!job) { |
| 514 | 526 | DRM_DEBUG("CPU job extension was attached to a GPU job.\n"); |
| @@ -533,21 +545,29 @@ v3d_get_cpu_reset_timestamp_params(struct drm_file *file_priv, · 18 unmodified lines | ||
| 533 | 545 | |
| 534 | 546 | syncs = u64_to_user_ptr(reset.syncs); |
| 535 | 547 | |
| 536 | − for (int i = 0; i < reset.count; i++) { | |
| 548 | + for (i = 0; i < reset.count; i++) { | |
| 537 | 549 | u32 sync; |
| 538 | 550 | |
| 539 | 551 | job->timestamp_query.queries[i].offset = reset.offset + 8 * i; |
| 540 | 552 | |
| 541 | 553 | if (copy_from_user(&sync, syncs++, sizeof(sync))) { |
| 542 | − kvfree(job->timestamp_query.queries); | |
| 543 | − return -EFAULT; | |
| 554 | + err = -EFAULT; | |
| 555 | + goto error; | |
| 544 | 556 | } |
| 545 | 557 | |
| 546 | 558 | job->timestamp_query.queries[i].syncobj = drm_syncobj_find(file_priv, sync); |
| 559 | + if (!job->timestamp_query.queries[i].syncobj) { | |
| 560 | + err = -ENOENT; | |
| 561 | + goto error; | |
| 562 | + } | |
| 547 | 563 | } |
| 548 | 564 | job->timestamp_query.count = reset.count; |
| 549 | 565 | |
| 550 | 566 | return 0; |
| 567 | + | |
| 568 | +error: | |
| 569 | + v3d_timestamp_query_info_free(&job->timestamp_query, i); | |
| 570 | + return err; | |
| 551 | 571 | } |
| 552 | 572 | |
| 553 | 573 | /* Get data for the copy timestamp query results job submission. */ |
| @@ -558,7 +578,8 @@ v3d_get_cpu_copy_query_results_params(struct drm_file *file_priv, · 4 unmodified lines | ||
| 558 | 578 | { |
| 559 | 579 | u32 __user *offsets, *syncs; |
| 560 | 580 | struct drm_v3d_copy_timestamp_query copy; |
| 561 | − int i; | |
| 581 | + unsigned int i; | |
| 582 | + int err; | |
| 562 | 583 | |
| 563 | 584 | if (!job) { |
| 564 | 585 | DRM_DEBUG("CPU job extension was attached to a GPU job.\n"); |
| @@ -591,18 +612,22 @@ v3d_get_cpu_copy_query_results_params(struct drm_file *file_priv, · 26 unmodified lines | ||
| 591 | 612 | u32 offset, sync; |
| 592 | 613 | |
| 593 | 614 | if (copy_from_user(&offset, offsets++, sizeof(offset))) { |
| 594 | − kvfree(job->timestamp_query.queries); | |
| 595 | − return -EFAULT; | |
| 615 | + err = -EFAULT; | |
| 616 | + goto error; | |
| 596 | 617 | } |
| 597 | 618 | |
| 598 | 619 | job->timestamp_query.queries[i].offset = offset; |
| 599 | 620 | |
| 600 | 621 | if (copy_from_user(&sync, syncs++, sizeof(sync))) { |
| 601 | − kvfree(job->timestamp_query.queries); | |
| 602 | − return -EFAULT; | |
| 622 | + err = -EFAULT; | |
| 623 | + goto error; | |
| 603 | 624 | } |
| 604 | 625 | |
| 605 | 626 | job->timestamp_query.queries[i].syncobj = drm_syncobj_find(file_priv, sync); |
| 627 | + if (!job->timestamp_query.queries[i].syncobj) { | |
| 628 | + err = -ENOENT; | |
| 629 | + goto error; | |
| 630 | + } | |
| 606 | 631 | } |
| 607 | 632 | job->timestamp_query.count = copy.count; |
| 608 | 633 | |
| @@ -613,6 +638,10 @@ v3d_get_cpu_copy_query_results_params(struct drm_file *file_priv, · 4 unmodified lines | ||
| 613 | 638 | job->copy.stride = copy.stride; |
| 614 | 639 | |
| 615 | 640 | return 0; |
| 641 | + | |
| 642 | +error: | |
| 643 | + v3d_timestamp_query_info_free(&job->timestamp_query, i); | |
| 644 | + return err; | |
| 616 | 645 | } |
| 617 | 646 | |
| 618 | 647 | static int |
| @@ -623,6 +652,8 @@ v3d_get_cpu_reset_performance_params(struct drm_file *file_priv, · 4 unmodified lines | ||
| 623 | 652 | u32 __user *syncs; |
| 624 | 653 | u64 __user *kperfmon_ids; |
| 625 | 654 | struct drm_v3d_reset_performance_query reset; |
| 655 | + unsigned int i, j; | |
| 656 | + int err; | |
| 626 | 657 | |
| 627 | 658 | if (!job) { |
| 628 | 659 | DRM_DEBUG("CPU job extension was attached to a GPU job.\n"); |
| @@ -637,6 +668,9 @@ v3d_get_cpu_reset_performance_params(struct drm_file *file_priv, · 8 unmodified lines | ||
| 637 | 668 | if (copy_from_user(&reset, ext, sizeof(reset))) |
| 638 | 669 | return -EFAULT; |
| 639 | 670 | |
| 671 | + if (reset.nperfmons > V3D_MAX_PERFMONS) | |
| 672 | + return -EINVAL; | |
| 673 | + | |
| 640 | 674 | job->job_type = V3D_CPU_JOB_TYPE_RESET_PERFORMANCE_QUERY; |
| 641 | 675 | |
| 642 | 676 | job->performance_query.queries = kvmalloc_array(reset.count, |
| @@ -648,39 +682,47 @@ v3d_get_cpu_reset_performance_params(struct drm_file *file_priv, · 5 unmodified lines | ||
| 648 | 682 | syncs = u64_to_user_ptr(reset.syncs); |
| 649 | 683 | kperfmon_ids = u64_to_user_ptr(reset.kperfmon_ids); |
| 650 | 684 | |
| 651 | − for (int i = 0; i < reset.count; i++) { | |
| 685 | + for (i = 0; i < reset.count; i++) { | |
| 652 | 686 | u32 sync; |
| 653 | 687 | u64 ids; |
| 654 | 688 | u32 __user *ids_pointer; |
| 655 | 689 | u32 id; |
| 656 | 690 | |
| 657 | 691 | if (copy_from_user(&sync, syncs++, sizeof(sync))) { |
| 658 | − kvfree(job->performance_query.queries); | |
| 659 | − return -EFAULT; | |
| 692 | + err = -EFAULT; | |
| 693 | + goto error; | |
| 660 | 694 | } |
| 661 | 695 | |
| 662 | − job->performance_query.queries[i].syncobj = drm_syncobj_find(file_priv, sync); | |
| 663 | − | |
| 664 | 696 | if (copy_from_user(&ids, kperfmon_ids++, sizeof(ids))) { |
| 665 | − kvfree(job->performance_query.queries); | |
| 666 | − return -EFAULT; | |
| 697 | + err = -EFAULT; | |
| 698 | + goto error; | |
| 667 | 699 | } |
| 668 | 700 | |
| 669 | 701 | ids_pointer = u64_to_user_ptr(ids); |
| 670 | 702 | |
| 671 | − for (int j = 0; j < reset.nperfmons; j++) { | |
| 703 | + for (j = 0; j < reset.nperfmons; j++) { | |
| 672 | 704 | if (copy_from_user(&id, ids_pointer++, sizeof(id))) { |
| 673 | − kvfree(job->performance_query.queries); | |
| 674 | − return -EFAULT; | |
| 705 | + err = -EFAULT; | |
| 706 | + goto error; | |
| 675 | 707 | } |
| 676 | 708 | |
| 677 | 709 | job->performance_query.queries[i].kperfmon_ids[j] = id; |
| 678 | 710 | } |
| 711 | + | |
| 712 | + job->performance_query.queries[i].syncobj = drm_syncobj_find(file_priv, sync); | |
| 713 | + if (!job->performance_query.queries[i].syncobj) { | |
| 714 | + err = -ENOENT; | |
| 715 | + goto error; | |
| 716 | + } | |
| 679 | 717 | } |
| 680 | 718 | job->performance_query.count = reset.count; |
| 681 | 719 | job->performance_query.nperfmons = reset.nperfmons; |
| 682 | 720 | |
| 683 | 721 | return 0; |
| 722 | + | |
| 723 | +error: | |
| 724 | + v3d_performance_query_info_free(&job->performance_query, i); | |
| 725 | + return err; | |
| 684 | 726 | } |
| 685 | 727 | |
| 686 | 728 | static int |
| @@ -691,6 +733,8 @@ v3d_get_cpu_copy_performance_query_params(struct drm_file *file_priv, · 4 unmodified lines | ||
| 691 | 733 | u32 __user *syncs; |
| 692 | 734 | u64 __user *kperfmon_ids; |
| 693 | 735 | struct drm_v3d_copy_performance_query copy; |
| 736 | + unsigned int i, j; | |
| 737 | + int err; | |
| 694 | 738 | |
| 695 | 739 | if (!job) { |
| 696 | 740 | DRM_DEBUG("CPU job extension was attached to a GPU job.\n"); |
| @@ -708,6 +752,9 @@ v3d_get_cpu_copy_performance_query_params(struct drm_file *file_priv, · 11 unmodified lines | ||
| 708 | 752 | if (copy.pad) |
| 709 | 753 | return -EINVAL; |
| 710 | 754 | |
| 755 | + if (copy.nperfmons > V3D_MAX_PERFMONS) | |
| 756 | + return -EINVAL; | |
| 757 | + | |
| 711 | 758 | job->job_type = V3D_CPU_JOB_TYPE_COPY_PERFORMANCE_QUERY; |
| 712 | 759 | |
| 713 | 760 | job->performance_query.queries = kvmalloc_array(copy.count, |
| @@ -719,34 +766,38 @@ v3d_get_cpu_copy_performance_query_params(struct drm_file *file_priv, · 5 unmodified lines | ||
| 719 | 766 | syncs = u64_to_user_ptr(copy.syncs); |
| 720 | 767 | kperfmon_ids = u64_to_user_ptr(copy.kperfmon_ids); |
| 721 | 768 | |
| 722 | − for (int i = 0; i < copy.count; i++) { | |
| 769 | + for (i = 0; i < copy.count; i++) { | |
| 723 | 770 | u32 sync; |
| 724 | 771 | u64 ids; |
| 725 | 772 | u32 __user *ids_pointer; |
| 726 | 773 | u32 id; |
| 727 | 774 | |
| 728 | 775 | if (copy_from_user(&sync, syncs++, sizeof(sync))) { |
| 729 | − kvfree(job->performance_query.queries); | |
| 730 | − return -EFAULT; | |
| 776 | + err = -EFAULT; | |
| 777 | + goto error; | |
| 731 | 778 | } |
| 732 | 779 | |
| 733 | − job->performance_query.queries[i].syncobj = drm_syncobj_find(file_priv, sync); | |
| 734 | − | |
| 735 | 780 | if (copy_from_user(&ids, kperfmon_ids++, sizeof(ids))) { |
| 736 | − kvfree(job->performance_query.queries); | |
| 737 | − return -EFAULT; | |
| 781 | + err = -EFAULT; | |
| 782 | + goto error; | |
| 738 | 783 | } |
| 739 | 784 | |
| 740 | 785 | ids_pointer = u64_to_user_ptr(ids); |
| 741 | 786 | |
| 742 | − for (int j = 0; j < copy.nperfmons; j++) { | |
| 787 | + for (j = 0; j < copy.nperfmons; j++) { | |
| 743 | 788 | if (copy_from_user(&id, ids_pointer++, sizeof(id))) { |
| 744 | − kvfree(job->performance_query.queries); | |
| 745 | − return -EFAULT; | |
| 789 | + err = -EFAULT; | |
| 790 | + goto error; | |
| 746 | 791 | } |
| 747 | 792 | |
| 748 | 793 | job->performance_query.queries[i].kperfmon_ids[j] = id; |
| 749 | 794 | } |
| 795 | + | |
| 796 | + job->performance_query.queries[i].syncobj = drm_syncobj_find(file_priv, sync); | |
| 797 | + if (!job->performance_query.queries[i].syncobj) { | |
| 798 | + err = -ENOENT; | |
| 799 | + goto error; | |
| 800 | + } | |
| 750 | 801 | } |
| 751 | 802 | job->performance_query.count = copy.count; |
| 752 | 803 | job->performance_query.nperfmons = copy.nperfmons; |
| @@ -759,6 +810,10 @@ v3d_get_cpu_copy_performance_query_params(struct drm_file *file_priv, · 6 unmodified lines | ||
| 759 | 810 | job->copy.stride = copy.stride; |
| 760 | 811 | |
| 761 | 812 | return 0; |
| 813 | + | |
| 814 | +error: | |
| 815 | + v3d_performance_query_info_free(&job->performance_query, i); | |
| 816 | + return err; | |
| 762 | 817 | } |
| 763 | 818 | |
| 764 | 819 | /* Whenever userspace sets ioctl extensions, v3d_get_extensions parses data |
| @@ -48,7 +48,7 @@ struct virtio_gpu_submit { · 47 unmodified lines | ||
| 48 | 48 | static int virtio_gpu_do_fence_wait(struct virtio_gpu_submit *submit, |
| 49 | 49 | struct dma_fence *in_fence) |
| 50 | 50 | { |
| 51 | − u32 context = submit->fence_ctx + submit->ring_idx; | |
| 51 | + u64 context = submit->fence_ctx + submit->ring_idx; | |
| 52 | 52 | |
| 53 | 53 | if (dma_fence_match_context(in_fence, context)) |
| 54 | 54 | return 0; |
| @@ -1,6 +1,8 @@ | ||
| 1 | +/* SPDX-License-Identifier: GPL-2.0 OR MIT */ | |
| 1 | 2 | /********************************************************** |
| 2 | − * Copyright 2021 VMware, Inc. | |
| 3 | − * SPDX-License-Identifier: GPL-2.0 OR MIT | |
| 3 | + * | |
| 4 | + * Copyright (c) 2021-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 4 | 6 | * |
| 5 | 7 | * Permission is hereby granted, free of charge, to any person |
| 6 | 8 | * obtaining a copy of this software and associated documentation |
| @@ -31,6 +33,10 @@ · 24 unmodified lines | ||
| 31 | 33 | |
| 32 | 34 | #include <drm/vmwgfx_drm.h> |
| 33 | 35 | |
| 36 | +#define SVGA3D_FLAGS_UPPER_32(svga3d_flags) ((svga3d_flags) >> 32) | |
| 37 | +#define SVGA3D_FLAGS_LOWER_32(svga3d_flags) \ | |
| 38 | + ((svga3d_flags) & ((uint64_t)U32_MAX)) | |
| 39 | + | |
| 34 | 40 | static inline u32 clamped_umul32(u32 a, u32 b) |
| 35 | 41 | { |
| 36 | 42 | uint64_t tmp = (uint64_t) a*b; |
| @@ -27,6 +27,8 @@ · 26 unmodified lines | ||
| 27 | 27 | **************************************************************************/ |
| 28 | 28 | |
| 29 | 29 | #include "vmwgfx_drv.h" |
| 30 | + | |
| 31 | +#include "vmwgfx_bo.h" | |
| 30 | 32 | #include <linux/highmem.h> |
| 31 | 33 | |
| 32 | 34 | /* |
| @@ -420,13 +422,105 @@ static int vmw_bo_cpu_blit_line(struct vmw_bo_blit_line_data *d, · 387 unmodified lines | ||
| 420 | 422 | return 0; |
| 421 | 423 | } |
| 422 | 424 | |
| 425 | +static void *map_external(struct vmw_bo *bo, struct iosys_map *map) | |
| 426 | +{ | |
| 427 | + struct vmw_private *vmw = | |
| 428 | + container_of(bo->tbo.bdev, struct vmw_private, bdev); | |
| 429 | + void *ptr = NULL; | |
| 430 | + int ret; | |
| 431 | + | |
| 432 | + if (bo->tbo.base.import_attach) { | |
| 433 | + ret = dma_buf_vmap(bo->tbo.base.dma_buf, map); | |
| 434 | + if (ret) { | |
| 435 | + drm_dbg_driver(&vmw->drm, | |
| 436 | + "Wasn't able to map external bo!\n"); | |
| 437 | + goto out; | |
| 438 | + } | |
| 439 | + ptr = map->vaddr; | |
| 440 | + } else { | |
| 441 | + ptr = vmw_bo_map_and_cache(bo); | |
| 442 | + } | |
| 443 | + | |
| 444 | +out: | |
| 445 | + return ptr; | |
| 446 | +} | |
| 447 | + | |
| 448 | +static void unmap_external(struct vmw_bo *bo, struct iosys_map *map) | |
| 449 | +{ | |
| 450 | + if (bo->tbo.base.import_attach) | |
| 451 | + dma_buf_vunmap(bo->tbo.base.dma_buf, map); | |
| 452 | + else | |
| 453 | + vmw_bo_unmap(bo); | |
| 454 | +} | |
| 455 | + | |
| 456 | +static int vmw_external_bo_copy(struct vmw_bo *dst, u32 dst_offset, | |
| 457 | + u32 dst_stride, struct vmw_bo *src, | |
| 458 | + u32 src_offset, u32 src_stride, | |
| 459 | + u32 width_in_bytes, u32 height, | |
| 460 | + struct vmw_diff_cpy *diff) | |
| 461 | +{ | |
| 462 | + struct vmw_private *vmw = | |
| 463 | + container_of(dst->tbo.bdev, struct vmw_private, bdev); | |
| 464 | + size_t dst_size = dst->tbo.resource->size; | |
| 465 | + size_t src_size = src->tbo.resource->size; | |
| 466 | + struct iosys_map dst_map = {0}; | |
| 467 | + struct iosys_map src_map = {0}; | |
| 468 | + int ret, i; | |
| 469 | + int x_in_bytes; | |
| 470 | + u8 *vsrc; | |
| 471 | + u8 *vdst; | |
| 472 | + | |
| 473 | + vsrc = map_external(src, &src_map); | |
| 474 | + if (!vsrc) { | |
| 475 | + drm_dbg_driver(&vmw->drm, "Wasn't able to map src\n"); | |
| 476 | + ret = -ENOMEM; | |
| 477 | + goto out; | |
| 478 | + } | |
| 479 | + | |
| 480 | + vdst = map_external(dst, &dst_map); | |
| 481 | + if (!vdst) { | |
| 482 | + drm_dbg_driver(&vmw->drm, "Wasn't able to map dst\n"); | |
| 483 | + ret = -ENOMEM; | |
| 484 | + goto out; | |
| 485 | + } | |
| 486 | + | |
| 487 | + vsrc += src_offset; | |
| 488 | + vdst += dst_offset; | |
| 489 | + if (src_stride == dst_stride) { | |
| 490 | + dst_size -= dst_offset; | |
| 491 | + src_size -= src_offset; | |
| 492 | + memcpy(vdst, vsrc, | |
| 493 | + min(dst_stride * height, min(dst_size, src_size))); | |
| 494 | + } else { | |
| 495 | + WARN_ON(dst_stride < width_in_bytes); | |
| 496 | + for (i = 0; i < height; ++i) { | |
| 497 | + memcpy(vdst, vsrc, width_in_bytes); | |
| 498 | + vsrc += src_stride; | |
| 499 | + vdst += dst_stride; | |
| 500 | + } | |
| 501 | + } | |
| 502 | + | |
| 503 | + x_in_bytes = (dst_offset % dst_stride); | |
| 504 | + diff->rect.x1 = x_in_bytes / diff->cpp; | |
| 505 | + diff->rect.y1 = ((dst_offset - x_in_bytes) / dst_stride); | |
| 506 | + diff->rect.x2 = diff->rect.x1 + width_in_bytes / diff->cpp; | |
| 507 | + diff->rect.y2 = diff->rect.y1 + height; | |
| 508 | + | |
| 509 | + ret = 0; | |
| 510 | +out: | |
| 511 | + unmap_external(src, &src_map); | |
| 512 | + unmap_external(dst, &dst_map); | |
| 513 | + | |
| 514 | + return ret; | |
| 515 | +} | |
| 516 | + | |
| 423 | 517 | /** |
| 424 | 518 | * vmw_bo_cpu_blit - in-kernel cpu blit. |
| 425 | 519 | * |
| 426 | − * @dst: Destination buffer object. | |
| 520 | + * @vmw_dst: Destination buffer object. | |
| 427 | 521 | * @dst_offset: Destination offset of blit start in bytes. |
| 428 | 522 | * @dst_stride: Destination stride in bytes. |
| 429 | − * @src: Source buffer object. | |
| 523 | + * @vmw_src: Source buffer object. | |
| 430 | 524 | * @src_offset: Source offset of blit start in bytes. |
| 431 | 525 | * @src_stride: Source stride in bytes. |
| 432 | 526 | * @w: Width of blit. |
| @@ -444,13 +538,15 @@ static int vmw_bo_cpu_blit_line(struct vmw_bo_blit_line_data *d, · 11 unmodified lines | ||
| 444 | 538 | * Neither of the buffer objects may be placed in PCI memory |
| 445 | 539 | * (Fixed memory in TTM terminology) when using this function. |
| 446 | 540 | */ |
| 447 | −int vmw_bo_cpu_blit(struct ttm_buffer_object *dst, | |
| 541 | +int vmw_bo_cpu_blit(struct vmw_bo *vmw_dst, | |
| 448 | 542 | u32 dst_offset, u32 dst_stride, |
| 449 | − struct ttm_buffer_object *src, | |
| 543 | + struct vmw_bo *vmw_src, | |
| 450 | 544 | u32 src_offset, u32 src_stride, |
| 451 | 545 | u32 w, u32 h, |
| 452 | 546 | struct vmw_diff_cpy *diff) |
| 453 | 547 | { |
| 548 | + struct ttm_buffer_object *src = &vmw_src->tbo; | |
| 549 | + struct ttm_buffer_object *dst = &vmw_dst->tbo; | |
| 454 | 550 | struct ttm_operation_ctx ctx = { |
| 455 | 551 | .interruptible = false, |
| 456 | 552 | .no_wait_gpu = false |
| @@ -460,6 +556,11 @@ int vmw_bo_cpu_blit(struct ttm_buffer_object *dst, · 3 unmodified lines | ||
| 460 | 556 | int ret = 0; |
| 461 | 557 | struct page **dst_pages = NULL; |
| 462 | 558 | struct page **src_pages = NULL; |
| 559 | + bool src_external = (src->ttm->page_flags & TTM_TT_FLAG_EXTERNAL) != 0; | |
| 560 | + bool dst_external = (dst->ttm->page_flags & TTM_TT_FLAG_EXTERNAL) != 0; | |
| 561 | + | |
| 562 | + if (WARN_ON(dst == src)) | |
| 563 | + return -EINVAL; | |
| 463 | 564 | |
| 464 | 565 | /* Buffer objects need to be either pinned or reserved: */ |
| 465 | 566 | if (!(dst->pin_count)) |
| @@ -479,6 +580,11 @@ int vmw_bo_cpu_blit(struct ttm_buffer_object *dst, · 13 unmodified lines | ||
| 479 | 580 | return ret; |
| 480 | 581 | } |
| 481 | 582 | |
| 583 | + if (src_external || dst_external) | |
| 584 | + return vmw_external_bo_copy(vmw_dst, dst_offset, dst_stride, | |
| 585 | + vmw_src, src_offset, src_stride, | |
| 586 | + w, h, diff); | |
| 587 | + | |
| 482 | 588 | if (!src->ttm->pages && src->ttm->sg) { |
| 483 | 589 | src_pages = kvmalloc_array(src->ttm->num_pages, |
| 484 | 590 | sizeof(struct page *), GFP_KERNEL); |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0 OR MIT |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright © 2011-2023 VMware, Inc., Palo Alto, CA., USA | |
| 5 | − * All Rights Reserved. | |
| 4 | + * Copyright (c) 2011-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 6 | 6 | * |
| 7 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 8 | 8 | * copy of this software and associated documentation files (the |
| @@ -28,15 +28,39 @@ · 19 unmodified lines | ||
| 28 | 28 | |
| 29 | 29 | #include "vmwgfx_bo.h" |
| 30 | 30 | #include "vmwgfx_drv.h" |
| 31 | − | |
| 31 | +#include "vmwgfx_resource_priv.h" | |
| 32 | 32 | |
| 33 | 33 | #include <drm/ttm/ttm_placement.h> |
| 34 | 34 | |
| 35 | 35 | static void vmw_bo_release(struct vmw_bo *vbo) |
| 36 | 36 | { |
| 37 | + struct vmw_resource *res; | |
| 38 | + | |
| 37 | 39 | WARN_ON(vbo->tbo.base.funcs && |
| 38 | 40 | kref_read(&vbo->tbo.base.refcount) != 0); |
| 39 | 41 | vmw_bo_unmap(vbo); |
| 42 | + | |
| 43 | + xa_destroy(&vbo->detached_resources); | |
| 44 | + WARN_ON(vbo->is_dumb && !vbo->dumb_surface); | |
| 45 | + if (vbo->is_dumb && vbo->dumb_surface) { | |
| 46 | + res = &vbo->dumb_surface->res; | |
| 47 | + WARN_ON(vbo != res->guest_memory_bo); | |
| 48 | + WARN_ON(!res->guest_memory_bo); | |
| 49 | + if (res->guest_memory_bo) { | |
| 50 | + /* Reserve and switch the backing mob. */ | |
| 51 | + mutex_lock(&res->dev_priv->cmdbuf_mutex); | |
| 52 | + (void)vmw_resource_reserve(res, false, true); | |
| 53 | + vmw_resource_mob_detach(res); | |
| 54 | + if (res->coherent) | |
| 55 | + vmw_bo_dirty_release(res->guest_memory_bo); | |
| 56 | + res->guest_memory_bo = NULL; | |
| 57 | + res->guest_memory_offset = 0; | |
| 58 | + vmw_resource_unreserve(res, false, false, false, NULL, | |
| 59 | + 0); | |
| 60 | + mutex_unlock(&res->dev_priv->cmdbuf_mutex); | |
| 61 | + } | |
| 62 | + vmw_surface_unreference(&vbo->dumb_surface); | |
| 63 | + } | |
| 40 | 64 | drm_gem_object_release(&vbo->tbo.base); |
| 41 | 65 | } |
| 42 | 66 | |
| @@ -325,19 +349,27 @@ void vmw_bo_pin_reserved(struct vmw_bo *vbo, bool pin) · 282 unmodified lines | ||
| 325 | 349 | * |
| 326 | 350 | */ |
| 327 | 351 | void *vmw_bo_map_and_cache(struct vmw_bo *vbo) |
| 352 | +{ | |
| 353 | + return vmw_bo_map_and_cache_size(vbo, vbo->tbo.base.size); | |
| 354 | +} | |
| 355 | + | |
| 356 | +void *vmw_bo_map_and_cache_size(struct vmw_bo *vbo, size_t size) | |
| 328 | 357 | { |
| 329 | 358 | struct ttm_buffer_object *bo = &vbo->tbo; |
| 330 | 359 | bool not_used; |
| 331 | 360 | void *virtual; |
| 332 | 361 | int ret; |
| 333 | 362 | |
| 363 | + atomic_inc(&vbo->map_count); | |
| 364 | + | |
| 334 | 365 | virtual = ttm_kmap_obj_virtual(&vbo->map, ¬_used); |
| 335 | 366 | if (virtual) |
| 336 | 367 | return virtual; |
| 337 | 368 | |
| 338 | − ret = ttm_bo_kmap(bo, 0, PFN_UP(bo->base.size), &vbo->map); | |
| 369 | + ret = ttm_bo_kmap(bo, 0, PFN_UP(size), &vbo->map); | |
| 339 | 370 | if (ret) |
| 340 | − DRM_ERROR("Buffer object map failed: %d.\n", ret); | |
| 371 | + DRM_ERROR("Buffer object map failed: %d (size: bo = %zu, map = %zu).\n", | |
| 372 | + ret, bo->base.size, size); | |
| 341 | 373 | |
| 342 | 374 | return ttm_kmap_obj_virtual(&vbo->map, ¬_used); |
| 343 | 375 | } |
| @@ -353,11 +385,17 @@ void *vmw_bo_map_and_cache(struct vmw_bo *vbo) · 9 unmodified lines | ||
| 353 | 385 | */ |
| 354 | 386 | void vmw_bo_unmap(struct vmw_bo *vbo) |
| 355 | 387 | { |
| 388 | + int map_count; | |
| 389 | + | |
| 356 | 390 | if (vbo->map.bo == NULL) |
| 357 | 391 | return; |
| 358 | 392 | |
| 359 | − ttm_bo_kunmap(&vbo->map); | |
| 360 | − vbo->map.bo = NULL; | |
| 393 | + map_count = atomic_dec_return(&vbo->map_count); | |
| 394 | + | |
| 395 | + if (!map_count) { | |
| 396 | + ttm_bo_kunmap(&vbo->map); | |
| 397 | + vbo->map.bo = NULL; | |
| 398 | + } | |
| 361 | 399 | } |
| 362 | 400 | |
| 363 | 401 | |
| @@ -390,6 +428,8 @@ static int vmw_bo_init(struct vmw_private *dev_priv, · 26 unmodified lines | ||
| 390 | 428 | BUILD_BUG_ON(TTM_MAX_BO_PRIORITY <= 3); |
| 391 | 429 | vmw_bo->tbo.priority = 3; |
| 392 | 430 | vmw_bo->res_tree = RB_ROOT; |
| 431 | + xa_init(&vmw_bo->detached_resources); | |
| 432 | + atomic_set(&vmw_bo->map_count, 0); | |
| 393 | 433 | |
| 394 | 434 | params->size = ALIGN(params->size, PAGE_SIZE); |
| 395 | 435 | drm_gem_private_object_init(vdev, &vmw_bo->tbo.base, params->size); |
| @@ -654,52 +694,6 @@ void vmw_bo_fence_single(struct ttm_buffer_object *bo, · 258 unmodified lines | ||
| 654 | 694 | dma_fence_put(&fence->base); |
| 655 | 695 | } |
| 656 | 696 | |
| 657 | − | |
| 658 | −/** | |
| 659 | − * vmw_dumb_create - Create a dumb kms buffer | |
| 660 | − * | |
| 661 | − * @file_priv: Pointer to a struct drm_file identifying the caller. | |
| 662 | − * @dev: Pointer to the drm device. | |
| 663 | − * @args: Pointer to a struct drm_mode_create_dumb structure | |
| 664 | − * Return: Zero on success, negative error code on failure. | |
| 665 | − * | |
| 666 | − * This is a driver callback for the core drm create_dumb functionality. | |
| 667 | − * Note that this is very similar to the vmw_bo_alloc ioctl, except | |
| 668 | − * that the arguments have a different format. | |
| 669 | − */ | |
| 670 | −int vmw_dumb_create(struct drm_file *file_priv, | |
| 671 | − struct drm_device *dev, | |
| 672 | − struct drm_mode_create_dumb *args) | |
| 673 | −{ | |
| 674 | − struct vmw_private *dev_priv = vmw_priv(dev); | |
| 675 | − struct vmw_bo *vbo; | |
| 676 | − int cpp = DIV_ROUND_UP(args->bpp, 8); | |
| 677 | − int ret; | |
| 678 | − | |
| 679 | − switch (cpp) { | |
| 680 | − case 1: /* DRM_FORMAT_C8 */ | |
| 681 | − case 2: /* DRM_FORMAT_RGB565 */ | |
| 682 | − case 4: /* DRM_FORMAT_XRGB8888 */ | |
| 683 | − break; | |
| 684 | − default: | |
| 685 | − /* | |
| 686 | − * Dumb buffers don't allow anything else. | |
| 687 | − * This is tested via IGT's dumb_buffers | |
| 688 | − */ | |
| 689 | − return -EINVAL; | |
| 690 | − } | |
| 691 | − | |
| 692 | − args->pitch = args->width * cpp; | |
| 693 | − args->size = ALIGN(args->pitch * args->height, PAGE_SIZE); | |
| 694 | − | |
| 695 | − ret = vmw_gem_object_create_with_handle(dev_priv, file_priv, | |
| 696 | − args->size, &args->handle, | |
| 697 | − &vbo); | |
| 698 | − /* drop reference from allocate - handle holds it now */ | |
| 699 | − drm_gem_object_put(&vbo->tbo.base); | |
| 700 | − return ret; | |
| 701 | −} | |
| 702 | − | |
| 703 | 697 | /** |
| 704 | 698 | * vmw_bo_swap_notify - swapout notify callback. |
| 705 | 699 | * |
| @@ -853,3 +847,43 @@ void vmw_bo_placement_set_default_accelerated(struct vmw_bo *bo) · 147 unmodified lines | ||
| 853 | 847 | |
| 854 | 848 | vmw_bo_placement_set(bo, domain, domain); |
| 855 | 849 | } |
| 850 | + | |
| 851 | +void vmw_bo_add_detached_resource(struct vmw_bo *vbo, struct vmw_resource *res) | |
| 852 | +{ | |
| 853 | + xa_store(&vbo->detached_resources, (unsigned long)res, res, GFP_KERNEL); | |
| 854 | +} | |
| 855 | + | |
| 856 | +void vmw_bo_del_detached_resource(struct vmw_bo *vbo, struct vmw_resource *res) | |
| 857 | +{ | |
| 858 | + xa_erase(&vbo->detached_resources, (unsigned long)res); | |
| 859 | +} | |
| 860 | + | |
| 861 | +struct vmw_surface *vmw_bo_surface(struct vmw_bo *vbo) | |
| 862 | +{ | |
| 863 | + unsigned long index; | |
| 864 | + struct vmw_resource *res = NULL; | |
| 865 | + struct vmw_surface *surf = NULL; | |
| 866 | + struct rb_node *rb_itr = vbo->res_tree.rb_node; | |
| 867 | + | |
| 868 | + if (vbo->is_dumb && vbo->dumb_surface) { | |
| 869 | + res = &vbo->dumb_surface->res; | |
| 870 | + goto out; | |
| 871 | + } | |
| 872 | + | |
| 873 | + xa_for_each(&vbo->detached_resources, index, res) { | |
| 874 | + if (res->func->res_type == vmw_res_surface) | |
| 875 | + goto out; | |
| 876 | + } | |
| 877 | + | |
| 878 | + for (rb_itr = rb_first(&vbo->res_tree); rb_itr; | |
| 879 | + rb_itr = rb_next(rb_itr)) { | |
| 880 | + res = rb_entry(rb_itr, struct vmw_resource, mob_node); | |
| 881 | + if (res->func->res_type == vmw_res_surface) | |
| 882 | + goto out; | |
| 883 | + } | |
| 884 | + | |
| 885 | +out: | |
| 886 | + if (res) | |
| 887 | + surf = vmw_res_to_srf(res); | |
| 888 | + return surf; | |
| 889 | +} | |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | /* SPDX-License-Identifier: GPL-2.0 OR MIT */ |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright 2023 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2023-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -35,11 +36,13 @@ · 27 unmodified lines | ||
| 35 | 36 | |
| 36 | 37 | #include <linux/rbtree_types.h> |
| 37 | 38 | #include <linux/types.h> |
| 39 | +#include <linux/xarray.h> | |
| 38 | 40 | |
| 39 | 41 | struct vmw_bo_dirty; |
| 40 | 42 | struct vmw_fence_obj; |
| 41 | 43 | struct vmw_private; |
| 42 | 44 | struct vmw_resource; |
| 45 | +struct vmw_surface; | |
| 43 | 46 | |
| 44 | 47 | enum vmw_bo_domain { |
| 45 | 48 | VMW_BO_DOMAIN_SYS = BIT(0), |
| @@ -68,6 +71,8 @@ struct vmw_bo_params { · 22 unmodified lines | ||
| 68 | 71 | * @map: Kmap object for semi-persistent mappings |
| 69 | 72 | * @res_tree: RB tree of resources using this buffer object as a backing MOB |
| 70 | 73 | * @res_prios: Eviction priority counts for attached resources |
| 74 | + * @map_count: The number of currently active maps. Will differ from the | |
| 75 | + * cpu_writers because it includes kernel maps. | |
| 71 | 76 | * @cpu_writers: Number of synccpu write grabs. Protected by reservation when |
| 72 | 77 | * increased. May be decreased without reservation. |
| 73 | 78 | * @dx_query_ctx: DX context if this buffer object is used as a DX query MOB |
| @@ -85,11 +90,16 @@ struct vmw_bo { · 11 unmodified lines | ||
| 85 | 90 | |
| 86 | 91 | struct rb_root res_tree; |
| 87 | 92 | u32 res_prios[TTM_MAX_BO_PRIORITY]; |
| 93 | + struct xarray detached_resources; | |
| 88 | 94 | |
| 95 | + atomic_t map_count; | |
| 89 | 96 | atomic_t cpu_writers; |
| 90 | 97 | /* Not ref-counted. Protected by binding_mutex */ |
| 91 | 98 | struct vmw_resource *dx_query_ctx; |
| 92 | 99 | struct vmw_bo_dirty *dirty; |
| 100 | + | |
| 101 | + bool is_dumb; | |
| 102 | + struct vmw_surface *dumb_surface; | |
| 93 | 103 | }; |
| 94 | 104 | |
| 95 | 105 | void vmw_bo_placement_set(struct vmw_bo *bo, u32 domain, u32 busy_domain); |
| @@ -124,15 +134,21 @@ void vmw_bo_fence_single(struct ttm_buffer_object *bo, · 28 unmodified lines | ||
| 124 | 134 | struct vmw_fence_obj *fence); |
| 125 | 135 | |
| 126 | 136 | void *vmw_bo_map_and_cache(struct vmw_bo *vbo); |
| 137 | +void *vmw_bo_map_and_cache_size(struct vmw_bo *vbo, size_t size); | |
| 127 | 138 | void vmw_bo_unmap(struct vmw_bo *vbo); |
| 128 | 139 | |
| 129 | 140 | void vmw_bo_move_notify(struct ttm_buffer_object *bo, |
| 130 | 141 | struct ttm_resource *mem); |
| 131 | 142 | void vmw_bo_swap_notify(struct ttm_buffer_object *bo); |
| 132 | 143 | |
| 144 | +void vmw_bo_add_detached_resource(struct vmw_bo *vbo, struct vmw_resource *res); | |
| 145 | +void vmw_bo_del_detached_resource(struct vmw_bo *vbo, struct vmw_resource *res); | |
| 146 | +struct vmw_surface *vmw_bo_surface(struct vmw_bo *vbo); | |
| 147 | + | |
| 133 | 148 | int vmw_user_bo_lookup(struct drm_file *filp, |
| 134 | 149 | u32 handle, |
| 135 | 150 | struct vmw_bo **out); |
| 151 | + | |
| 136 | 152 | /** |
| 137 | 153 | * vmw_bo_adjust_prio - Adjust the buffer object eviction priority |
| 138 | 154 | * according to attached resources |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | /* SPDX-License-Identifier: GPL-2.0 OR MIT */ |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright 2009-2023 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2009-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -762,6 +763,26 @@ extern int vmw_gmr_bind(struct vmw_private *dev_priv, · 754 unmodified lines | ||
| 762 | 763 | int gmr_id); |
| 763 | 764 | extern void vmw_gmr_unbind(struct vmw_private *dev_priv, int gmr_id); |
| 764 | 765 | |
| 766 | +/** | |
| 767 | + * User handles | |
| 768 | + */ | |
| 769 | +struct vmw_user_object { | |
| 770 | + struct vmw_surface *surface; | |
| 771 | + struct vmw_bo *buffer; | |
| 772 | +}; | |
| 773 | + | |
| 774 | +int vmw_user_object_lookup(struct vmw_private *dev_priv, struct drm_file *filp, | |
| 775 | + u32 handle, struct vmw_user_object *uo); | |
| 776 | +struct vmw_user_object *vmw_user_object_ref(struct vmw_user_object *uo); | |
| 777 | +void vmw_user_object_unref(struct vmw_user_object *uo); | |
| 778 | +bool vmw_user_object_is_null(struct vmw_user_object *uo); | |
| 779 | +struct vmw_surface *vmw_user_object_surface(struct vmw_user_object *uo); | |
| 780 | +struct vmw_bo *vmw_user_object_buffer(struct vmw_user_object *uo); | |
| 781 | +void *vmw_user_object_map(struct vmw_user_object *uo); | |
| 782 | +void *vmw_user_object_map_size(struct vmw_user_object *uo, size_t size); | |
| 783 | +void vmw_user_object_unmap(struct vmw_user_object *uo); | |
| 784 | +bool vmw_user_object_is_mapped(struct vmw_user_object *uo); | |
| 785 | + | |
| 765 | 786 | /** |
| 766 | 787 | * Resource utilities - vmwgfx_resource.c |
| 767 | 788 | */ |
| @@ -776,11 +797,6 @@ extern int vmw_resource_validate(struct vmw_resource *res, bool intr, · 8 unmodified lines | ||
| 776 | 797 | extern int vmw_resource_reserve(struct vmw_resource *res, bool interruptible, |
| 777 | 798 | bool no_backup); |
| 778 | 799 | extern bool vmw_resource_needs_backup(const struct vmw_resource *res); |
| 779 | −extern int vmw_user_lookup_handle(struct vmw_private *dev_priv, | |
| 780 | − struct drm_file *filp, | |
| 781 | − uint32_t handle, | |
| 782 | − struct vmw_surface **out_surf, | |
| 783 | − struct vmw_bo **out_buf); | |
| 784 | 800 | extern int vmw_user_resource_lookup_handle( |
| 785 | 801 | struct vmw_private *dev_priv, |
| 786 | 802 | struct ttm_object_file *tfile, |
| @@ -1057,9 +1073,6 @@ int vmw_kms_suspend(struct drm_device *dev); · 270 unmodified lines | ||
| 1057 | 1073 | int vmw_kms_resume(struct drm_device *dev); |
| 1058 | 1074 | void vmw_kms_lost_device(struct drm_device *dev); |
| 1059 | 1075 | |
| 1060 | −int vmw_dumb_create(struct drm_file *file_priv, | |
| 1061 | − struct drm_device *dev, | |
| 1062 | − struct drm_mode_create_dumb *args); | |
| 1063 | 1076 | extern int vmw_resource_pin(struct vmw_resource *res, bool interruptible); |
| 1064 | 1077 | extern void vmw_resource_unpin(struct vmw_resource *res); |
| 1065 | 1078 | extern enum vmw_res_type vmw_res_type(const struct vmw_resource *res); |
| @@ -1176,6 +1189,15 @@ extern int vmw_gb_surface_reference_ext_ioctl(struct drm_device *dev, · 110 unmodified lines | ||
| 1176 | 1189 | int vmw_gb_surface_define(struct vmw_private *dev_priv, |
| 1177 | 1190 | const struct vmw_surface_metadata *req, |
| 1178 | 1191 | struct vmw_surface **srf_out); |
| 1192 | +struct vmw_surface *vmw_lookup_surface_for_buffer(struct vmw_private *vmw, | |
| 1193 | + struct vmw_bo *bo, | |
| 1194 | + u32 handle); | |
| 1195 | +u32 vmw_lookup_surface_handle_for_buffer(struct vmw_private *vmw, | |
| 1196 | + struct vmw_bo *bo, | |
| 1197 | + u32 handle); | |
| 1198 | +int vmw_dumb_create(struct drm_file *file_priv, | |
| 1199 | + struct drm_device *dev, | |
| 1200 | + struct drm_mode_create_dumb *args); | |
| 1179 | 1201 | |
| 1180 | 1202 | /* |
| 1181 | 1203 | * Shader management - vmwgfx_shader.c |
| @@ -1331,9 +1353,9 @@ void vmw_diff_memcpy(struct vmw_diff_cpy *diff, u8 *dest, const u8 *src, · 149 unmodified lines | ||
| 1331 | 1353 | |
| 1332 | 1354 | void vmw_memcpy(struct vmw_diff_cpy *diff, u8 *dest, const u8 *src, size_t n); |
| 1333 | 1355 | |
| 1334 | −int vmw_bo_cpu_blit(struct ttm_buffer_object *dst, | |
| 1356 | +int vmw_bo_cpu_blit(struct vmw_bo *dst, | |
| 1335 | 1357 | u32 dst_offset, u32 dst_stride, |
| 1336 | − struct ttm_buffer_object *src, | |
| 1358 | + struct vmw_bo *src, | |
| 1337 | 1359 | u32 src_offset, u32 src_stride, |
| 1338 | 1360 | u32 w, u32 h, |
| 1339 | 1361 | struct vmw_diff_cpy *diff); |
| @@ -32,7 +32,6 @@ · 31 unmodified lines | ||
| 32 | 32 | #define VMW_FENCE_WRAP (1 << 31) |
| 33 | 33 | |
| 34 | 34 | struct vmw_fence_manager { |
| 35 | − int num_fence_objects; | |
| 36 | 35 | struct vmw_private *dev_priv; |
| 37 | 36 | spinlock_t lock; |
| 38 | 37 | struct list_head fence_list; |
| @@ -124,13 +123,13 @@ static void vmw_fence_obj_destroy(struct dma_fence *f) · 85 unmodified lines | ||
| 124 | 123 | { |
| 125 | 124 | struct vmw_fence_obj *fence = |
| 126 | 125 | container_of(f, struct vmw_fence_obj, base); |
| 127 | − | |
| 128 | 126 | struct vmw_fence_manager *fman = fman_from_fence(fence); |
| 129 | 127 | |
| 130 | − spin_lock(&fman->lock); | |
| 131 | − list_del_init(&fence->head); | |
| 132 | − --fman->num_fence_objects; | |
| 133 | − spin_unlock(&fman->lock); | |
| 128 | + if (!list_empty(&fence->head)) { | |
| 129 | + spin_lock(&fman->lock); | |
| 130 | + list_del_init(&fence->head); | |
| 131 | + spin_unlock(&fman->lock); | |
| 132 | + } | |
| 134 | 133 | fence->destroy(fence); |
| 135 | 134 | } |
| 136 | 135 | |
| @@ -257,7 +256,6 @@ static const struct dma_fence_ops vmw_fence_ops = { · 120 unmodified lines | ||
| 257 | 256 | .release = vmw_fence_obj_destroy, |
| 258 | 257 | }; |
| 259 | 258 | |
| 260 | − | |
| 261 | 259 | /* |
| 262 | 260 | * Execute signal actions on fences recently signaled. |
| 263 | 261 | * This is done from a workqueue so we don't have to execute |
| @@ -355,7 +353,6 @@ static int vmw_fence_obj_init(struct vmw_fence_manager *fman, · 91 unmodified lines | ||
| 355 | 353 | goto out_unlock; |
| 356 | 354 | } |
| 357 | 355 | list_add_tail(&fence->head, &fman->fence_list); |
| 358 | − ++fman->num_fence_objects; | |
| 359 | 356 | |
| 360 | 357 | out_unlock: |
| 361 | 358 | spin_unlock(&fman->lock); |
| @@ -403,7 +400,7 @@ static bool vmw_fence_goal_new_locked(struct vmw_fence_manager *fman, · 41 unmodified lines | ||
| 403 | 400 | u32 passed_seqno) |
| 404 | 401 | { |
| 405 | 402 | u32 goal_seqno; |
| 406 | − struct vmw_fence_obj *fence; | |
| 403 | + struct vmw_fence_obj *fence, *next_fence; | |
| 407 | 404 | |
| 408 | 405 | if (likely(!fman->seqno_valid)) |
| 409 | 406 | return false; |
| @@ -413,7 +410,7 @@ static bool vmw_fence_goal_new_locked(struct vmw_fence_manager *fman, · 3 unmodified lines | ||
| 413 | 410 | return false; |
| 414 | 411 | |
| 415 | 412 | fman->seqno_valid = false; |
| 416 | − list_for_each_entry(fence, &fman->fence_list, head) { | |
| 413 | + list_for_each_entry_safe(fence, next_fence, &fman->fence_list, head) { | |
| 417 | 414 | if (!list_empty(&fence->seq_passed_actions)) { |
| 418 | 415 | fman->seqno_valid = true; |
| 419 | 416 | vmw_fence_goal_write(fman->dev_priv, |
| @@ -1,6 +1,7 @@ | ||
| 1 | 1 | /* SPDX-License-Identifier: GPL-2.0 OR MIT */ |
| 2 | 2 | /* |
| 3 | − * Copyright 2021-2023 VMware, Inc. | |
| 3 | + * Copyright (c) 2021-2024 Broadcom. All Rights Reserved. The term | |
| 4 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 4 | 5 | * |
| 5 | 6 | * Permission is hereby granted, free of charge, to any person |
| 6 | 7 | * obtaining a copy of this software and associated documentation |
| @@ -78,6 +79,59 @@ static struct sg_table *vmw_gem_object_get_sg_table(struct drm_gem_object *obj) · 71 unmodified lines | ||
| 78 | 79 | return drm_prime_pages_to_sg(obj->dev, vmw_tt->dma_ttm.pages, vmw_tt->dma_ttm.num_pages); |
| 79 | 80 | } |
| 80 | 81 | |
| 82 | +static int vmw_gem_vmap(struct drm_gem_object *obj, struct iosys_map *map) | |
| 83 | +{ | |
| 84 | + struct ttm_buffer_object *bo = drm_gem_ttm_of_gem(obj); | |
| 85 | + int ret; | |
| 86 | + | |
| 87 | + if (obj->import_attach) { | |
| 88 | + ret = dma_buf_vmap(obj->import_attach->dmabuf, map); | |
| 89 | + if (!ret) { | |
| 90 | + if (drm_WARN_ON(obj->dev, map->is_iomem)) { | |
| 91 | + dma_buf_vunmap(obj->import_attach->dmabuf, map); | |
| 92 | + return -EIO; | |
| 93 | + } | |
| 94 | + } | |
| 95 | + } else { | |
| 96 | + ret = ttm_bo_vmap(bo, map); | |
| 97 | + } | |
| 98 | + | |
| 99 | + return ret; | |
| 100 | +} | |
| 101 | + | |
| 102 | +static void vmw_gem_vunmap(struct drm_gem_object *obj, struct iosys_map *map) | |
| 103 | +{ | |
| 104 | + if (obj->import_attach) | |
| 105 | + dma_buf_vunmap(obj->import_attach->dmabuf, map); | |
| 106 | + else | |
| 107 | + drm_gem_ttm_vunmap(obj, map); | |
| 108 | +} | |
| 109 | + | |
| 110 | +static int vmw_gem_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma) | |
| 111 | +{ | |
| 112 | + int ret; | |
| 113 | + | |
| 114 | + if (obj->import_attach) { | |
| 115 | + /* | |
| 116 | + * Reset both vm_ops and vm_private_data, so we don't end up with | |
| 117 | + * vm_ops pointing to our implementation if the dma-buf backend | |
| 118 | + * doesn't set those fields. | |
| 119 | + */ | |
| 120 | + vma->vm_private_data = NULL; | |
| 121 | + vma->vm_ops = NULL; | |
| 122 | + | |
| 123 | + ret = dma_buf_mmap(obj->dma_buf, vma, 0); | |
| 124 | + | |
| 125 | + /* Drop the reference drm_gem_mmap_obj() acquired.*/ | |
| 126 | + if (!ret) | |
| 127 | + drm_gem_object_put(obj); | |
| 128 | + | |
| 129 | + return ret; | |
| 130 | + } | |
| 131 | + | |
| 132 | + return drm_gem_ttm_mmap(obj, vma); | |
| 133 | +} | |
| 134 | + | |
| 81 | 135 | static const struct vm_operations_struct vmw_vm_ops = { |
| 82 | 136 | .pfn_mkwrite = vmw_bo_vm_mkwrite, |
| 83 | 137 | .page_mkwrite = vmw_bo_vm_mkwrite, |
| @@ -94,9 +148,9 @@ static const struct drm_gem_object_funcs vmw_gem_object_funcs = { · 10 unmodified lines | ||
| 94 | 148 | .pin = vmw_gem_object_pin, |
| 95 | 149 | .unpin = vmw_gem_object_unpin, |
| 96 | 150 | .get_sg_table = vmw_gem_object_get_sg_table, |
| 97 | − .vmap = drm_gem_ttm_vmap, | |
| 98 | − .vunmap = drm_gem_ttm_vunmap, | |
| 99 | − .mmap = drm_gem_ttm_mmap, | |
| 151 | + .vmap = vmw_gem_vmap, | |
| 152 | + .vunmap = vmw_gem_vunmap, | |
| 153 | + .mmap = vmw_gem_mmap, | |
| 100 | 154 | .vm_ops = &vmw_vm_ops, |
| 101 | 155 | }; |
| 102 | 156 | |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0 OR MIT |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright 2009-2023 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2009-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -193,13 +194,16 @@ static u32 vmw_du_cursor_mob_size(u32 w, u32 h) · 185 unmodified lines | ||
| 193 | 194 | */ |
| 194 | 195 | static u32 *vmw_du_cursor_plane_acquire_image(struct vmw_plane_state *vps) |
| 195 | 196 | { |
| 196 | − if (vps->surf) { | |
| 197 | − if (vps->surf_mapped) | |
| 198 | − return vmw_bo_map_and_cache(vps->surf->res.guest_memory_bo); | |
| 199 | − return vps->surf->snooper.image; | |
| 200 | − } else if (vps->bo) | |
| 201 | − return vmw_bo_map_and_cache(vps->bo); | |
| 202 | − return NULL; | |
| 197 | + struct vmw_surface *surf; | |
| 198 | + | |
| 199 | + if (vmw_user_object_is_null(&vps->uo)) | |
| 200 | + return NULL; | |
| 201 | + | |
| 202 | + surf = vmw_user_object_surface(&vps->uo); | |
| 203 | + if (surf && !vmw_user_object_is_mapped(&vps->uo)) | |
| 204 | + return surf->snooper.image; | |
| 205 | + | |
| 206 | + return vmw_user_object_map(&vps->uo); | |
| 203 | 207 | } |
| 204 | 208 | |
| 205 | 209 | static bool vmw_du_cursor_plane_has_changed(struct vmw_plane_state *old_vps, |
| @@ -536,22 +540,16 @@ void vmw_du_primary_plane_destroy(struct drm_plane *plane) · 330 unmodified lines | ||
| 536 | 540 | * vmw_du_plane_unpin_surf - unpins resource associated with a framebuffer surface |
| 537 | 541 | * |
| 538 | 542 | * @vps: plane state associated with the display surface |
| 539 | − * @unreference: true if we also want to unreference the display. | |
| 540 | 543 | */ |
| 541 | −void vmw_du_plane_unpin_surf(struct vmw_plane_state *vps, | |
| 542 | − bool unreference) | |
| 544 | +void vmw_du_plane_unpin_surf(struct vmw_plane_state *vps) | |
| 543 | 545 | { |
| 544 | − if (vps->surf) { | |
| 546 | + struct vmw_surface *surf = vmw_user_object_surface(&vps->uo); | |
| 547 | + | |
| 548 | + if (surf) { | |
| 545 | 549 | if (vps->pinned) { |
| 546 | − vmw_resource_unpin(&vps->surf->res); | |
| 550 | + vmw_resource_unpin(&surf->res); | |
| 547 | 551 | vps->pinned--; |
| 548 | 552 | } |
| 549 | − | |
| 550 | − if (unreference) { | |
| 551 | − if (vps->pinned) | |
| 552 | − DRM_ERROR("Surface still pinned\n"); | |
| 553 | − vmw_surface_unreference(&vps->surf); | |
| 554 | − } | |
| 555 | 553 | } |
| 556 | 554 | } |
| 557 | 555 | |
| @@ -572,7 +570,7 @@ vmw_du_plane_cleanup_fb(struct drm_plane *plane, · 14 unmodified lines | ||
| 572 | 570 | { |
| 573 | 571 | struct vmw_plane_state *vps = vmw_plane_state_to_vps(old_state); |
| 574 | 572 | |
| 575 | − vmw_du_plane_unpin_surf(vps, false); | |
| 573 | + vmw_du_plane_unpin_surf(vps); | |
| 576 | 574 | } |
| 577 | 575 | |
| 578 | 576 | |
| @@ -661,25 +659,14 @@ vmw_du_cursor_plane_cleanup_fb(struct drm_plane *plane, · 82 unmodified lines | ||
| 661 | 659 | struct vmw_cursor_plane *vcp = vmw_plane_to_vcp(plane); |
| 662 | 660 | struct vmw_plane_state *vps = vmw_plane_state_to_vps(old_state); |
| 663 | 661 | |
| 664 | − if (vps->surf_mapped) { | |
| 665 | − vmw_bo_unmap(vps->surf->res.guest_memory_bo); | |
| 666 | − vps->surf_mapped = false; | |
| 667 | − } | |
| 662 | + if (!vmw_user_object_is_null(&vps->uo)) | |
| 663 | + vmw_user_object_unmap(&vps->uo); | |
| 668 | 664 | |
| 669 | 665 | vmw_du_cursor_plane_unmap_cm(vps); |
| 670 | 666 | vmw_du_put_cursor_mob(vcp, vps); |
| 671 | 667 | |
| 672 | − vmw_du_plane_unpin_surf(vps, false); | |
| 673 | − | |
| 674 | − if (vps->surf) { | |
| 675 | − vmw_surface_unreference(&vps->surf); | |
| 676 | − vps->surf = NULL; | |
| 677 | − } | |
| 678 | − | |
| 679 | − if (vps->bo) { | |
| 680 | − vmw_bo_unreference(&vps->bo); | |
| 681 | − vps->bo = NULL; | |
| 682 | − } | |
| 668 | + vmw_du_plane_unpin_surf(vps); | |
| 669 | + vmw_user_object_unref(&vps->uo); | |
| 683 | 670 | } |
| 684 | 671 | |
| 685 | 672 | |
| @@ -698,64 +685,48 @@ vmw_du_cursor_plane_prepare_fb(struct drm_plane *plane, · 12 unmodified lines | ||
| 698 | 685 | struct drm_framebuffer *fb = new_state->fb; |
| 699 | 686 | struct vmw_cursor_plane *vcp = vmw_plane_to_vcp(plane); |
| 700 | 687 | struct vmw_plane_state *vps = vmw_plane_state_to_vps(new_state); |
| 688 | + struct vmw_bo *bo = NULL; | |
| 701 | 689 | int ret = 0; |
| 702 | 690 | |
| 703 | − if (vps->surf) { | |
| 704 | − if (vps->surf_mapped) { | |
| 705 | − vmw_bo_unmap(vps->surf->res.guest_memory_bo); | |
| 706 | − vps->surf_mapped = false; | |
| 707 | − } | |
| 708 | − vmw_surface_unreference(&vps->surf); | |
| 709 | − vps->surf = NULL; | |
| 710 | − } | |
| 711 | − | |
| 712 | − if (vps->bo) { | |
| 713 | − vmw_bo_unreference(&vps->bo); | |
| 714 | − vps->bo = NULL; | |
| 691 | + if (!vmw_user_object_is_null(&vps->uo)) { | |
| 692 | + vmw_user_object_unmap(&vps->uo); | |
| 693 | + vmw_user_object_unref(&vps->uo); | |
| 715 | 694 | } |
| 716 | 695 | |
| 717 | 696 | if (fb) { |
| 718 | 697 | if (vmw_framebuffer_to_vfb(fb)->bo) { |
| 719 | − vps->bo = vmw_framebuffer_to_vfbd(fb)->buffer; | |
| 720 | − vmw_bo_reference(vps->bo); | |
| 698 | + vps->uo.buffer = vmw_framebuffer_to_vfbd(fb)->buffer; | |
| 699 | + vps->uo.surface = NULL; | |
| 721 | 700 | } else { |
| 722 | − vps->surf = vmw_framebuffer_to_vfbs(fb)->surface; | |
| 723 | − vmw_surface_reference(vps->surf); | |
| 701 | + memcpy(&vps->uo, &vmw_framebuffer_to_vfbs(fb)->uo, sizeof(vps->uo)); | |
| 724 | 702 | } |
| 703 | + vmw_user_object_ref(&vps->uo); | |
| 725 | 704 | } |
| 726 | 705 | |
| 727 | − if (!vps->surf && vps->bo) { | |
| 728 | − const u32 size = new_state->crtc_w * new_state->crtc_h * sizeof(u32); | |
| 706 | + bo = vmw_user_object_buffer(&vps->uo); | |
| 707 | + if (bo) { | |
| 708 | + struct ttm_operation_ctx ctx = {false, false}; | |
| 729 | 709 | |
| 730 | − /* | |
| 731 | − * Not using vmw_bo_map_and_cache() helper here as we need to | |
| 732 | − * reserve the ttm_buffer_object first which | |
| 733 | − * vmw_bo_map_and_cache() omits. | |
| 734 | − */ | |
| 735 | − ret = ttm_bo_reserve(&vps->bo->tbo, true, false, NULL); | |
| 736 | − | |
| 737 | − if (unlikely(ret != 0)) | |
| 710 | + ret = ttm_bo_reserve(&bo->tbo, true, false, NULL); | |
| 711 | + if (ret != 0) | |
| 738 | 712 | return -ENOMEM; |
| 739 | 713 | |
| 740 | − ret = ttm_bo_kmap(&vps->bo->tbo, 0, PFN_UP(size), &vps->bo->map); | |
| 741 | − | |
| 742 | − ttm_bo_unreserve(&vps->bo->tbo); | |
| 743 | − | |
| 744 | − if (unlikely(ret != 0)) | |
| 714 | + ret = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); | |
| 715 | + if (ret != 0) | |
| 745 | 716 | return -ENOMEM; |
| 746 | − } else if (vps->surf && !vps->bo && vps->surf->res.guest_memory_bo) { | |
| 747 | 717 | |
| 748 | − WARN_ON(vps->surf->snooper.image); | |
| 749 | − ret = ttm_bo_reserve(&vps->surf->res.guest_memory_bo->tbo, true, false, | |
| 750 | − NULL); | |
| 751 | − if (unlikely(ret != 0)) | |
| 752 | − return -ENOMEM; | |
| 753 | − vmw_bo_map_and_cache(vps->surf->res.guest_memory_bo); | |
| 754 | − ttm_bo_unreserve(&vps->surf->res.guest_memory_bo->tbo); | |
| 755 | − vps->surf_mapped = true; | |
| 718 | + vmw_bo_pin_reserved(bo, true); | |
| 719 | + if (vmw_framebuffer_to_vfb(fb)->bo) { | |
| 720 | + const u32 size = new_state->crtc_w * new_state->crtc_h * sizeof(u32); | |
| 721 | + | |
| 722 | + (void)vmw_bo_map_and_cache_size(bo, size); | |
| 723 | + } else { | |
| 724 | + vmw_bo_map_and_cache(bo); | |
| 725 | + } | |
| 726 | + ttm_bo_unreserve(&bo->tbo); | |
| 756 | 727 | } |
| 757 | 728 | |
| 758 | − if (vps->surf || vps->bo) { | |
| 729 | + if (!vmw_user_object_is_null(&vps->uo)) { | |
| 759 | 730 | vmw_du_get_cursor_mob(vcp, vps); |
| 760 | 731 | vmw_du_cursor_plane_map_cm(vps); |
| 761 | 732 | } |
| @@ -777,14 +748,17 @@ vmw_du_cursor_plane_atomic_update(struct drm_plane *plane, · 15 unmodified lines | ||
| 777 | 748 | struct vmw_display_unit *du = vmw_crtc_to_du(crtc); |
| 778 | 749 | struct vmw_plane_state *vps = vmw_plane_state_to_vps(new_state); |
| 779 | 750 | struct vmw_plane_state *old_vps = vmw_plane_state_to_vps(old_state); |
| 751 | + struct vmw_bo *old_bo = NULL; | |
| 752 | + struct vmw_bo *new_bo = NULL; | |
| 780 | 753 | s32 hotspot_x, hotspot_y; |
| 754 | + int ret; | |
| 781 | 755 | |
| 782 | 756 | hotspot_x = du->hotspot_x + new_state->hotspot_x; |
| 783 | 757 | hotspot_y = du->hotspot_y + new_state->hotspot_y; |
| 784 | 758 | |
| 785 | − du->cursor_surface = vps->surf; | |
| 759 | + du->cursor_surface = vmw_user_object_surface(&vps->uo); | |
| 786 | 760 | |
| 787 | − if (!vps->surf && !vps->bo) { | |
| 761 | + if (vmw_user_object_is_null(&vps->uo)) { | |
| 788 | 762 | vmw_cursor_update_position(dev_priv, false, 0, 0); |
| 789 | 763 | return; |
| 790 | 764 | } |
| @@ -792,10 +766,26 @@ vmw_du_cursor_plane_atomic_update(struct drm_plane *plane, · 1 unmodified line | ||
| 792 | 766 | vps->cursor.hotspot_x = hotspot_x; |
| 793 | 767 | vps->cursor.hotspot_y = hotspot_y; |
| 794 | 768 | |
| 795 | − if (vps->surf) { | |
| 769 | + if (du->cursor_surface) | |
| 796 | 770 | du->cursor_age = du->cursor_surface->snooper.age; |
| 771 | + | |
| 772 | + if (!vmw_user_object_is_null(&old_vps->uo)) { | |
| 773 | + old_bo = vmw_user_object_buffer(&old_vps->uo); | |
| 774 | + ret = ttm_bo_reserve(&old_bo->tbo, false, false, NULL); | |
| 775 | + if (ret != 0) | |
| 776 | + return; | |
| 797 | 777 | } |
| 798 | 778 | |
| 779 | + if (!vmw_user_object_is_null(&vps->uo)) { | |
| 780 | + new_bo = vmw_user_object_buffer(&vps->uo); | |
| 781 | + if (old_bo != new_bo) { | |
| 782 | + ret = ttm_bo_reserve(&new_bo->tbo, false, false, NULL); | |
| 783 | + if (ret != 0) | |
| 784 | + return; | |
| 785 | + } else { | |
| 786 | + new_bo = NULL; | |
| 787 | + } | |
| 788 | + } | |
| 799 | 789 | if (!vmw_du_cursor_plane_has_changed(old_vps, vps)) { |
| 800 | 790 | /* |
| 801 | 791 | * If it hasn't changed, avoid making the device do extra |
| @@ -813,6 +803,11 @@ vmw_du_cursor_plane_atomic_update(struct drm_plane *plane, · 11 unmodified lines | ||
| 813 | 803 | hotspot_x, hotspot_y); |
| 814 | 804 | } |
| 815 | 805 | |
| 806 | + if (old_bo) | |
| 807 | + ttm_bo_unreserve(&old_bo->tbo); | |
| 808 | + if (new_bo) | |
| 809 | + ttm_bo_unreserve(&new_bo->tbo); | |
| 810 | + | |
| 816 | 811 | du->cursor_x = new_state->crtc_x + du->set_gui_x; |
| 817 | 812 | du->cursor_y = new_state->crtc_y + du->set_gui_y; |
| 818 | 813 | |
| @@ -913,7 +908,7 @@ int vmw_du_cursor_plane_atomic_check(struct drm_plane *plane, · 94 unmodified lines | ||
| 913 | 908 | } |
| 914 | 909 | |
| 915 | 910 | if (!vmw_framebuffer_to_vfb(fb)->bo) { |
| 916 | − surface = vmw_framebuffer_to_vfbs(fb)->surface; | |
| 911 | + surface = vmw_user_object_surface(&vmw_framebuffer_to_vfbs(fb)->uo); | |
| 917 | 912 | |
| 918 | 913 | WARN_ON(!surface); |
| 919 | 914 | |
| @@ -1074,12 +1069,7 @@ vmw_du_plane_duplicate_state(struct drm_plane *plane) · 154 unmodified lines | ||
| 1074 | 1069 | memset(&vps->cursor, 0, sizeof(vps->cursor)); |
| 1075 | 1070 | |
| 1076 | 1071 | /* Each ref counted resource needs to be acquired again */ |
| 1077 | − if (vps->surf) | |
| 1078 | − (void) vmw_surface_reference(vps->surf); | |
| 1079 | − | |
| 1080 | − if (vps->bo) | |
| 1081 | − (void) vmw_bo_reference(vps->bo); | |
| 1082 | − | |
| 1072 | + vmw_user_object_ref(&vps->uo); | |
| 1083 | 1073 | state = &vps->base; |
| 1084 | 1074 | |
| 1085 | 1075 | __drm_atomic_helper_plane_duplicate_state(plane, state); |
| @@ -1128,11 +1118,7 @@ vmw_du_plane_destroy_state(struct drm_plane *plane, · 42 unmodified lines | ||
| 1128 | 1118 | struct vmw_plane_state *vps = vmw_plane_state_to_vps(state); |
| 1129 | 1119 | |
| 1130 | 1120 | /* Should have been freed by cleanup_fb */ |
| 1131 | − if (vps->surf) | |
| 1132 | − vmw_surface_unreference(&vps->surf); | |
| 1133 | − | |
| 1134 | − if (vps->bo) | |
| 1135 | − vmw_bo_unreference(&vps->bo); | |
| 1121 | + vmw_user_object_unref(&vps->uo); | |
| 1136 | 1122 | |
| 1137 | 1123 | drm_atomic_helper_plane_destroy_state(plane, state); |
| 1138 | 1124 | } |
| @@ -1227,7 +1213,7 @@ static void vmw_framebuffer_surface_destroy(struct drm_framebuffer *framebuffer) · 88 unmodified lines | ||
| 1227 | 1213 | vmw_framebuffer_to_vfbs(framebuffer); |
| 1228 | 1214 | |
| 1229 | 1215 | drm_framebuffer_cleanup(framebuffer); |
| 1230 | − vmw_surface_unreference(&vfbs->surface); | |
| 1216 | + vmw_user_object_unref(&vfbs->uo); | |
| 1231 | 1217 | |
| 1232 | 1218 | kfree(vfbs); |
| 1233 | 1219 | } |
| @@ -1272,29 +1258,41 @@ int vmw_kms_readback(struct vmw_private *dev_priv, · 38 unmodified lines | ||
| 1272 | 1258 | return -ENOSYS; |
| 1273 | 1259 | } |
| 1274 | 1260 | |
| 1261 | +static int vmw_framebuffer_surface_create_handle(struct drm_framebuffer *fb, | |
| 1262 | + struct drm_file *file_priv, | |
| 1263 | + unsigned int *handle) | |
| 1264 | +{ | |
| 1265 | + struct vmw_framebuffer_surface *vfbs = vmw_framebuffer_to_vfbs(fb); | |
| 1266 | + struct vmw_bo *bo = vmw_user_object_buffer(&vfbs->uo); | |
| 1267 | + | |
| 1268 | + return drm_gem_handle_create(file_priv, &bo->tbo.base, handle); | |
| 1269 | +} | |
| 1275 | 1270 | |
| 1276 | 1271 | static const struct drm_framebuffer_funcs vmw_framebuffer_surface_funcs = { |
| 1272 | + .create_handle = vmw_framebuffer_surface_create_handle, | |
| 1277 | 1273 | .destroy = vmw_framebuffer_surface_destroy, |
| 1278 | 1274 | .dirty = drm_atomic_helper_dirtyfb, |
| 1279 | 1275 | }; |
| 1280 | 1276 | |
| 1281 | 1277 | static int vmw_kms_new_framebuffer_surface(struct vmw_private *dev_priv, |
| 1282 | − struct vmw_surface *surface, | |
| 1278 | + struct vmw_user_object *uo, | |
| 1283 | 1279 | struct vmw_framebuffer **out, |
| 1284 | 1280 | const struct drm_mode_fb_cmd2 |
| 1285 | − *mode_cmd, | |
| 1286 | − bool is_bo_proxy) | |
| 1281 | + *mode_cmd) | |
| 1287 | 1282 | |
| 1288 | 1283 | { |
| 1289 | 1284 | struct drm_device *dev = &dev_priv->drm; |
| 1290 | 1285 | struct vmw_framebuffer_surface *vfbs; |
| 1291 | 1286 | enum SVGA3dSurfaceFormat format; |
| 1287 | + struct vmw_surface *surface; | |
| 1292 | 1288 | int ret; |
| 1293 | 1289 | |
| 1294 | 1290 | /* 3D is only supported on HWv8 and newer hosts */ |
| 1295 | 1291 | if (dev_priv->active_display_unit == vmw_du_legacy) |
| 1296 | 1292 | return -ENOSYS; |
| 1297 | 1293 | |
| 1294 | + surface = vmw_user_object_surface(uo); | |
| 1295 | + | |
| 1298 | 1296 | /* |
| 1299 | 1297 | * Sanity checks. |
| 1300 | 1298 | */ |
| @@ -1357,8 +1355,8 @@ static int vmw_kms_new_framebuffer_surface(struct vmw_private *dev_priv, · 56 unmodified lines | ||
| 1357 | 1355 | } |
| 1358 | 1356 | |
| 1359 | 1357 | drm_helper_mode_fill_fb_struct(dev, &vfbs->base.base, mode_cmd); |
| 1360 | − vfbs->surface = vmw_surface_reference(surface); | |
| 1361 | − vfbs->is_bo_proxy = is_bo_proxy; | |
| 1358 | + memcpy(&vfbs->uo, uo, sizeof(vfbs->uo)); | |
| 1359 | + vmw_user_object_ref(&vfbs->uo); | |
| 1362 | 1360 | |
| 1363 | 1361 | *out = &vfbs->base; |
| 1364 | 1362 | |
| @@ -1370,7 +1368,7 @@ static int vmw_kms_new_framebuffer_surface(struct vmw_private *dev_priv, · 5 unmodified lines | ||
| 1370 | 1368 | return 0; |
| 1371 | 1369 | |
| 1372 | 1370 | out_err2: |
| 1373 | − vmw_surface_unreference(&surface); | |
| 1371 | + vmw_user_object_unref(&vfbs->uo); | |
| 1374 | 1372 | kfree(vfbs); |
| 1375 | 1373 | out_err1: |
| 1376 | 1374 | return ret; |
| @@ -1386,7 +1384,6 @@ static int vmw_framebuffer_bo_create_handle(struct drm_framebuffer *fb, · 9 unmodified lines | ||
| 1386 | 1384 | { |
| 1387 | 1385 | struct vmw_framebuffer_bo *vfbd = |
| 1388 | 1386 | vmw_framebuffer_to_vfbd(fb); |
| 1389 | − | |
| 1390 | 1387 | return drm_gem_handle_create(file_priv, &vfbd->buffer->tbo.base, handle); |
| 1391 | 1388 | } |
| 1392 | 1389 | |
| @@ -1407,86 +1404,6 @@ static const struct drm_framebuffer_funcs vmw_framebuffer_bo_funcs = { · 14 unmodified lines | ||
| 1407 | 1404 | .dirty = drm_atomic_helper_dirtyfb, |
| 1408 | 1405 | }; |
| 1409 | 1406 | |
| 1410 | −/** | |
| 1411 | − * vmw_create_bo_proxy - create a proxy surface for the buffer object | |
| 1412 | − * | |
| 1413 | − * @dev: DRM device | |
| 1414 | − * @mode_cmd: parameters for the new surface | |
| 1415 | − * @bo_mob: MOB backing the buffer object | |
| 1416 | − * @srf_out: newly created surface | |
| 1417 | − * | |
| 1418 | − * When the content FB is a buffer object, we create a surface as a proxy to the | |
| 1419 | − * same buffer. This way we can do a surface copy rather than a surface DMA. | |
| 1420 | − * This is a more efficient approach | |
| 1421 | − * | |
| 1422 | − * RETURNS: | |
| 1423 | − * 0 on success, error code otherwise | |
| 1424 | − */ | |
| 1425 | −static int vmw_create_bo_proxy(struct drm_device *dev, | |
| 1426 | − const struct drm_mode_fb_cmd2 *mode_cmd, | |
| 1427 | − struct vmw_bo *bo_mob, | |
| 1428 | − struct vmw_surface **srf_out) | |
| 1429 | −{ | |
| 1430 | − struct vmw_surface_metadata metadata = {0}; | |
| 1431 | − uint32_t format; | |
| 1432 | − struct vmw_resource *res; | |
| 1433 | − unsigned int bytes_pp; | |
| 1434 | − int ret; | |
| 1435 | − | |
| 1436 | − switch (mode_cmd->pixel_format) { | |
| 1437 | − case DRM_FORMAT_ARGB8888: | |
| 1438 | − case DRM_FORMAT_XRGB8888: | |
| 1439 | − format = SVGA3D_X8R8G8B8; | |
| 1440 | − bytes_pp = 4; | |
| 1441 | − break; | |
| 1442 | − | |
| 1443 | − case DRM_FORMAT_RGB565: | |
| 1444 | − case DRM_FORMAT_XRGB1555: | |
| 1445 | − format = SVGA3D_R5G6B5; | |
| 1446 | − bytes_pp = 2; | |
| 1447 | − break; | |
| 1448 | − | |
| 1449 | − case 8: | |
| 1450 | − format = SVGA3D_P8; | |
| 1451 | − bytes_pp = 1; | |
| 1452 | − break; | |
| 1453 | − | |
| 1454 | − default: | |
| 1455 | − DRM_ERROR("Invalid framebuffer format %p4cc\n", | |
| 1456 | − &mode_cmd->pixel_format); | |
| 1457 | − return -EINVAL; | |
| 1458 | − } | |
| 1459 | − | |
| 1460 | − metadata.format = format; | |
| 1461 | − metadata.mip_levels[0] = 1; | |
| 1462 | − metadata.num_sizes = 1; | |
| 1463 | − metadata.base_size.width = mode_cmd->pitches[0] / bytes_pp; | |
| 1464 | − metadata.base_size.height = mode_cmd->height; | |
| 1465 | − metadata.base_size.depth = 1; | |
| 1466 | − metadata.scanout = true; | |
| 1467 | − | |
| 1468 | − ret = vmw_gb_surface_define(vmw_priv(dev), &metadata, srf_out); | |
| 1469 | − if (ret) { | |
| 1470 | − DRM_ERROR("Failed to allocate proxy content buffer\n"); | |
| 1471 | − return ret; | |
| 1472 | − } | |
| 1473 | − | |
| 1474 | − res = &(*srf_out)->res; | |
| 1475 | − | |
| 1476 | − /* Reserve and switch the backing mob. */ | |
| 1477 | − mutex_lock(&res->dev_priv->cmdbuf_mutex); | |
| 1478 | − (void) vmw_resource_reserve(res, false, true); | |
| 1479 | − vmw_user_bo_unref(&res->guest_memory_bo); | |
| 1480 | − res->guest_memory_bo = vmw_user_bo_ref(bo_mob); | |
| 1481 | − res->guest_memory_offset = 0; | |
| 1482 | − vmw_resource_unreserve(res, false, false, false, NULL, 0); | |
| 1483 | − mutex_unlock(&res->dev_priv->cmdbuf_mutex); | |
| 1484 | − | |
| 1485 | − return 0; | |
| 1486 | −} | |
| 1487 | − | |
| 1488 | − | |
| 1489 | − | |
| 1490 | 1407 | static int vmw_kms_new_framebuffer_bo(struct vmw_private *dev_priv, |
| 1491 | 1408 | struct vmw_bo *bo, |
| 1492 | 1409 | struct vmw_framebuffer **out, |
| @@ -1565,55 +1482,24 @@ vmw_kms_srf_ok(struct vmw_private *dev_priv, uint32_t width, uint32_t height) · 72 unmodified lines | ||
| 1565 | 1482 | * vmw_kms_new_framebuffer - Create a new framebuffer. |
| 1566 | 1483 | * |
| 1567 | 1484 | * @dev_priv: Pointer to device private struct. |
| 1568 | − * @bo: Pointer to buffer object to wrap the kms framebuffer around. | |
| 1569 | − * Either @bo or @surface must be NULL. | |
| 1570 | − * @surface: Pointer to a surface to wrap the kms framebuffer around. | |
| 1571 | − * Either @bo or @surface must be NULL. | |
| 1572 | − * @only_2d: No presents will occur to this buffer object based framebuffer. | |
| 1573 | − * This helps the code to do some important optimizations. | |
| 1485 | + * @uo: Pointer to user object to wrap the kms framebuffer around. | |
| 1486 | + * Either the buffer or surface inside the user object must be NULL. | |
| 1574 | 1487 | * @mode_cmd: Frame-buffer metadata. |
| 1575 | 1488 | */ |
| 1576 | 1489 | struct vmw_framebuffer * |
| 1577 | 1490 | vmw_kms_new_framebuffer(struct vmw_private *dev_priv, |
| 1578 | − struct vmw_bo *bo, | |
| 1579 | − struct vmw_surface *surface, | |
| 1580 | − bool only_2d, | |
| 1491 | + struct vmw_user_object *uo, | |
| 1581 | 1492 | const struct drm_mode_fb_cmd2 *mode_cmd) |
| 1582 | 1493 | { |
| 1583 | 1494 | struct vmw_framebuffer *vfb = NULL; |
| 1584 | − bool is_bo_proxy = false; | |
| 1585 | 1495 | int ret; |
| 1586 | 1496 | |
| 1587 | − /* | |
| 1588 | − * We cannot use the SurfaceDMA command in an non-accelerated VM, | |
| 1589 | − * therefore, wrap the buffer object in a surface so we can use the | |
| 1590 | − * SurfaceCopy command. | |
| 1591 | − */ | |
| 1592 | − if (vmw_kms_srf_ok(dev_priv, mode_cmd->width, mode_cmd->height) && | |
| 1593 | − bo && only_2d && | |
| 1594 | − mode_cmd->width > 64 && /* Don't create a proxy for cursor */ | |
| 1595 | − dev_priv->active_display_unit == vmw_du_screen_target) { | |
| 1596 | − ret = vmw_create_bo_proxy(&dev_priv->drm, mode_cmd, | |
| 1597 | − bo, &surface); | |
| 1598 | − if (ret) | |
| 1599 | − return ERR_PTR(ret); | |
| 1600 | − | |
| 1601 | − is_bo_proxy = true; | |
| 1602 | − } | |
| 1603 | − | |
| 1604 | 1497 | /* Create the new framebuffer depending one what we have */ |
| 1605 | − if (surface) { | |
| 1606 | − ret = vmw_kms_new_framebuffer_surface(dev_priv, surface, &vfb, | |
| 1607 | − mode_cmd, | |
| 1608 | − is_bo_proxy); | |
| 1609 | − /* | |
| 1610 | − * vmw_create_bo_proxy() adds a reference that is no longer | |
| 1611 | − * needed | |
| 1612 | − */ | |
| 1613 | − if (is_bo_proxy) | |
| 1614 | − vmw_surface_unreference(&surface); | |
| 1615 | − } else if (bo) { | |
| 1616 | − ret = vmw_kms_new_framebuffer_bo(dev_priv, bo, &vfb, | |
| 1498 | + if (vmw_user_object_surface(uo)) { | |
| 1499 | + ret = vmw_kms_new_framebuffer_surface(dev_priv, uo, &vfb, | |
| 1500 | + mode_cmd); | |
| 1501 | + } else if (uo->buffer) { | |
| 1502 | + ret = vmw_kms_new_framebuffer_bo(dev_priv, uo->buffer, &vfb, | |
| 1617 | 1503 | mode_cmd); |
| 1618 | 1504 | } else { |
| 1619 | 1505 | BUG(); |
| @@ -1635,14 +1521,12 @@ static struct drm_framebuffer *vmw_kms_fb_create(struct drm_device *dev, · 15 unmodified lines | ||
| 1635 | 1521 | { |
| 1636 | 1522 | struct vmw_private *dev_priv = vmw_priv(dev); |
| 1637 | 1523 | struct vmw_framebuffer *vfb = NULL; |
| 1638 | − struct vmw_surface *surface = NULL; | |
| 1639 | − struct vmw_bo *bo = NULL; | |
| 1524 | + struct vmw_user_object uo = {0}; | |
| 1640 | 1525 | int ret; |
| 1641 | 1526 | |
| 1642 | 1527 | /* returns either a bo or surface */ |
| 1643 | − ret = vmw_user_lookup_handle(dev_priv, file_priv, | |
| 1644 | − mode_cmd->handles[0], | |
| 1645 | − &surface, &bo); | |
| 1528 | + ret = vmw_user_object_lookup(dev_priv, file_priv, mode_cmd->handles[0], | |
| 1529 | + &uo); | |
| 1646 | 1530 | if (ret) { |
| 1647 | 1531 | DRM_ERROR("Invalid buffer object handle %u (0x%x).\n", |
| 1648 | 1532 | mode_cmd->handles[0], mode_cmd->handles[0]); |
| @@ -1650,7 +1534,7 @@ static struct drm_framebuffer *vmw_kms_fb_create(struct drm_device *dev, · 1 unmodified line | ||
| 1650 | 1534 | } |
| 1651 | 1535 | |
| 1652 | 1536 | |
| 1653 | − if (!bo && | |
| 1537 | + if (vmw_user_object_surface(&uo) && | |
| 1654 | 1538 | !vmw_kms_srf_ok(dev_priv, mode_cmd->width, mode_cmd->height)) { |
| 1655 | 1539 | DRM_ERROR("Surface size cannot exceed %dx%d\n", |
| 1656 | 1540 | dev_priv->texture_max_width, |
| @@ -1659,20 +1543,15 @@ static struct drm_framebuffer *vmw_kms_fb_create(struct drm_device *dev, · 2 unmodified lines | ||
| 1659 | 1543 | } |
| 1660 | 1544 | |
| 1661 | 1545 | |
| 1662 | − vfb = vmw_kms_new_framebuffer(dev_priv, bo, surface, | |
| 1663 | − !(dev_priv->capabilities & SVGA_CAP_3D), | |
| 1664 | − mode_cmd); | |
| 1546 | + vfb = vmw_kms_new_framebuffer(dev_priv, &uo, mode_cmd); | |
| 1665 | 1547 | if (IS_ERR(vfb)) { |
| 1666 | 1548 | ret = PTR_ERR(vfb); |
| 1667 | 1549 | goto err_out; |
| 1668 | 1550 | } |
| 1669 | 1551 | |
| 1670 | 1552 | err_out: |
| 1671 | − /* vmw_user_lookup_handle takes one ref so does new_fb */ | |
| 1672 | − if (bo) | |
| 1673 | − vmw_user_bo_unref(&bo); | |
| 1674 | − if (surface) | |
| 1675 | − vmw_surface_unreference(&surface); | |
| 1553 | + /* vmw_user_object_lookup takes one ref so does new_fb */ | |
| 1554 | + vmw_user_object_unref(&uo); | |
| 1676 | 1555 | |
| 1677 | 1556 | if (ret) { |
| 1678 | 1557 | DRM_ERROR("failed to create vmw_framebuffer: %i\n", ret); |
| @@ -2584,72 +2463,6 @@ void vmw_kms_helper_validation_finish(struct vmw_private *dev_priv, · 905 unmodified lines | ||
| 2584 | 2463 | vmw_fence_obj_unreference(&fence); |
| 2585 | 2464 | } |
| 2586 | 2465 | |
| 2587 | −/** | |
| 2588 | − * vmw_kms_update_proxy - Helper function to update a proxy surface from | |
| 2589 | − * its backing MOB. | |
| 2590 | − * | |
| 2591 | − * @res: Pointer to the surface resource | |
| 2592 | − * @clips: Clip rects in framebuffer (surface) space. | |
| 2593 | − * @num_clips: Number of clips in @clips. | |
| 2594 | − * @increment: Integer with which to increment the clip counter when looping. | |
| 2595 | − * Used to skip a predetermined number of clip rects. | |
| 2596 | − * | |
| 2597 | − * This function makes sure the proxy surface is updated from its backing MOB | |
| 2598 | − * using the region given by @clips. The surface resource @res and its backing | |
| 2599 | − * MOB needs to be reserved and validated on call. | |
| 2600 | − */ | |
| 2601 | −int vmw_kms_update_proxy(struct vmw_resource *res, | |
| 2602 | − const struct drm_clip_rect *clips, | |
| 2603 | − unsigned num_clips, | |
| 2604 | − int increment) | |
| 2605 | −{ | |
| 2606 | − struct vmw_private *dev_priv = res->dev_priv; | |
| 2607 | − struct drm_vmw_size *size = &vmw_res_to_srf(res)->metadata.base_size; | |
| 2608 | − struct { | |
| 2609 | − SVGA3dCmdHeader header; | |
| 2610 | − SVGA3dCmdUpdateGBImage body; | |
| 2611 | − } *cmd; | |
| 2612 | − SVGA3dBox *box; | |
| 2613 | − size_t copy_size = 0; | |
| 2614 | − int i; | |
| 2615 | − | |
| 2616 | − if (!clips) | |
| 2617 | − return 0; | |
| 2618 | − | |
| 2619 | − cmd = VMW_CMD_RESERVE(dev_priv, sizeof(*cmd) * num_clips); | |
| 2620 | − if (!cmd) | |
| 2621 | − return -ENOMEM; | |
| 2622 | − | |
| 2623 | − for (i = 0; i < num_clips; ++i, clips += increment, ++cmd) { | |
| 2624 | − box = &cmd->body.box; | |
| 2625 | − | |
| 2626 | − cmd->header.id = SVGA_3D_CMD_UPDATE_GB_IMAGE; | |
| 2627 | − cmd->header.size = sizeof(cmd->body); | |
| 2628 | − cmd->body.image.sid = res->id; | |
| 2629 | − cmd->body.image.face = 0; | |
| 2630 | − cmd->body.image.mipmap = 0; | |
| 2631 | − | |
| 2632 | − if (clips->x1 > size->width || clips->x2 > size->width || | |
| 2633 | − clips->y1 > size->height || clips->y2 > size->height) { | |
| 2634 | − DRM_ERROR("Invalid clips outsize of framebuffer.\n"); | |
| 2635 | − return -EINVAL; | |
| 2636 | − } | |
| 2637 | − | |
| 2638 | − box->x = clips->x1; | |
| 2639 | − box->y = clips->y1; | |
| 2640 | − box->z = 0; | |
| 2641 | − box->w = clips->x2 - clips->x1; | |
| 2642 | − box->h = clips->y2 - clips->y1; | |
| 2643 | − box->d = 1; | |
| 2644 | − | |
| 2645 | − copy_size += sizeof(*cmd); | |
| 2646 | − } | |
| 2647 | − | |
| 2648 | − vmw_cmd_commit(dev_priv, copy_size); | |
| 2649 | − | |
| 2650 | − return 0; | |
| 2651 | −} | |
| 2652 | − | |
| 2653 | 2466 | /** |
| 2654 | 2467 | * vmw_kms_create_implicit_placement_property - Set up the implicit placement |
| 2655 | 2468 | * property. |
| @@ -2784,8 +2597,9 @@ int vmw_du_helper_plane_update(struct vmw_du_update_plane *update) · 128 unmodified lines | ||
| 2784 | 2597 | } else { |
| 2785 | 2598 | struct vmw_framebuffer_surface *vfbs = |
| 2786 | 2599 | container_of(update->vfb, typeof(*vfbs), base); |
| 2600 | + struct vmw_surface *surf = vmw_user_object_surface(&vfbs->uo); | |
| 2787 | 2601 | |
| 2788 | − ret = vmw_validation_add_resource(&val_ctx, &vfbs->surface->res, | |
| 2602 | + ret = vmw_validation_add_resource(&val_ctx, &surf->res, | |
| 2789 | 2603 | 0, VMW_RES_DIRTY_NONE, NULL, |
| 2790 | 2604 | NULL); |
| 2791 | 2605 | } |
| @@ -2941,3 +2755,93 @@ int vmw_connector_get_modes(struct drm_connector *connector) · 149 unmodified lines | ||
| 2941 | 2755 | |
| 2942 | 2756 | return num_modes; |
| 2943 | 2757 | } |
| 2758 | + | |
| 2759 | +struct vmw_user_object *vmw_user_object_ref(struct vmw_user_object *uo) | |
| 2760 | +{ | |
| 2761 | + if (uo->buffer) | |
| 2762 | + vmw_user_bo_ref(uo->buffer); | |
| 2763 | + else if (uo->surface) | |
| 2764 | + vmw_surface_reference(uo->surface); | |
| 2765 | + return uo; | |
| 2766 | +} | |
| 2767 | + | |
| 2768 | +void vmw_user_object_unref(struct vmw_user_object *uo) | |
| 2769 | +{ | |
| 2770 | + if (uo->buffer) | |
| 2771 | + vmw_user_bo_unref(&uo->buffer); | |
| 2772 | + else if (uo->surface) | |
| 2773 | + vmw_surface_unreference(&uo->surface); | |
| 2774 | +} | |
| 2775 | + | |
| 2776 | +struct vmw_bo * | |
| 2777 | +vmw_user_object_buffer(struct vmw_user_object *uo) | |
| 2778 | +{ | |
| 2779 | + if (uo->buffer) | |
| 2780 | + return uo->buffer; | |
| 2781 | + else if (uo->surface) | |
| 2782 | + return uo->surface->res.guest_memory_bo; | |
| 2783 | + return NULL; | |
| 2784 | +} | |
| 2785 | + | |
| 2786 | +struct vmw_surface * | |
| 2787 | +vmw_user_object_surface(struct vmw_user_object *uo) | |
| 2788 | +{ | |
| 2789 | + if (uo->buffer) | |
| 2790 | + return uo->buffer->dumb_surface; | |
| 2791 | + return uo->surface; | |
| 2792 | +} | |
| 2793 | + | |
| 2794 | +void *vmw_user_object_map(struct vmw_user_object *uo) | |
| 2795 | +{ | |
| 2796 | + struct vmw_bo *bo = vmw_user_object_buffer(uo); | |
| 2797 | + | |
| 2798 | + WARN_ON(!bo); | |
| 2799 | + return vmw_bo_map_and_cache(bo); | |
| 2800 | +} | |
| 2801 | + | |
| 2802 | +void *vmw_user_object_map_size(struct vmw_user_object *uo, size_t size) | |
| 2803 | +{ | |
| 2804 | + struct vmw_bo *bo = vmw_user_object_buffer(uo); | |
| 2805 | + | |
| 2806 | + WARN_ON(!bo); | |
| 2807 | + return vmw_bo_map_and_cache_size(bo, size); | |
| 2808 | +} | |
| 2809 | + | |
| 2810 | +void vmw_user_object_unmap(struct vmw_user_object *uo) | |
| 2811 | +{ | |
| 2812 | + struct vmw_bo *bo = vmw_user_object_buffer(uo); | |
| 2813 | + int ret; | |
| 2814 | + | |
| 2815 | + WARN_ON(!bo); | |
| 2816 | + | |
| 2817 | + /* Fence the mob creation so we are guarateed to have the mob */ | |
| 2818 | + ret = ttm_bo_reserve(&bo->tbo, false, false, NULL); | |
| 2819 | + if (ret != 0) | |
| 2820 | + return; | |
| 2821 | + | |
| 2822 | + vmw_bo_unmap(bo); | |
| 2823 | + vmw_bo_pin_reserved(bo, false); | |
| 2824 | + | |
| 2825 | + ttm_bo_unreserve(&bo->tbo); | |
| 2826 | +} | |
| 2827 | + | |
| 2828 | +bool vmw_user_object_is_mapped(struct vmw_user_object *uo) | |
| 2829 | +{ | |
| 2830 | + struct vmw_bo *bo; | |
| 2831 | + | |
| 2832 | + if (!uo || vmw_user_object_is_null(uo)) | |
| 2833 | + return false; | |
| 2834 | + | |
| 2835 | + bo = vmw_user_object_buffer(uo); | |
| 2836 | + | |
| 2837 | + if (WARN_ON(!bo)) | |
| 2838 | + return false; | |
| 2839 | + | |
| 2840 | + WARN_ON(bo->map.bo && !bo->map.virtual); | |
| 2841 | + return bo->map.virtual; | |
| 2842 | +} | |
| 2843 | + | |
| 2844 | +bool vmw_user_object_is_null(struct vmw_user_object *uo) | |
| 2845 | +{ | |
| 2846 | + return !uo->buffer && !uo->surface; | |
| 2847 | +} | |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | /* SPDX-License-Identifier: GPL-2.0 OR MIT */ |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright 2009-2023 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2009-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -221,11 +222,9 @@ struct vmw_framebuffer { · 213 unmodified lines | ||
| 221 | 222 | |
| 222 | 223 | struct vmw_framebuffer_surface { |
| 223 | 224 | struct vmw_framebuffer base; |
| 224 | − struct vmw_surface *surface; | |
| 225 | − bool is_bo_proxy; /* true if this is proxy surface for DMA buf */ | |
| 225 | + struct vmw_user_object uo; | |
| 226 | 226 | }; |
| 227 | 227 | |
| 228 | − | |
| 229 | 228 | struct vmw_framebuffer_bo { |
| 230 | 229 | struct vmw_framebuffer base; |
| 231 | 230 | struct vmw_bo *buffer; |
| @@ -277,8 +276,7 @@ struct vmw_cursor_plane_state { · 45 unmodified lines | ||
| 277 | 276 | */ |
| 278 | 277 | struct vmw_plane_state { |
| 279 | 278 | struct drm_plane_state base; |
| 280 | − struct vmw_surface *surf; | |
| 281 | − struct vmw_bo *bo; | |
| 279 | + struct vmw_user_object uo; | |
| 282 | 280 | |
| 283 | 281 | int content_fb_type; |
| 284 | 282 | unsigned long bo_size; |
| @@ -457,9 +455,7 @@ int vmw_kms_readback(struct vmw_private *dev_priv, · 172 unmodified lines | ||
| 457 | 455 | uint32_t num_clips); |
| 458 | 456 | struct vmw_framebuffer * |
| 459 | 457 | vmw_kms_new_framebuffer(struct vmw_private *dev_priv, |
| 460 | − struct vmw_bo *bo, | |
| 461 | − struct vmw_surface *surface, | |
| 462 | − bool only_2d, | |
| 458 | + struct vmw_user_object *uo, | |
| 463 | 459 | const struct drm_mode_fb_cmd2 *mode_cmd); |
| 464 | 460 | void vmw_guess_mode_timing(struct drm_display_mode *mode); |
| 465 | 461 | void vmw_kms_update_implicit_fb(struct vmw_private *dev_priv); |
| @@ -486,8 +482,7 @@ void vmw_du_plane_reset(struct drm_plane *plane); · 20 unmodified lines | ||
| 486 | 482 | struct drm_plane_state *vmw_du_plane_duplicate_state(struct drm_plane *plane); |
| 487 | 483 | void vmw_du_plane_destroy_state(struct drm_plane *plane, |
| 488 | 484 | struct drm_plane_state *state); |
| 489 | −void vmw_du_plane_unpin_surf(struct vmw_plane_state *vps, | |
| 490 | − bool unreference); | |
| 485 | +void vmw_du_plane_unpin_surf(struct vmw_plane_state *vps); | |
| 491 | 486 | |
| 492 | 487 | int vmw_du_crtc_atomic_check(struct drm_crtc *crtc, |
| 493 | 488 | struct drm_atomic_state *state); |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0 OR MIT |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright 2009-2023 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2009-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -147,8 +148,9 @@ static int vmw_ldu_fb_pin(struct vmw_framebuffer *vfb) · 139 unmodified lines | ||
| 147 | 148 | struct vmw_bo *buf; |
| 148 | 149 | int ret; |
| 149 | 150 | |
| 150 | − buf = vfb->bo ? vmw_framebuffer_to_vfbd(&vfb->base)->buffer : | |
| 151 | − vmw_framebuffer_to_vfbs(&vfb->base)->surface->res.guest_memory_bo; | |
| 151 | + buf = vfb->bo ? | |
| 152 | + vmw_framebuffer_to_vfbd(&vfb->base)->buffer : | |
| 153 | + vmw_user_object_buffer(&vmw_framebuffer_to_vfbs(&vfb->base)->uo); | |
| 152 | 154 | |
| 153 | 155 | if (!buf) |
| 154 | 156 | return 0; |
| @@ -169,8 +171,10 @@ static int vmw_ldu_fb_unpin(struct vmw_framebuffer *vfb) · 14 unmodified lines | ||
| 169 | 171 | struct vmw_private *dev_priv = vmw_priv(vfb->base.dev); |
| 170 | 172 | struct vmw_bo *buf; |
| 171 | 173 | |
| 172 | − buf = vfb->bo ? vmw_framebuffer_to_vfbd(&vfb->base)->buffer : | |
| 173 | − vmw_framebuffer_to_vfbs(&vfb->base)->surface->res.guest_memory_bo; | |
| 174 | + buf = vfb->bo ? | |
| 175 | + vmw_framebuffer_to_vfbd(&vfb->base)->buffer : | |
| 176 | + vmw_user_object_buffer(&vmw_framebuffer_to_vfbs(&vfb->base)->uo); | |
| 177 | + | |
| 174 | 178 | |
| 175 | 179 | if (WARN_ON(!buf)) |
| 176 | 180 | return 0; |
| @@ -92,7 +92,7 @@ static int vmw_overlay_send_put(struct vmw_private *dev_priv, · 91 unmodified lines | ||
| 92 | 92 | { |
| 93 | 93 | struct vmw_escape_video_flush *flush; |
| 94 | 94 | size_t fifo_size; |
| 95 | − bool have_so = (dev_priv->active_display_unit == vmw_du_screen_object); | |
| 95 | + bool have_so = (dev_priv->active_display_unit != vmw_du_legacy); | |
| 96 | 96 | int i, num_items; |
| 97 | 97 | SVGAGuestPtr ptr; |
| 98 | 98 | |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0 OR MIT |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright 2013 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2013-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -31,6 +32,7 @@ · 23 unmodified lines | ||
| 31 | 32 | */ |
| 32 | 33 | |
| 33 | 34 | #include "vmwgfx_drv.h" |
| 35 | +#include "vmwgfx_bo.h" | |
| 34 | 36 | #include "ttm_object.h" |
| 35 | 37 | #include <linux/dma-buf.h> |
| 36 | 38 | |
| @@ -88,13 +90,35 @@ int vmw_prime_handle_to_fd(struct drm_device *dev, · 51 unmodified lines | ||
| 88 | 90 | uint32_t handle, uint32_t flags, |
| 89 | 91 | int *prime_fd) |
| 90 | 92 | { |
| 93 | + struct vmw_private *vmw = vmw_priv(dev); | |
| 91 | 94 | struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile; |
| 95 | + struct vmw_bo *vbo; | |
| 92 | 96 | int ret; |
| 97 | + int surf_handle; | |
| 93 | 98 | |
| 94 | − if (handle > VMWGFX_NUM_MOB) | |
| 99 | + if (handle > VMWGFX_NUM_MOB) { | |
| 95 | 100 | ret = ttm_prime_handle_to_fd(tfile, handle, flags, prime_fd); |
| 96 | − else | |
| 97 | − ret = drm_gem_prime_handle_to_fd(dev, file_priv, handle, flags, prime_fd); | |
| 101 | + } else { | |
| 102 | + ret = vmw_user_bo_lookup(file_priv, handle, &vbo); | |
| 103 | + if (ret) | |
| 104 | + return ret; | |
| 105 | + if (vbo && vbo->is_dumb) { | |
| 106 | + ret = drm_gem_prime_handle_to_fd(dev, file_priv, handle, | |
| 107 | + flags, prime_fd); | |
| 108 | + } else { | |
| 109 | + surf_handle = vmw_lookup_surface_handle_for_buffer(vmw, | |
| 110 | + vbo, | |
| 111 | + handle); | |
| 112 | + if (surf_handle > 0) | |
| 113 | + ret = ttm_prime_handle_to_fd(tfile, surf_handle, | |
| 114 | + flags, prime_fd); | |
| 115 | + else | |
| 116 | + ret = drm_gem_prime_handle_to_fd(dev, file_priv, | |
| 117 | + handle, flags, | |
| 118 | + prime_fd); | |
| 119 | + } | |
| 120 | + vmw_user_bo_unref(&vbo); | |
| 121 | + } | |
| 98 | 122 | |
| 99 | 123 | return ret; |
| 100 | 124 | } |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0 OR MIT |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright 2009-2023 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2009-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -58,6 +59,7 @@ void vmw_resource_mob_attach(struct vmw_resource *res) · 50 unmodified lines | ||
| 58 | 59 | |
| 59 | 60 | rb_link_node(&res->mob_node, parent, new); |
| 60 | 61 | rb_insert_color(&res->mob_node, &gbo->res_tree); |
| 62 | + vmw_bo_del_detached_resource(gbo, res); | |
| 61 | 63 | |
| 62 | 64 | vmw_bo_prio_add(gbo, res->used_prio); |
| 63 | 65 | } |
| @@ -287,28 +289,35 @@ out_bad_resource: · 223 unmodified lines | ||
| 287 | 289 | * |
| 288 | 290 | * The pointer this pointed at by out_surf and out_buf needs to be null. |
| 289 | 291 | */ |
| 290 | −int vmw_user_lookup_handle(struct vmw_private *dev_priv, | |
| 292 | +int vmw_user_object_lookup(struct vmw_private *dev_priv, | |
| 291 | 293 | struct drm_file *filp, |
| 292 | − uint32_t handle, | |
| 293 | − struct vmw_surface **out_surf, | |
| 294 | − struct vmw_bo **out_buf) | |
| 294 | + u32 handle, | |
| 295 | + struct vmw_user_object *uo) | |
| 295 | 296 | { |
| 296 | 297 | struct ttm_object_file *tfile = vmw_fpriv(filp)->tfile; |
| 297 | 298 | struct vmw_resource *res; |
| 298 | 299 | int ret; |
| 299 | 300 | |
| 300 | − BUG_ON(*out_surf || *out_buf); | |
| 301 | + WARN_ON(uo->surface || uo->buffer); | |
| 301 | 302 | |
| 302 | 303 | ret = vmw_user_resource_lookup_handle(dev_priv, tfile, handle, |
| 303 | 304 | user_surface_converter, |
| 304 | 305 | &res); |
| 305 | 306 | if (!ret) { |
| 306 | − *out_surf = vmw_res_to_srf(res); | |
| 307 | + uo->surface = vmw_res_to_srf(res); | |
| 307 | 308 | return 0; |
| 308 | 309 | } |
| 309 | 310 | |
| 310 | − *out_surf = NULL; | |
| 311 | − ret = vmw_user_bo_lookup(filp, handle, out_buf); | |
| 311 | + uo->surface = NULL; | |
| 312 | + ret = vmw_user_bo_lookup(filp, handle, &uo->buffer); | |
| 313 | + if (!ret && !uo->buffer->is_dumb) { | |
| 314 | + uo->surface = vmw_lookup_surface_for_buffer(dev_priv, | |
| 315 | + uo->buffer, | |
| 316 | + handle); | |
| 317 | + if (uo->surface) | |
| 318 | + vmw_user_bo_unref(&uo->buffer); | |
| 319 | + } | |
| 320 | + | |
| 312 | 321 | return ret; |
| 313 | 322 | } |
| 314 | 323 | |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0 OR MIT |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright 2011-2023 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2011-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -240,7 +241,7 @@ static void vmw_sou_crtc_mode_set_nofb(struct drm_crtc *crtc) · 232 unmodified lines | ||
| 240 | 241 | struct vmw_connector_state *vmw_conn_state; |
| 241 | 242 | int x, y; |
| 242 | 243 | |
| 243 | − sou->buffer = vps->bo; | |
| 244 | + sou->buffer = vmw_user_object_buffer(&vps->uo); | |
| 244 | 245 | |
| 245 | 246 | conn_state = sou->base.connector.state; |
| 246 | 247 | vmw_conn_state = vmw_connector_state_to_vcs(conn_state); |
| @@ -376,10 +377,11 @@ vmw_sou_primary_plane_cleanup_fb(struct drm_plane *plane, · 129 unmodified lines | ||
| 376 | 377 | struct vmw_plane_state *vps = vmw_plane_state_to_vps(old_state); |
| 377 | 378 | struct drm_crtc *crtc = plane->state->crtc ? |
| 378 | 379 | plane->state->crtc : old_state->crtc; |
| 380 | + struct vmw_bo *bo = vmw_user_object_buffer(&vps->uo); | |
| 379 | 381 | |
| 380 | − if (vps->bo) | |
| 381 | − vmw_bo_unpin(vmw_priv(crtc->dev), vps->bo, false); | |
| 382 | − vmw_bo_unreference(&vps->bo); | |
| 382 | + if (bo) | |
| 383 | + vmw_bo_unpin(vmw_priv(crtc->dev), bo, false); | |
| 384 | + vmw_user_object_unref(&vps->uo); | |
| 383 | 385 | vps->bo_size = 0; |
| 384 | 386 | |
| 385 | 387 | vmw_du_plane_cleanup_fb(plane, old_state); |
| @@ -411,9 +413,10 @@ vmw_sou_primary_plane_prepare_fb(struct drm_plane *plane, · 25 unmodified lines | ||
| 411 | 413 | .bo_type = ttm_bo_type_device, |
| 412 | 414 | .pin = true |
| 413 | 415 | }; |
| 416 | + struct vmw_bo *bo = NULL; | |
| 414 | 417 | |
| 415 | 418 | if (!new_fb) { |
| 416 | − vmw_bo_unreference(&vps->bo); | |
| 419 | + vmw_user_object_unref(&vps->uo); | |
| 417 | 420 | vps->bo_size = 0; |
| 418 | 421 | |
| 419 | 422 | return 0; |
| @@ -422,17 +425,17 @@ vmw_sou_primary_plane_prepare_fb(struct drm_plane *plane, · 2 unmodified lines | ||
| 422 | 425 | bo_params.size = new_state->crtc_w * new_state->crtc_h * 4; |
| 423 | 426 | dev_priv = vmw_priv(crtc->dev); |
| 424 | 427 | |
| 425 | − if (vps->bo) { | |
| 428 | + bo = vmw_user_object_buffer(&vps->uo); | |
| 429 | + if (bo) { | |
| 426 | 430 | if (vps->bo_size == bo_params.size) { |
| 427 | 431 | /* |
| 428 | 432 | * Note that this might temporarily up the pin-count |
| 429 | 433 | * to 2, until cleanup_fb() is called. |
| 430 | 434 | */ |
| 431 | − return vmw_bo_pin_in_vram(dev_priv, vps->bo, | |
| 432 | − true); | |
| 435 | + return vmw_bo_pin_in_vram(dev_priv, bo, true); | |
| 433 | 436 | } |
| 434 | 437 | |
| 435 | − vmw_bo_unreference(&vps->bo); | |
| 438 | + vmw_user_object_unref(&vps->uo); | |
| 436 | 439 | vps->bo_size = 0; |
| 437 | 440 | } |
| 438 | 441 | |
| @@ -442,7 +445,7 @@ vmw_sou_primary_plane_prepare_fb(struct drm_plane *plane, · 3 unmodified lines | ||
| 442 | 445 | * resume the overlays, this is preferred to failing to alloc. |
| 443 | 446 | */ |
| 444 | 447 | vmw_overlay_pause_all(dev_priv); |
| 445 | − ret = vmw_bo_create(dev_priv, &bo_params, &vps->bo); | |
| 448 | + ret = vmw_gem_object_create(dev_priv, &bo_params, &vps->uo.buffer); | |
| 446 | 449 | vmw_overlay_resume_all(dev_priv); |
| 447 | 450 | if (ret) |
| 448 | 451 | return ret; |
| @@ -453,7 +456,7 @@ vmw_sou_primary_plane_prepare_fb(struct drm_plane *plane, · 4 unmodified lines | ||
| 453 | 456 | * TTM already thinks the buffer is pinned, but make sure the |
| 454 | 457 | * pin_count is upped. |
| 455 | 458 | */ |
| 456 | − return vmw_bo_pin_in_vram(dev_priv, vps->bo, true); | |
| 459 | + return vmw_bo_pin_in_vram(dev_priv, vps->uo.buffer, true); | |
| 457 | 460 | } |
| 458 | 461 | |
| 459 | 462 | static uint32_t vmw_sou_bo_fifo_size(struct vmw_du_update_plane *update, |
| @@ -580,6 +583,7 @@ static uint32_t vmw_sou_surface_pre_clip(struct vmw_du_update_plane *update, · 120 unmodified lines | ||
| 580 | 583 | { |
| 581 | 584 | struct vmw_kms_sou_dirty_cmd *blit = cmd; |
| 582 | 585 | struct vmw_framebuffer_surface *vfbs; |
| 586 | + struct vmw_surface *surf = NULL; | |
| 583 | 587 | |
| 584 | 588 | vfbs = container_of(update->vfb, typeof(*vfbs), base); |
| 585 | 589 | |
| @@ -587,7 +591,8 @@ static uint32_t vmw_sou_surface_pre_clip(struct vmw_du_update_plane *update, · 1 unmodified line | ||
| 587 | 591 | blit->header.size = sizeof(blit->body) + sizeof(SVGASignedRect) * |
| 588 | 592 | num_hits; |
| 589 | 593 | |
| 590 | − blit->body.srcImage.sid = vfbs->surface->res.id; | |
| 594 | + surf = vmw_user_object_surface(&vfbs->uo); | |
| 595 | + blit->body.srcImage.sid = surf->res.id; | |
| 591 | 596 | blit->body.destScreenId = update->du->unit; |
| 592 | 597 | |
| 593 | 598 | /* Update the source and destination bounding box later in post_clip */ |
| @@ -1104,7 +1109,7 @@ int vmw_kms_sou_do_surface_dirty(struct vmw_private *dev_priv, · 510 unmodified lines | ||
| 1104 | 1109 | int ret; |
| 1105 | 1110 | |
| 1106 | 1111 | if (!srf) |
| 1107 | − srf = &vfbs->surface->res; | |
| 1112 | + srf = &vmw_user_object_surface(&vfbs->uo)->res; | |
| 1108 | 1113 | |
| 1109 | 1114 | ret = vmw_validation_add_resource(&val_ctx, srf, 0, VMW_RES_DIRTY_NONE, |
| 1110 | 1115 | NULL, NULL); |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0 OR MIT |
| 2 | 2 | /****************************************************************************** |
| 3 | 3 | * |
| 4 | − * COPYRIGHT (C) 2014-2023 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2014-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -29,6 +30,7 @@ · 21 unmodified lines | ||
| 29 | 30 | #include "vmwgfx_kms.h" |
| 30 | 31 | #include "vmwgfx_vkms.h" |
| 31 | 32 | #include "vmw_surface_cache.h" |
| 33 | +#include <linux/fsnotify.h> | |
| 32 | 34 | |
| 33 | 35 | #include <drm/drm_atomic.h> |
| 34 | 36 | #include <drm/drm_atomic_helper.h> |
| @@ -500,7 +502,7 @@ static void vmw_stdu_bo_cpu_commit(struct vmw_kms_dirty *dirty) · 465 unmodified lines | ||
| 500 | 502 | container_of(dirty->unit, typeof(*stdu), base); |
| 501 | 503 | s32 width, height; |
| 502 | 504 | s32 src_pitch, dst_pitch; |
| 503 | − struct ttm_buffer_object *src_bo, *dst_bo; | |
| 505 | + struct vmw_bo *src_bo, *dst_bo; | |
| 504 | 506 | u32 src_offset, dst_offset; |
| 505 | 507 | struct vmw_diff_cpy diff = VMW_CPU_BLIT_DIFF_INITIALIZER(stdu->cpp); |
| 506 | 508 | |
| @@ -515,11 +517,11 @@ static void vmw_stdu_bo_cpu_commit(struct vmw_kms_dirty *dirty) · 8 unmodified lines | ||
| 515 | 517 | |
| 516 | 518 | /* Assume we are blitting from Guest (bo) to Host (display_srf) */ |
| 517 | 519 | src_pitch = stdu->display_srf->metadata.base_size.width * stdu->cpp; |
| 518 | − src_bo = &stdu->display_srf->res.guest_memory_bo->tbo; | |
| 520 | + src_bo = stdu->display_srf->res.guest_memory_bo; | |
| 519 | 521 | src_offset = ddirty->top * src_pitch + ddirty->left * stdu->cpp; |
| 520 | 522 | |
| 521 | 523 | dst_pitch = ddirty->pitch; |
| 522 | − dst_bo = &ddirty->buf->tbo; | |
| 524 | + dst_bo = ddirty->buf; | |
| 523 | 525 | dst_offset = ddirty->fb_top * dst_pitch + ddirty->fb_left * stdu->cpp; |
| 524 | 526 | |
| 525 | 527 | (void) vmw_bo_cpu_blit(dst_bo, dst_offset, dst_pitch, |
| @@ -735,7 +737,7 @@ int vmw_kms_stdu_surface_dirty(struct vmw_private *dev_priv, · 209 unmodified lines | ||
| 735 | 737 | int ret; |
| 736 | 738 | |
| 737 | 739 | if (!srf) |
| 738 | − srf = &vfbs->surface->res; | |
| 740 | + srf = &vmw_user_object_surface(&vfbs->uo)->res; | |
| 739 | 741 | |
| 740 | 742 | ret = vmw_validation_add_resource(&val_ctx, srf, 0, VMW_RES_DIRTY_NONE, |
| 741 | 743 | NULL, NULL); |
| @@ -746,12 +748,6 @@ int vmw_kms_stdu_surface_dirty(struct vmw_private *dev_priv, · 4 unmodified lines | ||
| 746 | 748 | if (ret) |
| 747 | 749 | goto out_unref; |
| 748 | 750 | |
| 749 | − if (vfbs->is_bo_proxy) { | |
| 750 | − ret = vmw_kms_update_proxy(srf, clips, num_clips, inc); | |
| 751 | − if (ret) | |
| 752 | − goto out_finish; | |
| 753 | − } | |
| 754 | − | |
| 755 | 751 | sdirty.base.fifo_commit = vmw_kms_stdu_surface_fifo_commit; |
| 756 | 752 | sdirty.base.clip = vmw_kms_stdu_surface_clip; |
| 757 | 753 | sdirty.base.fifo_reserve_size = sizeof(struct vmw_stdu_surface_copy) + |
| @@ -765,7 +761,7 @@ int vmw_kms_stdu_surface_dirty(struct vmw_private *dev_priv, · 7 unmodified lines | ||
| 765 | 761 | ret = vmw_kms_helper_dirty(dev_priv, framebuffer, clips, vclips, |
| 766 | 762 | dest_x, dest_y, num_clips, inc, |
| 767 | 763 | &sdirty.base); |
| 768 | −out_finish: | |
| 764 | + | |
| 769 | 765 | vmw_kms_helper_validation_finish(dev_priv, NULL, &val_ctx, out_fence, |
| 770 | 766 | NULL); |
| 771 | 767 | |
| @@ -877,6 +873,32 @@ vmw_stdu_connector_mode_valid(struct drm_connector *connector, · 105 unmodified lines | ||
| 877 | 873 | return MODE_OK; |
| 878 | 874 | } |
| 879 | 875 | |
| 876 | +/* | |
| 877 | + * Trigger a modeset if the X,Y position of the Screen Target changes. | |
| 878 | + * This is needed when multi-mon is cycled. The original Screen Target will have | |
| 879 | + * the same mode but its relative X,Y position in the topology will change. | |
| 880 | + */ | |
| 881 | +static int vmw_stdu_connector_atomic_check(struct drm_connector *conn, | |
| 882 | + struct drm_atomic_state *state) | |
| 883 | +{ | |
| 884 | + struct drm_connector_state *conn_state; | |
| 885 | + struct vmw_screen_target_display_unit *du; | |
| 886 | + struct drm_crtc_state *new_crtc_state; | |
| 887 | + | |
| 888 | + conn_state = drm_atomic_get_connector_state(state, conn); | |
| 889 | + du = vmw_connector_to_stdu(conn); | |
| 890 | + | |
| 891 | + if (!conn_state->crtc) | |
| 892 | + return 0; | |
| 893 | + | |
| 894 | + new_crtc_state = drm_atomic_get_new_crtc_state(state, conn_state->crtc); | |
| 895 | + if (du->base.gui_x != du->base.set_gui_x || | |
| 896 | + du->base.gui_y != du->base.set_gui_y) | |
| 897 | + new_crtc_state->mode_changed = true; | |
| 898 | + | |
| 899 | + return 0; | |
| 900 | +} | |
| 901 | + | |
| 880 | 902 | static const struct drm_connector_funcs vmw_stdu_connector_funcs = { |
| 881 | 903 | .dpms = vmw_du_connector_dpms, |
| 882 | 904 | .detect = vmw_du_connector_detect, |
| @@ -891,7 +913,8 @@ static const struct drm_connector_funcs vmw_stdu_connector_funcs = { · 8 unmodified lines | ||
| 891 | 913 | static const struct |
| 892 | 914 | drm_connector_helper_funcs vmw_stdu_connector_helper_funcs = { |
| 893 | 915 | .get_modes = vmw_connector_get_modes, |
| 894 | − .mode_valid = vmw_stdu_connector_mode_valid | |
| 916 | + .mode_valid = vmw_stdu_connector_mode_valid, | |
| 917 | + .atomic_check = vmw_stdu_connector_atomic_check, | |
| 895 | 918 | }; |
| 896 | 919 | |
| 897 | 920 | |
| @@ -918,9 +941,8 @@ vmw_stdu_primary_plane_cleanup_fb(struct drm_plane *plane, · 20 unmodified lines | ||
| 918 | 941 | { |
| 919 | 942 | struct vmw_plane_state *vps = vmw_plane_state_to_vps(old_state); |
| 920 | 943 | |
| 921 | − if (vps->surf) | |
| 944 | + if (vmw_user_object_surface(&vps->uo)) | |
| 922 | 945 | WARN_ON(!vps->pinned); |
| 923 | − | |
| 924 | 946 | vmw_du_plane_cleanup_fb(plane, old_state); |
| 925 | 947 | |
| 926 | 948 | vps->content_fb_type = SAME_AS_DISPLAY; |
| @@ -928,7 +950,6 @@ vmw_stdu_primary_plane_cleanup_fb(struct drm_plane *plane, · 1 unmodified line | ||
| 928 | 950 | } |
| 929 | 951 | |
| 930 | 952 | |
| 931 | − | |
| 932 | 953 | /** |
| 933 | 954 | * vmw_stdu_primary_plane_prepare_fb - Readies the display surface |
| 934 | 955 | * |
| @@ -952,13 +973,15 @@ vmw_stdu_primary_plane_prepare_fb(struct drm_plane *plane, · 17 unmodified lines | ||
| 952 | 973 | enum stdu_content_type new_content_type; |
| 953 | 974 | struct vmw_framebuffer_surface *new_vfbs; |
| 954 | 975 | uint32_t hdisplay = new_state->crtc_w, vdisplay = new_state->crtc_h; |
| 976 | + struct drm_plane_state *old_state = plane->state; | |
| 977 | + struct drm_rect rect; | |
| 955 | 978 | int ret; |
| 956 | 979 | |
| 957 | 980 | /* No FB to prepare */ |
| 958 | 981 | if (!new_fb) { |
| 959 | − if (vps->surf) { | |
| 982 | + if (vmw_user_object_surface(&vps->uo)) { | |
| 960 | 983 | WARN_ON(vps->pinned != 0); |
| 961 | − vmw_surface_unreference(&vps->surf); | |
| 984 | + vmw_user_object_unref(&vps->uo); | |
| 962 | 985 | } |
| 963 | 986 | |
| 964 | 987 | return 0; |
| @@ -968,8 +991,8 @@ vmw_stdu_primary_plane_prepare_fb(struct drm_plane *plane, · 3 unmodified lines | ||
| 968 | 991 | new_vfbs = (vfb->bo) ? NULL : vmw_framebuffer_to_vfbs(new_fb); |
| 969 | 992 | |
| 970 | 993 | if (new_vfbs && |
| 971 | − new_vfbs->surface->metadata.base_size.width == hdisplay && | |
| 972 | − new_vfbs->surface->metadata.base_size.height == vdisplay) | |
| 994 | + vmw_user_object_surface(&new_vfbs->uo)->metadata.base_size.width == hdisplay && | |
| 995 | + vmw_user_object_surface(&new_vfbs->uo)->metadata.base_size.height == vdisplay) | |
| 973 | 996 | new_content_type = SAME_AS_DISPLAY; |
| 974 | 997 | else if (vfb->bo) |
| 975 | 998 | new_content_type = SEPARATE_BO; |
| @@ -1007,29 +1030,29 @@ vmw_stdu_primary_plane_prepare_fb(struct drm_plane *plane, · 31 unmodified lines | ||
| 1007 | 1030 | metadata.num_sizes = 1; |
| 1008 | 1031 | metadata.scanout = true; |
| 1009 | 1032 | } else { |
| 1010 | − metadata = new_vfbs->surface->metadata; | |
| 1033 | + metadata = vmw_user_object_surface(&new_vfbs->uo)->metadata; | |
| 1011 | 1034 | } |
| 1012 | 1035 | |
| 1013 | 1036 | metadata.base_size.width = hdisplay; |
| 1014 | 1037 | metadata.base_size.height = vdisplay; |
| 1015 | 1038 | metadata.base_size.depth = 1; |
| 1016 | 1039 | |
| 1017 | − if (vps->surf) { | |
| 1040 | + if (vmw_user_object_surface(&vps->uo)) { | |
| 1018 | 1041 | struct drm_vmw_size cur_base_size = |
| 1019 | − vps->surf->metadata.base_size; | |
| 1042 | + vmw_user_object_surface(&vps->uo)->metadata.base_size; | |
| 1020 | 1043 | |
| 1021 | 1044 | if (cur_base_size.width != metadata.base_size.width || |
| 1022 | 1045 | cur_base_size.height != metadata.base_size.height || |
| 1023 | − vps->surf->metadata.format != metadata.format) { | |
| 1046 | + vmw_user_object_surface(&vps->uo)->metadata.format != metadata.format) { | |
| 1024 | 1047 | WARN_ON(vps->pinned != 0); |
| 1025 | − vmw_surface_unreference(&vps->surf); | |
| 1048 | + vmw_user_object_unref(&vps->uo); | |
| 1026 | 1049 | } |
| 1027 | 1050 | |
| 1028 | 1051 | } |
| 1029 | 1052 | |
| 1030 | − if (!vps->surf) { | |
| 1053 | + if (!vmw_user_object_surface(&vps->uo)) { | |
| 1031 | 1054 | ret = vmw_gb_surface_define(dev_priv, &metadata, |
| 1032 | − &vps->surf); | |
| 1055 | + &vps->uo.surface); | |
| 1033 | 1056 | if (ret != 0) { |
| 1034 | 1057 | DRM_ERROR("Couldn't allocate STDU surface.\n"); |
| 1035 | 1058 | return ret; |
| @@ -1042,18 +1065,19 @@ vmw_stdu_primary_plane_prepare_fb(struct drm_plane *plane, · 6 unmodified lines | ||
| 1042 | 1065 | * The only time we add a reference in prepare_fb is if the |
| 1043 | 1066 | * state object doesn't have a reference to begin with |
| 1044 | 1067 | */ |
| 1045 | − if (vps->surf) { | |
| 1068 | + if (vmw_user_object_surface(&vps->uo)) { | |
| 1046 | 1069 | WARN_ON(vps->pinned != 0); |
| 1047 | − vmw_surface_unreference(&vps->surf); | |
| 1070 | + vmw_user_object_unref(&vps->uo); | |
| 1048 | 1071 | } |
| 1049 | 1072 | |
| 1050 | − vps->surf = vmw_surface_reference(new_vfbs->surface); | |
| 1073 | + memcpy(&vps->uo, &new_vfbs->uo, sizeof(vps->uo)); | |
| 1074 | + vmw_user_object_ref(&vps->uo); | |
| 1051 | 1075 | } |
| 1052 | 1076 | |
| 1053 | − if (vps->surf) { | |
| 1077 | + if (vmw_user_object_surface(&vps->uo)) { | |
| 1054 | 1078 | |
| 1055 | 1079 | /* Pin new surface before flipping */ |
| 1056 | − ret = vmw_resource_pin(&vps->surf->res, false); | |
| 1080 | + ret = vmw_resource_pin(&vmw_user_object_surface(&vps->uo)->res, false); | |
| 1057 | 1081 | if (ret) |
| 1058 | 1082 | goto out_srf_unref; |
| 1059 | 1083 | |
| @@ -1062,6 +1086,34 @@ vmw_stdu_primary_plane_prepare_fb(struct drm_plane *plane, · 2 unmodified lines | ||
| 1062 | 1086 | |
| 1063 | 1087 | vps->content_fb_type = new_content_type; |
| 1064 | 1088 | |
| 1089 | + /* | |
| 1090 | + * The drm fb code will do blit's via the vmap interface, which doesn't | |
| 1091 | + * trigger vmw_bo page dirty tracking due to being kernel side (and thus | |
| 1092 | + * doesn't require mmap'ing) so we have to update the surface's dirty | |
| 1093 | + * regions by hand but we want to be careful to not overwrite the | |
| 1094 | + * resource if it has been written to by the gpu (res_dirty). | |
| 1095 | + */ | |
| 1096 | + if (vps->uo.buffer && vps->uo.buffer->is_dumb) { | |
| 1097 | + struct vmw_surface *surf = vmw_user_object_surface(&vps->uo); | |
| 1098 | + struct vmw_resource *res = &surf->res; | |
| 1099 | + | |
| 1100 | + if (!res->res_dirty && drm_atomic_helper_damage_merged(old_state, | |
| 1101 | + new_state, | |
| 1102 | + &rect)) { | |
| 1103 | + /* | |
| 1104 | + * At some point it might be useful to actually translate | |
| 1105 | + * (rect.x1, rect.y1) => start, and (rect.x2, rect.y2) => end, | |
| 1106 | + * but currently the fb code will just report the entire fb | |
| 1107 | + * dirty so in practice it doesn't matter. | |
| 1108 | + */ | |
| 1109 | + pgoff_t start = res->guest_memory_offset >> PAGE_SHIFT; | |
| 1110 | + pgoff_t end = __KERNEL_DIV_ROUND_UP(res->guest_memory_offset + | |
| 1111 | + res->guest_memory_size, | |
| 1112 | + PAGE_SIZE); | |
| 1113 | + vmw_resource_dirty_update(res, start, end); | |
| 1114 | + } | |
| 1115 | + } | |
| 1116 | + | |
| 1065 | 1117 | /* |
| 1066 | 1118 | * This should only happen if the buffer object is too large to create a |
| 1067 | 1119 | * proxy surface for. |
| @@ -1072,7 +1124,7 @@ vmw_stdu_primary_plane_prepare_fb(struct drm_plane *plane, · 4 unmodified lines | ||
| 1072 | 1124 | return 0; |
| 1073 | 1125 | |
| 1074 | 1126 | out_srf_unref: |
| 1075 | − vmw_surface_unreference(&vps->surf); | |
| 1127 | + vmw_user_object_unref(&vps->uo); | |
| 1076 | 1128 | return ret; |
| 1077 | 1129 | } |
| 1078 | 1130 | |
| @@ -1118,7 +1170,7 @@ vmw_stdu_bo_populate_update_cpu(struct vmw_du_update_plane *update, void *cmd, · 39 unmodified lines | ||
| 1118 | 1170 | struct vmw_diff_cpy diff = VMW_CPU_BLIT_DIFF_INITIALIZER(0); |
| 1119 | 1171 | struct vmw_stdu_update_gb_image *cmd_img = cmd; |
| 1120 | 1172 | struct vmw_stdu_update *cmd_update; |
| 1121 | − struct ttm_buffer_object *src_bo, *dst_bo; | |
| 1173 | + struct vmw_bo *src_bo, *dst_bo; | |
| 1122 | 1174 | u32 src_offset, dst_offset; |
| 1123 | 1175 | s32 src_pitch, dst_pitch; |
| 1124 | 1176 | s32 width, height; |
| @@ -1132,11 +1184,11 @@ vmw_stdu_bo_populate_update_cpu(struct vmw_du_update_plane *update, void *cmd, · 7 unmodified lines | ||
| 1132 | 1184 | |
| 1133 | 1185 | diff.cpp = stdu->cpp; |
| 1134 | 1186 | |
| 1135 | − dst_bo = &stdu->display_srf->res.guest_memory_bo->tbo; | |
| 1187 | + dst_bo = stdu->display_srf->res.guest_memory_bo; | |
| 1136 | 1188 | dst_pitch = stdu->display_srf->metadata.base_size.width * stdu->cpp; |
| 1137 | 1189 | dst_offset = bb->y1 * dst_pitch + bb->x1 * stdu->cpp; |
| 1138 | 1190 | |
| 1139 | − src_bo = &vfbbo->buffer->tbo; | |
| 1191 | + src_bo = vfbbo->buffer; | |
| 1140 | 1192 | src_pitch = update->vfb->base.pitches[0]; |
| 1141 | 1193 | src_offset = bo_update->fb_top * src_pitch + bo_update->fb_left * |
| 1142 | 1194 | stdu->cpp; |
| @@ -1214,14 +1266,8 @@ static uint32_t · 71 unmodified lines | ||
| 1214 | 1266 | vmw_stdu_surface_fifo_size_same_display(struct vmw_du_update_plane *update, |
| 1215 | 1267 | uint32_t num_hits) |
| 1216 | 1268 | { |
| 1217 | − struct vmw_framebuffer_surface *vfbs; | |
| 1218 | 1269 | uint32_t size = 0; |
| 1219 | 1270 | |
| 1220 | − vfbs = container_of(update->vfb, typeof(*vfbs), base); | |
| 1221 | − | |
| 1222 | − if (vfbs->is_bo_proxy) | |
| 1223 | − size += sizeof(struct vmw_stdu_update_gb_image) * num_hits; | |
| 1224 | − | |
| 1225 | 1271 | size += sizeof(struct vmw_stdu_update); |
| 1226 | 1272 | |
| 1227 | 1273 | return size; |
| @@ -1230,61 +1276,14 @@ vmw_stdu_surface_fifo_size_same_display(struct vmw_du_update_plane *update, · 2 unmodified lines | ||
| 1230 | 1276 | static uint32_t vmw_stdu_surface_fifo_size(struct vmw_du_update_plane *update, |
| 1231 | 1277 | uint32_t num_hits) |
| 1232 | 1278 | { |
| 1233 | − struct vmw_framebuffer_surface *vfbs; | |
| 1234 | 1279 | uint32_t size = 0; |
| 1235 | 1280 | |
| 1236 | − vfbs = container_of(update->vfb, typeof(*vfbs), base); | |
| 1237 | − | |
| 1238 | − if (vfbs->is_bo_proxy) | |
| 1239 | − size += sizeof(struct vmw_stdu_update_gb_image) * num_hits; | |
| 1240 | − | |
| 1241 | 1281 | size += sizeof(struct vmw_stdu_surface_copy) + sizeof(SVGA3dCopyBox) * |
| 1242 | 1282 | num_hits + sizeof(struct vmw_stdu_update); |
| 1243 | 1283 | |
| 1244 | 1284 | return size; |
| 1245 | 1285 | } |
| 1246 | 1286 | |
| 1247 | −static uint32_t | |
| 1248 | −vmw_stdu_surface_update_proxy(struct vmw_du_update_plane *update, void *cmd) | |
| 1249 | −{ | |
| 1250 | − struct vmw_framebuffer_surface *vfbs; | |
| 1251 | − struct drm_plane_state *state = update->plane->state; | |
| 1252 | − struct drm_plane_state *old_state = update->old_state; | |
| 1253 | − struct vmw_stdu_update_gb_image *cmd_update = cmd; | |
| 1254 | − struct drm_atomic_helper_damage_iter iter; | |
| 1255 | − struct drm_rect clip; | |
| 1256 | − uint32_t copy_size = 0; | |
| 1257 | − | |
| 1258 | − vfbs = container_of(update->vfb, typeof(*vfbs), base); | |
| 1259 | − | |
| 1260 | − /* | |
| 1261 | − * proxy surface is special where a buffer object type fb is wrapped | |
| 1262 | − * in a surface and need an update gb image command to sync with device. | |
| 1263 | − */ | |
| 1264 | − drm_atomic_helper_damage_iter_init(&iter, old_state, state); | |
| 1265 | − drm_atomic_for_each_plane_damage(&iter, &clip) { | |
| 1266 | − SVGA3dBox *box = &cmd_update->body.box; | |
| 1267 | − | |
| 1268 | − cmd_update->header.id = SVGA_3D_CMD_UPDATE_GB_IMAGE; | |
| 1269 | − cmd_update->header.size = sizeof(cmd_update->body); | |
| 1270 | − cmd_update->body.image.sid = vfbs->surface->res.id; | |
| 1271 | − cmd_update->body.image.face = 0; | |
| 1272 | − cmd_update->body.image.mipmap = 0; | |
| 1273 | − | |
| 1274 | − box->x = clip.x1; | |
| 1275 | − box->y = clip.y1; | |
| 1276 | − box->z = 0; | |
| 1277 | − box->w = drm_rect_width(&clip); | |
| 1278 | − box->h = drm_rect_height(&clip); | |
| 1279 | − box->d = 1; | |
| 1280 | − | |
| 1281 | − copy_size += sizeof(*cmd_update); | |
| 1282 | − cmd_update++; | |
| 1283 | − } | |
| 1284 | − | |
| 1285 | − return copy_size; | |
| 1286 | −} | |
| 1287 | − | |
| 1288 | 1287 | static uint32_t |
| 1289 | 1288 | vmw_stdu_surface_populate_copy(struct vmw_du_update_plane *update, void *cmd, |
| 1290 | 1289 | uint32_t num_hits) |
| @@ -1299,7 +1298,7 @@ vmw_stdu_surface_populate_copy(struct vmw_du_update_plane *update, void *cmd, · 8 unmodified lines | ||
| 1299 | 1298 | cmd_copy->header.id = SVGA_3D_CMD_SURFACE_COPY; |
| 1300 | 1299 | cmd_copy->header.size = sizeof(cmd_copy->body) + sizeof(SVGA3dCopyBox) * |
| 1301 | 1300 | num_hits; |
| 1302 | − cmd_copy->body.src.sid = vfbs->surface->res.id; | |
| 1301 | + cmd_copy->body.src.sid = vmw_user_object_surface(&vfbs->uo)->res.id; | |
| 1303 | 1302 | cmd_copy->body.dest.sid = stdu->display_srf->res.id; |
| 1304 | 1303 | |
| 1305 | 1304 | return sizeof(*cmd_copy); |
| @@ -1370,10 +1369,7 @@ static int vmw_stdu_plane_update_surface(struct vmw_private *dev_priv, · 64 unmodified lines | ||
| 1370 | 1369 | srf_update.mutex = &dev_priv->cmdbuf_mutex; |
| 1371 | 1370 | srf_update.intr = true; |
| 1372 | 1371 | |
| 1373 | − if (vfbs->is_bo_proxy) | |
| 1374 | − srf_update.post_prepare = vmw_stdu_surface_update_proxy; | |
| 1375 | − | |
| 1376 | − if (vfbs->surface->res.id != stdu->display_srf->res.id) { | |
| 1372 | + if (vmw_user_object_surface(&vfbs->uo)->res.id != stdu->display_srf->res.id) { | |
| 1377 | 1373 | srf_update.calc_fifo_size = vmw_stdu_surface_fifo_size; |
| 1378 | 1374 | srf_update.pre_clip = vmw_stdu_surface_populate_copy; |
| 1379 | 1375 | srf_update.clip = vmw_stdu_surface_populate_clip; |
| @@ -1417,7 +1413,7 @@ vmw_stdu_primary_plane_atomic_update(struct drm_plane *plane, · 37 unmodified lines | ||
| 1417 | 1413 | stdu = vmw_crtc_to_stdu(crtc); |
| 1418 | 1414 | dev_priv = vmw_priv(crtc->dev); |
| 1419 | 1415 | |
| 1420 | − stdu->display_srf = vps->surf; | |
| 1416 | + stdu->display_srf = vmw_user_object_surface(&vps->uo); | |
| 1421 | 1417 | stdu->content_fb_type = vps->content_fb_type; |
| 1422 | 1418 | stdu->cpp = vps->cpp; |
| 1423 | 1419 | |
| @@ -1,7 +1,8 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0 OR MIT |
| 2 | 2 | /************************************************************************** |
| 3 | 3 | * |
| 4 | − * Copyright 2009-2023 VMware, Inc., Palo Alto, CA., USA | |
| 4 | + * Copyright (c) 2009-2024 Broadcom. All Rights Reserved. The term | |
| 5 | + * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. | |
| 5 | 6 | * |
| 6 | 7 | * Permission is hereby granted, free of charge, to any person obtaining a |
| 7 | 8 | * copy of this software and associated documentation files (the |
| @@ -36,9 +37,6 @@ · 28 unmodified lines | ||
| 36 | 37 | #include <drm/ttm/ttm_placement.h> |
| 37 | 38 | |
| 38 | 39 | #define SVGA3D_FLAGS_64(upper32, lower32) (((uint64_t)upper32 << 32) | lower32) |
| 39 | −#define SVGA3D_FLAGS_UPPER_32(svga3d_flags) (svga3d_flags >> 32) | |
| 40 | −#define SVGA3D_FLAGS_LOWER_32(svga3d_flags) \ | |
| 41 | − (svga3d_flags & ((uint64_t)U32_MAX)) | |
| 42 | 40 | |
| 43 | 41 | /** |
| 44 | 42 | * struct vmw_user_surface - User-space visible surface resource |
| @@ -686,6 +684,14 @@ static void vmw_user_surface_base_release(struct ttm_base_object **p_base) · 641 unmodified lines | ||
| 686 | 684 | struct vmw_resource *res = &user_srf->srf.res; |
| 687 | 685 | |
| 688 | 686 | *p_base = NULL; |
| 687 | + | |
| 688 | + /* | |
| 689 | + * Dumb buffers own the resource and they'll unref the | |
| 690 | + * resource themselves | |
| 691 | + */ | |
| 692 | + if (res && res->guest_memory_bo && res->guest_memory_bo->is_dumb) | |
| 693 | + return; | |
| 694 | + | |
| 689 | 695 | vmw_resource_unreference(&res); |
| 690 | 696 | } |
| 691 | 697 | |
| @@ -812,7 +818,8 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data, · 120 unmodified lines | ||
| 812 | 818 | } |
| 813 | 819 | } |
| 814 | 820 | res->guest_memory_size = cur_bo_offset; |
| 815 | − if (metadata->scanout && | |
| 821 | + if (!file_priv->atomic && | |
| 822 | + metadata->scanout && | |
| 816 | 823 | metadata->num_sizes == 1 && |
| 817 | 824 | metadata->sizes[0].width == VMW_CURSOR_SNOOP_WIDTH && |
| 818 | 825 | metadata->sizes[0].height == VMW_CURSOR_SNOOP_HEIGHT && |
| @@ -864,6 +871,7 @@ int vmw_surface_define_ioctl(struct drm_device *dev, void *data, · 45 unmodified lines | ||
| 864 | 871 | vmw_resource_unreference(&res); |
| 865 | 872 | goto out_unlock; |
| 866 | 873 | } |
| 874 | + vmw_bo_add_detached_resource(res->guest_memory_bo, res); | |
| 867 | 875 | } |
| 868 | 876 | |
| 869 | 877 | tmp = vmw_resource_reference(&srf->res); |
| @@ -892,6 +900,113 @@ out_unlock: · 22 unmodified lines | ||
| 892 | 900 | return ret; |
| 893 | 901 | } |
| 894 | 902 | |
| 903 | +static struct vmw_user_surface * | |
| 904 | +vmw_lookup_user_surface_for_buffer(struct vmw_private *vmw, struct vmw_bo *bo, | |
| 905 | + u32 handle) | |
| 906 | +{ | |
| 907 | + struct vmw_user_surface *user_srf = NULL; | |
| 908 | + struct vmw_surface *surf; | |
| 909 | + struct ttm_base_object *base; | |
| 910 | + | |
| 911 | + surf = vmw_bo_surface(bo); | |
| 912 | + if (surf) { | |
| 913 | + rcu_read_lock(); | |
| 914 | + user_srf = container_of(surf, struct vmw_user_surface, srf); | |
| 915 | + base = &user_srf->prime.base; | |
| 916 | + if (base && !kref_get_unless_zero(&base->refcount)) { | |
| 917 | + drm_dbg_driver(&vmw->drm, | |
| 918 | + "%s: referencing a stale surface handle %d\n", | |
| 919 | + __func__, handle); | |
| 920 | + base = NULL; | |
| 921 | + user_srf = NULL; | |
| 922 | + } | |
| 923 | + rcu_read_unlock(); | |
| 924 | + } | |
| 925 | + | |
| 926 | + return user_srf; | |
| 927 | +} | |
| 928 | + | |
| 929 | +struct vmw_surface *vmw_lookup_surface_for_buffer(struct vmw_private *vmw, | |
| 930 | + struct vmw_bo *bo, | |
| 931 | + u32 handle) | |
| 932 | +{ | |
| 933 | + struct vmw_user_surface *user_srf = | |
| 934 | + vmw_lookup_user_surface_for_buffer(vmw, bo, handle); | |
| 935 | + struct vmw_surface *surf = NULL; | |
| 936 | + struct ttm_base_object *base; | |
| 937 | + | |
| 938 | + if (user_srf) { | |
| 939 | + surf = vmw_surface_reference(&user_srf->srf); | |
| 940 | + base = &user_srf->prime.base; | |
| 941 | + ttm_base_object_unref(&base); | |
| 942 | + } | |
| 943 | + return surf; | |
| 944 | +} | |
| 945 | + | |
| 946 | +u32 vmw_lookup_surface_handle_for_buffer(struct vmw_private *vmw, | |
| 947 | + struct vmw_bo *bo, | |
| 948 | + u32 handle) | |
| 949 | +{ | |
| 950 | + struct vmw_user_surface *user_srf = | |
| 951 | + vmw_lookup_user_surface_for_buffer(vmw, bo, handle); | |
| 952 | + int surf_handle = 0; | |
| 953 | + struct ttm_base_object *base; | |
| 954 | + | |
| 955 | + if (user_srf) { | |
| 956 | + base = &user_srf->prime.base; | |
| 957 | + surf_handle = (u32)base->handle; | |
| 958 | + ttm_base_object_unref(&base); | |
| 959 | + } | |
| 960 | + return surf_handle; | |
| 961 | +} | |
| 962 | + | |
| 963 | +static int vmw_buffer_prime_to_surface_base(struct vmw_private *dev_priv, | |
| 964 | + struct drm_file *file_priv, | |
| 965 | + u32 fd, u32 *handle, | |
| 966 | + struct ttm_base_object **base_p) | |
| 967 | +{ | |
| 968 | + struct ttm_base_object *base; | |
| 969 | + struct vmw_bo *bo; | |
| 970 | + struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile; | |
| 971 | + struct vmw_user_surface *user_srf; | |
| 972 | + int ret; | |
| 973 | + | |
| 974 | + ret = drm_gem_prime_fd_to_handle(&dev_priv->drm, file_priv, fd, handle); | |
| 975 | + if (ret) { | |
| 976 | + drm_warn(&dev_priv->drm, | |
| 977 | + "Wasn't able to find user buffer for fd = %u.\n", fd); | |
| 978 | + return ret; | |
| 979 | + } | |
| 980 | + | |
| 981 | + ret = vmw_user_bo_lookup(file_priv, *handle, &bo); | |
| 982 | + if (ret) { | |
| 983 | + drm_warn(&dev_priv->drm, | |
| 984 | + "Wasn't able to lookup user buffer for handle = %u.\n", *handle); | |
| 985 | + return ret; | |
| 986 | + } | |
| 987 | + | |
| 988 | + user_srf = vmw_lookup_user_surface_for_buffer(dev_priv, bo, *handle); | |
| 989 | + if (WARN_ON(!user_srf)) { | |
| 990 | + drm_warn(&dev_priv->drm, | |
| 991 | + "User surface fd %d (handle %d) is null.\n", fd, *handle); | |
| 992 | + ret = -EINVAL; | |
| 993 | + goto out; | |
| 994 | + } | |
| 995 | + | |
| 996 | + base = &user_srf->prime.base; | |
| 997 | + ret = ttm_ref_object_add(tfile, base, NULL, false); | |
| 998 | + if (ret) { | |
| 999 | + drm_warn(&dev_priv->drm, | |
| 1000 | + "Couldn't add an object ref for the buffer (%d).\n", *handle); | |
| 1001 | + goto out; | |
| 1002 | + } | |
| 1003 | + | |
| 1004 | + *base_p = base; | |
| 1005 | +out: | |
| 1006 | + vmw_user_bo_unref(&bo); | |
| 1007 | + | |
| 1008 | + return ret; | |
| 1009 | +} | |
| 895 | 1010 | |
| 896 | 1011 | static int |
| 897 | 1012 | vmw_surface_handle_reference(struct vmw_private *dev_priv, |
| @@ -901,15 +1016,19 @@ vmw_surface_handle_reference(struct vmw_private *dev_priv, · 3 unmodified lines | ||
| 901 | 1016 | struct ttm_base_object **base_p) |
| 902 | 1017 | { |
| 903 | 1018 | struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile; |
| 904 | − struct vmw_user_surface *user_srf; | |
| 1019 | + struct vmw_user_surface *user_srf = NULL; | |
| 905 | 1020 | uint32_t handle; |
| 906 | 1021 | struct ttm_base_object *base; |
| 907 | 1022 | int ret; |
| 908 | 1023 | |
| 909 | 1024 | if (handle_type == DRM_VMW_HANDLE_PRIME) { |
| 910 | 1025 | ret = ttm_prime_fd_to_handle(tfile, u_handle, &handle); |
| 911 | − if (unlikely(ret != 0)) | |
| 912 | − return ret; | |
| 1026 | + if (ret) | |
| 1027 | + return vmw_buffer_prime_to_surface_base(dev_priv, | |
| 1028 | + file_priv, | |
| 1029 | + u_handle, | |
| 1030 | + &handle, | |
| 1031 | + base_p); | |
| 913 | 1032 | } else { |
| 914 | 1033 | handle = u_handle; |
| 915 | 1034 | } |
| @@ -1503,7 +1622,12 @@ vmw_gb_surface_define_internal(struct drm_device *dev, · 587 unmodified lines | ||
| 1503 | 1622 | ret = vmw_user_bo_lookup(file_priv, req->base.buffer_handle, |
| 1504 | 1623 | &res->guest_memory_bo); |
| 1505 | 1624 | if (ret == 0) { |
| 1506 | − if (res->guest_memory_bo->tbo.base.size < res->guest_memory_size) { | |
| 1625 | + if (res->guest_memory_bo->is_dumb) { | |
| 1626 | + VMW_DEBUG_USER("Can't backup surface with a dumb buffer.\n"); | |
| 1627 | + vmw_user_bo_unref(&res->guest_memory_bo); | |
| 1628 | + ret = -EINVAL; | |
| 1629 | + goto out_unlock; | |
| 1630 | + } else if (res->guest_memory_bo->tbo.base.size < res->guest_memory_size) { | |
| 1507 | 1631 | VMW_DEBUG_USER("Surface backup buffer too small.\n"); |
| 1508 | 1632 | vmw_user_bo_unref(&res->guest_memory_bo); |
| 1509 | 1633 | ret = -EINVAL; |
| @@ -1560,6 +1684,7 @@ vmw_gb_surface_define_internal(struct drm_device *dev, · 50 unmodified lines | ||
| 1560 | 1684 | rep->handle = user_srf->prime.base.handle; |
| 1561 | 1685 | rep->backup_size = res->guest_memory_size; |
| 1562 | 1686 | if (res->guest_memory_bo) { |
| 1687 | + vmw_bo_add_detached_resource(res->guest_memory_bo, res); | |
| 1563 | 1688 | rep->buffer_map_handle = |
| 1564 | 1689 | drm_vma_node_offset_addr(&res->guest_memory_bo->tbo.base.vma_node); |
| 1565 | 1690 | rep->buffer_size = res->guest_memory_bo->tbo.base.size; |
| @@ -2100,3 +2225,142 @@ int vmw_gb_surface_define(struct vmw_private *dev_priv, · 534 unmodified lines | ||
| 2100 | 2225 | out_unlock: |
| 2101 | 2226 | return ret; |
| 2102 | 2227 | } |
| 2228 | + | |
| 2229 | +static SVGA3dSurfaceFormat vmw_format_bpp_to_svga(struct vmw_private *vmw, | |
| 2230 | + int bpp) | |
| 2231 | +{ | |
| 2232 | + switch (bpp) { | |
| 2233 | + case 8: /* DRM_FORMAT_C8 */ | |
| 2234 | + return SVGA3D_P8; | |
| 2235 | + case 16: /* DRM_FORMAT_RGB565 */ | |
| 2236 | + return SVGA3D_R5G6B5; | |
| 2237 | + case 32: /* DRM_FORMAT_XRGB8888 */ | |
| 2238 | + if (has_sm4_context(vmw)) | |
| 2239 | + return SVGA3D_B8G8R8X8_UNORM; | |
| 2240 | + return SVGA3D_X8R8G8B8; | |
| 2241 | + default: | |
| 2242 | + drm_warn(&vmw->drm, "Unsupported format bpp: %d\n", bpp); | |
| 2243 | + return SVGA3D_X8R8G8B8; | |
| 2244 | + } | |
| 2245 | +} | |
| 2246 | + | |
| 2247 | +/** | |
| 2248 | + * vmw_dumb_create - Create a dumb kms buffer | |
| 2249 | + * | |
| 2250 | + * @file_priv: Pointer to a struct drm_file identifying the caller. | |
| 2251 | + * @dev: Pointer to the drm device. | |
| 2252 | + * @args: Pointer to a struct drm_mode_create_dumb structure | |
| 2253 | + * Return: Zero on success, negative error code on failure. | |
| 2254 | + * | |
| 2255 | + * This is a driver callback for the core drm create_dumb functionality. | |
| 2256 | + * Note that this is very similar to the vmw_bo_alloc ioctl, except | |
| 2257 | + * that the arguments have a different format. | |
| 2258 | + */ | |
| 2259 | +int vmw_dumb_create(struct drm_file *file_priv, | |
| 2260 | + struct drm_device *dev, | |
| 2261 | + struct drm_mode_create_dumb *args) | |
| 2262 | +{ | |
| 2263 | + struct vmw_private *dev_priv = vmw_priv(dev); | |
| 2264 | + struct ttm_object_file *tfile = vmw_fpriv(file_priv)->tfile; | |
| 2265 | + struct vmw_bo *vbo = NULL; | |
| 2266 | + struct vmw_resource *res = NULL; | |
| 2267 | + union drm_vmw_gb_surface_create_ext_arg arg = { 0 }; | |
| 2268 | + struct drm_vmw_gb_surface_create_ext_req *req = &arg.req; | |
| 2269 | + int ret; | |
| 2270 | + struct drm_vmw_size drm_size = { | |
| 2271 | + .width = args->width, | |
| 2272 | + .height = args->height, | |
| 2273 | + .depth = 1, | |
| 2274 | + }; | |
| 2275 | + SVGA3dSurfaceFormat format = vmw_format_bpp_to_svga(dev_priv, args->bpp); | |
| 2276 | + const struct SVGA3dSurfaceDesc *desc = vmw_surface_get_desc(format); | |
| 2277 | + SVGA3dSurfaceAllFlags flags = SVGA3D_SURFACE_HINT_TEXTURE | | |
| 2278 | + SVGA3D_SURFACE_HINT_RENDERTARGET | | |
| 2279 | + SVGA3D_SURFACE_SCREENTARGET | | |
| 2280 | + SVGA3D_SURFACE_BIND_SHADER_RESOURCE | | |
| 2281 | + SVGA3D_SURFACE_BIND_RENDER_TARGET; | |
| 2282 | + | |
| 2283 | + /* | |
| 2284 | + * Without mob support we're just going to use raw memory buffer | |
| 2285 | + * because we wouldn't be able to support full surface coherency | |
| 2286 | + * without mobs. There also no reason to support surface coherency | |
| 2287 | + * without 3d (i.e. gpu usage on the host) because then all the | |
| 2288 | + * contents is going to be rendered guest side. | |
| 2289 | + */ | |
| 2290 | + if (!dev_priv->has_mob || !vmw_supports_3d(dev_priv)) { | |
| 2291 | + int cpp = DIV_ROUND_UP(args->bpp, 8); | |
| 2292 | + | |
| 2293 | + switch (cpp) { | |
| 2294 | + case 1: /* DRM_FORMAT_C8 */ | |
| 2295 | + case 2: /* DRM_FORMAT_RGB565 */ | |
| 2296 | + case 4: /* DRM_FORMAT_XRGB8888 */ | |
| 2297 | + break; | |
| 2298 | + default: | |
| 2299 | + /* | |
| 2300 | + * Dumb buffers don't allow anything else. | |
| 2301 | + * This is tested via IGT's dumb_buffers | |
| 2302 | + */ | |
| 2303 | + return -EINVAL; | |
| 2304 | + } | |
| 2305 | + | |
| 2306 | + args->pitch = args->width * cpp; | |
| 2307 | + args->size = ALIGN(args->pitch * args->height, PAGE_SIZE); | |
| 2308 | + | |
| 2309 | + ret = vmw_gem_object_create_with_handle(dev_priv, file_priv, | |
| 2310 | + args->size, &args->handle, | |
| 2311 | + &vbo); | |
| 2312 | + /* drop reference from allocate - handle holds it now */ | |
| 2313 | + drm_gem_object_put(&vbo->tbo.base); | |
| 2314 | + return ret; | |
| 2315 | + } | |
| 2316 | + | |
| 2317 | + req->version = drm_vmw_gb_surface_v1; | |
| 2318 | + req->multisample_pattern = SVGA3D_MS_PATTERN_NONE; | |
| 2319 | + req->quality_level = SVGA3D_MS_QUALITY_NONE; | |
| 2320 | + req->buffer_byte_stride = 0; | |
| 2321 | + req->must_be_zero = 0; | |
| 2322 | + req->base.svga3d_flags = SVGA3D_FLAGS_LOWER_32(flags); | |
| 2323 | + req->svga3d_flags_upper_32_bits = SVGA3D_FLAGS_UPPER_32(flags); | |
| 2324 | + req->base.format = (uint32_t)format; | |
| 2325 | + req->base.drm_surface_flags = drm_vmw_surface_flag_scanout; | |
| 2326 | + req->base.drm_surface_flags |= drm_vmw_surface_flag_shareable; | |
| 2327 | + req->base.drm_surface_flags |= drm_vmw_surface_flag_create_buffer; | |
| 2328 | + req->base.drm_surface_flags |= drm_vmw_surface_flag_coherent; | |
| 2329 | + req->base.base_size.width = args->width; | |
| 2330 | + req->base.base_size.height = args->height; | |
| 2331 | + req->base.base_size.depth = 1; | |
| 2332 | + req->base.array_size = 0; | |
| 2333 | + req->base.mip_levels = 1; | |
| 2334 | + req->base.multisample_count = 0; | |
| 2335 | + req->base.buffer_handle = SVGA3D_INVALID_ID; | |
| 2336 | + req->base.autogen_filter = SVGA3D_TEX_FILTER_NONE; | |
| 2337 | + ret = vmw_gb_surface_define_ext_ioctl(dev, &arg, file_priv); | |
| 2338 | + if (ret) { | |
| 2339 | + drm_warn(dev, "Unable to create a dumb buffer\n"); | |
| 2340 | + return ret; | |
| 2341 | + } | |
| 2342 | + | |
| 2343 | + args->handle = arg.rep.buffer_handle; | |
| 2344 | + args->size = arg.rep.buffer_size; | |
| 2345 | + args->pitch = vmw_surface_calculate_pitch(desc, &drm_size); | |
| 2346 | + | |
| 2347 | + ret = vmw_user_resource_lookup_handle(dev_priv, tfile, arg.rep.handle, | |
| 2348 | + user_surface_converter, | |
| 2349 | + &res); | |
| 2350 | + if (ret) { | |
| 2351 | + drm_err(dev, "Created resource handle doesn't exist!\n"); | |
| 2352 | + goto err; | |
| 2353 | + } | |
| 2354 | + | |
| 2355 | + vbo = res->guest_memory_bo; | |
| 2356 | + vbo->is_dumb = true; | |
| 2357 | + vbo->dumb_surface = vmw_res_to_srf(res); | |
| 2358 | + | |
| 2359 | +err: | |
| 2360 | + if (res) | |
| 2361 | + vmw_resource_unreference(&res); | |
| 2362 | + if (ret) | |
| 2363 | + ttm_ref_object_base_unref(tfile, arg.rep.handle); | |
| 2364 | + | |
| 2365 | + return ret; | |
| 2366 | +} | |
| @@ -75,7 +75,7 @@ done: · 74 unmodified lines | ||
| 75 | 75 | return ret; |
| 76 | 76 | } |
| 77 | 77 | |
| 78 | −static int | |
| 78 | +static void | |
| 79 | 79 | compute_crc(struct drm_crtc *crtc, |
| 80 | 80 | struct vmw_surface *surf, |
| 81 | 81 | u32 *crc) |
| @@ -101,8 +101,6 @@ compute_crc(struct drm_crtc *crtc, · 19 unmodified lines | ||
| 101 | 101 | } |
| 102 | 102 | |
| 103 | 103 | vmw_bo_unmap(bo); |
| 104 | − | |
| 105 | − return 0; | |
| 106 | 104 | } |
| 107 | 105 | |
| 108 | 106 | static void |
| @@ -116,7 +114,6 @@ crc_generate_worker(struct work_struct *work) · 7 unmodified lines | ||
| 116 | 114 | u64 frame_start, frame_end; |
| 117 | 115 | u32 crc32 = 0; |
| 118 | 116 | struct vmw_surface *surf = 0; |
| 119 | − int ret; | |
| 120 | 117 | |
| 121 | 118 | spin_lock_irq(&du->vkms.crc_state_lock); |
| 122 | 119 | crc_pending = du->vkms.crc_pending; |
| @@ -130,22 +127,24 @@ crc_generate_worker(struct work_struct *work) · 7 unmodified lines | ||
| 130 | 127 | return; |
| 131 | 128 | |
| 132 | 129 | spin_lock_irq(&du->vkms.crc_state_lock); |
| 133 | − surf = du->vkms.surface; | |
| 130 | + surf = vmw_surface_reference(du->vkms.surface); | |
| 134 | 131 | spin_unlock_irq(&du->vkms.crc_state_lock); |
| 135 | 132 | |
| 136 | − if (vmw_surface_sync(vmw, surf)) { | |
| 137 | − drm_warn(crtc->dev, "CRC worker wasn't able to sync the crc surface!\n"); | |
| 138 | − return; | |
| 139 | − } | |
| 133 | + if (surf) { | |
| 134 | + if (vmw_surface_sync(vmw, surf)) { | |
| 135 | + drm_warn( | |
| 136 | + crtc->dev, | |
| 137 | + "CRC worker wasn't able to sync the crc surface!\n"); | |
| 138 | + return; | |
| 139 | + } | |
| 140 | 140 | |
| 141 | − ret = compute_crc(crtc, surf, &crc32); | |
| 142 | − if (ret) | |
| 143 | − return; | |
| 141 | + compute_crc(crtc, surf, &crc32); | |
| 142 | + vmw_surface_unreference(&surf); | |
| 143 | + } | |
| 144 | 144 | |
| 145 | 145 | spin_lock_irq(&du->vkms.crc_state_lock); |
| 146 | 146 | frame_start = du->vkms.frame_start; |
| 147 | 147 | frame_end = du->vkms.frame_end; |
| 148 | − crc_pending = du->vkms.crc_pending; | |
| 149 | 148 | du->vkms.frame_start = 0; |
| 150 | 149 | du->vkms.frame_end = 0; |
| 151 | 150 | du->vkms.crc_pending = false; |
| @@ -164,7 +163,7 @@ vmw_vkms_vblank_simulate(struct hrtimer *timer) · 12 unmodified lines | ||
| 164 | 163 | struct vmw_display_unit *du = container_of(timer, struct vmw_display_unit, vkms.timer); |
| 165 | 164 | struct drm_crtc *crtc = &du->crtc; |
| 166 | 165 | struct vmw_private *vmw = vmw_priv(crtc->dev); |
| 167 | − struct vmw_surface *surf = NULL; | |
| 166 | + bool has_surface = false; | |
| 168 | 167 | u64 ret_overrun; |
| 169 | 168 | bool locked, ret; |
| 170 | 169 | |
| @@ -179,10 +178,10 @@ vmw_vkms_vblank_simulate(struct hrtimer *timer) · 8 unmodified lines | ||
| 179 | 178 | WARN_ON(!ret); |
| 180 | 179 | if (!locked) |
| 181 | 180 | return HRTIMER_RESTART; |
| 182 | − surf = du->vkms.surface; | |
| 181 | + has_surface = du->vkms.surface != NULL; | |
| 183 | 182 | vmw_vkms_unlock(crtc); |
| 184 | 183 | |
| 185 | − if (du->vkms.crc_enabled && surf) { | |
| 184 | + if (du->vkms.crc_enabled && has_surface) { | |
| 186 | 185 | u64 frame = drm_crtc_accurate_vblank_count(crtc); |
| 187 | 186 | |
| 188 | 187 | spin_lock(&du->vkms.crc_state_lock); |
| @@ -336,6 +335,8 @@ vmw_vkms_crtc_cleanup(struct drm_crtc *crtc) · 147 unmodified lines | ||
| 336 | 335 | { |
| 337 | 336 | struct vmw_display_unit *du = vmw_crtc_to_du(crtc); |
| 338 | 337 | |
| 338 | + if (du->vkms.surface) | |
| 339 | + vmw_surface_unreference(&du->vkms.surface); | |
| 339 | 340 | WARN_ON(work_pending(&du->vkms.crc_generator_work)); |
| 340 | 341 | hrtimer_cancel(&du->vkms.timer); |
| 341 | 342 | } |
| @@ -497,9 +498,12 @@ vmw_vkms_set_crc_surface(struct drm_crtc *crtc, · 155 unmodified lines | ||
| 497 | 498 | struct vmw_display_unit *du = vmw_crtc_to_du(crtc); |
| 498 | 499 | struct vmw_private *vmw = vmw_priv(crtc->dev); |
| 499 | 500 | |
| 500 | − if (vmw->vkms_enabled) { | |
| 501 | + if (vmw->vkms_enabled && du->vkms.surface != surf) { | |
| 501 | 502 | WARN_ON(atomic_read(&du->vkms.atomic_lock) != VMW_VKMS_LOCK_MODESET); |
| 502 | − du->vkms.surface = surf; | |
| 503 | + if (du->vkms.surface) | |
| 504 | + vmw_surface_unreference(&du->vkms.surface); | |
| 505 | + if (surf) | |
| 506 | + du->vkms.surface = vmw_surface_reference(surf); | |
| 503 | 507 | } |
| 504 | 508 | } |
| 505 | 509 | |
| @@ -25,12 +25,14 @@ $(obj)/generated/%_wa_oob.c $(obj)/generated/%_wa_oob.h: $(obj)/xe_gen_wa_oob \ · 24 unmodified lines | ||
| 25 | 25 | |
| 26 | 26 | uses_generated_oob := \ |
| 27 | 27 | $(obj)/xe_ggtt.o \ |
| 28 | + $(obj)/xe_device.o \ | |
| 28 | 29 | $(obj)/xe_gsc.o \ |
| 29 | 30 | $(obj)/xe_gt.o \ |
| 30 | 31 | $(obj)/xe_guc.o \ |
| 31 | 32 | $(obj)/xe_guc_ads.o \ |
| 32 | 33 | $(obj)/xe_guc_pc.o \ |
| 33 | 34 | $(obj)/xe_migrate.o \ |
| 35 | + $(obj)/xe_pat.o \ | |
| 34 | 36 | $(obj)/xe_ring_ops.o \ |
| 35 | 37 | $(obj)/xe_vm.o \ |
| 36 | 38 | $(obj)/xe_wa.o \ |
| @@ -132,6 +132,7 @@ static void xe_display_fini_noirq(void *arg) · 131 unmodified lines | ||
| 132 | 132 | return; |
| 133 | 133 | |
| 134 | 134 | intel_display_driver_remove_noirq(xe); |
| 135 | + intel_opregion_cleanup(xe); | |
| 135 | 136 | } |
| 136 | 137 | |
| 137 | 138 | int xe_display_init_noirq(struct xe_device *xe) |
| @@ -157,8 +158,10 @@ int xe_display_init_noirq(struct xe_device *xe) · 19 unmodified lines | ||
| 157 | 158 | intel_display_device_info_runtime_init(xe); |
| 158 | 159 | |
| 159 | 160 | err = intel_display_driver_probe_noirq(xe); |
| 160 | − if (err) | |
| 161 | + if (err) { | |
| 162 | + intel_opregion_cleanup(xe); | |
| 161 | 163 | return err; |
| 164 | + } | |
| 162 | 165 | |
| 163 | 166 | return devm_add_action_or_reset(xe->drm.dev, xe_display_fini_noirq, xe); |
| 164 | 167 | } |
| @@ -280,6 +283,27 @@ static bool suspend_to_idle(void) · 115 unmodified lines | ||
| 280 | 283 | return false; |
| 281 | 284 | } |
| 282 | 285 | |
| 286 | +static void xe_display_flush_cleanup_work(struct xe_device *xe) | |
| 287 | +{ | |
| 288 | + struct intel_crtc *crtc; | |
| 289 | + | |
| 290 | + for_each_intel_crtc(&xe->drm, crtc) { | |
| 291 | + struct drm_crtc_commit *commit; | |
| 292 | + | |
| 293 | + spin_lock(&crtc->base.commit_lock); | |
| 294 | + commit = list_first_entry_or_null(&crtc->base.commit_list, | |
| 295 | + struct drm_crtc_commit, commit_entry); | |
| 296 | + if (commit) | |
| 297 | + drm_crtc_commit_get(commit); | |
| 298 | + spin_unlock(&crtc->base.commit_lock); | |
| 299 | + | |
| 300 | + if (commit) { | |
| 301 | + wait_for_completion(&commit->cleanup_done); | |
| 302 | + drm_crtc_commit_put(commit); | |
| 303 | + } | |
| 304 | + } | |
| 305 | +} | |
| 306 | + | |
| 283 | 307 | void xe_display_pm_suspend(struct xe_device *xe, bool runtime) |
| 284 | 308 | { |
| 285 | 309 | bool s2idle = suspend_to_idle(); |
| @@ -297,6 +321,8 @@ void xe_display_pm_suspend(struct xe_device *xe, bool runtime) · 11 unmodified lines | ||
| 297 | 321 | if (!runtime) |
| 298 | 322 | intel_display_driver_suspend(xe); |
| 299 | 323 | |
| 324 | + xe_display_flush_cleanup_work(xe); | |
| 325 | + | |
| 300 | 326 | intel_dp_mst_suspend(xe); |
| 301 | 327 | |
| 302 | 328 | intel_hpd_cancel_work(xe); |
| @@ -7,6 +7,8 @@ · 6 unmodified lines | ||
| 7 | 7 | #include "intel_display_types.h" |
| 8 | 8 | #include "intel_dsb_buffer.h" |
| 9 | 9 | #include "xe_bo.h" |
| 10 | +#include "xe_device.h" | |
| 11 | +#include "xe_device_types.h" | |
| 10 | 12 | #include "xe_gt.h" |
| 11 | 13 | |
| 12 | 14 | u32 intel_dsb_buffer_ggtt_offset(struct intel_dsb_buffer *dsb_buf) |
| @@ -16,7 +18,10 @@ u32 intel_dsb_buffer_ggtt_offset(struct intel_dsb_buffer *dsb_buf) · 3 unmodified lines | ||
| 16 | 18 | |
| 17 | 19 | void intel_dsb_buffer_write(struct intel_dsb_buffer *dsb_buf, u32 idx, u32 val) |
| 18 | 20 | { |
| 21 | + struct xe_device *xe = dsb_buf->vma->bo->tile->xe; | |
| 22 | + | |
| 19 | 23 | iosys_map_wr(&dsb_buf->vma->bo->vmap, idx * 4, u32, val); |
| 24 | + xe_device_l2_flush(xe); | |
| 20 | 25 | } |
| 21 | 26 | |
| 22 | 27 | u32 intel_dsb_buffer_read(struct intel_dsb_buffer *dsb_buf, u32 idx) |
| @@ -26,9 +31,12 @@ u32 intel_dsb_buffer_read(struct intel_dsb_buffer *dsb_buf, u32 idx) · 3 unmodified lines | ||
| 26 | 31 | |
| 27 | 32 | void intel_dsb_buffer_memset(struct intel_dsb_buffer *dsb_buf, u32 idx, u32 val, size_t size) |
| 28 | 33 | { |
| 34 | + struct xe_device *xe = dsb_buf->vma->bo->tile->xe; | |
| 35 | + | |
| 29 | 36 | WARN_ON(idx > (dsb_buf->buf_size - size) / sizeof(*dsb_buf->cmd_buf)); |
| 30 | 37 | |
| 31 | 38 | iosys_map_memset(&dsb_buf->vma->bo->vmap, idx * 4, val, size); |
| 39 | + xe_device_l2_flush(xe); | |
| 32 | 40 | } |
| 33 | 41 | |
| 34 | 42 | bool intel_dsb_buffer_create(struct intel_crtc *crtc, struct intel_dsb_buffer *dsb_buf, size_t size) |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | ||
| 10 | 10 | #include "intel_fb.h" |
| 11 | 11 | #include "intel_fb_pin.h" |
| 12 | 12 | #include "xe_bo.h" |
| 13 | +#include "xe_device.h" | |
| 13 | 14 | #include "xe_ggtt.h" |
| 14 | 15 | #include "xe_gt.h" |
| 15 | 16 | #include "xe_pm.h" |
| @@ -304,6 +305,8 @@ static struct i915_vma *__xe_pin_fb_vma(const struct intel_framebuffer *fb, · 288 unmodified lines | ||
| 304 | 305 | if (ret) |
| 305 | 306 | goto err_unpin; |
| 306 | 307 | |
| 308 | + /* Ensure DPT writes are flushed */ | |
| 309 | + xe_device_l2_flush(xe); | |
| 307 | 310 | return vma; |
| 308 | 311 | |
| 309 | 312 | err_unpin: |
| @@ -80,6 +80,9 @@ · 79 unmodified lines | ||
| 80 | 80 | #define LE_CACHEABILITY_MASK REG_GENMASK(1, 0) |
| 81 | 81 | #define LE_CACHEABILITY(value) REG_FIELD_PREP(LE_CACHEABILITY_MASK, value) |
| 82 | 82 | |
| 83 | +#define XE2_GAMREQSTRM_CTRL XE_REG(0x4194) | |
| 84 | +#define CG_DIS_CNTLBUS REG_BIT(6) | |
| 85 | + | |
| 83 | 86 | #define CCS_AUX_INV XE_REG(0x4208) |
| 84 | 87 | |
| 85 | 88 | #define VD0_AUX_INV XE_REG(0x4218) |
| @@ -372,6 +375,11 @@ · 286 unmodified lines | ||
| 372 | 375 | |
| 373 | 376 | #define XEHPC_L3CLOS_MASK(i) XE_REG_MCR(0xb194 + (i) * 8) |
| 374 | 377 | |
| 378 | +#define XE2_GLOBAL_INVAL XE_REG(0xb404) | |
| 379 | + | |
| 380 | +#define SCRATCH1LPFC XE_REG(0xb474) | |
| 381 | +#define EN_L3_RW_CCS_CACHE_FLUSH REG_BIT(0) | |
| 382 | + | |
| 375 | 383 | #define XE2LPM_L3SQCREG5 XE_REG_MCR(0xb658) |
| 376 | 384 | |
| 377 | 385 | #define XE2_TDF_CTRL XE_REG(0xb418) |
| @@ -429,6 +437,7 @@ · 51 unmodified lines | ||
| 429 | 437 | #define DIS_FIX_EOT1_FLUSH REG_BIT(9) |
| 430 | 438 | |
| 431 | 439 | #define TDL_TSL_CHICKEN XE_REG_MCR(0xe4c4, XE_REG_OPTION_MASKED) |
| 440 | +#define STK_ID_RESTRICT REG_BIT(12) | |
| 432 | 441 | #define SLM_WMTP_RESTORE REG_BIT(11) |
| 433 | 442 | |
| 434 | 443 | #define ROW_CHICKEN XE_REG_MCR(0xe4f0, XE_REG_OPTION_MASKED) |
| @@ -1575,7 +1575,7 @@ struct xe_bo *xe_bo_create_from_data(struct xe_device *xe, struct xe_tile *tile, · 1574 unmodified lines | ||
| 1575 | 1575 | return bo; |
| 1576 | 1576 | } |
| 1577 | 1577 | |
| 1578 | −static void __xe_bo_unpin_map_no_vm(struct drm_device *drm, void *arg) | |
| 1578 | +static void __xe_bo_unpin_map_no_vm(void *arg) | |
| 1579 | 1579 | { |
| 1580 | 1580 | xe_bo_unpin_map_no_vm(arg); |
| 1581 | 1581 | } |
| @@ -1590,7 +1590,7 @@ struct xe_bo *xe_managed_bo_create_pin_map(struct xe_device *xe, struct xe_tile · 8 unmodified lines | ||
| 1590 | 1590 | if (IS_ERR(bo)) |
| 1591 | 1591 | return bo; |
| 1592 | 1592 | |
| 1593 | − ret = drmm_add_action_or_reset(&xe->drm, __xe_bo_unpin_map_no_vm, bo); | |
| 1593 | + ret = devm_add_action_or_reset(xe->drm.dev, __xe_bo_unpin_map_no_vm, bo); | |
| 1594 | 1594 | if (ret) |
| 1595 | 1595 | return ERR_PTR(ret); |
| 1596 | 1596 | |
| @@ -1638,7 +1638,7 @@ int xe_managed_bo_reinit_in_vram(struct xe_device *xe, struct xe_tile *tile, str · 41 unmodified lines | ||
| 1638 | 1638 | if (IS_ERR(bo)) |
| 1639 | 1639 | return PTR_ERR(bo); |
| 1640 | 1640 | |
| 1641 | − drmm_release_action(&xe->drm, __xe_bo_unpin_map_no_vm, *src); | |
| 1641 | + devm_release_action(xe->drm.dev, __xe_bo_unpin_map_no_vm, *src); | |
| 1642 | 1642 | *src = bo; |
| 1643 | 1643 | |
| 1644 | 1644 | return 0; |
| @@ -54,6 +54,9 @@ · 53 unmodified lines | ||
| 54 | 54 | #include "xe_vm.h" |
| 55 | 55 | #include "xe_vram.h" |
| 56 | 56 | #include "xe_wait_user_fence.h" |
| 57 | +#include "xe_wa.h" | |
| 58 | + | |
| 59 | +#include <generated/xe_wa_oob.h> | |
| 57 | 60 | |
| 58 | 61 | static int xe_file_open(struct drm_device *dev, struct drm_file *file) |
| 59 | 62 | { |
| @@ -87,9 +90,55 @@ static int xe_file_open(struct drm_device *dev, struct drm_file *file) · 27 unmodified lines | ||
| 87 | 90 | spin_unlock(&xe->clients.lock); |
| 88 | 91 | |
| 89 | 92 | file->driver_priv = xef; |
| 93 | + kref_init(&xef->refcount); | |
| 94 | + | |
| 90 | 95 | return 0; |
| 91 | 96 | } |
| 92 | 97 | |
| 98 | +static void xe_file_destroy(struct kref *ref) | |
| 99 | +{ | |
| 100 | + struct xe_file *xef = container_of(ref, struct xe_file, refcount); | |
| 101 | + struct xe_device *xe = xef->xe; | |
| 102 | + | |
| 103 | + xa_destroy(&xef->exec_queue.xa); | |
| 104 | + mutex_destroy(&xef->exec_queue.lock); | |
| 105 | + xa_destroy(&xef->vm.xa); | |
| 106 | + mutex_destroy(&xef->vm.lock); | |
| 107 | + | |
| 108 | + spin_lock(&xe->clients.lock); | |
| 109 | + xe->clients.count--; | |
| 110 | + spin_unlock(&xe->clients.lock); | |
| 111 | + | |
| 112 | + xe_drm_client_put(xef->client); | |
| 113 | + kfree(xef); | |
| 114 | +} | |
| 115 | + | |
| 116 | +/** | |
| 117 | + * xe_file_get() - Take a reference to the xe file object | |
| 118 | + * @xef: Pointer to the xe file | |
| 119 | + * | |
| 120 | + * Anyone with a pointer to xef must take a reference to the xe file | |
| 121 | + * object using this call. | |
| 122 | + * | |
| 123 | + * Return: xe file pointer | |
| 124 | + */ | |
| 125 | +struct xe_file *xe_file_get(struct xe_file *xef) | |
| 126 | +{ | |
| 127 | + kref_get(&xef->refcount); | |
| 128 | + return xef; | |
| 129 | +} | |
| 130 | + | |
| 131 | +/** | |
| 132 | + * xe_file_put() - Drop a reference to the xe file object | |
| 133 | + * @xef: Pointer to the xe file | |
| 134 | + * | |
| 135 | + * Used to drop reference to the xef object | |
| 136 | + */ | |
| 137 | +void xe_file_put(struct xe_file *xef) | |
| 138 | +{ | |
| 139 | + kref_put(&xef->refcount, xe_file_destroy); | |
| 140 | +} | |
| 141 | + | |
| 93 | 142 | static void xe_file_close(struct drm_device *dev, struct drm_file *file) |
| 94 | 143 | { |
| 95 | 144 | struct xe_device *xe = to_xe_device(dev); |
| @@ -98,6 +147,8 @@ static void xe_file_close(struct drm_device *dev, struct drm_file *file) · 2 unmodified lines | ||
| 98 | 147 | struct xe_exec_queue *q; |
| 99 | 148 | unsigned long idx; |
| 100 | 149 | |
| 150 | + xe_pm_runtime_get(xe); | |
| 151 | + | |
| 101 | 152 | /* |
| 102 | 153 | * No need for exec_queue.lock here as there is no contention for it |
| 103 | 154 | * when FD is closing as IOCTLs presumably can't be modifying the |
| @@ -108,21 +159,14 @@ static void xe_file_close(struct drm_device *dev, struct drm_file *file) · 4 unmodified lines | ||
| 108 | 159 | xe_exec_queue_kill(q); |
| 109 | 160 | xe_exec_queue_put(q); |
| 110 | 161 | } |
| 111 | − xa_destroy(&xef->exec_queue.xa); | |
| 112 | − mutex_destroy(&xef->exec_queue.lock); | |
| 113 | 162 | mutex_lock(&xef->vm.lock); |
| 114 | 163 | xa_for_each(&xef->vm.xa, idx, vm) |
| 115 | 164 | xe_vm_close_and_put(vm); |
| 116 | 165 | mutex_unlock(&xef->vm.lock); |
| 117 | − xa_destroy(&xef->vm.xa); | |
| 118 | − mutex_destroy(&xef->vm.lock); | |
| 119 | 166 | |
| 120 | − spin_lock(&xe->clients.lock); | |
| 121 | − xe->clients.count--; | |
| 122 | − spin_unlock(&xe->clients.lock); | |
| 167 | + xe_file_put(xef); | |
| 123 | 168 | |
| 124 | − xe_drm_client_put(xef->client); | |
| 125 | − kfree(xef); | |
| 169 | + xe_pm_runtime_put(xe); | |
| 126 | 170 | } |
| 127 | 171 | |
| 128 | 172 | static const struct drm_ioctl_desc xe_ioctls[] = { |
| @@ -779,6 +823,11 @@ void xe_device_td_flush(struct xe_device *xe) · 650 unmodified lines | ||
| 779 | 823 | if (!IS_DGFX(xe) || GRAPHICS_VER(xe) < 20) |
| 780 | 824 | return; |
| 781 | 825 | |
| 826 | + if (XE_WA(xe_root_mmio_gt(xe), 16023588340)) { | |
| 827 | + xe_device_l2_flush(xe); | |
| 828 | + return; | |
| 829 | + } | |
| 830 | + | |
| 782 | 831 | for_each_gt(gt, xe, id) { |
| 783 | 832 | if (xe_gt_is_media_type(gt)) |
| 784 | 833 | continue; |
| @@ -802,6 +851,30 @@ void xe_device_td_flush(struct xe_device *xe) · 17 unmodified lines | ||
| 802 | 851 | } |
| 803 | 852 | } |
| 804 | 853 | |
| 854 | +void xe_device_l2_flush(struct xe_device *xe) | |
| 855 | +{ | |
| 856 | + struct xe_gt *gt; | |
| 857 | + int err; | |
| 858 | + | |
| 859 | + gt = xe_root_mmio_gt(xe); | |
| 860 | + | |
| 861 | + if (!XE_WA(gt, 16023588340)) | |
| 862 | + return; | |
| 863 | + | |
| 864 | + err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT); | |
| 865 | + if (err) | |
| 866 | + return; | |
| 867 | + | |
| 868 | + spin_lock(>->global_invl_lock); | |
| 869 | + xe_mmio_write32(gt, XE2_GLOBAL_INVAL, 0x1); | |
| 870 | + | |
| 871 | + if (xe_mmio_wait32(gt, XE2_GLOBAL_INVAL, 0x1, 0x0, 150, NULL, true)) | |
| 872 | + xe_gt_err_once(gt, "Global invalidation timeout\n"); | |
| 873 | + spin_unlock(>->global_invl_lock); | |
| 874 | + | |
| 875 | + xe_force_wake_put(gt_to_fw(gt), XE_FW_GT); | |
| 876 | +} | |
| 877 | + | |
| 805 | 878 | u32 xe_device_ccs_bytes(struct xe_device *xe, u64 size) |
| 806 | 879 | { |
| 807 | 880 | return xe_device_has_flat_ccs(xe) ? |
| @@ -162,6 +162,7 @@ u64 xe_device_canonicalize_addr(struct xe_device *xe, u64 address); · 161 unmodified lines | ||
| 162 | 162 | u64 xe_device_uncanonicalize_addr(struct xe_device *xe, u64 address); |
| 163 | 163 | |
| 164 | 164 | void xe_device_td_flush(struct xe_device *xe); |
| 165 | +void xe_device_l2_flush(struct xe_device *xe); | |
| 165 | 166 | |
| 166 | 167 | static inline bool xe_device_wedged(struct xe_device *xe) |
| 167 | 168 | { |
| @@ -170,4 +171,7 @@ static inline bool xe_device_wedged(struct xe_device *xe) · 2 unmodified lines | ||
| 170 | 171 | |
| 171 | 172 | void xe_device_declare_wedged(struct xe_device *xe); |
| 172 | 173 | |
| 174 | +struct xe_file *xe_file_get(struct xe_file *xef); | |
| 175 | +void xe_file_put(struct xe_file *xef); | |
| 176 | + | |
| 173 | 177 | #endif |
| @@ -566,6 +566,9 @@ struct xe_file { · 565 unmodified lines | ||
| 566 | 566 | |
| 567 | 567 | /** @client: drm client */ |
| 568 | 568 | struct xe_drm_client *client; |
| 569 | + | |
| 570 | + /** @refcount: ref count of this xe file */ | |
| 571 | + struct kref refcount; | |
| 569 | 572 | }; |
| 570 | 573 | |
| 571 | 574 | #endif |
| @@ -251,11 +251,8 @@ static void show_run_ticks(struct drm_printer *p, struct drm_file *file) · 250 unmodified lines | ||
| 251 | 251 | |
| 252 | 252 | /* Accumulate all the exec queues from this client */ |
| 253 | 253 | mutex_lock(&xef->exec_queue.lock); |
| 254 | − xa_for_each(&xef->exec_queue.xa, i, q) { | |
| 254 | + xa_for_each(&xef->exec_queue.xa, i, q) | |
| 255 | 255 | xe_exec_queue_update_run_ticks(q); |
| 256 | − xef->run_ticks[q->class] += q->run_ticks - q->old_run_ticks; | |
| 257 | − q->old_run_ticks = q->run_ticks; | |
| 258 | − } | |
| 259 | 256 | mutex_unlock(&xef->exec_queue.lock); |
| 260 | 257 | |
| 261 | 258 | /* Get the total GPU cycles */ |
| @@ -37,6 +37,10 @@ static void __xe_exec_queue_free(struct xe_exec_queue *q) · 36 unmodified lines | ||
| 37 | 37 | { |
| 38 | 38 | if (q->vm) |
| 39 | 39 | xe_vm_put(q->vm); |
| 40 | + | |
| 41 | + if (q->xef) | |
| 42 | + xe_file_put(q->xef); | |
| 43 | + | |
| 40 | 44 | kfree(q); |
| 41 | 45 | } |
| 42 | 46 | |
| @@ -101,22 +105,35 @@ static struct xe_exec_queue *__xe_exec_queue_alloc(struct xe_device *xe, · 58 unmodified lines | ||
| 101 | 105 | |
| 102 | 106 | static int __xe_exec_queue_init(struct xe_exec_queue *q) |
| 103 | 107 | { |
| 108 | + struct xe_vm *vm = q->vm; | |
| 104 | 109 | int i, err; |
| 105 | 110 | |
| 111 | + if (vm) { | |
| 112 | + err = xe_vm_lock(vm, true); | |
| 113 | + if (err) | |
| 114 | + return err; | |
| 115 | + } | |
| 116 | + | |
| 106 | 117 | for (i = 0; i < q->width; ++i) { |
| 107 | 118 | q->lrc[i] = xe_lrc_create(q->hwe, q->vm, SZ_16K); |
| 108 | 119 | if (IS_ERR(q->lrc[i])) { |
| 109 | 120 | err = PTR_ERR(q->lrc[i]); |
| 110 | − goto err_lrc; | |
| 121 | + goto err_unlock; | |
| 111 | 122 | } |
| 112 | 123 | } |
| 113 | 124 | |
| 125 | + if (vm) | |
| 126 | + xe_vm_unlock(vm); | |
| 127 | + | |
| 114 | 128 | err = q->ops->init(q); |
| 115 | 129 | if (err) |
| 116 | 130 | goto err_lrc; |
| 117 | 131 | |
| 118 | 132 | return 0; |
| 119 | 133 | |
| 134 | +err_unlock: | |
| 135 | + if (vm) | |
| 136 | + xe_vm_unlock(vm); | |
| 120 | 137 | err_lrc: |
| 121 | 138 | for (i = i - 1; i >= 0; --i) |
| 122 | 139 | xe_lrc_put(q->lrc[i]); |
| @@ -136,15 +153,7 @@ struct xe_exec_queue *xe_exec_queue_create(struct xe_device *xe, struct xe_vm *v · 13 unmodified lines | ||
| 136 | 153 | if (IS_ERR(q)) |
| 137 | 154 | return q; |
| 138 | 155 | |
| 139 | − if (vm) { | |
| 140 | − err = xe_vm_lock(vm, true); | |
| 141 | − if (err) | |
| 142 | − goto err_post_alloc; | |
| 143 | − } | |
| 144 | − | |
| 145 | 156 | err = __xe_exec_queue_init(q); |
| 146 | − if (vm) | |
| 147 | − xe_vm_unlock(vm); | |
| 148 | 157 | if (err) |
| 149 | 158 | goto err_post_alloc; |
| 150 | 159 | |
| @@ -634,7 +643,6 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data, · 483 unmodified lines | ||
| 634 | 643 | |
| 635 | 644 | if (xe_vm_in_preempt_fence_mode(vm)) { |
| 636 | 645 | q->lr.context = dma_fence_context_alloc(1); |
| 637 | − spin_lock_init(&q->lr.lock); | |
| 638 | 646 | |
| 639 | 647 | err = xe_vm_add_compute_exec_queue(vm, q); |
| 640 | 648 | if (XE_IOCTL_DBG(xe, err)) |
| @@ -649,6 +657,7 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data, · 8 unmodified lines | ||
| 649 | 657 | goto kill_exec_queue; |
| 650 | 658 | |
| 651 | 659 | args->exec_queue_id = id; |
| 660 | + q->xef = xe_file_get(xef); | |
| 652 | 661 | |
| 653 | 662 | return 0; |
| 654 | 663 | |
| @@ -762,6 +771,7 @@ bool xe_exec_queue_is_idle(struct xe_exec_queue *q) · 107 unmodified lines | ||
| 762 | 771 | */ |
| 763 | 772 | void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q) |
| 764 | 773 | { |
| 774 | + struct xe_file *xef; | |
| 765 | 775 | struct xe_lrc *lrc; |
| 766 | 776 | u32 old_ts, new_ts; |
| 767 | 777 | |
| @@ -773,6 +783,8 @@ void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q) · 5 unmodified lines | ||
| 773 | 783 | if (!q->vm || !q->vm->xef) |
| 774 | 784 | return; |
| 775 | 785 | |
| 786 | + xef = q->vm->xef; | |
| 787 | + | |
| 776 | 788 | /* |
| 777 | 789 | * Only sample the first LRC. For parallel submission, all of them are |
| 778 | 790 | * scheduled together and we compensate that below by multiplying by |
| @@ -783,7 +795,7 @@ void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q) · 4 unmodified lines | ||
| 783 | 795 | */ |
| 784 | 796 | lrc = q->lrc[0]; |
| 785 | 797 | new_ts = xe_lrc_update_timestamp(lrc, &old_ts); |
| 786 | − q->run_ticks += (new_ts - old_ts) * q->width; | |
| 798 | + xef->run_ticks[q->class] += (new_ts - old_ts) * q->width; | |
| 787 | 799 | } |
| 788 | 800 | |
| 789 | 801 | void xe_exec_queue_kill(struct xe_exec_queue *q) |
| @@ -38,6 +38,9 @@ enum xe_exec_queue_priority { · 37 unmodified lines | ||
| 38 | 38 | * a kernel object. |
| 39 | 39 | */ |
| 40 | 40 | struct xe_exec_queue { |
| 41 | + /** @xef: Back pointer to xe file if this is user created exec queue */ | |
| 42 | + struct xe_file *xef; | |
| 43 | + | |
| 41 | 44 | /** @gt: graphics tile this exec queue can submit to */ |
| 42 | 45 | struct xe_gt *gt; |
| 43 | 46 | /** |
| @@ -123,8 +126,6 @@ struct xe_exec_queue { · 79 unmodified lines | ||
| 123 | 126 | u32 seqno; |
| 124 | 127 | /** @lr.link: link into VM's list of exec queues */ |
| 125 | 128 | struct list_head link; |
| 126 | − /** @lr.lock: preemption fences lock */ | |
| 127 | − spinlock_t lock; | |
| 128 | 129 | } lr; |
| 129 | 130 | |
| 130 | 131 | /** @ops: submission backend exec queue operations */ |
| @@ -139,10 +140,6 @@ struct xe_exec_queue { · 8 unmodified lines | ||
| 139 | 140 | * Protected by @vm's resv. Unused if @vm == NULL. |
| 140 | 141 | */ |
| 141 | 142 | u64 tlb_flush_seqno; |
| 142 | − /** @old_run_ticks: prior hw engine class run time in ticks for this exec queue */ | |
| 143 | − u64 old_run_ticks; | |
| 144 | − /** @run_ticks: hw engine class run time in ticks for this exec queue */ | |
| 145 | − u64 run_ticks; | |
| 146 | 143 | /** @lrc: logical ring context for this exec queue */ |
| 147 | 144 | struct xe_lrc *lrc[]; |
| 148 | 145 | }; |
| @@ -260,7 +260,7 @@ static int gsc_upload_and_init(struct xe_gsc *gsc) · 259 unmodified lines | ||
| 260 | 260 | struct xe_tile *tile = gt_to_tile(gt); |
| 261 | 261 | int ret; |
| 262 | 262 | |
| 263 | − if (XE_WA(gt, 14018094691)) { | |
| 263 | + if (XE_WA(tile->primary_gt, 14018094691)) { | |
| 264 | 264 | ret = xe_force_wake_get(gt_to_fw(tile->primary_gt), XE_FORCEWAKE_ALL); |
| 265 | 265 | |
| 266 | 266 | /* |
| @@ -278,7 +278,7 @@ static int gsc_upload_and_init(struct xe_gsc *gsc) · 11 unmodified lines | ||
| 278 | 278 | |
| 279 | 279 | ret = gsc_upload(gsc); |
| 280 | 280 | |
| 281 | − if (XE_WA(gt, 14018094691)) | |
| 281 | + if (XE_WA(tile->primary_gt, 14018094691)) | |
| 282 | 282 | xe_force_wake_put(gt_to_fw(tile->primary_gt), XE_FORCEWAKE_ALL); |
| 283 | 283 | |
| 284 | 284 | if (ret) |
| @@ -437,7 +437,7 @@ out: · 152 unmodified lines | ||
| 437 | 437 | return ret; |
| 438 | 438 | } |
| 439 | 439 | |
| 440 | −static void free_resources(struct drm_device *drm, void *arg) | |
| 440 | +static void free_resources(void *arg) | |
| 441 | 441 | { |
| 442 | 442 | struct xe_gsc *gsc = arg; |
| 443 | 443 | |
| @@ -501,7 +501,7 @@ int xe_gsc_init_post_hwconfig(struct xe_gsc *gsc) · 57 unmodified lines | ||
| 501 | 501 | gsc->q = q; |
| 502 | 502 | gsc->wq = wq; |
| 503 | 503 | |
| 504 | − err = drmm_add_action_or_reset(&xe->drm, free_resources, gsc); | |
| 504 | + err = devm_add_action_or_reset(xe->drm.dev, free_resources, gsc); | |
| 505 | 505 | if (err) |
| 506 | 506 | return err; |
| 507 | 507 | |
| @@ -11,6 +11,8 @@ · 10 unmodified lines | ||
| 11 | 11 | #include <drm/xe_drm.h> |
| 12 | 12 | #include <generated/xe_wa_oob.h> |
| 13 | 13 | |
| 14 | +#include <generated/xe_wa_oob.h> | |
| 15 | + | |
| 14 | 16 | #include "instructions/xe_gfxpipe_commands.h" |
| 15 | 17 | #include "instructions/xe_mi_commands.h" |
| 16 | 18 | #include "regs/xe_gt_regs.h" |
| @@ -95,6 +97,51 @@ void xe_gt_sanitize(struct xe_gt *gt) · 78 unmodified lines | ||
| 95 | 97 | gt->uc.guc.submission_state.enabled = false; |
| 96 | 98 | } |
| 97 | 99 | |
| 100 | +static void xe_gt_enable_host_l2_vram(struct xe_gt *gt) | |
| 101 | +{ | |
| 102 | + u32 reg; | |
| 103 | + int err; | |
| 104 | + | |
| 105 | + if (!XE_WA(gt, 16023588340)) | |
| 106 | + return; | |
| 107 | + | |
| 108 | + err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT); | |
| 109 | + if (WARN_ON(err)) | |
| 110 | + return; | |
| 111 | + | |
| 112 | + if (!xe_gt_is_media_type(gt)) { | |
| 113 | + xe_mmio_write32(gt, SCRATCH1LPFC, EN_L3_RW_CCS_CACHE_FLUSH); | |
| 114 | + reg = xe_mmio_read32(gt, XE2_GAMREQSTRM_CTRL); | |
| 115 | + reg |= CG_DIS_CNTLBUS; | |
| 116 | + xe_mmio_write32(gt, XE2_GAMREQSTRM_CTRL, reg); | |
| 117 | + } | |
| 118 | + | |
| 119 | + xe_gt_mcr_multicast_write(gt, XEHPC_L3CLOS_MASK(3), 0x3); | |
| 120 | + xe_force_wake_put(gt_to_fw(gt), XE_FW_GT); | |
| 121 | +} | |
| 122 | + | |
| 123 | +static void xe_gt_disable_host_l2_vram(struct xe_gt *gt) | |
| 124 | +{ | |
| 125 | + u32 reg; | |
| 126 | + int err; | |
| 127 | + | |
| 128 | + if (!XE_WA(gt, 16023588340)) | |
| 129 | + return; | |
| 130 | + | |
| 131 | + if (xe_gt_is_media_type(gt)) | |
| 132 | + return; | |
| 133 | + | |
| 134 | + err = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT); | |
| 135 | + if (WARN_ON(err)) | |
| 136 | + return; | |
| 137 | + | |
| 138 | + reg = xe_mmio_read32(gt, XE2_GAMREQSTRM_CTRL); | |
| 139 | + reg &= ~CG_DIS_CNTLBUS; | |
| 140 | + xe_mmio_write32(gt, XE2_GAMREQSTRM_CTRL, reg); | |
| 141 | + | |
| 142 | + xe_force_wake_put(gt_to_fw(gt), XE_FW_GT); | |
| 143 | +} | |
| 144 | + | |
| 98 | 145 | /** |
| 99 | 146 | * xe_gt_remove() - Clean up the GT structures before driver removal |
| 100 | 147 | * @gt: the GT object |
| @@ -111,6 +158,8 @@ void xe_gt_remove(struct xe_gt *gt) · 10 unmodified lines | ||
| 111 | 158 | |
| 112 | 159 | for (i = 0; i < XE_ENGINE_CLASS_MAX; ++i) |
| 113 | 160 | xe_hw_fence_irq_finish(>->fence_irq[i]); |
| 161 | + | |
| 162 | + xe_gt_disable_host_l2_vram(gt); | |
| 114 | 163 | } |
| 115 | 164 | |
| 116 | 165 | static void gt_reset_worker(struct work_struct *w); |
| @@ -339,6 +388,7 @@ int xe_gt_init_early(struct xe_gt *gt) · 222 unmodified lines | ||
| 339 | 388 | |
| 340 | 389 | xe_force_wake_init_gt(gt, gt_to_fw(gt)); |
| 341 | 390 | xe_pcode_init(gt); |
| 391 | + spin_lock_init(>->global_invl_lock); | |
| 342 | 392 | |
| 343 | 393 | return 0; |
| 344 | 394 | } |
| @@ -508,6 +558,7 @@ int xe_gt_init_hwconfig(struct xe_gt *gt) · 163 unmodified lines | ||
| 508 | 558 | |
| 509 | 559 | xe_gt_mcr_init_early(gt); |
| 510 | 560 | xe_pat_init(gt); |
| 561 | + xe_gt_enable_host_l2_vram(gt); | |
| 511 | 562 | |
| 512 | 563 | err = xe_uc_init(>->uc); |
| 513 | 564 | if (err) |
| @@ -643,6 +694,8 @@ static int do_gt_restart(struct xe_gt *gt) · 129 unmodified lines | ||
| 643 | 694 | |
| 644 | 695 | xe_pat_init(gt); |
| 645 | 696 | |
| 697 | + xe_gt_enable_host_l2_vram(gt); | |
| 698 | + | |
| 646 | 699 | xe_gt_mcr_set_implicit_defaults(gt); |
| 647 | 700 | xe_reg_sr_apply_mmio(>->reg_sr, gt); |
| 648 | 701 | |
| @@ -796,6 +849,8 @@ int xe_gt_suspend(struct xe_gt *gt) · 147 unmodified lines | ||
| 796 | 849 | |
| 797 | 850 | xe_gt_idle_disable_pg(gt); |
| 798 | 851 | |
| 852 | + xe_gt_disable_host_l2_vram(gt); | |
| 853 | + | |
| 799 | 854 | XE_WARN_ON(xe_force_wake_put(gt_to_fw(gt), XE_FORCEWAKE_ALL)); |
| 800 | 855 | xe_gt_dbg(gt, "suspended\n"); |
| 801 | 856 | |
| @@ -382,6 +382,18 @@ static void pf_queue_work_func(struct work_struct *w) · 381 unmodified lines | ||
| 382 | 382 | |
| 383 | 383 | static void acc_queue_work_func(struct work_struct *w); |
| 384 | 384 | |
| 385 | +static void pagefault_fini(void *arg) | |
| 386 | +{ | |
| 387 | + struct xe_gt *gt = arg; | |
| 388 | + struct xe_device *xe = gt_to_xe(gt); | |
| 389 | + | |
| 390 | + if (!xe->info.has_usm) | |
| 391 | + return; | |
| 392 | + | |
| 393 | + destroy_workqueue(gt->usm.acc_wq); | |
| 394 | + destroy_workqueue(gt->usm.pf_wq); | |
| 395 | +} | |
| 396 | + | |
| 385 | 397 | int xe_gt_pagefault_init(struct xe_gt *gt) |
| 386 | 398 | { |
| 387 | 399 | struct xe_device *xe = gt_to_xe(gt); |
| @@ -409,10 +421,12 @@ int xe_gt_pagefault_init(struct xe_gt *gt) · 21 unmodified lines | ||
| 409 | 421 | gt->usm.acc_wq = alloc_workqueue("xe_gt_access_counter_work_queue", |
| 410 | 422 | WQ_UNBOUND | WQ_HIGHPRI, |
| 411 | 423 | NUM_ACC_QUEUE); |
| 412 | − if (!gt->usm.acc_wq) | |
| 424 | + if (!gt->usm.acc_wq) { | |
| 425 | + destroy_workqueue(gt->usm.pf_wq); | |
| 413 | 426 | return -ENOMEM; |
| 427 | + } | |
| 414 | 428 | |
| 415 | − return 0; | |
| 429 | + return devm_add_action_or_reset(xe->drm.dev, pagefault_fini, gt); | |
| 416 | 430 | } |
| 417 | 431 | |
| 418 | 432 | void xe_gt_pagefault_reset(struct xe_gt *gt) |
| @@ -1927,6 +1927,7 @@ static int pf_validate_vf_config(struct xe_gt *gt, unsigned int vfid) · 1926 unmodified lines | ||
| 1927 | 1927 | { |
| 1928 | 1928 | struct xe_gt *primary_gt = gt_to_tile(gt)->primary_gt; |
| 1929 | 1929 | struct xe_device *xe = gt_to_xe(gt); |
| 1930 | + bool is_primary = !xe_gt_is_media_type(gt); | |
| 1930 | 1931 | bool valid_ggtt, valid_ctxs, valid_dbs; |
| 1931 | 1932 | bool valid_any, valid_all; |
| 1932 | 1933 | |
| @@ -1935,13 +1936,17 @@ static int pf_validate_vf_config(struct xe_gt *gt, unsigned int vfid) · 2 unmodified lines | ||
| 1935 | 1936 | valid_dbs = pf_get_vf_config_dbs(gt, vfid); |
| 1936 | 1937 | |
| 1937 | 1938 | /* note that GuC doorbells are optional */ |
| 1938 | − valid_any = valid_ggtt || valid_ctxs || valid_dbs; | |
| 1939 | − valid_all = valid_ggtt && valid_ctxs; | |
| 1939 | + valid_any = valid_ctxs || valid_dbs; | |
| 1940 | + valid_all = valid_ctxs; | |
| 1941 | + | |
| 1942 | + /* and GGTT/LMEM is configured on primary GT only */ | |
| 1943 | + valid_all = valid_all && valid_ggtt; | |
| 1944 | + valid_any = valid_any || (valid_ggtt && is_primary); | |
| 1940 | 1945 | |
| 1941 | 1946 | if (IS_DGFX(xe)) { |
| 1942 | 1947 | bool valid_lmem = pf_get_vf_config_ggtt(primary_gt, vfid); |
| 1943 | 1948 | |
| 1944 | − valid_any = valid_any || valid_lmem; | |
| 1949 | + valid_any = valid_any || (valid_lmem && is_primary); | |
| 1945 | 1950 | valid_all = valid_all && valid_lmem; |
| 1946 | 1951 | } |
| 1947 | 1952 | |
| @@ -850,7 +850,7 @@ static struct vf_runtime_reg *vf_lookup_reg(struct xe_gt *gt, u32 addr) · 849 unmodified lines | ||
| 850 | 850 | |
| 851 | 851 | xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt))); |
| 852 | 852 | |
| 853 | − return bsearch(&key, runtime->regs, runtime->regs_size, sizeof(key), | |
| 853 | + return bsearch(&key, runtime->regs, runtime->num_regs, sizeof(key), | |
| 854 | 854 | vf_runtime_reg_cmp); |
| 855 | 855 | } |
| 856 | 856 | |
| @@ -13,10 +13,13 @@ · 12 unmodified lines | ||
| 13 | 13 | #include "xe_guc.h" |
| 14 | 14 | #include "xe_guc_ct.h" |
| 15 | 15 | #include "xe_mmio.h" |
| 16 | +#include "xe_pm.h" | |
| 16 | 17 | #include "xe_sriov.h" |
| 17 | 18 | #include "xe_trace.h" |
| 18 | 19 | #include "regs/xe_guc_regs.h" |
| 19 | 20 | |
| 21 | +#define FENCE_STACK_BIT DMA_FENCE_FLAG_USER_BITS | |
| 22 | + | |
| 20 | 23 | /* |
| 21 | 24 | * TLB inval depends on pending commands in the CT queue and then the real |
| 22 | 25 | * invalidation time. Double up the time to process full CT queue |
| @@ -33,6 +36,24 @@ static long tlb_timeout_jiffies(struct xe_gt *gt) · 10 unmodified lines | ||
| 33 | 36 | return hw_tlb_timeout + 2 * delay; |
| 34 | 37 | } |
| 35 | 38 | |
| 39 | +static void | |
| 40 | +__invalidation_fence_signal(struct xe_device *xe, struct xe_gt_tlb_invalidation_fence *fence) | |
| 41 | +{ | |
| 42 | + bool stack = test_bit(FENCE_STACK_BIT, &fence->base.flags); | |
| 43 | + | |
| 44 | + trace_xe_gt_tlb_invalidation_fence_signal(xe, fence); | |
| 45 | + xe_gt_tlb_invalidation_fence_fini(fence); | |
| 46 | + dma_fence_signal(&fence->base); | |
| 47 | + if (!stack) | |
| 48 | + dma_fence_put(&fence->base); | |
| 49 | +} | |
| 50 | + | |
| 51 | +static void | |
| 52 | +invalidation_fence_signal(struct xe_device *xe, struct xe_gt_tlb_invalidation_fence *fence) | |
| 53 | +{ | |
| 54 | + list_del(&fence->link); | |
| 55 | + __invalidation_fence_signal(xe, fence); | |
| 56 | +} | |
| 36 | 57 | |
| 37 | 58 | static void xe_gt_tlb_fence_timeout(struct work_struct *work) |
| 38 | 59 | { |
| @@ -54,10 +75,8 @@ static void xe_gt_tlb_fence_timeout(struct work_struct *work) · 15 unmodified lines | ||
| 54 | 75 | xe_gt_err(gt, "TLB invalidation fence timeout, seqno=%d recv=%d", |
| 55 | 76 | fence->seqno, gt->tlb_invalidation.seqno_recv); |
| 56 | 77 | |
| 57 | − list_del(&fence->link); | |
| 58 | 78 | fence->base.error = -ETIME; |
| 59 | − dma_fence_signal(&fence->base); | |
| 60 | − dma_fence_put(&fence->base); | |
| 79 | + invalidation_fence_signal(xe, fence); | |
| 61 | 80 | } |
| 62 | 81 | if (!list_empty(>->tlb_invalidation.pending_fences)) |
| 63 | 82 | queue_delayed_work(system_wq, |
| @@ -87,21 +106,6 @@ int xe_gt_tlb_invalidation_init(struct xe_gt *gt) · 23 unmodified lines | ||
| 87 | 106 | return 0; |
| 88 | 107 | } |
| 89 | 108 | |
| 90 | −static void | |
| 91 | −__invalidation_fence_signal(struct xe_device *xe, struct xe_gt_tlb_invalidation_fence *fence) | |
| 92 | −{ | |
| 93 | − trace_xe_gt_tlb_invalidation_fence_signal(xe, fence); | |
| 94 | − dma_fence_signal(&fence->base); | |
| 95 | − dma_fence_put(&fence->base); | |
| 96 | −} | |
| 97 | − | |
| 98 | −static void | |
| 99 | −invalidation_fence_signal(struct xe_device *xe, struct xe_gt_tlb_invalidation_fence *fence) | |
| 100 | −{ | |
| 101 | − list_del(&fence->link); | |
| 102 | − __invalidation_fence_signal(xe, fence); | |
| 103 | −} | |
| 104 | − | |
| 105 | 109 | /** |
| 106 | 110 | * xe_gt_tlb_invalidation_reset - Initialize GT TLB invalidation reset |
| 107 | 111 | * @gt: graphics tile |
| @@ -111,7 +115,6 @@ invalidation_fence_signal(struct xe_device *xe, struct xe_gt_tlb_invalidation_fe · 3 unmodified lines | ||
| 111 | 115 | void xe_gt_tlb_invalidation_reset(struct xe_gt *gt) |
| 112 | 116 | { |
| 113 | 117 | struct xe_gt_tlb_invalidation_fence *fence, *next; |
| 114 | − struct xe_guc *guc = >->uc.guc; | |
| 115 | 118 | int pending_seqno; |
| 116 | 119 | |
| 117 | 120 | /* |
| @@ -134,7 +137,6 @@ void xe_gt_tlb_invalidation_reset(struct xe_gt *gt) · 16 unmodified lines | ||
| 134 | 137 | else |
| 135 | 138 | pending_seqno = gt->tlb_invalidation.seqno - 1; |
| 136 | 139 | WRITE_ONCE(gt->tlb_invalidation.seqno_recv, pending_seqno); |
| 137 | − wake_up_all(&guc->ct.wq); | |
| 138 | 140 | |
| 139 | 141 | list_for_each_entry_safe(fence, next, |
| 140 | 142 | >->tlb_invalidation.pending_fences, link) |
| @@ -165,6 +167,8 @@ static int send_tlb_invalidation(struct xe_guc *guc, · 24 unmodified lines | ||
| 165 | 167 | int seqno; |
| 166 | 168 | int ret; |
| 167 | 169 | |
| 170 | + xe_gt_assert(gt, fence); | |
| 171 | + | |
| 168 | 172 | /* |
| 169 | 173 | * XXX: The seqno algorithm relies on TLB invalidation being processed |
| 170 | 174 | * in order which they currently are, if that changes the algorithm will |
| @@ -173,10 +177,8 @@ static int send_tlb_invalidation(struct xe_guc *guc, · 2 unmodified lines | ||
| 173 | 177 | |
| 174 | 178 | mutex_lock(&guc->ct.lock); |
| 175 | 179 | seqno = gt->tlb_invalidation.seqno; |
| 176 | − if (fence) { | |
| 177 | − fence->seqno = seqno; | |
| 178 | − trace_xe_gt_tlb_invalidation_fence_send(xe, fence); | |
| 179 | − } | |
| 180 | + fence->seqno = seqno; | |
| 181 | + trace_xe_gt_tlb_invalidation_fence_send(xe, fence); | |
| 180 | 182 | action[1] = seqno; |
| 181 | 183 | ret = xe_guc_ct_send_locked(&guc->ct, action, len, |
| 182 | 184 | G2H_LEN_DW_TLB_INVALIDATE, 1); |
| @@ -209,7 +211,6 @@ static int send_tlb_invalidation(struct xe_guc *guc, · 26 unmodified lines | ||
| 209 | 211 | TLB_INVALIDATION_SEQNO_MAX; |
| 210 | 212 | if (!gt->tlb_invalidation.seqno) |
| 211 | 213 | gt->tlb_invalidation.seqno = 1; |
| 212 | − ret = seqno; | |
| 213 | 214 | } |
| 214 | 215 | mutex_unlock(&guc->ct.lock); |
| 215 | 216 | |
| @@ -223,14 +224,16 @@ static int send_tlb_invalidation(struct xe_guc *guc, · 7 unmodified lines | ||
| 223 | 224 | /** |
| 224 | 225 | * xe_gt_tlb_invalidation_guc - Issue a TLB invalidation on this GT for the GuC |
| 225 | 226 | * @gt: graphics tile |
| 227 | + * @fence: invalidation fence which will be signal on TLB invalidation | |
| 228 | + * completion | |
| 226 | 229 | * |
| 227 | 230 | * Issue a TLB invalidation for the GuC. Completion of TLB is asynchronous and |
| 228 | − * caller can use seqno + xe_gt_tlb_invalidation_wait to wait for completion. | |
| 231 | + * caller can use the invalidation fence to wait for completion. | |
| 229 | 232 | * |
| 230 | − * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, | |
| 231 | − * negative error code on error. | |
| 233 | + * Return: 0 on success, negative error code on error | |
| 232 | 234 | */ |
| 233 | −static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) | |
| 235 | +static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt, | |
| 236 | + struct xe_gt_tlb_invalidation_fence *fence) | |
| 234 | 237 | { |
| 235 | 238 | u32 action[] = { |
| 236 | 239 | XE_GUC_ACTION_TLB_INVALIDATION, |
| @@ -238,7 +241,7 @@ static int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) · 1 unmodified line | ||
| 238 | 241 | MAKE_INVAL_OP(XE_GUC_TLB_INVAL_GUC), |
| 239 | 242 | }; |
| 240 | 243 | |
| 241 | − return send_tlb_invalidation(>->uc.guc, NULL, action, | |
| 244 | + return send_tlb_invalidation(>->uc.guc, fence, action, | |
| 242 | 245 | ARRAY_SIZE(action)); |
| 243 | 246 | } |
| 244 | 247 | |
| @@ -257,13 +260,17 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) · 12 unmodified lines | ||
| 257 | 260 | |
| 258 | 261 | if (xe_guc_ct_enabled(>->uc.guc.ct) && |
| 259 | 262 | gt->uc.guc.submission_state.enabled) { |
| 260 | − int seqno; | |
| 261 | − | |
| 262 | − seqno = xe_gt_tlb_invalidation_guc(gt); | |
| 263 | − if (seqno <= 0) | |
| 264 | − return seqno; | |
| 263 | + struct xe_gt_tlb_invalidation_fence fence; | |
| 264 | + int ret; | |
| 265 | + | |
| 266 | + xe_gt_tlb_invalidation_fence_init(gt, &fence, true); | |
| 267 | + ret = xe_gt_tlb_invalidation_guc(gt, &fence); | |
| 268 | + if (ret < 0) { | |
| 269 | + xe_gt_tlb_invalidation_fence_fini(&fence); | |
| 270 | + return ret; | |
| 271 | + } | |
| 265 | 272 | |
| 266 | − xe_gt_tlb_invalidation_wait(gt, seqno); | |
| 273 | + xe_gt_tlb_invalidation_fence_wait(&fence); | |
| 267 | 274 | } else if (xe_device_uc_enabled(xe) && !xe_device_wedged(xe)) { |
| 268 | 275 | if (IS_SRIOV_VF(xe)) |
| 269 | 276 | return 0; |
| @@ -290,18 +297,16 @@ int xe_gt_tlb_invalidation_ggtt(struct xe_gt *gt) · 20 unmodified lines | ||
| 290 | 297 | * |
| 291 | 298 | * @gt: graphics tile |
| 292 | 299 | * @fence: invalidation fence which will be signal on TLB invalidation |
| 293 | − * completion, can be NULL | |
| 300 | + * completion | |
| 294 | 301 | * @start: start address |
| 295 | 302 | * @end: end address |
| 296 | 303 | * @asid: address space id |
| 297 | 304 | * |
| 298 | 305 | * Issue a range based TLB invalidation if supported, if not fallback to a full |
| 299 | − * TLB invalidation. Completion of TLB is asynchronous and caller can either use | |
| 300 | − * the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for | |
| 301 | − * completion. | |
| 306 | + * TLB invalidation. Completion of TLB is asynchronous and caller can use | |
| 307 | + * the invalidation fence to wait for completion. | |
| 302 | 308 | * |
| 303 | − * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, | |
| 304 | − * negative error code on error. | |
| 309 | + * Return: Negative error code on error, 0 on success | |
| 305 | 310 | */ |
| 306 | 311 | int xe_gt_tlb_invalidation_range(struct xe_gt *gt, |
| 307 | 312 | struct xe_gt_tlb_invalidation_fence *fence, |
| @@ -312,11 +317,11 @@ int xe_gt_tlb_invalidation_range(struct xe_gt *gt, · 4 unmodified lines | ||
| 312 | 317 | u32 action[MAX_TLB_INVALIDATION_LEN]; |
| 313 | 318 | int len = 0; |
| 314 | 319 | |
| 320 | + xe_gt_assert(gt, fence); | |
| 321 | + | |
| 315 | 322 | /* Execlists not supported */ |
| 316 | 323 | if (gt_to_xe(gt)->info.force_execlist) { |
| 317 | − if (fence) | |
| 318 | − __invalidation_fence_signal(xe, fence); | |
| 319 | − | |
| 324 | + __invalidation_fence_signal(xe, fence); | |
| 320 | 325 | return 0; |
| 321 | 326 | } |
| 322 | 327 | |
| @@ -382,12 +387,10 @@ int xe_gt_tlb_invalidation_range(struct xe_gt *gt, · 59 unmodified lines | ||
| 382 | 387 | * @vma: VMA to invalidate |
| 383 | 388 | * |
| 384 | 389 | * Issue a range based TLB invalidation if supported, if not fallback to a full |
| 385 | − * TLB invalidation. Completion of TLB is asynchronous and caller can either use | |
| 386 | − * the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for | |
| 387 | − * completion. | |
| 390 | + * TLB invalidation. Completion of TLB is asynchronous and caller can use | |
| 391 | + * the invalidation fence to wait for completion. | |
| 388 | 392 | * |
| 389 | − * Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, | |
| 390 | − * negative error code on error. | |
| 393 | + * Return: Negative error code on error, 0 on success | |
| 391 | 394 | */ |
| 392 | 395 | int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, |
| 393 | 396 | struct xe_gt_tlb_invalidation_fence *fence, |
| @@ -400,43 +403,6 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, · 6 unmodified lines | ||
| 400 | 403 | xe_vma_vm(vma)->usm.asid); |
| 401 | 404 | } |
| 402 | 405 | |
| 403 | −/** | |
| 404 | − * xe_gt_tlb_invalidation_wait - Wait for TLB to complete | |
| 405 | − * @gt: graphics tile | |
| 406 | − * @seqno: seqno to wait which was returned from xe_gt_tlb_invalidation | |
| 407 | − * | |
| 408 | − * Wait for tlb_timeout_jiffies() for a TLB invalidation to complete. | |
| 409 | − * | |
| 410 | − * Return: 0 on success, -ETIME on TLB invalidation timeout | |
| 411 | − */ | |
| 412 | −int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno) | |
| 413 | −{ | |
| 414 | − struct xe_guc *guc = >->uc.guc; | |
| 415 | − int ret; | |
| 416 | − | |
| 417 | − /* Execlists not supported */ | |
| 418 | − if (gt_to_xe(gt)->info.force_execlist) | |
| 419 | − return 0; | |
| 420 | − | |
| 421 | − /* | |
| 422 | − * XXX: See above, this algorithm only works if seqno are always in | |
| 423 | − * order | |
| 424 | − */ | |
| 425 | − ret = wait_event_timeout(guc->ct.wq, | |
| 426 | − tlb_invalidation_seqno_past(gt, seqno), | |
| 427 | − tlb_timeout_jiffies(gt)); | |
| 428 | − if (!ret) { | |
| 429 | − struct drm_printer p = xe_gt_err_printer(gt); | |
| 430 | − | |
| 431 | − xe_gt_err(gt, "TLB invalidation time'd out, seqno=%d, recv=%d\n", | |
| 432 | − seqno, gt->tlb_invalidation.seqno_recv); | |
| 433 | − xe_guc_ct_print(&guc->ct, &p, true); | |
| 434 | − return -ETIME; | |
| 435 | − } | |
| 436 | − | |
| 437 | − return 0; | |
| 438 | −} | |
| 439 | − | |
| 440 | 406 | /** |
| 441 | 407 | * xe_guc_tlb_invalidation_done_handler - TLB invalidation done handler |
| 442 | 408 | * @guc: guc |
| @@ -480,12 +446,7 @@ int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) · 37 unmodified lines | ||
| 480 | 446 | return 0; |
| 481 | 447 | } |
| 482 | 448 | |
| 483 | − /* | |
| 484 | − * wake_up_all() and wait_event_timeout() already have the correct | |
| 485 | − * barriers. | |
| 486 | − */ | |
| 487 | 449 | WRITE_ONCE(gt->tlb_invalidation.seqno_recv, msg[0]); |
| 488 | − wake_up_all(&guc->ct.wq); | |
| 489 | 450 | |
| 490 | 451 | list_for_each_entry_safe(fence, next, |
| 491 | 452 | >->tlb_invalidation.pending_fences, link) { |
| @@ -508,3 +469,59 @@ int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len) · 16 unmodified lines | ||
| 508 | 469 | |
| 509 | 470 | return 0; |
| 510 | 471 | } |
| 472 | + | |
| 473 | +static const char * | |
| 474 | +invalidation_fence_get_driver_name(struct dma_fence *dma_fence) | |
| 475 | +{ | |
| 476 | + return "xe"; | |
| 477 | +} | |
| 478 | + | |
| 479 | +static const char * | |
| 480 | +invalidation_fence_get_timeline_name(struct dma_fence *dma_fence) | |
| 481 | +{ | |
| 482 | + return "invalidation_fence"; | |
| 483 | +} | |
| 484 | + | |
| 485 | +static const struct dma_fence_ops invalidation_fence_ops = { | |
| 486 | + .get_driver_name = invalidation_fence_get_driver_name, | |
| 487 | + .get_timeline_name = invalidation_fence_get_timeline_name, | |
| 488 | +}; | |
| 489 | + | |
| 490 | +/** | |
| 491 | + * xe_gt_tlb_invalidation_fence_init - Initialize TLB invalidation fence | |
| 492 | + * @gt: GT | |
| 493 | + * @fence: TLB invalidation fence to initialize | |
| 494 | + * @stack: fence is stack variable | |
| 495 | + * | |
| 496 | + * Initialize TLB invalidation fence for use. xe_gt_tlb_invalidation_fence_fini | |
| 497 | + * must be called if fence is not signaled. | |
| 498 | + */ | |
| 499 | +void xe_gt_tlb_invalidation_fence_init(struct xe_gt *gt, | |
| 500 | + struct xe_gt_tlb_invalidation_fence *fence, | |
| 501 | + bool stack) | |
| 502 | +{ | |
| 503 | + xe_pm_runtime_get_noresume(gt_to_xe(gt)); | |
| 504 | + | |
| 505 | + spin_lock_irq(>->tlb_invalidation.lock); | |
| 506 | + dma_fence_init(&fence->base, &invalidation_fence_ops, | |
| 507 | + >->tlb_invalidation.lock, | |
| 508 | + dma_fence_context_alloc(1), 1); | |
| 509 | + spin_unlock_irq(>->tlb_invalidation.lock); | |
| 510 | + INIT_LIST_HEAD(&fence->link); | |
| 511 | + if (stack) | |
| 512 | + set_bit(FENCE_STACK_BIT, &fence->base.flags); | |
| 513 | + else | |
| 514 | + dma_fence_get(&fence->base); | |
| 515 | + fence->gt = gt; | |
| 516 | +} | |
| 517 | + | |
| 518 | +/** | |
| 519 | + * xe_gt_tlb_invalidation_fence_fini - Finalize TLB invalidation fence | |
| 520 | + * @fence: TLB invalidation fence to finalize | |
| 521 | + * | |
| 522 | + * Drop PM ref which fence took durinig init. | |
| 523 | + */ | |
| 524 | +void xe_gt_tlb_invalidation_fence_fini(struct xe_gt_tlb_invalidation_fence *fence) | |
| 525 | +{ | |
| 526 | + xe_pm_runtime_put(gt_to_xe(fence->gt)); | |
| 527 | +} | |
| @@ -23,7 +23,17 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt, · 22 unmodified lines | ||
| 23 | 23 | int xe_gt_tlb_invalidation_range(struct xe_gt *gt, |
| 24 | 24 | struct xe_gt_tlb_invalidation_fence *fence, |
| 25 | 25 | u64 start, u64 end, u32 asid); |
| 26 | −int xe_gt_tlb_invalidation_wait(struct xe_gt *gt, int seqno); | |
| 27 | 26 | int xe_guc_tlb_invalidation_done_handler(struct xe_guc *guc, u32 *msg, u32 len); |
| 28 | 27 | |
| 28 | +void xe_gt_tlb_invalidation_fence_init(struct xe_gt *gt, | |
| 29 | + struct xe_gt_tlb_invalidation_fence *fence, | |
| 30 | + bool stack); | |
| 31 | +void xe_gt_tlb_invalidation_fence_fini(struct xe_gt_tlb_invalidation_fence *fence); | |
| 32 | + | |
| 33 | +static inline void | |
| 34 | +xe_gt_tlb_invalidation_fence_wait(struct xe_gt_tlb_invalidation_fence *fence) | |
| 35 | +{ | |
| 36 | + dma_fence_wait(&fence->base, false); | |
| 37 | +} | |
| 38 | + | |
| 29 | 39 | #endif /* _XE_GT_TLB_INVALIDATION_ */ |
| @@ -8,6 +8,8 @@ · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | #include <linux/dma-fence.h> |
| 10 | 10 | |
| 11 | +struct xe_gt; | |
| 12 | + | |
| 11 | 13 | /** |
| 12 | 14 | * struct xe_gt_tlb_invalidation_fence - XE GT TLB invalidation fence |
| 13 | 15 | * |
| @@ -17,6 +19,8 @@ · 3 unmodified lines | ||
| 17 | 19 | struct xe_gt_tlb_invalidation_fence { |
| 18 | 20 | /** @base: dma fence base */ |
| 19 | 21 | struct dma_fence base; |
| 22 | + /** @gt: GT which fence belong to */ | |
| 23 | + struct xe_gt *gt; | |
| 20 | 24 | /** @link: link into list of pending tlb fences */ |
| 21 | 25 | struct list_head link; |
| 22 | 26 | /** @seqno: seqno of TLB invalidation to signal fence one */ |
| @@ -362,6 +362,12 @@ struct xe_gt { · 361 unmodified lines | ||
| 362 | 362 | */ |
| 363 | 363 | spinlock_t mcr_lock; |
| 364 | 364 | |
| 365 | + /** | |
| 366 | + * @global_invl_lock: protects the register for the duration | |
| 367 | + * of a global invalidation of l2 cache | |
| 368 | + */ | |
| 369 | + spinlock_t global_invl_lock; | |
| 370 | + | |
| 365 | 371 | /** @wa_active: keep track of active workarounds */ |
| 366 | 372 | struct { |
| 367 | 373 | /** @wa_active.gt: bitmap with active GT workarounds */ |
| @@ -327,6 +327,8 @@ static void xe_guc_ct_set_state(struct xe_guc_ct *ct, · 326 unmodified lines | ||
| 327 | 327 | xe_gt_assert(ct_to_gt(ct), ct->g2h_outstanding == 0 || |
| 328 | 328 | state == XE_GUC_CT_STATE_STOPPED); |
| 329 | 329 | |
| 330 | + if (ct->g2h_outstanding) | |
| 331 | + xe_pm_runtime_put(ct_to_xe(ct)); | |
| 330 | 332 | ct->g2h_outstanding = 0; |
| 331 | 333 | ct->state = state; |
| 332 | 334 | |
| @@ -495,10 +497,15 @@ static void h2g_reserve_space(struct xe_guc_ct *ct, u32 cmd_len) · 162 unmodified lines | ||
| 495 | 497 | static void __g2h_reserve_space(struct xe_guc_ct *ct, u32 g2h_len, u32 num_g2h) |
| 496 | 498 | { |
| 497 | 499 | xe_gt_assert(ct_to_gt(ct), g2h_len <= ct->ctbs.g2h.info.space); |
| 500 | + xe_gt_assert(ct_to_gt(ct), (!g2h_len && !num_g2h) || | |
| 501 | + (g2h_len && num_g2h)); | |
| 498 | 502 | |
| 499 | 503 | if (g2h_len) { |
| 500 | 504 | lockdep_assert_held(&ct->fast_lock); |
| 501 | 505 | |
| 506 | + if (!ct->g2h_outstanding) | |
| 507 | + xe_pm_runtime_get_noresume(ct_to_xe(ct)); | |
| 508 | + | |
| 502 | 509 | ct->ctbs.g2h.info.space -= g2h_len; |
| 503 | 510 | ct->g2h_outstanding += num_g2h; |
| 504 | 511 | } |
| @@ -511,7 +518,8 @@ static void __g2h_release_space(struct xe_guc_ct *ct, u32 g2h_len) · 6 unmodified lines | ||
| 511 | 518 | ct->ctbs.g2h.info.size - ct->ctbs.g2h.info.resv_space); |
| 512 | 519 | |
| 513 | 520 | ct->ctbs.g2h.info.space += g2h_len; |
| 514 | − --ct->g2h_outstanding; | |
| 521 | + if (!--ct->g2h_outstanding) | |
| 522 | + xe_pm_runtime_put(ct_to_xe(ct)); | |
| 515 | 523 | } |
| 516 | 524 | |
| 517 | 525 | static void g2h_release_space(struct xe_guc_ct *ct, u32 g2h_len) |
| @@ -284,7 +284,7 @@ static void guc_submit_fini(struct drm_device *drm, void *arg) · 283 unmodified lines | ||
| 284 | 284 | free_submit_wq(guc); |
| 285 | 285 | } |
| 286 | 286 | |
| 287 | −static void guc_submit_wedged_fini(struct drm_device *drm, void *arg) | |
| 287 | +static void guc_submit_wedged_fini(void *arg) | |
| 288 | 288 | { |
| 289 | 289 | struct xe_guc *guc = arg; |
| 290 | 290 | struct xe_exec_queue *q; |
| @@ -877,7 +877,7 @@ void xe_guc_submit_wedge(struct xe_guc *guc) · 586 unmodified lines | ||
| 877 | 877 | |
| 878 | 878 | xe_gt_assert(guc_to_gt(guc), guc_to_xe(guc)->wedged.mode); |
| 879 | 879 | |
| 880 | − err = drmm_add_action_or_reset(&guc_to_xe(guc)->drm, | |
| 880 | + err = devm_add_action_or_reset(guc_to_xe(guc)->drm.dev, | |
| 881 | 881 | guc_submit_wedged_fini, guc); |
| 882 | 882 | if (err) { |
| 883 | 883 | drm_err(&xe->drm, "Failed to register xe_guc_submit clean-up on wedged.mode=2. Although device is wedged.\n"); |
| @@ -1393,6 +1393,8 @@ static void guc_exec_queue_process_msg(struct xe_sched_msg *msg) · 509 unmodified lines | ||
| 1393 | 1393 | default: |
| 1394 | 1394 | XE_WARN_ON("Unknown message type"); |
| 1395 | 1395 | } |
| 1396 | + | |
| 1397 | + xe_pm_runtime_put(guc_to_xe(exec_queue_to_guc(msg->private_data))); | |
| 1396 | 1398 | } |
| 1397 | 1399 | |
| 1398 | 1400 | static const struct drm_sched_backend_ops drm_sched_ops = { |
| @@ -1482,6 +1484,8 @@ static void guc_exec_queue_kill(struct xe_exec_queue *q) · 83 unmodified lines | ||
| 1482 | 1484 | static void guc_exec_queue_add_msg(struct xe_exec_queue *q, struct xe_sched_msg *msg, |
| 1483 | 1485 | u32 opcode) |
| 1484 | 1486 | { |
| 1487 | + xe_pm_runtime_get_noresume(guc_to_xe(exec_queue_to_guc(q))); | |
| 1488 | + | |
| 1485 | 1489 | INIT_LIST_HEAD(&msg->link); |
| 1486 | 1490 | msg->opcode = opcode; |
| 1487 | 1491 | msg->private_data = q; |
| @@ -148,20 +148,20 @@ static const char *xe_hw_fence_get_driver_name(struct dma_fence *dma_fence) · 147 unmodified lines | ||
| 148 | 148 | { |
| 149 | 149 | struct xe_hw_fence *fence = to_xe_hw_fence(dma_fence); |
| 150 | 150 | |
| 151 | − return dev_name(gt_to_xe(fence->ctx->gt)->drm.dev); | |
| 151 | + return dev_name(fence->xe->drm.dev); | |
| 152 | 152 | } |
| 153 | 153 | |
| 154 | 154 | static const char *xe_hw_fence_get_timeline_name(struct dma_fence *dma_fence) |
| 155 | 155 | { |
| 156 | 156 | struct xe_hw_fence *fence = to_xe_hw_fence(dma_fence); |
| 157 | 157 | |
| 158 | − return fence->ctx->name; | |
| 158 | + return fence->name; | |
| 159 | 159 | } |
| 160 | 160 | |
| 161 | 161 | static bool xe_hw_fence_signaled(struct dma_fence *dma_fence) |
| 162 | 162 | { |
| 163 | 163 | struct xe_hw_fence *fence = to_xe_hw_fence(dma_fence); |
| 164 | − struct xe_device *xe = gt_to_xe(fence->ctx->gt); | |
| 164 | + struct xe_device *xe = fence->xe; | |
| 165 | 165 | u32 seqno = xe_map_rd(xe, &fence->seqno_map, 0, u32); |
| 166 | 166 | |
| 167 | 167 | return dma_fence->error || |
| @@ -253,7 +253,8 @@ void xe_hw_fence_init(struct dma_fence *fence, struct xe_hw_fence_ctx *ctx, · 85 unmodified lines | ||
| 253 | 253 | struct xe_hw_fence *hw_fence = |
| 254 | 254 | container_of(fence, typeof(*hw_fence), dma); |
| 255 | 255 | |
| 256 | − hw_fence->ctx = ctx; | |
| 256 | + hw_fence->xe = gt_to_xe(ctx->gt); | |
| 257 | + snprintf(hw_fence->name, sizeof(hw_fence->name), "%s", ctx->name); | |
| 257 | 258 | hw_fence->seqno_map = seqno_map; |
| 258 | 259 | INIT_LIST_HEAD(&hw_fence->irq_link); |
| 259 | 260 | |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <linux/list.h> |
| 13 | 13 | #include <linux/spinlock.h> |
| 14 | 14 | |
| 15 | +struct xe_device; | |
| 15 | 16 | struct xe_gt; |
| 16 | 17 | |
| 17 | 18 | /** |
| @@ -61,8 +62,10 @@ struct xe_hw_fence_ctx { · 43 unmodified lines | ||
| 61 | 62 | struct xe_hw_fence { |
| 62 | 63 | /** @dma: base dma fence for hardware fence context */ |
| 63 | 64 | struct dma_fence dma; |
| 64 | − /** @ctx: hardware fence context */ | |
| 65 | − struct xe_hw_fence_ctx *ctx; | |
| 65 | + /** @xe: Xe device for hw fence driver name */ | |
| 66 | + struct xe_device *xe; | |
| 67 | + /** @name: name of hardware fence context */ | |
| 68 | + char name[MAX_FENCE_NAME_LEN]; | |
| 66 | 69 | /** @seqno_map: I/O map for seqno */ |
| 67 | 70 | struct iosys_map seqno_map; |
| 68 | 71 | /** @irq_link: Link in struct xe_hw_fence_irq.pending */ |
| @@ -203,9 +203,10 @@ static int xe_hwmon_power_max_write(struct xe_hwmon *hwmon, int channel, long va · 202 unmodified lines | ||
| 203 | 203 | reg_val = xe_mmio_rmw32(hwmon->gt, rapl_limit, PKG_PWR_LIM_1_EN, 0); |
| 204 | 204 | reg_val = xe_mmio_read32(hwmon->gt, rapl_limit); |
| 205 | 205 | if (reg_val & PKG_PWR_LIM_1_EN) { |
| 206 | + drm_warn(>_to_xe(hwmon->gt)->drm, "PL1 disable is not supported!\n"); | |
| 206 | 207 | ret = -EOPNOTSUPP; |
| 207 | − goto unlock; | |
| 208 | 208 | } |
| 209 | + goto unlock; | |
| 209 | 210 | } |
| 210 | 211 | |
| 211 | 212 | /* Computation in 64-bits to avoid overflow. Round to nearest. */ |
| @@ -449,7 +450,7 @@ static int xe_hwmon_pcode_write_i1(struct xe_gt *gt, u32 uval) · 237 unmodified lines | ||
| 449 | 450 | { |
| 450 | 451 | return xe_pcode_write(gt, PCODE_MBOX(PCODE_POWER_SETUP, |
| 451 | 452 | POWER_SETUP_SUBCOMMAND_WRITE_I1, 0), |
| 452 | − uval); | |
| 453 | + (uval & POWER_SETUP_I1_DATA_MASK)); | |
| 453 | 454 | } |
| 454 | 455 | |
| 455 | 456 | static int xe_hwmon_power_curr_crit_read(struct xe_hwmon *hwmon, int channel, |
| @@ -1634,6 +1634,9 @@ struct xe_lrc_snapshot *xe_lrc_snapshot_capture(struct xe_lrc *lrc) · 1633 unmodified lines | ||
| 1634 | 1634 | if (!snapshot) |
| 1635 | 1635 | return NULL; |
| 1636 | 1636 | |
| 1637 | + if (lrc->bo && lrc->bo->vm) | |
| 1638 | + xe_vm_get(lrc->bo->vm); | |
| 1639 | + | |
| 1637 | 1640 | snapshot->context_desc = xe_lrc_ggtt_addr(lrc); |
| 1638 | 1641 | snapshot->indirect_context_desc = xe_lrc_indirect_ring_ggtt_addr(lrc); |
| 1639 | 1642 | snapshot->head = xe_lrc_ring_head(lrc); |
| @@ -1653,12 +1656,14 @@ struct xe_lrc_snapshot *xe_lrc_snapshot_capture(struct xe_lrc *lrc) · 13 unmodified lines | ||
| 1653 | 1656 | void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot) |
| 1654 | 1657 | { |
| 1655 | 1658 | struct xe_bo *bo; |
| 1659 | + struct xe_vm *vm; | |
| 1656 | 1660 | struct iosys_map src; |
| 1657 | 1661 | |
| 1658 | 1662 | if (!snapshot) |
| 1659 | 1663 | return; |
| 1660 | 1664 | |
| 1661 | 1665 | bo = snapshot->lrc_bo; |
| 1666 | + vm = bo->vm; | |
| 1662 | 1667 | snapshot->lrc_bo = NULL; |
| 1663 | 1668 | |
| 1664 | 1669 | snapshot->lrc_snapshot = kvmalloc(snapshot->lrc_size, GFP_KERNEL); |
| @@ -1678,6 +1683,8 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot) · 13 unmodified lines | ||
| 1678 | 1683 | xe_bo_unlock(bo); |
| 1679 | 1684 | put_bo: |
| 1680 | 1685 | xe_bo_put(bo); |
| 1686 | + if (vm) | |
| 1687 | + xe_vm_put(vm); | |
| 1681 | 1688 | } |
| 1682 | 1689 | |
| 1683 | 1690 | void xe_lrc_snapshot_print(struct xe_lrc_snapshot *snapshot, struct drm_printer *p) |
| @@ -1727,8 +1734,14 @@ void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot) · 43 unmodified lines | ||
| 1727 | 1734 | return; |
| 1728 | 1735 | |
| 1729 | 1736 | kvfree(snapshot->lrc_snapshot); |
| 1730 | − if (snapshot->lrc_bo) | |
| 1737 | + if (snapshot->lrc_bo) { | |
| 1738 | + struct xe_vm *vm; | |
| 1739 | + | |
| 1740 | + vm = snapshot->lrc_bo->vm; | |
| 1731 | 1741 | xe_bo_put(snapshot->lrc_bo); |
| 1742 | + if (vm) | |
| 1743 | + xe_vm_put(vm); | |
| 1744 | + } | |
| 1732 | 1745 | kfree(snapshot); |
| 1733 | 1746 | } |
| 1734 | 1747 | |
| @@ -30,7 +30,8 @@ static void tiles_fini(void *arg) · 29 unmodified lines | ||
| 30 | 30 | int id; |
| 31 | 31 | |
| 32 | 32 | for_each_tile(tile, xe, id) |
| 33 | − tile->mmio.regs = NULL; | |
| 33 | + if (tile != xe_device_get_root_tile(xe)) | |
| 34 | + tile->mmio.regs = NULL; | |
| 34 | 35 | } |
| 35 | 36 | |
| 36 | 37 | int xe_mmio_probe_tiles(struct xe_device *xe) |
| @@ -91,9 +92,11 @@ add_mmio_ext: · 54 unmodified lines | ||
| 91 | 92 | static void mmio_fini(void *arg) |
| 92 | 93 | { |
| 93 | 94 | struct xe_device *xe = arg; |
| 95 | + struct xe_tile *root_tile = xe_device_get_root_tile(xe); | |
| 94 | 96 | |
| 95 | 97 | pci_iounmap(to_pci_dev(xe->drm.dev), xe->mmio.regs); |
| 96 | 98 | xe->mmio.regs = NULL; |
| 99 | + root_tile->mmio.regs = NULL; | |
| 97 | 100 | } |
| 98 | 101 | |
| 99 | 102 | int xe_mmio_init(struct xe_device *xe) |
| @@ -121,12 +124,29 @@ int xe_mmio_init(struct xe_device *xe) · 21 unmodified lines | ||
| 121 | 124 | return devm_add_action_or_reset(xe->drm.dev, mmio_fini, xe); |
| 122 | 125 | } |
| 123 | 126 | |
| 127 | +static void mmio_flush_pending_writes(struct xe_gt *gt) | |
| 128 | +{ | |
| 129 | +#define DUMMY_REG_OFFSET 0x130030 | |
| 130 | + struct xe_tile *tile = gt_to_tile(gt); | |
| 131 | + int i; | |
| 132 | + | |
| 133 | + if (tile->xe->info.platform != XE_LUNARLAKE) | |
| 134 | + return; | |
| 135 | + | |
| 136 | + /* 4 dummy writes */ | |
| 137 | + for (i = 0; i < 4; i++) | |
| 138 | + writel(0, tile->mmio.regs + DUMMY_REG_OFFSET); | |
| 139 | +} | |
| 140 | + | |
| 124 | 141 | u8 xe_mmio_read8(struct xe_gt *gt, struct xe_reg reg) |
| 125 | 142 | { |
| 126 | 143 | struct xe_tile *tile = gt_to_tile(gt); |
| 127 | 144 | u32 addr = xe_mmio_adjusted_addr(gt, reg.addr); |
| 128 | 145 | u8 val; |
| 129 | 146 | |
| 147 | + /* Wa_15015404425 */ | |
| 148 | + mmio_flush_pending_writes(gt); | |
| 149 | + | |
| 130 | 150 | val = readb((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr); |
| 131 | 151 | trace_xe_reg_rw(gt, false, addr, val, sizeof(val)); |
| 132 | 152 | |
| @@ -139,6 +159,9 @@ u16 xe_mmio_read16(struct xe_gt *gt, struct xe_reg reg) · 6 unmodified lines | ||
| 139 | 159 | u32 addr = xe_mmio_adjusted_addr(gt, reg.addr); |
| 140 | 160 | u16 val; |
| 141 | 161 | |
| 162 | + /* Wa_15015404425 */ | |
| 163 | + mmio_flush_pending_writes(gt); | |
| 164 | + | |
| 142 | 165 | val = readw((reg.ext ? tile->mmio_ext.regs : tile->mmio.regs) + addr); |
| 143 | 166 | trace_xe_reg_rw(gt, false, addr, val, sizeof(val)); |
| 144 | 167 | |
| @@ -160,6 +183,9 @@ u32 xe_mmio_read32(struct xe_gt *gt, struct xe_reg reg) · 15 unmodified lines | ||
| 160 | 183 | u32 addr = xe_mmio_adjusted_addr(gt, reg.addr); |
| 161 | 184 | u32 val; |
| 162 | 185 | |
| 186 | + /* Wa_15015404425 */ | |
| 187 | + mmio_flush_pending_writes(gt); | |
| 188 | + | |
| 163 | 189 | if (!reg.vf && IS_SRIOV_VF(gt_to_xe(gt))) |
| 164 | 190 | val = xe_gt_sriov_vf_read32(gt, reg); |
| 165 | 191 | else |
| @@ -66,7 +66,6 @@ static struct ctl_table observation_ctl_table[] = { · 65 unmodified lines | ||
| 66 | 66 | .extra1 = SYSCTL_ZERO, |
| 67 | 67 | .extra2 = SYSCTL_ONE, |
| 68 | 68 | }, |
| 69 | − {} | |
| 70 | 69 | }; |
| 71 | 70 | |
| 72 | 71 | /** |
| @@ -7,6 +7,8 @@ · 6 unmodified lines | ||
| 7 | 7 | |
| 8 | 8 | #include <drm/xe_drm.h> |
| 9 | 9 | |
| 10 | +#include <generated/xe_wa_oob.h> | |
| 11 | + | |
| 10 | 12 | #include "regs/xe_reg_defs.h" |
| 11 | 13 | #include "xe_assert.h" |
| 12 | 14 | #include "xe_device.h" |
| @@ -15,6 +17,7 @@ · 2 unmodified lines | ||
| 15 | 17 | #include "xe_gt_mcr.h" |
| 16 | 18 | #include "xe_mmio.h" |
| 17 | 19 | #include "xe_sriov.h" |
| 20 | +#include "xe_wa.h" | |
| 18 | 21 | |
| 19 | 22 | #define _PAT_ATS 0x47fc |
| 20 | 23 | #define _PAT_INDEX(index) _PICK_EVEN_2RANGES(index, 8, \ |
| @@ -382,7 +385,13 @@ void xe_pat_init_early(struct xe_device *xe) · 361 unmodified lines | ||
| 382 | 385 | if (GRAPHICS_VER(xe) == 20) { |
| 383 | 386 | xe->pat.ops = &xe2_pat_ops; |
| 384 | 387 | xe->pat.table = xe2_pat_table; |
| 385 | − xe->pat.n_entries = ARRAY_SIZE(xe2_pat_table); | |
| 388 | + | |
| 389 | + /* Wa_16023588340. XXX: Should use XE_WA */ | |
| 390 | + if (GRAPHICS_VERx100(xe) == 2001) | |
| 391 | + xe->pat.n_entries = 28; /* Disable CLOS3 */ | |
| 392 | + else | |
| 393 | + xe->pat.n_entries = ARRAY_SIZE(xe2_pat_table); | |
| 394 | + | |
| 386 | 395 | xe->pat.idx[XE_CACHE_NONE] = 3; |
| 387 | 396 | xe->pat.idx[XE_CACHE_WT] = 15; |
| 388 | 397 | xe->pat.idx[XE_CACHE_WB] = 2; |
| @@ -91,13 +91,13 @@ int xe_pm_suspend(struct xe_device *xe) · 90 unmodified lines | ||
| 91 | 91 | for_each_gt(gt, xe, id) |
| 92 | 92 | xe_gt_suspend_prepare(gt); |
| 93 | 93 | |
| 94 | + xe_display_pm_suspend(xe, false); | |
| 95 | + | |
| 94 | 96 | /* FIXME: Super racey... */ |
| 95 | 97 | err = xe_bo_evict_all(xe); |
| 96 | 98 | if (err) |
| 97 | 99 | goto err; |
| 98 | 100 | |
| 99 | − xe_display_pm_suspend(xe, false); | |
| 100 | − | |
| 101 | 101 | for_each_gt(gt, xe, id) { |
| 102 | 102 | err = xe_gt_suspend(gt); |
| 103 | 103 | if (err) { |
| @@ -151,11 +151,11 @@ int xe_pm_resume(struct xe_device *xe) · 47 unmodified lines | ||
| 151 | 151 | |
| 152 | 152 | xe_irq_resume(xe); |
| 153 | 153 | |
| 154 | − xe_display_pm_resume(xe, false); | |
| 155 | − | |
| 156 | 154 | for_each_gt(gt, xe, id) |
| 157 | 155 | xe_gt_resume(gt); |
| 158 | 156 | |
| 157 | + xe_display_pm_resume(xe, false); | |
| 158 | + | |
| 159 | 159 | err = xe_bo_restore_user(xe); |
| 160 | 160 | if (err) |
| 161 | 161 | goto err; |
| @@ -363,10 +363,11 @@ int xe_pm_runtime_suspend(struct xe_device *xe) · 201 unmodified lines | ||
| 363 | 363 | mutex_unlock(&xe->mem_access.vram_userfault.lock); |
| 364 | 364 | |
| 365 | 365 | if (xe->d3cold.allowed) { |
| 366 | + xe_display_pm_suspend(xe, true); | |
| 367 | + | |
| 366 | 368 | err = xe_bo_evict_all(xe); |
| 367 | 369 | if (err) |
| 368 | 370 | goto out; |
| 369 | − xe_display_pm_suspend(xe, true); | |
| 370 | 371 | } |
| 371 | 372 | |
| 372 | 373 | for_each_gt(gt, xe, id) { |
| @@ -128,8 +128,9 @@ xe_preempt_fence_arm(struct xe_preempt_fence *pfence, struct xe_exec_queue *q, · 127 unmodified lines | ||
| 128 | 128 | { |
| 129 | 129 | list_del_init(&pfence->link); |
| 130 | 130 | pfence->q = xe_exec_queue_get(q); |
| 131 | + spin_lock_init(&pfence->lock); | |
| 131 | 132 | dma_fence_init(&pfence->base, &preempt_fence_ops, |
| 132 | − &q->lr.lock, context, seqno); | |
| 133 | + &pfence->lock, context, seqno); | |
| 133 | 134 | |
| 134 | 135 | return &pfence->base; |
| 135 | 136 | } |
| @@ -25,6 +25,8 @@ struct xe_preempt_fence { · 24 unmodified lines | ||
| 25 | 25 | struct xe_exec_queue *q; |
| 26 | 26 | /** @preempt_work: work struct which issues preemption */ |
| 27 | 27 | struct work_struct preempt_work; |
| 28 | + /** @lock: dma-fence fence lock */ | |
| 29 | + spinlock_t lock; | |
| 28 | 30 | /** @error: preempt fence is in error state */ |
| 29 | 31 | int error; |
| 30 | 32 | }; |
| @@ -1115,23 +1115,6 @@ struct invalidation_fence { · 1114 unmodified lines | ||
| 1115 | 1115 | u32 asid; |
| 1116 | 1116 | }; |
| 1117 | 1117 | |
| 1118 | −static const char * | |
| 1119 | −invalidation_fence_get_driver_name(struct dma_fence *dma_fence) | |
| 1120 | −{ | |
| 1121 | − return "xe"; | |
| 1122 | −} | |
| 1123 | − | |
| 1124 | −static const char * | |
| 1125 | −invalidation_fence_get_timeline_name(struct dma_fence *dma_fence) | |
| 1126 | −{ | |
| 1127 | − return "invalidation_fence"; | |
| 1128 | −} | |
| 1129 | − | |
| 1130 | −static const struct dma_fence_ops invalidation_fence_ops = { | |
| 1131 | − .get_driver_name = invalidation_fence_get_driver_name, | |
| 1132 | − .get_timeline_name = invalidation_fence_get_timeline_name, | |
| 1133 | −}; | |
| 1134 | − | |
| 1135 | 1118 | static void invalidation_fence_cb(struct dma_fence *fence, |
| 1136 | 1119 | struct dma_fence_cb *cb) |
| 1137 | 1120 | { |
| @@ -1170,15 +1153,8 @@ static int invalidation_fence_init(struct xe_gt *gt, · 32 unmodified lines | ||
| 1170 | 1153 | |
| 1171 | 1154 | trace_xe_gt_tlb_invalidation_fence_create(gt_to_xe(gt), &ifence->base); |
| 1172 | 1155 | |
| 1173 | − spin_lock_irq(>->tlb_invalidation.lock); | |
| 1174 | − dma_fence_init(&ifence->base.base, &invalidation_fence_ops, | |
| 1175 | − >->tlb_invalidation.lock, | |
| 1176 | − dma_fence_context_alloc(1), 1); | |
| 1177 | − spin_unlock_irq(>->tlb_invalidation.lock); | |
| 1178 | − | |
| 1179 | − INIT_LIST_HEAD(&ifence->base.link); | |
| 1156 | + xe_gt_tlb_invalidation_fence_init(gt, &ifence->base, false); | |
| 1180 | 1157 | |
| 1181 | − dma_fence_get(&ifence->base.base); /* Ref for caller */ | |
| 1182 | 1158 | ifence->fence = fence; |
| 1183 | 1159 | ifence->gt = gt; |
| 1184 | 1160 | ifence->start = start; |
| @@ -231,7 +231,7 @@ static void rtp_mark_active(struct xe_device *xe, · 230 unmodified lines | ||
| 231 | 231 | if (first == last) |
| 232 | 232 | bitmap_set(ctx->active_entries, first, 1); |
| 233 | 233 | else |
| 234 | − bitmap_set(ctx->active_entries, first, last - first + 2); | |
| 234 | + bitmap_set(ctx->active_entries, first, last - first + 1); | |
| 235 | 235 | } |
| 236 | 236 | |
| 237 | 237 | /** |
| @@ -171,12 +171,13 @@ void xe_sched_job_destroy(struct kref *ref) · 170 unmodified lines | ||
| 171 | 171 | struct xe_sched_job *job = |
| 172 | 172 | container_of(ref, struct xe_sched_job, refcount); |
| 173 | 173 | struct xe_device *xe = job_to_xe(job); |
| 174 | + struct xe_exec_queue *q = job->q; | |
| 174 | 175 | |
| 175 | 176 | xe_sched_job_free_fences(job); |
| 176 | − xe_exec_queue_put(job->q); | |
| 177 | 177 | dma_fence_put(job->fence); |
| 178 | 178 | drm_sched_job_cleanup(&job->drm); |
| 179 | 179 | job_free(job); |
| 180 | + xe_exec_queue_put(q); | |
| 180 | 181 | xe_pm_runtime_put(xe); |
| 181 | 182 | } |
| 182 | 183 | |
| @@ -53,14 +53,18 @@ static struct xe_user_fence *user_fence_create(struct xe_device *xe, u64 addr, · 52 unmodified lines | ||
| 53 | 53 | u64 value) |
| 54 | 54 | { |
| 55 | 55 | struct xe_user_fence *ufence; |
| 56 | + u64 __user *ptr = u64_to_user_ptr(addr); | |
| 57 | + | |
| 58 | + if (!access_ok(ptr, sizeof(ptr))) | |
| 59 | + return ERR_PTR(-EFAULT); | |
| 56 | 60 | |
| 57 | 61 | ufence = kmalloc(sizeof(*ufence), GFP_KERNEL); |
| 58 | 62 | if (!ufence) |
| 59 | − return NULL; | |
| 63 | + return ERR_PTR(-ENOMEM); | |
| 60 | 64 | |
| 61 | 65 | ufence->xe = xe; |
| 62 | 66 | kref_init(&ufence->refcount); |
| 63 | − ufence->addr = u64_to_user_ptr(addr); | |
| 67 | + ufence->addr = ptr; | |
| 64 | 68 | ufence->value = value; |
| 65 | 69 | ufence->mm = current->mm; |
| 66 | 70 | mmgrab(ufence->mm); |
| @@ -183,8 +187,8 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef, · 116 unmodified lines | ||
| 183 | 187 | } else { |
| 184 | 188 | sync->ufence = user_fence_create(xe, sync_in.addr, |
| 185 | 189 | sync_in.timeline_value); |
| 186 | − if (XE_IOCTL_DBG(xe, !sync->ufence)) | |
| 187 | − return -ENOMEM; | |
| 190 | + if (XE_IOCTL_DBG(xe, IS_ERR(sync->ufence))) | |
| 191 | + return PTR_ERR(sync->ufence); | |
| 188 | 192 | } |
| 189 | 193 | |
| 190 | 194 | break; |
| @@ -263,7 +267,7 @@ void xe_sync_entry_cleanup(struct xe_sync_entry *sync) · 72 unmodified lines | ||
| 263 | 267 | if (sync->fence) |
| 264 | 268 | dma_fence_put(sync->fence); |
| 265 | 269 | if (sync->chain_fence) |
| 266 | − dma_fence_put(&sync->chain_fence->base); | |
| 270 | + dma_fence_chain_free(sync->chain_fence); | |
| 267 | 271 | if (sync->ufence) |
| 268 | 272 | user_fence_put(sync->ufence); |
| 269 | 273 | } |
| @@ -309,7 +309,7 @@ DECLARE_EVENT_CLASS(xe_hw_fence, · 308 unmodified lines | ||
| 309 | 309 | TP_ARGS(fence), |
| 310 | 310 | |
| 311 | 311 | TP_STRUCT__entry( |
| 312 | − __string(dev, __dev_name_gt(fence->ctx->gt)) | |
| 312 | + __string(dev, __dev_name_xe(fence->xe)) | |
| 313 | 313 | __field(u64, ctx) |
| 314 | 314 | __field(u32, seqno) |
| 315 | 315 | __field(struct xe_hw_fence *, fence) |
| @@ -150,7 +150,7 @@ static int xe_ttm_vram_mgr_new(struct ttm_resource_manager *man, · 149 unmodified lines | ||
| 150 | 150 | } while (remaining_size); |
| 151 | 151 | |
| 152 | 152 | if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { |
| 153 | − if (!drm_buddy_block_trim(mm, vres->base.size, &vres->blocks)) | |
| 153 | + if (!drm_buddy_block_trim(mm, NULL, vres->base.size, &vres->blocks)) | |
| 154 | 154 | size = vres->base.size; |
| 155 | 155 | } |
| 156 | 156 | |
| @@ -1601,6 +1601,10 @@ static void vm_destroy_work_func(struct work_struct *w) · 1600 unmodified lines | ||
| 1601 | 1601 | XE_WARN_ON(vm->pt_root[id]); |
| 1602 | 1602 | |
| 1603 | 1603 | trace_xe_vm_free(vm); |
| 1604 | + | |
| 1605 | + if (vm->xef) | |
| 1606 | + xe_file_put(vm->xef); | |
| 1607 | + | |
| 1604 | 1608 | kfree(vm); |
| 1605 | 1609 | } |
| 1606 | 1610 | |
| @@ -1916,7 +1920,7 @@ int xe_vm_create_ioctl(struct drm_device *dev, void *data, · 309 unmodified lines | ||
| 1916 | 1920 | } |
| 1917 | 1921 | |
| 1918 | 1922 | args->vm_id = id; |
| 1919 | − vm->xef = xef; | |
| 1923 | + vm->xef = xe_file_get(xef); | |
| 1920 | 1924 | |
| 1921 | 1925 | /* Record BO memory for VM pagetable created against client */ |
| 1922 | 1926 | for_each_tile(tile, xe, id) |
| @@ -3337,10 +3341,11 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) · 1414 unmodified lines | ||
| 3337 | 3341 | { |
| 3338 | 3342 | struct xe_device *xe = xe_vma_vm(vma)->xe; |
| 3339 | 3343 | struct xe_tile *tile; |
| 3340 | − u32 tile_needs_invalidate = 0; | |
| 3341 | − int seqno[XE_MAX_TILES_PER_DEVICE]; | |
| 3344 | + struct xe_gt_tlb_invalidation_fence | |
| 3345 | + fence[XE_MAX_TILES_PER_DEVICE * XE_MAX_GT_PER_TILE]; | |
| 3342 | 3346 | u8 id; |
| 3343 | − int ret; | |
| 3347 | + u32 fence_id = 0; | |
| 3348 | + int ret = 0; | |
| 3344 | 3349 | |
| 3345 | 3350 | xe_assert(xe, !xe_vma_is_null(vma)); |
| 3346 | 3351 | trace_xe_vma_invalidate(vma); |
| @@ -3365,29 +3370,43 @@ int xe_vm_invalidate_vma(struct xe_vma *vma) · 18 unmodified lines | ||
| 3365 | 3370 | |
| 3366 | 3371 | for_each_tile(tile, xe, id) { |
| 3367 | 3372 | if (xe_pt_zap_ptes(tile, vma)) { |
| 3368 | − tile_needs_invalidate |= BIT(id); | |
| 3369 | 3373 | xe_device_wmb(xe); |
| 3370 | − /* | |
| 3371 | − * FIXME: We potentially need to invalidate multiple | |
| 3372 | − * GTs within the tile | |
| 3373 | − */ | |
| 3374 | − seqno[id] = xe_gt_tlb_invalidation_vma(tile->primary_gt, NULL, vma); | |
| 3375 | − if (seqno[id] < 0) | |
| 3376 | − return seqno[id]; | |
| 3377 | − } | |
| 3378 | − } | |
| 3374 | + xe_gt_tlb_invalidation_fence_init(tile->primary_gt, | |
| 3375 | + &fence[fence_id], | |
| 3376 | + true); | |
| 3377 | + | |
| 3378 | + ret = xe_gt_tlb_invalidation_vma(tile->primary_gt, | |
| 3379 | + &fence[fence_id], vma); | |
| 3380 | + if (ret < 0) { | |
| 3381 | + xe_gt_tlb_invalidation_fence_fini(&fence[fence_id]); | |
| 3382 | + goto wait; | |
| 3383 | + } | |
| 3384 | + ++fence_id; | |
| 3379 | 3385 | |
| 3380 | − for_each_tile(tile, xe, id) { | |
| 3381 | − if (tile_needs_invalidate & BIT(id)) { | |
| 3382 | − ret = xe_gt_tlb_invalidation_wait(tile->primary_gt, seqno[id]); | |
| 3383 | − if (ret < 0) | |
| 3384 | − return ret; | |
| 3386 | + if (!tile->media_gt) | |
| 3387 | + continue; | |
| 3388 | + | |
| 3389 | + xe_gt_tlb_invalidation_fence_init(tile->media_gt, | |
| 3390 | + &fence[fence_id], | |
| 3391 | + true); | |
| 3392 | + | |
| 3393 | + ret = xe_gt_tlb_invalidation_vma(tile->media_gt, | |
| 3394 | + &fence[fence_id], vma); | |
| 3395 | + if (ret < 0) { | |
| 3396 | + xe_gt_tlb_invalidation_fence_fini(&fence[fence_id]); | |
| 3397 | + goto wait; | |
| 3398 | + } | |
| 3399 | + ++fence_id; | |
| 3385 | 3400 | } |
| 3386 | 3401 | } |
| 3387 | 3402 | |
| 3403 | +wait: | |
| 3404 | + for (id = 0; id < fence_id; ++id) | |
| 3405 | + xe_gt_tlb_invalidation_fence_wait(&fence[id]); | |
| 3406 | + | |
| 3388 | 3407 | vma->tile_invalidated = vma->tile_mask; |
| 3389 | 3408 | |
| 3390 | − return 0; | |
| 3409 | + return ret; | |
| 3391 | 3410 | } |
| 3392 | 3411 | |
| 3393 | 3412 | struct xe_vm_snapshot { |
| @@ -486,6 +486,10 @@ static const struct xe_rtp_entry_sr engine_was[] = { · 485 unmodified lines | ||
| 486 | 486 | XE_RTP_RULES(GRAPHICS_VERSION(2004), FUNC(xe_rtp_match_first_render_or_compute)), |
| 487 | 487 | XE_RTP_ACTIONS(SET(TDL_TSL_CHICKEN, SLM_WMTP_RESTORE)) |
| 488 | 488 | }, |
| 489 | + { XE_RTP_NAME("14021402888"), | |
| 490 | + XE_RTP_RULES(GRAPHICS_VERSION(2004), ENGINE_CLASS(RENDER)), | |
| 491 | + XE_RTP_ACTIONS(SET(HALF_SLICE_CHICKEN7, CLEAR_OPTIMIZATION_DISABLE)) | |
| 492 | + }, | |
| 489 | 493 | |
| 490 | 494 | /* Xe2_HPG */ |
| 491 | 495 | |
| @@ -538,6 +542,20 @@ static const struct xe_rtp_entry_sr engine_was[] = { · 46 unmodified lines | ||
| 538 | 542 | XE_RTP_RULES(GRAPHICS_VERSION(2001), ENGINE_CLASS(RENDER)), |
| 539 | 543 | XE_RTP_ACTIONS(SET(HALF_SLICE_CHICKEN7, CLEAR_OPTIMIZATION_DISABLE)) |
| 540 | 544 | }, |
| 545 | + { XE_RTP_NAME("14021821874"), | |
| 546 | + XE_RTP_RULES(GRAPHICS_VERSION(2001), FUNC(xe_rtp_match_first_render_or_compute)), | |
| 547 | + XE_RTP_ACTIONS(SET(TDL_TSL_CHICKEN, STK_ID_RESTRICT)) | |
| 548 | + }, | |
| 549 | + | |
| 550 | + /* Xe2_LPM */ | |
| 551 | + | |
| 552 | + { XE_RTP_NAME("16021639441"), | |
| 553 | + XE_RTP_RULES(MEDIA_VERSION(2000)), | |
| 554 | + XE_RTP_ACTIONS(SET(CSFE_CHICKEN1(0), | |
| 555 | + GHWSP_CSB_REPORT_DIS | | |
| 556 | + PPHWSP_CSB_AND_TIMESTAMP_REPORT_DIS, | |
| 557 | + XE_RTP_ACTION_FLAG(ENGINE_BASE))) | |
| 558 | + }, | |
| 541 | 559 | |
| 542 | 560 | /* Xe2_HPM */ |
| 543 | 561 | |
| @@ -29,3 +29,4 @@ · 28 unmodified lines | ||
| 29 | 29 | 13011645652 GRAPHICS_VERSION(2004) |
| 30 | 30 | 22019338487 MEDIA_VERSION(2000) |
| 31 | 31 | GRAPHICS_VERSION(2001) |
| 32 | +16023588340 GRAPHICS_VERSION(2001) | |
| @@ -288,12 +288,22 @@ int amd_sfh_hid_client_init(struct amd_mp2_dev *privdata) · 287 unmodified lines | ||
| 288 | 288 | mp2_ops->start(privdata, info); |
| 289 | 289 | cl_data->sensor_sts[i] = amd_sfh_wait_for_response |
| 290 | 290 | (privdata, cl_data->sensor_idx[i], SENSOR_ENABLED); |
| 291 | + | |
| 292 | + if (cl_data->sensor_sts[i] == SENSOR_ENABLED) | |
| 293 | + cl_data->is_any_sensor_enabled = true; | |
| 294 | + } | |
| 295 | + | |
| 296 | + if (!cl_data->is_any_sensor_enabled || | |
| 297 | + (mp2_ops->discovery_status && mp2_ops->discovery_status(privdata) == 0)) { | |
| 298 | + dev_warn(dev, "Failed to discover, sensors not enabled is %d\n", | |
| 299 | + cl_data->is_any_sensor_enabled); | |
| 300 | + rc = -EOPNOTSUPP; | |
| 301 | + goto cleanup; | |
| 291 | 302 | } |
| 292 | 303 | |
| 293 | 304 | for (i = 0; i < cl_data->num_hid_devices; i++) { |
| 294 | 305 | cl_data->cur_hid_dev = i; |
| 295 | 306 | if (cl_data->sensor_sts[i] == SENSOR_ENABLED) { |
| 296 | − cl_data->is_any_sensor_enabled = true; | |
| 297 | 307 | rc = amdtp_hid_probe(i, cl_data); |
| 298 | 308 | if (rc) |
| 299 | 309 | goto cleanup; |
| @@ -305,12 +315,6 @@ int amd_sfh_hid_client_init(struct amd_mp2_dev *privdata) · 5 unmodified lines | ||
| 305 | 315 | cl_data->sensor_sts[i]); |
| 306 | 316 | } |
| 307 | 317 | |
| 308 | − if (!cl_data->is_any_sensor_enabled || | |
| 309 | − (mp2_ops->discovery_status && mp2_ops->discovery_status(privdata) == 0)) { | |
| 310 | − dev_warn(dev, "Failed to discover, sensors not enabled is %d\n", cl_data->is_any_sensor_enabled); | |
| 311 | − rc = -EOPNOTSUPP; | |
| 312 | − goto cleanup; | |
| 313 | − } | |
| 314 | 318 | schedule_delayed_work(&cl_data->work_buffer, msecs_to_jiffies(AMD_SFH_IDLE_LOOP)); |
| 315 | 319 | return 0; |
| 316 | 320 | |
| @@ -171,11 +171,13 @@ err_hid_data: · 170 unmodified lines | ||
| 171 | 171 | void amdtp_hid_remove(struct amdtp_cl_data *cli_data) |
| 172 | 172 | { |
| 173 | 173 | int i; |
| 174 | + struct amdtp_hid_data *hid_data; | |
| 174 | 175 | |
| 175 | 176 | for (i = 0; i < cli_data->num_hid_devices; ++i) { |
| 176 | 177 | if (cli_data->hid_sensor_hubs[i]) { |
| 177 | − kfree(cli_data->hid_sensor_hubs[i]->driver_data); | |
| 178 | + hid_data = cli_data->hid_sensor_hubs[i]->driver_data; | |
| 178 | 179 | hid_destroy_device(cli_data->hid_sensor_hubs[i]); |
| 180 | + kfree(hid_data); | |
| 179 | 181 | cli_data->hid_sensor_hubs[i] = NULL; |
| 180 | 182 | } |
| 181 | 183 | } |
| @@ -3,7 +3,7 @@ menu "HID-BPF support" · 2 unmodified lines | ||
| 3 | 3 | |
| 4 | 4 | config HID_BPF |
| 5 | 5 | bool "HID-BPF support" |
| 6 | − depends on BPF | |
| 6 | + depends on BPF_JIT | |
| 7 | 7 | depends on BPF_SYSCALL |
| 8 | 8 | depends on DYNAMIC_FTRACE_WITH_DIRECT_CALLS |
| 9 | 9 | help |
| @@ -183,6 +183,10 @@ static int hid_bpf_reg(void *kdata, struct bpf_link *link) · 182 unmodified lines | ||
| 183 | 183 | struct hid_device *hdev; |
| 184 | 184 | int count, err = 0; |
| 185 | 185 | |
| 186 | + /* prevent multiple attach of the same struct_ops */ | |
| 187 | + if (ops->hdev) | |
| 188 | + return -EINVAL; | |
| 189 | + | |
| 186 | 190 | hdev = hid_get_device(ops->hid_id); |
| 187 | 191 | if (IS_ERR(hdev)) |
| 188 | 192 | return PTR_ERR(hdev); |
| @@ -248,6 +252,7 @@ static void hid_bpf_unreg(void *kdata, struct bpf_link *link) · 59 unmodified lines | ||
| 248 | 252 | |
| 249 | 253 | list_del_rcu(&ops->list); |
| 250 | 254 | synchronize_srcu(&hdev->bpf.srcu); |
| 255 | + ops->hdev = NULL; | |
| 251 | 256 | |
| 252 | 257 | reconnect = hdev->bpf.rdesc_ops == ops; |
| 253 | 258 | if (reconnect) |
| @@ -1248,6 +1248,9 @@ static const struct hid_device_id asus_devices[] = { · 1247 unmodified lines | ||
| 1248 | 1248 | { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, |
| 1249 | 1249 | USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY), |
| 1250 | 1250 | QUIRK_USE_KBD_BACKLIGHT | QUIRK_ROG_NKEY_KEYBOARD }, |
| 1251 | + { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, | |
| 1252 | + USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X), | |
| 1253 | + QUIRK_USE_KBD_BACKLIGHT | QUIRK_ROG_NKEY_KEYBOARD }, | |
| 1251 | 1254 | { HID_USB_DEVICE(USB_VENDOR_ID_ASUSTEK, |
| 1252 | 1255 | USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD), |
| 1253 | 1256 | QUIRK_ROG_CLAYMORE_II_KEYBOARD }, |
| @@ -106,7 +106,7 @@ static void cougar_fix_g6_mapping(void) · 105 unmodified lines | ||
| 106 | 106 | static __u8 *cougar_report_fixup(struct hid_device *hdev, __u8 *rdesc, |
| 107 | 107 | unsigned int *rsize) |
| 108 | 108 | { |
| 109 | − if (rdesc[2] == 0x09 && rdesc[3] == 0x02 && | |
| 109 | + if (*rsize >= 117 && rdesc[2] == 0x09 && rdesc[3] == 0x02 && | |
| 110 | 110 | (rdesc[115] | rdesc[116] << 8) >= HID_MAX_USAGES) { |
| 111 | 111 | hid_info(hdev, |
| 112 | 112 | "usage count exceeds max: fixing up report descriptor\n"); |
| @@ -210,6 +210,7 @@ · 209 unmodified lines | ||
| 210 | 210 | #define USB_DEVICE_ID_ASUSTEK_ROG_NKEY_KEYBOARD3 0x1a30 |
| 211 | 211 | #define USB_DEVICE_ID_ASUSTEK_ROG_Z13_LIGHTBAR 0x18c6 |
| 212 | 212 | #define USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY 0x1abe |
| 213 | +#define USB_DEVICE_ID_ASUSTEK_ROG_NKEY_ALLY_X 0x1b4c | |
| 213 | 214 | #define USB_DEVICE_ID_ASUSTEK_ROG_CLAYMORE_II_KEYBOARD 0x196b |
| 214 | 215 | #define USB_DEVICE_ID_ASUSTEK_FX503VD_KEYBOARD 0x1869 |
| 215 | 216 | |
| @@ -520,6 +521,8 @@ · 304 unmodified lines | ||
| 520 | 521 | #define USB_DEVICE_ID_GENERAL_TOUCH_WIN8_PIT_E100 0xe100 |
| 521 | 522 | |
| 522 | 523 | #define I2C_VENDOR_ID_GOODIX 0x27c6 |
| 524 | +#define I2C_DEVICE_ID_GOODIX_01E8 0x01e8 | |
| 525 | +#define I2C_DEVICE_ID_GOODIX_01E9 0x01e9 | |
| 523 | 526 | #define I2C_DEVICE_ID_GOODIX_01F0 0x01f0 |
| 524 | 527 | |
| 525 | 528 | #define USB_VENDOR_ID_GOODTOUCH 0x1aad |
| @@ -1441,6 +1441,30 @@ static int mt_event(struct hid_device *hid, struct hid_field *field, · 1440 unmodified lines | ||
| 1441 | 1441 | return 0; |
| 1442 | 1442 | } |
| 1443 | 1443 | |
| 1444 | +static __u8 *mt_report_fixup(struct hid_device *hdev, __u8 *rdesc, | |
| 1445 | + unsigned int *size) | |
| 1446 | +{ | |
| 1447 | + if (hdev->vendor == I2C_VENDOR_ID_GOODIX && | |
| 1448 | + (hdev->product == I2C_DEVICE_ID_GOODIX_01E8 || | |
| 1449 | + hdev->product == I2C_DEVICE_ID_GOODIX_01E9)) { | |
| 1450 | + if (rdesc[607] == 0x15) { | |
| 1451 | + rdesc[607] = 0x25; | |
| 1452 | + dev_info( | |
| 1453 | + &hdev->dev, | |
| 1454 | + "GT7868Q report descriptor fixup is applied.\n"); | |
| 1455 | + } else { | |
| 1456 | + dev_info( | |
| 1457 | + &hdev->dev, | |
| 1458 | + "The byte is not expected for fixing the report descriptor. \ | |
| 1459 | +It's possible that the touchpad firmware is not suitable for applying the fix. \ | |
| 1460 | +got: %x\n", | |
| 1461 | + rdesc[607]); | |
| 1462 | + } | |
| 1463 | + } | |
| 1464 | + | |
| 1465 | + return rdesc; | |
| 1466 | +} | |
| 1467 | + | |
| 1444 | 1468 | static void mt_report(struct hid_device *hid, struct hid_report *report) |
| 1445 | 1469 | { |
| 1446 | 1470 | struct mt_device *td = hid_get_drvdata(hid); |
| @@ -2035,6 +2059,14 @@ static const struct hid_device_id mt_devices[] = { · 588 unmodified lines | ||
| 2035 | 2059 | MT_BT_DEVICE(USB_VENDOR_ID_FRUCTEL, |
| 2036 | 2060 | USB_DEVICE_ID_GAMETEL_MT_MODE) }, |
| 2037 | 2061 | |
| 2062 | + /* Goodix GT7868Q devices */ | |
| 2063 | + { .driver_data = MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU, | |
| 2064 | + HID_DEVICE(BUS_I2C, HID_GROUP_ANY, I2C_VENDOR_ID_GOODIX, | |
| 2065 | + I2C_DEVICE_ID_GOODIX_01E8) }, | |
| 2066 | + { .driver_data = MT_CLS_WIN_8_FORCE_MULTI_INPUT_NSMU, | |
| 2067 | + HID_DEVICE(BUS_I2C, HID_GROUP_ANY, I2C_VENDOR_ID_GOODIX, | |
| 2068 | + I2C_DEVICE_ID_GOODIX_01E8) }, | |
| 2069 | + | |
| 2038 | 2070 | /* GoodTouch panels */ |
| 2039 | 2071 | { .driver_data = MT_CLS_NSMU, |
| 2040 | 2072 | MT_USB_DEVICE(USB_VENDOR_ID_GOODTOUCH, |
| @@ -2270,6 +2302,7 @@ static struct hid_driver mt_driver = { · 229 unmodified lines | ||
| 2270 | 2302 | .feature_mapping = mt_feature_mapping, |
| 2271 | 2303 | .usage_table = mt_grabbed_usages, |
| 2272 | 2304 | .event = mt_event, |
| 2305 | + .report_fixup = mt_report_fixup, | |
| 2273 | 2306 | .report = mt_report, |
| 2274 | 2307 | .suspend = pm_ptr(mt_suspend), |
| 2275 | 2308 | .reset_resume = pm_ptr(mt_reset_resume), |
| @@ -692,78 +692,28 @@ static bool wacom_is_art_pen(int tool_id) · 691 unmodified lines | ||
| 692 | 692 | |
| 693 | 693 | static int wacom_intuos_get_tool_type(int tool_id) |
| 694 | 694 | { |
| 695 | − int tool_type = BTN_TOOL_PEN; | |
| 696 | − | |
| 697 | − if (wacom_is_art_pen(tool_id)) | |
| 698 | − return tool_type; | |
| 699 | − | |
| 700 | 695 | switch (tool_id) { |
| 701 | 696 | case 0x812: /* Inking pen */ |
| 702 | 697 | case 0x801: /* Intuos3 Inking pen */ |
| 703 | 698 | case 0x12802: /* Intuos4/5 Inking Pen */ |
| 704 | 699 | case 0x012: |
| 705 | − tool_type = BTN_TOOL_PENCIL; | |
| 706 | − break; | |
| 707 | − | |
| 708 | − case 0x822: /* Pen */ | |
| 709 | − case 0x842: | |
| 710 | − case 0x852: | |
| 711 | − case 0x823: /* Intuos3 Grip Pen */ | |
| 712 | − case 0x813: /* Intuos3 Classic Pen */ | |
| 713 | − case 0x802: /* Intuos4/5 13HD/24HD General Pen */ | |
| 714 | − case 0x8e2: /* IntuosHT2 pen */ | |
| 715 | − case 0x022: | |
| 716 | − case 0x200: /* Pro Pen 3 */ | |
| 717 | − case 0x04200: /* Pro Pen 3 */ | |
| 718 | − case 0x10842: /* MobileStudio Pro Pro Pen slim */ | |
| 719 | − case 0x14802: /* Intuos4/5 13HD/24HD Classic Pen */ | |
| 720 | − case 0x16802: /* Cintiq 13HD Pro Pen */ | |
| 721 | − case 0x18802: /* DTH2242 Pen */ | |
| 722 | − case 0x10802: /* Intuos4/5 13HD/24HD General Pen */ | |
| 723 | − case 0x80842: /* Intuos Pro and Cintiq Pro 3D Pen */ | |
| 724 | − tool_type = BTN_TOOL_PEN; | |
| 725 | − break; | |
| 700 | + return BTN_TOOL_PENCIL; | |
| 726 | 701 | |
| 727 | 702 | case 0x832: /* Stroke pen */ |
| 728 | 703 | case 0x032: |
| 729 | − tool_type = BTN_TOOL_BRUSH; | |
| 730 | − break; | |
| 704 | + return BTN_TOOL_BRUSH; | |
| 731 | 705 | |
| 732 | 706 | case 0x007: /* Mouse 4D and 2D */ |
| 733 | 707 | case 0x09c: |
| 734 | 708 | case 0x094: |
| 735 | 709 | case 0x017: /* Intuos3 2D Mouse */ |
| 736 | 710 | case 0x806: /* Intuos4 Mouse */ |
| 737 | − tool_type = BTN_TOOL_MOUSE; | |
| 738 | − break; | |
| 711 | + return BTN_TOOL_MOUSE; | |
| 739 | 712 | |
| 740 | 713 | case 0x096: /* Lens cursor */ |
| 741 | 714 | case 0x097: /* Intuos3 Lens cursor */ |
| 742 | 715 | case 0x006: /* Intuos4 Lens cursor */ |
| 743 | − tool_type = BTN_TOOL_LENS; | |
| 744 | − break; | |
| 745 | − | |
| 746 | − case 0x82a: /* Eraser */ | |
| 747 | − case 0x84a: | |
| 748 | − case 0x85a: | |
| 749 | − case 0x91a: | |
| 750 | − case 0xd1a: | |
| 751 | − case 0x0fa: | |
| 752 | − case 0x82b: /* Intuos3 Grip Pen Eraser */ | |
| 753 | − case 0x81b: /* Intuos3 Classic Pen Eraser */ | |
| 754 | − case 0x91b: /* Intuos3 Airbrush Eraser */ | |
| 755 | − case 0x80c: /* Intuos4/5 13HD/24HD Marker Pen Eraser */ | |
| 756 | − case 0x80a: /* Intuos4/5 13HD/24HD General Pen Eraser */ | |
| 757 | − case 0x90a: /* Intuos4/5 13HD/24HD Airbrush Eraser */ | |
| 758 | − case 0x1480a: /* Intuos4/5 13HD/24HD Classic Pen Eraser */ | |
| 759 | − case 0x1090a: /* Intuos4/5 13HD/24HD Airbrush Eraser */ | |
| 760 | − case 0x1080c: /* Intuos4/5 13HD/24HD Art Pen Eraser */ | |
| 761 | − case 0x1084a: /* MobileStudio Pro Pro Pen slim Eraser */ | |
| 762 | − case 0x1680a: /* Cintiq 13HD Pro Pen Eraser */ | |
| 763 | − case 0x1880a: /* DTH2242 Eraser */ | |
| 764 | − case 0x1080a: /* Intuos4/5 13HD/24HD General Pen Eraser */ | |
| 765 | − tool_type = BTN_TOOL_RUBBER; | |
| 766 | − break; | |
| 716 | + return BTN_TOOL_LENS; | |
| 767 | 717 | |
| 768 | 718 | case 0xd12: |
| 769 | 719 | case 0x912: |
| @@ -771,10 +721,13 @@ static int wacom_intuos_get_tool_type(int tool_id) · 1 unmodified line | ||
| 771 | 721 | case 0x913: /* Intuos3 Airbrush */ |
| 772 | 722 | case 0x902: /* Intuos4/5 13HD/24HD Airbrush */ |
| 773 | 723 | case 0x10902: /* Intuos4/5 13HD/24HD Airbrush */ |
| 774 | − tool_type = BTN_TOOL_AIRBRUSH; | |
| 775 | − break; | |
| 724 | + return BTN_TOOL_AIRBRUSH; | |
| 725 | + | |
| 726 | + default: | |
| 727 | + if (tool_id & 0x0008) | |
| 728 | + return BTN_TOOL_RUBBER; | |
| 729 | + return BTN_TOOL_PEN; | |
| 776 | 730 | } |
| 777 | − return tool_type; | |
| 778 | 731 | } |
| 779 | 732 | |
| 780 | 733 | static void wacom_exit_report(struct wacom_wac *wacom) |
| @@ -1925,12 +1878,14 @@ static void wacom_map_usage(struct input_dev *input, struct hid_usage *usage, · 1144 unmodified lines | ||
| 1925 | 1878 | int fmax = field->logical_maximum; |
| 1926 | 1879 | unsigned int equivalent_usage = wacom_equivalent_usage(usage->hid); |
| 1927 | 1880 | int resolution_code = code; |
| 1928 | − int resolution = hidinput_calc_abs_res(field, resolution_code); | |
| 1881 | + int resolution; | |
| 1929 | 1882 | |
| 1930 | 1883 | if (equivalent_usage == HID_DG_TWIST) { |
| 1931 | 1884 | resolution_code = ABS_RZ; |
| 1932 | 1885 | } |
| 1933 | 1886 | |
| 1887 | + resolution = hidinput_calc_abs_res(field, resolution_code); | |
| 1888 | + | |
| 1934 | 1889 | if (equivalent_usage == HID_GD_X) { |
| 1935 | 1890 | fmin += features->offset_left; |
| 1936 | 1891 | fmax -= features->offset_right; |
| @@ -22,23 +22,23 @@ · 21 unmodified lines | ||
| 22 | 22 | #include <linux/util_macros.h> |
| 23 | 23 | |
| 24 | 24 | /* Indexes for the sysfs hooks */ |
| 25 | − | |
| 26 | −#define INPUT 0 | |
| 27 | −#define MIN 1 | |
| 28 | −#define MAX 2 | |
| 29 | −#define CONTROL 3 | |
| 30 | −#define OFFSET 3 | |
| 31 | −#define AUTOMIN 4 | |
| 32 | −#define THERM 5 | |
| 33 | −#define HYSTERSIS 6 | |
| 34 | − | |
| 25 | +enum adt_sysfs_id { | |
| 26 | + INPUT = 0, | |
| 27 | + MIN = 1, | |
| 28 | + MAX = 2, | |
| 29 | + CONTROL = 3, | |
| 30 | + OFFSET = 3, // Dup | |
| 31 | + AUTOMIN = 4, | |
| 32 | + THERM = 5, | |
| 33 | + HYSTERSIS = 6, | |
| 35 | 34 | /* |
| 36 | 35 | * These are unique identifiers for the sysfs functions - unlike the |
| 37 | 36 | * numbers above, these are not also indexes into an array |
| 38 | 37 | */ |
| 38 | + ALARM = 9, | |
| 39 | + FAULT = 10, | |
| 40 | +}; | |
| 39 | 41 | |
| 40 | −#define ALARM 9 | |
| 41 | −#define FAULT 10 | |
| 42 | 42 | |
| 43 | 43 | /* 7475 Common Registers */ |
| 44 | 44 | |
| @@ -420,7 +420,7 @@ static const struct ec_board_info board_info_strix_b550_i_gaming = { · 419 unmodified lines | ||
| 420 | 420 | |
| 421 | 421 | static const struct ec_board_info board_info_strix_x570_e_gaming = { |
| 422 | 422 | .sensors = SENSOR_SET_TEMP_CHIPSET_CPU_MB | |
| 423 | − SENSOR_TEMP_T_SENSOR | SENSOR_TEMP_VRM | | |
| 423 | + SENSOR_TEMP_T_SENSOR | | |
| 424 | 424 | SENSOR_FAN_CHIPSET | SENSOR_CURR_CPU | |
| 425 | 425 | SENSOR_IN_CPU_CORE, |
| 426 | 426 | .mutex_path = ASUS_HW_ACCESS_MUTEX_ASMX, |
| @@ -427,7 +427,7 @@ static int pt5161l_read(struct device *dev, enum hwmon_sensor_types type, · 426 unmodified lines | ||
| 427 | 427 | struct pt5161l_data *data = dev_get_drvdata(dev); |
| 428 | 428 | int ret; |
| 429 | 429 | u8 buf[8]; |
| 430 | − long adc_code; | |
| 430 | + u32 adc_code; | |
| 431 | 431 | |
| 432 | 432 | switch (attr) { |
| 433 | 433 | case hwmon_temp_input: |
| @@ -449,7 +449,7 @@ static int pt5161l_read(struct device *dev, enum hwmon_sensor_types type, · 15 unmodified lines | ||
| 449 | 449 | |
| 450 | 450 | adc_code = buf[3] << 24 | buf[2] << 16 | buf[1] << 8 | buf[0]; |
| 451 | 451 | if (adc_code == 0 || adc_code >= 0x3ff) { |
| 452 | − dev_dbg(dev, "Invalid adc_code %lx\n", adc_code); | |
| 452 | + dev_dbg(dev, "Invalid adc_code %x\n", adc_code); | |
| 453 | 453 | return -EIO; |
| 454 | 454 | } |
| 455 | 455 | |
| @@ -986,12 +986,17 @@ static int __maybe_unused geni_i2c_runtime_resume(struct device *dev) · 985 unmodified lines | ||
| 986 | 986 | return ret; |
| 987 | 987 | |
| 988 | 988 | ret = clk_prepare_enable(gi2c->core_clk); |
| 989 | − if (ret) | |
| 989 | + if (ret) { | |
| 990 | + geni_icc_disable(&gi2c->se); | |
| 990 | 991 | return ret; |
| 992 | + } | |
| 991 | 993 | |
| 992 | 994 | ret = geni_se_resources_on(&gi2c->se); |
| 993 | − if (ret) | |
| 995 | + if (ret) { | |
| 996 | + clk_disable_unprepare(gi2c->core_clk); | |
| 997 | + geni_icc_disable(&gi2c->se); | |
| 994 | 998 | return ret; |
| 999 | + } | |
| 995 | 1000 | |
| 996 | 1001 | enable_irq(gi2c->irq); |
| 997 | 1002 | gi2c->suspended = 0; |
| @@ -1802,9 +1802,9 @@ static int tegra_i2c_probe(struct platform_device *pdev) · 1801 unmodified lines | ||
| 1802 | 1802 | * domain. |
| 1803 | 1803 | * |
| 1804 | 1804 | * VI I2C device shouldn't be marked as IRQ-safe because VI I2C won't |
| 1805 | − * be used for atomic transfers. | |
| 1805 | + * be used for atomic transfers. ACPI device is not IRQ safe also. | |
| 1806 | 1806 | */ |
| 1807 | − if (!IS_VI(i2c_dev)) | |
| 1807 | + if (!IS_VI(i2c_dev) && !has_acpi_companion(i2c_dev->dev)) | |
| 1808 | 1808 | pm_runtime_irq_safe(i2c_dev->dev); |
| 1809 | 1809 | |
| 1810 | 1810 | pm_runtime_enable(i2c_dev->dev); |
| @@ -18,7 +18,7 @@ · 17 unmodified lines | ||
| 18 | 18 | |
| 19 | 19 | enum testunit_cmds { |
| 20 | 20 | TU_CMD_READ_BYTES = 1, /* save 0 for ABORT, RESET or similar */ |
| 21 | − TU_CMD_HOST_NOTIFY, | |
| 21 | + TU_CMD_SMBUS_HOST_NOTIFY, | |
| 22 | 22 | TU_CMD_SMBUS_BLOCK_PROC_CALL, |
| 23 | 23 | TU_NUM_CMDS |
| 24 | 24 | }; |
| @@ -60,7 +60,7 @@ static void i2c_slave_testunit_work(struct work_struct *work) · 35 unmodified lines | ||
| 60 | 60 | msg.len = tu->regs[TU_REG_DATAH]; |
| 61 | 61 | break; |
| 62 | 62 | |
| 63 | − case TU_CMD_HOST_NOTIFY: | |
| 63 | + case TU_CMD_SMBUS_HOST_NOTIFY: | |
| 64 | 64 | msg.addr = 0x08; |
| 65 | 65 | msg.flags = 0; |
| 66 | 66 | msg.len = 3; |
| @@ -34,6 +34,7 @@ static int smbus_do_alert(struct device *dev, void *addrp) · 33 unmodified lines | ||
| 34 | 34 | struct i2c_client *client = i2c_verify_client(dev); |
| 35 | 35 | struct alert_data *data = addrp; |
| 36 | 36 | struct i2c_driver *driver; |
| 37 | + int ret; | |
| 37 | 38 | |
| 38 | 39 | if (!client || client->addr != data->addr) |
| 39 | 40 | return 0; |
| @@ -47,16 +48,47 @@ static int smbus_do_alert(struct device *dev, void *addrp) · 7 unmodified lines | ||
| 47 | 48 | device_lock(dev); |
| 48 | 49 | if (client->dev.driver) { |
| 49 | 50 | driver = to_i2c_driver(client->dev.driver); |
| 50 | − if (driver->alert) | |
| 51 | + if (driver->alert) { | |
| 52 | + /* Stop iterating after we find the device */ | |
| 51 | 53 | driver->alert(client, data->type, data->data); |
| 52 | − else | |
| 54 | + ret = -EBUSY; | |
| 55 | + } else { | |
| 53 | 56 | dev_warn(&client->dev, "no driver alert()!\n"); |
| 54 | − } else | |
| 57 | + ret = -EOPNOTSUPP; | |
| 58 | + } | |
| 59 | + } else { | |
| 55 | 60 | dev_dbg(&client->dev, "alert with no driver\n"); |
| 61 | + ret = -ENODEV; | |
| 62 | + } | |
| 63 | + device_unlock(dev); | |
| 64 | + | |
| 65 | + return ret; | |
| 66 | +} | |
| 67 | + | |
| 68 | +/* Same as above, but call back all drivers with alert handler */ | |
| 69 | + | |
| 70 | +static int smbus_do_alert_force(struct device *dev, void *addrp) | |
| 71 | +{ | |
| 72 | + struct i2c_client *client = i2c_verify_client(dev); | |
| 73 | + struct alert_data *data = addrp; | |
| 74 | + struct i2c_driver *driver; | |
| 75 | + | |
| 76 | + if (!client || (client->flags & I2C_CLIENT_TEN)) | |
| 77 | + return 0; | |
| 78 | + | |
| 79 | + /* | |
| 80 | + * Drivers should either disable alerts, or provide at least | |
| 81 | + * a minimal handler. Lock so the driver won't change. | |
| 82 | + */ | |
| 83 | + device_lock(dev); | |
| 84 | + if (client->dev.driver) { | |
| 85 | + driver = to_i2c_driver(client->dev.driver); | |
| 86 | + if (driver->alert) | |
| 87 | + driver->alert(client, data->type, data->data); | |
| 88 | + } | |
| 56 | 89 | device_unlock(dev); |
| 57 | 90 | |
| 58 | − /* Stop iterating after we find the device */ | |
| 59 | − return -EBUSY; | |
| 91 | + return 0; | |
| 60 | 92 | } |
| 61 | 93 | |
| 62 | 94 | /* |
| @@ -67,6 +99,7 @@ static irqreturn_t smbus_alert(int irq, void *d) · 4 unmodified lines | ||
| 67 | 99 | { |
| 68 | 100 | struct i2c_smbus_alert *alert = d; |
| 69 | 101 | struct i2c_client *ara; |
| 102 | + unsigned short prev_addr = I2C_CLIENT_END; /* Not a valid address */ | |
| 70 | 103 | |
| 71 | 104 | ara = alert->ara; |
| 72 | 105 | |
| @@ -94,8 +127,25 @@ static irqreturn_t smbus_alert(int irq, void *d) · 21 unmodified lines | ||
| 94 | 127 | data.addr, data.data); |
| 95 | 128 | |
| 96 | 129 | /* Notify driver for the device which issued the alert */ |
| 97 | − device_for_each_child(&ara->adapter->dev, &data, | |
| 98 | − smbus_do_alert); | |
| 130 | + status = device_for_each_child(&ara->adapter->dev, &data, | |
| 131 | + smbus_do_alert); | |
| 132 | + /* | |
| 133 | + * If we read the same address more than once, and the alert | |
| 134 | + * was not handled by a driver, it won't do any good to repeat | |
| 135 | + * the loop because it will never terminate. Try again, this | |
| 136 | + * time calling the alert handlers of all devices connected to | |
| 137 | + * the bus, and abort the loop afterwards. If this helps, we | |
| 138 | + * are all set. If it doesn't, there is nothing else we can do, | |
| 139 | + * so we might as well abort the loop. | |
| 140 | + * Note: This assumes that a driver with alert handler handles | |
| 141 | + * the alert properly and clears it if necessary. | |
| 142 | + */ | |
| 143 | + if (data.addr == prev_addr && status != -EBUSY) { | |
| 144 | + device_for_each_child(&ara->adapter->dev, &data, | |
| 145 | + smbus_do_alert_force); | |
| 146 | + break; | |
| 147 | + } | |
| 148 | + prev_addr = data.addr; | |
| 99 | 149 | } |
| 100 | 150 | |
| 101 | 151 | return IRQ_HANDLED; |
| @@ -46,6 +46,9 @@ int input_mt_init_slots(struct input_dev *dev, unsigned int num_slots, · 45 unmodified lines | ||
| 46 | 46 | return 0; |
| 47 | 47 | if (mt) |
| 48 | 48 | return mt->num_slots != num_slots ? -EINVAL : 0; |
| 49 | + /* Arbitrary limit for avoiding too large memory allocation. */ | |
| 50 | + if (num_slots > 1024) | |
| 51 | + return -EINVAL; | |
| 49 | 52 | |
| 50 | 53 | mt = kzalloc(struct_size(mt, slots, num_slots), GFP_KERNEL); |
| 51 | 54 | if (!mt) |
| @@ -182,8 +182,11 @@ static int adc_joystick_set_axes(struct device *dev, struct adc_joystick *joy) · 181 unmodified lines | ||
| 182 | 182 | swap(range[0], range[1]); |
| 183 | 183 | } |
| 184 | 184 | |
| 185 | − fwnode_property_read_u32(child, "abs-fuzz", &fuzz); | |
| 186 | − fwnode_property_read_u32(child, "abs-flat", &flat); | |
| 185 | + if (fwnode_property_read_u32(child, "abs-fuzz", &fuzz)) | |
| 186 | + fuzz = 0; | |
| 187 | + | |
| 188 | + if (fwnode_property_read_u32(child, "abs-flat", &flat)) | |
| 189 | + flat = 0; | |
| 187 | 190 | |
| 188 | 191 | input_set_abs_params(joy->input, axes[i].code, |
| 189 | 192 | range[0], range[1], fuzz, flat); |
| @@ -417,6 +417,20 @@ static int uinput_validate_absinfo(struct input_dev *dev, unsigned int code, · 416 unmodified lines | ||
| 417 | 417 | return -EINVAL; |
| 418 | 418 | } |
| 419 | 419 | |
| 420 | + /* | |
| 421 | + * Limit number of contacts to a reasonable value (100). This | |
| 422 | + * ensures that we need less than 2 pages for struct input_mt | |
| 423 | + * (we are not using in-kernel slot assignment so not going to | |
| 424 | + * allocate memory for the "red" table), and we should have no | |
| 425 | + * trouble getting this much memory. | |
| 426 | + */ | |
| 427 | + if (code == ABS_MT_SLOT && max > 99) { | |
| 428 | + printk(KERN_DEBUG | |
| 429 | + "%s: unreasonably large number of slots requested: %d\n", | |
| 430 | + UINPUT_NAME, max); | |
| 431 | + return -EINVAL; | |
| 432 | + } | |
| 433 | + | |
| 420 | 434 | return 0; |
| 421 | 435 | } |
| 422 | 436 | |
| @@ -91,48 +91,6 @@ static int cypress_ps2_ext_cmd(struct psmouse *psmouse, u8 prefix, u8 nibble) · 90 unmodified lines | ||
| 91 | 91 | return rc; |
| 92 | 92 | } |
| 93 | 93 | |
| 94 | −static int cypress_ps2_read_cmd_status(struct psmouse *psmouse, | |
| 95 | − u8 cmd, u8 *param) | |
| 96 | −{ | |
| 97 | − struct ps2dev *ps2dev = &psmouse->ps2dev; | |
| 98 | − enum psmouse_state old_state; | |
| 99 | − int pktsize; | |
| 100 | − int rc; | |
| 101 | − | |
| 102 | − ps2_begin_command(ps2dev); | |
| 103 | − | |
| 104 | − old_state = psmouse->state; | |
| 105 | − psmouse->state = PSMOUSE_CMD_MODE; | |
| 106 | − psmouse->pktcnt = 0; | |
| 107 | − | |
| 108 | − pktsize = (cmd == CYTP_CMD_READ_TP_METRICS) ? 8 : 3; | |
| 109 | − memset(param, 0, pktsize); | |
| 110 | − | |
| 111 | − rc = cypress_ps2_sendbyte(psmouse, PSMOUSE_CMD_GETINFO & 0xff); | |
| 112 | − if (rc) | |
| 113 | − goto out; | |
| 114 | − | |
| 115 | − if (!wait_event_timeout(ps2dev->wait, | |
| 116 | − psmouse->pktcnt >= pktsize, | |
| 117 | − msecs_to_jiffies(CYTP_CMD_TIMEOUT))) { | |
| 118 | − rc = -ETIMEDOUT; | |
| 119 | − goto out; | |
| 120 | − } | |
| 121 | − | |
| 122 | − memcpy(param, psmouse->packet, pktsize); | |
| 123 | − | |
| 124 | − psmouse_dbg(psmouse, "Command 0x%02x response data (0x): %*ph\n", | |
| 125 | − cmd, pktsize, param); | |
| 126 | − | |
| 127 | −out: | |
| 128 | − psmouse->state = old_state; | |
| 129 | − psmouse->pktcnt = 0; | |
| 130 | − | |
| 131 | − ps2_end_command(ps2dev); | |
| 132 | − | |
| 133 | − return rc; | |
| 134 | −} | |
| 135 | − | |
| 136 | 94 | static bool cypress_verify_cmd_state(struct psmouse *psmouse, u8 cmd, u8* param) |
| 137 | 95 | { |
| 138 | 96 | bool rate_match = false; |
| @@ -166,6 +124,8 @@ static bool cypress_verify_cmd_state(struct psmouse *psmouse, u8 cmd, u8* param) · 27 unmodified lines | ||
| 166 | 124 | static int cypress_send_ext_cmd(struct psmouse *psmouse, u8 cmd, u8 *param) |
| 167 | 125 | { |
| 168 | 126 | u8 cmd_prefix = PSMOUSE_CMD_SETRES & 0xff; |
| 127 | + unsigned int resp_size = cmd == CYTP_CMD_READ_TP_METRICS ? 8 : 3; | |
| 128 | + unsigned int ps2_cmd = (PSMOUSE_CMD_GETINFO & 0xff) | (resp_size << 8); | |
| 169 | 129 | int tries = CYTP_PS2_CMD_TRIES; |
| 170 | 130 | int error; |
| 171 | 131 | |
| @@ -179,10 +139,18 @@ static int cypress_send_ext_cmd(struct psmouse *psmouse, u8 cmd, u8 *param) · 7 unmodified lines | ||
| 179 | 139 | cypress_ps2_ext_cmd(psmouse, cmd_prefix, DECODE_CMD_BB(cmd)); |
| 180 | 140 | cypress_ps2_ext_cmd(psmouse, cmd_prefix, DECODE_CMD_AA(cmd)); |
| 181 | 141 | |
| 182 | − error = cypress_ps2_read_cmd_status(psmouse, cmd, param); | |
| 183 | − if (!error && cypress_verify_cmd_state(psmouse, cmd, param)) | |
| 184 | − return 0; | |
| 142 | + error = ps2_command(&psmouse->ps2dev, param, ps2_cmd); | |
| 143 | + if (error) { | |
| 144 | + psmouse_dbg(psmouse, "Command 0x%02x failed: %d\n", | |
| 145 | + cmd, error); | |
| 146 | + } else { | |
| 147 | + psmouse_dbg(psmouse, | |
| 148 | + "Command 0x%02x response data (0x): %*ph\n", | |
| 149 | + cmd, resp_size, param); | |
| 185 | 150 | |
| 151 | + if (cypress_verify_cmd_state(psmouse, cmd, param)) | |
| 152 | + return 0; | |
| 153 | + } | |
| 186 | 154 | } while (--tries > 0); |
| 187 | 155 | |
| 188 | 156 | return -EIO; |
| @@ -189,6 +189,7 @@ static const char * const smbus_pnp_ids[] = { · 188 unmodified lines | ||
| 189 | 189 | "LEN2054", /* E480 */ |
| 190 | 190 | "LEN2055", /* E580 */ |
| 191 | 191 | "LEN2068", /* T14 Gen 1 */ |
| 192 | + "SYN3015", /* HP EliteBook 840 G2 */ | |
| 192 | 193 | "SYN3052", /* HP EliteBook 840 G4 */ |
| 193 | 194 | "SYN3221", /* HP 15-ay000 */ |
| 194 | 195 | "SYN323d", /* HP Spectre X360 13-w013dx */ |
| @@ -83,6 +83,7 @@ static inline void i8042_write_command(int val) · 82 unmodified lines | ||
| 83 | 83 | #define SERIO_QUIRK_KBDRESET BIT(12) |
| 84 | 84 | #define SERIO_QUIRK_DRITEK BIT(13) |
| 85 | 85 | #define SERIO_QUIRK_NOPNP BIT(14) |
| 86 | +#define SERIO_QUIRK_FORCENORESTORE BIT(15) | |
| 86 | 87 | |
| 87 | 88 | /* Quirk table for different mainboards. Options similar or identical to i8042 |
| 88 | 89 | * module parameters. |
| @@ -626,6 +627,15 @@ static const struct dmi_system_id i8042_dmi_quirk_table[] __initconst = { · 537 unmodified lines | ||
| 626 | 627 | }, |
| 627 | 628 | .driver_data = (void *)(SERIO_QUIRK_NOMUX) |
| 628 | 629 | }, |
| 630 | + { | |
| 631 | + /* Fujitsu Lifebook E756 */ | |
| 632 | + /* https://bugzilla.suse.com/show_bug.cgi?id=1229056 */ | |
| 633 | + .matches = { | |
| 634 | + DMI_MATCH(DMI_SYS_VENDOR, "FUJITSU"), | |
| 635 | + DMI_MATCH(DMI_PRODUCT_NAME, "LIFEBOOK E756"), | |
| 636 | + }, | |
| 637 | + .driver_data = (void *)(SERIO_QUIRK_NOMUX) | |
| 638 | + }, | |
| 629 | 639 | { |
| 630 | 640 | /* Fujitsu Lifebook E5411 */ |
| 631 | 641 | .matches = { |
| @@ -1149,18 +1159,10 @@ static const struct dmi_system_id i8042_dmi_quirk_table[] __initconst = { · 517 unmodified lines | ||
| 1149 | 1159 | SERIO_QUIRK_NOLOOP | SERIO_QUIRK_NOPNP) |
| 1150 | 1160 | }, |
| 1151 | 1161 | { |
| 1152 | − /* | |
| 1153 | − * Setting SERIO_QUIRK_NOMUX or SERIO_QUIRK_RESET_ALWAYS makes | |
| 1154 | − * the keyboard very laggy for ~5 seconds after boot and | |
| 1155 | − * sometimes also after resume. | |
| 1156 | − * However both are required for the keyboard to not fail | |
| 1157 | − * completely sometimes after boot or resume. | |
| 1158 | − */ | |
| 1159 | 1162 | .matches = { |
| 1160 | 1163 | DMI_MATCH(DMI_BOARD_NAME, "N150CU"), |
| 1161 | 1164 | }, |
| 1162 | − .driver_data = (void *)(SERIO_QUIRK_NOMUX | SERIO_QUIRK_RESET_ALWAYS | | |
| 1163 | − SERIO_QUIRK_NOLOOP | SERIO_QUIRK_NOPNP) | |
| 1165 | + .driver_data = (void *)(SERIO_QUIRK_FORCENORESTORE) | |
| 1164 | 1166 | }, |
| 1165 | 1167 | { |
| 1166 | 1168 | .matches = { |
| @@ -1685,6 +1687,8 @@ static void __init i8042_check_quirks(void) · 518 unmodified lines | ||
| 1685 | 1687 | if (quirks & SERIO_QUIRK_NOPNP) |
| 1686 | 1688 | i8042_nopnp = true; |
| 1687 | 1689 | #endif |
| 1690 | + if (quirks & SERIO_QUIRK_FORCENORESTORE) | |
| 1691 | + i8042_forcenorestore = true; | |
| 1688 | 1692 | } |
| 1689 | 1693 | #else |
| 1690 | 1694 | static inline void i8042_check_quirks(void) {} |
| @@ -1718,7 +1722,7 @@ static int __init i8042_platform_init(void) · 27 unmodified lines | ||
| 1718 | 1722 | |
| 1719 | 1723 | i8042_check_quirks(); |
| 1720 | 1724 | |
| 1721 | − pr_debug("Active quirks (empty means none):%s%s%s%s%s%s%s%s%s%s%s%s%s\n", | |
| 1725 | + pr_debug("Active quirks (empty means none):%s%s%s%s%s%s%s%s%s%s%s%s%s%s\n", | |
| 1722 | 1726 | i8042_nokbd ? " nokbd" : "", |
| 1723 | 1727 | i8042_noaux ? " noaux" : "", |
| 1724 | 1728 | i8042_nomux ? " nomux" : "", |
| @@ -1738,10 +1742,11 @@ static int __init i8042_platform_init(void) · 13 unmodified lines | ||
| 1738 | 1742 | "", |
| 1739 | 1743 | #endif |
| 1740 | 1744 | #ifdef CONFIG_PNP |
| 1741 | − i8042_nopnp ? " nopnp" : ""); | |
| 1745 | + i8042_nopnp ? " nopnp" : "", | |
| 1742 | 1746 | #else |
| 1743 | − ""); | |
| 1747 | + "", | |
| 1744 | 1748 | #endif |
| 1749 | + i8042_forcenorestore ? " forcenorestore" : ""); | |
| 1745 | 1750 | |
| 1746 | 1751 | retval = i8042_pnp_init(); |
| 1747 | 1752 | if (retval) |
| @@ -115,6 +115,10 @@ module_param_named(nopnp, i8042_nopnp, bool, 0); · 114 unmodified lines | ||
| 115 | 115 | MODULE_PARM_DESC(nopnp, "Do not use PNP to detect controller settings"); |
| 116 | 116 | #endif |
| 117 | 117 | |
| 118 | +static bool i8042_forcenorestore; | |
| 119 | +module_param_named(forcenorestore, i8042_forcenorestore, bool, 0); | |
| 120 | +MODULE_PARM_DESC(forcenorestore, "Force no restore on s3 resume, copying s2idle behaviour"); | |
| 121 | + | |
| 118 | 122 | #define DEBUG |
| 119 | 123 | #ifdef DEBUG |
| 120 | 124 | static bool i8042_debug; |
| @@ -1232,7 +1236,7 @@ static int i8042_pm_suspend(struct device *dev) · 1111 unmodified lines | ||
| 1232 | 1236 | { |
| 1233 | 1237 | int i; |
| 1234 | 1238 | |
| 1235 | − if (pm_suspend_via_firmware()) | |
| 1239 | + if (!i8042_forcenorestore && pm_suspend_via_firmware()) | |
| 1236 | 1240 | i8042_controller_reset(true); |
| 1237 | 1241 | |
| 1238 | 1242 | /* Set up serio interrupts for system wakeup. */ |
| @@ -1248,7 +1252,7 @@ static int i8042_pm_suspend(struct device *dev) · 9 unmodified lines | ||
| 1248 | 1252 | |
| 1249 | 1253 | static int i8042_pm_resume_noirq(struct device *dev) |
| 1250 | 1254 | { |
| 1251 | − if (!pm_resume_via_firmware()) | |
| 1255 | + if (i8042_forcenorestore || !pm_resume_via_firmware()) | |
| 1252 | 1256 | i8042_interrupt(0, NULL); |
| 1253 | 1257 | |
| 1254 | 1258 | return 0; |
| @@ -1271,7 +1275,7 @@ static int i8042_pm_resume(struct device *dev) · 16 unmodified lines | ||
| 1271 | 1275 | * not restore the controller state to whatever it had been at boot |
| 1272 | 1276 | * time, so we do not need to do anything. |
| 1273 | 1277 | */ |
| 1274 | − if (!pm_suspend_via_firmware()) | |
| 1278 | + if (i8042_forcenorestore || !pm_suspend_via_firmware()) | |
| 1275 | 1279 | return 0; |
| 1276 | 1280 | |
| 1277 | 1281 | /* |
| @@ -824,7 +824,7 @@ static void ads7846_read_state(struct ads7846 *ts) · 823 unmodified lines | ||
| 824 | 824 | m = &ts->msg[msg_idx]; |
| 825 | 825 | error = spi_sync(ts->spi, m); |
| 826 | 826 | if (error) { |
| 827 | − dev_err(&ts->spi->dev, "spi_sync --> %d\n", error); | |
| 827 | + dev_err_ratelimited(&ts->spi->dev, "spi_sync --> %d\n", error); | |
| 828 | 828 | packet->ignore = true; |
| 829 | 829 | return; |
| 830 | 830 | } |
| @@ -871,7 +871,7 @@ static void cyttsp4_get_mt_touches(struct cyttsp4_mt_data *md, int num_cur_tch) · 870 unmodified lines | ||
| 871 | 871 | struct cyttsp4_touch tch; |
| 872 | 872 | int sig; |
| 873 | 873 | int i, j, t = 0; |
| 874 | − int ids[max(CY_TMA1036_MAX_TCH, CY_TMA4XX_MAX_TCH)]; | |
| 874 | + int ids[MAX(CY_TMA1036_MAX_TCH, CY_TMA4XX_MAX_TCH)]; | |
| 875 | 875 | |
| 876 | 876 | memset(ids, 0, si->si_ofs.tch_abs[CY_TCH_T].max * sizeof(int)); |
| 877 | 877 | for (i = 0; i < num_cur_tch; i++) { |
| @@ -1474,6 +1474,10 @@ static const struct edt_i2c_chip_data edt_ft6236_data = { · 1473 unmodified lines | ||
| 1474 | 1474 | .max_support_points = 2, |
| 1475 | 1475 | }; |
| 1476 | 1476 | |
| 1477 | +static const struct edt_i2c_chip_data edt_ft8201_data = { | |
| 1478 | + .max_support_points = 10, | |
| 1479 | +}; | |
| 1480 | + | |
| 1477 | 1481 | static const struct edt_i2c_chip_data edt_ft8719_data = { |
| 1478 | 1482 | .max_support_points = 10, |
| 1479 | 1483 | }; |
| @@ -1485,6 +1489,7 @@ static const struct i2c_device_id edt_ft5x06_ts_id[] = { · 5 unmodified lines | ||
| 1485 | 1489 | { .name = "ft5452", .driver_data = (long)&edt_ft5452_data }, |
| 1486 | 1490 | /* Note no edt- prefix for compatibility with the ft6236.c driver */ |
| 1487 | 1491 | { .name = "ft6236", .driver_data = (long)&edt_ft6236_data }, |
| 1492 | + { .name = "ft8201", .driver_data = (long)&edt_ft8201_data }, | |
| 1488 | 1493 | { .name = "ft8719", .driver_data = (long)&edt_ft8719_data }, |
| 1489 | 1494 | { /* sentinel */ } |
| 1490 | 1495 | }; |
| @@ -1500,6 +1505,7 @@ static const struct of_device_id edt_ft5x06_of_match[] = { · 9 unmodified lines | ||
| 1500 | 1505 | { .compatible = "focaltech,ft5452", .data = &edt_ft5452_data }, |
| 1501 | 1506 | /* Note focaltech vendor prefix for compatibility with ft6236.c */ |
| 1502 | 1507 | { .compatible = "focaltech,ft6236", .data = &edt_ft6236_data }, |
| 1508 | + { .compatible = "focaltech,ft8201", .data = &edt_ft8201_data }, | |
| 1503 | 1509 | { .compatible = "focaltech,ft8719", .data = &edt_ft8719_data }, |
| 1504 | 1510 | { /* sentinel */ } |
| 1505 | 1511 | }; |
| @@ -130,17 +130,6 @@ static int himax_bus_read(struct himax_ts_data *ts, u32 address, void *dst, · 129 unmodified lines | ||
| 130 | 130 | return 0; |
| 131 | 131 | } |
| 132 | 132 | |
| 133 | −static int himax_read_mcu(struct himax_ts_data *ts, u32 address, u32 *dst) | |
| 134 | −{ | |
| 135 | − int error; | |
| 136 | − | |
| 137 | − error = himax_bus_read(ts, address, dst, sizeof(dst)); | |
| 138 | − if (error) | |
| 139 | − return error; | |
| 140 | − | |
| 141 | − return 0; | |
| 142 | −} | |
| 143 | − | |
| 144 | 133 | static void himax_reset(struct himax_ts_data *ts) |
| 145 | 134 | { |
| 146 | 135 | gpiod_set_value_cansleep(ts->gpiod_rst, 1); |
| @@ -160,7 +149,8 @@ static int himax_read_product_id(struct himax_ts_data *ts, u32 *product_id) · 13 unmodified lines | ||
| 160 | 149 | { |
| 161 | 150 | int error; |
| 162 | 151 | |
| 163 | − error = himax_read_mcu(ts, HIMAX_REG_ADDR_ICID, product_id); | |
| 152 | + error = himax_bus_read(ts, HIMAX_REG_ADDR_ICID, product_id, | |
| 153 | + sizeof(*product_id)); | |
| 164 | 154 | if (error) |
| 165 | 155 | return error; |
| 166 | 156 | |
| @@ -1777,7 +1777,7 @@ static int arm_smmu_handle_evt(struct arm_smmu_device *smmu, u64 *evt) · 1776 unmodified lines | ||
| 1777 | 1777 | goto out_unlock; |
| 1778 | 1778 | } |
| 1779 | 1779 | |
| 1780 | − iommu_report_device_fault(master->dev, &fault_evt); | |
| 1780 | + ret = iommu_report_device_fault(master->dev, &fault_evt); | |
| 1781 | 1781 | out_unlock: |
| 1782 | 1782 | mutex_unlock(&smmu->streams_mutex); |
| 1783 | 1783 | return ret; |
| @@ -1944,6 +1944,7 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8 · 1943 unmodified lines | ||
| 1944 | 1944 | { |
| 1945 | 1945 | struct intel_iommu *iommu = info->iommu; |
| 1946 | 1946 | struct context_entry *context; |
| 1947 | + u16 did; | |
| 1947 | 1948 | |
| 1948 | 1949 | spin_lock(&iommu->lock); |
| 1949 | 1950 | context = iommu_context_addr(iommu, bus, devfn, 0); |
| @@ -1952,10 +1953,11 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8 · 2 unmodified lines | ||
| 1952 | 1953 | return; |
| 1953 | 1954 | } |
| 1954 | 1955 | |
| 1956 | + did = context_domain_id(context); | |
| 1955 | 1957 | context_clear_entry(context); |
| 1956 | 1958 | __iommu_flush_cache(iommu, context, sizeof(*context)); |
| 1957 | 1959 | spin_unlock(&iommu->lock); |
| 1958 | − intel_context_flush_present(info, context, true); | |
| 1960 | + intel_context_flush_present(info, context, did, true); | |
| 1959 | 1961 | } |
| 1960 | 1962 | |
| 1961 | 1963 | static int domain_setup_first_level(struct intel_iommu *iommu, |
| @@ -4249,6 +4251,7 @@ static int context_flip_pri(struct device_domain_info *info, bool enable) · 2287 unmodified lines | ||
| 4249 | 4251 | struct intel_iommu *iommu = info->iommu; |
| 4250 | 4252 | u8 bus = info->bus, devfn = info->devfn; |
| 4251 | 4253 | struct context_entry *context; |
| 4254 | + u16 did; | |
| 4252 | 4255 | |
| 4253 | 4256 | spin_lock(&iommu->lock); |
| 4254 | 4257 | if (context_copied(iommu, bus, devfn)) { |
| @@ -4261,6 +4264,7 @@ static int context_flip_pri(struct device_domain_info *info, bool enable) · 6 unmodified lines | ||
| 4261 | 4264 | spin_unlock(&iommu->lock); |
| 4262 | 4265 | return -ENODEV; |
| 4263 | 4266 | } |
| 4267 | + did = context_domain_id(context); | |
| 4264 | 4268 | |
| 4265 | 4269 | if (enable) |
| 4266 | 4270 | context_set_sm_pre(context); |
| @@ -4269,7 +4273,7 @@ static int context_flip_pri(struct device_domain_info *info, bool enable) · 2 unmodified lines | ||
| 4269 | 4273 | |
| 4270 | 4274 | if (!ecap_coherent(iommu->ecap)) |
| 4271 | 4275 | clflush_cache_range(context, sizeof(*context)); |
| 4272 | − intel_context_flush_present(info, context, true); | |
| 4276 | + intel_context_flush_present(info, context, did, true); | |
| 4273 | 4277 | spin_unlock(&iommu->lock); |
| 4274 | 4278 | |
| 4275 | 4279 | return 0; |
| @@ -1154,7 +1154,7 @@ void cache_tag_flush_range_np(struct dmar_domain *domain, unsigned long start, · 1153 unmodified lines | ||
| 1154 | 1154 | |
| 1155 | 1155 | void intel_context_flush_present(struct device_domain_info *info, |
| 1156 | 1156 | struct context_entry *context, |
| 1157 | − bool affect_domains); | |
| 1157 | + u16 did, bool affect_domains); | |
| 1158 | 1158 | |
| 1159 | 1159 | #ifdef CONFIG_INTEL_IOMMU_SVM |
| 1160 | 1160 | void intel_svm_check(struct intel_iommu *iommu); |
| @@ -683,6 +683,7 @@ static void device_pasid_table_teardown(struct device *dev, u8 bus, u8 devfn) · 682 unmodified lines | ||
| 683 | 683 | struct device_domain_info *info = dev_iommu_priv_get(dev); |
| 684 | 684 | struct intel_iommu *iommu = info->iommu; |
| 685 | 685 | struct context_entry *context; |
| 686 | + u16 did; | |
| 686 | 687 | |
| 687 | 688 | spin_lock(&iommu->lock); |
| 688 | 689 | context = iommu_context_addr(iommu, bus, devfn, false); |
| @@ -691,10 +692,11 @@ static void device_pasid_table_teardown(struct device *dev, u8 bus, u8 devfn) · 2 unmodified lines | ||
| 691 | 692 | return; |
| 692 | 693 | } |
| 693 | 694 | |
| 695 | + did = context_domain_id(context); | |
| 694 | 696 | context_clear_entry(context); |
| 695 | 697 | __iommu_flush_cache(iommu, context, sizeof(*context)); |
| 696 | 698 | spin_unlock(&iommu->lock); |
| 697 | − intel_context_flush_present(info, context, false); | |
| 699 | + intel_context_flush_present(info, context, did, false); | |
| 698 | 700 | } |
| 699 | 701 | |
| 700 | 702 | static int pci_pasid_table_teardown(struct pci_dev *pdev, u16 alias, void *data) |
| @@ -885,10 +887,9 @@ static void __context_flush_dev_iotlb(struct device_domain_info *info) · 184 unmodified lines | ||
| 885 | 887 | */ |
| 886 | 888 | void intel_context_flush_present(struct device_domain_info *info, |
| 887 | 889 | struct context_entry *context, |
| 888 | − bool flush_domains) | |
| 890 | + u16 did, bool flush_domains) | |
| 889 | 891 | { |
| 890 | 892 | struct intel_iommu *iommu = info->iommu; |
| 891 | − u16 did = context_domain_id(context); | |
| 892 | 893 | struct pasid_entry *pte; |
| 893 | 894 | int i; |
| 894 | 895 | |
| @@ -115,6 +115,59 @@ static struct iopf_group *iopf_group_alloc(struct iommu_fault_param *iopf_param, · 114 unmodified lines | ||
| 115 | 115 | return group; |
| 116 | 116 | } |
| 117 | 117 | |
| 118 | +static struct iommu_attach_handle *find_fault_handler(struct device *dev, | |
| 119 | + struct iopf_fault *evt) | |
| 120 | +{ | |
| 121 | + struct iommu_fault *fault = &evt->fault; | |
| 122 | + struct iommu_attach_handle *attach_handle; | |
| 123 | + | |
| 124 | + if (fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID) { | |
| 125 | + attach_handle = iommu_attach_handle_get(dev->iommu_group, | |
| 126 | + fault->prm.pasid, 0); | |
| 127 | + if (IS_ERR(attach_handle)) { | |
| 128 | + const struct iommu_ops *ops = dev_iommu_ops(dev); | |
| 129 | + | |
| 130 | + if (!ops->user_pasid_table) | |
| 131 | + return NULL; | |
| 132 | + /* | |
| 133 | + * The iommu driver for this device supports user- | |
| 134 | + * managed PASID table. Therefore page faults for | |
| 135 | + * any PASID should go through the NESTING domain | |
| 136 | + * attached to the device RID. | |
| 137 | + */ | |
| 138 | + attach_handle = iommu_attach_handle_get( | |
| 139 | + dev->iommu_group, IOMMU_NO_PASID, | |
| 140 | + IOMMU_DOMAIN_NESTED); | |
| 141 | + if (IS_ERR(attach_handle)) | |
| 142 | + return NULL; | |
| 143 | + } | |
| 144 | + } else { | |
| 145 | + attach_handle = iommu_attach_handle_get(dev->iommu_group, | |
| 146 | + IOMMU_NO_PASID, 0); | |
| 147 | + | |
| 148 | + if (IS_ERR(attach_handle)) | |
| 149 | + return NULL; | |
| 150 | + } | |
| 151 | + | |
| 152 | + if (!attach_handle->domain->iopf_handler) | |
| 153 | + return NULL; | |
| 154 | + | |
| 155 | + return attach_handle; | |
| 156 | +} | |
| 157 | + | |
| 158 | +static void iopf_error_response(struct device *dev, struct iopf_fault *evt) | |
| 159 | +{ | |
| 160 | + const struct iommu_ops *ops = dev_iommu_ops(dev); | |
| 161 | + struct iommu_fault *fault = &evt->fault; | |
| 162 | + struct iommu_page_response resp = { | |
| 163 | + .pasid = fault->prm.pasid, | |
| 164 | + .grpid = fault->prm.grpid, | |
| 165 | + .code = IOMMU_PAGE_RESP_INVALID | |
| 166 | + }; | |
| 167 | + | |
| 168 | + ops->page_response(dev, evt, &resp); | |
| 169 | +} | |
| 170 | + | |
| 118 | 171 | /** |
| 119 | 172 | * iommu_report_device_fault() - Report fault event to device driver |
| 120 | 173 | * @dev: the device |
| @@ -153,23 +206,39 @@ static struct iopf_group *iopf_group_alloc(struct iommu_fault_param *iopf_param, · 32 unmodified lines | ||
| 153 | 206 | * handling framework should guarantee that the iommu domain could only be |
| 154 | 207 | * freed after the device has stopped generating page faults (or the iommu |
| 155 | 208 | * hardware has been set to block the page faults) and the pending page faults |
| 156 | − * have been flushed. | |
| 209 | + * have been flushed. In case no page fault handler is attached or no iopf params | |
| 210 | + * are setup, then the ops->page_response() is called to complete the evt. | |
| 211 | + * | |
| 212 | + * Returns 0 on success, or an error in case of a bad/failed iopf setup. | |
| 157 | 213 | */ |
| 158 | −void iommu_report_device_fault(struct device *dev, struct iopf_fault *evt) | |
| 214 | +int iommu_report_device_fault(struct device *dev, struct iopf_fault *evt) | |
| 159 | 215 | { |
| 216 | + struct iommu_attach_handle *attach_handle; | |
| 160 | 217 | struct iommu_fault *fault = &evt->fault; |
| 161 | 218 | struct iommu_fault_param *iopf_param; |
| 162 | 219 | struct iopf_group abort_group = {}; |
| 163 | 220 | struct iopf_group *group; |
| 164 | 221 | |
| 222 | + attach_handle = find_fault_handler(dev, evt); | |
| 223 | + if (!attach_handle) | |
| 224 | + goto err_bad_iopf; | |
| 225 | + | |
| 226 | + /* | |
| 227 | + * Something has gone wrong if a fault capable domain is attached but no | |
| 228 | + * iopf_param is setup | |
| 229 | + */ | |
| 165 | 230 | iopf_param = iopf_get_dev_fault_param(dev); |
| 166 | 231 | if (WARN_ON(!iopf_param)) |
| 167 | − return; | |
| 232 | + goto err_bad_iopf; | |
| 168 | 233 | |
| 169 | 234 | if (!(fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE)) { |
| 170 | − report_partial_fault(iopf_param, fault); | |
| 235 | + int ret; | |
| 236 | + | |
| 237 | + ret = report_partial_fault(iopf_param, fault); | |
| 171 | 238 | iopf_put_dev_fault_param(iopf_param); |
| 172 | 239 | /* A request that is not the last does not need to be ack'd */ |
| 240 | + | |
| 241 | + return ret; | |
| 173 | 242 | } |
| 174 | 243 | |
| 175 | 244 | /* |
| @@ -184,38 +253,7 @@ void iommu_report_device_fault(struct device *dev, struct iopf_fault *evt) · 8 unmodified lines | ||
| 184 | 253 | if (group == &abort_group) |
| 185 | 254 | goto err_abort; |
| 186 | 255 | |
| 187 | − if (fault->prm.flags & IOMMU_FAULT_PAGE_REQUEST_PASID_VALID) { | |
| 188 | − group->attach_handle = iommu_attach_handle_get(dev->iommu_group, | |
| 189 | − fault->prm.pasid, | |
| 190 | − 0); | |
| 191 | − if (IS_ERR(group->attach_handle)) { | |
| 192 | − const struct iommu_ops *ops = dev_iommu_ops(dev); | |
| 193 | − | |
| 194 | − if (!ops->user_pasid_table) | |
| 195 | − goto err_abort; | |
| 196 | − | |
| 197 | − /* | |
| 198 | − * The iommu driver for this device supports user- | |
| 199 | − * managed PASID table. Therefore page faults for | |
| 200 | − * any PASID should go through the NESTING domain | |
| 201 | − * attached to the device RID. | |
| 202 | − */ | |
| 203 | − group->attach_handle = | |
| 204 | − iommu_attach_handle_get(dev->iommu_group, | |
| 205 | − IOMMU_NO_PASID, | |
| 206 | − IOMMU_DOMAIN_NESTED); | |
| 207 | − if (IS_ERR(group->attach_handle)) | |
| 208 | − goto err_abort; | |
| 209 | − } | |
| 210 | − } else { | |
| 211 | − group->attach_handle = | |
| 212 | − iommu_attach_handle_get(dev->iommu_group, IOMMU_NO_PASID, 0); | |
| 213 | − if (IS_ERR(group->attach_handle)) | |
| 214 | − goto err_abort; | |
| 215 | − } | |
| 216 | − | |
| 217 | − if (!group->attach_handle->domain->iopf_handler) | |
| 218 | − goto err_abort; | |
| 256 | + group->attach_handle = attach_handle; | |
| 219 | 257 | |
| 220 | 258 | /* |
| 221 | 259 | * On success iopf_handler must call iopf_group_response() and |
| @@ -224,7 +262,7 @@ void iommu_report_device_fault(struct device *dev, struct iopf_fault *evt) · 2 unmodified lines | ||
| 224 | 262 | if (group->attach_handle->domain->iopf_handler(group)) |
| 225 | 263 | goto err_abort; |
| 226 | 264 | |
| 227 | − return; | |
| 265 | + return 0; | |
| 228 | 266 | |
| 229 | 267 | err_abort: |
| 230 | 268 | dev_warn_ratelimited(dev, "iopf with pasid %d aborted\n", |
| @@ -234,6 +272,14 @@ err_abort: · 3 unmodified lines | ||
| 234 | 272 | __iopf_free_group(group); |
| 235 | 273 | else |
| 236 | 274 | iopf_free_group(group); |
| 275 | + | |
| 276 | + return 0; | |
| 277 | + | |
| 278 | +err_bad_iopf: | |
| 279 | + if (fault->type == IOMMU_FAULT_PAGE_REQ) | |
| 280 | + iopf_error_response(dev, evt); | |
| 281 | + | |
| 282 | + return -EINVAL; | |
| 237 | 283 | } |
| 238 | 284 | EXPORT_SYMBOL_GPL(iommu_report_device_fault); |
| 239 | 285 | |
| @@ -552,9 +552,8 @@ static int arm_v7s_map_pages(struct io_pgtable_ops *ops, unsigned long iova, · 551 unmodified lines | ||
| 552 | 552 | paddr >= (1ULL << data->iop.cfg.oas))) |
| 553 | 553 | return -ERANGE; |
| 554 | 554 | |
| 555 | − /* If no access, then nothing to do */ | |
| 556 | 555 | if (!(prot & (IOMMU_READ | IOMMU_WRITE))) |
| 557 | − return 0; | |
| 556 | + return -EINVAL; | |
| 558 | 557 | |
| 559 | 558 | while (pgcount--) { |
| 560 | 559 | ret = __arm_v7s_map(data, iova, paddr, pgsize, prot, 1, data->pgd, |
| @@ -515,9 +515,8 @@ static int arm_lpae_map_pages(struct io_pgtable_ops *ops, unsigned long iova, · 514 unmodified lines | ||
| 515 | 515 | if (WARN_ON(iaext || paddr >> cfg->oas)) |
| 516 | 516 | return -ERANGE; |
| 517 | 517 | |
| 518 | − /* If no access, then nothing to do */ | |
| 519 | 518 | if (!(iommu_prot & (IOMMU_READ | IOMMU_WRITE))) |
| 520 | − return 0; | |
| 519 | + return -EINVAL; | |
| 521 | 520 | |
| 522 | 521 | prot = arm_lpae_prot_to_pte(data, iommu_prot); |
| 523 | 522 | ret = __arm_lpae_map(data, iova, paddr, pgsize, pgcount, prot, lvl, |
| @@ -245,9 +245,8 @@ static int dart_map_pages(struct io_pgtable_ops *ops, unsigned long iova, · 244 unmodified lines | ||
| 245 | 245 | if (WARN_ON(paddr >> cfg->oas)) |
| 246 | 246 | return -ERANGE; |
| 247 | 247 | |
| 248 | − /* If no access, then nothing to do */ | |
| 249 | 248 | if (!(iommu_prot & (IOMMU_READ | IOMMU_WRITE))) |
| 250 | − return 0; | |
| 249 | + return -EINVAL; | |
| 251 | 250 | |
| 252 | 251 | tbl = dart_get_table(data, iova); |
| 253 | 252 | |
| @@ -526,7 +526,7 @@ iommufd_device_do_replace(struct iommufd_device *idev, · 525 unmodified lines | ||
| 526 | 526 | err_unresv: |
| 527 | 527 | if (hwpt_is_paging(hwpt)) |
| 528 | 528 | iommufd_group_remove_reserved_iova(igroup, |
| 529 | − to_hwpt_paging(old_hwpt)); | |
| 529 | + to_hwpt_paging(hwpt)); | |
| 530 | 530 | err_unlock: |
| 531 | 531 | mutex_unlock(&idev->igroup->lock); |
| 532 | 532 | return ERR_PTR(rc); |
| @@ -213,6 +213,10 @@ int iommufd_ioas_map(struct iommufd_ucmd *ucmd) · 212 unmodified lines | ||
| 213 | 213 | if (cmd->iova >= ULONG_MAX || cmd->length >= ULONG_MAX) |
| 214 | 214 | return -EOVERFLOW; |
| 215 | 215 | |
| 216 | + if (!(cmd->flags & | |
| 217 | + (IOMMU_IOAS_MAP_WRITEABLE | IOMMU_IOAS_MAP_READABLE))) | |
| 218 | + return -EINVAL; | |
| 219 | + | |
| 216 | 220 | ioas = iommufd_get_ioas(ucmd->ictx, cmd->ioas_id); |
| 217 | 221 | if (IS_ERR(ioas)) |
| 218 | 222 | return PTR_ERR(ioas); |
| @@ -253,6 +257,10 @@ int iommufd_ioas_copy(struct iommufd_ucmd *ucmd) · 34 unmodified lines | ||
| 253 | 257 | cmd->dst_iova >= ULONG_MAX) |
| 254 | 258 | return -EOVERFLOW; |
| 255 | 259 | |
| 260 | + if (!(cmd->flags & | |
| 261 | + (IOMMU_IOAS_MAP_WRITEABLE | IOMMU_IOAS_MAP_READABLE))) | |
| 262 | + return -EINVAL; | |
| 263 | + | |
| 256 | 264 | src_ioas = iommufd_get_ioas(ucmd->ictx, cmd->src_ioas_id); |
| 257 | 265 | if (IS_ERR(src_ioas)) |
| 258 | 266 | return PTR_ERR(src_ioas); |
| @@ -273,7 +273,7 @@ static int mock_domain_read_and_clear_dirty(struct iommu_domain *domain, · 272 unmodified lines | ||
| 273 | 273 | return 0; |
| 274 | 274 | } |
| 275 | 275 | |
| 276 | −const struct iommu_dirty_ops dirty_ops = { | |
| 276 | +static const struct iommu_dirty_ops dirty_ops = { | |
| 277 | 277 | .set_dirty_tracking = mock_domain_set_dirty_tracking, |
| 278 | 278 | .read_and_clear_dirty = mock_domain_read_and_clear_dirty, |
| 279 | 279 | }; |
| @@ -18,11 +18,13 @@ struct fwnode_handle *cpuintc_handle; · 17 unmodified lines | ||
| 18 | 18 | |
| 19 | 19 | static u32 lpic_gsi_to_irq(u32 gsi) |
| 20 | 20 | { |
| 21 | + int irq = 0; | |
| 22 | + | |
| 21 | 23 | /* Only pch irqdomain transferring is required for LoongArch. */ |
| 22 | 24 | if (gsi >= GSI_MIN_PCH_IRQ && gsi <= GSI_MAX_PCH_IRQ) |
| 23 | − return acpi_register_gsi(NULL, gsi, ACPI_LEVEL_SENSITIVE, ACPI_ACTIVE_HIGH); | |
| 25 | + irq = acpi_register_gsi(NULL, gsi, ACPI_LEVEL_SENSITIVE, ACPI_ACTIVE_HIGH); | |
| 24 | 26 | |
| 25 | − return 0; | |
| 27 | + return (irq > 0) ? irq : 0; | |
| 26 | 28 | } |
| 27 | 29 | |
| 28 | 30 | static struct fwnode_handle *lpic_get_gsi_domain_id(u32 gsi) |
| @@ -64,6 +64,20 @@ struct mbigen_device { · 63 unmodified lines | ||
| 64 | 64 | void __iomem *base; |
| 65 | 65 | }; |
| 66 | 66 | |
| 67 | +static inline unsigned int get_mbigen_node_offset(unsigned int nid) | |
| 68 | +{ | |
| 69 | + unsigned int offset = nid * MBIGEN_NODE_OFFSET; | |
| 70 | + | |
| 71 | + /* | |
| 72 | + * To avoid touched clear register in unexpected way, we need to directly | |
| 73 | + * skip clear register when access to more than 10 mbigen nodes. | |
| 74 | + */ | |
| 75 | + if (nid >= (REG_MBIGEN_CLEAR_OFFSET / MBIGEN_NODE_OFFSET)) | |
| 76 | + offset += MBIGEN_NODE_OFFSET; | |
| 77 | + | |
| 78 | + return offset; | |
| 79 | +} | |
| 80 | + | |
| 67 | 81 | static inline unsigned int get_mbigen_vec_reg(irq_hw_number_t hwirq) |
| 68 | 82 | { |
| 69 | 83 | unsigned int nid, pin; |
| @@ -72,8 +86,7 @@ static inline unsigned int get_mbigen_vec_reg(irq_hw_number_t hwirq) · 2 unmodified lines | ||
| 72 | 86 | nid = hwirq / IRQS_PER_MBIGEN_NODE + 1; |
| 73 | 87 | pin = hwirq % IRQS_PER_MBIGEN_NODE; |
| 74 | 88 | |
| 75 | − return pin * 4 + nid * MBIGEN_NODE_OFFSET | |
| 76 | − + REG_MBIGEN_VEC_OFFSET; | |
| 89 | + return pin * 4 + get_mbigen_node_offset(nid) + REG_MBIGEN_VEC_OFFSET; | |
| 77 | 90 | } |
| 78 | 91 | |
| 79 | 92 | static inline void get_mbigen_type_reg(irq_hw_number_t hwirq, |
| @@ -88,8 +101,7 @@ static inline void get_mbigen_type_reg(irq_hw_number_t hwirq, · 8 unmodified lines | ||
| 88 | 101 | *mask = 1 << (irq_ofst % 32); |
| 89 | 102 | ofst = irq_ofst / 32 * 4; |
| 90 | 103 | |
| 91 | − *addr = ofst + nid * MBIGEN_NODE_OFFSET | |
| 92 | − + REG_MBIGEN_TYPE_OFFSET; | |
| 104 | + *addr = ofst + get_mbigen_node_offset(nid) + REG_MBIGEN_TYPE_OFFSET; | |
| 93 | 105 | } |
| 94 | 106 | |
| 95 | 107 | static inline void get_mbigen_clear_reg(irq_hw_number_t hwirq, |
| @@ -178,7 +178,7 @@ struct meson_gpio_irq_controller { · 177 unmodified lines | ||
| 178 | 178 | void __iomem *base; |
| 179 | 179 | u32 channel_irqs[MAX_NUM_CHANNEL]; |
| 180 | 180 | DECLARE_BITMAP(channel_map, MAX_NUM_CHANNEL); |
| 181 | − spinlock_t lock; | |
| 181 | + raw_spinlock_t lock; | |
| 182 | 182 | }; |
| 183 | 183 | |
| 184 | 184 | static void meson_gpio_irq_update_bits(struct meson_gpio_irq_controller *ctl, |
| @@ -187,14 +187,14 @@ static void meson_gpio_irq_update_bits(struct meson_gpio_irq_controller *ctl, · 2 unmodified lines | ||
| 187 | 187 | unsigned long flags; |
| 188 | 188 | u32 tmp; |
| 189 | 189 | |
| 190 | − spin_lock_irqsave(&ctl->lock, flags); | |
| 190 | + raw_spin_lock_irqsave(&ctl->lock, flags); | |
| 191 | 191 | |
| 192 | 192 | tmp = readl_relaxed(ctl->base + reg); |
| 193 | 193 | tmp &= ~mask; |
| 194 | 194 | tmp |= val; |
| 195 | 195 | writel_relaxed(tmp, ctl->base + reg); |
| 196 | 196 | |
| 197 | − spin_unlock_irqrestore(&ctl->lock, flags); | |
| 197 | + raw_spin_unlock_irqrestore(&ctl->lock, flags); | |
| 198 | 198 | } |
| 199 | 199 | |
| 200 | 200 | static void meson_gpio_irq_init_dummy(struct meson_gpio_irq_controller *ctl) |
| @@ -244,12 +244,12 @@ meson_gpio_irq_request_channel(struct meson_gpio_irq_controller *ctl, · 43 unmodified lines | ||
| 244 | 244 | unsigned long flags; |
| 245 | 245 | unsigned int idx; |
| 246 | 246 | |
| 247 | − spin_lock_irqsave(&ctl->lock, flags); | |
| 247 | + raw_spin_lock_irqsave(&ctl->lock, flags); | |
| 248 | 248 | |
| 249 | 249 | /* Find a free channel */ |
| 250 | 250 | idx = find_first_zero_bit(ctl->channel_map, ctl->params->nr_channels); |
| 251 | 251 | if (idx >= ctl->params->nr_channels) { |
| 252 | − spin_unlock_irqrestore(&ctl->lock, flags); | |
| 252 | + raw_spin_unlock_irqrestore(&ctl->lock, flags); | |
| 253 | 253 | pr_err("No channel available\n"); |
| 254 | 254 | return -ENOSPC; |
| 255 | 255 | } |
| @@ -257,7 +257,7 @@ meson_gpio_irq_request_channel(struct meson_gpio_irq_controller *ctl, · 1 unmodified line | ||
| 257 | 257 | /* Mark the channel as used */ |
| 258 | 258 | set_bit(idx, ctl->channel_map); |
| 259 | 259 | |
| 260 | − spin_unlock_irqrestore(&ctl->lock, flags); | |
| 260 | + raw_spin_unlock_irqrestore(&ctl->lock, flags); | |
| 261 | 261 | |
| 262 | 262 | /* |
| 263 | 263 | * Setup the mux of the channel to route the signal of the pad |
| @@ -567,7 +567,7 @@ static int meson_gpio_irq_of_init(struct device_node *node, struct device_node * · 303 unmodified lines | ||
| 567 | 567 | if (!ctl) |
| 568 | 568 | return -ENOMEM; |
| 569 | 569 | |
| 570 | − spin_lock_init(&ctl->lock); | |
| 570 | + raw_spin_lock_init(&ctl->lock); | |
| 571 | 571 | |
| 572 | 572 | ctl->base = of_iomap(node, 0); |
| 573 | 573 | if (!ctl->base) { |
| @@ -161,9 +161,9 @@ static int pic32_irq_domain_map(struct irq_domain *d, unsigned int virq, · 160 unmodified lines | ||
| 161 | 161 | return ret; |
| 162 | 162 | } |
| 163 | 163 | |
| 164 | −int pic32_irq_domain_xlate(struct irq_domain *d, struct device_node *ctrlr, | |
| 165 | − const u32 *intspec, unsigned int intsize, | |
| 166 | − irq_hw_number_t *out_hwirq, unsigned int *out_type) | |
| 164 | +static int pic32_irq_domain_xlate(struct irq_domain *d, struct device_node *ctrlr, | |
| 165 | + const u32 *intspec, unsigned int intsize, | |
| 166 | + irq_hw_number_t *out_hwirq, unsigned int *out_type) | |
| 167 | 167 | { |
| 168 | 168 | struct evic_chip_data *priv = d->host_data; |
| 169 | 169 | |
| @@ -32,15 +32,10 @@ static void aplic_msi_irq_unmask(struct irq_data *d) · 31 unmodified lines | ||
| 32 | 32 | aplic_irq_unmask(d); |
| 33 | 33 | } |
| 34 | 34 | |
| 35 | −static void aplic_msi_irq_eoi(struct irq_data *d) | |
| 35 | +static void aplic_msi_irq_retrigger_level(struct irq_data *d) | |
| 36 | 36 | { |
| 37 | 37 | struct aplic_priv *priv = irq_data_get_irq_chip_data(d); |
| 38 | 38 | |
| 39 | − /* | |
| 40 | − * EOI handling is required only for level-triggered interrupts | |
| 41 | − * when APLIC is in MSI mode. | |
| 42 | − */ | |
| 43 | − | |
| 44 | 39 | switch (irqd_get_trigger_type(d)) { |
| 45 | 40 | case IRQ_TYPE_LEVEL_LOW: |
| 46 | 41 | case IRQ_TYPE_LEVEL_HIGH: |
| @@ -59,6 +54,29 @@ static void aplic_msi_irq_eoi(struct irq_data *d) · 12 unmodified lines | ||
| 59 | 54 | } |
| 60 | 55 | } |
| 61 | 56 | |
| 57 | +static void aplic_msi_irq_eoi(struct irq_data *d) | |
| 58 | +{ | |
| 59 | + /* | |
| 60 | + * EOI handling is required only for level-triggered interrupts | |
| 61 | + * when APLIC is in MSI mode. | |
| 62 | + */ | |
| 63 | + aplic_msi_irq_retrigger_level(d); | |
| 64 | +} | |
| 65 | + | |
| 66 | +static int aplic_msi_irq_set_type(struct irq_data *d, unsigned int type) | |
| 67 | +{ | |
| 68 | + int rc = aplic_irq_set_type(d, type); | |
| 69 | + | |
| 70 | + if (rc) | |
| 71 | + return rc; | |
| 72 | + /* | |
| 73 | + * Updating sourcecfg register for level-triggered interrupts | |
| 74 | + * requires interrupt retriggering when APLIC is in MSI mode. | |
| 75 | + */ | |
| 76 | + aplic_msi_irq_retrigger_level(d); | |
| 77 | + return 0; | |
| 78 | +} | |
| 79 | + | |
| 62 | 80 | static void aplic_msi_write_msg(struct irq_data *d, struct msi_msg *msg) |
| 63 | 81 | { |
| 64 | 82 | unsigned int group_index, hart_index, guest_index, val; |
| @@ -130,7 +148,7 @@ static const struct msi_domain_template aplic_msi_template = { · 65 unmodified lines | ||
| 130 | 148 | .name = "APLIC-MSI", |
| 131 | 149 | .irq_mask = aplic_msi_irq_mask, |
| 132 | 150 | .irq_unmask = aplic_msi_irq_unmask, |
| 133 | − .irq_set_type = aplic_irq_set_type, | |
| 151 | + .irq_set_type = aplic_msi_irq_set_type, | |
| 134 | 152 | .irq_eoi = aplic_msi_irq_eoi, |
| 135 | 153 | #ifdef CONFIG_SMP |
| 136 | 154 | .irq_set_affinity = irq_chip_set_affinity_parent, |
| @@ -270,7 +270,7 @@ static const struct irq_domain_ops sun6i_r_intc_domain_ops = { · 269 unmodified lines | ||
| 270 | 270 | |
| 271 | 271 | static int sun6i_r_intc_suspend(void) |
| 272 | 272 | { |
| 273 | − u32 buf[BITS_TO_U32(max(SUN6I_NR_TOP_LEVEL_IRQS, SUN6I_NR_MUX_BITS))]; | |
| 273 | + u32 buf[BITS_TO_U32(MAX(SUN6I_NR_TOP_LEVEL_IRQS, SUN6I_NR_MUX_BITS))]; | |
| 274 | 274 | int i; |
| 275 | 275 | |
| 276 | 276 | /* Wake IRQs are enabled during system sleep and shutdown. */ |
| @@ -189,7 +189,7 @@ static int __init xilinx_intc_of_init(struct device_node *intc, · 188 unmodified lines | ||
| 189 | 189 | irqc->intr_mask = 0; |
| 190 | 190 | } |
| 191 | 191 | |
| 192 | − if (irqc->intr_mask >> irqc->nr_irq) | |
| 192 | + if ((u64)irqc->intr_mask >> irqc->nr_irq) | |
| 193 | 193 | pr_warn("irq-xilinx: mismatch in kind-of-intr param\n"); |
| 194 | 194 | |
| 195 | 195 | pr_info("irq-xilinx: %pOF: num_irq=%d, edge=0x%x\n", |
| @@ -1181,8 +1181,26 @@ static int do_resume(struct dm_ioctl *param) · 1180 unmodified lines | ||
| 1181 | 1181 | suspend_flags &= ~DM_SUSPEND_LOCKFS_FLAG; |
| 1182 | 1182 | if (param->flags & DM_NOFLUSH_FLAG) |
| 1183 | 1183 | suspend_flags |= DM_SUSPEND_NOFLUSH_FLAG; |
| 1184 | − if (!dm_suspended_md(md)) | |
| 1185 | − dm_suspend(md, suspend_flags); | |
| 1184 | + if (!dm_suspended_md(md)) { | |
| 1185 | + r = dm_suspend(md, suspend_flags); | |
| 1186 | + if (r) { | |
| 1187 | + down_write(&_hash_lock); | |
| 1188 | + hc = dm_get_mdptr(md); | |
| 1189 | + if (hc && !hc->new_map) { | |
| 1190 | + hc->new_map = new_map; | |
| 1191 | + new_map = NULL; | |
| 1192 | + } else { | |
| 1193 | + r = -ENXIO; | |
| 1194 | + } | |
| 1195 | + up_write(&_hash_lock); | |
| 1196 | + if (new_map) { | |
| 1197 | + dm_sync_table(md); | |
| 1198 | + dm_table_destroy(new_map); | |
| 1199 | + } | |
| 1200 | + dm_put(md); | |
| 1201 | + return r; | |
| 1202 | + } | |
| 1203 | + } | |
| 1186 | 1204 | |
| 1187 | 1205 | old_size = dm_get_size(md); |
| 1188 | 1206 | old_map = dm_swap_table(md, new_map); |
| @@ -2737,7 +2737,7 @@ static int dm_wait_for_bios_completion(struct mapped_device *md, unsigned int ta · 2736 unmodified lines | ||
| 2737 | 2737 | break; |
| 2738 | 2738 | |
| 2739 | 2739 | if (signal_pending_state(task_state, current)) { |
| 2740 | − r = -EINTR; | |
| 2740 | + r = -ERESTARTSYS; | |
| 2741 | 2741 | break; |
| 2742 | 2742 | } |
| 2743 | 2743 | |
| @@ -2762,7 +2762,7 @@ static int dm_wait_for_completion(struct mapped_device *md, unsigned int task_st · 18 unmodified lines | ||
| 2762 | 2762 | break; |
| 2763 | 2763 | |
| 2764 | 2764 | if (signal_pending_state(task_state, current)) { |
| 2765 | − r = -EINTR; | |
| 2765 | + r = -ERESTARTSYS; | |
| 2766 | 2766 | break; |
| 2767 | 2767 | } |
| 2768 | 2768 | |
| @@ -277,7 +277,7 @@ static void sm_metadata_destroy(struct dm_space_map *sm) · 276 unmodified lines | ||
| 277 | 277 | { |
| 278 | 278 | struct sm_metadata *smm = container_of(sm, struct sm_metadata, sm); |
| 279 | 279 | |
| 280 | − kfree(smm); | |
| 280 | + kvfree(smm); | |
| 281 | 281 | } |
| 282 | 282 | |
| 283 | 283 | static int sm_metadata_get_nr_blocks(struct dm_space_map *sm, dm_block_t *count) |
| @@ -772,7 +772,7 @@ struct dm_space_map *dm_sm_metadata_init(void) · 488 unmodified lines | ||
| 772 | 772 | { |
| 773 | 773 | struct sm_metadata *smm; |
| 774 | 774 | |
| 775 | − smm = kmalloc(sizeof(*smm), GFP_KERNEL); | |
| 775 | + smm = kvmalloc(sizeof(*smm), GFP_KERNEL); | |
| 776 | 776 | if (!smm) |
| 777 | 777 | return ERR_PTR(-ENOMEM); |
| 778 | 778 | |
| @@ -617,6 +617,12 @@ static int choose_first_rdev(struct r1conf *conf, struct r1bio *r1_bio, · 616 unmodified lines | ||
| 617 | 617 | return -1; |
| 618 | 618 | } |
| 619 | 619 | |
| 620 | +static bool rdev_in_recovery(struct md_rdev *rdev, struct r1bio *r1_bio) | |
| 621 | +{ | |
| 622 | + return !test_bit(In_sync, &rdev->flags) && | |
| 623 | + rdev->recovery_offset < r1_bio->sector + r1_bio->sectors; | |
| 624 | +} | |
| 625 | + | |
| 620 | 626 | static int choose_bb_rdev(struct r1conf *conf, struct r1bio *r1_bio, |
| 621 | 627 | int *max_sectors) |
| 622 | 628 | { |
| @@ -635,6 +641,7 @@ static int choose_bb_rdev(struct r1conf *conf, struct r1bio *r1_bio, · 12 unmodified lines | ||
| 635 | 641 | |
| 636 | 642 | rdev = conf->mirrors[disk].rdev; |
| 637 | 643 | if (!rdev || test_bit(Faulty, &rdev->flags) || |
| 644 | + rdev_in_recovery(rdev, r1_bio) || | |
| 638 | 645 | test_bit(WriteMostly, &rdev->flags)) |
| 639 | 646 | continue; |
| 640 | 647 | |
| @@ -673,7 +680,8 @@ static int choose_slow_rdev(struct r1conf *conf, struct r1bio *r1_bio, · 32 unmodified lines | ||
| 673 | 680 | |
| 674 | 681 | rdev = conf->mirrors[disk].rdev; |
| 675 | 682 | if (!rdev || test_bit(Faulty, &rdev->flags) || |
| 676 | − !test_bit(WriteMostly, &rdev->flags)) | |
| 683 | + !test_bit(WriteMostly, &rdev->flags) || | |
| 684 | + rdev_in_recovery(rdev, r1_bio)) | |
| 677 | 685 | continue; |
| 678 | 686 | |
| 679 | 687 | /* there are no bad blocks, we can use this disk */ |
| @@ -733,9 +741,7 @@ static bool rdev_readable(struct md_rdev *rdev, struct r1bio *r1_bio) · 53 unmodified lines | ||
| 733 | 741 | if (!rdev || test_bit(Faulty, &rdev->flags)) |
| 734 | 742 | return false; |
| 735 | 743 | |
| 736 | − /* still in recovery */ | |
| 737 | − if (!test_bit(In_sync, &rdev->flags) && | |
| 738 | − rdev->recovery_offset < r1_bio->sector + r1_bio->sectors) | |
| 744 | + if (rdev_in_recovery(rdev, r1_bio)) | |
| 739 | 745 | return false; |
| 740 | 746 | |
| 741 | 747 | /* don't read from slow disk unless have to */ |
| @@ -25,8 +25,11 @@ · 24 unmodified lines | ||
| 25 | 25 | #endif |
| 26 | 26 | |
| 27 | 27 | /* MACRO definitions */ |
| 28 | +#ifndef MIN | |
| 28 | 29 | #define MAX(X, Y) ((X) >= (Y) ? (X) : (Y)) |
| 29 | 30 | #define MIN(X, Y) ((X) <= (Y) ? (X) : (Y)) |
| 31 | +#endif | |
| 32 | + | |
| 30 | 33 | #define INRANGE(X, Y, Z) \ |
| 31 | 34 | ((((X) <= (Y)) && ((Y) <= (Z))) || \ |
| 32 | 35 | (((Z) <= (Y)) && ((Y) <= (X))) ? 1 : 0) |
| @@ -3,13 +3,14 @@ config VIDEO_INTEL_IPU6 · 2 unmodified lines | ||
| 3 | 3 | depends on ACPI || COMPILE_TEST |
| 4 | 4 | depends on VIDEO_DEV |
| 5 | 5 | depends on X86 && X86_64 && HAS_DMA |
| 6 | + depends on IPU_BRIDGE || !IPU_BRIDGE | |
| 7 | + select AUXILIARY_BUS | |
| 6 | 8 | select DMA_OPS |
| 7 | 9 | select IOMMU_IOVA |
| 8 | 10 | select VIDEO_V4L2_SUBDEV_API |
| 9 | 11 | select MEDIA_CONTROLLER |
| 10 | 12 | select VIDEOBUF2_DMA_CONTIG |
| 11 | 13 | select V4L2_FWNODE |
| 12 | − select IPU_BRIDGE | |
| 13 | 14 | help |
| 14 | 15 | This is the 6th Gen Intel Image Processing Unit, found in Intel SoCs |
| 15 | 16 | and used for capturing images and video from camera sensors. |
| @@ -23,40 +23,11 @@ static int dvb_usb_force_pid_filter_usage; · 22 unmodified lines | ||
| 23 | 23 | module_param_named(force_pid_filter_usage, dvb_usb_force_pid_filter_usage, int, 0444); |
| 24 | 24 | MODULE_PARM_DESC(force_pid_filter_usage, "force all dvb-usb-devices to use a PID filter, if any (default: 0)."); |
| 25 | 25 | |
| 26 | −static int dvb_usb_check_bulk_endpoint(struct dvb_usb_device *d, u8 endpoint) | |
| 27 | −{ | |
| 28 | − if (endpoint) { | |
| 29 | − int ret; | |
| 30 | − | |
| 31 | − ret = usb_pipe_type_check(d->udev, usb_sndbulkpipe(d->udev, endpoint)); | |
| 32 | − if (ret) | |
| 33 | − return ret; | |
| 34 | − ret = usb_pipe_type_check(d->udev, usb_rcvbulkpipe(d->udev, endpoint)); | |
| 35 | − if (ret) | |
| 36 | − return ret; | |
| 37 | − } | |
| 38 | − return 0; | |
| 39 | −} | |
| 40 | − | |
| 41 | −static void dvb_usb_clear_halt(struct dvb_usb_device *d, u8 endpoint) | |
| 42 | −{ | |
| 43 | − if (endpoint) { | |
| 44 | − usb_clear_halt(d->udev, usb_sndbulkpipe(d->udev, endpoint)); | |
| 45 | − usb_clear_halt(d->udev, usb_rcvbulkpipe(d->udev, endpoint)); | |
| 46 | − } | |
| 47 | −} | |
| 48 | − | |
| 49 | 26 | static int dvb_usb_adapter_init(struct dvb_usb_device *d, short *adapter_nrs) |
| 50 | 27 | { |
| 51 | 28 | struct dvb_usb_adapter *adap; |
| 52 | 29 | int ret, n, o; |
| 53 | 30 | |
| 54 | − ret = dvb_usb_check_bulk_endpoint(d, d->props.generic_bulk_ctrl_endpoint); | |
| 55 | − if (ret) | |
| 56 | − return ret; | |
| 57 | − ret = dvb_usb_check_bulk_endpoint(d, d->props.generic_bulk_ctrl_endpoint_response); | |
| 58 | − if (ret) | |
| 59 | − return ret; | |
| 60 | 31 | for (n = 0; n < d->props.num_adapters; n++) { |
| 61 | 32 | adap = &d->adapter[n]; |
| 62 | 33 | adap->dev = d; |
| @@ -132,8 +103,10 @@ static int dvb_usb_adapter_init(struct dvb_usb_device *d, short *adapter_nrs) · 69 unmodified lines | ||
| 132 | 103 | * when reloading the driver w/o replugging the device |
| 133 | 104 | * sometimes a timeout occurs, this helps |
| 134 | 105 | */ |
| 135 | − dvb_usb_clear_halt(d, d->props.generic_bulk_ctrl_endpoint); | |
| 136 | − dvb_usb_clear_halt(d, d->props.generic_bulk_ctrl_endpoint_response); | |
| 106 | + if (d->props.generic_bulk_ctrl_endpoint != 0) { | |
| 107 | + usb_clear_halt(d->udev, usb_sndbulkpipe(d->udev, d->props.generic_bulk_ctrl_endpoint)); | |
| 108 | + usb_clear_halt(d->udev, usb_rcvbulkpipe(d->udev, d->props.generic_bulk_ctrl_endpoint)); | |
| 109 | + } | |
| 137 | 110 | |
| 138 | 111 | return 0; |
| 139 | 112 | |
| @@ -2680,6 +2680,10 @@ static void uvc_ctrl_init_ctrl(struct uvc_video_chain *chain, · 2679 unmodified lines | ||
| 2680 | 2680 | for (i = 0; i < ARRAY_SIZE(uvc_ctrl_mappings); ++i) { |
| 2681 | 2681 | const struct uvc_control_mapping *mapping = &uvc_ctrl_mappings[i]; |
| 2682 | 2682 | |
| 2683 | + if (!uvc_entity_match_guid(ctrl->entity, mapping->entity) || | |
| 2684 | + ctrl->info.selector != mapping->selector) | |
| 2685 | + continue; | |
| 2686 | + | |
| 2683 | 2687 | /* Let the device provide a custom mapping. */ |
| 2684 | 2688 | if (mapping->filter_mapping) { |
| 2685 | 2689 | mapping = mapping->filter_mapping(chain, ctrl); |
| @@ -2687,9 +2691,7 @@ static void uvc_ctrl_init_ctrl(struct uvc_video_chain *chain, · 1 unmodified line | ||
| 2687 | 2691 | continue; |
| 2688 | 2692 | } |
| 2689 | 2693 | |
| 2690 | − if (uvc_entity_match_guid(ctrl->entity, mapping->entity) && | |
| 2691 | − ctrl->info.selector == mapping->selector) | |
| 2692 | − __uvc_ctrl_add_mapping(chain, ctrl, mapping); | |
| 2694 | + __uvc_ctrl_add_mapping(chain, ctrl, mapping); | |
| 2693 | 2695 | } |
| 2694 | 2696 | } |
| 2695 | 2697 | |
| @@ -587,7 +587,7 @@ config NSM · 586 unmodified lines | ||
| 587 | 587 | |
| 588 | 588 | config MARVELL_CN10K_DPI |
| 589 | 589 | tristate "Octeon CN10K DPI driver" |
| 590 | − depends on PCI | |
| 590 | + depends on PCI && PCI_IOV | |
| 591 | 591 | depends on ARCH_THUNDER || (COMPILE_TEST && 64BIT) |
| 592 | 592 | help |
| 593 | 593 | Enables Octeon CN10K DMA packet interface (DPI) driver which |
| @@ -233,6 +233,49 @@ static void ee1004_cleanup_bus_data(void *data) · 232 unmodified lines | ||
| 233 | 233 | mutex_unlock(&ee1004_bus_lock); |
| 234 | 234 | } |
| 235 | 235 | |
| 236 | +static int ee1004_init_bus_data(struct i2c_client *client) | |
| 237 | +{ | |
| 238 | + struct ee1004_bus_data *bd; | |
| 239 | + int err, cnr = 0; | |
| 240 | + | |
| 241 | + bd = ee1004_get_bus_data(client->adapter); | |
| 242 | + if (!bd) | |
| 243 | + return dev_err_probe(&client->dev, -ENOSPC, "Only %d busses supported", | |
| 244 | + EE1004_MAX_BUSSES); | |
| 245 | + | |
| 246 | + i2c_set_clientdata(client, bd); | |
| 247 | + | |
| 248 | + if (++bd->dev_count == 1) { | |
| 249 | + /* Use 2 dummy devices for page select command */ | |
| 250 | + for (cnr = 0; cnr < EE1004_NUM_PAGES; cnr++) { | |
| 251 | + struct i2c_client *cl; | |
| 252 | + | |
| 253 | + cl = i2c_new_dummy_device(client->adapter, EE1004_ADDR_SET_PAGE + cnr); | |
| 254 | + if (IS_ERR(cl)) { | |
| 255 | + err = PTR_ERR(cl); | |
| 256 | + goto err_out; | |
| 257 | + } | |
| 258 | + | |
| 259 | + bd->set_page[cnr] = cl; | |
| 260 | + } | |
| 261 | + | |
| 262 | + /* Remember current page to avoid unneeded page select */ | |
| 263 | + err = ee1004_get_current_page(bd); | |
| 264 | + if (err < 0) | |
| 265 | + goto err_out; | |
| 266 | + | |
| 267 | + dev_dbg(&client->dev, "Currently selected page: %d\n", err); | |
| 268 | + bd->current_page = err; | |
| 269 | + } | |
| 270 | + | |
| 271 | + return 0; | |
| 272 | + | |
| 273 | +err_out: | |
| 274 | + ee1004_cleanup(cnr, bd); | |
| 275 | + | |
| 276 | + return err; | |
| 277 | +} | |
| 278 | + | |
| 236 | 279 | static int ee1004_probe(struct i2c_client *client) |
| 237 | 280 | { |
| 238 | 281 | struct nvmem_config config = { |
| @@ -251,9 +294,8 @@ static int ee1004_probe(struct i2c_client *client) · 12 unmodified lines | ||
| 251 | 294 | .compat = true, |
| 252 | 295 | .base_dev = &client->dev, |
| 253 | 296 | }; |
| 254 | − struct ee1004_bus_data *bd; | |
| 255 | 297 | struct nvmem_device *ndev; |
| 256 | − int err, cnr = 0; | |
| 298 | + int err; | |
| 257 | 299 | |
| 258 | 300 | /* Make sure we can operate on this adapter */ |
| 259 | 301 | if (!i2c_check_functionality(client->adapter, |
| @@ -264,46 +306,21 @@ static int ee1004_probe(struct i2c_client *client) · 4 unmodified lines | ||
| 264 | 306 | |
| 265 | 307 | mutex_lock(&ee1004_bus_lock); |
| 266 | 308 | |
| 267 | − bd = ee1004_get_bus_data(client->adapter); | |
| 268 | − if (!bd) { | |
| 309 | + err = ee1004_init_bus_data(client); | |
| 310 | + if (err < 0) { | |
| 269 | 311 | mutex_unlock(&ee1004_bus_lock); |
| 270 | − return dev_err_probe(&client->dev, -ENOSPC, | |
| 271 | − "Only %d busses supported", EE1004_MAX_BUSSES); | |
| 272 | − } | |
| 273 | − | |
| 274 | − err = devm_add_action_or_reset(&client->dev, ee1004_cleanup_bus_data, bd); | |
| 275 | − if (err < 0) | |
| 276 | 312 | return err; |
| 277 | − | |
| 278 | − i2c_set_clientdata(client, bd); | |
| 279 | − | |
| 280 | − if (++bd->dev_count == 1) { | |
| 281 | − /* Use 2 dummy devices for page select command */ | |
| 282 | − for (cnr = 0; cnr < EE1004_NUM_PAGES; cnr++) { | |
| 283 | − struct i2c_client *cl; | |
| 284 | − | |
| 285 | − cl = i2c_new_dummy_device(client->adapter, EE1004_ADDR_SET_PAGE + cnr); | |
| 286 | − if (IS_ERR(cl)) { | |
| 287 | − mutex_unlock(&ee1004_bus_lock); | |
| 288 | − return PTR_ERR(cl); | |
| 289 | − } | |
| 290 | − bd->set_page[cnr] = cl; | |
| 291 | − } | |
| 292 | − | |
| 293 | − /* Remember current page to avoid unneeded page select */ | |
| 294 | − err = ee1004_get_current_page(bd); | |
| 295 | − if (err < 0) { | |
| 296 | − mutex_unlock(&ee1004_bus_lock); | |
| 297 | − return err; | |
| 298 | − } | |
| 299 | − dev_dbg(&client->dev, "Currently selected page: %d\n", err); | |
| 300 | − bd->current_page = err; | |
| 301 | 313 | } |
| 302 | 314 | |
| 303 | 315 | ee1004_probe_temp_sensor(client); |
| 304 | 316 | |
| 305 | 317 | mutex_unlock(&ee1004_bus_lock); |
| 306 | 318 | |
| 319 | + err = devm_add_action_or_reset(&client->dev, ee1004_cleanup_bus_data, | |
| 320 | + i2c_get_clientdata(client)); | |
| 321 | + if (err < 0) | |
| 322 | + return err; | |
| 323 | + | |
| 307 | 324 | ndev = devm_nvmem_register(&client->dev, &config); |
| 308 | 325 | if (IS_ERR(ndev)) |
| 309 | 326 | return PTR_ERR(ndev); |
| @@ -2085,16 +2085,6 @@ err_invoke: · 2084 unmodified lines | ||
| 2085 | 2085 | return err; |
| 2086 | 2086 | } |
| 2087 | 2087 | |
| 2088 | −static int is_attach_rejected(struct fastrpc_user *fl) | |
| 2089 | −{ | |
| 2090 | − /* Check if the device node is non-secure */ | |
| 2091 | − if (!fl->is_secure_dev) { | |
| 2092 | − dev_dbg(&fl->cctx->rpdev->dev, "untrusted app trying to attach to privileged DSP PD\n"); | |
| 2093 | − return -EACCES; | |
| 2094 | − } | |
| 2095 | − return 0; | |
| 2096 | −} | |
| 2097 | − | |
| 2098 | 2088 | static long fastrpc_device_ioctl(struct file *file, unsigned int cmd, |
| 2099 | 2089 | unsigned long arg) |
| 2100 | 2090 | { |
| @@ -2107,19 +2097,13 @@ static long fastrpc_device_ioctl(struct file *file, unsigned int cmd, · 6 unmodified lines | ||
| 2107 | 2097 | err = fastrpc_invoke(fl, argp); |
| 2108 | 2098 | break; |
| 2109 | 2099 | case FASTRPC_IOCTL_INIT_ATTACH: |
| 2110 | − err = is_attach_rejected(fl); | |
| 2111 | − if (!err) | |
| 2112 | − err = fastrpc_init_attach(fl, ROOT_PD); | |
| 2100 | + err = fastrpc_init_attach(fl, ROOT_PD); | |
| 2113 | 2101 | break; |
| 2114 | 2102 | case FASTRPC_IOCTL_INIT_ATTACH_SNS: |
| 2115 | − err = is_attach_rejected(fl); | |
| 2116 | − if (!err) | |
| 2117 | − err = fastrpc_init_attach(fl, SENSORS_PD); | |
| 2103 | + err = fastrpc_init_attach(fl, SENSORS_PD); | |
| 2118 | 2104 | break; |
| 2119 | 2105 | case FASTRPC_IOCTL_INIT_CREATE_STATIC: |
| 2120 | − err = is_attach_rejected(fl); | |
| 2121 | − if (!err) | |
| 2122 | − err = fastrpc_init_create_static_process(fl, argp); | |
| 2106 | + err = fastrpc_init_create_static_process(fl, argp); | |
| 2123 | 2107 | break; |
| 2124 | 2108 | case FASTRPC_IOCTL_INIT_CREATE: |
| 2125 | 2109 | err = fastrpc_init_create_process(fl, argp); |
| @@ -182,6 +182,21 @@ static void lkdtm_REFCOUNT_SUB_AND_TEST_NEGATIVE(void) · 181 unmodified lines | ||
| 182 | 182 | check_negative(&neg, 3); |
| 183 | 183 | } |
| 184 | 184 | |
| 185 | +/* | |
| 186 | + * A refcount_sub_and_test() by zero when the counter is at zero should act like | |
| 187 | + * refcount_sub_and_test() above when going negative. | |
| 188 | + */ | |
| 189 | +static void lkdtm_REFCOUNT_SUB_AND_TEST_ZERO(void) | |
| 190 | +{ | |
| 191 | + refcount_t neg = REFCOUNT_INIT(0); | |
| 192 | + | |
| 193 | + pr_info("attempting bad refcount_sub_and_test() at zero\n"); | |
| 194 | + if (refcount_sub_and_test(0, &neg)) | |
| 195 | + pr_warn("Weird: refcount_sub_and_test() reported zero\n"); | |
| 196 | + | |
| 197 | + check_negative(&neg, 0); | |
| 198 | +} | |
| 199 | + | |
| 185 | 200 | static void check_from_zero(refcount_t *ref) |
| 186 | 201 | { |
| 187 | 202 | switch (refcount_read(ref)) { |
| @@ -400,6 +415,7 @@ static struct crashtype crashtypes[] = { · 212 unmodified lines | ||
| 400 | 415 | CRASHTYPE(REFCOUNT_DEC_NEGATIVE), |
| 401 | 416 | CRASHTYPE(REFCOUNT_DEC_AND_TEST_NEGATIVE), |
| 402 | 417 | CRASHTYPE(REFCOUNT_SUB_AND_TEST_NEGATIVE), |
| 418 | + CRASHTYPE(REFCOUNT_SUB_AND_TEST_ZERO), | |
| 403 | 419 | CRASHTYPE(REFCOUNT_INC_ZERO), |
| 404 | 420 | CRASHTYPE(REFCOUNT_ADD_ZERO), |
| 405 | 421 | CRASHTYPE(REFCOUNT_INC_SATURATED), |
| @@ -3125,13 +3125,13 @@ static ssize_t mtf_test_write(struct file *file, const char __user *buf, · 3124 unmodified lines | ||
| 3125 | 3125 | test->buffer = kzalloc(BUFFER_SIZE, GFP_KERNEL); |
| 3126 | 3126 | #ifdef CONFIG_HIGHMEM |
| 3127 | 3127 | test->highmem = alloc_pages(GFP_KERNEL | __GFP_HIGHMEM, BUFFER_ORDER); |
| 3128 | + if (!test->highmem) { | |
| 3129 | + count = -ENOMEM; | |
| 3130 | + goto free_test_buffer; | |
| 3131 | + } | |
| 3128 | 3132 | #endif |
| 3129 | 3133 | |
| 3130 | −#ifdef CONFIG_HIGHMEM | |
| 3131 | − if (test->buffer && test->highmem) { | |
| 3132 | −#else | |
| 3133 | 3134 | if (test->buffer) { |
| 3134 | −#endif | |
| 3135 | 3135 | mutex_lock(&mmc_test_lock); |
| 3136 | 3136 | mmc_test_run(test, testcase); |
| 3137 | 3137 | mutex_unlock(&mmc_test_lock); |
| @@ -3139,6 +3139,7 @@ static ssize_t mtf_test_write(struct file *file, const char __user *buf, · 1 unmodified line | ||
| 3139 | 3139 | |
| 3140 | 3140 | #ifdef CONFIG_HIGHMEM |
| 3141 | 3141 | __free_pages(test->highmem, BUFFER_ORDER); |
| 3142 | +free_test_buffer: | |
| 3142 | 3143 | #endif |
| 3143 | 3144 | kfree(test->buffer); |
| 3144 | 3145 | kfree(test); |
| @@ -3299,6 +3299,10 @@ int dw_mci_probe(struct dw_mci *host) · 3298 unmodified lines | ||
| 3299 | 3299 | host->biu_clk = devm_clk_get(host->dev, "biu"); |
| 3300 | 3300 | if (IS_ERR(host->biu_clk)) { |
| 3301 | 3301 | dev_dbg(host->dev, "biu clock not available\n"); |
| 3302 | + ret = PTR_ERR(host->biu_clk); | |
| 3303 | + if (ret == -EPROBE_DEFER) | |
| 3304 | + return ret; | |
| 3305 | + | |
| 3302 | 3306 | } else { |
| 3303 | 3307 | ret = clk_prepare_enable(host->biu_clk); |
| 3304 | 3308 | if (ret) { |
| @@ -3310,6 +3314,10 @@ int dw_mci_probe(struct dw_mci *host) · 5 unmodified lines | ||
| 3310 | 3314 | host->ciu_clk = devm_clk_get(host->dev, "ciu"); |
| 3311 | 3315 | if (IS_ERR(host->ciu_clk)) { |
| 3312 | 3316 | dev_dbg(host->dev, "ciu clock not available\n"); |
| 3317 | + ret = PTR_ERR(host->ciu_clk); | |
| 3318 | + if (ret == -EPROBE_DEFER) | |
| 3319 | + goto err_clk_biu; | |
| 3320 | + | |
| 3313 | 3321 | host->bus_hz = host->pdata->bus_hz; |
| 3314 | 3322 | } else { |
| 3315 | 3323 | ret = clk_prepare_enable(host->ciu_clk); |
| @@ -1230,7 +1230,7 @@ static bool msdc_cmd_done(struct msdc_host *host, int events, · 1229 unmodified lines | ||
| 1230 | 1230 | } |
| 1231 | 1231 | |
| 1232 | 1232 | if (!sbc_error && !(events & MSDC_INT_CMDRDY)) { |
| 1233 | − if (events & MSDC_INT_CMDTMO || | |
| 1233 | + if ((events & MSDC_INT_CMDTMO && !host->hs400_tuning) || | |
| 1234 | 1234 | (!mmc_op_tuning(cmd->opcode) && !host->hs400_tuning)) |
| 1235 | 1235 | /* |
| 1236 | 1236 | * should not clear fifo/interrupt as the tune data |
| @@ -1323,9 +1323,9 @@ static void msdc_start_command(struct msdc_host *host, · 86 unmodified lines | ||
| 1323 | 1323 | static void msdc_cmd_next(struct msdc_host *host, |
| 1324 | 1324 | struct mmc_request *mrq, struct mmc_command *cmd) |
| 1325 | 1325 | { |
| 1326 | − if ((cmd->error && | |
| 1327 | − !(cmd->error == -EILSEQ && | |
| 1328 | − (mmc_op_tuning(cmd->opcode) || host->hs400_tuning))) || | |
| 1326 | + if ((cmd->error && !host->hs400_tuning && | |
| 1327 | + !(cmd->error == -EILSEQ && | |
| 1328 | + mmc_op_tuning(cmd->opcode))) || | |
| 1329 | 1329 | (mrq->sbc && mrq->sbc->error)) |
| 1330 | 1330 | msdc_request_done(host, mrq); |
| 1331 | 1331 | else if (cmd == mrq->sbc) |
| @@ -427,6 +427,8 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs, · 426 unmodified lines | ||
| 427 | 427 | struct netlink_ext_ack *extack) |
| 428 | 428 | { |
| 429 | 429 | struct net_device *bond_dev = xs->xso.dev; |
| 430 | + struct net_device *real_dev; | |
| 431 | + netdevice_tracker tracker; | |
| 430 | 432 | struct bond_ipsec *ipsec; |
| 431 | 433 | struct bonding *bond; |
| 432 | 434 | struct slave *slave; |
| @@ -438,74 +440,80 @@ static int bond_ipsec_add_sa(struct xfrm_state *xs, · 5 unmodified lines | ||
| 438 | 440 | rcu_read_lock(); |
| 439 | 441 | bond = netdev_priv(bond_dev); |
| 440 | 442 | slave = rcu_dereference(bond->curr_active_slave); |
| 441 | − if (!slave) { | |
| 442 | − rcu_read_unlock(); | |
| 443 | − return -ENODEV; | |
| 443 | + real_dev = slave ? slave->dev : NULL; | |
| 444 | + netdev_hold(real_dev, &tracker, GFP_ATOMIC); | |
| 445 | + rcu_read_unlock(); | |
| 446 | + if (!real_dev) { | |
| 447 | + err = -ENODEV; | |
| 448 | + goto out; | |
| 444 | 449 | } |
| 445 | 450 | |
| 446 | − if (!slave->dev->xfrmdev_ops || | |
| 447 | − !slave->dev->xfrmdev_ops->xdo_dev_state_add || | |
| 448 | − netif_is_bond_master(slave->dev)) { | |
| 451 | + if (!real_dev->xfrmdev_ops || | |
| 452 | + !real_dev->xfrmdev_ops->xdo_dev_state_add || | |
| 453 | + netif_is_bond_master(real_dev)) { | |
| 449 | 454 | NL_SET_ERR_MSG_MOD(extack, "Slave does not support ipsec offload"); |
| 450 | − rcu_read_unlock(); | |
| 451 | − return -EINVAL; | |
| 455 | + err = -EINVAL; | |
| 456 | + goto out; | |
| 452 | 457 | } |
| 453 | 458 | |
| 454 | − ipsec = kmalloc(sizeof(*ipsec), GFP_ATOMIC); | |
| 459 | + ipsec = kmalloc(sizeof(*ipsec), GFP_KERNEL); | |
| 455 | 460 | if (!ipsec) { |
| 456 | − rcu_read_unlock(); | |
| 457 | − return -ENOMEM; | |
| 461 | + err = -ENOMEM; | |
| 462 | + goto out; | |
| 458 | 463 | } |
| 459 | − xs->xso.real_dev = slave->dev; | |
| 460 | 464 | |
| 461 | − err = slave->dev->xfrmdev_ops->xdo_dev_state_add(xs, extack); | |
| 465 | + xs->xso.real_dev = real_dev; | |
| 466 | + err = real_dev->xfrmdev_ops->xdo_dev_state_add(xs, extack); | |
| 462 | 467 | if (!err) { |
| 463 | 468 | ipsec->xs = xs; |
| 464 | 469 | INIT_LIST_HEAD(&ipsec->list); |
| 465 | − spin_lock_bh(&bond->ipsec_lock); | |
| 470 | + mutex_lock(&bond->ipsec_lock); | |
| 466 | 471 | list_add(&ipsec->list, &bond->ipsec_list); |
| 467 | − spin_unlock_bh(&bond->ipsec_lock); | |
| 472 | + mutex_unlock(&bond->ipsec_lock); | |
| 468 | 473 | } else { |
| 469 | 474 | kfree(ipsec); |
| 470 | 475 | } |
| 471 | − rcu_read_unlock(); | |
| 476 | +out: | |
| 477 | + netdev_put(real_dev, &tracker); | |
| 472 | 478 | return err; |
| 473 | 479 | } |
| 474 | 480 | |
| 475 | 481 | static void bond_ipsec_add_sa_all(struct bonding *bond) |
| 476 | 482 | { |
| 477 | 483 | struct net_device *bond_dev = bond->dev; |
| 484 | + struct net_device *real_dev; | |
| 478 | 485 | struct bond_ipsec *ipsec; |
| 479 | 486 | struct slave *slave; |
| 480 | 487 | |
| 481 | − rcu_read_lock(); | |
| 482 | − slave = rcu_dereference(bond->curr_active_slave); | |
| 483 | − if (!slave) | |
| 484 | − goto out; | |
| 488 | + slave = rtnl_dereference(bond->curr_active_slave); | |
| 489 | + real_dev = slave ? slave->dev : NULL; | |
| 490 | + if (!real_dev) | |
| 491 | + return; | |
| 485 | 492 | |
| 486 | − if (!slave->dev->xfrmdev_ops || | |
| 487 | − !slave->dev->xfrmdev_ops->xdo_dev_state_add || | |
| 488 | − netif_is_bond_master(slave->dev)) { | |
| 489 | − spin_lock_bh(&bond->ipsec_lock); | |
| 493 | + mutex_lock(&bond->ipsec_lock); | |
| 494 | + if (!real_dev->xfrmdev_ops || | |
| 495 | + !real_dev->xfrmdev_ops->xdo_dev_state_add || | |
| 496 | + netif_is_bond_master(real_dev)) { | |
| 490 | 497 | if (!list_empty(&bond->ipsec_list)) |
| 491 | − slave_warn(bond_dev, slave->dev, | |
| 498 | + slave_warn(bond_dev, real_dev, | |
| 492 | 499 | "%s: no slave xdo_dev_state_add\n", |
| 493 | 500 | __func__); |
| 494 | − spin_unlock_bh(&bond->ipsec_lock); | |
| 495 | 501 | goto out; |
| 496 | 502 | } |
| 497 | 503 | |
| 498 | − spin_lock_bh(&bond->ipsec_lock); | |
| 499 | 504 | list_for_each_entry(ipsec, &bond->ipsec_list, list) { |
| 500 | − ipsec->xs->xso.real_dev = slave->dev; | |
| 501 | − if (slave->dev->xfrmdev_ops->xdo_dev_state_add(ipsec->xs, NULL)) { | |
| 502 | − slave_warn(bond_dev, slave->dev, "%s: failed to add SA\n", __func__); | |
| 505 | + /* If new state is added before ipsec_lock acquired */ | |
| 506 | + if (ipsec->xs->xso.real_dev == real_dev) | |
| 507 | + continue; | |
| 508 | + | |
| 509 | + ipsec->xs->xso.real_dev = real_dev; | |
| 510 | + if (real_dev->xfrmdev_ops->xdo_dev_state_add(ipsec->xs, NULL)) { | |
| 511 | + slave_warn(bond_dev, real_dev, "%s: failed to add SA\n", __func__); | |
| 503 | 512 | ipsec->xs->xso.real_dev = NULL; |
| 504 | 513 | } |
| 505 | 514 | } |
| 506 | − spin_unlock_bh(&bond->ipsec_lock); | |
| 507 | 515 | out: |
| 508 | − rcu_read_unlock(); | |
| 516 | + mutex_unlock(&bond->ipsec_lock); | |
| 509 | 517 | } |
| 510 | 518 | |
| 511 | 519 | /** |
| @@ -515,6 +523,8 @@ out: · 3 unmodified lines | ||
| 515 | 523 | static void bond_ipsec_del_sa(struct xfrm_state *xs) |
| 516 | 524 | { |
| 517 | 525 | struct net_device *bond_dev = xs->xso.dev; |
| 526 | + struct net_device *real_dev; | |
| 527 | + netdevice_tracker tracker; | |
| 518 | 528 | struct bond_ipsec *ipsec; |
| 519 | 529 | struct bonding *bond; |
| 520 | 530 | struct slave *slave; |
| @@ -525,6 +535,9 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) · 4 unmodified lines | ||
| 525 | 535 | rcu_read_lock(); |
| 526 | 536 | bond = netdev_priv(bond_dev); |
| 527 | 537 | slave = rcu_dereference(bond->curr_active_slave); |
| 538 | + real_dev = slave ? slave->dev : NULL; | |
| 539 | + netdev_hold(real_dev, &tracker, GFP_ATOMIC); | |
| 540 | + rcu_read_unlock(); | |
| 528 | 541 | |
| 529 | 542 | if (!slave) |
| 530 | 543 | goto out; |
| @@ -532,18 +545,19 @@ static void bond_ipsec_del_sa(struct xfrm_state *xs) · 1 unmodified line | ||
| 532 | 545 | if (!xs->xso.real_dev) |
| 533 | 546 | goto out; |
| 534 | 547 | |
| 535 | − WARN_ON(xs->xso.real_dev != slave->dev); | |
| 548 | + WARN_ON(xs->xso.real_dev != real_dev); | |
| 536 | 549 | |
| 537 | − if (!slave->dev->xfrmdev_ops || | |
| 538 | − !slave->dev->xfrmdev_ops->xdo_dev_state_delete || | |
| 539 | − netif_is_bond_master(slave->dev)) { | |
| 540 | − slave_warn(bond_dev, slave->dev, "%s: no slave xdo_dev_state_delete\n", __func__); | |
| 550 | + if (!real_dev->xfrmdev_ops || | |
| 551 | + !real_dev->xfrmdev_ops->xdo_dev_state_delete || | |
| 552 | + netif_is_bond_master(real_dev)) { | |
| 553 | + slave_warn(bond_dev, real_dev, "%s: no slave xdo_dev_state_delete\n", __func__); | |
| 541 | 554 | goto out; |
| 542 | 555 | } |
| 543 | 556 | |
| 544 | − slave->dev->xfrmdev_ops->xdo_dev_state_delete(xs); | |
| 557 | + real_dev->xfrmdev_ops->xdo_dev_state_delete(xs); | |
| 545 | 558 | out: |
| 546 | − spin_lock_bh(&bond->ipsec_lock); | |
| 559 | + netdev_put(real_dev, &tracker); | |
| 560 | + mutex_lock(&bond->ipsec_lock); | |
| 547 | 561 | list_for_each_entry(ipsec, &bond->ipsec_list, list) { |
| 548 | 562 | if (ipsec->xs == xs) { |
| 549 | 563 | list_del(&ipsec->list); |
| @@ -551,41 +565,72 @@ out: · 1 unmodified line | ||
| 551 | 565 | break; |
| 552 | 566 | } |
| 553 | 567 | } |
| 554 | − spin_unlock_bh(&bond->ipsec_lock); | |
| 555 | − rcu_read_unlock(); | |
| 568 | + mutex_unlock(&bond->ipsec_lock); | |
| 556 | 569 | } |
| 557 | 570 | |
| 558 | 571 | static void bond_ipsec_del_sa_all(struct bonding *bond) |
| 559 | 572 | { |
| 560 | 573 | struct net_device *bond_dev = bond->dev; |
| 574 | + struct net_device *real_dev; | |
| 561 | 575 | struct bond_ipsec *ipsec; |
| 562 | 576 | struct slave *slave; |
| 563 | 577 | |
| 564 | − rcu_read_lock(); | |
| 565 | − slave = rcu_dereference(bond->curr_active_slave); | |
| 566 | − if (!slave) { | |
| 567 | − rcu_read_unlock(); | |
| 578 | + slave = rtnl_dereference(bond->curr_active_slave); | |
| 579 | + real_dev = slave ? slave->dev : NULL; | |
| 580 | + if (!real_dev) | |
| 568 | 581 | return; |
| 569 | − } | |
| 570 | 582 | |
| 571 | − spin_lock_bh(&bond->ipsec_lock); | |
| 583 | + mutex_lock(&bond->ipsec_lock); | |
| 572 | 584 | list_for_each_entry(ipsec, &bond->ipsec_list, list) { |
| 573 | 585 | if (!ipsec->xs->xso.real_dev) |
| 574 | 586 | continue; |
| 575 | 587 | |
| 576 | − if (!slave->dev->xfrmdev_ops || | |
| 577 | − !slave->dev->xfrmdev_ops->xdo_dev_state_delete || | |
| 578 | − netif_is_bond_master(slave->dev)) { | |
| 579 | − slave_warn(bond_dev, slave->dev, | |
| 588 | + if (!real_dev->xfrmdev_ops || | |
| 589 | + !real_dev->xfrmdev_ops->xdo_dev_state_delete || | |
| 590 | + netif_is_bond_master(real_dev)) { | |
| 591 | + slave_warn(bond_dev, real_dev, | |
| 580 | 592 | "%s: no slave xdo_dev_state_delete\n", |
| 581 | 593 | __func__); |
| 582 | 594 | } else { |
| 583 | − slave->dev->xfrmdev_ops->xdo_dev_state_delete(ipsec->xs); | |
| 595 | + real_dev->xfrmdev_ops->xdo_dev_state_delete(ipsec->xs); | |
| 596 | + if (real_dev->xfrmdev_ops->xdo_dev_state_free) | |
| 597 | + real_dev->xfrmdev_ops->xdo_dev_state_free(ipsec->xs); | |
| 584 | 598 | } |
| 585 | − ipsec->xs->xso.real_dev = NULL; | |
| 586 | 599 | } |
| 587 | − spin_unlock_bh(&bond->ipsec_lock); | |
| 600 | + mutex_unlock(&bond->ipsec_lock); | |
| 601 | +} | |
| 602 | + | |
| 603 | +static void bond_ipsec_free_sa(struct xfrm_state *xs) | |
| 604 | +{ | |
| 605 | + struct net_device *bond_dev = xs->xso.dev; | |
| 606 | + struct net_device *real_dev; | |
| 607 | + netdevice_tracker tracker; | |
| 608 | + struct bonding *bond; | |
| 609 | + struct slave *slave; | |
| 610 | + | |
| 611 | + if (!bond_dev) | |
| 612 | + return; | |
| 613 | + | |
| 614 | + rcu_read_lock(); | |
| 615 | + bond = netdev_priv(bond_dev); | |
| 616 | + slave = rcu_dereference(bond->curr_active_slave); | |
| 617 | + real_dev = slave ? slave->dev : NULL; | |
| 618 | + netdev_hold(real_dev, &tracker, GFP_ATOMIC); | |
| 588 | 619 | rcu_read_unlock(); |
| 620 | + | |
| 621 | + if (!slave) | |
| 622 | + goto out; | |
| 623 | + | |
| 624 | + if (!xs->xso.real_dev) | |
| 625 | + goto out; | |
| 626 | + | |
| 627 | + WARN_ON(xs->xso.real_dev != real_dev); | |
| 628 | + | |
| 629 | + if (real_dev && real_dev->xfrmdev_ops && | |
| 630 | + real_dev->xfrmdev_ops->xdo_dev_state_free) | |
| 631 | + real_dev->xfrmdev_ops->xdo_dev_state_free(xs); | |
| 632 | +out: | |
| 633 | + netdev_put(real_dev, &tracker); | |
| 589 | 634 | } |
| 590 | 635 | |
| 591 | 636 | /** |
| @@ -599,39 +644,36 @@ static bool bond_ipsec_offload_ok(struct sk_buff *skb, struct xfrm_state *xs) · 7 unmodified lines | ||
| 599 | 644 | struct net_device *real_dev; |
| 600 | 645 | struct slave *curr_active; |
| 601 | 646 | struct bonding *bond; |
| 602 | − int err; | |
| 647 | + bool ok = false; | |
| 603 | 648 | |
| 604 | 649 | bond = netdev_priv(bond_dev); |
| 605 | 650 | rcu_read_lock(); |
| 606 | 651 | curr_active = rcu_dereference(bond->curr_active_slave); |
| 652 | + if (!curr_active) | |
| 653 | + goto out; | |
| 607 | 654 | real_dev = curr_active->dev; |
| 608 | 655 | |
| 609 | − if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP) { | |
| 610 | − err = false; | |
| 656 | + if (BOND_MODE(bond) != BOND_MODE_ACTIVEBACKUP) | |
| 611 | 657 | goto out; |
| 612 | − } | |
| 613 | 658 | |
| 614 | − if (!xs->xso.real_dev) { | |
| 615 | − err = false; | |
| 659 | + if (!xs->xso.real_dev) | |
| 616 | 660 | goto out; |
| 617 | − } | |
| 618 | 661 | |
| 619 | 662 | if (!real_dev->xfrmdev_ops || |
| 620 | 663 | !real_dev->xfrmdev_ops->xdo_dev_offload_ok || |
| 621 | − netif_is_bond_master(real_dev)) { | |
| 622 | − err = false; | |
| 664 | + netif_is_bond_master(real_dev)) | |
| 623 | 665 | goto out; |
| 624 | − } | |
| 625 | 666 | |
| 626 | − err = real_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs); | |
| 667 | + ok = real_dev->xfrmdev_ops->xdo_dev_offload_ok(skb, xs); | |
| 627 | 668 | out: |
| 628 | 669 | rcu_read_unlock(); |
| 629 | − return err; | |
| 670 | + return ok; | |
| 630 | 671 | } |
| 631 | 672 | |
| 632 | 673 | static const struct xfrmdev_ops bond_xfrmdev_ops = { |
| 633 | 674 | .xdo_dev_state_add = bond_ipsec_add_sa, |
| 634 | 675 | .xdo_dev_state_delete = bond_ipsec_del_sa, |
| 676 | + .xdo_dev_state_free = bond_ipsec_free_sa, | |
| 635 | 677 | .xdo_dev_offload_ok = bond_ipsec_offload_ok, |
| 636 | 678 | }; |
| 637 | 679 | #endif /* CONFIG_XFRM_OFFLOAD */ |
| @@ -5882,7 +5924,7 @@ void bond_setup(struct net_device *bond_dev) · 5244 unmodified lines | ||
| 5882 | 5924 | /* set up xfrm device ops (only supported in active-backup right now) */ |
| 5883 | 5925 | bond_dev->xfrmdev_ops = &bond_xfrmdev_ops; |
| 5884 | 5926 | INIT_LIST_HEAD(&bond->ipsec_list); |
| 5885 | − spin_lock_init(&bond->ipsec_lock); | |
| 5927 | + mutex_init(&bond->ipsec_lock); | |
| 5886 | 5928 | #endif /* CONFIG_XFRM_OFFLOAD */ |
| 5887 | 5929 | |
| 5888 | 5930 | /* don't acquire bond device's netif_tx_lock when transmitting */ |
| @@ -5931,6 +5973,10 @@ static void bond_uninit(struct net_device *bond_dev) · 42 unmodified lines | ||
| 5931 | 5973 | __bond_release_one(bond_dev, slave->dev, true, true); |
| 5932 | 5974 | netdev_info(bond_dev, "Released all slaves\n"); |
| 5933 | 5975 | |
| 5976 | +#ifdef CONFIG_XFRM_OFFLOAD | |
| 5977 | + mutex_destroy(&bond->ipsec_lock); | |
| 5978 | +#endif /* CONFIG_XFRM_OFFLOAD */ | |
| 5979 | + | |
| 5934 | 5980 | bond_set_slave_arr(bond, NULL, NULL); |
| 5935 | 5981 | |
| 5936 | 5982 | list_del_rcu(&bond->bond_list); |
| @@ -936,7 +936,7 @@ static int bond_option_active_slave_set(struct bonding *bond, · 935 unmodified lines | ||
| 936 | 936 | /* check to see if we are clearing active */ |
| 937 | 937 | if (!slave_dev) { |
| 938 | 938 | netdev_dbg(bond->dev, "Clearing current active slave\n"); |
| 939 | − RCU_INIT_POINTER(bond->curr_active_slave, NULL); | |
| 939 | + bond_change_active_slave(bond, NULL); | |
| 940 | 940 | bond_select_active_slave(bond); |
| 941 | 941 | } else { |
| 942 | 942 | struct slave *old_active = rtnl_dereference(bond->curr_active_slave); |
| @@ -215,7 +215,7 @@ static int es58x_devlink_info_get(struct devlink *devlink, · 214 unmodified lines | ||
| 215 | 215 | struct es58x_sw_version *fw_ver = &es58x_dev->firmware_version; |
| 216 | 216 | struct es58x_sw_version *bl_ver = &es58x_dev->bootloader_version; |
| 217 | 217 | struct es58x_hw_revision *hw_rev = &es58x_dev->hardware_revision; |
| 218 | − char buf[max(sizeof("xx.xx.xx"), sizeof("axxx/xxx"))]; | |
| 218 | + char buf[MAX(sizeof("xx.xx.xx"), sizeof("axxx/xxx"))]; | |
| 219 | 219 | int ret = 0; |
| 220 | 220 | |
| 221 | 221 | if (es58x_sw_version_is_valid(fw_ver)) { |
| @@ -675,8 +675,10 @@ static int bcm_sf2_mdio_register(struct dsa_switch *ds) · 674 unmodified lines | ||
| 675 | 675 | of_remove_property(child, prop); |
| 676 | 676 | |
| 677 | 677 | phydev = of_phy_find_device(child); |
| 678 | − if (phydev) | |
| 678 | + if (phydev) { | |
| 679 | 679 | phy_device_remove(phydev); |
| 680 | + phy_device_free(phydev); | |
| 681 | + } | |
| 680 | 682 | } |
| 681 | 683 | |
| 682 | 684 | err = mdiobus_register(priv->user_mii_bus); |
| @@ -2578,7 +2578,11 @@ static u32 ksz_get_phy_flags(struct dsa_switch *ds, int port) · 2577 unmodified lines | ||
| 2578 | 2578 | if (!port) |
| 2579 | 2579 | return MICREL_KSZ8_P1_ERRATA; |
| 2580 | 2580 | break; |
| 2581 | + case KSZ8567_CHIP_ID: | |
| 2581 | 2582 | case KSZ9477_CHIP_ID: |
| 2583 | + case KSZ9567_CHIP_ID: | |
| 2584 | + case KSZ9896_CHIP_ID: | |
| 2585 | + case KSZ9897_CHIP_ID: | |
| 2582 | 2586 | /* KSZ9477 Errata DS80000754C |
| 2583 | 2587 | * |
| 2584 | 2588 | * Module 4: Energy Efficient Ethernet (EEE) feature select must |
| @@ -2588,6 +2592,13 @@ static u32 ksz_get_phy_flags(struct dsa_switch *ds, int port) · 3 unmodified lines | ||
| 2588 | 2592 | * controls. If not disabled, the PHY ports can auto-negotiate |
| 2589 | 2593 | * to enable EEE, and this feature can cause link drops when |
| 2590 | 2594 | * linked to another device supporting EEE. |
| 2595 | + * | |
| 2596 | + * The same item appears in the errata for the KSZ9567, KSZ9896, | |
| 2597 | + * and KSZ9897. | |
| 2598 | + * | |
| 2599 | + * A similar item appears in the errata for the KSZ8567, but | |
| 2600 | + * provides an alternative workaround. For now, use the simple | |
| 2601 | + * workaround of disabling the EEE feature for this device too. | |
| 2591 | 2602 | */ |
| 2592 | 2603 | return MICREL_NO_EEE; |
| 2593 | 2604 | } |
| @@ -3764,6 +3775,11 @@ static int ksz_port_set_mac_address(struct dsa_switch *ds, int port, · 1170 unmodified lines | ||
| 3764 | 3775 | return -EBUSY; |
| 3765 | 3776 | } |
| 3766 | 3777 | |
| 3778 | + /* Need to initialize variable as the code to fill in settings may | |
| 3779 | + * not be executed. | |
| 3780 | + */ | |
| 3781 | + wol.wolopts = 0; | |
| 3782 | + | |
| 3767 | 3783 | ksz_get_wol(ds, dp->index, &wol); |
| 3768 | 3784 | if (wol.wolopts & WAKE_MAGIC) { |
| 3769 | 3785 | dev_err(ds->dev, |
| @@ -266,7 +266,6 @@ static int ksz_ptp_enable_mode(struct ksz_device *dev) · 265 unmodified lines | ||
| 266 | 266 | struct ksz_port *prt; |
| 267 | 267 | struct dsa_port *dp; |
| 268 | 268 | bool tag_en = false; |
| 269 | − int ret; | |
| 270 | 269 | |
| 271 | 270 | dsa_switch_for_each_user_port(dp, dev->ds) { |
| 272 | 271 | prt = &dev->ports[dp->index]; |
| @@ -277,9 +276,7 @@ static int ksz_ptp_enable_mode(struct ksz_device *dev) · 4 unmodified lines | ||
| 277 | 276 | } |
| 278 | 277 | |
| 279 | 278 | if (tag_en) { |
| 280 | − ret = ptp_schedule_worker(ptp_data->clock, 0); | |
| 281 | − if (ret) | |
| 282 | − return ret; | |
| 279 | + ptp_schedule_worker(ptp_data->clock, 0); | |
| 283 | 280 | } else { |
| 284 | 281 | ptp_cancel_worker_sync(ptp_data->clock); |
| 285 | 282 | } |
| @@ -457,7 +457,8 @@ static irqreturn_t mv88e6xxx_g1_atu_prob_irq_thread_fn(int irq, void *dev_id) · 456 unmodified lines | ||
| 457 | 457 | trace_mv88e6xxx_atu_full_violation(chip->dev, spid, |
| 458 | 458 | entry.portvec, entry.mac, |
| 459 | 459 | fid); |
| 460 | − chip->ports[spid].atu_full_violation++; | |
| 460 | + if (spid < ARRAY_SIZE(chip->ports)) | |
| 461 | + chip->ports[spid].atu_full_violation++; | |
| 461 | 462 | } |
| 462 | 463 | |
| 463 | 464 | return IRQ_HANDLED; |
| @@ -61,11 +61,46 @@ static int felix_cpu_port_for_conduit(struct dsa_switch *ds, · 60 unmodified lines | ||
| 61 | 61 | return cpu_dp->index; |
| 62 | 62 | } |
| 63 | 63 | |
| 64 | +/** | |
| 65 | + * felix_update_tag_8021q_rx_rule - Update VCAP ES0 tag_8021q rule after | |
| 66 | + * vlan_filtering change | |
| 67 | + * @outer_tagging_rule: Pointer to VCAP filter on which the update is performed | |
| 68 | + * @vlan_filtering: Current bridge VLAN filtering setting | |
| 69 | + * | |
| 70 | + * Source port identification for tag_8021q is done using VCAP ES0 rules on the | |
| 71 | + * CPU port(s). The ES0 tag B (inner tag from the packet) can be configured as | |
| 72 | + * either: | |
| 73 | + * - push_inner_tag=0: the inner tag is never pushed into the frame | |
| 74 | + * (and we lose info about the classified VLAN). This is | |
| 75 | + * good when the classified VLAN is a discardable quantity | |
| 76 | + * for the software RX path: it is either set to | |
| 77 | + * OCELOT_STANDALONE_PVID, or to | |
| 78 | + * ocelot_vlan_unaware_pvid(bridge). | |
| 79 | + * - push_inner_tag=1: the inner tag is always pushed. This is good when the | |
| 80 | + * classified VLAN is not a discardable quantity (the port | |
| 81 | + * is under a VLAN-aware bridge, and software needs to | |
| 82 | + * continue processing the packet in the same VLAN as the | |
| 83 | + * hardware). | |
| 84 | + * The point is that what is good for a VLAN-unaware port is not good for a | |
| 85 | + * VLAN-aware port, and vice versa. Thus, the RX tagging rules must be kept in | |
| 86 | + * sync with the VLAN filtering state of the port. | |
| 87 | + */ | |
| 88 | +static void | |
| 89 | +felix_update_tag_8021q_rx_rule(struct ocelot_vcap_filter *outer_tagging_rule, | |
| 90 | + bool vlan_filtering) | |
| 91 | +{ | |
| 92 | + if (vlan_filtering) | |
| 93 | + outer_tagging_rule->action.push_inner_tag = OCELOT_ES0_TAG; | |
| 94 | + else | |
| 95 | + outer_tagging_rule->action.push_inner_tag = OCELOT_NO_ES0_TAG; | |
| 96 | +} | |
| 97 | + | |
| 64 | 98 | /* Set up VCAP ES0 rules for pushing a tag_8021q VLAN towards the CPU such that |
| 65 | 99 | * the tagger can perform RX source port identification. |
| 66 | 100 | */ |
| 67 | 101 | static int felix_tag_8021q_vlan_add_rx(struct dsa_switch *ds, int port, |
| 68 | − int upstream, u16 vid) | |
| 102 | + int upstream, u16 vid, | |
| 103 | + bool vlan_filtering) | |
| 69 | 104 | { |
| 70 | 105 | struct ocelot_vcap_filter *outer_tagging_rule; |
| 71 | 106 | struct ocelot *ocelot = ds->priv; |
| @@ -96,6 +131,14 @@ static int felix_tag_8021q_vlan_add_rx(struct dsa_switch *ds, int port, · 24 unmodified lines | ||
| 96 | 131 | outer_tagging_rule->action.tag_a_tpid_sel = OCELOT_TAG_TPID_SEL_8021AD; |
| 97 | 132 | outer_tagging_rule->action.tag_a_vid_sel = 1; |
| 98 | 133 | outer_tagging_rule->action.vid_a_val = vid; |
| 134 | + felix_update_tag_8021q_rx_rule(outer_tagging_rule, vlan_filtering); | |
| 135 | + outer_tagging_rule->action.tag_b_tpid_sel = OCELOT_TAG_TPID_SEL_8021Q; | |
| 136 | + /* Leave TAG_B_VID_SEL at 0 (Classified VID + VID_B_VAL). Since we also | |
| 137 | + * leave VID_B_VAL at 0, this makes ES0 tag B (the inner tag) equal to | |
| 138 | + * the classified VID, which we need to see in the DSA tagger's receive | |
| 139 | + * path. Note: the inner tag is only visible in the packet when pushed | |
| 140 | + * (push_inner_tag == OCELOT_ES0_TAG). | |
| 141 | + */ | |
| 99 | 142 | |
| 100 | 143 | err = ocelot_vcap_filter_add(ocelot, outer_tagging_rule, NULL); |
| 101 | 144 | if (err) |
| @@ -227,6 +270,7 @@ static int felix_tag_8021q_vlan_del_tx(struct dsa_switch *ds, int port, u16 vid) · 125 unmodified lines | ||
| 227 | 270 | static int felix_tag_8021q_vlan_add(struct dsa_switch *ds, int port, u16 vid, |
| 228 | 271 | u16 flags) |
| 229 | 272 | { |
| 273 | + struct dsa_port *dp = dsa_to_port(ds, port); | |
| 230 | 274 | struct dsa_port *cpu_dp; |
| 231 | 275 | int err; |
| 232 | 276 | |
| @@ -234,11 +278,12 @@ static int felix_tag_8021q_vlan_add(struct dsa_switch *ds, int port, u16 vid, · 1 unmodified line | ||
| 234 | 278 | * membership, which we aren't. So we don't need to add any VCAP filter |
| 235 | 279 | * for the CPU port. |
| 236 | 280 | */ |
| 237 | − if (!dsa_is_user_port(ds, port)) | |
| 281 | + if (!dsa_port_is_user(dp)) | |
| 238 | 282 | return 0; |
| 239 | 283 | |
| 240 | 284 | dsa_switch_for_each_cpu_port(cpu_dp, ds) { |
| 241 | − err = felix_tag_8021q_vlan_add_rx(ds, port, cpu_dp->index, vid); | |
| 285 | + err = felix_tag_8021q_vlan_add_rx(ds, port, cpu_dp->index, vid, | |
| 286 | + dsa_port_is_vlan_filtering(dp)); | |
| 242 | 287 | if (err) |
| 243 | 288 | return err; |
| 244 | 289 | } |
| @@ -258,10 +303,11 @@ add_tx_failed: · 13 unmodified lines | ||
| 258 | 303 | |
| 259 | 304 | static int felix_tag_8021q_vlan_del(struct dsa_switch *ds, int port, u16 vid) |
| 260 | 305 | { |
| 306 | + struct dsa_port *dp = dsa_to_port(ds, port); | |
| 261 | 307 | struct dsa_port *cpu_dp; |
| 262 | 308 | int err; |
| 263 | 309 | |
| 264 | − if (!dsa_is_user_port(ds, port)) | |
| 310 | + if (!dsa_port_is_user(dp)) | |
| 265 | 311 | return 0; |
| 266 | 312 | |
| 267 | 313 | dsa_switch_for_each_cpu_port(cpu_dp, ds) { |
| @@ -278,11 +324,41 @@ static int felix_tag_8021q_vlan_del(struct dsa_switch *ds, int port, u16 vid) · 10 unmodified lines | ||
| 278 | 324 | |
| 279 | 325 | del_tx_failed: |
| 280 | 326 | dsa_switch_for_each_cpu_port(cpu_dp, ds) |
| 281 | − felix_tag_8021q_vlan_add_rx(ds, port, cpu_dp->index, vid); | |
| 327 | + felix_tag_8021q_vlan_add_rx(ds, port, cpu_dp->index, vid, | |
| 328 | + dsa_port_is_vlan_filtering(dp)); | |
| 282 | 329 | |
| 283 | 330 | return err; |
| 284 | 331 | } |
| 285 | 332 | |
| 333 | +static int felix_update_tag_8021q_rx_rules(struct dsa_switch *ds, int port, | |
| 334 | + bool vlan_filtering) | |
| 335 | +{ | |
| 336 | + struct ocelot_vcap_filter *outer_tagging_rule; | |
| 337 | + struct ocelot_vcap_block *block_vcap_es0; | |
| 338 | + struct ocelot *ocelot = ds->priv; | |
| 339 | + struct dsa_port *cpu_dp; | |
| 340 | + unsigned long cookie; | |
| 341 | + int err; | |
| 342 | + | |
| 343 | + block_vcap_es0 = &ocelot->block[VCAP_ES0]; | |
| 344 | + | |
| 345 | + dsa_switch_for_each_cpu_port(cpu_dp, ds) { | |
| 346 | + cookie = OCELOT_VCAP_ES0_TAG_8021Q_RXVLAN(ocelot, port, | |
| 347 | + cpu_dp->index); | |
| 348 | + | |
| 349 | + outer_tagging_rule = ocelot_vcap_block_find_filter_by_id(block_vcap_es0, | |
| 350 | + cookie, false); | |
| 351 | + | |
| 352 | + felix_update_tag_8021q_rx_rule(outer_tagging_rule, vlan_filtering); | |
| 353 | + | |
| 354 | + err = ocelot_vcap_filter_replace(ocelot, outer_tagging_rule); | |
| 355 | + if (err) | |
| 356 | + return err; | |
| 357 | + } | |
| 358 | + | |
| 359 | + return 0; | |
| 360 | +} | |
| 361 | + | |
| 286 | 362 | static int felix_trap_get_cpu_port(struct dsa_switch *ds, |
| 287 | 363 | const struct ocelot_vcap_filter *trap) |
| 288 | 364 | { |
| @@ -528,7 +604,19 @@ static int felix_tag_8021q_setup(struct dsa_switch *ds) · 239 unmodified lines | ||
| 528 | 604 | * so we need to be careful that there are no extra frames to be |
| 529 | 605 | * dequeued over MMIO, since we would never know to discard them. |
| 530 | 606 | */ |
| 607 | + ocelot_lock_xtr_grp_bh(ocelot, 0); | |
| 531 | 608 | ocelot_drain_cpu_queue(ocelot, 0); |
| 609 | + ocelot_unlock_xtr_grp_bh(ocelot, 0); | |
| 610 | + | |
| 611 | + /* Problem: when using push_inner_tag=1 for ES0 tag B, we lose info | |
| 612 | + * about whether the received packets were VLAN-tagged on the wire, | |
| 613 | + * since they are always tagged on egress towards the CPU port. | |
| 614 | + * | |
| 615 | + * Since using push_inner_tag=1 is unavoidable for VLAN-aware bridges, | |
| 616 | + * we must work around the fallout by untagging in software to make | |
| 617 | + * untagged reception work more or less as expected. | |
| 618 | + */ | |
| 619 | + ds->untag_vlan_aware_bridge_pvid = true; | |
| 532 | 620 | |
| 533 | 621 | return 0; |
| 534 | 622 | } |
| @@ -554,6 +642,8 @@ static void felix_tag_8021q_teardown(struct dsa_switch *ds) · 19 unmodified lines | ||
| 554 | 642 | ocelot_port_teardown_dsa_8021q_cpu(ocelot, dp->index); |
| 555 | 643 | |
| 556 | 644 | dsa_tag_8021q_unregister(ds); |
| 645 | + | |
| 646 | + ds->untag_vlan_aware_bridge_pvid = false; | |
| 557 | 647 | } |
| 558 | 648 | |
| 559 | 649 | static unsigned long felix_tag_8021q_get_host_fwd_mask(struct dsa_switch *ds) |
| @@ -1008,8 +1098,23 @@ static int felix_vlan_filtering(struct dsa_switch *ds, int port, bool enabled, · 448 unmodified lines | ||
| 1008 | 1098 | struct netlink_ext_ack *extack) |
| 1009 | 1099 | { |
| 1010 | 1100 | struct ocelot *ocelot = ds->priv; |
| 1101 | + bool using_tag_8021q; | |
| 1102 | + struct felix *felix; | |
| 1103 | + int err; | |
| 1011 | 1104 | |
| 1012 | − return ocelot_port_vlan_filtering(ocelot, port, enabled, extack); | |
| 1105 | + err = ocelot_port_vlan_filtering(ocelot, port, enabled, extack); | |
| 1106 | + if (err) | |
| 1107 | + return err; | |
| 1108 | + | |
| 1109 | + felix = ocelot_to_felix(ocelot); | |
| 1110 | + using_tag_8021q = felix->tag_proto == DSA_TAG_PROTO_OCELOT_8021Q; | |
| 1111 | + if (using_tag_8021q) { | |
| 1112 | + err = felix_update_tag_8021q_rx_rules(ds, port, enabled); | |
| 1113 | + if (err) | |
| 1114 | + return err; | |
| 1115 | + } | |
| 1116 | + | |
| 1117 | + return 0; | |
| 1013 | 1118 | } |
| 1014 | 1119 | |
| 1015 | 1120 | static int felix_vlan_add(struct dsa_switch *ds, int port, |
| @@ -1518,6 +1623,8 @@ static void felix_port_deferred_xmit(struct kthread_work *work) · 502 unmodified lines | ||
| 1518 | 1623 | int port = xmit_work->dp->index; |
| 1519 | 1624 | int retries = 10; |
| 1520 | 1625 | |
| 1626 | + ocelot_lock_inj_grp(ocelot, 0); | |
| 1627 | + | |
| 1521 | 1628 | do { |
| 1522 | 1629 | if (ocelot_can_inject(ocelot, 0)) |
| 1523 | 1630 | break; |
| @@ -1526,6 +1633,7 @@ static void felix_port_deferred_xmit(struct kthread_work *work) · 2 unmodified lines | ||
| 1526 | 1633 | } while (--retries); |
| 1527 | 1634 | |
| 1528 | 1635 | if (!retries) { |
| 1636 | + ocelot_unlock_inj_grp(ocelot, 0); | |
| 1529 | 1637 | dev_err(ocelot->dev, "port %d failed to inject skb\n", |
| 1530 | 1638 | port); |
| 1531 | 1639 | ocelot_port_purge_txtstamp_skb(ocelot, port, skb); |
| @@ -1535,6 +1643,8 @@ static void felix_port_deferred_xmit(struct kthread_work *work) · 3 unmodified lines | ||
| 1535 | 1643 | |
| 1536 | 1644 | ocelot_port_inject_frame(ocelot, port, 0, rew_op, skb); |
| 1537 | 1645 | |
| 1646 | + ocelot_unlock_inj_grp(ocelot, 0); | |
| 1647 | + | |
| 1538 | 1648 | consume_skb(skb); |
| 1539 | 1649 | kfree(xmit_work); |
| 1540 | 1650 | } |
| @@ -1694,6 +1804,8 @@ static bool felix_check_xtr_pkt(struct ocelot *ocelot) · 153 unmodified lines | ||
| 1694 | 1804 | if (!felix->info->quirk_no_xtr_irq) |
| 1695 | 1805 | return false; |
| 1696 | 1806 | |
| 1807 | + ocelot_lock_xtr_grp(ocelot, grp); | |
| 1808 | + | |
| 1697 | 1809 | while (ocelot_read(ocelot, QS_XTR_DATA_PRESENT) & BIT(grp)) { |
| 1698 | 1810 | struct sk_buff *skb; |
| 1699 | 1811 | unsigned int type; |
| @@ -1730,6 +1842,8 @@ out: · 30 unmodified lines | ||
| 1730 | 1842 | ocelot_drain_cpu_queue(ocelot, 0); |
| 1731 | 1843 | } |
| 1732 | 1844 | |
| 1845 | + ocelot_unlock_xtr_grp(ocelot, grp); | |
| 1846 | + | |
| 1733 | 1847 | return true; |
| 1734 | 1848 | } |
| 1735 | 1849 | |
| @@ -40,6 +40,10 @@ · 39 unmodified lines | ||
| 40 | 40 | #define VSC73XX_BLOCK_ARBITER 0x5 /* Only subblock 0 */ |
| 41 | 41 | #define VSC73XX_BLOCK_SYSTEM 0x7 /* Only subblock 0 */ |
| 42 | 42 | |
| 43 | +/* MII Block subblock */ | |
| 44 | +#define VSC73XX_BLOCK_MII_INTERNAL 0x0 /* Internal MDIO subblock */ | |
| 45 | +#define VSC73XX_BLOCK_MII_EXTERNAL 0x1 /* External MDIO subblock */ | |
| 46 | + | |
| 43 | 47 | #define CPU_PORT 6 /* CPU port */ |
| 44 | 48 | |
| 45 | 49 | /* MAC Block registers */ |
| @@ -225,6 +229,8 @@ · 179 unmodified lines | ||
| 225 | 229 | #define VSC73XX_MII_CMD 0x1 |
| 226 | 230 | #define VSC73XX_MII_DATA 0x2 |
| 227 | 231 | |
| 232 | +#define VSC73XX_MII_STAT_BUSY BIT(3) | |
| 233 | + | |
| 228 | 234 | /* Arbiter block 5 registers */ |
| 229 | 235 | #define VSC73XX_ARBEMPTY 0x0c |
| 230 | 236 | #define VSC73XX_ARBDISC 0x0e |
| @@ -299,6 +305,7 @@ · 68 unmodified lines | ||
| 299 | 305 | #define IS_739X(a) (IS_7395(a) || IS_7398(a)) |
| 300 | 306 | |
| 301 | 307 | #define VSC73XX_POLL_SLEEP_US 1000 |
| 308 | +#define VSC73XX_MDIO_POLL_SLEEP_US 5 | |
| 302 | 309 | #define VSC73XX_POLL_TIMEOUT_US 10000 |
| 303 | 310 | |
| 304 | 311 | struct vsc73xx_counter { |
| @@ -527,6 +534,22 @@ static int vsc73xx_detect(struct vsc73xx *vsc) · 222 unmodified lines | ||
| 527 | 534 | return 0; |
| 528 | 535 | } |
| 529 | 536 | |
| 537 | +static int vsc73xx_mdio_busy_check(struct vsc73xx *vsc) | |
| 538 | +{ | |
| 539 | + int ret, err; | |
| 540 | + u32 val; | |
| 541 | + | |
| 542 | + ret = read_poll_timeout(vsc73xx_read, err, | |
| 543 | + err < 0 || !(val & VSC73XX_MII_STAT_BUSY), | |
| 544 | + VSC73XX_MDIO_POLL_SLEEP_US, | |
| 545 | + VSC73XX_POLL_TIMEOUT_US, false, vsc, | |
| 546 | + VSC73XX_BLOCK_MII, VSC73XX_BLOCK_MII_INTERNAL, | |
| 547 | + VSC73XX_MII_STAT, &val); | |
| 548 | + if (ret) | |
| 549 | + return ret; | |
| 550 | + return err; | |
| 551 | +} | |
| 552 | + | |
| 530 | 553 | static int vsc73xx_phy_read(struct dsa_switch *ds, int phy, int regnum) |
| 531 | 554 | { |
| 532 | 555 | struct vsc73xx *vsc = ds->priv; |
| @@ -534,12 +557,20 @@ static int vsc73xx_phy_read(struct dsa_switch *ds, int phy, int regnum) · 1 unmodified line | ||
| 534 | 557 | u32 val; |
| 535 | 558 | int ret; |
| 536 | 559 | |
| 560 | + ret = vsc73xx_mdio_busy_check(vsc); | |
| 561 | + if (ret) | |
| 562 | + return ret; | |
| 563 | + | |
| 537 | 564 | /* Setting bit 26 means "read" */ |
| 538 | 565 | cmd = BIT(26) | (phy << 21) | (regnum << 16); |
| 539 | 566 | ret = vsc73xx_write(vsc, VSC73XX_BLOCK_MII, 0, 1, cmd); |
| 540 | 567 | if (ret) |
| 541 | 568 | return ret; |
| 542 | − msleep(2); | |
| 569 | + | |
| 570 | + ret = vsc73xx_mdio_busy_check(vsc); | |
| 571 | + if (ret) | |
| 572 | + return ret; | |
| 573 | + | |
| 543 | 574 | ret = vsc73xx_read(vsc, VSC73XX_BLOCK_MII, 0, 2, &val); |
| 544 | 575 | if (ret) |
| 545 | 576 | return ret; |
| @@ -563,18 +594,11 @@ static int vsc73xx_phy_write(struct dsa_switch *ds, int phy, int regnum, · 17 unmodified lines | ||
| 563 | 594 | u32 cmd; |
| 564 | 595 | int ret; |
| 565 | 596 | |
| 566 | − /* It was found through tedious experiments that this router | |
| 567 | − * chip really hates to have it's PHYs reset. They | |
| 568 | − * never recover if that happens: autonegotiation stops | |
| 569 | − * working after a reset. Just filter out this command. | |
| 570 | − * (Resetting the whole chip is OK.) | |
| 571 | − */ | |
| 572 | − if (regnum == 0 && (val & BIT(15))) { | |
| 573 | − dev_info(vsc->dev, "reset PHY - disallowed\n"); | |
| 574 | − return 0; | |
| 575 | − } | |
| 597 | + ret = vsc73xx_mdio_busy_check(vsc); | |
| 598 | + if (ret) | |
| 599 | + return ret; | |
| 576 | 600 | |
| 577 | − cmd = (phy << 21) | (regnum << 16); | |
| 601 | + cmd = (phy << 21) | (regnum << 16) | val; | |
| 578 | 602 | ret = vsc73xx_write(vsc, VSC73XX_BLOCK_MII, 0, 1, cmd); |
| 579 | 603 | if (ret) |
| 580 | 604 | return ret; |
| @@ -957,6 +981,11 @@ static void vsc73xx_mac_link_up(struct phylink_config *config, · 376 unmodified lines | ||
| 957 | 981 | |
| 958 | 982 | if (duplex == DUPLEX_FULL) |
| 959 | 983 | val |= VSC73XX_MAC_CFG_FDX; |
| 984 | + else | |
| 985 | + /* In datasheet description ("Port Mode Procedure" in 5.6.2) | |
| 986 | + * this bit is configured only for half duplex. | |
| 987 | + */ | |
| 988 | + val |= VSC73XX_MAC_CFG_WEXC_DIS; | |
| 960 | 989 | |
| 961 | 990 | /* This routine is described in the datasheet (below ARBDISC register |
| 962 | 991 | * description) |
| @@ -967,7 +996,6 @@ static void vsc73xx_mac_link_up(struct phylink_config *config, · 4 unmodified lines | ||
| 967 | 996 | get_random_bytes(&seed, 1); |
| 968 | 997 | val |= seed << VSC73XX_MAC_CFG_SEED_OFFSET; |
| 969 | 998 | val |= VSC73XX_MAC_CFG_SEED_LOAD; |
| 970 | − val |= VSC73XX_MAC_CFG_WEXC_DIS; | |
| 971 | 999 | |
| 972 | 1000 | /* Those bits are responsible for MTU only. Kernel takes care about MTU, |
| 973 | 1001 | * let's enable +8 bytes frame length unconditionally. |
| @@ -5056,7 +5056,7 @@ void bnxt_del_one_usr_fltr(struct bnxt *bp, struct bnxt_filter_base *fltr) · 5055 unmodified lines | ||
| 5056 | 5056 | list_del_init(&fltr->list); |
| 5057 | 5057 | } |
| 5058 | 5058 | |
| 5059 | −void bnxt_clear_usr_fltrs(struct bnxt *bp, bool all) | |
| 5059 | +static void bnxt_clear_usr_fltrs(struct bnxt *bp, bool all) | |
| 5060 | 5060 | { |
| 5061 | 5061 | struct bnxt_filter_base *usr_fltr, *tmp; |
| 5062 | 5062 | |
| @@ -7591,19 +7591,20 @@ static bool bnxt_need_reserve_rings(struct bnxt *bp) · 2528 unmodified lines | ||
| 7591 | 7591 | int rx = bp->rx_nr_rings, stat; |
| 7592 | 7592 | int vnic, grp = rx; |
| 7593 | 7593 | |
| 7594 | − if (hw_resc->resv_tx_rings != bp->tx_nr_rings && | |
| 7595 | − bp->hwrm_spec_code >= 0x10601) | |
| 7596 | − return true; | |
| 7597 | − | |
| 7598 | 7594 | /* Old firmware does not need RX ring reservations but we still |
| 7599 | 7595 | * need to setup a default RSS map when needed. With new firmware |
| 7600 | 7596 | * we go through RX ring reservations first and then set up the |
| 7601 | 7597 | * RSS map for the successfully reserved RX rings when needed. |
| 7602 | 7598 | */ |
| 7603 | − if (!BNXT_NEW_RM(bp)) { | |
| 7599 | + if (!BNXT_NEW_RM(bp)) | |
| 7604 | 7600 | bnxt_check_rss_tbl_no_rmgr(bp); |
| 7601 | + | |
| 7602 | + if (hw_resc->resv_tx_rings != bp->tx_nr_rings && | |
| 7603 | + bp->hwrm_spec_code >= 0x10601) | |
| 7604 | + return true; | |
| 7605 | + | |
| 7606 | + if (!BNXT_NEW_RM(bp)) | |
| 7605 | 7607 | return false; |
| 7606 | − } | |
| 7607 | 7608 | |
| 7608 | 7609 | vnic = bnxt_get_total_vnics(bp, rx); |
| 7609 | 7610 | |
| @@ -7649,8 +7650,8 @@ static int bnxt_get_avail_msix(struct bnxt *bp, int num); · 39 unmodified lines | ||
| 7649 | 7650 | static int __bnxt_reserve_rings(struct bnxt *bp) |
| 7650 | 7651 | { |
| 7651 | 7652 | struct bnxt_hw_rings hwr = {0}; |
| 7653 | + int rx_rings, old_rx_rings, rc; | |
| 7652 | 7654 | int cp = bp->cp_nr_rings; |
| 7653 | − int rx_rings, rc; | |
| 7654 | 7655 | int ulp_msix = 0; |
| 7655 | 7656 | bool sh = false; |
| 7656 | 7657 | int tx_cp; |
| @@ -7684,6 +7685,7 @@ static int __bnxt_reserve_rings(struct bnxt *bp) · 27 unmodified lines | ||
| 7684 | 7685 | hwr.grp = bp->rx_nr_rings; |
| 7685 | 7686 | hwr.rss_ctx = bnxt_get_total_rss_ctxs(bp, &hwr); |
| 7686 | 7687 | hwr.stat = bnxt_get_func_stat_ctxs(bp); |
| 7688 | + old_rx_rings = bp->hw_resc.resv_rx_rings; | |
| 7687 | 7689 | |
| 7688 | 7690 | rc = bnxt_hwrm_reserve_rings(bp, &hwr); |
| 7689 | 7691 | if (rc) |
| @@ -7738,7 +7740,8 @@ static int __bnxt_reserve_rings(struct bnxt *bp) · 48 unmodified lines | ||
| 7738 | 7740 | if (!bnxt_rings_ok(bp, &hwr)) |
| 7739 | 7741 | return -ENOMEM; |
| 7740 | 7742 | |
| 7741 | − if (!netif_is_rxfh_configured(bp->dev)) | |
| 7743 | + if (old_rx_rings != bp->hw_resc.resv_rx_rings && | |
| 7744 | + !netif_is_rxfh_configured(bp->dev)) | |
| 7742 | 7745 | bnxt_set_dflt_rss_indir_tbl(bp, NULL); |
| 7743 | 7746 | |
| 7744 | 7747 | if (!bnxt_ulp_registered(bp->edev) && BNXT_NEW_RM(bp)) { |
| @@ -10245,7 +10248,7 @@ static void bnxt_hwrm_realloc_rss_ctx_vnic(struct bnxt *bp) · 2500 unmodified lines | ||
| 10245 | 10248 | } |
| 10246 | 10249 | } |
| 10247 | 10250 | |
| 10248 | −void bnxt_clear_rss_ctxs(struct bnxt *bp) | |
| 10251 | +static void bnxt_clear_rss_ctxs(struct bnxt *bp) | |
| 10249 | 10252 | { |
| 10250 | 10253 | struct ethtool_rxfh_context *ctx; |
| 10251 | 10254 | unsigned long context; |
| @@ -2790,7 +2790,6 @@ void bnxt_set_ring_params(struct bnxt *); · 2789 unmodified lines | ||
| 2790 | 2790 | int bnxt_set_rx_skb_mode(struct bnxt *bp, bool page_mode); |
| 2791 | 2791 | void bnxt_insert_usr_fltr(struct bnxt *bp, struct bnxt_filter_base *fltr); |
| 2792 | 2792 | void bnxt_del_one_usr_fltr(struct bnxt *bp, struct bnxt_filter_base *fltr); |
| 2793 | −void bnxt_clear_usr_fltrs(struct bnxt *bp, bool all); | |
| 2794 | 2793 | int bnxt_hwrm_func_drv_rgtr(struct bnxt *bp, unsigned long *bmap, |
| 2795 | 2794 | int bmap_size, bool async_only); |
| 2796 | 2795 | int bnxt_hwrm_func_drv_unrgtr(struct bnxt *bp); |
| @@ -2842,7 +2841,6 @@ int bnxt_hwrm_vnic_rss_cfg_p5(struct bnxt *bp, struct bnxt_vnic_info *vnic); · 45 unmodified lines | ||
| 2842 | 2841 | int __bnxt_setup_vnic_p5(struct bnxt *bp, struct bnxt_vnic_info *vnic); |
| 2843 | 2842 | void bnxt_del_one_rss_ctx(struct bnxt *bp, struct bnxt_rss_ctx *rss_ctx, |
| 2844 | 2843 | bool all); |
| 2845 | −void bnxt_clear_rss_ctxs(struct bnxt *bp); | |
| 2846 | 2844 | int bnxt_open_nic(struct bnxt *, bool, bool); |
| 2847 | 2845 | int bnxt_half_open_nic(struct bnxt *bp); |
| 2848 | 2846 | void bnxt_half_close_nic(struct bnxt *bp); |
| @@ -968,9 +968,6 @@ static int bnxt_set_channels(struct net_device *dev, · 967 unmodified lines | ||
| 968 | 968 | return -EINVAL; |
| 969 | 969 | } |
| 970 | 970 | |
| 971 | − bnxt_clear_usr_fltrs(bp, true); | |
| 972 | − if (BNXT_SUPPORTS_MULTI_RSS_CTX(bp)) | |
| 973 | − bnxt_clear_rss_ctxs(bp); | |
| 974 | 971 | if (netif_running(dev)) { |
| 975 | 972 | if (BNXT_PF(bp)) { |
| 976 | 973 | /* TODO CHIMP_FW: Send message to all VF's |
| @@ -1863,8 +1860,14 @@ static void bnxt_modify_rss(struct bnxt *bp, struct ethtool_rxfh_context *ctx, · 886 unmodified lines | ||
| 1863 | 1860 | } |
| 1864 | 1861 | |
| 1865 | 1862 | static int bnxt_rxfh_context_check(struct bnxt *bp, |
| 1863 | + const struct ethtool_rxfh_param *rxfh, | |
| 1866 | 1864 | struct netlink_ext_ack *extack) |
| 1867 | 1865 | { |
| 1866 | + if (rxfh->hfunc && rxfh->hfunc != ETH_RSS_HASH_TOP) { | |
| 1867 | + NL_SET_ERR_MSG_MOD(extack, "RSS hash function not supported"); | |
| 1868 | + return -EOPNOTSUPP; | |
| 1869 | + } | |
| 1870 | + | |
| 1868 | 1871 | if (!BNXT_SUPPORTS_MULTI_RSS_CTX(bp)) { |
| 1869 | 1872 | NL_SET_ERR_MSG_MOD(extack, "RSS contexts not supported"); |
| 1870 | 1873 | return -EOPNOTSUPP; |
| @@ -1888,7 +1891,7 @@ static int bnxt_create_rxfh_context(struct net_device *dev, · 17 unmodified lines | ||
| 1888 | 1891 | struct bnxt_vnic_info *vnic; |
| 1889 | 1892 | int rc; |
| 1890 | 1893 | |
| 1891 | − rc = bnxt_rxfh_context_check(bp, extack); | |
| 1894 | + rc = bnxt_rxfh_context_check(bp, rxfh, extack); | |
| 1892 | 1895 | if (rc) |
| 1893 | 1896 | return rc; |
| 1894 | 1897 | |
| @@ -1915,8 +1918,12 @@ static int bnxt_create_rxfh_context(struct net_device *dev, · 20 unmodified lines | ||
| 1915 | 1918 | if (rc) |
| 1916 | 1919 | goto out; |
| 1917 | 1920 | |
| 1921 | + /* Populate defaults in the context */ | |
| 1918 | 1922 | bnxt_set_dflt_rss_indir_tbl(bp, ctx); |
| 1923 | + ctx->hfunc = ETH_RSS_HASH_TOP; | |
| 1919 | 1924 | memcpy(vnic->rss_hash_key, bp->rss_hash_key, HW_HASH_KEY_SIZE); |
| 1925 | + memcpy(ethtool_rxfh_context_key(ctx), | |
| 1926 | + bp->rss_hash_key, HW_HASH_KEY_SIZE); | |
| 1920 | 1927 | |
| 1921 | 1928 | rc = bnxt_hwrm_vnic_alloc(bp, vnic, 0, bp->rx_nr_rings); |
| 1922 | 1929 | if (rc) { |
| @@ -1953,7 +1960,7 @@ static int bnxt_modify_rxfh_context(struct net_device *dev, · 30 unmodified lines | ||
| 1953 | 1960 | struct bnxt_rss_ctx *rss_ctx; |
| 1954 | 1961 | int rc; |
| 1955 | 1962 | |
| 1956 | − rc = bnxt_rxfh_context_check(bp, extack); | |
| 1963 | + rc = bnxt_rxfh_context_check(bp, rxfh, extack); | |
| 1957 | 1964 | if (rc) |
| 1958 | 1965 | return rc; |
| 1959 | 1966 | |
| @@ -1990,7 +1997,6 @@ static int bnxt_set_rxfh(struct net_device *dev, · 30 unmodified lines | ||
| 1990 | 1997 | |
| 1991 | 1998 | bnxt_modify_rss(bp, NULL, NULL, rxfh); |
| 1992 | 1999 | |
| 1993 | − bnxt_clear_usr_fltrs(bp, false); | |
| 1994 | 2000 | if (netif_running(bp->dev)) { |
| 1995 | 2001 | bnxt_close_nic(bp, false, false); |
| 1996 | 2002 | rc = bnxt_open_nic(bp, false, false); |
| @@ -5280,7 +5286,7 @@ void bnxt_ethtool_free(struct bnxt *bp) · 3283 unmodified lines | ||
| 5280 | 5286 | const struct ethtool_ops bnxt_ethtool_ops = { |
| 5281 | 5287 | .cap_link_lanes_supported = 1, |
| 5282 | 5288 | .cap_rss_ctx_supported = 1, |
| 5283 | − .rxfh_max_context_id = BNXT_MAX_ETH_RSS_CTX, | |
| 5289 | + .rxfh_max_num_contexts = BNXT_MAX_ETH_RSS_CTX + 1, | |
| 5284 | 5290 | .rxfh_indir_space = BNXT_MAX_RSS_TABLE_ENTRIES_P5, |
| 5285 | 5291 | .rxfh_priv_size = sizeof(struct bnxt_rss_ctx), |
| 5286 | 5292 | .supported_coalesce_params = ETHTOOL_COALESCE_USECS | |
| @@ -297,11 +297,6 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons, · 296 unmodified lines | ||
| 297 | 297 | * redirect is coming from a frame received by the |
| 298 | 298 | * bnxt_en driver. |
| 299 | 299 | */ |
| 300 | − rx_buf = &rxr->rx_buf_ring[cons]; | |
| 301 | − mapping = rx_buf->mapping - bp->rx_dma_offset; | |
| 302 | − dma_unmap_page_attrs(&pdev->dev, mapping, | |
| 303 | − BNXT_RX_PAGE_SIZE, bp->rx_dir, | |
| 304 | − DMA_ATTR_WEAK_ORDERING); | |
| 305 | 300 | |
| 306 | 301 | /* if we are unable to allocate a new buffer, abort and reuse */ |
| 307 | 302 | if (bnxt_alloc_rx_data(bp, rxr, rxr->rx_prod, GFP_ATOMIC)) { |
| @@ -42,19 +42,15 @@ void bcmgenet_get_wol(struct net_device *dev, struct ethtool_wolinfo *wol) · 41 unmodified lines | ||
| 42 | 42 | struct bcmgenet_priv *priv = netdev_priv(dev); |
| 43 | 43 | struct device *kdev = &priv->pdev->dev; |
| 44 | 44 | |
| 45 | − if (dev->phydev) { | |
| 45 | + if (dev->phydev) | |
| 46 | 46 | phy_ethtool_get_wol(dev->phydev, wol); |
| 47 | − if (wol->supported) | |
| 48 | − return; | |
| 49 | − } | |
| 50 | 47 | |
| 51 | − if (!device_can_wakeup(kdev)) { | |
| 52 | − wol->supported = 0; | |
| 53 | − wol->wolopts = 0; | |
| 48 | + /* MAC is not wake-up capable, return what the PHY does */ | |
| 49 | + if (!device_can_wakeup(kdev)) | |
| 54 | 50 | return; |
| 55 | − } | |
| 56 | 51 | |
| 57 | − wol->supported = WAKE_MAGIC | WAKE_MAGICSECURE | WAKE_FILTER; | |
| 52 | + /* Overlay MAC capabilities with that of the PHY queried before */ | |
| 53 | + wol->supported |= WAKE_MAGIC | WAKE_MAGICSECURE | WAKE_FILTER; | |
| 58 | 54 | wol->wolopts = priv->wolopts; |
| 59 | 55 | memset(wol->sopass, 0, sizeof(wol->sopass)); |
| 60 | 56 | |
| @@ -5250,8 +5250,8 @@ static int __maybe_unused macb_suspend(struct device *dev) · 5249 unmodified lines | ||
| 5250 | 5250 | if (bp->wol & MACB_WOL_ENABLED) { |
| 5251 | 5251 | /* Check for IP address in WOL ARP mode */ |
| 5252 | 5252 | idev = __in_dev_get_rcu(bp->dev); |
| 5253 | − if (idev && idev->ifa_list) | |
| 5254 | − ifa = rcu_access_pointer(idev->ifa_list); | |
| 5253 | + if (idev) | |
| 5254 | + ifa = rcu_dereference(idev->ifa_list); | |
| 5255 | 5255 | if ((bp->wolopts & WAKE_ARP) && !ifa) { |
| 5256 | 5256 | netdev_err(netdev, "IP address not assigned as required by WoL walk ARP\n"); |
| 5257 | 5257 | return -EOPNOTSUPP; |
| @@ -1054,18 +1054,12 @@ static int phy_interface_mode(u8 lmac_type) · 1053 unmodified lines | ||
| 1054 | 1054 | |
| 1055 | 1055 | static int bgx_lmac_enable(struct bgx *bgx, u8 lmacid) |
| 1056 | 1056 | { |
| 1057 | − struct lmac *lmac, **priv; | |
| 1057 | + struct lmac *lmac; | |
| 1058 | 1058 | u64 cfg; |
| 1059 | 1059 | |
| 1060 | 1060 | lmac = &bgx->lmac[lmacid]; |
| 1061 | 1061 | lmac->bgx = bgx; |
| 1062 | 1062 | |
| 1063 | − lmac->netdev = alloc_netdev_dummy(sizeof(struct lmac *)); | |
| 1064 | − if (!lmac->netdev) | |
| 1065 | − return -ENOMEM; | |
| 1066 | − priv = netdev_priv(lmac->netdev); | |
| 1067 | − *priv = lmac; | |
| 1068 | − | |
| 1069 | 1063 | if ((lmac->lmac_type == BGX_MODE_SGMII) || |
| 1070 | 1064 | (lmac->lmac_type == BGX_MODE_QSGMII) || |
| 1071 | 1065 | (lmac->lmac_type == BGX_MODE_RGMII)) { |
| @@ -1191,7 +1185,6 @@ static void bgx_lmac_disable(struct bgx *bgx, u8 lmacid) · 119 unmodified lines | ||
| 1191 | 1185 | (lmac->lmac_type != BGX_MODE_10G_KR) && lmac->phydev) |
| 1192 | 1186 | phy_disconnect(lmac->phydev); |
| 1193 | 1187 | |
| 1194 | − free_netdev(lmac->netdev); | |
| 1195 | 1188 | lmac->phydev = NULL; |
| 1196 | 1189 | } |
| 1197 | 1190 | |
| @@ -1653,6 +1646,23 @@ static int bgx_probe(struct pci_dev *pdev, const struct pci_device_id *ent) · 455 unmodified lines | ||
| 1653 | 1646 | |
| 1654 | 1647 | bgx_get_qlm_mode(bgx); |
| 1655 | 1648 | |
| 1649 | + for (lmac = 0; lmac < bgx->lmac_count; lmac++) { | |
| 1650 | + struct lmac *lmacp, **priv; | |
| 1651 | + | |
| 1652 | + lmacp = &bgx->lmac[lmac]; | |
| 1653 | + lmacp->netdev = alloc_netdev_dummy(sizeof(struct lmac *)); | |
| 1654 | + | |
| 1655 | + if (!lmacp->netdev) { | |
| 1656 | + for (int i = 0; i < lmac; i++) | |
| 1657 | + free_netdev(bgx->lmac[i].netdev); | |
| 1658 | + err = -ENOMEM; | |
| 1659 | + goto err_enable; | |
| 1660 | + } | |
| 1661 | + | |
| 1662 | + priv = netdev_priv(lmacp->netdev); | |
| 1663 | + *priv = lmacp; | |
| 1664 | + } | |
| 1665 | + | |
| 1656 | 1666 | err = bgx_init_phy(bgx); |
| 1657 | 1667 | if (err) |
| 1658 | 1668 | goto err_enable; |
| @@ -1692,8 +1702,10 @@ static void bgx_remove(struct pci_dev *pdev) · 33 unmodified lines | ||
| 1692 | 1702 | u8 lmac; |
| 1693 | 1703 | |
| 1694 | 1704 | /* Disable all LMACs */ |
| 1695 | − for (lmac = 0; lmac < bgx->lmac_count; lmac++) | |
| 1705 | + for (lmac = 0; lmac < bgx->lmac_count; lmac++) { | |
| 1696 | 1706 | bgx_lmac_disable(bgx, lmac); |
| 1707 | + free_netdev(bgx->lmac[lmac].netdev); | |
| 1708 | + } | |
| 1697 | 1709 | |
| 1698 | 1710 | pci_free_irq(pdev, GMPX_GMI_TX_INT, bgx); |
| 1699 | 1711 | |
| @@ -1244,7 +1244,8 @@ static u64 hash_filter_ntuple(struct ch_filter_specification *fs, · 1243 unmodified lines | ||
| 1244 | 1244 | * in the Compressed Filter Tuple. |
| 1245 | 1245 | */ |
| 1246 | 1246 | if (tp->vlan_shift >= 0 && fs->mask.ivlan) |
| 1247 | − ntuple |= (FT_VLAN_VLD_F | fs->val.ivlan) << tp->vlan_shift; | |
| 1247 | + ntuple |= (u64)(FT_VLAN_VLD_F | | |
| 1248 | + fs->val.ivlan) << tp->vlan_shift; | |
| 1248 | 1249 | |
| 1249 | 1250 | if (tp->port_shift >= 0 && fs->mask.iport) |
| 1250 | 1251 | ntuple |= (u64)fs->val.iport << tp->port_shift; |
| @@ -572,7 +572,7 @@ static bool ftgmac100_rx_packet(struct ftgmac100 *priv, int *processed) · 571 unmodified lines | ||
| 572 | 572 | (*processed)++; |
| 573 | 573 | return true; |
| 574 | 574 | |
| 575 | − drop: | |
| 575 | +drop: | |
| 576 | 576 | /* Clean rxdes0 (which resets own bit) */ |
| 577 | 577 | rxdes->rxdes0 = cpu_to_le32(status & priv->rxdes0_edorr_mask); |
| 578 | 578 | priv->rx_pointer = ftgmac100_next_rx_pointer(priv, pointer); |
| @@ -656,6 +656,11 @@ static bool ftgmac100_tx_complete_packet(struct ftgmac100 *priv) · 77 unmodified lines | ||
| 656 | 656 | ftgmac100_free_tx_packet(priv, pointer, skb, txdes, ctl_stat); |
| 657 | 657 | txdes->txdes0 = cpu_to_le32(ctl_stat & priv->txdes0_edotr_mask); |
| 658 | 658 | |
| 659 | + /* Ensure the descriptor config is visible before setting the tx | |
| 660 | + * pointer. | |
| 661 | + */ | |
| 662 | + smp_wmb(); | |
| 663 | + | |
| 659 | 664 | priv->tx_clean_pointer = ftgmac100_next_tx_pointer(priv, pointer); |
| 660 | 665 | |
| 661 | 666 | return true; |
| @@ -809,6 +814,11 @@ static netdev_tx_t ftgmac100_hard_start_xmit(struct sk_buff *skb, · 147 unmodified lines | ||
| 809 | 814 | dma_wmb(); |
| 810 | 815 | first->txdes0 = cpu_to_le32(f_ctl_stat); |
| 811 | 816 | |
| 817 | + /* Ensure the descriptor config is visible before setting the tx | |
| 818 | + * pointer. | |
| 819 | + */ | |
| 820 | + smp_wmb(); | |
| 821 | + | |
| 812 | 822 | /* Update next TX pointer */ |
| 813 | 823 | priv->tx_pointer = pointer; |
| 814 | 824 | |
| @@ -829,7 +839,7 @@ static netdev_tx_t ftgmac100_hard_start_xmit(struct sk_buff *skb, · 14 unmodified lines | ||
| 829 | 839 | |
| 830 | 840 | return NETDEV_TX_OK; |
| 831 | 841 | |
| 832 | − dma_err: | |
| 842 | +dma_err: | |
| 833 | 843 | if (net_ratelimit()) |
| 834 | 844 | netdev_err(netdev, "map tx fragment failed\n"); |
| 835 | 845 | |
| @@ -851,7 +861,7 @@ static netdev_tx_t ftgmac100_hard_start_xmit(struct sk_buff *skb, · 15 unmodified lines | ||
| 851 | 861 | * last fragment, so we know ftgmac100_free_tx_packet() |
| 852 | 862 | * hasn't freed the skb yet. |
| 853 | 863 | */ |
| 854 | − drop: | |
| 864 | +drop: | |
| 855 | 865 | /* Drop the packet */ |
| 856 | 866 | dev_kfree_skb_any(skb); |
| 857 | 867 | netdev->stats.tx_dropped++; |
| @@ -1344,7 +1354,7 @@ static void ftgmac100_reset(struct ftgmac100 *priv) · 486 unmodified lines | ||
| 1344 | 1354 | ftgmac100_init_all(priv, true); |
| 1345 | 1355 | |
| 1346 | 1356 | netdev_dbg(netdev, "Reset done !\n"); |
| 1347 | − bail: | |
| 1357 | +bail: | |
| 1348 | 1358 | if (priv->mii_bus) |
| 1349 | 1359 | mutex_unlock(&priv->mii_bus->mdio_lock); |
| 1350 | 1360 | if (netdev->phydev) |
| @@ -1543,15 +1553,15 @@ static int ftgmac100_open(struct net_device *netdev) · 192 unmodified lines | ||
| 1543 | 1553 | |
| 1544 | 1554 | return 0; |
| 1545 | 1555 | |
| 1546 | − err_ncsi: | |
| 1556 | +err_ncsi: | |
| 1547 | 1557 | napi_disable(&priv->napi); |
| 1548 | 1558 | netif_stop_queue(netdev); |
| 1549 | − err_alloc: | |
| 1559 | +err_alloc: | |
| 1550 | 1560 | ftgmac100_free_buffers(priv); |
| 1551 | 1561 | free_irq(netdev->irq, netdev); |
| 1552 | − err_irq: | |
| 1562 | +err_irq: | |
| 1553 | 1563 | netif_napi_del(&priv->napi); |
| 1554 | − err_hw: | |
| 1564 | +err_hw: | |
| 1555 | 1565 | iowrite32(0, priv->base + FTGMAC100_OFFSET_IER); |
| 1556 | 1566 | ftgmac100_free_rings(priv); |
| 1557 | 1567 | return err; |
| @@ -2638,13 +2638,14 @@ static int dpaa2_switch_refill_bp(struct ethsw_core *ethsw) · 2637 unmodified lines | ||
| 2638 | 2638 | |
| 2639 | 2639 | static int dpaa2_switch_seed_bp(struct ethsw_core *ethsw) |
| 2640 | 2640 | { |
| 2641 | − int *count, i; | |
| 2641 | + int *count, ret, i; | |
| 2642 | 2642 | |
| 2643 | 2643 | for (i = 0; i < DPAA2_ETHSW_NUM_BUFS; i += BUFS_PER_CMD) { |
| 2644 | + ret = dpaa2_switch_add_bufs(ethsw, ethsw->bpid); | |
| 2644 | 2645 | count = ðsw->buf_count; |
| 2645 | − *count += dpaa2_switch_add_bufs(ethsw, ethsw->bpid); | |
| 2646 | + *count += ret; | |
| 2646 | 2647 | |
| 2647 | − if (unlikely(*count < BUFS_PER_CMD)) | |
| 2648 | + if (unlikely(ret < BUFS_PER_CMD)) | |
| 2648 | 2649 | return -ENOMEM; |
| 2649 | 2650 | } |
| 2650 | 2651 | |
| @@ -775,6 +775,9 @@ void fec_ptp_stop(struct platform_device *pdev) · 774 unmodified lines | ||
| 775 | 775 | struct net_device *ndev = platform_get_drvdata(pdev); |
| 776 | 776 | struct fec_enet_private *fep = netdev_priv(ndev); |
| 777 | 777 | |
| 778 | + if (fep->pps_enable) | |
| 779 | + fec_ptp_enable_pps(fep, 0); | |
| 780 | + | |
| 778 | 781 | cancel_delayed_work_sync(&fep->time_keep); |
| 779 | 782 | hrtimer_cancel(&fep->perout_timer); |
| 780 | 783 | if (fep->ptp_clock) |
| @@ -495,7 +495,7 @@ static int gve_set_channels(struct net_device *netdev, · 494 unmodified lines | ||
| 495 | 495 | return -EINVAL; |
| 496 | 496 | } |
| 497 | 497 | |
| 498 | − if (!netif_carrier_ok(netdev)) { | |
| 498 | + if (!netif_running(netdev)) { | |
| 499 | 499 | priv->tx_cfg.num_queues = new_tx; |
| 500 | 500 | priv->rx_cfg.num_queues = new_rx; |
| 501 | 501 | return 0; |
| @@ -1566,7 +1566,7 @@ static int gve_set_xdp(struct gve_priv *priv, struct bpf_prog *prog, · 1565 unmodified lines | ||
| 1566 | 1566 | u32 status; |
| 1567 | 1567 | |
| 1568 | 1568 | old_prog = READ_ONCE(priv->xdp_prog); |
| 1569 | − if (!netif_carrier_ok(priv->dev)) { | |
| 1569 | + if (!netif_running(priv->dev)) { | |
| 1570 | 1570 | WRITE_ONCE(priv->xdp_prog, prog); |
| 1571 | 1571 | if (old_prog) |
| 1572 | 1572 | bpf_prog_put(old_prog); |
| @@ -1847,7 +1847,7 @@ int gve_adjust_queues(struct gve_priv *priv, · 274 unmodified lines | ||
| 1847 | 1847 | rx_alloc_cfg.qcfg = &new_rx_config; |
| 1848 | 1848 | tx_alloc_cfg.num_rings = new_tx_config.num_queues; |
| 1849 | 1849 | |
| 1850 | − if (netif_carrier_ok(priv->dev)) { | |
| 1850 | + if (netif_running(priv->dev)) { | |
| 1851 | 1851 | err = gve_adjust_config(priv, &tx_alloc_cfg, &rx_alloc_cfg); |
| 1852 | 1852 | return err; |
| 1853 | 1853 | } |
| @@ -2064,7 +2064,7 @@ static int gve_set_features(struct net_device *netdev, · 210 unmodified lines | ||
| 2064 | 2064 | |
| 2065 | 2065 | if ((netdev->features & NETIF_F_LRO) != (features & NETIF_F_LRO)) { |
| 2066 | 2066 | netdev->features ^= NETIF_F_LRO; |
| 2067 | − if (netif_carrier_ok(netdev)) { | |
| 2067 | + if (netif_running(netdev)) { | |
| 2068 | 2068 | err = gve_adjust_config(priv, &tx_alloc_cfg, &rx_alloc_cfg); |
| 2069 | 2069 | if (err) |
| 2070 | 2070 | goto revert_features; |
| @@ -2359,7 +2359,7 @@ err: · 288 unmodified lines | ||
| 2359 | 2359 | |
| 2360 | 2360 | int gve_reset(struct gve_priv *priv, bool attempt_teardown) |
| 2361 | 2361 | { |
| 2362 | − bool was_up = netif_carrier_ok(priv->dev); | |
| 2362 | + bool was_up = netif_running(priv->dev); | |
| 2363 | 2363 | int err; |
| 2364 | 2364 | |
| 2365 | 2365 | dev_info(&priv->pdev->dev, "Performing reset\n"); |
| @@ -2700,7 +2700,7 @@ static void gve_shutdown(struct pci_dev *pdev) · 334 unmodified lines | ||
| 2700 | 2700 | { |
| 2701 | 2701 | struct net_device *netdev = pci_get_drvdata(pdev); |
| 2702 | 2702 | struct gve_priv *priv = netdev_priv(netdev); |
| 2703 | − bool was_up = netif_carrier_ok(priv->dev); | |
| 2703 | + bool was_up = netif_running(priv->dev); | |
| 2704 | 2704 | |
| 2705 | 2705 | rtnl_lock(); |
| 2706 | 2706 | if (was_up && gve_close(priv->dev)) { |
| @@ -2718,7 +2718,7 @@ static int gve_suspend(struct pci_dev *pdev, pm_message_t state) · 11 unmodified lines | ||
| 2718 | 2718 | { |
| 2719 | 2719 | struct net_device *netdev = pci_get_drvdata(pdev); |
| 2720 | 2720 | struct gve_priv *priv = netdev_priv(netdev); |
| 2721 | − bool was_up = netif_carrier_ok(priv->dev); | |
| 2721 | + bool was_up = netif_running(priv->dev); | |
| 2722 | 2722 | |
| 2723 | 2723 | priv->suspend_cnt++; |
| 2724 | 2724 | rtnl_lock(); |
| @@ -5724,6 +5724,9 @@ static int hns3_reset_notify_uninit_enet(struct hnae3_handle *handle) · 5723 unmodified lines | ||
| 5724 | 5724 | struct net_device *netdev = handle->kinfo.netdev; |
| 5725 | 5725 | struct hns3_nic_priv *priv = netdev_priv(netdev); |
| 5726 | 5726 | |
| 5727 | + if (!test_bit(HNS3_NIC_STATE_DOWN, &priv->state)) | |
| 5728 | + hns3_nic_net_stop(netdev); | |
| 5729 | + | |
| 5727 | 5730 | if (!test_and_clear_bit(HNS3_NIC_STATE_INITED, &priv->state)) { |
| 5728 | 5731 | netdev_warn(netdev, "already uninitialized\n"); |
| 5729 | 5732 | return 0; |
| @@ -1598,8 +1598,7 @@ static void hclge_query_reg_info_of_ssu(struct hclge_dev *hdev) · 1597 unmodified lines | ||
| 1598 | 1598 | { |
| 1599 | 1599 | u32 loop_para[HCLGE_MOD_MSG_PARA_ARRAY_MAX_SIZE] = {0}; |
| 1600 | 1600 | struct hclge_mod_reg_common_msg msg; |
| 1601 | − u8 i, j, num; | |
| 1602 | − u32 loop_time; | |
| 1601 | + u8 i, j, num, loop_time; | |
| 1603 | 1602 | |
| 1604 | 1603 | num = ARRAY_SIZE(hclge_ssu_reg_common_msg); |
| 1605 | 1604 | for (i = 0; i < num; i++) { |
| @@ -1609,7 +1608,8 @@ static void hclge_query_reg_info_of_ssu(struct hclge_dev *hdev) · 3 unmodified lines | ||
| 1609 | 1608 | loop_time = 1; |
| 1610 | 1609 | loop_para[0] = 0; |
| 1611 | 1610 | if (msg.need_para) { |
| 1612 | − loop_time = hdev->ae_dev->dev_specs.tnl_num; | |
| 1611 | + loop_time = min(hdev->ae_dev->dev_specs.tnl_num, | |
| 1612 | + HCLGE_MOD_MSG_PARA_ARRAY_MAX_SIZE); | |
| 1613 | 1613 | for (j = 0; j < loop_time; j++) |
| 1614 | 1614 | loop_para[j] = j + 1; |
| 1615 | 1615 | } |
| @@ -2653,8 +2653,17 @@ static int hclge_cfg_mac_speed_dup_h(struct hnae3_handle *handle, int speed, · 2652 unmodified lines | ||
| 2653 | 2653 | { |
| 2654 | 2654 | struct hclge_vport *vport = hclge_get_vport(handle); |
| 2655 | 2655 | struct hclge_dev *hdev = vport->back; |
| 2656 | + int ret; | |
| 2657 | + | |
| 2658 | + ret = hclge_cfg_mac_speed_dup(hdev, speed, duplex, lane_num); | |
| 2659 | + | |
| 2660 | + if (ret) | |
| 2661 | + return ret; | |
| 2656 | 2662 | |
| 2657 | − return hclge_cfg_mac_speed_dup(hdev, speed, duplex, lane_num); | |
| 2663 | + hdev->hw.mac.req_speed = speed; | |
| 2664 | + hdev->hw.mac.req_duplex = duplex; | |
| 2665 | + | |
| 2666 | + return 0; | |
| 2658 | 2667 | } |
| 2659 | 2668 | |
| 2660 | 2669 | static int hclge_set_autoneg_en(struct hclge_dev *hdev, bool enable) |
| @@ -2956,17 +2965,20 @@ static int hclge_mac_init(struct hclge_dev *hdev) · 295 unmodified lines | ||
| 2956 | 2965 | if (!test_bit(HCLGE_STATE_RST_HANDLING, &hdev->state)) |
| 2957 | 2966 | hdev->hw.mac.duplex = HCLGE_MAC_FULL; |
| 2958 | 2967 | |
| 2959 | − ret = hclge_cfg_mac_speed_dup_hw(hdev, hdev->hw.mac.speed, | |
| 2960 | − hdev->hw.mac.duplex, hdev->hw.mac.lane_num); | |
| 2961 | − if (ret) | |
| 2962 | − return ret; | |
| 2963 | − | |
| 2964 | 2968 | if (hdev->hw.mac.support_autoneg) { |
| 2965 | 2969 | ret = hclge_set_autoneg_en(hdev, hdev->hw.mac.autoneg); |
| 2966 | 2970 | if (ret) |
| 2967 | 2971 | return ret; |
| 2968 | 2972 | } |
| 2969 | 2973 | |
| 2974 | + if (!hdev->hw.mac.autoneg) { | |
| 2975 | + ret = hclge_cfg_mac_speed_dup_hw(hdev, hdev->hw.mac.req_speed, | |
| 2976 | + hdev->hw.mac.req_duplex, | |
| 2977 | + hdev->hw.mac.lane_num); | |
| 2978 | + if (ret) | |
| 2979 | + return ret; | |
| 2980 | + } | |
| 2981 | + | |
| 2970 | 2982 | mac->link = 0; |
| 2971 | 2983 | |
| 2972 | 2984 | if (mac->user_fec_mode & BIT(HNAE3_FEC_USER_DEF)) { |
| @@ -11444,7 +11456,7 @@ static void hclge_pci_uninit(struct hclge_dev *hdev) · 8471 unmodified lines | ||
| 11444 | 11456 | |
| 11445 | 11457 | pcim_iounmap(pdev, hdev->hw.hw.io_base); |
| 11446 | 11458 | pci_free_irq_vectors(pdev); |
| 11447 | − pci_release_mem_regions(pdev); | |
| 11459 | + pci_release_regions(pdev); | |
| 11448 | 11460 | pci_disable_device(pdev); |
| 11449 | 11461 | } |
| 11450 | 11462 | |
| @@ -11516,8 +11528,8 @@ static void hclge_reset_done(struct hnae3_ae_dev *ae_dev) · 65 unmodified lines | ||
| 11516 | 11528 | dev_err(&hdev->pdev->dev, "fail to rebuild, ret=%d\n", ret); |
| 11517 | 11529 | |
| 11518 | 11530 | hdev->reset_type = HNAE3_NONE_RESET; |
| 11519 | − clear_bit(HCLGE_STATE_RST_HANDLING, &hdev->state); | |
| 11520 | − up(&hdev->reset_sem); | |
| 11531 | + if (test_and_clear_bit(HCLGE_STATE_RST_HANDLING, &hdev->state)) | |
| 11532 | + up(&hdev->reset_sem); | |
| 11521 | 11533 | } |
| 11522 | 11534 | |
| 11523 | 11535 | static void hclge_clear_resetting_state(struct hclge_dev *hdev) |
| @@ -191,6 +191,9 @@ static void hclge_mac_adjust_link(struct net_device *netdev) · 190 unmodified lines | ||
| 191 | 191 | if (ret) |
| 192 | 192 | netdev_err(netdev, "failed to adjust link.\n"); |
| 193 | 193 | |
| 194 | + hdev->hw.mac.req_speed = (u32)speed; | |
| 195 | + hdev->hw.mac.req_duplex = (u8)duplex; | |
| 196 | + | |
| 194 | 197 | ret = hclge_cfg_flowctrl(hdev); |
| 195 | 198 | if (ret) |
| 196 | 199 | netdev_err(netdev, "failed to configure flow control.\n"); |
| @@ -1747,8 +1747,8 @@ static void hclgevf_reset_done(struct hnae3_ae_dev *ae_dev) · 1746 unmodified lines | ||
| 1747 | 1747 | ret); |
| 1748 | 1748 | |
| 1749 | 1749 | hdev->reset_type = HNAE3_NONE_RESET; |
| 1750 | − clear_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state); | |
| 1751 | − up(&hdev->reset_sem); | |
| 1750 | + if (test_and_clear_bit(HCLGEVF_STATE_RST_HANDLING, &hdev->state)) | |
| 1751 | + up(&hdev->reset_sem); | |
| 1752 | 1752 | } |
| 1753 | 1753 | |
| 1754 | 1754 | static u32 hclgevf_get_fw_version(struct hnae3_handle *handle) |
| @@ -337,7 +337,7 @@ int ice_devlink_create_pf_port(struct ice_pf *pf) · 336 unmodified lines | ||
| 337 | 337 | return -EIO; |
| 338 | 338 | |
| 339 | 339 | attrs.flavour = DEVLINK_PORT_FLAVOUR_PHYSICAL; |
| 340 | − attrs.phys.port_number = pf->hw.bus.func; | |
| 340 | + attrs.phys.port_number = pf->hw.pf_id; | |
| 341 | 341 | |
| 342 | 342 | /* As FW supports only port split options for whole device, |
| 343 | 343 | * set port split options only for first PF. |
| @@ -455,7 +455,7 @@ int ice_devlink_create_vf_port(struct ice_vf *vf) · 111 unmodified lines | ||
| 455 | 455 | return -EINVAL; |
| 456 | 456 | |
| 457 | 457 | attrs.flavour = DEVLINK_PORT_FLAVOUR_PCI_VF; |
| 458 | − attrs.pci_vf.pf = pf->hw.bus.func; | |
| 458 | + attrs.pci_vf.pf = pf->hw.pf_id; | |
| 459 | 459 | attrs.pci_vf.vf = vf->vf_id; |
| 460 | 460 | |
| 461 | 461 | ice_devlink_set_switch_id(pf, &attrs.switch_id); |
| @@ -765,18 +765,17 @@ static inline struct xsk_buff_pool *ice_get_xp_from_qid(struct ice_vsi *vsi, · 764 unmodified lines | ||
| 765 | 765 | } |
| 766 | 766 | |
| 767 | 767 | /** |
| 768 | − * ice_xsk_pool - get XSK buffer pool bound to a ring | |
| 768 | + * ice_rx_xsk_pool - assign XSK buff pool to Rx ring | |
| 769 | 769 | * @ring: Rx ring to use |
| 770 | 770 | * |
| 771 | − * Returns a pointer to xsk_buff_pool structure if there is a buffer pool | |
| 772 | − * present, NULL otherwise. | |
| 771 | + * Sets XSK buff pool pointer on Rx ring. | |
| 773 | 772 | */ |
| 774 | −static inline struct xsk_buff_pool *ice_xsk_pool(struct ice_rx_ring *ring) | |
| 773 | +static inline void ice_rx_xsk_pool(struct ice_rx_ring *ring) | |
| 775 | 774 | { |
| 776 | 775 | struct ice_vsi *vsi = ring->vsi; |
| 777 | 776 | u16 qid = ring->q_index; |
| 778 | 777 | |
| 779 | − return ice_get_xp_from_qid(vsi, qid); | |
| 778 | + WRITE_ONCE(ring->xsk_pool, ice_get_xp_from_qid(vsi, qid)); | |
| 780 | 779 | } |
| 781 | 780 | |
| 782 | 781 | /** |
| @@ -801,7 +800,7 @@ static inline void ice_tx_xsk_pool(struct ice_vsi *vsi, u16 qid) · 18 unmodified lines | ||
| 801 | 800 | if (!ring) |
| 802 | 801 | return; |
| 803 | 802 | |
| 804 | − ring->xsk_pool = ice_get_xp_from_qid(vsi, qid); | |
| 803 | + WRITE_ONCE(ring->xsk_pool, ice_get_xp_from_qid(vsi, qid)); | |
| 805 | 804 | } |
| 806 | 805 | |
| 807 | 806 | /** |
| @@ -512,6 +512,25 @@ static void ice_xsk_pool_fill_cb(struct ice_rx_ring *ring) · 511 unmodified lines | ||
| 512 | 512 | xsk_pool_fill_cb(ring->xsk_pool, &desc); |
| 513 | 513 | } |
| 514 | 514 | |
| 515 | +/** | |
| 516 | + * ice_get_frame_sz - calculate xdp_buff::frame_sz | |
| 517 | + * @rx_ring: the ring being configured | |
| 518 | + * | |
| 519 | + * Return frame size based on underlying PAGE_SIZE | |
| 520 | + */ | |
| 521 | +static unsigned int ice_get_frame_sz(struct ice_rx_ring *rx_ring) | |
| 522 | +{ | |
| 523 | + unsigned int frame_sz; | |
| 524 | + | |
| 525 | +#if (PAGE_SIZE >= 8192) | |
| 526 | + frame_sz = rx_ring->rx_buf_len; | |
| 527 | +#else | |
| 528 | + frame_sz = ice_rx_pg_size(rx_ring) / 2; | |
| 529 | +#endif | |
| 530 | + | |
| 531 | + return frame_sz; | |
| 532 | +} | |
| 533 | + | |
| 515 | 534 | /** |
| 516 | 535 | * ice_vsi_cfg_rxq - Configure an Rx queue |
| 517 | 536 | * @ring: the ring being configured |
| @@ -536,7 +555,7 @@ static int ice_vsi_cfg_rxq(struct ice_rx_ring *ring) · 18 unmodified lines | ||
| 536 | 555 | return err; |
| 537 | 556 | } |
| 538 | 557 | |
| 539 | − ring->xsk_pool = ice_xsk_pool(ring); | |
| 558 | + ice_rx_xsk_pool(ring); | |
| 540 | 559 | if (ring->xsk_pool) { |
| 541 | 560 | xdp_rxq_info_unreg(&ring->xdp_rxq); |
| 542 | 561 | |
| @@ -576,7 +595,7 @@ static int ice_vsi_cfg_rxq(struct ice_rx_ring *ring) · 33 unmodified lines | ||
| 576 | 595 | } |
| 577 | 596 | } |
| 578 | 597 | |
| 579 | − xdp_init_buff(&ring->xdp, ice_rx_pg_size(ring) / 2, &ring->xdp_rxq); | |
| 598 | + xdp_init_buff(&ring->xdp, ice_get_frame_sz(ring), &ring->xdp_rxq); | |
| 580 | 599 | ring->xdp.data = NULL; |
| 581 | 600 | ring->xdp_ext.pkt_ctx = &ring->pkt_ctx; |
| 582 | 601 | err = ice_setup_rx_ctx(ring); |
| @@ -597,7 +616,7 @@ static int ice_vsi_cfg_rxq(struct ice_rx_ring *ring) · 14 unmodified lines | ||
| 597 | 616 | return 0; |
| 598 | 617 | } |
| 599 | 618 | |
| 600 | − ok = ice_alloc_rx_bufs_zc(ring, num_bufs); | |
| 619 | + ok = ice_alloc_rx_bufs_zc(ring, ring->xsk_pool, num_bufs); | |
| 601 | 620 | if (!ok) { |
| 602 | 621 | u16 pf_q = ring->vsi->rxq_map[ring->q_index]; |
| 603 | 622 | |
| @@ -4673,10 +4673,10 @@ static int ice_get_port_fec_stats(struct ice_hw *hw, u16 pcs_quad, u16 pcs_port, · 4672 unmodified lines | ||
| 4673 | 4673 | if (err) |
| 4674 | 4674 | return err; |
| 4675 | 4675 | |
| 4676 | − fec_stats->uncorrectable_blocks.total = (fec_corr_high_val << 16) + | |
| 4677 | − fec_corr_low_val; | |
| 4678 | − fec_stats->corrected_blocks.total = (fec_uncorr_high_val << 16) + | |
| 4679 | − fec_uncorr_low_val; | |
| 4676 | + fec_stats->corrected_blocks.total = (fec_corr_high_val << 16) + | |
| 4677 | + fec_corr_low_val; | |
| 4678 | + fec_stats->uncorrectable_blocks.total = (fec_uncorr_high_val << 16) + | |
| 4679 | + fec_uncorr_low_val; | |
| 4680 | 4680 | return 0; |
| 4681 | 4681 | } |
| 4682 | 4682 | |
| @@ -559,6 +559,8 @@ ice_prepare_for_reset(struct ice_pf *pf, enum ice_reset_req reset_type) · 558 unmodified lines | ||
| 559 | 559 | if (test_bit(ICE_PREPARED_FOR_RESET, pf->state)) |
| 560 | 560 | return; |
| 561 | 561 | |
| 562 | + synchronize_irq(pf->oicr_irq.virq); | |
| 563 | + | |
| 562 | 564 | ice_unplug_aux_dev(pf); |
| 563 | 565 | |
| 564 | 566 | /* Notify VFs of impending reset */ |
| @@ -2948,7 +2950,7 @@ static void ice_vsi_rx_napi_schedule(struct ice_vsi *vsi) · 2383 unmodified lines | ||
| 2948 | 2950 | ice_for_each_rxq(vsi, i) { |
| 2949 | 2951 | struct ice_rx_ring *rx_ring = vsi->rx_rings[i]; |
| 2950 | 2952 | |
| 2951 | − if (rx_ring->xsk_pool) | |
| 2953 | + if (READ_ONCE(rx_ring->xsk_pool)) | |
| 2952 | 2954 | napi_schedule(&rx_ring->q_vector->napi); |
| 2953 | 2955 | } |
| 2954 | 2956 | } |
| @@ -1477,6 +1477,10 @@ void ice_ptp_link_change(struct ice_pf *pf, u8 port, bool linkup) · 1476 unmodified lines | ||
| 1477 | 1477 | /* Update cached link status for this port immediately */ |
| 1478 | 1478 | ptp_port->link_up = linkup; |
| 1479 | 1479 | |
| 1480 | + /* Skip HW writes if reset is in progress */ | |
| 1481 | + if (pf->hw.reset_ongoing) | |
| 1482 | + return; | |
| 1483 | + | |
| 1480 | 1484 | switch (hw->ptp.phy_model) { |
| 1481 | 1485 | case ICE_PHY_E810: |
| 1482 | 1486 | /* Do not reconfigure E810 PHY */ |
| @@ -456,7 +456,7 @@ void ice_free_rx_ring(struct ice_rx_ring *rx_ring) · 455 unmodified lines | ||
| 456 | 456 | if (rx_ring->vsi->type == ICE_VSI_PF) |
| 457 | 457 | if (xdp_rxq_info_is_reg(&rx_ring->xdp_rxq)) |
| 458 | 458 | xdp_rxq_info_unreg(&rx_ring->xdp_rxq); |
| 459 | − rx_ring->xdp_prog = NULL; | |
| 459 | + WRITE_ONCE(rx_ring->xdp_prog, NULL); | |
| 460 | 460 | if (rx_ring->xsk_pool) { |
| 461 | 461 | kfree(rx_ring->xdp_buf); |
| 462 | 462 | rx_ring->xdp_buf = NULL; |
| @@ -521,30 +521,6 @@ err: · 58 unmodified lines | ||
| 521 | 521 | return -ENOMEM; |
| 522 | 522 | } |
| 523 | 523 | |
| 524 | −/** | |
| 525 | − * ice_rx_frame_truesize | |
| 526 | − * @rx_ring: ptr to Rx ring | |
| 527 | − * @size: size | |
| 528 | − * | |
| 529 | − * calculate the truesize with taking into the account PAGE_SIZE of | |
| 530 | − * underlying arch | |
| 531 | − */ | |
| 532 | −static unsigned int | |
| 533 | −ice_rx_frame_truesize(struct ice_rx_ring *rx_ring, const unsigned int size) | |
| 534 | −{ | |
| 535 | − unsigned int truesize; | |
| 536 | − | |
| 537 | −#if (PAGE_SIZE < 8192) | |
| 538 | − truesize = ice_rx_pg_size(rx_ring) / 2; /* Must be power-of-2 */ | |
| 539 | −#else | |
| 540 | − truesize = rx_ring->rx_offset ? | |
| 541 | − SKB_DATA_ALIGN(rx_ring->rx_offset + size) + | |
| 542 | − SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) : | |
| 543 | − SKB_DATA_ALIGN(size); | |
| 544 | −#endif | |
| 545 | − return truesize; | |
| 546 | −} | |
| 547 | − | |
| 548 | 524 | /** |
| 549 | 525 | * ice_run_xdp - Executes an XDP program on initialized xdp_buff |
| 550 | 526 | * @rx_ring: Rx ring |
| @@ -837,16 +813,15 @@ ice_can_reuse_rx_page(struct ice_rx_buf *rx_buf) · 286 unmodified lines | ||
| 837 | 813 | if (!dev_page_is_reusable(page)) |
| 838 | 814 | return false; |
| 839 | 815 | |
| 840 | −#if (PAGE_SIZE < 8192) | |
| 841 | 816 | /* if we are only owner of page we can reuse it */ |
| 842 | 817 | if (unlikely(rx_buf->pgcnt - pagecnt_bias > 1)) |
| 843 | 818 | return false; |
| 844 | −#else | |
| 819 | +#if (PAGE_SIZE >= 8192) | |
| 845 | 820 | #define ICE_LAST_OFFSET \ |
| 846 | − (SKB_WITH_OVERHEAD(PAGE_SIZE) - ICE_RXBUF_2048) | |
| 821 | + (SKB_WITH_OVERHEAD(PAGE_SIZE) - ICE_RXBUF_3072) | |
| 847 | 822 | if (rx_buf->page_offset > ICE_LAST_OFFSET) |
| 848 | 823 | return false; |
| 849 | −#endif /* PAGE_SIZE < 8192) */ | |
| 824 | +#endif /* PAGE_SIZE >= 8192) */ | |
| 850 | 825 | |
| 851 | 826 | /* If we have drained the page fragment pool we need to update |
| 852 | 827 | * the pagecnt_bias and page count so that we fully restock the |
| @@ -949,12 +924,7 @@ ice_get_rx_buf(struct ice_rx_ring *rx_ring, const unsigned int size, · 96 unmodified lines | ||
| 949 | 924 | struct ice_rx_buf *rx_buf; |
| 950 | 925 | |
| 951 | 926 | rx_buf = &rx_ring->rx_buf[ntc]; |
| 952 | − rx_buf->pgcnt = | |
| 953 | −#if (PAGE_SIZE < 8192) | |
| 954 | − page_count(rx_buf->page); | |
| 955 | −#else | |
| 956 | − 0; | |
| 957 | −#endif | |
| 927 | + rx_buf->pgcnt = page_count(rx_buf->page); | |
| 958 | 928 | prefetchw(rx_buf->page); |
| 959 | 929 | |
| 960 | 930 | if (!size) |
| @@ -1160,11 +1130,6 @@ int ice_clean_rx_irq(struct ice_rx_ring *rx_ring, int budget) · 199 unmodified lines | ||
| 1160 | 1130 | bool failure; |
| 1161 | 1131 | u32 first; |
| 1162 | 1132 | |
| 1163 | − /* Frame size depend on rx_ring setup when PAGE_SIZE=4K */ | |
| 1164 | −#if (PAGE_SIZE < 8192) | |
| 1165 | − xdp->frame_sz = ice_rx_frame_truesize(rx_ring, 0); | |
| 1166 | −#endif | |
| 1167 | − | |
| 1168 | 1133 | xdp_prog = READ_ONCE(rx_ring->xdp_prog); |
| 1169 | 1134 | if (xdp_prog) { |
| 1170 | 1135 | xdp_ring = rx_ring->xdp_ring; |
| @@ -1223,10 +1188,6 @@ int ice_clean_rx_irq(struct ice_rx_ring *rx_ring, int budget) · 52 unmodified lines | ||
| 1223 | 1188 | hard_start = page_address(rx_buf->page) + rx_buf->page_offset - |
| 1224 | 1189 | offset; |
| 1225 | 1190 | xdp_prepare_buff(xdp, hard_start, offset, size, !!offset); |
| 1226 | −#if (PAGE_SIZE > 4096) | |
| 1227 | − /* At larger PAGE_SIZE, frame_sz depend on len size */ | |
| 1228 | − xdp->frame_sz = ice_rx_frame_truesize(rx_ring, size); | |
| 1229 | −#endif | |
| 1230 | 1191 | xdp_buff_clear_frags_flag(xdp); |
| 1231 | 1192 | } else if (ice_add_xdp_frag(rx_ring, xdp, rx_buf, size)) { |
| 1232 | 1193 | break; |
| @@ -1521,10 +1482,11 @@ int ice_napi_poll(struct napi_struct *napi, int budget) · 288 unmodified lines | ||
| 1521 | 1482 | * budget and be more aggressive about cleaning up the Tx descriptors. |
| 1522 | 1483 | */ |
| 1523 | 1484 | ice_for_each_tx_ring(tx_ring, q_vector->tx) { |
| 1485 | + struct xsk_buff_pool *xsk_pool = READ_ONCE(tx_ring->xsk_pool); | |
| 1524 | 1486 | bool wd; |
| 1525 | 1487 | |
| 1526 | − if (tx_ring->xsk_pool) | |
| 1527 | − wd = ice_xmit_zc(tx_ring); | |
| 1488 | + if (xsk_pool) | |
| 1489 | + wd = ice_xmit_zc(tx_ring, xsk_pool); | |
| 1528 | 1490 | else if (ice_ring_is_xdp(tx_ring)) |
| 1529 | 1491 | wd = true; |
| 1530 | 1492 | else |
| @@ -1550,6 +1512,7 @@ int ice_napi_poll(struct napi_struct *napi, int budget) · 19 unmodified lines | ||
| 1550 | 1512 | budget_per_ring = budget; |
| 1551 | 1513 | |
| 1552 | 1514 | ice_for_each_rx_ring(rx_ring, q_vector->rx) { |
| 1515 | + struct xsk_buff_pool *xsk_pool = READ_ONCE(rx_ring->xsk_pool); | |
| 1553 | 1516 | int cleaned; |
| 1554 | 1517 | |
| 1555 | 1518 | /* A dedicated path for zero-copy allows making a single |
| @@ -1557,7 +1520,7 @@ int ice_napi_poll(struct napi_struct *napi, int budget) · 1 unmodified line | ||
| 1557 | 1520 | * ice_clean_rx_irq function and makes the codebase cleaner. |
| 1558 | 1521 | */ |
| 1559 | 1522 | cleaned = rx_ring->xsk_pool ? |
| 1560 | − ice_clean_rx_irq_zc(rx_ring, budget_per_ring) : | |
| 1523 | + ice_clean_rx_irq_zc(rx_ring, xsk_pool, budget_per_ring) : | |
| 1561 | 1524 | ice_clean_rx_irq(rx_ring, budget_per_ring); |
| 1562 | 1525 | work_done += cleaned; |
| 1563 | 1526 | /* if we clean as many as budgeted, we must not be done */ |
| @@ -52,10 +52,8 @@ static void ice_qp_reset_stats(struct ice_vsi *vsi, u16 q_idx) · 51 unmodified lines | ||
| 52 | 52 | static void ice_qp_clean_rings(struct ice_vsi *vsi, u16 q_idx) |
| 53 | 53 | { |
| 54 | 54 | ice_clean_tx_ring(vsi->tx_rings[q_idx]); |
| 55 | − if (ice_is_xdp_ena_vsi(vsi)) { | |
| 56 | − synchronize_rcu(); | |
| 55 | + if (ice_is_xdp_ena_vsi(vsi)) | |
| 57 | 56 | ice_clean_tx_ring(vsi->xdp_rings[q_idx]); |
| 58 | − } | |
| 59 | 57 | ice_clean_rx_ring(vsi->rx_rings[q_idx]); |
| 60 | 58 | } |
| 61 | 59 | |
| @@ -112,25 +110,29 @@ ice_qvec_dis_irq(struct ice_vsi *vsi, struct ice_rx_ring *rx_ring, · 50 unmodified lines | ||
| 112 | 110 | * ice_qvec_cfg_msix - Enable IRQ for given queue vector |
| 113 | 111 | * @vsi: the VSI that contains queue vector |
| 114 | 112 | * @q_vector: queue vector |
| 113 | + * @qid: queue index | |
| 115 | 114 | */ |
| 116 | 115 | static void |
| 117 | −ice_qvec_cfg_msix(struct ice_vsi *vsi, struct ice_q_vector *q_vector) | |
| 116 | +ice_qvec_cfg_msix(struct ice_vsi *vsi, struct ice_q_vector *q_vector, u16 qid) | |
| 118 | 117 | { |
| 119 | 118 | u16 reg_idx = q_vector->reg_idx; |
| 120 | 119 | struct ice_pf *pf = vsi->back; |
| 121 | 120 | struct ice_hw *hw = &pf->hw; |
| 122 | − struct ice_tx_ring *tx_ring; | |
| 123 | − struct ice_rx_ring *rx_ring; | |
| 121 | + int q, _qid = qid; | |
| 124 | 122 | |
| 125 | 123 | ice_cfg_itr(hw, q_vector); |
| 126 | 124 | |
| 127 | − ice_for_each_tx_ring(tx_ring, q_vector->tx) | |
| 128 | − ice_cfg_txq_interrupt(vsi, tx_ring->reg_idx, reg_idx, | |
| 129 | − q_vector->tx.itr_idx); | |
| 125 | + for (q = 0; q < q_vector->num_ring_tx; q++) { | |
| 126 | + ice_cfg_txq_interrupt(vsi, _qid, reg_idx, q_vector->tx.itr_idx); | |
| 127 | + _qid++; | |
| 128 | + } | |
| 129 | + | |
| 130 | + _qid = qid; | |
| 130 | 131 | |
| 131 | − ice_for_each_rx_ring(rx_ring, q_vector->rx) | |
| 132 | − ice_cfg_rxq_interrupt(vsi, rx_ring->reg_idx, reg_idx, | |
| 133 | − q_vector->rx.itr_idx); | |
| 132 | + for (q = 0; q < q_vector->num_ring_rx; q++) { | |
| 133 | + ice_cfg_rxq_interrupt(vsi, _qid, reg_idx, q_vector->rx.itr_idx); | |
| 134 | + _qid++; | |
| 135 | + } | |
| 134 | 136 | |
| 135 | 137 | ice_flush(hw); |
| 136 | 138 | } |
| @@ -164,6 +166,7 @@ static int ice_qp_dis(struct ice_vsi *vsi, u16 q_idx) · 27 unmodified lines | ||
| 164 | 166 | struct ice_tx_ring *tx_ring; |
| 165 | 167 | struct ice_rx_ring *rx_ring; |
| 166 | 168 | int timeout = 50; |
| 169 | + int fail = 0; | |
| 167 | 170 | int err; |
| 168 | 171 | |
| 169 | 172 | if (q_idx >= vsi->num_rxq || q_idx >= vsi->num_txq) |
| @@ -180,15 +183,17 @@ static int ice_qp_dis(struct ice_vsi *vsi, u16 q_idx) · 10 unmodified lines | ||
| 180 | 183 | usleep_range(1000, 2000); |
| 181 | 184 | } |
| 182 | 185 | |
| 186 | + synchronize_net(); | |
| 187 | + netif_carrier_off(vsi->netdev); | |
| 188 | + netif_tx_stop_queue(netdev_get_tx_queue(vsi->netdev, q_idx)); | |
| 189 | + | |
| 183 | 190 | ice_qvec_dis_irq(vsi, rx_ring, q_vector); |
| 184 | 191 | ice_qvec_toggle_napi(vsi, q_vector, false); |
| 185 | 192 | |
| 186 | − netif_tx_stop_queue(netdev_get_tx_queue(vsi->netdev, q_idx)); | |
| 187 | − | |
| 188 | 193 | ice_fill_txq_meta(vsi, tx_ring, &txq_meta); |
| 189 | 194 | err = ice_vsi_stop_tx_ring(vsi, ICE_NO_RESET, 0, tx_ring, &txq_meta); |
| 190 | − if (err) | |
| 191 | − return err; | |
| 195 | + if (!fail) | |
| 196 | + fail = err; | |
| 192 | 197 | if (ice_is_xdp_ena_vsi(vsi)) { |
| 193 | 198 | struct ice_tx_ring *xdp_ring = vsi->xdp_rings[q_idx]; |
| 194 | 199 | |
| @@ -196,17 +201,15 @@ static int ice_qp_dis(struct ice_vsi *vsi, u16 q_idx) · 1 unmodified line | ||
| 196 | 201 | ice_fill_txq_meta(vsi, xdp_ring, &txq_meta); |
| 197 | 202 | err = ice_vsi_stop_tx_ring(vsi, ICE_NO_RESET, 0, xdp_ring, |
| 198 | 203 | &txq_meta); |
| 199 | − if (err) | |
| 200 | − return err; | |
| 204 | + if (!fail) | |
| 205 | + fail = err; | |
| 201 | 206 | } |
| 202 | − err = ice_vsi_ctrl_one_rx_ring(vsi, false, q_idx, true); | |
| 203 | − if (err) | |
| 204 | − return err; | |
| 205 | 207 | |
| 208 | + ice_vsi_ctrl_one_rx_ring(vsi, false, q_idx, false); | |
| 206 | 209 | ice_qp_clean_rings(vsi, q_idx); |
| 207 | 210 | ice_qp_reset_stats(vsi, q_idx); |
| 208 | 211 | |
| 209 | − return 0; | |
| 212 | + return fail; | |
| 210 | 213 | } |
| 211 | 214 | |
| 212 | 215 | /** |
| @@ -219,40 +222,48 @@ static int ice_qp_dis(struct ice_vsi *vsi, u16 q_idx) · 6 unmodified lines | ||
| 219 | 222 | static int ice_qp_ena(struct ice_vsi *vsi, u16 q_idx) |
| 220 | 223 | { |
| 221 | 224 | struct ice_q_vector *q_vector; |
| 225 | + int fail = 0; | |
| 226 | + bool link_up; | |
| 222 | 227 | int err; |
| 223 | 228 | |
| 224 | 229 | err = ice_vsi_cfg_single_txq(vsi, vsi->tx_rings, q_idx); |
| 225 | − if (err) | |
| 226 | − return err; | |
| 230 | + if (!fail) | |
| 231 | + fail = err; | |
| 227 | 232 | |
| 228 | 233 | if (ice_is_xdp_ena_vsi(vsi)) { |
| 229 | 234 | struct ice_tx_ring *xdp_ring = vsi->xdp_rings[q_idx]; |
| 230 | 235 | |
| 231 | 236 | err = ice_vsi_cfg_single_txq(vsi, vsi->xdp_rings, q_idx); |
| 232 | − if (err) | |
| 233 | − return err; | |
| 237 | + if (!fail) | |
| 238 | + fail = err; | |
| 234 | 239 | ice_set_ring_xdp(xdp_ring); |
| 235 | 240 | ice_tx_xsk_pool(vsi, q_idx); |
| 236 | 241 | } |
| 237 | 242 | |
| 238 | 243 | err = ice_vsi_cfg_single_rxq(vsi, q_idx); |
| 239 | − if (err) | |
| 240 | − return err; | |
| 244 | + if (!fail) | |
| 245 | + fail = err; | |
| 241 | 246 | |
| 242 | 247 | q_vector = vsi->rx_rings[q_idx]->q_vector; |
| 243 | − ice_qvec_cfg_msix(vsi, q_vector); | |
| 248 | + ice_qvec_cfg_msix(vsi, q_vector, q_idx); | |
| 244 | 249 | |
| 245 | 250 | err = ice_vsi_ctrl_one_rx_ring(vsi, true, q_idx, true); |
| 246 | − if (err) | |
| 247 | − return err; | |
| 251 | + if (!fail) | |
| 252 | + fail = err; | |
| 248 | 253 | |
| 249 | 254 | ice_qvec_toggle_napi(vsi, q_vector, true); |
| 250 | 255 | ice_qvec_ena_irq(vsi, q_vector); |
| 251 | 256 | |
| 252 | − netif_tx_start_queue(netdev_get_tx_queue(vsi->netdev, q_idx)); | |
| 257 | + /* make sure NAPI sees updated ice_{t,x}_ring::xsk_pool */ | |
| 258 | + synchronize_net(); | |
| 259 | + ice_get_link_status(vsi->port_info, &link_up); | |
| 260 | + if (link_up) { | |
| 261 | + netif_tx_start_queue(netdev_get_tx_queue(vsi->netdev, q_idx)); | |
| 262 | + netif_carrier_on(vsi->netdev); | |
| 263 | + } | |
| 253 | 264 | clear_bit(ICE_CFG_BUSY, vsi->state); |
| 254 | 265 | |
| 255 | − return 0; | |
| 266 | + return fail; | |
| 256 | 267 | } |
| 257 | 268 | |
| 258 | 269 | /** |
| @@ -459,6 +470,7 @@ static u16 ice_fill_rx_descs(struct xsk_buff_pool *pool, struct xdp_buff **xdp, · 200 unmodified lines | ||
| 459 | 470 | /** |
| 460 | 471 | * __ice_alloc_rx_bufs_zc - allocate a number of Rx buffers |
| 461 | 472 | * @rx_ring: Rx ring |
| 473 | + * @xsk_pool: XSK buffer pool to pick buffers to be filled by HW | |
| 462 | 474 | * @count: The number of buffers to allocate |
| 463 | 475 | * |
| 464 | 476 | * Place the @count of descriptors onto Rx ring. Handle the ring wrap |
| @@ -467,7 +479,8 @@ static u16 ice_fill_rx_descs(struct xsk_buff_pool *pool, struct xdp_buff **xdp, · 2 unmodified lines | ||
| 467 | 479 | * |
| 468 | 480 | * Returns true if all allocations were successful, false if any fail. |
| 469 | 481 | */ |
| 470 | −static bool __ice_alloc_rx_bufs_zc(struct ice_rx_ring *rx_ring, u16 count) | |
| 482 | +static bool __ice_alloc_rx_bufs_zc(struct ice_rx_ring *rx_ring, | |
| 483 | + struct xsk_buff_pool *xsk_pool, u16 count) | |
| 471 | 484 | { |
| 472 | 485 | u32 nb_buffs_extra = 0, nb_buffs = 0; |
| 473 | 486 | union ice_32b_rx_flex_desc *rx_desc; |
| @@ -479,8 +492,7 @@ static bool __ice_alloc_rx_bufs_zc(struct ice_rx_ring *rx_ring, u16 count) · 5 unmodified lines | ||
| 479 | 492 | xdp = ice_xdp_buf(rx_ring, ntu); |
| 480 | 493 | |
| 481 | 494 | if (ntu + count >= rx_ring->count) { |
| 482 | − nb_buffs_extra = ice_fill_rx_descs(rx_ring->xsk_pool, xdp, | |
| 483 | − rx_desc, | |
| 495 | + nb_buffs_extra = ice_fill_rx_descs(xsk_pool, xdp, rx_desc, | |
| 484 | 496 | rx_ring->count - ntu); |
| 485 | 497 | if (nb_buffs_extra != rx_ring->count - ntu) { |
| 486 | 498 | ntu += nb_buffs_extra; |
| @@ -493,7 +505,7 @@ static bool __ice_alloc_rx_bufs_zc(struct ice_rx_ring *rx_ring, u16 count) · 6 unmodified lines | ||
| 493 | 505 | ice_release_rx_desc(rx_ring, 0); |
| 494 | 506 | } |
| 495 | 507 | |
| 496 | − nb_buffs = ice_fill_rx_descs(rx_ring->xsk_pool, xdp, rx_desc, count); | |
| 508 | + nb_buffs = ice_fill_rx_descs(xsk_pool, xdp, rx_desc, count); | |
| 497 | 509 | |
| 498 | 510 | ntu += nb_buffs; |
| 499 | 511 | if (ntu == rx_ring->count) |
| @@ -509,6 +521,7 @@ exit: · 9 unmodified lines | ||
| 509 | 521 | /** |
| 510 | 522 | * ice_alloc_rx_bufs_zc - allocate a number of Rx buffers |
| 511 | 523 | * @rx_ring: Rx ring |
| 524 | + * @xsk_pool: XSK buffer pool to pick buffers to be filled by HW | |
| 512 | 525 | * @count: The number of buffers to allocate |
| 513 | 526 | * |
| 514 | 527 | * Wrapper for internal allocation routine; figure out how many tail |
| @@ -516,7 +529,8 @@ exit: · 1 unmodified line | ||
| 516 | 529 | * |
| 517 | 530 | * Returns true if all calls to internal alloc routine succeeded |
| 518 | 531 | */ |
| 519 | −bool ice_alloc_rx_bufs_zc(struct ice_rx_ring *rx_ring, u16 count) | |
| 532 | +bool ice_alloc_rx_bufs_zc(struct ice_rx_ring *rx_ring, | |
| 533 | + struct xsk_buff_pool *xsk_pool, u16 count) | |
| 520 | 534 | { |
| 521 | 535 | u16 rx_thresh = ICE_RING_QUARTER(rx_ring); |
| 522 | 536 | u16 leftover, i, tail_bumps; |
| @@ -525,9 +539,9 @@ bool ice_alloc_rx_bufs_zc(struct ice_rx_ring *rx_ring, u16 count) · 2 unmodified lines | ||
| 525 | 539 | leftover = count - (tail_bumps * rx_thresh); |
| 526 | 540 | |
| 527 | 541 | for (i = 0; i < tail_bumps; i++) |
| 528 | − if (!__ice_alloc_rx_bufs_zc(rx_ring, rx_thresh)) | |
| 542 | + if (!__ice_alloc_rx_bufs_zc(rx_ring, xsk_pool, rx_thresh)) | |
| 529 | 543 | return false; |
| 530 | − return __ice_alloc_rx_bufs_zc(rx_ring, leftover); | |
| 544 | + return __ice_alloc_rx_bufs_zc(rx_ring, xsk_pool, leftover); | |
| 531 | 545 | } |
| 532 | 546 | |
| 533 | 547 | /** |
| @@ -596,8 +610,10 @@ out: · 62 unmodified lines | ||
| 596 | 610 | /** |
| 597 | 611 | * ice_clean_xdp_irq_zc - produce AF_XDP descriptors to CQ |
| 598 | 612 | * @xdp_ring: XDP Tx ring |
| 613 | + * @xsk_pool: AF_XDP buffer pool pointer | |
| 599 | 614 | */ |
| 600 | −static u32 ice_clean_xdp_irq_zc(struct ice_tx_ring *xdp_ring) | |
| 615 | +static u32 ice_clean_xdp_irq_zc(struct ice_tx_ring *xdp_ring, | |
| 616 | + struct xsk_buff_pool *xsk_pool) | |
| 601 | 617 | { |
| 602 | 618 | u16 ntc = xdp_ring->next_to_clean; |
| 603 | 619 | struct ice_tx_desc *tx_desc; |
| @@ -648,7 +664,7 @@ skip: · 44 unmodified lines | ||
| 648 | 664 | if (xdp_ring->next_to_clean >= cnt) |
| 649 | 665 | xdp_ring->next_to_clean -= cnt; |
| 650 | 666 | if (xsk_frames) |
| 651 | − xsk_tx_completed(xdp_ring->xsk_pool, xsk_frames); | |
| 667 | + xsk_tx_completed(xsk_pool, xsk_frames); | |
| 652 | 668 | |
| 653 | 669 | return completed_frames; |
| 654 | 670 | } |
| @@ -657,6 +673,7 @@ skip: · 2 unmodified lines | ||
| 657 | 673 | * ice_xmit_xdp_tx_zc - AF_XDP ZC handler for XDP_TX |
| 658 | 674 | * @xdp: XDP buffer to xmit |
| 659 | 675 | * @xdp_ring: XDP ring to produce descriptor onto |
| 676 | + * @xsk_pool: AF_XDP buffer pool pointer | |
| 660 | 677 | * |
| 661 | 678 | * note that this function works directly on xdp_buff, no need to convert |
| 662 | 679 | * it to xdp_frame. xdp_buff pointer is stored to ice_tx_buf so that cleaning |
| @@ -666,7 +683,8 @@ skip: · 3 unmodified lines | ||
| 666 | 683 | * was not enough space on XDP ring |
| 667 | 684 | */ |
| 668 | 685 | static int ice_xmit_xdp_tx_zc(struct xdp_buff *xdp, |
| 669 | − struct ice_tx_ring *xdp_ring) | |
| 686 | + struct ice_tx_ring *xdp_ring, | |
| 687 | + struct xsk_buff_pool *xsk_pool) | |
| 670 | 688 | { |
| 671 | 689 | struct skb_shared_info *sinfo = NULL; |
| 672 | 690 | u32 size = xdp->data_end - xdp->data; |
| @@ -680,7 +698,7 @@ static int ice_xmit_xdp_tx_zc(struct xdp_buff *xdp, · 7 unmodified lines | ||
| 680 | 698 | |
| 681 | 699 | free_space = ICE_DESC_UNUSED(xdp_ring); |
| 682 | 700 | if (free_space < ICE_RING_QUARTER(xdp_ring)) |
| 683 | − free_space += ice_clean_xdp_irq_zc(xdp_ring); | |
| 701 | + free_space += ice_clean_xdp_irq_zc(xdp_ring, xsk_pool); | |
| 684 | 702 | |
| 685 | 703 | if (unlikely(!free_space)) |
| 686 | 704 | goto busy; |
| @@ -700,7 +718,7 @@ static int ice_xmit_xdp_tx_zc(struct xdp_buff *xdp, · 13 unmodified lines | ||
| 700 | 718 | dma_addr_t dma; |
| 701 | 719 | |
| 702 | 720 | dma = xsk_buff_xdp_get_dma(xdp); |
| 703 | − xsk_buff_raw_dma_sync_for_device(xdp_ring->xsk_pool, dma, size); | |
| 721 | + xsk_buff_raw_dma_sync_for_device(xsk_pool, dma, size); | |
| 704 | 722 | |
| 705 | 723 | tx_buf->xdp = xdp; |
| 706 | 724 | tx_buf->type = ICE_TX_BUF_XSK_TX; |
| @@ -742,12 +760,14 @@ busy: · 35 unmodified lines | ||
| 742 | 760 | * @xdp: xdp_buff used as input to the XDP program |
| 743 | 761 | * @xdp_prog: XDP program to run |
| 744 | 762 | * @xdp_ring: ring to be used for XDP_TX action |
| 763 | + * @xsk_pool: AF_XDP buffer pool pointer | |
| 745 | 764 | * |
| 746 | 765 | * Returns any of ICE_XDP_{PASS, CONSUMED, TX, REDIR} |
| 747 | 766 | */ |
| 748 | 767 | static int |
| 749 | 768 | ice_run_xdp_zc(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp, |
| 750 | − struct bpf_prog *xdp_prog, struct ice_tx_ring *xdp_ring) | |
| 769 | + struct bpf_prog *xdp_prog, struct ice_tx_ring *xdp_ring, | |
| 770 | + struct xsk_buff_pool *xsk_pool) | |
| 751 | 771 | { |
| 752 | 772 | int err, result = ICE_XDP_PASS; |
| 753 | 773 | u32 act; |
| @@ -758,7 +778,7 @@ ice_run_xdp_zc(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp, · 4 unmodified lines | ||
| 758 | 778 | err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); |
| 759 | 779 | if (!err) |
| 760 | 780 | return ICE_XDP_REDIR; |
| 761 | − if (xsk_uses_need_wakeup(rx_ring->xsk_pool) && err == -ENOBUFS) | |
| 781 | + if (xsk_uses_need_wakeup(xsk_pool) && err == -ENOBUFS) | |
| 762 | 782 | result = ICE_XDP_EXIT; |
| 763 | 783 | else |
| 764 | 784 | result = ICE_XDP_CONSUMED; |
| @@ -769,7 +789,7 @@ ice_run_xdp_zc(struct ice_rx_ring *rx_ring, struct xdp_buff *xdp, · 4 unmodified lines | ||
| 769 | 789 | case XDP_PASS: |
| 770 | 790 | break; |
| 771 | 791 | case XDP_TX: |
| 772 | − result = ice_xmit_xdp_tx_zc(xdp, xdp_ring); | |
| 792 | + result = ice_xmit_xdp_tx_zc(xdp, xdp_ring, xsk_pool); | |
| 773 | 793 | if (result == ICE_XDP_CONSUMED) |
| 774 | 794 | goto out_failure; |
| 775 | 795 | break; |
| @@ -821,14 +841,16 @@ ice_add_xsk_frag(struct ice_rx_ring *rx_ring, struct xdp_buff *first, · 45 unmodified lines | ||
| 821 | 841 | /** |
| 822 | 842 | * ice_clean_rx_irq_zc - consumes packets from the hardware ring |
| 823 | 843 | * @rx_ring: AF_XDP Rx ring |
| 844 | + * @xsk_pool: AF_XDP buffer pool pointer | |
| 824 | 845 | * @budget: NAPI budget |
| 825 | 846 | * |
| 826 | 847 | * Returns number of processed packets on success, remaining budget on failure. |
| 827 | 848 | */ |
| 828 | −int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget) | |
| 849 | +int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, | |
| 850 | + struct xsk_buff_pool *xsk_pool, | |
| 851 | + int budget) | |
| 829 | 852 | { |
| 830 | 853 | unsigned int total_rx_bytes = 0, total_rx_packets = 0; |
| 831 | − struct xsk_buff_pool *xsk_pool = rx_ring->xsk_pool; | |
| 832 | 854 | u32 ntc = rx_ring->next_to_clean; |
| 833 | 855 | u32 ntu = rx_ring->next_to_use; |
| 834 | 856 | struct xdp_buff *first = NULL; |
| @@ -891,7 +913,8 @@ int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget) · 56 unmodified lines | ||
| 891 | 913 | if (ice_is_non_eop(rx_ring, rx_desc)) |
| 892 | 914 | continue; |
| 893 | 915 | |
| 894 | − xdp_res = ice_run_xdp_zc(rx_ring, first, xdp_prog, xdp_ring); | |
| 916 | + xdp_res = ice_run_xdp_zc(rx_ring, first, xdp_prog, xdp_ring, | |
| 917 | + xsk_pool); | |
| 895 | 918 | if (likely(xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR))) { |
| 896 | 919 | xdp_xmit |= xdp_res; |
| 897 | 920 | } else if (xdp_res == ICE_XDP_EXIT) { |
| @@ -940,7 +963,8 @@ construct_skb: · 42 unmodified lines | ||
| 940 | 963 | rx_ring->next_to_clean = ntc; |
| 941 | 964 | entries_to_alloc = ICE_RX_DESC_UNUSED(rx_ring); |
| 942 | 965 | if (entries_to_alloc > ICE_RING_QUARTER(rx_ring)) |
| 943 | − failure |= !ice_alloc_rx_bufs_zc(rx_ring, entries_to_alloc); | |
| 966 | + failure |= !ice_alloc_rx_bufs_zc(rx_ring, xsk_pool, | |
| 967 | + entries_to_alloc); | |
| 944 | 968 | |
| 945 | 969 | ice_finalize_xdp_rx(xdp_ring, xdp_xmit, 0); |
| 946 | 970 | ice_update_rx_ring_stats(rx_ring, total_rx_packets, total_rx_bytes); |
| @@ -963,17 +987,19 @@ construct_skb: · 16 unmodified lines | ||
| 963 | 987 | /** |
| 964 | 988 | * ice_xmit_pkt - produce a single HW Tx descriptor out of AF_XDP descriptor |
| 965 | 989 | * @xdp_ring: XDP ring to produce the HW Tx descriptor on |
| 990 | + * @xsk_pool: XSK buffer pool to pick buffers to be consumed by HW | |
| 966 | 991 | * @desc: AF_XDP descriptor to pull the DMA address and length from |
| 967 | 992 | * @total_bytes: bytes accumulator that will be used for stats update |
| 968 | 993 | */ |
| 969 | −static void ice_xmit_pkt(struct ice_tx_ring *xdp_ring, struct xdp_desc *desc, | |
| 994 | +static void ice_xmit_pkt(struct ice_tx_ring *xdp_ring, | |
| 995 | + struct xsk_buff_pool *xsk_pool, struct xdp_desc *desc, | |
| 970 | 996 | unsigned int *total_bytes) |
| 971 | 997 | { |
| 972 | 998 | struct ice_tx_desc *tx_desc; |
| 973 | 999 | dma_addr_t dma; |
| 974 | 1000 | |
| 975 | − dma = xsk_buff_raw_get_dma(xdp_ring->xsk_pool, desc->addr); | |
| 976 | − xsk_buff_raw_dma_sync_for_device(xdp_ring->xsk_pool, dma, desc->len); | |
| 1001 | + dma = xsk_buff_raw_get_dma(xsk_pool, desc->addr); | |
| 1002 | + xsk_buff_raw_dma_sync_for_device(xsk_pool, dma, desc->len); | |
| 977 | 1003 | |
| 978 | 1004 | tx_desc = ICE_TX_DESC(xdp_ring, xdp_ring->next_to_use++); |
| 979 | 1005 | tx_desc->buf_addr = cpu_to_le64(dma); |
| @@ -986,10 +1012,13 @@ static void ice_xmit_pkt(struct ice_tx_ring *xdp_ring, struct xdp_desc *desc, · 6 unmodified lines | ||
| 986 | 1012 | /** |
| 987 | 1013 | * ice_xmit_pkt_batch - produce a batch of HW Tx descriptors out of AF_XDP descriptors |
| 988 | 1014 | * @xdp_ring: XDP ring to produce the HW Tx descriptors on |
| 1015 | + * @xsk_pool: XSK buffer pool to pick buffers to be consumed by HW | |
| 989 | 1016 | * @descs: AF_XDP descriptors to pull the DMA addresses and lengths from |
| 990 | 1017 | * @total_bytes: bytes accumulator that will be used for stats update |
| 991 | 1018 | */ |
| 992 | −static void ice_xmit_pkt_batch(struct ice_tx_ring *xdp_ring, struct xdp_desc *descs, | |
| 1019 | +static void ice_xmit_pkt_batch(struct ice_tx_ring *xdp_ring, | |
| 1020 | + struct xsk_buff_pool *xsk_pool, | |
| 1021 | + struct xdp_desc *descs, | |
| 993 | 1022 | unsigned int *total_bytes) |
| 994 | 1023 | { |
| 995 | 1024 | u16 ntu = xdp_ring->next_to_use; |
| @@ -999,8 +1028,8 @@ static void ice_xmit_pkt_batch(struct ice_tx_ring *xdp_ring, struct xdp_desc *de · 3 unmodified lines | ||
| 999 | 1028 | loop_unrolled_for(i = 0; i < PKTS_PER_BATCH; i++) { |
| 1000 | 1029 | dma_addr_t dma; |
| 1001 | 1030 | |
| 1002 | − dma = xsk_buff_raw_get_dma(xdp_ring->xsk_pool, descs[i].addr); | |
| 1003 | − xsk_buff_raw_dma_sync_for_device(xdp_ring->xsk_pool, dma, descs[i].len); | |
| 1031 | + dma = xsk_buff_raw_get_dma(xsk_pool, descs[i].addr); | |
| 1032 | + xsk_buff_raw_dma_sync_for_device(xsk_pool, dma, descs[i].len); | |
| 1004 | 1033 | |
| 1005 | 1034 | tx_desc = ICE_TX_DESC(xdp_ring, ntu++); |
| 1006 | 1035 | tx_desc->buf_addr = cpu_to_le64(dma); |
| @@ -1016,60 +1045,69 @@ static void ice_xmit_pkt_batch(struct ice_tx_ring *xdp_ring, struct xdp_desc *de · 9 unmodified lines | ||
| 1016 | 1045 | /** |
| 1017 | 1046 | * ice_fill_tx_hw_ring - produce the number of Tx descriptors onto ring |
| 1018 | 1047 | * @xdp_ring: XDP ring to produce the HW Tx descriptors on |
| 1048 | + * @xsk_pool: XSK buffer pool to pick buffers to be consumed by HW | |
| 1019 | 1049 | * @descs: AF_XDP descriptors to pull the DMA addresses and lengths from |
| 1020 | 1050 | * @nb_pkts: count of packets to be send |
| 1021 | 1051 | * @total_bytes: bytes accumulator that will be used for stats update |
| 1022 | 1052 | */ |
| 1023 | −static void ice_fill_tx_hw_ring(struct ice_tx_ring *xdp_ring, struct xdp_desc *descs, | |
| 1024 | − u32 nb_pkts, unsigned int *total_bytes) | |
| 1053 | +static void ice_fill_tx_hw_ring(struct ice_tx_ring *xdp_ring, | |
| 1054 | + struct xsk_buff_pool *xsk_pool, | |
| 1055 | + struct xdp_desc *descs, u32 nb_pkts, | |
| 1056 | + unsigned int *total_bytes) | |
| 1025 | 1057 | { |
| 1026 | 1058 | u32 batched, leftover, i; |
| 1027 | 1059 | |
| 1028 | 1060 | batched = ALIGN_DOWN(nb_pkts, PKTS_PER_BATCH); |
| 1029 | 1061 | leftover = nb_pkts & (PKTS_PER_BATCH - 1); |
| 1030 | 1062 | for (i = 0; i < batched; i += PKTS_PER_BATCH) |
| 1031 | − ice_xmit_pkt_batch(xdp_ring, &descs[i], total_bytes); | |
| 1063 | + ice_xmit_pkt_batch(xdp_ring, xsk_pool, &descs[i], total_bytes); | |
| 1032 | 1064 | for (; i < batched + leftover; i++) |
| 1033 | − ice_xmit_pkt(xdp_ring, &descs[i], total_bytes); | |
| 1065 | + ice_xmit_pkt(xdp_ring, xsk_pool, &descs[i], total_bytes); | |
| 1034 | 1066 | } |
| 1035 | 1067 | |
| 1036 | 1068 | /** |
| 1037 | 1069 | * ice_xmit_zc - take entries from XSK Tx ring and place them onto HW Tx ring |
| 1038 | 1070 | * @xdp_ring: XDP ring to produce the HW Tx descriptors on |
| 1071 | + * @xsk_pool: AF_XDP buffer pool pointer | |
| 1039 | 1072 | * |
| 1040 | 1073 | * Returns true if there is no more work that needs to be done, false otherwise |
| 1041 | 1074 | */ |
| 1042 | −bool ice_xmit_zc(struct ice_tx_ring *xdp_ring) | |
| 1075 | +bool ice_xmit_zc(struct ice_tx_ring *xdp_ring, struct xsk_buff_pool *xsk_pool) | |
| 1043 | 1076 | { |
| 1044 | − struct xdp_desc *descs = xdp_ring->xsk_pool->tx_descs; | |
| 1077 | + struct xdp_desc *descs = xsk_pool->tx_descs; | |
| 1045 | 1078 | u32 nb_pkts, nb_processed = 0; |
| 1046 | 1079 | unsigned int total_bytes = 0; |
| 1047 | 1080 | int budget; |
| 1048 | 1081 | |
| 1049 | − ice_clean_xdp_irq_zc(xdp_ring); | |
| 1082 | + ice_clean_xdp_irq_zc(xdp_ring, xsk_pool); | |
| 1083 | + | |
| 1084 | + if (!netif_carrier_ok(xdp_ring->vsi->netdev) || | |
| 1085 | + !netif_running(xdp_ring->vsi->netdev)) | |
| 1086 | + return true; | |
| 1050 | 1087 | |
| 1051 | 1088 | budget = ICE_DESC_UNUSED(xdp_ring); |
| 1052 | 1089 | budget = min_t(u16, budget, ICE_RING_QUARTER(xdp_ring)); |
| 1053 | 1090 | |
| 1054 | − nb_pkts = xsk_tx_peek_release_desc_batch(xdp_ring->xsk_pool, budget); | |
| 1091 | + nb_pkts = xsk_tx_peek_release_desc_batch(xsk_pool, budget); | |
| 1055 | 1092 | if (!nb_pkts) |
| 1056 | 1093 | return true; |
| 1057 | 1094 | |
| 1058 | 1095 | if (xdp_ring->next_to_use + nb_pkts >= xdp_ring->count) { |
| 1059 | 1096 | nb_processed = xdp_ring->count - xdp_ring->next_to_use; |
| 1060 | − ice_fill_tx_hw_ring(xdp_ring, descs, nb_processed, &total_bytes); | |
| 1097 | + ice_fill_tx_hw_ring(xdp_ring, xsk_pool, descs, nb_processed, | |
| 1098 | + &total_bytes); | |
| 1061 | 1099 | xdp_ring->next_to_use = 0; |
| 1062 | 1100 | } |
| 1063 | 1101 | |
| 1064 | − ice_fill_tx_hw_ring(xdp_ring, &descs[nb_processed], nb_pkts - nb_processed, | |
| 1065 | − &total_bytes); | |
| 1102 | + ice_fill_tx_hw_ring(xdp_ring, xsk_pool, &descs[nb_processed], | |
| 1103 | + nb_pkts - nb_processed, &total_bytes); | |
| 1066 | 1104 | |
| 1067 | 1105 | ice_set_rs_bit(xdp_ring); |
| 1068 | 1106 | ice_xdp_ring_update_tail(xdp_ring); |
| 1069 | 1107 | ice_update_tx_ring_stats(xdp_ring, nb_pkts, total_bytes); |
| 1070 | 1108 | |
| 1071 | − if (xsk_uses_need_wakeup(xdp_ring->xsk_pool)) | |
| 1072 | − xsk_set_tx_need_wakeup(xdp_ring->xsk_pool); | |
| 1109 | + if (xsk_uses_need_wakeup(xsk_pool)) | |
| 1110 | + xsk_set_tx_need_wakeup(xsk_pool); | |
| 1073 | 1111 | |
| 1074 | 1112 | return nb_pkts < budget; |
| 1075 | 1113 | } |
| @@ -1091,7 +1129,7 @@ ice_xsk_wakeup(struct net_device *netdev, u32 queue_id, · 15 unmodified lines | ||
| 1091 | 1129 | struct ice_vsi *vsi = np->vsi; |
| 1092 | 1130 | struct ice_tx_ring *ring; |
| 1093 | 1131 | |
| 1094 | − if (test_bit(ICE_VSI_DOWN, vsi->state)) | |
| 1132 | + if (test_bit(ICE_VSI_DOWN, vsi->state) || !netif_carrier_ok(netdev)) | |
| 1095 | 1133 | return -ENETDOWN; |
| 1096 | 1134 | |
| 1097 | 1135 | if (!ice_is_xdp_ena_vsi(vsi)) |
| @@ -1102,7 +1140,7 @@ ice_xsk_wakeup(struct net_device *netdev, u32 queue_id, · 4 unmodified lines | ||
| 1102 | 1140 | |
| 1103 | 1141 | ring = vsi->rx_rings[queue_id]->xdp_ring; |
| 1104 | 1142 | |
| 1105 | − if (!ring->xsk_pool) | |
| 1143 | + if (!READ_ONCE(ring->xsk_pool)) | |
| 1106 | 1144 | return -EINVAL; |
| 1107 | 1145 | |
| 1108 | 1146 | /* The idea here is that if NAPI is running, mark a miss, so |
| @@ -20,16 +20,20 @@ struct ice_vsi; · 19 unmodified lines | ||
| 20 | 20 | #ifdef CONFIG_XDP_SOCKETS |
| 21 | 21 | int ice_xsk_pool_setup(struct ice_vsi *vsi, struct xsk_buff_pool *pool, |
| 22 | 22 | u16 qid); |
| 23 | −int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, int budget); | |
| 23 | +int ice_clean_rx_irq_zc(struct ice_rx_ring *rx_ring, | |
| 24 | + struct xsk_buff_pool *xsk_pool, | |
| 25 | + int budget); | |
| 24 | 26 | int ice_xsk_wakeup(struct net_device *netdev, u32 queue_id, u32 flags); |
| 25 | −bool ice_alloc_rx_bufs_zc(struct ice_rx_ring *rx_ring, u16 count); | |
| 27 | +bool ice_alloc_rx_bufs_zc(struct ice_rx_ring *rx_ring, | |
| 28 | + struct xsk_buff_pool *xsk_pool, u16 count); | |
| 26 | 29 | bool ice_xsk_any_rx_ring_ena(struct ice_vsi *vsi); |
| 27 | 30 | void ice_xsk_clean_rx_ring(struct ice_rx_ring *rx_ring); |
| 28 | 31 | void ice_xsk_clean_xdp_ring(struct ice_tx_ring *xdp_ring); |
| 29 | −bool ice_xmit_zc(struct ice_tx_ring *xdp_ring); | |
| 32 | +bool ice_xmit_zc(struct ice_tx_ring *xdp_ring, struct xsk_buff_pool *xsk_pool); | |
| 30 | 33 | int ice_realloc_zc_buf(struct ice_vsi *vsi, bool zc); |
| 31 | 34 | #else |
| 32 | −static inline bool ice_xmit_zc(struct ice_tx_ring __always_unused *xdp_ring) | |
| 35 | +static inline bool ice_xmit_zc(struct ice_tx_ring __always_unused *xdp_ring, | |
| 36 | + struct xsk_buff_pool __always_unused *xsk_pool) | |
| 33 | 37 | { |
| 34 | 38 | return false; |
| 35 | 39 | } |
| @@ -44,6 +48,7 @@ ice_xsk_pool_setup(struct ice_vsi __always_unused *vsi, · 8 unmodified lines | ||
| 44 | 48 | |
| 45 | 49 | static inline int |
| 46 | 50 | ice_clean_rx_irq_zc(struct ice_rx_ring __always_unused *rx_ring, |
| 51 | + struct xsk_buff_pool __always_unused *xsk_pool, | |
| 47 | 52 | int __always_unused budget) |
10 more lines aren’t shown here.
This diff is large: its 144 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 386 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 106 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 322 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 338 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 286 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 252 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 249 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 141 lines aren’t shown here.
This diff is large: its 140 lines aren’t shown here.
This diff is large: its 219 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 315 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 365 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 219 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 153 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 193 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 186 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 254 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 151 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 223 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 152 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 136 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 312 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 129 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 412 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 163 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 157 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 233 lines aren’t shown here.
This diff is large: its 176 lines aren’t shown here.
This diff is large: its 677 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 320 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 292 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 212 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 358 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 126 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 910 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 179 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 584 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 665 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 341 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.