Backmerge to get dfce90259d74 ("Backmerge i915 security patches from commit 'ea0b163b13ff' into drm-next") and thus 100d46bd72ec ("Merge Intel Gen8/Gen9 graphics fixes from Jon Bloomfield.").
Unified · Split
| @@ -108,6 +108,10 @@ Jason Gunthorpe <jgg@ziepe.ca> <jgg@mellanox.com> · 107 unmodified lines | ||
| 108 | 108 | Jason Gunthorpe <jgg@ziepe.ca> <jgunthorpe@obsidianresearch.com> |
| 109 | 109 | Javi Merino <javi.merino@kernel.org> <javi.merino@arm.com> |
| 110 | 110 | <javier@osg.samsung.com> <javier.martinez@collabora.co.uk> |
| 111 | +Jayachandran C <c.jayachandran@gmail.com> <jayachandranc@netlogicmicro.com> | |
| 112 | +Jayachandran C <c.jayachandran@gmail.com> <jchandra@broadcom.com> | |
| 113 | +Jayachandran C <c.jayachandran@gmail.com> <jchandra@digeo.com> | |
| 114 | +Jayachandran C <c.jayachandran@gmail.com> <jnair@caviumnetworks.com> | |
| 111 | 115 | Jean Tourrilhes <jt@hpl.hp.com> |
| 112 | 116 | <jean-philippe@linaro.org> <jean-philippe.brucker@arm.com> |
| 113 | 117 | Jeff Garzik <jgarzik@pretzel.yyz.us> |
| @@ -196,7 +200,8 @@ Oleksij Rempel <linux@rempel-privat.de> <o.rempel@pengutronix.de> · 82 unmodified lines | ||
| 196 | 200 | Oleksij Rempel <linux@rempel-privat.de> <ore@pengutronix.de> |
| 197 | 201 | Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it> |
| 198 | 202 | Patrick Mochel <mochel@digitalimplant.org> |
| 199 | −Paul Burton <paul.burton@mips.com> <paul.burton@imgtec.com> | |
| 203 | +Paul Burton <paulburton@kernel.org> <paul.burton@imgtec.com> | |
| 204 | +Paul Burton <paulburton@kernel.org> <paul.burton@mips.com> | |
| 200 | 205 | Peter A Jonsson <pj@ludd.ltu.se> |
| 201 | 206 | Peter Oruba <peter@oruba.de> |
| 202 | 207 | Peter Oruba <peter.oruba@amd.com> |
| @@ -229,6 +234,7 @@ Shuah Khan <shuah@kernel.org> <shuahkhan@gmail.com> · 26 unmodified lines | ||
| 229 | 234 | Shuah Khan <shuah@kernel.org> <shuah.khan@hp.com> |
| 230 | 235 | Shuah Khan <shuah@kernel.org> <shuahkh@osg.samsung.com> |
| 231 | 236 | Shuah Khan <shuah@kernel.org> <shuah.kh@samsung.com> |
| 237 | +Simon Arlott <simon@octiron.net> <simon@fire.lp0.eu> | |
| 232 | 238 | Simon Kelley <simon@thekelleys.org.uk> |
| 233 | 239 | Stéphane Witzmann <stephane.witzmann@ubpmes.univ-bpclermont.fr> |
| 234 | 240 | Stephen Hemminger <shemminger@osdl.org> |
| @@ -1637,6 +1637,10 @@ S: Panoramastrasse 18 · 1636 unmodified lines | ||
| 1637 | 1637 | S: D-69126 Heidelberg |
| 1638 | 1638 | S: Germany |
| 1639 | 1639 | |
| 1640 | +N: Simon Horman | |
| 1641 | +M: horms@verge.net.au | |
| 1642 | +D: Renesas ARM/ARM64 SoC maintainer | |
| 1643 | + | |
| 1640 | 1644 | N: Christopher Horn |
| 1641 | 1645 | E: chorn@warwick.net |
| 1642 | 1646 | D: Miscellaneous sysctl hacks |
| @@ -615,8 +615,8 @@ on an IO device and is an example of this type. · 614 unmodified lines | ||
| 615 | 615 | Protections |
| 616 | 616 | ----------- |
| 617 | 617 | |
| 618 | −A cgroup is protected to be allocated upto the configured amount of | |
| 619 | −the resource if the usages of all its ancestors are under their | |
| 618 | +A cgroup is protected upto the configured amount of the resource | |
| 619 | +as long as the usages of all its ancestors are under their | |
| 620 | 620 | protected levels. Protections can be hard guarantees or best effort |
| 621 | 621 | soft boundaries. Protections can also be over-committed in which case |
| 622 | 622 | only upto the amount available to the parent is protected among |
| @@ -1096,7 +1096,10 @@ PAGE_SIZE multiple when read back. · 473 unmodified lines | ||
| 1096 | 1096 | is within its effective min boundary, the cgroup's memory |
| 1097 | 1097 | won't be reclaimed under any conditions. If there is no |
| 1098 | 1098 | unprotected reclaimable memory available, OOM killer |
| 1099 | − is invoked. | |
| 1099 | + is invoked. Above the effective min boundary (or | |
| 1100 | + effective low boundary if it is higher), pages are reclaimed | |
| 1101 | + proportionally to the overage, reducing reclaim pressure for | |
| 1102 | + smaller overages. | |
| 1100 | 1103 | |
| 1101 | 1104 | Effective min boundary is limited by memory.min values of |
| 1102 | 1105 | all ancestor cgroups. If there is memory.min overcommitment |
| @@ -1118,7 +1121,10 @@ PAGE_SIZE multiple when read back. · 15 unmodified lines | ||
| 1118 | 1121 | Best-effort memory protection. If the memory usage of a |
| 1119 | 1122 | cgroup is within its effective low boundary, the cgroup's |
| 1120 | 1123 | memory won't be reclaimed unless memory can be reclaimed |
| 1121 | − from unprotected cgroups. | |
| 1124 | + from unprotected cgroups. Above the effective low boundary (or | |
| 1125 | + effective min boundary if it is higher), pages are reclaimed | |
| 1126 | + proportionally to the overage, reducing reclaim pressure for | |
| 1127 | + smaller overages. | |
| 1122 | 1128 | |
| 1123 | 1129 | Effective low boundary is limited by memory.low values of |
| 1124 | 1130 | all ancestor cgroups. If there is memory.low overcommitment |
| @@ -2482,8 +2488,10 @@ system performance due to overreclaim, to the point where the feature · 1357 unmodified lines | ||
| 2482 | 2488 | becomes self-defeating. |
| 2483 | 2489 | |
| 2484 | 2490 | The memory.low boundary on the other hand is a top-down allocated |
| 2485 | −reserve. A cgroup enjoys reclaim protection when it's within its low, | |
| 2486 | −which makes delegation of subtrees possible. | |
| 2491 | +reserve. A cgroup enjoys reclaim protection when it's within its | |
| 2492 | +effective low, which makes delegation of subtrees possible. It also | |
| 2493 | +enjoys having reclaim pressure proportional to its overage when | |
| 2494 | +above its effective low. | |
| 2487 | 2495 | |
| 2488 | 2496 | The original high boundary, the hard limit, is defined as a strict |
| 2489 | 2497 | limit that can not budge, even if the OOM killer has to be called. |
| @@ -5302,6 +5302,10 @@ · 5301 unmodified lines | ||
| 5302 | 5302 | the unplug protocol |
| 5303 | 5303 | never -- do not unplug even if version check succeeds |
| 5304 | 5304 | |
| 5305 | + xen_legacy_crash [X86,XEN] | |
| 5306 | + Crash from Xen panic notifier, without executing late | |
| 5307 | + panic() code such as dumping handler. | |
| 5308 | + | |
| 5305 | 5309 | xen_nopvspin [X86,XEN] |
| 5306 | 5310 | Disables the ticketlock slowpath using Xen PV |
| 5307 | 5311 | optimizations. |
| @@ -154,11 +154,18 @@ return virtual addresses to userspace from a 48-bit range. · 153 unmodified lines | ||
| 154 | 154 | |
| 155 | 155 | Software can "opt-in" to receiving VAs from a 52-bit space by |
| 156 | 156 | specifying an mmap hint parameter that is larger than 48-bit. |
| 157 | + | |
| 157 | 158 | For example: |
| 158 | − maybe_high_address = mmap(~0UL, size, prot, flags,...); | |
| 159 | + | |
| 160 | +.. code-block:: c | |
| 161 | + | |
| 162 | + maybe_high_address = mmap(~0UL, size, prot, flags,...); | |
| 159 | 163 | |
| 160 | 164 | It is also possible to build a debug kernel that returns addresses |
| 161 | 165 | from a 52-bit space by enabling the following kernel config options: |
| 166 | + | |
| 167 | +.. code-block:: sh | |
| 168 | + | |
| 162 | 169 | CONFIG_EXPERT=y && CONFIG_ARM64_FORCE_52BIT=y |
| 163 | 170 | |
| 164 | 171 | Note that this option is only intended for debugging applications |
| @@ -91,6 +91,11 @@ stable kernels. · 90 unmodified lines | ||
| 91 | 91 | | ARM | MMU-500 | #841119,826419 | N/A | |
| 92 | 92 | +----------------+-----------------+-----------------+-----------------------------+ |
| 93 | 93 | +----------------+-----------------+-----------------+-----------------------------+ |
| 94 | +| Broadcom | Brahma-B53 | N/A | ARM64_ERRATUM_845719 | | |
| 95 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 96 | +| Broadcom | Brahma-B53 | N/A | ARM64_ERRATUM_843419 | | |
| 97 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 98 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 94 | 99 | | Cavium | ThunderX ITS | #22375,24313 | CAVIUM_ERRATUM_22375 | |
| 95 | 100 | +----------------+-----------------+-----------------+-----------------------------+ |
| 96 | 101 | | Cavium | ThunderX ITS | #23144 | CAVIUM_ERRATUM_23144 | |
| @@ -107,6 +112,8 @@ stable kernels. · 10 unmodified lines | ||
| 107 | 112 | +----------------+-----------------+-----------------+-----------------------------+ |
| 108 | 113 | | Cavium | ThunderX2 SMMUv3| #126 | N/A | |
| 109 | 114 | +----------------+-----------------+-----------------+-----------------------------+ |
| 115 | +| Cavium | ThunderX2 Core | #219 | CAVIUM_TX2_ERRATUM_219 | | |
| 116 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 110 | 117 | +----------------+-----------------+-----------------+-----------------------------+ |
| 111 | 118 | | Freescale/NXP | LS2080A/LS1043A | A-008585 | FSL_ERRATUM_A008585 | |
| 112 | 119 | +----------------+-----------------+-----------------+-----------------------------+ |
| @@ -124,7 +131,7 @@ stable kernels. · 11 unmodified lines | ||
| 124 | 131 | +----------------+-----------------+-----------------+-----------------------------+ |
| 125 | 132 | | Qualcomm Tech. | Kryo/Falkor v1 | E1003 | QCOM_FALKOR_ERRATUM_1003 | |
| 126 | 133 | +----------------+-----------------+-----------------+-----------------------------+ |
| 127 | −| Qualcomm Tech. | Falkor v1 | E1009 | QCOM_FALKOR_ERRATUM_1009 | | |
| 134 | +| Qualcomm Tech. | Kryo/Falkor v1 | E1009 | QCOM_FALKOR_ERRATUM_1009 | | |
| 128 | 135 | +----------------+-----------------+-----------------+-----------------------------+ |
| 129 | 136 | | Qualcomm Tech. | QDF2400 ITS | E0065 | QCOM_QDF2400_ERRATUM_0065 | |
| 130 | 137 | +----------------+-----------------+-----------------+-----------------------------+ |
| @@ -38,6 +38,7 @@ Core utilities · 37 unmodified lines | ||
| 38 | 38 | protection-keys |
| 39 | 39 | ../RCU/index |
| 40 | 40 | gcc-plugins |
| 41 | + symbol-namespaces | |
| 41 | 42 | |
| 42 | 43 | |
| 43 | 44 | Interfaces for kernel debugging |
| @@ -98,6 +98,10 @@ limited. The actual limit depends on the hardware and the kernel · 97 unmodified lines | ||
| 98 | 98 | configuration, but it is a good practice to use `kmalloc` for objects |
| 99 | 99 | smaller than page size. |
| 100 | 100 | |
| 101 | +The address of a chunk allocated with `kmalloc` is aligned to at least | |
| 102 | +ARCH_KMALLOC_MINALIGN bytes. For sizes which are a power of two, the | |
| 103 | +alignment is also guaranteed to be at least the respective size. | |
| 104 | + | |
| 101 | 105 | For large allocations you can use :c:func:`vmalloc` and |
| 102 | 106 | :c:func:`vzalloc`, or directly request pages from the page |
| 103 | 107 | allocator. The memory allocated by `vmalloc` and related functions is |
File renamed without changes.
| @@ -41,6 +41,9 @@ smaller binary while the latter is 1.1 - 2 times faster. · 40 unmodified lines | ||
| 41 | 41 | Both KASAN modes work with both SLUB and SLAB memory allocators. |
| 42 | 42 | For better bug detection and nicer reporting, enable CONFIG_STACKTRACE. |
| 43 | 43 | |
| 44 | +To augment reports with last allocation and freeing stack of the physical page, | |
| 45 | +it is recommended to enable also CONFIG_PAGE_OWNER and boot with page_owner=on. | |
| 46 | + | |
| 44 | 47 | To disable instrumentation for specific files or directories, add a line |
| 45 | 48 | similar to the following to the respective kernel Makefile: |
| 46 | 49 | |
| @@ -89,6 +89,22 @@ To build, save output files in a separate directory with KBUILD_OUTPUT :: · 88 unmodified lines | ||
| 89 | 89 | |
| 90 | 90 | $ export KBUILD_OUTPUT=/tmp/kselftest; make TARGETS="size timers" kselftest |
| 91 | 91 | |
| 92 | +Additionally you can use the "SKIP_TARGETS" variable on the make command | |
| 93 | +line to specify one or more targets to exclude from the TARGETS list. | |
| 94 | + | |
| 95 | +To run all tests but a single subsystem:: | |
| 96 | + | |
| 97 | + $ make -C tools/testing/selftests SKIP_TARGETS=ptrace run_tests | |
| 98 | + | |
| 99 | +You can specify multiple tests to skip:: | |
| 100 | + | |
| 101 | + $ make SKIP_TARGETS="size timers" kselftest | |
| 102 | + | |
| 103 | +You can also specify a restricted list of tests to run together with a | |
| 104 | +dedicated skiplist:: | |
| 105 | + | |
| 106 | + $ make TARGETS="bpf breakpoints size timers" SKIP_TARGETS=bpf kselftest | |
| 107 | + | |
| 92 | 108 | See the top-level tools/testing/selftests/Makefile for the list of all |
| 93 | 109 | possible targets. |
| 94 | 110 | |
| @@ -496,12 +496,12 @@ properties: · 495 unmodified lines | ||
| 496 | 496 | |
| 497 | 497 | - description: Theobroma Systems RK3368-uQ7 with Haikou baseboard |
| 498 | 498 | items: |
| 499 | − - const: tsd,rk3368-uq7-haikou | |
| 499 | + - const: tsd,rk3368-lion-haikou | |
| 500 | 500 | - const: rockchip,rk3368 |
| 501 | 501 | |
| 502 | 502 | - description: Theobroma Systems RK3399-Q7 with Haikou baseboard |
| 503 | 503 | items: |
| 504 | − - const: tsd,rk3399-q7-haikou | |
| 504 | + - const: tsd,rk3399-puma-haikou | |
| 505 | 505 | - const: rockchip,rk3399 |
| 506 | 506 | |
| 507 | 507 | - description: Tronsmart Orion R68 Meta |
| @@ -37,6 +37,8 @@ Optional properties: · 36 unmodified lines | ||
| 37 | 37 | Documentation/devicetree/bindings/reserved-memory/reserved-memory.txt) |
| 38 | 38 | to be used for the framebuffer; if not present, the framebuffer may |
| 39 | 39 | be located anywhere in memory. |
| 40 | + - arm,malidp-arqos-high-level: integer of u32 value describing the ARQoS | |
| 41 | + levels of DP500's QoS signaling. | |
| 40 | 42 | |
| 41 | 43 | |
| 42 | 44 | Example: |
| @@ -54,6 +56,7 @@ Example: · 11 unmodified lines | ||
| 54 | 56 | clocks = <&oscclk2>, <&fpgaosc0>, <&fpgaosc1>, <&fpgaosc1>; |
| 55 | 57 | clock-names = "pxlclk", "mclk", "aclk", "pclk"; |
| 56 | 58 | arm,malidp-output-port-lines = /bits/ 8 <8 8 8>; |
| 59 | + arm,malidp-arqos-high-level = <0xd000d000>; | |
| 57 | 60 | port { |
| 58 | 61 | dp0_output: endpoint { |
| 59 | 62 | remote-endpoint = <&tda998x_2_input>; |
| @@ -13,6 +13,7 @@ Required properties: · 12 unmodified lines | ||
| 13 | 13 | |
| 14 | 14 | - compatible : Shall contain one or more of |
| 15 | 15 | - "renesas,r8a774a1-hdmi" for R8A774A1 (RZ/G2M) compatible HDMI TX |
| 16 | + - "renesas,r8a774b1-hdmi" for R8A774B1 (RZ/G2N) compatible HDMI TX | |
| 16 | 17 | - "renesas,r8a7795-hdmi" for R8A7795 (R-Car H3) compatible HDMI TX |
| 17 | 18 | - "renesas,r8a7796-hdmi" for R8A7796 (R-Car M3-W) compatible HDMI TX |
| 18 | 19 | - "renesas,r8a77965-hdmi" for R8A77965 (R-Car M3-N) compatible HDMI TX |
| @@ -10,6 +10,7 @@ Required properties: · 9 unmodified lines | ||
| 10 | 10 | - "renesas,r8a7743-lvds" for R8A7743 (RZ/G1M) compatible LVDS encoders |
| 11 | 11 | - "renesas,r8a7744-lvds" for R8A7744 (RZ/G1N) compatible LVDS encoders |
| 12 | 12 | - "renesas,r8a774a1-lvds" for R8A774A1 (RZ/G2M) compatible LVDS encoders |
| 13 | + - "renesas,r8a774b1-lvds" for R8A774B1 (RZ/G2N) compatible LVDS encoders | |
| 13 | 14 | - "renesas,r8a774c0-lvds" for R8A774C0 (RZ/G2E) compatible LVDS encoders |
| 14 | 15 | - "renesas,r8a7790-lvds" for R8A7790 (R-Car H2) compatible LVDS encoders |
| 15 | 16 | - "renesas,r8a7791-lvds" for R8A7791 (R-Car M2-W) compatible LVDS encoders |
| @@ -27,19 +27,22 @@ Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.txt. · 26 unmodified lines | ||
| 27 | 27 | |
| 28 | 28 | Required properties (all function blocks): |
| 29 | 29 | - compatible: "mediatek,<chip>-disp-<function>", one of |
| 30 | − "mediatek,<chip>-disp-ovl" - overlay (4 layers, blending, csc) | |
| 31 | − "mediatek,<chip>-disp-rdma" - read DMA / line buffer | |
| 32 | − "mediatek,<chip>-disp-wdma" - write DMA | |
| 33 | − "mediatek,<chip>-disp-color" - color processor | |
| 34 | − "mediatek,<chip>-disp-aal" - adaptive ambient light controller | |
| 35 | − "mediatek,<chip>-disp-gamma" - gamma correction | |
| 36 | − "mediatek,<chip>-disp-merge" - merge streams from two RDMA sources | |
| 37 | − "mediatek,<chip>-disp-split" - split stream to two encoders | |
| 38 | − "mediatek,<chip>-disp-ufoe" - data compression engine | |
| 39 | − "mediatek,<chip>-dsi" - DSI controller, see mediatek,dsi.txt | |
| 40 | − "mediatek,<chip>-dpi" - DPI controller, see mediatek,dpi.txt | |
| 41 | − "mediatek,<chip>-disp-mutex" - display mutex | |
| 42 | − "mediatek,<chip>-disp-od" - overdrive | |
| 30 | + "mediatek,<chip>-disp-ovl" - overlay (4 layers, blending, csc) | |
| 31 | + "mediatek,<chip>-disp-ovl-2l" - overlay (2 layers, blending, csc) | |
| 32 | + "mediatek,<chip>-disp-rdma" - read DMA / line buffer | |
| 33 | + "mediatek,<chip>-disp-wdma" - write DMA | |
| 34 | + "mediatek,<chip>-disp-ccorr" - color correction | |
| 35 | + "mediatek,<chip>-disp-color" - color processor | |
| 36 | + "mediatek,<chip>-disp-dither" - dither | |
| 37 | + "mediatek,<chip>-disp-aal" - adaptive ambient light controller | |
| 38 | + "mediatek,<chip>-disp-gamma" - gamma correction | |
| 39 | + "mediatek,<chip>-disp-merge" - merge streams from two RDMA sources | |
| 40 | + "mediatek,<chip>-disp-split" - split stream to two encoders | |
| 41 | + "mediatek,<chip>-disp-ufoe" - data compression engine | |
| 42 | + "mediatek,<chip>-dsi" - DSI controller, see mediatek,dsi.txt | |
| 43 | + "mediatek,<chip>-dpi" - DPI controller, see mediatek,dpi.txt | |
| 44 | + "mediatek,<chip>-disp-mutex" - display mutex | |
| 45 | + "mediatek,<chip>-disp-od" - overdrive | |
| 43 | 46 | the supported chips are mt2701, mt2712 and mt8173. |
| 44 | 47 | - reg: Physical base address and length of the function block register space |
| 45 | 48 | - interrupts: The interrupt signal from the function block (required, except for |
| @@ -49,6 +52,7 @@ Required properties (all function blocks): · 3 unmodified lines | ||
| 49 | 52 | For most function blocks this is just a single clock input. Only the DSI and |
| 50 | 53 | DPI controller nodes have multiple clock inputs. These are documented in |
| 51 | 54 | mediatek,dsi.txt and mediatek,dpi.txt, respectively. |
| 55 | + An exception is that the mt8183 mutex is always free running with no clocks property. | |
| 52 | 56 | |
| 53 | 57 | Required properties (DMA function blocks): |
| 54 | 58 | - compatible: Should be one of |
| @@ -7,7 +7,7 @@ channel output. · 6 unmodified lines | ||
| 7 | 7 | |
| 8 | 8 | Required properties: |
| 9 | 9 | - compatible: "mediatek,<chip>-dsi" |
| 10 | − the supported chips are mt2701 and mt8173. | |
| 10 | + the supported chips are mt2701, mt8173 and mt8183. | |
| 11 | 11 | - reg: Physical base address and length of the controller's registers |
| 12 | 12 | - interrupts: The interrupt signal from the function block. |
| 13 | 13 | - clocks: device clocks |
| @@ -26,7 +26,7 @@ The MIPI TX configuration module controls the MIPI D-PHY. · 12 unmodified lines | ||
| 26 | 26 | |
| 27 | 27 | Required properties: |
| 28 | 28 | - compatible: "mediatek,<chip>-mipi-tx" |
| 29 | − the supported chips are mt2701 and mt8173. | |
| 29 | + the supported chips are mt2701, mt8173 and mt8183. | |
| 30 | 30 | - reg: Physical base address and length of the controller's registers |
| 31 | 31 | - clocks: PLL reference clock |
| 32 | 32 | - clock-output-names: name of the output clock line to the DSI encoder |
| @@ -8,6 +8,7 @@ Required Properties: · 7 unmodified lines | ||
| 8 | 8 | - "renesas,du-r8a7745" for R8A7745 (RZ/G1E) compatible DU |
| 9 | 9 | - "renesas,du-r8a77470" for R8A77470 (RZ/G1C) compatible DU |
| 10 | 10 | - "renesas,du-r8a774a1" for R8A774A1 (RZ/G2M) compatible DU |
| 11 | + - "renesas,du-r8a774b1" for R8A774B1 (RZ/G2N) compatible DU | |
| 11 | 12 | - "renesas,du-r8a774c0" for R8A774C0 (RZ/G2E) compatible DU |
| 12 | 13 | - "renesas,du-r8a7779" for R8A7779 (R-Car H1) compatible DU |
| 13 | 14 | - "renesas,du-r8a7790" for R8A7790 (R-Car H2) compatible DU |
| @@ -60,6 +61,7 @@ corresponding to each DU output. · 46 unmodified lines | ||
| 60 | 61 | R8A7745 (RZ/G1E) DPAD 0 DPAD 1 - - |
| 61 | 62 | R8A77470 (RZ/G1C) DPAD 0 DPAD 1 LVDS 0 - |
| 62 | 63 | R8A774A1 (RZ/G2M) DPAD 0 HDMI 0 LVDS 0 - |
| 64 | + R8A774B1 (RZ/G2N) DPAD 0 HDMI 0 LVDS 0 - | |
| 63 | 65 | R8A774C0 (RZ/G2E) DPAD 0 LVDS 0 LVDS 1 - |
| 64 | 66 | R8A7779 (R-Car H1) DPAD 0 DPAD 1 - - |
| 65 | 67 | R8A7790 (R-Car H2) DPAD 0 LVDS 0 LVDS 1 - |
| @@ -20,6 +20,10 @@ Required properties: · 19 unmodified lines | ||
| 20 | 20 | "rockchip,rk3228-vop"; |
| 21 | 21 | "rockchip,rk3328-vop"; |
| 22 | 22 | |
| 23 | +- reg: Must contain one entry corresponding to the base address and length | |
| 24 | + of the register space. Can optionally contain a second entry | |
| 25 | + corresponding to the CRTC gamma LUT address. | |
| 26 | + | |
| 23 | 27 | - interrupts: should contain a list of all VOP IP block interrupts in the |
| 24 | 28 | order: VSYNC, LCD_SYSTEM. The interrupt specifier |
| 25 | 29 | format depends on the interrupt controller used. |
| @@ -48,7 +52,7 @@ Example: · 22 unmodified lines | ||
| 48 | 52 | SoC specific DT entry: |
| 49 | 53 | vopb: vopb@ff930000 { |
| 50 | 54 | compatible = "rockchip,rk3288-vop"; |
| 51 | − reg = <0xff930000 0x19c>; | |
| 55 | + reg = <0x0 0xff930000 0x0 0x19c>, <0x0 0xff931000 0x0 0x1000>; | |
| 52 | 56 | interrupts = <GIC_SPI 15 IRQ_TYPE_LEVEL_HIGH>; |
| 53 | 57 | clocks = <&cru ACLK_VOP0>, <&cru DCLK_VOP0>, <&cru HCLK_VOP0>; |
| 54 | 58 | clock-names = "aclk_vop", "dclk_vop", "hclk_vop"; |
| @@ -1,8 +1,11 @@ | ||
| 1 | 1 | * Advanced Interrupt Controller (AIC) |
| 2 | 2 | |
| 3 | 3 | Required properties: |
| 4 | −- compatible: Should be "atmel,<chip>-aic" | |
| 5 | − <chip> can be "at91rm9200", "sama5d2", "sama5d3" or "sama5d4" | |
| 4 | +- compatible: Should be: | |
| 5 | + - "atmel,<chip>-aic" where <chip> can be "at91rm9200", "sama5d2", | |
| 6 | + "sama5d3" or "sama5d4" | |
| 7 | + - "microchip,<chip>-aic" where <chip> can be "sam9x60" | |
| 8 | + | |
| 6 | 9 | - interrupt-controller: Identifies the node as an interrupt controller. |
| 7 | 10 | - #interrupt-cells: The number of cells to define the interrupts. It should be 3. |
| 8 | 11 | The first cell is the IRQ number (aka "Peripheral IDentifier" on datasheet). |
| @@ -1,7 +1,7 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) |
| 2 | 2 | %YAML 1.2 |
| 3 | 3 | --- |
| 4 | −$id: http://devicetree.org/schemas/arm/allwinner,sun4i-a10-csi.yaml# | |
| 4 | +$id: http://devicetree.org/schemas/media/allwinner,sun4i-a10-csi.yaml# | |
| 5 | 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# |
| 6 | 6 | |
| 7 | 7 | title: Allwinner A10 CMOS Sensor Interface (CSI) Device Tree Bindings |
| @@ -27,14 +27,12 @@ properties: · 19 unmodified lines | ||
| 27 | 27 | clocks: |
| 28 | 28 | items: |
| 29 | 29 | - description: The CSI interface clock |
| 30 | − - description: The CSI module clock | |
| 31 | 30 | - description: The CSI ISP clock |
| 32 | 31 | - description: The CSI DRAM clock |
| 33 | 32 | |
| 34 | 33 | clock-names: |
| 35 | 34 | items: |
| 36 | 35 | - const: bus |
| 37 | − - const: mod | |
| 38 | 36 | - const: isp |
| 39 | 37 | - const: ram |
| 40 | 38 | |
| @@ -89,9 +87,8 @@ examples: · 48 unmodified lines | ||
| 89 | 87 | compatible = "allwinner,sun7i-a20-csi0"; |
| 90 | 88 | reg = <0x01c09000 0x1000>; |
| 91 | 89 | interrupts = <GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>; |
| 92 | − clocks = <&ccu CLK_AHB_CSI0>, <&ccu CLK_CSI0>, | |
| 93 | − <&ccu CLK_CSI_SCLK>, <&ccu CLK_DRAM_CSI0>; | |
| 94 | − clock-names = "bus", "mod", "isp", "ram"; | |
| 90 | + clocks = <&ccu CLK_AHB_CSI0>, <&ccu CLK_CSI_SCLK>, <&ccu CLK_DRAM_CSI0>; | |
| 91 | + clock-names = "bus", "isp", "ram"; | |
| 95 | 92 | resets = <&ccu RST_CSI0>; |
| 96 | 93 | |
| 97 | 94 | port { |
| @@ -33,13 +33,13 @@ patternProperties: · 32 unmodified lines | ||
| 33 | 33 | allOf: |
| 34 | 34 | - $ref: "/schemas/types.yaml#/definitions/string" |
| 35 | 35 | - enum: [ ADC0, ADC1, ADC10, ADC11, ADC12, ADC13, ADC14, ADC15, |
| 36 | − ADC2, ADC3, ADC4, ADC5, ADC6, ADC7, ADC8, ADC9, BMCINT, ESPI, | |
| 37 | − ESPIALT, FSI1, FSI2, FWSPIABR, FWSPID, FWSPIWP, GPIT0, GPIT1, | |
| 38 | − GPIT2, GPIT3, GPIT4, GPIT5, GPIT6, GPIT7, GPIU0, GPIU1, GPIU2, | |
| 39 | − GPIU3, GPIU4, GPIU5, GPIU6, GPIU7, I2C1, I2C10, I2C11, I2C12, | |
| 40 | − I2C13, I2C14, I2C15, I2C16, I2C2, I2C3, I2C4, I2C5, I2C6, I2C7, | |
| 41 | − I2C8, I2C9, I3C3, I3C4, I3C5, I3C6, JTAGM, LHPD, LHSIRQ, LPC, | |
| 42 | − LPCHC, LPCPD, LPCPME, LPCSMI, LSIRQ, MACLINK1, MACLINK2, | |
| 36 | + ADC2, ADC3, ADC4, ADC5, ADC6, ADC7, ADC8, ADC9, BMCINT, EMMC, | |
| 37 | + ESPI, ESPIALT, FSI1, FSI2, FWSPIABR, FWSPID, FWSPIWP, GPIT0, | |
| 38 | + GPIT1, GPIT2, GPIT3, GPIT4, GPIT5, GPIT6, GPIT7, GPIU0, GPIU1, | |
| 39 | + GPIU2, GPIU3, GPIU4, GPIU5, GPIU6, GPIU7, I2C1, I2C10, I2C11, | |
| 40 | + I2C12, I2C13, I2C14, I2C15, I2C16, I2C2, I2C3, I2C4, I2C5, I2C6, | |
| 41 | + I2C7, I2C8, I2C9, I3C3, I3C4, I3C5, I3C6, JTAGM, LHPD, LHSIRQ, | |
| 42 | + LPC, LPCHC, LPCPD, LPCPME, LPCSMI, LSIRQ, MACLINK1, MACLINK2, | |
| 43 | 43 | MACLINK3, MACLINK4, MDIO1, MDIO2, MDIO3, MDIO4, NCTS1, NCTS2, |
| 44 | 44 | NCTS3, NCTS4, NDCD1, NDCD2, NDCD3, NDCD4, NDSR1, NDSR2, NDSR3, |
| 45 | 45 | NDSR4, NDTR1, NDTR2, NDTR3, NDTR4, NRI1, NRI2, NRI3, NRI4, NRTS1, |
| @@ -48,47 +48,45 @@ patternProperties: · 2 unmodified lines | ||
| 48 | 48 | PWM8, PWM9, RGMII1, RGMII2, RGMII3, RGMII4, RMII1, RMII2, RMII3, |
| 49 | 49 | RMII4, RXD1, RXD2, RXD3, RXD4, SALT1, SALT10, SALT11, SALT12, |
| 50 | 50 | SALT13, SALT14, SALT15, SALT16, SALT2, SALT3, SALT4, SALT5, |
| 51 | − SALT6, SALT7, SALT8, SALT9, SD1, SD2, SD3, SD3DAT4, SD3DAT5, | |
| 52 | − SD3DAT6, SD3DAT7, SGPM1, SGPS1, SIOONCTRL, SIOPBI, SIOPBO, | |
| 53 | − SIOPWREQ, SIOPWRGD, SIOS3, SIOS5, SIOSCI, SPI1, SPI1ABR, SPI1CS1, | |
| 54 | − SPI1WP, SPI2, SPI2CS1, SPI2CS2, TACH0, TACH1, TACH10, TACH11, | |
| 55 | − TACH12, TACH13, TACH14, TACH15, TACH2, TACH3, TACH4, TACH5, | |
| 56 | − TACH6, TACH7, TACH8, TACH9, THRU0, THRU1, THRU2, THRU3, TXD1, | |
| 57 | − TXD2, TXD3, TXD4, UART10, UART11, UART12, UART13, UART6, UART7, | |
| 58 | − UART8, UART9, VB, VGAHS, VGAVS, WDTRST1, WDTRST2, WDTRST3, | |
| 59 | − WDTRST4, ] | |
| 51 | + SALT6, SALT7, SALT8, SALT9, SD1, SD2, SGPM1, SGPS1, SIOONCTRL, | |
| 52 | + SIOPBI, SIOPBO, SIOPWREQ, SIOPWRGD, SIOS3, SIOS5, SIOSCI, SPI1, | |
| 53 | + SPI1ABR, SPI1CS1, SPI1WP, SPI2, SPI2CS1, SPI2CS2, TACH0, TACH1, | |
| 54 | + TACH10, TACH11, TACH12, TACH13, TACH14, TACH15, TACH2, TACH3, | |
| 55 | + TACH4, TACH5, TACH6, TACH7, TACH8, TACH9, THRU0, THRU1, THRU2, | |
| 56 | + THRU3, TXD1, TXD2, TXD3, TXD4, UART10, UART11, UART12, UART13, | |
| 57 | + UART6, UART7, UART8, UART9, VB, VGAHS, VGAVS, WDTRST1, WDTRST2, | |
| 58 | + WDTRST3, WDTRST4, ] | |
| 60 | 59 | groups: |
| 61 | 60 | allOf: |
| 62 | 61 | - $ref: "/schemas/types.yaml#/definitions/string" |
| 63 | 62 | - enum: [ ADC0, ADC1, ADC10, ADC11, ADC12, ADC13, ADC14, ADC15, |
| 64 | − ADC2, ADC3, ADC4, ADC5, ADC6, ADC7, ADC8, ADC9, BMCINT, ESPI, | |
| 65 | − ESPIALT, FSI1, FSI2, FWSPIABR, FWSPID, FWQSPID, FWSPIWP, GPIT0, | |
| 66 | − GPIT1, GPIT2, GPIT3, GPIT4, GPIT5, GPIT6, GPIT7, GPIU0, GPIU1, | |
| 67 | − GPIU2, GPIU3, GPIU4, GPIU5, GPIU6, GPIU7, HVI3C3, HVI3C4, I2C1, | |
| 68 | − I2C10, I2C11, I2C12, I2C13, I2C14, I2C15, I2C16, I2C2, I2C3, | |
| 69 | − I2C4, I2C5, I2C6, I2C7, I2C8, I2C9, I3C3, I3C4, I3C5, I3C6, | |
| 70 | − JTAGM, LHPD, LHSIRQ, LPC, LPCHC, LPCPD, LPCPME, LPCSMI, LSIRQ, | |
| 71 | − MACLINK1, MACLINK2, MACLINK3, MACLINK4, MDIO1, MDIO2, MDIO3, | |
| 72 | − MDIO4, NCTS1, NCTS2, NCTS3, NCTS4, NDCD1, NDCD2, NDCD3, NDCD4, | |
| 73 | − NDSR1, NDSR2, NDSR3, NDSR4, NDTR1, NDTR2, NDTR3, NDTR4, NRI1, | |
| 74 | − NRI2, NRI3, NRI4, NRTS1, NRTS2, NRTS3, NRTS4, OSCCLK, PEWAKE, | |
| 75 | − PWM0, PWM1, PWM10G0, PWM10G1, PWM11G0, PWM11G1, PWM12G0, PWM12G1, | |
| 76 | − PWM13G0, PWM13G1, PWM14G0, PWM14G1, PWM15G0, PWM15G1, PWM2, PWM3, | |
| 77 | − PWM4, PWM5, PWM6, PWM7, PWM8G0, PWM8G1, PWM9G0, PWM9G1, QSPI1, | |
| 78 | − QSPI2, RGMII1, RGMII2, RGMII3, RGMII4, RMII1, RMII2, RMII3, | |
| 79 | − RMII4, RXD1, RXD2, RXD3, RXD4, SALT1, SALT10G0, SALT10G1, | |
| 80 | − SALT11G0, SALT11G1, SALT12G0, SALT12G1, SALT13G0, SALT13G1, | |
| 81 | − SALT14G0, SALT14G1, SALT15G0, SALT15G1, SALT16G0, SALT16G1, | |
| 82 | − SALT2, SALT3, SALT4, SALT5, SALT6, SALT7, SALT8, SALT9G0, | |
| 83 | − SALT9G1, SD1, SD2, SD3, SD3DAT4, SD3DAT5, SD3DAT6, SD3DAT7, | |
| 84 | − SGPM1, SGPS1, SIOONCTRL, SIOPBI, SIOPBO, SIOPWREQ, SIOPWRGD, | |
| 85 | − SIOS3, SIOS5, SIOSCI, SPI1, SPI1ABR, SPI1CS1, SPI1WP, SPI2, | |
| 86 | − SPI2CS1, SPI2CS2, TACH0, TACH1, TACH10, TACH11, TACH12, TACH13, | |
| 87 | − TACH14, TACH15, TACH2, TACH3, TACH4, TACH5, TACH6, TACH7, TACH8, | |
| 88 | − TACH9, THRU0, THRU1, THRU2, THRU3, TXD1, TXD2, TXD3, TXD4, | |
| 89 | − UART10, UART11, UART12G0, UART12G1, UART13G0, UART13G1, UART6, | |
| 90 | − UART7, UART8, UART9, VB, VGAHS, VGAVS, WDTRST1, WDTRST2, WDTRST3, | |
| 91 | − WDTRST4, ] | |
| 63 | + ADC2, ADC3, ADC4, ADC5, ADC6, ADC7, ADC8, ADC9, BMCINT, EMMCG1, | |
| 64 | + EMMCG4, EMMCG8, ESPI, ESPIALT, FSI1, FSI2, FWSPIABR, FWSPID, | |
| 65 | + FWQSPID, FWSPIWP, GPIT0, GPIT1, GPIT2, GPIT3, GPIT4, GPIT5, | |
| 66 | + GPIT6, GPIT7, GPIU0, GPIU1, GPIU2, GPIU3, GPIU4, GPIU5, GPIU6, | |
| 67 | + GPIU7, HVI3C3, HVI3C4, I2C1, I2C10, I2C11, I2C12, I2C13, I2C14, | |
| 68 | + I2C15, I2C16, I2C2, I2C3, I2C4, I2C5, I2C6, I2C7, I2C8, I2C9, | |
| 69 | + I3C3, I3C4, I3C5, I3C6, JTAGM, LHPD, LHSIRQ, LPC, LPCHC, LPCPD, | |
| 70 | + LPCPME, LPCSMI, LSIRQ, MACLINK1, MACLINK2, MACLINK3, MACLINK4, | |
| 71 | + MDIO1, MDIO2, MDIO3, MDIO4, NCTS1, NCTS2, NCTS3, NCTS4, NDCD1, | |
| 72 | + NDCD2, NDCD3, NDCD4, NDSR1, NDSR2, NDSR3, NDSR4, NDTR1, NDTR2, | |
| 73 | + NDTR3, NDTR4, NRI1, NRI2, NRI3, NRI4, NRTS1, NRTS2, NRTS3, NRTS4, | |
| 74 | + OSCCLK, PEWAKE, PWM0, PWM1, PWM10G0, PWM10G1, PWM11G0, PWM11G1, | |
| 75 | + PWM12G0, PWM12G1, PWM13G0, PWM13G1, PWM14G0, PWM14G1, PWM15G0, | |
| 76 | + PWM15G1, PWM2, PWM3, PWM4, PWM5, PWM6, PWM7, PWM8G0, PWM8G1, | |
| 77 | + PWM9G0, PWM9G1, QSPI1, QSPI2, RGMII1, RGMII2, RGMII3, RGMII4, | |
| 78 | + RMII1, RMII2, RMII3, RMII4, RXD1, RXD2, RXD3, RXD4, SALT1, | |
| 79 | + SALT10G0, SALT10G1, SALT11G0, SALT11G1, SALT12G0, SALT12G1, | |
| 80 | + SALT13G0, SALT13G1, SALT14G0, SALT14G1, SALT15G0, SALT15G1, | |
| 81 | + SALT16G0, SALT16G1, SALT2, SALT3, SALT4, SALT5, SALT6, SALT7, | |
| 82 | + SALT8, SALT9G0, SALT9G1, SD1, SD2, SD3, SGPM1, SGPS1, SIOONCTRL, | |
| 83 | + SIOPBI, SIOPBO, SIOPWREQ, SIOPWRGD, SIOS3, SIOS5, SIOSCI, SPI1, | |
| 84 | + SPI1ABR, SPI1CS1, SPI1WP, SPI2, SPI2CS1, SPI2CS2, TACH0, TACH1, | |
| 85 | + TACH10, TACH11, TACH12, TACH13, TACH14, TACH15, TACH2, TACH3, | |
| 86 | + TACH4, TACH5, TACH6, TACH7, TACH8, TACH9, THRU0, THRU1, THRU2, | |
| 87 | + THRU3, TXD1, TXD2, TXD3, TXD4, UART10, UART11, UART12G0, | |
| 88 | + UART12G1, UART13G0, UART13G1, UART6, UART7, UART8, UART9, VB, | |
| 89 | + VGAHS, VGAVS, WDTRST1, WDTRST2, WDTRST3, WDTRST4, ] | |
| 92 | 90 | |
| 93 | 91 | required: |
| 94 | 92 | - compatible |
| @@ -30,8 +30,8 @@ if: · 29 unmodified lines | ||
| 30 | 30 | properties: |
| 31 | 31 | compatible: |
| 32 | 32 | enum: |
| 33 | − - const: regulator-fixed | |
| 34 | − - const: regulator-fixed-clock | |
| 33 | + - regulator-fixed | |
| 34 | + - regulator-fixed-clock | |
| 35 | 35 | |
| 36 | 36 | regulator-name: true |
| 37 | 37 | |
| @@ -24,15 +24,17 @@ description: | · 23 unmodified lines | ||
| 24 | 24 | |
| 25 | 25 | properties: |
| 26 | 26 | compatible: |
| 27 | − items: | |
| 28 | − - enum: | |
| 29 | − - sifive,rocket0 | |
| 30 | − - sifive,e5 | |
| 31 | − - sifive,e51 | |
| 32 | − - sifive,u54-mc | |
| 33 | − - sifive,u54 | |
| 34 | − - sifive,u5 | |
| 35 | − - const: riscv | |
| 27 | + oneOf: | |
| 28 | + - items: | |
| 29 | + - enum: | |
| 30 | + - sifive,rocket0 | |
| 31 | + - sifive,e5 | |
| 32 | + - sifive,e51 | |
| 33 | + - sifive,u54-mc | |
| 34 | + - sifive,u54 | |
| 35 | + - sifive,u5 | |
| 36 | + - const: riscv | |
| 37 | + - const: riscv # Simulator only | |
| 36 | 38 | description: |
| 37 | 39 | Identifies that the hart uses the RISC-V instruction set |
| 38 | 40 | and identifies the type of the hart. |
| @@ -66,12 +68,8 @@ properties: · 27 unmodified lines | ||
| 66 | 68 | insensitive, letters in the riscv,isa string must be all |
| 67 | 69 | lowercase to simplify parsing. |
| 68 | 70 | |
| 69 | − timebase-frequency: | |
| 70 | − type: integer | |
| 71 | − minimum: 1 | |
| 72 | − description: | |
| 73 | − Specifies the clock frequency of the system timer in Hz. | |
| 74 | − This value is common to all harts on a single system image. | |
| 71 | + # RISC-V requires 'timebase-frequency' in /cpus, so disallow it here | |
| 72 | + timebase-frequency: false | |
| 75 | 73 | |
| 76 | 74 | interrupt-controller: |
| 77 | 75 | type: object |
| @@ -93,7 +91,6 @@ properties: · 15 unmodified lines | ||
| 93 | 91 | |
| 94 | 92 | required: |
| 95 | 93 | - riscv,isa |
| 96 | − - timebase-frequency | |
| 97 | 94 | - interrupt-controller |
| 98 | 95 | |
| 99 | 96 | examples: |
| @@ -26,6 +26,8 @@ Required properties: · 25 unmodified lines | ||
| 26 | 26 | - "renesas,hscif-r8a77470" for R8A77470 (RZ/G1C) HSCIF compatible UART. |
| 27 | 27 | - "renesas,scif-r8a774a1" for R8A774A1 (RZ/G2M) SCIF compatible UART. |
| 28 | 28 | - "renesas,hscif-r8a774a1" for R8A774A1 (RZ/G2M) HSCIF compatible UART. |
| 29 | + - "renesas,scif-r8a774b1" for R8A774B1 (RZ/G2N) SCIF compatible UART. | |
| 30 | + - "renesas,hscif-r8a774b1" for R8A774B1 (RZ/G2N) HSCIF compatible UART. | |
| 29 | 31 | - "renesas,scif-r8a774c0" for R8A774C0 (RZ/G2E) SCIF compatible UART. |
| 30 | 32 | - "renesas,hscif-r8a774c0" for R8A774C0 (RZ/G2E) HSCIF compatible UART. |
| 31 | 33 | - "renesas,scif-r8a7778" for R8A7778 (R-Car M1) SCIF compatible UART. |
| @@ -85,8 +85,8 @@ A child node must exist to represent the core DWC2 IP block. The name of · 84 unmodified lines | ||
| 85 | 85 | the node is not important. The content of the node is defined in dwc2.txt. |
| 86 | 86 | |
| 87 | 87 | PHY documentation is provided in the following places: |
| 88 | −- Documentation/devicetree/bindings/phy/meson-g12a-usb2-phy.txt | |
| 89 | −- Documentation/devicetree/bindings/phy/meson-g12a-usb3-pcie-phy.txt | |
| 88 | +- Documentation/devicetree/bindings/phy/amlogic,meson-g12a-usb2-phy.yaml | |
| 89 | +- Documentation/devicetree/bindings/phy/amlogic,meson-g12a-usb3-pcie-phy.yaml | |
| 90 | 90 | |
| 91 | 91 | Example device nodes: |
| 92 | 92 | usb: usb@ffe09000 { |
| @@ -63,7 +63,11 @@ properties: · 62 unmodified lines | ||
| 63 | 63 | description: |
| 64 | 64 | Set this flag to force EHCI reset after resume. |
| 65 | 65 | |
| 66 | − phys: true | |
| 66 | + phys: | |
| 67 | + description: PHY specifier for the USB PHY | |
| 68 | + | |
| 69 | + phy-names: | |
| 70 | + const: usb | |
| 67 | 71 | |
| 68 | 72 | required: |
| 69 | 73 | - compatible |
| @@ -89,6 +93,7 @@ examples: · 19 unmodified lines | ||
| 89 | 93 | interrupts = <39>; |
| 90 | 94 | clocks = <&ahb_gates 1>; |
| 91 | 95 | phys = <&usbphy 1>; |
| 96 | + phy-names = "usb"; | |
| 92 | 97 | }; |
| 93 | 98 | |
| 94 | 99 | ... |
| @@ -67,7 +67,11 @@ properties: · 66 unmodified lines | ||
| 67 | 67 | description: |
| 68 | 68 | Overrides the detected port count |
| 69 | 69 | |
| 70 | − phys: true | |
| 70 | + phys: | |
| 71 | + description: PHY specifier for the USB PHY | |
| 72 | + | |
| 73 | + phy-names: | |
| 74 | + const: usb | |
| 71 | 75 | |
| 72 | 76 | required: |
| 73 | 77 | - compatible |
| @@ -84,6 +88,7 @@ examples: · 10 unmodified lines | ||
| 84 | 88 | interrupts = <64>; |
| 85 | 89 | clocks = <&usb_clk 6>, <&ahb_gates 2>; |
| 86 | 90 | phys = <&usbphy 1>; |
| 91 | + phy-names = "usb"; | |
| 87 | 92 | }; |
| 88 | 93 | |
| 89 | 94 | ... |
| @@ -33,7 +33,7 @@ Required properties: · 32 unmodified lines | ||
| 33 | 33 | "dma_ck": dma_bus clock for data transfer by DMA, |
| 34 | 34 | "xhci_ck": controller clock |
| 35 | 35 | |
| 36 | − - phys : see usb-hcd.txt in the current directory | |
| 36 | + - phys : see usb-hcd.yaml in the current directory | |
| 37 | 37 | |
| 38 | 38 | Optional properties: |
| 39 | 39 | - wakeup-source : enable USB remote wakeup; |
| @@ -53,7 +53,7 @@ Optional properties: · 13 unmodified lines | ||
| 53 | 53 | See: Documentation/devicetree/bindings/pinctrl/pinctrl-bindings.txt |
| 54 | 54 | - imod-interval-ns: default interrupt moderation interval is 5000ns |
| 55 | 55 | |
| 56 | −additionally the properties from usb-hcd.txt (in the current directory) are | |
| 56 | +additionally the properties from usb-hcd.yaml (in the current directory) are | |
| 57 | 57 | supported. |
| 58 | 58 | |
| 59 | 59 | Example: |
| @@ -17,7 +17,7 @@ Required properties: · 16 unmodified lines | ||
| 17 | 17 | - clock-names : must contain "sys_ck" for clock of controller, |
| 18 | 18 | the following clocks are optional: |
| 19 | 19 | "ref_ck", "mcu_ck" and "dma_ck"; |
| 20 | − - phys : see usb-hcd.txt in the current directory | |
| 20 | + - phys : see usb-hcd.yaml in the current directory | |
| 21 | 21 | - dr_mode : should be one of "host", "peripheral" or "otg", |
| 22 | 22 | refer to usb/generic.txt |
| 23 | 23 | |
| @@ -60,7 +60,7 @@ Optional properties: · 36 unmodified lines | ||
| 60 | 60 | - mediatek,u3p-dis-msk : mask to disable u3ports, bit0 for u3port0, |
| 61 | 61 | bit1 for u3port1, ... etc; |
| 62 | 62 | |
| 63 | −additionally the properties from usb-hcd.txt (in the current directory) are | |
| 63 | +additionally the properties from usb-hcd.yaml (in the current directory) are | |
| 64 | 64 | supported. |
| 65 | 65 | |
| 66 | 66 | Sub-nodes: |
| @@ -18,8 +18,13 @@ properties: · 17 unmodified lines | ||
| 18 | 18 | description: |
| 19 | 19 | List of all the USB PHYs on this HCD |
| 20 | 20 | |
| 21 | + phy-names: | |
| 22 | + description: | |
| 23 | + Name specifier for the USB PHY | |
| 24 | + | |
| 21 | 25 | examples: |
| 22 | 26 | - | |
| 23 | 27 | usb { |
| 24 | 28 | phys = <&usb2_phy1>, <&usb3_phy1>; |
| 29 | + phy-names = "usb"; | |
| 25 | 30 | }; |
| @@ -6,7 +6,7 @@ Required properties: · 5 unmodified lines | ||
| 6 | 6 | - reg : Should contain 1 register ranges(address and length) |
| 7 | 7 | - interrupts : UHCI controller interrupt |
| 8 | 8 | |
| 9 | −additionally the properties from usb-hcd.txt (in the current directory) are | |
| 9 | +additionally the properties from usb-hcd.yaml (in the current directory) are | |
| 10 | 10 | supported. |
| 11 | 11 | |
| 12 | 12 | Example: |
| @@ -41,9 +41,9 @@ Optional properties: · 40 unmodified lines | ||
| 41 | 41 | - usb3-lpm-capable: determines if platform is USB3 LPM capable |
| 42 | 42 | - quirk-broken-port-ped: set if the controller has broken port disable mechanism |
| 43 | 43 | - imod-interval-ns: default interrupt moderation interval is 5000ns |
| 44 | − - phys : see usb-hcd.txt in the current directory | |
| 44 | + - phys : see usb-hcd.yaml in the current directory | |
| 45 | 45 | |
| 46 | −additionally the properties from usb-hcd.txt (in the current directory) are | |
| 46 | +additionally the properties from usb-hcd.yaml (in the current directory) are | |
| 47 | 47 | supported. |
| 48 | 48 | |
| 49 | 49 | |
| @@ -118,13 +118,13 @@ Kernel Functions and Structures Reference · 117 unmodified lines | ||
| 118 | 118 | Reservation Objects |
| 119 | 119 | ------------------- |
| 120 | 120 | |
| 121 | −.. kernel-doc:: drivers/dma-buf/reservation.c | |
| 121 | +.. kernel-doc:: drivers/dma-buf/dma-resv.c | |
| 122 | 122 | :doc: Reservation Object Overview |
| 123 | 123 | |
| 124 | −.. kernel-doc:: drivers/dma-buf/reservation.c | |
| 124 | +.. kernel-doc:: drivers/dma-buf/dma-resv.c | |
| 125 | 125 | :export: |
| 126 | 126 | |
| 127 | −.. kernel-doc:: include/linux/reservation.h | |
| 127 | +.. kernel-doc:: include/linux/dma-resv.h | |
| 128 | 128 | :internal: |
| 129 | 129 | |
| 130 | 130 | DMA Fences |
| @@ -79,16 +79,71 @@ AMDGPU XGMI Support · 78 unmodified lines | ||
| 79 | 79 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c |
| 80 | 80 | :internal: |
| 81 | 81 | |
| 82 | −AMDGPU RAS debugfs control interface | |
| 83 | −==================================== | |
| 82 | +AMDGPU RAS Support | |
| 83 | +================== | |
| 84 | + | |
| 85 | +The AMDGPU RAS interfaces are exposed via sysfs (for informational queries) and | |
| 86 | +debugfs (for error injection). | |
| 87 | + | |
| 88 | +RAS debugfs/sysfs Control and Error Injection Interfaces | |
| 89 | +-------------------------------------------------------- | |
| 84 | 90 | |
| 85 | 91 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c |
| 86 | 92 | :doc: AMDGPU RAS debugfs control interface |
| 87 | 93 | |
| 94 | +RAS Reboot Behavior for Unrecoverable Errors | |
| 95 | +-------------------------------------------------------- | |
| 96 | + | |
| 97 | +.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | |
| 98 | + :doc: AMDGPU RAS Reboot Behavior for Unrecoverable Errors | |
| 99 | + | |
| 100 | +RAS Error Count sysfs Interface | |
| 101 | +------------------------------- | |
| 102 | + | |
| 103 | +.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | |
| 104 | + :doc: AMDGPU RAS sysfs Error Count Interface | |
| 105 | + | |
| 106 | +RAS EEPROM debugfs Interface | |
| 107 | +---------------------------- | |
| 108 | + | |
| 109 | +.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | |
| 110 | + :doc: AMDGPU RAS debugfs EEPROM table reset interface | |
| 111 | + | |
| 112 | +RAS VRAM Bad Pages sysfs Interface | |
| 113 | +---------------------------------- | |
| 114 | + | |
| 115 | +.. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | |
| 116 | + :doc: AMDGPU RAS sysfs gpu_vram_bad_pages Interface | |
| 88 | 117 | |
| 89 | 118 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c |
| 90 | 119 | :internal: |
| 91 | 120 | |
| 121 | +Sample Code | |
| 122 | +----------- | |
| 123 | +Sample code for testing error injection can be found here: | |
| 124 | +https://cgit.freedesktop.org/mesa/drm/tree/tests/amdgpu/ras_tests.c | |
| 125 | + | |
| 126 | +This is part of the libdrm amdgpu unit tests which cover several areas of the GPU. | |
| 127 | +There are four sets of tests: | |
| 128 | + | |
| 129 | +RAS Basic Test | |
| 130 | + | |
| 131 | +The test verifies the RAS feature enabled status and makes sure the necessary sysfs and debugfs files | |
| 132 | +are present. | |
| 133 | + | |
| 134 | +RAS Query Test | |
| 135 | + | |
| 136 | +This test checks the RAS availability and enablement status for each supported IP block as well as | |
| 137 | +the error counts. | |
| 138 | + | |
| 139 | +RAS Inject Test | |
| 140 | + | |
| 141 | +This test injects errors for each IP. | |
| 142 | + | |
| 143 | +RAS Disable Test | |
| 144 | + | |
| 145 | +This test tests disabling of RAS features for each IP block. | |
| 146 | + | |
| 92 | 147 | |
| 93 | 148 | GPU Power/Thermal Controls and Monitoring |
| 94 | 149 | ========================================= |
| @@ -130,11 +185,11 @@ pp_od_clk_voltage · 35 unmodified lines | ||
| 130 | 185 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c |
| 131 | 186 | :doc: pp_od_clk_voltage |
| 132 | 187 | |
| 133 | −pp_dpm_sclk pp_dpm_mclk pp_dpm_pcie | |
| 134 | −~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
| 188 | +pp_dpm_* | |
| 189 | +~~~~~~~~ | |
| 135 | 190 | |
| 136 | 191 | .. kernel-doc:: drivers/gpu/drm/amd/amdgpu/amdgpu_pm.c |
| 137 | − :doc: pp_dpm_sclk pp_dpm_mclk pp_dpm_pcie | |
| 192 | + :doc: pp_dpm_sclk pp_dpm_mclk pp_dpm_socclk pp_dpm_fclk pp_dpm_dcefclk pp_dpm_pcie | |
| 138 | 193 | |
| 139 | 194 | pp_power_profile_mode |
| 140 | 195 | ~~~~~~~~~~~~~~~~~~~~~ |
| @@ -77,9 +77,6 @@ Atomic State Reset and Initialization · 76 unmodified lines | ||
| 77 | 77 | Atomic State Helper Reference |
| 78 | 78 | ----------------------------- |
| 79 | 79 | |
| 80 | −.. kernel-doc:: include/drm/drm_atomic_state_helper.h | |
| 81 | − :internal: | |
| 82 | − | |
| 83 | 80 | .. kernel-doc:: drivers/gpu/drm/drm_atomic_state_helper.c |
| 84 | 81 | :export: |
| 85 | 82 | |
| @@ -7,6 +7,22 @@ TODO list · 6 unmodified lines | ||
| 7 | 7 | This section contains a list of smaller janitorial tasks in the kernel DRM |
| 8 | 8 | graphics subsystem useful as newbie projects. Or for slow rainy days. |
| 9 | 9 | |
| 10 | +Difficulty | |
| 11 | +---------- | |
| 12 | + | |
| 13 | +To make it easier task are categorized into different levels: | |
| 14 | + | |
| 15 | +Starter: Good tasks to get started with the DRM subsystem. | |
| 16 | + | |
| 17 | +Intermediate: Tasks which need some experience with working in the DRM | |
| 18 | +subsystem, or some specific GPU/display graphics knowledge. For debugging issue | |
| 19 | +it's good to have the relevant hardware (or a virtual driver set up) available | |
| 20 | +for testing. | |
| 21 | + | |
| 22 | +Advanced: Tricky tasks that need fairly good understanding of the DRM subsystem | |
| 23 | +and graphics topics. Generally need the relevant hardware for development and | |
| 24 | +testing. | |
| 25 | + | |
| 10 | 26 | Subsystem-wide refactorings |
| 11 | 27 | =========================== |
| 12 | 28 | |
| @@ -20,6 +36,8 @@ implementations), and then remove it. · 7 unmodified lines | ||
| 20 | 36 | |
| 21 | 37 | Contact: Daniel Vetter, respective driver maintainers |
| 22 | 38 | |
| 39 | +Level: Intermediate | |
| 40 | + | |
| 23 | 41 | Convert existing KMS drivers to atomic modesetting |
| 24 | 42 | -------------------------------------------------- |
| 25 | 43 | |
| @@ -38,6 +56,8 @@ do by directly using the new atomic helper driver callbacks. · 12 unmodified lines | ||
| 38 | 56 | |
| 39 | 57 | Contact: Daniel Vetter, respective driver maintainers |
| 40 | 58 | |
| 59 | +Level: Advanced | |
| 60 | + | |
| 41 | 61 | Clean up the clipped coordination confusion around planes |
| 42 | 62 | --------------------------------------------------------- |
| 43 | 63 | |
| @@ -50,6 +70,8 @@ helpers. · 6 unmodified lines | ||
| 50 | 70 | |
| 51 | 71 | Contact: Ville Syrjälä, Daniel Vetter, driver maintainers |
| 52 | 72 | |
| 73 | +Level: Advanced | |
| 74 | + | |
| 53 | 75 | Convert early atomic drivers to async commit helpers |
| 54 | 76 | ---------------------------------------------------- |
| 55 | 77 | |
| @@ -63,6 +85,8 @@ events for atomic commits correctly. But fixing these bugs is good anyway. · 7 unmodified lines | ||
| 63 | 85 | |
| 64 | 86 | Contact: Daniel Vetter, respective driver maintainers |
| 65 | 87 | |
| 88 | +Level: Advanced | |
| 89 | + | |
| 66 | 90 | Fallout from atomic KMS |
| 67 | 91 | ----------------------- |
| 68 | 92 | |
| @@ -91,6 +115,8 @@ interfaces to fix these issues: · 22 unmodified lines | ||
| 91 | 115 | |
| 92 | 116 | Contact: Daniel Vetter |
| 93 | 117 | |
| 118 | +Level: Intermediate | |
| 119 | + | |
| 94 | 120 | Get rid of dev->struct_mutex from GEM drivers |
| 95 | 121 | --------------------------------------------- |
| 96 | 122 | |
| @@ -114,6 +140,8 @@ fine-grained per-buffer object and per-context lockings scheme. Currently only t · 17 unmodified lines | ||
| 114 | 140 | |
| 115 | 141 | Contact: Daniel Vetter, respective driver maintainers |
| 116 | 142 | |
| 143 | +Level: Advanced | |
| 144 | + | |
| 117 | 145 | Convert instances of dev_info/dev_err/dev_warn to their DRM_DEV_* equivalent |
| 118 | 146 | ---------------------------------------------------------------------------- |
| 119 | 147 | |
| @@ -129,6 +157,8 @@ are better. · 9 unmodified lines | ||
| 129 | 157 | |
| 130 | 158 | Contact: Sean Paul, Maintainer of the driver you plan to convert |
| 131 | 159 | |
| 160 | +Level: Starter | |
| 161 | + | |
| 132 | 162 | Convert drivers to use simple modeset suspend/resume |
| 133 | 163 | ---------------------------------------------------- |
| 134 | 164 | |
| @@ -139,6 +169,8 @@ of the atomic suspend/resume code in older atomic modeset drivers. · 4 unmodified lines | ||
| 139 | 169 | |
| 140 | 170 | Contact: Maintainer of the driver you plan to convert |
| 141 | 171 | |
| 172 | +Level: Intermediate | |
| 173 | + | |
| 142 | 174 | Convert drivers to use drm_fb_helper_fbdev_setup/teardown() |
| 143 | 175 | ----------------------------------------------------------- |
| 144 | 176 | |
| @@ -157,6 +189,8 @@ probably use drm_fb_helper_fbdev_teardown(). · 12 unmodified lines | ||
| 157 | 189 | |
| 158 | 190 | Contact: Maintainer of the driver you plan to convert |
| 159 | 191 | |
| 192 | +Level: Intermediate | |
| 193 | + | |
| 160 | 194 | Clean up mmap forwarding |
| 161 | 195 | ------------------------ |
| 162 | 196 | |
| @@ -166,14 +200,16 @@ There's drm_gem_prime_mmap() for this now, but still needs to be rolled out. · 3 unmodified lines | ||
| 166 | 200 | |
| 167 | 201 | Contact: Daniel Vetter |
| 168 | 202 | |
| 203 | +Level: Intermediate | |
| 204 | + | |
| 169 | 205 | Generic fbdev defio support |
| 170 | 206 | --------------------------- |
| 171 | 207 | |
| 172 | 208 | The defio support code in the fbdev core has some very specific requirements, |
| 173 | −which means drivers need to have a special framebuffer for fbdev. Which prevents | |
| 174 | −us from using the generic fbdev emulation code everywhere. The main issue is | |
| 175 | −that it uses some fields in struct page itself, which breaks shmem gem objects | |
| 176 | −(and other things). | |
| 209 | +which means drivers need to have a special framebuffer for fbdev. The main | |
| 210 | +issue is that it uses some fields in struct page itself, which breaks shmem | |
| 211 | +gem objects (and other things). To support defio, affected drivers require | |
| 212 | +the use of a shadow buffer, which may add CPU and memory overhead. | |
| 177 | 213 | |
| 178 | 214 | Possible solution would be to write our own defio mmap code in the drm fbdev |
| 179 | 215 | emulation. It would need to fully wrap the existing mmap ops, forwarding |
| @@ -196,6 +232,8 @@ Might be good to also have some igt testcases for this. · 16 unmodified lines | ||
| 196 | 232 | |
| 197 | 233 | Contact: Daniel Vetter, Noralf Tronnes |
| 198 | 234 | |
| 235 | +Level: Advanced | |
| 236 | + | |
| 199 | 237 | idr_init_base() |
| 200 | 238 | --------------- |
| 201 | 239 | |
| @@ -206,6 +244,8 @@ efficient. · 4 unmodified lines | ||
| 206 | 244 | |
| 207 | 245 | Contact: Daniel Vetter |
| 208 | 246 | |
| 247 | +Level: Starter | |
| 248 | + | |
| 209 | 249 | struct drm_gem_object_funcs |
| 210 | 250 | --------------------------- |
| 211 | 251 | |
| @@ -216,6 +256,8 @@ We also need a 2nd version of the CMA define that doesn't require the · 4 unmodified lines | ||
| 216 | 256 | vmapping to be present (different hook for prime importing). Plus this needs to |
| 217 | 257 | be rolled out to all drivers using their own implementations, too. |
| 218 | 258 | |
| 259 | +Level: Intermediate | |
| 260 | + | |
| 219 | 261 | Use DRM_MODESET_LOCK_ALL_* helpers instead of boilerplate |
| 220 | 262 | --------------------------------------------------------- |
| 221 | 263 | |
| @@ -231,6 +273,8 @@ As a reference, take a look at the conversions already completed in drm core. · 9 unmodified lines | ||
| 231 | 273 | |
| 232 | 274 | Contact: Sean Paul, respective driver maintainers |
| 233 | 275 | |
| 276 | +Level: Starter | |
| 277 | + | |
| 234 | 278 | Rename CMA helpers to DMA helpers |
| 235 | 279 | --------------------------------- |
| 236 | 280 | |
| @@ -241,6 +285,9 @@ no one knows what that means) since underneath they just use dma_alloc_coherent. · 4 unmodified lines | ||
| 241 | 285 | |
| 242 | 286 | Contact: Laurent Pinchart, Daniel Vetter |
| 243 | 287 | |
| 288 | +Level: Intermediate (mostly because it is a huge tasks without good partial | |
| 289 | +milestones, not technically itself that challenging) | |
| 290 | + | |
| 244 | 291 | Convert direct mode.vrefresh accesses to use drm_mode_vrefresh() |
| 245 | 292 | ---------------------------------------------------------------- |
| 246 | 293 | |
| @@ -259,6 +306,8 @@ drm_display_mode to avoid future use. · 12 unmodified lines | ||
| 259 | 306 | |
| 260 | 307 | Contact: Sean Paul |
| 261 | 308 | |
| 309 | +Level: Starter | |
| 310 | + | |
| 262 | 311 | Remove drm_display_mode.hsync |
| 263 | 312 | ----------------------------- |
| 264 | 313 | |
| @@ -269,6 +318,8 @@ it to use drm_mode_hsync() instead. · 4 unmodified lines | ||
| 269 | 318 | |
| 270 | 319 | Contact: Sean Paul |
| 271 | 320 | |
| 321 | +Level: Starter | |
| 322 | + | |
| 272 | 323 | drm_fb_helper tasks |
| 273 | 324 | ------------------- |
| 274 | 325 | |
| @@ -284,6 +335,8 @@ drm_fb_helper tasks · 9 unmodified lines | ||
| 284 | 335 | removed: drm_fb_helper_single_add_all_connectors(), |
| 285 | 336 | drm_fb_helper_add_one_connector() and drm_fb_helper_remove_one_connector(). |
| 286 | 337 | |
| 338 | +Level: Intermediate | |
| 339 | + | |
| 287 | 340 | connector register/unregister fixes |
| 288 | 341 | ----------------------------------- |
| 289 | 342 | |
| @@ -296,21 +349,11 @@ connector register/unregister fixes · 6 unmodified lines | ||
| 296 | 349 | drm_dp_aux_init, and moving the actual registering into a late_register |
| 297 | 350 | callback as recommended in the kerneldoc. |
| 298 | 351 | |
| 352 | +Level: Intermediate | |
| 353 | + | |
| 299 | 354 | Core refactorings |
| 300 | 355 | ================= |
| 301 | 356 | |
| 302 | −Clean up the DRM header mess | |
| 303 | −---------------------------- | |
| 304 | − | |
| 305 | −The DRM subsystem originally had only one huge global header, ``drmP.h``. This | |
| 306 | −is now split up, but many source files still include it. The remaining part of | |
| 307 | −the cleanup work here is to replace any ``#include <drm/drmP.h>`` by only the | |
| 308 | −headers needed (and fixing up any missing pre-declarations in the headers). | |
| 309 | − | |
| 310 | −In the end no .c file should need to include ``drmP.h`` anymore. | |
| 311 | − | |
| 312 | −Contact: Daniel Vetter | |
| 313 | − | |
| 314 | 357 | Make panic handling work |
| 315 | 358 | ------------------------ |
| 316 | 359 | |
| @@ -350,6 +393,8 @@ This is a really varied tasks with lots of little bits and pieces: · 33 unmodified lines | ||
| 350 | 393 | |
| 351 | 394 | Contact: Daniel Vetter |
| 352 | 395 | |
| 396 | +Level: Advanced | |
| 397 | + | |
| 353 | 398 | Clean up the debugfs support |
| 354 | 399 | ---------------------------- |
| 355 | 400 | |
| @@ -379,6 +424,8 @@ There's a bunch of issues with it: · 23 unmodified lines | ||
| 379 | 424 | |
| 380 | 425 | Contact: Daniel Vetter |
| 381 | 426 | |
| 427 | +Level: Intermediate | |
| 428 | + | |
| 382 | 429 | KMS cleanups |
| 383 | 430 | ------------ |
| 384 | 431 | |
| @@ -394,6 +441,8 @@ Some of these date from the very introduction of KMS in 2008 ... · 9 unmodified lines | ||
| 394 | 441 | end, for which we could add drm_*_cleanup_kfree(). And then there's the (for |
| 395 | 442 | historical reasons) misnamed drm_primary_helper_destroy() function. |
| 396 | 443 | |
| 444 | +Level: Intermediate | |
| 445 | + | |
| 397 | 446 | Better Testing |
| 398 | 447 | ============== |
| 399 | 448 | |
| @@ -402,6 +451,8 @@ Enable trinity for DRM · 2 unmodified lines | ||
| 402 | 451 | |
| 403 | 452 | And fix up the fallout. Should be really interesting ... |
| 404 | 453 | |
| 454 | +Level: Advanced | |
| 455 | + | |
| 405 | 456 | Make KMS tests in i-g-t generic |
| 406 | 457 | ------------------------------- |
| 407 | 458 | |
| @@ -415,6 +466,8 @@ converting things over. For modeset tests we also first need a bit of · 7 unmodified lines | ||
| 415 | 466 | infrastructure to use dumb buffers for untiled buffers, to be able to run all |
| 416 | 467 | the non-i915 specific modeset tests. |
| 417 | 468 | |
| 469 | +Level: Advanced | |
| 470 | + | |
| 418 | 471 | Extend virtual test driver (VKMS) |
| 419 | 472 | --------------------------------- |
| 420 | 473 | |
| @@ -424,6 +477,8 @@ fit the available time. · 3 unmodified lines | ||
| 424 | 477 | |
| 425 | 478 | Contact: Daniel Vetter |
| 426 | 479 | |
| 480 | +Level: See details | |
| 481 | + | |
| 427 | 482 | Backlight Refactoring |
| 428 | 483 | --------------------- |
| 429 | 484 | |
| @@ -437,6 +492,8 @@ Plan to fix this: · 7 unmodified lines | ||
| 437 | 492 | |
| 438 | 493 | Contact: Daniel Vetter |
| 439 | 494 | |
| 495 | +Level: Intermediate | |
| 496 | + | |
| 440 | 497 | Driver Specific |
| 441 | 498 | =============== |
| 442 | 499 | |
| @@ -450,13 +507,6 @@ See drivers/gpu/drm/amd/display/TODO for tasks. · 7 unmodified lines | ||
| 450 | 507 | |
| 451 | 508 | Contact: Harry Wentland, Alex Deucher |
| 452 | 509 | |
| 453 | −i915 | |
| 454 | −---- | |
| 455 | − | |
| 456 | −- Our early/late pm callbacks could be removed in favour of using | |
| 457 | − device_link_add to model the dependency between i915 and snd_had. See | |
| 458 | − https://dri.freedesktop.org/docs/drm/driver-api/device_link.html | |
| 459 | − | |
| 460 | 510 | Bootsplash |
| 461 | 511 | ========== |
| 462 | 512 | |
| @@ -472,5 +522,36 @@ for fbdev. · 9 unmodified lines | ||
| 472 | 522 | |
| 473 | 523 | Contact: Sam Ravnborg |
| 474 | 524 | |
| 525 | +Level: Advanced | |
| 526 | + | |
| 475 | 527 | Outside DRM |
| 476 | 528 | =========== |
| 529 | + | |
| 530 | +Convert fbdev drivers to DRM | |
| 531 | +---------------------------- | |
| 532 | + | |
| 533 | +There are plenty of fbdev drivers for older hardware. Some hwardware has | |
| 534 | +become obsolete, but some still provides good(-enough) framebuffers. The | |
| 535 | +drivers that are still useful should be converted to DRM and afterwards | |
| 536 | +removed from fbdev. | |
| 537 | + | |
| 538 | +Very simple fbdev drivers can best be converted by starting with a new | |
| 539 | +DRM driver. Simple KMS helpers and SHMEM should be able to handle any | |
| 540 | +existing hardware. The new driver's call-back functions are filled from | |
| 541 | +existing fbdev code. | |
| 542 | + | |
| 543 | +More complex fbdev drivers can be refactored step-by-step into a DRM | |
| 544 | +driver with the help of the DRM fbconv helpers. [1] These helpers provide | |
| 545 | +the transition layer between the DRM core infrastructure and the fbdev | |
| 546 | +driver interface. Create a new DRM driver on top of the fbconv helpers, | |
| 547 | +copy over the fbdev driver, and hook it up to the DRM code. Examples for | |
| 548 | +several fbdev drivers are available at [1] and a tutorial of this process | |
| 549 | +available at [2]. The result is a primitive DRM driver that can run X11 | |
| 550 | +and Weston. | |
| 551 | + | |
| 552 | + - [1] https://gitlab.freedesktop.org/tzimmermann/linux/tree/fbconv | |
| 553 | + - [2] https://gitlab.freedesktop.org/tzimmermann/linux/blob/fbconv/drivers/gpu/drm/drm_fbconv_helper.c | |
| 554 | + | |
| 555 | +Contact: Thomas Zimmermann <tzimmermann@suse.de> | |
| 556 | + | |
| 557 | +Level: Advanced | |
| @@ -7,6 +7,7 @@ Linux Hardware Monitoring · 6 unmodified lines | ||
| 7 | 7 | |
| 8 | 8 | hwmon-kernel-api |
| 9 | 9 | pmbus-core |
| 10 | + inspur-ipsps1 | |
| 10 | 11 | submitting-patches |
| 11 | 12 | sysfs-interface |
| 12 | 13 | userspace-tools |
| @@ -1,5 +1,5 @@ | ||
| 1 | 1 | Kernel driver inspur-ipsps1 |
| 2 | −======================= | |
| 2 | +=========================== | |
| 3 | 3 | |
| 4 | 4 | Supported chips: |
| 5 | 5 | |
| @@ -21,10 +21,17 @@ Supported chips: · 20 unmodified lines | ||
| 21 | 21 | |
| 22 | 22 | * AMD Family 14h processors: "Brazos" (C/E/G/Z-Series) |
| 23 | 23 | |
| 24 | −* AMD Family 15h processors: "Bulldozer" (FX-Series), "Trinity", "Kaveri", "Carrizo" | |
| 24 | +* AMD Family 15h processors: "Bulldozer" (FX-Series), "Trinity", "Kaveri", | |
| 25 | + "Carrizo", "Stoney Ridge", "Bristol Ridge" | |
| 25 | 26 | |
| 26 | 27 | * AMD Family 16h processors: "Kabini", "Mullins" |
| 27 | 28 | |
| 29 | +* AMD Family 17h processors: "Zen", "Zen 2" | |
| 30 | + | |
| 31 | +* AMD Family 18h processors: "Hygon Dhyana" | |
| 32 | + | |
| 33 | +* AMD Family 19h processors: "Zen 3" | |
| 34 | + | |
| 28 | 35 | Prefix: 'k10temp' |
| 29 | 36 | |
| 30 | 37 | Addresses scanned: PCI space |
| @@ -110,3 +117,12 @@ The maximum value for Tctl is available in the file temp1_max. · 79 unmodified lines | ||
| 110 | 117 | If the BIOS has enabled hardware temperature control, the threshold at |
| 111 | 118 | which the processor will throttle itself to avoid damage is available in |
| 112 | 119 | temp1_crit and temp1_crit_hyst. |
| 120 | + | |
| 121 | +On some AMD CPUs, there is a difference between the die temperature (Tdie) and | |
| 122 | +the reported temperature (Tctl). Tdie is the real measured temperature, and | |
| 123 | +Tctl is used for fan control. While Tctl is always available as temp1_input, | |
| 124 | +the driver exports Tdie temperature as temp2_input for those CPUs which support | |
| 125 | +it. | |
| 126 | + | |
| 127 | +Models from 17h family report relative temperature, the driver aims to | |
| 128 | +compensate and report the real temperature. | |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −============================================================== | |
| 4 | −Linux* Base Driver for the Intel(R) PRO/100 Family of Adapters | |
| 5 | −============================================================== | |
| 3 | +============================================================= | |
| 4 | +Linux Base Driver for the Intel(R) PRO/100 Family of Adapters | |
| 5 | +============================================================= | |
| 6 | 6 | |
| 7 | 7 | June 1, 2018 |
| 8 | 8 | |
| @@ -21,7 +21,7 @@ Contents · 12 unmodified lines | ||
| 21 | 21 | In This Release |
| 22 | 22 | =============== |
| 23 | 23 | |
| 24 | −This file describes the Linux* Base Driver for the Intel(R) PRO/100 Family of | |
| 24 | +This file describes the Linux Base Driver for the Intel(R) PRO/100 Family of | |
| 25 | 25 | Adapters. This driver includes support for Itanium(R)2-based systems. |
| 26 | 26 | |
| 27 | 27 | For questions related to hardware requirements, refer to the documentation |
| @@ -138,9 +138,9 @@ version 1.6 or later is required for this functionality. · 110 unmodified lines | ||
| 138 | 138 | The latest release of ethtool can be found from |
| 139 | 139 | https://www.kernel.org/pub/software/network/ethtool/ |
| 140 | 140 | |
| 141 | −Enabling Wake on LAN* (WoL) | |
| 142 | −--------------------------- | |
| 143 | −WoL is provided through the ethtool* utility. For instructions on | |
| 141 | +Enabling Wake on LAN (WoL) | |
| 142 | +-------------------------- | |
| 143 | +WoL is provided through the ethtool utility. For instructions on | |
| 144 | 144 | enabling WoL with ethtool, refer to the ethtool man page. WoL will be |
| 145 | 145 | enabled on the system during the next shut down or reboot. For this |
| 146 | 146 | driver version, in order to enable WoL, the e100 driver must be loaded |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −=========================================================== | |
| 4 | −Linux* Base Driver for Intel(R) Ethernet Network Connection | |
| 5 | −=========================================================== | |
| 3 | +========================================================== | |
| 4 | +Linux Base Driver for Intel(R) Ethernet Network Connection | |
| 5 | +========================================================== | |
| 6 | 6 | |
| 7 | 7 | Intel Gigabit Linux driver. |
| 8 | 8 | Copyright(c) 1999 - 2013 Intel Corporation. |
| @@ -438,10 +438,10 @@ ethtool · 429 unmodified lines | ||
| 438 | 438 | The latest release of ethtool can be found from |
| 439 | 439 | https://www.kernel.org/pub/software/network/ethtool/ |
| 440 | 440 | |
| 441 | −Enabling Wake on LAN* (WoL) | |
| 442 | −--------------------------- | |
| 441 | +Enabling Wake on LAN (WoL) | |
| 442 | +-------------------------- | |
| 443 | 443 | |
| 444 | − WoL is configured through the ethtool* utility. | |
| 444 | + WoL is configured through the ethtool utility. | |
| 445 | 445 | |
| 446 | 446 | WoL will be enabled on the system during the next shut down or reboot. |
| 447 | 447 | For this driver version, in order to enable WoL, the e1000 driver must be |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −====================================================== | |
| 4 | −Linux* Driver for Intel(R) Ethernet Network Connection | |
| 5 | −====================================================== | |
| 3 | +===================================================== | |
| 4 | +Linux Driver for Intel(R) Ethernet Network Connection | |
| 5 | +===================================================== | |
| 6 | 6 | |
| 7 | 7 | Intel Gigabit Linux driver. |
| 8 | 8 | Copyright(c) 2008-2018 Intel Corporation. |
| @@ -338,7 +338,7 @@ and higher cannot be forced. Use the autonegotiation advertising setting to · 329 unmodified lines | ||
| 338 | 338 | manually set devices for 1 Gbps and higher. |
| 339 | 339 | |
| 340 | 340 | Speed, duplex, and autonegotiation advertising are configured through the |
| 341 | −ethtool* utility. | |
| 341 | +ethtool utility. | |
| 342 | 342 | |
| 343 | 343 | Caution: Only experienced network administrators should force speed and duplex |
| 344 | 344 | or change autonegotiation advertising manually. The settings at the switch must |
| @@ -351,9 +351,9 @@ will not attempt to auto-negotiate with its link partner since those adapters · 6 unmodified lines | ||
| 351 | 351 | operate only in full duplex and only at their native speed. |
| 352 | 352 | |
| 353 | 353 | |
| 354 | −Enabling Wake on LAN* (WoL) | |
| 355 | −--------------------------- | |
| 356 | −WoL is configured through the ethtool* utility. | |
| 354 | +Enabling Wake on LAN (WoL) | |
| 355 | +-------------------------- | |
| 356 | +WoL is configured through the ethtool utility. | |
| 357 | 357 | |
| 358 | 358 | WoL will be enabled on the system during the next shut down or reboot. For |
| 359 | 359 | this driver version, in order to enable WoL, the e1000e driver must be loaded |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −============================================================== | |
| 4 | −Linux* Base Driver for Intel(R) Ethernet Multi-host Controller | |
| 5 | −============================================================== | |
| 3 | +============================================================= | |
| 4 | +Linux Base Driver for Intel(R) Ethernet Multi-host Controller | |
| 5 | +============================================================= | |
| 6 | 6 | |
| 7 | 7 | August 20, 2018 |
| 8 | 8 | Copyright(c) 2015-2018 Intel Corporation. |
| @@ -120,8 +120,8 @@ rx-flow-hash tcp4|udp4|ah4|esp4|sctp4|tcp6|udp6|ah6|esp6|sctp6 m|v|t|s|d|f|n|r · 111 unmodified lines | ||
| 120 | 120 | Known Issues/Troubleshooting |
| 121 | 121 | ============================ |
| 122 | 122 | |
| 123 | −Enabling SR-IOV in a 64-bit Microsoft* Windows Server* 2012/R2 guest OS under Linux KVM | |
| 124 | −--------------------------------------------------------------------------------------- | |
| 123 | +Enabling SR-IOV in a 64-bit Microsoft Windows Server 2012/R2 guest OS under Linux KVM | |
| 124 | +------------------------------------------------------------------------------------- | |
| 125 | 125 | KVM Hypervisor/VMM supports direct assignment of a PCIe device to a VM. This |
| 126 | 126 | includes traditional PCIe devices, as well as SR-IOV-capable devices based on |
| 127 | 127 | the Intel Ethernet Controller XL710. |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −================================================================== | |
| 4 | −Linux* Base Driver for the Intel(R) Ethernet Controller 700 Series | |
| 5 | −================================================================== | |
| 3 | +================================================================= | |
| 4 | +Linux Base Driver for the Intel(R) Ethernet Controller 700 Series | |
| 5 | +================================================================= | |
| 6 | 6 | |
| 7 | 7 | Intel 40 Gigabit Linux driver. |
| 8 | 8 | Copyright(c) 1999-2018 Intel Corporation. |
| @@ -384,7 +384,7 @@ NOTE: You cannot set the speed for devices based on the Intel(R) Ethernet · 375 unmodified lines | ||
| 384 | 384 | Network Adapter XXV710 based devices. |
| 385 | 385 | |
| 386 | 386 | Speed, duplex, and autonegotiation advertising are configured through the |
| 387 | −ethtool* utility. | |
| 387 | +ethtool utility. | |
| 388 | 388 | |
| 389 | 389 | Caution: Only experienced network administrators should force speed and duplex |
| 390 | 390 | or change autonegotiation advertising manually. The settings at the switch must |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −================================================================== | |
| 4 | −Linux* Base Driver for Intel(R) Ethernet Adaptive Virtual Function | |
| 5 | −================================================================== | |
| 3 | +================================================================= | |
| 4 | +Linux Base Driver for Intel(R) Ethernet Adaptive Virtual Function | |
| 5 | +================================================================= | |
| 6 | 6 | |
| 7 | 7 | Intel Ethernet Adaptive Virtual Function Linux driver. |
| 8 | 8 | Copyright(c) 2013-2018 Intel Corporation. |
| @@ -19,7 +19,7 @@ Contents · 10 unmodified lines | ||
| 19 | 19 | Overview |
| 20 | 20 | ======== |
| 21 | 21 | |
| 22 | −This file describes the iavf Linux* Base Driver. This driver was formerly | |
| 22 | +This file describes the iavf Linux Base Driver. This driver was formerly | |
| 23 | 23 | called i40evf. |
| 24 | 24 | |
| 25 | 25 | The iavf driver supports the below mentioned virtual function devices and |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −=================================================================== | |
| 4 | −Linux* Base Driver for the Intel(R) Ethernet Connection E800 Series | |
| 5 | −=================================================================== | |
| 3 | +================================================================== | |
| 4 | +Linux Base Driver for the Intel(R) Ethernet Connection E800 Series | |
| 5 | +================================================================== | |
| 6 | 6 | |
| 7 | 7 | Intel ice Linux driver. |
| 8 | 8 | Copyright(c) 2018 Intel Corporation. |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −=========================================================== | |
| 4 | −Linux* Base Driver for Intel(R) Ethernet Network Connection | |
| 5 | −=========================================================== | |
| 3 | +========================================================== | |
| 4 | +Linux Base Driver for Intel(R) Ethernet Network Connection | |
| 5 | +========================================================== | |
| 6 | 6 | |
| 7 | 7 | Intel Gigabit Linux driver. |
| 8 | 8 | Copyright(c) 1999-2018 Intel Corporation. |
| @@ -129,9 +129,9 @@ version is required for this functionality. Download it at: · 120 unmodified lines | ||
| 129 | 129 | https://www.kernel.org/pub/software/network/ethtool/ |
| 130 | 130 | |
| 131 | 131 | |
| 132 | −Enabling Wake on LAN* (WoL) | |
| 133 | −--------------------------- | |
| 134 | −WoL is configured through the ethtool* utility. | |
| 132 | +Enabling Wake on LAN (WoL) | |
| 133 | +-------------------------- | |
| 134 | +WoL is configured through the ethtool utility. | |
| 135 | 135 | |
| 136 | 136 | WoL will be enabled on the system during the next shut down or reboot. For |
| 137 | 137 | this driver version, in order to enable WoL, the igb driver must be loaded |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −============================================================ | |
| 4 | −Linux* Base Virtual Function Driver for Intel(R) 1G Ethernet | |
| 5 | −============================================================ | |
| 3 | +=========================================================== | |
| 4 | +Linux Base Virtual Function Driver for Intel(R) 1G Ethernet | |
| 5 | +=========================================================== | |
| 6 | 6 | |
| 7 | 7 | Intel Gigabit Virtual Function Linux driver. |
| 8 | 8 | Copyright(c) 1999-2018 Intel Corporation. |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −============================================================================= | |
| 4 | −Linux* Base Driver for the Intel(R) Ethernet 10 Gigabit PCI Express Adapters | |
| 5 | −============================================================================= | |
| 3 | +=========================================================================== | |
| 4 | +Linux Base Driver for the Intel(R) Ethernet 10 Gigabit PCI Express Adapters | |
| 5 | +=========================================================================== | |
| 6 | 6 | |
| 7 | 7 | Intel 10 Gigabit Linux driver. |
| 8 | 8 | Copyright(c) 1999-2018 Intel Corporation. |
| @@ -519,8 +519,8 @@ The offload is also supported for ixgbe's VFs, but the VF must be set as · 510 unmodified lines | ||
| 519 | 519 | Known Issues/Troubleshooting |
| 520 | 520 | ============================ |
| 521 | 521 | |
| 522 | −Enabling SR-IOV in a 64-bit Microsoft* Windows Server* 2012/R2 guest OS | |
| 523 | −----------------------------------------------------------------------- | |
| 522 | +Enabling SR-IOV in a 64-bit Microsoft Windows Server 2012/R2 guest OS | |
| 523 | +--------------------------------------------------------------------- | |
| 524 | 524 | Linux KVM Hypervisor/VMM supports direct assignment of a PCIe device to a VM. |
| 525 | 525 | This includes traditional PCIe devices, as well as SR-IOV-capable devices based |
| 526 | 526 | on the Intel Ethernet Controller XL710. |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −============================================================= | |
| 4 | −Linux* Base Virtual Function Driver for Intel(R) 10G Ethernet | |
| 5 | −============================================================= | |
| 3 | +============================================================ | |
| 4 | +Linux Base Virtual Function Driver for Intel(R) 10G Ethernet | |
| 5 | +============================================================ | |
| 6 | 6 | |
| 7 | 7 | Intel 10 Gigabit Virtual Function Linux driver. |
| 8 | 8 | Copyright(c) 1999-2018 Intel Corporation. |
| @@ -1,8 +1,8 @@ | ||
| 1 | 1 | .. SPDX-License-Identifier: GPL-2.0+ |
| 2 | 2 | |
| 3 | −========================================================== | |
| 4 | −Linux* Driver for the Pensando(R) Ethernet adapter family | |
| 5 | −========================================================== | |
| 3 | +======================================================== | |
| 4 | +Linux Driver for the Pensando(R) Ethernet adapter family | |
| 5 | +======================================================== | |
| 6 | 6 | |
| 7 | 7 | Pensando Linux Ethernet driver. |
| 8 | 8 | Copyright(c) 2019 Pensando Systems, Inc |
| @@ -36,8 +36,10 @@ Support · 27 unmodified lines | ||
| 36 | 36 | ======= |
| 37 | 37 | For general Linux networking support, please use the netdev mailing |
| 38 | 38 | list, which is monitored by Pensando personnel:: |
| 39 | + | |
| 39 | 40 | netdev@vger.kernel.org |
| 40 | 41 | |
| 41 | 42 | For more specific support needs, please use the Pensando driver support |
| 42 | 43 | email:: |
| 43 | − drivers@pensando.io | |
| 44 | + | |
| 45 | + drivers@pensando.io | |
| @@ -207,8 +207,8 @@ TCP variables: · 206 unmodified lines | ||
| 207 | 207 | |
| 208 | 208 | somaxconn - INTEGER |
| 209 | 209 | Limit of socket listen() backlog, known in userspace as SOMAXCONN. |
| 210 | − Defaults to 128. See also tcp_max_syn_backlog for additional tuning | |
| 211 | − for TCP sockets. | |
| 210 | + Defaults to 4096. (Was 128 before linux-5.4) | |
| 211 | + See also tcp_max_syn_backlog for additional tuning for TCP sockets. | |
| 212 | 212 | |
| 213 | 213 | tcp_abort_on_overflow - BOOLEAN |
| 214 | 214 | If listening service is too slow to accept new connections, |
| @@ -408,11 +408,14 @@ tcp_max_orphans - INTEGER · 193 unmodified lines | ||
| 408 | 408 | up to ~64K of unswappable memory. |
| 409 | 409 | |
| 410 | 410 | tcp_max_syn_backlog - INTEGER |
| 411 | − Maximal number of remembered connection requests, which have not | |
| 412 | − received an acknowledgment from connecting client. | |
| 411 | + Maximal number of remembered connection requests (SYN_RECV), | |
| 412 | + which have not received an acknowledgment from connecting client. | |
| 413 | + This is a per-listener limit. | |
| 413 | 414 | The minimal value is 128 for low memory machines, and it will |
| 414 | 415 | increase in proportion to the memory of machine. |
| 415 | 416 | If server suffers from overload, try increasing this number. |
| 417 | + Remember to also check /proc/sys/net/core/somaxconn | |
| 418 | + A SYN_RECV request socket consumes about 304 bytes of memory. | |
| 416 | 419 | |
| 417 | 420 | tcp_max_tw_buckets - INTEGER |
| 418 | 421 | Maximal number of timewait sockets held by system simultaneously. |
| @@ -92,16 +92,16 @@ under some conditions. · 91 unmodified lines | ||
| 92 | 92 | Part III: Registering a Network Device to DIM |
| 93 | 93 | ============================================== |
| 94 | 94 | |
| 95 | −Net DIM API exposes the main function net_dim(struct net_dim *dim, | |
| 96 | −struct net_dim_sample end_sample). This function is the entry point to the Net | |
| 95 | +Net DIM API exposes the main function net_dim(struct dim *dim, | |
| 96 | +struct dim_sample end_sample). This function is the entry point to the Net | |
| 97 | 97 | DIM algorithm and has to be called every time the driver would like to check if |
| 98 | 98 | it should change interrupt moderation parameters. The driver should provide two |
| 99 | −data structures: struct net_dim and struct net_dim_sample. Struct net_dim | |
| 99 | +data structures: struct dim and struct dim_sample. Struct dim | |
| 100 | 100 | describes the state of DIM for a specific object (RX queue, TX queue, |
| 101 | 101 | other queues, etc.). This includes the current selected profile, previous data |
| 102 | 102 | samples, the callback function provided by the driver and more. |
| 103 | −Struct net_dim_sample describes a data sample, which will be compared to the | |
| 104 | −data sample stored in struct net_dim in order to decide on the algorithm's next | |
| 103 | +Struct dim_sample describes a data sample, which will be compared to the | |
| 104 | +data sample stored in struct dim in order to decide on the algorithm's next | |
| 105 | 105 | step. The sample should include bytes, packets and interrupts, measured by |
| 106 | 106 | the driver. |
| 107 | 107 | |
| @@ -110,9 +110,9 @@ main net_dim() function. The recommended method is to call net_dim() on each · 2 unmodified lines | ||
| 110 | 110 | interrupt. Since Net DIM has a built-in moderation and it might decide to skip |
| 111 | 111 | iterations under certain conditions, there is no need to moderate the net_dim() |
| 112 | 112 | calls as well. As mentioned above, the driver needs to provide an object of type |
| 113 | −struct net_dim to the net_dim() function call. It is advised for each entity | |
| 114 | −using Net DIM to hold a struct net_dim as part of its data structure and use it | |
| 115 | −as the main Net DIM API object. The struct net_dim_sample should hold the latest | |
| 113 | +struct dim to the net_dim() function call. It is advised for each entity | |
| 114 | +using Net DIM to hold a struct dim as part of its data structure and use it | |
| 115 | +as the main Net DIM API object. The struct dim_sample should hold the latest | |
| 116 | 116 | bytes, packets and interrupts count. No need to perform any calculations, just |
| 117 | 117 | include the raw data. |
| 118 | 118 | |
| @@ -132,19 +132,19 @@ usage is not complete but it should make the outline of the usage clear. · 13 unmodified lines | ||
| 132 | 132 | |
| 133 | 133 | my_driver.c: |
| 134 | 134 | |
| 135 | −#include <linux/net_dim.h> | |
| 135 | +#include <linux/dim.h> | |
| 136 | 136 | |
| 137 | 137 | /* Callback for net DIM to schedule on a decision to change moderation */ |
| 138 | 138 | void my_driver_do_dim_work(struct work_struct *work) |
| 139 | 139 | { |
| 140 | − /* Get struct net_dim from struct work_struct */ | |
| 141 | − struct net_dim *dim = container_of(work, struct net_dim, | |
| 142 | − work); | |
| 140 | + /* Get struct dim from struct work_struct */ | |
| 141 | + struct dim *dim = container_of(work, struct dim, | |
| 142 | + work); | |
| 143 | 143 | /* Do interrupt moderation related stuff */ |
| 144 | 144 | ... |
| 145 | 145 | |
| 146 | 146 | /* Signal net DIM work is done and it should move to next iteration */ |
| 147 | − dim->state = NET_DIM_START_MEASURE; | |
| 147 | + dim->state = DIM_START_MEASURE; | |
| 148 | 148 | } |
| 149 | 149 | |
| 150 | 150 | /* My driver's interrupt handler */ |
| @@ -152,13 +152,13 @@ int my_driver_handle_interrupt(struct my_driver_entity *my_entity, ...) · 1 unmodified line | ||
| 152 | 152 | { |
| 153 | 153 | ... |
| 154 | 154 | /* A struct to hold current measured data */ |
| 155 | − struct net_dim_sample dim_sample; | |
| 155 | + struct dim_sample dim_sample; | |
| 156 | 156 | ... |
| 157 | 157 | /* Initiate data sample struct with current data */ |
| 158 | − net_dim_sample(my_entity->events, | |
| 159 | − my_entity->packets, | |
| 160 | − my_entity->bytes, | |
| 161 | − &dim_sample); | |
| 158 | + dim_update_sample(my_entity->events, | |
| 159 | + my_entity->packets, | |
| 160 | + my_entity->bytes, | |
| 161 | + &dim_sample); | |
| 162 | 162 | /* Call net DIM */ |
| 163 | 163 | net_dim(&my_entity->dim, dim_sample); |
| 164 | 164 | ... |
| @@ -436,6 +436,10 @@ by the driver: · 435 unmodified lines | ||
| 436 | 436 | encryption. |
| 437 | 437 | * ``tx_tls_ooo`` - number of TX packets which were part of a TLS stream |
| 438 | 438 | but did not arrive in the expected order. |
| 439 | + * ``tx_tls_skip_no_sync_data`` - number of TX packets which were part of | |
| 440 | + a TLS stream and arrived out-of-order, but skipped the HW offload routine | |
| 441 | + and went to the regular transmit flow as they were retransmissions of the | |
| 442 | + connection handshake. | |
| 439 | 443 | * ``tx_tls_drop_no_sync_data`` - number of TX packets which were part of |
| 440 | 444 | a TLS stream dropped, because they arrived out of order and associated |
| 441 | 445 | record could not be found. |
| @@ -56,7 +56,7 @@ instead of ``double-indenting`` the ``case`` labels. E.g.: · 55 unmodified lines | ||
| 56 | 56 | case 'K': |
| 57 | 57 | case 'k': |
| 58 | 58 | mem <<= 10; |
| 59 | − /* fall through */ | |
| 59 | + fallthrough; | |
| 60 | 60 | default: |
| 61 | 61 | break; |
| 62 | 62 | } |
| @@ -122,14 +122,27 @@ memory adjacent to the stack (when built without `CONFIG_VMAP_STACK=y`) · 121 unmodified lines | ||
| 122 | 122 | |
| 123 | 123 | Implicit switch case fall-through |
| 124 | 124 | --------------------------------- |
| 125 | −The C language allows switch cases to "fall through" when | |
| 126 | −a "break" statement is missing at the end of a case. This, | |
| 127 | −however, introduces ambiguity in the code, as it's not always | |
| 128 | −clear if the missing break is intentional or a bug. As there | |
| 129 | −have been a long list of flaws `due to missing "break" statements | |
| 125 | +The C language allows switch cases to "fall-through" when a "break" statement | |
| 126 | +is missing at the end of a case. This, however, introduces ambiguity in the | |
| 127 | +code, as it's not always clear if the missing break is intentional or a bug. | |
| 128 | + | |
| 129 | +As there have been a long list of flaws `due to missing "break" statements | |
| 130 | 130 | <https://cwe.mitre.org/data/definitions/484.html>`_, we no longer allow |
| 131 | −"implicit fall-through". In order to identify an intentional fall-through | |
| 132 | −case, we have adopted the marking used by static analyzers: a comment | |
| 133 | −saying `/* Fall through */`. Once the C++17 `__attribute__((fallthrough))` | |
| 134 | −is more widely handled by C compilers, static analyzers, and IDEs, we can | |
| 135 | −switch to using that instead. | |
| 131 | +"implicit fall-through". | |
| 132 | + | |
| 133 | +In order to identify intentional fall-through cases, we have adopted a | |
| 134 | +pseudo-keyword macro 'fallthrough' which expands to gcc's extension | |
| 135 | +__attribute__((__fallthrough__)). `Statement Attributes | |
| 136 | +<https://gcc.gnu.org/onlinedocs/gcc/Statement-Attributes.html>`_ | |
| 137 | + | |
| 138 | +When the C17/C18 [[fallthrough]] syntax is more commonly supported by | |
| 139 | +C compilers, static analyzers, and IDEs, we can switch to using that syntax | |
| 140 | +for the macro pseudo-keyword. | |
| 141 | + | |
| 142 | +All switch/case blocks must end in one of: | |
| 143 | + | |
| 144 | + break; | |
| 145 | + fallthrough; | |
| 146 | + continue; | |
| 147 | + goto <label>; | |
| 148 | + return [expression]; | |
| @@ -1,109 +0,0 @@ | ||
| 1 | −============ | |
| 2 | −Diamonds Rio | |
| 3 | −============ | |
| 4 | − | |
| 5 | −Copyright (C) 1999, 2000 Bruce Tenison | |
| 6 | − | |
| 7 | −Portions Copyright (C) 1999, 2000 David Nelson | |
| 8 | − | |
| 9 | −Thanks to David Nelson for guidance and the usage of the scanner.txt | |
| 10 | −and scanner.c files to model our driver and this informative file. | |
| 11 | − | |
| 12 | −Mar. 2, 2000 | |
| 13 | − | |
| 14 | −Changes | |
| 15 | −======= | |
| 16 | − | |
| 17 | −- Initial Revision | |
| 18 | − | |
| 19 | − | |
| 20 | −Overview | |
| 21 | −======== | |
| 22 | − | |
| 23 | −This README will address issues regarding how to configure the kernel | |
| 24 | −to access a RIO 500 mp3 player. | |
| 25 | −Before I explain how to use this to access the Rio500 please be warned: | |
| 26 | − | |
| 27 | −.. warning:: | |
| 28 | − | |
| 29 | − Please note that this software is still under development. The authors | |
| 30 | − are in no way responsible for any damage that may occur, no matter how | |
| 31 | − inconsequential. | |
| 32 | − | |
| 33 | −It seems that the Rio has a problem when sending .mp3 with low batteries. | |
| 34 | −I suggest when the batteries are low and you want to transfer stuff that you | |
| 35 | −replace it with a fresh one. In my case, what happened is I lost two 16kb | |
| 36 | −blocks (they are no longer usable to store information to it). But I don't | |
| 37 | −know if that's normal or not; it could simply be a problem with the flash | |
| 38 | −memory. | |
| 39 | − | |
| 40 | −In an extreme case, I left my Rio playing overnight and the batteries wore | |
| 41 | −down to nothing and appear to have corrupted the flash memory. My RIO | |
| 42 | −needed to be replaced as a result. Diamond tech support is aware of the | |
| 43 | −problem. Do NOT allow your batteries to wear down to nothing before | |
| 44 | −changing them. It appears RIO 500 firmware does not handle low battery | |
| 45 | −power well at all. | |
| 46 | − | |
| 47 | −On systems with OHCI controllers, the kernel OHCI code appears to have | |
| 48 | −power on problems with some chipsets. If you are having problems | |
| 49 | −connecting to your RIO 500, try turning it on first and then plugging it | |
| 50 | −into the USB cable. | |
| 51 | − | |
| 52 | −Contact Information | |
| 53 | −------------------- | |
| 54 | − | |
| 55 | − The main page for the project is hosted at sourceforge.net in the following | |
| 56 | − URL: <http://rio500.sourceforge.net>. You can also go to the project's | |
| 57 | − sourceforge home page at: <http://sourceforge.net/projects/rio500/>. | |
| 58 | − There is also a mailing list: rio500-users@lists.sourceforge.net | |
| 59 | − | |
| 60 | −Authors | |
| 61 | −------- | |
| 62 | − | |
| 63 | −Most of the code was written by Cesar Miquel <miquel@df.uba.ar>. Keith | |
| 64 | −Clayton <kclayton@jps.net> is incharge of the PPC port and making sure | |
| 65 | −things work there. Bruce Tenison <btenison@dibbs.net> is adding support | |
| 66 | −for .fon files and also does testing. The program will mostly sure be | |
| 67 | −re-written and Pete Ikusz along with the rest will re-design it. I would | |
| 68 | −also like to thank Tri Nguyen <tmn_3022000@hotmail.com> who provided use | |
| 69 | −with some important information regarding the communication with the Rio. | |
| 70 | − | |
| 71 | −Additional Information and userspace tools | |
| 72 | − | |
| 73 | − http://rio500.sourceforge.net/ | |
| 74 | − | |
| 75 | − | |
| 76 | −Requirements | |
| 77 | −============ | |
| 78 | − | |
| 79 | −A host with a USB port running a Linux kernel with RIO 500 support enabled. | |
| 80 | − | |
| 81 | −The driver is a module called rio500, which should be automatically loaded | |
| 82 | −as you plug in your device. If that fails you can manually load it with | |
| 83 | − | |
| 84 | − modprobe rio500 | |
| 85 | − | |
| 86 | −Udev should automatically create a device node as soon as plug in your device. | |
| 87 | −If that fails, you can manually add a device for the USB rio500:: | |
| 88 | − | |
| 89 | − mknod /dev/usb/rio500 c 180 64 | |
| 90 | − | |
| 91 | −In that case, set appropriate permissions for /dev/usb/rio500 (don't forget | |
| 92 | −about group and world permissions). Both read and write permissions are | |
| 93 | −required for proper operation. | |
| 94 | − | |
| 95 | −That's it. The Rio500 Utils at: http://rio500.sourceforge.net should | |
| 96 | −be able to access the rio500. | |
| 97 | − | |
| 98 | −Limits | |
| 99 | −====== | |
| 100 | − | |
| 101 | −You can use only a single rio500 device at a time with your computer. | |
| 102 | − | |
| 103 | −Bugs | |
| 104 | −==== | |
| 105 | − | |
| 106 | −If you encounter any problems feel free to drop me an email. | |
| 107 | − | |
| 108 | −Bruce Tenison | |
| 109 | −btenison@dibbs.net | |
| @@ -1251,6 +1251,7 @@ F: Documentation/devicetree/bindings/display/arm,hdlcd.txt · 1250 unmodified lines | ||
| 1251 | 1251 | ARM KOMEDA DRM-KMS DRIVER |
| 1252 | 1252 | M: James (Qian) Wang <james.qian.wang@arm.com> |
| 1253 | 1253 | M: Liviu Dudau <liviu.dudau@arm.com> |
| 1254 | +M: Mihail Atanassov <mihail.atanassov@arm.com> | |
| 1254 | 1255 | L: Mali DP Maintainers <malidp@foss.arm.com> |
| 1255 | 1256 | S: Supported |
| 1256 | 1257 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| @@ -2167,12 +2168,10 @@ F: arch/arm64/boot/dts/realtek/ · 910 unmodified lines | ||
| 2167 | 2168 | F: Documentation/devicetree/bindings/arm/realtek.yaml |
| 2168 | 2169 | |
| 2169 | 2170 | ARM/RENESAS ARM64 ARCHITECTURE |
| 2170 | −M: Simon Horman <horms@verge.net.au> | |
| 2171 | 2171 | M: Geert Uytterhoeven <geert+renesas@glider.be> |
| 2172 | 2172 | M: Magnus Damm <magnus.damm@gmail.com> |
| 2173 | 2173 | L: linux-renesas-soc@vger.kernel.org |
| 2174 | 2174 | Q: http://patchwork.kernel.org/project/linux-renesas-soc/list/ |
| 2175 | −T: git git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas.git next | |
| 2176 | 2175 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/geert/renesas-devel.git next |
| 2177 | 2176 | S: Supported |
| 2178 | 2177 | F: arch/arm64/boot/dts/renesas/ |
| @@ -2284,12 +2283,10 @@ S: Maintained · 105 unmodified lines | ||
| 2284 | 2283 | F: drivers/media/platform/s5p-mfc/ |
| 2285 | 2284 | |
| 2286 | 2285 | ARM/SHMOBILE ARM ARCHITECTURE |
| 2287 | −M: Simon Horman <horms@verge.net.au> | |
| 2288 | 2286 | M: Geert Uytterhoeven <geert+renesas@glider.be> |
| 2289 | 2287 | M: Magnus Damm <magnus.damm@gmail.com> |
| 2290 | 2288 | L: linux-renesas-soc@vger.kernel.org |
| 2291 | 2289 | Q: http://patchwork.kernel.org/project/linux-renesas-soc/list/ |
| 2292 | −T: git git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas.git next | |
| 2293 | 2290 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/geert/renesas-devel.git next |
| 2294 | 2291 | S: Supported |
| 2295 | 2292 | F: arch/arm/boot/dts/emev2* |
| @@ -2329,11 +2326,13 @@ F: drivers/edac/altera_edac. · 33 unmodified lines | ||
| 2329 | 2326 | |
| 2330 | 2327 | ARM/SPREADTRUM SoC SUPPORT |
| 2331 | 2328 | M: Orson Zhai <orsonzhai@gmail.com> |
| 2332 | −M: Baolin Wang <baolin.wang@linaro.org> | |
| 2329 | +M: Baolin Wang <baolin.wang7@gmail.com> | |
| 2333 | 2330 | M: Chunyan Zhang <zhang.lyra@gmail.com> |
| 2334 | 2331 | S: Maintained |
| 2335 | 2332 | F: arch/arm64/boot/dts/sprd |
| 2336 | 2333 | N: sprd |
| 2334 | +N: sc27xx | |
| 2335 | +N: sc2731 | |
| 2337 | 2336 | |
| 2338 | 2337 | ARM/STI ARCHITECTURE |
| 2339 | 2338 | M: Patrice Chotard <patrice.chotard@st.com> |
| @@ -3057,6 +3056,7 @@ M: Daniel Borkmann <daniel@iogearbox.net> · 717 unmodified lines | ||
| 3057 | 3056 | R: Martin KaFai Lau <kafai@fb.com> |
| 3058 | 3057 | R: Song Liu <songliubraving@fb.com> |
| 3059 | 3058 | R: Yonghong Song <yhs@fb.com> |
| 3059 | +R: Andrii Nakryiko <andriin@fb.com> | |
| 3060 | 3060 | L: netdev@vger.kernel.org |
| 3061 | 3061 | L: bpf@vger.kernel.org |
| 3062 | 3062 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git |
| @@ -3102,7 +3102,7 @@ S: Supported · 39 unmodified lines | ||
| 3102 | 3102 | F: arch/arm64/net/ |
| 3103 | 3103 | |
| 3104 | 3104 | BPF JIT for MIPS (32-BIT AND 64-BIT) |
| 3105 | −M: Paul Burton <paul.burton@mips.com> | |
| 3105 | +M: Paul Burton <paulburton@kernel.org> | |
| 3106 | 3106 | L: netdev@vger.kernel.org |
| 3107 | 3107 | L: bpf@vger.kernel.org |
| 3108 | 3108 | S: Maintained |
| @@ -3189,7 +3189,7 @@ N: bcm216* · 80 unmodified lines | ||
| 3189 | 3189 | N: kona |
| 3190 | 3190 | F: arch/arm/mach-bcm/ |
| 3191 | 3191 | |
| 3192 | −BROADCOM BCM2835 ARM ARCHITECTURE | |
| 3192 | +BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE | |
| 3193 | 3193 | M: Eric Anholt <eric@anholt.net> |
| 3194 | 3194 | M: Stefan Wahren <wahrenst@gmx.net> |
| 3195 | 3195 | L: bcm-kernel-feedback-list@broadcom.com |
| @@ -3197,6 +3197,7 @@ L: linux-rpi-kernel@lists.infradead.org (moderated for non-subscribers) · 1 unmodified line | ||
| 3197 | 3197 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 3198 | 3198 | T: git git://github.com/anholt/linux |
| 3199 | 3199 | S: Maintained |
| 3200 | +N: bcm2711 | |
| 3200 | 3201 | N: bcm2835 |
| 3201 | 3202 | F: drivers/staging/vc04_services |
| 3202 | 3203 | |
| @@ -3243,8 +3244,6 @@ S: Maintained · 40 unmodified lines | ||
| 3243 | 3244 | F: drivers/usb/gadget/udc/bcm63xx_udc.* |
| 3244 | 3245 | |
| 3245 | 3246 | BROADCOM BCM7XXX ARM ARCHITECTURE |
| 3246 | −M: Brian Norris <computersforpeace@gmail.com> | |
| 3247 | −M: Gregory Fong <gregory.0xf0@gmail.com> | |
| 3248 | 3247 | M: Florian Fainelli <f.fainelli@gmail.com> |
| 3249 | 3248 | M: bcm-kernel-feedback-list@broadcom.com |
| 3250 | 3249 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| @@ -3742,7 +3741,6 @@ F: drivers/crypto/cavium/cpt/ · 491 unmodified lines | ||
| 3742 | 3741 | |
| 3743 | 3742 | CAVIUM THUNDERX2 ARM64 SOC |
| 3744 | 3743 | M: Robert Richter <rrichter@cavium.com> |
| 3745 | −M: Jayachandran C <jnair@caviumnetworks.com> | |
| 3746 | 3744 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 3747 | 3745 | S: Maintained |
| 3748 | 3746 | F: arch/arm64/boot/dts/cavium/thunder2-99xx* |
| @@ -8017,7 +8015,7 @@ S: Maintained · 4268 unmodified lines | ||
| 8017 | 8015 | F: drivers/usb/atm/ueagle-atm.c |
| 8018 | 8016 | |
| 8019 | 8017 | IMGTEC ASCII LCD DRIVER |
| 8020 | −M: Paul Burton <paul.burton@mips.com> | |
| 8018 | +M: Paul Burton <paulburton@kernel.org> | |
| 8021 | 8019 | S: Maintained |
| 8022 | 8020 | F: Documentation/devicetree/bindings/auxdisplay/img-ascii-lcd.txt |
| 8023 | 8021 | F: drivers/auxdisplay/img-ascii-lcd.c |
| @@ -9138,7 +9136,7 @@ F: drivers/auxdisplay/ks0108.c · 1114 unmodified lines | ||
| 9138 | 9136 | F: include/linux/ks0108.h |
| 9139 | 9137 | |
| 9140 | 9138 | L3MDEV |
| 9141 | −M: David Ahern <dsa@cumulusnetworks.com> | |
| 9139 | +M: David Ahern <dsahern@kernel.org> | |
| 9142 | 9140 | L: netdev@vger.kernel.org |
| 9143 | 9141 | S: Maintained |
| 9144 | 9142 | F: net/l3mdev |
| @@ -9199,6 +9197,7 @@ M: Pavel Machek <pavel@ucw.cz> · 54 unmodified lines | ||
| 9199 | 9197 | R: Dan Murphy <dmurphy@ti.com> |
| 9200 | 9198 | L: linux-leds@vger.kernel.org |
| 9201 | 9199 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/j.anaszewski/linux-leds.git |
| 9200 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/pavel/linux-leds.git | |
| 9202 | 9201 | S: Maintained |
| 9203 | 9202 | F: Documentation/devicetree/bindings/leds/ |
| 9204 | 9203 | F: drivers/leds/ |
| @@ -10270,7 +10269,7 @@ MEDIATEK ETHERNET DRIVER · 1065 unmodified lines | ||
| 10270 | 10269 | M: Felix Fietkau <nbd@openwrt.org> |
| 10271 | 10270 | M: John Crispin <john@phrozen.org> |
| 10272 | 10271 | M: Sean Wang <sean.wang@mediatek.com> |
| 10273 | −M: Nelson Chang <nelson.chang@mediatek.com> | |
| 10272 | +M: Mark Lee <Mark-MC.Lee@mediatek.com> | |
| 10274 | 10273 | L: netdev@vger.kernel.org |
| 10275 | 10274 | S: Maintained |
| 10276 | 10275 | F: drivers/net/ethernet/mediatek/ |
| @@ -10533,8 +10532,12 @@ F: mm/memblock.c · 256 unmodified lines | ||
| 10533 | 10532 | F: Documentation/core-api/boot-time-mm.rst |
| 10534 | 10533 | |
| 10535 | 10534 | MEMORY MANAGEMENT |
| 10535 | +M: Andrew Morton <akpm@linux-foundation.org> | |
| 10536 | 10536 | L: linux-mm@kvack.org |
| 10537 | 10537 | W: http://www.linux-mm.org |
| 10538 | +T: quilt https://ozlabs.org/~akpm/mmotm/ | |
| 10539 | +T: quilt https://ozlabs.org/~akpm/mmots/ | |
| 10540 | +T: git git://github.com/hnaz/linux-mm.git | |
| 10538 | 10541 | S: Maintained |
| 10539 | 10542 | F: include/linux/mm.h |
| 10540 | 10543 | F: include/linux/gfp.h |
| @@ -10843,7 +10846,7 @@ F: drivers/usb/image/microtek.* · 302 unmodified lines | ||
| 10843 | 10846 | |
| 10844 | 10847 | MIPS |
| 10845 | 10848 | M: Ralf Baechle <ralf@linux-mips.org> |
| 10846 | −M: Paul Burton <paul.burton@mips.com> | |
| 10849 | +M: Paul Burton <paulburton@kernel.org> | |
| 10847 | 10850 | M: James Hogan <jhogan@kernel.org> |
| 10848 | 10851 | L: linux-mips@vger.kernel.org |
| 10849 | 10852 | W: http://www.linux-mips.org/ |
| @@ -10857,7 +10860,7 @@ F: arch/mips/ · 7 unmodified lines | ||
| 10857 | 10860 | F: drivers/platform/mips/ |
| 10858 | 10861 | |
| 10859 | 10862 | MIPS BOSTON DEVELOPMENT BOARD |
| 10860 | −M: Paul Burton <paul.burton@mips.com> | |
| 10863 | +M: Paul Burton <paulburton@kernel.org> | |
| 10861 | 10864 | L: linux-mips@vger.kernel.org |
| 10862 | 10865 | S: Maintained |
| 10863 | 10866 | F: Documentation/devicetree/bindings/clock/img,boston-clock.txt |
| @@ -10867,7 +10870,7 @@ F: drivers/clk/imgtec/clk-boston.c · 3 unmodified lines | ||
| 10867 | 10870 | F: include/dt-bindings/clock/boston-clock.h |
| 10868 | 10871 | |
| 10869 | 10872 | MIPS GENERIC PLATFORM |
| 10870 | −M: Paul Burton <paul.burton@mips.com> | |
| 10873 | +M: Paul Burton <paulburton@kernel.org> | |
| 10871 | 10874 | L: linux-mips@vger.kernel.org |
| 10872 | 10875 | S: Supported |
| 10873 | 10876 | F: Documentation/devicetree/bindings/power/mti,mips-cpc.txt |
| @@ -11422,7 +11425,6 @@ F: include/trace/events/tcp.h · 548 unmodified lines | ||
| 11422 | 11425 | NETWORKING [TLS] |
| 11423 | 11426 | M: Boris Pismenny <borisp@mellanox.com> |
| 11424 | 11427 | M: Aviad Yehezkel <aviadye@mellanox.com> |
| 11425 | −M: Dave Watson <davejwatson@fb.com> | |
| 11426 | 11428 | M: John Fastabend <john.fastabend@gmail.com> |
| 11427 | 11429 | M: Daniel Borkmann <daniel@iogearbox.net> |
| 11428 | 11430 | M: Jakub Kicinski <jakub.kicinski@netronome.com> |
| @@ -11559,6 +11561,7 @@ NSDEPS · 130 unmodified lines | ||
| 11559 | 11561 | M: Matthias Maennich <maennich@google.com> |
| 11560 | 11562 | S: Maintained |
| 11561 | 11563 | F: scripts/nsdeps |
| 11564 | +F: Documentation/core-api/symbol-namespaces.rst | |
| 11562 | 11565 | |
| 11563 | 11566 | NTB AMD DRIVER |
| 11564 | 11567 | M: Shyam Sundar S K <Shyam-sundar.S-k@amd.com> |
| @@ -12326,12 +12329,15 @@ F: arch/parisc/ · 761 unmodified lines | ||
| 12326 | 12329 | F: Documentation/parisc/ |
| 12327 | 12330 | F: drivers/parisc/ |
| 12328 | 12331 | F: drivers/char/agp/parisc-agp.c |
| 12332 | +F: drivers/input/misc/hp_sdc_rtc.c | |
| 12329 | 12333 | F: drivers/input/serio/gscps2.c |
| 12334 | +F: drivers/input/serio/hp_sdc* | |
| 12330 | 12335 | F: drivers/parport/parport_gsc.* |
| 12331 | 12336 | F: drivers/tty/serial/8250/8250_gsc.c |
| 12332 | 12337 | F: drivers/video/fbdev/sti* |
| 12333 | 12338 | F: drivers/video/console/sti* |
| 12334 | 12339 | F: drivers/video/logo/logo_parisc* |
| 12340 | +F: include/linux/hp_sdc.h | |
| 12335 | 12341 | |
| 12336 | 12342 | PARMAN |
| 12337 | 12343 | M: Jiri Pirko <jiri@mellanox.com> |
| @@ -13375,7 +13381,7 @@ S: Maintained · 1037 unmodified lines | ||
| 13375 | 13381 | F: drivers/scsi/qla1280.[ch] |
| 13376 | 13382 | |
| 13377 | 13383 | QLOGIC QLA2XXX FC-SCSI DRIVER |
| 13378 | −M: qla2xxx-upstream@qlogic.com | |
| 13384 | +M: hmadhani@marvell.com | |
| 13379 | 13385 | L: linux-scsi@vger.kernel.org |
| 13380 | 13386 | S: Supported |
| 13381 | 13387 | F: Documentation/scsi/LICENSE.qla2xxx |
| @@ -13916,7 +13922,7 @@ F: drivers/mtd/nand/raw/r852.h · 534 unmodified lines | ||
| 13916 | 13922 | |
| 13917 | 13923 | RISC-V ARCHITECTURE |
| 13918 | 13924 | M: Paul Walmsley <paul.walmsley@sifive.com> |
| 13919 | −M: Palmer Dabbelt <palmer@sifive.com> | |
| 13925 | +M: Palmer Dabbelt <palmer@dabbelt.com> | |
| 13920 | 13926 | M: Albert Ou <aou@eecs.berkeley.edu> |
| 13921 | 13927 | L: linux-riscv@lists.infradead.org |
| 13922 | 13928 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux.git |
| @@ -14793,7 +14799,7 @@ F: drivers/media/usb/siano/ · 870 unmodified lines | ||
| 14793 | 14799 | F: drivers/media/mmc/siano/ |
| 14794 | 14800 | |
| 14795 | 14801 | SIFIVE DRIVERS |
| 14796 | −M: Palmer Dabbelt <palmer@sifive.com> | |
| 14802 | +M: Palmer Dabbelt <palmer@dabbelt.com> | |
| 14797 | 14803 | M: Paul Walmsley <paul.walmsley@sifive.com> |
| 14798 | 14804 | L: linux-riscv@lists.infradead.org |
| 14799 | 14805 | T: git git://github.com/sifive/riscv-linux.git |
| @@ -14803,7 +14809,7 @@ N: sifive · 3 unmodified lines | ||
| 14803 | 14809 | |
| 14804 | 14810 | SIFIVE FU540 SYSTEM-ON-CHIP |
| 14805 | 14811 | M: Paul Walmsley <paul.walmsley@sifive.com> |
| 14806 | −M: Palmer Dabbelt <palmer@sifive.com> | |
| 14812 | +M: Palmer Dabbelt <palmer@dabbelt.com> | |
| 14807 | 14813 | L: linux-riscv@lists.infradead.org |
| 14808 | 14814 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/pjw/sifive.git |
| 14809 | 14815 | S: Supported |
| @@ -16778,13 +16784,6 @@ W: http://www.linux-usb.org/usbnet · 1968 unmodified lines | ||
| 16778 | 16784 | S: Maintained |
| 16779 | 16785 | F: drivers/net/usb/dm9601.c |
| 16780 | 16786 | |
| 16781 | −USB DIAMOND RIO500 DRIVER | |
| 16782 | −M: Cesar Miquel <miquel@df.uba.ar> | |
| 16783 | −L: rio500-users@lists.sourceforge.net | |
| 16784 | −W: http://rio500.sourceforge.net | |
| 16785 | −S: Maintained | |
| 16786 | −F: drivers/usb/misc/rio500* | |
| 16787 | − | |
| 16788 | 16787 | USB EHCI DRIVER |
| 16789 | 16788 | M: Alan Stern <stern@rowland.harvard.edu> |
| 16790 | 16789 | L: linux-usb@vger.kernel.org |
| @@ -17352,6 +17351,12 @@ F: include/linux/vbox_utils.h · 561 unmodified lines | ||
| 17352 | 17351 | F: include/uapi/linux/vbox*.h |
| 17353 | 17352 | F: drivers/virt/vboxguest/ |
| 17354 | 17353 | |
| 17354 | +VIRTUAL BOX SHARED FOLDER VFS DRIVER: | |
| 17355 | +M: Hans de Goede <hdegoede@redhat.com> | |
| 17356 | +L: linux-fsdevel@vger.kernel.org | |
| 17357 | +S: Maintained | |
| 17358 | +F: drivers/staging/vboxsf/* | |
| 17359 | + | |
| 17355 | 17360 | VIRTUAL SERIO DEVICE DRIVER |
| 17356 | 17361 | M: Stephen Chandler Paul <thatslyude@gmail.com> |
| 17357 | 17362 | S: Maintained |
| @@ -17451,7 +17456,7 @@ F: include/linux/regulator/ · 93 unmodified lines | ||
| 17451 | 17456 | K: regulator_get_optional |
| 17452 | 17457 | |
| 17453 | 17458 | VRF |
| 17454 | −M: David Ahern <dsa@cumulusnetworks.com> | |
| 17459 | +M: David Ahern <dsahern@kernel.org> | |
| 17455 | 17460 | M: Shrijeet Mukherjee <shrijeet@gmail.com> |
| 17456 | 17461 | L: netdev@vger.kernel.org |
| 17457 | 17462 | S: Maintained |
| @@ -18052,6 +18057,7 @@ F: Documentation/vm/zsmalloc.rst · 594 unmodified lines | ||
| 18052 | 18057 | ZSWAP COMPRESSED SWAP CACHING |
| 18053 | 18058 | M: Seth Jennings <sjenning@redhat.com> |
| 18054 | 18059 | M: Dan Streetman <ddstreet@ieee.org> |
| 18060 | +M: Vitaly Wool <vitaly.wool@konsulko.com> | |
| 18055 | 18061 | L: linux-mm@kvack.org |
| 18056 | 18062 | S: Maintained |
| 18057 | 18063 | F: mm/zswap.c |
| @@ -2,8 +2,8 @@ · 1 unmodified line | ||
| 2 | 2 | VERSION = 5 |
| 3 | 3 | PATCHLEVEL = 4 |
| 4 | 4 | SUBLEVEL = 0 |
| 5 | −EXTRAVERSION = -rc2 | |
| 6 | −NAME = Nesting Opossum | |
| 5 | +EXTRAVERSION = -rc7 | |
| 6 | +NAME = Kleptomaniac Octopus | |
| 7 | 7 | |
| 8 | 8 | # *DOCUMENTATION* |
| 9 | 9 | # To see a list of typical targets execute "make help" |
| @@ -599,7 +599,7 @@ endif · 589 unmodified lines | ||
| 599 | 599 | # in addition to whatever we do anyway. |
| 600 | 600 | # Just "make" or "make all" shall build modules as well |
| 601 | 601 | |
| 602 | −ifneq ($(filter all _all modules,$(MAKECMDGOALS)),) | |
| 602 | +ifneq ($(filter all _all modules nsdeps,$(MAKECMDGOALS)),) | |
| 603 | 603 | KBUILD_MODULES := 1 |
| 604 | 604 | endif |
| 605 | 605 | |
| @@ -1037,7 +1037,7 @@ export KBUILD_VMLINUX_OBJS := $(head-y) $(init-y) $(core-y) $(libs-y2) \ · 431 unmodified lines | ||
| 1037 | 1037 | export KBUILD_VMLINUX_LIBS := $(libs-y1) |
| 1038 | 1038 | export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds |
| 1039 | 1039 | export LDFLAGS_vmlinux |
| 1040 | −# used by scripts/package/Makefile | |
| 1040 | +# used by scripts/Makefile.package | |
| 1041 | 1041 | export KBUILD_ALLDIRS := $(sort $(filter-out arch/%,$(vmlinux-alldirs)) LICENSES arch include scripts tools) |
| 1042 | 1042 | |
| 1043 | 1043 | vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_OBJS) $(KBUILD_VMLINUX_LIBS) |
| @@ -1217,9 +1217,8 @@ PHONY += kselftest · 173 unmodified lines | ||
| 1217 | 1217 | kselftest: |
| 1218 | 1218 | $(Q)$(MAKE) -C $(srctree)/tools/testing/selftests run_tests |
| 1219 | 1219 | |
| 1220 | −PHONY += kselftest-clean | |
| 1221 | −kselftest-clean: | |
| 1222 | − $(Q)$(MAKE) -C $(srctree)/tools/testing/selftests clean | |
| 1220 | +kselftest-%: FORCE | |
| 1221 | + $(Q)$(MAKE) -C $(srctree)/tools/testing/selftests $* | |
| 1223 | 1222 | |
| 1224 | 1223 | PHONY += kselftest-merge |
| 1225 | 1224 | kselftest-merge: |
| @@ -65,6 +65,14 @@ · 64 unmodified lines | ||
| 65 | 65 | clock-frequency = <33333333>; |
| 66 | 66 | }; |
| 67 | 67 | |
| 68 | + reg_5v0: regulator-5v0 { | |
| 69 | + compatible = "regulator-fixed"; | |
| 70 | + | |
| 71 | + regulator-name = "5v0-supply"; | |
| 72 | + regulator-min-microvolt = <5000000>; | |
| 73 | + regulator-max-microvolt = <5000000>; | |
| 74 | + }; | |
| 75 | + | |
| 68 | 76 | cpu_intc: cpu-interrupt-controller { |
| 69 | 77 | compatible = "snps,archs-intc"; |
| 70 | 78 | interrupt-controller; |
| @@ -264,6 +272,21 @@ · 193 unmodified lines | ||
| 264 | 272 | clocks = <&input_clk>; |
| 265 | 273 | cs-gpios = <&creg_gpio 0 GPIO_ACTIVE_LOW>, |
| 266 | 274 | <&creg_gpio 1 GPIO_ACTIVE_LOW>; |
| 275 | + | |
| 276 | + spi-flash@0 { | |
| 277 | + compatible = "sst26wf016b", "jedec,spi-nor"; | |
| 278 | + reg = <0>; | |
| 279 | + #address-cells = <1>; | |
| 280 | + #size-cells = <1>; | |
| 281 | + spi-max-frequency = <4000000>; | |
| 282 | + }; | |
| 283 | + | |
| 284 | + adc@1 { | |
| 285 | + compatible = "ti,adc108s102"; | |
| 286 | + reg = <1>; | |
| 287 | + vref-supply = <®_5v0>; | |
| 288 | + spi-max-frequency = <1000000>; | |
| 289 | + }; | |
| 267 | 290 | }; |
| 268 | 291 | |
| 269 | 292 | creg_gpio: gpio@14b0 { |
| @@ -32,6 +32,8 @@ CONFIG_INET=y · 31 unmodified lines | ||
| 32 | 32 | CONFIG_DEVTMPFS=y |
| 33 | 33 | # CONFIG_STANDALONE is not set |
| 34 | 34 | # CONFIG_PREVENT_FIRMWARE_BUILD is not set |
| 35 | +CONFIG_MTD=y | |
| 36 | +CONFIG_MTD_SPI_NOR=y | |
| 35 | 37 | CONFIG_SCSI=y |
| 36 | 38 | CONFIG_BLK_DEV_SD=y |
| 37 | 39 | CONFIG_NETDEVICES=y |
| @@ -55,6 +57,8 @@ CONFIG_GPIO_SYSFS=y · 17 unmodified lines | ||
| 55 | 57 | CONFIG_GPIO_DWAPB=y |
| 56 | 58 | CONFIG_GPIO_SNPS_CREG=y |
| 57 | 59 | # CONFIG_HWMON is not set |
| 60 | +CONFIG_REGULATOR=y | |
| 61 | +CONFIG_REGULATOR_FIXED_VOLTAGE=y | |
| 58 | 62 | CONFIG_DRM=y |
| 59 | 63 | # CONFIG_DRM_FBDEV_EMULATION is not set |
| 60 | 64 | CONFIG_DRM_UDL=y |
| @@ -72,6 +76,8 @@ CONFIG_MMC_SDHCI_PLTFM=y · 11 unmodified lines | ||
| 72 | 76 | CONFIG_MMC_DW=y |
| 73 | 77 | CONFIG_DMADEVICES=y |
| 74 | 78 | CONFIG_DW_AXI_DMAC=y |
| 79 | +CONFIG_IIO=y | |
| 80 | +CONFIG_TI_ADC108S102=y | |
| 75 | 81 | CONFIG_EXT3_FS=y |
| 76 | 82 | CONFIG_VFAT_FS=y |
| 77 | 83 | CONFIG_TMPFS=y |
| @@ -614,8 +614,8 @@ static int arc_pmu_device_probe(struct platform_device *pdev) · 613 unmodified lines | ||
| 614 | 614 | /* loop thru all available h/w condition indexes */ |
| 615 | 615 | for (i = 0; i < cc_bcr.c; i++) { |
| 616 | 616 | write_aux_reg(ARC_REG_CC_INDEX, i); |
| 617 | − cc_name.indiv.word0 = read_aux_reg(ARC_REG_CC_NAME0); | |
| 618 | − cc_name.indiv.word1 = read_aux_reg(ARC_REG_CC_NAME1); | |
| 617 | + cc_name.indiv.word0 = le32_to_cpu(read_aux_reg(ARC_REG_CC_NAME0)); | |
| 618 | + cc_name.indiv.word1 = le32_to_cpu(read_aux_reg(ARC_REG_CC_NAME1)); | |
| 619 | 619 | |
| 620 | 620 | arc_pmu_map_hw_event(i, cc_name.str); |
| 621 | 621 | arc_pmu_add_raw_event_attr(i, cc_name.str); |
| @@ -111,13 +111,13 @@ · 110 unmodified lines | ||
| 111 | 111 | reg = <0x70>; |
| 112 | 112 | #address-cells = <1>; |
| 113 | 113 | #size-cells = <0>; |
| 114 | + i2c-mux-idle-disconnect; | |
| 114 | 115 | |
| 115 | 116 | i2c@0 { |
| 116 | 117 | /* FMC A */ |
| 117 | 118 | #address-cells = <1>; |
| 118 | 119 | #size-cells = <0>; |
| 119 | 120 | reg = <0>; |
| 120 | − i2c-mux-idle-disconnect; | |
| 121 | 121 | }; |
| 122 | 122 | |
| 123 | 123 | i2c@1 { |
| @@ -125,7 +125,6 @@ · 1 unmodified line | ||
| 125 | 125 | #address-cells = <1>; |
| 126 | 126 | #size-cells = <0>; |
| 127 | 127 | reg = <1>; |
| 128 | − i2c-mux-idle-disconnect; | |
| 129 | 128 | }; |
| 130 | 129 | |
| 131 | 130 | i2c@2 { |
| @@ -133,7 +132,6 @@ · 1 unmodified line | ||
| 133 | 132 | #address-cells = <1>; |
| 134 | 133 | #size-cells = <0>; |
| 135 | 134 | reg = <2>; |
| 136 | − i2c-mux-idle-disconnect; | |
| 137 | 135 | }; |
| 138 | 136 | |
| 139 | 137 | i2c@3 { |
| @@ -141,7 +139,6 @@ · 1 unmodified line | ||
| 141 | 139 | #address-cells = <1>; |
| 142 | 140 | #size-cells = <0>; |
| 143 | 141 | reg = <3>; |
| 144 | − i2c-mux-idle-disconnect; | |
| 145 | 142 | }; |
| 146 | 143 | |
| 147 | 144 | i2c@4 { |
| @@ -149,14 +146,12 @@ · 1 unmodified line | ||
| 149 | 146 | #address-cells = <1>; |
| 150 | 147 | #size-cells = <0>; |
| 151 | 148 | reg = <4>; |
| 152 | − i2c-mux-idle-disconnect; | |
| 153 | 149 | }; |
| 154 | 150 | |
| 155 | 151 | i2c@5 { |
| 156 | 152 | #address-cells = <1>; |
| 157 | 153 | #size-cells = <0>; |
| 158 | 154 | reg = <5>; |
| 159 | − i2c-mux-idle-disconnect; | |
| 160 | 155 | |
| 161 | 156 | ina230@40 { compatible = "ti,ina230"; reg = <0x40>; shunt-resistor = <5000>; }; |
| 162 | 157 | ina230@41 { compatible = "ti,ina230"; reg = <0x41>; shunt-resistor = <5000>; }; |
| @@ -182,14 +177,12 @@ · 19 unmodified lines | ||
| 182 | 177 | #address-cells = <1>; |
| 183 | 178 | #size-cells = <0>; |
| 184 | 179 | reg = <6>; |
| 185 | − i2c-mux-idle-disconnect; | |
| 186 | 180 | }; |
| 187 | 181 | |
| 188 | 182 | i2c@7 { |
| 189 | 183 | #address-cells = <1>; |
| 190 | 184 | #size-cells = <0>; |
| 191 | 185 | reg = <7>; |
| 192 | − i2c-mux-idle-disconnect; | |
| 193 | 186 | |
| 194 | 187 | u41: pca9575@20 { |
| 195 | 188 | compatible = "nxp,pca9575"; |
| @@ -113,6 +113,7 @@ · 112 unmodified lines | ||
| 113 | 113 | #address-cells = <1>; |
| 114 | 114 | #size-cells = <0>; |
| 115 | 115 | pinctrl-0 = <&emmc_gpio34 &gpclk2_gpio43>; |
| 116 | + bus-width = <4>; | |
| 116 | 117 | mmc-pwrseq = <&wifi_pwrseq>; |
| 117 | 118 | non-removable; |
| 118 | 119 | status = "okay"; |
| @@ -9,6 +9,14 @@ · 8 unmodified lines | ||
| 9 | 9 | reg = <0 0x40000000>; |
| 10 | 10 | }; |
| 11 | 11 | |
| 12 | + leds { | |
| 13 | + /* | |
| 14 | + * Since there is no upstream GPIO driver yet, | |
| 15 | + * remove the incomplete node. | |
| 16 | + */ | |
| 17 | + /delete-node/ act; | |
| 18 | + }; | |
| 19 | + | |
| 12 | 20 | reg_3v3: fixed-regulator { |
| 13 | 21 | compatible = "regulator-fixed"; |
| 14 | 22 | regulator-name = "3V3"; |
| @@ -328,6 +328,10 @@ · 327 unmodified lines | ||
| 328 | 328 | pinctrl-0 = <&pinctrl_pwm3>; |
| 329 | 329 | }; |
| 330 | 330 | |
| 331 | +&snvs_pwrkey { | |
| 332 | + status = "okay"; | |
| 333 | +}; | |
| 334 | + | |
| 331 | 335 | &ssi2 { |
| 332 | 336 | status = "okay"; |
| 333 | 337 | }; |
| @@ -207,6 +207,10 @@ · 206 unmodified lines | ||
| 207 | 207 | vin-supply = <&sw1c_reg>; |
| 208 | 208 | }; |
| 209 | 209 | |
| 210 | +&snvs_poweroff { | |
| 211 | + status = "okay"; | |
| 212 | +}; | |
| 213 | + | |
| 210 | 214 | &iomuxc { |
| 211 | 215 | pinctrl-names = "default"; |
| 212 | 216 | pinctrl-0 = <&pinctrl_hog>; |
| @@ -230,6 +230,8 @@ · 229 unmodified lines | ||
| 230 | 230 | accelerometer@1c { |
| 231 | 231 | compatible = "fsl,mma8451"; |
| 232 | 232 | reg = <0x1c>; |
| 233 | + pinctrl-names = "default"; | |
| 234 | + pinctrl-0 = <&pinctrl_mma8451_int>; | |
| 233 | 235 | interrupt-parent = <&gpio6>; |
| 234 | 236 | interrupts = <31 IRQ_TYPE_LEVEL_LOW>; |
| 235 | 237 | }; |
| @@ -628,6 +630,12 @@ · 392 unmodified lines | ||
| 628 | 630 | >; |
| 629 | 631 | }; |
| 630 | 632 | |
| 633 | + pinctrl_mma8451_int: mma8451intgrp { | |
| 634 | + fsl,pins = < | |
| 635 | + MX6QDL_PAD_EIM_BCLK__GPIO6_IO31 0xb0b1 | |
| 636 | + >; | |
| 637 | + }; | |
| 638 | + | |
| 631 | 639 | pinctrl_pwm3: pwm1grp { |
| 632 | 640 | fsl,pins = < |
| 633 | 641 | MX6QDL_PAD_SD4_DAT1__PWM3_OUT 0x1b0b1 |
| @@ -448,7 +448,7 @@ · 447 unmodified lines | ||
| 448 | 448 | compatible = "fsl,imx7d-gpt", "fsl,imx6sx-gpt"; |
| 449 | 449 | reg = <0x302d0000 0x10000>; |
| 450 | 450 | interrupts = <GIC_SPI 55 IRQ_TYPE_LEVEL_HIGH>; |
| 451 | − clocks = <&clks IMX7D_CLK_DUMMY>, | |
| 451 | + clocks = <&clks IMX7D_GPT1_ROOT_CLK>, | |
| 452 | 452 | <&clks IMX7D_GPT1_ROOT_CLK>; |
| 453 | 453 | clock-names = "ipg", "per"; |
| 454 | 454 | }; |
| @@ -457,7 +457,7 @@ · 2 unmodified lines | ||
| 457 | 457 | compatible = "fsl,imx7d-gpt", "fsl,imx6sx-gpt"; |
| 458 | 458 | reg = <0x302e0000 0x10000>; |
| 459 | 459 | interrupts = <GIC_SPI 54 IRQ_TYPE_LEVEL_HIGH>; |
| 460 | − clocks = <&clks IMX7D_CLK_DUMMY>, | |
| 460 | + clocks = <&clks IMX7D_GPT2_ROOT_CLK>, | |
| 461 | 461 | <&clks IMX7D_GPT2_ROOT_CLK>; |
| 462 | 462 | clock-names = "ipg", "per"; |
| 463 | 463 | status = "disabled"; |
| @@ -467,7 +467,7 @@ · 3 unmodified lines | ||
| 467 | 467 | compatible = "fsl,imx7d-gpt", "fsl,imx6sx-gpt"; |
| 468 | 468 | reg = <0x302f0000 0x10000>; |
| 469 | 469 | interrupts = <GIC_SPI 53 IRQ_TYPE_LEVEL_HIGH>; |
| 470 | − clocks = <&clks IMX7D_CLK_DUMMY>, | |
| 470 | + clocks = <&clks IMX7D_GPT3_ROOT_CLK>, | |
| 471 | 471 | <&clks IMX7D_GPT3_ROOT_CLK>; |
| 472 | 472 | clock-names = "ipg", "per"; |
| 473 | 473 | status = "disabled"; |
| @@ -477,7 +477,7 @@ · 3 unmodified lines | ||
| 477 | 477 | compatible = "fsl,imx7d-gpt", "fsl,imx6sx-gpt"; |
| 478 | 478 | reg = <0x30300000 0x10000>; |
| 479 | 479 | interrupts = <GIC_SPI 52 IRQ_TYPE_LEVEL_HIGH>; |
| 480 | − clocks = <&clks IMX7D_CLK_DUMMY>, | |
| 480 | + clocks = <&clks IMX7D_GPT4_ROOT_CLK>, | |
| 481 | 481 | <&clks IMX7D_GPT4_ROOT_CLK>; |
| 482 | 482 | clock-names = "ipg", "per"; |
| 483 | 483 | status = "disabled"; |
| @@ -192,3 +192,7 @@ · 191 unmodified lines | ||
| 192 | 192 | &twl_gpio { |
| 193 | 193 | ti,use-leds; |
| 194 | 194 | }; |
| 195 | + | |
| 196 | +&twl_keypad { | |
| 197 | + status = "disabled"; | |
| 198 | +}; | |
| @@ -66,9 +66,21 @@ · 65 unmodified lines | ||
| 66 | 66 | pinctrl-1 = <&ephy_leds_pins>; |
| 67 | 67 | status = "okay"; |
| 68 | 68 | |
| 69 | + gmac0: mac@0 { | |
| 70 | + compatible = "mediatek,eth-mac"; | |
| 71 | + reg = <0>; | |
| 72 | + phy-mode = "2500base-x"; | |
| 73 | + fixed-link { | |
| 74 | + speed = <2500>; | |
| 75 | + full-duplex; | |
| 76 | + pause; | |
| 77 | + }; | |
| 78 | + }; | |
| 79 | + | |
| 69 | 80 | gmac1: mac@1 { |
| 70 | 81 | compatible = "mediatek,eth-mac"; |
| 71 | 82 | reg = <1>; |
| 83 | + phy-mode = "gmii"; | |
| 72 | 84 | phy-handle = <&phy0>; |
| 73 | 85 | }; |
| 74 | 86 | |
| @@ -78,7 +90,6 @@ · 3 unmodified lines | ||
| 78 | 90 | |
| 79 | 91 | phy0: ethernet-phy@0 { |
| 80 | 92 | reg = <0>; |
| 81 | − phy-mode = "gmii"; | |
| 82 | 93 | }; |
| 83 | 94 | }; |
| 84 | 95 | }; |
| @@ -468,14 +468,12 @@ · 467 unmodified lines | ||
| 468 | 468 | compatible = "mediatek,mt7629-sgmiisys", "syscon"; |
| 469 | 469 | reg = <0x1b128000 0x3000>; |
| 470 | 470 | #clock-cells = <1>; |
| 471 | − mediatek,physpeed = "2500"; | |
| 472 | 471 | }; |
| 473 | 472 | |
| 474 | 473 | sgmiisys1: syscon@1b130000 { |
| 475 | 474 | compatible = "mediatek,mt7629-sgmiisys", "syscon"; |
| 476 | 475 | reg = <0x1b130000 0x3000>; |
| 477 | 476 | #clock-cells = <1>; |
| 478 | − mediatek,physpeed = "2500"; | |
| 479 | 477 | }; |
| 480 | 478 | }; |
| 481 | 479 | }; |
| @@ -369,7 +369,7 @@ · 368 unmodified lines | ||
| 369 | 369 | compatible = "ti,wl1285", "ti,wl1283"; |
| 370 | 370 | reg = <2>; |
| 371 | 371 | /* gpio_100 with gpmc_wait2 pad as wakeirq */ |
| 372 | − interrupts-extended = <&gpio4 4 IRQ_TYPE_EDGE_RISING>, | |
| 372 | + interrupts-extended = <&gpio4 4 IRQ_TYPE_LEVEL_HIGH>, | |
| 373 | 373 | <&omap4_pmx_core 0x4e>; |
| 374 | 374 | interrupt-names = "irq", "wakeup"; |
| 375 | 375 | ref-clock-frequency = <26000000>; |
| @@ -474,7 +474,7 @@ · 473 unmodified lines | ||
| 474 | 474 | compatible = "ti,wl1271"; |
| 475 | 475 | reg = <2>; |
| 476 | 476 | /* gpio_53 with gpmc_ncs3 pad as wakeup */ |
| 477 | − interrupts-extended = <&gpio2 21 IRQ_TYPE_EDGE_RISING>, | |
| 477 | + interrupts-extended = <&gpio2 21 IRQ_TYPE_LEVEL_HIGH>, | |
| 478 | 478 | <&omap4_pmx_core 0x3a>; |
| 479 | 479 | interrupt-names = "irq", "wakeup"; |
| 480 | 480 | ref-clock-frequency = <38400000>; |
| @@ -512,7 +512,7 @@ · 511 unmodified lines | ||
| 512 | 512 | compatible = "ti,wl1281"; |
| 513 | 513 | reg = <2>; |
| 514 | 514 | interrupt-parent = <&gpio1>; |
| 515 | − interrupts = <21 IRQ_TYPE_EDGE_RISING>; /* gpio 53 */ | |
| 515 | + interrupts = <21 IRQ_TYPE_LEVEL_HIGH>; /* gpio 53 */ | |
| 516 | 516 | ref-clock-frequency = <26000000>; |
| 517 | 517 | tcxo-clock-frequency = <26000000>; |
| 518 | 518 | }; |
| @@ -69,7 +69,7 @@ · 68 unmodified lines | ||
| 69 | 69 | compatible = "ti,wl1271"; |
| 70 | 70 | reg = <2>; |
| 71 | 71 | interrupt-parent = <&gpio2>; |
| 72 | − interrupts = <9 IRQ_TYPE_EDGE_RISING>; /* gpio 41 */ | |
| 72 | + interrupts = <9 IRQ_TYPE_LEVEL_HIGH>; /* gpio 41 */ | |
| 73 | 73 | ref-clock-frequency = <38400000>; |
| 74 | 74 | }; |
| 75 | 75 | }; |
| @@ -362,7 +362,7 @@ · 361 unmodified lines | ||
| 362 | 362 | pinctrl-names = "default"; |
| 363 | 363 | pinctrl-0 = <&wlcore_irq_pin>; |
| 364 | 364 | interrupt-parent = <&gpio1>; |
| 365 | − interrupts = <14 IRQ_TYPE_EDGE_RISING>; /* gpio 14 */ | |
| 365 | + interrupts = <14 IRQ_TYPE_LEVEL_HIGH>; /* gpio 14 */ | |
| 366 | 366 | ref-clock-frequency = <26000000>; |
| 367 | 367 | }; |
| 368 | 368 | }; |
| @@ -1146,7 +1146,7 @@ · 1145 unmodified lines | ||
| 1146 | 1146 | }; |
| 1147 | 1147 | }; |
| 1148 | 1148 | |
| 1149 | − gpu_cm: clock-controller@1500 { | |
| 1149 | + gpu_cm: gpu_cm@1500 { | |
| 1150 | 1150 | compatible = "ti,omap4-cm"; |
| 1151 | 1151 | reg = <0x1500 0x100>; |
| 1152 | 1152 | #address-cells = <1>; |
| @@ -609,13 +609,13 @@ · 608 unmodified lines | ||
| 609 | 609 | <STM32_PINMUX('F', 6, AF9)>; /* QSPI_BK1_IO3 */ |
| 610 | 610 | bias-disable; |
| 611 | 611 | drive-push-pull; |
| 612 | − slew-rate = <3>; | |
| 612 | + slew-rate = <1>; | |
| 613 | 613 | }; |
| 614 | 614 | pins2 { |
| 615 | 615 | pinmux = <STM32_PINMUX('B', 6, AF10)>; /* QSPI_BK1_NCS */ |
| 616 | 616 | bias-pull-up; |
| 617 | 617 | drive-push-pull; |
| 618 | − slew-rate = <3>; | |
| 618 | + slew-rate = <1>; | |
| 619 | 619 | }; |
| 620 | 620 | }; |
| 621 | 621 | |
| @@ -637,13 +637,13 @@ · 15 unmodified lines | ||
| 637 | 637 | <STM32_PINMUX('G', 7, AF11)>; /* QSPI_BK2_IO3 */ |
| 638 | 638 | bias-disable; |
| 639 | 639 | drive-push-pull; |
| 640 | − slew-rate = <3>; | |
| 640 | + slew-rate = <1>; | |
| 641 | 641 | }; |
| 642 | 642 | pins2 { |
| 643 | 643 | pinmux = <STM32_PINMUX('C', 0, AF10)>; /* QSPI_BK2_NCS */ |
| 644 | 644 | bias-pull-up; |
| 645 | 645 | drive-push-pull; |
| 646 | − slew-rate = <3>; | |
| 646 | + slew-rate = <1>; | |
| 647 | 647 | }; |
| 648 | 648 | }; |
| 649 | 649 | |
| @@ -183,14 +183,12 @@ · 182 unmodified lines | ||
| 183 | 183 | |
| 184 | 184 | ov5640: camera@3c { |
| 185 | 185 | compatible = "ovti,ov5640"; |
| 186 | − pinctrl-names = "default"; | |
| 187 | − pinctrl-0 = <&ov5640_pins>; | |
| 188 | 186 | reg = <0x3c>; |
| 189 | 187 | clocks = <&clk_ext_camera>; |
| 190 | 188 | clock-names = "xclk"; |
| 191 | 189 | DOVDD-supply = <&v2v8>; |
| 192 | − powerdown-gpios = <&stmfx_pinctrl 18 GPIO_ACTIVE_HIGH>; | |
| 193 | − reset-gpios = <&stmfx_pinctrl 19 GPIO_ACTIVE_LOW>; | |
| 190 | + powerdown-gpios = <&stmfx_pinctrl 18 (GPIO_ACTIVE_HIGH | GPIO_PUSH_PULL)>; | |
| 191 | + reset-gpios = <&stmfx_pinctrl 19 (GPIO_ACTIVE_LOW | GPIO_PUSH_PULL)>; | |
| 194 | 192 | rotation = <180>; |
| 195 | 193 | status = "okay"; |
| 196 | 194 | |
| @@ -223,15 +221,8 @@ · 26 unmodified lines | ||
| 223 | 221 | |
| 224 | 222 | joystick_pins: joystick { |
| 225 | 223 | pins = "gpio0", "gpio1", "gpio2", "gpio3", "gpio4"; |
| 226 | − drive-push-pull; | |
| 227 | 224 | bias-pull-down; |
| 228 | 225 | }; |
| 229 | − | |
| 230 | − ov5640_pins: camera { | |
| 231 | − pins = "agpio2", "agpio3"; /* stmfx pins 18 & 19 */ | |
| 232 | − drive-push-pull; | |
| 233 | − output-low; | |
| 234 | − }; | |
| 235 | 226 | }; |
| 236 | 227 | }; |
| 237 | 228 | }; |
| @@ -932,7 +932,7 @@ · 931 unmodified lines | ||
| 932 | 932 | interrupt-names = "int0", "int1"; |
| 933 | 933 | clocks = <&rcc CK_HSE>, <&rcc FDCAN_K>; |
| 934 | 934 | clock-names = "hclk", "cclk"; |
| 935 | − bosch,mram-cfg = <0x1400 0 0 32 0 0 2 2>; | |
| 935 | + bosch,mram-cfg = <0x0 0 0 32 0 0 2 2>; | |
| 936 | 936 | status = "disabled"; |
| 937 | 937 | }; |
| 938 | 938 | |
| @@ -945,7 +945,7 @@ · 6 unmodified lines | ||
| 945 | 945 | interrupt-names = "int0", "int1"; |
| 946 | 946 | clocks = <&rcc CK_HSE>, <&rcc FDCAN_K>; |
| 947 | 947 | clock-names = "hclk", "cclk"; |
| 948 | − bosch,mram-cfg = <0x0 0 0 32 0 0 2 2>; | |
| 948 | + bosch,mram-cfg = <0x1400 0 0 32 0 0 2 2>; | |
| 949 | 949 | status = "disabled"; |
| 950 | 950 | }; |
| 951 | 951 | |
| @@ -520,6 +520,7 @@ · 519 unmodified lines | ||
| 520 | 520 | interrupts = <39>; |
| 521 | 521 | clocks = <&ccu CLK_AHB_EHCI0>; |
| 522 | 522 | phys = <&usbphy 1>; |
| 523 | + phy-names = "usb"; | |
| 523 | 524 | status = "disabled"; |
| 524 | 525 | }; |
| 525 | 526 | |
| @@ -529,6 +530,7 @@ · 3 unmodified lines | ||
| 529 | 530 | interrupts = <64>; |
| 530 | 531 | clocks = <&ccu CLK_USB_OHCI0>, <&ccu CLK_AHB_OHCI0>; |
| 531 | 532 | phys = <&usbphy 1>; |
| 533 | + phy-names = "usb"; | |
| 532 | 534 | status = "disabled"; |
| 533 | 535 | }; |
| 534 | 536 | |
| @@ -608,6 +610,7 @@ · 73 unmodified lines | ||
| 608 | 610 | interrupts = <40>; |
| 609 | 611 | clocks = <&ccu CLK_AHB_EHCI1>; |
| 610 | 612 | phys = <&usbphy 2>; |
| 613 | + phy-names = "usb"; | |
| 611 | 614 | status = "disabled"; |
| 612 | 615 | }; |
| 613 | 616 | |
| @@ -617,6 +620,7 @@ · 3 unmodified lines | ||
| 617 | 620 | interrupts = <65>; |
| 618 | 621 | clocks = <&ccu CLK_USB_OHCI1>, <&ccu CLK_AHB_OHCI1>; |
| 619 | 622 | phys = <&usbphy 2>; |
| 623 | + phy-names = "usb"; | |
| 620 | 624 | status = "disabled"; |
| 621 | 625 | }; |
| 622 | 626 | |
| @@ -391,6 +391,7 @@ · 390 unmodified lines | ||
| 391 | 391 | interrupts = <39>; |
| 392 | 392 | clocks = <&ccu CLK_AHB_EHCI>; |
| 393 | 393 | phys = <&usbphy 1>; |
| 394 | + phy-names = "usb"; | |
| 394 | 395 | status = "disabled"; |
| 395 | 396 | }; |
| 396 | 397 | |
| @@ -400,6 +401,7 @@ · 3 unmodified lines | ||
| 400 | 401 | interrupts = <40>; |
| 401 | 402 | clocks = <&ccu CLK_USB_OHCI>, <&ccu CLK_AHB_OHCI>; |
| 402 | 403 | phys = <&usbphy 1>; |
| 404 | + phy-names = "usb"; | |
| 403 | 405 | status = "disabled"; |
| 404 | 406 | }; |
| 405 | 407 | |
| @@ -545,6 +545,7 @@ · 544 unmodified lines | ||
| 545 | 545 | clocks = <&ccu CLK_AHB1_EHCI0>; |
| 546 | 546 | resets = <&ccu RST_AHB1_EHCI0>; |
| 547 | 547 | phys = <&usbphy 1>; |
| 548 | + phy-names = "usb"; | |
| 548 | 549 | status = "disabled"; |
| 549 | 550 | }; |
| 550 | 551 | |
| @@ -555,6 +556,7 @@ · 4 unmodified lines | ||
| 555 | 556 | clocks = <&ccu CLK_AHB1_OHCI0>, <&ccu CLK_USB_OHCI0>; |
| 556 | 557 | resets = <&ccu RST_AHB1_OHCI0>; |
| 557 | 558 | phys = <&usbphy 1>; |
| 559 | + phy-names = "usb"; | |
| 558 | 560 | status = "disabled"; |
| 559 | 561 | }; |
| 560 | 562 | |
| @@ -565,6 +567,7 @@ · 4 unmodified lines | ||
| 565 | 567 | clocks = <&ccu CLK_AHB1_EHCI1>; |
| 566 | 568 | resets = <&ccu RST_AHB1_EHCI1>; |
| 567 | 569 | phys = <&usbphy 2>; |
| 570 | + phy-names = "usb"; | |
| 568 | 571 | status = "disabled"; |
| 569 | 572 | }; |
| 570 | 573 | |
| @@ -575,6 +578,7 @@ · 4 unmodified lines | ||
| 575 | 578 | clocks = <&ccu CLK_AHB1_OHCI1>, <&ccu CLK_USB_OHCI1>; |
| 576 | 579 | resets = <&ccu RST_AHB1_OHCI1>; |
| 577 | 580 | phys = <&usbphy 2>; |
| 581 | + phy-names = "usb"; | |
| 578 | 582 | status = "disabled"; |
| 579 | 583 | }; |
| 580 | 584 | |
| @@ -380,9 +380,8 @@ · 379 unmodified lines | ||
| 380 | 380 | compatible = "allwinner,sun7i-a20-csi0"; |
| 381 | 381 | reg = <0x01c09000 0x1000>; |
| 382 | 382 | interrupts = <GIC_SPI 42 IRQ_TYPE_LEVEL_HIGH>; |
| 383 | − clocks = <&ccu CLK_AHB_CSI0>, <&ccu CLK_CSI0>, | |
| 384 | − <&ccu CLK_CSI_SCLK>, <&ccu CLK_DRAM_CSI0>; | |
| 385 | − clock-names = "bus", "mod", "isp", "ram"; | |
| 383 | + clocks = <&ccu CLK_AHB_CSI0>, <&ccu CLK_CSI_SCLK>, <&ccu CLK_DRAM_CSI0>; | |
| 384 | + clock-names = "bus", "isp", "ram"; | |
| 386 | 385 | resets = <&ccu RST_CSI0>; |
| 387 | 386 | status = "disabled"; |
| 388 | 387 | }; |
| @@ -623,6 +622,7 @@ · 234 unmodified lines | ||
| 623 | 622 | interrupts = <GIC_SPI 39 IRQ_TYPE_LEVEL_HIGH>; |
| 624 | 623 | clocks = <&ccu CLK_AHB_EHCI0>; |
| 625 | 624 | phys = <&usbphy 1>; |
| 625 | + phy-names = "usb"; | |
| 626 | 626 | status = "disabled"; |
| 627 | 627 | }; |
| 628 | 628 | |
| @@ -632,6 +632,7 @@ · 3 unmodified lines | ||
| 632 | 632 | interrupts = <GIC_SPI 64 IRQ_TYPE_LEVEL_HIGH>; |
| 633 | 633 | clocks = <&ccu CLK_USB_OHCI0>, <&ccu CLK_AHB_OHCI0>; |
| 634 | 634 | phys = <&usbphy 1>; |
| 635 | + phy-names = "usb"; | |
| 635 | 636 | status = "disabled"; |
| 636 | 637 | }; |
| 637 | 638 | |
| @@ -714,6 +715,7 @@ · 76 unmodified lines | ||
| 714 | 715 | interrupts = <GIC_SPI 40 IRQ_TYPE_LEVEL_HIGH>; |
| 715 | 716 | clocks = <&ccu CLK_AHB_EHCI1>; |
| 716 | 717 | phys = <&usbphy 2>; |
| 718 | + phy-names = "usb"; | |
| 717 | 719 | status = "disabled"; |
| 718 | 720 | }; |
| 719 | 721 | |
| @@ -723,6 +725,7 @@ · 3 unmodified lines | ||
| 723 | 725 | interrupts = <GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>; |
| 724 | 726 | clocks = <&ccu CLK_USB_OHCI1>, <&ccu CLK_AHB_OHCI1>; |
| 725 | 727 | phys = <&usbphy 2>; |
| 728 | + phy-names = "usb"; | |
| 726 | 729 | status = "disabled"; |
| 727 | 730 | }; |
| 728 | 731 | |
| @@ -307,6 +307,7 @@ · 306 unmodified lines | ||
| 307 | 307 | clocks = <&ccu CLK_BUS_EHCI>; |
| 308 | 308 | resets = <&ccu RST_BUS_EHCI>; |
| 309 | 309 | phys = <&usbphy 1>; |
| 310 | + phy-names = "usb"; | |
| 310 | 311 | status = "disabled"; |
| 311 | 312 | }; |
| 312 | 313 | |
| @@ -317,6 +318,7 @@ · 4 unmodified lines | ||
| 317 | 318 | clocks = <&ccu CLK_BUS_OHCI>, <&ccu CLK_USB_OHCI>; |
| 318 | 319 | resets = <&ccu RST_BUS_OHCI>; |
| 319 | 320 | phys = <&usbphy 1>; |
| 321 | + phy-names = "usb"; | |
| 320 | 322 | status = "disabled"; |
| 321 | 323 | }; |
| 322 | 324 | |
| @@ -192,6 +192,7 @@ · 191 unmodified lines | ||
| 192 | 192 | vqmmc-supply = <®_dldo1>; |
| 193 | 193 | non-removable; |
| 194 | 194 | wakeup-source; |
| 195 | + keep-power-in-suspend; | |
| 195 | 196 | status = "okay"; |
| 196 | 197 | |
| 197 | 198 | brcmf: wifi@1 { |
| @@ -632,6 +632,7 @@ · 631 unmodified lines | ||
| 632 | 632 | clocks = <&ccu CLK_BUS_EHCI0>; |
| 633 | 633 | resets = <&ccu RST_BUS_EHCI0>; |
| 634 | 634 | phys = <&usbphy 1>; |
| 635 | + phy-names = "usb"; | |
| 635 | 636 | status = "disabled"; |
| 636 | 637 | }; |
| 637 | 638 | |
| @@ -643,6 +644,7 @@ · 5 unmodified lines | ||
| 643 | 644 | clocks = <&ccu CLK_BUS_OHCI0>, <&ccu CLK_USB_OHCI0>; |
| 644 | 645 | resets = <&ccu RST_BUS_OHCI0>; |
| 645 | 646 | phys = <&usbphy 1>; |
| 647 | + phy-names = "usb"; | |
| 646 | 648 | status = "disabled"; |
| 647 | 649 | }; |
| 648 | 650 | |
| @@ -654,6 +656,7 @@ · 5 unmodified lines | ||
| 654 | 656 | clocks = <&ccu CLK_BUS_EHCI1>; |
| 655 | 657 | resets = <&ccu RST_BUS_EHCI1>; |
| 656 | 658 | phys = <&usbphy 2>; |
| 659 | + phy-names = "usb"; | |
| 657 | 660 | status = "disabled"; |
| 658 | 661 | }; |
| 659 | 662 | |
| @@ -273,6 +273,7 @@ · 272 unmodified lines | ||
| 273 | 273 | clocks = <&ccu CLK_BUS_EHCI1>; |
| 274 | 274 | resets = <&ccu RST_BUS_EHCI1>; |
| 275 | 275 | phys = <&usbphy 1>; |
| 276 | + phy-names = "usb"; | |
| 276 | 277 | status = "disabled"; |
| 277 | 278 | }; |
| 278 | 279 | |
| @@ -284,6 +285,7 @@ · 5 unmodified lines | ||
| 284 | 285 | <&ccu CLK_USB_OHCI1>; |
| 285 | 286 | resets = <&ccu RST_BUS_OHCI1>; |
| 286 | 287 | phys = <&usbphy 1>; |
| 288 | + phy-names = "usb"; | |
| 287 | 289 | status = "disabled"; |
| 288 | 290 | }; |
| 289 | 291 | |
| @@ -294,6 +296,7 @@ · 4 unmodified lines | ||
| 294 | 296 | clocks = <&ccu CLK_BUS_EHCI2>; |
| 295 | 297 | resets = <&ccu RST_BUS_EHCI2>; |
| 296 | 298 | phys = <&usbphy 2>; |
| 299 | + phy-names = "usb"; | |
| 297 | 300 | status = "disabled"; |
| 298 | 301 | }; |
| 299 | 302 | |
| @@ -305,6 +308,7 @@ · 5 unmodified lines | ||
| 305 | 308 | <&ccu CLK_USB_OHCI2>; |
| 306 | 309 | resets = <&ccu RST_BUS_OHCI2>; |
| 307 | 310 | phys = <&usbphy 2>; |
| 311 | + phy-names = "usb"; | |
| 308 | 312 | status = "disabled"; |
| 309 | 313 | }; |
| 310 | 314 | |
| @@ -346,6 +346,7 @@ · 345 unmodified lines | ||
| 346 | 346 | clocks = <&usb_clocks CLK_BUS_HCI0>; |
| 347 | 347 | resets = <&usb_clocks RST_USB0_HCI>; |
| 348 | 348 | phys = <&usbphy1>; |
| 349 | + phy-names = "usb"; | |
| 349 | 350 | status = "disabled"; |
| 350 | 351 | }; |
| 351 | 352 | |
| @@ -357,6 +358,7 @@ · 5 unmodified lines | ||
| 357 | 358 | <&usb_clocks CLK_USB_OHCI0>; |
| 358 | 359 | resets = <&usb_clocks RST_USB0_HCI>; |
| 359 | 360 | phys = <&usbphy1>; |
| 361 | + phy-names = "usb"; | |
| 360 | 362 | status = "disabled"; |
| 361 | 363 | }; |
| 362 | 364 | |
| @@ -378,6 +380,7 @@ · 15 unmodified lines | ||
| 378 | 380 | clocks = <&usb_clocks CLK_BUS_HCI1>; |
| 379 | 381 | resets = <&usb_clocks RST_USB1_HCI>; |
| 380 | 382 | phys = <&usbphy2>; |
| 383 | + phy-names = "usb"; | |
| 381 | 384 | status = "disabled"; |
| 382 | 385 | }; |
| 383 | 386 | |
| @@ -407,6 +410,7 @@ · 23 unmodified lines | ||
| 407 | 410 | clocks = <&usb_clocks CLK_BUS_HCI2>; |
| 408 | 411 | resets = <&usb_clocks RST_USB2_HCI>; |
| 409 | 412 | phys = <&usbphy3>; |
| 413 | + phy-names = "usb"; | |
| 410 | 414 | status = "disabled"; |
| 411 | 415 | }; |
| 412 | 416 | |
| @@ -418,6 +422,7 @@ · 5 unmodified lines | ||
| 418 | 422 | <&usb_clocks CLK_USB_OHCI2>; |
| 419 | 423 | resets = <&usb_clocks RST_USB2_HCI>; |
| 420 | 424 | phys = <&usbphy3>; |
| 425 | + phy-names = "usb"; | |
| 421 | 426 | status = "disabled"; |
| 422 | 427 | }; |
| 423 | 428 | |
| @@ -304,6 +304,7 @@ · 303 unmodified lines | ||
| 304 | 304 | clocks = <&ccu CLK_BUS_EHCI1>, <&ccu CLK_BUS_OHCI1>; |
| 305 | 305 | resets = <&ccu RST_BUS_EHCI1>, <&ccu RST_BUS_OHCI1>; |
| 306 | 306 | phys = <&usbphy 1>; |
| 307 | + phy-names = "usb"; | |
| 307 | 308 | status = "disabled"; |
| 308 | 309 | }; |
| 309 | 310 | |
| @@ -315,6 +316,7 @@ · 5 unmodified lines | ||
| 315 | 316 | <&ccu CLK_USB_OHCI1>; |
| 316 | 317 | resets = <&ccu RST_BUS_EHCI1>, <&ccu RST_BUS_OHCI1>; |
| 317 | 318 | phys = <&usbphy 1>; |
| 319 | + phy-names = "usb"; | |
| 318 | 320 | status = "disabled"; |
| 319 | 321 | }; |
| 320 | 322 | |
| @@ -325,6 +327,7 @@ · 4 unmodified lines | ||
| 325 | 327 | clocks = <&ccu CLK_BUS_EHCI2>, <&ccu CLK_BUS_OHCI2>; |
| 326 | 328 | resets = <&ccu RST_BUS_EHCI2>, <&ccu RST_BUS_OHCI2>; |
| 327 | 329 | phys = <&usbphy 2>; |
| 330 | + phy-names = "usb"; | |
| 328 | 331 | status = "disabled"; |
| 329 | 332 | }; |
| 330 | 333 | |
| @@ -336,6 +339,7 @@ · 5 unmodified lines | ||
| 336 | 339 | <&ccu CLK_USB_OHCI2>; |
| 337 | 340 | resets = <&ccu RST_BUS_EHCI2>, <&ccu RST_BUS_OHCI2>; |
| 338 | 341 | phys = <&usbphy 2>; |
| 342 | + phy-names = "usb"; | |
| 339 | 343 | status = "disabled"; |
| 340 | 344 | }; |
| 341 | 345 | |
| @@ -346,6 +350,7 @@ · 4 unmodified lines | ||
| 346 | 350 | clocks = <&ccu CLK_BUS_EHCI3>, <&ccu CLK_BUS_OHCI3>; |
| 347 | 351 | resets = <&ccu RST_BUS_EHCI3>, <&ccu RST_BUS_OHCI3>; |
| 348 | 352 | phys = <&usbphy 3>; |
| 353 | + phy-names = "usb"; | |
| 349 | 354 | status = "disabled"; |
| 350 | 355 | }; |
| 351 | 356 | |
| @@ -357,6 +362,7 @@ · 5 unmodified lines | ||
| 357 | 362 | <&ccu CLK_USB_OHCI3>; |
| 358 | 363 | resets = <&ccu RST_BUS_EHCI3>, <&ccu RST_BUS_OHCI3>; |
| 359 | 364 | phys = <&usbphy 3>; |
| 365 | + phy-names = "usb"; | |
| 360 | 366 | status = "disabled"; |
| 361 | 367 | }; |
| 362 | 368 | |
| @@ -602,6 +602,7 @@ · 601 unmodified lines | ||
| 602 | 602 | #address-cells = <1>; |
| 603 | 603 | #size-cells = <0>; |
| 604 | 604 | reg = <0x70>; |
| 605 | + i2c-mux-idle-disconnect; | |
| 605 | 606 | |
| 606 | 607 | sff0_i2c: i2c@1 { |
| 607 | 608 | #address-cells = <1>; |
| @@ -640,6 +641,7 @@ · 32 unmodified lines | ||
| 640 | 641 | reg = <0x71>; |
| 641 | 642 | #address-cells = <1>; |
| 642 | 643 | #size-cells = <0>; |
| 644 | + i2c-mux-idle-disconnect; | |
| 643 | 645 | |
| 644 | 646 | sff5_i2c: i2c@1 { |
| 645 | 647 | #address-cells = <1>; |
| @@ -91,7 +91,6 @@ CONFIG_USB_SERIAL_PL2303=m · 90 unmodified lines | ||
| 91 | 91 | CONFIG_USB_SERIAL_CYBERJACK=m |
| 92 | 92 | CONFIG_USB_SERIAL_XIRCOM=m |
| 93 | 93 | CONFIG_USB_SERIAL_OMNINET=m |
| 94 | −CONFIG_USB_RIO500=m | |
| 95 | 94 | CONFIG_EXT2_FS=m |
| 96 | 95 | CONFIG_EXT3_FS=m |
| 97 | 96 | CONFIG_MSDOS_FS=y |
| @@ -195,7 +195,6 @@ CONFIG_USB_SERIAL_XIRCOM=m · 194 unmodified lines | ||
| 195 | 195 | CONFIG_USB_SERIAL_OMNINET=m |
| 196 | 196 | CONFIG_USB_EMI62=m |
| 197 | 197 | CONFIG_USB_EMI26=m |
| 198 | −CONFIG_USB_RIO500=m | |
| 199 | 198 | CONFIG_USB_LEGOTOWER=m |
| 200 | 199 | CONFIG_USB_LCD=m |
| 201 | 200 | CONFIG_USB_CYTHERM=m |
| @@ -167,6 +167,7 @@ CONFIG_FB=y · 166 unmodified lines | ||
| 167 | 167 | CONFIG_FIRMWARE_EDID=y |
| 168 | 168 | CONFIG_FB_DA8XX=y |
| 169 | 169 | CONFIG_BACKLIGHT_PWM=m |
| 170 | +CONFIG_BACKLIGHT_GPIO=m | |
| 170 | 171 | CONFIG_FRAMEBUFFER_CONSOLE=y |
| 171 | 172 | CONFIG_LOGO=y |
| 172 | 173 | CONFIG_SOUND=m |
| @@ -276,6 +276,7 @@ CONFIG_VIDEO_OV5640=m · 275 unmodified lines | ||
| 276 | 276 | CONFIG_VIDEO_OV5645=m |
| 277 | 277 | CONFIG_IMX_IPUV3_CORE=y |
| 278 | 278 | CONFIG_DRM=y |
| 279 | +CONFIG_DRM_MSM=y | |
| 279 | 280 | CONFIG_DRM_PANEL_LVDS=y |
| 280 | 281 | CONFIG_DRM_PANEL_SIMPLE=y |
| 281 | 282 | CONFIG_DRM_PANEL_SEIKO_43WVF1G=y |
| @@ -356,15 +356,15 @@ CONFIG_DRM_OMAP_CONNECTOR_HDMI=m · 355 unmodified lines | ||
| 356 | 356 | CONFIG_DRM_OMAP_CONNECTOR_ANALOG_TV=m |
| 357 | 357 | CONFIG_DRM_OMAP_PANEL_DPI=m |
| 358 | 358 | CONFIG_DRM_OMAP_PANEL_DSI_CM=m |
| 359 | −CONFIG_DRM_OMAP_PANEL_SONY_ACX565AKM=m | |
| 360 | −CONFIG_DRM_OMAP_PANEL_LGPHILIPS_LB035Q02=m | |
| 361 | −CONFIG_DRM_OMAP_PANEL_SHARP_LS037V7DW01=m | |
| 362 | −CONFIG_DRM_OMAP_PANEL_TPO_TD028TTEC1=m | |
| 363 | −CONFIG_DRM_OMAP_PANEL_TPO_TD043MTEA1=m | |
| 364 | −CONFIG_DRM_OMAP_PANEL_NEC_NL8048HL11=m | |
| 365 | 359 | CONFIG_DRM_TILCDC=m |
| 366 | 360 | CONFIG_DRM_PANEL_SIMPLE=m |
| 367 | 361 | CONFIG_DRM_TI_TFP410=m |
| 362 | +CONFIG_DRM_PANEL_LG_LB035Q02=m | |
| 363 | +CONFIG_DRM_PANEL_NEC_NL8048HL11=m | |
| 364 | +CONFIG_DRM_PANEL_SHARP_LS037V7DW01=m | |
| 365 | +CONFIG_DRM_PANEL_SONY_ACX565AKM=m | |
| 366 | +CONFIG_DRM_PANEL_TPO_TD028TTEC1=m | |
| 367 | +CONFIG_DRM_PANEL_TPO_TD043MTEA1=m | |
| 368 | 368 | CONFIG_FB=y |
| 369 | 369 | CONFIG_FIRMWARE_EDID=y |
| 370 | 370 | CONFIG_FB_MODE_HELPERS=y |
| @@ -581,7 +581,6 @@ CONFIG_USB_SERIAL_XIRCOM=m · 580 unmodified lines | ||
| 581 | 581 | CONFIG_USB_SERIAL_OMNINET=m |
| 582 | 582 | CONFIG_USB_EMI62=m |
| 583 | 583 | CONFIG_USB_EMI26=m |
| 584 | −CONFIG_USB_RIO500=m | |
| 585 | 584 | CONFIG_USB_LEGOTOWER=m |
| 586 | 585 | CONFIG_USB_LCD=m |
| 587 | 586 | CONFIG_USB_CYTHERM=m |
| @@ -327,7 +327,6 @@ CONFIG_USB_EMI62=m · 326 unmodified lines | ||
| 327 | 327 | CONFIG_USB_EMI26=m |
| 328 | 328 | CONFIG_USB_ADUTUX=m |
| 329 | 329 | CONFIG_USB_SEVSEG=m |
| 330 | −CONFIG_USB_RIO500=m | |
| 331 | 330 | CONFIG_USB_LEGOTOWER=m |
| 332 | 331 | CONFIG_USB_LCD=m |
| 333 | 332 | CONFIG_USB_CYPRESS_CY7C63=m |
| @@ -189,7 +189,6 @@ CONFIG_USB_SERIAL_XIRCOM=m · 188 unmodified lines | ||
| 189 | 189 | CONFIG_USB_SERIAL_OMNINET=m |
| 190 | 190 | CONFIG_USB_EMI62=m |
| 191 | 191 | CONFIG_USB_EMI26=m |
| 192 | −CONFIG_USB_RIO500=m | |
| 193 | 192 | CONFIG_USB_LEGOTOWER=m |
| 194 | 193 | CONFIG_USB_LCD=m |
| 195 | 194 | CONFIG_USB_CYTHERM=m |
| @@ -98,6 +98,7 @@ config CRYPTO_AES_ARM_CE · 97 unmodified lines | ||
| 98 | 98 | tristate "Accelerated AES using ARMv8 Crypto Extensions" |
| 99 | 99 | depends on KERNEL_MODE_NEON |
| 100 | 100 | select CRYPTO_BLKCIPHER |
| 101 | + select CRYPTO_LIB_AES | |
| 101 | 102 | select CRYPTO_SIMD |
| 102 | 103 | help |
| 103 | 104 | Use an implementation of AES in CBC, CTR and XTS modes that uses |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | #include <asm/assembler.h> |
| 10 | 10 | |
| 11 | 11 | .text |
| 12 | + .arch armv8-a | |
| 12 | 13 | .fpu crypto-neon-fp-armv8 |
| 13 | 14 | .align 3 |
| 14 | 15 | |
| @@ -82,7 +82,7 @@ · 81 unmodified lines | ||
| 82 | 82 | #ifndef __ASSEMBLY__ |
| 83 | 83 | |
| 84 | 84 | #ifdef CONFIG_CPU_CP15_MMU |
| 85 | −static inline unsigned int get_domain(void) | |
| 85 | +static __always_inline unsigned int get_domain(void) | |
| 86 | 86 | { |
| 87 | 87 | unsigned int domain; |
| 88 | 88 | |
| @@ -94,7 +94,7 @@ static inline unsigned int get_domain(void) · 5 unmodified lines | ||
| 94 | 94 | return domain; |
| 95 | 95 | } |
| 96 | 96 | |
| 97 | −static inline void set_domain(unsigned val) | |
| 97 | +static __always_inline void set_domain(unsigned int val) | |
| 98 | 98 | { |
| 99 | 99 | asm volatile( |
| 100 | 100 | "mcr p15, 0, %0, c3, c0 @ set domain" |
| @@ -102,12 +102,12 @@ static inline void set_domain(unsigned val) · 1 unmodified line | ||
| 102 | 102 | isb(); |
| 103 | 103 | } |
| 104 | 104 | #else |
| 105 | −static inline unsigned int get_domain(void) | |
| 105 | +static __always_inline unsigned int get_domain(void) | |
| 106 | 106 | { |
| 107 | 107 | return 0; |
| 108 | 108 | } |
| 109 | 109 | |
| 110 | −static inline void set_domain(unsigned val) | |
| 110 | +static __always_inline void set_domain(unsigned int val) | |
| 111 | 111 | { |
| 112 | 112 | } |
| 113 | 113 | #endif |
| @@ -22,7 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | * perform such accesses (eg, via list poison values) which could then |
| 23 | 23 | * be exploited for priviledge escalation. |
| 24 | 24 | */ |
| 25 | −static inline unsigned int uaccess_save_and_enable(void) | |
| 25 | +static __always_inline unsigned int uaccess_save_and_enable(void) | |
| 26 | 26 | { |
| 27 | 27 | #ifdef CONFIG_CPU_SW_DOMAIN_PAN |
| 28 | 28 | unsigned int old_domain = get_domain(); |
| @@ -37,7 +37,7 @@ static inline unsigned int uaccess_save_and_enable(void) · 8 unmodified lines | ||
| 37 | 37 | #endif |
| 38 | 38 | } |
| 39 | 39 | |
| 40 | −static inline void uaccess_restore(unsigned int flags) | |
| 40 | +static __always_inline void uaccess_restore(unsigned int flags) | |
| 41 | 41 | { |
| 42 | 42 | #ifdef CONFIG_CPU_SW_DOMAIN_PAN |
| 43 | 43 | /* Restore the user access mask */ |
| @@ -68,7 +68,7 @@ ENDPROC(__vet_atags) · 67 unmodified lines | ||
| 68 | 68 | * The following fragment of code is executed with the MMU on in MMU mode, |
| 69 | 69 | * and uses absolute addresses; this is not position independent. |
| 70 | 70 | * |
| 71 | − * r0 = cp#15 control register | |
| 71 | + * r0 = cp#15 control register (exc_ret for M-class) | |
| 72 | 72 | * r1 = machine ID |
| 73 | 73 | * r2 = atags/dtb pointer |
| 74 | 74 | * r9 = processor ID |
| @@ -137,7 +137,8 @@ __mmap_switched_data: · 62 unmodified lines | ||
| 137 | 137 | #ifdef CONFIG_CPU_CP15 |
| 138 | 138 | .long cr_alignment @ r3 |
| 139 | 139 | #else |
| 140 | − .long 0 @ r3 | |
| 140 | +M_CLASS(.long exc_ret) @ r3 | |
| 141 | +AR_CLASS(.long 0) @ r3 | |
| 141 | 142 | #endif |
| 142 | 143 | .size __mmap_switched_data, . - __mmap_switched_data |
| 143 | 144 | |
| @@ -201,6 +201,8 @@ M_CLASS(streq r3, [r12, #PMSAv8_MAIR1]) · 200 unmodified lines | ||
| 201 | 201 | bic r0, r0, #V7M_SCB_CCR_IC |
| 202 | 202 | #endif |
| 203 | 203 | str r0, [r12, V7M_SCB_CCR] |
| 204 | + /* Pass exc_ret to __mmap_switched */ | |
| 205 | + mov r0, r10 | |
| 204 | 206 | #endif /* CONFIG_CPU_CP15 elif CONFIG_CPU_V7M */ |
| 205 | 207 | ret lr |
| 206 | 208 | ENDPROC(__after_proc_init) |
| @@ -462,8 +462,8 @@ static s8 dm365_queue_priority_mapping[][2] = { · 461 unmodified lines | ||
| 462 | 462 | }; |
| 463 | 463 | |
| 464 | 464 | static const struct dma_slave_map dm365_edma_map[] = { |
| 465 | − { "davinci-mcbsp.0", "tx", EDMA_FILTER_PARAM(0, 2) }, | |
| 466 | − { "davinci-mcbsp.0", "rx", EDMA_FILTER_PARAM(0, 3) }, | |
| 465 | + { "davinci-mcbsp", "tx", EDMA_FILTER_PARAM(0, 2) }, | |
| 466 | + { "davinci-mcbsp", "rx", EDMA_FILTER_PARAM(0, 3) }, | |
| 467 | 467 | { "davinci_voicecodec", "tx", EDMA_FILTER_PARAM(0, 2) }, |
| 468 | 468 | { "davinci_voicecodec", "rx", EDMA_FILTER_PARAM(0, 3) }, |
| 469 | 469 | { "spi_davinci.2", "tx", EDMA_FILTER_PARAM(0, 10) }, |
| @@ -89,6 +89,13 @@ static struct iommu_platform_data omap3_iommu_pdata = { · 88 unmodified lines | ||
| 89 | 89 | .reset_name = "mmu", |
| 90 | 90 | .assert_reset = omap_device_assert_hardreset, |
| 91 | 91 | .deassert_reset = omap_device_deassert_hardreset, |
| 92 | + .device_enable = omap_device_enable, | |
| 93 | + .device_idle = omap_device_idle, | |
| 94 | +}; | |
| 95 | + | |
| 96 | +static struct iommu_platform_data omap3_iommu_isp_pdata = { | |
| 97 | + .device_enable = omap_device_enable, | |
| 98 | + .device_idle = omap_device_idle, | |
| 92 | 99 | }; |
| 93 | 100 | |
| 94 | 101 | static int omap3_sbc_t3730_twl_callback(struct device *dev, |
| @@ -424,6 +431,8 @@ static struct iommu_platform_data omap4_iommu_pdata = { · 329 unmodified lines | ||
| 424 | 431 | .reset_name = "mmu_cache", |
| 425 | 432 | .assert_reset = omap_device_assert_hardreset, |
| 426 | 433 | .deassert_reset = omap_device_deassert_hardreset, |
| 434 | + .device_enable = omap_device_enable, | |
| 435 | + .device_idle = omap_device_idle, | |
| 427 | 436 | }; |
| 428 | 437 | #endif |
| 429 | 438 | |
| @@ -617,6 +626,8 @@ static struct of_dev_auxdata omap_auxdata_lookup[] = { · 187 unmodified lines | ||
| 617 | 626 | #ifdef CONFIG_ARCH_OMAP3 |
| 618 | 627 | OF_DEV_AUXDATA("ti,omap2-iommu", 0x5d000000, "5d000000.mmu", |
| 619 | 628 | &omap3_iommu_pdata), |
| 629 | + OF_DEV_AUXDATA("ti,omap2-iommu", 0x480bd400, "480bd400.mmu", | |
| 630 | + &omap3_iommu_isp_pdata), | |
| 620 | 631 | OF_DEV_AUXDATA("ti,omap3-smartreflex-core", 0x480cb000, |
| 621 | 632 | "480cb000.smartreflex", &omap_sr_pdata[OMAP_SR_CORE]), |
| 622 | 633 | OF_DEV_AUXDATA("ti,omap3-smartreflex-mpu-iva", 0x480c9000, |
| @@ -481,14 +481,18 @@ static void sunxi_mc_smp_cpu_die(unsigned int l_cpu) · 480 unmodified lines | ||
| 481 | 481 | static int sunxi_cpu_powerdown(unsigned int cpu, unsigned int cluster) |
| 482 | 482 | { |
| 483 | 483 | u32 reg; |
| 484 | + int gating_bit = cpu; | |
| 484 | 485 | |
| 485 | 486 | pr_debug("%s: cluster %u cpu %u\n", __func__, cluster, cpu); |
| 486 | 487 | if (cpu >= SUNXI_CPUS_PER_CLUSTER || cluster >= SUNXI_NR_CLUSTERS) |
| 487 | 488 | return -EINVAL; |
| 488 | 489 | |
| 490 | + if (is_a83t && cpu == 0) | |
| 491 | + gating_bit = 4; | |
| 492 | + | |
| 489 | 493 | /* gate processor power */ |
| 490 | 494 | reg = readl(prcm_base + PRCM_PWROFF_GATING_REG(cluster)); |
| 491 | − reg |= PRCM_PWROFF_GATING_REG_CORE(cpu); | |
| 495 | + reg |= PRCM_PWROFF_GATING_REG_CORE(gating_bit); | |
| 492 | 496 | writel(reg, prcm_base + PRCM_PWROFF_GATING_REG(cluster)); |
| 493 | 497 | udelay(20); |
| 494 | 498 | |
| @@ -324,7 +324,7 @@ union offset_union { · 323 unmodified lines | ||
| 324 | 324 | __put32_unaligned_check("strbt", val, addr) |
| 325 | 325 | |
| 326 | 326 | static void |
| 327 | −do_alignment_finish_ldst(unsigned long addr, unsigned long instr, struct pt_regs *regs, union offset_union offset) | |
| 327 | +do_alignment_finish_ldst(unsigned long addr, u32 instr, struct pt_regs *regs, union offset_union offset) | |
| 328 | 328 | { |
| 329 | 329 | if (!LDST_U_BIT(instr)) |
| 330 | 330 | offset.un = -offset.un; |
| @@ -337,7 +337,7 @@ do_alignment_finish_ldst(unsigned long addr, unsigned long instr, struct pt_regs · 6 unmodified lines | ||
| 337 | 337 | } |
| 338 | 338 | |
| 339 | 339 | static int |
| 340 | −do_alignment_ldrhstrh(unsigned long addr, unsigned long instr, struct pt_regs *regs) | |
| 340 | +do_alignment_ldrhstrh(unsigned long addr, u32 instr, struct pt_regs *regs) | |
| 341 | 341 | { |
| 342 | 342 | unsigned int rd = RD_BITS(instr); |
| 343 | 343 | |
| @@ -386,8 +386,7 @@ do_alignment_ldrhstrh(unsigned long addr, unsigned long instr, struct pt_regs *r · 42 unmodified lines | ||
| 386 | 386 | } |
| 387 | 387 | |
| 388 | 388 | static int |
| 389 | −do_alignment_ldrdstrd(unsigned long addr, unsigned long instr, | |
| 390 | − struct pt_regs *regs) | |
| 389 | +do_alignment_ldrdstrd(unsigned long addr, u32 instr, struct pt_regs *regs) | |
| 391 | 390 | { |
| 392 | 391 | unsigned int rd = RD_BITS(instr); |
| 393 | 392 | unsigned int rd2; |
| @@ -449,7 +448,7 @@ do_alignment_ldrdstrd(unsigned long addr, unsigned long instr, · 55 unmodified lines | ||
| 449 | 448 | } |
| 450 | 449 | |
| 451 | 450 | static int |
| 452 | −do_alignment_ldrstr(unsigned long addr, unsigned long instr, struct pt_regs *regs) | |
| 451 | +do_alignment_ldrstr(unsigned long addr, u32 instr, struct pt_regs *regs) | |
| 453 | 452 | { |
| 454 | 453 | unsigned int rd = RD_BITS(instr); |
| 455 | 454 | |
| @@ -498,7 +497,7 @@ do_alignment_ldrstr(unsigned long addr, unsigned long instr, struct pt_regs *reg · 42 unmodified lines | ||
| 498 | 497 | * PU = 10 A B |
| 499 | 498 | */ |
| 500 | 499 | static int |
| 501 | −do_alignment_ldmstm(unsigned long addr, unsigned long instr, struct pt_regs *regs) | |
| 500 | +do_alignment_ldmstm(unsigned long addr, u32 instr, struct pt_regs *regs) | |
| 502 | 501 | { |
| 503 | 502 | unsigned int rd, rn, correction, nr_regs, regbits; |
| 504 | 503 | unsigned long eaddr, newaddr; |
| @@ -539,7 +538,7 @@ do_alignment_ldmstm(unsigned long addr, unsigned long instr, struct pt_regs *reg · 34 unmodified lines | ||
| 539 | 538 | * processor for us. |
| 540 | 539 | */ |
| 541 | 540 | if (addr != eaddr) { |
| 542 | − pr_err("LDMSTM: PC = %08lx, instr = %08lx, " | |
| 541 | + pr_err("LDMSTM: PC = %08lx, instr = %08x, " | |
| 543 | 542 | "addr = %08lx, eaddr = %08lx\n", |
| 544 | 543 | instruction_pointer(regs), instr, addr, eaddr); |
| 545 | 544 | show_regs(regs); |
| @@ -716,10 +715,10 @@ thumb2arm(u16 tinstr) · 170 unmodified lines | ||
| 716 | 715 | * 2. Register name Rt from ARMv7 is same as Rd from ARMv6 (Rd is Rt) |
| 717 | 716 | */ |
| 718 | 717 | static void * |
| 719 | −do_alignment_t32_to_handler(unsigned long *pinstr, struct pt_regs *regs, | |
| 718 | +do_alignment_t32_to_handler(u32 *pinstr, struct pt_regs *regs, | |
| 720 | 719 | union offset_union *poffset) |
| 721 | 720 | { |
| 722 | − unsigned long instr = *pinstr; | |
| 721 | + u32 instr = *pinstr; | |
| 723 | 722 | u16 tinst1 = (instr >> 16) & 0xffff; |
| 724 | 723 | u16 tinst2 = instr & 0xffff; |
| 725 | 724 | |
| @@ -767,17 +766,48 @@ do_alignment_t32_to_handler(unsigned long *pinstr, struct pt_regs *regs, · 41 unmodified lines | ||
| 767 | 766 | return NULL; |
| 768 | 767 | } |
| 769 | 768 | |
| 769 | +static int alignment_get_arm(struct pt_regs *regs, u32 *ip, u32 *inst) | |
| 770 | +{ | |
| 771 | + u32 instr = 0; | |
| 772 | + int fault; | |
| 773 | + | |
| 774 | + if (user_mode(regs)) | |
| 775 | + fault = get_user(instr, ip); | |
| 776 | + else | |
| 777 | + fault = probe_kernel_address(ip, instr); | |
| 778 | + | |
| 779 | + *inst = __mem_to_opcode_arm(instr); | |
| 780 | + | |
| 781 | + return fault; | |
| 782 | +} | |
| 783 | + | |
| 784 | +static int alignment_get_thumb(struct pt_regs *regs, u16 *ip, u16 *inst) | |
| 785 | +{ | |
| 786 | + u16 instr = 0; | |
| 787 | + int fault; | |
| 788 | + | |
| 789 | + if (user_mode(regs)) | |
| 790 | + fault = get_user(instr, ip); | |
| 791 | + else | |
| 792 | + fault = probe_kernel_address(ip, instr); | |
| 793 | + | |
| 794 | + *inst = __mem_to_opcode_thumb16(instr); | |
| 795 | + | |
| 796 | + return fault; | |
| 797 | +} | |
| 798 | + | |
| 770 | 799 | static int |
| 771 | 800 | do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs) |
| 772 | 801 | { |
| 773 | 802 | union offset_union uninitialized_var(offset); |
| 774 | − unsigned long instr = 0, instrptr; | |
| 775 | − int (*handler)(unsigned long addr, unsigned long instr, struct pt_regs *regs); | |
| 803 | + unsigned long instrptr; | |
| 804 | + int (*handler)(unsigned long addr, u32 instr, struct pt_regs *regs); | |
| 776 | 805 | unsigned int type; |
| 777 | − unsigned int fault; | |
| 806 | + u32 instr = 0; | |
| 778 | 807 | u16 tinstr = 0; |
| 779 | 808 | int isize = 4; |
| 780 | 809 | int thumb2_32b = 0; |
| 810 | + int fault; | |
| 781 | 811 | |
| 782 | 812 | if (interrupts_enabled(regs)) |
| 783 | 813 | local_irq_enable(); |
| @@ -786,15 +816,14 @@ do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs) · 2 unmodified lines | ||
| 786 | 816 | |
| 787 | 817 | if (thumb_mode(regs)) { |
| 788 | 818 | u16 *ptr = (u16 *)(instrptr & ~1); |
| 789 | − fault = probe_kernel_address(ptr, tinstr); | |
| 790 | − tinstr = __mem_to_opcode_thumb16(tinstr); | |
| 819 | + | |
| 820 | + fault = alignment_get_thumb(regs, ptr, &tinstr); | |
| 791 | 821 | if (!fault) { |
| 792 | 822 | if (cpu_architecture() >= CPU_ARCH_ARMv7 && |
| 793 | 823 | IS_T32(tinstr)) { |
| 794 | 824 | /* Thumb-2 32-bit */ |
| 795 | − u16 tinst2 = 0; | |
| 796 | − fault = probe_kernel_address(ptr + 1, tinst2); | |
| 797 | − tinst2 = __mem_to_opcode_thumb16(tinst2); | |
| 825 | + u16 tinst2; | |
| 826 | + fault = alignment_get_thumb(regs, ptr + 1, &tinst2); | |
| 798 | 827 | instr = __opcode_thumb32_compose(tinstr, tinst2); |
| 799 | 828 | thumb2_32b = 1; |
| 800 | 829 | } else { |
| @@ -803,8 +832,7 @@ do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs) · 2 unmodified lines | ||
| 803 | 832 | } |
| 804 | 833 | } |
| 805 | 834 | } else { |
| 806 | − fault = probe_kernel_address((void *)instrptr, instr); | |
| 807 | − instr = __mem_to_opcode_arm(instr); | |
| 835 | + fault = alignment_get_arm(regs, (void *)instrptr, &instr); | |
| 808 | 836 | } |
| 809 | 837 | |
| 810 | 838 | if (fault) { |
| @@ -926,7 +954,7 @@ do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs) · 115 unmodified lines | ||
| 926 | 954 | * Oops, we didn't handle the instruction. |
| 927 | 955 | */ |
| 928 | 956 | pr_err("Alignment trap: not handling instruction " |
| 929 | − "%0*lx at [<%08lx>]\n", | |
| 957 | + "%0*x at [<%08lx>]\n", | |
| 930 | 958 | isize << 1, |
| 931 | 959 | isize == 2 ? tinstr : instr, instrptr); |
| 932 | 960 | ai_skipped += 1; |
| @@ -936,7 +964,7 @@ do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs) · 3 unmodified lines | ||
| 936 | 964 | ai_user += 1; |
| 937 | 965 | |
| 938 | 966 | if (ai_usermode & UM_WARN) |
| 939 | − printk("Alignment trap: %s (%d) PC=0x%08lx Instr=0x%0*lx " | |
| 967 | + printk("Alignment trap: %s (%d) PC=0x%08lx Instr=0x%0*x " | |
| 940 | 968 | "Address=0x%08lx FSR 0x%03x\n", current->comm, |
| 941 | 969 | task_pid_nr(current), instrptr, |
| 942 | 970 | isize << 1, |
| @@ -132,13 +132,11 @@ __v7m_setup_cont: · 131 unmodified lines | ||
| 132 | 132 | dsb |
| 133 | 133 | mov r6, lr @ save LR |
| 134 | 134 | ldr sp, =init_thread_union + THREAD_START_SP |
| 135 | − stmia sp, {r0-r3, r12} | |
| 136 | 135 | cpsie i |
| 137 | 136 | svc #0 |
| 138 | 137 | 1: cpsid i |
| 139 | − ldr r0, =exc_ret | |
| 140 | − orr lr, lr, #EXC_RET_THREADMODE_PROCESSSTACK | |
| 141 | − str lr, [r0] | |
| 138 | + /* Calculate exc_ret */ | |
| 139 | + orr r10, lr, #EXC_RET_THREADMODE_PROCESSSTACK | |
| 142 | 140 | ldmia sp, {r0-r3, r12} |
| 143 | 141 | str r5, [r12, #11 * 4] @ restore the original SVC vector entry |
| 144 | 142 | mov lr, r6 @ restore LR |
| @@ -110,7 +110,6 @@ config ARM64 · 109 unmodified lines | ||
| 110 | 110 | select GENERIC_STRNLEN_USER |
| 111 | 111 | select GENERIC_TIME_VSYSCALL |
| 112 | 112 | select GENERIC_GETTIMEOFDAY |
| 113 | − select GENERIC_COMPAT_VDSO if (!CPU_BIG_ENDIAN && COMPAT) | |
| 114 | 113 | select HANDLE_DOMAIN_IRQ |
| 115 | 114 | select HARDIRQS_SW_RESEND |
| 116 | 115 | select HAVE_PCI |
| @@ -617,6 +616,23 @@ config CAVIUM_ERRATUM_30115 · 500 unmodified lines | ||
| 617 | 616 | |
| 618 | 617 | If unsure, say Y. |
| 619 | 618 | |
| 619 | +config CAVIUM_TX2_ERRATUM_219 | |
| 620 | + bool "Cavium ThunderX2 erratum 219: PRFM between TTBR change and ISB fails" | |
| 621 | + default y | |
| 622 | + help | |
| 623 | + On Cavium ThunderX2, a load, store or prefetch instruction between a | |
| 624 | + TTBR update and the corresponding context synchronizing operation can | |
| 625 | + cause a spurious Data Abort to be delivered to any hardware thread in | |
| 626 | + the CPU core. | |
| 627 | + | |
| 628 | + Work around the issue by avoiding the problematic code sequence and | |
| 629 | + trapping KVM guest TTBRx_EL1 writes to EL2 when SMT is enabled. The | |
| 630 | + trap handler performs the corresponding register access, skips the | |
| 631 | + instruction and ensures context synchronization by virtue of the | |
| 632 | + exception return. | |
| 633 | + | |
| 634 | + If unsure, say Y. | |
| 635 | + | |
| 620 | 636 | config QCOM_FALKOR_ERRATUM_1003 |
| 621 | 637 | bool "Falkor E1003: Incorrect translation due to ASID change" |
| 622 | 638 | default y |
| @@ -1159,7 +1175,7 @@ menuconfig COMPAT · 536 unmodified lines | ||
| 1159 | 1175 | if COMPAT |
| 1160 | 1176 | |
| 1161 | 1177 | config KUSER_HELPERS |
| 1162 | − bool "Enable kuser helpers page for 32 bit applications" | |
| 1178 | + bool "Enable kuser helpers page for 32-bit applications" | |
| 1163 | 1179 | default y |
| 1164 | 1180 | help |
| 1165 | 1181 | Warning: disabling this option may break 32-bit user programs. |
| @@ -1185,6 +1201,18 @@ config KUSER_HELPERS · 19 unmodified lines | ||
| 1185 | 1201 | Say N here only if you are absolutely certain that you do not |
| 1186 | 1202 | need these helpers; otherwise, the safe option is to say Y. |
| 1187 | 1203 | |
| 1204 | +config COMPAT_VDSO | |
| 1205 | + bool "Enable vDSO for 32-bit applications" | |
| 1206 | + depends on !CPU_BIG_ENDIAN && "$(CROSS_COMPILE_COMPAT)" != "" | |
| 1207 | + select GENERIC_COMPAT_VDSO | |
| 1208 | + default y | |
| 1209 | + help | |
| 1210 | + Place in the process address space of 32-bit applications an | |
| 1211 | + ELF shared object providing fast implementations of gettimeofday | |
| 1212 | + and clock_gettime. | |
| 1213 | + | |
| 1214 | + You must have a 32-bit build of glibc 2.22 or later for programs | |
| 1215 | + to seamlessly take advantage of this. | |
| 1188 | 1216 | |
| 1189 | 1217 | menuconfig ARMV8_DEPRECATED |
| 1190 | 1218 | bool "Emulate deprecated/obsolete ARMv8 instructions" |
| @@ -53,22 +53,6 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable) · 52 unmodified lines | ||
| 53 | 53 | endif |
| 54 | 54 | endif |
| 55 | 55 | |
| 56 | −ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y) | |
| 57 | − CROSS_COMPILE_COMPAT ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%) | |
| 58 | − | |
| 59 | − ifeq ($(CONFIG_CC_IS_CLANG), y) | |
| 60 | − $(warning CROSS_COMPILE_COMPAT is clang, the compat vDSO will not be built) | |
| 61 | − else ifeq ($(strip $(CROSS_COMPILE_COMPAT)),) | |
| 62 | − $(warning CROSS_COMPILE_COMPAT not defined or empty, the compat vDSO will not be built) | |
| 63 | − else ifeq ($(shell which $(CROSS_COMPILE_COMPAT)gcc 2> /dev/null),) | |
| 64 | − $(error $(CROSS_COMPILE_COMPAT)gcc not found, check CROSS_COMPILE_COMPAT) | |
| 65 | − else | |
| 66 | − export CROSS_COMPILE_COMPAT | |
| 67 | − export CONFIG_COMPAT_VDSO := y | |
| 68 | − compat_vdso := -DCONFIG_COMPAT_VDSO=1 | |
| 69 | − endif | |
| 70 | −endif | |
| 71 | − | |
| 72 | 56 | KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) \ |
| 73 | 57 | $(compat_vdso) $(cc_has_k_constraint) |
| 74 | 58 | KBUILD_CFLAGS += -fno-asynchronous-unwind-tables |
| @@ -63,3 +63,12 @@ · 62 unmodified lines | ||
| 63 | 63 | reg = <1>; |
| 64 | 64 | }; |
| 65 | 65 | }; |
| 66 | + | |
| 67 | +®_dc1sw { | |
| 68 | + /* | |
| 69 | + * Ethernet PHY needs 30ms to properly power up and some more | |
| 70 | + * to initialize. 100ms should be plenty of time to finish | |
| 71 | + * whole process. | |
| 72 | + */ | |
| 73 | + regulator-enable-ramp-delay = <100000>; | |
| 74 | +}; | |
| @@ -104,6 +104,7 @@ · 103 unmodified lines | ||
| 104 | 104 | |
| 105 | 105 | &ehci0 { |
| 106 | 106 | phys = <&usbphy 0>; |
| 107 | + phy-names = "usb"; | |
| 107 | 108 | status = "okay"; |
| 108 | 109 | }; |
| 109 | 110 | |
| @@ -150,6 +151,7 @@ · 40 unmodified lines | ||
| 150 | 151 | |
| 151 | 152 | &ohci0 { |
| 152 | 153 | phys = <&usbphy 0>; |
| 154 | + phy-names = "usb"; | |
| 153 | 155 | status = "okay"; |
| 154 | 156 | }; |
| 155 | 157 | |
| @@ -159,6 +159,12 @@ · 158 unmodified lines | ||
| 159 | 159 | }; |
| 160 | 160 | |
| 161 | 161 | ®_dc1sw { |
| 162 | + /* | |
| 163 | + * Ethernet PHY needs 30ms to properly power up and some more | |
| 164 | + * to initialize. 100ms should be plenty of time to finish | |
| 165 | + * whole process. | |
| 166 | + */ | |
| 167 | + regulator-enable-ramp-delay = <100000>; | |
| 162 | 168 | regulator-name = "vcc-phy"; |
| 163 | 169 | }; |
| 164 | 170 | |
| @@ -142,15 +142,6 @@ · 141 unmodified lines | ||
| 142 | 142 | clock-output-names = "ext-osc32k"; |
| 143 | 143 | }; |
| 144 | 144 | |
| 145 | − pmu { | |
| 146 | − compatible = "arm,cortex-a53-pmu"; | |
| 147 | − interrupts = <GIC_SPI 152 IRQ_TYPE_LEVEL_HIGH>, | |
| 148 | − <GIC_SPI 153 IRQ_TYPE_LEVEL_HIGH>, | |
| 149 | − <GIC_SPI 154 IRQ_TYPE_LEVEL_HIGH>, | |
| 150 | − <GIC_SPI 155 IRQ_TYPE_LEVEL_HIGH>; | |
| 151 | − interrupt-affinity = <&cpu0>, <&cpu1>, <&cpu2>, <&cpu3>; | |
| 152 | − }; | |
| 153 | − | |
| 154 | 145 | psci { |
| 155 | 146 | compatible = "arm,psci-0.2"; |
| 156 | 147 | method = "smc"; |
| @@ -553,6 +544,7 @@ · 396 unmodified lines | ||
| 553 | 544 | resets = <&ccu RST_BUS_OHCI1>, |
| 554 | 545 | <&ccu RST_BUS_EHCI1>; |
| 555 | 546 | phys = <&usbphy 1>; |
| 547 | + phy-names = "usb"; | |
| 556 | 548 | status = "disabled"; |
| 557 | 549 | }; |
| 558 | 550 | |
| @@ -564,6 +556,7 @@ · 5 unmodified lines | ||
| 564 | 556 | <&ccu CLK_USB_OHCI1>; |
| 565 | 557 | resets = <&ccu RST_BUS_OHCI1>; |
| 566 | 558 | phys = <&usbphy 1>; |
| 559 | + phy-names = "usb"; | |
| 567 | 560 | status = "disabled"; |
| 568 | 561 | }; |
| 569 | 562 | |
| @@ -547,6 +547,7 @@ · 546 unmodified lines | ||
| 547 | 547 | resets = <&ccu RST_BUS_OHCI3>, |
| 548 | 548 | <&ccu RST_BUS_EHCI3>; |
| 549 | 549 | phys = <&usb2phy 3>; |
| 550 | + phy-names = "usb"; | |
| 550 | 551 | status = "disabled"; |
| 551 | 552 | }; |
| 552 | 553 | |
| @@ -558,6 +559,7 @@ · 5 unmodified lines | ||
| 558 | 559 | <&ccu CLK_USB_OHCI3>; |
| 559 | 560 | resets = <&ccu RST_BUS_OHCI3>; |
| 560 | 561 | phys = <&usb2phy 3>; |
| 562 | + phy-names = "usb"; | |
| 561 | 563 | status = "disabled"; |
| 562 | 564 | }; |
| 563 | 565 | |
| @@ -42,13 +42,14 @@ · 41 unmodified lines | ||
| 42 | 42 | |
| 43 | 43 | pinmux: pinmux@14029c { |
| 44 | 44 | compatible = "pinctrl-single"; |
| 45 | − reg = <0x0014029c 0x250>; | |
| 45 | + reg = <0x0014029c 0x26c>; | |
| 46 | 46 | #address-cells = <1>; |
| 47 | 47 | #size-cells = <1>; |
| 48 | 48 | pinctrl-single,register-width = <32>; |
| 49 | 49 | pinctrl-single,function-mask = <0xf>; |
| 50 | 50 | pinctrl-single,gpio-range = < |
| 51 | − &range 0 154 MODE_GPIO | |
| 51 | + &range 0 91 MODE_GPIO | |
| 52 | + &range 95 60 MODE_GPIO | |
| 52 | 53 | >; |
| 53 | 54 | range: gpio-range { |
| 54 | 55 | #pinctrl-single,gpio-range-cells = <3>; |
| @@ -464,8 +464,7 @@ · 463 unmodified lines | ||
| 464 | 464 | <&pinmux 108 16 27>, |
| 465 | 465 | <&pinmux 135 77 6>, |
| 466 | 466 | <&pinmux 141 67 4>, |
| 467 | − <&pinmux 145 149 6>, | |
| 468 | − <&pinmux 151 91 4>; | |
| 467 | + <&pinmux 145 149 6>; | |
| 469 | 468 | }; |
| 470 | 469 | |
| 471 | 470 | i2c1: i2c@e0000 { |
| @@ -127,7 +127,7 @@ · 126 unmodified lines | ||
| 127 | 127 | status = "okay"; |
| 128 | 128 | |
| 129 | 129 | i2c-mux@77 { |
| 130 | − compatible = "nxp,pca9847"; | |
| 130 | + compatible = "nxp,pca9547"; | |
| 131 | 131 | reg = <0x77>; |
| 132 | 132 | #address-cells = <1>; |
| 133 | 133 | #size-cells = <0>; |
| @@ -33,7 +33,7 @@ · 32 unmodified lines | ||
| 33 | 33 | i-cache-line-size = <64>; |
| 34 | 34 | i-cache-sets = <192>; |
| 35 | 35 | next-level-cache = <&cluster0_l2>; |
| 36 | − cpu-idle-states = <&cpu_pw20>; | |
| 36 | + cpu-idle-states = <&cpu_pw15>; | |
| 37 | 37 | }; |
| 38 | 38 | |
| 39 | 39 | cpu@1 { |
| @@ -49,7 +49,7 @@ · 9 unmodified lines | ||
| 49 | 49 | i-cache-line-size = <64>; |
| 50 | 50 | i-cache-sets = <192>; |
| 51 | 51 | next-level-cache = <&cluster0_l2>; |
| 52 | − cpu-idle-states = <&cpu_pw20>; | |
| 52 | + cpu-idle-states = <&cpu_pw15>; | |
| 53 | 53 | }; |
| 54 | 54 | |
| 55 | 55 | cpu@100 { |
| @@ -65,7 +65,7 @@ · 9 unmodified lines | ||
| 65 | 65 | i-cache-line-size = <64>; |
| 66 | 66 | i-cache-sets = <192>; |
| 67 | 67 | next-level-cache = <&cluster1_l2>; |
| 68 | − cpu-idle-states = <&cpu_pw20>; | |
| 68 | + cpu-idle-states = <&cpu_pw15>; | |
| 69 | 69 | }; |
| 70 | 70 | |
| 71 | 71 | cpu@101 { |
| @@ -81,7 +81,7 @@ · 9 unmodified lines | ||
| 81 | 81 | i-cache-line-size = <64>; |
| 82 | 82 | i-cache-sets = <192>; |
| 83 | 83 | next-level-cache = <&cluster1_l2>; |
| 84 | − cpu-idle-states = <&cpu_pw20>; | |
| 84 | + cpu-idle-states = <&cpu_pw15>; | |
| 85 | 85 | }; |
| 86 | 86 | |
| 87 | 87 | cpu@200 { |
| @@ -97,7 +97,7 @@ · 9 unmodified lines | ||
| 97 | 97 | i-cache-line-size = <64>; |
| 98 | 98 | i-cache-sets = <192>; |
| 99 | 99 | next-level-cache = <&cluster2_l2>; |
| 100 | − cpu-idle-states = <&cpu_pw20>; | |
| 100 | + cpu-idle-states = <&cpu_pw15>; | |
| 101 | 101 | }; |
| 102 | 102 | |
| 103 | 103 | cpu@201 { |
| @@ -113,7 +113,7 @@ · 9 unmodified lines | ||
| 113 | 113 | i-cache-line-size = <64>; |
| 114 | 114 | i-cache-sets = <192>; |
| 115 | 115 | next-level-cache = <&cluster2_l2>; |
| 116 | − cpu-idle-states = <&cpu_pw20>; | |
| 116 | + cpu-idle-states = <&cpu_pw15>; | |
| 117 | 117 | }; |
| 118 | 118 | |
| 119 | 119 | cpu@300 { |
| @@ -129,7 +129,7 @@ · 9 unmodified lines | ||
| 129 | 129 | i-cache-line-size = <64>; |
| 130 | 130 | i-cache-sets = <192>; |
| 131 | 131 | next-level-cache = <&cluster3_l2>; |
| 132 | − cpu-idle-states = <&cpu_pw20>; | |
| 132 | + cpu-idle-states = <&cpu_pw15>; | |
| 133 | 133 | }; |
| 134 | 134 | |
| 135 | 135 | cpu@301 { |
| @@ -145,7 +145,7 @@ · 9 unmodified lines | ||
| 145 | 145 | i-cache-line-size = <64>; |
| 146 | 146 | i-cache-sets = <192>; |
| 147 | 147 | next-level-cache = <&cluster3_l2>; |
| 148 | − cpu-idle-states = <&cpu_pw20>; | |
| 148 | + cpu-idle-states = <&cpu_pw15>; | |
| 149 | 149 | }; |
| 150 | 150 | |
| 151 | 151 | cpu@400 { |
| @@ -161,7 +161,7 @@ · 9 unmodified lines | ||
| 161 | 161 | i-cache-line-size = <64>; |
| 162 | 162 | i-cache-sets = <192>; |
| 163 | 163 | next-level-cache = <&cluster4_l2>; |
| 164 | − cpu-idle-states = <&cpu_pw20>; | |
| 164 | + cpu-idle-states = <&cpu_pw15>; | |
| 165 | 165 | }; |
| 166 | 166 | |
| 167 | 167 | cpu@401 { |
| @@ -177,7 +177,7 @@ · 9 unmodified lines | ||
| 177 | 177 | i-cache-line-size = <64>; |
| 178 | 178 | i-cache-sets = <192>; |
| 179 | 179 | next-level-cache = <&cluster4_l2>; |
| 180 | − cpu-idle-states = <&cpu_pw20>; | |
| 180 | + cpu-idle-states = <&cpu_pw15>; | |
| 181 | 181 | }; |
| 182 | 182 | |
| 183 | 183 | cpu@500 { |
| @@ -193,7 +193,7 @@ · 9 unmodified lines | ||
| 193 | 193 | i-cache-line-size = <64>; |
| 194 | 194 | i-cache-sets = <192>; |
| 195 | 195 | next-level-cache = <&cluster5_l2>; |
| 196 | − cpu-idle-states = <&cpu_pw20>; | |
| 196 | + cpu-idle-states = <&cpu_pw15>; | |
| 197 | 197 | }; |
| 198 | 198 | |
| 199 | 199 | cpu@501 { |
| @@ -209,7 +209,7 @@ · 9 unmodified lines | ||
| 209 | 209 | i-cache-line-size = <64>; |
| 210 | 210 | i-cache-sets = <192>; |
| 211 | 211 | next-level-cache = <&cluster5_l2>; |
| 212 | − cpu-idle-states = <&cpu_pw20>; | |
| 212 | + cpu-idle-states = <&cpu_pw15>; | |
| 213 | 213 | }; |
| 214 | 214 | |
| 215 | 215 | cpu@600 { |
| @@ -225,7 +225,7 @@ · 9 unmodified lines | ||
| 225 | 225 | i-cache-line-size = <64>; |
| 226 | 226 | i-cache-sets = <192>; |
| 227 | 227 | next-level-cache = <&cluster6_l2>; |
| 228 | − cpu-idle-states = <&cpu_pw20>; | |
| 228 | + cpu-idle-states = <&cpu_pw15>; | |
| 229 | 229 | }; |
| 230 | 230 | |
| 231 | 231 | cpu@601 { |
| @@ -241,7 +241,7 @@ · 9 unmodified lines | ||
| 241 | 241 | i-cache-line-size = <64>; |
| 242 | 242 | i-cache-sets = <192>; |
| 243 | 243 | next-level-cache = <&cluster6_l2>; |
| 244 | − cpu-idle-states = <&cpu_pw20>; | |
| 244 | + cpu-idle-states = <&cpu_pw15>; | |
| 245 | 245 | }; |
| 246 | 246 | |
| 247 | 247 | cpu@700 { |
| @@ -257,7 +257,7 @@ · 9 unmodified lines | ||
| 257 | 257 | i-cache-line-size = <64>; |
| 258 | 258 | i-cache-sets = <192>; |
| 259 | 259 | next-level-cache = <&cluster7_l2>; |
| 260 | − cpu-idle-states = <&cpu_pw20>; | |
| 260 | + cpu-idle-states = <&cpu_pw15>; | |
| 261 | 261 | }; |
| 262 | 262 | |
| 263 | 263 | cpu@701 { |
| @@ -273,7 +273,7 @@ · 9 unmodified lines | ||
| 273 | 273 | i-cache-line-size = <64>; |
| 274 | 274 | i-cache-sets = <192>; |
| 275 | 275 | next-level-cache = <&cluster7_l2>; |
| 276 | − cpu-idle-states = <&cpu_pw20>; | |
| 276 | + cpu-idle-states = <&cpu_pw15>; | |
| 277 | 277 | }; |
| 278 | 278 | |
| 279 | 279 | cluster0_l2: l2-cache0 { |
| @@ -340,9 +340,9 @@ · 60 unmodified lines | ||
| 340 | 340 | cache-level = <2>; |
| 341 | 341 | }; |
| 342 | 342 | |
| 343 | − cpu_pw20: cpu-pw20 { | |
| 343 | + cpu_pw15: cpu-pw15 { | |
| 344 | 344 | compatible = "arm,idle-state"; |
| 345 | − idle-state-name = "PW20"; | |
| 345 | + idle-state-name = "PW15"; | |
| 346 | 346 | arm,psci-suspend-param = <0x0>; |
| 347 | 347 | entry-latency-us = <2000>; |
| 348 | 348 | exit-latency-us = <2000>; |
| @@ -394,7 +394,7 @@ · 393 unmodified lines | ||
| 394 | 394 | }; |
| 395 | 395 | |
| 396 | 396 | sdma2: dma-controller@302c0000 { |
| 397 | − compatible = "fsl,imx8mm-sdma", "fsl,imx7d-sdma"; | |
| 397 | + compatible = "fsl,imx8mm-sdma", "fsl,imx8mq-sdma"; | |
| 398 | 398 | reg = <0x302c0000 0x10000>; |
| 399 | 399 | interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>; |
| 400 | 400 | clocks = <&clk IMX8MM_CLK_SDMA2_ROOT>, |
| @@ -405,7 +405,7 @@ · 4 unmodified lines | ||
| 405 | 405 | }; |
| 406 | 406 | |
| 407 | 407 | sdma3: dma-controller@302b0000 { |
| 408 | − compatible = "fsl,imx8mm-sdma", "fsl,imx7d-sdma"; | |
| 408 | + compatible = "fsl,imx8mm-sdma", "fsl,imx8mq-sdma"; | |
| 409 | 409 | reg = <0x302b0000 0x10000>; |
| 410 | 410 | interrupts = <GIC_SPI 34 IRQ_TYPE_LEVEL_HIGH>; |
| 411 | 411 | clocks = <&clk IMX8MM_CLK_SDMA3_ROOT>, |
| @@ -694,7 +694,7 @@ · 282 unmodified lines | ||
| 694 | 694 | compatible = "fsl,imx8mm-usdhc", "fsl,imx7d-usdhc"; |
| 695 | 695 | reg = <0x30b40000 0x10000>; |
| 696 | 696 | interrupts = <GIC_SPI 22 IRQ_TYPE_LEVEL_HIGH>; |
| 697 | − clocks = <&clk IMX8MM_CLK_DUMMY>, | |
| 697 | + clocks = <&clk IMX8MM_CLK_IPG_ROOT>, | |
| 698 | 698 | <&clk IMX8MM_CLK_NAND_USDHC_BUS>, |
| 699 | 699 | <&clk IMX8MM_CLK_USDHC1_ROOT>; |
| 700 | 700 | clock-names = "ipg", "ahb", "per"; |
| @@ -710,7 +710,7 @@ · 9 unmodified lines | ||
| 710 | 710 | compatible = "fsl,imx8mm-usdhc", "fsl,imx7d-usdhc"; |
| 711 | 711 | reg = <0x30b50000 0x10000>; |
| 712 | 712 | interrupts = <GIC_SPI 23 IRQ_TYPE_LEVEL_HIGH>; |
| 713 | − clocks = <&clk IMX8MM_CLK_DUMMY>, | |
| 713 | + clocks = <&clk IMX8MM_CLK_IPG_ROOT>, | |
| 714 | 714 | <&clk IMX8MM_CLK_NAND_USDHC_BUS>, |
| 715 | 715 | <&clk IMX8MM_CLK_USDHC2_ROOT>; |
| 716 | 716 | clock-names = "ipg", "ahb", "per"; |
| @@ -724,7 +724,7 @@ · 7 unmodified lines | ||
| 724 | 724 | compatible = "fsl,imx8mm-usdhc", "fsl,imx7d-usdhc"; |
| 725 | 725 | reg = <0x30b60000 0x10000>; |
| 726 | 726 | interrupts = <GIC_SPI 24 IRQ_TYPE_LEVEL_HIGH>; |
| 727 | − clocks = <&clk IMX8MM_CLK_DUMMY>, | |
| 727 | + clocks = <&clk IMX8MM_CLK_IPG_ROOT>, | |
| 728 | 728 | <&clk IMX8MM_CLK_NAND_USDHC_BUS>, |
| 729 | 729 | <&clk IMX8MM_CLK_USDHC3_ROOT>; |
| 730 | 730 | clock-names = "ipg", "ahb", "per"; |
| @@ -737,7 +737,7 @@ · 6 unmodified lines | ||
| 737 | 737 | }; |
| 738 | 738 | |
| 739 | 739 | sdma1: dma-controller@30bd0000 { |
| 740 | − compatible = "fsl,imx8mm-sdma", "fsl,imx7d-sdma"; | |
| 740 | + compatible = "fsl,imx8mm-sdma", "fsl,imx8mq-sdma"; | |
| 741 | 741 | reg = <0x30bd0000 0x10000>; |
| 742 | 742 | interrupts = <GIC_SPI 2 IRQ_TYPE_LEVEL_HIGH>; |
| 743 | 743 | clocks = <&clk IMX8MM_CLK_SDMA1_ROOT>, |
| @@ -288,7 +288,7 @@ · 287 unmodified lines | ||
| 288 | 288 | }; |
| 289 | 289 | |
| 290 | 290 | sdma3: dma-controller@302b0000 { |
| 291 | − compatible = "fsl,imx8mn-sdma", "fsl,imx7d-sdma"; | |
| 291 | + compatible = "fsl,imx8mn-sdma", "fsl,imx8mq-sdma"; | |
| 292 | 292 | reg = <0x302b0000 0x10000>; |
| 293 | 293 | interrupts = <GIC_SPI 34 IRQ_TYPE_LEVEL_HIGH>; |
| 294 | 294 | clocks = <&clk IMX8MN_CLK_SDMA3_ROOT>, |
| @@ -299,7 +299,7 @@ · 4 unmodified lines | ||
| 299 | 299 | }; |
| 300 | 300 | |
| 301 | 301 | sdma2: dma-controller@302c0000 { |
| 302 | − compatible = "fsl,imx8mn-sdma", "fsl,imx7d-sdma"; | |
| 302 | + compatible = "fsl,imx8mn-sdma", "fsl,imx8mq-sdma"; | |
| 303 | 303 | reg = <0x302c0000 0x10000>; |
| 304 | 304 | interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>; |
| 305 | 305 | clocks = <&clk IMX8MN_CLK_SDMA2_ROOT>, |
| @@ -569,7 +569,7 @@ · 263 unmodified lines | ||
| 569 | 569 | compatible = "fsl,imx8mn-usdhc", "fsl,imx7d-usdhc"; |
| 570 | 570 | reg = <0x30b40000 0x10000>; |
| 571 | 571 | interrupts = <GIC_SPI 22 IRQ_TYPE_LEVEL_HIGH>; |
| 572 | − clocks = <&clk IMX8MN_CLK_DUMMY>, | |
| 572 | + clocks = <&clk IMX8MN_CLK_IPG_ROOT>, | |
| 573 | 573 | <&clk IMX8MN_CLK_NAND_USDHC_BUS>, |
| 574 | 574 | <&clk IMX8MN_CLK_USDHC1_ROOT>; |
| 575 | 575 | clock-names = "ipg", "ahb", "per"; |
| @@ -585,7 +585,7 @@ · 9 unmodified lines | ||
| 585 | 585 | compatible = "fsl,imx8mn-usdhc", "fsl,imx7d-usdhc"; |
| 586 | 586 | reg = <0x30b50000 0x10000>; |
| 587 | 587 | interrupts = <GIC_SPI 23 IRQ_TYPE_LEVEL_HIGH>; |
| 588 | − clocks = <&clk IMX8MN_CLK_DUMMY>, | |
| 588 | + clocks = <&clk IMX8MN_CLK_IPG_ROOT>, | |
| 589 | 589 | <&clk IMX8MN_CLK_NAND_USDHC_BUS>, |
| 590 | 590 | <&clk IMX8MN_CLK_USDHC2_ROOT>; |
| 591 | 591 | clock-names = "ipg", "ahb", "per"; |
| @@ -599,7 +599,7 @@ · 7 unmodified lines | ||
| 599 | 599 | compatible = "fsl,imx8mn-usdhc", "fsl,imx7d-usdhc"; |
| 600 | 600 | reg = <0x30b60000 0x10000>; |
| 601 | 601 | interrupts = <GIC_SPI 24 IRQ_TYPE_LEVEL_HIGH>; |
| 602 | − clocks = <&clk IMX8MN_CLK_DUMMY>, | |
| 602 | + clocks = <&clk IMX8MN_CLK_IPG_ROOT>, | |
| 603 | 603 | <&clk IMX8MN_CLK_NAND_USDHC_BUS>, |
| 604 | 604 | <&clk IMX8MN_CLK_USDHC3_ROOT>; |
| 605 | 605 | clock-names = "ipg", "ahb", "per"; |
| @@ -612,7 +612,7 @@ · 6 unmodified lines | ||
| 612 | 612 | }; |
| 613 | 613 | |
| 614 | 614 | sdma1: dma-controller@30bd0000 { |
| 615 | − compatible = "fsl,imx8mn-sdma", "fsl,imx7d-sdma"; | |
| 615 | + compatible = "fsl,imx8mn-sdma", "fsl,imx8mq-sdma"; | |
| 616 | 616 | reg = <0x30bd0000 0x10000>; |
| 617 | 617 | interrupts = <GIC_SPI 2 IRQ_TYPE_LEVEL_HIGH>; |
| 618 | 618 | clocks = <&clk IMX8MN_CLK_SDMA1_ROOT>, |
| @@ -88,9 +88,9 @@ · 87 unmodified lines | ||
| 88 | 88 | regulator-name = "0V9_ARM"; |
| 89 | 89 | regulator-min-microvolt = <900000>; |
| 90 | 90 | regulator-max-microvolt = <1000000>; |
| 91 | − gpios = <&gpio3 19 GPIO_ACTIVE_HIGH>; | |
| 92 | − states = <1000000 0x0 | |
| 93 | − 900000 0x1>; | |
| 91 | + gpios = <&gpio3 16 GPIO_ACTIVE_HIGH>; | |
| 92 | + states = <1000000 0x1 | |
| 93 | + 900000 0x0>; | |
| 94 | 94 | regulator-always-on; |
| 95 | 95 | }; |
| 96 | 96 | }; |
| @@ -850,7 +850,7 @@ · 849 unmodified lines | ||
| 850 | 850 | "fsl,imx7d-usdhc"; |
| 851 | 851 | reg = <0x30b40000 0x10000>; |
| 852 | 852 | interrupts = <GIC_SPI 22 IRQ_TYPE_LEVEL_HIGH>; |
| 853 | − clocks = <&clk IMX8MQ_CLK_DUMMY>, | |
| 853 | + clocks = <&clk IMX8MQ_CLK_IPG_ROOT>, | |
| 854 | 854 | <&clk IMX8MQ_CLK_NAND_USDHC_BUS>, |
| 855 | 855 | <&clk IMX8MQ_CLK_USDHC1_ROOT>; |
| 856 | 856 | clock-names = "ipg", "ahb", "per"; |
| @@ -867,7 +867,7 @@ · 10 unmodified lines | ||
| 867 | 867 | "fsl,imx7d-usdhc"; |
| 868 | 868 | reg = <0x30b50000 0x10000>; |
| 869 | 869 | interrupts = <GIC_SPI 23 IRQ_TYPE_LEVEL_HIGH>; |
| 870 | − clocks = <&clk IMX8MQ_CLK_DUMMY>, | |
| 870 | + clocks = <&clk IMX8MQ_CLK_IPG_ROOT>, | |
| 871 | 871 | <&clk IMX8MQ_CLK_NAND_USDHC_BUS>, |
| 872 | 872 | <&clk IMX8MQ_CLK_USDHC2_ROOT>; |
| 873 | 873 | clock-names = "ipg", "ahb", "per"; |
| @@ -60,11 +60,6 @@ · 59 unmodified lines | ||
| 60 | 60 | gpio = <&gpiosb 0 GPIO_ACTIVE_HIGH>; |
| 61 | 61 | }; |
| 62 | 62 | |
| 63 | − usb3_phy: usb3-phy { | |
| 64 | − compatible = "usb-nop-xceiv"; | |
| 65 | − vcc-supply = <&exp_usb3_vbus>; | |
| 66 | − }; | |
| 67 | − | |
| 68 | 63 | vsdc_reg: vsdc-reg { |
| 69 | 64 | compatible = "regulator-gpio"; |
| 70 | 65 | regulator-name = "vsdc"; |
| @@ -255,10 +250,16 @@ · 184 unmodified lines | ||
| 255 | 250 | status = "okay"; |
| 256 | 251 | }; |
| 257 | 252 | |
| 253 | +&comphy2 { | |
| 254 | + connector { | |
| 255 | + compatible = "usb-a-connector"; | |
| 256 | + phy-supply = <&exp_usb3_vbus>; | |
| 257 | + }; | |
| 258 | +}; | |
| 259 | + | |
| 258 | 260 | &usb3 { |
| 259 | 261 | status = "okay"; |
| 260 | 262 | phys = <&comphy2 0>; |
| 261 | − usb-phy = <&usb3_phy>; | |
| 262 | 263 | }; |
| 263 | 264 | |
| 264 | 265 | &mdio { |
| @@ -44,7 +44,7 @@ · 43 unmodified lines | ||
| 44 | 44 | power-supply = <&pp3300_disp>; |
| 45 | 45 | |
| 46 | 46 | panel-timing { |
| 47 | − clock-frequency = <266604720>; | |
| 47 | + clock-frequency = <266666667>; | |
| 48 | 48 | hactive = <2400>; |
| 49 | 49 | hfront-porch = <48>; |
| 50 | 50 | hback-porch = <84>; |
| @@ -644,7 +644,7 @@ · 643 unmodified lines | ||
| 644 | 644 | status = "okay"; |
| 645 | 645 | |
| 646 | 646 | u2phy0_host: host-port { |
| 647 | − phy-supply = <&vcc5v0_host>; | |
| 647 | + phy-supply = <&vcc5v0_typec>; | |
| 648 | 648 | status = "okay"; |
| 649 | 649 | }; |
| 650 | 650 | |
| @@ -712,7 +712,7 @@ · 61 unmodified lines | ||
| 712 | 712 | |
| 713 | 713 | &usbdrd_dwc3_0 { |
| 714 | 714 | status = "okay"; |
| 715 | − dr_mode = "otg"; | |
| 715 | + dr_mode = "host"; | |
| 716 | 716 | }; |
| 717 | 717 | |
| 718 | 718 | &usbdrd3_1 { |
| @@ -173,7 +173,7 @@ · 172 unmodified lines | ||
| 173 | 173 | regulator-always-on; |
| 174 | 174 | regulator-boot-on; |
| 175 | 175 | regulator-min-microvolt = <800000>; |
| 176 | − regulator-max-microvolt = <1400000>; | |
| 176 | + regulator-max-microvolt = <1700000>; | |
| 177 | 177 | vin-supply = <&vcc5v0_sys>; |
| 178 | 178 | }; |
| 179 | 179 | }; |
| @@ -247,8 +247,8 @@ · 67 unmodified lines | ||
| 247 | 247 | rk808: pmic@1b { |
| 248 | 248 | compatible = "rockchip,rk808"; |
| 249 | 249 | reg = <0x1b>; |
| 250 | − interrupt-parent = <&gpio1>; | |
| 251 | − interrupts = <21 IRQ_TYPE_LEVEL_LOW>; | |
| 250 | + interrupt-parent = <&gpio3>; | |
| 251 | + interrupts = <10 IRQ_TYPE_LEVEL_LOW>; | |
| 252 | 252 | #clock-cells = <1>; |
| 253 | 253 | clock-output-names = "xin32k", "rk808-clkout2"; |
| 254 | 254 | pinctrl-names = "default"; |
| @@ -574,7 +574,7 @@ · 319 unmodified lines | ||
| 574 | 574 | |
| 575 | 575 | pmic { |
| 576 | 576 | pmic_int_l: pmic-int-l { |
| 577 | − rockchip,pins = <1 RK_PC5 RK_FUNC_GPIO &pcfg_pull_up>; | |
| 577 | + rockchip,pins = <3 RK_PB2 RK_FUNC_GPIO &pcfg_pull_up>; | |
| 578 | 578 | }; |
| 579 | 579 | |
| 580 | 580 | vsel1_gpio: vsel1-gpio { |
| @@ -624,7 +624,6 @@ · 43 unmodified lines | ||
| 624 | 624 | |
| 625 | 625 | &sdmmc { |
| 626 | 626 | bus-width = <4>; |
| 627 | − cap-mmc-highspeed; | |
| 628 | 627 | cap-sd-highspeed; |
| 629 | 628 | cd-gpios = <&gpio0 7 GPIO_ACTIVE_LOW>; |
| 630 | 629 | disable-wp; |
| @@ -636,8 +635,7 @@ · 5 unmodified lines | ||
| 636 | 635 | |
| 637 | 636 | &sdhci { |
| 638 | 637 | bus-width = <8>; |
| 639 | − mmc-hs400-1_8v; | |
| 640 | − mmc-hs400-enhanced-strobe; | |
| 638 | + mmc-hs200-1_8v; | |
| 641 | 639 | non-removable; |
| 642 | 640 | status = "okay"; |
| 643 | 641 | }; |
| @@ -78,10 +78,9 @@ alternative_else_nop_endif · 77 unmodified lines | ||
| 78 | 78 | /* |
| 79 | 79 | * Remove the address tag from a virtual address, if present. |
| 80 | 80 | */ |
| 81 | − .macro clear_address_tag, dst, addr | |
| 82 | − tst \addr, #(1 << 55) | |
| 83 | − bic \dst, \addr, #(0xff << 56) | |
| 84 | − csel \dst, \dst, \addr, eq | |
| 81 | + .macro untagged_addr, dst, addr | |
| 82 | + sbfx \dst, \addr, #0, #56 | |
| 83 | + and \dst, \dst, \addr | |
| 85 | 84 | .endm |
| 86 | 85 | |
| 87 | 86 | #endif |
| @@ -321,7 +321,8 @@ static inline s64 __lse_atomic64_dec_if_positive(atomic64_t *v) · 320 unmodified lines | ||
| 321 | 321 | } |
| 322 | 322 | |
| 323 | 323 | #define __CMPXCHG_CASE(w, sfx, name, sz, mb, cl...) \ |
| 324 | −static inline u##sz __lse__cmpxchg_case_##name##sz(volatile void *ptr, \ | |
| 324 | +static __always_inline u##sz \ | |
| 325 | +__lse__cmpxchg_case_##name##sz(volatile void *ptr, \ | |
| 325 | 326 | u##sz old, \ |
| 326 | 327 | u##sz new) \ |
| 327 | 328 | { \ |
| @@ -362,7 +363,8 @@ __CMPXCHG_CASE(x, , mb_, 64, al, "memory") · 34 unmodified lines | ||
| 362 | 363 | #undef __CMPXCHG_CASE |
| 363 | 364 | |
| 364 | 365 | #define __CMPXCHG_DBL(name, mb, cl...) \ |
| 365 | −static inline long __lse__cmpxchg_double##name(unsigned long old1, \ | |
| 366 | +static __always_inline long \ | |
| 367 | +__lse__cmpxchg_double##name(unsigned long old1, \ | |
| 366 | 368 | unsigned long old2, \ |
| 367 | 369 | unsigned long new1, \ |
| 368 | 370 | unsigned long new2, \ |
| @@ -52,7 +52,9 @@ · 51 unmodified lines | ||
| 52 | 52 | #define ARM64_HAS_IRQ_PRIO_MASKING 42 |
| 53 | 53 | #define ARM64_HAS_DCPODP 43 |
| 54 | 54 | #define ARM64_WORKAROUND_1463225 44 |
| 55 | +#define ARM64_WORKAROUND_CAVIUM_TX2_219_TVM 45 | |
| 56 | +#define ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM 46 | |
| 55 | 57 | |
| 56 | −#define ARM64_NCAPS 45 | |
| 58 | +#define ARM64_NCAPS 47 | |
| 57 | 59 | |
| 58 | 60 | #endif /* __ASM_CPUCAPS_H */ |
| @@ -79,6 +79,7 @@ · 78 unmodified lines | ||
| 79 | 79 | #define CAVIUM_CPU_PART_THUNDERX_83XX 0x0A3 |
| 80 | 80 | #define CAVIUM_CPU_PART_THUNDERX2 0x0AF |
| 81 | 81 | |
| 82 | +#define BRCM_CPU_PART_BRAHMA_B53 0x100 | |
| 82 | 83 | #define BRCM_CPU_PART_VULCAN 0x516 |
| 83 | 84 | |
| 84 | 85 | #define QCOM_CPU_PART_FALKOR_V1 0x800 |
| @@ -105,6 +106,7 @@ · 20 unmodified lines | ||
| 105 | 106 | #define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX) |
| 106 | 107 | #define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX) |
| 107 | 108 | #define MIDR_CAVIUM_THUNDERX2 MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX2) |
| 109 | +#define MIDR_BRAHMA_B53 MIDR_CPU_MODEL(ARM_CPU_IMP_BRCM, BRCM_CPU_PART_BRAHMA_B53) | |
| 108 | 110 | #define MIDR_BRCM_VULCAN MIDR_CPU_MODEL(ARM_CPU_IMP_BRCM, BRCM_CPU_PART_VULCAN) |
| 109 | 111 | #define MIDR_QCOM_FALKOR_V1 MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_FALKOR_V1) |
| 110 | 112 | #define MIDR_QCOM_FALKOR MIDR_CPU_MODEL(ARM_CPU_IMP_QCOM, QCOM_CPU_PART_FALKOR) |
| @@ -215,12 +215,18 @@ static inline unsigned long kaslr_offset(void) · 214 unmodified lines | ||
| 215 | 215 | * up with a tagged userland pointer. Clear the tag to get a sane pointer to |
| 216 | 216 | * pass on to access_ok(), for instance. |
| 217 | 217 | */ |
| 218 | −#define untagged_addr(addr) \ | |
| 218 | +#define __untagged_addr(addr) \ | |
| 219 | 219 | ((__force __typeof__(addr))sign_extend64((__force u64)(addr), 55)) |
| 220 | 220 | |
| 221 | +#define untagged_addr(addr) ({ \ | |
| 222 | + u64 __addr = (__force u64)addr; \ | |
| 223 | + __addr &= __untagged_addr(__addr); \ | |
| 224 | + (__force __typeof__(addr))__addr; \ | |
| 225 | +}) | |
| 226 | + | |
| 221 | 227 | #ifdef CONFIG_KASAN_SW_TAGS |
| 222 | 228 | #define __tag_shifted(tag) ((u64)(tag) << 56) |
| 223 | −#define __tag_reset(addr) untagged_addr(addr) | |
| 229 | +#define __tag_reset(addr) __untagged_addr(addr) | |
| 224 | 230 | #define __tag_get(addr) (__u8)((u64)(addr) >> 56) |
| 225 | 231 | #else |
| 226 | 232 | #define __tag_shifted(tag) 0UL |
| @@ -32,11 +32,11 @@ · 31 unmodified lines | ||
| 32 | 32 | #define PROT_DEFAULT (_PROT_DEFAULT | PTE_MAYBE_NG) |
| 33 | 33 | #define PROT_SECT_DEFAULT (_PROT_SECT_DEFAULT | PMD_MAYBE_NG) |
| 34 | 34 | |
| 35 | −#define PROT_DEVICE_nGnRnE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRnE)) | |
| 36 | −#define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRE)) | |
| 37 | −#define PROT_NORMAL_NC (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_NC)) | |
| 38 | −#define PROT_NORMAL_WT (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_WT)) | |
| 39 | −#define PROT_NORMAL (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_DIRTY | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL)) | |
| 35 | +#define PROT_DEVICE_nGnRnE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRnE)) | |
| 36 | +#define PROT_DEVICE_nGnRE (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_DEVICE_nGnRE)) | |
| 37 | +#define PROT_NORMAL_NC (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_NC)) | |
| 38 | +#define PROT_NORMAL_WT (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL_WT)) | |
| 39 | +#define PROT_NORMAL (PROT_DEFAULT | PTE_PXN | PTE_UXN | PTE_WRITE | PTE_ATTRINDX(MT_NORMAL)) | |
| 40 | 40 | |
| 41 | 41 | #define PROT_SECT_DEVICE_nGnRE (PROT_SECT_DEFAULT | PMD_SECT_PXN | PMD_SECT_UXN | PMD_ATTRINDX(MT_DEVICE_nGnRE)) |
| 42 | 42 | #define PROT_SECT_NORMAL (PROT_SECT_DEFAULT | PMD_SECT_PXN | PMD_SECT_UXN | PMD_ATTRINDX(MT_NORMAL)) |
| @@ -80,8 +80,9 @@ · 37 unmodified lines | ||
| 80 | 80 | #define PAGE_S2_DEVICE __pgprot(_PROT_DEFAULT | PAGE_S2_MEMATTR(DEVICE_nGnRE) | PTE_S2_RDONLY | PTE_S2_XN) |
| 81 | 81 | |
| 82 | 82 | #define PAGE_NONE __pgprot(((_PAGE_DEFAULT) & ~PTE_VALID) | PTE_PROT_NONE | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN) |
| 83 | −#define PAGE_SHARED __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE) | |
| 84 | −#define PAGE_SHARED_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_NG | PTE_PXN | PTE_WRITE) | |
| 83 | +/* shared+writable pages are clean by default, hence PTE_RDONLY|PTE_WRITE */ | |
| 84 | +#define PAGE_SHARED __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN | PTE_WRITE) | |
| 85 | +#define PAGE_SHARED_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_WRITE) | |
| 85 | 86 | #define PAGE_READONLY __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN | PTE_UXN) |
| 86 | 87 | #define PAGE_READONLY_EXEC __pgprot(_PAGE_DEFAULT | PTE_USER | PTE_RDONLY | PTE_NG | PTE_PXN) |
| 87 | 88 | #define PAGE_EXECONLY __pgprot(_PAGE_DEFAULT | PTE_RDONLY | PTE_NG | PTE_PXN) |
| @@ -283,23 +283,6 @@ static inline void set_pte_at(struct mm_struct *mm, unsigned long addr, · 282 unmodified lines | ||
| 283 | 283 | set_pte(ptep, pte); |
| 284 | 284 | } |
| 285 | 285 | |
| 286 | −#define __HAVE_ARCH_PTE_SAME | |
| 287 | −static inline int pte_same(pte_t pte_a, pte_t pte_b) | |
| 288 | −{ | |
| 289 | − pteval_t lhs, rhs; | |
| 290 | − | |
| 291 | − lhs = pte_val(pte_a); | |
| 292 | − rhs = pte_val(pte_b); | |
| 293 | − | |
| 294 | − if (pte_present(pte_a)) | |
| 295 | − lhs &= ~PTE_RDONLY; | |
| 296 | − | |
| 297 | − if (pte_present(pte_b)) | |
| 298 | − rhs &= ~PTE_RDONLY; | |
| 299 | − | |
| 300 | − return (lhs == rhs); | |
| 301 | −} | |
| 302 | − | |
| 303 | 286 | /* |
| 304 | 287 | * Huge pte definitions. |
| 305 | 288 | */ |
| @@ -876,9 +859,6 @@ static inline void update_mmu_cache(struct vm_area_struct *vma, · 570 unmodified lines | ||
| 876 | 859 | |
| 877 | 860 | #define update_mmu_cache_pmd(vma, address, pmd) do { } while (0) |
| 878 | 861 | |
| 879 | −#define kc_vaddr_to_offset(v) ((v) & ~PAGE_END) | |
| 880 | −#define kc_offset_to_vaddr(o) ((o) | PAGE_END) | |
| 881 | − | |
| 882 | 862 | #ifdef CONFIG_ARM64_PA_BITS_52 |
| 883 | 863 | #define phys_to_ttbr(addr) (((addr) | ((addr) >> 46)) & TTBR_BADDR_MASK_52) |
| 884 | 864 | #else |
| @@ -212,7 +212,7 @@ · 211 unmodified lines | ||
| 212 | 212 | #define SYS_FAR_EL1 sys_reg(3, 0, 6, 0, 0) |
| 213 | 213 | #define SYS_PAR_EL1 sys_reg(3, 0, 7, 4, 0) |
| 214 | 214 | |
| 215 | −#define SYS_PAR_EL1_F BIT(1) | |
| 215 | +#define SYS_PAR_EL1_F BIT(0) | |
| 216 | 216 | #define SYS_PAR_EL1_FST GENMASK(6, 1) |
| 217 | 217 | |
| 218 | 218 | /*** Statistical Profiling Extension ***/ |
| @@ -20,7 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | |
| 21 | 21 | #define dmb(option) __asm__ __volatile__ ("dmb " #option : : : "memory") |
| 22 | 22 | |
| 23 | −#if __LINUX_ARM_ARCH__ >= 8 | |
| 23 | +#if __LINUX_ARM_ARCH__ >= 8 && defined(CONFIG_AS_DMB_ISHLD) | |
| 24 | 24 | #define aarch32_smp_mb() dmb(ish) |
| 25 | 25 | #define aarch32_smp_rmb() dmb(ishld) |
| 26 | 26 | #define aarch32_smp_wmb() dmb(ishst) |
| @@ -30,13 +30,6 @@ int __arm64_get_clock_mode(struct timekeeper *tk) · 29 unmodified lines | ||
| 30 | 30 | } |
| 31 | 31 | #define __arch_get_clock_mode __arm64_get_clock_mode |
| 32 | 32 | |
| 33 | −static __always_inline | |
| 34 | −int __arm64_use_vsyscall(struct vdso_data *vdata) | |
| 35 | −{ | |
| 36 | − return !vdata[CS_HRES_COARSE].clock_mode; | |
| 37 | −} | |
| 38 | −#define __arch_use_vsyscall __arm64_use_vsyscall | |
| 39 | − | |
| 40 | 33 | static __always_inline |
| 41 | 34 | void __arm64_update_vsyscall(struct vdso_data *vdata, struct timekeeper *tk) |
| 42 | 35 | { |
| @@ -1,33 +0,0 @@ | ||
| 1 | −/* SPDX-License-Identifier: GPL-2.0-only */ | |
| 2 | −/* | |
| 3 | − * Copyright (C) 2012 ARM Limited | |
| 4 | − */ | |
| 5 | −#ifndef __ASM_VDSO_DATAPAGE_H | |
| 6 | −#define __ASM_VDSO_DATAPAGE_H | |
| 7 | − | |
| 8 | −#ifndef __ASSEMBLY__ | |
| 9 | − | |
| 10 | −struct vdso_data { | |
| 11 | − __u64 cs_cycle_last; /* Timebase at clocksource init */ | |
| 12 | − __u64 raw_time_sec; /* Raw time */ | |
| 13 | − __u64 raw_time_nsec; | |
| 14 | − __u64 xtime_clock_sec; /* Kernel time */ | |
| 15 | − __u64 xtime_clock_nsec; | |
| 16 | − __u64 xtime_coarse_sec; /* Coarse time */ | |
| 17 | − __u64 xtime_coarse_nsec; | |
| 18 | − __u64 wtm_clock_sec; /* Wall to monotonic time */ | |
| 19 | − __u64 wtm_clock_nsec; | |
| 20 | − __u32 tb_seq_count; /* Timebase sequence counter */ | |
| 21 | − /* cs_* members must be adjacent and in this order (ldp accesses) */ | |
| 22 | − __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */ | |
| 23 | − __u32 cs_shift; /* Clocksource shift (mono = raw) */ | |
| 24 | − __u32 cs_raw_mult; /* Raw clocksource multiplier */ | |
| 25 | − __u32 tz_minuteswest; /* Whacky timezone stuff */ | |
| 26 | − __u32 tz_dsttime; | |
| 27 | − __u32 use_syscall; | |
| 28 | − __u32 hrtimer_res; | |
| 29 | −}; | |
| 30 | − | |
| 31 | −#endif /* !__ASSEMBLY__ */ | |
| 32 | − | |
| 33 | −#endif /* __ASM_VDSO_DATAPAGE_H */ | |
| @@ -174,6 +174,9 @@ static void __init register_insn_emulation(struct insn_emulation_ops *ops) · 173 unmodified lines | ||
| 174 | 174 | struct insn_emulation *insn; |
| 175 | 175 | |
| 176 | 176 | insn = kzalloc(sizeof(*insn), GFP_KERNEL); |
| 177 | + if (!insn) | |
| 178 | + return; | |
| 179 | + | |
| 177 | 180 | insn->ops = ops; |
| 178 | 181 | insn->min = INSN_UNDEF; |
| 179 | 182 | |
| @@ -233,6 +236,8 @@ static void __init register_insn_emulation_sysctl(void) · 53 unmodified lines | ||
| 233 | 236 | |
| 234 | 237 | insns_sysctl = kcalloc(nr_insn_emulated + 1, sizeof(*sysctl), |
| 235 | 238 | GFP_KERNEL); |
| 239 | + if (!insns_sysctl) | |
| 240 | + return; | |
| 236 | 241 | |
| 237 | 242 | raw_spin_lock_irqsave(&insn_emulation_lock, flags); |
| 238 | 243 | list_for_each_entry(insn, &insn_emulation, node) { |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <asm/cpu.h> |
| 13 | 13 | #include <asm/cputype.h> |
| 14 | 14 | #include <asm/cpufeature.h> |
| 15 | +#include <asm/smp_plat.h> | |
| 15 | 16 | |
| 16 | 17 | static bool __maybe_unused |
| 17 | 18 | is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope) |
| @@ -128,8 +129,8 @@ static void install_bp_hardening_cb(bp_hardening_cb_t fn, · 110 unmodified lines | ||
| 128 | 129 | int cpu, slot = -1; |
| 129 | 130 | |
| 130 | 131 | /* |
| 131 | − * enable_smccc_arch_workaround_1() passes NULL for the hyp_vecs | |
| 132 | − * start/end if we're a guest. Skip the hyp-vectors work. | |
| 132 | + * detect_harden_bp_fw() passes NULL for the hyp_vecs start/end if | |
| 133 | + * we're a guest. Skip the hyp-vectors work. | |
| 133 | 134 | */ |
| 134 | 135 | if (!hyp_vecs_start) { |
| 135 | 136 | __this_cpu_write(bp_hardening_data.fn, fn); |
| @@ -488,6 +489,7 @@ static const struct midr_range arm64_ssb_cpus[] = { · 352 unmodified lines | ||
| 488 | 489 | MIDR_ALL_VERSIONS(MIDR_CORTEX_A35), |
| 489 | 490 | MIDR_ALL_VERSIONS(MIDR_CORTEX_A53), |
| 490 | 491 | MIDR_ALL_VERSIONS(MIDR_CORTEX_A55), |
| 492 | + MIDR_ALL_VERSIONS(MIDR_BRAHMA_B53), | |
| 491 | 493 | {}, |
| 492 | 494 | }; |
| 493 | 495 | |
| @@ -572,6 +574,7 @@ static const struct midr_range spectre_v2_safe_list[] = { · 78 unmodified lines | ||
| 572 | 574 | MIDR_ALL_VERSIONS(MIDR_CORTEX_A35), |
| 573 | 575 | MIDR_ALL_VERSIONS(MIDR_CORTEX_A53), |
| 574 | 576 | MIDR_ALL_VERSIONS(MIDR_CORTEX_A55), |
| 577 | + MIDR_ALL_VERSIONS(MIDR_BRAHMA_B53), | |
| 575 | 578 | { /* sentinel */ } |
| 576 | 579 | }; |
| 577 | 580 | |
| @@ -623,6 +626,30 @@ check_branch_predictor(const struct arm64_cpu_capabilities *entry, int scope) · 45 unmodified lines | ||
| 623 | 626 | return (need_wa > 0); |
| 624 | 627 | } |
| 625 | 628 | |
| 629 | +static const __maybe_unused struct midr_range tx2_family_cpus[] = { | |
| 630 | + MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN), | |
| 631 | + MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2), | |
| 632 | + {}, | |
| 633 | +}; | |
| 634 | + | |
| 635 | +static bool __maybe_unused | |
| 636 | +needs_tx2_tvm_workaround(const struct arm64_cpu_capabilities *entry, | |
| 637 | + int scope) | |
| 638 | +{ | |
| 639 | + int i; | |
| 640 | + | |
| 641 | + if (!is_affected_midr_range_list(entry, scope) || | |
| 642 | + !is_hyp_mode_available()) | |
| 643 | + return false; | |
| 644 | + | |
| 645 | + for_each_possible_cpu(i) { | |
| 646 | + if (MPIDR_AFFINITY_LEVEL(cpu_logical_map(i), 0) != 0) | |
| 647 | + return true; | |
| 648 | + } | |
| 649 | + | |
| 650 | + return false; | |
| 651 | +} | |
| 652 | + | |
| 626 | 653 | #ifdef CONFIG_HARDEN_EL2_VECTORS |
| 627 | 654 | |
| 628 | 655 | static const struct midr_range arm64_harden_el2_vectors[] = { |
| @@ -634,17 +661,23 @@ static const struct midr_range arm64_harden_el2_vectors[] = { · 5 unmodified lines | ||
| 634 | 661 | #endif |
| 635 | 662 | |
| 636 | 663 | #ifdef CONFIG_ARM64_WORKAROUND_REPEAT_TLBI |
| 637 | − | |
| 638 | −static const struct midr_range arm64_repeat_tlbi_cpus[] = { | |
| 664 | +static const struct arm64_cpu_capabilities arm64_repeat_tlbi_list[] = { | |
| 639 | 665 | #ifdef CONFIG_QCOM_FALKOR_ERRATUM_1009 |
| 640 | − MIDR_RANGE(MIDR_QCOM_FALKOR_V1, 0, 0, 0, 0), | |
| 666 | + { | |
| 667 | + ERRATA_MIDR_REV(MIDR_QCOM_FALKOR_V1, 0, 0) | |
| 668 | + }, | |
| 669 | + { | |
| 670 | + .midr_range.model = MIDR_QCOM_KRYO, | |
| 671 | + .matches = is_kryo_midr, | |
| 672 | + }, | |
| 641 | 673 | #endif |
| 642 | 674 | #ifdef CONFIG_ARM64_ERRATUM_1286807 |
| 643 | − MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 3, 0), | |
| 675 | + { | |
| 676 | + ERRATA_MIDR_RANGE(MIDR_CORTEX_A76, 0, 0, 3, 0), | |
| 677 | + }, | |
| 644 | 678 | #endif |
| 645 | 679 | {}, |
| 646 | 680 | }; |
| 647 | − | |
| 648 | 681 | #endif |
| 649 | 682 | |
| 650 | 683 | #ifdef CONFIG_CAVIUM_ERRATUM_27456 |
| @@ -712,6 +745,33 @@ static const struct midr_range erratum_1418040_list[] = { · 61 unmodified lines | ||
| 712 | 745 | }; |
| 713 | 746 | #endif |
| 714 | 747 | |
| 748 | +#ifdef CONFIG_ARM64_ERRATUM_845719 | |
| 749 | +static const struct midr_range erratum_845719_list[] = { | |
| 750 | + /* Cortex-A53 r0p[01234] */ | |
| 751 | + MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4), | |
| 752 | + /* Brahma-B53 r0p[0] */ | |
| 753 | + MIDR_REV(MIDR_BRAHMA_B53, 0, 0), | |
| 754 | + {}, | |
| 755 | +}; | |
| 756 | +#endif | |
| 757 | + | |
| 758 | +#ifdef CONFIG_ARM64_ERRATUM_843419 | |
| 759 | +static const struct arm64_cpu_capabilities erratum_843419_list[] = { | |
| 760 | + { | |
| 761 | + /* Cortex-A53 r0p[01234] */ | |
| 762 | + .matches = is_affected_midr_range, | |
| 763 | + ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4), | |
| 764 | + MIDR_FIXED(0x4, BIT(8)), | |
| 765 | + }, | |
| 766 | + { | |
| 767 | + /* Brahma-B53 r0p[0] */ | |
| 768 | + .matches = is_affected_midr_range, | |
| 769 | + ERRATA_MIDR_REV(MIDR_BRAHMA_B53, 0, 0), | |
| 770 | + }, | |
| 771 | + {}, | |
| 772 | +}; | |
| 773 | +#endif | |
| 774 | + | |
| 715 | 775 | const struct arm64_cpu_capabilities arm64_errata[] = { |
| 716 | 776 | #ifdef CONFIG_ARM64_WORKAROUND_CLEAN_CACHE |
| 717 | 777 | { |
| @@ -743,19 +803,18 @@ const struct arm64_cpu_capabilities arm64_errata[] = { · 25 unmodified lines | ||
| 743 | 803 | #endif |
| 744 | 804 | #ifdef CONFIG_ARM64_ERRATUM_843419 |
| 745 | 805 | { |
| 746 | − /* Cortex-A53 r0p[01234] */ | |
| 747 | 806 | .desc = "ARM erratum 843419", |
| 748 | 807 | .capability = ARM64_WORKAROUND_843419, |
| 749 | − ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4), | |
| 750 | − MIDR_FIXED(0x4, BIT(8)), | |
| 808 | + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, | |
| 809 | + .matches = cpucap_multi_entry_cap_matches, | |
| 810 | + .match_list = erratum_843419_list, | |
| 751 | 811 | }, |
| 752 | 812 | #endif |
| 753 | 813 | #ifdef CONFIG_ARM64_ERRATUM_845719 |
| 754 | 814 | { |
| 755 | − /* Cortex-A53 r0p[01234] */ | |
| 756 | 815 | .desc = "ARM erratum 845719", |
| 757 | 816 | .capability = ARM64_WORKAROUND_845719, |
| 758 | − ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4), | |
| 817 | + ERRATA_MIDR_RANGE_LIST(erratum_845719_list), | |
| 759 | 818 | }, |
| 760 | 819 | #endif |
| 761 | 820 | #ifdef CONFIG_CAVIUM_ERRATUM_23154 |
| @@ -791,6 +850,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = { · 29 unmodified lines | ||
| 791 | 850 | { |
| 792 | 851 | .desc = "Qualcomm Technologies Falkor/Kryo erratum 1003", |
| 793 | 852 | .capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003, |
| 853 | + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, | |
| 794 | 854 | .matches = cpucap_multi_entry_cap_matches, |
| 795 | 855 | .match_list = qcom_erratum_1003_list, |
| 796 | 856 | }, |
| @@ -799,7 +859,9 @@ const struct arm64_cpu_capabilities arm64_errata[] = { · 2 unmodified lines | ||
| 799 | 859 | { |
| 800 | 860 | .desc = "Qualcomm erratum 1009, ARM erratum 1286807", |
| 801 | 861 | .capability = ARM64_WORKAROUND_REPEAT_TLBI, |
| 802 | − ERRATA_MIDR_RANGE_LIST(arm64_repeat_tlbi_cpus), | |
| 862 | + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, | |
| 863 | + .matches = cpucap_multi_entry_cap_matches, | |
| 864 | + .match_list = arm64_repeat_tlbi_list, | |
| 803 | 865 | }, |
| 804 | 866 | #endif |
| 805 | 867 | #ifdef CONFIG_ARM64_ERRATUM_858921 |
| @@ -851,6 +913,19 @@ const struct arm64_cpu_capabilities arm64_errata[] = { · 45 unmodified lines | ||
| 851 | 913 | .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, |
| 852 | 914 | .matches = has_cortex_a76_erratum_1463225, |
| 853 | 915 | }, |
| 916 | +#endif | |
| 917 | +#ifdef CONFIG_CAVIUM_TX2_ERRATUM_219 | |
| 918 | + { | |
| 919 | + .desc = "Cavium ThunderX2 erratum 219 (KVM guest sysreg trapping)", | |
| 920 | + .capability = ARM64_WORKAROUND_CAVIUM_TX2_219_TVM, | |
| 921 | + ERRATA_MIDR_RANGE_LIST(tx2_family_cpus), | |
| 922 | + .matches = needs_tx2_tvm_workaround, | |
| 923 | + }, | |
| 924 | + { | |
| 925 | + .desc = "Cavium ThunderX2 erratum 219 (PRFM removal)", | |
| 926 | + .capability = ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM, | |
| 927 | + ERRATA_MIDR_RANGE_LIST(tx2_family_cpus), | |
| 928 | + }, | |
| 854 | 929 | #endif |
| 855 | 930 | { |
| 856 | 931 | } |
| @@ -136,6 +136,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar0[] = { · 135 unmodified lines | ||
| 136 | 136 | |
| 137 | 137 | static const struct arm64_ftr_bits ftr_id_aa64isar1[] = { |
| 138 | 138 | ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SB_SHIFT, 4, 0), |
| 139 | + ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FRINTTS_SHIFT, 4, 0), | |
| 139 | 140 | ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH), |
| 140 | 141 | FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_GPI_SHIFT, 4, 0), |
| 141 | 142 | ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_PTR_AUTH), |
| @@ -175,11 +176,16 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr1[] = { · 33 unmodified lines | ||
| 175 | 176 | }; |
| 176 | 177 | |
| 177 | 178 | static const struct arm64_ftr_bits ftr_id_aa64zfr0[] = { |
| 178 | − ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SM4_SHIFT, 4, 0), | |
| 179 | − ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SHA3_SHIFT, 4, 0), | |
| 180 | − ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BITPERM_SHIFT, 4, 0), | |
| 181 | − ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_AES_SHIFT, 4, 0), | |
| 182 | − ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SVEVER_SHIFT, 4, 0), | |
| 179 | + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE), | |
| 180 | + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SM4_SHIFT, 4, 0), | |
| 181 | + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE), | |
| 182 | + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SHA3_SHIFT, 4, 0), | |
| 183 | + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE), | |
| 184 | + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_BITPERM_SHIFT, 4, 0), | |
| 185 | + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE), | |
| 186 | + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_AES_SHIFT, 4, 0), | |
| 187 | + ARM64_FTR_BITS(FTR_VISIBLE_IF_IS_ENABLED(CONFIG_ARM64_SVE), | |
| 188 | + FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ZFR0_SVEVER_SHIFT, 4, 0), | |
| 183 | 189 | ARM64_FTR_END, |
| 184 | 190 | }; |
| 185 | 191 | |
| @@ -604,7 +604,7 @@ el1_da: · 603 unmodified lines | ||
| 604 | 604 | */ |
| 605 | 605 | mrs x3, far_el1 |
| 606 | 606 | inherit_daif pstate=x23, tmp=x2 |
| 607 | − clear_address_tag x0, x3 | |
| 607 | + untagged_addr x0, x3 | |
| 608 | 608 | mov x2, sp // struct pt_regs |
| 609 | 609 | bl do_mem_abort |
| 610 | 610 | |
| @@ -680,7 +680,7 @@ alternative_if ARM64_HAS_IRQ_PRIO_MASKING · 69 unmodified lines | ||
| 680 | 680 | orr x24, x24, x0 |
| 681 | 681 | alternative_else_nop_endif |
| 682 | 682 | cbnz x24, 1f // preempt count != 0 || NMI return path |
| 683 | − bl preempt_schedule_irq // irq en/disable is done inside | |
| 683 | + bl arm64_preempt_schedule_irq // irq en/disable is done inside | |
| 684 | 684 | 1: |
| 685 | 685 | #endif |
| 686 | 686 | |
| @@ -775,6 +775,7 @@ el0_sync_compat: · 88 unmodified lines | ||
| 775 | 775 | b.ge el0_dbg |
| 776 | 776 | b el0_inv |
| 777 | 777 | el0_svc_compat: |
| 778 | + gic_prio_kentry_setup tmp=x1 | |
| 778 | 779 | mov x0, sp |
| 779 | 780 | bl el0_svc_compat_handler |
| 780 | 781 | b ret_to_user |
| @@ -807,7 +808,7 @@ el0_da: · 26 unmodified lines | ||
| 807 | 808 | mrs x26, far_el1 |
| 808 | 809 | ct_user_exit_irqoff |
| 809 | 810 | enable_daif |
| 810 | − clear_address_tag x0, x26 | |
| 811 | + untagged_addr x0, x26 | |
| 811 | 812 | mov x1, x25 |
| 812 | 813 | mov x2, sp |
| 813 | 814 | bl do_mem_abort |
| @@ -1070,7 +1071,9 @@ alternative_insn isb, nop, ARM64_WORKAROUND_QCOM_FALKOR_E1003 · 256 unmodified lines | ||
| 1070 | 1071 | #else |
| 1071 | 1072 | ldr x30, =vectors |
| 1072 | 1073 | #endif |
| 1074 | +alternative_if_not ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM | |
| 1073 | 1075 | prfm plil1strm, [x30, #(1b - tramp_vectors)] |
| 1076 | +alternative_else_nop_endif | |
| 1074 | 1077 | msr vbar_el1, x30 |
| 1075 | 1078 | add x30, x30, #(1b - tramp_vectors) |
| 1076 | 1079 | isb |
| @@ -121,10 +121,16 @@ int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) · 120 unmodified lines | ||
| 121 | 121 | |
| 122 | 122 | /* |
| 123 | 123 | * Ensure updated trampoline is visible to instruction |
| 124 | − * fetch before we patch in the branch. | |
| 124 | + * fetch before we patch in the branch. Although the | |
| 125 | + * architecture doesn't require an IPI in this case, | |
| 126 | + * Neoverse-N1 erratum #1542419 does require one | |
| 127 | + * if the TLB maintenance in module_enable_ro() is | |
| 128 | + * skipped due to rodata_enabled. It doesn't seem worth | |
| 129 | + * it to make it conditional given that this is | |
| 130 | + * certainly not a fast-path. | |
| 125 | 131 | */ |
| 126 | − __flush_icache_range((unsigned long)&dst[0], | |
| 127 | − (unsigned long)&dst[1]); | |
| 132 | + flush_icache_range((unsigned long)&dst[0], | |
| 133 | + (unsigned long)&dst[1]); | |
| 128 | 134 | } |
| 129 | 135 | addr = (unsigned long)dst; |
| 130 | 136 | #else /* CONFIG_ARM64_MODULE_PLTS */ |
| @@ -201,6 +201,7 @@ static int create_safe_exec_page(void *src_start, size_t length, · 200 unmodified lines | ||
| 201 | 201 | gfp_t mask) |
| 202 | 202 | { |
| 203 | 203 | int rc = 0; |
| 204 | + pgd_t *trans_pgd; | |
| 204 | 205 | pgd_t *pgdp; |
| 205 | 206 | pud_t *pudp; |
| 206 | 207 | pmd_t *pmdp; |
| @@ -215,7 +216,13 @@ static int create_safe_exec_page(void *src_start, size_t length, · 8 unmodified lines | ||
| 215 | 216 | memcpy((void *)dst, src_start, length); |
| 216 | 217 | __flush_icache_range(dst, dst + length); |
| 217 | 218 | |
| 218 | − pgdp = pgd_offset_raw(allocator(mask), dst_addr); | |
| 219 | + trans_pgd = allocator(mask); | |
| 220 | + if (!trans_pgd) { | |
| 221 | + rc = -ENOMEM; | |
| 222 | + goto out; | |
| 223 | + } | |
| 224 | + | |
| 225 | + pgdp = pgd_offset_raw(trans_pgd, dst_addr); | |
| 219 | 226 | if (pgd_none(READ_ONCE(*pgdp))) { |
| 220 | 227 | pudp = allocator(mask); |
| 221 | 228 | if (!pudp) { |
| @@ -17,6 +17,7 @@ · 16 unmodified lines | ||
| 17 | 17 | #include <linux/sched/task.h> |
| 18 | 18 | #include <linux/sched/task_stack.h> |
| 19 | 19 | #include <linux/kernel.h> |
| 20 | +#include <linux/lockdep.h> | |
| 20 | 21 | #include <linux/mm.h> |
| 21 | 22 | #include <linux/stddef.h> |
| 22 | 23 | #include <linux/sysctl.h> |
| @@ -44,6 +45,7 @@ · 21 unmodified lines | ||
| 44 | 45 | #include <asm/alternative.h> |
| 45 | 46 | #include <asm/arch_gicv3.h> |
| 46 | 47 | #include <asm/compat.h> |
| 48 | +#include <asm/cpufeature.h> | |
| 47 | 49 | #include <asm/cacheflush.h> |
| 48 | 50 | #include <asm/exec.h> |
| 49 | 51 | #include <asm/fpsimd.h> |
| @@ -332,22 +334,27 @@ void arch_release_task_struct(struct task_struct *tsk) · 282 unmodified lines | ||
| 332 | 334 | fpsimd_release_task(tsk); |
| 333 | 335 | } |
| 334 | 336 | |
| 335 | −/* | |
| 336 | − * src and dst may temporarily have aliased sve_state after task_struct | |
| 337 | − * is copied. We cannot fix this properly here, because src may have | |
| 338 | − * live SVE state and dst's thread_info may not exist yet, so tweaking | |
| 339 | − * either src's or dst's TIF_SVE is not safe. | |
| 340 | − * | |
| 341 | − * The unaliasing is done in copy_thread() instead. This works because | |
| 342 | − * dst is not schedulable or traceable until both of these functions | |
| 343 | − * have been called. | |
| 344 | − */ | |
| 345 | 337 | int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) |
| 346 | 338 | { |
| 347 | 339 | if (current->mm) |
| 348 | 340 | fpsimd_preserve_current_state(); |
| 349 | 341 | *dst = *src; |
| 350 | 342 | |
| 343 | + /* We rely on the above assignment to initialize dst's thread_flags: */ | |
| 344 | + BUILD_BUG_ON(!IS_ENABLED(CONFIG_THREAD_INFO_IN_TASK)); | |
| 345 | + | |
| 346 | + /* | |
| 347 | + * Detach src's sve_state (if any) from dst so that it does not | |
| 348 | + * get erroneously used or freed prematurely. dst's sve_state | |
| 349 | + * will be allocated on demand later on if dst uses SVE. | |
| 350 | + * For consistency, also clear TIF_SVE here: this could be done | |
| 351 | + * later in copy_process(), but to avoid tripping up future | |
| 352 | + * maintainers it is best not to leave TIF_SVE and sve_state in | |
| 353 | + * an inconsistent state, even temporarily. | |
| 354 | + */ | |
| 355 | + dst->thread.sve_state = NULL; | |
| 356 | + clear_tsk_thread_flag(dst, TIF_SVE); | |
| 357 | + | |
| 351 | 358 | return 0; |
| 352 | 359 | } |
| 353 | 360 | |
| @@ -360,13 +367,6 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start, · 6 unmodified lines | ||
| 360 | 367 | |
| 361 | 368 | memset(&p->thread.cpu_context, 0, sizeof(struct cpu_context)); |
| 362 | 369 | |
| 363 | − /* | |
| 364 | − * Unalias p->thread.sve_state (if any) from the parent task | |
| 365 | − * and disable discard SVE state for p: | |
| 366 | − */ | |
| 367 | − clear_tsk_thread_flag(p, TIF_SVE); | |
| 368 | − p->thread.sve_state = NULL; | |
| 369 | − | |
| 370 | 370 | /* |
| 371 | 371 | * In case p was allocated the same task_struct pointer as some |
| 372 | 372 | * other recently-exited task, make sure p is disassociated from |
| @@ -633,3 +633,19 @@ static int __init tagged_addr_init(void) · 260 unmodified lines | ||
| 633 | 633 | |
| 634 | 634 | core_initcall(tagged_addr_init); |
| 635 | 635 | #endif /* CONFIG_ARM64_TAGGED_ADDR_ABI */ |
| 636 | + | |
| 637 | +asmlinkage void __sched arm64_preempt_schedule_irq(void) | |
| 638 | +{ | |
| 639 | + lockdep_assert_irqs_disabled(); | |
| 640 | + | |
| 641 | + /* | |
| 642 | + * Preempting a task from an IRQ means we leave copies of PSTATE | |
| 643 | + * on the stack. cpufeature's enable calls may modify PSTATE, but | |
| 644 | + * resuming one of these preempted tasks would undo those changes. | |
| 645 | + * | |
| 646 | + * Only allow a task to be preempted once cpufeatures have been | |
| 647 | + * enabled. | |
| 648 | + */ | |
| 649 | + if (static_branch_likely(&arm64_const_caps_ready)) | |
| 650 | + preempt_schedule_irq(); | |
| 651 | +} | |
| @@ -8,15 +8,21 @@ · 7 unmodified lines | ||
| 8 | 8 | ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32 |
| 9 | 9 | include $(srctree)/lib/vdso/Makefile |
| 10 | 10 | |
| 11 | −COMPATCC := $(CROSS_COMPILE_COMPAT)gcc | |
| 11 | +# Same as cc-*option, but using CC_COMPAT instead of CC | |
| 12 | +ifeq ($(CONFIG_CC_IS_CLANG), y) | |
| 13 | +CC_COMPAT ?= $(CC) | |
| 14 | +else | |
| 15 | +CC_COMPAT ?= $(CROSS_COMPILE_COMPAT)gcc | |
| 16 | +endif | |
| 12 | 17 | |
| 13 | −# Same as cc-*option, but using COMPATCC instead of CC | |
| 14 | 18 | cc32-option = $(call try-run,\ |
| 15 | − $(COMPATCC) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2)) | |
| 19 | + $(CC_COMPAT) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2)) | |
| 16 | 20 | cc32-disable-warning = $(call try-run,\ |
| 17 | − $(COMPATCC) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1))) | |
| 21 | + $(CC_COMPAT) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1))) | |
| 18 | 22 | cc32-ldoption = $(call try-run,\ |
| 19 | − $(COMPATCC) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2)) | |
| 23 | + $(CC_COMPAT) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2)) | |
| 24 | +cc32-as-instr = $(call try-run,\ | |
| 25 | + printf "%b\n" "$(1)" | $(CC_COMPAT) $(VDSO_AFLAGS) -c -x assembler -o "$$TMP" -,$(2),$(3)) | |
| 20 | 26 | |
| 21 | 27 | # We cannot use the global flags to compile the vDSO files, the main reason |
| 22 | 28 | # being that the 32-bit compiler may be older than the main (64-bit) compiler |
| @@ -25,22 +31,21 @@ cc32-ldoption = $(call try-run,\ · 2 unmodified lines | ||
| 25 | 31 | # arm64 one. |
| 26 | 32 | # As a result we set our own flags here. |
| 27 | 33 | |
| 28 | −# From top-level Makefile | |
| 29 | −# NOSTDINC_FLAGS | |
| 30 | −VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(COMPATCC) -print-file-name=include) | |
| 34 | +# KBUILD_CPPFLAGS and NOSTDINC_FLAGS from top-level Makefile | |
| 35 | +VDSO_CPPFLAGS := -D__KERNEL__ -nostdinc -isystem $(shell $(CC_COMPAT) -print-file-name=include) | |
| 31 | 36 | VDSO_CPPFLAGS += $(LINUXINCLUDE) |
| 32 | −VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS) | |
| 33 | 37 | |
| 34 | 38 | # Common C and assembly flags |
| 35 | 39 | # From top-level Makefile |
| 36 | 40 | VDSO_CAFLAGS := $(VDSO_CPPFLAGS) |
| 41 | +ifneq ($(shell $(CC_COMPAT) --version 2>&1 | head -n 1 | grep clang),) | |
| 42 | +VDSO_CAFLAGS += --target=$(notdir $(CROSS_COMPILE_COMPAT:%-=%)) | |
| 43 | +endif | |
| 44 | + | |
| 37 | 45 | VDSO_CAFLAGS += $(call cc32-option,-fno-PIE) |
| 38 | 46 | ifdef CONFIG_DEBUG_INFO |
| 39 | 47 | VDSO_CAFLAGS += -g |
| 40 | 48 | endif |
| 41 | −ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(COMPATCC)), y) | |
| 42 | −VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO | |
| 43 | −endif | |
| 44 | 49 | |
| 45 | 50 | # From arm Makefile |
| 46 | 51 | VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm) |
| @@ -55,6 +60,7 @@ endif · 8 unmodified lines | ||
| 55 | 60 | VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector |
| 56 | 61 | VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING |
| 57 | 62 | |
| 63 | + | |
| 58 | 64 | # Try to compile for ARMv8. If the compiler is too old and doesn't support it, |
| 59 | 65 | # fall back to v7. There is no easy way to check for what architecture the code |
| 60 | 66 | # is being compiled, so define a macro specifying that (see arch/arm/Makefile). |
| @@ -91,6 +97,12 @@ VDSO_CFLAGS += -Wno-int-to-pointer-cast · 30 unmodified lines | ||
| 91 | 97 | VDSO_AFLAGS := $(VDSO_CAFLAGS) |
| 92 | 98 | VDSO_AFLAGS += -D__ASSEMBLY__ |
| 93 | 99 | |
| 100 | +# Check for binutils support for dmb ishld | |
| 101 | +dmbinstr := $(call cc32-as-instr,dmb ishld,-DCONFIG_AS_DMB_ISHLD=1) | |
| 102 | + | |
| 103 | +VDSO_CFLAGS += $(dmbinstr) | |
| 104 | +VDSO_AFLAGS += $(dmbinstr) | |
| 105 | + | |
| 94 | 106 | VDSO_LDFLAGS := $(VDSO_CPPFLAGS) |
| 95 | 107 | # From arm vDSO Makefile |
| 96 | 108 | VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1 |
| @@ -159,14 +171,14 @@ quiet_cmd_vdsold_and_vdso_check = LD32 $@ · 62 unmodified lines | ||
| 159 | 171 | cmd_vdsold_and_vdso_check = $(cmd_vdsold); $(cmd_vdso_check) |
| 160 | 172 | |
| 161 | 173 | quiet_cmd_vdsold = LD32 $@ |
| 162 | − cmd_vdsold = $(COMPATCC) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \ | |
| 174 | + cmd_vdsold = $(CC_COMPAT) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \ | |
| 163 | 175 | -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@ |
| 164 | 176 | quiet_cmd_vdsocc = CC32 $@ |
| 165 | − cmd_vdsocc = $(COMPATCC) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $< | |
| 177 | + cmd_vdsocc = $(CC_COMPAT) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $< | |
| 166 | 178 | quiet_cmd_vdsocc_gettimeofday = CC32 $@ |
| 167 | − cmd_vdsocc_gettimeofday = $(COMPATCC) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $< | |
| 179 | + cmd_vdsocc_gettimeofday = $(CC_COMPAT) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $< | |
| 168 | 180 | quiet_cmd_vdsoas = AS32 $@ |
| 169 | − cmd_vdsoas = $(COMPATCC) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $< | |
| 181 | + cmd_vdsoas = $(CC_COMPAT) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $< | |
| 170 | 182 | |
| 171 | 183 | quiet_cmd_vdsomunge = MUNGE $@ |
| 172 | 184 | cmd_vdsomunge = $(obj)/$(munge) $< $@ |
| @@ -124,6 +124,9 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu) · 123 unmodified lines | ||
| 124 | 124 | { |
| 125 | 125 | u64 hcr = vcpu->arch.hcr_el2; |
| 126 | 126 | |
| 127 | + if (cpus_have_const_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM)) | |
| 128 | + hcr |= HCR_TVM; | |
| 129 | + | |
| 127 | 130 | write_sysreg(hcr, hcr_el2); |
| 128 | 131 | |
| 129 | 132 | if (cpus_have_const_cap(ARM64_HAS_RAS_EXTN) && (hcr & HCR_VSE)) |
| @@ -174,8 +177,10 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu) · 44 unmodified lines | ||
| 174 | 177 | * the crucial bit is "On taking a vSError interrupt, |
| 175 | 178 | * HCR_EL2.VSE is cleared to 0." |
| 176 | 179 | */ |
| 177 | − if (vcpu->arch.hcr_el2 & HCR_VSE) | |
| 178 | − vcpu->arch.hcr_el2 = read_sysreg(hcr_el2); | |
| 180 | + if (vcpu->arch.hcr_el2 & HCR_VSE) { | |
| 181 | + vcpu->arch.hcr_el2 &= ~HCR_VSE; | |
| 182 | + vcpu->arch.hcr_el2 |= read_sysreg(hcr_el2) & HCR_VSE; | |
| 183 | + } | |
| 179 | 184 | |
| 180 | 185 | if (has_vhe()) |
| 181 | 186 | deactivate_traps_vhe(); |
| @@ -380,6 +385,61 @@ static bool __hyp_text __hyp_handle_fpsimd(struct kvm_vcpu *vcpu) · 198 unmodified lines | ||
| 380 | 385 | return true; |
| 381 | 386 | } |
| 382 | 387 | |
| 388 | +static bool __hyp_text handle_tx2_tvm(struct kvm_vcpu *vcpu) | |
| 389 | +{ | |
| 390 | + u32 sysreg = esr_sys64_to_sysreg(kvm_vcpu_get_hsr(vcpu)); | |
| 391 | + int rt = kvm_vcpu_sys_get_rt(vcpu); | |
| 392 | + u64 val = vcpu_get_reg(vcpu, rt); | |
| 393 | + | |
| 394 | + /* | |
| 395 | + * The normal sysreg handling code expects to see the traps, | |
| 396 | + * let's not do anything here. | |
| 397 | + */ | |
| 398 | + if (vcpu->arch.hcr_el2 & HCR_TVM) | |
| 399 | + return false; | |
| 400 | + | |
| 401 | + switch (sysreg) { | |
| 402 | + case SYS_SCTLR_EL1: | |
| 403 | + write_sysreg_el1(val, SYS_SCTLR); | |
| 404 | + break; | |
| 405 | + case SYS_TTBR0_EL1: | |
| 406 | + write_sysreg_el1(val, SYS_TTBR0); | |
| 407 | + break; | |
| 408 | + case SYS_TTBR1_EL1: | |
| 409 | + write_sysreg_el1(val, SYS_TTBR1); | |
| 410 | + break; | |
| 411 | + case SYS_TCR_EL1: | |
| 412 | + write_sysreg_el1(val, SYS_TCR); | |
| 413 | + break; | |
| 414 | + case SYS_ESR_EL1: | |
| 415 | + write_sysreg_el1(val, SYS_ESR); | |
| 416 | + break; | |
| 417 | + case SYS_FAR_EL1: | |
| 418 | + write_sysreg_el1(val, SYS_FAR); | |
| 419 | + break; | |
| 420 | + case SYS_AFSR0_EL1: | |
| 421 | + write_sysreg_el1(val, SYS_AFSR0); | |
| 422 | + break; | |
| 423 | + case SYS_AFSR1_EL1: | |
| 424 | + write_sysreg_el1(val, SYS_AFSR1); | |
| 425 | + break; | |
| 426 | + case SYS_MAIR_EL1: | |
| 427 | + write_sysreg_el1(val, SYS_MAIR); | |
| 428 | + break; | |
| 429 | + case SYS_AMAIR_EL1: | |
| 430 | + write_sysreg_el1(val, SYS_AMAIR); | |
| 431 | + break; | |
| 432 | + case SYS_CONTEXTIDR_EL1: | |
| 433 | + write_sysreg_el1(val, SYS_CONTEXTIDR); | |
| 434 | + break; | |
| 435 | + default: | |
| 436 | + return false; | |
| 437 | + } | |
| 438 | + | |
| 439 | + __kvm_skip_instr(vcpu); | |
| 440 | + return true; | |
| 441 | +} | |
| 442 | + | |
| 383 | 443 | /* |
| 384 | 444 | * Return true when we were able to fixup the guest exit and should return to |
| 385 | 445 | * the guest, false when we should restore the host state and return to the |
| @@ -399,6 +459,11 @@ static bool __hyp_text fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) · 13 unmodified lines | ||
| 399 | 459 | if (*exit_code != ARM_EXCEPTION_TRAP) |
| 400 | 460 | goto exit; |
| 401 | 461 | |
| 462 | + if (cpus_have_const_cap(ARM64_WORKAROUND_CAVIUM_TX2_219_TVM) && | |
| 463 | + kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_SYS64 && | |
| 464 | + handle_tx2_tvm(vcpu)) | |
| 465 | + return true; | |
| 466 | + | |
| 402 | 467 | /* |
| 403 | 468 | * We trap the first access to the FP/SIMD to save the host context |
| 404 | 469 | * and restore the guest context lazily. |
| @@ -632,6 +632,8 @@ static void reset_pmcr(struct kvm_vcpu *vcpu, const struct sys_reg_desc *r) · 631 unmodified lines | ||
| 632 | 632 | */ |
| 633 | 633 | val = ((pmcr & ~ARMV8_PMU_PMCR_MASK) |
| 634 | 634 | | (ARMV8_PMU_PMCR_MASK & 0xdecafbad)) & (~ARMV8_PMU_PMCR_E); |
| 635 | + if (!system_supports_32bit_el0()) | |
| 636 | + val |= ARMV8_PMU_PMCR_LC; | |
| 635 | 637 | __vcpu_sys_reg(vcpu, r->reg) = val; |
| 636 | 638 | } |
| 637 | 639 | |
| @@ -682,6 +684,8 @@ static bool access_pmcr(struct kvm_vcpu *vcpu, struct sys_reg_params *p, · 44 unmodified lines | ||
| 682 | 684 | val = __vcpu_sys_reg(vcpu, PMCR_EL0); |
| 683 | 685 | val &= ~ARMV8_PMU_PMCR_MASK; |
| 684 | 686 | val |= p->regval & ARMV8_PMU_PMCR_MASK; |
| 687 | + if (!system_supports_32bit_el0()) | |
| 688 | + val |= ARMV8_PMU_PMCR_LC; | |
| 685 | 689 | __vcpu_sys_reg(vcpu, PMCR_EL0) = val; |
| 686 | 690 | kvm_pmu_handle_pmcr(vcpu, val); |
| 687 | 691 | kvm_vcpu_pmu_restore_guest(vcpu); |
| @@ -113,6 +113,15 @@ static inline bool is_ttbr1_addr(unsigned long addr) · 112 unmodified lines | ||
| 113 | 113 | return arch_kasan_reset_tag(addr) >= PAGE_OFFSET; |
| 114 | 114 | } |
| 115 | 115 | |
| 116 | +static inline unsigned long mm_to_pgd_phys(struct mm_struct *mm) | |
| 117 | +{ | |
| 118 | + /* Either init_pg_dir or swapper_pg_dir */ | |
| 119 | + if (mm == &init_mm) | |
| 120 | + return __pa_symbol(mm->pgd); | |
| 121 | + | |
| 122 | + return (unsigned long)virt_to_phys(mm->pgd); | |
| 123 | +} | |
| 124 | + | |
| 116 | 125 | /* |
| 117 | 126 | * Dump out the page tables associated with 'addr' in the currently active mm. |
| 118 | 127 | */ |
| @@ -141,7 +150,7 @@ static void show_pte(unsigned long addr) · 22 unmodified lines | ||
| 141 | 150 | |
| 142 | 151 | pr_alert("%s pgtable: %luk pages, %llu-bit VAs, pgdp=%016lx\n", |
| 143 | 152 | mm == &init_mm ? "swapper" : "user", PAGE_SIZE / SZ_1K, |
| 144 | − vabits_actual, (unsigned long)virt_to_phys(mm->pgd)); | |
| 153 | + vabits_actual, mm_to_pgd_phys(mm)); | |
| 145 | 154 | pgdp = pgd_offset(mm, addr); |
| 146 | 155 | pgd = READ_ONCE(*pgdp); |
| 147 | 156 | pr_alert("[%016lx] pgd=%016llx", addr, pgd_val(pgd)); |
| @@ -259,14 +268,18 @@ static bool __kprobes is_spurious_el1_translation_fault(unsigned long addr, · 111 unmodified lines | ||
| 259 | 268 | par = read_sysreg(par_el1); |
| 260 | 269 | local_irq_restore(flags); |
| 261 | 270 | |
| 271 | + /* | |
| 272 | + * If we now have a valid translation, treat the translation fault as | |
| 273 | + * spurious. | |
| 274 | + */ | |
| 262 | 275 | if (!(par & SYS_PAR_EL1_F)) |
| 263 | − return false; | |
| 276 | + return true; | |
| 264 | 277 | |
| 265 | 278 | /* |
| 266 | 279 | * If we got a different type of fault from the AT instruction, |
| 267 | 280 | * treat the translation fault as spurious. |
| 268 | 281 | */ |
| 269 | − dfsc = FIELD_PREP(SYS_PAR_EL1_FST, par); | |
| 282 | + dfsc = FIELD_GET(SYS_PAR_EL1_FST, par); | |
| 270 | 283 | return (dfsc & ESR_ELx_FSC_TYPE) != ESR_ELx_FSC_FAULT; |
| 271 | 284 | } |
| 272 | 285 | |
| @@ -84,7 +84,7 @@ void __init prom_init(void) · 83 unmodified lines | ||
| 84 | 84 | * Here we will start up CPU1 in the background and ask it to |
| 85 | 85 | * reconfigure itself then go back to sleep. |
| 86 | 86 | */ |
| 87 | − memcpy((void *)0xa0000200, &bmips_smp_movevec, 0x20); | |
| 87 | + memcpy((void *)0xa0000200, bmips_smp_movevec, 0x20); | |
| 88 | 88 | __sync(); |
| 89 | 89 | set_c0_cause(C_SW0); |
| 90 | 90 | cpumask_set_cpu(1, &bmips_booted_mask); |
| @@ -571,7 +571,6 @@ CONFIG_USB_SERIAL_OMNINET=m · 570 unmodified lines | ||
| 571 | 571 | CONFIG_USB_EMI62=m |
| 572 | 572 | CONFIG_USB_EMI26=m |
| 573 | 573 | CONFIG_USB_ADUTUX=m |
| 574 | −CONFIG_USB_RIO500=m | |
| 575 | 574 | CONFIG_USB_LEGOTOWER=m |
| 576 | 575 | CONFIG_USB_LCD=m |
| 577 | 576 | CONFIG_USB_CYPRESS_CY7C63=m |
| @@ -314,7 +314,6 @@ CONFIG_USB_SERIAL_SAFE_PADDED=y · 313 unmodified lines | ||
| 314 | 314 | CONFIG_USB_SERIAL_CYBERJACK=m |
| 315 | 315 | CONFIG_USB_SERIAL_XIRCOM=m |
| 316 | 316 | CONFIG_USB_SERIAL_OMNINET=m |
| 317 | −CONFIG_USB_RIO500=m | |
| 318 | 317 | CONFIG_USB_LEGOTOWER=m |
| 319 | 318 | CONFIG_USB_LCD=m |
| 320 | 319 | CONFIG_USB_CYTHERM=m |
| @@ -43,7 +43,7 @@ · 42 unmodified lines | ||
| 43 | 43 | |
| 44 | 44 | /* O32 stack has to be 8-byte aligned. */ |
| 45 | 45 | static u64 o32_stk[4096]; |
| 46 | −#define O32_STK &o32_stk[sizeof(o32_stk)] | |
| 46 | +#define O32_STK (&o32_stk[ARRAY_SIZE(o32_stk)]) | |
| 47 | 47 | |
| 48 | 48 | #define __PROM_O32(fun, arg) fun arg __asm__(#fun); \ |
| 49 | 49 | __asm__(#fun " = call_o32") |
| @@ -75,11 +75,11 @@ static inline int register_bmips_smp_ops(void) · 74 unmodified lines | ||
| 75 | 75 | #endif |
| 76 | 76 | } |
| 77 | 77 | |
| 78 | −extern char bmips_reset_nmi_vec; | |
| 79 | −extern char bmips_reset_nmi_vec_end; | |
| 80 | −extern char bmips_smp_movevec; | |
| 81 | −extern char bmips_smp_int_vec; | |
| 82 | −extern char bmips_smp_int_vec_end; | |
| 78 | +extern char bmips_reset_nmi_vec[]; | |
| 79 | +extern char bmips_reset_nmi_vec_end[]; | |
| 80 | +extern char bmips_smp_movevec[]; | |
| 81 | +extern char bmips_smp_int_vec[]; | |
| 82 | +extern char bmips_smp_int_vec_end[]; | |
| 83 | 83 | |
| 84 | 84 | extern int bmips_smp_enabled; |
| 85 | 85 | extern int bmips_cpu_offset; |
| @@ -77,8 +77,8 @@ extern unsigned long __xchg_called_with_bad_pointer(void) · 76 unmodified lines | ||
| 77 | 77 | extern unsigned long __xchg_small(volatile void *ptr, unsigned long val, |
| 78 | 78 | unsigned int size); |
| 79 | 79 | |
| 80 | −static inline unsigned long __xchg(volatile void *ptr, unsigned long x, | |
| 81 | − int size) | |
| 80 | +static __always_inline | |
| 81 | +unsigned long __xchg(volatile void *ptr, unsigned long x, int size) | |
| 82 | 82 | { |
| 83 | 83 | switch (size) { |
| 84 | 84 | case 1: |
| @@ -153,8 +153,9 @@ static inline unsigned long __xchg(volatile void *ptr, unsigned long x, · 68 unmodified lines | ||
| 153 | 153 | extern unsigned long __cmpxchg_small(volatile void *ptr, unsigned long old, |
| 154 | 154 | unsigned long new, unsigned int size); |
| 155 | 155 | |
| 156 | −static inline unsigned long __cmpxchg(volatile void *ptr, unsigned long old, | |
| 157 | − unsigned long new, unsigned int size) | |
| 156 | +static __always_inline | |
| 157 | +unsigned long __cmpxchg(volatile void *ptr, unsigned long old, | |
| 158 | + unsigned long new, unsigned int size) | |
| 158 | 159 | { |
| 159 | 160 | switch (size) { |
| 160 | 161 | case 1: |
| @@ -24,6 +24,8 @@ · 23 unmodified lines | ||
| 24 | 24 | |
| 25 | 25 | #define VDSO_HAS_CLOCK_GETRES 1 |
| 26 | 26 | |
| 27 | +#define __VDSO_USE_SYSCALL ULLONG_MAX | |
| 28 | + | |
| 27 | 29 | #ifdef CONFIG_MIPS_CLOCK_VSYSCALL |
| 28 | 30 | |
| 29 | 31 | static __always_inline long gettimeofday_fallback( |
| @@ -205,7 +207,7 @@ static __always_inline u64 __arch_get_hw_counter(s32 clock_mode) · 175 unmodified lines | ||
| 205 | 207 | break; |
| 206 | 208 | #endif |
| 207 | 209 | default: |
| 208 | − cycle_now = 0; | |
| 210 | + cycle_now = __VDSO_USE_SYSCALL; | |
| 209 | 211 | break; |
| 210 | 212 | } |
| 211 | 213 | |
| @@ -28,13 +28,6 @@ int __mips_get_clock_mode(struct timekeeper *tk) · 27 unmodified lines | ||
| 28 | 28 | } |
| 29 | 29 | #define __arch_get_clock_mode __mips_get_clock_mode |
| 30 | 30 | |
| 31 | −static __always_inline | |
| 32 | −int __mips_use_vsyscall(struct vdso_data *vdata) | |
| 33 | −{ | |
| 34 | − return (vdata[CS_HRES_COARSE].clock_mode != VDSO_CLOCK_NONE); | |
| 35 | −} | |
| 36 | −#define __arch_use_vsyscall __mips_use_vsyscall | |
| 37 | − | |
| 38 | 31 | /* The asm-generic header needs to be included after the definitions above */ |
| 39 | 32 | #include <asm-generic/vdso/vsyscall.h> |
| 40 | 33 | |
| @@ -6,5 +6,16 @@ · 5 unmodified lines | ||
| 6 | 6 | #define HWCAP_MIPS_R6 (1 << 0) |
| 7 | 7 | #define HWCAP_MIPS_MSA (1 << 1) |
| 8 | 8 | #define HWCAP_MIPS_CRC32 (1 << 2) |
| 9 | +#define HWCAP_MIPS_MIPS16 (1 << 3) | |
| 10 | +#define HWCAP_MIPS_MDMX (1 << 4) | |
| 11 | +#define HWCAP_MIPS_MIPS3D (1 << 5) | |
| 12 | +#define HWCAP_MIPS_SMARTMIPS (1 << 6) | |
| 13 | +#define HWCAP_MIPS_DSP (1 << 7) | |
| 14 | +#define HWCAP_MIPS_DSP2 (1 << 8) | |
| 15 | +#define HWCAP_MIPS_DSP3 (1 << 9) | |
| 16 | +#define HWCAP_MIPS_MIPS16E2 (1 << 10) | |
| 17 | +#define HWCAP_LOONGSON_MMI (1 << 11) | |
| 18 | +#define HWCAP_LOONGSON_EXT (1 << 12) | |
| 19 | +#define HWCAP_LOONGSON_EXT2 (1 << 13) | |
| 9 | 20 | |
| 10 | 21 | #endif /* _UAPI_ASM_HWCAP_H */ |
| @@ -2180,6 +2180,39 @@ void cpu_probe(void) · 2179 unmodified lines | ||
| 2180 | 2180 | elf_hwcap |= HWCAP_MIPS_MSA; |
| 2181 | 2181 | } |
| 2182 | 2182 | |
| 2183 | + if (cpu_has_mips16) | |
| 2184 | + elf_hwcap |= HWCAP_MIPS_MIPS16; | |
| 2185 | + | |
| 2186 | + if (cpu_has_mdmx) | |
| 2187 | + elf_hwcap |= HWCAP_MIPS_MDMX; | |
| 2188 | + | |
| 2189 | + if (cpu_has_mips3d) | |
| 2190 | + elf_hwcap |= HWCAP_MIPS_MIPS3D; | |
| 2191 | + | |
| 2192 | + if (cpu_has_smartmips) | |
| 2193 | + elf_hwcap |= HWCAP_MIPS_SMARTMIPS; | |
| 2194 | + | |
| 2195 | + if (cpu_has_dsp) | |
| 2196 | + elf_hwcap |= HWCAP_MIPS_DSP; | |
| 2197 | + | |
| 2198 | + if (cpu_has_dsp2) | |
| 2199 | + elf_hwcap |= HWCAP_MIPS_DSP2; | |
| 2200 | + | |
| 2201 | + if (cpu_has_dsp3) | |
| 2202 | + elf_hwcap |= HWCAP_MIPS_DSP3; | |
| 2203 | + | |
| 2204 | + if (cpu_has_mips16e2) | |
| 2205 | + elf_hwcap |= HWCAP_MIPS_MIPS16E2; | |
| 2206 | + | |
| 2207 | + if (cpu_has_loongson_mmi) | |
| 2208 | + elf_hwcap |= HWCAP_LOONGSON_MMI; | |
| 2209 | + | |
| 2210 | + if (cpu_has_loongson_ext) | |
| 2211 | + elf_hwcap |= HWCAP_LOONGSON_EXT; | |
| 2212 | + | |
| 2213 | + if (cpu_has_loongson_ext2) | |
| 2214 | + elf_hwcap |= HWCAP_LOONGSON_EXT2; | |
| 2215 | + | |
| 2183 | 2216 | if (cpu_has_vz) |
| 2184 | 2217 | cpu_probe_vz(c); |
| 2185 | 2218 | |
| @@ -464,10 +464,10 @@ static void bmips_wr_vec(unsigned long dst, char *start, char *end) · 463 unmodified lines | ||
| 464 | 464 | |
| 465 | 465 | static inline void bmips_nmi_handler_setup(void) |
| 466 | 466 | { |
| 467 | − bmips_wr_vec(BMIPS_NMI_RESET_VEC, &bmips_reset_nmi_vec, | |
| 468 | − &bmips_reset_nmi_vec_end); | |
| 469 | − bmips_wr_vec(BMIPS_WARM_RESTART_VEC, &bmips_smp_int_vec, | |
| 470 | − &bmips_smp_int_vec_end); | |
| 467 | + bmips_wr_vec(BMIPS_NMI_RESET_VEC, bmips_reset_nmi_vec, | |
| 468 | + bmips_reset_nmi_vec_end); | |
| 469 | + bmips_wr_vec(BMIPS_WARM_RESTART_VEC, bmips_smp_int_vec, | |
| 470 | + bmips_smp_int_vec_end); | |
| 471 | 471 | } |
| 472 | 472 | |
| 473 | 473 | struct reset_vec_info { |
| @@ -66,6 +66,10 @@ else · 65 unmodified lines | ||
| 66 | 66 | $(call cc-option,-march=mips64r2,-mips64r2 -U_MIPS_ISA -D_MIPS_ISA=_MIPS_ISA_MIPS64) |
| 67 | 67 | endif |
| 68 | 68 | |
| 69 | +# Some -march= flags enable MMI instructions, and GCC complains about that | |
| 70 | +# support being enabled alongside -msoft-float. Thus explicitly disable MMI. | |
| 71 | +cflags-y += $(call cc-option,-mno-loongson-mmi) | |
| 72 | + | |
| 69 | 73 | # |
| 70 | 74 | # Loongson Machines' Support |
| 71 | 75 | # |
| @@ -653,6 +653,13 @@ static void build_restore_pagemask(u32 **p, struct uasm_reloc **r, · 652 unmodified lines | ||
| 653 | 653 | int restore_scratch) |
| 654 | 654 | { |
| 655 | 655 | if (restore_scratch) { |
| 656 | + /* | |
| 657 | + * Ensure the MFC0 below observes the value written to the | |
| 658 | + * KScratch register by the prior MTC0. | |
| 659 | + */ | |
| 660 | + if (scratch_reg >= 0) | |
| 661 | + uasm_i_ehb(p); | |
| 662 | + | |
| 656 | 663 | /* Reset default page size */ |
| 657 | 664 | if (PM_DEFAULT_MASK >> 16) { |
| 658 | 665 | uasm_i_lui(p, tmp, PM_DEFAULT_MASK >> 16); |
| @@ -667,12 +674,10 @@ static void build_restore_pagemask(u32 **p, struct uasm_reloc **r, · 8 unmodified lines | ||
| 667 | 674 | uasm_i_mtc0(p, 0, C0_PAGEMASK); |
| 668 | 675 | uasm_il_b(p, r, lid); |
| 669 | 676 | } |
| 670 | − if (scratch_reg >= 0) { | |
| 671 | − uasm_i_ehb(p); | |
| 677 | + if (scratch_reg >= 0) | |
| 672 | 678 | UASM_i_MFC0(p, 1, c0_kscratch(), scratch_reg); |
| 673 | − } else { | |
| 679 | + else | |
| 674 | 680 | UASM_i_LW(p, 1, scratchpad_offset(0), 0); |
| 675 | − } | |
| 676 | 681 | } else { |
| 677 | 682 | /* Reset default page size */ |
| 678 | 683 | if (PM_DEFAULT_MASK >> 16) { |
| @@ -921,6 +926,10 @@ build_get_pgd_vmalloc64(u32 **p, struct uasm_label **l, struct uasm_reloc **r, · 242 unmodified lines | ||
| 921 | 926 | } |
| 922 | 927 | if (mode != not_refill && check_for_high_segbits) { |
| 923 | 928 | uasm_l_large_segbits_fault(l, *p); |
| 929 | + | |
| 930 | + if (mode == refill_scratch && scratch_reg >= 0) | |
| 931 | + uasm_i_ehb(p); | |
| 932 | + | |
| 924 | 933 | /* |
| 925 | 934 | * We get here if we are an xsseg address, or if we are |
| 926 | 935 | * an xuseg address above (PGDIR_SHIFT+PGDIR_BITS) boundary. |
| @@ -939,12 +948,10 @@ build_get_pgd_vmalloc64(u32 **p, struct uasm_label **l, struct uasm_reloc **r, · 12 unmodified lines | ||
| 939 | 948 | uasm_i_jr(p, ptr); |
| 940 | 949 | |
| 941 | 950 | if (mode == refill_scratch) { |
| 942 | − if (scratch_reg >= 0) { | |
| 943 | − uasm_i_ehb(p); | |
| 951 | + if (scratch_reg >= 0) | |
| 944 | 952 | UASM_i_MFC0(p, 1, c0_kscratch(), scratch_reg); |
| 945 | − } else { | |
| 953 | + else | |
| 946 | 954 | UASM_i_LW(p, 1, scratchpad_offset(0), 0); |
| 947 | − } | |
| 948 | 955 | } else { |
| 949 | 956 | uasm_i_nop(p); |
| 950 | 957 | } |
| @@ -15,6 +15,7 @@ ccflags-vdso := \ · 14 unmodified lines | ||
| 15 | 15 | $(filter -mmicromips,$(KBUILD_CFLAGS)) \ |
| 16 | 16 | $(filter -march=%,$(KBUILD_CFLAGS)) \ |
| 17 | 17 | $(filter -m%-float,$(KBUILD_CFLAGS)) \ |
| 18 | + $(filter -mno-loongson-%,$(KBUILD_CFLAGS)) \ | |
| 18 | 19 | -D__VDSO__ |
| 19 | 20 | |
| 20 | 21 | ifdef CONFIG_CC_IS_CLANG |
| @@ -22,7 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | |
| 23 | 23 | #define ARCH_DMA_MINALIGN L1_CACHE_BYTES |
| 24 | 24 | |
| 25 | −#define __read_mostly __attribute__((__section__(".data..read_mostly"))) | |
| 25 | +#define __read_mostly __section(.data..read_mostly) | |
| 26 | 26 | |
| 27 | 27 | void parisc_cache_init(void); /* initializes cache-flushing */ |
| 28 | 28 | void disable_sr_hashing_asm(int); /* low level support for above */ |
| @@ -52,7 +52,7 @@ · 51 unmodified lines | ||
| 52 | 52 | }) |
| 53 | 53 | |
| 54 | 54 | #ifdef CONFIG_SMP |
| 55 | −# define __lock_aligned __attribute__((__section__(".data..lock_aligned"))) | |
| 55 | +# define __lock_aligned __section(.data..lock_aligned) | |
| 56 | 56 | #endif |
| 57 | 57 | |
| 58 | 58 | #endif /* __PARISC_LDCW_H */ |
| @@ -2125,7 +2125,7 @@ ftrace_regs_caller: · 2124 unmodified lines | ||
| 2125 | 2125 | copy %rp, %r26 |
| 2126 | 2126 | LDREG -FTRACE_FRAME_SIZE-PT_SZ_ALGN(%sp), %r25 |
| 2127 | 2127 | ldo -8(%r25), %r25 |
| 2128 | − copy %r3, %arg2 | |
| 2128 | + ldo -FTRACE_FRAME_SIZE(%r1), %arg2 | |
| 2129 | 2129 | b,l ftrace_function_trampoline, %rp |
| 2130 | 2130 | copy %r1, %arg3 /* struct pt_regs */ |
| 2131 | 2131 | |
| @@ -3,7 +3,7 @@ · 2 unmodified lines | ||
| 3 | 3 | * arch/parisc/mm/ioremap.c |
| 4 | 4 | * |
| 5 | 5 | * (C) Copyright 1995 1996 Linus Torvalds |
| 6 | − * (C) Copyright 2001-2006 Helge Deller <deller@gmx.de> | |
| 6 | + * (C) Copyright 2001-2019 Helge Deller <deller@gmx.de> | |
| 7 | 7 | * (C) Copyright 2005 Kyle McMartin <kyle@parisc-linux.org> |
| 8 | 8 | */ |
| 9 | 9 | |
| @@ -84,7 +84,7 @@ void __iomem * __ioremap(unsigned long phys_addr, unsigned long size, unsigned l · 74 unmodified lines | ||
| 84 | 84 | addr = (void __iomem *) area->addr; |
| 85 | 85 | if (ioremap_page_range((unsigned long)addr, (unsigned long)addr + size, |
| 86 | 86 | phys_addr, pgprot)) { |
| 87 | − vfree(addr); | |
| 87 | + vunmap(addr); | |
| 88 | 88 | return NULL; |
| 89 | 89 | } |
| 90 | 90 | |
| @@ -92,9 +92,11 @@ void __iomem * __ioremap(unsigned long phys_addr, unsigned long size, unsigned l · 1 unmodified line | ||
| 92 | 92 | } |
| 93 | 93 | EXPORT_SYMBOL(__ioremap); |
| 94 | 94 | |
| 95 | −void iounmap(const volatile void __iomem *addr) | |
| 95 | +void iounmap(const volatile void __iomem *io_addr) | |
| 96 | 96 | { |
| 97 | − if (addr > high_memory) | |
| 98 | − return vfree((void *) (PAGE_MASK & (unsigned long __force) addr)); | |
| 97 | + unsigned long addr = (unsigned long)io_addr & PAGE_MASK; | |
| 98 | + | |
| 99 | + if (is_vmalloc_addr((void *)addr)) | |
| 100 | + vunmap((void *)addr); | |
| 99 | 101 | } |
| 100 | 102 | EXPORT_SYMBOL(iounmap); |
| @@ -91,6 +91,7 @@ · 90 unmodified lines | ||
| 91 | 91 | |
| 92 | 92 | static inline void kuap_update_sr(u32 sr, u32 addr, u32 end) |
| 93 | 93 | { |
| 94 | + addr &= 0xf0000000; /* align addr to start of segment */ | |
| 94 | 95 | barrier(); /* make sure thread.kuap is updated before playing with SRs */ |
| 95 | 96 | while (addr < end) { |
| 96 | 97 | mtsrin(sr, addr); |
| @@ -35,6 +35,10 @@ static inline void radix__flush_all_lpid(unsigned int lpid) · 34 unmodified lines | ||
| 35 | 35 | { |
| 36 | 36 | WARN_ON(1); |
| 37 | 37 | } |
| 38 | +static inline void radix__flush_all_lpid_guest(unsigned int lpid) | |
| 39 | +{ | |
| 40 | + WARN_ON(1); | |
| 41 | +} | |
| 38 | 42 | #endif |
| 39 | 43 | |
| 40 | 44 | extern void radix__flush_hugetlb_tlb_range(struct vm_area_struct *vma, |
| @@ -175,4 +175,7 @@ do { \ · 174 unmodified lines | ||
| 175 | 175 | ARCH_DLINFO_CACHE_GEOMETRY; \ |
| 176 | 176 | } while (0) |
| 177 | 177 | |
| 178 | +/* Relocate the kernel image to @final_address */ | |
| 179 | +void relocate(unsigned long final_address); | |
| 180 | + | |
| 178 | 181 | #endif /* _ASM_POWERPC_ELF_H */ |
| @@ -3249,7 +3249,20 @@ static void setup_secure_guest(unsigned long kbase, unsigned long fdt) · 3248 unmodified lines | ||
| 3249 | 3249 | /* Switch to secure mode. */ |
| 3250 | 3250 | prom_printf("Switching to secure mode.\n"); |
| 3251 | 3251 | |
| 3252 | + /* | |
| 3253 | + * The ultravisor will do an integrity check of the kernel image but we | |
| 3254 | + * relocated it so the check will fail. Restore the original image by | |
| 3255 | + * relocating it back to the kernel virtual base address. | |
| 3256 | + */ | |
| 3257 | + if (IS_ENABLED(CONFIG_RELOCATABLE)) | |
| 3258 | + relocate(KERNELBASE); | |
| 3259 | + | |
| 3252 | 3260 | ret = enter_secure_mode(kbase, fdt); |
| 3261 | + | |
| 3262 | + /* Relocate the kernel again. */ | |
| 3263 | + if (IS_ENABLED(CONFIG_RELOCATABLE)) | |
| 3264 | + relocate(kbase); | |
| 3265 | + | |
| 3253 | 3266 | if (ret != U_SUCCESS) { |
| 3254 | 3267 | prom_printf("Returned %d from switching to secure mode.\n", ret); |
| 3255 | 3268 | prom_rtas_os_term("Switch to secure mode failed.\n"); |
| @@ -26,7 +26,8 @@ _end enter_prom $MEM_FUNCS reloc_offset __secondary_hold · 25 unmodified lines | ||
| 26 | 26 | __secondary_hold_acknowledge __secondary_hold_spinloop __start |
| 27 | 27 | logo_linux_clut224 btext_prepare_BAT |
| 28 | 28 | reloc_got2 kernstart_addr memstart_addr linux_banner _stext |
| 29 | −__prom_init_toc_start __prom_init_toc_end btext_setup_display TOC." | |
| 29 | +__prom_init_toc_start __prom_init_toc_end btext_setup_display TOC. | |
| 30 | +relocate" | |
| 30 | 31 | |
| 31 | 32 | NM="$1" |
| 32 | 33 | OBJ="$2" |
| @@ -1921,6 +1921,7 @@ END_FTR_SECTION_IFSET(CPU_FTR_ARCH_207S) · 1920 unmodified lines | ||
| 1921 | 1921 | mtspr SPRN_PCR, r6 |
| 1922 | 1922 | 18: |
| 1923 | 1923 | /* Signal secondary CPUs to continue */ |
| 1924 | + li r0, 0 | |
| 1924 | 1925 | stb r0,VCORE_IN_GUEST(r5) |
| 1925 | 1926 | 19: lis r8,0x7fff /* MAX_INT@h */ |
| 1926 | 1927 | mtspr SPRN_HDEC,r8 |
| @@ -1217,6 +1217,7 @@ int kvmppc_xive_connect_vcpu(struct kvm_device *dev, · 1216 unmodified lines | ||
| 1217 | 1217 | struct kvmppc_xive *xive = dev->private; |
| 1218 | 1218 | struct kvmppc_xive_vcpu *xc; |
| 1219 | 1219 | int i, r = -EBUSY; |
| 1220 | + u32 vp_id; | |
| 1220 | 1221 | |
| 1221 | 1222 | pr_devel("connect_vcpu(cpu=%d)\n", cpu); |
| 1222 | 1223 | |
| @@ -1228,25 +1229,32 @@ int kvmppc_xive_connect_vcpu(struct kvm_device *dev, · 5 unmodified lines | ||
| 1228 | 1229 | return -EPERM; |
| 1229 | 1230 | if (vcpu->arch.irq_type != KVMPPC_IRQ_DEFAULT) |
| 1230 | 1231 | return -EBUSY; |
| 1231 | − if (kvmppc_xive_find_server(vcpu->kvm, cpu)) { | |
| 1232 | − pr_devel("Duplicate !\n"); | |
| 1233 | − return -EEXIST; | |
| 1234 | − } | |
| 1235 | 1232 | if (cpu >= (KVM_MAX_VCPUS * vcpu->kvm->arch.emul_smt_mode)) { |
| 1236 | 1233 | pr_devel("Out of bounds !\n"); |
| 1237 | 1234 | return -EINVAL; |
| 1238 | 1235 | } |
| 1239 | − xc = kzalloc(sizeof(*xc), GFP_KERNEL); | |
| 1240 | − if (!xc) | |
| 1241 | − return -ENOMEM; | |
| 1242 | 1236 | |
| 1243 | 1237 | /* We need to synchronize with queue provisioning */ |
| 1244 | 1238 | mutex_lock(&xive->lock); |
| 1239 | + | |
| 1240 | + vp_id = kvmppc_xive_vp(xive, cpu); | |
| 1241 | + if (kvmppc_xive_vp_in_use(xive->kvm, vp_id)) { | |
| 1242 | + pr_devel("Duplicate !\n"); | |
| 1243 | + r = -EEXIST; | |
| 1244 | + goto bail; | |
| 1245 | + } | |
| 1246 | + | |
| 1247 | + xc = kzalloc(sizeof(*xc), GFP_KERNEL); | |
| 1248 | + if (!xc) { | |
| 1249 | + r = -ENOMEM; | |
| 1250 | + goto bail; | |
| 1251 | + } | |
| 1252 | + | |
| 1245 | 1253 | vcpu->arch.xive_vcpu = xc; |
| 1246 | 1254 | xc->xive = xive; |
| 1247 | 1255 | xc->vcpu = vcpu; |
| 1248 | 1256 | xc->server_num = cpu; |
| 1249 | − xc->vp_id = kvmppc_xive_vp(xive, cpu); | |
| 1257 | + xc->vp_id = vp_id; | |
| 1250 | 1258 | xc->mfrr = 0xff; |
| 1251 | 1259 | xc->valid = true; |
| 1252 | 1260 | |
| @@ -220,6 +220,18 @@ static inline u32 kvmppc_xive_vp(struct kvmppc_xive *xive, u32 server) · 219 unmodified lines | ||
| 220 | 220 | return xive->vp_base + kvmppc_pack_vcpu_id(xive->kvm, server); |
| 221 | 221 | } |
| 222 | 222 | |
| 223 | +static inline bool kvmppc_xive_vp_in_use(struct kvm *kvm, u32 vp_id) | |
| 224 | +{ | |
| 225 | + struct kvm_vcpu *vcpu = NULL; | |
| 226 | + int i; | |
| 227 | + | |
| 228 | + kvm_for_each_vcpu(i, vcpu, kvm) { | |
| 229 | + if (vcpu->arch.xive_vcpu && vp_id == vcpu->arch.xive_vcpu->vp_id) | |
| 230 | + return true; | |
| 231 | + } | |
| 232 | + return false; | |
| 233 | +} | |
| 234 | + | |
| 223 | 235 | /* |
| 224 | 236 | * Mapping between guest priorities and host priorities |
| 225 | 237 | * is as follow. |
| @@ -106,6 +106,7 @@ int kvmppc_xive_native_connect_vcpu(struct kvm_device *dev, · 105 unmodified lines | ||
| 106 | 106 | struct kvmppc_xive *xive = dev->private; |
| 107 | 107 | struct kvmppc_xive_vcpu *xc = NULL; |
| 108 | 108 | int rc; |
| 109 | + u32 vp_id; | |
| 109 | 110 | |
| 110 | 111 | pr_devel("native_connect_vcpu(server=%d)\n", server_num); |
| 111 | 112 | |
| @@ -124,7 +125,8 @@ int kvmppc_xive_native_connect_vcpu(struct kvm_device *dev, · 12 unmodified lines | ||
| 124 | 125 | |
| 125 | 126 | mutex_lock(&xive->lock); |
| 126 | 127 | |
| 127 | − if (kvmppc_xive_find_server(vcpu->kvm, server_num)) { | |
| 128 | + vp_id = kvmppc_xive_vp(xive, server_num); | |
| 129 | + if (kvmppc_xive_vp_in_use(xive->kvm, vp_id)) { | |
| 128 | 130 | pr_devel("Duplicate !\n"); |
| 129 | 131 | rc = -EEXIST; |
| 130 | 132 | goto bail; |
| @@ -141,7 +143,7 @@ int kvmppc_xive_native_connect_vcpu(struct kvm_device *dev, · 10 unmodified lines | ||
| 141 | 143 | xc->vcpu = vcpu; |
| 142 | 144 | xc->server_num = server_num; |
| 143 | 145 | |
| 144 | − xc->vp_id = kvmppc_xive_vp(xive, server_num); | |
| 146 | + xc->vp_id = vp_id; | |
| 145 | 147 | xc->valid = true; |
| 146 | 148 | vcpu->arch.irq_type = KVMPPC_IRQ_XIVE; |
| 147 | 149 | |
| @@ -1141,6 +1141,19 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *fp) · 1140 unmodified lines | ||
| 1141 | 1141 | goto out_addrs; |
| 1142 | 1142 | } |
| 1143 | 1143 | |
| 1144 | + /* | |
| 1145 | + * If we have seen a tail call, we need a second pass. | |
| 1146 | + * This is because bpf_jit_emit_common_epilogue() is called | |
| 1147 | + * from bpf_jit_emit_tail_call() with a not yet stable ctx->seen. | |
| 1148 | + */ | |
| 1149 | + if (cgctx.seen & SEEN_TAILCALL) { | |
| 1150 | + cgctx.idx = 0; | |
| 1151 | + if (bpf_jit_build_body(fp, 0, &cgctx, addrs, false)) { | |
| 1152 | + fp = org_fp; | |
| 1153 | + goto out_addrs; | |
| 1154 | + } | |
| 1155 | + } | |
| 1156 | + | |
| 1144 | 1157 | /* |
| 1145 | 1158 | * Pretend to build prologue, given the features we've seen. This will |
| 1146 | 1159 | * update ctgtx.idx as it pretends to output instructions, then we can |
| @@ -761,6 +761,7 @@ static int spufs_init_fs_context(struct fs_context *fc) · 760 unmodified lines | ||
| 761 | 761 | ctx->gid = current_gid(); |
| 762 | 762 | ctx->mode = 0755; |
| 763 | 763 | |
| 764 | + fc->fs_private = ctx; | |
| 764 | 765 | fc->s_fs_info = sbi; |
| 765 | 766 | fc->ops = &spufs_context_ops; |
| 766 | 767 | return 0; |
| @@ -42,7 +42,7 @@ void pnv_pcibios_bus_add_device(struct pci_dev *pdev) · 41 unmodified lines | ||
| 42 | 42 | { |
| 43 | 43 | struct pci_dn *pdn = pci_get_pdn(pdev); |
| 44 | 44 | |
| 45 | − if (eeh_has_flag(EEH_FORCE_DISABLED)) | |
| 45 | + if (!pdn || eeh_has_flag(EEH_FORCE_DISABLED)) | |
| 46 | 46 | return; |
| 47 | 47 | |
| 48 | 48 | dev_dbg(&pdev->dev, "EEH: Setting up device\n"); |
| @@ -146,20 +146,25 @@ static int pnv_smp_cpu_disable(void) · 145 unmodified lines | ||
| 146 | 146 | return 0; |
| 147 | 147 | } |
| 148 | 148 | |
| 149 | +static void pnv_flush_interrupts(void) | |
| 150 | +{ | |
| 151 | + if (cpu_has_feature(CPU_FTR_ARCH_300)) { | |
| 152 | + if (xive_enabled()) | |
| 153 | + xive_flush_interrupt(); | |
| 154 | + else | |
| 155 | + icp_opal_flush_interrupt(); | |
| 156 | + } else { | |
| 157 | + icp_native_flush_interrupt(); | |
| 158 | + } | |
| 159 | +} | |
| 160 | + | |
| 149 | 161 | static void pnv_smp_cpu_kill_self(void) |
| 150 | 162 | { |
| 163 | + unsigned long srr1, unexpected_mask, wmask; | |
| 151 | 164 | unsigned int cpu; |
| 152 | − unsigned long srr1, wmask; | |
| 153 | 165 | u64 lpcr_val; |
| 154 | 166 | |
| 155 | 167 | /* Standard hot unplug procedure */ |
| 156 | − /* | |
| 157 | − * This hard disables local interurpts, ensuring we have no lazy | |
| 158 | − * irqs pending. | |
| 159 | − */ | |
| 160 | − WARN_ON(irqs_disabled()); | |
| 161 | − hard_irq_disable(); | |
| 162 | − WARN_ON(lazy_irq_pending()); | |
| 163 | 168 | |
| 164 | 169 | idle_task_exit(); |
| 165 | 170 | current->active_mm = NULL; /* for sanity */ |
| @@ -172,6 +177,27 @@ static void pnv_smp_cpu_kill_self(void) · 6 unmodified lines | ||
| 172 | 177 | if (cpu_has_feature(CPU_FTR_ARCH_207S)) |
| 173 | 178 | wmask = SRR1_WAKEMASK_P8; |
| 174 | 179 | |
| 180 | + /* | |
| 181 | + * This turns the irq soft-disabled state we're called with, into a | |
| 182 | + * hard-disabled state with pending irq_happened interrupts cleared. | |
| 183 | + * | |
| 184 | + * PACA_IRQ_DEC - Decrementer should be ignored. | |
| 185 | + * PACA_IRQ_HMI - Can be ignored, processing is done in real mode. | |
| 186 | + * PACA_IRQ_DBELL, EE, PMI - Unexpected. | |
| 187 | + */ | |
| 188 | + hard_irq_disable(); | |
| 189 | + if (generic_check_cpu_restart(cpu)) | |
| 190 | + goto out; | |
| 191 | + | |
| 192 | + unexpected_mask = ~(PACA_IRQ_DEC | PACA_IRQ_HMI | PACA_IRQ_HARD_DIS); | |
| 193 | + if (local_paca->irq_happened & unexpected_mask) { | |
| 194 | + if (local_paca->irq_happened & PACA_IRQ_EE) | |
| 195 | + pnv_flush_interrupts(); | |
| 196 | + DBG("CPU%d Unexpected exit while offline irq_happened=%lx!\n", | |
| 197 | + cpu, local_paca->irq_happened); | |
| 198 | + } | |
| 199 | + local_paca->irq_happened = PACA_IRQ_HARD_DIS; | |
| 200 | + | |
| 175 | 201 | /* |
| 176 | 202 | * We don't want to take decrementer interrupts while we are |
| 177 | 203 | * offline, so clear LPCR:PECE1. We keep PECE2 (and |
| @@ -197,6 +223,7 @@ static void pnv_smp_cpu_kill_self(void) · 19 unmodified lines | ||
| 197 | 223 | |
| 198 | 224 | srr1 = pnv_cpu_offline(cpu); |
| 199 | 225 | |
| 226 | + WARN_ON_ONCE(!irqs_disabled()); | |
| 200 | 227 | WARN_ON(lazy_irq_pending()); |
| 201 | 228 | |
| 202 | 229 | /* |
| @@ -212,13 +239,7 @@ static void pnv_smp_cpu_kill_self(void) · 9 unmodified lines | ||
| 212 | 239 | */ |
| 213 | 240 | if (((srr1 & wmask) == SRR1_WAKEEE) || |
| 214 | 241 | ((srr1 & wmask) == SRR1_WAKEHVI)) { |
| 215 | − if (cpu_has_feature(CPU_FTR_ARCH_300)) { | |
| 216 | − if (xive_enabled()) | |
| 217 | − xive_flush_interrupt(); | |
| 218 | − else | |
| 219 | − icp_opal_flush_interrupt(); | |
| 220 | − } else | |
| 221 | − icp_native_flush_interrupt(); | |
| 242 | + pnv_flush_interrupts(); | |
| 222 | 243 | } else if ((srr1 & wmask) == SRR1_WAKEHDBELL) { |
| 223 | 244 | unsigned long msg = PPC_DBELL_TYPE(PPC_DBELL_SERVER); |
| 224 | 245 | asm volatile(PPC_MSGCLR(%0) : : "r" (msg)); |
| @@ -266,7 +287,7 @@ static void pnv_smp_cpu_kill_self(void) · 41 unmodified lines | ||
| 266 | 287 | */ |
| 267 | 288 | lpcr_val = mfspr(SPRN_LPCR) | (u64)LPCR_PECE1; |
| 268 | 289 | pnv_program_cpu_hotplug_lpcr(cpu, lpcr_val); |
| 269 | − | |
| 290 | +out: | |
| 270 | 291 | DBG("CPU%d coming online...\n", cpu); |
| 271 | 292 | } |
| 272 | 293 | |
| @@ -1419,6 +1419,9 @@ void __init pseries_lpar_read_hblkrm_characteristics(void) · 1418 unmodified lines | ||
| 1419 | 1419 | unsigned char local_buffer[SPLPAR_TLB_BIC_MAXLENGTH]; |
| 1420 | 1420 | int call_status, len, idx, bpsize; |
| 1421 | 1421 | |
| 1422 | + if (!firmware_has_feature(FW_FEATURE_BLOCK_REMOVE)) | |
| 1423 | + return; | |
| 1424 | + | |
| 1422 | 1425 | spin_lock(&rtas_data_buf_lock); |
| 1423 | 1426 | memset(rtas_data_buf, 0, RTAS_DATA_BUF_SIZE); |
| 1424 | 1427 | call_status = rtas_call(rtas_token("ibm,get-system-parameter"), 3, 1, |
| @@ -13,6 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | compatible = "sifive,hifive-unleashed-a00", "sifive,fu540-c000"; |
| 14 | 14 | |
| 15 | 15 | chosen { |
| 16 | + stdout-path = "serial0"; | |
| 16 | 17 | }; |
| 17 | 18 | |
| 18 | 19 | cpus { |
| @@ -12,7 +12,6 @@ · 11 unmodified lines | ||
| 12 | 12 | |
| 13 | 13 | #include <asm/asm.h> |
| 14 | 14 | |
| 15 | −#ifdef CONFIG_GENERIC_BUG | |
| 16 | 15 | #define __INSN_LENGTH_MASK _UL(0x3) |
| 17 | 16 | #define __INSN_LENGTH_32 _UL(0x3) |
| 18 | 17 | #define __COMPRESSED_INSN_MASK _UL(0xffff) |
| @@ -20,7 +19,6 @@ · 1 unmodified line | ||
| 20 | 19 | #define __BUG_INSN_32 _UL(0x00100073) /* ebreak */ |
| 21 | 20 | #define __BUG_INSN_16 _UL(0x9002) /* c.ebreak */ |
| 22 | 21 | |
| 23 | −#ifndef __ASSEMBLY__ | |
| 24 | 22 | typedef u32 bug_insn_t; |
| 25 | 23 | |
| 26 | 24 | #ifdef CONFIG_GENERIC_BUG_RELATIVE_POINTERS |
| @@ -43,6 +41,7 @@ typedef u32 bug_insn_t; · 16 unmodified lines | ||
| 43 | 41 | RISCV_SHORT " %2" |
| 44 | 42 | #endif |
| 45 | 43 | |
| 44 | +#ifdef CONFIG_GENERIC_BUG | |
| 46 | 45 | #define __BUG_FLAGS(flags) \ |
| 47 | 46 | do { \ |
| 48 | 47 | __asm__ __volatile__ ( \ |
| @@ -58,14 +57,10 @@ do { \ · 9 unmodified lines | ||
| 58 | 57 | "i" (flags), \ |
| 59 | 58 | "i" (sizeof(struct bug_entry))); \ |
| 60 | 59 | } while (0) |
| 61 | − | |
| 62 | −#endif /* !__ASSEMBLY__ */ | |
| 63 | 60 | #else /* CONFIG_GENERIC_BUG */ |
| 64 | −#ifndef __ASSEMBLY__ | |
| 65 | 61 | #define __BUG_FLAGS(flags) do { \ |
| 66 | 62 | __asm__ __volatile__ ("ebreak\n"); \ |
| 67 | 63 | } while (0) |
| 68 | −#endif /* !__ASSEMBLY__ */ | |
| 69 | 64 | #endif /* CONFIG_GENERIC_BUG */ |
| 70 | 65 | |
| 71 | 66 | #define BUG() do { \ |
| @@ -79,15 +74,10 @@ do { \ · 7 unmodified lines | ||
| 79 | 74 | |
| 80 | 75 | #include <asm-generic/bug.h> |
| 81 | 76 | |
| 82 | −#ifndef __ASSEMBLY__ | |
| 83 | − | |
| 84 | 77 | struct pt_regs; |
| 85 | 78 | struct task_struct; |
| 86 | 79 | |
| 87 | −extern void die(struct pt_regs *regs, const char *str); | |
| 88 | −extern void do_trap(struct pt_regs *regs, int signo, int code, | |
| 89 | − unsigned long addr); | |
| 90 | − | |
| 91 | −#endif /* !__ASSEMBLY__ */ | |
| 80 | +void die(struct pt_regs *regs, const char *str); | |
| 81 | +void do_trap(struct pt_regs *regs, int signo, int code, unsigned long addr); | |
| 92 | 82 | |
| 93 | 83 | #endif /* _ASM_RISCV_BUG_H */ |
| @@ -13,6 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | |
| 14 | 14 | #include <linux/types.h> |
| 15 | 15 | #include <asm/mmiowb.h> |
| 16 | +#include <asm/pgtable.h> | |
| 16 | 17 | |
| 17 | 18 | extern void __iomem *ioremap(phys_addr_t offset, unsigned long size); |
| 18 | 19 | |
| @@ -161,6 +162,12 @@ static inline u64 __raw_readq(const volatile void __iomem *addr) · 142 unmodified lines | ||
| 161 | 162 | #define writeq(v,c) ({ __io_bw(); writeq_cpu((v),(c)); __io_aw(); }) |
| 162 | 163 | #endif |
| 163 | 164 | |
| 165 | +/* | |
| 166 | + * I/O port access constants. | |
| 167 | + */ | |
| 168 | +#define IO_SPACE_LIMIT (PCI_IO_SIZE - 1) | |
| 169 | +#define PCI_IOBASE ((void __iomem *)PCI_IO_START) | |
| 170 | + | |
| 164 | 171 | /* |
| 165 | 172 | * Emulation routines for the port-mapped IO space used by some PCI drivers. |
| 166 | 173 | * These are defined as being "fully synchronous", but also "not guaranteed to |
| @@ -7,6 +7,9 @@ · 6 unmodified lines | ||
| 7 | 7 | #ifndef _ASM_RISCV_IRQ_H |
| 8 | 8 | #define _ASM_RISCV_IRQ_H |
| 9 | 9 | |
| 10 | +#include <linux/interrupt.h> | |
| 11 | +#include <linux/linkage.h> | |
| 12 | + | |
| 10 | 13 | #define NR_IRQS 0 |
| 11 | 14 | |
| 12 | 15 | void riscv_timer_interrupt(void); |
| @@ -7,6 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | #define _ASM_RISCV_PGTABLE_H |
| 8 | 8 | |
| 9 | 9 | #include <linux/mmzone.h> |
| 10 | +#include <linux/sizes.h> | |
| 10 | 11 | |
| 11 | 12 | #include <asm/pgtable-bits.h> |
| 12 | 13 | |
| @@ -86,14 +87,7 @@ extern pgd_t swapper_pg_dir[]; · 73 unmodified lines | ||
| 86 | 87 | #define VMALLOC_SIZE (KERN_VIRT_SIZE >> 1) |
| 87 | 88 | #define VMALLOC_END (PAGE_OFFSET - 1) |
| 88 | 89 | #define VMALLOC_START (PAGE_OFFSET - VMALLOC_SIZE) |
| 89 | − | |
| 90 | −#define FIXADDR_TOP VMALLOC_START | |
| 91 | −#ifdef CONFIG_64BIT | |
| 92 | −#define FIXADDR_SIZE PMD_SIZE | |
| 93 | −#else | |
| 94 | −#define FIXADDR_SIZE PGDIR_SIZE | |
| 95 | −#endif | |
| 96 | −#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) | |
| 90 | +#define PCI_IO_SIZE SZ_16M | |
| 97 | 91 | |
| 98 | 92 | /* |
| 99 | 93 | * Roughly size the vmemmap space to be large enough to fit enough |
| @@ -108,6 +102,17 @@ extern pgd_t swapper_pg_dir[]; · 8 unmodified lines | ||
| 108 | 102 | |
| 109 | 103 | #define vmemmap ((struct page *)VMEMMAP_START) |
| 110 | 104 | |
| 105 | +#define PCI_IO_END VMEMMAP_START | |
| 106 | +#define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE) | |
| 107 | +#define FIXADDR_TOP PCI_IO_START | |
| 108 | + | |
| 109 | +#ifdef CONFIG_64BIT | |
| 110 | +#define FIXADDR_SIZE PMD_SIZE | |
| 111 | +#else | |
| 112 | +#define FIXADDR_SIZE PGDIR_SIZE | |
| 113 | +#endif | |
| 114 | +#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) | |
| 115 | + | |
| 111 | 116 | /* |
| 112 | 117 | * ZERO_PAGE is a global shared page that is always zero, |
| 113 | 118 | * used for zero-mapped memory areas, etc. |
| @@ -184,10 +189,7 @@ static inline pte_t pfn_pte(unsigned long pfn, pgprot_t prot) · 70 unmodified lines | ||
| 184 | 189 | return __pte((pfn << _PAGE_PFN_SHIFT) | pgprot_val(prot)); |
| 185 | 190 | } |
| 186 | 191 | |
| 187 | −static inline pte_t mk_pte(struct page *page, pgprot_t prot) | |
| 188 | −{ | |
| 189 | − return pfn_pte(page_to_pfn(page), prot); | |
| 190 | −} | |
| 192 | +#define mk_pte(page, prot) pfn_pte(page_to_pfn(page), prot) | |
| 191 | 193 | |
| 192 | 194 | #define pte_index(addr) (((addr) >> PAGE_SHIFT) & (PTRS_PER_PTE - 1)) |
| 193 | 195 | |
| @@ -428,9 +430,7 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma, · 234 unmodified lines | ||
| 428 | 430 | #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val(pte) }) |
| 429 | 431 | #define __swp_entry_to_pte(x) ((pte_t) { (x).val }) |
| 430 | 432 | |
| 431 | −#ifdef CONFIG_FLATMEM | |
| 432 | 433 | #define kern_addr_valid(addr) (1) /* FIXME */ |
| 433 | −#endif | |
| 434 | 434 | |
| 435 | 435 | extern void *dtb_early_va; |
| 436 | 436 | extern void setup_bootmem(void); |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | #ifndef _ASM_RISCV_SWITCH_TO_H |
| 7 | 7 | #define _ASM_RISCV_SWITCH_TO_H |
| 8 | 8 | |
| 9 | +#include <linux/sched/task_stack.h> | |
| 9 | 10 | #include <asm/processor.h> |
| 10 | 11 | #include <asm/ptrace.h> |
| 11 | 12 | #include <asm/csr.h> |
| @@ -10,10 +10,6 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <linux/mm_types.h> |
| 11 | 11 | #include <asm/smp.h> |
| 12 | 12 | |
| 13 | −/* | |
| 14 | − * Flush entire local TLB. 'sfence.vma' implicitly fences with the instruction | |
| 15 | − * cache as well, so a 'fence.i' is not necessary. | |
| 16 | − */ | |
| 17 | 13 | static inline void local_flush_tlb_all(void) |
| 18 | 14 | { |
| 19 | 15 | __asm__ __volatile__ ("sfence.vma" : : : "memory"); |
| @@ -10,6 +10,7 @@ · 9 unmodified lines | ||
| 10 | 10 | #include <asm/processor.h> |
| 11 | 11 | #include <asm/hwcap.h> |
| 12 | 12 | #include <asm/smp.h> |
| 13 | +#include <asm/switch_to.h> | |
| 13 | 14 | |
| 14 | 15 | unsigned long elf_hwcap __read_mostly; |
| 15 | 16 | #ifdef CONFIG_FPU |
| @@ -273,12 +273,11 @@ restore_all: · 272 unmodified lines | ||
| 273 | 273 | resume_kernel: |
| 274 | 274 | REG_L s0, TASK_TI_PREEMPT_COUNT(tp) |
| 275 | 275 | bnez s0, restore_all |
| 276 | −need_resched: | |
| 277 | 276 | REG_L s0, TASK_TI_FLAGS(tp) |
| 278 | 277 | andi s0, s0, _TIF_NEED_RESCHED |
| 279 | 278 | beqz s0, restore_all |
| 280 | 279 | call preempt_schedule_irq |
| 281 | − j need_resched | |
| 280 | + j restore_all | |
| 282 | 281 | #endif |
| 283 | 282 | |
| 284 | 283 | work_pending: |
| @@ -0,0 +1,21 @@ | ||
| 1 | +/* SPDX-License-Identifier: GPL-2.0 */ | |
| 2 | +/* | |
| 3 | + * Copyright (C) 2019 SiFive, Inc. | |
| 4 | + */ | |
| 5 | +#ifndef __ASM_HEAD_H | |
| 6 | +#define __ASM_HEAD_H | |
| 7 | + | |
| 8 | +#include <linux/linkage.h> | |
| 9 | +#include <linux/init.h> | |
| 10 | + | |
| 11 | +extern atomic_t hart_lottery; | |
| 12 | + | |
| 13 | +asmlinkage void do_page_fault(struct pt_regs *regs); | |
| 14 | +asmlinkage void __init setup_vm(uintptr_t dtb_pa); | |
| 15 | + | |
| 16 | +extern void *__cpu_up_stack_pointer[]; | |
| 17 | +extern void *__cpu_up_task_pointer[]; | |
| 18 | + | |
| 19 | +void __init parse_dtb(void); | |
| 20 | + | |
| 21 | +#endif /* __ASM_HEAD_H */ | |
| @@ -24,7 +24,7 @@ int arch_show_interrupts(struct seq_file *p, int prec) · 23 unmodified lines | ||
| 24 | 24 | return 0; |
| 25 | 25 | } |
| 26 | 26 | |
| 27 | −asmlinkage void __irq_entry do_IRQ(struct pt_regs *regs) | |
| 27 | +asmlinkage __visible void __irq_entry do_IRQ(struct pt_regs *regs) | |
| 28 | 28 | { |
| 29 | 29 | struct pt_regs *old_regs = set_irq_regs(regs); |
| 30 | 30 | |
| @@ -8,6 +8,7 @@ · 7 unmodified lines | ||
| 8 | 8 | #include <linux/elf.h> |
| 9 | 9 | #include <linux/kernel.h> |
| 10 | 10 | #include <linux/module.h> |
| 11 | +#include <linux/moduleloader.h> | |
| 11 | 12 | |
| 12 | 13 | unsigned long module_emit_got_entry(struct module *mod, unsigned long val) |
| 13 | 14 | { |
| @@ -7,6 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | * Copyright (C) 2017 SiFive |
| 8 | 8 | */ |
| 9 | 9 | |
| 10 | +#include <linux/cpu.h> | |
| 10 | 11 | #include <linux/kernel.h> |
| 11 | 12 | #include <linux/sched.h> |
| 12 | 13 | #include <linux/sched/task_stack.h> |
| @@ -19,6 +20,7 @@ · 6 unmodified lines | ||
| 19 | 20 | #include <asm/csr.h> |
| 20 | 21 | #include <asm/string.h> |
| 21 | 22 | #include <asm/switch_to.h> |
| 23 | +#include <asm/thread_info.h> | |
| 22 | 24 | |
| 23 | 25 | extern asmlinkage void ret_from_fork(void); |
| 24 | 26 | extern asmlinkage void ret_from_kernel_thread(void); |
| @@ -148,7 +148,7 @@ long arch_ptrace(struct task_struct *child, long request, · 147 unmodified lines | ||
| 148 | 148 | * Allows PTRACE_SYSCALL to work. These are called from entry.S in |
| 149 | 149 | * {handle,ret_from}_syscall. |
| 150 | 150 | */ |
| 151 | −void do_syscall_trace_enter(struct pt_regs *regs) | |
| 151 | +__visible void do_syscall_trace_enter(struct pt_regs *regs) | |
| 152 | 152 | { |
| 153 | 153 | if (test_thread_flag(TIF_SYSCALL_TRACE)) |
| 154 | 154 | if (tracehook_report_syscall_entry(regs)) |
| @@ -162,7 +162,7 @@ void do_syscall_trace_enter(struct pt_regs *regs) · 7 unmodified lines | ||
| 162 | 162 | audit_syscall_entry(regs->a7, regs->a0, regs->a1, regs->a2, regs->a3); |
| 163 | 163 | } |
| 164 | 164 | |
| 165 | −void do_syscall_trace_exit(struct pt_regs *regs) | |
| 165 | +__visible void do_syscall_trace_exit(struct pt_regs *regs) | |
| 166 | 166 | { |
| 167 | 167 | audit_syscall_exit(regs); |
| 168 | 168 | |
| @@ -4,6 +4,7 @@ · 3 unmodified lines | ||
| 4 | 4 | */ |
| 5 | 5 | |
| 6 | 6 | #include <linux/reboot.h> |
| 7 | +#include <linux/pm.h> | |
| 7 | 8 | #include <asm/sbi.h> |
| 8 | 9 | |
| 9 | 10 | static void default_power_off(void) |
| @@ -24,6 +24,8 @@ · 23 unmodified lines | ||
| 24 | 24 | #include <asm/tlbflush.h> |
| 25 | 25 | #include <asm/thread_info.h> |
| 26 | 26 | |
| 27 | +#include "head.h" | |
| 28 | + | |
| 27 | 29 | #ifdef CONFIG_DUMMY_CONSOLE |
| 28 | 30 | struct screen_info screen_info = { |
| 29 | 31 | .orig_video_lines = 30, |
| @@ -26,7 +26,7 @@ struct rt_sigframe { · 25 unmodified lines | ||
| 26 | 26 | |
| 27 | 27 | #ifdef CONFIG_FPU |
| 28 | 28 | static long restore_fp_state(struct pt_regs *regs, |
| 29 | − union __riscv_fp_state *sc_fpregs) | |
| 29 | + union __riscv_fp_state __user *sc_fpregs) | |
| 30 | 30 | { |
| 31 | 31 | long err; |
| 32 | 32 | struct __riscv_d_ext_state __user *state = &sc_fpregs->d; |
| @@ -53,7 +53,7 @@ static long restore_fp_state(struct pt_regs *regs, · 20 unmodified lines | ||
| 53 | 53 | } |
| 54 | 54 | |
| 55 | 55 | static long save_fp_state(struct pt_regs *regs, |
| 56 | − union __riscv_fp_state *sc_fpregs) | |
| 56 | + union __riscv_fp_state __user *sc_fpregs) | |
| 57 | 57 | { |
| 58 | 58 | long err; |
| 59 | 59 | struct __riscv_d_ext_state __user *state = &sc_fpregs->d; |
| @@ -292,8 +292,8 @@ static void do_signal(struct pt_regs *regs) · 232 unmodified lines | ||
| 292 | 292 | * notification of userspace execution resumption |
| 293 | 293 | * - triggered by the _TIF_WORK_MASK flags |
| 294 | 294 | */ |
| 295 | −asmlinkage void do_notify_resume(struct pt_regs *regs, | |
| 296 | − unsigned long thread_info_flags) | |
| 295 | +asmlinkage __visible void do_notify_resume(struct pt_regs *regs, | |
| 296 | + unsigned long thread_info_flags) | |
| 297 | 297 | { |
| 298 | 298 | /* Handle pending signal delivery */ |
| 299 | 299 | if (thread_info_flags & _TIF_SIGPENDING) |
| @@ -8,7 +8,9 @@ · 7 unmodified lines | ||
| 8 | 8 | * Copyright (C) 2017 SiFive |
| 9 | 9 | */ |
| 10 | 10 | |
| 11 | +#include <linux/cpu.h> | |
| 11 | 12 | #include <linux/interrupt.h> |
| 13 | +#include <linux/profile.h> | |
| 12 | 14 | #include <linux/smp.h> |
| 13 | 15 | #include <linux/sched.h> |
| 14 | 16 | #include <linux/seq_file.h> |
| @@ -29,6 +29,9 @@ · 28 unmodified lines | ||
| 29 | 29 | #include <asm/tlbflush.h> |
| 30 | 30 | #include <asm/sections.h> |
| 31 | 31 | #include <asm/sbi.h> |
| 32 | +#include <asm/smp.h> | |
| 33 | + | |
| 34 | +#include "head.h" | |
| 32 | 35 | |
| 33 | 36 | void *__cpu_up_stack_pointer[NR_CPUS]; |
| 34 | 37 | void *__cpu_up_task_pointer[NR_CPUS]; |
| @@ -130,7 +133,7 @@ void __init smp_cpus_done(unsigned int max_cpus) · 95 unmodified lines | ||
| 130 | 133 | /* |
| 131 | 134 | * C entry point for a secondary processor. |
| 132 | 135 | */ |
| 133 | −asmlinkage void __init smp_callin(void) | |
| 136 | +asmlinkage __visible void __init smp_callin(void) | |
| 134 | 137 | { |
| 135 | 138 | struct mm_struct *mm = &init_mm; |
| 136 | 139 | |
| @@ -8,6 +8,7 @@ · 7 unmodified lines | ||
| 8 | 8 | #include <linux/syscalls.h> |
| 9 | 9 | #include <asm-generic/syscalls.h> |
| 10 | 10 | #include <asm/vdso.h> |
| 11 | +#include <asm/syscall.h> | |
| 11 | 12 | |
| 12 | 13 | #undef __SYSCALL |
| 13 | 14 | #define __SYSCALL(nr, call) [nr] = (call), |
| @@ -7,6 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | #include <linux/clocksource.h> |
| 8 | 8 | #include <linux/delay.h> |
| 9 | 9 | #include <asm/sbi.h> |
| 10 | +#include <asm/processor.h> | |
| 10 | 11 | |
| 11 | 12 | unsigned long riscv_timebase; |
| 12 | 13 | EXPORT_SYMBOL_GPL(riscv_timebase); |
| @@ -3,6 +3,7 @@ · 2 unmodified lines | ||
| 3 | 3 | * Copyright (C) 2012 Regents of the University of California |
| 4 | 4 | */ |
| 5 | 5 | |
| 6 | +#include <linux/cpu.h> | |
| 6 | 7 | #include <linux/kernel.h> |
| 7 | 8 | #include <linux/init.h> |
| 8 | 9 | #include <linux/sched.h> |
| @@ -83,7 +84,7 @@ static void do_trap_error(struct pt_regs *regs, int signo, int code, · 74 unmodified lines | ||
| 83 | 84 | } |
| 84 | 85 | |
| 85 | 86 | #define DO_ERROR_INFO(name, signo, code, str) \ |
| 86 | −asmlinkage void name(struct pt_regs *regs) \ | |
| 87 | +asmlinkage __visible void name(struct pt_regs *regs) \ | |
| 87 | 88 | { \ |
| 88 | 89 | do_trap_error(regs, signo, code, regs->sepc, "Oops - " str); \ |
| 89 | 90 | } |
| @@ -111,7 +112,6 @@ DO_ERROR_INFO(do_trap_ecall_s, · 21 unmodified lines | ||
| 111 | 112 | DO_ERROR_INFO(do_trap_ecall_m, |
| 112 | 113 | SIGILL, ILL_ILLTRP, "environment call from M-mode"); |
| 113 | 114 | |
| 114 | −#ifdef CONFIG_GENERIC_BUG | |
| 115 | 115 | static inline unsigned long get_break_insn_length(unsigned long pc) |
| 116 | 116 | { |
| 117 | 117 | bug_insn_t insn; |
| @@ -120,28 +120,15 @@ static inline unsigned long get_break_insn_length(unsigned long pc) · 2 unmodified lines | ||
| 120 | 120 | return 0; |
| 121 | 121 | return (((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32) ? 4UL : 2UL); |
| 122 | 122 | } |
| 123 | −#endif /* CONFIG_GENERIC_BUG */ | |
| 124 | 123 | |
| 125 | −asmlinkage void do_trap_break(struct pt_regs *regs) | |
| 124 | +asmlinkage __visible void do_trap_break(struct pt_regs *regs) | |
| 126 | 125 | { |
| 127 | −#ifdef CONFIG_GENERIC_BUG | |
| 128 | − if (!user_mode(regs)) { | |
| 129 | − enum bug_trap_type type; | |
| 130 | − | |
| 131 | − type = report_bug(regs->sepc, regs); | |
| 132 | − switch (type) { | |
| 133 | − case BUG_TRAP_TYPE_NONE: | |
| 134 | − break; | |
| 135 | − case BUG_TRAP_TYPE_WARN: | |
| 136 | − regs->sepc += get_break_insn_length(regs->sepc); | |
| 137 | − break; | |
| 138 | − case BUG_TRAP_TYPE_BUG: | |
| 139 | − die(regs, "Kernel BUG"); | |
| 140 | − } | |
| 141 | − } | |
| 142 | −#endif /* CONFIG_GENERIC_BUG */ | |
| 143 | − | |
| 144 | − force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *)(regs->sepc)); | |
| 126 | + if (user_mode(regs)) | |
| 127 | + force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *)regs->sepc); | |
| 128 | + else if (report_bug(regs->sepc, regs) == BUG_TRAP_TYPE_WARN) | |
| 129 | + regs->sepc += get_break_insn_length(regs->sepc); | |
| 130 | + else | |
| 131 | + die(regs, "Kernel BUG"); | |
| 145 | 132 | } |
| 146 | 133 | |
| 147 | 134 | #ifdef CONFIG_GENERIC_BUG |
| @@ -6,6 +6,7 @@ · 5 unmodified lines | ||
| 6 | 6 | * Copyright (C) 2015 Regents of the University of California |
| 7 | 7 | */ |
| 8 | 8 | |
| 9 | +#include <linux/elf.h> | |
| 9 | 10 | #include <linux/mm.h> |
| 10 | 11 | #include <linux/slab.h> |
| 11 | 12 | #include <linux/binfmts.h> |
| @@ -25,7 +26,7 @@ static union { · 13 unmodified lines | ||
| 25 | 26 | struct vdso_data data; |
| 26 | 27 | u8 page[PAGE_SIZE]; |
| 27 | 28 | } vdso_data_store __page_aligned_data; |
| 28 | −struct vdso_data *vdso_data = &vdso_data_store.data; | |
| 29 | +static struct vdso_data *vdso_data = &vdso_data_store.data; | |
| 29 | 30 | |
| 30 | 31 | static int __init vdso_init(void) |
| 31 | 32 | { |
| @@ -7,6 +7,7 @@ · 6 unmodified lines | ||
| 7 | 7 | #include <linux/mm.h> |
| 8 | 8 | #include <asm/tlbflush.h> |
| 9 | 9 | #include <asm/cacheflush.h> |
| 10 | +#include <asm/mmu_context.h> | |
| 10 | 11 | |
| 11 | 12 | /* |
| 12 | 13 | * When necessary, performs a deferred icache flush for the given MM context, |
| @@ -18,6 +18,8 @@ · 17 unmodified lines | ||
| 18 | 18 | #include <asm/ptrace.h> |
| 19 | 19 | #include <asm/tlbflush.h> |
| 20 | 20 | |
| 21 | +#include "../kernel/head.h" | |
| 22 | + | |
| 21 | 23 | /* |
| 22 | 24 | * This routine handles page faults. It determines the address and the |
| 23 | 25 | * problem, and then passes it off to one of the appropriate routines. |
| @@ -19,6 +19,8 @@ · 18 unmodified lines | ||
| 19 | 19 | #include <asm/pgtable.h> |
| 20 | 20 | #include <asm/io.h> |
| 21 | 21 | |
| 22 | +#include "../kernel/head.h" | |
| 23 | + | |
| 22 | 24 | unsigned long empty_zero_page[PAGE_SIZE / sizeof(unsigned long)] |
| 23 | 25 | __page_aligned_bss; |
| 24 | 26 | EXPORT_SYMBOL(empty_zero_page); |
| @@ -337,8 +339,7 @@ static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size) · 312 unmodified lines | ||
| 337 | 339 | */ |
| 338 | 340 | |
| 339 | 341 | #ifndef __riscv_cmodel_medany |
| 340 | −#error "setup_vm() is called from head.S before relocate so it should " | |
| 341 | − "not use absolute addressing." | |
| 342 | +#error "setup_vm() is called from head.S before relocate so it should not use absolute addressing." | |
| 342 | 343 | #endif |
| 343 | 344 | |
| 344 | 345 | asmlinkage void __init setup_vm(uintptr_t dtb_pa) |
| @@ -458,7 +459,7 @@ void __init paging_init(void) · 113 unmodified lines | ||
| 458 | 459 | zone_sizes_init(); |
| 459 | 460 | } |
| 460 | 461 | |
| 461 | −#ifdef CONFIG_SPARSEMEM | |
| 462 | +#ifdef CONFIG_SPARSEMEM_VMEMMAP | |
| 462 | 463 | int __meminit vmemmap_populate(unsigned long start, unsigned long end, int node, |
| 463 | 464 | struct vmem_altmap *altmap) |
| 464 | 465 | { |
| @@ -142,7 +142,7 @@ static irqreturn_t l2_int_handler(int irq, void *device) · 141 unmodified lines | ||
| 142 | 142 | return IRQ_HANDLED; |
| 143 | 143 | } |
| 144 | 144 | |
| 145 | −int __init sifive_l2_init(void) | |
| 145 | +static int __init sifive_l2_init(void) | |
| 146 | 146 | { |
| 147 | 147 | struct device_node *np; |
| 148 | 148 | struct resource res; |
| @@ -101,10 +101,18 @@ static void handle_relocs(unsigned long offset) · 100 unmodified lines | ||
| 101 | 101 | dynsym = (Elf64_Sym *) vmlinux.dynsym_start; |
| 102 | 102 | for (rela = rela_start; rela < rela_end; rela++) { |
| 103 | 103 | loc = rela->r_offset + offset; |
| 104 | − val = rela->r_addend + offset; | |
| 104 | + val = rela->r_addend; | |
| 105 | 105 | r_sym = ELF64_R_SYM(rela->r_info); |
| 106 | − if (r_sym) | |
| 107 | − val += dynsym[r_sym].st_value; | |
| 106 | + if (r_sym) { | |
| 107 | + if (dynsym[r_sym].st_shndx != SHN_UNDEF) | |
| 108 | + val += dynsym[r_sym].st_value + offset; | |
| 109 | + } else { | |
| 110 | + /* | |
| 111 | + * 0 == undefined symbol table index (STN_UNDEF), | |
| 112 | + * used for R_390_RELATIVE, only add KASLR offset | |
| 113 | + */ | |
| 114 | + val += offset; | |
| 115 | + } | |
| 108 | 116 | r_type = ELF64_R_TYPE(rela->r_info); |
| 109 | 117 | rc = arch_kexec_do_relocs(r_type, (void *) loc, val, 0); |
| 110 | 118 | if (rc) |
| @@ -83,7 +83,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n); · 82 unmodified lines | ||
| 83 | 83 | __rc; \ |
| 84 | 84 | }) |
| 85 | 85 | |
| 86 | −static inline int __put_user_fn(void *x, void __user *ptr, unsigned long size) | |
| 86 | +static __always_inline int __put_user_fn(void *x, void __user *ptr, unsigned long size) | |
| 87 | 87 | { |
| 88 | 88 | unsigned long spec = 0x010000UL; |
| 89 | 89 | int rc; |
| @@ -113,7 +113,7 @@ static inline int __put_user_fn(void *x, void __user *ptr, unsigned long size) · 23 unmodified lines | ||
| 113 | 113 | return rc; |
| 114 | 114 | } |
| 115 | 115 | |
| 116 | −static inline int __get_user_fn(void *x, const void __user *ptr, unsigned long size) | |
| 116 | +static __always_inline int __get_user_fn(void *x, const void __user *ptr, unsigned long size) | |
| 117 | 117 | { |
| 118 | 118 | unsigned long spec = 0x01UL; |
| 119 | 119 | int rc; |
| @@ -35,6 +35,7 @@ struct unwind_state { · 34 unmodified lines | ||
| 35 | 35 | struct task_struct *task; |
| 36 | 36 | struct pt_regs *regs; |
| 37 | 37 | unsigned long sp, ip; |
| 38 | + bool reuse_sp; | |
| 38 | 39 | int graph_idx; |
| 39 | 40 | bool reliable; |
| 40 | 41 | bool error; |
| @@ -69,18 +69,26 @@ DEVICE_ATTR(idle_count, 0444, show_idle_count, NULL); · 68 unmodified lines | ||
| 69 | 69 | static ssize_t show_idle_time(struct device *dev, |
| 70 | 70 | struct device_attribute *attr, char *buf) |
| 71 | 71 | { |
| 72 | + unsigned long long now, idle_time, idle_enter, idle_exit, in_idle; | |
| 72 | 73 | struct s390_idle_data *idle = &per_cpu(s390_idle, dev->id); |
| 73 | − unsigned long long now, idle_time, idle_enter, idle_exit; | |
| 74 | 74 | unsigned int seq; |
| 75 | 75 | |
| 76 | 76 | do { |
| 77 | − now = get_tod_clock(); | |
| 78 | 77 | seq = read_seqcount_begin(&idle->seqcount); |
| 79 | 78 | idle_time = READ_ONCE(idle->idle_time); |
| 80 | 79 | idle_enter = READ_ONCE(idle->clock_idle_enter); |
| 81 | 80 | idle_exit = READ_ONCE(idle->clock_idle_exit); |
| 82 | 81 | } while (read_seqcount_retry(&idle->seqcount, seq)); |
| 83 | − idle_time += idle_enter ? ((idle_exit ? : now) - idle_enter) : 0; | |
| 82 | + in_idle = 0; | |
| 83 | + now = get_tod_clock(); | |
| 84 | + if (idle_enter) { | |
| 85 | + if (idle_exit) { | |
| 86 | + in_idle = idle_exit - idle_enter; | |
| 87 | + } else if (now > idle_enter) { | |
| 88 | + in_idle = now - idle_enter; | |
| 89 | + } | |
| 90 | + } | |
| 91 | + idle_time += in_idle; | |
| 84 | 92 | return sprintf(buf, "%llu\n", idle_time >> 12); |
| 85 | 93 | } |
| 86 | 94 | DEVICE_ATTR(idle_time_us, 0444, show_idle_time, NULL); |
| @@ -88,17 +96,24 @@ DEVICE_ATTR(idle_time_us, 0444, show_idle_time, NULL); · 1 unmodified line | ||
| 88 | 96 | u64 arch_cpu_idle_time(int cpu) |
| 89 | 97 | { |
| 90 | 98 | struct s390_idle_data *idle = &per_cpu(s390_idle, cpu); |
| 91 | − unsigned long long now, idle_enter, idle_exit; | |
| 99 | + unsigned long long now, idle_enter, idle_exit, in_idle; | |
| 92 | 100 | unsigned int seq; |
| 93 | 101 | |
| 94 | 102 | do { |
| 95 | − now = get_tod_clock(); | |
| 96 | 103 | seq = read_seqcount_begin(&idle->seqcount); |
| 97 | 104 | idle_enter = READ_ONCE(idle->clock_idle_enter); |
| 98 | 105 | idle_exit = READ_ONCE(idle->clock_idle_exit); |
| 99 | 106 | } while (read_seqcount_retry(&idle->seqcount, seq)); |
| 100 | − | |
| 101 | − return cputime_to_nsecs(idle_enter ? ((idle_exit ?: now) - idle_enter) : 0); | |
| 107 | + in_idle = 0; | |
| 108 | + now = get_tod_clock(); | |
| 109 | + if (idle_enter) { | |
| 110 | + if (idle_exit) { | |
| 111 | + in_idle = idle_exit - idle_enter; | |
| 112 | + } else if (now > idle_enter) { | |
| 113 | + in_idle = now - idle_enter; | |
| 114 | + } | |
| 115 | + } | |
| 116 | + return cputime_to_nsecs(in_idle); | |
| 102 | 117 | } |
| 103 | 118 | |
| 104 | 119 | void arch_cpu_idle_enter(void) |
| @@ -27,6 +27,7 @@ int arch_kexec_do_relocs(int r_type, void *loc, unsigned long val, · 26 unmodified lines | ||
| 27 | 27 | *(u32 *)loc = val; |
| 28 | 28 | break; |
| 29 | 29 | case R_390_64: /* Direct 64 bit. */ |
| 30 | + case R_390_GLOB_DAT: | |
| 30 | 31 | *(u64 *)loc = val; |
| 31 | 32 | break; |
| 32 | 33 | case R_390_PC16: /* PC relative 16 bit. */ |
| @@ -46,10 +46,15 @@ bool unwind_next_frame(struct unwind_state *state) · 45 unmodified lines | ||
| 46 | 46 | |
| 47 | 47 | regs = state->regs; |
| 48 | 48 | if (unlikely(regs)) { |
| 49 | − sp = READ_ONCE_NOCHECK(regs->gprs[15]); | |
| 50 | − if (unlikely(outside_of_stack(state, sp))) { | |
| 51 | − if (!update_stack_info(state, sp)) | |
| 52 | − goto out_err; | |
| 49 | + if (state->reuse_sp) { | |
| 50 | + sp = state->sp; | |
| 51 | + state->reuse_sp = false; | |
| 52 | + } else { | |
| 53 | + sp = READ_ONCE_NOCHECK(regs->gprs[15]); | |
| 54 | + if (unlikely(outside_of_stack(state, sp))) { | |
| 55 | + if (!update_stack_info(state, sp)) | |
| 56 | + goto out_err; | |
| 57 | + } | |
| 53 | 58 | } |
| 54 | 59 | sf = (struct stack_frame *) sp; |
| 55 | 60 | ip = READ_ONCE_NOCHECK(sf->gprs[8]); |
| @@ -107,9 +112,9 @@ void __unwind_start(struct unwind_state *state, struct task_struct *task, · 51 unmodified lines | ||
| 107 | 112 | { |
| 108 | 113 | struct stack_info *info = &state->stack_info; |
| 109 | 114 | unsigned long *mask = &state->stack_mask; |
| 115 | + bool reliable, reuse_sp; | |
| 110 | 116 | struct stack_frame *sf; |
| 111 | 117 | unsigned long ip; |
| 112 | − bool reliable; | |
| 113 | 118 | |
| 114 | 119 | memset(state, 0, sizeof(*state)); |
| 115 | 120 | state->task = task; |
| @@ -134,10 +139,12 @@ void __unwind_start(struct unwind_state *state, struct task_struct *task, · 18 unmodified lines | ||
| 134 | 139 | if (regs) { |
| 135 | 140 | ip = READ_ONCE_NOCHECK(regs->psw.addr); |
| 136 | 141 | reliable = true; |
| 142 | + reuse_sp = true; | |
| 137 | 143 | } else { |
| 138 | 144 | sf = (struct stack_frame *) sp; |
| 139 | 145 | ip = READ_ONCE_NOCHECK(sf->gprs[8]); |
| 140 | 146 | reliable = false; |
| 147 | + reuse_sp = false; | |
| 141 | 148 | } |
| 142 | 149 | |
| 143 | 150 | #ifdef CONFIG_FUNCTION_GRAPH_TRACER |
| @@ -151,5 +158,6 @@ void __unwind_start(struct unwind_state *state, struct task_struct *task, · 7 unmodified lines | ||
| 151 | 158 | state->sp = sp; |
| 152 | 159 | state->ip = ip; |
| 153 | 160 | state->reliable = reliable; |
| 161 | + state->reuse_sp = reuse_sp; | |
| 154 | 162 | } |
| 155 | 163 | EXPORT_SYMBOL_GPL(__unwind_start); |
| @@ -298,16 +298,16 @@ static int cmm_timeout_handler(struct ctl_table *ctl, int write, · 297 unmodified lines | ||
| 298 | 298 | } |
| 299 | 299 | |
| 300 | 300 | if (write) { |
| 301 | − len = *lenp; | |
| 302 | − if (copy_from_user(buf, buffer, | |
| 303 | − len > sizeof(buf) ? sizeof(buf) : len)) | |
| 301 | + len = min(*lenp, sizeof(buf)); | |
| 302 | + if (copy_from_user(buf, buffer, len)) | |
| 304 | 303 | return -EFAULT; |
| 305 | − buf[sizeof(buf) - 1] = '\0'; | |
| 304 | + buf[len - 1] = '\0'; | |
| 306 | 305 | cmm_skip_blanks(buf, &p); |
| 307 | 306 | nr = simple_strtoul(p, &p, 0); |
| 308 | 307 | cmm_skip_blanks(p, &p); |
| 309 | 308 | seconds = simple_strtoul(p, &p, 0); |
| 310 | 309 | cmm_set_timeout(nr, seconds); |
| 310 | + *ppos += *lenp; | |
| 311 | 311 | } else { |
| 312 | 312 | len = sprintf(buf, "%ld %ld\n", |
| 313 | 313 | cmm_timeout_pages, cmm_timeout_seconds); |
| @@ -315,9 +315,9 @@ static int cmm_timeout_handler(struct ctl_table *ctl, int write, · 1 unmodified line | ||
| 315 | 315 | len = *lenp; |
| 316 | 316 | if (copy_to_user(buffer, buf, len)) |
| 317 | 317 | return -EFAULT; |
| 318 | + *lenp = len; | |
| 319 | + *ppos += len; | |
| 318 | 320 | } |
| 319 | − *lenp = len; | |
| 320 | − *ppos += len; | |
| 321 | 321 | return 0; |
| 322 | 322 | } |
| 323 | 323 | |
| @@ -29,7 +29,6 @@ config SPARC · 28 unmodified lines | ||
| 29 | 29 | select RTC_DRV_M48T59 |
| 30 | 30 | select RTC_SYSTOHC |
| 31 | 31 | select HAVE_ARCH_JUMP_LABEL if SPARC64 |
| 32 | − select HAVE_FAST_GUP if SPARC64 | |
| 33 | 32 | select GENERIC_IRQ_SHOW |
| 34 | 33 | select ARCH_WANT_IPC_PARSE_VERSION |
| 35 | 34 | select GENERIC_PCI_IOMAP |
| @@ -1403,8 +1403,12 @@ static blk_status_t ubd_queue_rq(struct blk_mq_hw_ctx *hctx, · 1402 unmodified lines | ||
| 1403 | 1403 | |
| 1404 | 1404 | spin_unlock_irq(&ubd_dev->lock); |
| 1405 | 1405 | |
| 1406 | − if (ret < 0) | |
| 1407 | − blk_mq_requeue_request(req, true); | |
| 1406 | + if (ret < 0) { | |
| 1407 | + if (ret == -ENOMEM) | |
| 1408 | + res = BLK_STS_RESOURCE; | |
| 1409 | + else | |
| 1410 | + res = BLK_STS_DEV_RESOURCE; | |
| 1411 | + } | |
| 1408 | 1412 | |
| 1409 | 1413 | return res; |
| 1410 | 1414 | } |
| @@ -20,30 +20,6 @@ · 19 unmodified lines | ||
| 20 | 20 | */ |
| 21 | 21 | struct mem_vector immovable_mem[MAX_NUMNODES*2]; |
| 22 | 22 | |
| 23 | −/* | |
| 24 | − * Max length of 64-bit hex address string is 19, prefix "0x" + 16 hex | |
| 25 | − * digits, and '\0' for termination. | |
| 26 | − */ | |
| 27 | −#define MAX_ADDR_LEN 19 | |
| 28 | − | |
| 29 | −static acpi_physical_address get_cmdline_acpi_rsdp(void) | |
| 30 | −{ | |
| 31 | − acpi_physical_address addr = 0; | |
| 32 | − | |
| 33 | −#ifdef CONFIG_KEXEC | |
| 34 | − char val[MAX_ADDR_LEN] = { }; | |
| 35 | − int ret; | |
| 36 | − | |
| 37 | − ret = cmdline_find_option("acpi_rsdp", val, MAX_ADDR_LEN); | |
| 38 | − if (ret < 0) | |
| 39 | − return 0; | |
| 40 | − | |
| 41 | − if (kstrtoull(val, 16, &addr)) | |
| 42 | − return 0; | |
| 43 | −#endif | |
| 44 | − return addr; | |
| 45 | −} | |
| 46 | − | |
| 47 | 23 | /* |
| 48 | 24 | * Search EFI system tables for RSDP. If both ACPI_20_TABLE_GUID and |
| 49 | 25 | * ACPI_TABLE_GUID are found, take the former, which has more features. |
| @@ -298,6 +274,30 @@ acpi_physical_address get_rsdp_addr(void) · 248 unmodified lines | ||
| 298 | 274 | } |
| 299 | 275 | |
| 300 | 276 | #if defined(CONFIG_RANDOMIZE_BASE) && defined(CONFIG_MEMORY_HOTREMOVE) |
| 277 | +/* | |
| 278 | + * Max length of 64-bit hex address string is 19, prefix "0x" + 16 hex | |
| 279 | + * digits, and '\0' for termination. | |
| 280 | + */ | |
| 281 | +#define MAX_ADDR_LEN 19 | |
| 282 | + | |
| 283 | +static acpi_physical_address get_cmdline_acpi_rsdp(void) | |
| 284 | +{ | |
| 285 | + acpi_physical_address addr = 0; | |
| 286 | + | |
| 287 | +#ifdef CONFIG_KEXEC | |
| 288 | + char val[MAX_ADDR_LEN] = { }; | |
| 289 | + int ret; | |
| 290 | + | |
| 291 | + ret = cmdline_find_option("acpi_rsdp", val, MAX_ADDR_LEN); | |
| 292 | + if (ret < 0) | |
| 293 | + return 0; | |
| 294 | + | |
| 295 | + if (kstrtoull(val, 16, &addr)) | |
| 296 | + return 0; | |
| 297 | +#endif | |
| 298 | + return addr; | |
| 299 | +} | |
| 300 | + | |
| 301 | 301 | /* Compute SRAT address from RSDP. */ |
| 302 | 302 | static unsigned long get_acpi_srat_table(void) |
| 303 | 303 | { |
| @@ -13,6 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <asm/e820/types.h> |
| 14 | 14 | #include <asm/setup.h> |
| 15 | 15 | #include <asm/desc.h> |
| 16 | +#include <asm/boot.h> | |
| 16 | 17 | |
| 17 | 18 | #include "../string.h" |
| 18 | 19 | #include "eboot.h" |
| @@ -813,7 +814,8 @@ efi_main(struct efi_config *c, struct boot_params *boot_params) · 794 unmodified lines | ||
| 813 | 814 | status = efi_relocate_kernel(sys_table, &bzimage_addr, |
| 814 | 815 | hdr->init_size, hdr->init_size, |
| 815 | 816 | hdr->pref_address, |
| 816 | − hdr->kernel_alignment); | |
| 817 | + hdr->kernel_alignment, | |
| 818 | + LOAD_PHYSICAL_ADDR); | |
| 817 | 819 | if (status != EFI_SUCCESS) { |
| 818 | 820 | efi_printk(sys_table, "efi_relocate_kernel() failed!\n"); |
| 819 | 821 | goto fail; |
| @@ -345,6 +345,7 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, · 344 unmodified lines | ||
| 345 | 345 | { |
| 346 | 346 | const unsigned long kernel_total_size = VO__end - VO__text; |
| 347 | 347 | unsigned long virt_addr = LOAD_PHYSICAL_ADDR; |
| 348 | + unsigned long needed_size; | |
| 348 | 349 | |
| 349 | 350 | /* Retain x86 boot parameters pointer passed from startup_32/64. */ |
| 350 | 351 | boot_params = rmode; |
| @@ -379,26 +380,38 @@ asmlinkage __visible void *extract_kernel(void *rmode, memptr heap, · 28 unmodified lines | ||
| 379 | 380 | free_mem_ptr = heap; /* Heap */ |
| 380 | 381 | free_mem_end_ptr = heap + BOOT_HEAP_SIZE; |
| 381 | 382 | |
| 383 | + /* | |
| 384 | + * The memory hole needed for the kernel is the larger of either | |
| 385 | + * the entire decompressed kernel plus relocation table, or the | |
| 386 | + * entire decompressed kernel plus .bss and .brk sections. | |
| 387 | + * | |
| 388 | + * On X86_64, the memory is mapped with PMD pages. Round the | |
| 389 | + * size up so that the full extent of PMD pages mapped is | |
| 390 | + * included in the check against the valid memory table | |
| 391 | + * entries. This ensures the full mapped area is usable RAM | |
| 392 | + * and doesn't include any reserved areas. | |
| 393 | + */ | |
| 394 | + needed_size = max(output_len, kernel_total_size); | |
| 395 | +#ifdef CONFIG_X86_64 | |
| 396 | + needed_size = ALIGN(needed_size, MIN_KERNEL_ALIGN); | |
| 397 | +#endif | |
| 398 | + | |
| 382 | 399 | /* Report initial kernel position details. */ |
| 383 | 400 | debug_putaddr(input_data); |
| 384 | 401 | debug_putaddr(input_len); |
| 385 | 402 | debug_putaddr(output); |
| 386 | 403 | debug_putaddr(output_len); |
| 387 | 404 | debug_putaddr(kernel_total_size); |
| 405 | + debug_putaddr(needed_size); | |
| 388 | 406 | |
| 389 | 407 | #ifdef CONFIG_X86_64 |
| 390 | 408 | /* Report address of 32-bit trampoline */ |
| 391 | 409 | debug_putaddr(trampoline_32bit); |
| 392 | 410 | #endif |
| 393 | 411 | |
| 394 | − /* | |
| 395 | − * The memory hole needed for the kernel is the larger of either | |
| 396 | − * the entire decompressed kernel plus relocation table, or the | |
| 397 | − * entire decompressed kernel plus .bss and .brk sections. | |
| 398 | − */ | |
| 399 | 412 | choose_random_location((unsigned long)input_data, input_len, |
| 400 | 413 | (unsigned long *)&output, |
| 401 | − max(output_len, kernel_total_size), | |
| 414 | + needed_size, | |
| 402 | 415 | &virt_addr); |
| 403 | 416 | |
| 404 | 417 | /* Validate memory location choices. */ |
| @@ -5,12 +5,14 @@ · 4 unmodified lines | ||
| 5 | 5 | #include <linux/init.h> |
| 6 | 6 | #include <linux/slab.h> |
| 7 | 7 | #include <linux/delay.h> |
| 8 | +#include <linux/jiffies.h> | |
| 8 | 9 | #include <asm/apicdef.h> |
| 9 | 10 | #include <asm/nmi.h> |
| 10 | 11 | |
| 11 | 12 | #include "../perf_event.h" |
| 12 | 13 | |
| 13 | −static DEFINE_PER_CPU(unsigned int, perf_nmi_counter); | |
| 14 | +static DEFINE_PER_CPU(unsigned long, perf_nmi_tstamp); | |
| 15 | +static unsigned long perf_nmi_window; | |
| 14 | 16 | |
| 15 | 17 | static __initconst const u64 amd_hw_cache_event_ids |
| 16 | 18 | [PERF_COUNT_HW_CACHE_MAX] |
| @@ -641,11 +643,12 @@ static void amd_pmu_disable_event(struct perf_event *event) · 624 unmodified lines | ||
| 641 | 643 | * handler when multiple PMCs are active or PMC overflow while handling some |
| 642 | 644 | * other source of an NMI. |
| 643 | 645 | * |
| 644 | − * Attempt to mitigate this by using the number of active PMCs to determine | |
| 645 | − * whether to return NMI_HANDLED if the perf NMI handler did not handle/reset | |
| 646 | − * any PMCs. The per-CPU perf_nmi_counter variable is set to a minimum of the | |
| 647 | − * number of active PMCs or 2. The value of 2 is used in case an NMI does not | |
| 648 | − * arrive at the LAPIC in time to be collapsed into an already pending NMI. | |
| 646 | + * Attempt to mitigate this by creating an NMI window in which un-handled NMIs | |
| 647 | + * received during this window will be claimed. This prevents extending the | |
| 648 | + * window past when it is possible that latent NMIs should be received. The | |
| 649 | + * per-CPU perf_nmi_tstamp will be set to the window end time whenever perf has | |
| 650 | + * handled a counter. When an un-handled NMI is received, it will be claimed | |
| 651 | + * only if arriving within that window. | |
| 649 | 652 | */ |
| 650 | 653 | static int amd_pmu_handle_irq(struct pt_regs *regs) |
| 651 | 654 | { |
| @@ -663,21 +666,19 @@ static int amd_pmu_handle_irq(struct pt_regs *regs) · 11 unmodified lines | ||
| 663 | 666 | handled = x86_pmu_handle_irq(regs); |
| 664 | 667 | |
| 665 | 668 | /* |
| 666 | − * If a counter was handled, record the number of possible remaining | |
| 667 | − * NMIs that can occur. | |
| 669 | + * If a counter was handled, record a timestamp such that un-handled | |
| 670 | + * NMIs will be claimed if arriving within that window. | |
| 668 | 671 | */ |
| 669 | 672 | if (handled) { |
| 670 | − this_cpu_write(perf_nmi_counter, | |
| 671 | − min_t(unsigned int, 2, active)); | |
| 673 | + this_cpu_write(perf_nmi_tstamp, | |
| 674 | + jiffies + perf_nmi_window); | |
| 672 | 675 | |
| 673 | 676 | return handled; |
| 674 | 677 | } |
| 675 | 678 | |
| 676 | − if (!this_cpu_read(perf_nmi_counter)) | |
| 679 | + if (time_after(jiffies, this_cpu_read(perf_nmi_tstamp))) | |
| 677 | 680 | return NMI_DONE; |
| 678 | 681 | |
| 679 | − this_cpu_dec(perf_nmi_counter); | |
| 680 | − | |
| 681 | 682 | return NMI_HANDLED; |
| 682 | 683 | } |
| 683 | 684 | |
| @@ -909,6 +910,9 @@ static int __init amd_core_pmu_init(void) · 225 unmodified lines | ||
| 909 | 910 | if (!boot_cpu_has(X86_FEATURE_PERFCTR_CORE)) |
| 910 | 911 | return 0; |
| 911 | 912 | |
| 913 | + /* Avoid calulating the value each time in the NMI handler */ | |
| 914 | + perf_nmi_window = msecs_to_jiffies(100); | |
| 915 | + | |
| 912 | 916 | switch (boot_cpu_data.x86) { |
| 913 | 917 | case 0x15: |
| 914 | 918 | pr_cont("Fam15h "); |
| @@ -377,7 +377,8 @@ static inline void perf_ibs_disable_event(struct perf_ibs *perf_ibs, · 376 unmodified lines | ||
| 377 | 377 | struct hw_perf_event *hwc, u64 config) |
| 378 | 378 | { |
| 379 | 379 | config &= ~perf_ibs->cnt_mask; |
| 380 | − wrmsrl(hwc->config_base, config); | |
| 380 | + if (boot_cpu_data.x86 == 0x10) | |
| 381 | + wrmsrl(hwc->config_base, config); | |
| 381 | 382 | config &= ~perf_ibs->enable_mask; |
| 382 | 383 | wrmsrl(hwc->config_base, config); |
| 383 | 384 | } |
| @@ -553,7 +554,8 @@ static struct perf_ibs perf_ibs_op = { · 169 unmodified lines | ||
| 553 | 554 | }, |
| 554 | 555 | .msr = MSR_AMD64_IBSOPCTL, |
| 555 | 556 | .config_mask = IBS_OP_CONFIG_MASK, |
| 556 | − .cnt_mask = IBS_OP_MAX_CNT, | |
| 557 | + .cnt_mask = IBS_OP_MAX_CNT | IBS_OP_CUR_CNT | | |
| 558 | + IBS_OP_CUR_CNT_RAND, | |
| 557 | 559 | .enable_mask = IBS_OP_ENABLE, |
| 558 | 560 | .valid_mask = IBS_OP_VAL, |
| 559 | 561 | .max_period = IBS_OP_MAX_CNT << 4, |
| @@ -614,7 +616,7 @@ fail: · 54 unmodified lines | ||
| 614 | 616 | if (event->attr.sample_type & PERF_SAMPLE_RAW) |
| 615 | 617 | offset_max = perf_ibs->offset_max; |
| 616 | 618 | else if (check_rip) |
| 617 | − offset_max = 2; | |
| 619 | + offset_max = 3; | |
| 618 | 620 | else |
| 619 | 621 | offset_max = 1; |
| 620 | 622 | do { |
| @@ -4983,6 +4983,8 @@ __init int intel_pmu_init(void) · 4982 unmodified lines | ||
| 4983 | 4983 | case INTEL_FAM6_SKYLAKE: |
| 4984 | 4984 | case INTEL_FAM6_KABYLAKE_L: |
| 4985 | 4985 | case INTEL_FAM6_KABYLAKE: |
| 4986 | + case INTEL_FAM6_COMETLAKE_L: | |
| 4987 | + case INTEL_FAM6_COMETLAKE: | |
| 4986 | 4988 | x86_add_quirk(intel_pebs_isolation_quirk); |
| 4987 | 4989 | x86_pmu.late_ack = true; |
| 4988 | 4990 | memcpy(hw_cache_event_ids, skl_hw_cache_event_ids, sizeof(hw_cache_event_ids)); |
| @@ -5031,6 +5033,8 @@ __init int intel_pmu_init(void) · 42 unmodified lines | ||
| 5031 | 5033 | /* fall through */ |
| 5032 | 5034 | case INTEL_FAM6_ICELAKE_L: |
| 5033 | 5035 | case INTEL_FAM6_ICELAKE: |
| 5036 | + case INTEL_FAM6_TIGERLAKE_L: | |
| 5037 | + case INTEL_FAM6_TIGERLAKE: | |
| 5034 | 5038 | x86_pmu.late_ack = true; |
| 5035 | 5039 | memcpy(hw_cache_event_ids, skl_hw_cache_event_ids, sizeof(hw_cache_event_ids)); |
| 5036 | 5040 | memcpy(hw_cache_extra_regs, skl_hw_cache_extra_regs, sizeof(hw_cache_extra_regs)); |
| @@ -45,46 +45,49 @@ · 44 unmodified lines | ||
| 45 | 45 | * MSR_CORE_C3_RESIDENCY: CORE C3 Residency Counter |
| 46 | 46 | * perf code: 0x01 |
| 47 | 47 | * Available model: NHM,WSM,SNB,IVB,HSW,BDW,SKL,GLM, |
| 48 | − CNL | |
| 48 | + * CNL,KBL,CML | |
| 49 | 49 | * Scope: Core |
| 50 | 50 | * MSR_CORE_C6_RESIDENCY: CORE C6 Residency Counter |
| 51 | 51 | * perf code: 0x02 |
| 52 | 52 | * Available model: SLM,AMT,NHM,WSM,SNB,IVB,HSW,BDW, |
| 53 | − * SKL,KNL,GLM,CNL | |
| 53 | + * SKL,KNL,GLM,CNL,KBL,CML,ICL,TGL | |
| 54 | 54 | * Scope: Core |
| 55 | 55 | * MSR_CORE_C7_RESIDENCY: CORE C7 Residency Counter |
| 56 | 56 | * perf code: 0x03 |
| 57 | − * Available model: SNB,IVB,HSW,BDW,SKL,CNL | |
| 57 | + * Available model: SNB,IVB,HSW,BDW,SKL,CNL,KBL,CML, | |
| 58 | + * ICL,TGL | |
| 58 | 59 | * Scope: Core |
| 59 | 60 | * MSR_PKG_C2_RESIDENCY: Package C2 Residency Counter. |
| 60 | 61 | * perf code: 0x00 |
| 61 | − * Available model: SNB,IVB,HSW,BDW,SKL,KNL,GLM,CNL | |
| 62 | + * Available model: SNB,IVB,HSW,BDW,SKL,KNL,GLM,CNL, | |
| 63 | + * KBL,CML,ICL,TGL | |
| 62 | 64 | * Scope: Package (physical package) |
| 63 | 65 | * MSR_PKG_C3_RESIDENCY: Package C3 Residency Counter. |
| 64 | 66 | * perf code: 0x01 |
| 65 | 67 | * Available model: NHM,WSM,SNB,IVB,HSW,BDW,SKL,KNL, |
| 66 | − * GLM,CNL | |
| 68 | + * GLM,CNL,KBL,CML,ICL,TGL | |
| 67 | 69 | * Scope: Package (physical package) |
| 68 | 70 | * MSR_PKG_C6_RESIDENCY: Package C6 Residency Counter. |
| 69 | 71 | * perf code: 0x02 |
| 70 | 72 | * Available model: SLM,AMT,NHM,WSM,SNB,IVB,HSW,BDW |
| 71 | − * SKL,KNL,GLM,CNL | |
| 73 | + * SKL,KNL,GLM,CNL,KBL,CML,ICL,TGL | |
| 72 | 74 | * Scope: Package (physical package) |
| 73 | 75 | * MSR_PKG_C7_RESIDENCY: Package C7 Residency Counter. |
| 74 | 76 | * perf code: 0x03 |
| 75 | − * Available model: NHM,WSM,SNB,IVB,HSW,BDW,SKL,CNL | |
| 77 | + * Available model: NHM,WSM,SNB,IVB,HSW,BDW,SKL,CNL, | |
| 78 | + * KBL,CML,ICL,TGL | |
| 76 | 79 | * Scope: Package (physical package) |
| 77 | 80 | * MSR_PKG_C8_RESIDENCY: Package C8 Residency Counter. |
| 78 | 81 | * perf code: 0x04 |
| 79 | − * Available model: HSW ULT,KBL,CNL | |
| 82 | + * Available model: HSW ULT,KBL,CNL,CML,ICL,TGL | |
| 80 | 83 | * Scope: Package (physical package) |
| 81 | 84 | * MSR_PKG_C9_RESIDENCY: Package C9 Residency Counter. |
| 82 | 85 | * perf code: 0x05 |
| 83 | − * Available model: HSW ULT,KBL,CNL | |
| 86 | + * Available model: HSW ULT,KBL,CNL,CML,ICL,TGL | |
| 84 | 87 | * Scope: Package (physical package) |
| 85 | 88 | * MSR_PKG_C10_RESIDENCY: Package C10 Residency Counter. |
| 86 | 89 | * perf code: 0x06 |
| 87 | − * Available model: HSW ULT,KBL,GLM,CNL | |
| 90 | + * Available model: HSW ULT,KBL,GLM,CNL,CML,ICL,TGL | |
| 88 | 91 | * Scope: Package (physical package) |
| 89 | 92 | * |
| 90 | 93 | */ |
| @@ -544,6 +547,19 @@ static const struct cstate_model cnl_cstates __initconst = { · 453 unmodified lines | ||
| 544 | 547 | BIT(PERF_CSTATE_PKG_C10_RES), |
| 545 | 548 | }; |
| 546 | 549 | |
| 550 | +static const struct cstate_model icl_cstates __initconst = { | |
| 551 | + .core_events = BIT(PERF_CSTATE_CORE_C6_RES) | | |
| 552 | + BIT(PERF_CSTATE_CORE_C7_RES), | |
| 553 | + | |
| 554 | + .pkg_events = BIT(PERF_CSTATE_PKG_C2_RES) | | |
| 555 | + BIT(PERF_CSTATE_PKG_C3_RES) | | |
| 556 | + BIT(PERF_CSTATE_PKG_C6_RES) | | |
| 557 | + BIT(PERF_CSTATE_PKG_C7_RES) | | |
| 558 | + BIT(PERF_CSTATE_PKG_C8_RES) | | |
| 559 | + BIT(PERF_CSTATE_PKG_C9_RES) | | |
| 560 | + BIT(PERF_CSTATE_PKG_C10_RES), | |
| 561 | +}; | |
| 562 | + | |
| 547 | 563 | static const struct cstate_model slm_cstates __initconst = { |
| 548 | 564 | .core_events = BIT(PERF_CSTATE_CORE_C1_RES) | |
| 549 | 565 | BIT(PERF_CSTATE_CORE_C6_RES), |
| @@ -614,6 +630,8 @@ static const struct x86_cpu_id intel_cstates_match[] __initconst = { · 64 unmodified lines | ||
| 614 | 630 | |
| 615 | 631 | X86_CSTATES_MODEL(INTEL_FAM6_KABYLAKE_L, hswult_cstates), |
| 616 | 632 | X86_CSTATES_MODEL(INTEL_FAM6_KABYLAKE, hswult_cstates), |
| 633 | + X86_CSTATES_MODEL(INTEL_FAM6_COMETLAKE_L, hswult_cstates), | |
| 634 | + X86_CSTATES_MODEL(INTEL_FAM6_COMETLAKE, hswult_cstates), | |
| 617 | 635 | |
| 618 | 636 | X86_CSTATES_MODEL(INTEL_FAM6_CANNONLAKE_L, cnl_cstates), |
| 619 | 637 | |
| @@ -625,8 +643,10 @@ static const struct x86_cpu_id intel_cstates_match[] __initconst = { · 5 unmodified lines | ||
| 625 | 643 | |
| 626 | 644 | X86_CSTATES_MODEL(INTEL_FAM6_ATOM_GOLDMONT_PLUS, glm_cstates), |
| 627 | 645 | |
| 628 | − X86_CSTATES_MODEL(INTEL_FAM6_ICELAKE_L, snb_cstates), | |
| 629 | − X86_CSTATES_MODEL(INTEL_FAM6_ICELAKE, snb_cstates), | |
| 646 | + X86_CSTATES_MODEL(INTEL_FAM6_ICELAKE_L, icl_cstates), | |
| 647 | + X86_CSTATES_MODEL(INTEL_FAM6_ICELAKE, icl_cstates), | |
| 648 | + X86_CSTATES_MODEL(INTEL_FAM6_TIGERLAKE_L, icl_cstates), | |
| 649 | + X86_CSTATES_MODEL(INTEL_FAM6_TIGERLAKE, icl_cstates), | |
| 630 | 650 | { }, |
| 631 | 651 | }; |
| 632 | 652 | MODULE_DEVICE_TABLE(x86cpu, intel_cstates_match); |
| @@ -627,7 +627,7 @@ static struct topa *topa_alloc(int cpu, gfp_t gfp) · 626 unmodified lines | ||
| 627 | 627 | * link as the 2nd entry in the table |
| 628 | 628 | */ |
| 629 | 629 | if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries)) { |
| 630 | − TOPA_ENTRY(&tp->topa, 1)->base = page_to_phys(p); | |
| 630 | + TOPA_ENTRY(&tp->topa, 1)->base = page_to_phys(p) >> TOPA_SHIFT; | |
| 631 | 631 | TOPA_ENTRY(&tp->topa, 1)->end = 1; |
| 632 | 632 | } |
| 633 | 633 | |
| @@ -502,10 +502,8 @@ void uncore_pmu_event_start(struct perf_event *event, int flags) · 501 unmodified lines | ||
| 502 | 502 | local64_set(&event->hw.prev_count, uncore_read_counter(box, event)); |
| 503 | 503 | uncore_enable_event(box, event); |
| 504 | 504 | |
| 505 | − if (box->n_active == 1) { | |
| 506 | − uncore_enable_box(box); | |
| 505 | + if (box->n_active == 1) | |
| 507 | 506 | uncore_pmu_start_hrtimer(box); |
| 508 | − } | |
| 509 | 507 | } |
| 510 | 508 | |
| 511 | 509 | void uncore_pmu_event_stop(struct perf_event *event, int flags) |
| @@ -529,10 +527,8 @@ void uncore_pmu_event_stop(struct perf_event *event, int flags) · 17 unmodified lines | ||
| 529 | 527 | WARN_ON_ONCE(hwc->state & PERF_HES_STOPPED); |
| 530 | 528 | hwc->state |= PERF_HES_STOPPED; |
| 531 | 529 | |
| 532 | − if (box->n_active == 0) { | |
| 533 | − uncore_disable_box(box); | |
| 530 | + if (box->n_active == 0) | |
| 534 | 531 | uncore_pmu_cancel_hrtimer(box); |
| 535 | − } | |
| 536 | 532 | } |
| 537 | 533 | |
| 538 | 534 | if ((flags & PERF_EF_UPDATE) && !(hwc->state & PERF_HES_UPTODATE)) { |
| @@ -778,6 +774,40 @@ static int uncore_pmu_event_init(struct perf_event *event) · 239 unmodified lines | ||
| 778 | 774 | return ret; |
| 779 | 775 | } |
| 780 | 776 | |
| 777 | +static void uncore_pmu_enable(struct pmu *pmu) | |
| 778 | +{ | |
| 779 | + struct intel_uncore_pmu *uncore_pmu; | |
| 780 | + struct intel_uncore_box *box; | |
| 781 | + | |
| 782 | + uncore_pmu = container_of(pmu, struct intel_uncore_pmu, pmu); | |
| 783 | + if (!uncore_pmu) | |
| 784 | + return; | |
| 785 | + | |
| 786 | + box = uncore_pmu_to_box(uncore_pmu, smp_processor_id()); | |
| 787 | + if (!box) | |
| 788 | + return; | |
| 789 | + | |
| 790 | + if (uncore_pmu->type->ops->enable_box) | |
| 791 | + uncore_pmu->type->ops->enable_box(box); | |
| 792 | +} | |
| 793 | + | |
| 794 | +static void uncore_pmu_disable(struct pmu *pmu) | |
| 795 | +{ | |
| 796 | + struct intel_uncore_pmu *uncore_pmu; | |
| 797 | + struct intel_uncore_box *box; | |
| 798 | + | |
| 799 | + uncore_pmu = container_of(pmu, struct intel_uncore_pmu, pmu); | |
| 800 | + if (!uncore_pmu) | |
| 801 | + return; | |
| 802 | + | |
| 803 | + box = uncore_pmu_to_box(uncore_pmu, smp_processor_id()); | |
| 804 | + if (!box) | |
| 805 | + return; | |
| 806 | + | |
| 807 | + if (uncore_pmu->type->ops->disable_box) | |
| 808 | + uncore_pmu->type->ops->disable_box(box); | |
| 809 | +} | |
| 810 | + | |
| 781 | 811 | static ssize_t uncore_get_attr_cpumask(struct device *dev, |
| 782 | 812 | struct device_attribute *attr, char *buf) |
| 783 | 813 | { |
| @@ -803,6 +833,8 @@ static int uncore_pmu_register(struct intel_uncore_pmu *pmu) · 19 unmodified lines | ||
| 803 | 833 | pmu->pmu = (struct pmu) { |
| 804 | 834 | .attr_groups = pmu->type->attr_groups, |
| 805 | 835 | .task_ctx_nr = perf_invalid_context, |
| 836 | + .pmu_enable = uncore_pmu_enable, | |
| 837 | + .pmu_disable = uncore_pmu_disable, | |
| 806 | 838 | .event_init = uncore_pmu_event_init, |
| 807 | 839 | .add = uncore_pmu_event_add, |
| 808 | 840 | .del = uncore_pmu_event_del, |
| @@ -441,18 +441,6 @@ static inline int uncore_freerunning_hw_config(struct intel_uncore_box *box, · 440 unmodified lines | ||
| 441 | 441 | return -EINVAL; |
| 442 | 442 | } |
| 443 | 443 | |
| 444 | −static inline void uncore_disable_box(struct intel_uncore_box *box) | |
| 445 | −{ | |
| 446 | − if (box->pmu->type->ops->disable_box) | |
| 447 | − box->pmu->type->ops->disable_box(box); | |
| 448 | −} | |
| 449 | − | |
| 450 | −static inline void uncore_enable_box(struct intel_uncore_box *box) | |
| 451 | −{ | |
| 452 | − if (box->pmu->type->ops->enable_box) | |
| 453 | − box->pmu->type->ops->enable_box(box); | |
| 454 | −} | |
| 455 | − | |
| 456 | 444 | static inline void uncore_disable_event(struct intel_uncore_box *box, |
| 457 | 445 | struct perf_event *event) |
| 458 | 446 | { |
| @@ -89,7 +89,14 @@ static bool test_intel(int idx, void *data) · 88 unmodified lines | ||
| 89 | 89 | case INTEL_FAM6_SKYLAKE_X: |
| 90 | 90 | case INTEL_FAM6_KABYLAKE_L: |
| 91 | 91 | case INTEL_FAM6_KABYLAKE: |
| 92 | + case INTEL_FAM6_COMETLAKE_L: | |
| 93 | + case INTEL_FAM6_COMETLAKE: | |
| 92 | 94 | case INTEL_FAM6_ICELAKE_L: |
| 95 | + case INTEL_FAM6_ICELAKE: | |
| 96 | + case INTEL_FAM6_ICELAKE_X: | |
| 97 | + case INTEL_FAM6_ICELAKE_D: | |
| 98 | + case INTEL_FAM6_TIGERLAKE_L: | |
| 99 | + case INTEL_FAM6_TIGERLAKE: | |
| 93 | 100 | if (idx == PERF_MSR_SMI || idx == PERF_MSR_PPERF) |
| 94 | 101 | return true; |
| 95 | 102 | break; |
| @@ -260,11 +260,21 @@ void __init hv_apic_init(void) · 259 unmodified lines | ||
| 260 | 260 | } |
| 261 | 261 | |
| 262 | 262 | if (ms_hyperv.hints & HV_X64_APIC_ACCESS_RECOMMENDED) { |
| 263 | − pr_info("Hyper-V: Using MSR based APIC access\n"); | |
| 263 | + pr_info("Hyper-V: Using enlightened APIC (%s mode)", | |
| 264 | + x2apic_enabled() ? "x2apic" : "xapic"); | |
| 265 | + /* | |
| 266 | + * With x2apic, architectural x2apic MSRs are equivalent to the | |
| 267 | + * respective synthetic MSRs, so there's no need to override | |
| 268 | + * the apic accessors. The only exception is | |
| 269 | + * hv_apic_eoi_write, because it benefits from lazy EOI when | |
| 270 | + * available, but it works for both xapic and x2apic modes. | |
| 271 | + */ | |
| 264 | 272 | apic_set_eoi_write(hv_apic_eoi_write); |
| 265 | − apic->read = hv_apic_read; | |
| 266 | − apic->write = hv_apic_write; | |
| 267 | − apic->icr_write = hv_apic_icr_write; | |
| 268 | − apic->icr_read = hv_apic_icr_read; | |
| 273 | + if (!x2apic_enabled()) { | |
| 274 | + apic->read = hv_apic_read; | |
| 275 | + apic->write = hv_apic_write; | |
| 276 | + apic->icr_write = hv_apic_icr_write; | |
| 277 | + apic->icr_read = hv_apic_icr_read; | |
| 278 | + } | |
| 269 | 279 | } |
| 270 | 280 | } |
| @@ -1,4 +1,4 @@ | ||
| 1 | −// SPDX-License-Identifier: GPL-2.0 | |
| 1 | +/* SPDX-License-Identifier: GPL-2.0 */ | |
| 2 | 2 | |
| 3 | 3 | #ifndef _ASM_X86_CPU_ENTRY_AREA_H |
| 4 | 4 | #define _ASM_X86_CPU_ENTRY_AREA_H |
| @@ -83,6 +83,9 @@ · 82 unmodified lines | ||
| 83 | 83 | #define INTEL_FAM6_TIGERLAKE_L 0x8C |
| 84 | 84 | #define INTEL_FAM6_TIGERLAKE 0x8D |
| 85 | 85 | |
| 86 | +#define INTEL_FAM6_COMETLAKE 0xA5 | |
| 87 | +#define INTEL_FAM6_COMETLAKE_L 0xA6 | |
| 88 | + | |
| 86 | 89 | /* "Small Core" Processors (Atom) */ |
| 87 | 90 | |
| 88 | 91 | #define INTEL_FAM6_ATOM_BONNELL 0x1C /* Diamondville, Pineview */ |
| @@ -1189,7 +1189,7 @@ struct kvm_x86_ops { · 1188 unmodified lines | ||
| 1189 | 1189 | int (*set_nested_state)(struct kvm_vcpu *vcpu, |
| 1190 | 1190 | struct kvm_nested_state __user *user_kvm_nested_state, |
| 1191 | 1191 | struct kvm_nested_state *kvm_state); |
| 1192 | − void (*get_vmcs12_pages)(struct kvm_vcpu *vcpu); | |
| 1192 | + bool (*get_vmcs12_pages)(struct kvm_vcpu *vcpu); | |
| 1193 | 1193 | |
| 1194 | 1194 | int (*smi_allowed)(struct kvm_vcpu *vcpu); |
| 1195 | 1195 | int (*pre_enter_smm)(struct kvm_vcpu *vcpu, char *smstate); |
| @@ -21,7 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | #define MWAIT_ECX_INTERRUPT_BREAK 0x1 |
| 22 | 22 | #define MWAITX_ECX_TIMER_ENABLE BIT(1) |
| 23 | 23 | #define MWAITX_MAX_LOOPS ((u32)-1) |
| 24 | −#define MWAITX_DISABLE_CSTATES 0xf | |
| 24 | +#define MWAITX_DISABLE_CSTATES 0xf0 | |
| 25 | 25 | |
| 26 | 26 | static inline void __monitor(const void *eax, unsigned long ecx, |
| 27 | 27 | unsigned long edx) |
| @@ -1,4 +1,4 @@ | ||
| 1 | −// SPDX-License-Identifier: GPL-2.0 | |
| 1 | +/* SPDX-License-Identifier: GPL-2.0 */ | |
| 2 | 2 | #ifndef _ASM_X86_PTI_H |
| 3 | 3 | #define _ASM_X86_PTI_H |
| 4 | 4 | #ifndef __ASSEMBLY__ |
| @@ -734,5 +734,28 @@ do { \ · 733 unmodified lines | ||
| 734 | 734 | if (unlikely(__gu_err)) goto err_label; \ |
| 735 | 735 | } while (0) |
| 736 | 736 | |
| 737 | +/* | |
| 738 | + * We want the unsafe accessors to always be inlined and use | |
| 739 | + * the error labels - thus the macro games. | |
| 740 | + */ | |
| 741 | +#define unsafe_copy_loop(dst, src, len, type, label) \ | |
| 742 | + while (len >= sizeof(type)) { \ | |
| 743 | + unsafe_put_user(*(type *)src,(type __user *)dst,label); \ | |
| 744 | + dst += sizeof(type); \ | |
| 745 | + src += sizeof(type); \ | |
| 746 | + len -= sizeof(type); \ | |
| 747 | + } | |
| 748 | + | |
| 749 | +#define unsafe_copy_to_user(_dst,_src,_len,label) \ | |
| 750 | +do { \ | |
| 751 | + char __user *__ucu_dst = (_dst); \ | |
| 752 | + const char *__ucu_src = (_src); \ | |
| 753 | + size_t __ucu_len = (_len); \ | |
| 754 | + unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u64, label); \ | |
| 755 | + unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u32, label); \ | |
| 756 | + unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u16, label); \ | |
| 757 | + unsafe_copy_loop(__ucu_dst, __ucu_src, __ucu_len, u8, label); \ | |
| 758 | +} while (0) | |
| 759 | + | |
| 737 | 760 | #endif /* _ASM_X86_UACCESS_H */ |
| 738 | 761 | |
| @@ -4,6 +4,7 @@ · 3 unmodified lines | ||
| 4 | 4 | |
| 5 | 5 | #include <asm/cpufeatures.h> |
| 6 | 6 | #include <asm/alternative.h> |
| 7 | +#include <linux/stringify.h> | |
| 7 | 8 | |
| 8 | 9 | /* |
| 9 | 10 | * The hypercall definitions differ in the low word of the %edx argument |
| @@ -20,8 +21,8 @@ · 10 unmodified lines | ||
| 20 | 21 | */ |
| 21 | 22 | |
| 22 | 23 | /* Old port-based version */ |
| 23 | −#define VMWARE_HYPERVISOR_PORT "0x5658" | |
| 24 | −#define VMWARE_HYPERVISOR_PORT_HB "0x5659" | |
| 24 | +#define VMWARE_HYPERVISOR_PORT 0x5658 | |
| 25 | +#define VMWARE_HYPERVISOR_PORT_HB 0x5659 | |
| 25 | 26 | |
| 26 | 27 | /* Current vmcall / vmmcall version */ |
| 27 | 28 | #define VMWARE_HYPERVISOR_HB BIT(0) |
| @@ -29,7 +30,8 @@ · 1 unmodified line | ||
| 29 | 30 | |
| 30 | 31 | /* The low bandwidth call. The low word of edx is presumed clear. */ |
| 31 | 32 | #define VMWARE_HYPERCALL \ |
| 32 | − ALTERNATIVE_2("movw $" VMWARE_HYPERVISOR_PORT ", %%dx; inl (%%dx)", \ | |
| 33 | + ALTERNATIVE_2("movw $" __stringify(VMWARE_HYPERVISOR_PORT) ", %%dx; " \ | |
| 34 | + "inl (%%dx), %%eax", \ | |
| 33 | 35 | "vmcall", X86_FEATURE_VMCALL, \ |
| 34 | 36 | "vmmcall", X86_FEATURE_VMW_VMMCALL) |
| 35 | 37 | |
| @@ -38,7 +40,8 @@ · 2 unmodified lines | ||
| 38 | 40 | * HB and OUT bits set. |
| 39 | 41 | */ |
| 40 | 42 | #define VMWARE_HYPERCALL_HB_OUT \ |
| 41 | − ALTERNATIVE_2("movw $" VMWARE_HYPERVISOR_PORT_HB ", %%dx; rep outsb", \ | |
| 43 | + ALTERNATIVE_2("movw $" __stringify(VMWARE_HYPERVISOR_PORT_HB) ", %%dx; " \ | |
| 44 | + "rep outsb", \ | |
| 42 | 45 | "vmcall", X86_FEATURE_VMCALL, \ |
| 43 | 46 | "vmmcall", X86_FEATURE_VMW_VMMCALL) |
| 44 | 47 | |
| @@ -47,7 +50,8 @@ · 2 unmodified lines | ||
| 47 | 50 | * HB bit set. |
| 48 | 51 | */ |
| 49 | 52 | #define VMWARE_HYPERCALL_HB_IN \ |
| 50 | − ALTERNATIVE_2("movw $" VMWARE_HYPERVISOR_PORT_HB ", %%dx; rep insb", \ | |
| 53 | + ALTERNATIVE_2("movw $" __stringify(VMWARE_HYPERVISOR_PORT_HB) ", %%dx; " \ | |
| 54 | + "rep insb", \ | |
| 51 | 55 | "vmcall", X86_FEATURE_VMCALL, \ |
| 52 | 56 | "vmmcall", X86_FEATURE_VMW_VMMCALL) |
| 53 | 57 | #endif |
| @@ -1586,9 +1586,6 @@ static void setup_local_APIC(void) · 1585 unmodified lines | ||
| 1586 | 1586 | { |
| 1587 | 1587 | int cpu = smp_processor_id(); |
| 1588 | 1588 | unsigned int value; |
| 1589 | −#ifdef CONFIG_X86_32 | |
| 1590 | − int logical_apicid, ldr_apicid; | |
| 1591 | −#endif | |
| 1592 | 1589 | |
| 1593 | 1590 | if (disable_apic) { |
| 1594 | 1591 | disable_ioapic_support(); |
| @@ -1626,16 +1623,21 @@ static void setup_local_APIC(void) · 31 unmodified lines | ||
| 1626 | 1623 | apic->init_apic_ldr(); |
| 1627 | 1624 | |
| 1628 | 1625 | #ifdef CONFIG_X86_32 |
| 1629 | − /* | |
| 1630 | − * APIC LDR is initialized. If logical_apicid mapping was | |
| 1631 | − * initialized during get_smp_config(), make sure it matches the | |
| 1632 | − * actual value. | |
| 1633 | − */ | |
| 1634 | − logical_apicid = early_per_cpu(x86_cpu_to_logical_apicid, cpu); | |
| 1635 | − ldr_apicid = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR)); | |
| 1636 | − WARN_ON(logical_apicid != BAD_APICID && logical_apicid != ldr_apicid); | |
| 1637 | − /* always use the value from LDR */ | |
| 1638 | − early_per_cpu(x86_cpu_to_logical_apicid, cpu) = ldr_apicid; | |
| 1626 | + if (apic->dest_logical) { | |
| 1627 | + int logical_apicid, ldr_apicid; | |
| 1628 | + | |
| 1629 | + /* | |
| 1630 | + * APIC LDR is initialized. If logical_apicid mapping was | |
| 1631 | + * initialized during get_smp_config(), make sure it matches | |
| 1632 | + * the actual value. | |
| 1633 | + */ | |
| 1634 | + logical_apicid = early_per_cpu(x86_cpu_to_logical_apicid, cpu); | |
| 1635 | + ldr_apicid = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR)); | |
| 1636 | + if (logical_apicid != BAD_APICID) | |
| 1637 | + WARN_ON(logical_apicid != ldr_apicid); | |
| 1638 | + /* Always use the value from LDR. */ | |
| 1639 | + early_per_cpu(x86_cpu_to_logical_apicid, cpu) = ldr_apicid; | |
| 1640 | + } | |
| 1639 | 1641 | #endif |
| 1640 | 1642 | |
| 1641 | 1643 | /* |
| @@ -156,7 +156,8 @@ static int x2apic_dead_cpu(unsigned int dead_cpu) · 155 unmodified lines | ||
| 156 | 156 | { |
| 157 | 157 | struct cluster_mask *cmsk = per_cpu(cluster_masks, dead_cpu); |
| 158 | 158 | |
| 159 | − cpumask_clear_cpu(dead_cpu, &cmsk->mask); | |
| 159 | + if (cmsk) | |
| 160 | + cpumask_clear_cpu(dead_cpu, &cmsk->mask); | |
| 160 | 161 | free_cpumask_var(per_cpu(ipi_mask, dead_cpu)); |
| 161 | 162 | return 0; |
| 162 | 163 | } |
| @@ -216,6 +216,10 @@ static void __init ms_hyperv_init_platform(void) · 215 unmodified lines | ||
| 216 | 216 | int hv_host_info_ecx; |
| 217 | 217 | int hv_host_info_edx; |
| 218 | 218 | |
| 219 | +#ifdef CONFIG_PARAVIRT | |
| 220 | + pv_info.name = "Hyper-V"; | |
| 221 | +#endif | |
| 222 | + | |
| 219 | 223 | /* |
| 220 | 224 | * Extract the features and hints |
| 221 | 225 | */ |
| @@ -522,6 +522,10 @@ int rdtgroup_mondata_show(struct seq_file *m, void *arg) · 521 unmodified lines | ||
| 522 | 522 | int ret = 0; |
| 523 | 523 | |
| 524 | 524 | rdtgrp = rdtgroup_kn_lock_live(of->kn); |
| 525 | + if (!rdtgrp) { | |
| 526 | + ret = -ENOENT; | |
| 527 | + goto out; | |
| 528 | + } | |
| 525 | 529 | |
| 526 | 530 | md.priv = of->kn->priv; |
| 527 | 531 | resid = md.u.rid; |
| @@ -49,7 +49,7 @@ · 48 unmodified lines | ||
| 49 | 49 | #define VMWARE_CMD_VCPU_RESERVED 31 |
| 50 | 50 | |
| 51 | 51 | #define VMWARE_PORT(cmd, eax, ebx, ecx, edx) \ |
| 52 | − __asm__("inl (%%dx)" : \ | |
| 52 | + __asm__("inl (%%dx), %%eax" : \ | |
| 53 | 53 | "=a"(eax), "=c"(ecx), "=d"(edx), "=b"(ebx) : \ |
| 54 | 54 | "a"(VMWARE_HYPERVISOR_MAGIC), \ |
| 55 | 55 | "c"(VMWARE_CMD_##cmd), \ |
| @@ -94,6 +94,13 @@ static bool in_exception_stack(unsigned long *stack, struct stack_info *info) · 93 unmodified lines | ||
| 94 | 94 | BUILD_BUG_ON(N_EXCEPTION_STACKS != 6); |
| 95 | 95 | |
| 96 | 96 | begin = (unsigned long)__this_cpu_read(cea_exception_stacks); |
| 97 | + /* | |
| 98 | + * Handle the case where stack trace is collected _before_ | |
| 99 | + * cea_exception_stacks had been initialized. | |
| 100 | + */ | |
| 101 | + if (!begin) | |
| 102 | + return false; | |
| 103 | + | |
| 97 | 104 | end = begin + sizeof(struct cea_exception_stacks); |
| 98 | 105 | /* Bail if @stack is outside the exception stack area. */ |
| 99 | 106 | if (stk < begin || stk >= end) |
| @@ -222,13 +222,31 @@ unsigned long __head __startup_64(unsigned long physaddr, · 221 unmodified lines | ||
| 222 | 222 | * we might write invalid pmds, when the kernel is relocated |
| 223 | 223 | * cleanup_highmap() fixes this up along with the mappings |
| 224 | 224 | * beyond _end. |
| 225 | + * | |
| 226 | + * Only the region occupied by the kernel image has so far | |
| 227 | + * been checked against the table of usable memory regions | |
| 228 | + * provided by the firmware, so invalidate pages outside that | |
| 229 | + * region. A page table entry that maps to a reserved area of | |
| 230 | + * memory would allow processor speculation into that area, | |
| 231 | + * and on some hardware (particularly the UV platform) even | |
| 232 | + * speculative access to some reserved areas is caught as an | |
| 233 | + * error, causing the BIOS to halt the system. | |
| 225 | 234 | */ |
| 226 | 235 | |
| 227 | 236 | pmd = fixup_pointer(level2_kernel_pgt, physaddr); |
| 228 | − for (i = 0; i < PTRS_PER_PMD; i++) { | |
| 237 | + | |
| 238 | + /* invalidate pages before the kernel image */ | |
| 239 | + for (i = 0; i < pmd_index((unsigned long)_text); i++) | |
| 240 | + pmd[i] &= ~_PAGE_PRESENT; | |
| 241 | + | |
| 242 | + /* fixup pages that are part of the kernel image */ | |
| 243 | + for (; i <= pmd_index((unsigned long)_end); i++) | |
| 229 | 244 | if (pmd[i] & _PAGE_PRESENT) |
| 230 | 245 | pmd[i] += load_delta; |
| 231 | − } | |
| 246 | + | |
| 247 | + /* invalidate pages after the kernel image */ | |
| 248 | + for (; i < PTRS_PER_PMD; i++) | |
| 249 | + pmd[i] &= ~_PAGE_PRESENT; | |
| 232 | 250 | |
| 233 | 251 | /* |
| 234 | 252 | * Fixup phys_base - remove the memory encryption mask to obtain |
| @@ -1,4 +1,4 @@ | ||
| 1 | −// SPDX-License-Identifier: GPL-2.0 | |
| 1 | +/* SPDX-License-Identifier: GPL-2.0 */ | |
| 2 | 2 | // |
| 3 | 3 | // Code shared between 32 and 64 bit |
| 4 | 4 | |
| @@ -1505,6 +1505,9 @@ void __init tsc_init(void) · 1504 unmodified lines | ||
| 1505 | 1505 | return; |
| 1506 | 1506 | } |
| 1507 | 1507 | |
| 1508 | + if (tsc_clocksource_reliable || no_tsc_watchdog) | |
| 1509 | + clocksource_tsc_early.flags &= ~CLOCK_SOURCE_MUST_VERIFY; | |
| 1510 | + | |
| 1508 | 1511 | clocksource_register_khz(&clocksource_tsc_early, tsc_khz); |
| 1509 | 1512 | detect_art(); |
| 1510 | 1513 | } |
| @@ -363,7 +363,7 @@ static inline void do_cpuid_7_mask(struct kvm_cpuid_entry2 *entry, int index) · 362 unmodified lines | ||
| 363 | 363 | |
| 364 | 364 | /* cpuid 7.0.ecx*/ |
| 365 | 365 | const u32 kvm_cpuid_7_0_ecx_x86_features = |
| 366 | − F(AVX512VBMI) | F(LA57) | F(PKU) | 0 /*OSPKE*/ | | |
| 366 | + F(AVX512VBMI) | F(LA57) | F(PKU) | 0 /*OSPKE*/ | F(RDPID) | | |
| 367 | 367 | F(AVX512_VPOPCNTDQ) | F(UMIP) | F(AVX512_VBMI2) | F(GFNI) | |
| 368 | 368 | F(VAES) | F(VPCLMULQDQ) | F(AVX512_VNNI) | F(AVX512_BITALG) | |
| 369 | 369 | F(CLDEMOTE) | F(MOVDIRI) | F(MOVDIR64B) | 0 /*WAITPKG*/; |
| @@ -111,11 +111,6 @@ static inline int apic_enabled(struct kvm_lapic *apic) · 110 unmodified lines | ||
| 111 | 111 | (LVT_MASK | APIC_MODE_MASK | APIC_INPUT_POLARITY | \ |
| 112 | 112 | APIC_LVT_REMOTE_IRR | APIC_LVT_LEVEL_TRIGGER) |
| 113 | 113 | |
| 114 | −static inline u8 kvm_xapic_id(struct kvm_lapic *apic) | |
| 115 | −{ | |
| 116 | − return kvm_lapic_get_reg(apic, APIC_ID) >> 24; | |
| 117 | −} | |
| 118 | − | |
| 119 | 114 | static inline u32 kvm_x2apic_id(struct kvm_lapic *apic) |
| 120 | 115 | { |
| 121 | 116 | return apic->vcpu->vcpu_id; |
| @@ -242,4 +242,9 @@ static inline enum lapic_mode kvm_apic_mode(u64 apic_base) · 241 unmodified lines | ||
| 242 | 242 | return apic_base & (MSR_IA32_APICBASE_ENABLE | X2APIC_ENABLE); |
| 243 | 243 | } |
| 244 | 244 | |
| 245 | +static inline u8 kvm_xapic_id(struct kvm_lapic *apic) | |
| 246 | +{ | |
| 247 | + return kvm_lapic_get_reg(apic, APIC_ID) >> 24; | |
| 248 | +} | |
| 249 | + | |
| 245 | 250 | #endif |
| @@ -734,8 +734,14 @@ static int get_npt_level(struct kvm_vcpu *vcpu) · 733 unmodified lines | ||
| 734 | 734 | static void svm_set_efer(struct kvm_vcpu *vcpu, u64 efer) |
| 735 | 735 | { |
| 736 | 736 | vcpu->arch.efer = efer; |
| 737 | − if (!npt_enabled && !(efer & EFER_LMA)) | |
| 738 | − efer &= ~EFER_LME; | |
| 737 | + | |
| 738 | + if (!npt_enabled) { | |
| 739 | + /* Shadow paging assumes NX to be available. */ | |
| 740 | + efer |= EFER_NX; | |
| 741 | + | |
| 742 | + if (!(efer & EFER_LMA)) | |
| 743 | + efer &= ~EFER_LME; | |
| 744 | + } | |
| 739 | 745 | |
| 740 | 746 | to_svm(vcpu)->vmcb->save.efer = efer | EFER_SVME; |
| 741 | 747 | mark_dirty(to_svm(vcpu)->vmcb, VMCB_CR); |
| @@ -4591,6 +4597,7 @@ static int avic_handle_ldr_update(struct kvm_vcpu *vcpu) · 3849 unmodified lines | ||
| 4591 | 4597 | int ret = 0; |
| 4592 | 4598 | struct vcpu_svm *svm = to_svm(vcpu); |
| 4593 | 4599 | u32 ldr = kvm_lapic_get_reg(vcpu->arch.apic, APIC_LDR); |
| 4600 | + u32 id = kvm_xapic_id(vcpu->arch.apic); | |
| 4594 | 4601 | |
| 4595 | 4602 | if (ldr == svm->ldr_reg) |
| 4596 | 4603 | return 0; |
| @@ -4598,7 +4605,7 @@ static int avic_handle_ldr_update(struct kvm_vcpu *vcpu) · 1 unmodified line | ||
| 4598 | 4605 | avic_invalidate_logical_id_entry(vcpu); |
| 4599 | 4606 | |
| 4600 | 4607 | if (ldr) |
| 4601 | − ret = avic_ldr_write(vcpu, vcpu->vcpu_id, ldr); | |
| 4608 | + ret = avic_ldr_write(vcpu, id, ldr); | |
| 4602 | 4609 | |
| 4603 | 4610 | if (!ret) |
| 4604 | 4611 | svm->ldr_reg = ldr; |
| @@ -4610,8 +4617,7 @@ static int avic_handle_apic_id_update(struct kvm_vcpu *vcpu) · 5 unmodified lines | ||
| 4610 | 4617 | { |
| 4611 | 4618 | u64 *old, *new; |
| 4612 | 4619 | struct vcpu_svm *svm = to_svm(vcpu); |
| 4613 | − u32 apic_id_reg = kvm_lapic_get_reg(vcpu->arch.apic, APIC_ID); | |
| 4614 | − u32 id = (apic_id_reg >> 24) & 0xff; | |
| 4620 | + u32 id = kvm_xapic_id(vcpu->arch.apic); | |
| 4615 | 4621 | |
| 4616 | 4622 | if (vcpu->vcpu_id == id) |
| 4617 | 4623 | return 0; |
| @@ -2917,7 +2917,7 @@ static int nested_vmx_check_vmentry_hw(struct kvm_vcpu *vcpu) · 2916 unmodified lines | ||
| 2917 | 2917 | static inline bool nested_vmx_prepare_msr_bitmap(struct kvm_vcpu *vcpu, |
| 2918 | 2918 | struct vmcs12 *vmcs12); |
| 2919 | 2919 | |
| 2920 | −static void nested_get_vmcs12_pages(struct kvm_vcpu *vcpu) | |
| 2920 | +static bool nested_get_vmcs12_pages(struct kvm_vcpu *vcpu) | |
| 2921 | 2921 | { |
| 2922 | 2922 | struct vmcs12 *vmcs12 = get_vmcs12(vcpu); |
| 2923 | 2923 | struct vcpu_vmx *vmx = to_vmx(vcpu); |
| @@ -2937,19 +2937,18 @@ static void nested_get_vmcs12_pages(struct kvm_vcpu *vcpu) · 13 unmodified lines | ||
| 2937 | 2937 | vmx->nested.apic_access_page = NULL; |
| 2938 | 2938 | } |
| 2939 | 2939 | page = kvm_vcpu_gpa_to_page(vcpu, vmcs12->apic_access_addr); |
| 2940 | − /* | |
| 2941 | − * If translation failed, no matter: This feature asks | |
| 2942 | − * to exit when accessing the given address, and if it | |
| 2943 | − * can never be accessed, this feature won't do | |
| 2944 | − * anything anyway. | |
| 2945 | − */ | |
| 2946 | 2940 | if (!is_error_page(page)) { |
| 2947 | 2941 | vmx->nested.apic_access_page = page; |
| 2948 | 2942 | hpa = page_to_phys(vmx->nested.apic_access_page); |
| 2949 | 2943 | vmcs_write64(APIC_ACCESS_ADDR, hpa); |
| 2950 | 2944 | } else { |
| 2951 | − secondary_exec_controls_clearbit(vmx, | |
| 2952 | − SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES); | |
| 2945 | + pr_debug_ratelimited("%s: no backing 'struct page' for APIC-access address in vmcs12\n", | |
| 2946 | + __func__); | |
| 2947 | + vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR; | |
| 2948 | + vcpu->run->internal.suberror = | |
| 2949 | + KVM_INTERNAL_ERROR_EMULATION; | |
| 2950 | + vcpu->run->internal.ndata = 0; | |
| 2951 | + return false; | |
| 2953 | 2952 | } |
| 2954 | 2953 | } |
| 2955 | 2954 | |
| @@ -2994,6 +2993,7 @@ static void nested_get_vmcs12_pages(struct kvm_vcpu *vcpu) · 38 unmodified lines | ||
| 2994 | 2993 | exec_controls_setbit(vmx, CPU_BASED_USE_MSR_BITMAPS); |
| 2995 | 2994 | else |
| 2996 | 2995 | exec_controls_clearbit(vmx, CPU_BASED_USE_MSR_BITMAPS); |
| 2996 | + return true; | |
| 2997 | 2997 | } |
| 2998 | 2998 | |
| 2999 | 2999 | /* |
| @@ -3032,13 +3032,15 @@ static void load_vmcs12_host_state(struct kvm_vcpu *vcpu, · 32 unmodified lines | ||
| 3032 | 3032 | /* |
| 3033 | 3033 | * If from_vmentry is false, this is being called from state restore (either RSM |
| 3034 | 3034 | * or KVM_SET_NESTED_STATE). Otherwise it's called from vmlaunch/vmresume. |
| 3035 | −+ * | |
| 3036 | −+ * Returns: | |
| 3037 | −+ * 0 - success, i.e. proceed with actual VMEnter | |
| 3038 | −+ * 1 - consistency check VMExit | |
| 3039 | −+ * -1 - consistency check VMFail | |
| 3035 | + * | |
| 3036 | + * Returns: | |
| 3037 | + * NVMX_ENTRY_SUCCESS: Entered VMX non-root mode | |
| 3038 | + * NVMX_ENTRY_VMFAIL: Consistency check VMFail | |
| 3039 | + * NVMX_ENTRY_VMEXIT: Consistency check VMExit | |
| 3040 | + * NVMX_ENTRY_KVM_INTERNAL_ERROR: KVM internal error | |
| 3040 | 3041 | */ |
| 3041 | −int nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, bool from_vmentry) | |
| 3042 | +enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, | |
| 3043 | + bool from_vmentry) | |
| 3042 | 3044 | { |
| 3043 | 3045 | struct vcpu_vmx *vmx = to_vmx(vcpu); |
| 3044 | 3046 | struct vmcs12 *vmcs12 = get_vmcs12(vcpu); |
| @@ -3081,11 +3083,12 @@ int nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, bool from_vmentry) · 36 unmodified lines | ||
| 3081 | 3083 | prepare_vmcs02_early(vmx, vmcs12); |
| 3082 | 3084 | |
| 3083 | 3085 | if (from_vmentry) { |
| 3084 | − nested_get_vmcs12_pages(vcpu); | |
| 3086 | + if (unlikely(!nested_get_vmcs12_pages(vcpu))) | |
| 3087 | + return NVMX_VMENTRY_KVM_INTERNAL_ERROR; | |
| 3085 | 3088 | |
| 3086 | 3089 | if (nested_vmx_check_vmentry_hw(vcpu)) { |
| 3087 | 3090 | vmx_switch_vmcs(vcpu, &vmx->vmcs01); |
| 3088 | − return -1; | |
| 3091 | + return NVMX_VMENTRY_VMFAIL; | |
| 3089 | 3092 | } |
| 3090 | 3093 | |
| 3091 | 3094 | if (nested_vmx_check_guest_state(vcpu, vmcs12, &exit_qual)) |
| @@ -3149,7 +3152,7 @@ int nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, bool from_vmentry) · 57 unmodified lines | ||
| 3149 | 3152 | * returned as far as L1 is concerned. It will only return (and set |
| 3150 | 3153 | * the success flag) when L2 exits (see nested_vmx_vmexit()). |
| 3151 | 3154 | */ |
| 3152 | − return 0; | |
| 3155 | + return NVMX_VMENTRY_SUCCESS; | |
| 3153 | 3156 | |
| 3154 | 3157 | /* |
| 3155 | 3158 | * A failed consistency check that leads to a VMExit during L1's |
| @@ -3165,14 +3168,14 @@ vmentry_fail_vmexit: · 9 unmodified lines | ||
| 3165 | 3168 | vmx_switch_vmcs(vcpu, &vmx->vmcs01); |
| 3166 | 3169 | |
| 3167 | 3170 | if (!from_vmentry) |
| 3168 | − return 1; | |
| 3171 | + return NVMX_VMENTRY_VMEXIT; | |
| 3169 | 3172 | |
| 3170 | 3173 | load_vmcs12_host_state(vcpu, vmcs12); |
| 3171 | 3174 | vmcs12->vm_exit_reason = exit_reason | VMX_EXIT_REASONS_FAILED_VMENTRY; |
| 3172 | 3175 | vmcs12->exit_qualification = exit_qual; |
| 3173 | 3176 | if (enable_shadow_vmcs || vmx->nested.hv_evmcs) |
| 3174 | 3177 | vmx->nested.need_vmcs12_to_shadow_sync = true; |
| 3175 | − return 1; | |
| 3178 | + return NVMX_VMENTRY_VMEXIT; | |
| 3176 | 3179 | } |
| 3177 | 3180 | |
| 3178 | 3181 | /* |
| @@ -3182,9 +3185,9 @@ vmentry_fail_vmexit: · 3 unmodified lines | ||
| 3182 | 3185 | static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch) |
| 3183 | 3186 | { |
| 3184 | 3187 | struct vmcs12 *vmcs12; |
| 3188 | + enum nvmx_vmentry_status status; | |
| 3185 | 3189 | struct vcpu_vmx *vmx = to_vmx(vcpu); |
| 3186 | 3190 | u32 interrupt_shadow = vmx_get_interrupt_shadow(vcpu); |
| 3187 | − int ret; | |
| 3188 | 3191 | |
| 3189 | 3192 | if (!nested_vmx_check_permission(vcpu)) |
| 3190 | 3193 | return 1; |
| @@ -3244,13 +3247,9 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch) · 53 unmodified lines | ||
| 3244 | 3247 | * the nested entry. |
| 3245 | 3248 | */ |
| 3246 | 3249 | vmx->nested.nested_run_pending = 1; |
| 3247 | − ret = nested_vmx_enter_non_root_mode(vcpu, true); | |
| 3248 | − vmx->nested.nested_run_pending = !ret; | |
| 3249 | − if (ret > 0) | |
| 3250 | − return 1; | |
| 3251 | − else if (ret) | |
| 3252 | − return nested_vmx_failValid(vcpu, | |
| 3253 | − VMXERR_ENTRY_INVALID_CONTROL_FIELD); | |
| 3250 | + status = nested_vmx_enter_non_root_mode(vcpu, true); | |
| 3251 | + if (unlikely(status != NVMX_VMENTRY_SUCCESS)) | |
| 3252 | + goto vmentry_failed; | |
| 3254 | 3253 | |
| 3255 | 3254 | /* Hide L1D cache contents from the nested guest. */ |
| 3256 | 3255 | vmx->vcpu.arch.l1tf_flush_l1d = true; |
| @@ -3281,6 +3280,15 @@ static int nested_vmx_run(struct kvm_vcpu *vcpu, bool launch) · 24 unmodified lines | ||
| 3281 | 3280 | return kvm_vcpu_halt(vcpu); |
| 3282 | 3281 | } |
| 3283 | 3282 | return 1; |
| 3283 | + | |
| 3284 | +vmentry_failed: | |
| 3285 | + vmx->nested.nested_run_pending = 0; | |
| 3286 | + if (status == NVMX_VMENTRY_KVM_INTERNAL_ERROR) | |
| 3287 | + return 0; | |
| 3288 | + if (status == NVMX_VMENTRY_VMEXIT) | |
| 3289 | + return 1; | |
| 3290 | + WARN_ON_ONCE(status != NVMX_VMENTRY_VMFAIL); | |
| 3291 | + return nested_vmx_failValid(vcpu, VMXERR_ENTRY_INVALID_CONTROL_FIELD); | |
| 3284 | 3292 | } |
| 3285 | 3293 | |
| 3286 | 3294 | /* |
| @@ -6,6 +6,16 @@ · 5 unmodified lines | ||
| 6 | 6 | #include "vmcs12.h" |
| 7 | 7 | #include "vmx.h" |
| 8 | 8 | |
| 9 | +/* | |
| 10 | + * Status returned by nested_vmx_enter_non_root_mode(): | |
| 11 | + */ | |
| 12 | +enum nvmx_vmentry_status { | |
| 13 | + NVMX_VMENTRY_SUCCESS, /* Entered VMX non-root mode */ | |
| 14 | + NVMX_VMENTRY_VMFAIL, /* Consistency check VMFail */ | |
| 15 | + NVMX_VMENTRY_VMEXIT, /* Consistency check VMExit */ | |
| 16 | + NVMX_VMENTRY_KVM_INTERNAL_ERROR,/* KVM internal error */ | |
| 17 | +}; | |
| 18 | + | |
| 9 | 19 | void vmx_leave_nested(struct kvm_vcpu *vcpu); |
| 10 | 20 | void nested_vmx_setup_ctls_msrs(struct nested_vmx_msrs *msrs, u32 ept_caps, |
| 11 | 21 | bool apicv); |
| @@ -13,7 +23,8 @@ void nested_vmx_hardware_unsetup(void); · 1 unmodified line | ||
| 13 | 23 | __init int nested_vmx_hardware_setup(int (*exit_handlers[])(struct kvm_vcpu *)); |
| 14 | 24 | void nested_vmx_vcpu_setup(void); |
| 15 | 25 | void nested_vmx_free_vcpu(struct kvm_vcpu *vcpu); |
| 16 | −int nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, bool from_vmentry); | |
| 26 | +enum nvmx_vmentry_status nested_vmx_enter_non_root_mode(struct kvm_vcpu *vcpu, | |
| 27 | + bool from_vmentry); | |
| 17 | 28 | bool nested_vmx_exit_reflected(struct kvm_vcpu *vcpu, u32 exit_reason); |
| 18 | 29 | void nested_vmx_vmexit(struct kvm_vcpu *vcpu, u32 exit_reason, |
| 19 | 30 | u32 exit_intr_info, unsigned long exit_qualification); |
| @@ -969,17 +969,9 @@ static bool update_transition_efer(struct vcpu_vmx *vmx, int efer_offset) · 968 unmodified lines | ||
| 969 | 969 | u64 guest_efer = vmx->vcpu.arch.efer; |
| 970 | 970 | u64 ignore_bits = 0; |
| 971 | 971 | |
| 972 | − if (!enable_ept) { | |
| 973 | − /* | |
| 974 | − * NX is needed to handle CR0.WP=1, CR4.SMEP=1. Testing | |
| 975 | − * host CPUID is more efficient than testing guest CPUID | |
| 976 | − * or CR4. Host SMEP is anyway a requirement for guest SMEP. | |
| 977 | − */ | |
| 978 | − if (boot_cpu_has(X86_FEATURE_SMEP)) | |
| 979 | − guest_efer |= EFER_NX; | |
| 980 | − else if (!(guest_efer & EFER_NX)) | |
| 981 | − ignore_bits |= EFER_NX; | |
| 982 | − } | |
| 972 | + /* Shadow paging assumes NX to be available. */ | |
| 973 | + if (!enable_ept) | |
| 974 | + guest_efer |= EFER_NX; | |
| 983 | 975 | |
| 984 | 976 | /* |
| 985 | 977 | * LMA and LME handled by hardware; SCE meaningless outside long mode. |
| @@ -5543,14 +5535,6 @@ static int handle_encls(struct kvm_vcpu *vcpu) · 4557 unmodified lines | ||
| 5543 | 5535 | return 1; |
| 5544 | 5536 | } |
| 5545 | 5537 | |
| 5546 | −static int handle_unexpected_vmexit(struct kvm_vcpu *vcpu) | |
| 5547 | −{ | |
| 5548 | − kvm_skip_emulated_instruction(vcpu); | |
| 5549 | − WARN_ONCE(1, "Unexpected VM-Exit Reason = 0x%x", | |
| 5550 | − vmcs_read32(VM_EXIT_REASON)); | |
| 5551 | − return 1; | |
| 5552 | −} | |
| 5553 | − | |
| 5554 | 5538 | /* |
| 5555 | 5539 | * The exit handlers return 1 if the exit was handled fully and guest execution |
| 5556 | 5540 | * may resume. Otherwise they set the kvm_run parameter to indicate what needs |
| @@ -5602,15 +5586,11 @@ static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu) = { · 45 unmodified lines | ||
| 5602 | 5586 | [EXIT_REASON_INVVPID] = handle_vmx_instruction, |
| 5603 | 5587 | [EXIT_REASON_RDRAND] = handle_invalid_op, |
| 5604 | 5588 | [EXIT_REASON_RDSEED] = handle_invalid_op, |
| 5605 | − [EXIT_REASON_XSAVES] = handle_unexpected_vmexit, | |
| 5606 | − [EXIT_REASON_XRSTORS] = handle_unexpected_vmexit, | |
| 5607 | 5589 | [EXIT_REASON_PML_FULL] = handle_pml_full, |
| 5608 | 5590 | [EXIT_REASON_INVPCID] = handle_invpcid, |
| 5609 | 5591 | [EXIT_REASON_VMFUNC] = handle_vmx_instruction, |
| 5610 | 5592 | [EXIT_REASON_PREEMPTION_TIMER] = handle_preemption_timer, |
| 5611 | 5593 | [EXIT_REASON_ENCLS] = handle_encls, |
| 5612 | − [EXIT_REASON_UMWAIT] = handle_unexpected_vmexit, | |
| 5613 | − [EXIT_REASON_TPAUSE] = handle_unexpected_vmexit, | |
| 5614 | 5594 | }; |
| 5615 | 5595 | |
| 5616 | 5596 | static const int kvm_vmx_max_exit_handlers = |
| @@ -360,8 +360,7 @@ EXPORT_SYMBOL_GPL(kvm_set_apic_base); · 359 unmodified lines | ||
| 360 | 360 | asmlinkage __visible void kvm_spurious_fault(void) |
| 361 | 361 | { |
| 362 | 362 | /* Fault while not rebooting. We want the trace. */ |
| 363 | − if (!kvm_rebooting) | |
| 364 | − BUG(); | |
| 363 | + BUG_ON(!kvm_rebooting); | |
| 365 | 364 | } |
| 366 | 365 | EXPORT_SYMBOL_GPL(kvm_spurious_fault); |
| 367 | 366 | |
| @@ -2537,6 +2536,7 @@ static int kvm_pv_enable_async_pf(struct kvm_vcpu *vcpu, u64 data) · 2169 unmodified lines | ||
| 2537 | 2536 | static void kvmclock_reset(struct kvm_vcpu *vcpu) |
| 2538 | 2537 | { |
| 2539 | 2538 | vcpu->arch.pv_time_enabled = false; |
| 2539 | + vcpu->arch.time = 0; | |
| 2540 | 2540 | } |
| 2541 | 2541 | |
| 2542 | 2542 | static void kvm_vcpu_flush_tlb(struct kvm_vcpu *vcpu, bool invalidate_gpa) |
| @@ -2702,8 +2702,6 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · 159 unmodified lines | ||
| 2702 | 2702 | case MSR_KVM_SYSTEM_TIME: { |
| 2703 | 2703 | struct kvm_arch *ka = &vcpu->kvm->arch; |
| 2704 | 2704 | |
| 2705 | − kvmclock_reset(vcpu); | |
| 2706 | − | |
| 2707 | 2705 | if (vcpu->vcpu_id == 0 && !msr_info->host_initiated) { |
| 2708 | 2706 | bool tmp = (msr == MSR_KVM_SYSTEM_TIME); |
| 2709 | 2707 | |
| @@ -2717,14 +2715,13 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · 7 unmodified lines | ||
| 2717 | 2715 | kvm_make_request(KVM_REQ_GLOBAL_CLOCK_UPDATE, vcpu); |
| 2718 | 2716 | |
| 2719 | 2717 | /* we verify if the enable bit is set... */ |
| 2718 | + vcpu->arch.pv_time_enabled = false; | |
| 2720 | 2719 | if (!(data & 1)) |
| 2721 | 2720 | break; |
| 2722 | 2721 | |
| 2723 | − if (kvm_gfn_to_hva_cache_init(vcpu->kvm, | |
| 2722 | + if (!kvm_gfn_to_hva_cache_init(vcpu->kvm, | |
| 2724 | 2723 | &vcpu->arch.pv_time, data & ~1ULL, |
| 2725 | 2724 | sizeof(struct pvclock_vcpu_time_info))) |
| 2726 | − vcpu->arch.pv_time_enabled = false; | |
| 2727 | − else | |
| 2728 | 2725 | vcpu->arch.pv_time_enabled = true; |
| 2729 | 2726 | |
| 2730 | 2727 | break; |
| @@ -7941,8 +7938,12 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · 5210 unmodified lines | ||
| 7941 | 7938 | bool req_immediate_exit = false; |
| 7942 | 7939 | |
| 7943 | 7940 | if (kvm_request_pending(vcpu)) { |
| 7944 | − if (kvm_check_request(KVM_REQ_GET_VMCS12_PAGES, vcpu)) | |
| 7945 | − kvm_x86_ops->get_vmcs12_pages(vcpu); | |
| 7941 | + if (kvm_check_request(KVM_REQ_GET_VMCS12_PAGES, vcpu)) { | |
| 7942 | + if (unlikely(!kvm_x86_ops->get_vmcs12_pages(vcpu))) { | |
| 7943 | + r = 0; | |
| 7944 | + goto out; | |
| 7945 | + } | |
| 7946 | + } | |
| 7946 | 7947 | if (kvm_check_request(KVM_REQ_MMU_RELOAD, vcpu)) |
| 7947 | 7948 | kvm_mmu_unload(vcpu); |
| 7948 | 7949 | if (kvm_check_request(KVM_REQ_MIGRATE_TIMER, vcpu)) |
| @@ -113,8 +113,8 @@ static void delay_mwaitx(unsigned long __loops) · 112 unmodified lines | ||
| 113 | 113 | __monitorx(raw_cpu_ptr(&cpu_tss_rw), 0, 0); |
| 114 | 114 | |
| 115 | 115 | /* |
| 116 | − * AMD, like Intel, supports the EAX hint and EAX=0xf | |
| 117 | − * means, do not enter any deep C-state and we use it | |
| 116 | + * AMD, like Intel's MWAIT version, supports the EAX hint and | |
| 117 | + * EAX=0xf0 means, do not enter any deep C-state and we use it | |
| 118 | 118 | * here in delay() to minimize wakeup latency. |
| 119 | 119 | */ |
| 120 | 120 | __mwaitx(MWAITX_DISABLE_CSTATES, delay, MWAITX_ECX_TIMER_ENABLE); |
| @@ -917,9 +917,6 @@ static void __init kexec_enter_virtual_mode(void) · 916 unmodified lines | ||
| 917 | 917 | |
| 918 | 918 | if (efi_enabled(EFI_OLD_MEMMAP) && (__supported_pte_mask & _PAGE_NX)) |
| 919 | 919 | runtime_code_page_mkexec(); |
| 920 | − | |
| 921 | − /* clean DUMMY object */ | |
| 922 | − efi_delete_dummy_variable(); | |
| 923 | 920 | #endif |
| 924 | 921 | } |
| 925 | 922 | |
| @@ -269,19 +269,41 @@ void xen_reboot(int reason) · 268 unmodified lines | ||
| 269 | 269 | BUG(); |
| 270 | 270 | } |
| 271 | 271 | |
| 272 | +static int reboot_reason = SHUTDOWN_reboot; | |
| 273 | +static bool xen_legacy_crash; | |
| 272 | 274 | void xen_emergency_restart(void) |
| 273 | 275 | { |
| 274 | − xen_reboot(SHUTDOWN_reboot); | |
| 276 | + xen_reboot(reboot_reason); | |
| 275 | 277 | } |
| 276 | 278 | |
| 277 | 279 | static int |
| 278 | 280 | xen_panic_event(struct notifier_block *this, unsigned long event, void *ptr) |
| 279 | 281 | { |
| 280 | − if (!kexec_crash_loaded()) | |
| 281 | − xen_reboot(SHUTDOWN_crash); | |
| 282 | + if (!kexec_crash_loaded()) { | |
| 283 | + if (xen_legacy_crash) | |
| 284 | + xen_reboot(SHUTDOWN_crash); | |
| 285 | + | |
| 286 | + reboot_reason = SHUTDOWN_crash; | |
| 287 | + | |
| 288 | + /* | |
| 289 | + * If panic_timeout==0 then we are supposed to wait forever. | |
| 290 | + * However, to preserve original dom0 behavior we have to drop | |
| 291 | + * into hypervisor. (domU behavior is controlled by its | |
| 292 | + * config file) | |
| 293 | + */ | |
| 294 | + if (panic_timeout == 0) | |
| 295 | + panic_timeout = -1; | |
| 296 | + } | |
| 282 | 297 | return NOTIFY_DONE; |
| 283 | 298 | } |
| 284 | 299 | |
| 300 | +static int __init parse_xen_legacy_crash(char *arg) | |
| 301 | +{ | |
| 302 | + xen_legacy_crash = true; | |
| 303 | + return 0; | |
| 304 | +} | |
| 305 | +early_param("xen_legacy_crash", parse_xen_legacy_crash); | |
| 306 | + | |
| 285 | 307 | static struct notifier_block xen_panic_block = { |
| 286 | 308 | .notifier_call = xen_panic_event, |
| 287 | 309 | .priority = INT_MIN |
| @@ -117,6 +117,14 @@ static void __init xen_banner(void) · 116 unmodified lines | ||
| 117 | 117 | printk(KERN_INFO "Xen version: %d.%d%s%s\n", |
| 118 | 118 | version >> 16, version & 0xffff, extra.extraversion, |
| 119 | 119 | xen_feature(XENFEAT_mmu_pt_update_preserve_ad) ? " (preserve-AD)" : ""); |
| 120 | + | |
| 121 | +#ifdef CONFIG_X86_32 | |
| 122 | + pr_warn("WARNING! WARNING! WARNING! WARNING! WARNING! WARNING! WARNING!\n" | |
| 123 | + "Support for running as 32-bit PV-guest under Xen will soon be removed\n" | |
| 124 | + "from the Linux kernel!\n" | |
| 125 | + "Please use either a 64-bit kernel or switch to HVM or PVH mode!\n" | |
| 126 | + "WARNING! WARNING! WARNING! WARNING! WARNING! WARNING! WARNING!\n"); | |
| 127 | +#endif | |
| 120 | 128 | } |
| 121 | 129 | |
| 122 | 130 | static void __init xen_pv_init_platform(void) |
| @@ -56,7 +56,7 @@ · 55 unmodified lines | ||
| 56 | 56 | reg = <0xf0100000 0x03f00000>; |
| 57 | 57 | |
| 58 | 58 | // BUS_ADDRESS(3) CPU_PHYSICAL(1) SIZE(2) |
| 59 | − ranges = <0x01000000 0x0 0xf0000000 0xf0000000 0x0 0x00010000>, | |
| 59 | + ranges = <0x01000000 0x0 0x00000000 0xf0000000 0x0 0x00010000>, | |
| 60 | 60 | <0x02000000 0x0 0xf4000000 0xf4000000 0x0 0x08000000>; |
| 61 | 61 | |
| 62 | 62 | // PCI_DEVICE(3) INT#(1) CONTROLLER(PHANDLE) CONTROLLER_DATA(2) |
| @@ -148,7 +148,7 @@ static inline void change_bit(unsigned int bit, volatile unsigned long *p) · 147 unmodified lines | ||
| 148 | 148 | " getex %0\n" |
| 149 | 149 | " beqz %0, 1b\n" |
| 150 | 150 | : "=&a" (tmp) |
| 151 | − : "a" (~mask), "a" (p) | |
| 151 | + : "a" (mask), "a" (p) | |
| 152 | 152 | : "memory"); |
| 153 | 153 | } |
| 154 | 154 | |
| @@ -100,7 +100,7 @@ do { \ · 99 unmodified lines | ||
| 100 | 100 | case 4: __put_user_asm(x, ptr, retval, 4, "s32i", __cb); break; \ |
| 101 | 101 | case 8: { \ |
| 102 | 102 | __typeof__(*ptr) __v64 = x; \ |
| 103 | − retval = __copy_to_user(ptr, &__v64, 8); \ | |
| 103 | + retval = __copy_to_user(ptr, &__v64, 8) ? -EFAULT : 0; \ | |
| 104 | 104 | break; \ |
| 105 | 105 | } \ |
| 106 | 106 | default: __put_user_bad(); \ |
| @@ -132,14 +132,14 @@ do { \ · 25 unmodified lines | ||
| 132 | 132 | #define __check_align_1 "" |
| 133 | 133 | |
| 134 | 134 | #define __check_align_2 \ |
| 135 | − " _bbci.l %3, 0, 1f \n" \ | |
| 136 | − " movi %0, %4 \n" \ | |
| 135 | + " _bbci.l %[addr], 0, 1f \n" \ | |
| 136 | + " movi %[err], %[efault] \n" \ | |
| 137 | 137 | " _j 2f \n" |
| 138 | 138 | |
| 139 | 139 | #define __check_align_4 \ |
| 140 | − " _bbsi.l %3, 0, 0f \n" \ | |
| 141 | − " _bbci.l %3, 1, 1f \n" \ | |
| 142 | − "0: movi %0, %4 \n" \ | |
| 140 | + " _bbsi.l %[addr], 0, 0f \n" \ | |
| 141 | + " _bbci.l %[addr], 1, 1f \n" \ | |
| 142 | + "0: movi %[err], %[efault] \n" \ | |
| 143 | 143 | " _j 2f \n" |
| 144 | 144 | |
| 145 | 145 | |
| @@ -151,40 +151,40 @@ do { \ · 5 unmodified lines | ||
| 151 | 151 | * WARNING: If you modify this macro at all, verify that the |
| 152 | 152 | * __check_align_* macros still work. |
| 153 | 153 | */ |
| 154 | −#define __put_user_asm(x, addr, err, align, insn, cb) \ | |
| 154 | +#define __put_user_asm(x_, addr_, err_, align, insn, cb)\ | |
| 155 | 155 | __asm__ __volatile__( \ |
| 156 | 156 | __check_align_##align \ |
| 157 | − "1: "insn" %2, %3, 0 \n" \ | |
| 157 | + "1: "insn" %[x], %[addr], 0 \n" \ | |
| 158 | 158 | "2: \n" \ |
| 159 | 159 | " .section .fixup,\"ax\" \n" \ |
| 160 | 160 | " .align 4 \n" \ |
| 161 | 161 | " .literal_position \n" \ |
| 162 | 162 | "5: \n" \ |
| 163 | − " movi %1, 2b \n" \ | |
| 164 | − " movi %0, %4 \n" \ | |
| 165 | − " jx %1 \n" \ | |
| 163 | + " movi %[tmp], 2b \n" \ | |
| 164 | + " movi %[err], %[efault] \n" \ | |
| 165 | + " jx %[tmp] \n" \ | |
| 166 | 166 | " .previous \n" \ |
| 167 | 167 | " .section __ex_table,\"a\" \n" \ |
| 168 | 168 | " .long 1b, 5b \n" \ |
| 169 | 169 | " .previous" \ |
| 170 | − :"=r" (err), "=r" (cb) \ | |
| 171 | − :"r" ((int)(x)), "r" (addr), "i" (-EFAULT), "0" (err)) | |
| 170 | + :[err] "+r"(err_), [tmp] "=r"(cb) \ | |
| 171 | + :[x] "r"(x_), [addr] "r"(addr_), [efault] "i"(-EFAULT)) | |
| 172 | 172 | |
| 173 | 173 | #define __get_user_nocheck(x, ptr, size) \ |
| 174 | 174 | ({ \ |
| 175 | − long __gu_err, __gu_val; \ | |
| 176 | − __get_user_size(__gu_val, (ptr), (size), __gu_err); \ | |
| 177 | − (x) = (__force __typeof__(*(ptr)))__gu_val; \ | |
| 175 | + long __gu_err; \ | |
| 176 | + __get_user_size((x), (ptr), (size), __gu_err); \ | |
| 178 | 177 | __gu_err; \ |
| 179 | 178 | }) |
| 180 | 179 | |
| 181 | 180 | #define __get_user_check(x, ptr, size) \ |
| 182 | 181 | ({ \ |
| 183 | − long __gu_err = -EFAULT, __gu_val = 0; \ | |
| 182 | + long __gu_err = -EFAULT; \ | |
| 184 | 183 | const __typeof__(*(ptr)) *__gu_addr = (ptr); \ |
| 185 | − if (access_ok(__gu_addr, size)) \ | |
| 186 | − __get_user_size(__gu_val, __gu_addr, (size), __gu_err); \ | |
| 187 | − (x) = (__force __typeof__(*(ptr)))__gu_val; \ | |
| 184 | + if (access_ok(__gu_addr, size)) \ | |
| 185 | + __get_user_size((x), __gu_addr, (size), __gu_err); \ | |
| 186 | + else \ | |
| 187 | + (x) = 0; \ | |
| 188 | 188 | __gu_err; \ |
| 189 | 189 | }) |
| 190 | 190 | |
| @@ -198,8 +198,17 @@ do { \ · 7 unmodified lines | ||
| 198 | 198 | case 1: __get_user_asm(x, ptr, retval, 1, "l8ui", __cb); break;\ |
| 199 | 199 | case 2: __get_user_asm(x, ptr, retval, 2, "l16ui", __cb); break;\ |
| 200 | 200 | case 4: __get_user_asm(x, ptr, retval, 4, "l32i", __cb); break;\ |
| 201 | − case 8: retval = __copy_from_user(&x, ptr, 8); break; \ | |
| 202 | − default: (x) = __get_user_bad(); \ | |
| 201 | + case 8: { \ | |
| 202 | + u64 __x; \ | |
| 203 | + if (unlikely(__copy_from_user(&__x, ptr, 8))) { \ | |
| 204 | + retval = -EFAULT; \ | |
| 205 | + (x) = 0; \ | |
| 206 | + } else { \ | |
| 207 | + (x) = *(__force __typeof__((ptr)))&__x; \ | |
| 208 | + } \ | |
| 209 | + break; \ | |
| 210 | + } \ | |
| 211 | + default: (x) = 0; __get_user_bad(); \ | |
| 203 | 212 | } \ |
| 204 | 213 | } while (0) |
| 205 | 214 | |
| @@ -208,25 +217,28 @@ do { \ · 2 unmodified lines | ||
| 208 | 217 | * WARNING: If you modify this macro at all, verify that the |
| 209 | 218 | * __check_align_* macros still work. |
| 210 | 219 | */ |
| 211 | −#define __get_user_asm(x, addr, err, align, insn, cb) \ | |
| 212 | −__asm__ __volatile__( \ | |
| 213 | − __check_align_##align \ | |
| 214 | − "1: "insn" %2, %3, 0 \n" \ | |
| 215 | − "2: \n" \ | |
| 216 | − " .section .fixup,\"ax\" \n" \ | |
| 217 | − " .align 4 \n" \ | |
| 218 | − " .literal_position \n" \ | |
| 219 | − "5: \n" \ | |
| 220 | − " movi %1, 2b \n" \ | |
| 221 | − " movi %2, 0 \n" \ | |
| 222 | − " movi %0, %4 \n" \ | |
| 223 | − " jx %1 \n" \ | |
| 224 | − " .previous \n" \ | |
| 225 | − " .section __ex_table,\"a\" \n" \ | |
| 226 | − " .long 1b, 5b \n" \ | |
| 227 | − " .previous" \ | |
| 228 | − :"=r" (err), "=r" (cb), "=r" (x) \ | |
| 229 | − :"r" (addr), "i" (-EFAULT), "0" (err)) | |
| 220 | +#define __get_user_asm(x_, addr_, err_, align, insn, cb) \ | |
| 221 | +do { \ | |
| 222 | + u32 __x = 0; \ | |
| 223 | + __asm__ __volatile__( \ | |
| 224 | + __check_align_##align \ | |
| 225 | + "1: "insn" %[x], %[addr], 0 \n" \ | |
| 226 | + "2: \n" \ | |
| 227 | + " .section .fixup,\"ax\" \n" \ | |
| 228 | + " .align 4 \n" \ | |
| 229 | + " .literal_position \n" \ | |
| 230 | + "5: \n" \ | |
| 231 | + " movi %[tmp], 2b \n" \ | |
| 232 | + " movi %[err], %[efault] \n" \ | |
| 233 | + " jx %[tmp] \n" \ | |
| 234 | + " .previous \n" \ | |
| 235 | + " .section __ex_table,\"a\" \n" \ | |
| 236 | + " .long 1b, 5b \n" \ | |
| 237 | + " .previous" \ | |
| 238 | + :[err] "+r"(err_), [tmp] "=r"(cb), [x] "+r"(__x) \ | |
| 239 | + :[addr] "r"(addr_), [efault] "i"(-EFAULT)); \ | |
| 240 | + (x_) = (__force __typeof__(*(addr_)))__x; \ | |
| 241 | +} while (0) | |
| 230 | 242 | |
| 231 | 243 | |
| 232 | 244 | /* |
| @@ -119,13 +119,6 @@ EXPORT_SYMBOL(__invalidate_icache_range); · 118 unmodified lines | ||
| 119 | 119 | // FIXME EXPORT_SYMBOL(screen_info); |
| 120 | 120 | #endif |
| 121 | 121 | |
| 122 | −EXPORT_SYMBOL(outsb); | |
| 123 | −EXPORT_SYMBOL(outsw); | |
| 124 | −EXPORT_SYMBOL(outsl); | |
| 125 | −EXPORT_SYMBOL(insb); | |
| 126 | −EXPORT_SYMBOL(insw); | |
| 127 | −EXPORT_SYMBOL(insl); | |
| 128 | − | |
| 129 | 122 | extern long common_exception_return; |
| 130 | 123 | EXPORT_SYMBOL(common_exception_return); |
| 131 | 124 | |
| @@ -934,9 +934,14 @@ static int blkcg_print_stat(struct seq_file *sf, void *v) · 933 unmodified lines | ||
| 934 | 934 | int i; |
| 935 | 935 | bool has_stats = false; |
| 936 | 936 | |
| 937 | + spin_lock_irq(&blkg->q->queue_lock); | |
| 938 | + | |
| 939 | + if (!blkg->online) | |
| 940 | + goto skip; | |
| 941 | + | |
| 937 | 942 | dname = blkg_dev_name(blkg); |
| 938 | 943 | if (!dname) |
| 939 | − continue; | |
| 944 | + goto skip; | |
| 940 | 945 | |
| 941 | 946 | /* |
| 942 | 947 | * Hooray string manipulation, count is the size written NOT |
| @@ -946,8 +951,6 @@ static int blkcg_print_stat(struct seq_file *sf, void *v) · 3 unmodified lines | ||
| 946 | 951 | */ |
| 947 | 952 | off += scnprintf(buf+off, size-off, "%s ", dname); |
| 948 | 953 | |
| 949 | − spin_lock_irq(&blkg->q->queue_lock); | |
| 950 | − | |
| 951 | 954 | blkg_rwstat_recursive_sum(blkg, NULL, |
| 952 | 955 | offsetof(struct blkcg_gq, stat_bytes), &rwstat); |
| 953 | 956 | rbytes = rwstat.cnt[BLKG_RWSTAT_READ]; |
| @@ -960,8 +963,6 @@ static int blkcg_print_stat(struct seq_file *sf, void *v) · 6 unmodified lines | ||
| 960 | 963 | wios = rwstat.cnt[BLKG_RWSTAT_WRITE]; |
| 961 | 964 | dios = rwstat.cnt[BLKG_RWSTAT_DISCARD]; |
| 962 | 965 | |
| 963 | − spin_unlock_irq(&blkg->q->queue_lock); | |
| 964 | − | |
| 965 | 966 | if (rbytes || wbytes || rios || wios) { |
| 966 | 967 | has_stats = true; |
| 967 | 968 | off += scnprintf(buf+off, size-off, |
| @@ -999,6 +1000,8 @@ static int blkcg_print_stat(struct seq_file *sf, void *v) · 31 unmodified lines | ||
| 999 | 1000 | seq_commit(sf, -1); |
| 1000 | 1001 | } |
| 1001 | 1002 | } |
| 1003 | + skip: | |
| 1004 | + spin_unlock_irq(&blkg->q->queue_lock); | |
| 1002 | 1005 | } |
| 1003 | 1006 | |
| 1004 | 1007 | rcu_read_unlock(); |
| @@ -1362,7 +1365,7 @@ int blkcg_activate_policy(struct request_queue *q, · 357 unmodified lines | ||
| 1362 | 1365 | const struct blkcg_policy *pol) |
| 1363 | 1366 | { |
| 1364 | 1367 | struct blkg_policy_data *pd_prealloc = NULL; |
| 1365 | − struct blkcg_gq *blkg; | |
| 1368 | + struct blkcg_gq *blkg, *pinned_blkg = NULL; | |
| 1366 | 1369 | int ret; |
| 1367 | 1370 | |
| 1368 | 1371 | if (blkcg_policy_enabled(q, pol)) |
| @@ -1370,49 +1373,82 @@ int blkcg_activate_policy(struct request_queue *q, · 1 unmodified line | ||
| 1370 | 1373 | |
| 1371 | 1374 | if (queue_is_mq(q)) |
| 1372 | 1375 | blk_mq_freeze_queue(q); |
| 1373 | −pd_prealloc: | |
| 1374 | − if (!pd_prealloc) { | |
| 1375 | − pd_prealloc = pol->pd_alloc_fn(GFP_KERNEL, q, &blkcg_root); | |
| 1376 | − if (!pd_prealloc) { | |
| 1377 | − ret = -ENOMEM; | |
| 1378 | − goto out_bypass_end; | |
| 1379 | − } | |
| 1380 | − } | |
| 1381 | − | |
| 1376 | +retry: | |
| 1382 | 1377 | spin_lock_irq(&q->queue_lock); |
| 1383 | 1378 | |
| 1384 | − /* blkg_list is pushed at the head, reverse walk to init parents first */ | |
| 1379 | + /* blkg_list is pushed at the head, reverse walk to allocate parents first */ | |
| 1385 | 1380 | list_for_each_entry_reverse(blkg, &q->blkg_list, q_node) { |
| 1386 | 1381 | struct blkg_policy_data *pd; |
| 1387 | 1382 | |
| 1388 | 1383 | if (blkg->pd[pol->plid]) |
| 1389 | 1384 | continue; |
| 1390 | 1385 | |
| 1391 | − pd = pol->pd_alloc_fn(GFP_NOWAIT | __GFP_NOWARN, q, &blkcg_root); | |
| 1392 | − if (!pd) | |
| 1393 | − swap(pd, pd_prealloc); | |
| 1386 | + /* If prealloc matches, use it; otherwise try GFP_NOWAIT */ | |
| 1387 | + if (blkg == pinned_blkg) { | |
| 1388 | + pd = pd_prealloc; | |
| 1389 | + pd_prealloc = NULL; | |
| 1390 | + } else { | |
| 1391 | + pd = pol->pd_alloc_fn(GFP_NOWAIT | __GFP_NOWARN, q, | |
| 1392 | + blkg->blkcg); | |
| 1393 | + } | |
| 1394 | + | |
| 1394 | 1395 | if (!pd) { |
| 1396 | + /* | |
| 1397 | + * GFP_NOWAIT failed. Free the existing one and | |
| 1398 | + * prealloc for @blkg w/ GFP_KERNEL. | |
| 1399 | + */ | |
| 1400 | + if (pinned_blkg) | |
| 1401 | + blkg_put(pinned_blkg); | |
| 1402 | + blkg_get(blkg); | |
| 1403 | + pinned_blkg = blkg; | |
| 1404 | + | |
| 1395 | 1405 | spin_unlock_irq(&q->queue_lock); |
| 1396 | − goto pd_prealloc; | |
| 1406 | + | |
| 1407 | + if (pd_prealloc) | |
| 1408 | + pol->pd_free_fn(pd_prealloc); | |
| 1409 | + pd_prealloc = pol->pd_alloc_fn(GFP_KERNEL, q, | |
| 1410 | + blkg->blkcg); | |
| 1411 | + if (pd_prealloc) | |
| 1412 | + goto retry; | |
| 1413 | + else | |
| 1414 | + goto enomem; | |
| 1397 | 1415 | } |
| 1398 | 1416 | |
| 1399 | 1417 | blkg->pd[pol->plid] = pd; |
| 1400 | 1418 | pd->blkg = blkg; |
| 1401 | 1419 | pd->plid = pol->plid; |
| 1402 | − if (pol->pd_init_fn) | |
| 1403 | − pol->pd_init_fn(pd); | |
| 1404 | 1420 | } |
| 1405 | 1421 | |
| 1422 | + /* all allocated, init in the same order */ | |
| 1423 | + if (pol->pd_init_fn) | |
| 1424 | + list_for_each_entry_reverse(blkg, &q->blkg_list, q_node) | |
| 1425 | + pol->pd_init_fn(blkg->pd[pol->plid]); | |
| 1426 | + | |
| 1406 | 1427 | __set_bit(pol->plid, q->blkcg_pols); |
| 1407 | 1428 | ret = 0; |
| 1408 | 1429 | |
| 1409 | 1430 | spin_unlock_irq(&q->queue_lock); |
| 1410 | −out_bypass_end: | |
| 1431 | +out: | |
| 1411 | 1432 | if (queue_is_mq(q)) |
| 1412 | 1433 | blk_mq_unfreeze_queue(q); |
| 1434 | + if (pinned_blkg) | |
| 1435 | + blkg_put(pinned_blkg); | |
| 1413 | 1436 | if (pd_prealloc) |
| 1414 | 1437 | pol->pd_free_fn(pd_prealloc); |
| 1415 | 1438 | return ret; |
| 1439 | + | |
| 1440 | +enomem: | |
| 1441 | + /* alloc failed, nothing's initialized yet, free everything */ | |
| 1442 | + spin_lock_irq(&q->queue_lock); | |
| 1443 | + list_for_each_entry(blkg, &q->blkg_list, q_node) { | |
| 1444 | + if (blkg->pd[pol->plid]) { | |
| 1445 | + pol->pd_free_fn(blkg->pd[pol->plid]); | |
| 1446 | + blkg->pd[pol->plid] = NULL; | |
| 1447 | + } | |
| 1448 | + } | |
| 1449 | + spin_unlock_irq(&q->queue_lock); | |
| 1450 | + ret = -ENOMEM; | |
| 1451 | + goto out; | |
| 1416 | 1452 | } |
| 1417 | 1453 | EXPORT_SYMBOL_GPL(blkcg_activate_policy); |
| 1418 | 1454 | |
| @@ -2110,10 +2110,10 @@ static ssize_t ioc_weight_write(struct kernfs_open_file *of, char *buf, · 2109 unmodified lines | ||
| 2110 | 2110 | goto einval; |
| 2111 | 2111 | } |
| 2112 | 2112 | |
| 2113 | − spin_lock_irq(&iocg->ioc->lock); | |
| 2113 | + spin_lock(&iocg->ioc->lock); | |
| 2114 | 2114 | iocg->cfg_weight = v; |
| 2115 | 2115 | weight_updated(iocg); |
| 2116 | − spin_unlock_irq(&iocg->ioc->lock); | |
| 2116 | + spin_unlock(&iocg->ioc->lock); | |
| 2117 | 2117 | |
| 2118 | 2118 | blkg_conf_finish(&ctx); |
| 2119 | 2119 | return nbytes; |
| @@ -160,24 +160,27 @@ bool rq_depth_calc_max_depth(struct rq_depth *rqd) · 159 unmodified lines | ||
| 160 | 160 | return ret; |
| 161 | 161 | } |
| 162 | 162 | |
| 163 | −void rq_depth_scale_up(struct rq_depth *rqd) | |
| 163 | +/* Returns true on success and false if scaling up wasn't possible */ | |
| 164 | +bool rq_depth_scale_up(struct rq_depth *rqd) | |
| 164 | 165 | { |
| 165 | 166 | /* |
| 166 | 167 | * Hit max in previous round, stop here |
| 167 | 168 | */ |
| 168 | 169 | if (rqd->scaled_max) |
| 169 | − return; | |
| 170 | + return false; | |
| 170 | 171 | |
| 171 | 172 | rqd->scale_step--; |
| 172 | 173 | |
| 173 | 174 | rqd->scaled_max = rq_depth_calc_max_depth(rqd); |
| 175 | + return true; | |
| 174 | 176 | } |
| 175 | 177 | |
| 176 | 178 | /* |
| 177 | 179 | * Scale rwb down. If 'hard_throttle' is set, do it quicker, since we |
| 178 | − * had a latency violation. | |
| 180 | + * had a latency violation. Returns true on success and returns false if | |
| 181 | + * scaling down wasn't possible. | |
| 179 | 182 | */ |
| 180 | −void rq_depth_scale_down(struct rq_depth *rqd, bool hard_throttle) | |
| 183 | +bool rq_depth_scale_down(struct rq_depth *rqd, bool hard_throttle) | |
| 181 | 184 | { |
| 182 | 185 | /* |
| 183 | 186 | * Stop scaling down when we've hit the limit. This also prevents |
| @@ -185,7 +188,7 @@ void rq_depth_scale_down(struct rq_depth *rqd, bool hard_throttle) · 1 unmodified line | ||
| 185 | 188 | * keep up. |
| 186 | 189 | */ |
| 187 | 190 | if (rqd->max_depth == 1) |
| 188 | − return; | |
| 191 | + return false; | |
| 189 | 192 | |
| 190 | 193 | if (rqd->scale_step < 0 && hard_throttle) |
| 191 | 194 | rqd->scale_step = 0; |
| @@ -194,6 +197,7 @@ void rq_depth_scale_down(struct rq_depth *rqd, bool hard_throttle) · 2 unmodified lines | ||
| 194 | 197 | |
| 195 | 198 | rqd->scaled_max = false; |
| 196 | 199 | rq_depth_calc_max_depth(rqd); |
| 200 | + return true; | |
| 197 | 201 | } |
| 198 | 202 | |
| 199 | 203 | struct rq_qos_wait_data { |
| @@ -108,16 +108,13 @@ static inline void rq_qos_add(struct request_queue *q, struct rq_qos *rqos) · 107 unmodified lines | ||
| 108 | 108 | |
| 109 | 109 | static inline void rq_qos_del(struct request_queue *q, struct rq_qos *rqos) |
| 110 | 110 | { |
| 111 | − struct rq_qos *cur, *prev = NULL; | |
| 112 | − for (cur = q->rq_qos; cur; cur = cur->next) { | |
| 113 | − if (cur == rqos) { | |
| 114 | − if (prev) | |
| 115 | − prev->next = rqos->next; | |
| 116 | − else | |
| 117 | − q->rq_qos = cur; | |
| 111 | + struct rq_qos **cur; | |
| 112 | + | |
| 113 | + for (cur = &q->rq_qos; *cur; cur = &(*cur)->next) { | |
| 114 | + if (*cur == rqos) { | |
| 115 | + *cur = rqos->next; | |
| 118 | 116 | break; |
| 119 | 117 | } |
| 120 | − prev = cur; | |
| 121 | 118 | } |
| 122 | 119 | |
| 123 | 120 | blk_mq_debugfs_unregister_rqos(rqos); |
| @@ -130,8 +127,8 @@ void rq_qos_wait(struct rq_wait *rqw, void *private_data, · 6 unmodified lines | ||
| 130 | 127 | acquire_inflight_cb_t *acquire_inflight_cb, |
| 131 | 128 | cleanup_cb_t *cleanup_cb); |
| 132 | 129 | bool rq_wait_inc_below(struct rq_wait *rq_wait, unsigned int limit); |
| 133 | −void rq_depth_scale_up(struct rq_depth *rqd); | |
| 134 | −void rq_depth_scale_down(struct rq_depth *rqd, bool hard_throttle); | |
| 130 | +bool rq_depth_scale_up(struct rq_depth *rqd); | |
| 131 | +bool rq_depth_scale_down(struct rq_depth *rqd, bool hard_throttle); | |
| 135 | 132 | bool rq_depth_calc_max_depth(struct rq_depth *rqd); |
| 136 | 133 | |
| 137 | 134 | void __rq_qos_cleanup(struct rq_qos *rqos, struct bio *bio); |
| @@ -308,7 +308,8 @@ static void calc_wb_limits(struct rq_wb *rwb) · 307 unmodified lines | ||
| 308 | 308 | |
| 309 | 309 | static void scale_up(struct rq_wb *rwb) |
| 310 | 310 | { |
| 311 | − rq_depth_scale_up(&rwb->rq_depth); | |
| 311 | + if (!rq_depth_scale_up(&rwb->rq_depth)) | |
| 312 | + return; | |
| 312 | 313 | calc_wb_limits(rwb); |
| 313 | 314 | rwb->unknown_cnt = 0; |
| 314 | 315 | rwb_wake_all(rwb); |
| @@ -317,7 +318,8 @@ static void scale_up(struct rq_wb *rwb) · 2 unmodified lines | ||
| 317 | 318 | |
| 318 | 319 | static void scale_down(struct rq_wb *rwb, bool hard_throttle) |
| 319 | 320 | { |
| 320 | − rq_depth_scale_down(&rwb->rq_depth, hard_throttle); | |
| 321 | + if (!rq_depth_scale_down(&rwb->rq_depth, hard_throttle)) | |
| 322 | + return; | |
| 321 | 323 | calc_wb_limits(rwb); |
| 322 | 324 | rwb->unknown_cnt = 0; |
| 323 | 325 | rwb_trace_step(rwb, "scale down"); |
| @@ -616,7 +616,8 @@ out: · 615 unmodified lines | ||
| 616 | 616 | |
| 617 | 617 | static inline bool elv_support_iosched(struct request_queue *q) |
| 618 | 618 | { |
| 619 | − if (q->tag_set && (q->tag_set->flags & BLK_MQ_F_NO_SCHED)) | |
| 619 | + if (!q->mq_ops || | |
| 620 | + (q->tag_set && (q->tag_set->flags & BLK_MQ_F_NO_SCHED))) | |
| 620 | 621 | return false; |
| 621 | 622 | return true; |
| 622 | 623 | } |
| @@ -905,8 +905,8 @@ void acpi_cppc_processor_exit(struct acpi_processor *pr) · 904 unmodified lines | ||
| 905 | 905 | pcc_data[pcc_ss_id]->refcount--; |
| 906 | 906 | if (!pcc_data[pcc_ss_id]->refcount) { |
| 907 | 907 | pcc_mbox_free_channel(pcc_data[pcc_ss_id]->pcc_channel); |
| 908 | − pcc_data[pcc_ss_id]->pcc_channel_acquired = 0; | |
| 909 | 908 | kfree(pcc_data[pcc_ss_id]); |
| 909 | + pcc_data[pcc_ss_id] = NULL; | |
| 910 | 910 | } |
| 911 | 911 | } |
| 912 | 912 | } |
| @@ -403,7 +403,7 @@ static int __init hmat_parse_proximity_domain(union acpi_subtable_headers *heade · 402 unmodified lines | ||
| 403 | 403 | pr_info("HMAT: Memory Flags:%04x Processor Domain:%d Memory Domain:%d\n", |
| 404 | 404 | p->flags, p->processor_PD, p->memory_PD); |
| 405 | 405 | |
| 406 | − if (p->flags & ACPI_HMAT_MEMORY_PD_VALID) { | |
| 406 | + if (p->flags & ACPI_HMAT_MEMORY_PD_VALID && hmat_revision == 1) { | |
| 407 | 407 | target = find_mem_target(p->memory_PD); |
| 408 | 408 | if (!target) { |
| 409 | 409 | pr_debug("HMAT: Memory Domain missing from SRAT\n"); |
| @@ -1322,7 +1322,7 @@ static ssize_t scrub_show(struct device *dev, · 1321 unmodified lines | ||
| 1322 | 1322 | nfit_device_lock(dev); |
| 1323 | 1323 | nd_desc = dev_get_drvdata(dev); |
| 1324 | 1324 | if (!nd_desc) { |
| 1325 | − device_unlock(dev); | |
| 1325 | + nfit_device_unlock(dev); | |
| 1326 | 1326 | return rc; |
| 1327 | 1327 | } |
| 1328 | 1328 | acpi_desc = to_acpi_desc(nd_desc); |
| @@ -290,14 +290,13 @@ static int acpi_processor_notifier(struct notifier_block *nb, · 289 unmodified lines | ||
| 290 | 290 | unsigned long event, void *data) |
| 291 | 291 | { |
| 292 | 292 | struct cpufreq_policy *policy = data; |
| 293 | − int cpu = policy->cpu; | |
| 294 | 293 | |
| 295 | 294 | if (event == CPUFREQ_CREATE_POLICY) { |
| 296 | − acpi_thermal_cpufreq_init(cpu); | |
| 297 | − acpi_processor_ppc_init(cpu); | |
| 295 | + acpi_thermal_cpufreq_init(policy); | |
| 296 | + acpi_processor_ppc_init(policy); | |
| 298 | 297 | } else if (event == CPUFREQ_REMOVE_POLICY) { |
| 299 | − acpi_processor_ppc_exit(cpu); | |
| 300 | − acpi_thermal_cpufreq_exit(cpu); | |
| 298 | + acpi_processor_ppc_exit(policy); | |
| 299 | + acpi_thermal_cpufreq_exit(policy); | |
| 301 | 300 | } |
| 302 | 301 | |
| 303 | 302 | return 0; |
| @@ -81,10 +81,10 @@ static int acpi_processor_get_platform_limit(struct acpi_processor *pr) · 80 unmodified lines | ||
| 81 | 81 | pr->performance_platform_limit = (int)ppc; |
| 82 | 82 | |
| 83 | 83 | if (ppc >= pr->performance->state_count || |
| 84 | − unlikely(!dev_pm_qos_request_active(&pr->perflib_req))) | |
| 84 | + unlikely(!freq_qos_request_active(&pr->perflib_req))) | |
| 85 | 85 | return 0; |
| 86 | 86 | |
| 87 | − ret = dev_pm_qos_update_request(&pr->perflib_req, | |
| 87 | + ret = freq_qos_update_request(&pr->perflib_req, | |
| 88 | 88 | pr->performance->states[ppc].core_frequency * 1000); |
| 89 | 89 | if (ret < 0) { |
| 90 | 90 | pr_warn("Failed to update perflib freq constraint: CPU%d (%d)\n", |
| @@ -157,26 +157,36 @@ void acpi_processor_ignore_ppc_init(void) · 66 unmodified lines | ||
| 157 | 157 | ignore_ppc = 0; |
| 158 | 158 | } |
| 159 | 159 | |
| 160 | −void acpi_processor_ppc_init(int cpu) | |
| 160 | +void acpi_processor_ppc_init(struct cpufreq_policy *policy) | |
| 161 | 161 | { |
| 162 | − struct acpi_processor *pr = per_cpu(processors, cpu); | |
| 163 | − int ret; | |
| 162 | + unsigned int cpu; | |
| 164 | 163 | |
| 165 | − ret = dev_pm_qos_add_request(get_cpu_device(cpu), | |
| 166 | − &pr->perflib_req, DEV_PM_QOS_MAX_FREQUENCY, | |
| 167 | − INT_MAX); | |
| 168 | − if (ret < 0) { | |
| 169 | − pr_err("Failed to add freq constraint for CPU%d (%d)\n", cpu, | |
| 170 | − ret); | |
| 171 | − return; | |
| 164 | + for_each_cpu(cpu, policy->related_cpus) { | |
| 165 | + struct acpi_processor *pr = per_cpu(processors, cpu); | |
| 166 | + int ret; | |
| 167 | + | |
| 168 | + if (!pr) | |
| 169 | + continue; | |
| 170 | + | |
| 171 | + ret = freq_qos_add_request(&policy->constraints, | |
| 172 | + &pr->perflib_req, | |
| 173 | + FREQ_QOS_MAX, INT_MAX); | |
| 174 | + if (ret < 0) | |
| 175 | + pr_err("Failed to add freq constraint for CPU%d (%d)\n", | |
| 176 | + cpu, ret); | |
| 172 | 177 | } |
| 173 | 178 | } |
| 174 | 179 | |
| 175 | −void acpi_processor_ppc_exit(int cpu) | |
| 180 | +void acpi_processor_ppc_exit(struct cpufreq_policy *policy) | |
| 176 | 181 | { |
| 177 | − struct acpi_processor *pr = per_cpu(processors, cpu); | |
| 182 | + unsigned int cpu; | |
| 178 | 183 | |
| 179 | − dev_pm_qos_remove_request(&pr->perflib_req); | |
| 184 | + for_each_cpu(cpu, policy->related_cpus) { | |
| 185 | + struct acpi_processor *pr = per_cpu(processors, cpu); | |
| 186 | + | |
| 187 | + if (pr) | |
| 188 | + freq_qos_remove_request(&pr->perflib_req); | |
| 189 | + } | |
| 180 | 190 | } |
| 181 | 191 | |
| 182 | 192 | static int acpi_processor_get_performance_control(struct acpi_processor *pr) |
| @@ -105,7 +105,7 @@ static int cpufreq_set_cur_state(unsigned int cpu, int state) · 104 unmodified lines | ||
| 105 | 105 | |
| 106 | 106 | pr = per_cpu(processors, i); |
| 107 | 107 | |
| 108 | − if (unlikely(!dev_pm_qos_request_active(&pr->thermal_req))) | |
| 108 | + if (unlikely(!freq_qos_request_active(&pr->thermal_req))) | |
| 109 | 109 | continue; |
| 110 | 110 | |
| 111 | 111 | policy = cpufreq_cpu_get(i); |
| @@ -116,7 +116,7 @@ static int cpufreq_set_cur_state(unsigned int cpu, int state) · 4 unmodified lines | ||
| 116 | 116 | |
| 117 | 117 | cpufreq_cpu_put(policy); |
| 118 | 118 | |
| 119 | − ret = dev_pm_qos_update_request(&pr->thermal_req, max_freq); | |
| 119 | + ret = freq_qos_update_request(&pr->thermal_req, max_freq); | |
| 120 | 120 | if (ret < 0) { |
| 121 | 121 | pr_warn("Failed to update thermal freq constraint: CPU%d (%d)\n", |
| 122 | 122 | pr->id, ret); |
| @@ -125,26 +125,36 @@ static int cpufreq_set_cur_state(unsigned int cpu, int state) · 2 unmodified lines | ||
| 125 | 125 | return 0; |
| 126 | 126 | } |
| 127 | 127 | |
| 128 | −void acpi_thermal_cpufreq_init(int cpu) | |
| 128 | +void acpi_thermal_cpufreq_init(struct cpufreq_policy *policy) | |
| 129 | 129 | { |
| 130 | − struct acpi_processor *pr = per_cpu(processors, cpu); | |
| 131 | − int ret; | |
| 132 | − | |
| 133 | − ret = dev_pm_qos_add_request(get_cpu_device(cpu), | |
| 134 | − &pr->thermal_req, DEV_PM_QOS_MAX_FREQUENCY, | |
| 135 | − INT_MAX); | |
| 136 | − if (ret < 0) { | |
| 137 | − pr_err("Failed to add freq constraint for CPU%d (%d)\n", cpu, | |
| 138 | − ret); | |
| 139 | − return; | |
| 130 | + unsigned int cpu; | |
| 131 | + | |
| 132 | + for_each_cpu(cpu, policy->related_cpus) { | |
| 133 | + struct acpi_processor *pr = per_cpu(processors, cpu); | |
| 134 | + int ret; | |
| 135 | + | |
| 136 | + if (!pr) | |
| 137 | + continue; | |
| 138 | + | |
| 139 | + ret = freq_qos_add_request(&policy->constraints, | |
| 140 | + &pr->thermal_req, | |
| 141 | + FREQ_QOS_MAX, INT_MAX); | |
| 142 | + if (ret < 0) | |
| 143 | + pr_err("Failed to add freq constraint for CPU%d (%d)\n", | |
| 144 | + cpu, ret); | |
| 140 | 145 | } |
| 141 | 146 | } |
| 142 | 147 | |
| 143 | −void acpi_thermal_cpufreq_exit(int cpu) | |
| 148 | +void acpi_thermal_cpufreq_exit(struct cpufreq_policy *policy) | |
| 144 | 149 | { |
| 145 | − struct acpi_processor *pr = per_cpu(processors, cpu); | |
| 150 | + unsigned int cpu; | |
| 151 | + | |
| 152 | + for_each_cpu(cpu, policy->related_cpus) { | |
| 153 | + struct acpi_processor *pr = per_cpu(processors, policy->cpu); | |
| 146 | 154 | |
| 147 | − dev_pm_qos_remove_request(&pr->thermal_req); | |
| 155 | + if (pr) | |
| 156 | + freq_qos_remove_request(&pr->thermal_req); | |
| 157 | + } | |
| 148 | 158 | } |
| 149 | 159 | #else /* ! CONFIG_CPU_FREQ */ |
| 150 | 160 | static int cpufreq_get_max_state(unsigned int cpu) |
| @@ -361,19 +361,6 @@ static const struct dmi_system_id acpisleep_dmi_table[] __initconst = { · 360 unmodified lines | ||
| 361 | 361 | DMI_MATCH(DMI_PRODUCT_NAME, "80E3"), |
| 362 | 362 | }, |
| 363 | 363 | }, |
| 364 | − /* | |
| 365 | − * https://bugzilla.kernel.org/show_bug.cgi?id=196907 | |
| 366 | − * Some Dell XPS13 9360 cannot do suspend-to-idle using the Low Power | |
| 367 | − * S0 Idle firmware interface. | |
| 368 | − */ | |
| 369 | − { | |
| 370 | − .callback = init_default_s3, | |
| 371 | − .ident = "Dell XPS13 9360", | |
| 372 | − .matches = { | |
| 373 | − DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."), | |
| 374 | − DMI_MATCH(DMI_PRODUCT_NAME, "XPS 13 9360"), | |
| 375 | − }, | |
| 376 | − }, | |
| 377 | 364 | /* |
| 378 | 365 | * ThinkPad X1 Tablet(2016) cannot do suspend-to-idle using |
| 379 | 366 | * the Low Power S0 Idle firmware interface (see |
| @@ -409,9 +409,11 @@ static int amba_device_try_add(struct amba_device *dev, struct resource *parent) · 408 unmodified lines | ||
| 409 | 409 | */ |
| 410 | 410 | rstc = of_reset_control_array_get_optional_shared(dev->dev.of_node); |
| 411 | 411 | if (IS_ERR(rstc)) { |
| 412 | − if (PTR_ERR(rstc) != -EPROBE_DEFER) | |
| 413 | − dev_err(&dev->dev, "Can't get amba reset!\n"); | |
| 414 | − return PTR_ERR(rstc); | |
| 412 | + ret = PTR_ERR(rstc); | |
| 413 | + if (ret != -EPROBE_DEFER) | |
| 414 | + dev_err(&dev->dev, "can't get reset: %d\n", | |
| 415 | + ret); | |
| 416 | + goto err_reset; | |
| 415 | 417 | } |
| 416 | 418 | reset_control_deassert(rstc); |
| 417 | 419 | reset_control_put(rstc); |
| @@ -472,6 +474,12 @@ static int amba_device_try_add(struct amba_device *dev, struct resource *parent) · 54 unmodified lines | ||
| 472 | 474 | release_resource(&dev->res); |
| 473 | 475 | err_out: |
| 474 | 476 | return ret; |
| 477 | + | |
| 478 | + err_reset: | |
| 479 | + amba_put_disable_pclk(dev); | |
| 480 | + iounmap(tmp); | |
| 481 | + dev_pm_domain_detach(&dev->dev, true); | |
| 482 | + goto err_release; | |
| 475 | 483 | } |
| 476 | 484 | |
| 477 | 485 | /* |
| @@ -57,6 +57,7 @@ · 56 unmodified lines | ||
| 57 | 57 | #include <linux/sched/signal.h> |
| 58 | 58 | #include <linux/sched/mm.h> |
| 59 | 59 | #include <linux/seq_file.h> |
| 60 | +#include <linux/string.h> | |
| 60 | 61 | #include <linux/uaccess.h> |
| 61 | 62 | #include <linux/pid_namespace.h> |
| 62 | 63 | #include <linux/security.h> |
| @@ -66,6 +67,7 @@ · 3 unmodified lines | ||
| 66 | 67 | #include <linux/task_work.h> |
| 67 | 68 | |
| 68 | 69 | #include <uapi/linux/android/binder.h> |
| 70 | +#include <uapi/linux/android/binderfs.h> | |
| 69 | 71 | |
| 70 | 72 | #include <asm/cacheflush.h> |
| 71 | 73 | |
| @@ -95,10 +97,6 @@ DEFINE_SHOW_ATTRIBUTE(proc); · 23 unmodified lines | ||
| 95 | 97 | #define SZ_1K 0x400 |
| 96 | 98 | #endif |
| 97 | 99 | |
| 98 | −#ifndef SZ_4M | |
| 99 | −#define SZ_4M 0x400000 | |
| 100 | −#endif | |
| 101 | − | |
| 102 | 100 | #define FORBIDDEN_MMAP_FLAGS (VM_WRITE) |
| 103 | 101 | |
| 104 | 102 | enum { |
| @@ -2876,7 +2874,7 @@ static void binder_transaction(struct binder_proc *proc, · 2771 unmodified lines | ||
| 2876 | 2874 | e->target_handle = tr->target.handle; |
| 2877 | 2875 | e->data_size = tr->data_size; |
| 2878 | 2876 | e->offsets_size = tr->offsets_size; |
| 2879 | − e->context_name = proc->context->name; | |
| 2877 | + strscpy(e->context_name, proc->context->name, BINDERFS_MAX_NAME); | |
| 2880 | 2878 | |
| 2881 | 2879 | if (reply) { |
| 2882 | 2880 | binder_inner_proc_lock(proc); |
| @@ -5175,9 +5173,6 @@ static int binder_mmap(struct file *filp, struct vm_area_struct *vma) · 2292 unmodified lines | ||
| 5175 | 5173 | if (proc->tsk != current->group_leader) |
| 5176 | 5174 | return -EINVAL; |
| 5177 | 5175 | |
| 5178 | − if ((vma->vm_end - vma->vm_start) > SZ_4M) | |
| 5179 | − vma->vm_end = vma->vm_start + SZ_4M; | |
| 5180 | − | |
| 5181 | 5176 | binder_debug(BINDER_DEBUG_OPEN_CLOSE, |
| 5182 | 5177 | "%s: %d %lx-%lx (%ld K) vma %lx pagep %lx\n", |
| 5183 | 5178 | __func__, proc->pid, vma->vm_start, vma->vm_end, |
| @@ -22,6 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | #include <asm/cacheflush.h> |
| 23 | 23 | #include <linux/uaccess.h> |
| 24 | 24 | #include <linux/highmem.h> |
| 25 | +#include <linux/sizes.h> | |
| 25 | 26 | #include "binder_alloc.h" |
| 26 | 27 | #include "binder_trace.h" |
| 27 | 28 | |
| @@ -156,7 +157,7 @@ static struct binder_buffer *binder_alloc_prepare_to_free_locked( · 128 unmodified lines | ||
| 156 | 157 | } |
| 157 | 158 | |
| 158 | 159 | /** |
| 159 | − * binder_alloc_buffer_lookup() - get buffer given user ptr | |
| 160 | + * binder_alloc_prepare_to_free() - get buffer given user ptr | |
| 160 | 161 | * @alloc: binder_alloc for this proc |
| 161 | 162 | * @user_ptr: User pointer to buffer data |
| 162 | 163 | * |
| @@ -689,7 +690,9 @@ int binder_alloc_mmap_handler(struct binder_alloc *alloc, · 526 unmodified lines | ||
| 689 | 690 | alloc->buffer = (void __user *)vma->vm_start; |
| 690 | 691 | mutex_unlock(&binder_alloc_mmap_lock); |
| 691 | 692 | |
| 692 | − alloc->pages = kcalloc((vma->vm_end - vma->vm_start) / PAGE_SIZE, | |
| 693 | + alloc->buffer_size = min_t(unsigned long, vma->vm_end - vma->vm_start, | |
| 694 | + SZ_4M); | |
| 695 | + alloc->pages = kcalloc(alloc->buffer_size / PAGE_SIZE, | |
| 693 | 696 | sizeof(alloc->pages[0]), |
| 694 | 697 | GFP_KERNEL); |
| 695 | 698 | if (alloc->pages == NULL) { |
| @@ -697,7 +700,6 @@ int binder_alloc_mmap_handler(struct binder_alloc *alloc, · 1 unmodified line | ||
| 697 | 700 | failure_string = "alloc page array"; |
| 698 | 701 | goto err_alloc_pages_failed; |
| 699 | 702 | } |
| 700 | − alloc->buffer_size = vma->vm_end - vma->vm_start; | |
| 701 | 703 | |
| 702 | 704 | buffer = kzalloc(sizeof(*buffer), GFP_KERNEL); |
| 703 | 705 | if (!buffer) { |
| @@ -130,7 +130,7 @@ struct binder_transaction_log_entry { · 129 unmodified lines | ||
| 130 | 130 | int return_error_line; |
| 131 | 131 | uint32_t return_error; |
| 132 | 132 | uint32_t return_error_param; |
| 133 | − const char *context_name; | |
| 133 | + char context_name[BINDERFS_MAX_NAME + 1]; | |
| 134 | 134 | }; |
| 135 | 135 | |
| 136 | 136 | struct binder_transaction_log { |
| @@ -1600,7 +1600,9 @@ static void ahci_intel_pcs_quirk(struct pci_dev *pdev, struct ahci_host_priv *hp · 1599 unmodified lines | ||
| 1600 | 1600 | */ |
| 1601 | 1601 | if (!id || id->vendor != PCI_VENDOR_ID_INTEL) |
| 1602 | 1602 | return; |
| 1603 | − if (((enum board_ids) id->driver_data) < board_ahci_pcs7) | |
| 1603 | + | |
| 1604 | + /* Skip applying the quirk on Denverton and beyond */ | |
| 1605 | + if (((enum board_ids) id->driver_data) >= board_ahci_pcs7) | |
| 1604 | 1606 | return; |
| 1605 | 1607 | |
| 1606 | 1608 | /* |
| @@ -153,17 +153,13 @@ int ahci_platform_enable_regulators(struct ahci_host_priv *hpriv) · 152 unmodified lines | ||
| 153 | 153 | { |
| 154 | 154 | int rc, i; |
| 155 | 155 | |
| 156 | − if (hpriv->ahci_regulator) { | |
| 157 | − rc = regulator_enable(hpriv->ahci_regulator); | |
| 158 | − if (rc) | |
| 159 | − return rc; | |
| 160 | − } | |
| 156 | + rc = regulator_enable(hpriv->ahci_regulator); | |
| 157 | + if (rc) | |
| 158 | + return rc; | |
| 161 | 159 | |
| 162 | − if (hpriv->phy_regulator) { | |
| 163 | − rc = regulator_enable(hpriv->phy_regulator); | |
| 164 | − if (rc) | |
| 165 | − goto disable_ahci_pwrs; | |
| 166 | − } | |
| 160 | + rc = regulator_enable(hpriv->phy_regulator); | |
| 161 | + if (rc) | |
| 162 | + goto disable_ahci_pwrs; | |
| 167 | 163 | |
| 168 | 164 | for (i = 0; i < hpriv->nports; i++) { |
| 169 | 165 | if (!hpriv->target_pwrs[i]) |
| @@ -181,11 +177,9 @@ disable_target_pwrs: · 11 unmodified lines | ||
| 181 | 177 | if (hpriv->target_pwrs[i]) |
| 182 | 178 | regulator_disable(hpriv->target_pwrs[i]); |
| 183 | 179 | |
| 184 | − if (hpriv->phy_regulator) | |
| 185 | − regulator_disable(hpriv->phy_regulator); | |
| 180 | + regulator_disable(hpriv->phy_regulator); | |
| 186 | 181 | disable_ahci_pwrs: |
| 187 | − if (hpriv->ahci_regulator) | |
| 188 | − regulator_disable(hpriv->ahci_regulator); | |
| 182 | + regulator_disable(hpriv->ahci_regulator); | |
| 189 | 183 | return rc; |
| 190 | 184 | } |
| 191 | 185 | EXPORT_SYMBOL_GPL(ahci_platform_enable_regulators); |
| @@ -207,10 +201,8 @@ void ahci_platform_disable_regulators(struct ahci_host_priv *hpriv) · 15 unmodified lines | ||
| 207 | 201 | regulator_disable(hpriv->target_pwrs[i]); |
| 208 | 202 | } |
| 209 | 203 | |
| 210 | − if (hpriv->ahci_regulator) | |
| 211 | − regulator_disable(hpriv->ahci_regulator); | |
| 212 | − if (hpriv->phy_regulator) | |
| 213 | − regulator_disable(hpriv->phy_regulator); | |
| 204 | + regulator_disable(hpriv->ahci_regulator); | |
| 205 | + regulator_disable(hpriv->phy_regulator); | |
| 214 | 206 | } |
| 215 | 207 | EXPORT_SYMBOL_GPL(ahci_platform_disable_regulators); |
| 216 | 208 | /** |
| @@ -359,7 +351,7 @@ static int ahci_platform_get_regulator(struct ahci_host_priv *hpriv, u32 port, · 142 unmodified lines | ||
| 359 | 351 | struct regulator *target_pwr; |
| 360 | 352 | int rc = 0; |
| 361 | 353 | |
| 362 | − target_pwr = regulator_get_optional(dev, "target"); | |
| 354 | + target_pwr = regulator_get(dev, "target"); | |
| 363 | 355 | |
| 364 | 356 | if (!IS_ERR(target_pwr)) |
| 365 | 357 | hpriv->target_pwrs[port] = target_pwr; |
| @@ -436,16 +428,14 @@ struct ahci_host_priv *ahci_platform_get_resources(struct platform_device *pdev, · 70 unmodified lines | ||
| 436 | 428 | hpriv->clks[i] = clk; |
| 437 | 429 | } |
| 438 | 430 | |
| 439 | − hpriv->ahci_regulator = devm_regulator_get_optional(dev, "ahci"); | |
| 431 | + hpriv->ahci_regulator = devm_regulator_get(dev, "ahci"); | |
| 440 | 432 | if (IS_ERR(hpriv->ahci_regulator)) { |
| 441 | 433 | rc = PTR_ERR(hpriv->ahci_regulator); |
| 442 | − if (rc == -EPROBE_DEFER) | |
| 434 | + if (rc != 0) | |
| 443 | 435 | goto err_out; |
| 444 | − rc = 0; | |
| 445 | − hpriv->ahci_regulator = NULL; | |
| 446 | 436 | } |
| 447 | 437 | |
| 448 | − hpriv->phy_regulator = devm_regulator_get_optional(dev, "phy"); | |
| 438 | + hpriv->phy_regulator = devm_regulator_get(dev, "phy"); | |
| 449 | 439 | if (IS_ERR(hpriv->phy_regulator)) { |
| 450 | 440 | rc = PTR_ERR(hpriv->phy_regulator); |
| 451 | 441 | if (rc == -EPROBE_DEFER) |
| @@ -4791,27 +4791,6 @@ void ata_scsi_hotplug(struct work_struct *work) · 4790 unmodified lines | ||
| 4791 | 4791 | return; |
| 4792 | 4792 | } |
| 4793 | 4793 | |
| 4794 | − /* | |
| 4795 | − * XXX - UGLY HACK | |
| 4796 | − * | |
| 4797 | − * The block layer suspend/resume path is fundamentally broken due | |
| 4798 | − * to freezable kthreads and workqueue and may deadlock if a block | |
| 4799 | − * device gets removed while resume is in progress. I don't know | |
| 4800 | − * what the solution is short of removing freezable kthreads and | |
| 4801 | − * workqueues altogether. | |
| 4802 | − * | |
| 4803 | − * The following is an ugly hack to avoid kicking off device | |
| 4804 | − * removal while freezer is active. This is a joke but does avoid | |
| 4805 | − * this particular deadlock scenario. | |
| 4806 | − * | |
| 4807 | − * https://bugzilla.kernel.org/show_bug.cgi?id=62801 | |
| 4808 | − * http://marc.info/?l=linux-kernel&m=138695698516487 | |
| 4809 | − */ | |
| 4810 | −#ifdef CONFIG_FREEZER | |
| 4811 | − while (pm_freezing) | |
| 4812 | − msleep(10); | |
| 4813 | −#endif | |
| 4814 | − | |
| 4815 | 4794 | DPRINTK("ENTER\n"); |
| 4816 | 4795 | mutex_lock(&ap->scsi_scan_mutex); |
| 4817 | 4796 | |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | */ |
| 10 | 10 | |
| 11 | 11 | #include <linux/acpi.h> |
| 12 | +#include <linux/cpufreq.h> | |
| 12 | 13 | #include <linux/device.h> |
| 13 | 14 | #include <linux/err.h> |
| 14 | 15 | #include <linux/fwnode.h> |
| @@ -3179,6 +3180,8 @@ void device_shutdown(void) · 3164 unmodified lines | ||
| 3179 | 3180 | wait_for_device_probe(); |
| 3180 | 3181 | device_block_probing(); |
| 3181 | 3182 | |
| 3183 | + cpufreq_suspend(); | |
| 3184 | + | |
| 3182 | 3185 | spin_lock(&devices_kset->list_lock); |
| 3183 | 3186 | /* |
| 3184 | 3187 | * Walk the devices list backward, shutting down each in turn. |
| @@ -540,6 +540,9 @@ static ssize_t soft_offline_page_store(struct device *dev, · 539 unmodified lines | ||
| 540 | 540 | pfn >>= PAGE_SHIFT; |
| 541 | 541 | if (!pfn_valid(pfn)) |
| 542 | 542 | return -ENXIO; |
| 543 | + /* Only online pages can be soft-offlined (esp., not ZONE_DEVICE). */ | |
| 544 | + if (!pfn_to_online_page(pfn)) | |
| 545 | + return -EIO; | |
| 543 | 546 | ret = soft_offline_page(pfn_to_page(pfn), 0); |
| 544 | 547 | return ret == 0 ? count : ret; |
| 545 | 548 | } |
| @@ -241,12 +241,8 @@ struct resource *platform_get_resource_byname(struct platform_device *dev, · 240 unmodified lines | ||
| 241 | 241 | } |
| 242 | 242 | EXPORT_SYMBOL_GPL(platform_get_resource_byname); |
| 243 | 243 | |
| 244 | −/** | |
| 245 | − * platform_get_irq_byname - get an IRQ for a device by name | |
| 246 | − * @dev: platform device | |
| 247 | − * @name: IRQ name | |
| 248 | − */ | |
| 249 | −int platform_get_irq_byname(struct platform_device *dev, const char *name) | |
| 244 | +static int __platform_get_irq_byname(struct platform_device *dev, | |
| 245 | + const char *name) | |
| 250 | 246 | { |
| 251 | 247 | struct resource *r; |
| 252 | 248 | |
| @@ -262,11 +258,47 @@ int platform_get_irq_byname(struct platform_device *dev, const char *name) · 9 unmodified lines | ||
| 262 | 258 | if (r) |
| 263 | 259 | return r->start; |
| 264 | 260 | |
| 265 | − dev_err(&dev->dev, "IRQ %s not found\n", name); | |
| 266 | 261 | return -ENXIO; |
| 267 | 262 | } |
| 263 | + | |
| 264 | +/** | |
| 265 | + * platform_get_irq_byname - get an IRQ for a device by name | |
| 266 | + * @dev: platform device | |
| 267 | + * @name: IRQ name | |
| 268 | + * | |
| 269 | + * Get an IRQ like platform_get_irq(), but then by name rather then by index. | |
| 270 | + * | |
| 271 | + * Return: IRQ number on success, negative error number on failure. | |
| 272 | + */ | |
| 273 | +int platform_get_irq_byname(struct platform_device *dev, const char *name) | |
| 274 | +{ | |
| 275 | + int ret; | |
| 276 | + | |
| 277 | + ret = __platform_get_irq_byname(dev, name); | |
| 278 | + if (ret < 0 && ret != -EPROBE_DEFER) | |
| 279 | + dev_err(&dev->dev, "IRQ %s not found\n", name); | |
| 280 | + | |
| 281 | + return ret; | |
| 282 | +} | |
| 268 | 283 | EXPORT_SYMBOL_GPL(platform_get_irq_byname); |
| 269 | 284 | |
| 285 | +/** | |
| 286 | + * platform_get_irq_byname_optional - get an optional IRQ for a device by name | |
| 287 | + * @dev: platform device | |
| 288 | + * @name: IRQ name | |
| 289 | + * | |
| 290 | + * Get an optional IRQ by name like platform_get_irq_byname(). Except that it | |
| 291 | + * does not print an error message if an IRQ can not be obtained. | |
| 292 | + * | |
| 293 | + * Return: IRQ number on success, negative error number on failure. | |
| 294 | + */ | |
| 295 | +int platform_get_irq_byname_optional(struct platform_device *dev, | |
| 296 | + const char *name) | |
| 297 | +{ | |
| 298 | + return __platform_get_irq_byname(dev, name); | |
| 299 | +} | |
| 300 | +EXPORT_SYMBOL_GPL(platform_get_irq_byname_optional); | |
| 301 | + | |
| 270 | 302 | /** |
| 271 | 303 | * platform_add_devices - add a numbers of platform devices |
| 272 | 304 | * @devs: array of platform devices to add |
| @@ -115,20 +115,10 @@ s32 dev_pm_qos_read_value(struct device *dev, enum dev_pm_qos_req_type type) · 114 unmodified lines | ||
| 115 | 115 | |
| 116 | 116 | spin_lock_irqsave(&dev->power.lock, flags); |
| 117 | 117 | |
| 118 | − switch (type) { | |
| 119 | − case DEV_PM_QOS_RESUME_LATENCY: | |
| 118 | + if (type == DEV_PM_QOS_RESUME_LATENCY) { | |
| 120 | 119 | ret = IS_ERR_OR_NULL(qos) ? PM_QOS_RESUME_LATENCY_NO_CONSTRAINT |
| 121 | 120 | : pm_qos_read_value(&qos->resume_latency); |
| 122 | − break; | |
| 123 | − case DEV_PM_QOS_MIN_FREQUENCY: | |
| 124 | − ret = IS_ERR_OR_NULL(qos) ? PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE | |
| 125 | − : pm_qos_read_value(&qos->min_frequency); | |
| 126 | − break; | |
| 127 | − case DEV_PM_QOS_MAX_FREQUENCY: | |
| 128 | − ret = IS_ERR_OR_NULL(qos) ? PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE | |
| 129 | − : pm_qos_read_value(&qos->max_frequency); | |
| 130 | − break; | |
| 131 | − default: | |
| 121 | + } else { | |
| 132 | 122 | WARN_ON(1); |
| 133 | 123 | ret = 0; |
| 134 | 124 | } |
| @@ -169,14 +159,6 @@ static int apply_constraint(struct dev_pm_qos_request *req, · 34 unmodified lines | ||
| 169 | 159 | req->dev->power.set_latency_tolerance(req->dev, value); |
| 170 | 160 | } |
| 171 | 161 | break; |
| 172 | − case DEV_PM_QOS_MIN_FREQUENCY: | |
| 173 | − ret = pm_qos_update_target(&qos->min_frequency, | |
| 174 | − &req->data.pnode, action, value); | |
| 175 | − break; | |
| 176 | − case DEV_PM_QOS_MAX_FREQUENCY: | |
| 177 | − ret = pm_qos_update_target(&qos->max_frequency, | |
| 178 | − &req->data.pnode, action, value); | |
| 179 | − break; | |
| 180 | 162 | case DEV_PM_QOS_FLAGS: |
| 181 | 163 | ret = pm_qos_update_flags(&qos->flags, &req->data.flr, |
| 182 | 164 | action, value); |
| @@ -227,24 +209,6 @@ static int dev_pm_qos_constraints_allocate(struct device *dev) · 44 unmodified lines | ||
| 227 | 209 | c->no_constraint_value = PM_QOS_LATENCY_TOLERANCE_NO_CONSTRAINT; |
| 228 | 210 | c->type = PM_QOS_MIN; |
| 229 | 211 | |
| 230 | − c = &qos->min_frequency; | |
| 231 | − plist_head_init(&c->list); | |
| 232 | − c->target_value = PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE; | |
| 233 | − c->default_value = PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE; | |
| 234 | − c->no_constraint_value = PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE; | |
| 235 | − c->type = PM_QOS_MAX; | |
| 236 | − c->notifiers = ++n; | |
| 237 | − BLOCKING_INIT_NOTIFIER_HEAD(n); | |
| 238 | − | |
| 239 | − c = &qos->max_frequency; | |
| 240 | − plist_head_init(&c->list); | |
| 241 | − c->target_value = PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE; | |
| 242 | − c->default_value = PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE; | |
| 243 | − c->no_constraint_value = PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE; | |
| 244 | − c->type = PM_QOS_MIN; | |
| 245 | − c->notifiers = ++n; | |
| 246 | − BLOCKING_INIT_NOTIFIER_HEAD(n); | |
| 247 | − | |
| 248 | 212 | INIT_LIST_HEAD(&qos->flags.list); |
| 249 | 213 | |
| 250 | 214 | spin_lock_irq(&dev->power.lock); |
| @@ -305,18 +269,6 @@ void dev_pm_qos_constraints_destroy(struct device *dev) · 54 unmodified lines | ||
| 305 | 269 | memset(req, 0, sizeof(*req)); |
| 306 | 270 | } |
| 307 | 271 | |
| 308 | − c = &qos->min_frequency; | |
| 309 | − plist_for_each_entry_safe(req, tmp, &c->list, data.pnode) { | |
| 310 | − apply_constraint(req, PM_QOS_REMOVE_REQ, PM_QOS_MIN_FREQUENCY_DEFAULT_VALUE); | |
| 311 | − memset(req, 0, sizeof(*req)); | |
| 312 | − } | |
| 313 | − | |
| 314 | − c = &qos->max_frequency; | |
| 315 | − plist_for_each_entry_safe(req, tmp, &c->list, data.pnode) { | |
| 316 | − apply_constraint(req, PM_QOS_REMOVE_REQ, PM_QOS_MAX_FREQUENCY_DEFAULT_VALUE); | |
| 317 | − memset(req, 0, sizeof(*req)); | |
| 318 | − } | |
| 319 | − | |
| 320 | 272 | f = &qos->flags; |
| 321 | 273 | list_for_each_entry_safe(req, tmp, &f->list, data.flr.node) { |
| 322 | 274 | apply_constraint(req, PM_QOS_REMOVE_REQ, PM_QOS_DEFAULT_VALUE); |
| @@ -428,8 +380,6 @@ static int __dev_pm_qos_update_request(struct dev_pm_qos_request *req, · 105 unmodified lines | ||
| 428 | 380 | switch(req->type) { |
| 429 | 381 | case DEV_PM_QOS_RESUME_LATENCY: |
| 430 | 382 | case DEV_PM_QOS_LATENCY_TOLERANCE: |
| 431 | − case DEV_PM_QOS_MIN_FREQUENCY: | |
| 432 | − case DEV_PM_QOS_MAX_FREQUENCY: | |
| 433 | 383 | curr_value = req->data.pnode.prio; |
| 434 | 384 | break; |
| 435 | 385 | case DEV_PM_QOS_FLAGS: |
| @@ -557,14 +507,6 @@ int dev_pm_qos_add_notifier(struct device *dev, struct notifier_block *notifier, · 121 unmodified lines | ||
| 557 | 507 | ret = blocking_notifier_chain_register(dev->power.qos->resume_latency.notifiers, |
| 558 | 508 | notifier); |
| 559 | 509 | break; |
| 560 | − case DEV_PM_QOS_MIN_FREQUENCY: | |
| 561 | − ret = blocking_notifier_chain_register(dev->power.qos->min_frequency.notifiers, | |
| 562 | − notifier); | |
| 563 | − break; | |
| 564 | − case DEV_PM_QOS_MAX_FREQUENCY: | |
| 565 | − ret = blocking_notifier_chain_register(dev->power.qos->max_frequency.notifiers, | |
| 566 | − notifier); | |
| 567 | − break; | |
| 568 | 510 | default: |
| 569 | 511 | WARN_ON(1); |
| 570 | 512 | ret = -EINVAL; |
| @@ -604,14 +546,6 @@ int dev_pm_qos_remove_notifier(struct device *dev, · 33 unmodified lines | ||
| 604 | 546 | ret = blocking_notifier_chain_unregister(dev->power.qos->resume_latency.notifiers, |
| 605 | 547 | notifier); |
| 606 | 548 | break; |
| 607 | − case DEV_PM_QOS_MIN_FREQUENCY: | |
| 608 | − ret = blocking_notifier_chain_unregister(dev->power.qos->min_frequency.notifiers, | |
| 609 | − notifier); | |
| 610 | − break; | |
| 611 | − case DEV_PM_QOS_MAX_FREQUENCY: | |
| 612 | − ret = blocking_notifier_chain_unregister(dev->power.qos->max_frequency.notifiers, | |
| 613 | − notifier); | |
| 614 | − break; | |
| 615 | 549 | default: |
| 616 | 550 | WARN_ON(1); |
| 617 | 551 | ret = -EINVAL; |
| @@ -786,7 +786,6 @@ int __drbd_send_protocol(struct drbd_connection *connection, enum drbd_packet cm · 785 unmodified lines | ||
| 786 | 786 | |
| 787 | 787 | if (nc->tentative && connection->agreed_pro_version < 92) { |
| 788 | 788 | rcu_read_unlock(); |
| 789 | − mutex_unlock(&sock->mutex); | |
| 790 | 789 | drbd_err(connection, "--dry-run is not supported by peer"); |
| 791 | 790 | return -EOPNOTSUPP; |
| 792 | 791 | } |
| @@ -248,8 +248,8 @@ static void nbd_put(struct nbd_device *nbd) · 247 unmodified lines | ||
| 248 | 248 | if (refcount_dec_and_mutex_lock(&nbd->refs, |
| 249 | 249 | &nbd_index_mutex)) { |
| 250 | 250 | idr_remove(&nbd_index_idr, nbd->index); |
| 251 | − mutex_unlock(&nbd_index_mutex); | |
| 252 | 251 | nbd_dev_remove(nbd); |
| 252 | + mutex_unlock(&nbd_index_mutex); | |
| 253 | 253 | } |
| 254 | 254 | } |
| 255 | 255 | |
| @@ -385,17 +385,16 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req, · 129 unmodified lines | ||
| 385 | 385 | struct nbd_device *nbd = cmd->nbd; |
| 386 | 386 | struct nbd_config *config; |
| 387 | 387 | |
| 388 | + if (!mutex_trylock(&cmd->lock)) | |
| 389 | + return BLK_EH_RESET_TIMER; | |
| 390 | + | |
| 388 | 391 | if (!refcount_inc_not_zero(&nbd->config_refs)) { |
| 389 | 392 | cmd->status = BLK_STS_TIMEOUT; |
| 393 | + mutex_unlock(&cmd->lock); | |
| 390 | 394 | goto done; |
| 391 | 395 | } |
| 392 | 396 | config = nbd->config; |
| 393 | 397 | |
| 394 | − if (!mutex_trylock(&cmd->lock)) { | |
| 395 | − nbd_config_put(nbd); | |
| 396 | − return BLK_EH_RESET_TIMER; | |
| 397 | − } | |
| 398 | − | |
| 399 | 398 | if (config->num_connections > 1) { |
| 400 | 399 | dev_err_ratelimited(nbd_to_dev(nbd), |
| 401 | 400 | "Connection timed out, retrying (%d/%d alive)\n", |
| @@ -711,6 +710,12 @@ static struct nbd_cmd *nbd_read_stat(struct nbd_device *nbd, int index) · 309 unmodified lines | ||
| 711 | 710 | ret = -ENOENT; |
| 712 | 711 | goto out; |
| 713 | 712 | } |
| 713 | + if (cmd->status != BLK_STS_OK) { | |
| 714 | + dev_err(disk_to_dev(nbd->disk), "Command already handled %p\n", | |
| 715 | + req); | |
| 716 | + ret = -ENOENT; | |
| 717 | + goto out; | |
| 718 | + } | |
| 714 | 719 | if (test_bit(NBD_CMD_REQUEUED, &cmd->flags)) { |
| 715 | 720 | dev_err(disk_to_dev(nbd->disk), "Raced with timeout on req %p\n", |
| 716 | 721 | req); |
| @@ -792,7 +797,10 @@ static bool nbd_clear_req(struct request *req, void *data, bool reserved) · 75 unmodified lines | ||
| 792 | 797 | { |
| 793 | 798 | struct nbd_cmd *cmd = blk_mq_rq_to_pdu(req); |
| 794 | 799 | |
| 800 | + mutex_lock(&cmd->lock); | |
| 795 | 801 | cmd->status = BLK_STS_IOERR; |
| 802 | + mutex_unlock(&cmd->lock); | |
| 803 | + | |
| 796 | 804 | blk_mq_complete_request(req); |
| 797 | 805 | return true; |
| 798 | 806 | } |
| @@ -972,6 +980,25 @@ static blk_status_t nbd_queue_rq(struct blk_mq_hw_ctx *hctx, · 173 unmodified lines | ||
| 972 | 980 | return ret; |
| 973 | 981 | } |
| 974 | 982 | |
| 983 | +static struct socket *nbd_get_socket(struct nbd_device *nbd, unsigned long fd, | |
| 984 | + int *err) | |
| 985 | +{ | |
| 986 | + struct socket *sock; | |
| 987 | + | |
| 988 | + *err = 0; | |
| 989 | + sock = sockfd_lookup(fd, err); | |
| 990 | + if (!sock) | |
| 991 | + return NULL; | |
| 992 | + | |
| 993 | + if (sock->ops->shutdown == sock_no_shutdown) { | |
| 994 | + dev_err(disk_to_dev(nbd->disk), "Unsupported socket: shutdown callout must be supported.\n"); | |
| 995 | + *err = -EINVAL; | |
| 996 | + return NULL; | |
| 997 | + } | |
| 998 | + | |
| 999 | + return sock; | |
| 1000 | +} | |
| 1001 | + | |
| 975 | 1002 | static int nbd_add_socket(struct nbd_device *nbd, unsigned long arg, |
| 976 | 1003 | bool netlink) |
| 977 | 1004 | { |
| @@ -981,7 +1008,7 @@ static int nbd_add_socket(struct nbd_device *nbd, unsigned long arg, · 3 unmodified lines | ||
| 981 | 1008 | struct nbd_sock *nsock; |
| 982 | 1009 | int err; |
| 983 | 1010 | |
| 984 | − sock = sockfd_lookup(arg, &err); | |
| 1011 | + sock = nbd_get_socket(nbd, arg, &err); | |
| 985 | 1012 | if (!sock) |
| 986 | 1013 | return err; |
| 987 | 1014 | |
| @@ -1033,7 +1060,7 @@ static int nbd_reconnect_socket(struct nbd_device *nbd, unsigned long arg) · 45 unmodified lines | ||
| 1033 | 1060 | int i; |
| 1034 | 1061 | int err; |
| 1035 | 1062 | |
| 1036 | − sock = sockfd_lookup(arg, &err); | |
| 1063 | + sock = nbd_get_socket(nbd, arg, &err); | |
| 1037 | 1064 | if (!sock) |
| 1038 | 1065 | return err; |
| 1039 | 1066 | |
| @@ -142,8 +142,7 @@ static blk_status_t null_zone_reset(struct nullb_cmd *cmd, sector_t sector) · 141 unmodified lines | ||
| 142 | 142 | zone->wp = zone->start; |
| 143 | 143 | break; |
| 144 | 144 | default: |
| 145 | − cmd->error = BLK_STS_NOTSUPP; | |
| 146 | − break; | |
| 145 | + return BLK_STS_NOTSUPP; | |
| 147 | 146 | } |
| 148 | 147 | return BLK_STS_OK; |
| 149 | 148 | } |
| @@ -6639,10 +6639,13 @@ static int rbd_add_acquire_lock(struct rbd_device *rbd_dev) · 6638 unmodified lines | ||
| 6639 | 6639 | queue_delayed_work(rbd_dev->task_wq, &rbd_dev->lock_dwork, 0); |
| 6640 | 6640 | ret = wait_for_completion_killable_timeout(&rbd_dev->acquire_wait, |
| 6641 | 6641 | ceph_timeout_jiffies(rbd_dev->opts->lock_timeout)); |
| 6642 | − if (ret > 0) | |
| 6642 | + if (ret > 0) { | |
| 6643 | 6643 | ret = rbd_dev->acquire_err; |
| 6644 | − else if (!ret) | |
| 6645 | − ret = -ETIMEDOUT; | |
| 6644 | + } else { | |
| 6645 | + cancel_delayed_work_sync(&rbd_dev->lock_dwork); | |
| 6646 | + if (!ret) | |
| 6647 | + ret = -ETIMEDOUT; | |
| 6648 | + } | |
| 6646 | 6649 | |
| 6647 | 6650 | if (ret) { |
| 6648 | 6651 | rbd_warn(rbd_dev, "failed to acquire exclusive lock: %ld", ret); |
| @@ -413,13 +413,14 @@ static void reset_bdev(struct zram *zram) · 412 unmodified lines | ||
| 413 | 413 | static ssize_t backing_dev_show(struct device *dev, |
| 414 | 414 | struct device_attribute *attr, char *buf) |
| 415 | 415 | { |
| 416 | + struct file *file; | |
| 416 | 417 | struct zram *zram = dev_to_zram(dev); |
| 417 | − struct file *file = zram->backing_dev; | |
| 418 | 418 | char *p; |
| 419 | 419 | ssize_t ret; |
| 420 | 420 | |
| 421 | 421 | down_read(&zram->init_lock); |
| 422 | − if (!zram->backing_dev) { | |
| 422 | + file = zram->backing_dev; | |
| 423 | + if (!file) { | |
| 423 | 424 | memcpy(buf, "none\n", 5); |
| 424 | 425 | up_read(&zram->init_lock); |
| 425 | 426 | return 5; |
| @@ -74,6 +74,7 @@ static const char * const clock_names[SYSC_MAX_CLOCKS] = { · 73 unmodified lines | ||
| 74 | 74 | * @clk_disable_quirk: module specific clock disable quirk |
| 75 | 75 | * @reset_done_quirk: module specific reset done quirk |
| 76 | 76 | * @module_enable_quirk: module specific enable quirk |
| 77 | + * @module_disable_quirk: module specific disable quirk | |
| 77 | 78 | */ |
| 78 | 79 | struct sysc { |
| 79 | 80 | struct device *dev; |
| @@ -100,6 +101,7 @@ struct sysc { · 20 unmodified lines | ||
| 100 | 101 | void (*clk_disable_quirk)(struct sysc *sysc); |
| 101 | 102 | void (*reset_done_quirk)(struct sysc *sysc); |
| 102 | 103 | void (*module_enable_quirk)(struct sysc *sysc); |
| 104 | + void (*module_disable_quirk)(struct sysc *sysc); | |
| 103 | 105 | }; |
| 104 | 106 | |
| 105 | 107 | static void sysc_parse_dts_quirks(struct sysc *ddata, struct device_node *np, |
| @@ -959,6 +961,9 @@ static int sysc_disable_module(struct device *dev) · 853 unmodified lines | ||
| 959 | 961 | if (ddata->offsets[SYSC_SYSCONFIG] == -ENODEV) |
| 960 | 962 | return 0; |
| 961 | 963 | |
| 964 | + if (ddata->module_disable_quirk) | |
| 965 | + ddata->module_disable_quirk(ddata); | |
| 966 | + | |
| 962 | 967 | regbits = ddata->cap->regbits; |
| 963 | 968 | reg = sysc_read(ddata, ddata->offsets[SYSC_SYSCONFIG]); |
| 964 | 969 | |
| @@ -1248,6 +1253,9 @@ static const struct sysc_revision_quirk sysc_revision_quirks[] = { · 283 unmodified lines | ||
| 1248 | 1253 | SYSC_MODULE_QUIRK_SGX), |
| 1249 | 1254 | SYSC_QUIRK("wdt", 0, 0, 0x10, 0x14, 0x502a0500, 0xfffff0f0, |
| 1250 | 1255 | SYSC_MODULE_QUIRK_WDT), |
| 1256 | + /* Watchdog on am3 and am4 */ | |
| 1257 | + SYSC_QUIRK("wdt", 0x44e35000, 0, 0x10, 0x14, 0x502a0500, 0xfffff0f0, | |
| 1258 | + SYSC_MODULE_QUIRK_WDT | SYSC_QUIRK_SWSUP_SIDLE), | |
| 1251 | 1259 | |
| 1252 | 1260 | #ifdef DEBUG |
| 1253 | 1261 | SYSC_QUIRK("adc", 0, 0, 0x10, -1, 0x47300001, 0xffffffff, 0), |
| @@ -1440,14 +1448,14 @@ static void sysc_reset_done_quirk_wdt(struct sysc *ddata) · 186 unmodified lines | ||
| 1440 | 1448 | !(val & 0x10), 100, |
| 1441 | 1449 | MAX_MODULE_SOFTRESET_WAIT); |
| 1442 | 1450 | if (error) |
| 1443 | − dev_warn(ddata->dev, "wdt disable spr failed\n"); | |
| 1451 | + dev_warn(ddata->dev, "wdt disable step1 failed\n"); | |
| 1444 | 1452 | |
| 1445 | − sysc_write(ddata, wps, 0x5555); | |
| 1453 | + sysc_write(ddata, spr, 0x5555); | |
| 1446 | 1454 | error = readl_poll_timeout(ddata->module_va + wps, val, |
| 1447 | 1455 | !(val & 0x10), 100, |
| 1448 | 1456 | MAX_MODULE_SOFTRESET_WAIT); |
| 1449 | 1457 | if (error) |
| 1450 | − dev_warn(ddata->dev, "wdt disable wps failed\n"); | |
| 1458 | + dev_warn(ddata->dev, "wdt disable step2 failed\n"); | |
| 1451 | 1459 | } |
| 1452 | 1460 | |
| 1453 | 1461 | static void sysc_init_module_quirks(struct sysc *ddata) |
| @@ -1471,8 +1479,10 @@ static void sysc_init_module_quirks(struct sysc *ddata) · 17 unmodified lines | ||
| 1471 | 1479 | if (ddata->cfg.quirks & SYSC_MODULE_QUIRK_SGX) |
| 1472 | 1480 | ddata->module_enable_quirk = sysc_module_enable_quirk_sgx; |
| 1473 | 1481 | |
| 1474 | − if (ddata->cfg.quirks & SYSC_MODULE_QUIRK_WDT) | |
| 1482 | + if (ddata->cfg.quirks & SYSC_MODULE_QUIRK_WDT) { | |
| 1475 | 1483 | ddata->reset_done_quirk = sysc_reset_done_quirk_wdt; |
| 1484 | + ddata->module_disable_quirk = sysc_reset_done_quirk_wdt; | |
| 1485 | + } | |
| 1476 | 1486 | } |
| 1477 | 1487 | |
| 1478 | 1488 | static int sysc_clockdomain_init(struct sysc *ddata) |
| @@ -297,7 +297,10 @@ static int clk_main_probe_frequency(struct regmap *regmap) · 296 unmodified lines | ||
| 297 | 297 | regmap_read(regmap, AT91_CKGR_MCFR, &mcfr); |
| 298 | 298 | if (mcfr & AT91_PMC_MAINRDY) |
| 299 | 299 | return 0; |
| 300 | − usleep_range(MAINF_LOOP_MIN_WAIT, MAINF_LOOP_MAX_WAIT); | |
| 300 | + if (system_state < SYSTEM_RUNNING) | |
| 301 | + udelay(MAINF_LOOP_MIN_WAIT); | |
| 302 | + else | |
| 303 | + usleep_range(MAINF_LOOP_MIN_WAIT, MAINF_LOOP_MAX_WAIT); | |
| 301 | 304 | } while (time_before(prep_time, timeout)); |
| 302 | 305 | |
| 303 | 306 | return -ETIMEDOUT; |
| @@ -43,6 +43,7 @@ static const struct clk_pll_characteristics upll_characteristics = { · 42 unmodified lines | ||
| 43 | 43 | }; |
| 44 | 44 | |
| 45 | 45 | static const struct clk_programmable_layout sam9x60_programmable_layout = { |
| 46 | + .pres_mask = 0xff, | |
| 46 | 47 | .pres_shift = 8, |
| 47 | 48 | .css_mask = 0x1f, |
| 48 | 49 | .have_slck_mck = 0, |
| @@ -76,7 +76,10 @@ static int clk_slow_osc_prepare(struct clk_hw *hw) · 75 unmodified lines | ||
| 76 | 76 | |
| 77 | 77 | writel(tmp | osc->bits->cr_osc32en, sckcr); |
| 78 | 78 | |
| 79 | − usleep_range(osc->startup_usec, osc->startup_usec + 1); | |
| 79 | + if (system_state < SYSTEM_RUNNING) | |
| 80 | + udelay(osc->startup_usec); | |
| 81 | + else | |
| 82 | + usleep_range(osc->startup_usec, osc->startup_usec + 1); | |
| 80 | 83 | |
| 81 | 84 | return 0; |
| 82 | 85 | } |
| @@ -187,7 +190,10 @@ static int clk_slow_rc_osc_prepare(struct clk_hw *hw) · 104 unmodified lines | ||
| 187 | 190 | |
| 188 | 191 | writel(readl(sckcr) | osc->bits->cr_rcen, sckcr); |
| 189 | 192 | |
| 190 | − usleep_range(osc->startup_usec, osc->startup_usec + 1); | |
| 193 | + if (system_state < SYSTEM_RUNNING) | |
| 194 | + udelay(osc->startup_usec); | |
| 195 | + else | |
| 196 | + usleep_range(osc->startup_usec, osc->startup_usec + 1); | |
| 191 | 197 | |
| 192 | 198 | return 0; |
| 193 | 199 | } |
| @@ -288,7 +294,10 @@ static int clk_sam9x5_slow_set_parent(struct clk_hw *hw, u8 index) · 94 unmodified lines | ||
| 288 | 294 | |
| 289 | 295 | writel(tmp, sckcr); |
| 290 | 296 | |
| 291 | − usleep_range(SLOWCK_SW_TIME_USEC, SLOWCK_SW_TIME_USEC + 1); | |
| 297 | + if (system_state < SYSTEM_RUNNING) | |
| 298 | + udelay(SLOWCK_SW_TIME_USEC); | |
| 299 | + else | |
| 300 | + usleep_range(SLOWCK_SW_TIME_USEC, SLOWCK_SW_TIME_USEC + 1); | |
| 292 | 301 | |
| 293 | 302 | return 0; |
| 294 | 303 | } |
| @@ -533,7 +542,10 @@ static int clk_sama5d4_slow_osc_prepare(struct clk_hw *hw) · 238 unmodified lines | ||
| 533 | 542 | return 0; |
| 534 | 543 | } |
| 535 | 544 | |
| 536 | − usleep_range(osc->startup_usec, osc->startup_usec + 1); | |
| 545 | + if (system_state < SYSTEM_RUNNING) | |
| 546 | + udelay(osc->startup_usec); | |
| 547 | + else | |
| 548 | + usleep_range(osc->startup_usec, osc->startup_usec + 1); | |
| 537 | 549 | osc->prepared = true; |
| 538 | 550 | |
| 539 | 551 | return 0; |
| @@ -266,10 +266,11 @@ static int aspeed_g6_clk_enable(struct clk_hw *hw) · 265 unmodified lines | ||
| 266 | 266 | |
| 267 | 267 | /* Enable clock */ |
| 268 | 268 | if (gate->flags & CLK_GATE_SET_TO_DISABLE) { |
| 269 | − regmap_write(gate->map, get_clock_reg(gate), clk); | |
| 270 | − } else { | |
| 271 | − /* Use set to clear register */ | |
| 269 | + /* Clock is clear to enable, so use set to clear register */ | |
| 272 | 270 | regmap_write(gate->map, get_clock_reg(gate) + 0x04, clk); |
| 271 | + } else { | |
| 272 | + /* Clock is set to enable, so use write to set register */ | |
| 273 | + regmap_write(gate->map, get_clock_reg(gate), clk); | |
| 273 | 274 | } |
| 274 | 275 | |
| 275 | 276 | if (gate->reset_idx >= 0) { |
| @@ -638,7 +638,7 @@ static int imx8mm_clocks_probe(struct platform_device *pdev) · 637 unmodified lines | ||
| 638 | 638 | clks[IMX8MM_CLK_A53_DIV], |
| 639 | 639 | clks[IMX8MM_CLK_A53_SRC], |
| 640 | 640 | clks[IMX8MM_ARM_PLL_OUT], |
| 641 | − clks[IMX8MM_CLK_24M]); | |
| 641 | + clks[IMX8MM_SYS_PLL1_800M]); | |
| 642 | 642 | |
| 643 | 643 | imx_check_clocks(clks, ARRAY_SIZE(clks)); |
| 644 | 644 | |
| @@ -610,7 +610,7 @@ static int imx8mn_clocks_probe(struct platform_device *pdev) · 609 unmodified lines | ||
| 610 | 610 | clks[IMX8MN_CLK_A53_DIV], |
| 611 | 611 | clks[IMX8MN_CLK_A53_SRC], |
| 612 | 612 | clks[IMX8MN_ARM_PLL_OUT], |
| 613 | − clks[IMX8MN_CLK_24M]); | |
| 613 | + clks[IMX8MN_SYS_PLL1_800M]); | |
| 614 | 614 | |
| 615 | 615 | imx_check_clocks(clks, ARRAY_SIZE(clks)); |
| 616 | 616 | |
| @@ -343,6 +343,7 @@ static struct clk_regmap g12a_cpu_clk_premux0 = { · 342 unmodified lines | ||
| 343 | 343 | .offset = HHI_SYS_CPU_CLK_CNTL0, |
| 344 | 344 | .mask = 0x3, |
| 345 | 345 | .shift = 0, |
| 346 | + .flags = CLK_MUX_ROUND_CLOSEST, | |
| 346 | 347 | }, |
| 347 | 348 | .hw.init = &(struct clk_init_data){ |
| 348 | 349 | .name = "cpu_clk_dyn0_sel", |
| @@ -353,8 +354,7 @@ static struct clk_regmap g12a_cpu_clk_premux0 = { · 4 unmodified lines | ||
| 353 | 354 | { .hw = &g12a_fclk_div3.hw }, |
| 354 | 355 | }, |
| 355 | 356 | .num_parents = 3, |
| 356 | − /* This sub-tree is used a parking clock */ | |
| 357 | − .flags = CLK_SET_RATE_NO_REPARENT, | |
| 357 | + .flags = CLK_SET_RATE_PARENT, | |
| 358 | 358 | }, |
| 359 | 359 | }; |
| 360 | 360 | |
| @@ -410,6 +410,7 @@ static struct clk_regmap g12a_cpu_clk_postmux0 = { · 49 unmodified lines | ||
| 410 | 410 | .offset = HHI_SYS_CPU_CLK_CNTL0, |
| 411 | 411 | .mask = 0x1, |
| 412 | 412 | .shift = 2, |
| 413 | + .flags = CLK_MUX_ROUND_CLOSEST, | |
| 413 | 414 | }, |
| 414 | 415 | .hw.init = &(struct clk_init_data){ |
| 415 | 416 | .name = "cpu_clk_dyn0", |
| @@ -466,6 +467,7 @@ static struct clk_regmap g12a_cpu_clk_dyn = { · 50 unmodified lines | ||
| 466 | 467 | .offset = HHI_SYS_CPU_CLK_CNTL0, |
| 467 | 468 | .mask = 0x1, |
| 468 | 469 | .shift = 10, |
| 470 | + .flags = CLK_MUX_ROUND_CLOSEST, | |
| 469 | 471 | }, |
| 470 | 472 | .hw.init = &(struct clk_init_data){ |
| 471 | 473 | .name = "cpu_clk_dyn", |
| @@ -485,6 +487,7 @@ static struct clk_regmap g12a_cpu_clk = { · 13 unmodified lines | ||
| 485 | 487 | .offset = HHI_SYS_CPU_CLK_CNTL0, |
| 486 | 488 | .mask = 0x1, |
| 487 | 489 | .shift = 11, |
| 490 | + .flags = CLK_MUX_ROUND_CLOSEST, | |
| 488 | 491 | }, |
| 489 | 492 | .hw.init = &(struct clk_init_data){ |
| 490 | 493 | .name = "cpu_clk", |
| @@ -504,6 +507,7 @@ static struct clk_regmap g12b_cpu_clk = { · 13 unmodified lines | ||
| 504 | 507 | .offset = HHI_SYS_CPU_CLK_CNTL0, |
| 505 | 508 | .mask = 0x1, |
| 506 | 509 | .shift = 11, |
| 510 | + .flags = CLK_MUX_ROUND_CLOSEST, | |
| 507 | 511 | }, |
| 508 | 512 | .hw.init = &(struct clk_init_data){ |
| 509 | 513 | .name = "cpu_clk", |
| @@ -523,6 +527,7 @@ static struct clk_regmap g12b_cpub_clk_premux0 = { · 13 unmodified lines | ||
| 523 | 527 | .offset = HHI_SYS_CPUB_CLK_CNTL, |
| 524 | 528 | .mask = 0x3, |
| 525 | 529 | .shift = 0, |
| 530 | + .flags = CLK_MUX_ROUND_CLOSEST, | |
| 526 | 531 | }, |
| 527 | 532 | .hw.init = &(struct clk_init_data){ |
| 528 | 533 | .name = "cpub_clk_dyn0_sel", |
| @@ -533,6 +538,7 @@ static struct clk_regmap g12b_cpub_clk_premux0 = { · 4 unmodified lines | ||
| 533 | 538 | { .hw = &g12a_fclk_div3.hw }, |
| 534 | 539 | }, |
| 535 | 540 | .num_parents = 3, |
| 541 | + .flags = CLK_SET_RATE_PARENT, | |
| 536 | 542 | }, |
| 537 | 543 | }; |
| 538 | 544 | |
| @@ -567,6 +573,7 @@ static struct clk_regmap g12b_cpub_clk_postmux0 = { · 28 unmodified lines | ||
| 567 | 573 | .offset = HHI_SYS_CPUB_CLK_CNTL, |
| 568 | 574 | .mask = 0x1, |
| 569 | 575 | .shift = 2, |
| 576 | + .flags = CLK_MUX_ROUND_CLOSEST, | |
| 570 | 577 | }, |
| 571 | 578 | .hw.init = &(struct clk_init_data){ |
| 572 | 579 | .name = "cpub_clk_dyn0", |
| @@ -644,6 +651,7 @@ static struct clk_regmap g12b_cpub_clk_dyn = { · 71 unmodified lines | ||
| 644 | 651 | .offset = HHI_SYS_CPUB_CLK_CNTL, |
| 645 | 652 | .mask = 0x1, |
| 646 | 653 | .shift = 10, |
| 654 | + .flags = CLK_MUX_ROUND_CLOSEST, | |
| 647 | 655 | }, |
| 648 | 656 | .hw.init = &(struct clk_init_data){ |
| 649 | 657 | .name = "cpub_clk_dyn", |
| @@ -663,6 +671,7 @@ static struct clk_regmap g12b_cpub_clk = { · 13 unmodified lines | ||
| 663 | 671 | .offset = HHI_SYS_CPUB_CLK_CNTL, |
| 664 | 672 | .mask = 0x1, |
| 665 | 673 | .shift = 11, |
| 674 | + .flags = CLK_MUX_ROUND_CLOSEST, | |
| 666 | 675 | }, |
| 667 | 676 | .hw.init = &(struct clk_init_data){ |
| 668 | 677 | .name = "cpub_clk", |
| @@ -935,6 +935,7 @@ static struct clk_regmap gxbb_sar_adc_clk_div = { · 934 unmodified lines | ||
| 935 | 935 | &gxbb_sar_adc_clk_sel.hw |
| 936 | 936 | }, |
| 937 | 937 | .num_parents = 1, |
| 938 | + .flags = CLK_SET_RATE_PARENT, | |
| 938 | 939 | }, |
| 939 | 940 | }; |
| 940 | 941 | |
| @@ -165,12 +165,18 @@ static const unsigned long exynos5x_clk_regs[] __initconst = { · 164 unmodified lines | ||
| 165 | 165 | GATE_BUS_CPU, |
| 166 | 166 | GATE_SCLK_CPU, |
| 167 | 167 | CLKOUT_CMU_CPU, |
| 168 | + CPLL_CON0, | |
| 169 | + DPLL_CON0, | |
| 168 | 170 | EPLL_CON0, |
| 169 | 171 | EPLL_CON1, |
| 170 | 172 | EPLL_CON2, |
| 171 | 173 | RPLL_CON0, |
| 172 | 174 | RPLL_CON1, |
| 173 | 175 | RPLL_CON2, |
| 176 | + IPLL_CON0, | |
| 177 | + SPLL_CON0, | |
| 178 | + VPLL_CON0, | |
| 179 | + MPLL_CON0, | |
| 174 | 180 | SRC_TOP0, |
| 175 | 181 | SRC_TOP1, |
| 176 | 182 | SRC_TOP2, |
| @@ -1172,8 +1178,6 @@ static const struct samsung_gate_clock exynos5x_gate_clks[] __initconst = { · 995 unmodified lines | ||
| 1172 | 1178 | GATE(CLK_SCLK_ISP_SENSOR2, "sclk_isp_sensor2", "dout_isp_sensor2", |
| 1173 | 1179 | GATE_TOP_SCLK_ISP, 12, CLK_SET_RATE_PARENT, 0), |
| 1174 | 1180 | |
| 1175 | − GATE(CLK_G3D, "g3d", "mout_user_aclk_g3d", GATE_IP_G3D, 9, 0, 0), | |
| 1176 | − | |
| 1177 | 1181 | /* CDREX */ |
| 1178 | 1182 | GATE(CLK_CLKM_PHY0, "clkm_phy0", "dout_sclk_cdrex", |
| 1179 | 1183 | GATE_BUS_CDREX0, 0, 0, 0), |
| @@ -1248,6 +1252,15 @@ static struct exynos5_subcmu_reg_dump exynos5x_gsc_suspend_regs[] = { · 68 unmodified lines | ||
| 1248 | 1252 | { DIV2_RATIO0, 0, 0x30 }, /* DIV dout_gscl_blk_300 */ |
| 1249 | 1253 | }; |
| 1250 | 1254 | |
| 1255 | +static const struct samsung_gate_clock exynos5x_g3d_gate_clks[] __initconst = { | |
| 1256 | + GATE(CLK_G3D, "g3d", "mout_user_aclk_g3d", GATE_IP_G3D, 9, 0, 0), | |
| 1257 | +}; | |
| 1258 | + | |
| 1259 | +static struct exynos5_subcmu_reg_dump exynos5x_g3d_suspend_regs[] = { | |
| 1260 | + { GATE_IP_G3D, 0x3ff, 0x3ff }, /* G3D gates */ | |
| 1261 | + { SRC_TOP5, 0, BIT(16) }, /* MUX mout_user_aclk_g3d */ | |
| 1262 | +}; | |
| 1263 | + | |
| 1251 | 1264 | static const struct samsung_div_clock exynos5x_mfc_div_clks[] __initconst = { |
| 1252 | 1265 | DIV(0, "dout_mfc_blk", "mout_user_aclk333", DIV4_RATIO, 0, 2), |
| 1253 | 1266 | }; |
| @@ -1320,6 +1333,14 @@ static const struct exynos5_subcmu_info exynos5x_gsc_subcmu = { · 66 unmodified lines | ||
| 1320 | 1333 | .pd_name = "GSC", |
| 1321 | 1334 | }; |
| 1322 | 1335 | |
| 1336 | +static const struct exynos5_subcmu_info exynos5x_g3d_subcmu = { | |
| 1337 | + .gate_clks = exynos5x_g3d_gate_clks, | |
| 1338 | + .nr_gate_clks = ARRAY_SIZE(exynos5x_g3d_gate_clks), | |
| 1339 | + .suspend_regs = exynos5x_g3d_suspend_regs, | |
| 1340 | + .nr_suspend_regs = ARRAY_SIZE(exynos5x_g3d_suspend_regs), | |
| 1341 | + .pd_name = "G3D", | |
| 1342 | +}; | |
| 1343 | + | |
| 1323 | 1344 | static const struct exynos5_subcmu_info exynos5x_mfc_subcmu = { |
| 1324 | 1345 | .div_clks = exynos5x_mfc_div_clks, |
| 1325 | 1346 | .nr_div_clks = ARRAY_SIZE(exynos5x_mfc_div_clks), |
| @@ -1351,6 +1372,7 @@ static const struct exynos5_subcmu_info exynos5800_mau_subcmu = { · 25 unmodified lines | ||
| 1351 | 1372 | static const struct exynos5_subcmu_info *exynos5x_subcmus[] = { |
| 1352 | 1373 | &exynos5x_disp_subcmu, |
| 1353 | 1374 | &exynos5x_gsc_subcmu, |
| 1375 | + &exynos5x_g3d_subcmu, | |
| 1354 | 1376 | &exynos5x_mfc_subcmu, |
| 1355 | 1377 | &exynos5x_mscl_subcmu, |
| 1356 | 1378 | }; |
| @@ -1358,6 +1380,7 @@ static const struct exynos5_subcmu_info *exynos5x_subcmus[] = { · 1 unmodified line | ||
| 1358 | 1380 | static const struct exynos5_subcmu_info *exynos5800_subcmus[] = { |
| 1359 | 1381 | &exynos5x_disp_subcmu, |
| 1360 | 1382 | &exynos5x_gsc_subcmu, |
| 1383 | + &exynos5x_g3d_subcmu, | |
| 1361 | 1384 | &exynos5x_mfc_subcmu, |
| 1362 | 1385 | &exynos5x_mscl_subcmu, |
| 1363 | 1386 | &exynos5800_mau_subcmu, |
| @@ -13,6 +13,7 @@ · 12 unmodified lines | ||
| 13 | 13 | #include <linux/of_device.h> |
| 14 | 14 | #include <linux/platform_device.h> |
| 15 | 15 | #include <linux/pm_runtime.h> |
| 16 | +#include <linux/slab.h> | |
| 16 | 17 | |
| 17 | 18 | #include <dt-bindings/clock/exynos5433.h> |
| 18 | 19 | |
| @@ -5584,6 +5585,8 @@ static int __init exynos5433_cmu_probe(struct platform_device *pdev) · 5565 unmodified lines | ||
| 5584 | 5585 | |
| 5585 | 5586 | data->clk_save = samsung_clk_alloc_reg_dump(info->clk_regs, |
| 5586 | 5587 | info->nr_clk_regs); |
| 5588 | + if (!data->clk_save) | |
| 5589 | + return -ENOMEM; | |
| 5587 | 5590 | data->nr_clk_save = info->nr_clk_regs; |
| 5588 | 5591 | data->clk_suspend = info->suspend_regs; |
| 5589 | 5592 | data->nr_clk_suspend = info->nr_suspend_regs; |
| @@ -5592,12 +5595,19 @@ static int __init exynos5433_cmu_probe(struct platform_device *pdev) · 2 unmodified lines | ||
| 5592 | 5595 | if (data->nr_pclks > 0) { |
| 5593 | 5596 | data->pclks = devm_kcalloc(dev, sizeof(struct clk *), |
| 5594 | 5597 | data->nr_pclks, GFP_KERNEL); |
| 5595 | − | |
| 5598 | + if (!data->pclks) { | |
| 5599 | + kfree(data->clk_save); | |
| 5600 | + return -ENOMEM; | |
| 5601 | + } | |
| 5596 | 5602 | for (i = 0; i < data->nr_pclks; i++) { |
| 5597 | 5603 | struct clk *clk = of_clk_get(dev->of_node, i); |
| 5598 | 5604 | |
| 5599 | − if (IS_ERR(clk)) | |
| 5605 | + if (IS_ERR(clk)) { | |
| 5606 | + kfree(data->clk_save); | |
| 5607 | + while (--i >= 0) | |
| 5608 | + clk_put(data->pclks[i]); | |
| 5600 | 5609 | return PTR_ERR(clk); |
| 5610 | + } | |
| 5601 | 5611 | data->pclks[i] = clk; |
| 5602 | 5612 | } |
| 5603 | 5613 | } |
| @@ -1224,7 +1224,7 @@ static int sun9i_a80_ccu_probe(struct platform_device *pdev) · 1223 unmodified lines | ||
| 1224 | 1224 | |
| 1225 | 1225 | /* Enforce d1 = 0, d2 = 0 for Audio PLL */ |
| 1226 | 1226 | val = readl(reg + SUN9I_A80_PLL_AUDIO_REG); |
| 1227 | − val &= (BIT(16) & BIT(18)); | |
| 1227 | + val &= ~(BIT(16) | BIT(18)); | |
| 1228 | 1228 | writel(val, reg + SUN9I_A80_PLL_AUDIO_REG); |
| 1229 | 1229 | |
| 1230 | 1230 | /* Enforce P = 1 for both CPU cluster PLLs */ |
| @@ -1080,8 +1080,8 @@ static struct clk ** __init sunxi_divs_clk_setup(struct device_node *node, · 1079 unmodified lines | ||
| 1080 | 1080 | rate_hw, rate_ops, |
| 1081 | 1081 | gate_hw, &clk_gate_ops, |
| 1082 | 1082 | clkflags | |
| 1083 | − data->div[i].critical ? | |
| 1084 | − CLK_IS_CRITICAL : 0); | |
| 1083 | + (data->div[i].critical ? | |
| 1084 | + CLK_IS_CRITICAL : 0)); | |
| 1085 | 1085 | |
| 1086 | 1086 | WARN_ON(IS_ERR(clk_data->clks[i])); |
| 1087 | 1087 | } |
| @@ -174,7 +174,6 @@ static void __init of_dra7_atl_clock_setup(struct device_node *node) · 173 unmodified lines | ||
| 174 | 174 | struct clk_init_data init = { NULL }; |
| 175 | 175 | const char **parent_names = NULL; |
| 176 | 176 | struct clk *clk; |
| 177 | − int ret; | |
| 178 | 177 | |
| 179 | 178 | clk_hw = kzalloc(sizeof(*clk_hw), GFP_KERNEL); |
| 180 | 179 | if (!clk_hw) { |
| @@ -207,11 +206,6 @@ static void __init of_dra7_atl_clock_setup(struct device_node *node) · 26 unmodified lines | ||
| 207 | 206 | clk = ti_clk_register(NULL, &clk_hw->hw, node->name); |
| 208 | 207 | |
| 209 | 208 | if (!IS_ERR(clk)) { |
| 210 | − ret = ti_clk_add_alias(NULL, clk, node->name); | |
| 211 | − if (ret) { | |
| 212 | − clk_unregister(clk); | |
| 213 | − goto cleanup; | |
| 214 | − } | |
| 215 | 209 | of_clk_add_provider(node, of_clk_src_simple_get, clk); |
| 216 | 210 | kfree(parent_names); |
| 217 | 211 | return; |
| @@ -100,11 +100,12 @@ static bool _omap4_is_timeout(union omap4_timeout *time, u32 timeout) · 99 unmodified lines | ||
| 100 | 100 | * can be from a timer that requires pm_runtime access, which |
| 101 | 101 | * will eventually bring us here with timekeeping_suspended, |
| 102 | 102 | * during both suspend entry and resume paths. This happens |
| 103 | − * at least on am43xx platform. | |
| 103 | + * at least on am43xx platform. Account for flakeyness | |
| 104 | + * with udelay() by multiplying the timeout value by 2. | |
| 104 | 105 | */ |
| 105 | 106 | if (unlikely(_early_timeout || timekeeping_suspended)) { |
| 106 | 107 | if (time->cycles++ < timeout) { |
| 107 | − udelay(1); | |
| 108 | + udelay(1 * 2); | |
| 108 | 109 | return false; |
| 109 | 110 | } |
| 110 | 111 | } else { |
| @@ -328,12 +328,13 @@ static int sh_mtu2_register(struct sh_mtu2_channel *ch, const char *name) · 327 unmodified lines | ||
| 328 | 328 | return 0; |
| 329 | 329 | } |
| 330 | 330 | |
| 331 | +static const unsigned int sh_mtu2_channel_offsets[] = { | |
| 332 | + 0x300, 0x380, 0x000, | |
| 333 | +}; | |
| 334 | + | |
| 331 | 335 | static int sh_mtu2_setup_channel(struct sh_mtu2_channel *ch, unsigned int index, |
| 332 | 336 | struct sh_mtu2_device *mtu) |
| 333 | 337 | { |
| 334 | − static const unsigned int channel_offsets[] = { | |
| 335 | − 0x300, 0x380, 0x000, | |
| 336 | − }; | |
| 337 | 338 | char name[6]; |
| 338 | 339 | int irq; |
| 339 | 340 | int ret; |
| @@ -356,7 +357,7 @@ static int sh_mtu2_setup_channel(struct sh_mtu2_channel *ch, unsigned int index, · 16 unmodified lines | ||
| 356 | 357 | return ret; |
| 357 | 358 | } |
| 358 | 359 | |
| 359 | − ch->base = mtu->mapbase + channel_offsets[index]; | |
| 360 | + ch->base = mtu->mapbase + sh_mtu2_channel_offsets[index]; | |
| 360 | 361 | ch->index = index; |
| 361 | 362 | |
| 362 | 363 | return sh_mtu2_register(ch, dev_name(&mtu->pdev->dev)); |
| @@ -408,7 +409,12 @@ static int sh_mtu2_setup(struct sh_mtu2_device *mtu, · 45 unmodified lines | ||
| 408 | 409 | } |
| 409 | 410 | |
| 410 | 411 | /* Allocate and setup the channels. */ |
| 411 | − mtu->num_channels = 3; | |
| 412 | + ret = platform_irq_count(pdev); | |
| 413 | + if (ret < 0) | |
| 414 | + goto err_unmap; | |
| 415 | + | |
| 416 | + mtu->num_channels = min_t(unsigned int, ret, | |
| 417 | + ARRAY_SIZE(sh_mtu2_channel_offsets)); | |
| 412 | 418 | |
| 413 | 419 | mtu->channels = kcalloc(mtu->num_channels, sizeof(*mtu->channels), |
| 414 | 420 | GFP_KERNEL); |
| @@ -268,15 +268,12 @@ static int __init mtk_syst_init(struct device_node *node) · 267 unmodified lines | ||
| 268 | 268 | |
| 269 | 269 | ret = timer_of_init(node, &to); |
| 270 | 270 | if (ret) |
| 271 | − goto err; | |
| 271 | + return ret; | |
| 272 | 272 | |
| 273 | 273 | clockevents_config_and_register(&to.clkevt, timer_of_rate(&to), |
| 274 | 274 | TIMER_SYNC_TICKS, 0xffffffff); |
| 275 | 275 | |
| 276 | 276 | return 0; |
| 277 | −err: | |
| 278 | − timer_of_cleanup(&to); | |
| 279 | − return ret; | |
| 280 | 277 | } |
| 281 | 278 | |
| 282 | 279 | static int __init mtk_gpt_init(struct device_node *node) |
| @@ -293,7 +290,7 @@ static int __init mtk_gpt_init(struct device_node *node) · 10 unmodified lines | ||
| 293 | 290 | |
| 294 | 291 | ret = timer_of_init(node, &to); |
| 295 | 292 | if (ret) |
| 296 | − goto err; | |
| 293 | + return ret; | |
| 297 | 294 | |
| 298 | 295 | /* Configure clock source */ |
| 299 | 296 | mtk_gpt_setup(&to, TIMER_CLK_SRC, GPT_CTRL_OP_FREERUN); |
| @@ -311,9 +308,6 @@ static int __init mtk_gpt_init(struct device_node *node) · 11 unmodified lines | ||
| 311 | 308 | mtk_gpt_enable_irq(&to, TIMER_CLK_EVT); |
| 312 | 309 | |
| 313 | 310 | return 0; |
| 314 | −err: | |
| 315 | − timer_of_cleanup(&to); | |
| 316 | − return ret; | |
| 317 | 311 | } |
| 318 | 312 | TIMER_OF_DECLARE(mtk_mt6577, "mediatek,mt6577-timer", mtk_gpt_init); |
| 319 | 313 | TIMER_OF_DECLARE(mtk_mt6765, "mediatek,mt6765-timer", mtk_syst_init); |
| @@ -720,7 +720,7 @@ static ssize_t store_##file_name \ · 719 unmodified lines | ||
| 720 | 720 | if (ret != 1) \ |
| 721 | 721 | return -EINVAL; \ |
| 722 | 722 | \ |
| 723 | − ret = dev_pm_qos_update_request(policy->object##_freq_req, val);\ | |
| 723 | + ret = freq_qos_update_request(policy->object##_freq_req, val);\ | |
| 724 | 724 | return ret >= 0 ? count : ret; \ |
| 725 | 725 | } |
| 726 | 726 | |
| @@ -1202,19 +1202,21 @@ static struct cpufreq_policy *cpufreq_policy_alloc(unsigned int cpu) · 475 unmodified lines | ||
| 1202 | 1202 | goto err_free_real_cpus; |
| 1203 | 1203 | } |
| 1204 | 1204 | |
| 1205 | + freq_constraints_init(&policy->constraints); | |
| 1206 | + | |
| 1205 | 1207 | policy->nb_min.notifier_call = cpufreq_notifier_min; |
| 1206 | 1208 | policy->nb_max.notifier_call = cpufreq_notifier_max; |
| 1207 | 1209 | |
| 1208 | − ret = dev_pm_qos_add_notifier(dev, &policy->nb_min, | |
| 1209 | − DEV_PM_QOS_MIN_FREQUENCY); | |
| 1210 | + ret = freq_qos_add_notifier(&policy->constraints, FREQ_QOS_MIN, | |
| 1211 | + &policy->nb_min); | |
| 1210 | 1212 | if (ret) { |
| 1211 | 1213 | dev_err(dev, "Failed to register MIN QoS notifier: %d (%*pbl)\n", |
| 1212 | 1214 | ret, cpumask_pr_args(policy->cpus)); |
| 1213 | 1215 | goto err_kobj_remove; |
| 1214 | 1216 | } |
| 1215 | 1217 | |
| 1216 | − ret = dev_pm_qos_add_notifier(dev, &policy->nb_max, | |
| 1217 | − DEV_PM_QOS_MAX_FREQUENCY); | |
| 1218 | + ret = freq_qos_add_notifier(&policy->constraints, FREQ_QOS_MAX, | |
| 1219 | + &policy->nb_max); | |
| 1218 | 1220 | if (ret) { |
| 1219 | 1221 | dev_err(dev, "Failed to register MAX QoS notifier: %d (%*pbl)\n", |
| 1220 | 1222 | ret, cpumask_pr_args(policy->cpus)); |
| @@ -1232,8 +1234,8 @@ static struct cpufreq_policy *cpufreq_policy_alloc(unsigned int cpu) · 11 unmodified lines | ||
| 1232 | 1234 | return policy; |
| 1233 | 1235 | |
| 1234 | 1236 | err_min_qos_notifier: |
| 1235 | − dev_pm_qos_remove_notifier(dev, &policy->nb_min, | |
| 1236 | − DEV_PM_QOS_MIN_FREQUENCY); | |
| 1237 | + freq_qos_remove_notifier(&policy->constraints, FREQ_QOS_MIN, | |
| 1238 | + &policy->nb_min); | |
| 1237 | 1239 | err_kobj_remove: |
| 1238 | 1240 | cpufreq_policy_put_kobj(policy); |
| 1239 | 1241 | err_free_real_cpus: |
| @@ -1250,7 +1252,6 @@ err_free_policy: · 10 unmodified lines | ||
| 1250 | 1252 | |
| 1251 | 1253 | static void cpufreq_policy_free(struct cpufreq_policy *policy) |
| 1252 | 1254 | { |
| 1253 | − struct device *dev = get_cpu_device(policy->cpu); | |
| 1254 | 1255 | unsigned long flags; |
| 1255 | 1256 | int cpu; |
| 1256 | 1257 | |
| @@ -1262,10 +1263,13 @@ static void cpufreq_policy_free(struct cpufreq_policy *policy) · 5 unmodified lines | ||
| 1262 | 1263 | per_cpu(cpufreq_cpu_data, cpu) = NULL; |
| 1263 | 1264 | write_unlock_irqrestore(&cpufreq_driver_lock, flags); |
| 1264 | 1265 | |
| 1265 | − dev_pm_qos_remove_notifier(dev, &policy->nb_max, | |
| 1266 | − DEV_PM_QOS_MAX_FREQUENCY); | |
| 1267 | − dev_pm_qos_remove_notifier(dev, &policy->nb_min, | |
| 1268 | − DEV_PM_QOS_MIN_FREQUENCY); | |
| 1266 | + freq_qos_remove_notifier(&policy->constraints, FREQ_QOS_MAX, | |
| 1267 | + &policy->nb_max); | |
| 1268 | + freq_qos_remove_notifier(&policy->constraints, FREQ_QOS_MIN, | |
| 1269 | + &policy->nb_min); | |
| 1270 | + | |
| 1271 | + /* Cancel any pending policy->update work before freeing the policy. */ | |
| 1272 | + cancel_work_sync(&policy->update); | |
| 1269 | 1273 | |
| 1270 | 1274 | if (policy->max_freq_req) { |
| 1271 | 1275 | /* |
| @@ -1274,10 +1278,10 @@ static void cpufreq_policy_free(struct cpufreq_policy *policy) · 2 unmodified lines | ||
| 1274 | 1278 | */ |
| 1275 | 1279 | blocking_notifier_call_chain(&cpufreq_policy_notifier_list, |
| 1276 | 1280 | CPUFREQ_REMOVE_POLICY, policy); |
| 1277 | − dev_pm_qos_remove_request(policy->max_freq_req); | |
| 1281 | + freq_qos_remove_request(policy->max_freq_req); | |
| 1278 | 1282 | } |
| 1279 | 1283 | |
| 1280 | − dev_pm_qos_remove_request(policy->min_freq_req); | |
| 1284 | + freq_qos_remove_request(policy->min_freq_req); | |
| 1281 | 1285 | kfree(policy->min_freq_req); |
| 1282 | 1286 | |
| 1283 | 1287 | cpufreq_policy_put_kobj(policy); |
| @@ -1357,8 +1361,6 @@ static int cpufreq_online(unsigned int cpu) · 73 unmodified lines | ||
| 1357 | 1361 | cpumask_and(policy->cpus, policy->cpus, cpu_online_mask); |
| 1358 | 1362 | |
| 1359 | 1363 | if (new_policy) { |
| 1360 | − struct device *dev = get_cpu_device(cpu); | |
| 1361 | − | |
| 1362 | 1364 | for_each_cpu(j, policy->related_cpus) { |
| 1363 | 1365 | per_cpu(cpufreq_cpu_data, j) = policy; |
| 1364 | 1366 | add_cpu_dev_symlink(policy, j); |
| @@ -1369,36 +1371,31 @@ static int cpufreq_online(unsigned int cpu) · 4 unmodified lines | ||
| 1369 | 1371 | if (!policy->min_freq_req) |
| 1370 | 1372 | goto out_destroy_policy; |
| 1371 | 1373 | |
| 1372 | − ret = dev_pm_qos_add_request(dev, policy->min_freq_req, | |
| 1373 | − DEV_PM_QOS_MIN_FREQUENCY, | |
| 1374 | − policy->min); | |
| 1374 | + ret = freq_qos_add_request(&policy->constraints, | |
| 1375 | + policy->min_freq_req, FREQ_QOS_MIN, | |
| 1376 | + policy->min); | |
| 1375 | 1377 | if (ret < 0) { |
| 1376 | 1378 | /* |
| 1377 | − * So we don't call dev_pm_qos_remove_request() for an | |
| 1379 | + * So we don't call freq_qos_remove_request() for an | |
| 1378 | 1380 | * uninitialized request. |
| 1379 | 1381 | */ |
| 1380 | 1382 | kfree(policy->min_freq_req); |
| 1381 | 1383 | policy->min_freq_req = NULL; |
| 1382 | − | |
| 1383 | − dev_err(dev, "Failed to add min-freq constraint (%d)\n", | |
| 1384 | − ret); | |
| 1385 | 1384 | goto out_destroy_policy; |
| 1386 | 1385 | } |
| 1387 | 1386 | |
| 1388 | 1387 | /* |
| 1389 | 1388 | * This must be initialized right here to avoid calling |
| 1390 | − * dev_pm_qos_remove_request() on uninitialized request in case | |
| 1389 | + * freq_qos_remove_request() on uninitialized request in case | |
| 1391 | 1390 | * of errors. |
| 1392 | 1391 | */ |
| 1393 | 1392 | policy->max_freq_req = policy->min_freq_req + 1; |
| 1394 | 1393 | |
| 1395 | − ret = dev_pm_qos_add_request(dev, policy->max_freq_req, | |
| 1396 | − DEV_PM_QOS_MAX_FREQUENCY, | |
| 1397 | − policy->max); | |
| 1394 | + ret = freq_qos_add_request(&policy->constraints, | |
| 1395 | + policy->max_freq_req, FREQ_QOS_MAX, | |
| 1396 | + policy->max); | |
| 1398 | 1397 | if (ret < 0) { |
| 1399 | 1398 | policy->max_freq_req = NULL; |
| 1400 | − dev_err(dev, "Failed to add max-freq constraint (%d)\n", | |
| 1401 | − ret); | |
| 1402 | 1399 | goto out_destroy_policy; |
| 1403 | 1400 | } |
| 1404 | 1401 | |
| @@ -2374,7 +2371,6 @@ int cpufreq_set_policy(struct cpufreq_policy *policy, · 969 unmodified lines | ||
| 2374 | 2371 | struct cpufreq_policy *new_policy) |
| 2375 | 2372 | { |
| 2376 | 2373 | struct cpufreq_governor *old_gov; |
| 2377 | − struct device *cpu_dev = get_cpu_device(policy->cpu); | |
| 2378 | 2374 | int ret; |
| 2379 | 2375 | |
| 2380 | 2376 | pr_debug("setting new policy for CPU %u: %u - %u kHz\n", |
| @@ -2386,8 +2382,8 @@ int cpufreq_set_policy(struct cpufreq_policy *policy, · 5 unmodified lines | ||
| 2386 | 2382 | * PM QoS framework collects all the requests from users and provide us |
| 2387 | 2383 | * the final aggregated value here. |
| 2388 | 2384 | */ |
| 2389 | − new_policy->min = dev_pm_qos_read_value(cpu_dev, DEV_PM_QOS_MIN_FREQUENCY); | |
| 2390 | − new_policy->max = dev_pm_qos_read_value(cpu_dev, DEV_PM_QOS_MAX_FREQUENCY); | |
| 2385 | + new_policy->min = freq_qos_read_value(&policy->constraints, FREQ_QOS_MIN); | |
| 2386 | + new_policy->max = freq_qos_read_value(&policy->constraints, FREQ_QOS_MAX); | |
| 2391 | 2387 | |
| 2392 | 2388 | /* verify the cpu speed can be set within this limit */ |
| 2393 | 2389 | ret = cpufreq_driver->verify(new_policy); |
| @@ -2518,7 +2514,7 @@ static int cpufreq_boost_set_sw(int state) · 124 unmodified lines | ||
| 2518 | 2514 | break; |
| 2519 | 2515 | } |
| 2520 | 2516 | |
| 2521 | − ret = dev_pm_qos_update_request(policy->max_freq_req, policy->max); | |
| 2517 | + ret = freq_qos_update_request(policy->max_freq_req, policy->max); | |
| 2522 | 2518 | if (ret < 0) |
| 2523 | 2519 | break; |
| 2524 | 2520 | } |
| @@ -2737,14 +2733,6 @@ int cpufreq_unregister_driver(struct cpufreq_driver *driver) · 212 unmodified lines | ||
| 2737 | 2733 | } |
| 2738 | 2734 | EXPORT_SYMBOL_GPL(cpufreq_unregister_driver); |
| 2739 | 2735 | |
| 2740 | −/* | |
| 2741 | − * Stop cpufreq at shutdown to make sure it isn't holding any locks | |
| 2742 | − * or mutexes when secondary CPUs are halted. | |
| 2743 | − */ | |
| 2744 | −static struct syscore_ops cpufreq_syscore_ops = { | |
| 2745 | − .shutdown = cpufreq_suspend, | |
| 2746 | −}; | |
| 2747 | − | |
| 2748 | 2736 | struct kobject *cpufreq_global_kobject; |
| 2749 | 2737 | EXPORT_SYMBOL(cpufreq_global_kobject); |
| 2750 | 2738 | |
| @@ -2756,8 +2744,6 @@ static int __init cpufreq_core_init(void) · 5 unmodified lines | ||
| 2756 | 2744 | cpufreq_global_kobject = kobject_create_and_add("cpufreq", &cpu_subsys.dev_root->kobj); |
| 2757 | 2745 | BUG_ON(!cpufreq_global_kobject); |
| 2758 | 2746 | |
| 2759 | − register_syscore_ops(&cpufreq_syscore_ops); | |
| 2760 | − | |
| 2761 | 2747 | return 0; |
| 2762 | 2748 | } |
| 2763 | 2749 | module_param(off, int, 0444); |
| @@ -847,11 +847,9 @@ static void intel_pstate_hwp_force_min_perf(int cpu) · 846 unmodified lines | ||
| 847 | 847 | value |= HWP_MAX_PERF(min_perf); |
| 848 | 848 | value |= HWP_MIN_PERF(min_perf); |
| 849 | 849 | |
| 850 | − /* Set EPP/EPB to min */ | |
| 850 | + /* Set EPP to min */ | |
| 851 | 851 | if (boot_cpu_has(X86_FEATURE_HWP_EPP)) |
| 852 | 852 | value |= HWP_ENERGY_PERF_PREFERENCE(HWP_EPP_POWERSAVE); |
| 853 | − else | |
| 854 | − intel_pstate_set_epb(cpu, HWP_EPP_BALANCE_POWERSAVE); | |
| 855 | 853 | |
| 856 | 854 | wrmsrl_on_cpu(cpu, MSR_HWP_REQUEST, value); |
| 857 | 855 | } |
| @@ -1088,10 +1086,10 @@ static ssize_t store_no_turbo(struct kobject *a, struct kobj_attribute *b, · 230 unmodified lines | ||
| 1088 | 1086 | |
| 1089 | 1087 | static struct cpufreq_driver intel_pstate; |
| 1090 | 1088 | |
| 1091 | −static void update_qos_request(enum dev_pm_qos_req_type type) | |
| 1089 | +static void update_qos_request(enum freq_qos_req_type type) | |
| 1092 | 1090 | { |
| 1093 | 1091 | int max_state, turbo_max, freq, i, perf_pct; |
| 1094 | − struct dev_pm_qos_request *req; | |
| 1092 | + struct freq_qos_request *req; | |
| 1095 | 1093 | struct cpufreq_policy *policy; |
| 1096 | 1094 | |
| 1097 | 1095 | for_each_possible_cpu(i) { |
| @@ -1112,7 +1110,7 @@ static void update_qos_request(enum dev_pm_qos_req_type type) · 14 unmodified lines | ||
| 1112 | 1110 | else |
| 1113 | 1111 | turbo_max = cpu->pstate.turbo_pstate; |
| 1114 | 1112 | |
| 1115 | − if (type == DEV_PM_QOS_MIN_FREQUENCY) { | |
| 1113 | + if (type == FREQ_QOS_MIN) { | |
| 1116 | 1114 | perf_pct = global.min_perf_pct; |
| 1117 | 1115 | } else { |
| 1118 | 1116 | req++; |
| @@ -1122,7 +1120,7 @@ static void update_qos_request(enum dev_pm_qos_req_type type) · 3 unmodified lines | ||
| 1122 | 1120 | freq = DIV_ROUND_UP(turbo_max * perf_pct, 100); |
| 1123 | 1121 | freq *= cpu->pstate.scaling; |
| 1124 | 1122 | |
| 1125 | − if (dev_pm_qos_update_request(req, freq) < 0) | |
| 1123 | + if (freq_qos_update_request(req, freq) < 0) | |
| 1126 | 1124 | pr_warn("Failed to update freq constraint: CPU%d\n", i); |
| 1127 | 1125 | } |
| 1128 | 1126 | } |
| @@ -1153,7 +1151,7 @@ static ssize_t store_max_perf_pct(struct kobject *a, struct kobj_attribute *b, · 24 unmodified lines | ||
| 1153 | 1151 | if (intel_pstate_driver == &intel_pstate) |
| 1154 | 1152 | intel_pstate_update_policies(); |
| 1155 | 1153 | else |
| 1156 | − update_qos_request(DEV_PM_QOS_MAX_FREQUENCY); | |
| 1154 | + update_qos_request(FREQ_QOS_MAX); | |
| 1157 | 1155 | |
| 1158 | 1156 | mutex_unlock(&intel_pstate_driver_lock); |
| 1159 | 1157 | |
| @@ -1187,7 +1185,7 @@ static ssize_t store_min_perf_pct(struct kobject *a, struct kobj_attribute *b, · 27 unmodified lines | ||
| 1187 | 1185 | if (intel_pstate_driver == &intel_pstate) |
| 1188 | 1186 | intel_pstate_update_policies(); |
| 1189 | 1187 | else |
| 1190 | − update_qos_request(DEV_PM_QOS_MIN_FREQUENCY); | |
| 1188 | + update_qos_request(FREQ_QOS_MIN); | |
| 1191 | 1189 | |
| 1192 | 1190 | mutex_unlock(&intel_pstate_driver_lock); |
| 1193 | 1191 | |
| @@ -2381,7 +2379,7 @@ static unsigned int intel_cpufreq_fast_switch(struct cpufreq_policy *policy, · 1187 unmodified lines | ||
| 2381 | 2379 | static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy) |
| 2382 | 2380 | { |
| 2383 | 2381 | int max_state, turbo_max, min_freq, max_freq, ret; |
| 2384 | − struct dev_pm_qos_request *req; | |
| 2382 | + struct freq_qos_request *req; | |
| 2385 | 2383 | struct cpudata *cpu; |
| 2386 | 2384 | struct device *dev; |
| 2387 | 2385 | |
| @@ -2416,15 +2414,15 @@ static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy) · 28 unmodified lines | ||
| 2416 | 2414 | max_freq = DIV_ROUND_UP(turbo_max * global.max_perf_pct, 100); |
| 2417 | 2415 | max_freq *= cpu->pstate.scaling; |
| 2418 | 2416 | |
| 2419 | − ret = dev_pm_qos_add_request(dev, req, DEV_PM_QOS_MIN_FREQUENCY, | |
| 2420 | − min_freq); | |
| 2417 | + ret = freq_qos_add_request(&policy->constraints, req, FREQ_QOS_MIN, | |
| 2418 | + min_freq); | |
| 2421 | 2419 | if (ret < 0) { |
| 2422 | 2420 | dev_err(dev, "Failed to add min-freq constraint (%d)\n", ret); |
| 2423 | 2421 | goto free_req; |
| 2424 | 2422 | } |
| 2425 | 2423 | |
| 2426 | − ret = dev_pm_qos_add_request(dev, req + 1, DEV_PM_QOS_MAX_FREQUENCY, | |
| 2427 | − max_freq); | |
| 2424 | + ret = freq_qos_add_request(&policy->constraints, req + 1, FREQ_QOS_MAX, | |
| 2425 | + max_freq); | |
| 2428 | 2426 | if (ret < 0) { |
| 2429 | 2427 | dev_err(dev, "Failed to add max-freq constraint (%d)\n", ret); |
| 2430 | 2428 | goto remove_min_req; |
| @@ -2435,7 +2433,7 @@ static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy) · 4 unmodified lines | ||
| 2435 | 2433 | return 0; |
| 2436 | 2434 | |
| 2437 | 2435 | remove_min_req: |
| 2438 | − dev_pm_qos_remove_request(req); | |
| 2436 | + freq_qos_remove_request(req); | |
| 2439 | 2437 | free_req: |
| 2440 | 2438 | kfree(req); |
| 2441 | 2439 | pstate_exit: |
| @@ -2446,12 +2444,12 @@ pstate_exit: · 4 unmodified lines | ||
| 2446 | 2444 | |
| 2447 | 2445 | static int intel_cpufreq_cpu_exit(struct cpufreq_policy *policy) |
| 2448 | 2446 | { |
| 2449 | − struct dev_pm_qos_request *req; | |
| 2447 | + struct freq_qos_request *req; | |
| 2450 | 2448 | |
| 2451 | 2449 | req = policy->driver_data; |
| 2452 | 2450 | |
| 2453 | − dev_pm_qos_remove_request(req + 1); | |
| 2454 | − dev_pm_qos_remove_request(req); | |
| 2451 | + freq_qos_remove_request(req + 1); | |
| 2452 | + freq_qos_remove_request(req); | |
| 2455 | 2453 | kfree(req); |
| 2456 | 2454 | |
| 2457 | 2455 | return intel_pstate_cpu_exit(policy); |
| @@ -65,7 +65,7 @@ EXPORT_SYMBOL_GPL(cbe_cpufreq_set_pmode_pmi); · 64 unmodified lines | ||
| 65 | 65 | static void cbe_cpufreq_handle_pmi(pmi_message_t pmi_msg) |
| 66 | 66 | { |
| 67 | 67 | struct cpufreq_policy *policy; |
| 68 | − struct dev_pm_qos_request *req; | |
| 68 | + struct freq_qos_request *req; | |
| 69 | 69 | u8 node, slow_mode; |
| 70 | 70 | int cpu, ret; |
| 71 | 71 | |
| @@ -86,7 +86,7 @@ static void cbe_cpufreq_handle_pmi(pmi_message_t pmi_msg) · 14 unmodified lines | ||
| 86 | 86 | |
| 87 | 87 | req = policy->driver_data; |
| 88 | 88 | |
| 89 | − ret = dev_pm_qos_update_request(req, | |
| 89 | + ret = freq_qos_update_request(req, | |
| 90 | 90 | policy->freq_table[slow_mode].frequency); |
| 91 | 91 | if (ret < 0) |
| 92 | 92 | pr_warn("Failed to update freq constraint: %d\n", ret); |
| @@ -103,7 +103,7 @@ static struct pmi_handler cbe_pmi_handler = { · 10 unmodified lines | ||
| 103 | 103 | |
| 104 | 104 | void cbe_cpufreq_pmi_policy_init(struct cpufreq_policy *policy) |
| 105 | 105 | { |
| 106 | − struct dev_pm_qos_request *req; | |
| 106 | + struct freq_qos_request *req; | |
| 107 | 107 | int ret; |
| 108 | 108 | |
| 109 | 109 | if (!cbe_cpufreq_has_pmi) |
| @@ -113,9 +113,8 @@ void cbe_cpufreq_pmi_policy_init(struct cpufreq_policy *policy) · 3 unmodified lines | ||
| 113 | 113 | if (!req) |
| 114 | 114 | return; |
| 115 | 115 | |
| 116 | − ret = dev_pm_qos_add_request(get_cpu_device(policy->cpu), req, | |
| 117 | − DEV_PM_QOS_MAX_FREQUENCY, | |
| 118 | − policy->freq_table[0].frequency); | |
| 116 | + ret = freq_qos_add_request(&policy->constraints, req, FREQ_QOS_MAX, | |
| 117 | + policy->freq_table[0].frequency); | |
| 119 | 118 | if (ret < 0) { |
| 120 | 119 | pr_err("Failed to add freq constraint (%d)\n", ret); |
| 121 | 120 | kfree(req); |
| @@ -128,10 +127,10 @@ EXPORT_SYMBOL_GPL(cbe_cpufreq_pmi_policy_init); · 6 unmodified lines | ||
| 128 | 127 | |
| 129 | 128 | void cbe_cpufreq_pmi_policy_exit(struct cpufreq_policy *policy) |
| 130 | 129 | { |
| 131 | − struct dev_pm_qos_request *req = policy->driver_data; | |
| 130 | + struct freq_qos_request *req = policy->driver_data; | |
| 132 | 131 | |
| 133 | 132 | if (cbe_cpufreq_has_pmi) { |
| 134 | − dev_pm_qos_remove_request(req); | |
| 133 | + freq_qos_remove_request(req); | |
| 135 | 134 | kfree(req); |
| 136 | 135 | } |
| 137 | 136 | } |
| @@ -95,6 +95,10 @@ static int __init haltpoll_init(void) · 94 unmodified lines | ||
| 95 | 95 | int ret; |
| 96 | 96 | struct cpuidle_driver *drv = &haltpoll_driver; |
| 97 | 97 | |
| 98 | + /* Do not load haltpoll if idle= is passed */ | |
| 99 | + if (boot_option_idle_override != IDLE_NO_OVERRIDE) | |
| 100 | + return -ENODEV; | |
| 101 | + | |
| 98 | 102 | cpuidle_poll_state_init(drv); |
| 99 | 103 | |
| 100 | 104 | if (!kvm_para_available() || |
| @@ -1297,7 +1297,7 @@ static void make_established(struct sock *sk, u32 snd_isn, unsigned int opt) · 1296 unmodified lines | ||
| 1297 | 1297 | tp->write_seq = snd_isn; |
| 1298 | 1298 | tp->snd_nxt = snd_isn; |
| 1299 | 1299 | tp->snd_una = snd_isn; |
| 1300 | − inet_sk(sk)->inet_id = tp->write_seq ^ jiffies; | |
| 1300 | + inet_sk(sk)->inet_id = prandom_u32(); | |
| 1301 | 1301 | assign_rxopt(sk, opt); |
| 1302 | 1302 | |
| 1303 | 1303 | if (tp->rcv_wnd > (RCV_BUFSIZ_M << 10)) |
| @@ -1702,7 +1702,7 @@ int chtls_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, · 1701 unmodified lines | ||
| 1702 | 1702 | return peekmsg(sk, msg, len, nonblock, flags); |
| 1703 | 1703 | |
| 1704 | 1704 | if (sk_can_busy_loop(sk) && |
| 1705 | − skb_queue_empty(&sk->sk_receive_queue) && | |
| 1705 | + skb_queue_empty_lockless(&sk->sk_receive_queue) && | |
| 1706 | 1706 | sk->sk_state == TCP_ESTABLISHED) |
| 1707 | 1707 | sk_busy_loop(sk, nonblock); |
| 1708 | 1708 | |
| @@ -45,10 +45,10 @@ static char *dmabuffs_dname(struct dentry *dentry, char *buffer, int buflen) · 44 unmodified lines | ||
| 45 | 45 | size_t ret = 0; |
| 46 | 46 | |
| 47 | 47 | dmabuf = dentry->d_fsdata; |
| 48 | − mutex_lock(&dmabuf->lock); | |
| 48 | + dma_resv_lock(dmabuf->resv, NULL); | |
| 49 | 49 | if (dmabuf->name) |
| 50 | 50 | ret = strlcpy(name, dmabuf->name, DMA_BUF_NAME_LEN); |
| 51 | − mutex_unlock(&dmabuf->lock); | |
| 51 | + dma_resv_unlock(dmabuf->resv); | |
| 52 | 52 | |
| 53 | 53 | return dynamic_dname(dentry, buffer, buflen, "/%s:%s", |
| 54 | 54 | dentry->d_name.name, ret > 0 ? name : ""); |
| @@ -334,7 +334,7 @@ static long dma_buf_set_name(struct dma_buf *dmabuf, const char __user *buf) · 279 unmodified lines | ||
| 334 | 334 | if (IS_ERR(name)) |
| 335 | 335 | return PTR_ERR(name); |
| 336 | 336 | |
| 337 | − mutex_lock(&dmabuf->lock); | |
| 337 | + dma_resv_lock(dmabuf->resv, NULL); | |
| 338 | 338 | if (!list_empty(&dmabuf->attachments)) { |
| 339 | 339 | ret = -EBUSY; |
| 340 | 340 | kfree(name); |
| @@ -344,7 +344,7 @@ static long dma_buf_set_name(struct dma_buf *dmabuf, const char __user *buf) · 3 unmodified lines | ||
| 344 | 344 | dmabuf->name = name; |
| 345 | 345 | |
| 346 | 346 | out_unlock: |
| 347 | − mutex_unlock(&dmabuf->lock); | |
| 347 | + dma_resv_unlock(dmabuf->resv); | |
| 348 | 348 | return ret; |
| 349 | 349 | } |
| 350 | 350 | |
| @@ -403,10 +403,10 @@ static void dma_buf_show_fdinfo(struct seq_file *m, struct file *file) · 52 unmodified lines | ||
| 403 | 403 | /* Don't count the temporary reference taken inside procfs seq_show */ |
| 404 | 404 | seq_printf(m, "count:\t%ld\n", file_count(dmabuf->file) - 1); |
| 405 | 405 | seq_printf(m, "exp_name:\t%s\n", dmabuf->exp_name); |
| 406 | − mutex_lock(&dmabuf->lock); | |
| 406 | + dma_resv_lock(dmabuf->resv, NULL); | |
| 407 | 407 | if (dmabuf->name) |
| 408 | 408 | seq_printf(m, "name:\t%s\n", dmabuf->name); |
| 409 | − mutex_unlock(&dmabuf->lock); | |
| 409 | + dma_resv_unlock(dmabuf->resv); | |
| 410 | 410 | } |
| 411 | 411 | |
| 412 | 412 | static const struct file_operations dma_buf_fops = { |
| @@ -525,6 +525,10 @@ struct dma_buf *dma_buf_export(const struct dma_buf_export_info *exp_info) · 112 unmodified lines | ||
| 525 | 525 | return ERR_PTR(-EINVAL); |
| 526 | 526 | } |
| 527 | 527 | |
| 528 | + if (WARN_ON(exp_info->ops->cache_sgt_mapping && | |
| 529 | + exp_info->ops->dynamic_mapping)) | |
| 530 | + return ERR_PTR(-EINVAL); | |
| 531 | + | |
| 528 | 532 | if (!try_module_get(exp_info->owner)) |
| 529 | 533 | return ERR_PTR(-ENOENT); |
| 530 | 534 | |
| @@ -645,10 +649,11 @@ void dma_buf_put(struct dma_buf *dmabuf) · 114 unmodified lines | ||
| 645 | 649 | EXPORT_SYMBOL_GPL(dma_buf_put); |
| 646 | 650 | |
| 647 | 651 | /** |
| 648 | − * dma_buf_attach - Add the device to dma_buf's attachments list; optionally, | |
| 652 | + * dma_buf_dynamic_attach - Add the device to dma_buf's attachments list; optionally, | |
| 649 | 653 | * calls attach() of dma_buf_ops to allow device-specific attach functionality |
| 650 | − * @dmabuf: [in] buffer to attach device to. | |
| 651 | − * @dev: [in] device to be attached. | |
| 654 | + * @dmabuf: [in] buffer to attach device to. | |
| 655 | + * @dev: [in] device to be attached. | |
| 656 | + * @dynamic_mapping: [in] calling convention for map/unmap | |
| 652 | 657 | * |
| 653 | 658 | * Returns struct dma_buf_attachment pointer for this attachment. Attachments |
| 654 | 659 | * must be cleaned up by calling dma_buf_detach(). |
| @@ -662,8 +667,9 @@ EXPORT_SYMBOL_GPL(dma_buf_put); · 7 unmodified lines | ||
| 662 | 667 | * accessible to @dev, and cannot be moved to a more suitable place. This is |
| 663 | 668 | * indicated with the error code -EBUSY. |
| 664 | 669 | */ |
| 665 | −struct dma_buf_attachment *dma_buf_attach(struct dma_buf *dmabuf, | |
| 666 | − struct device *dev) | |
| 670 | +struct dma_buf_attachment * | |
| 671 | +dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev, | |
| 672 | + bool dynamic_mapping) | |
| 667 | 673 | { |
| 668 | 674 | struct dma_buf_attachment *attach; |
| 669 | 675 | int ret; |
| @@ -677,25 +683,69 @@ struct dma_buf_attachment *dma_buf_attach(struct dma_buf *dmabuf, · 7 unmodified lines | ||
| 677 | 683 | |
| 678 | 684 | attach->dev = dev; |
| 679 | 685 | attach->dmabuf = dmabuf; |
| 680 | − | |
| 681 | − mutex_lock(&dmabuf->lock); | |
| 686 | + attach->dynamic_mapping = dynamic_mapping; | |
| 682 | 687 | |
| 683 | 688 | if (dmabuf->ops->attach) { |
| 684 | 689 | ret = dmabuf->ops->attach(dmabuf, attach); |
| 685 | 690 | if (ret) |
| 686 | 691 | goto err_attach; |
| 687 | 692 | } |
| 693 | + dma_resv_lock(dmabuf->resv, NULL); | |
| 688 | 694 | list_add(&attach->node, &dmabuf->attachments); |
| 695 | + dma_resv_unlock(dmabuf->resv); | |
| 689 | 696 | |
| 690 | − mutex_unlock(&dmabuf->lock); | |
| 697 | + /* When either the importer or the exporter can't handle dynamic | |
| 698 | + * mappings we cache the mapping here to avoid issues with the | |
| 699 | + * reservation object lock. | |
| 700 | + */ | |
| 701 | + if (dma_buf_attachment_is_dynamic(attach) != | |
| 702 | + dma_buf_is_dynamic(dmabuf)) { | |
| 703 | + struct sg_table *sgt; | |
| 704 | + | |
| 705 | + if (dma_buf_is_dynamic(attach->dmabuf)) | |
| 706 | + dma_resv_lock(attach->dmabuf->resv, NULL); | |
| 707 | + | |
| 708 | + sgt = dmabuf->ops->map_dma_buf(attach, DMA_BIDIRECTIONAL); | |
| 709 | + if (!sgt) | |
| 710 | + sgt = ERR_PTR(-ENOMEM); | |
| 711 | + if (IS_ERR(sgt)) { | |
| 712 | + ret = PTR_ERR(sgt); | |
| 713 | + goto err_unlock; | |
| 714 | + } | |
| 715 | + if (dma_buf_is_dynamic(attach->dmabuf)) | |
| 716 | + dma_resv_unlock(attach->dmabuf->resv); | |
| 717 | + attach->sgt = sgt; | |
| 718 | + attach->dir = DMA_BIDIRECTIONAL; | |
| 719 | + } | |
| 691 | 720 | |
| 692 | 721 | return attach; |
| 693 | 722 | |
| 694 | 723 | err_attach: |
| 695 | 724 | kfree(attach); |
| 696 | − mutex_unlock(&dmabuf->lock); | |
| 725 | + return ERR_PTR(ret); | |
| 726 | + | |
| 727 | +err_unlock: | |
| 728 | + if (dma_buf_is_dynamic(attach->dmabuf)) | |
| 729 | + dma_resv_unlock(attach->dmabuf->resv); | |
| 730 | + | |
| 731 | + dma_buf_detach(dmabuf, attach); | |
| 697 | 732 | return ERR_PTR(ret); |
| 698 | 733 | } |
| 734 | +EXPORT_SYMBOL_GPL(dma_buf_dynamic_attach); | |
| 735 | + | |
| 736 | +/** | |
| 737 | + * dma_buf_attach - Wrapper for dma_buf_dynamic_attach | |
| 738 | + * @dmabuf: [in] buffer to attach device to. | |
| 739 | + * @dev: [in] device to be attached. | |
| 740 | + * | |
| 741 | + * Wrapper to call dma_buf_dynamic_attach() for drivers which still use a static | |
| 742 | + * mapping. | |
| 743 | + */ | |
| 744 | +struct dma_buf_attachment *dma_buf_attach(struct dma_buf *dmabuf, | |
| 745 | + struct device *dev) | |
| 746 | +{ | |
| 747 | + return dma_buf_dynamic_attach(dmabuf, dev, false); | |
| 748 | +} | |
| 699 | 749 | EXPORT_SYMBOL_GPL(dma_buf_attach); |
| 700 | 750 | |
| 701 | 751 | /** |
| @@ -711,15 +761,22 @@ void dma_buf_detach(struct dma_buf *dmabuf, struct dma_buf_attachment *attach) · 9 unmodified lines | ||
| 711 | 761 | if (WARN_ON(!dmabuf || !attach)) |
| 712 | 762 | return; |
| 713 | 763 | |
| 714 | − if (attach->sgt) | |
| 764 | + if (attach->sgt) { | |
| 765 | + if (dma_buf_is_dynamic(attach->dmabuf)) | |
| 766 | + dma_resv_lock(attach->dmabuf->resv, NULL); | |
| 767 | + | |
| 715 | 768 | dmabuf->ops->unmap_dma_buf(attach, attach->sgt, attach->dir); |
| 716 | 769 | |
| 717 | − mutex_lock(&dmabuf->lock); | |
| 770 | + if (dma_buf_is_dynamic(attach->dmabuf)) | |
| 771 | + dma_resv_unlock(attach->dmabuf->resv); | |
| 772 | + } | |
| 773 | + | |
| 774 | + dma_resv_lock(dmabuf->resv, NULL); | |
| 718 | 775 | list_del(&attach->node); |
| 776 | + dma_resv_unlock(dmabuf->resv); | |
| 719 | 777 | if (dmabuf->ops->detach) |
| 720 | 778 | dmabuf->ops->detach(dmabuf, attach); |
| 721 | 779 | |
| 722 | − mutex_unlock(&dmabuf->lock); | |
| 723 | 780 | kfree(attach); |
| 724 | 781 | } |
| 725 | 782 | EXPORT_SYMBOL_GPL(dma_buf_detach); |
| @@ -749,6 +806,9 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach, · 23 unmodified lines | ||
| 749 | 806 | if (WARN_ON(!attach || !attach->dmabuf)) |
| 750 | 807 | return ERR_PTR(-EINVAL); |
| 751 | 808 | |
| 809 | + if (dma_buf_attachment_is_dynamic(attach)) | |
| 810 | + dma_resv_assert_held(attach->dmabuf->resv); | |
| 811 | + | |
| 752 | 812 | if (attach->sgt) { |
| 753 | 813 | /* |
| 754 | 814 | * Two mappings with different directions for the same |
| @@ -761,6 +821,9 @@ struct sg_table *dma_buf_map_attachment(struct dma_buf_attachment *attach, · 6 unmodified lines | ||
| 761 | 821 | return attach->sgt; |
| 762 | 822 | } |
| 763 | 823 | |
| 824 | + if (dma_buf_is_dynamic(attach->dmabuf)) | |
| 825 | + dma_resv_assert_held(attach->dmabuf->resv); | |
| 826 | + | |
| 764 | 827 | sg_table = attach->dmabuf->ops->map_dma_buf(attach, direction); |
| 765 | 828 | if (!sg_table) |
| 766 | 829 | sg_table = ERR_PTR(-ENOMEM); |
| @@ -793,9 +856,15 @@ void dma_buf_unmap_attachment(struct dma_buf_attachment *attach, · 26 unmodified lines | ||
| 793 | 856 | if (WARN_ON(!attach || !attach->dmabuf || !sg_table)) |
| 794 | 857 | return; |
| 795 | 858 | |
| 859 | + if (dma_buf_attachment_is_dynamic(attach)) | |
| 860 | + dma_resv_assert_held(attach->dmabuf->resv); | |
| 861 | + | |
| 796 | 862 | if (attach->sgt == sg_table) |
| 797 | 863 | return; |
| 798 | 864 | |
| 865 | + if (dma_buf_is_dynamic(attach->dmabuf)) | |
| 866 | + dma_resv_assert_held(attach->dmabuf->resv); | |
| 867 | + | |
| 799 | 868 | attach->dmabuf->ops->unmap_dma_buf(attach, sg_table, direction); |
| 800 | 869 | } |
| 801 | 870 | EXPORT_SYMBOL_GPL(dma_buf_unmap_attachment); |
| @@ -1171,13 +1240,10 @@ static int dma_buf_debug_show(struct seq_file *s, void *unused) · 369 unmodified lines | ||
| 1171 | 1240 | "size", "flags", "mode", "count", "ino"); |
| 1172 | 1241 | |
| 1173 | 1242 | list_for_each_entry(buf_obj, &db_list.head, list_node) { |
| 1174 | − ret = mutex_lock_interruptible(&buf_obj->lock); | |
| 1175 | 1243 | |
| 1176 | − if (ret) { | |
| 1177 | − seq_puts(s, | |
| 1178 | − "\tERROR locking buffer object: skipping\n"); | |
| 1179 | − continue; | |
| 1180 | − } | |
| 1244 | + ret = dma_resv_lock_interruptible(buf_obj->resv, NULL); | |
| 1245 | + if (ret) | |
| 1246 | + goto error_unlock; | |
| 1181 | 1247 | |
| 1182 | 1248 | seq_printf(s, "%08zu\t%08x\t%08x\t%08ld\t%s\t%08lu\t%s\n", |
| 1183 | 1249 | buf_obj->size, |
| @@ -1223,19 +1289,23 @@ static int dma_buf_debug_show(struct seq_file *s, void *unused) · 39 unmodified lines | ||
| 1223 | 1289 | seq_printf(s, "\t%s\n", dev_name(attach_obj->dev)); |
| 1224 | 1290 | attach_count++; |
| 1225 | 1291 | } |
| 1292 | + dma_resv_unlock(buf_obj->resv); | |
| 1226 | 1293 | |
| 1227 | 1294 | seq_printf(s, "Total %d devices attached\n\n", |
| 1228 | 1295 | attach_count); |
| 1229 | 1296 | |
| 1230 | 1297 | count++; |
| 1231 | 1298 | size += buf_obj->size; |
| 1232 | − mutex_unlock(&buf_obj->lock); | |
| 1233 | 1299 | } |
| 1234 | 1300 | |
| 1235 | 1301 | seq_printf(s, "\nTotal %d objects, %zu bytes\n", count, size); |
| 1236 | 1302 | |
| 1237 | 1303 | mutex_unlock(&db_list.lock); |
| 1238 | 1304 | return 0; |
| 1305 | + | |
| 1306 | +error_unlock: | |
| 1307 | + mutex_unlock(&db_list.lock); | |
| 1308 | + return ret; | |
| 1239 | 1309 | } |
| 1240 | 1310 | |
| 1241 | 1311 | DEFINE_SHOW_ATTRIBUTE(dma_buf_debug); |
| @@ -1707,6 +1707,14 @@ static void sdma_add_scripts(struct sdma_engine *sdma, · 1706 unmodified lines | ||
| 1707 | 1707 | if (!sdma->script_number) |
| 1708 | 1708 | sdma->script_number = SDMA_SCRIPT_ADDRS_ARRAY_SIZE_V1; |
| 1709 | 1709 | |
| 1710 | + if (sdma->script_number > sizeof(struct sdma_script_start_addrs) | |
| 1711 | + / sizeof(s32)) { | |
| 1712 | + dev_err(sdma->dev, | |
| 1713 | + "SDMA script number %d not match with firmware.\n", | |
| 1714 | + sdma->script_number); | |
| 1715 | + return; | |
| 1716 | + } | |
| 1717 | + | |
| 1710 | 1718 | for (i = 0; i < sdma->script_number; i++) |
| 1711 | 1719 | if (addr_arr[i] > 0) |
| 1712 | 1720 | saddr_arr[i] = addr_arr[i]; |
| @@ -694,6 +694,25 @@ static int bam_dma_terminate_all(struct dma_chan *chan) · 693 unmodified lines | ||
| 694 | 694 | |
| 695 | 695 | /* remove all transactions, including active transaction */ |
| 696 | 696 | spin_lock_irqsave(&bchan->vc.lock, flag); |
| 697 | + /* | |
| 698 | + * If we have transactions queued, then some might be committed to the | |
| 699 | + * hardware in the desc fifo. The only way to reset the desc fifo is | |
| 700 | + * to do a hardware reset (either by pipe or the entire block). | |
| 701 | + * bam_chan_init_hw() will trigger a pipe reset, and also reinit the | |
| 702 | + * pipe. If the pipe is left disabled (default state after pipe reset) | |
| 703 | + * and is accessed by a connected hardware engine, a fatal error in | |
| 704 | + * the BAM will occur. There is a small window where this could happen | |
| 705 | + * with bam_chan_init_hw(), but it is assumed that the caller has | |
| 706 | + * stopped activity on any attached hardware engine. Make sure to do | |
| 707 | + * this first so that the BAM hardware doesn't cause memory corruption | |
| 708 | + * by accessing freed resources. | |
| 709 | + */ | |
| 710 | + if (!list_empty(&bchan->desc_list)) { | |
| 711 | + async_desc = list_first_entry(&bchan->desc_list, | |
| 712 | + struct bam_async_desc, desc_node); | |
| 713 | + bam_chan_init_hw(bchan, async_desc->dir); | |
| 714 | + } | |
| 715 | + | |
| 697 | 716 | list_for_each_entry_safe(async_desc, tmp, |
| 698 | 717 | &bchan->desc_list, desc_node) { |
| 699 | 718 | list_add(&async_desc->vd.node, &bchan->vc.desc_issued); |
| @@ -134,6 +134,10 @@ · 133 unmodified lines | ||
| 134 | 134 | #define SPRD_DMA_SRC_TRSF_STEP_OFFSET 0 |
| 135 | 135 | #define SPRD_DMA_TRSF_STEP_MASK GENMASK(15, 0) |
| 136 | 136 | |
| 137 | +/* SPRD DMA_SRC_BLK_STEP register definition */ | |
| 138 | +#define SPRD_DMA_LLIST_HIGH_MASK GENMASK(31, 28) | |
| 139 | +#define SPRD_DMA_LLIST_HIGH_SHIFT 28 | |
| 140 | + | |
| 137 | 141 | /* define DMA channel mode & trigger mode mask */ |
| 138 | 142 | #define SPRD_DMA_CHN_MODE_MASK GENMASK(7, 0) |
| 139 | 143 | #define SPRD_DMA_TRG_MODE_MASK GENMASK(7, 0) |
| @@ -208,6 +212,7 @@ struct sprd_dma_dev { · 68 unmodified lines | ||
| 208 | 212 | struct sprd_dma_chn channels[0]; |
| 209 | 213 | }; |
| 210 | 214 | |
| 215 | +static void sprd_dma_free_desc(struct virt_dma_desc *vd); | |
| 211 | 216 | static bool sprd_dma_filter_fn(struct dma_chan *chan, void *param); |
| 212 | 217 | static struct of_dma_filter_info sprd_dma_info = { |
| 213 | 218 | .filter_fn = sprd_dma_filter_fn, |
| @@ -609,12 +614,19 @@ static int sprd_dma_alloc_chan_resources(struct dma_chan *chan) · 395 unmodified lines | ||
| 609 | 614 | static void sprd_dma_free_chan_resources(struct dma_chan *chan) |
| 610 | 615 | { |
| 611 | 616 | struct sprd_dma_chn *schan = to_sprd_dma_chan(chan); |
| 617 | + struct virt_dma_desc *cur_vd = NULL; | |
| 612 | 618 | unsigned long flags; |
| 613 | 619 | |
| 614 | 620 | spin_lock_irqsave(&schan->vc.lock, flags); |
| 621 | + if (schan->cur_desc) | |
| 622 | + cur_vd = &schan->cur_desc->vd; | |
| 623 | + | |
| 615 | 624 | sprd_dma_stop(schan); |
| 616 | 625 | spin_unlock_irqrestore(&schan->vc.lock, flags); |
| 617 | 626 | |
| 627 | + if (cur_vd) | |
| 628 | + sprd_dma_free_desc(cur_vd); | |
| 629 | + | |
| 618 | 630 | vchan_free_chan_resources(&schan->vc); |
| 619 | 631 | pm_runtime_put(chan->device->dev); |
| 620 | 632 | } |
| @@ -717,6 +729,7 @@ static int sprd_dma_fill_desc(struct dma_chan *chan, · 96 unmodified lines | ||
| 717 | 729 | u32 int_mode = flags & SPRD_DMA_INT_MASK; |
| 718 | 730 | int src_datawidth, dst_datawidth, src_step, dst_step; |
| 719 | 731 | u32 temp, fix_mode = 0, fix_en = 0; |
| 732 | + phys_addr_t llist_ptr; | |
| 720 | 733 | |
| 721 | 734 | if (dir == DMA_MEM_TO_DEV) { |
| 722 | 735 | src_step = sprd_dma_get_step(slave_cfg->src_addr_width); |
| @@ -814,13 +827,16 @@ static int sprd_dma_fill_desc(struct dma_chan *chan, · 91 unmodified lines | ||
| 814 | 827 | * Set the link-list pointer point to next link-list |
| 815 | 828 | * configuration's physical address. |
| 816 | 829 | */ |
| 817 | − hw->llist_ptr = schan->linklist.phy_addr + temp; | |
| 830 | + llist_ptr = schan->linklist.phy_addr + temp; | |
| 831 | + hw->llist_ptr = lower_32_bits(llist_ptr); | |
| 832 | + hw->src_blk_step = (upper_32_bits(llist_ptr) << SPRD_DMA_LLIST_HIGH_SHIFT) & | |
| 833 | + SPRD_DMA_LLIST_HIGH_MASK; | |
| 818 | 834 | } else { |
| 819 | 835 | hw->llist_ptr = 0; |
| 836 | + hw->src_blk_step = 0; | |
| 820 | 837 | } |
| 821 | 838 | |
| 822 | 839 | hw->frg_step = 0; |
| 823 | − hw->src_blk_step = 0; | |
| 824 | 840 | hw->des_blk_step = 0; |
| 825 | 841 | return 0; |
| 826 | 842 | } |
| @@ -1023,15 +1039,22 @@ static int sprd_dma_resume(struct dma_chan *chan) · 196 unmodified lines | ||
| 1023 | 1039 | static int sprd_dma_terminate_all(struct dma_chan *chan) |
| 1024 | 1040 | { |
| 1025 | 1041 | struct sprd_dma_chn *schan = to_sprd_dma_chan(chan); |
| 1042 | + struct virt_dma_desc *cur_vd = NULL; | |
| 1026 | 1043 | unsigned long flags; |
| 1027 | 1044 | LIST_HEAD(head); |
| 1028 | 1045 | |
| 1029 | 1046 | spin_lock_irqsave(&schan->vc.lock, flags); |
| 1047 | + if (schan->cur_desc) | |
| 1048 | + cur_vd = &schan->cur_desc->vd; | |
| 1049 | + | |
| 1030 | 1050 | sprd_dma_stop(schan); |
| 1031 | 1051 | |
| 1032 | 1052 | vchan_get_all_descriptors(&schan->vc, &head); |
| 1033 | 1053 | spin_unlock_irqrestore(&schan->vc.lock, flags); |
| 1034 | 1054 | |
| 1055 | + if (cur_vd) | |
| 1056 | + sprd_dma_free_desc(cur_vd); | |
| 1057 | + | |
| 1035 | 1058 | vchan_dma_desc_free_list(&schan->vc, &head); |
| 1036 | 1059 | return 0; |
| 1037 | 1060 | } |
| @@ -40,6 +40,7 @@ · 39 unmodified lines | ||
| 40 | 40 | #define ADMA_CH_CONFIG_MAX_BURST_SIZE 16 |
| 41 | 41 | #define ADMA_CH_CONFIG_WEIGHT_FOR_WRR(val) ((val) & 0xf) |
| 42 | 42 | #define ADMA_CH_CONFIG_MAX_BUFS 8 |
| 43 | +#define TEGRA186_ADMA_CH_CONFIG_OUTSTANDING_REQS(reqs) (reqs << 4) | |
| 43 | 44 | |
| 44 | 45 | #define ADMA_CH_FIFO_CTRL 0x2c |
| 45 | 46 | #define TEGRA210_ADMA_CH_FIFO_CTRL_TXSIZE(val) (((val) & 0xf) << 8) |
| @@ -77,6 +78,7 @@ struct tegra_adma; · 31 unmodified lines | ||
| 77 | 78 | * @ch_req_tx_shift: Register offset for AHUB transmit channel select. |
| 78 | 79 | * @ch_req_rx_shift: Register offset for AHUB receive channel select. |
| 79 | 80 | * @ch_base_offset: Register offset of DMA channel registers. |
| 81 | + * @has_outstanding_reqs: If DMA channel can have outstanding requests. | |
| 80 | 82 | * @ch_fifo_ctrl: Default value for channel FIFO CTRL register. |
| 81 | 83 | * @ch_req_mask: Mask for Tx or Rx channel select. |
| 82 | 84 | * @ch_req_max: Maximum number of Tx or Rx channels available. |
| @@ -95,6 +97,7 @@ struct tegra_adma_chip_data { · 12 unmodified lines | ||
| 95 | 97 | unsigned int ch_req_max; |
| 96 | 98 | unsigned int ch_reg_size; |
| 97 | 99 | unsigned int nr_channels; |
| 100 | + bool has_outstanding_reqs; | |
| 98 | 101 | }; |
| 99 | 102 | |
| 100 | 103 | /* |
| @@ -594,6 +597,8 @@ static int tegra_adma_set_xfer_params(struct tegra_adma_chan *tdc, · 493 unmodified lines | ||
| 594 | 597 | ADMA_CH_CTRL_FLOWCTRL_EN; |
| 595 | 598 | ch_regs->config |= cdata->adma_get_burst_config(burst_size); |
| 596 | 599 | ch_regs->config |= ADMA_CH_CONFIG_WEIGHT_FOR_WRR(1); |
| 600 | + if (cdata->has_outstanding_reqs) | |
| 601 | + ch_regs->config |= TEGRA186_ADMA_CH_CONFIG_OUTSTANDING_REQS(8); | |
| 597 | 602 | ch_regs->fifo_ctrl = cdata->ch_fifo_ctrl; |
| 598 | 603 | ch_regs->tc = desc->period_len & ADMA_CH_TC_COUNT_MASK; |
| 599 | 604 | |
| @@ -778,6 +783,7 @@ static const struct tegra_adma_chip_data tegra210_chip_data = { · 178 unmodified lines | ||
| 778 | 783 | .ch_req_tx_shift = 28, |
| 779 | 784 | .ch_req_rx_shift = 24, |
| 780 | 785 | .ch_base_offset = 0, |
| 786 | + .has_outstanding_reqs = false, | |
| 781 | 787 | .ch_fifo_ctrl = TEGRA210_FIFO_CTRL_DEFAULT, |
| 782 | 788 | .ch_req_mask = 0xf, |
| 783 | 789 | .ch_req_max = 10, |
| @@ -792,6 +798,7 @@ static const struct tegra_adma_chip_data tegra186_chip_data = { · 8 unmodified lines | ||
| 792 | 798 | .ch_req_tx_shift = 27, |
| 793 | 799 | .ch_req_rx_shift = 22, |
| 794 | 800 | .ch_base_offset = 0x10000, |
| 801 | + .has_outstanding_reqs = true, | |
| 795 | 802 | .ch_fifo_ctrl = TEGRA186_FIFO_CTRL_DEFAULT, |
| 796 | 803 | .ch_req_mask = 0x1f, |
| 797 | 804 | .ch_req_max = 20, |
| @@ -586,9 +586,22 @@ static struct dma_async_tx_descriptor *cppi41_dma_prep_slave_sg( · 585 unmodified lines | ||
| 586 | 586 | enum dma_transfer_direction dir, unsigned long tx_flags, void *context) |
| 587 | 587 | { |
| 588 | 588 | struct cppi41_channel *c = to_cpp41_chan(chan); |
| 589 | + struct dma_async_tx_descriptor *txd = NULL; | |
| 590 | + struct cppi41_dd *cdd = c->cdd; | |
| 589 | 591 | struct cppi41_desc *d; |
| 590 | 592 | struct scatterlist *sg; |
| 591 | 593 | unsigned int i; |
| 594 | + int error; | |
| 595 | + | |
| 596 | + error = pm_runtime_get(cdd->ddev.dev); | |
| 597 | + if (error < 0) { | |
| 598 | + pm_runtime_put_noidle(cdd->ddev.dev); | |
| 599 | + | |
| 600 | + return NULL; | |
| 601 | + } | |
| 602 | + | |
| 603 | + if (cdd->is_suspended) | |
| 604 | + goto err_out_not_ready; | |
| 592 | 605 | |
| 593 | 606 | d = c->desc; |
| 594 | 607 | for_each_sg(sgl, sg, sg_len, i) { |
| @@ -611,7 +624,13 @@ static struct dma_async_tx_descriptor *cppi41_dma_prep_slave_sg( · 16 unmodified lines | ||
| 611 | 624 | d++; |
| 612 | 625 | } |
| 613 | 626 | |
| 614 | − return &c->txd; | |
| 627 | + txd = &c->txd; | |
| 628 | + | |
| 629 | +err_out_not_ready: | |
| 630 | + pm_runtime_mark_last_busy(cdd->ddev.dev); | |
| 631 | + pm_runtime_put_autosuspend(cdd->ddev.dev); | |
| 632 | + | |
| 633 | + return txd; | |
| 615 | 634 | } |
| 616 | 635 | |
| 617 | 636 | static void cppi41_compute_td_desc(struct cppi41_desc *d) |
| @@ -68,6 +68,9 @@ · 67 unmodified lines | ||
| 68 | 68 | #define XILINX_DMA_DMACR_CIRC_EN BIT(1) |
| 69 | 69 | #define XILINX_DMA_DMACR_RUNSTOP BIT(0) |
| 70 | 70 | #define XILINX_DMA_DMACR_FSYNCSRC_MASK GENMASK(6, 5) |
| 71 | +#define XILINX_DMA_DMACR_DELAY_MASK GENMASK(31, 24) | |
| 72 | +#define XILINX_DMA_DMACR_FRAME_COUNT_MASK GENMASK(23, 16) | |
| 73 | +#define XILINX_DMA_DMACR_MASTER_MASK GENMASK(11, 8) | |
| 71 | 74 | |
| 72 | 75 | #define XILINX_DMA_REG_DMASR 0x0004 |
| 73 | 76 | #define XILINX_DMA_DMASR_EOL_LATE_ERR BIT(15) |
| @@ -1354,7 +1357,8 @@ static void xilinx_dma_start_transfer(struct xilinx_dma_chan *chan) · 1280 unmodified lines | ||
| 1354 | 1357 | node); |
| 1355 | 1358 | hw = &segment->hw; |
| 1356 | 1359 | |
| 1357 | − xilinx_write(chan, XILINX_DMA_REG_SRCDSTADDR, hw->buf_addr); | |
| 1360 | + xilinx_write(chan, XILINX_DMA_REG_SRCDSTADDR, | |
| 1361 | + xilinx_prep_dma_addr_t(hw->buf_addr)); | |
| 1358 | 1362 | |
| 1359 | 1363 | /* Start the transfer */ |
| 1360 | 1364 | dma_ctrl_write(chan, XILINX_DMA_REG_BTT, |
| @@ -2117,8 +2121,10 @@ int xilinx_vdma_channel_set_config(struct dma_chan *dchan, · 756 unmodified lines | ||
| 2117 | 2121 | chan->config.gen_lock = cfg->gen_lock; |
| 2118 | 2122 | chan->config.master = cfg->master; |
| 2119 | 2123 | |
| 2124 | + dmacr &= ~XILINX_DMA_DMACR_GENLOCK_EN; | |
| 2120 | 2125 | if (cfg->gen_lock && chan->genlock) { |
| 2121 | 2126 | dmacr |= XILINX_DMA_DMACR_GENLOCK_EN; |
| 2127 | + dmacr &= ~XILINX_DMA_DMACR_MASTER_MASK; | |
| 2122 | 2128 | dmacr |= cfg->master << XILINX_DMA_DMACR_MASTER_SHIFT; |
| 2123 | 2129 | } |
| 2124 | 2130 | |
| @@ -2134,11 +2140,13 @@ int xilinx_vdma_channel_set_config(struct dma_chan *dchan, · 9 unmodified lines | ||
| 2134 | 2140 | chan->config.delay = cfg->delay; |
| 2135 | 2141 | |
| 2136 | 2142 | if (cfg->coalesc <= XILINX_DMA_DMACR_FRAME_COUNT_MAX) { |
| 2143 | + dmacr &= ~XILINX_DMA_DMACR_FRAME_COUNT_MASK; | |
| 2137 | 2144 | dmacr |= cfg->coalesc << XILINX_DMA_DMACR_FRAME_COUNT_SHIFT; |
| 2138 | 2145 | chan->config.coalesc = cfg->coalesc; |
| 2139 | 2146 | } |
| 2140 | 2147 | |
| 2141 | 2148 | if (cfg->delay <= XILINX_DMA_DMACR_DELAY_MAX) { |
| 2149 | + dmacr &= ~XILINX_DMA_DMACR_DELAY_MASK; | |
| 2142 | 2150 | dmacr |= cfg->delay << XILINX_DMA_DMACR_DELAY_SHIFT; |
| 2143 | 2151 | chan->config.delay = cfg->delay; |
| 2144 | 2152 | } |
| @@ -553,7 +553,11 @@ void ghes_edac_unregister(struct ghes *ghes) · 552 unmodified lines | ||
| 553 | 553 | if (!ghes_pvt) |
| 554 | 554 | return; |
| 555 | 555 | |
| 556 | + if (atomic_dec_return(&ghes_init)) | |
| 557 | + return; | |
| 558 | + | |
| 556 | 559 | mci = ghes_pvt->mci; |
| 560 | + ghes_pvt = NULL; | |
| 557 | 561 | edac_mc_del_mc(mci->pdev); |
| 558 | 562 | edac_mc_free(mci); |
| 559 | 563 | } |
| @@ -408,7 +408,7 @@ static void __init save_mem_devices(const struct dmi_header *dm, void *v) · 407 unmodified lines | ||
| 408 | 408 | bytes = ~0ull; |
| 409 | 409 | else if (size & 0x8000) |
| 410 | 410 | bytes = (u64)(size & 0x7fff) << 10; |
| 411 | − else if (size != 0x7fff) | |
| 411 | + else if (size != 0x7fff || dm->length < 0x20) | |
| 412 | 412 | bytes = (u64)size << 20; |
| 413 | 413 | else |
| 414 | 414 | bytes = (u64)get_unaligned((u32 *)&d[0x1C]) << 20; |
| @@ -182,6 +182,7 @@ config RESET_ATTACK_MITIGATION · 181 unmodified lines | ||
| 182 | 182 | |
| 183 | 183 | config EFI_RCI2_TABLE |
| 184 | 184 | bool "EFI Runtime Configuration Interface Table Version 2 Support" |
| 185 | + depends on X86 || COMPILE_TEST | |
| 185 | 186 | help |
| 186 | 187 | Displays the content of the Runtime Configuration Interface |
| 187 | 188 | Table version 2 on Dell EMC PowerEdge systems as a binary |
| @@ -381,7 +381,7 @@ static void cper_print_pcie(const char *pfx, const struct cper_sec_pcie *pcie, · 380 unmodified lines | ||
| 381 | 381 | printk("%s""vendor_id: 0x%04x, device_id: 0x%04x\n", pfx, |
| 382 | 382 | pcie->device_id.vendor_id, pcie->device_id.device_id); |
| 383 | 383 | p = pcie->device_id.class_code; |
| 384 | − printk("%s""class_code: %02x%02x%02x\n", pfx, p[0], p[1], p[2]); | |
| 384 | + printk("%s""class_code: %02x%02x%02x\n", pfx, p[2], p[1], p[0]); | |
| 385 | 385 | } |
| 386 | 386 | if (pcie->validation_bits & CPER_PCIE_VALID_SERIAL_NUMBER) |
| 387 | 387 | printk("%s""serial number: 0x%04x, 0x%04x\n", pfx, |
| @@ -267,6 +267,9 @@ static __init int efivar_ssdt_load(void) · 266 unmodified lines | ||
| 267 | 267 | void *data; |
| 268 | 268 | int ret; |
| 269 | 269 | |
| 270 | + if (!efivar_ssdt[0]) | |
| 271 | + return 0; | |
| 272 | + | |
| 270 | 273 | ret = efivar_init(efivar_ssdt_iter, &entries, true, &entries); |
| 271 | 274 | |
| 272 | 275 | list_for_each_entry_safe(entry, aux, &entries, list) { |
| @@ -551,7 +554,7 @@ int __init efi_config_parse_tables(void *config_tables, int count, int sz, · 278 unmodified lines | ||
| 551 | 554 | sizeof(*seed) + size); |
| 552 | 555 | if (seed != NULL) { |
| 553 | 556 | pr_notice("seeding entropy pool\n"); |
| 554 | − add_device_randomness(seed->bits, seed->size); | |
| 557 | + add_bootloader_randomness(seed->bits, seed->size); | |
| 555 | 558 | early_memunmap(seed, sizeof(*seed) + size); |
| 556 | 559 | } else { |
| 557 | 560 | pr_err("Could not map UEFI random seed!\n"); |
| @@ -52,6 +52,7 @@ lib-$(CONFIG_EFI_ARMSTUB) += arm-stub.o fdt.o string.o random.o \ · 51 unmodified lines | ||
| 52 | 52 | |
| 53 | 53 | lib-$(CONFIG_ARM) += arm32-stub.o |
| 54 | 54 | lib-$(CONFIG_ARM64) += arm64-stub.o |
| 55 | +CFLAGS_arm32-stub.o := -DTEXT_OFFSET=$(TEXT_OFFSET) | |
| 55 | 56 | CFLAGS_arm64-stub.o := -DTEXT_OFFSET=$(TEXT_OFFSET) |
| 56 | 57 | |
| 57 | 58 | # |
| @@ -195,6 +195,7 @@ efi_status_t handle_kernel_image(efi_system_table_t *sys_table, · 194 unmodified lines | ||
| 195 | 195 | unsigned long dram_base, |
| 196 | 196 | efi_loaded_image_t *image) |
| 197 | 197 | { |
| 198 | + unsigned long kernel_base; | |
| 198 | 199 | efi_status_t status; |
| 199 | 200 | |
| 200 | 201 | /* |
| @@ -204,9 +205,18 @@ efi_status_t handle_kernel_image(efi_system_table_t *sys_table, · 3 unmodified lines | ||
| 204 | 205 | * loaded. These assumptions are made by the decompressor, |
| 205 | 206 | * before any memory map is available. |
| 206 | 207 | */ |
| 207 | − dram_base = round_up(dram_base, SZ_128M); | |
| 208 | + kernel_base = round_up(dram_base, SZ_128M); | |
| 208 | 209 | |
| 209 | − status = reserve_kernel_base(sys_table, dram_base, reserve_addr, | |
| 210 | + /* | |
| 211 | + * Note that some platforms (notably, the Raspberry Pi 2) put | |
| 212 | + * spin-tables and other pieces of firmware at the base of RAM, | |
| 213 | + * abusing the fact that the window of TEXT_OFFSET bytes at the | |
| 214 | + * base of the kernel image is only partially used at the moment. | |
| 215 | + * (Up to 5 pages are used for the swapper page tables) | |
| 216 | + */ | |
| 217 | + kernel_base += TEXT_OFFSET - 5 * PAGE_SIZE; | |
| 218 | + | |
| 219 | + status = reserve_kernel_base(sys_table, kernel_base, reserve_addr, | |
| 210 | 220 | reserve_size); |
| 211 | 221 | if (status != EFI_SUCCESS) { |
| 212 | 222 | pr_efi_err(sys_table, "Unable to allocate memory for uncompressed kernel.\n"); |
| @@ -220,7 +230,7 @@ efi_status_t handle_kernel_image(efi_system_table_t *sys_table, · 7 unmodified lines | ||
| 220 | 230 | *image_size = image->image_size; |
| 221 | 231 | status = efi_relocate_kernel(sys_table, image_addr, *image_size, |
| 222 | 232 | *image_size, |
| 223 | − dram_base + MAX_UNCOMP_KERNEL_SIZE, 0); | |
| 233 | + kernel_base + MAX_UNCOMP_KERNEL_SIZE, 0, 0); | |
| 224 | 234 | if (status != EFI_SUCCESS) { |
| 225 | 235 | pr_efi_err(sys_table, "Failed to relocate kernel.\n"); |
| 226 | 236 | efi_free(sys_table, *reserve_size, *reserve_addr); |
| @@ -260,11 +260,11 @@ fail: · 259 unmodified lines | ||
| 260 | 260 | } |
| 261 | 261 | |
| 262 | 262 | /* |
| 263 | − * Allocate at the lowest possible address. | |
| 263 | + * Allocate at the lowest possible address that is not below 'min'. | |
| 264 | 264 | */ |
| 265 | −efi_status_t efi_low_alloc(efi_system_table_t *sys_table_arg, | |
| 266 | − unsigned long size, unsigned long align, | |
| 267 | − unsigned long *addr) | |
| 265 | +efi_status_t efi_low_alloc_above(efi_system_table_t *sys_table_arg, | |
| 266 | + unsigned long size, unsigned long align, | |
| 267 | + unsigned long *addr, unsigned long min) | |
| 268 | 268 | { |
| 269 | 269 | unsigned long map_size, desc_size, buff_size; |
| 270 | 270 | efi_memory_desc_t *map; |
| @@ -311,13 +311,8 @@ efi_status_t efi_low_alloc(efi_system_table_t *sys_table_arg, · 40 unmodified lines | ||
| 311 | 311 | start = desc->phys_addr; |
| 312 | 312 | end = start + desc->num_pages * EFI_PAGE_SIZE; |
| 313 | 313 | |
| 314 | − /* | |
| 315 | − * Don't allocate at 0x0. It will confuse code that | |
| 316 | − * checks pointers against NULL. Skip the first 8 | |
| 317 | − * bytes so we start at a nice even number. | |
| 318 | − */ | |
| 319 | − if (start == 0x0) | |
| 320 | − start += 8; | |
| 314 | + if (start < min) | |
| 315 | + start = min; | |
| 321 | 316 | |
| 322 | 317 | start = round_up(start, align); |
| 323 | 318 | if ((start + size) > end) |
| @@ -698,7 +693,8 @@ efi_status_t efi_relocate_kernel(efi_system_table_t *sys_table_arg, · 374 unmodified lines | ||
| 698 | 693 | unsigned long image_size, |
| 699 | 694 | unsigned long alloc_size, |
| 700 | 695 | unsigned long preferred_addr, |
| 701 | − unsigned long alignment) | |
| 696 | + unsigned long alignment, | |
| 697 | + unsigned long min_addr) | |
| 702 | 698 | { |
| 703 | 699 | unsigned long cur_image_addr; |
| 704 | 700 | unsigned long new_addr = 0; |
| @@ -731,8 +727,8 @@ efi_status_t efi_relocate_kernel(efi_system_table_t *sys_table_arg, · 26 unmodified lines | ||
| 731 | 727 | * possible. |
| 732 | 728 | */ |
| 733 | 729 | if (status != EFI_SUCCESS) { |
| 734 | − status = efi_low_alloc(sys_table_arg, alloc_size, alignment, | |
| 735 | − &new_addr); | |
| 730 | + status = efi_low_alloc_above(sys_table_arg, alloc_size, | |
| 731 | + alignment, &new_addr, min_addr); | |
| 736 | 732 | } |
| 737 | 733 | if (status != EFI_SUCCESS) { |
| 738 | 734 | pr_efi_err(sys_table_arg, "Failed to allocate usable memory for kernel.\n"); |
| @@ -76,7 +76,7 @@ static u16 checksum(void) · 75 unmodified lines | ||
| 76 | 76 | return chksum; |
| 77 | 77 | } |
| 78 | 78 | |
| 79 | −int __init efi_rci2_sysfs_init(void) | |
| 79 | +static int __init efi_rci2_sysfs_init(void) | |
| 80 | 80 | { |
| 81 | 81 | struct kobject *tables_kobj; |
| 82 | 82 | int ret = -ENOMEM; |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | #include <linux/init.h> |
| 15 | 15 | #include <linux/proc_fs.h> |
| 16 | 16 | #include <linux/efi.h> |
| 17 | +#include <linux/security.h> | |
| 17 | 18 | #include <linux/slab.h> |
| 18 | 19 | #include <linux/uaccess.h> |
| 19 | 20 | |
| @@ -717,6 +718,13 @@ static long efi_test_ioctl(struct file *file, unsigned int cmd, · 697 unmodified lines | ||
| 717 | 718 | |
| 718 | 719 | static int efi_test_open(struct inode *inode, struct file *file) |
| 719 | 720 | { |
| 721 | + int ret = security_locked_down(LOCKDOWN_EFI_TEST); | |
| 722 | + | |
| 723 | + if (ret) | |
| 724 | + return ret; | |
| 725 | + | |
| 726 | + if (!capable(CAP_SYS_ADMIN)) | |
| 727 | + return -EACCES; | |
| 720 | 728 | /* |
| 721 | 729 | * nothing special to do here |
| 722 | 730 | * We do accept multiple open files at the same time as we |
| @@ -40,7 +40,7 @@ int __init efi_tpm_eventlog_init(void) · 39 unmodified lines | ||
| 40 | 40 | { |
| 41 | 41 | struct linux_efi_tpm_eventlog *log_tbl; |
| 42 | 42 | struct efi_tcg2_final_events_table *final_tbl; |
| 43 | − unsigned int tbl_size; | |
| 43 | + int tbl_size; | |
| 44 | 44 | int ret = 0; |
| 45 | 45 | |
| 46 | 46 | if (efi.tpm_log == EFI_INVALID_TABLE_ADDR) { |
| @@ -75,16 +75,29 @@ int __init efi_tpm_eventlog_init(void) · 28 unmodified lines | ||
| 75 | 75 | goto out; |
| 76 | 76 | } |
| 77 | 77 | |
| 78 | − tbl_size = tpm2_calc_event_log_size((void *)efi.tpm_final_log | |
| 79 | − + sizeof(final_tbl->version) | |
| 80 | − + sizeof(final_tbl->nr_events), | |
| 81 | − final_tbl->nr_events, | |
| 82 | − log_tbl->log); | |
| 78 | + tbl_size = 0; | |
| 79 | + if (final_tbl->nr_events != 0) { | |
| 80 | + void *events = (void *)efi.tpm_final_log | |
| 81 | + + sizeof(final_tbl->version) | |
| 82 | + + sizeof(final_tbl->nr_events); | |
| 83 | + | |
| 84 | + tbl_size = tpm2_calc_event_log_size(events, | |
| 85 | + final_tbl->nr_events, | |
| 86 | + log_tbl->log); | |
| 87 | + } | |
| 88 | + | |
| 89 | + if (tbl_size < 0) { | |
| 90 | + pr_err(FW_BUG "Failed to parse event in TPM Final Events Log\n"); | |
| 91 | + ret = -EINVAL; | |
| 92 | + goto out_calc; | |
| 93 | + } | |
| 94 | + | |
| 83 | 95 | memblock_reserve((unsigned long)final_tbl, |
| 84 | 96 | tbl_size + sizeof(*final_tbl)); |
| 85 | − early_memunmap(final_tbl, sizeof(*final_tbl)); | |
| 86 | 97 | efi_tpm_final_log_size = tbl_size; |
| 87 | 98 | |
| 99 | +out_calc: | |
| 100 | + early_memunmap(final_tbl, sizeof(*final_tbl)); | |
| 88 | 101 | out: |
| 89 | 102 | early_memunmap(log_tbl, sizeof(*log_tbl)); |
| 90 | 103 | return ret; |
| @@ -52,7 +52,7 @@ static int vpd_decode_entry(const u32 max_len, const u8 *input_buf, · 51 unmodified lines | ||
| 52 | 52 | if (max_len - consumed < *entry_len) |
| 53 | 53 | return VPD_FAIL; |
| 54 | 54 | |
| 55 | − consumed += decoded_len; | |
| 55 | + consumed += *entry_len; | |
| 56 | 56 | *_consumed = consumed; |
| 57 | 57 | return VPD_OK; |
| 58 | 58 | } |
| @@ -530,11 +530,12 @@ static void sprd_eic_handle_one_type(struct gpio_chip *chip) · 529 unmodified lines | ||
| 530 | 530 | } |
| 531 | 531 | |
| 532 | 532 | for_each_set_bit(n, ®, SPRD_EIC_PER_BANK_NR) { |
| 533 | − girq = irq_find_mapping(chip->irq.domain, | |
| 534 | − bank * SPRD_EIC_PER_BANK_NR + n); | |
| 533 | + u32 offset = bank * SPRD_EIC_PER_BANK_NR + n; | |
| 534 | + | |
| 535 | + girq = irq_find_mapping(chip->irq.domain, offset); | |
| 535 | 536 | |
| 536 | 537 | generic_handle_irq(girq); |
| 537 | − sprd_eic_toggle_trigger(chip, girq, n); | |
| 538 | + sprd_eic_toggle_trigger(chip, girq, offset); | |
| 538 | 539 | } |
| 539 | 540 | } |
| 540 | 541 | } |
| @@ -293,8 +293,9 @@ static void intel_mid_irq_handler(struct irq_desc *desc) · 292 unmodified lines | ||
| 293 | 293 | chip->irq_eoi(data); |
| 294 | 294 | } |
| 295 | 295 | |
| 296 | −static void intel_mid_irq_init_hw(struct intel_mid_gpio *priv) | |
| 296 | +static int intel_mid_irq_init_hw(struct gpio_chip *chip) | |
| 297 | 297 | { |
| 298 | + struct intel_mid_gpio *priv = gpiochip_get_data(chip); | |
| 298 | 299 | void __iomem *reg; |
| 299 | 300 | unsigned base; |
| 300 | 301 | |
| @@ -309,6 +310,8 @@ static void intel_mid_irq_init_hw(struct intel_mid_gpio *priv) · 8 unmodified lines | ||
| 309 | 310 | reg = gpio_reg(&priv->chip, base, GEDR); |
| 310 | 311 | writel(~0, reg); |
| 311 | 312 | } |
| 313 | + | |
| 314 | + return 0; | |
| 312 | 315 | } |
| 313 | 316 | |
| 314 | 317 | static int __maybe_unused intel_gpio_runtime_idle(struct device *dev) |
| @@ -372,6 +375,7 @@ static int intel_gpio_probe(struct pci_dev *pdev, · 57 unmodified lines | ||
| 372 | 375 | |
| 373 | 376 | girq = &priv->chip.irq; |
| 374 | 377 | girq->chip = &intel_mid_irqchip; |
| 378 | + girq->init_hw = intel_mid_irq_init_hw; | |
| 375 | 379 | girq->parent_handler = intel_mid_irq_handler; |
| 376 | 380 | girq->num_parents = 1; |
| 377 | 381 | girq->parents = devm_kcalloc(&pdev->dev, girq->num_parents, |
| @@ -384,9 +388,8 @@ static int intel_gpio_probe(struct pci_dev *pdev, · 6 unmodified lines | ||
| 384 | 388 | girq->default_type = IRQ_TYPE_NONE; |
| 385 | 389 | girq->handler = handle_simple_irq; |
| 386 | 390 | |
| 387 | − intel_mid_irq_init_hw(priv); | |
| 388 | − | |
| 389 | 391 | pci_set_drvdata(pdev, priv); |
| 392 | + | |
| 390 | 393 | retval = devm_gpiochip_add_data(&pdev->dev, &priv->chip, priv); |
| 391 | 394 | if (retval) { |
| 392 | 395 | dev_err(&pdev->dev, "gpiochip_add error %d\n", retval); |
| @@ -294,8 +294,9 @@ static struct irq_chip lp_irqchip = { · 293 unmodified lines | ||
| 294 | 294 | .flags = IRQCHIP_SKIP_SET_WAKE, |
| 295 | 295 | }; |
| 296 | 296 | |
| 297 | −static void lp_gpio_irq_init_hw(struct lp_gpio *lg) | |
| 297 | +static int lp_gpio_irq_init_hw(struct gpio_chip *chip) | |
| 298 | 298 | { |
| 299 | + struct lp_gpio *lg = gpiochip_get_data(chip); | |
| 299 | 300 | unsigned long reg; |
| 300 | 301 | unsigned base; |
| 301 | 302 | |
| @@ -307,6 +308,8 @@ static void lp_gpio_irq_init_hw(struct lp_gpio *lg) · 5 unmodified lines | ||
| 307 | 308 | reg = lp_gpio_reg(&lg->chip, base, LP_INT_STAT); |
| 308 | 309 | outl(0xffffffff, reg); |
| 309 | 310 | } |
| 311 | + | |
| 312 | + return 0; | |
| 310 | 313 | } |
| 311 | 314 | |
| 312 | 315 | static int lp_gpio_probe(struct platform_device *pdev) |
| @@ -364,6 +367,7 @@ static int lp_gpio_probe(struct platform_device *pdev) · 51 unmodified lines | ||
| 364 | 367 | |
| 365 | 368 | girq = &gc->irq; |
| 366 | 369 | girq->chip = &lp_irqchip; |
| 370 | + girq->init_hw = lp_gpio_irq_init_hw; | |
| 367 | 371 | girq->parent_handler = lp_gpio_irq_handler; |
| 368 | 372 | girq->num_parents = 1; |
| 369 | 373 | girq->parents = devm_kcalloc(&pdev->dev, girq->num_parents, |
| @@ -373,9 +377,7 @@ static int lp_gpio_probe(struct platform_device *pdev) · 3 unmodified lines | ||
| 373 | 377 | return -ENOMEM; |
| 374 | 378 | girq->parents[0] = (unsigned)irq_rc->start; |
| 375 | 379 | girq->default_type = IRQ_TYPE_NONE; |
| 376 | − girq->handler = handle_simple_irq; | |
| 377 | − | |
| 378 | − lp_gpio_irq_init_hw(lg); | |
| 380 | + girq->handler = handle_bad_irq; | |
| 379 | 381 | } |
| 380 | 382 | |
| 381 | 383 | ret = devm_gpiochip_add_data(dev, gc, lg); |
| @@ -192,13 +192,13 @@ static int max77620_gpio_set_debounce(struct max77620_gpio *mgpio, · 191 unmodified lines | ||
| 192 | 192 | case 0: |
| 193 | 193 | val = MAX77620_CNFG_GPIO_DBNC_None; |
| 194 | 194 | break; |
| 195 | − case 1 ... 8: | |
| 195 | + case 1000 ... 8000: | |
| 196 | 196 | val = MAX77620_CNFG_GPIO_DBNC_8ms; |
| 197 | 197 | break; |
| 198 | − case 9 ... 16: | |
| 198 | + case 9000 ... 16000: | |
| 199 | 199 | val = MAX77620_CNFG_GPIO_DBNC_16ms; |
| 200 | 200 | break; |
| 201 | − case 17 ... 32: | |
| 201 | + case 17000 ... 32000: | |
| 202 | 202 | val = MAX77620_CNFG_GPIO_DBNC_32ms; |
| 203 | 203 | break; |
| 204 | 204 | default: |
| @@ -397,7 +397,6 @@ static int mrfld_gpio_probe(struct pci_dev *pdev, const struct pci_device_id *id · 396 unmodified lines | ||
| 397 | 397 | { |
| 398 | 398 | const struct mrfld_gpio_pinrange *range; |
| 399 | 399 | const char *pinctrl_dev_name; |
| 400 | − struct gpio_irq_chip *girq; | |
| 401 | 400 | struct mrfld_gpio *priv; |
| 402 | 401 | u32 gpio_base, irq_base; |
| 403 | 402 | void __iomem *base; |
| @@ -445,21 +444,6 @@ static int mrfld_gpio_probe(struct pci_dev *pdev, const struct pci_device_id *id · 41 unmodified lines | ||
| 445 | 444 | |
| 446 | 445 | raw_spin_lock_init(&priv->lock); |
| 447 | 446 | |
| 448 | − girq = &priv->chip.irq; | |
| 449 | − girq->chip = &mrfld_irqchip; | |
| 450 | − girq->parent_handler = mrfld_irq_handler; | |
| 451 | − girq->num_parents = 1; | |
| 452 | − girq->parents = devm_kcalloc(&pdev->dev, girq->num_parents, | |
| 453 | − sizeof(*girq->parents), | |
| 454 | − GFP_KERNEL); | |
| 455 | − if (!girq->parents) | |
| 456 | − return -ENOMEM; | |
| 457 | − girq->parents[0] = pdev->irq; | |
| 458 | − girq->default_type = IRQ_TYPE_NONE; | |
| 459 | − girq->handler = handle_bad_irq; | |
| 460 | − | |
| 461 | − mrfld_irq_init_hw(priv); | |
| 462 | − | |
| 463 | 447 | pci_set_drvdata(pdev, priv); |
| 464 | 448 | retval = devm_gpiochip_add_data(&pdev->dev, &priv->chip, priv); |
| 465 | 449 | if (retval) { |
| @@ -481,6 +465,18 @@ static int mrfld_gpio_probe(struct pci_dev *pdev, const struct pci_device_id *id · 15 unmodified lines | ||
| 481 | 465 | } |
| 482 | 466 | } |
| 483 | 467 | |
| 468 | + retval = gpiochip_irqchip_add(&priv->chip, &mrfld_irqchip, irq_base, | |
| 469 | + handle_bad_irq, IRQ_TYPE_NONE); | |
| 470 | + if (retval) { | |
| 471 | + dev_err(&pdev->dev, "could not connect irqchip to gpiochip\n"); | |
| 472 | + return retval; | |
| 473 | + } | |
| 474 | + | |
| 475 | + mrfld_irq_init_hw(priv); | |
| 476 | + | |
| 477 | + gpiochip_set_chained_irqchip(&priv->chip, &mrfld_irqchip, pdev->irq, | |
| 478 | + mrfld_irq_handler); | |
| 479 | + | |
| 484 | 480 | return 0; |
| 485 | 481 | } |
| 486 | 482 | |
| @@ -317,7 +317,7 @@ struct gpio_desc *gpiod_get_from_of_node(struct device_node *node, · 316 unmodified lines | ||
| 317 | 317 | transitory = flags & OF_GPIO_TRANSITORY; |
| 318 | 318 | |
| 319 | 319 | ret = gpiod_request(desc, label); |
| 320 | − if (ret == -EBUSY && (flags & GPIOD_FLAGS_BIT_NONEXCLUSIVE)) | |
| 320 | + if (ret == -EBUSY && (dflags & GPIOD_FLAGS_BIT_NONEXCLUSIVE)) | |
| 321 | 321 | return desc; |
| 322 | 322 | if (ret) |
| 323 | 323 | return ERR_PTR(ret); |
| @@ -86,6 +86,7 @@ static int gpiochip_add_irqchip(struct gpio_chip *gpiochip, · 85 unmodified lines | ||
| 86 | 86 | struct lock_class_key *lock_key, |
| 87 | 87 | struct lock_class_key *request_key); |
| 88 | 88 | static void gpiochip_irqchip_remove(struct gpio_chip *gpiochip); |
| 89 | +static int gpiochip_irqchip_init_hw(struct gpio_chip *gpiochip); | |
| 89 | 90 | static int gpiochip_irqchip_init_valid_mask(struct gpio_chip *gpiochip); |
| 90 | 91 | static void gpiochip_irqchip_free_valid_mask(struct gpio_chip *gpiochip); |
| 91 | 92 | |
| @@ -1406,6 +1407,10 @@ int gpiochip_add_data_with_key(struct gpio_chip *chip, void *data, · 1314 unmodified lines | ||
| 1406 | 1407 | |
| 1407 | 1408 | machine_gpiochip_add(chip); |
| 1408 | 1409 | |
| 1410 | + ret = gpiochip_irqchip_init_hw(chip); | |
| 1411 | + if (ret) | |
| 1412 | + goto err_remove_acpi_chip; | |
| 1413 | + | |
| 1409 | 1414 | ret = gpiochip_irqchip_init_valid_mask(chip); |
| 1410 | 1415 | if (ret) |
| 1411 | 1416 | goto err_remove_acpi_chip; |
| @@ -1622,6 +1627,16 @@ static struct gpio_chip *find_chip_by_name(const char *name) · 210 unmodified lines | ||
| 1622 | 1627 | * The following is irqchip helper code for gpiochips. |
| 1623 | 1628 | */ |
| 1624 | 1629 | |
| 1630 | +static int gpiochip_irqchip_init_hw(struct gpio_chip *gc) | |
| 1631 | +{ | |
| 1632 | + struct gpio_irq_chip *girq = &gc->irq; | |
| 1633 | + | |
| 1634 | + if (!girq->init_hw) | |
| 1635 | + return 0; | |
| 1636 | + | |
| 1637 | + return girq->init_hw(gc); | |
| 1638 | +} | |
| 1639 | + | |
| 1625 | 1640 | static int gpiochip_irqchip_init_valid_mask(struct gpio_chip *gc) |
| 1626 | 1641 | { |
| 1627 | 1642 | struct gpio_irq_chip *girq = &gc->irq; |
| @@ -2446,8 +2461,13 @@ static inline int gpiochip_add_irqchip(struct gpio_chip *gpiochip, · 818 unmodified lines | ||
| 2446 | 2461 | { |
| 2447 | 2462 | return 0; |
| 2448 | 2463 | } |
| 2449 | − | |
| 2450 | 2464 | static void gpiochip_irqchip_remove(struct gpio_chip *gpiochip) {} |
| 2465 | + | |
| 2466 | +static inline int gpiochip_irqchip_init_hw(struct gpio_chip *gpiochip) | |
| 2467 | +{ | |
| 2468 | + return 0; | |
| 2469 | +} | |
| 2470 | + | |
| 2451 | 2471 | static inline int gpiochip_irqchip_init_valid_mask(struct gpio_chip *gpiochip) |
| 2452 | 2472 | { |
| 2453 | 2473 | return 0; |
| @@ -3070,8 +3090,10 @@ int gpiod_direction_output(struct gpio_desc *desc, int value) · 616 unmodified lines | ||
| 3070 | 3090 | if (!ret) |
| 3071 | 3091 | goto set_output_value; |
| 3072 | 3092 | /* Emulate open drain by not actively driving the line high */ |
| 3073 | − if (value) | |
| 3074 | − return gpiod_direction_input(desc); | |
| 3093 | + if (value) { | |
| 3094 | + ret = gpiod_direction_input(desc); | |
| 3095 | + goto set_output_flag; | |
| 3096 | + } | |
| 3075 | 3097 | } |
| 3076 | 3098 | else if (test_bit(FLAG_OPEN_SOURCE, &desc->flags)) { |
| 3077 | 3099 | ret = gpio_set_config(gc, gpio_chip_hwgpio(desc), |
| @@ -3079,8 +3101,10 @@ int gpiod_direction_output(struct gpio_desc *desc, int value) · 1 unmodified line | ||
| 3079 | 3101 | if (!ret) |
| 3080 | 3102 | goto set_output_value; |
| 3081 | 3103 | /* Emulate open source by not actively driving the line low */ |
| 3082 | − if (!value) | |
| 3083 | − return gpiod_direction_input(desc); | |
| 3104 | + if (!value) { | |
| 3105 | + ret = gpiod_direction_input(desc); | |
| 3106 | + goto set_output_flag; | |
| 3107 | + } | |
| 3084 | 3108 | } else { |
| 3085 | 3109 | gpio_set_config(gc, gpio_chip_hwgpio(desc), |
| 3086 | 3110 | PIN_CONFIG_DRIVE_PUSH_PULL); |
| @@ -3088,6 +3112,17 @@ int gpiod_direction_output(struct gpio_desc *desc, int value) · 1 unmodified line | ||
| 3088 | 3112 | |
| 3089 | 3113 | set_output_value: |
| 3090 | 3114 | return gpiod_direction_output_raw_commit(desc, value); |
| 3115 | + | |
| 3116 | +set_output_flag: | |
| 3117 | + /* | |
| 3118 | + * When emulating open-source or open-drain functionalities by not | |
| 3119 | + * actively driving the line (setting mode to input) we still need to | |
| 3120 | + * set the IS_OUT flag or otherwise we won't be able to set the line | |
| 3121 | + * value anymore. | |
| 3122 | + */ | |
| 3123 | + if (ret == 0) | |
| 3124 | + set_bit(FLAG_IS_OUT, &desc->flags); | |
| 3125 | + return ret; | |
| 3091 | 3126 | } |
| 3092 | 3127 | EXPORT_SYMBOL_GPL(gpiod_direction_output); |
| 3093 | 3128 | |
| @@ -3448,8 +3483,6 @@ static void gpio_set_open_drain_value_commit(struct gpio_desc *desc, bool value) · 354 unmodified lines | ||
| 3448 | 3483 | |
| 3449 | 3484 | if (value) { |
| 3450 | 3485 | ret = chip->direction_input(chip, offset); |
| 3451 | − if (!ret) | |
| 3452 | − clear_bit(FLAG_IS_OUT, &desc->flags); | |
| 3453 | 3486 | } else { |
| 3454 | 3487 | ret = chip->direction_output(chip, offset, 0); |
| 3455 | 3488 | if (!ret) |
| @@ -3479,8 +3512,6 @@ static void gpio_set_open_source_value_commit(struct gpio_desc *desc, bool value · 23 unmodified lines | ||
| 3479 | 3512 | set_bit(FLAG_IS_OUT, &desc->flags); |
| 3480 | 3513 | } else { |
| 3481 | 3514 | ret = chip->direction_input(chip, offset); |
| 3482 | − if (!ret) | |
| 3483 | − clear_bit(FLAG_IS_OUT, &desc->flags); | |
| 3484 | 3515 | } |
| 3485 | 3516 | trace_gpio_direction(desc_to_gpio(desc), !value, ret); |
| 3486 | 3517 | if (ret < 0) |
| @@ -97,6 +97,20 @@ config DRM_KMS_FB_HELPER · 96 unmodified lines | ||
| 97 | 97 | help |
| 98 | 98 | FBDEV helpers for KMS drivers. |
| 99 | 99 | |
| 100 | +config DRM_DEBUG_DP_MST_TOPOLOGY_REFS | |
| 101 | + bool "Enable refcount backtrace history in the DP MST helpers" | |
| 102 | + select STACKDEPOT | |
| 103 | + depends on DRM_KMS_HELPER | |
| 104 | + depends on DEBUG_KERNEL | |
| 105 | + depends on EXPERT | |
| 106 | + help | |
| 107 | + Enables debug tracing for topology refs in DRM's DP MST helpers. A | |
| 108 | + history of each topology reference/dereference will be printed to the | |
| 109 | + kernel log once a port or branch device's topology refcount reaches 0. | |
| 110 | + | |
| 111 | + This has the potential to use a lot of memory and print some very | |
| 112 | + large kernel messages. If in doubt, say "N". | |
| 113 | + | |
| 100 | 114 | config DRM_FBDEV_EMULATION |
| 101 | 115 | bool "Enable legacy fbdev support for your modesetting driver" |
| 102 | 116 | depends on DRM |
| @@ -169,6 +183,13 @@ config DRM_TTM · 66 unmodified lines | ||
| 169 | 183 | GPU memory types. Will be enabled automatically if a device driver |
| 170 | 184 | uses it. |
| 171 | 185 | |
| 186 | +config DRM_TTM_DMA_PAGE_POOL | |
| 187 | + bool | |
| 188 | + depends on DRM_TTM && (SWIOTLB || INTEL_IOMMU) | |
| 189 | + default y | |
| 190 | + help | |
| 191 | + Choose this if you need the TTM dma page pool | |
| 192 | + | |
| 172 | 193 | config DRM_VRAM_HELPER |
| 173 | 194 | tristate |
| 174 | 195 | depends on DRM |
| @@ -236,9 +257,9 @@ config DRM_AMDGPU · 61 unmodified lines | ||
| 236 | 257 | tristate "AMD GPU" |
| 237 | 258 | depends on DRM && PCI && MMU |
| 238 | 259 | select FW_LOADER |
| 239 | − select DRM_KMS_HELPER | |
| 260 | + select DRM_KMS_HELPER | |
| 240 | 261 | select DRM_SCHED |
| 241 | − select DRM_TTM | |
| 262 | + select DRM_TTM | |
| 242 | 263 | select POWER_SUPPLY |
| 243 | 264 | select HWMON |
| 244 | 265 | select BACKLIGHT_CLASS_DEVICE |
| @@ -267,6 +288,7 @@ config DRM_VKMS · 22 unmodified lines | ||
| 267 | 288 | tristate "Virtual KMS (EXPERIMENTAL)" |
| 268 | 289 | depends on DRM |
| 269 | 290 | select DRM_KMS_HELPER |
| 291 | + select CRC32 | |
| 270 | 292 | default n |
| 271 | 293 | help |
| 272 | 294 | Virtual Kernel Mode-Setting (VKMS) is used for testing or for |
| @@ -407,7 +429,7 @@ config DRM_R128 · 134 unmodified lines | ||
| 407 | 429 | |
| 408 | 430 | config DRM_I810 |
| 409 | 431 | tristate "Intel I810" |
| 410 | − # !PREEMPT because of missing ioctl locking | |
| 432 | + # !PREEMPTION because of missing ioctl locking | |
| 411 | 433 | depends on DRM && AGP && AGP_INTEL && (!PREEMPTION || BROKEN) |
| 412 | 434 | help |
| 413 | 435 | Choose this option if you have an Intel I810 graphics card. If M is |
| @@ -53,8 +53,9 @@ amdgpu-y += amdgpu_device.o amdgpu_kms.o \ · 52 unmodified lines | ||
| 53 | 53 | amdgpu_ucode.o amdgpu_bo_list.o amdgpu_ctx.o amdgpu_sync.o \ |
| 54 | 54 | amdgpu_gtt_mgr.o amdgpu_vram_mgr.o amdgpu_virt.o amdgpu_atomfirmware.o \ |
| 55 | 55 | amdgpu_vf_error.o amdgpu_sched.o amdgpu_debugfs.o amdgpu_ids.o \ |
| 56 | − amdgpu_gmc.o amdgpu_xgmi.o amdgpu_csa.o amdgpu_ras.o amdgpu_vm_cpu.o \ | |
| 57 | − amdgpu_vm_sdma.o amdgpu_discovery.o amdgpu_ras_eeprom.o smu_v11_0_i2c.o | |
| 56 | + amdgpu_gmc.o amdgpu_mmhub.o amdgpu_xgmi.o amdgpu_csa.o amdgpu_ras.o amdgpu_vm_cpu.o \ | |
| 57 | + amdgpu_vm_sdma.o amdgpu_discovery.o amdgpu_ras_eeprom.o amdgpu_nbio.o \ | |
| 58 | + amdgpu_umc.o smu_v11_0_i2c.o | |
| 58 | 59 | |
| 59 | 60 | amdgpu-$(CONFIG_PERF_EVENTS) += amdgpu_pmu.o |
| 60 | 61 | |
| @@ -67,7 +68,7 @@ amdgpu-$(CONFIG_DRM_AMDGPU_SI)+= si.o gmc_v6_0.o gfx_v6_0.o si_ih.o si_dma.o dce · 6 unmodified lines | ||
| 67 | 68 | amdgpu-y += \ |
| 68 | 69 | vi.o mxgpu_vi.o nbio_v6_1.o soc15.o emu_soc.o mxgpu_ai.o nbio_v7_0.o vega10_reg_init.o \ |
| 69 | 70 | vega20_reg_init.o nbio_v7_4.o nbio_v2_3.o nv.o navi10_reg_init.o navi14_reg_init.o \ |
| 70 | − arct_reg_init.o navi12_reg_init.o | |
| 71 | + arct_reg_init.o navi12_reg_init.o mxgpu_nv.o | |
| 71 | 72 | |
| 72 | 73 | # add DF block |
| 73 | 74 | amdgpu-y += \ |
| @@ -83,7 +84,7 @@ amdgpu-y += \ · 9 unmodified lines | ||
| 83 | 84 | |
| 84 | 85 | # add UMC block |
| 85 | 86 | amdgpu-y += \ |
| 86 | − umc_v6_1.o | |
| 87 | + umc_v6_1.o umc_v6_0.o | |
| 87 | 88 | |
| 88 | 89 | # add IH block |
| 89 | 90 | amdgpu-y += \ |
| @@ -73,6 +73,7 @@ · 72 unmodified lines | ||
| 73 | 73 | #include "amdgpu_gmc.h" |
| 74 | 74 | #include "amdgpu_gfx.h" |
| 75 | 75 | #include "amdgpu_sdma.h" |
| 76 | +#include "amdgpu_nbio.h" | |
| 76 | 77 | #include "amdgpu_dm.h" |
| 77 | 78 | #include "amdgpu_virt.h" |
| 78 | 79 | #include "amdgpu_csa.h" |
| @@ -106,6 +107,8 @@ struct amdgpu_mgpu_info · 27 unmodified lines | ||
| 106 | 107 | uint32_t num_apu; |
| 107 | 108 | }; |
| 108 | 109 | |
| 110 | +#define AMDGPU_MAX_TIMEOUT_PARAM_LENGTH 256 | |
| 111 | + | |
| 109 | 112 | /* |
| 110 | 113 | * Modules parameters. |
| 111 | 114 | */ |
| @@ -122,6 +125,7 @@ extern int amdgpu_disp_priority; · 10 unmodified lines | ||
| 122 | 125 | extern int amdgpu_hw_i2c; |
| 123 | 126 | extern int amdgpu_pcie_gen2; |
| 124 | 127 | extern int amdgpu_msi; |
| 128 | +extern char amdgpu_lockup_timeout[AMDGPU_MAX_TIMEOUT_PARAM_LENGTH]; | |
| 125 | 129 | extern int amdgpu_dpm; |
| 126 | 130 | extern int amdgpu_fw_load_type; |
| 127 | 131 | extern int amdgpu_aspm; |
| @@ -135,6 +139,7 @@ extern int amdgpu_vm_fragment_size; · 7 unmodified lines | ||
| 135 | 139 | extern int amdgpu_vm_fault_stop; |
| 136 | 140 | extern int amdgpu_vm_debug; |
| 137 | 141 | extern int amdgpu_vm_update_mode; |
| 142 | +extern int amdgpu_exp_hw_support; | |
| 138 | 143 | extern int amdgpu_dc; |
| 139 | 144 | extern int amdgpu_sched_jobs; |
| 140 | 145 | extern int amdgpu_sched_hw_submission; |
| @@ -146,11 +151,7 @@ extern uint amdgpu_sdma_phase_quantum; · 5 unmodified lines | ||
| 146 | 151 | extern char *amdgpu_disable_cu; |
| 147 | 152 | extern char *amdgpu_virtual_display; |
| 148 | 153 | extern uint amdgpu_pp_feature_mask; |
| 149 | −extern int amdgpu_ngg; | |
| 150 | −extern int amdgpu_prim_buf_per_se; | |
| 151 | −extern int amdgpu_pos_buf_per_se; | |
| 152 | −extern int amdgpu_cntl_sb_buf_per_se; | |
| 153 | −extern int amdgpu_param_buf_per_se; | |
| 154 | +extern uint amdgpu_force_long_training; | |
| 154 | 155 | extern int amdgpu_job_hang_limit; |
| 155 | 156 | extern int amdgpu_lbpw; |
| 156 | 157 | extern int amdgpu_compute_multipipe; |
| @@ -167,6 +168,12 @@ extern int amdgpu_mcbp; · 10 unmodified lines | ||
| 167 | 168 | extern int amdgpu_discovery; |
| 168 | 169 | extern int amdgpu_mes; |
| 169 | 170 | extern int amdgpu_noretry; |
| 171 | +extern int amdgpu_force_asic_type; | |
| 172 | +#ifdef CONFIG_HSA_AMD | |
| 173 | +extern int sched_policy; | |
| 174 | +#else | |
| 175 | +static const int sched_policy = KFD_SCHED_POLICY_HWS; | |
| 176 | +#endif | |
| 170 | 177 | |
| 171 | 178 | #ifdef CONFIG_DRM_AMDGPU_SI |
| 172 | 179 | extern int amdgpu_si_support; |
| @@ -283,6 +290,9 @@ struct amdgpu_ip_block_version { · 110 unmodified lines | ||
| 283 | 290 | const struct amd_ip_funcs *funcs; |
| 284 | 291 | }; |
| 285 | 292 | |
| 293 | +#define HW_REV(_Major, _Minor, _Rev) \ | |
| 294 | + ((((uint32_t) (_Major)) << 16) | ((uint32_t) (_Minor) << 8) | ((uint32_t) (_Rev))) | |
| 295 | + | |
| 286 | 296 | struct amdgpu_ip_block { |
| 287 | 297 | struct amdgpu_ip_block_status status; |
| 288 | 298 | const struct amdgpu_ip_block_version *version; |
| @@ -425,7 +435,6 @@ struct amdgpu_fpriv { · 136 unmodified lines | ||
| 425 | 435 | }; |
| 426 | 436 | |
| 427 | 437 | int amdgpu_file_to_fpriv(struct file *filp, struct amdgpu_fpriv **fpriv); |
| 428 | −int amdgpu_device_get_job_timeout_settings(struct amdgpu_device *adev); | |
| 429 | 438 | |
| 430 | 439 | int amdgpu_ib_get(struct amdgpu_device *adev, struct amdgpu_vm *vm, |
| 431 | 440 | unsigned size, struct amdgpu_ib *ib); |
| @@ -477,7 +486,6 @@ struct amdgpu_cs_parser { · 45 unmodified lines | ||
| 477 | 486 | uint64_t bytes_moved_vis_threshold; |
| 478 | 487 | uint64_t bytes_moved; |
| 479 | 488 | uint64_t bytes_moved_vis; |
| 480 | − struct amdgpu_bo_list_entry *evictable; | |
| 481 | 489 | |
| 482 | 490 | /* user fence */ |
| 483 | 491 | struct amdgpu_bo_list_entry uf_entry; |
| @@ -624,6 +632,11 @@ struct amdgpu_fw_vram_usage { · 140 unmodified lines | ||
| 624 | 632 | u64 size; |
| 625 | 633 | struct amdgpu_bo *reserved_bo; |
| 626 | 634 | void *va; |
| 635 | + | |
| 636 | + /* Offset on the top of VRAM, used as c2p write buffer. | |
| 637 | + */ | |
| 638 | + u64 mem_train_fb_loc; | |
| 639 | + bool mem_train_support; | |
| 627 | 640 | }; |
| 628 | 641 | |
| 629 | 642 | /* |
| @@ -644,71 +657,14 @@ typedef void (*amdgpu_wreg64_t)(struct amdgpu_device*, uint32_t, uint64_t); · 14 unmodified lines | ||
| 644 | 657 | typedef uint32_t (*amdgpu_block_rreg_t)(struct amdgpu_device*, uint32_t, uint32_t); |
| 645 | 658 | typedef void (*amdgpu_block_wreg_t)(struct amdgpu_device*, uint32_t, uint32_t, uint32_t); |
| 646 | 659 | |
| 647 | − | |
| 648 | −/* | |
| 649 | − * amdgpu nbio functions | |
| 650 | − * | |
| 651 | − */ | |
| 652 | −struct nbio_hdp_flush_reg { | |
| 653 | − u32 ref_and_mask_cp0; | |
| 654 | − u32 ref_and_mask_cp1; | |
| 655 | − u32 ref_and_mask_cp2; | |
| 656 | − u32 ref_and_mask_cp3; | |
| 657 | − u32 ref_and_mask_cp4; | |
| 658 | − u32 ref_and_mask_cp5; | |
| 659 | − u32 ref_and_mask_cp6; | |
| 660 | − u32 ref_and_mask_cp7; | |
| 661 | − u32 ref_and_mask_cp8; | |
| 662 | − u32 ref_and_mask_cp9; | |
| 663 | − u32 ref_and_mask_sdma0; | |
| 664 | − u32 ref_and_mask_sdma1; | |
| 665 | − u32 ref_and_mask_sdma2; | |
| 666 | − u32 ref_and_mask_sdma3; | |
| 667 | − u32 ref_and_mask_sdma4; | |
| 668 | − u32 ref_and_mask_sdma5; | |
| 669 | − u32 ref_and_mask_sdma6; | |
| 670 | − u32 ref_and_mask_sdma7; | |
| 671 | −}; | |
| 672 | − | |
| 673 | 660 | struct amdgpu_mmio_remap { |
| 674 | 661 | u32 reg_offset; |
| 675 | 662 | resource_size_t bus_addr; |
| 676 | 663 | }; |
| 677 | 664 | |
| 678 | −struct amdgpu_nbio_funcs { | |
| 679 | − const struct nbio_hdp_flush_reg *hdp_flush_reg; | |
| 680 | − u32 (*get_hdp_flush_req_offset)(struct amdgpu_device *adev); | |
| 681 | − u32 (*get_hdp_flush_done_offset)(struct amdgpu_device *adev); | |
| 682 | − u32 (*get_pcie_index_offset)(struct amdgpu_device *adev); | |
| 683 | − u32 (*get_pcie_data_offset)(struct amdgpu_device *adev); | |
| 684 | − u32 (*get_rev_id)(struct amdgpu_device *adev); | |
| 685 | − void (*mc_access_enable)(struct amdgpu_device *adev, bool enable); | |
| 686 | − void (*hdp_flush)(struct amdgpu_device *adev, struct amdgpu_ring *ring); | |
| 687 | − u32 (*get_memsize)(struct amdgpu_device *adev); | |
| 688 | − void (*sdma_doorbell_range)(struct amdgpu_device *adev, int instance, | |
| 689 | − bool use_doorbell, int doorbell_index, int doorbell_size); | |
| 690 | − void (*vcn_doorbell_range)(struct amdgpu_device *adev, bool use_doorbell, | |
| 691 | − int doorbell_index, int instance); | |
| 692 | − void (*enable_doorbell_aperture)(struct amdgpu_device *adev, | |
| 693 | − bool enable); | |
| 694 | − void (*enable_doorbell_selfring_aperture)(struct amdgpu_device *adev, | |
| 695 | − bool enable); | |
| 696 | − void (*ih_doorbell_range)(struct amdgpu_device *adev, | |
| 697 | − bool use_doorbell, int doorbell_index); | |
| 698 | − void (*update_medium_grain_clock_gating)(struct amdgpu_device *adev, | |
| 699 | − bool enable); | |
| 700 | − void (*update_medium_grain_light_sleep)(struct amdgpu_device *adev, | |
| 701 | − bool enable); | |
| 702 | − void (*get_clockgating_state)(struct amdgpu_device *adev, | |
| 703 | − u32 *flags); | |
| 704 | − void (*ih_control)(struct amdgpu_device *adev); | |
| 705 | − void (*init_registers)(struct amdgpu_device *adev); | |
| 706 | − void (*detect_hw_virt)(struct amdgpu_device *adev); | |
| 707 | − void (*remap_hdp_registers)(struct amdgpu_device *adev); | |
| 708 | −}; | |
| 709 | − | |
| 710 | 665 | struct amdgpu_df_funcs { |
| 711 | 666 | void (*sw_init)(struct amdgpu_device *adev); |
| 667 | + void (*sw_fini)(struct amdgpu_device *adev); | |
| 712 | 668 | void (*enable_broadcast_mode)(struct amdgpu_device *adev, |
| 713 | 669 | bool enable); |
| 714 | 670 | u32 (*get_fb_channel_number)(struct amdgpu_device *adev); |
| @@ -813,6 +769,7 @@ struct amdgpu_device { · 98 unmodified lines | ||
| 813 | 769 | uint8_t *bios; |
| 814 | 770 | uint32_t bios_size; |
| 815 | 771 | struct amdgpu_bo *stolen_vga_memory; |
| 772 | + struct amdgpu_bo *discovery_memory; | |
| 816 | 773 | uint32_t bios_scratch_reg_offset; |
| 817 | 774 | uint32_t bios_scratch[AMDGPU_BIOS_NUM_SCRATCH]; |
| 818 | 775 | |
| @@ -921,6 +878,12 @@ struct amdgpu_device { · 102 unmodified lines | ||
| 921 | 878 | u32 cg_flags; |
| 922 | 879 | u32 pg_flags; |
| 923 | 880 | |
| 881 | + /* nbio */ | |
| 882 | + struct amdgpu_nbio nbio; | |
| 883 | + | |
| 884 | + /* mmhub */ | |
| 885 | + struct amdgpu_mmhub mmhub; | |
| 886 | + | |
| 924 | 887 | /* gfx */ |
| 925 | 888 | struct amdgpu_gfx gfx; |
| 926 | 889 | |
| @@ -974,9 +937,7 @@ struct amdgpu_device { · 47 unmodified lines | ||
| 974 | 937 | /* soc15 register offset based on ip, instance and segment */ |
| 975 | 938 | uint32_t *reg_offset[MAX_HWIP][HWIP_MAX_INSTANCE]; |
| 976 | 939 | |
| 977 | − const struct amdgpu_nbio_funcs *nbio_funcs; | |
| 978 | 940 | const struct amdgpu_df_funcs *df_funcs; |
| 979 | − const struct amdgpu_mmhub_funcs *mmhub_funcs; | |
| 980 | 941 | |
| 981 | 942 | /* delayed work_func for deferring clockgating during resume */ |
| 982 | 943 | struct delayed_work delayed_init_work; |
| @@ -1009,8 +970,6 @@ struct amdgpu_device { · 26 unmodified lines | ||
| 1009 | 970 | int asic_reset_res; |
| 1010 | 971 | struct work_struct xgmi_reset_work; |
| 1011 | 972 | |
| 1012 | − bool in_baco_reset; | |
| 1013 | − | |
| 1014 | 973 | long gfx_timeout; |
| 1015 | 974 | long sdma_timeout; |
| 1016 | 975 | long video_timeout; |
| @@ -1018,6 +977,9 @@ struct amdgpu_device { · 1 unmodified line | ||
| 1018 | 977 | |
| 1019 | 978 | uint64_t unique_id; |
| 1020 | 979 | uint64_t df_perfmon_config_assign_mask[AMDGPU_MAX_DF_PERFMONS]; |
| 980 | + | |
| 981 | + /* device pstate */ | |
| 982 | + int pstate; | |
| 1021 | 983 | }; |
| 1022 | 984 | |
| 1023 | 985 | static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_bo_device *bdev) |
| @@ -1032,6 +994,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 8 unmodified lines | ||
| 1032 | 994 | void amdgpu_device_fini(struct amdgpu_device *adev); |
| 1033 | 995 | int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev); |
| 1034 | 996 | |
| 997 | +void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, | |
| 998 | + uint32_t *buf, size_t size, bool write); | |
| 1035 | 999 | uint32_t amdgpu_mm_rreg(struct amdgpu_device *adev, uint32_t reg, |
| 1036 | 1000 | uint32_t acc_flags); |
| 1037 | 1001 | void amdgpu_mm_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v, |
| @@ -63,45 +63,10 @@ void amdgpu_amdkfd_fini(void) · 62 unmodified lines | ||
| 63 | 63 | |
| 64 | 64 | void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev) |
| 65 | 65 | { |
| 66 | − const struct kfd2kgd_calls *kfd2kgd; | |
| 67 | − | |
| 68 | − switch (adev->asic_type) { | |
| 69 | −#ifdef CONFIG_DRM_AMDGPU_CIK | |
| 70 | − case CHIP_KAVERI: | |
| 71 | − case CHIP_HAWAII: | |
| 72 | − kfd2kgd = amdgpu_amdkfd_gfx_7_get_functions(); | |
| 73 | − break; | |
| 74 | −#endif | |
| 75 | − case CHIP_CARRIZO: | |
| 76 | − case CHIP_TONGA: | |
| 77 | − case CHIP_FIJI: | |
| 78 | − case CHIP_POLARIS10: | |
| 79 | − case CHIP_POLARIS11: | |
| 80 | − case CHIP_POLARIS12: | |
| 81 | − case CHIP_VEGAM: | |
| 82 | − kfd2kgd = amdgpu_amdkfd_gfx_8_0_get_functions(); | |
| 83 | − break; | |
| 84 | − case CHIP_VEGA10: | |
| 85 | − case CHIP_VEGA12: | |
| 86 | − case CHIP_VEGA20: | |
| 87 | − case CHIP_RAVEN: | |
| 88 | − kfd2kgd = amdgpu_amdkfd_gfx_9_0_get_functions(); | |
| 89 | − break; | |
| 90 | − case CHIP_ARCTURUS: | |
| 91 | − kfd2kgd = amdgpu_amdkfd_arcturus_get_functions(); | |
| 92 | − break; | |
| 93 | − case CHIP_NAVI10: | |
| 94 | − case CHIP_NAVI14: | |
| 95 | − case CHIP_NAVI12: | |
| 96 | − kfd2kgd = amdgpu_amdkfd_gfx_10_0_get_functions(); | |
| 97 | − break; | |
| 98 | − default: | |
| 99 | − dev_info(adev->dev, "kfd not supported on this ASIC\n"); | |
| 100 | − return; | |
| 101 | − } | |
| 66 | + bool vf = amdgpu_sriov_vf(adev); | |
| 102 | 67 | |
| 103 | 68 | adev->kfd.dev = kgd2kfd_probe((struct kgd_dev *)adev, |
| 104 | − adev->pdev, kfd2kgd); | |
| 69 | + adev->pdev, adev->asic_type, vf); | |
| 105 | 70 | |
| 106 | 71 | if (adev->kfd.dev) |
| 107 | 72 | amdgpu_amdkfd_total_mem_size += adev->gmc.real_vram_size; |
| @@ -165,14 +130,6 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · 57 unmodified lines | ||
| 165 | 130 | adev->gfx.mec.queue_bitmap, |
| 166 | 131 | KGD_MAX_QUEUES); |
| 167 | 132 | |
| 168 | − /* remove the KIQ bit as well */ | |
| 169 | − if (adev->gfx.kiq.ring.sched.ready) | |
| 170 | − clear_bit(amdgpu_gfx_mec_queue_to_bit(adev, | |
| 171 | − adev->gfx.kiq.ring.me - 1, | |
| 172 | − adev->gfx.kiq.ring.pipe, | |
| 173 | − adev->gfx.kiq.ring.queue), | |
| 174 | − gpu_resources.queue_bitmap); | |
| 175 | − | |
| 176 | 133 | /* According to linux/bitmap.h we shouldn't use bitmap_clear if |
| 177 | 134 | * nbits is not compile time constant |
| 178 | 135 | */ |
| @@ -202,7 +159,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) · 23 unmodified lines | ||
| 202 | 159 | adev->doorbell_index.last_non_cp; |
| 203 | 160 | } |
| 204 | 161 | |
| 205 | − kgd2kfd_device_init(adev->kfd.dev, &gpu_resources); | |
| 162 | + kgd2kfd_device_init(adev->kfd.dev, adev->ddev, &gpu_resources); | |
| 206 | 163 | } |
| 207 | 164 | } |
| 208 | 165 | |
| @@ -709,38 +666,14 @@ int amdgpu_amdkfd_evict_userptr(struct kgd_mem *mem, struct mm_struct *mm) · 500 unmodified lines | ||
| 709 | 666 | return 0; |
| 710 | 667 | } |
| 711 | 668 | |
| 712 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_7_get_functions(void) | |
| 713 | −{ | |
| 714 | − return NULL; | |
| 715 | −} | |
| 716 | − | |
| 717 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_8_0_get_functions(void) | |
| 718 | −{ | |
| 719 | − return NULL; | |
| 720 | −} | |
| 721 | − | |
| 722 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_9_0_get_functions(void) | |
| 723 | −{ | |
| 724 | − return NULL; | |
| 725 | −} | |
| 726 | − | |
| 727 | −struct kfd2kgd_calls *amdgpu_amdkfd_arcturus_get_functions(void) | |
| 728 | −{ | |
| 729 | − return NULL; | |
| 730 | −} | |
| 731 | − | |
| 732 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_10_0_get_functions(void) | |
| 733 | −{ | |
| 734 | − return NULL; | |
| 735 | −} | |
| 736 | − | |
| 737 | 669 | struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, struct pci_dev *pdev, |
| 738 | − const struct kfd2kgd_calls *f2g) | |
| 670 | + unsigned int asic_type, bool vf) | |
| 739 | 671 | { |
| 740 | 672 | return NULL; |
| 741 | 673 | } |
| 742 | 674 | |
| 743 | 675 | bool kgd2kfd_device_init(struct kfd_dev *kfd, |
| 676 | + struct drm_device *ddev, | |
| 744 | 677 | const struct kgd2kfd_shared_resources *gpu_resources) |
| 745 | 678 | { |
| 746 | 679 | return false; |
| @@ -57,7 +57,7 @@ struct kgd_mem { · 56 unmodified lines | ||
| 57 | 57 | unsigned int mapped_to_gpu_memory; |
| 58 | 58 | uint64_t va; |
| 59 | 59 | |
| 60 | − uint32_t mapping_flags; | |
| 60 | + uint32_t alloc_flags; | |
| 61 | 61 | |
| 62 | 62 | atomic_t invalid; |
| 63 | 63 | struct amdkfd_process_info *process_info; |
| @@ -137,12 +137,6 @@ int amdgpu_amdkfd_submit_ib(struct kgd_dev *kgd, enum kgd_engine_type engine, · 73 unmodified lines | ||
| 137 | 137 | void amdgpu_amdkfd_set_compute_idle(struct kgd_dev *kgd, bool idle); |
| 138 | 138 | bool amdgpu_amdkfd_have_atomics_support(struct kgd_dev *kgd); |
| 139 | 139 | |
| 140 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_7_get_functions(void); | |
| 141 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_8_0_get_functions(void); | |
| 142 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_9_0_get_functions(void); | |
| 143 | −struct kfd2kgd_calls *amdgpu_amdkfd_arcturus_get_functions(void); | |
| 144 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_10_0_get_functions(void); | |
| 145 | − | |
| 146 | 140 | bool amdgpu_amdkfd_is_kfd_vmid(struct amdgpu_device *adev, u32 vmid); |
| 147 | 141 | |
| 148 | 142 | int amdgpu_amdkfd_pre_reset(struct amdgpu_device *adev); |
| @@ -179,10 +173,17 @@ uint64_t amdgpu_amdkfd_get_mmio_remap_phys_addr(struct kgd_dev *kgd); · 30 unmodified lines | ||
| 179 | 173 | uint32_t amdgpu_amdkfd_get_num_gws(struct kgd_dev *kgd); |
| 180 | 174 | uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct kgd_dev *dst, struct kgd_dev *src); |
| 181 | 175 | |
| 176 | +/* Read user wptr from a specified user address space with page fault | |
| 177 | + * disabled. The memory must be pinned and mapped to the hardware when | |
| 178 | + * this is called in hqd_load functions, so it should never fault in | |
| 179 | + * the first place. This resolves a circular lock dependency involving | |
| 180 | + * four locks, including the DQM lock and mmap_sem. | |
| 181 | + */ | |
| 182 | 182 | #define read_user_wptr(mmptr, wptr, dst) \ |
| 183 | 183 | ({ \ |
| 184 | 184 | bool valid = false; \ |
| 185 | 185 | if ((mmptr) && (wptr)) { \ |
| 186 | + pagefault_disable(); \ | |
| 186 | 187 | if ((mmptr) == current->mm) { \ |
| 187 | 188 | valid = !get_user((dst), (wptr)); \ |
| 188 | 189 | } else if (current->mm == NULL) { \ |
| @@ -190,6 +191,7 @@ uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct kgd_dev *dst, struct kgd_dev *s · 1 unmodified line | ||
| 190 | 191 | valid = !get_user((dst), (wptr)); \ |
| 191 | 192 | unuse_mm(mmptr); \ |
| 192 | 193 | } \ |
| 194 | + pagefault_enable(); \ | |
| 193 | 195 | } \ |
| 194 | 196 | valid; \ |
| 195 | 197 | }) |
| @@ -240,8 +242,9 @@ void amdgpu_amdkfd_unreserve_memory_limit(struct amdgpu_bo *bo); · 44 unmodified lines | ||
| 240 | 242 | int kgd2kfd_init(void); |
| 241 | 243 | void kgd2kfd_exit(void); |
| 242 | 244 | struct kfd_dev *kgd2kfd_probe(struct kgd_dev *kgd, struct pci_dev *pdev, |
| 243 | − const struct kfd2kgd_calls *f2g); | |
| 245 | + unsigned int asic_type, bool vf); | |
| 244 | 246 | bool kgd2kfd_device_init(struct kfd_dev *kfd, |
| 247 | + struct drm_device *ddev, | |
| 245 | 248 | const struct kgd2kfd_shared_resources *gpu_resources); |
| 246 | 249 | void kgd2kfd_device_exit(struct kfd_dev *kfd); |
| 247 | 250 | void kgd2kfd_suspend(struct kfd_dev *kfd); |
| @@ -19,10 +19,6 @@ · 18 unmodified lines | ||
| 19 | 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR |
| 20 | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | 21 | */ |
| 22 | − | |
| 23 | −#undef pr_fmt | |
| 24 | −#define pr_fmt(fmt) "kfd2kgd: " fmt | |
| 25 | − | |
| 26 | 22 | #include <linux/module.h> |
| 27 | 23 | #include <linux/fdtable.h> |
| 28 | 24 | #include <linux/uaccess.h> |
| @@ -69,11 +65,11 @@ static inline struct v9_sdma_mqd *get_sdma_mqd(void *mqd) · 40 unmodified lines | ||
| 69 | 65 | return (struct v9_sdma_mqd *)mqd; |
| 70 | 66 | } |
| 71 | 67 | |
| 72 | −static uint32_t get_sdma_base_addr(struct amdgpu_device *adev, | |
| 68 | +static uint32_t get_sdma_rlc_reg_offset(struct amdgpu_device *adev, | |
| 73 | 69 | unsigned int engine_id, |
| 74 | 70 | unsigned int queue_id) |
| 75 | 71 | { |
| 76 | − uint32_t base[8] = { | |
| 72 | + uint32_t sdma_engine_reg_base[8] = { | |
| 77 | 73 | SOC15_REG_OFFSET(SDMA0, 0, |
| 78 | 74 | mmSDMA0_RLC0_RB_CNTL) - mmSDMA0_RLC0_RB_CNTL, |
| 79 | 75 | SOC15_REG_OFFSET(SDMA1, 0, |
| @@ -91,111 +87,82 @@ static uint32_t get_sdma_base_addr(struct amdgpu_device *adev, · 11 unmodified lines | ||
| 91 | 87 | SOC15_REG_OFFSET(SDMA7, 0, |
| 92 | 88 | mmSDMA7_RLC0_RB_CNTL) - mmSDMA7_RLC0_RB_CNTL |
| 93 | 89 | }; |
| 94 | − uint32_t retval; | |
| 95 | 90 | |
| 96 | − retval = base[engine_id] + queue_id * (mmSDMA0_RLC1_RB_CNTL - | |
| 97 | − mmSDMA0_RLC0_RB_CNTL); | |
| 91 | + uint32_t retval = sdma_engine_reg_base[engine_id] | |
| 92 | + + queue_id * (mmSDMA0_RLC1_RB_CNTL - mmSDMA0_RLC0_RB_CNTL); | |
| 98 | 93 | |
| 99 | − pr_debug("sdma base address: 0x%x\n", retval); | |
| 94 | + pr_debug("RLC register offset for SDMA%d RLC%d: 0x%x\n", engine_id, | |
| 95 | + queue_id, retval); | |
| 100 | 96 | |
| 101 | 97 | return retval; |
| 102 | 98 | } |
| 103 | 99 | |
| 104 | −static u32 sdma_v4_0_get_reg_offset(struct amdgpu_device *adev, | |
| 105 | − u32 instance, u32 offset) | |
| 106 | −{ | |
| 107 | − switch (instance) { | |
| 108 | − case 0: | |
| 109 | − return (adev->reg_offset[SDMA0_HWIP][0][0] + offset); | |
| 110 | − case 1: | |
| 111 | − return (adev->reg_offset[SDMA1_HWIP][0][1] + offset); | |
| 112 | − case 2: | |
| 113 | − return (adev->reg_offset[SDMA2_HWIP][0][1] + offset); | |
| 114 | − case 3: | |
| 115 | − return (adev->reg_offset[SDMA3_HWIP][0][1] + offset); | |
| 116 | − case 4: | |
| 117 | − return (adev->reg_offset[SDMA4_HWIP][0][1] + offset); | |
| 118 | − case 5: | |
| 119 | − return (adev->reg_offset[SDMA5_HWIP][0][1] + offset); | |
| 120 | − case 6: | |
| 121 | − return (adev->reg_offset[SDMA6_HWIP][0][1] + offset); | |
| 122 | − case 7: | |
| 123 | − return (adev->reg_offset[SDMA7_HWIP][0][1] + offset); | |
| 124 | − default: | |
| 125 | − break; | |
| 126 | − } | |
| 127 | − return 0; | |
| 128 | −} | |
| 129 | − | |
| 130 | 100 | static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, |
| 131 | 101 | uint32_t __user *wptr, struct mm_struct *mm) |
| 132 | 102 | { |
| 133 | 103 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 134 | 104 | struct v9_sdma_mqd *m; |
| 135 | − uint32_t sdma_base_addr, sdmax_gfx_context_cntl; | |
| 105 | + uint32_t sdma_rlc_reg_offset; | |
| 136 | 106 | unsigned long end_jiffies; |
| 137 | 107 | uint32_t data; |
| 138 | 108 | uint64_t data64; |
| 139 | 109 | uint64_t __user *wptr64 = (uint64_t __user *)wptr; |
| 140 | 110 | |
| 141 | 111 | m = get_sdma_mqd(mqd); |
| 142 | − sdma_base_addr = get_sdma_base_addr(adev, m->sdma_engine_id, | |
| 112 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, m->sdma_engine_id, | |
| 143 | 113 | m->sdma_queue_id); |
| 144 | − sdmax_gfx_context_cntl = sdma_v4_0_get_reg_offset(adev, | |
| 145 | − m->sdma_engine_id, mmSDMA0_GFX_CONTEXT_CNTL); | |
| 146 | 114 | |
| 147 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 115 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 148 | 116 | m->sdmax_rlcx_rb_cntl & (~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK)); |
| 149 | 117 | |
| 150 | 118 | end_jiffies = msecs_to_jiffies(2000) + jiffies; |
| 151 | 119 | while (true) { |
| 152 | − data = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 120 | + data = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 153 | 121 | if (data & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 154 | 122 | break; |
| 155 | − if (time_after(jiffies, end_jiffies)) | |
| 123 | + if (time_after(jiffies, end_jiffies)) { | |
| 124 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 156 | 125 | return -ETIME; |
| 126 | + } | |
| 157 | 127 | usleep_range(500, 1000); |
| 158 | 128 | } |
| 159 | − data = RREG32(sdmax_gfx_context_cntl); | |
| 160 | − data = REG_SET_FIELD(data, SDMA0_GFX_CONTEXT_CNTL, | |
| 161 | − RESUME_CTX, 0); | |
| 162 | − WREG32(sdmax_gfx_context_cntl, data); | |
| 163 | 129 | |
| 164 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL_OFFSET, | |
| 130 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL_OFFSET, | |
| 165 | 131 | m->sdmax_rlcx_doorbell_offset); |
| 166 | 132 | |
| 167 | 133 | data = REG_SET_FIELD(m->sdmax_rlcx_doorbell, SDMA0_RLC0_DOORBELL, |
| 168 | 134 | ENABLE, 1); |
| 169 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, data); | |
| 170 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR, m->sdmax_rlcx_rb_rptr); | |
| 171 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_HI, | |
| 135 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, data); | |
| 136 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR, | |
| 137 | + m->sdmax_rlcx_rb_rptr); | |
| 138 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_HI, | |
| 172 | 139 | m->sdmax_rlcx_rb_rptr_hi); |
| 173 | 140 | |
| 174 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 1); | |
| 141 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 1); | |
| 175 | 142 | if (read_user_wptr(mm, wptr64, data64)) { |
| 176 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, | |
| 143 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, | |
| 177 | 144 | lower_32_bits(data64)); |
| 178 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 145 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 179 | 146 | upper_32_bits(data64)); |
| 180 | 147 | } else { |
| 181 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, | |
| 148 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, | |
| 182 | 149 | m->sdmax_rlcx_rb_rptr); |
| 183 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 150 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 184 | 151 | m->sdmax_rlcx_rb_rptr_hi); |
| 185 | 152 | } |
| 186 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 0); | |
| 153 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 0); | |
| 187 | 154 | |
| 188 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE, m->sdmax_rlcx_rb_base); | |
| 189 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE_HI, | |
| 155 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE, m->sdmax_rlcx_rb_base); | |
| 156 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE_HI, | |
| 190 | 157 | m->sdmax_rlcx_rb_base_hi); |
| 191 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 158 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 192 | 159 | m->sdmax_rlcx_rb_rptr_addr_lo); |
| 193 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 160 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 194 | 161 | m->sdmax_rlcx_rb_rptr_addr_hi); |
| 195 | 162 | |
| 196 | 163 | data = REG_SET_FIELD(m->sdmax_rlcx_rb_cntl, SDMA0_RLC0_RB_CNTL, |
| 197 | 164 | RB_ENABLE, 1); |
| 198 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, data); | |
| 165 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, data); | |
| 199 | 166 | |
| 200 | 167 | return 0; |
| 201 | 168 | } |
| @@ -205,7 +172,8 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, · 3 unmodified lines | ||
| 205 | 172 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 206 | 173 | { |
| 207 | 174 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 208 | − uint32_t sdma_base_addr = get_sdma_base_addr(adev, engine_id, queue_id); | |
| 175 | + uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, | |
| 176 | + engine_id, queue_id); | |
| 209 | 177 | uint32_t i = 0, reg; |
| 210 | 178 | #undef HQD_N_REGS |
| 211 | 179 | #define HQD_N_REGS (19+6+7+10) |
| @@ -215,15 +183,15 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, · 3 unmodified lines | ||
| 215 | 183 | return -ENOMEM; |
| 216 | 184 | |
| 217 | 185 | for (reg = mmSDMA0_RLC0_RB_CNTL; reg <= mmSDMA0_RLC0_DOORBELL; reg++) |
| 218 | − DUMP_REG(sdma_base_addr + reg); | |
| 186 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 219 | 187 | for (reg = mmSDMA0_RLC0_STATUS; reg <= mmSDMA0_RLC0_CSA_ADDR_HI; reg++) |
| 220 | − DUMP_REG(sdma_base_addr + reg); | |
| 188 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 221 | 189 | for (reg = mmSDMA0_RLC0_IB_SUB_REMAIN; |
| 222 | 190 | reg <= mmSDMA0_RLC0_MINOR_PTR_UPDATE; reg++) |
| 223 | − DUMP_REG(sdma_base_addr + reg); | |
| 191 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 224 | 192 | for (reg = mmSDMA0_RLC0_MIDCMD_DATA0; |
| 225 | 193 | reg <= mmSDMA0_RLC0_MIDCMD_CNTL; reg++) |
| 226 | − DUMP_REG(sdma_base_addr + reg); | |
| 194 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 227 | 195 | |
| 228 | 196 | WARN_ON_ONCE(i != HQD_N_REGS); |
| 229 | 197 | *n_regs = i; |
| @@ -235,14 +203,14 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 5 unmodified lines | ||
| 235 | 203 | { |
| 236 | 204 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 237 | 205 | struct v9_sdma_mqd *m; |
| 238 | − uint32_t sdma_base_addr; | |
| 206 | + uint32_t sdma_rlc_reg_offset; | |
| 239 | 207 | uint32_t sdma_rlc_rb_cntl; |
| 240 | 208 | |
| 241 | 209 | m = get_sdma_mqd(mqd); |
| 242 | − sdma_base_addr = get_sdma_base_addr(adev, m->sdma_engine_id, | |
| 210 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, m->sdma_engine_id, | |
| 243 | 211 | m->sdma_queue_id); |
| 244 | 212 | |
| 245 | − sdma_rlc_rb_cntl = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 213 | + sdma_rlc_rb_cntl = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 246 | 214 | |
| 247 | 215 | if (sdma_rlc_rb_cntl & SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK) |
| 248 | 216 | return true; |
| @@ -255,40 +223,42 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, · 6 unmodified lines | ||
| 255 | 223 | { |
| 256 | 224 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 257 | 225 | struct v9_sdma_mqd *m; |
| 258 | − uint32_t sdma_base_addr; | |
| 226 | + uint32_t sdma_rlc_reg_offset; | |
| 259 | 227 | uint32_t temp; |
| 260 | 228 | unsigned long end_jiffies = (utimeout * HZ / 1000) + jiffies; |
| 261 | 229 | |
| 262 | 230 | m = get_sdma_mqd(mqd); |
| 263 | − sdma_base_addr = get_sdma_base_addr(adev, m->sdma_engine_id, | |
| 231 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, m->sdma_engine_id, | |
| 264 | 232 | m->sdma_queue_id); |
| 265 | 233 | |
| 266 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 234 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 267 | 235 | temp = temp & ~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK; |
| 268 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 236 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 269 | 237 | |
| 270 | 238 | while (true) { |
| 271 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 239 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 272 | 240 | if (temp & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 273 | 241 | break; |
| 274 | − if (time_after(jiffies, end_jiffies)) | |
| 242 | + if (time_after(jiffies, end_jiffies)) { | |
| 243 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 275 | 244 | return -ETIME; |
| 245 | + } | |
| 276 | 246 | usleep_range(500, 1000); |
| 277 | 247 | } |
| 278 | 248 | |
| 279 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, 0); | |
| 280 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 281 | − RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL) | | |
| 249 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, 0); | |
| 250 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 251 | + RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL) | | |
| 282 | 252 | SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK); |
| 283 | 253 | |
| 284 | − m->sdmax_rlcx_rb_rptr = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR); | |
| 254 | + m->sdmax_rlcx_rb_rptr = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR); | |
| 285 | 255 | m->sdmax_rlcx_rb_rptr_hi = |
| 286 | − RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_HI); | |
| 256 | + RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_HI); | |
| 287 | 257 | |
| 288 | 258 | return 0; |
| 289 | 259 | } |
| 290 | 260 | |
| 291 | −static const struct kfd2kgd_calls kfd2kgd = { | |
| 261 | +const struct kfd2kgd_calls arcturus_kfd2kgd = { | |
| 292 | 262 | .program_sh_mem_settings = kgd_gfx_v9_program_sh_mem_settings, |
| 293 | 263 | .set_pasid_vmid_mapping = kgd_gfx_v9_set_pasid_vmid_mapping, |
| 294 | 264 | .init_interrupts = kgd_gfx_v9_init_interrupts, |
| @@ -304,20 +274,11 @@ static const struct kfd2kgd_calls kfd2kgd = { · 9 unmodified lines | ||
| 304 | 274 | .address_watch_execute = kgd_gfx_v9_address_watch_execute, |
| 305 | 275 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, |
| 306 | 276 | .address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset, |
| 307 | − .get_atc_vmid_pasid_mapping_pasid = | |
| 308 | − kgd_gfx_v9_get_atc_vmid_pasid_mapping_pasid, | |
| 309 | − .get_atc_vmid_pasid_mapping_valid = | |
| 310 | − kgd_gfx_v9_get_atc_vmid_pasid_mapping_valid, | |
| 311 | − .set_scratch_backing_va = kgd_gfx_v9_set_scratch_backing_va, | |
| 277 | + .get_atc_vmid_pasid_mapping_info = | |
| 278 | + kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, | |
| 312 | 279 | .get_tile_config = kgd_gfx_v9_get_tile_config, |
| 313 | 280 | .set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base, |
| 314 | 281 | .invalidate_tlbs = kgd_gfx_v9_invalidate_tlbs, |
| 315 | 282 | .invalidate_tlbs_vmid = kgd_gfx_v9_invalidate_tlbs_vmid, |
| 316 | 283 | .get_hive_id = amdgpu_amdkfd_get_hive_id, |
| 317 | 284 | }; |
| 318 | − | |
| 319 | −struct kfd2kgd_calls *amdgpu_amdkfd_arcturus_get_functions(void) | |
| 320 | −{ | |
| 321 | − return (struct kfd2kgd_calls *)&kfd2kgd; | |
| 322 | −} | |
| 323 | − | |
| @@ -19,18 +19,9 @@ · 18 unmodified lines | ||
| 19 | 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR |
| 20 | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | 21 | */ |
| 22 | −#undef pr_fmt | |
| 23 | −#define pr_fmt(fmt) "kfd2kgd: " fmt | |
| 24 | − | |
| 25 | −#include <linux/module.h> | |
| 26 | −#include <linux/fdtable.h> | |
| 27 | −#include <linux/uaccess.h> | |
| 28 | −#include <linux/firmware.h> | |
| 29 | 22 | #include <linux/mmu_context.h> |
| 30 | 23 | #include "amdgpu.h" |
| 31 | 24 | #include "amdgpu_amdkfd.h" |
| 32 | −#include "amdgpu_ucode.h" | |
| 33 | −#include "soc15_hw_ip.h" | |
| 34 | 25 | #include "gc/gc_10_1_0_offset.h" |
| 35 | 26 | #include "gc/gc_10_1_0_sh_mask.h" |
| 36 | 27 | #include "navi10_enum.h" |
| @@ -42,6 +33,7 @@ · 5 unmodified lines | ||
| 42 | 33 | #include "v10_structs.h" |
| 43 | 34 | #include "nv.h" |
| 44 | 35 | #include "nvd.h" |
| 36 | +#include "gfxhub_v2_0.h" | |
| 45 | 37 | |
| 46 | 38 | enum hqd_dequeue_request_type { |
| 47 | 39 | NO_ACTION = 0, |
| @@ -50,63 +42,6 @@ enum hqd_dequeue_request_type { · 2 unmodified lines | ||
| 50 | 42 | SAVE_WAVES |
| 51 | 43 | }; |
| 52 | 44 | |
| 53 | −/* | |
| 54 | − * Register access functions | |
| 55 | − */ | |
| 56 | − | |
| 57 | −static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, | |
| 58 | − uint32_t sh_mem_config, | |
| 59 | − uint32_t sh_mem_ape1_base, uint32_t sh_mem_ape1_limit, | |
| 60 | − uint32_t sh_mem_bases); | |
| 61 | −static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, unsigned int pasid, | |
| 62 | − unsigned int vmid); | |
| 63 | −static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id); | |
| 64 | −static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, | |
| 65 | − uint32_t queue_id, uint32_t __user *wptr, | |
| 66 | − uint32_t wptr_shift, uint32_t wptr_mask, | |
| 67 | − struct mm_struct *mm); | |
| 68 | −static int kgd_hqd_dump(struct kgd_dev *kgd, | |
| 69 | − uint32_t pipe_id, uint32_t queue_id, | |
| 70 | − uint32_t (**dump)[2], uint32_t *n_regs); | |
| 71 | −static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, | |
| 72 | − uint32_t __user *wptr, struct mm_struct *mm); | |
| 73 | −static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, | |
| 74 | − uint32_t engine_id, uint32_t queue_id, | |
| 75 | − uint32_t (**dump)[2], uint32_t *n_regs); | |
| 76 | −static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, | |
| 77 | − uint32_t pipe_id, uint32_t queue_id); | |
| 78 | −static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd); | |
| 79 | −static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd, | |
| 80 | − enum kfd_preempt_type reset_type, | |
| 81 | − unsigned int utimeout, uint32_t pipe_id, | |
| 82 | − uint32_t queue_id); | |
| 83 | −static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, | |
| 84 | − unsigned int utimeout); | |
| 85 | −#if 0 | |
| 86 | −static uint32_t get_watch_base_addr(struct amdgpu_device *adev); | |
| 87 | −#endif | |
| 88 | −static int kgd_address_watch_disable(struct kgd_dev *kgd); | |
| 89 | −static int kgd_address_watch_execute(struct kgd_dev *kgd, | |
| 90 | − unsigned int watch_point_id, | |
| 91 | − uint32_t cntl_val, | |
| 92 | − uint32_t addr_hi, | |
| 93 | − uint32_t addr_lo); | |
| 94 | −static int kgd_wave_control_execute(struct kgd_dev *kgd, | |
| 95 | − uint32_t gfx_index_val, | |
| 96 | − uint32_t sq_cmd); | |
| 97 | −static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd, | |
| 98 | − unsigned int watch_point_id, | |
| 99 | − unsigned int reg_offset); | |
| 100 | − | |
| 101 | −static bool get_atc_vmid_pasid_mapping_valid(struct kgd_dev *kgd, | |
| 102 | − uint8_t vmid); | |
| 103 | −static uint16_t get_atc_vmid_pasid_mapping_pasid(struct kgd_dev *kgd, | |
| 104 | − uint8_t vmid); | |
| 105 | −static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, | |
| 106 | − uint64_t page_table_base); | |
| 107 | −static int invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid); | |
| 108 | −static int invalidate_tlbs_vmid(struct kgd_dev *kgd, uint16_t vmid); | |
| 109 | − | |
| 110 | 45 | /* Because of REG_GET_FIELD() being used, we put this function in the |
| 111 | 46 | * asic specific file. |
| 112 | 47 | */ |
| @@ -139,37 +74,6 @@ static int amdgpu_amdkfd_get_tile_config(struct kgd_dev *kgd, · 26 unmodified lines | ||
| 139 | 74 | return 0; |
| 140 | 75 | } |
| 141 | 76 | |
| 142 | −static const struct kfd2kgd_calls kfd2kgd = { | |
| 143 | − .program_sh_mem_settings = kgd_program_sh_mem_settings, | |
| 144 | − .set_pasid_vmid_mapping = kgd_set_pasid_vmid_mapping, | |
| 145 | − .init_interrupts = kgd_init_interrupts, | |
| 146 | − .hqd_load = kgd_hqd_load, | |
| 147 | − .hqd_sdma_load = kgd_hqd_sdma_load, | |
| 148 | − .hqd_dump = kgd_hqd_dump, | |
| 149 | − .hqd_sdma_dump = kgd_hqd_sdma_dump, | |
| 150 | − .hqd_is_occupied = kgd_hqd_is_occupied, | |
| 151 | − .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | |
| 152 | − .hqd_destroy = kgd_hqd_destroy, | |
| 153 | − .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | |
| 154 | − .address_watch_disable = kgd_address_watch_disable, | |
| 155 | − .address_watch_execute = kgd_address_watch_execute, | |
| 156 | − .wave_control_execute = kgd_wave_control_execute, | |
| 157 | − .address_watch_get_offset = kgd_address_watch_get_offset, | |
| 158 | − .get_atc_vmid_pasid_mapping_pasid = | |
| 159 | − get_atc_vmid_pasid_mapping_pasid, | |
| 160 | − .get_atc_vmid_pasid_mapping_valid = | |
| 161 | − get_atc_vmid_pasid_mapping_valid, | |
| 162 | − .invalidate_tlbs = invalidate_tlbs, | |
| 163 | − .invalidate_tlbs_vmid = invalidate_tlbs_vmid, | |
| 164 | − .set_vm_context_page_table_base = set_vm_context_page_table_base, | |
| 165 | − .get_tile_config = amdgpu_amdkfd_get_tile_config, | |
| 166 | −}; | |
| 167 | − | |
| 168 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_10_0_get_functions() | |
| 169 | −{ | |
| 170 | − return (struct kfd2kgd_calls *)&kfd2kgd; | |
| 171 | −} | |
| 172 | − | |
| 173 | 77 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) |
| 174 | 78 | { |
| 175 | 79 | return (struct amdgpu_device *)kgd; |
| @@ -250,11 +154,6 @@ static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, unsigned int pasid, · 74 unmodified lines | ||
| 250 | 154 | ATC_VMID0_PASID_MAPPING__VALID_MASK; |
| 251 | 155 | |
| 252 | 156 | pr_debug("pasid 0x%x vmid %d, reg value %x\n", pasid, vmid, pasid_mapping); |
| 253 | − /* | |
| 254 | − * need to do this twice, once for gfx and once for mmhub | |
| 255 | − * for ATC add 16 to VMID for mmhub, for IH different registers. | |
| 256 | − * ATC_VMID0..15 registers are separate from ATC_VMID16..31. | |
| 257 | − */ | |
| 258 | 157 | |
| 259 | 158 | pr_debug("ATHUB, reg %x\n", SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) + vmid); |
| 260 | 159 | WREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) + vmid, |
| @@ -306,11 +205,11 @@ static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id) · 45 unmodified lines | ||
| 306 | 205 | return 0; |
| 307 | 206 | } |
| 308 | 207 | |
| 309 | −static uint32_t get_sdma_base_addr(struct amdgpu_device *adev, | |
| 208 | +static uint32_t get_sdma_rlc_reg_offset(struct amdgpu_device *adev, | |
| 310 | 209 | unsigned int engine_id, |
| 311 | 210 | unsigned int queue_id) |
| 312 | 211 | { |
| 313 | − uint32_t base[2] = { | |
| 212 | + uint32_t sdma_engine_reg_base[2] = { | |
| 314 | 213 | SOC15_REG_OFFSET(SDMA0, 0, |
| 315 | 214 | mmSDMA0_RLC0_RB_CNTL) - mmSDMA0_RLC0_RB_CNTL, |
| 316 | 215 | /* On gfx10, mmSDMA1_xxx registers are defined NOT based |
| @@ -322,12 +221,12 @@ static uint32_t get_sdma_base_addr(struct amdgpu_device *adev, · 5 unmodified lines | ||
| 322 | 221 | SOC15_REG_OFFSET(SDMA1, 0, |
| 323 | 222 | mmSDMA1_RLC0_RB_CNTL) - mmSDMA0_RLC0_RB_CNTL |
| 324 | 223 | }; |
| 325 | − uint32_t retval; | |
| 326 | 224 | |
| 327 | − retval = base[engine_id] + queue_id * (mmSDMA0_RLC1_RB_CNTL - | |
| 328 | − mmSDMA0_RLC0_RB_CNTL); | |
| 225 | + uint32_t retval = sdma_engine_reg_base[engine_id] | |
| 226 | + + queue_id * (mmSDMA0_RLC1_RB_CNTL - mmSDMA0_RLC0_RB_CNTL); | |
| 329 | 227 | |
| 330 | − pr_debug("sdma base address: 0x%x\n", retval); | |
| 228 | + pr_debug("RLC register offset for SDMA%d RLC%d: 0x%x\n", engine_id, | |
| 229 | + queue_id, retval); | |
| 331 | 230 | |
| 332 | 231 | return retval; |
| 333 | 232 | } |
| @@ -488,72 +387,67 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, · 154 unmodified lines | ||
| 488 | 387 | { |
| 489 | 388 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 490 | 389 | struct v10_sdma_mqd *m; |
| 491 | − uint32_t sdma_base_addr, sdmax_gfx_context_cntl; | |
| 390 | + uint32_t sdma_rlc_reg_offset; | |
| 492 | 391 | unsigned long end_jiffies; |
| 493 | 392 | uint32_t data; |
| 494 | 393 | uint64_t data64; |
| 495 | 394 | uint64_t __user *wptr64 = (uint64_t __user *)wptr; |
| 496 | 395 | |
| 497 | 396 | m = get_sdma_mqd(mqd); |
| 498 | − sdma_base_addr = get_sdma_base_addr(adev, m->sdma_engine_id, | |
| 397 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, m->sdma_engine_id, | |
| 499 | 398 | m->sdma_queue_id); |
| 500 | − pr_debug("sdma load base addr %x for engine %d, queue %d\n", sdma_base_addr, m->sdma_engine_id, m->sdma_queue_id); | |
| 501 | − sdmax_gfx_context_cntl = m->sdma_engine_id ? | |
| 502 | − SOC15_REG_OFFSET(SDMA1, 0, mmSDMA1_GFX_CONTEXT_CNTL) : | |
| 503 | − SOC15_REG_OFFSET(SDMA0, 0, mmSDMA0_GFX_CONTEXT_CNTL); | |
| 504 | 399 | |
| 505 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 400 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 506 | 401 | m->sdmax_rlcx_rb_cntl & (~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK)); |
| 507 | 402 | |
| 508 | 403 | end_jiffies = msecs_to_jiffies(2000) + jiffies; |
| 509 | 404 | while (true) { |
| 510 | − data = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 405 | + data = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 511 | 406 | if (data & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 512 | 407 | break; |
| 513 | − if (time_after(jiffies, end_jiffies)) | |
| 408 | + if (time_after(jiffies, end_jiffies)) { | |
| 409 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 514 | 410 | return -ETIME; |
| 411 | + } | |
| 515 | 412 | usleep_range(500, 1000); |
| 516 | 413 | } |
| 517 | − data = RREG32(sdmax_gfx_context_cntl); | |
| 518 | − data = REG_SET_FIELD(data, SDMA0_GFX_CONTEXT_CNTL, | |
| 519 | − RESUME_CTX, 0); | |
| 520 | − WREG32(sdmax_gfx_context_cntl, data); | |
| 521 | 414 | |
| 522 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL_OFFSET, | |
| 415 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL_OFFSET, | |
| 523 | 416 | m->sdmax_rlcx_doorbell_offset); |
| 524 | 417 | |
| 525 | 418 | data = REG_SET_FIELD(m->sdmax_rlcx_doorbell, SDMA0_RLC0_DOORBELL, |
| 526 | 419 | ENABLE, 1); |
| 527 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, data); | |
| 528 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR, m->sdmax_rlcx_rb_rptr); | |
| 529 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_HI, | |
| 420 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, data); | |
| 421 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR, | |
| 422 | + m->sdmax_rlcx_rb_rptr); | |
| 423 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_HI, | |
| 530 | 424 | m->sdmax_rlcx_rb_rptr_hi); |
| 531 | 425 | |
| 532 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 1); | |
| 426 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 1); | |
| 533 | 427 | if (read_user_wptr(mm, wptr64, data64)) { |
| 534 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, | |
| 428 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, | |
| 535 | 429 | lower_32_bits(data64)); |
| 536 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 430 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 537 | 431 | upper_32_bits(data64)); |
| 538 | 432 | } else { |
| 539 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, | |
| 433 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, | |
| 540 | 434 | m->sdmax_rlcx_rb_rptr); |
| 541 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 435 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 542 | 436 | m->sdmax_rlcx_rb_rptr_hi); |
| 543 | 437 | } |
| 544 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 0); | |
| 438 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 0); | |
| 545 | 439 | |
| 546 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE, m->sdmax_rlcx_rb_base); | |
| 547 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE_HI, | |
| 440 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE, m->sdmax_rlcx_rb_base); | |
| 441 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE_HI, | |
| 548 | 442 | m->sdmax_rlcx_rb_base_hi); |
| 549 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 443 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 550 | 444 | m->sdmax_rlcx_rb_rptr_addr_lo); |
| 551 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 445 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 552 | 446 | m->sdmax_rlcx_rb_rptr_addr_hi); |
| 553 | 447 | |
| 554 | 448 | data = REG_SET_FIELD(m->sdmax_rlcx_rb_cntl, SDMA0_RLC0_RB_CNTL, |
| 555 | 449 | RB_ENABLE, 1); |
| 556 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, data); | |
| 450 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, data); | |
| 557 | 451 | |
| 558 | 452 | return 0; |
| 559 | 453 | } |
| @@ -563,28 +457,26 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, · 3 unmodified lines | ||
| 563 | 457 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 564 | 458 | { |
| 565 | 459 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 566 | − uint32_t sdma_base_addr = get_sdma_base_addr(adev, engine_id, queue_id); | |
| 460 | + uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, | |
| 461 | + engine_id, queue_id); | |
| 567 | 462 | uint32_t i = 0, reg; |
| 568 | 463 | #undef HQD_N_REGS |
| 569 | 464 | #define HQD_N_REGS (19+6+7+10) |
| 570 | 465 | |
| 571 | − pr_debug("sdma dump engine id %d queue_id %d\n", engine_id, queue_id); | |
| 572 | − pr_debug("sdma base addr %x\n", sdma_base_addr); | |
| 573 | − | |
| 574 | 466 | *dump = kmalloc(HQD_N_REGS*2*sizeof(uint32_t), GFP_KERNEL); |
| 575 | 467 | if (*dump == NULL) |
| 576 | 468 | return -ENOMEM; |
| 577 | 469 | |
| 578 | 470 | for (reg = mmSDMA0_RLC0_RB_CNTL; reg <= mmSDMA0_RLC0_DOORBELL; reg++) |
| 579 | − DUMP_REG(sdma_base_addr + reg); | |
| 471 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 580 | 472 | for (reg = mmSDMA0_RLC0_STATUS; reg <= mmSDMA0_RLC0_CSA_ADDR_HI; reg++) |
| 581 | − DUMP_REG(sdma_base_addr + reg); | |
| 473 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 582 | 474 | for (reg = mmSDMA0_RLC0_IB_SUB_REMAIN; |
| 583 | 475 | reg <= mmSDMA0_RLC0_MINOR_PTR_UPDATE; reg++) |
| 584 | − DUMP_REG(sdma_base_addr + reg); | |
| 476 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 585 | 477 | for (reg = mmSDMA0_RLC0_MIDCMD_DATA0; |
| 586 | 478 | reg <= mmSDMA0_RLC0_MIDCMD_CNTL; reg++) |
| 587 | − DUMP_REG(sdma_base_addr + reg); | |
| 479 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 588 | 480 | |
| 589 | 481 | WARN_ON_ONCE(i != HQD_N_REGS); |
| 590 | 482 | *n_regs = i; |
| @@ -618,14 +510,14 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 27 unmodified lines | ||
| 618 | 510 | { |
| 619 | 511 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 620 | 512 | struct v10_sdma_mqd *m; |
| 621 | − uint32_t sdma_base_addr; | |
| 513 | + uint32_t sdma_rlc_reg_offset; | |
| 622 | 514 | uint32_t sdma_rlc_rb_cntl; |
| 623 | 515 | |
| 624 | 516 | m = get_sdma_mqd(mqd); |
| 625 | − sdma_base_addr = get_sdma_base_addr(adev, m->sdma_engine_id, | |
| 517 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, m->sdma_engine_id, | |
| 626 | 518 | m->sdma_queue_id); |
| 627 | 519 | |
| 628 | − sdma_rlc_rb_cntl = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 520 | + sdma_rlc_rb_cntl = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 629 | 521 | |
| 630 | 522 | if (sdma_rlc_rb_cntl & SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK) |
| 631 | 523 | return true; |
| @@ -746,59 +638,52 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, · 114 unmodified lines | ||
| 746 | 638 | { |
| 747 | 639 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 748 | 640 | struct v10_sdma_mqd *m; |
| 749 | − uint32_t sdma_base_addr; | |
| 641 | + uint32_t sdma_rlc_reg_offset; | |
| 750 | 642 | uint32_t temp; |
| 751 | 643 | unsigned long end_jiffies = (utimeout * HZ / 1000) + jiffies; |
| 752 | 644 | |
| 753 | 645 | m = get_sdma_mqd(mqd); |
| 754 | − sdma_base_addr = get_sdma_base_addr(adev, m->sdma_engine_id, | |
| 646 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, m->sdma_engine_id, | |
| 755 | 647 | m->sdma_queue_id); |
| 756 | 648 | |
| 757 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 649 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 758 | 650 | temp = temp & ~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK; |
| 759 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 651 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 760 | 652 | |
| 761 | 653 | while (true) { |
| 762 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 654 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 763 | 655 | if (temp & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 764 | 656 | break; |
| 765 | − if (time_after(jiffies, end_jiffies)) | |
| 657 | + if (time_after(jiffies, end_jiffies)) { | |
| 658 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 766 | 659 | return -ETIME; |
| 660 | + } | |
| 767 | 661 | usleep_range(500, 1000); |
| 768 | 662 | } |
| 769 | 663 | |
| 770 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, 0); | |
| 771 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 772 | − RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL) | | |
| 664 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, 0); | |
| 665 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 666 | + RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL) | | |
| 773 | 667 | SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK); |
| 774 | 668 | |
| 775 | − m->sdmax_rlcx_rb_rptr = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR); | |
| 669 | + m->sdmax_rlcx_rb_rptr = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR); | |
| 776 | 670 | m->sdmax_rlcx_rb_rptr_hi = |
| 777 | − RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_HI); | |
| 671 | + RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_HI); | |
| 778 | 672 | |
| 779 | 673 | return 0; |
| 780 | 674 | } |
| 781 | 675 | |
| 782 | −static bool get_atc_vmid_pasid_mapping_valid(struct kgd_dev *kgd, | |
| 783 | − uint8_t vmid) | |
| 676 | +static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | |
| 677 | + uint8_t vmid, uint16_t *p_pasid) | |
| 784 | 678 | { |
| 785 | − uint32_t reg; | |
| 679 | + uint32_t value; | |
| 786 | 680 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; |
| 787 | 681 | |
| 788 | − reg = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) | |
| 682 | + value = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) | |
| 789 | 683 | + vmid); |
| 790 | − return reg & ATC_VMID0_PASID_MAPPING__VALID_MASK; | |
| 791 | −} | |
| 792 | − | |
| 793 | −static uint16_t get_atc_vmid_pasid_mapping_pasid(struct kgd_dev *kgd, | |
| 794 | − uint8_t vmid) | |
| 795 | −{ | |
| 796 | − uint32_t reg; | |
| 797 | − struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | |
| 684 | + *p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK; | |
| 798 | 685 | |
| 799 | − reg = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) | |
| 800 | − + vmid); | |
| 801 | − return reg & ATC_VMID0_PASID_MAPPING__PASID_MASK; | |
| 686 | + return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | |
| 802 | 687 | } |
| 803 | 688 | |
| 804 | 689 | static int invalidate_tlbs_with_kiq(struct amdgpu_device *adev, uint16_t pasid) |
| @@ -830,6 +715,8 @@ static int invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid) · 25 unmodified lines | ||
| 830 | 715 | { |
| 831 | 716 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; |
| 832 | 717 | int vmid; |
| 718 | + uint16_t queried_pasid; | |
| 719 | + bool ret; | |
| 833 | 720 | struct amdgpu_ring *ring = &adev->gfx.kiq.ring; |
| 834 | 721 | |
| 835 | 722 | if (amdgpu_emu_mode == 0 && ring->sched.ready) |
| @@ -838,13 +725,13 @@ static int invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid) · 2 unmodified lines | ||
| 838 | 725 | for (vmid = 0; vmid < 16; vmid++) { |
| 839 | 726 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) |
| 840 | 727 | continue; |
| 841 | − if (get_atc_vmid_pasid_mapping_valid(kgd, vmid)) { | |
| 842 | − if (get_atc_vmid_pasid_mapping_pasid(kgd, vmid) | |
| 843 | − == pasid) { | |
| 844 | − amdgpu_gmc_flush_gpu_tlb(adev, vmid, | |
| 845 | − AMDGPU_GFXHUB_0, 0); | |
| 846 | − break; | |
| 847 | − } | |
| 728 | + | |
| 729 | + ret = get_atc_vmid_pasid_mapping_info(kgd, vmid, | |
| 730 | + &queried_pasid); | |
| 731 | + if (ret && queried_pasid == pasid) { | |
| 732 | + amdgpu_gmc_flush_gpu_tlb(adev, vmid, | |
| 733 | + AMDGPU_GFXHUB_0, 0); | |
| 734 | + break; | |
| 848 | 735 | } |
| 849 | 736 | } |
| 850 | 737 | |
| @@ -914,7 +801,6 @@ static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, · 63 unmodified lines | ||
| 914 | 801 | uint64_t page_table_base) |
| 915 | 802 | { |
| 916 | 803 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 917 | − uint64_t base = page_table_base | AMDGPU_PTE_VALID; | |
| 918 | 804 | |
| 919 | 805 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) { |
| 920 | 806 | pr_err("trying to set page table base for wrong VMID %u\n", |
| @@ -922,18 +808,31 @@ static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, · 1 unmodified line | ||
| 922 | 808 | return; |
| 923 | 809 | } |
| 924 | 810 | |
| 925 | − /* TODO: take advantage of per-process address space size. For | |
| 926 | − * now, all processes share the same address space size, like | |
| 927 | − * on GFX8 and older. | |
| 928 | − */ | |
| 929 | − WREG32(SOC15_REG_OFFSET(GC, 0, mmGCVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32) + (vmid*2), 0); | |
| 930 | − WREG32(SOC15_REG_OFFSET(GC, 0, mmGCVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32) + (vmid*2), 0); | |
| 931 | − | |
| 932 | − WREG32(SOC15_REG_OFFSET(GC, 0, mmGCVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32) + (vmid*2), | |
| 933 | − lower_32_bits(adev->vm_manager.max_pfn - 1)); | |
| 934 | − WREG32(SOC15_REG_OFFSET(GC, 0, mmGCVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32) + (vmid*2), | |
| 935 | − upper_32_bits(adev->vm_manager.max_pfn - 1)); | |
| 936 | − | |
| 937 | − WREG32(SOC15_REG_OFFSET(GC, 0, mmGCVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32) + (vmid*2), lower_32_bits(base)); | |
| 938 | − WREG32(SOC15_REG_OFFSET(GC, 0, mmGCVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_HI32) + (vmid*2), upper_32_bits(base)); | |
| 811 | + /* SDMA is on gfxhub as well for Navi1* series */ | |
| 812 | + gfxhub_v2_0_setup_vm_pt_regs(adev, vmid, page_table_base); | |
| 939 | 813 | } |
| 814 | + | |
| 815 | +const struct kfd2kgd_calls gfx_v10_kfd2kgd = { | |
| 816 | + .program_sh_mem_settings = kgd_program_sh_mem_settings, | |
| 817 | + .set_pasid_vmid_mapping = kgd_set_pasid_vmid_mapping, | |
| 818 | + .init_interrupts = kgd_init_interrupts, | |
| 819 | + .hqd_load = kgd_hqd_load, | |
| 820 | + .hqd_sdma_load = kgd_hqd_sdma_load, | |
| 821 | + .hqd_dump = kgd_hqd_dump, | |
| 822 | + .hqd_sdma_dump = kgd_hqd_sdma_dump, | |
| 823 | + .hqd_is_occupied = kgd_hqd_is_occupied, | |
| 824 | + .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | |
| 825 | + .hqd_destroy = kgd_hqd_destroy, | |
| 826 | + .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | |
| 827 | + .address_watch_disable = kgd_address_watch_disable, | |
| 828 | + .address_watch_execute = kgd_address_watch_execute, | |
| 829 | + .wave_control_execute = kgd_wave_control_execute, | |
| 830 | + .address_watch_get_offset = kgd_address_watch_get_offset, | |
| 831 | + .get_atc_vmid_pasid_mapping_info = | |
| 832 | + get_atc_vmid_pasid_mapping_info, | |
| 833 | + .get_tile_config = amdgpu_amdkfd_get_tile_config, | |
| 834 | + .set_vm_context_page_table_base = set_vm_context_page_table_base, | |
| 835 | + .invalidate_tlbs = invalidate_tlbs, | |
| 836 | + .invalidate_tlbs_vmid = invalidate_tlbs_vmid, | |
| 837 | + .get_hive_id = amdgpu_amdkfd_get_hive_id, | |
| 838 | +}; | |
| @@ -20,8 +20,6 @@ · 19 unmodified lines | ||
| 20 | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | 21 | */ |
| 22 | 22 | |
| 23 | −#include <linux/fdtable.h> | |
| 24 | −#include <linux/uaccess.h> | |
| 25 | 23 | #include <linux/mmu_context.h> |
| 26 | 24 | |
| 27 | 25 | #include "amdgpu.h" |
| @@ -86,65 +84,6 @@ union TCP_WATCH_CNTL_BITS { · 58 unmodified lines | ||
| 86 | 84 | float f32All; |
| 87 | 85 | }; |
| 88 | 86 | |
| 89 | −/* | |
| 90 | − * Register access functions | |
| 91 | − */ | |
| 92 | − | |
| 93 | −static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, | |
| 94 | − uint32_t sh_mem_config, uint32_t sh_mem_ape1_base, | |
| 95 | − uint32_t sh_mem_ape1_limit, uint32_t sh_mem_bases); | |
| 96 | − | |
| 97 | −static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, unsigned int pasid, | |
| 98 | − unsigned int vmid); | |
| 99 | − | |
| 100 | −static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id); | |
| 101 | −static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, | |
| 102 | − uint32_t queue_id, uint32_t __user *wptr, | |
| 103 | − uint32_t wptr_shift, uint32_t wptr_mask, | |
| 104 | − struct mm_struct *mm); | |
| 105 | −static int kgd_hqd_dump(struct kgd_dev *kgd, | |
| 106 | − uint32_t pipe_id, uint32_t queue_id, | |
| 107 | − uint32_t (**dump)[2], uint32_t *n_regs); | |
| 108 | −static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, | |
| 109 | − uint32_t __user *wptr, struct mm_struct *mm); | |
| 110 | −static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, | |
| 111 | − uint32_t engine_id, uint32_t queue_id, | |
| 112 | − uint32_t (**dump)[2], uint32_t *n_regs); | |
| 113 | −static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, | |
| 114 | − uint32_t pipe_id, uint32_t queue_id); | |
| 115 | − | |
| 116 | −static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd, | |
| 117 | − enum kfd_preempt_type reset_type, | |
| 118 | − unsigned int utimeout, uint32_t pipe_id, | |
| 119 | − uint32_t queue_id); | |
| 120 | −static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd); | |
| 121 | −static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, | |
| 122 | − unsigned int utimeout); | |
| 123 | −static int kgd_address_watch_disable(struct kgd_dev *kgd); | |
| 124 | −static int kgd_address_watch_execute(struct kgd_dev *kgd, | |
| 125 | − unsigned int watch_point_id, | |
| 126 | − uint32_t cntl_val, | |
| 127 | − uint32_t addr_hi, | |
| 128 | − uint32_t addr_lo); | |
| 129 | −static int kgd_wave_control_execute(struct kgd_dev *kgd, | |
| 130 | − uint32_t gfx_index_val, | |
| 131 | − uint32_t sq_cmd); | |
| 132 | −static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd, | |
| 133 | − unsigned int watch_point_id, | |
| 134 | − unsigned int reg_offset); | |
| 135 | − | |
| 136 | −static bool get_atc_vmid_pasid_mapping_valid(struct kgd_dev *kgd, uint8_t vmid); | |
| 137 | −static uint16_t get_atc_vmid_pasid_mapping_pasid(struct kgd_dev *kgd, | |
| 138 | − uint8_t vmid); | |
| 139 | − | |
| 140 | −static void set_scratch_backing_va(struct kgd_dev *kgd, | |
| 141 | − uint64_t va, uint32_t vmid); | |
| 142 | −static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, | |
| 143 | − uint64_t page_table_base); | |
| 144 | −static int invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid); | |
| 145 | −static int invalidate_tlbs_vmid(struct kgd_dev *kgd, uint16_t vmid); | |
| 146 | −static uint32_t read_vmid_from_vmfault_reg(struct kgd_dev *kgd); | |
| 147 | − | |
| 148 | 87 | /* Because of REG_GET_FIELD() being used, we put this function in the |
| 149 | 88 | * asic specific file. |
| 150 | 89 | */ |
| @@ -170,37 +109,6 @@ static int get_tile_config(struct kgd_dev *kgd, · 19 unmodified lines | ||
| 170 | 109 | return 0; |
| 171 | 110 | } |
| 172 | 111 | |
| 173 | −static const struct kfd2kgd_calls kfd2kgd = { | |
| 174 | − .program_sh_mem_settings = kgd_program_sh_mem_settings, | |
| 175 | − .set_pasid_vmid_mapping = kgd_set_pasid_vmid_mapping, | |
| 176 | − .init_interrupts = kgd_init_interrupts, | |
| 177 | − .hqd_load = kgd_hqd_load, | |
| 178 | − .hqd_sdma_load = kgd_hqd_sdma_load, | |
| 179 | − .hqd_dump = kgd_hqd_dump, | |
| 180 | − .hqd_sdma_dump = kgd_hqd_sdma_dump, | |
| 181 | − .hqd_is_occupied = kgd_hqd_is_occupied, | |
| 182 | − .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | |
| 183 | − .hqd_destroy = kgd_hqd_destroy, | |
| 184 | − .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | |
| 185 | − .address_watch_disable = kgd_address_watch_disable, | |
| 186 | − .address_watch_execute = kgd_address_watch_execute, | |
| 187 | − .wave_control_execute = kgd_wave_control_execute, | |
| 188 | − .address_watch_get_offset = kgd_address_watch_get_offset, | |
| 189 | − .get_atc_vmid_pasid_mapping_pasid = get_atc_vmid_pasid_mapping_pasid, | |
| 190 | − .get_atc_vmid_pasid_mapping_valid = get_atc_vmid_pasid_mapping_valid, | |
| 191 | − .set_scratch_backing_va = set_scratch_backing_va, | |
| 192 | − .get_tile_config = get_tile_config, | |
| 193 | − .set_vm_context_page_table_base = set_vm_context_page_table_base, | |
| 194 | − .invalidate_tlbs = invalidate_tlbs, | |
| 195 | − .invalidate_tlbs_vmid = invalidate_tlbs_vmid, | |
| 196 | − .read_vmid_from_vmfault_reg = read_vmid_from_vmfault_reg, | |
| 197 | −}; | |
| 198 | − | |
| 199 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_7_get_functions(void) | |
| 200 | −{ | |
| 201 | − return (struct kfd2kgd_calls *)&kfd2kgd; | |
| 202 | −} | |
| 203 | − | |
| 204 | 112 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) |
| 205 | 113 | { |
| 206 | 114 | return (struct amdgpu_device *)kgd; |
| @@ -303,14 +211,15 @@ static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id) · 96 unmodified lines | ||
| 303 | 211 | return 0; |
| 304 | 212 | } |
| 305 | 213 | |
| 306 | −static inline uint32_t get_sdma_base_addr(struct cik_sdma_rlc_registers *m) | |
| 214 | +static inline uint32_t get_sdma_rlc_reg_offset(struct cik_sdma_rlc_registers *m) | |
| 307 | 215 | { |
| 308 | 216 | uint32_t retval; |
| 309 | 217 | |
| 310 | 218 | retval = m->sdma_engine_id * SDMA1_REGISTER_OFFSET + |
| 311 | 219 | m->sdma_queue_id * KFD_CIK_SDMA_QUEUE_OFFSET; |
| 312 | 220 | |
| 313 | − pr_debug("sdma base address: 0x%x\n", retval); | |
| 221 | + pr_debug("RLC register offset for SDMA%d RLC%d: 0x%x\n", | |
| 222 | + m->sdma_engine_id, m->sdma_queue_id, retval); | |
| 314 | 223 | |
| 315 | 224 | return retval; |
| 316 | 225 | } |
| @@ -413,60 +322,52 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, · 96 unmodified lines | ||
| 413 | 322 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 414 | 323 | struct cik_sdma_rlc_registers *m; |
| 415 | 324 | unsigned long end_jiffies; |
| 416 | − uint32_t sdma_base_addr; | |
| 325 | + uint32_t sdma_rlc_reg_offset; | |
| 417 | 326 | uint32_t data; |
| 418 | 327 | |
| 419 | 328 | m = get_sdma_mqd(mqd); |
| 420 | − sdma_base_addr = get_sdma_base_addr(m); | |
| 329 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(m); | |
| 421 | 330 | |
| 422 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 331 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 423 | 332 | m->sdma_rlc_rb_cntl & (~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK)); |
| 424 | 333 | |
| 425 | 334 | end_jiffies = msecs_to_jiffies(2000) + jiffies; |
| 426 | 335 | while (true) { |
| 427 | − data = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 336 | + data = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 428 | 337 | if (data & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 429 | 338 | break; |
| 430 | − if (time_after(jiffies, end_jiffies)) | |
| 339 | + if (time_after(jiffies, end_jiffies)) { | |
| 340 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 431 | 341 | return -ETIME; |
| 342 | + } | |
| 432 | 343 | usleep_range(500, 1000); |
| 433 | 344 | } |
| 434 | − if (m->sdma_engine_id) { | |
| 435 | − data = RREG32(mmSDMA1_GFX_CONTEXT_CNTL); | |
| 436 | − data = REG_SET_FIELD(data, SDMA1_GFX_CONTEXT_CNTL, | |
| 437 | − RESUME_CTX, 0); | |
| 438 | − WREG32(mmSDMA1_GFX_CONTEXT_CNTL, data); | |
| 439 | − } else { | |
| 440 | − data = RREG32(mmSDMA0_GFX_CONTEXT_CNTL); | |
| 441 | − data = REG_SET_FIELD(data, SDMA0_GFX_CONTEXT_CNTL, | |
| 442 | − RESUME_CTX, 0); | |
| 443 | − WREG32(mmSDMA0_GFX_CONTEXT_CNTL, data); | |
| 444 | − } | |
| 445 | 345 | |
| 446 | 346 | data = REG_SET_FIELD(m->sdma_rlc_doorbell, SDMA0_RLC0_DOORBELL, |
| 447 | 347 | ENABLE, 1); |
| 448 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, data); | |
| 449 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR, m->sdma_rlc_rb_rptr); | |
| 348 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, data); | |
| 349 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR, | |
| 350 | + m->sdma_rlc_rb_rptr); | |
| 450 | 351 | |
| 451 | 352 | if (read_user_wptr(mm, wptr, data)) |
| 452 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, data); | |
| 353 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, data); | |
| 453 | 354 | else |
| 454 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, | |
| 355 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, | |
| 455 | 356 | m->sdma_rlc_rb_rptr); |
| 456 | 357 | |
| 457 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_VIRTUAL_ADDR, | |
| 358 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_VIRTUAL_ADDR, | |
| 458 | 359 | m->sdma_rlc_virtual_addr); |
| 459 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE, m->sdma_rlc_rb_base); | |
| 460 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE_HI, | |
| 360 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE, m->sdma_rlc_rb_base); | |
| 361 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE_HI, | |
| 461 | 362 | m->sdma_rlc_rb_base_hi); |
| 462 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 363 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 463 | 364 | m->sdma_rlc_rb_rptr_addr_lo); |
| 464 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 365 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 465 | 366 | m->sdma_rlc_rb_rptr_addr_hi); |
| 466 | 367 | |
| 467 | 368 | data = REG_SET_FIELD(m->sdma_rlc_rb_cntl, SDMA0_RLC0_RB_CNTL, |
| 468 | 369 | RB_ENABLE, 1); |
| 469 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, data); | |
| 370 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, data); | |
| 470 | 371 | |
| 471 | 372 | return 0; |
| 472 | 373 | } |
| @@ -524,13 +425,13 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 51 unmodified lines | ||
| 524 | 425 | { |
| 525 | 426 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 526 | 427 | struct cik_sdma_rlc_registers *m; |
| 527 | − uint32_t sdma_base_addr; | |
| 428 | + uint32_t sdma_rlc_reg_offset; | |
| 528 | 429 | uint32_t sdma_rlc_rb_cntl; |
| 529 | 430 | |
| 530 | 431 | m = get_sdma_mqd(mqd); |
| 531 | − sdma_base_addr = get_sdma_base_addr(m); | |
| 432 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(m); | |
| 532 | 433 | |
| 533 | − sdma_rlc_rb_cntl = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 434 | + sdma_rlc_rb_cntl = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 534 | 435 | |
| 535 | 436 | if (sdma_rlc_rb_cntl & SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK) |
| 536 | 437 | return true; |
| @@ -645,32 +546,34 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, · 108 unmodified lines | ||
| 645 | 546 | { |
| 646 | 547 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 647 | 548 | struct cik_sdma_rlc_registers *m; |
| 648 | − uint32_t sdma_base_addr; | |
| 549 | + uint32_t sdma_rlc_reg_offset; | |
| 649 | 550 | uint32_t temp; |
| 650 | 551 | unsigned long end_jiffies = (utimeout * HZ / 1000) + jiffies; |
| 651 | 552 | |
| 652 | 553 | m = get_sdma_mqd(mqd); |
| 653 | − sdma_base_addr = get_sdma_base_addr(m); | |
| 554 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(m); | |
| 654 | 555 | |
| 655 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 556 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 656 | 557 | temp = temp & ~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK; |
| 657 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 558 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 658 | 559 | |
| 659 | 560 | while (true) { |
| 660 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 561 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 661 | 562 | if (temp & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 662 | 563 | break; |
| 663 | − if (time_after(jiffies, end_jiffies)) | |
| 564 | + if (time_after(jiffies, end_jiffies)) { | |
| 565 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 664 | 566 | return -ETIME; |
| 567 | + } | |
| 665 | 568 | usleep_range(500, 1000); |
| 666 | 569 | } |
| 667 | 570 | |
| 668 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, 0); | |
| 669 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 670 | − RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL) | | |
| 571 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, 0); | |
| 572 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 573 | + RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL) | | |
| 671 | 574 | SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK); |
| 672 | 575 | |
| 673 | − m->sdma_rlc_rb_rptr = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR); | |
| 576 | + m->sdma_rlc_rb_rptr = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR); | |
| 674 | 577 | |
| 675 | 578 | return 0; |
| 676 | 579 | } |
| @@ -758,24 +661,16 @@ static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd, · 81 unmodified lines | ||
| 758 | 661 | return watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + reg_offset]; |
| 759 | 662 | } |
| 760 | 663 | |
| 761 | −static bool get_atc_vmid_pasid_mapping_valid(struct kgd_dev *kgd, | |
| 762 | − uint8_t vmid) | |
| 664 | +static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | |
| 665 | + uint8_t vmid, uint16_t *p_pasid) | |
| 763 | 666 | { |
| 764 | − uint32_t reg; | |
| 667 | + uint32_t value; | |
| 765 | 668 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; |
| 766 | 669 | |
| 767 | − reg = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); | |
| 768 | − return reg & ATC_VMID0_PASID_MAPPING__VALID_MASK; | |
| 769 | −} | |
| 770 | − | |
| 771 | −static uint16_t get_atc_vmid_pasid_mapping_pasid(struct kgd_dev *kgd, | |
| 772 | − uint8_t vmid) | |
| 773 | −{ | |
| 774 | − uint32_t reg; | |
| 775 | − struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | |
| 670 | + value = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); | |
| 671 | + *p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK; | |
| 776 | 672 | |
| 777 | − reg = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); | |
| 778 | − return reg & ATC_VMID0_PASID_MAPPING__PASID_MASK; | |
| 673 | + return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | |
| 779 | 674 | } |
| 780 | 675 | |
| 781 | 676 | static void set_scratch_backing_va(struct kgd_dev *kgd, |
| @@ -855,3 +750,28 @@ static uint32_t read_vmid_from_vmfault_reg(struct kgd_dev *kgd) · 73 unmodified lines | ||
| 855 | 750 | |
| 856 | 751 | return REG_GET_FIELD(status, VM_CONTEXT1_PROTECTION_FAULT_STATUS, VMID); |
| 857 | 752 | } |
| 753 | + | |
| 754 | +const struct kfd2kgd_calls gfx_v7_kfd2kgd = { | |
| 755 | + .program_sh_mem_settings = kgd_program_sh_mem_settings, | |
| 756 | + .set_pasid_vmid_mapping = kgd_set_pasid_vmid_mapping, | |
| 757 | + .init_interrupts = kgd_init_interrupts, | |
| 758 | + .hqd_load = kgd_hqd_load, | |
| 759 | + .hqd_sdma_load = kgd_hqd_sdma_load, | |
| 760 | + .hqd_dump = kgd_hqd_dump, | |
| 761 | + .hqd_sdma_dump = kgd_hqd_sdma_dump, | |
| 762 | + .hqd_is_occupied = kgd_hqd_is_occupied, | |
| 763 | + .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | |
| 764 | + .hqd_destroy = kgd_hqd_destroy, | |
| 765 | + .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | |
| 766 | + .address_watch_disable = kgd_address_watch_disable, | |
| 767 | + .address_watch_execute = kgd_address_watch_execute, | |
| 768 | + .wave_control_execute = kgd_wave_control_execute, | |
| 769 | + .address_watch_get_offset = kgd_address_watch_get_offset, | |
| 770 | + .get_atc_vmid_pasid_mapping_info = get_atc_vmid_pasid_mapping_info, | |
| 771 | + .set_scratch_backing_va = set_scratch_backing_va, | |
| 772 | + .get_tile_config = get_tile_config, | |
| 773 | + .set_vm_context_page_table_base = set_vm_context_page_table_base, | |
| 774 | + .invalidate_tlbs = invalidate_tlbs, | |
| 775 | + .invalidate_tlbs_vmid = invalidate_tlbs_vmid, | |
| 776 | + .read_vmid_from_vmfault_reg = read_vmid_from_vmfault_reg, | |
| 777 | +}; | |
| @@ -20,9 +20,6 @@ · 19 unmodified lines | ||
| 20 | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | 21 | */ |
| 22 | 22 | |
| 23 | −#include <linux/module.h> | |
| 24 | −#include <linux/fdtable.h> | |
| 25 | −#include <linux/uaccess.h> | |
| 26 | 23 | #include <linux/mmu_context.h> |
| 27 | 24 | |
| 28 | 25 | #include "amdgpu.h" |
| @@ -44,62 +41,6 @@ enum hqd_dequeue_request_type { · 15 unmodified lines | ||
| 44 | 41 | RESET_WAVES |
| 45 | 42 | }; |
| 46 | 43 | |
| 47 | −/* | |
| 48 | − * Register access functions | |
| 49 | − */ | |
| 50 | − | |
| 51 | −static void kgd_program_sh_mem_settings(struct kgd_dev *kgd, uint32_t vmid, | |
| 52 | − uint32_t sh_mem_config, | |
| 53 | − uint32_t sh_mem_ape1_base, uint32_t sh_mem_ape1_limit, | |
| 54 | − uint32_t sh_mem_bases); | |
| 55 | −static int kgd_set_pasid_vmid_mapping(struct kgd_dev *kgd, unsigned int pasid, | |
| 56 | − unsigned int vmid); | |
| 57 | −static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id); | |
| 58 | −static int kgd_hqd_load(struct kgd_dev *kgd, void *mqd, uint32_t pipe_id, | |
| 59 | − uint32_t queue_id, uint32_t __user *wptr, | |
| 60 | − uint32_t wptr_shift, uint32_t wptr_mask, | |
| 61 | − struct mm_struct *mm); | |
| 62 | −static int kgd_hqd_dump(struct kgd_dev *kgd, | |
| 63 | − uint32_t pipe_id, uint32_t queue_id, | |
| 64 | − uint32_t (**dump)[2], uint32_t *n_regs); | |
| 65 | −static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, | |
| 66 | − uint32_t __user *wptr, struct mm_struct *mm); | |
| 67 | −static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, | |
| 68 | − uint32_t engine_id, uint32_t queue_id, | |
| 69 | − uint32_t (**dump)[2], uint32_t *n_regs); | |
| 70 | −static bool kgd_hqd_is_occupied(struct kgd_dev *kgd, uint64_t queue_address, | |
| 71 | − uint32_t pipe_id, uint32_t queue_id); | |
| 72 | −static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd); | |
| 73 | −static int kgd_hqd_destroy(struct kgd_dev *kgd, void *mqd, | |
| 74 | − enum kfd_preempt_type reset_type, | |
| 75 | − unsigned int utimeout, uint32_t pipe_id, | |
| 76 | − uint32_t queue_id); | |
| 77 | −static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, | |
| 78 | − unsigned int utimeout); | |
| 79 | −static int kgd_address_watch_disable(struct kgd_dev *kgd); | |
| 80 | −static int kgd_address_watch_execute(struct kgd_dev *kgd, | |
| 81 | − unsigned int watch_point_id, | |
| 82 | − uint32_t cntl_val, | |
| 83 | − uint32_t addr_hi, | |
| 84 | − uint32_t addr_lo); | |
| 85 | −static int kgd_wave_control_execute(struct kgd_dev *kgd, | |
| 86 | − uint32_t gfx_index_val, | |
| 87 | − uint32_t sq_cmd); | |
| 88 | −static uint32_t kgd_address_watch_get_offset(struct kgd_dev *kgd, | |
| 89 | − unsigned int watch_point_id, | |
| 90 | − unsigned int reg_offset); | |
| 91 | − | |
| 92 | −static bool get_atc_vmid_pasid_mapping_valid(struct kgd_dev *kgd, | |
| 93 | − uint8_t vmid); | |
| 94 | −static uint16_t get_atc_vmid_pasid_mapping_pasid(struct kgd_dev *kgd, | |
| 95 | − uint8_t vmid); | |
| 96 | −static void set_scratch_backing_va(struct kgd_dev *kgd, | |
| 97 | − uint64_t va, uint32_t vmid); | |
| 98 | −static void set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, | |
| 99 | − uint64_t page_table_base); | |
| 100 | −static int invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid); | |
| 101 | −static int invalidate_tlbs_vmid(struct kgd_dev *kgd, uint16_t vmid); | |
| 102 | − | |
| 103 | 44 | /* Because of REG_GET_FIELD() being used, we put this function in the |
| 104 | 45 | * asic specific file. |
| 105 | 46 | */ |
| @@ -125,38 +66,6 @@ static int get_tile_config(struct kgd_dev *kgd, · 19 unmodified lines | ||
| 125 | 66 | return 0; |
| 126 | 67 | } |
| 127 | 68 | |
| 128 | −static const struct kfd2kgd_calls kfd2kgd = { | |
| 129 | − .program_sh_mem_settings = kgd_program_sh_mem_settings, | |
| 130 | − .set_pasid_vmid_mapping = kgd_set_pasid_vmid_mapping, | |
| 131 | − .init_interrupts = kgd_init_interrupts, | |
| 132 | − .hqd_load = kgd_hqd_load, | |
| 133 | − .hqd_sdma_load = kgd_hqd_sdma_load, | |
| 134 | − .hqd_dump = kgd_hqd_dump, | |
| 135 | − .hqd_sdma_dump = kgd_hqd_sdma_dump, | |
| 136 | − .hqd_is_occupied = kgd_hqd_is_occupied, | |
| 137 | − .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | |
| 138 | − .hqd_destroy = kgd_hqd_destroy, | |
| 139 | − .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | |
| 140 | − .address_watch_disable = kgd_address_watch_disable, | |
| 141 | − .address_watch_execute = kgd_address_watch_execute, | |
| 142 | − .wave_control_execute = kgd_wave_control_execute, | |
| 143 | − .address_watch_get_offset = kgd_address_watch_get_offset, | |
| 144 | − .get_atc_vmid_pasid_mapping_pasid = | |
| 145 | − get_atc_vmid_pasid_mapping_pasid, | |
| 146 | − .get_atc_vmid_pasid_mapping_valid = | |
| 147 | − get_atc_vmid_pasid_mapping_valid, | |
| 148 | − .set_scratch_backing_va = set_scratch_backing_va, | |
| 149 | − .get_tile_config = get_tile_config, | |
| 150 | − .set_vm_context_page_table_base = set_vm_context_page_table_base, | |
| 151 | − .invalidate_tlbs = invalidate_tlbs, | |
| 152 | − .invalidate_tlbs_vmid = invalidate_tlbs_vmid, | |
| 153 | −}; | |
| 154 | − | |
| 155 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_8_0_get_functions(void) | |
| 156 | −{ | |
| 157 | − return (struct kfd2kgd_calls *)&kfd2kgd; | |
| 158 | −} | |
| 159 | − | |
| 160 | 69 | static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) |
| 161 | 70 | { |
| 162 | 71 | return (struct amdgpu_device *)kgd; |
| @@ -260,13 +169,15 @@ static int kgd_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id) · 97 unmodified lines | ||
| 260 | 169 | return 0; |
| 261 | 170 | } |
| 262 | 171 | |
| 263 | −static inline uint32_t get_sdma_base_addr(struct vi_sdma_mqd *m) | |
| 172 | +static inline uint32_t get_sdma_rlc_reg_offset(struct vi_sdma_mqd *m) | |
| 264 | 173 | { |
| 265 | 174 | uint32_t retval; |
| 266 | 175 | |
| 267 | 176 | retval = m->sdma_engine_id * SDMA1_REGISTER_OFFSET + |
| 268 | 177 | m->sdma_queue_id * KFD_VI_SDMA_QUEUE_OFFSET; |
| 269 | − pr_debug("sdma base address: 0x%x\n", retval); | |
| 178 | + | |
| 179 | + pr_debug("RLC register offset for SDMA%d RLC%d: 0x%x\n", | |
| 180 | + m->sdma_engine_id, m->sdma_queue_id, retval); | |
| 270 | 181 | |
| 271 | 182 | return retval; |
| 272 | 183 | } |
| @@ -398,59 +309,51 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, · 125 unmodified lines | ||
| 398 | 309 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 399 | 310 | struct vi_sdma_mqd *m; |
| 400 | 311 | unsigned long end_jiffies; |
| 401 | − uint32_t sdma_base_addr; | |
| 312 | + uint32_t sdma_rlc_reg_offset; | |
| 402 | 313 | uint32_t data; |
| 403 | 314 | |
| 404 | 315 | m = get_sdma_mqd(mqd); |
| 405 | − sdma_base_addr = get_sdma_base_addr(m); | |
| 406 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 316 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(m); | |
| 317 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 407 | 318 | m->sdmax_rlcx_rb_cntl & (~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK)); |
| 408 | 319 | |
| 409 | 320 | end_jiffies = msecs_to_jiffies(2000) + jiffies; |
| 410 | 321 | while (true) { |
| 411 | − data = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 322 | + data = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 412 | 323 | if (data & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 413 | 324 | break; |
| 414 | − if (time_after(jiffies, end_jiffies)) | |
| 325 | + if (time_after(jiffies, end_jiffies)) { | |
| 326 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 415 | 327 | return -ETIME; |
| 328 | + } | |
| 416 | 329 | usleep_range(500, 1000); |
| 417 | 330 | } |
| 418 | − if (m->sdma_engine_id) { | |
| 419 | − data = RREG32(mmSDMA1_GFX_CONTEXT_CNTL); | |
| 420 | − data = REG_SET_FIELD(data, SDMA1_GFX_CONTEXT_CNTL, | |
| 421 | − RESUME_CTX, 0); | |
| 422 | − WREG32(mmSDMA1_GFX_CONTEXT_CNTL, data); | |
| 423 | − } else { | |
| 424 | − data = RREG32(mmSDMA0_GFX_CONTEXT_CNTL); | |
| 425 | − data = REG_SET_FIELD(data, SDMA0_GFX_CONTEXT_CNTL, | |
| 426 | − RESUME_CTX, 0); | |
| 427 | − WREG32(mmSDMA0_GFX_CONTEXT_CNTL, data); | |
| 428 | − } | |
| 429 | 331 | |
| 430 | 332 | data = REG_SET_FIELD(m->sdmax_rlcx_doorbell, SDMA0_RLC0_DOORBELL, |
| 431 | 333 | ENABLE, 1); |
| 432 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, data); | |
| 433 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR, m->sdmax_rlcx_rb_rptr); | |
| 334 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, data); | |
| 335 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR, | |
| 336 | + m->sdmax_rlcx_rb_rptr); | |
| 434 | 337 | |
| 435 | 338 | if (read_user_wptr(mm, wptr, data)) |
| 436 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, data); | |
| 339 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, data); | |
| 437 | 340 | else |
| 438 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, | |
| 341 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, | |
| 439 | 342 | m->sdmax_rlcx_rb_rptr); |
| 440 | 343 | |
| 441 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_VIRTUAL_ADDR, | |
| 344 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_VIRTUAL_ADDR, | |
| 442 | 345 | m->sdmax_rlcx_virtual_addr); |
| 443 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE, m->sdmax_rlcx_rb_base); | |
| 444 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE_HI, | |
| 346 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE, m->sdmax_rlcx_rb_base); | |
| 347 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE_HI, | |
| 445 | 348 | m->sdmax_rlcx_rb_base_hi); |
| 446 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 349 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 447 | 350 | m->sdmax_rlcx_rb_rptr_addr_lo); |
| 448 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 351 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 449 | 352 | m->sdmax_rlcx_rb_rptr_addr_hi); |
| 450 | 353 | |
| 451 | 354 | data = REG_SET_FIELD(m->sdmax_rlcx_rb_cntl, SDMA0_RLC0_RB_CNTL, |
| 452 | 355 | RB_ENABLE, 1); |
| 453 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, data); | |
| 356 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, data); | |
| 454 | 357 | |
| 455 | 358 | return 0; |
| 456 | 359 | } |
| @@ -517,13 +420,13 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 60 unmodified lines | ||
| 517 | 420 | { |
| 518 | 421 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 519 | 422 | struct vi_sdma_mqd *m; |
| 520 | − uint32_t sdma_base_addr; | |
| 423 | + uint32_t sdma_rlc_reg_offset; | |
| 521 | 424 | uint32_t sdma_rlc_rb_cntl; |
| 522 | 425 | |
| 523 | 426 | m = get_sdma_mqd(mqd); |
| 524 | − sdma_base_addr = get_sdma_base_addr(m); | |
| 427 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(m); | |
| 525 | 428 | |
| 526 | − sdma_rlc_rb_cntl = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 429 | + sdma_rlc_rb_cntl = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 527 | 430 | |
| 528 | 431 | if (sdma_rlc_rb_cntl & SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK) |
| 529 | 432 | return true; |
| @@ -641,54 +544,48 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, · 111 unmodified lines | ||
| 641 | 544 | { |
| 642 | 545 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 643 | 546 | struct vi_sdma_mqd *m; |
| 644 | − uint32_t sdma_base_addr; | |
| 547 | + uint32_t sdma_rlc_reg_offset; | |
| 645 | 548 | uint32_t temp; |
| 646 | 549 | unsigned long end_jiffies = (utimeout * HZ / 1000) + jiffies; |
| 647 | 550 | |
| 648 | 551 | m = get_sdma_mqd(mqd); |
| 649 | − sdma_base_addr = get_sdma_base_addr(m); | |
| 552 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(m); | |
| 650 | 553 | |
| 651 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 554 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 652 | 555 | temp = temp & ~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK; |
| 653 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 556 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 654 | 557 | |
| 655 | 558 | while (true) { |
| 656 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 559 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 657 | 560 | if (temp & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 658 | 561 | break; |
| 659 | − if (time_after(jiffies, end_jiffies)) | |
| 562 | + if (time_after(jiffies, end_jiffies)) { | |
| 563 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 660 | 564 | return -ETIME; |
| 565 | + } | |
| 661 | 566 | usleep_range(500, 1000); |
| 662 | 567 | } |
| 663 | 568 | |
| 664 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, 0); | |
| 665 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 666 | − RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL) | | |
| 569 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, 0); | |
| 570 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 571 | + RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL) | | |
| 667 | 572 | SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK); |
| 668 | 573 | |
| 669 | − m->sdmax_rlcx_rb_rptr = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR); | |
| 574 | + m->sdmax_rlcx_rb_rptr = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR); | |
| 670 | 575 | |
| 671 | 576 | return 0; |
| 672 | 577 | } |
| 673 | 578 | |
| 674 | −static bool get_atc_vmid_pasid_mapping_valid(struct kgd_dev *kgd, | |
| 675 | − uint8_t vmid) | |
| 579 | +static bool get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | |
| 580 | + uint8_t vmid, uint16_t *p_pasid) | |
| 676 | 581 | { |
| 677 | − uint32_t reg; | |
| 582 | + uint32_t value; | |
| 678 | 583 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; |
| 679 | 584 | |
| 680 | − reg = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); | |
| 681 | − return reg & ATC_VMID0_PASID_MAPPING__VALID_MASK; | |
| 682 | −} | |
| 585 | + value = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); | |
| 586 | + *p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK; | |
| 683 | 587 | |
| 684 | −static uint16_t get_atc_vmid_pasid_mapping_pasid(struct kgd_dev *kgd, | |
| 685 | − uint8_t vmid) | |
| 686 | −{ | |
| 687 | − uint32_t reg; | |
| 688 | − struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | |
| 689 | − | |
| 690 | − reg = RREG32(mmATC_VMID0_PASID_MAPPING + vmid); | |
| 691 | − return reg & ATC_VMID0_PASID_MAPPING__PASID_MASK; | |
| 588 | + return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | |
| 692 | 589 | } |
| 693 | 590 | |
| 694 | 591 | static int kgd_address_watch_disable(struct kgd_dev *kgd) |
| @@ -798,3 +695,28 @@ static int invalidate_tlbs_vmid(struct kgd_dev *kgd, uint16_t vmid) · 103 unmodified lines | ||
| 798 | 695 | RREG32(mmVM_INVALIDATE_RESPONSE); |
| 799 | 696 | return 0; |
| 800 | 697 | } |
| 698 | + | |
| 699 | +const struct kfd2kgd_calls gfx_v8_kfd2kgd = { | |
| 700 | + .program_sh_mem_settings = kgd_program_sh_mem_settings, | |
| 701 | + .set_pasid_vmid_mapping = kgd_set_pasid_vmid_mapping, | |
| 702 | + .init_interrupts = kgd_init_interrupts, | |
| 703 | + .hqd_load = kgd_hqd_load, | |
| 704 | + .hqd_sdma_load = kgd_hqd_sdma_load, | |
| 705 | + .hqd_dump = kgd_hqd_dump, | |
| 706 | + .hqd_sdma_dump = kgd_hqd_sdma_dump, | |
| 707 | + .hqd_is_occupied = kgd_hqd_is_occupied, | |
| 708 | + .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | |
| 709 | + .hqd_destroy = kgd_hqd_destroy, | |
| 710 | + .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | |
| 711 | + .address_watch_disable = kgd_address_watch_disable, | |
| 712 | + .address_watch_execute = kgd_address_watch_execute, | |
| 713 | + .wave_control_execute = kgd_wave_control_execute, | |
| 714 | + .address_watch_get_offset = kgd_address_watch_get_offset, | |
| 715 | + .get_atc_vmid_pasid_mapping_info = | |
| 716 | + get_atc_vmid_pasid_mapping_info, | |
| 717 | + .set_scratch_backing_va = set_scratch_backing_va, | |
| 718 | + .get_tile_config = get_tile_config, | |
| 719 | + .set_vm_context_page_table_base = set_vm_context_page_table_base, | |
| 720 | + .invalidate_tlbs = invalidate_tlbs, | |
| 721 | + .invalidate_tlbs_vmid = invalidate_tlbs_vmid, | |
| 722 | +}; | |
| @@ -19,17 +19,10 @@ · 18 unmodified lines | ||
| 19 | 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR |
| 20 | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | 21 | */ |
| 22 | − | |
| 23 | −#define pr_fmt(fmt) "kfd2kgd: " fmt | |
| 24 | − | |
| 25 | −#include <linux/module.h> | |
| 26 | −#include <linux/fdtable.h> | |
| 27 | −#include <linux/uaccess.h> | |
| 28 | 22 | #include <linux/mmu_context.h> |
| 29 | 23 | |
| 30 | 24 | #include "amdgpu.h" |
| 31 | 25 | #include "amdgpu_amdkfd.h" |
| 32 | −#include "soc15_hw_ip.h" | |
| 33 | 26 | #include "gc/gc_9_0_offset.h" |
| 34 | 27 | #include "gc/gc_9_0_sh_mask.h" |
| 35 | 28 | #include "vega10_enum.h" |
| @@ -50,9 +43,6 @@ · 14 unmodified lines | ||
| 50 | 43 | #include "gmc_v9_0.h" |
| 51 | 44 | |
| 52 | 45 | |
| 53 | −#define V9_PIPE_PER_MEC (4) | |
| 54 | −#define V9_QUEUES_PER_PIPE_MEC (8) | |
| 55 | − | |
| 56 | 46 | enum hqd_dequeue_request_type { |
| 57 | 47 | NO_ACTION = 0, |
| 58 | 48 | DRAIN_PIPE, |
| @@ -226,22 +216,21 @@ int kgd_gfx_v9_init_interrupts(struct kgd_dev *kgd, uint32_t pipe_id) · 167 unmodified lines | ||
| 226 | 216 | return 0; |
| 227 | 217 | } |
| 228 | 218 | |
| 229 | −static uint32_t get_sdma_base_addr(struct amdgpu_device *adev, | |
| 219 | +static uint32_t get_sdma_rlc_reg_offset(struct amdgpu_device *adev, | |
| 230 | 220 | unsigned int engine_id, |
| 231 | 221 | unsigned int queue_id) |
| 232 | 222 | { |
| 233 | − uint32_t base[2] = { | |
| 223 | + uint32_t sdma_engine_reg_base[2] = { | |
| 234 | 224 | SOC15_REG_OFFSET(SDMA0, 0, |
| 235 | 225 | mmSDMA0_RLC0_RB_CNTL) - mmSDMA0_RLC0_RB_CNTL, |
| 236 | 226 | SOC15_REG_OFFSET(SDMA1, 0, |
| 237 | 227 | mmSDMA1_RLC0_RB_CNTL) - mmSDMA1_RLC0_RB_CNTL |
| 238 | 228 | }; |
| 239 | − uint32_t retval; | |
| 229 | + uint32_t retval = sdma_engine_reg_base[engine_id] | |
| 230 | + + queue_id * (mmSDMA0_RLC1_RB_CNTL - mmSDMA0_RLC0_RB_CNTL); | |
| 240 | 231 | |
| 241 | − retval = base[engine_id] + queue_id * (mmSDMA0_RLC1_RB_CNTL - | |
| 242 | − mmSDMA0_RLC0_RB_CNTL); | |
| 243 | − | |
| 244 | − pr_debug("sdma base address: 0x%x\n", retval); | |
| 232 | + pr_debug("RLC register offset for SDMA%d RLC%d: 0x%x\n", engine_id, | |
| 233 | + queue_id, retval); | |
| 245 | 234 | |
| 246 | 235 | return retval; |
| 247 | 236 | } |
| @@ -388,71 +377,67 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd, · 140 unmodified lines | ||
| 388 | 377 | { |
| 389 | 378 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 390 | 379 | struct v9_sdma_mqd *m; |
| 391 | − uint32_t sdma_base_addr, sdmax_gfx_context_cntl; | |
| 380 | + uint32_t sdma_rlc_reg_offset; | |
| 392 | 381 | unsigned long end_jiffies; |
| 393 | 382 | uint32_t data; |
| 394 | 383 | uint64_t data64; |
| 395 | 384 | uint64_t __user *wptr64 = (uint64_t __user *)wptr; |
| 396 | 385 | |
| 397 | 386 | m = get_sdma_mqd(mqd); |
| 398 | − sdma_base_addr = get_sdma_base_addr(adev, m->sdma_engine_id, | |
| 387 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, m->sdma_engine_id, | |
| 399 | 388 | m->sdma_queue_id); |
| 400 | − sdmax_gfx_context_cntl = m->sdma_engine_id ? | |
| 401 | − SOC15_REG_OFFSET(SDMA1, 0, mmSDMA1_GFX_CONTEXT_CNTL) : | |
| 402 | − SOC15_REG_OFFSET(SDMA0, 0, mmSDMA0_GFX_CONTEXT_CNTL); | |
| 403 | 389 | |
| 404 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 390 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 405 | 391 | m->sdmax_rlcx_rb_cntl & (~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK)); |
| 406 | 392 | |
| 407 | 393 | end_jiffies = msecs_to_jiffies(2000) + jiffies; |
| 408 | 394 | while (true) { |
| 409 | − data = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 395 | + data = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 410 | 396 | if (data & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 411 | 397 | break; |
| 412 | − if (time_after(jiffies, end_jiffies)) | |
| 398 | + if (time_after(jiffies, end_jiffies)) { | |
| 399 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 413 | 400 | return -ETIME; |
| 401 | + } | |
| 414 | 402 | usleep_range(500, 1000); |
| 415 | 403 | } |
| 416 | − data = RREG32(sdmax_gfx_context_cntl); | |
| 417 | − data = REG_SET_FIELD(data, SDMA0_GFX_CONTEXT_CNTL, | |
| 418 | − RESUME_CTX, 0); | |
| 419 | − WREG32(sdmax_gfx_context_cntl, data); | |
| 420 | 404 | |
| 421 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL_OFFSET, | |
| 405 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL_OFFSET, | |
| 422 | 406 | m->sdmax_rlcx_doorbell_offset); |
| 423 | 407 | |
| 424 | 408 | data = REG_SET_FIELD(m->sdmax_rlcx_doorbell, SDMA0_RLC0_DOORBELL, |
| 425 | 409 | ENABLE, 1); |
| 426 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, data); | |
| 427 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR, m->sdmax_rlcx_rb_rptr); | |
| 428 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_HI, | |
| 410 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, data); | |
| 411 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR, | |
| 412 | + m->sdmax_rlcx_rb_rptr); | |
| 413 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_HI, | |
| 429 | 414 | m->sdmax_rlcx_rb_rptr_hi); |
| 430 | 415 | |
| 431 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 1); | |
| 416 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 1); | |
| 432 | 417 | if (read_user_wptr(mm, wptr64, data64)) { |
| 433 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, | |
| 418 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, | |
| 434 | 419 | lower_32_bits(data64)); |
| 435 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 420 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 436 | 421 | upper_32_bits(data64)); |
| 437 | 422 | } else { |
| 438 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR, | |
| 423 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR, | |
| 439 | 424 | m->sdmax_rlcx_rb_rptr); |
| 440 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 425 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_WPTR_HI, | |
| 441 | 426 | m->sdmax_rlcx_rb_rptr_hi); |
| 442 | 427 | } |
| 443 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 0); | |
| 428 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_MINOR_PTR_UPDATE, 0); | |
| 444 | 429 | |
| 445 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE, m->sdmax_rlcx_rb_base); | |
| 446 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_BASE_HI, | |
| 430 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE, m->sdmax_rlcx_rb_base); | |
| 431 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_BASE_HI, | |
| 447 | 432 | m->sdmax_rlcx_rb_base_hi); |
| 448 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 433 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_LO, | |
| 449 | 434 | m->sdmax_rlcx_rb_rptr_addr_lo); |
| 450 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 435 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_ADDR_HI, | |
| 451 | 436 | m->sdmax_rlcx_rb_rptr_addr_hi); |
| 452 | 437 | |
| 453 | 438 | data = REG_SET_FIELD(m->sdmax_rlcx_rb_cntl, SDMA0_RLC0_RB_CNTL, |
| 454 | 439 | RB_ENABLE, 1); |
| 455 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, data); | |
| 440 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, data); | |
| 456 | 441 | |
| 457 | 442 | return 0; |
| 458 | 443 | } |
| @@ -462,7 +447,8 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, · 3 unmodified lines | ||
| 462 | 447 | uint32_t (**dump)[2], uint32_t *n_regs) |
| 463 | 448 | { |
| 464 | 449 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 465 | − uint32_t sdma_base_addr = get_sdma_base_addr(adev, engine_id, queue_id); | |
| 450 | + uint32_t sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, | |
| 451 | + engine_id, queue_id); | |
| 466 | 452 | uint32_t i = 0, reg; |
| 467 | 453 | #undef HQD_N_REGS |
| 468 | 454 | #define HQD_N_REGS (19+6+7+10) |
| @@ -472,15 +458,15 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd, · 3 unmodified lines | ||
| 472 | 458 | return -ENOMEM; |
| 473 | 459 | |
| 474 | 460 | for (reg = mmSDMA0_RLC0_RB_CNTL; reg <= mmSDMA0_RLC0_DOORBELL; reg++) |
| 475 | − DUMP_REG(sdma_base_addr + reg); | |
| 461 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 476 | 462 | for (reg = mmSDMA0_RLC0_STATUS; reg <= mmSDMA0_RLC0_CSA_ADDR_HI; reg++) |
| 477 | − DUMP_REG(sdma_base_addr + reg); | |
| 463 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 478 | 464 | for (reg = mmSDMA0_RLC0_IB_SUB_REMAIN; |
| 479 | 465 | reg <= mmSDMA0_RLC0_MINOR_PTR_UPDATE; reg++) |
| 480 | − DUMP_REG(sdma_base_addr + reg); | |
| 466 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 481 | 467 | for (reg = mmSDMA0_RLC0_MIDCMD_DATA0; |
| 482 | 468 | reg <= mmSDMA0_RLC0_MIDCMD_CNTL; reg++) |
| 483 | − DUMP_REG(sdma_base_addr + reg); | |
| 469 | + DUMP_REG(sdma_rlc_reg_offset + reg); | |
| 484 | 470 | |
| 485 | 471 | WARN_ON_ONCE(i != HQD_N_REGS); |
| 486 | 472 | *n_regs = i; |
| @@ -514,14 +500,14 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd) · 27 unmodified lines | ||
| 514 | 500 | { |
| 515 | 501 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 516 | 502 | struct v9_sdma_mqd *m; |
| 517 | − uint32_t sdma_base_addr; | |
| 503 | + uint32_t sdma_rlc_reg_offset; | |
| 518 | 504 | uint32_t sdma_rlc_rb_cntl; |
| 519 | 505 | |
| 520 | 506 | m = get_sdma_mqd(mqd); |
| 521 | − sdma_base_addr = get_sdma_base_addr(adev, m->sdma_engine_id, | |
| 507 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, m->sdma_engine_id, | |
| 522 | 508 | m->sdma_queue_id); |
| 523 | 509 | |
| 524 | − sdma_rlc_rb_cntl = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 510 | + sdma_rlc_rb_cntl = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 525 | 511 | |
| 526 | 512 | if (sdma_rlc_rb_cntl & SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK) |
| 527 | 513 | return true; |
| @@ -584,59 +570,52 @@ static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd, · 56 unmodified lines | ||
| 584 | 570 | { |
| 585 | 571 | struct amdgpu_device *adev = get_amdgpu_device(kgd); |
| 586 | 572 | struct v9_sdma_mqd *m; |
| 587 | − uint32_t sdma_base_addr; | |
| 573 | + uint32_t sdma_rlc_reg_offset; | |
| 588 | 574 | uint32_t temp; |
| 589 | 575 | unsigned long end_jiffies = (utimeout * HZ / 1000) + jiffies; |
| 590 | 576 | |
| 591 | 577 | m = get_sdma_mqd(mqd); |
| 592 | − sdma_base_addr = get_sdma_base_addr(adev, m->sdma_engine_id, | |
| 578 | + sdma_rlc_reg_offset = get_sdma_rlc_reg_offset(adev, m->sdma_engine_id, | |
| 593 | 579 | m->sdma_queue_id); |
| 594 | 580 | |
| 595 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL); | |
| 581 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL); | |
| 596 | 582 | temp = temp & ~SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK; |
| 597 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 583 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, temp); | |
| 598 | 584 | |
| 599 | 585 | while (true) { |
| 600 | − temp = RREG32(sdma_base_addr + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 586 | + temp = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_CONTEXT_STATUS); | |
| 601 | 587 | if (temp & SDMA0_RLC0_CONTEXT_STATUS__IDLE_MASK) |
| 602 | 588 | break; |
| 603 | − if (time_after(jiffies, end_jiffies)) | |
| 589 | + if (time_after(jiffies, end_jiffies)) { | |
| 590 | + pr_err("SDMA RLC not idle in %s\n", __func__); | |
| 604 | 591 | return -ETIME; |
| 592 | + } | |
| 605 | 593 | usleep_range(500, 1000); |
| 606 | 594 | } |
| 607 | 595 | |
| 608 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_DOORBELL, 0); | |
| 609 | − WREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL, | |
| 610 | − RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_CNTL) | | |
| 596 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_DOORBELL, 0); | |
| 597 | + WREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL, | |
| 598 | + RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_CNTL) | | |
| 611 | 599 | SDMA0_RLC0_RB_CNTL__RB_ENABLE_MASK); |
| 612 | 600 | |
| 613 | − m->sdmax_rlcx_rb_rptr = RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR); | |
| 601 | + m->sdmax_rlcx_rb_rptr = RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR); | |
| 614 | 602 | m->sdmax_rlcx_rb_rptr_hi = |
| 615 | − RREG32(sdma_base_addr + mmSDMA0_RLC0_RB_RPTR_HI); | |
| 603 | + RREG32(sdma_rlc_reg_offset + mmSDMA0_RLC0_RB_RPTR_HI); | |
| 616 | 604 | |
| 617 | 605 | return 0; |
| 618 | 606 | } |
| 619 | 607 | |
| 620 | −bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_valid(struct kgd_dev *kgd, | |
| 621 | − uint8_t vmid) | |
| 608 | +bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | |
| 609 | + uint8_t vmid, uint16_t *p_pasid) | |
| 622 | 610 | { |
| 623 | − uint32_t reg; | |
| 611 | + uint32_t value; | |
| 624 | 612 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; |
| 625 | 613 | |
| 626 | − reg = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) | |
| 614 | + value = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) | |
| 627 | 615 | + vmid); |
| 628 | − return reg & ATC_VMID0_PASID_MAPPING__VALID_MASK; | |
| 629 | −} | |
| 630 | − | |
| 631 | −uint16_t kgd_gfx_v9_get_atc_vmid_pasid_mapping_pasid(struct kgd_dev *kgd, | |
| 632 | − uint8_t vmid) | |
| 633 | −{ | |
| 634 | − uint32_t reg; | |
| 635 | − struct amdgpu_device *adev = (struct amdgpu_device *) kgd; | |
| 616 | + *p_pasid = value & ATC_VMID0_PASID_MAPPING__PASID_MASK; | |
| 636 | 617 | |
| 637 | − reg = RREG32(SOC15_REG_OFFSET(ATHUB, 0, mmATC_VMID0_PASID_MAPPING) | |
| 638 | − + vmid); | |
| 639 | − return reg & ATC_VMID0_PASID_MAPPING__PASID_MASK; | |
| 618 | + return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | |
| 640 | 619 | } |
| 641 | 620 | |
| 642 | 621 | static int invalidate_tlbs_with_kiq(struct amdgpu_device *adev, uint16_t pasid, |
| @@ -671,6 +650,8 @@ int kgd_gfx_v9_invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid) · 28 unmodified lines | ||
| 671 | 650 | { |
| 672 | 651 | struct amdgpu_device *adev = (struct amdgpu_device *) kgd; |
| 673 | 652 | int vmid, i; |
| 653 | + uint16_t queried_pasid; | |
| 654 | + bool ret; | |
| 674 | 655 | struct amdgpu_ring *ring = &adev->gfx.kiq.ring; |
| 675 | 656 | uint32_t flush_type = 0; |
| 676 | 657 | |
| @@ -686,14 +667,14 @@ int kgd_gfx_v9_invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid) · 9 unmodified lines | ||
| 686 | 667 | for (vmid = 0; vmid < 16; vmid++) { |
| 687 | 668 | if (!amdgpu_amdkfd_is_kfd_vmid(adev, vmid)) |
| 688 | 669 | continue; |
| 689 | − if (kgd_gfx_v9_get_atc_vmid_pasid_mapping_valid(kgd, vmid)) { | |
| 690 | − if (kgd_gfx_v9_get_atc_vmid_pasid_mapping_pasid(kgd, vmid) | |
| 691 | − == pasid) { | |
| 692 | − for (i = 0; i < adev->num_vmhubs; i++) | |
| 693 | − amdgpu_gmc_flush_gpu_tlb(adev, vmid, | |
| 694 | − i, flush_type); | |
| 695 | − break; | |
| 696 | − } | |
| 670 | + | |
| 671 | + ret = kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(kgd, vmid, | |
| 672 | + &queried_pasid); | |
| 673 | + if (ret && queried_pasid == pasid) { | |
| 674 | + for (i = 0; i < adev->num_vmhubs; i++) | |
| 675 | + amdgpu_gmc_flush_gpu_tlb(adev, vmid, | |
| 676 | + i, flush_type); | |
| 677 | + break; | |
| 697 | 678 | } |
| 698 | 679 | } |
| 699 | 680 | |
| @@ -777,15 +758,6 @@ uint32_t kgd_gfx_v9_address_watch_get_offset(struct kgd_dev *kgd, · 77 unmodified lines | ||
| 777 | 758 | return 0; |
| 778 | 759 | } |
| 779 | 760 | |
| 780 | −void kgd_gfx_v9_set_scratch_backing_va(struct kgd_dev *kgd, | |
| 781 | − uint64_t va, uint32_t vmid) | |
| 782 | −{ | |
| 783 | − /* No longer needed on GFXv9. The scratch base address is | |
| 784 | − * passed to the shader by the CP. It's the user mode driver's | |
| 785 | − * responsibility. | |
| 786 | − */ | |
| 787 | −} | |
| 788 | − | |
| 789 | 761 | void kgd_gfx_v9_set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, |
| 790 | 762 | uint64_t page_table_base) |
| 791 | 763 | { |
| @@ -811,7 +783,7 @@ void kgd_gfx_v9_set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmi · 19 unmodified lines | ||
| 811 | 783 | gfxhub_v1_0_setup_vm_pt_regs(adev, vmid, page_table_base); |
| 812 | 784 | } |
| 813 | 785 | |
| 814 | −static const struct kfd2kgd_calls kfd2kgd = { | |
| 786 | +const struct kfd2kgd_calls gfx_v9_kfd2kgd = { | |
| 815 | 787 | .program_sh_mem_settings = kgd_gfx_v9_program_sh_mem_settings, |
| 816 | 788 | .set_pasid_vmid_mapping = kgd_gfx_v9_set_pasid_vmid_mapping, |
| 817 | 789 | .init_interrupts = kgd_gfx_v9_init_interrupts, |
| @@ -827,19 +799,11 @@ static const struct kfd2kgd_calls kfd2kgd = { · 9 unmodified lines | ||
| 827 | 799 | .address_watch_execute = kgd_gfx_v9_address_watch_execute, |
| 828 | 800 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, |
| 829 | 801 | .address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset, |
| 830 | − .get_atc_vmid_pasid_mapping_pasid = | |
| 831 | − kgd_gfx_v9_get_atc_vmid_pasid_mapping_pasid, | |
| 832 | − .get_atc_vmid_pasid_mapping_valid = | |
| 833 | − kgd_gfx_v9_get_atc_vmid_pasid_mapping_valid, | |
| 834 | − .set_scratch_backing_va = kgd_gfx_v9_set_scratch_backing_va, | |
| 802 | + .get_atc_vmid_pasid_mapping_info = | |
| 803 | + kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, | |
| 835 | 804 | .get_tile_config = kgd_gfx_v9_get_tile_config, |
| 836 | 805 | .set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base, |
| 837 | 806 | .invalidate_tlbs = kgd_gfx_v9_invalidate_tlbs, |
| 838 | 807 | .invalidate_tlbs_vmid = kgd_gfx_v9_invalidate_tlbs_vmid, |
| 839 | 808 | .get_hive_id = amdgpu_amdkfd_get_hive_id, |
| 840 | 809 | }; |
| 841 | − | |
| 842 | −struct kfd2kgd_calls *amdgpu_amdkfd_gfx_9_0_get_functions(void) | |
| 843 | −{ | |
| 844 | − return (struct kfd2kgd_calls *)&kfd2kgd; | |
| 845 | −} | |
| @@ -55,14 +55,10 @@ uint32_t kgd_gfx_v9_address_watch_get_offset(struct kgd_dev *kgd, · 54 unmodified lines | ||
| 55 | 55 | unsigned int watch_point_id, |
| 56 | 56 | unsigned int reg_offset); |
| 57 | 57 | |
| 58 | −bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_valid(struct kgd_dev *kgd, | |
| 59 | − uint8_t vmid); | |
| 60 | −uint16_t kgd_gfx_v9_get_atc_vmid_pasid_mapping_pasid(struct kgd_dev *kgd, | |
| 61 | − uint8_t vmid); | |
| 58 | +bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct kgd_dev *kgd, | |
| 59 | + uint8_t vmid, uint16_t *p_pasid); | |
| 62 | 60 | void kgd_gfx_v9_set_vm_context_page_table_base(struct kgd_dev *kgd, uint32_t vmid, |
| 63 | 61 | uint64_t page_table_base); |
| 64 | −void kgd_gfx_v9_set_scratch_backing_va(struct kgd_dev *kgd, | |
| 65 | − uint64_t va, uint32_t vmid); | |
| 66 | 62 | int kgd_gfx_v9_invalidate_tlbs(struct kgd_dev *kgd, uint16_t pasid); |
| 67 | 63 | int kgd_gfx_v9_invalidate_tlbs_vmid(struct kgd_dev *kgd, uint16_t vmid); |
| 68 | 64 | int kgd_gfx_v9_get_tile_config(struct kgd_dev *kgd, |
| @@ -19,9 +19,6 @@ · 18 unmodified lines | ||
| 19 | 19 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR |
| 20 | 20 | * OTHER DEALINGS IN THE SOFTWARE. |
| 21 | 21 | */ |
| 22 | − | |
| 23 | −#define pr_fmt(fmt) "kfd2kgd: " fmt | |
| 24 | − | |
| 25 | 22 | #include <linux/dma-buf.h> |
| 26 | 23 | #include <linux/list.h> |
| 27 | 24 | #include <linux/pagemap.h> |
| @@ -33,11 +30,6 @@ · 5 unmodified lines | ||
| 33 | 30 | #include "amdgpu_amdkfd.h" |
| 34 | 31 | #include "amdgpu_dma_buf.h" |
| 35 | 32 | |
| 36 | −/* Special VM and GART address alignment needed for VI pre-Fiji due to | |
| 37 | − * a HW bug. | |
| 38 | − */ | |
| 39 | −#define VI_BO_SIZE_ALIGN (0x8000) | |
| 40 | − | |
| 41 | 33 | /* BO flag to indicate a KFD userptr BO */ |
| 42 | 34 | #define AMDGPU_AMDKFD_USERPTR_BO (1ULL << 63) |
| 43 | 35 | |
| @@ -349,13 +341,46 @@ static int vm_update_pds(struct amdgpu_vm *vm, struct amdgpu_sync *sync) · 305 unmodified lines | ||
| 349 | 341 | struct amdgpu_device *adev = amdgpu_ttm_adev(pd->tbo.bdev); |
| 350 | 342 | int ret; |
| 351 | 343 | |
| 352 | − ret = amdgpu_vm_update_directories(adev, vm); | |
| 344 | + ret = amdgpu_vm_update_pdes(adev, vm, false); | |
| 353 | 345 | if (ret) |
| 354 | 346 | return ret; |
| 355 | 347 | |
| 356 | 348 | return amdgpu_sync_fence(NULL, sync, vm->last_update, false); |
| 357 | 349 | } |
| 358 | 350 | |
| 351 | +static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) | |
| 352 | +{ | |
| 353 | + struct amdgpu_device *bo_adev = amdgpu_ttm_adev(mem->bo->tbo.bdev); | |
| 354 | + bool coherent = mem->alloc_flags & ALLOC_MEM_FLAGS_COHERENT; | |
| 355 | + uint32_t mapping_flags; | |
| 356 | + | |
| 357 | + mapping_flags = AMDGPU_VM_PAGE_READABLE; | |
| 358 | + if (mem->alloc_flags & ALLOC_MEM_FLAGS_WRITABLE) | |
| 359 | + mapping_flags |= AMDGPU_VM_PAGE_WRITEABLE; | |
| 360 | + if (mem->alloc_flags & ALLOC_MEM_FLAGS_EXECUTABLE) | |
| 361 | + mapping_flags |= AMDGPU_VM_PAGE_EXECUTABLE; | |
| 362 | + | |
| 363 | + switch (adev->asic_type) { | |
| 364 | + case CHIP_ARCTURUS: | |
| 365 | + if (mem->alloc_flags & ALLOC_MEM_FLAGS_VRAM) { | |
| 366 | + if (bo_adev == adev) | |
| 367 | + mapping_flags |= coherent ? | |
| 368 | + AMDGPU_VM_MTYPE_CC : AMDGPU_VM_MTYPE_RW; | |
| 369 | + else | |
| 370 | + mapping_flags |= AMDGPU_VM_MTYPE_UC; | |
| 371 | + } else { | |
| 372 | + mapping_flags |= coherent ? | |
| 373 | + AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC; | |
| 374 | + } | |
| 375 | + break; | |
| 376 | + default: | |
| 377 | + mapping_flags |= coherent ? | |
| 378 | + AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC; | |
| 379 | + } | |
| 380 | + | |
| 381 | + return amdgpu_gem_va_map_flags(adev, mapping_flags); | |
| 382 | +} | |
| 383 | + | |
| 359 | 384 | /* add_bo_to_vm - Add a BO to a VM |
| 360 | 385 | * |
| 361 | 386 | * Everything that needs to bo done only once when a BO is first added |
| @@ -404,8 +429,7 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, · 42 unmodified lines | ||
| 404 | 429 | } |
| 405 | 430 | |
| 406 | 431 | bo_va_entry->va = va; |
| 407 | − bo_va_entry->pte_flags = amdgpu_gmc_get_pte_flags(adev, | |
| 408 | − mem->mapping_flags); | |
| 432 | + bo_va_entry->pte_flags = get_pte_flags(adev, mem); | |
| 409 | 433 | bo_va_entry->kgd_dev = (void *)adev; |
| 410 | 434 | list_add(&bo_va_entry->bo_list, list_bo_va); |
| 411 | 435 | |
| @@ -586,7 +610,7 @@ static int reserve_bo_and_vm(struct kgd_mem *mem, · 174 unmodified lines | ||
| 586 | 610 | amdgpu_vm_get_pd_bo(vm, &ctx->list, &ctx->vm_pd[0]); |
| 587 | 611 | |
| 588 | 612 | ret = ttm_eu_reserve_buffers(&ctx->ticket, &ctx->list, |
| 589 | − false, &ctx->duplicates, true); | |
| 613 | + false, &ctx->duplicates); | |
| 590 | 614 | if (!ret) |
| 591 | 615 | ctx->reserved = true; |
| 592 | 616 | else { |
| @@ -659,7 +683,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, · 66 unmodified lines | ||
| 659 | 683 | } |
| 660 | 684 | |
| 661 | 685 | ret = ttm_eu_reserve_buffers(&ctx->ticket, &ctx->list, |
| 662 | − false, &ctx->duplicates, true); | |
| 686 | + false, &ctx->duplicates); | |
| 663 | 687 | if (!ret) |
| 664 | 688 | ctx->reserved = true; |
| 665 | 689 | else |
| @@ -1079,10 +1103,8 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( · 413 unmodified lines | ||
| 1079 | 1103 | uint64_t user_addr = 0; |
| 1080 | 1104 | struct amdgpu_bo *bo; |
| 1081 | 1105 | struct amdgpu_bo_param bp; |
| 1082 | − int byte_align; | |
| 1083 | 1106 | u32 domain, alloc_domain; |
| 1084 | 1107 | u64 alloc_flags; |
| 1085 | − uint32_t mapping_flags; | |
| 1086 | 1108 | int ret; |
| 1087 | 1109 | |
| 1088 | 1110 | /* |
| @@ -1135,25 +1157,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( · 46 unmodified lines | ||
| 1135 | 1157 | if ((*mem)->aql_queue) |
| 1136 | 1158 | size = size >> 1; |
| 1137 | 1159 | |
| 1138 | − /* Workaround for TLB bug on older VI chips */ | |
| 1139 | − byte_align = (adev->family == AMDGPU_FAMILY_VI && | |
| 1140 | − adev->asic_type != CHIP_FIJI && | |
| 1141 | − adev->asic_type != CHIP_POLARIS10 && | |
| 1142 | − adev->asic_type != CHIP_POLARIS11 && | |
| 1143 | − adev->asic_type != CHIP_POLARIS12 && | |
| 1144 | − adev->asic_type != CHIP_VEGAM) ? | |
| 1145 | − VI_BO_SIZE_ALIGN : 1; | |
| 1146 | − | |
| 1147 | − mapping_flags = AMDGPU_VM_PAGE_READABLE; | |
| 1148 | − if (flags & ALLOC_MEM_FLAGS_WRITABLE) | |
| 1149 | − mapping_flags |= AMDGPU_VM_PAGE_WRITEABLE; | |
| 1150 | − if (flags & ALLOC_MEM_FLAGS_EXECUTABLE) | |
| 1151 | − mapping_flags |= AMDGPU_VM_PAGE_EXECUTABLE; | |
| 1152 | − if (flags & ALLOC_MEM_FLAGS_COHERENT) | |
| 1153 | − mapping_flags |= AMDGPU_VM_MTYPE_UC; | |
| 1154 | − else | |
| 1155 | − mapping_flags |= AMDGPU_VM_MTYPE_NC; | |
| 1156 | − (*mem)->mapping_flags = mapping_flags; | |
| 1160 | + (*mem)->alloc_flags = flags; | |
| 1157 | 1161 | |
| 1158 | 1162 | amdgpu_sync_create(&(*mem)->sync); |
| 1159 | 1163 | |
| @@ -1168,7 +1172,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( · 8 unmodified lines | ||
| 1168 | 1172 | |
| 1169 | 1173 | memset(&bp, 0, sizeof(bp)); |
| 1170 | 1174 | bp.size = size; |
| 1171 | − bp.byte_align = byte_align; | |
| 1175 | + bp.byte_align = 1; | |
| 1172 | 1176 | bp.domain = alloc_domain; |
| 1173 | 1177 | bp.flags = alloc_flags; |
| 1174 | 1178 | bp.type = bo_type; |
| @@ -1626,9 +1630,10 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct kgd_dev *kgd, · 451 unmodified lines | ||
| 1626 | 1630 | |
| 1627 | 1631 | INIT_LIST_HEAD(&(*mem)->bo_va_list); |
| 1628 | 1632 | mutex_init(&(*mem)->lock); |
| 1629 | − (*mem)->mapping_flags = | |
| 1630 | − AMDGPU_VM_PAGE_READABLE | AMDGPU_VM_PAGE_WRITEABLE | | |
| 1631 | − AMDGPU_VM_PAGE_EXECUTABLE | AMDGPU_VM_MTYPE_NC; | |
| 1633 | + (*mem)->alloc_flags = | |
| 1634 | + ((bo->preferred_domains & AMDGPU_GEM_DOMAIN_VRAM) ? | |
| 1635 | + ALLOC_MEM_FLAGS_VRAM : ALLOC_MEM_FLAGS_GTT) | | |
| 1636 | + ALLOC_MEM_FLAGS_WRITABLE | ALLOC_MEM_FLAGS_EXECUTABLE; | |
| 1632 | 1637 | |
| 1633 | 1638 | (*mem)->bo = amdgpu_bo_ref(bo); |
| 1634 | 1639 | (*mem)->va = va; |
| @@ -1797,8 +1802,7 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) · 162 unmodified lines | ||
| 1797 | 1802 | } |
| 1798 | 1803 | |
| 1799 | 1804 | /* Reserve all BOs and page tables for validation */ |
| 1800 | − ret = ttm_eu_reserve_buffers(&ticket, &resv_list, false, &duplicates, | |
| 1801 | − true); | |
| 1805 | + ret = ttm_eu_reserve_buffers(&ticket, &resv_list, false, &duplicates); | |
| 1802 | 1806 | WARN(!list_empty(&duplicates), "Duplicates should be empty"); |
| 1803 | 1807 | if (ret) |
| 1804 | 1808 | goto out_free; |
| @@ -1996,7 +2000,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) · 191 unmodified lines | ||
| 1996 | 2000 | } |
| 1997 | 2001 | |
| 1998 | 2002 | ret = ttm_eu_reserve_buffers(&ctx.ticket, &ctx.list, |
| 1999 | − false, &duplicate_save, true); | |
| 2003 | + false, &duplicate_save); | |
| 2000 | 2004 | if (ret) { |
| 2001 | 2005 | pr_debug("Memory eviction: TTM Reserve Failed. Try again\n"); |
| 2002 | 2006 | goto ttm_reserve_fail; |
| @@ -2038,6 +2038,11 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) · 2037 unmodified lines | ||
| 2038 | 2038 | if (adev->is_atom_fw) { |
| 2039 | 2039 | amdgpu_atomfirmware_scratch_regs_init(adev); |
| 2040 | 2040 | amdgpu_atomfirmware_allocate_fb_scratch(adev); |
| 2041 | + ret = amdgpu_atomfirmware_get_mem_train_fb_loc(adev); | |
| 2042 | + if (ret) { | |
| 2043 | + DRM_ERROR("Failed to get mem train fb location.\n"); | |
| 2044 | + return ret; | |
| 2045 | + } | |
| 2041 | 2046 | } else { |
| 2042 | 2047 | amdgpu_atombios_scratch_regs_init(adev); |
| 2043 | 2048 | amdgpu_atombios_allocate_fb_scratch(adev); |
| @@ -27,6 +27,7 @@ · 26 unmodified lines | ||
| 27 | 27 | #include "amdgpu_atomfirmware.h" |
| 28 | 28 | #include "atom.h" |
| 29 | 29 | #include "atombios.h" |
| 30 | +#include "soc15_hw_ip.h" | |
| 30 | 31 | |
| 31 | 32 | bool amdgpu_atomfirmware_gpu_supports_virtualization(struct amdgpu_device *adev) |
| 32 | 33 | { |
| @@ -120,65 +121,14 @@ union vram_info { · 87 unmodified lines | ||
| 120 | 121 | struct atom_vram_info_header_v2_3 v23; |
| 121 | 122 | struct atom_vram_info_header_v2_4 v24; |
| 122 | 123 | }; |
| 123 | −/* | |
| 124 | − * Return vram width from integrated system info table, if available, | |
| 125 | − * or 0 if not. | |
| 126 | − */ | |
| 127 | −int amdgpu_atomfirmware_get_vram_width(struct amdgpu_device *adev) | |
| 128 | −{ | |
| 129 | − struct amdgpu_mode_info *mode_info = &adev->mode_info; | |
| 130 | − int index; | |
| 131 | − u16 data_offset, size; | |
| 132 | − union igp_info *igp_info; | |
| 133 | − union vram_info *vram_info; | |
| 134 | − u32 mem_channel_number; | |
| 135 | − u32 mem_channel_width; | |
| 136 | − u8 frev, crev; | |
| 137 | − | |
| 138 | − if (adev->flags & AMD_IS_APU) | |
| 139 | − index = get_index_into_master_table(atom_master_list_of_data_tables_v2_1, | |
| 140 | − integratedsysteminfo); | |
| 141 | − else | |
| 142 | − index = get_index_into_master_table(atom_master_list_of_data_tables_v2_1, | |
| 143 | − vram_info); | |
| 144 | 124 | |
| 145 | − /* get any igp specific overrides */ | |
| 146 | − if (amdgpu_atom_parse_data_header(mode_info->atom_context, index, &size, | |
| 147 | − &frev, &crev, &data_offset)) { | |
| 148 | − if (adev->flags & AMD_IS_APU) { | |
| 149 | − igp_info = (union igp_info *) | |
| 150 | − (mode_info->atom_context->bios + data_offset); | |
| 151 | − switch (crev) { | |
| 152 | − case 11: | |
| 153 | − mem_channel_number = igp_info->v11.umachannelnumber; | |
| 154 | − /* channel width is 64 */ | |
| 155 | − return mem_channel_number * 64; | |
| 156 | − default: | |
| 157 | − return 0; | |
| 158 | − } | |
| 159 | − } else { | |
| 160 | − vram_info = (union vram_info *) | |
| 161 | − (mode_info->atom_context->bios + data_offset); | |
| 162 | − switch (crev) { | |
| 163 | − case 3: | |
| 164 | − mem_channel_number = vram_info->v23.vram_module[0].channel_num; | |
| 165 | − mem_channel_width = vram_info->v23.vram_module[0].channel_width; | |
| 166 | − return mem_channel_number * (1 << mem_channel_width); | |
| 167 | − case 4: | |
| 168 | − mem_channel_number = vram_info->v24.vram_module[0].channel_num; | |
| 169 | − mem_channel_width = vram_info->v24.vram_module[0].channel_width; | |
| 170 | − return mem_channel_number * (1 << mem_channel_width); | |
| 171 | − default: | |
| 172 | − return 0; | |
| 173 | − } | |
| 174 | − } | |
| 175 | − } | |
| 176 | − | |
| 177 | − return 0; | |
| 178 | −} | |
| 125 | +union vram_module { | |
| 126 | + struct atom_vram_module_v9 v9; | |
| 127 | + struct atom_vram_module_v10 v10; | |
| 128 | +}; | |
| 179 | 129 | |
| 180 | −static int convert_atom_mem_type_to_vram_type (struct amdgpu_device *adev, | |
| 181 | − int atom_mem_type) | |
| 130 | +static int convert_atom_mem_type_to_vram_type(struct amdgpu_device *adev, | |
| 131 | + int atom_mem_type) | |
| 182 | 132 | { |
| 183 | 133 | int vram_type; |
| 184 | 134 | |
| @@ -219,19 +169,25 @@ static int convert_atom_mem_type_to_vram_type (struct amdgpu_device *adev, · 34 unmodified lines | ||
| 219 | 169 | |
| 220 | 170 | return vram_type; |
| 221 | 171 | } |
| 222 | −/* | |
| 223 | − * Return vram type from either integrated system info table | |
| 224 | − * or umc info table, if available, or 0 (TYPE_UNKNOWN) if not | |
| 225 | − */ | |
| 226 | −int amdgpu_atomfirmware_get_vram_type(struct amdgpu_device *adev) | |
| 172 | + | |
| 173 | + | |
| 174 | +int | |
| 175 | +amdgpu_atomfirmware_get_vram_info(struct amdgpu_device *adev, | |
| 176 | + int *vram_width, int *vram_type, | |
| 177 | + int *vram_vendor) | |
| 227 | 178 | { |
| 228 | 179 | struct amdgpu_mode_info *mode_info = &adev->mode_info; |
| 229 | − int index; | |
| 180 | + int index, i = 0; | |
| 230 | 181 | u16 data_offset, size; |
| 231 | 182 | union igp_info *igp_info; |
| 232 | 183 | union vram_info *vram_info; |
| 184 | + union vram_module *vram_module; | |
| 233 | 185 | u8 frev, crev; |
| 234 | 186 | u8 mem_type; |
| 187 | + u8 mem_vendor; | |
| 188 | + u32 mem_channel_number; | |
| 189 | + u32 mem_channel_width; | |
| 190 | + u32 module_id; | |
| 235 | 191 | |
| 236 | 192 | if (adev->flags & AMD_IS_APU) |
| 237 | 193 | index = get_index_into_master_table(atom_master_list_of_data_tables_v2_1, |
| @@ -239,6 +195,7 @@ int amdgpu_atomfirmware_get_vram_type(struct amdgpu_device *adev) · 1 unmodified line | ||
| 239 | 195 | else |
| 240 | 196 | index = get_index_into_master_table(atom_master_list_of_data_tables_v2_1, |
| 241 | 197 | vram_info); |
| 198 | + | |
| 242 | 199 | if (amdgpu_atom_parse_data_header(mode_info->atom_context, |
| 243 | 200 | index, &size, |
| 244 | 201 | &frev, &crev, &data_offset)) { |
| @@ -247,25 +204,67 @@ int amdgpu_atomfirmware_get_vram_type(struct amdgpu_device *adev) · 2 unmodified lines | ||
| 247 | 204 | (mode_info->atom_context->bios + data_offset); |
| 248 | 205 | switch (crev) { |
| 249 | 206 | case 11: |
| 207 | + mem_channel_number = igp_info->v11.umachannelnumber; | |
| 208 | + /* channel width is 64 */ | |
| 209 | + if (vram_width) | |
| 210 | + *vram_width = mem_channel_number * 64; | |
| 250 | 211 | mem_type = igp_info->v11.memorytype; |
| 251 | − return convert_atom_mem_type_to_vram_type(adev, mem_type); | |
| 212 | + if (vram_type) | |
| 213 | + *vram_type = convert_atom_mem_type_to_vram_type(adev, mem_type); | |
| 214 | + break; | |
| 252 | 215 | default: |
| 253 | − return 0; | |
| 216 | + return -EINVAL; | |
| 254 | 217 | } |
| 255 | 218 | } else { |
| 256 | 219 | vram_info = (union vram_info *) |
| 257 | 220 | (mode_info->atom_context->bios + data_offset); |
| 221 | + module_id = (RREG32(adev->bios_scratch_reg_offset + 4) & 0x00ff0000) >> 16; | |
| 258 | 222 | switch (crev) { |
| 259 | 223 | case 3: |
| 260 | − mem_type = vram_info->v23.vram_module[0].memory_type; | |
| 261 | − return convert_atom_mem_type_to_vram_type(adev, mem_type); | |
| 224 | + if (module_id > vram_info->v23.vram_module_num) | |
| 225 | + module_id = 0; | |
| 226 | + vram_module = (union vram_module *)vram_info->v23.vram_module; | |
| 227 | + while (i < module_id) { | |
| 228 | + vram_module = (union vram_module *) | |
| 229 | + ((u8 *)vram_module + vram_module->v9.vram_module_size); | |
| 230 | + i++; | |
| 231 | + } | |
| 232 | + mem_type = vram_module->v9.memory_type; | |
| 233 | + if (vram_type) | |
| 234 | + *vram_type = convert_atom_mem_type_to_vram_type(adev, mem_type); | |
| 235 | + mem_channel_number = vram_module->v9.channel_num; | |
| 236 | + mem_channel_width = vram_module->v9.channel_width; | |
| 237 | + if (vram_width) | |
| 238 | + *vram_width = mem_channel_number * (1 << mem_channel_width); | |
| 239 | + mem_vendor = (vram_module->v9.vender_rev_id) & 0xF; | |
| 240 | + if (vram_vendor) | |
| 241 | + *vram_vendor = mem_vendor; | |
| 242 | + break; | |
| 262 | 243 | case 4: |
| 263 | − mem_type = vram_info->v24.vram_module[0].memory_type; | |
| 264 | − return convert_atom_mem_type_to_vram_type(adev, mem_type); | |
| 244 | + if (module_id > vram_info->v24.vram_module_num) | |
| 245 | + module_id = 0; | |
| 246 | + vram_module = (union vram_module *)vram_info->v24.vram_module; | |
| 247 | + while (i < module_id) { | |
| 248 | + vram_module = (union vram_module *) | |
| 249 | + ((u8 *)vram_module + vram_module->v10.vram_module_size); | |
| 250 | + i++; | |
| 251 | + } | |
| 252 | + mem_type = vram_module->v10.memory_type; | |
| 253 | + if (vram_type) | |
| 254 | + *vram_type = convert_atom_mem_type_to_vram_type(adev, mem_type); | |
| 255 | + mem_channel_number = vram_module->v10.channel_num; | |
| 256 | + mem_channel_width = vram_module->v10.channel_width; | |
| 257 | + if (vram_width) | |
| 258 | + *vram_width = mem_channel_number * (1 << mem_channel_width); | |
| 259 | + mem_vendor = (vram_module->v10.vender_rev_id) & 0xF; | |
| 260 | + if (vram_vendor) | |
| 261 | + *vram_vendor = mem_vendor; | |
| 262 | + break; | |
| 265 | 263 | default: |
| 266 | − return 0; | |
| 264 | + return -EINVAL; | |
| 267 | 265 | } |
| 268 | 266 | } |
| 267 | + | |
| 269 | 268 | } |
| 270 | 269 | |
| 271 | 270 | return 0; |
| @@ -464,3 +463,138 @@ int amdgpu_atomfirmware_get_gfx_info(struct amdgpu_device *adev) · 192 unmodified lines | ||
| 464 | 463 | } |
| 465 | 464 | return -EINVAL; |
| 466 | 465 | } |
| 466 | + | |
| 467 | +/* | |
| 468 | + * Check if VBIOS supports GDDR6 training data save/restore | |
| 469 | + */ | |
| 470 | +static bool gddr6_mem_train_vbios_support(struct amdgpu_device *adev) | |
| 471 | +{ | |
| 472 | + uint16_t data_offset; | |
| 473 | + int index; | |
| 474 | + | |
| 475 | + index = get_index_into_master_table(atom_master_list_of_data_tables_v2_1, | |
| 476 | + firmwareinfo); | |
| 477 | + if (amdgpu_atom_parse_data_header(adev->mode_info.atom_context, index, NULL, | |
| 478 | + NULL, NULL, &data_offset)) { | |
| 479 | + struct atom_firmware_info_v3_1 *firmware_info = | |
| 480 | + (struct atom_firmware_info_v3_1 *)(adev->mode_info.atom_context->bios + | |
| 481 | + data_offset); | |
| 482 | + | |
| 483 | + DRM_DEBUG("atom firmware capability:0x%08x.\n", | |
| 484 | + le32_to_cpu(firmware_info->firmware_capability)); | |
| 485 | + | |
| 486 | + if (le32_to_cpu(firmware_info->firmware_capability) & | |
| 487 | + ATOM_FIRMWARE_CAP_ENABLE_2STAGE_BIST_TRAINING) | |
| 488 | + return true; | |
| 489 | + } | |
| 490 | + | |
| 491 | + return false; | |
| 492 | +} | |
| 493 | + | |
| 494 | +static int gddr6_mem_train_support(struct amdgpu_device *adev) | |
| 495 | +{ | |
| 496 | + int ret; | |
| 497 | + uint32_t major, minor, revision, hw_v; | |
| 498 | + | |
| 499 | + if (gddr6_mem_train_vbios_support(adev)) { | |
| 500 | + amdgpu_discovery_get_ip_version(adev, MP0_HWID, &major, &minor, &revision); | |
| 501 | + hw_v = HW_REV(major, minor, revision); | |
| 502 | + /* | |
| 503 | + * treat 0 revision as a special case since register for MP0 and MMHUB is missing | |
| 504 | + * for some Navi10 A0, preventing driver from discovering the hwip information since | |
| 505 | + * none of the functions will be initialized, it should not cause any problems | |
| 506 | + */ | |
| 507 | + switch (hw_v) { | |
| 508 | + case HW_REV(11, 0, 0): | |
| 509 | + case HW_REV(11, 0, 5): | |
| 510 | + ret = 1; | |
| 511 | + break; | |
| 512 | + default: | |
| 513 | + DRM_ERROR("memory training vbios supports but psp hw(%08x)" | |
| 514 | + " doesn't support!\n", hw_v); | |
| 515 | + ret = -1; | |
| 516 | + break; | |
| 517 | + } | |
| 518 | + } else { | |
| 519 | + ret = 0; | |
| 520 | + hw_v = -1; | |
| 521 | + } | |
| 522 | + | |
| 523 | + | |
| 524 | + DRM_DEBUG("mp0 hw_v %08x, ret:%d.\n", hw_v, ret); | |
| 525 | + return ret; | |
| 526 | +} | |
| 527 | + | |
| 528 | +int amdgpu_atomfirmware_get_mem_train_fb_loc(struct amdgpu_device *adev) | |
| 529 | +{ | |
| 530 | + struct atom_context *ctx = adev->mode_info.atom_context; | |
| 531 | + unsigned char *bios = ctx->bios; | |
| 532 | + struct vram_reserve_block *reserved_block; | |
| 533 | + int index, block_number; | |
| 534 | + uint8_t frev, crev; | |
| 535 | + uint16_t data_offset, size; | |
| 536 | + uint32_t start_address_in_kb; | |
| 537 | + uint64_t offset; | |
| 538 | + int ret; | |
| 539 | + | |
| 540 | + adev->fw_vram_usage.mem_train_support = false; | |
| 541 | + | |
| 542 | + if (adev->asic_type != CHIP_NAVI10 && | |
| 543 | + adev->asic_type != CHIP_NAVI14) | |
| 544 | + return 0; | |
| 545 | + | |
| 546 | + if (amdgpu_sriov_vf(adev)) | |
| 547 | + return 0; | |
| 548 | + | |
| 549 | + ret = gddr6_mem_train_support(adev); | |
| 550 | + if (ret == -1) | |
| 551 | + return -EINVAL; | |
| 552 | + else if (ret == 0) | |
| 553 | + return 0; | |
| 554 | + | |
| 555 | + index = get_index_into_master_table(atom_master_list_of_data_tables_v2_1, | |
| 556 | + vram_usagebyfirmware); | |
| 557 | + ret = amdgpu_atom_parse_data_header(ctx, index, &size, &frev, &crev, | |
| 558 | + &data_offset); | |
| 559 | + if (ret == 0) { | |
| 560 | + DRM_ERROR("parse data header failed.\n"); | |
| 561 | + return -EINVAL; | |
| 562 | + } | |
| 563 | + | |
| 564 | + DRM_DEBUG("atom firmware common table header size:0x%04x, frev:0x%02x," | |
| 565 | + " crev:0x%02x, data_offset:0x%04x.\n", size, frev, crev, data_offset); | |
| 566 | + /* only support 2.1+ */ | |
| 567 | + if (((uint16_t)frev << 8 | crev) < 0x0201) { | |
| 568 | + DRM_ERROR("frev:0x%02x, crev:0x%02x < 2.1 !\n", frev, crev); | |
| 569 | + return -EINVAL; | |
| 570 | + } | |
| 571 | + | |
| 572 | + reserved_block = (struct vram_reserve_block *) | |
| 573 | + (bios + data_offset + sizeof(struct atom_common_table_header)); | |
| 574 | + block_number = ((unsigned int)size - sizeof(struct atom_common_table_header)) | |
| 575 | + / sizeof(struct vram_reserve_block); | |
| 576 | + reserved_block += (block_number > 0) ? block_number-1 : 0; | |
| 577 | + DRM_DEBUG("block_number:0x%04x, last block: 0x%08xkb sz, %dkb fw, %dkb drv.\n", | |
| 578 | + block_number, | |
| 579 | + le32_to_cpu(reserved_block->start_address_in_kb), | |
| 580 | + le16_to_cpu(reserved_block->used_by_firmware_in_kb), | |
| 581 | + le16_to_cpu(reserved_block->used_by_driver_in_kb)); | |
| 582 | + if (reserved_block->used_by_firmware_in_kb > 0) { | |
| 583 | + start_address_in_kb = le32_to_cpu(reserved_block->start_address_in_kb); | |
| 584 | + offset = (uint64_t)start_address_in_kb * ONE_KiB; | |
| 585 | + if ((offset & (ONE_MiB - 1)) < (4 * ONE_KiB + 1) ) { | |
| 586 | + offset -= ONE_MiB; | |
| 587 | + } | |
| 588 | + | |
| 589 | + offset &= ~(ONE_MiB - 1); | |
| 590 | + adev->fw_vram_usage.mem_train_fb_loc = offset; | |
| 591 | + adev->fw_vram_usage.mem_train_support = true; | |
| 592 | + DRM_DEBUG("mem_train_fb_loc:0x%09llx.\n", offset); | |
| 593 | + ret = 0; | |
| 594 | + } else { | |
| 595 | + DRM_ERROR("used_by_firmware_in_kb is 0!\n"); | |
| 596 | + ret = -EINVAL; | |
| 597 | + } | |
| 598 | + | |
| 599 | + return ret; | |
| 600 | +} | |
| @@ -29,8 +29,9 @@ · 28 unmodified lines | ||
| 29 | 29 | bool amdgpu_atomfirmware_gpu_supports_virtualization(struct amdgpu_device *adev); |
| 30 | 30 | void amdgpu_atomfirmware_scratch_regs_init(struct amdgpu_device *adev); |
| 31 | 31 | int amdgpu_atomfirmware_allocate_fb_scratch(struct amdgpu_device *adev); |
| 32 | −int amdgpu_atomfirmware_get_vram_width(struct amdgpu_device *adev); | |
| 33 | −int amdgpu_atomfirmware_get_vram_type(struct amdgpu_device *adev); | |
| 32 | +int amdgpu_atomfirmware_get_vram_info(struct amdgpu_device *adev, | |
| 33 | + int *vram_width, int *vram_type, int *vram_vendor); | |
| 34 | +int amdgpu_atomfirmware_get_mem_train_fb_loc(struct amdgpu_device *adev); | |
| 34 | 35 | int amdgpu_atomfirmware_get_clock_info(struct amdgpu_device *adev); |
| 35 | 36 | int amdgpu_atomfirmware_get_gfx_info(struct amdgpu_device *adev); |
| 36 | 37 | bool amdgpu_atomfirmware_mem_ecc_supported(struct amdgpu_device *adev); |
| @@ -613,17 +613,7 @@ static bool amdgpu_atpx_detect(void) · 612 unmodified lines | ||
| 613 | 613 | bool d3_supported = false; |
| 614 | 614 | struct pci_dev *parent_pdev; |
| 615 | 615 | |
| 616 | − while ((pdev = pci_get_class(PCI_CLASS_DISPLAY_VGA << 8, pdev)) != NULL) { | |
| 617 | − vga_count++; | |
| 618 | − | |
| 619 | − has_atpx |= (amdgpu_atpx_pci_probe_handle(pdev) == true); | |
| 620 | − | |
| 621 | − parent_pdev = pci_upstream_bridge(pdev); | |
| 622 | − d3_supported |= parent_pdev && parent_pdev->bridge_d3; | |
| 623 | − amdgpu_atpx_get_quirks(pdev); | |
| 624 | − } | |
| 625 | − | |
| 626 | − while ((pdev = pci_get_class(PCI_CLASS_DISPLAY_OTHER << 8, pdev)) != NULL) { | |
| 616 | + while ((pdev = pci_get_class(PCI_BASE_CLASS_DISPLAY << 16, pdev)) != NULL) { | |
| 627 | 617 | vga_count++; |
| 628 | 618 | |
| 629 | 619 | has_atpx |= (amdgpu_atpx_pci_probe_handle(pdev) == true); |
| @@ -33,7 +33,7 @@ static int amdgpu_benchmark_do_move(struct amdgpu_device *adev, unsigned size, · 32 unmodified lines | ||
| 33 | 33 | { |
| 34 | 34 | unsigned long start_jiffies; |
| 35 | 35 | unsigned long end_jiffies; |
| 36 | − struct dma_fence *fence = NULL; | |
| 36 | + struct dma_fence *fence; | |
| 37 | 37 | int i, r; |
| 38 | 38 | |
| 39 | 39 | start_jiffies = jiffies; |
| @@ -44,16 +44,14 @@ static int amdgpu_benchmark_do_move(struct amdgpu_device *adev, unsigned size, · 4 unmodified lines | ||
| 44 | 44 | if (r) |
| 45 | 45 | goto exit_do_move; |
| 46 | 46 | r = dma_fence_wait(fence, false); |
| 47 | + dma_fence_put(fence); | |
| 47 | 48 | if (r) |
| 48 | 49 | goto exit_do_move; |
| 49 | − dma_fence_put(fence); | |
| 50 | 50 | } |
| 51 | 51 | end_jiffies = jiffies; |
| 52 | 52 | r = jiffies_to_msecs(end_jiffies - start_jiffies); |
| 53 | 53 | |
| 54 | 54 | exit_do_move: |
| 55 | − if (fence) | |
| 56 | − dma_fence_put(fence); | |
| 57 | 55 | return r; |
| 58 | 56 | } |
| 59 | 57 | |
| @@ -140,7 +140,12 @@ int amdgpu_bo_list_create(struct amdgpu_device *adev, struct drm_file *filp, · 139 unmodified lines | ||
| 140 | 140 | return 0; |
| 141 | 141 | |
| 142 | 142 | error_free: |
| 143 | − while (i--) { | |
| 143 | + for (i = 0; i < last_entry; ++i) { | |
| 144 | + struct amdgpu_bo *bo = ttm_to_amdgpu_bo(array[i].tv.bo); | |
| 145 | + | |
| 146 | + amdgpu_bo_unref(&bo); | |
| 147 | + } | |
| 148 | + for (i = first_userptr; i < num_entries; ++i) { | |
| 144 | 149 | struct amdgpu_bo *bo = ttm_to_amdgpu_bo(array[i].tv.bo); |
| 145 | 150 | |
| 146 | 151 | amdgpu_bo_unref(&bo); |
| @@ -270,7 +275,7 @@ int amdgpu_bo_list_ioctl(struct drm_device *dev, void *data, · 123 unmodified lines | ||
| 270 | 275 | |
| 271 | 276 | r = amdgpu_bo_create_list_entry_array(&args->in, &info); |
| 272 | 277 | if (r) |
| 273 | − goto error_free; | |
| 278 | + return r; | |
| 274 | 279 | |
| 275 | 280 | switch (args->in.operation) { |
| 276 | 281 | case AMDGPU_BO_LIST_OP_CREATE: |
| @@ -283,8 +288,7 @@ int amdgpu_bo_list_ioctl(struct drm_device *dev, void *data, · 6 unmodified lines | ||
| 283 | 288 | r = idr_alloc(&fpriv->bo_list_handles, list, 1, 0, GFP_KERNEL); |
| 284 | 289 | mutex_unlock(&fpriv->bo_list_lock); |
| 285 | 290 | if (r < 0) { |
| 286 | − amdgpu_bo_list_put(list); | |
| 287 | − return r; | |
| 291 | + goto error_put_list; | |
| 288 | 292 | } |
| 289 | 293 | |
| 290 | 294 | handle = r; |
| @@ -306,9 +310,8 @@ int amdgpu_bo_list_ioctl(struct drm_device *dev, void *data, · 15 unmodified lines | ||
| 306 | 310 | mutex_unlock(&fpriv->bo_list_lock); |
| 307 | 311 | |
| 308 | 312 | if (IS_ERR(old)) { |
| 309 | − amdgpu_bo_list_put(list); | |
| 310 | 313 | r = PTR_ERR(old); |
| 311 | − goto error_free; | |
| 314 | + goto error_put_list; | |
| 312 | 315 | } |
| 313 | 316 | |
| 314 | 317 | amdgpu_bo_list_put(old); |
| @@ -325,8 +328,10 @@ int amdgpu_bo_list_ioctl(struct drm_device *dev, void *data, · 10 unmodified lines | ||
| 325 | 328 | |
| 326 | 329 | return 0; |
| 327 | 330 | |
| 331 | +error_put_list: | |
| 332 | + amdgpu_bo_list_put(list); | |
| 333 | + | |
| 328 | 334 | error_free: |
| 329 | − if (info) | |
| 330 | − kvfree(info); | |
| 335 | + kvfree(info); | |
| 331 | 336 | return r; |
| 332 | 337 | } |
| @@ -1019,8 +1019,12 @@ amdgpu_connector_dvi_detect(struct drm_connector *connector, bool force) · 1018 unmodified lines | ||
| 1019 | 1019 | */ |
| 1020 | 1020 | if (amdgpu_connector->shared_ddc && (ret == connector_status_connected)) { |
| 1021 | 1021 | struct drm_connector *list_connector; |
| 1022 | + struct drm_connector_list_iter iter; | |
| 1022 | 1023 | struct amdgpu_connector *list_amdgpu_connector; |
| 1023 | − list_for_each_entry(list_connector, &dev->mode_config.connector_list, head) { | |
| 1024 | + | |
| 1025 | + drm_connector_list_iter_begin(dev, &iter); | |
| 1026 | + drm_for_each_connector_iter(list_connector, | |
| 1027 | + &iter) { | |
| 1024 | 1028 | if (connector == list_connector) |
| 1025 | 1029 | continue; |
| 1026 | 1030 | list_amdgpu_connector = to_amdgpu_connector(list_connector); |
| @@ -1037,6 +1041,7 @@ amdgpu_connector_dvi_detect(struct drm_connector *connector, bool force) · 10 unmodified lines | ||
| 1037 | 1041 | } |
| 1038 | 1042 | } |
| 1039 | 1043 | } |
| 1044 | + drm_connector_list_iter_end(&iter); | |
| 1040 | 1045 | } |
| 1041 | 1046 | } |
| 1042 | 1047 | } |
| @@ -1494,6 +1499,7 @@ amdgpu_connector_add(struct amdgpu_device *adev, · 451 unmodified lines | ||
| 1494 | 1499 | { |
| 1495 | 1500 | struct drm_device *dev = adev->ddev; |
| 1496 | 1501 | struct drm_connector *connector; |
| 1502 | + struct drm_connector_list_iter iter; | |
| 1497 | 1503 | struct amdgpu_connector *amdgpu_connector; |
| 1498 | 1504 | struct amdgpu_connector_atom_dig *amdgpu_dig_connector; |
| 1499 | 1505 | struct drm_encoder *encoder; |
| @@ -1508,10 +1514,12 @@ amdgpu_connector_add(struct amdgpu_device *adev, · 8 unmodified lines | ||
| 1508 | 1514 | return; |
| 1509 | 1515 | |
| 1510 | 1516 | /* see if we already added it */ |
| 1511 | − list_for_each_entry(connector, &dev->mode_config.connector_list, head) { | |
| 1517 | + drm_connector_list_iter_begin(dev, &iter); | |
| 1518 | + drm_for_each_connector_iter(connector, &iter) { | |
| 1512 | 1519 | amdgpu_connector = to_amdgpu_connector(connector); |
| 1513 | 1520 | if (amdgpu_connector->connector_id == connector_id) { |
| 1514 | 1521 | amdgpu_connector->devices |= supported_device; |
| 1522 | + drm_connector_list_iter_end(&iter); | |
| 1515 | 1523 | return; |
| 1516 | 1524 | } |
| 1517 | 1525 | if (amdgpu_connector->ddc_bus && i2c_bus->valid) { |
| @@ -1526,6 +1534,7 @@ amdgpu_connector_add(struct amdgpu_device *adev, · 8 unmodified lines | ||
| 1526 | 1534 | } |
| 1527 | 1535 | } |
| 1528 | 1536 | } |
| 1537 | + drm_connector_list_iter_end(&iter); | |
| 1529 | 1538 | |
| 1530 | 1539 | /* check if it's a dp bridge */ |
| 1531 | 1540 | list_for_each_entry(encoder, &dev->mode_config.encoder_list, head) { |
| @@ -35,6 +35,7 @@ · 34 unmodified lines | ||
| 35 | 35 | #include "amdgpu_trace.h" |
| 36 | 36 | #include "amdgpu_gmc.h" |
| 37 | 37 | #include "amdgpu_gem.h" |
| 38 | +#include "amdgpu_ras.h" | |
| 38 | 39 | |
| 39 | 40 | static int amdgpu_cs_user_fence_chunk(struct amdgpu_cs_parser *p, |
| 40 | 41 | struct drm_amdgpu_cs_chunk_fence *data, |
| @@ -449,75 +450,12 @@ retry: · 408 unmodified lines | ||
| 449 | 450 | return r; |
| 450 | 451 | } |
| 451 | 452 | |
| 452 | −/* Last resort, try to evict something from the current working set */ | |
| 453 | −static bool amdgpu_cs_try_evict(struct amdgpu_cs_parser *p, | |
| 454 | − struct amdgpu_bo *validated) | |
| 455 | −{ | |
| 456 | − uint32_t domain = validated->allowed_domains; | |
| 457 | − struct ttm_operation_ctx ctx = { true, false }; | |
| 458 | − int r; | |
| 459 | − | |
| 460 | − if (!p->evictable) | |
| 461 | − return false; | |
| 462 | − | |
| 463 | − for (;&p->evictable->tv.head != &p->validated; | |
| 464 | − p->evictable = list_prev_entry(p->evictable, tv.head)) { | |
| 465 | − | |
| 466 | − struct amdgpu_bo_list_entry *candidate = p->evictable; | |
| 467 | − struct amdgpu_bo *bo = ttm_to_amdgpu_bo(candidate->tv.bo); | |
| 468 | − struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | |
| 469 | − bool update_bytes_moved_vis; | |
| 470 | − uint32_t other; | |
| 471 | − | |
| 472 | − /* If we reached our current BO we can forget it */ | |
| 473 | − if (bo == validated) | |
| 474 | − break; | |
| 475 | − | |
| 476 | − /* We can't move pinned BOs here */ | |
| 477 | − if (bo->pin_count) | |
| 478 | − continue; | |
| 479 | − | |
| 480 | − other = amdgpu_mem_type_to_domain(bo->tbo.mem.mem_type); | |
| 481 | − | |
| 482 | − /* Check if this BO is in one of the domains we need space for */ | |
| 483 | − if (!(other & domain)) | |
| 484 | − continue; | |
| 485 | − | |
| 486 | − /* Check if we can move this BO somewhere else */ | |
| 487 | − other = bo->allowed_domains & ~domain; | |
| 488 | − if (!other) | |
| 489 | − continue; | |
| 490 | − | |
| 491 | − /* Good we can try to move this BO somewhere else */ | |
| 492 | − update_bytes_moved_vis = | |
| 493 | − !amdgpu_gmc_vram_full_visible(&adev->gmc) && | |
| 494 | − amdgpu_bo_in_cpu_visible_vram(bo); | |
| 495 | − amdgpu_bo_placement_from_domain(bo, other); | |
| 496 | − r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); | |
| 497 | − p->bytes_moved += ctx.bytes_moved; | |
| 498 | − if (update_bytes_moved_vis) | |
| 499 | − p->bytes_moved_vis += ctx.bytes_moved; | |
| 500 | − | |
| 501 | − if (unlikely(r)) | |
| 502 | − break; | |
| 503 | − | |
| 504 | − p->evictable = list_prev_entry(p->evictable, tv.head); | |
| 505 | − list_move(&candidate->tv.head, &p->validated); | |
| 506 | − | |
| 507 | − return true; | |
| 508 | − } | |
| 509 | − | |
| 510 | − return false; | |
| 511 | −} | |
| 512 | − | |
| 513 | 453 | static int amdgpu_cs_validate(void *param, struct amdgpu_bo *bo) |
| 514 | 454 | { |
| 515 | 455 | struct amdgpu_cs_parser *p = param; |
| 516 | 456 | int r; |
| 517 | 457 | |
| 518 | − do { | |
| 519 | − r = amdgpu_cs_bo_validate(p, bo); | |
| 520 | − } while (r == -ENOMEM && amdgpu_cs_try_evict(p, bo)); | |
| 458 | + r = amdgpu_cs_bo_validate(p, bo); | |
| 521 | 459 | if (r) |
| 522 | 460 | return r; |
| 523 | 461 | |
| @@ -536,7 +474,6 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p, · 12 unmodified lines | ||
| 536 | 474 | |
| 537 | 475 | list_for_each_entry(lobj, validated, tv.head) { |
| 538 | 476 | struct amdgpu_bo *bo = ttm_to_amdgpu_bo(lobj->tv.bo); |
| 539 | − bool binding_userptr = false; | |
| 540 | 477 | struct mm_struct *usermm; |
| 541 | 478 | |
| 542 | 479 | usermm = amdgpu_ttm_tt_get_usermm(bo->tbo.ttm); |
| @@ -553,20 +490,14 @@ static int amdgpu_cs_list_validate(struct amdgpu_cs_parser *p, · 10 unmodified lines | ||
| 553 | 490 | |
| 554 | 491 | amdgpu_ttm_tt_set_user_pages(bo->tbo.ttm, |
| 555 | 492 | lobj->user_pages); |
| 556 | − binding_userptr = true; | |
| 557 | 493 | } |
| 558 | 494 | |
| 559 | − if (p->evictable == lobj) | |
| 560 | − p->evictable = NULL; | |
| 561 | − | |
| 562 | 495 | r = amdgpu_cs_validate(p, bo); |
| 563 | 496 | if (r) |
| 564 | 497 | return r; |
| 565 | 498 | |
| 566 | − if (binding_userptr) { | |
| 567 | − kvfree(lobj->user_pages); | |
| 568 | − lobj->user_pages = NULL; | |
| 569 | − } | |
| 499 | + kvfree(lobj->user_pages); | |
| 500 | + lobj->user_pages = NULL; | |
| 570 | 501 | } |
| 571 | 502 | return 0; |
| 572 | 503 | } |
| @@ -650,7 +581,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, · 77 unmodified lines | ||
| 650 | 581 | } |
| 651 | 582 | |
| 652 | 583 | r = ttm_eu_reserve_buffers(&p->ticket, &p->validated, true, |
| 653 | − &duplicates, false); | |
| 584 | + &duplicates); | |
| 654 | 585 | if (unlikely(r != 0)) { |
| 655 | 586 | if (r != -ERESTARTSYS) |
| 656 | 587 | DRM_ERROR("ttm_eu_reserve_buffers failed.\n"); |
| @@ -661,9 +592,6 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, · 4 unmodified lines | ||
| 661 | 592 | &p->bytes_moved_vis_threshold); |
| 662 | 593 | p->bytes_moved = 0; |
| 663 | 594 | p->bytes_moved_vis = 0; |
| 664 | − p->evictable = list_last_entry(&p->validated, | |
| 665 | − struct amdgpu_bo_list_entry, | |
| 666 | − tv.head); | |
| 667 | 595 | |
| 668 | 596 | r = amdgpu_vm_validate_pt_bos(p->adev, &fpriv->vm, |
| 669 | 597 | amdgpu_cs_validate, p); |
| @@ -915,7 +843,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p) · 245 unmodified lines | ||
| 915 | 843 | if (r) |
| 916 | 844 | return r; |
| 917 | 845 | |
| 918 | − r = amdgpu_vm_update_directories(adev, vm); | |
| 846 | + r = amdgpu_vm_update_pdes(adev, vm, false); | |
| 919 | 847 | if (r) |
| 920 | 848 | return r; |
| 921 | 849 | |
| @@ -1359,6 +1287,9 @@ int amdgpu_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 437 unmodified lines | ||
| 1359 | 1287 | bool reserved_buffers = false; |
| 1360 | 1288 | int i, r; |
| 1361 | 1289 | |
| 1290 | + if (amdgpu_ras_intr_triggered()) | |
| 1291 | + return -EHWPOISON; | |
| 1292 | + | |
| 1362 | 1293 | if (!adev->accel_working) |
| 1363 | 1294 | return -EBUSY; |
| 1364 | 1295 | |
| @@ -80,7 +80,7 @@ int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 79 unmodified lines | ||
| 80 | 80 | list_add(&csa_tv.head, &list); |
| 81 | 81 | amdgpu_vm_get_pd_bo(vm, &list, &pd); |
| 82 | 82 | |
| 83 | − r = ttm_eu_reserve_buffers(&ticket, &list, true, NULL, false); | |
| 83 | + r = ttm_eu_reserve_buffers(&ticket, &list, true, NULL); | |
| 84 | 84 | if (r) { |
| 85 | 85 | DRM_ERROR("failed to reserve CSA,PD BOs: err=%d\n", r); |
| 86 | 86 | return r; |
| @@ -604,8 +604,11 @@ void amdgpu_ctx_mgr_entity_fini(struct amdgpu_ctx_mgr *mgr) · 603 unmodified lines | ||
| 604 | 604 | continue; |
| 605 | 605 | } |
| 606 | 606 | |
| 607 | − for (i = 0; i < num_entities; i++) | |
| 607 | + for (i = 0; i < num_entities; i++) { | |
| 608 | + mutex_lock(&ctx->adev->lock_reset); | |
| 608 | 609 | drm_sched_entity_fini(&ctx->entities[0][i].entity); |
| 610 | + mutex_unlock(&ctx->adev->lock_reset); | |
| 611 | + } | |
| 609 | 612 | } |
| 610 | 613 | } |
| 611 | 614 | |
| @@ -859,6 +859,9 @@ static int amdgpu_debugfs_test_ib(struct seq_file *m, void *data) · 858 unmodified lines | ||
| 859 | 859 | struct amdgpu_device *adev = dev->dev_private; |
| 860 | 860 | int r = 0, i; |
| 861 | 861 | |
| 862 | + /* Avoid accidently unparking the sched thread during GPU reset */ | |
| 863 | + mutex_lock(&adev->lock_reset); | |
| 864 | + | |
| 862 | 865 | /* hold on the scheduler */ |
| 863 | 866 | for (i = 0; i < AMDGPU_MAX_RINGS; i++) { |
| 864 | 867 | struct amdgpu_ring *ring = adev->rings[i]; |
| @@ -884,6 +887,8 @@ static int amdgpu_debugfs_test_ib(struct seq_file *m, void *data) · 19 unmodified lines | ||
| 884 | 887 | kthread_unpark(ring->sched.thread); |
| 885 | 888 | } |
| 886 | 889 | |
| 890 | + mutex_unlock(&adev->lock_reset); | |
| 891 | + | |
| 887 | 892 | return 0; |
| 888 | 893 | } |
| 889 | 894 | |
| @@ -1036,6 +1041,9 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) · 146 unmodified lines | ||
| 1036 | 1041 | if (!fences) |
| 1037 | 1042 | return -ENOMEM; |
| 1038 | 1043 | |
| 1044 | + /* Avoid accidently unparking the sched thread during GPU reset */ | |
| 1045 | + mutex_lock(&adev->lock_reset); | |
| 1046 | + | |
| 1039 | 1047 | /* stop the scheduler */ |
| 1040 | 1048 | kthread_park(ring->sched.thread); |
| 1041 | 1049 | |
| @@ -1075,10 +1083,11 @@ failure: · 33 unmodified lines | ||
| 1075 | 1083 | /* restart the scheduler */ |
| 1076 | 1084 | kthread_unpark(ring->sched.thread); |
| 1077 | 1085 | |
| 1086 | + mutex_unlock(&adev->lock_reset); | |
| 1087 | + | |
| 1078 | 1088 | ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, resched); |
| 1079 | 1089 | |
| 1080 | − if (fences) | |
| 1081 | − kfree(fences); | |
| 1090 | + kfree(fences); | |
| 1082 | 1091 | |
| 1083 | 1092 | return 0; |
| 1084 | 1093 | } |
| @@ -1090,8 +1099,8 @@ int amdgpu_debugfs_init(struct amdgpu_device *adev) · 5 unmodified lines | ||
| 1090 | 1099 | { |
| 1091 | 1100 | adev->debugfs_preempt = |
| 1092 | 1101 | debugfs_create_file("amdgpu_preempt_ib", 0600, |
| 1093 | − adev->ddev->primary->debugfs_root, | |
| 1094 | − (void *)adev, &fops_ib_preempt); | |
| 1102 | + adev->ddev->primary->debugfs_root, adev, | |
| 1103 | + &fops_ib_preempt); | |
| 1095 | 1104 | if (!(adev->debugfs_preempt)) { |
| 1096 | 1105 | DRM_ERROR("unable to create amdgpu_preempt_ib debugsfs file\n"); |
| 1097 | 1106 | return -EIO; |
| @@ -1103,8 +1112,7 @@ int amdgpu_debugfs_init(struct amdgpu_device *adev) · 5 unmodified lines | ||
| 1103 | 1112 | |
| 1104 | 1113 | void amdgpu_debugfs_preempt_cleanup(struct amdgpu_device *adev) |
| 1105 | 1114 | { |
| 1106 | − if (adev->debugfs_preempt) | |
| 1107 | − debugfs_remove(adev->debugfs_preempt); | |
| 1115 | + debugfs_remove(adev->debugfs_preempt); | |
| 1108 | 1116 | } |
| 1109 | 1117 | |
| 1110 | 1118 | #else |
| @@ -65,6 +65,8 @@ · 64 unmodified lines | ||
| 65 | 65 | #include "amdgpu_ras.h" |
| 66 | 66 | #include "amdgpu_pmu.h" |
| 67 | 67 | |
| 68 | +#include <linux/suspend.h> | |
| 69 | + | |
| 68 | 70 | MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); |
| 69 | 71 | MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); |
| 70 | 72 | MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); |
| @@ -78,7 +80,7 @@ MODULE_FIRMWARE("amdgpu/navi12_gpu_info.bin"); · 7 unmodified lines | ||
| 78 | 80 | |
| 79 | 81 | #define AMDGPU_RESUME_MS 2000 |
| 80 | 82 | |
| 81 | −static const char *amdgpu_asic_name[] = { | |
| 83 | +const char *amdgpu_asic_name[] = { | |
| 82 | 84 | "TAHITI", |
| 83 | 85 | "PITCAIRN", |
| 84 | 86 | "VERDE", |
| @@ -151,6 +153,36 @@ bool amdgpu_device_is_px(struct drm_device *dev) · 66 unmodified lines | ||
| 151 | 153 | return false; |
| 152 | 154 | } |
| 153 | 155 | |
| 156 | +/** | |
| 157 | + * VRAM access helper functions. | |
| 158 | + * | |
| 159 | + * amdgpu_device_vram_access - read/write a buffer in vram | |
| 160 | + * | |
| 161 | + * @adev: amdgpu_device pointer | |
| 162 | + * @pos: offset of the buffer in vram | |
| 163 | + * @buf: virtual address of the buffer in system memory | |
| 164 | + * @size: read/write size, sizeof(@buf) must > @size | |
| 165 | + * @write: true - write to vram, otherwise - read from vram | |
| 166 | + */ | |
| 167 | +void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, | |
| 168 | + uint32_t *buf, size_t size, bool write) | |
| 169 | +{ | |
| 170 | + uint64_t last; | |
| 171 | + unsigned long flags; | |
| 172 | + | |
| 173 | + last = size - 4; | |
| 174 | + for (last += pos; pos <= last; pos += 4) { | |
| 175 | + spin_lock_irqsave(&adev->mmio_idx_lock, flags); | |
| 176 | + WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000); | |
| 177 | + WREG32_NO_KIQ(mmMM_INDEX_HI, pos >> 31); | |
| 178 | + if (write) | |
| 179 | + WREG32_NO_KIQ(mmMM_DATA, *buf++); | |
| 180 | + else | |
| 181 | + *buf++ = RREG32_NO_KIQ(mmMM_DATA); | |
| 182 | + spin_unlock_irqrestore(&adev->mmio_idx_lock, flags); | |
| 183 | + } | |
| 184 | +} | |
| 185 | + | |
| 154 | 186 | /* |
| 155 | 187 | * MMIO register access helper functions. |
| 156 | 188 | */ |
| @@ -1023,12 +1055,6 @@ static int amdgpu_device_check_arguments(struct amdgpu_device *adev) · 866 unmodified lines | ||
| 1023 | 1055 | |
| 1024 | 1056 | amdgpu_device_check_block_size(adev); |
| 1025 | 1057 | |
| 1026 | − ret = amdgpu_device_get_job_timeout_settings(adev); | |
| 1027 | − if (ret) { | |
| 1028 | − dev_err(adev->dev, "invalid lockup_timeout parameter syntax\n"); | |
| 1029 | − return ret; | |
| 1030 | − } | |
| 1031 | − | |
| 1032 | 1058 | adev->firmware.load_type = amdgpu_ucode_get_load_type(adev, amdgpu_fw_load_type); |
| 1033 | 1059 | |
| 1034 | 1060 | return ret; |
| @@ -1469,6 +1495,9 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev) · 434 unmodified lines | ||
| 1469 | 1495 | (const struct gpu_info_firmware_v1_0 *)(adev->firmware.gpu_info_fw->data + |
| 1470 | 1496 | le32_to_cpu(hdr->header.ucode_array_offset_bytes)); |
| 1471 | 1497 | |
| 1498 | + if (amdgpu_discovery && adev->asic_type >= CHIP_NAVI10) | |
| 1499 | + goto parse_soc_bounding_box; | |
| 1500 | + | |
| 1472 | 1501 | adev->gfx.config.max_shader_engines = le32_to_cpu(gpu_info_fw->gc_num_se); |
| 1473 | 1502 | adev->gfx.config.max_cu_per_sh = le32_to_cpu(gpu_info_fw->gc_num_cu_per_sh); |
| 1474 | 1503 | adev->gfx.config.max_sh_per_se = le32_to_cpu(gpu_info_fw->gc_num_sh_per_se); |
| @@ -1496,7 +1525,13 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev) · 21 unmodified lines | ||
| 1496 | 1525 | adev->gfx.config.num_packer_per_sc = |
| 1497 | 1526 | le32_to_cpu(gpu_info_fw->num_packer_per_sc); |
| 1498 | 1527 | } |
| 1528 | + | |
| 1529 | +parse_soc_bounding_box: | |
| 1499 | 1530 | #ifdef CONFIG_DRM_AMD_DC_DCN2_0 |
| 1531 | + /* | |
| 1532 | + * soc bounding box info is not integrated in disocovery table, | |
| 1533 | + * we always need to parse it from gpu info firmware. | |
| 1534 | + */ | |
| 1500 | 1535 | if (hdr->version_minor == 2) { |
| 1501 | 1536 | const struct gpu_info_firmware_v1_2 *gpu_info_fw = |
| 1502 | 1537 | (const struct gpu_info_firmware_v1_2 *)(adev->firmware.gpu_info_fw->data + |
| @@ -1613,6 +1648,9 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) · 110 unmodified lines | ||
| 1613 | 1648 | if (r) |
| 1614 | 1649 | return r; |
| 1615 | 1650 | |
| 1651 | + if (amdgpu_discovery && adev->asic_type >= CHIP_NAVI10) | |
| 1652 | + amdgpu_discovery_get_gfx_info(adev); | |
| 1653 | + | |
| 1616 | 1654 | amdgpu_amdkfd_device_probe(adev); |
| 1617 | 1655 | |
| 1618 | 1656 | if (amdgpu_sriov_vf(adev)) { |
| @@ -1622,7 +1660,7 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) · 3 unmodified lines | ||
| 1622 | 1660 | } |
| 1623 | 1661 | |
| 1624 | 1662 | adev->pm.pp_feature = amdgpu_pp_feature_mask; |
| 1625 | − if (amdgpu_sriov_vf(adev)) | |
| 1663 | + if (amdgpu_sriov_vf(adev) || sched_policy == KFD_SCHED_POLICY_NO_HWS) | |
| 1626 | 1664 | adev->pm.pp_feature &= ~PP_GFXOFF_MASK; |
| 1627 | 1665 | |
| 1628 | 1666 | for (i = 0; i < adev->num_ip_blocks; i++) { |
| @@ -1839,6 +1877,19 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev) · 210 unmodified lines | ||
| 1839 | 1877 | if (r) |
| 1840 | 1878 | goto init_failed; |
| 1841 | 1879 | |
| 1880 | + /* | |
| 1881 | + * retired pages will be loaded from eeprom and reserved here, | |
| 1882 | + * it should be called after amdgpu_device_ip_hw_init_phase2 since | |
| 1883 | + * for some ASICs the RAS EEPROM code relies on SMU fully functioning | |
| 1884 | + * for I2C communication which only true at this point. | |
| 1885 | + * recovery_init may fail, but it can free all resources allocated by | |
| 1886 | + * itself and its failure should not stop amdgpu init process. | |
| 1887 | + * | |
| 1888 | + * Note: theoretically, this should be called before all vram allocations | |
| 1889 | + * to protect retired page from abusing | |
| 1890 | + */ | |
| 1891 | + amdgpu_ras_recovery_init(adev); | |
| 1892 | + | |
| 1842 | 1893 | if (adev->gmc.xgmi.num_physical_nodes > 1) |
| 1843 | 1894 | amdgpu_xgmi_add_device(adev); |
| 1844 | 1895 | amdgpu_amdkfd_device_init(adev); |
| @@ -2006,6 +2057,7 @@ out: · 161 unmodified lines | ||
| 2006 | 2057 | */ |
| 2007 | 2058 | static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) |
| 2008 | 2059 | { |
| 2060 | + struct amdgpu_gpu_instance *gpu_instance; | |
| 2009 | 2061 | int i = 0, r; |
| 2010 | 2062 | |
| 2011 | 2063 | for (i = 0; i < adev->num_ip_blocks; i++) { |
| @@ -2031,8 +2083,39 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) · 19 unmodified lines | ||
| 2031 | 2083 | if (r) |
| 2032 | 2084 | DRM_ERROR("enable mgpu fan boost failed (%d).\n", r); |
| 2033 | 2085 | |
| 2034 | − /* set to low pstate by default */ | |
| 2035 | − amdgpu_xgmi_set_pstate(adev, 0); | |
| 2086 | + | |
| 2087 | + if (adev->gmc.xgmi.num_physical_nodes > 1) { | |
| 2088 | + mutex_lock(&mgpu_info.mutex); | |
| 2089 | + | |
| 2090 | + /* | |
| 2091 | + * Reset device p-state to low as this was booted with high. | |
| 2092 | + * | |
| 2093 | + * This should be performed only after all devices from the same | |
| 2094 | + * hive get initialized. | |
| 2095 | + * | |
| 2096 | + * However, it's unknown how many device in the hive in advance. | |
| 2097 | + * As this is counted one by one during devices initializations. | |
| 2098 | + * | |
| 2099 | + * So, we wait for all XGMI interlinked devices initialized. | |
| 2100 | + * This may bring some delays as those devices may come from | |
| 2101 | + * different hives. But that should be OK. | |
| 2102 | + */ | |
| 2103 | + if (mgpu_info.num_dgpu == adev->gmc.xgmi.num_physical_nodes) { | |
| 2104 | + for (i = 0; i < mgpu_info.num_gpu; i++) { | |
| 2105 | + gpu_instance = &(mgpu_info.gpu_ins[i]); | |
| 2106 | + if (gpu_instance->adev->flags & AMD_IS_APU) | |
| 2107 | + continue; | |
| 2108 | + | |
| 2109 | + r = amdgpu_xgmi_set_pstate(gpu_instance->adev, 0); | |
| 2110 | + if (r) { | |
| 2111 | + DRM_ERROR("pstate setting failed (%d).\n", r); | |
| 2112 | + break; | |
| 2113 | + } | |
| 2114 | + } | |
| 2115 | + } | |
| 2116 | + | |
| 2117 | + mutex_unlock(&mgpu_info.mutex); | |
| 2118 | + } | |
| 2036 | 2119 | |
| 2037 | 2120 | return 0; |
| 2038 | 2121 | } |
| @@ -2220,6 +2303,12 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev) · 181 unmodified lines | ||
| 2220 | 2303 | /* displays are handled in phase1 */ |
| 2221 | 2304 | if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_DCE) |
| 2222 | 2305 | continue; |
| 2306 | + /* PSP lost connection when err_event_athub occurs */ | |
| 2307 | + if (amdgpu_ras_intr_triggered() && | |
| 2308 | + adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_PSP) { | |
| 2309 | + adev->ip_blocks[i].status.hw = false; | |
| 2310 | + continue; | |
| 2311 | + } | |
| 2223 | 2312 | /* XXX handle errors */ |
| 2224 | 2313 | r = adev->ip_blocks[i].version->funcs->suspend(adev); |
| 2225 | 2314 | /* XXX handle errors */ |
| @@ -2231,17 +2320,17 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev) · 5 unmodified lines | ||
| 2231 | 2320 | /* handle putting the SMC in the appropriate state */ |
| 2232 | 2321 | if (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_SMC) { |
| 2233 | 2322 | if (is_support_sw_smu(adev)) { |
| 2234 | − /* todo */ | |
| 2323 | + r = smu_set_mp1_state(&adev->smu, adev->mp1_state); | |
| 2235 | 2324 | } else if (adev->powerplay.pp_funcs && |
| 2236 | 2325 | adev->powerplay.pp_funcs->set_mp1_state) { |
| 2237 | 2326 | r = adev->powerplay.pp_funcs->set_mp1_state( |
| 2238 | 2327 | adev->powerplay.pp_handle, |
| 2239 | 2328 | adev->mp1_state); |
| 2240 | − if (r) { | |
| 2241 | − DRM_ERROR("SMC failed to set mp1 state %d, %d\n", | |
| 2242 | − adev->mp1_state, r); | |
| 2243 | − return r; | |
| 2244 | − } | |
| 2329 | + } | |
| 2330 | + if (r) { | |
| 2331 | + DRM_ERROR("SMC failed to set mp1 state %d, %d\n", | |
| 2332 | + adev->mp1_state, r); | |
| 2333 | + return r; | |
| 2245 | 2334 | } |
| 2246 | 2335 | } |
| 2247 | 2336 | |
| @@ -2556,6 +2645,73 @@ static void amdgpu_device_xgmi_reset_func(struct work_struct *__work) · 308 unmodified lines | ||
| 2556 | 2645 | adev->asic_reset_res, adev->ddev->unique); |
| 2557 | 2646 | } |
| 2558 | 2647 | |
| 2648 | +static int amdgpu_device_get_job_timeout_settings(struct amdgpu_device *adev) | |
| 2649 | +{ | |
| 2650 | + char *input = amdgpu_lockup_timeout; | |
| 2651 | + char *timeout_setting = NULL; | |
| 2652 | + int index = 0; | |
| 2653 | + long timeout; | |
| 2654 | + int ret = 0; | |
| 2655 | + | |
| 2656 | + /* | |
| 2657 | + * By default timeout for non compute jobs is 10000. | |
| 2658 | + * And there is no timeout enforced on compute jobs. | |
| 2659 | + * In SR-IOV or passthrough mode, timeout for compute | |
| 2660 | + * jobs are 10000 by default. | |
| 2661 | + */ | |
| 2662 | + adev->gfx_timeout = msecs_to_jiffies(10000); | |
| 2663 | + adev->sdma_timeout = adev->video_timeout = adev->gfx_timeout; | |
| 2664 | + if (amdgpu_sriov_vf(adev) || amdgpu_passthrough(adev)) | |
| 2665 | + adev->compute_timeout = adev->gfx_timeout; | |
| 2666 | + else | |
| 2667 | + adev->compute_timeout = MAX_SCHEDULE_TIMEOUT; | |
| 2668 | + | |
| 2669 | + if (strnlen(input, AMDGPU_MAX_TIMEOUT_PARAM_LENGTH)) { | |
| 2670 | + while ((timeout_setting = strsep(&input, ",")) && | |
| 2671 | + strnlen(timeout_setting, AMDGPU_MAX_TIMEOUT_PARAM_LENGTH)) { | |
| 2672 | + ret = kstrtol(timeout_setting, 0, &timeout); | |
| 2673 | + if (ret) | |
| 2674 | + return ret; | |
| 2675 | + | |
| 2676 | + if (timeout == 0) { | |
| 2677 | + index++; | |
| 2678 | + continue; | |
| 2679 | + } else if (timeout < 0) { | |
| 2680 | + timeout = MAX_SCHEDULE_TIMEOUT; | |
| 2681 | + } else { | |
| 2682 | + timeout = msecs_to_jiffies(timeout); | |
| 2683 | + } | |
| 2684 | + | |
| 2685 | + switch (index++) { | |
| 2686 | + case 0: | |
| 2687 | + adev->gfx_timeout = timeout; | |
| 2688 | + break; | |
| 2689 | + case 1: | |
| 2690 | + adev->compute_timeout = timeout; | |
| 2691 | + break; | |
| 2692 | + case 2: | |
| 2693 | + adev->sdma_timeout = timeout; | |
| 2694 | + break; | |
| 2695 | + case 3: | |
| 2696 | + adev->video_timeout = timeout; | |
| 2697 | + break; | |
| 2698 | + default: | |
| 2699 | + break; | |
| 2700 | + } | |
| 2701 | + } | |
| 2702 | + /* | |
| 2703 | + * There is only one value specified and | |
| 2704 | + * it should apply to all non-compute jobs. | |
| 2705 | + */ | |
| 2706 | + if (index == 1) { | |
| 2707 | + adev->sdma_timeout = adev->video_timeout = adev->gfx_timeout; | |
| 2708 | + if (amdgpu_sriov_vf(adev) || amdgpu_passthrough(adev)) | |
| 2709 | + adev->compute_timeout = adev->gfx_timeout; | |
| 2710 | + } | |
| 2711 | + } | |
| 2712 | + | |
| 2713 | + return ret; | |
| 2714 | +} | |
| 2559 | 2715 | |
| 2560 | 2716 | /** |
| 2561 | 2717 | * amdgpu_device_init - initialize the driver |
| @@ -2583,7 +2739,12 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 21 unmodified lines | ||
| 2583 | 2739 | adev->ddev = ddev; |
| 2584 | 2740 | adev->pdev = pdev; |
| 2585 | 2741 | adev->flags = flags; |
| 2586 | − adev->asic_type = flags & AMD_ASIC_MASK; | |
| 2742 | + | |
| 2743 | + if (amdgpu_force_asic_type >= 0 && amdgpu_force_asic_type < CHIP_LAST) | |
| 2744 | + adev->asic_type = amdgpu_force_asic_type; | |
| 2745 | + else | |
| 2746 | + adev->asic_type = flags & AMD_ASIC_MASK; | |
| 2747 | + | |
| 2587 | 2748 | adev->usec_timeout = AMDGPU_MAX_USEC_TIMEOUT; |
| 2588 | 2749 | if (amdgpu_emu_mode == 1) |
| 2589 | 2750 | adev->usec_timeout *= 2; |
| @@ -2726,6 +2887,12 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 136 unmodified lines | ||
| 2726 | 2887 | if (r) |
| 2727 | 2888 | return r; |
| 2728 | 2889 | |
| 2890 | + r = amdgpu_device_get_job_timeout_settings(adev); | |
| 2891 | + if (r) { | |
| 2892 | + dev_err(adev->dev, "invalid lockup_timeout parameter syntax\n"); | |
| 2893 | + return r; | |
| 2894 | + } | |
| 2895 | + | |
| 2729 | 2896 | /* doorbell bar mapping and doorbell index init*/ |
| 2730 | 2897 | amdgpu_device_doorbell_init(adev); |
| 2731 | 2898 | |
| @@ -2885,6 +3052,13 @@ fence_driver_init: · 153 unmodified lines | ||
| 2885 | 3052 | DRM_INFO("amdgpu: acceleration disabled, skipping benchmarks\n"); |
| 2886 | 3053 | } |
| 2887 | 3054 | |
| 3055 | + /* | |
| 3056 | + * Register gpu instance before amdgpu_device_enable_mgpu_fan_boost. | |
| 3057 | + * Otherwise the mgpu fan boost feature will be skipped due to the | |
| 3058 | + * gpu instance is counted less. | |
| 3059 | + */ | |
| 3060 | + amdgpu_register_gpu_instance(adev); | |
| 3061 | + | |
| 2888 | 3062 | /* enable clockgating, etc. after ib tests, etc. since some blocks require |
| 2889 | 3063 | * explicit gating rather than handling it automatically. |
| 2890 | 3064 | */ |
| @@ -3007,6 +3181,7 @@ int amdgpu_device_suspend(struct drm_device *dev, bool suspend, bool fbcon) · 116 unmodified lines | ||
| 3007 | 3181 | struct amdgpu_device *adev; |
| 3008 | 3182 | struct drm_crtc *crtc; |
| 3009 | 3183 | struct drm_connector *connector; |
| 3184 | + struct drm_connector_list_iter iter; | |
| 3010 | 3185 | int r; |
| 3011 | 3186 | |
| 3012 | 3187 | if (dev == NULL || dev->dev_private == NULL) { |
| @@ -3029,9 +3204,11 @@ int amdgpu_device_suspend(struct drm_device *dev, bool suspend, bool fbcon) · 16 unmodified lines | ||
| 3029 | 3204 | if (!amdgpu_device_has_dc_support(adev)) { |
| 3030 | 3205 | /* turn off display hw */ |
| 3031 | 3206 | drm_modeset_lock_all(dev); |
| 3032 | − list_for_each_entry(connector, &dev->mode_config.connector_list, head) { | |
| 3033 | − drm_helper_connector_dpms(connector, DRM_MODE_DPMS_OFF); | |
| 3034 | − } | |
| 3207 | + drm_connector_list_iter_begin(dev, &iter); | |
| 3208 | + drm_for_each_connector_iter(connector, &iter) | |
| 3209 | + drm_helper_connector_dpms(connector, | |
| 3210 | + DRM_MODE_DPMS_OFF); | |
| 3211 | + drm_connector_list_iter_end(&iter); | |
| 3035 | 3212 | drm_modeset_unlock_all(dev); |
| 3036 | 3213 | /* unpin the front buffers and cursors */ |
| 3037 | 3214 | list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) { |
| @@ -3082,15 +3259,11 @@ int amdgpu_device_suspend(struct drm_device *dev, bool suspend, bool fbcon) · 44 unmodified lines | ||
| 3082 | 3259 | */ |
| 3083 | 3260 | amdgpu_bo_evict_vram(adev); |
| 3084 | 3261 | |
| 3085 | − pci_save_state(dev->pdev); | |
| 3086 | 3262 | if (suspend) { |
| 3263 | + pci_save_state(dev->pdev); | |
| 3087 | 3264 | /* Shut down the device */ |
| 3088 | 3265 | pci_disable_device(dev->pdev); |
| 3089 | 3266 | pci_set_power_state(dev->pdev, PCI_D3hot); |
| 3090 | − } else { | |
| 3091 | − r = amdgpu_asic_reset(adev); | |
| 3092 | − if (r) | |
| 3093 | − DRM_ERROR("amdgpu asic reset failed\n"); | |
| 3094 | 3267 | } |
| 3095 | 3268 | |
| 3096 | 3269 | return 0; |
| @@ -3110,6 +3283,7 @@ int amdgpu_device_suspend(struct drm_device *dev, bool suspend, bool fbcon) · 13 unmodified lines | ||
| 3110 | 3283 | int amdgpu_device_resume(struct drm_device *dev, bool resume, bool fbcon) |
| 3111 | 3284 | { |
| 3112 | 3285 | struct drm_connector *connector; |
| 3286 | + struct drm_connector_list_iter iter; | |
| 3113 | 3287 | struct amdgpu_device *adev = dev->dev_private; |
| 3114 | 3288 | struct drm_crtc *crtc; |
| 3115 | 3289 | int r = 0; |
| @@ -3180,9 +3354,13 @@ int amdgpu_device_resume(struct drm_device *dev, bool resume, bool fbcon) · 64 unmodified lines | ||
| 3180 | 3354 | |
| 3181 | 3355 | /* turn on display hw */ |
| 3182 | 3356 | drm_modeset_lock_all(dev); |
| 3183 | − list_for_each_entry(connector, &dev->mode_config.connector_list, head) { | |
| 3184 | − drm_helper_connector_dpms(connector, DRM_MODE_DPMS_ON); | |
| 3185 | − } | |
| 3357 | + | |
| 3358 | + drm_connector_list_iter_begin(dev, &iter); | |
| 3359 | + drm_for_each_connector_iter(connector, &iter) | |
| 3360 | + drm_helper_connector_dpms(connector, | |
| 3361 | + DRM_MODE_DPMS_ON); | |
| 3362 | + drm_connector_list_iter_end(&iter); | |
| 3363 | + | |
| 3186 | 3364 | drm_modeset_unlock_all(dev); |
| 3187 | 3365 | } |
| 3188 | 3366 | amdgpu_fbdev_set_suspend(adev, 0); |
| @@ -3628,11 +3806,6 @@ static int amdgpu_do_asic_reset(struct amdgpu_hive_info *hive, · 439 unmodified lines | ||
| 3628 | 3806 | break; |
| 3629 | 3807 | } |
| 3630 | 3808 | } |
| 3631 | − | |
| 3632 | − list_for_each_entry(tmp_adev, device_list_handle, | |
| 3633 | − gmc.xgmi.head) { | |
| 3634 | − amdgpu_ras_reserve_bad_pages(tmp_adev); | |
| 3635 | − } | |
| 3636 | 3809 | } |
| 3637 | 3810 | } |
| 3638 | 3811 | |
| @@ -3736,25 +3909,18 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev, bool trylock) · 97 unmodified lines | ||
| 3736 | 3909 | adev->mp1_state = PP_MP1_STATE_NONE; |
| 3737 | 3910 | break; |
| 3738 | 3911 | } |
| 3739 | − /* Block kfd: SRIOV would do it separately */ | |
| 3740 | − if (!amdgpu_sriov_vf(adev)) | |
| 3741 | − amdgpu_amdkfd_pre_reset(adev); | |
| 3742 | 3912 | |
| 3743 | 3913 | return true; |
| 3744 | 3914 | } |
| 3745 | 3915 | |
| 3746 | 3916 | static void amdgpu_device_unlock_adev(struct amdgpu_device *adev) |
| 3747 | 3917 | { |
| 3748 | − /*unlock kfd: SRIOV would do it separately */ | |
| 3749 | − if (!amdgpu_sriov_vf(adev)) | |
| 3750 | − amdgpu_amdkfd_post_reset(adev); | |
| 3751 | 3918 | amdgpu_vf_error_trans_all(adev); |
| 3752 | 3919 | adev->mp1_state = PP_MP1_STATE_NONE; |
| 3753 | 3920 | adev->in_gpu_reset = 0; |
| 3754 | 3921 | mutex_unlock(&adev->lock_reset); |
| 3755 | 3922 | } |
| 3756 | 3923 | |
| 3757 | − | |
| 3758 | 3924 | /** |
| 3759 | 3925 | * amdgpu_device_gpu_recover - reset the asic and recover scheduler |
| 3760 | 3926 | * |
| @@ -3774,11 +3940,24 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 13 unmodified lines | ||
| 3774 | 3940 | struct amdgpu_hive_info *hive = NULL; |
| 3775 | 3941 | struct amdgpu_device *tmp_adev = NULL; |
| 3776 | 3942 | int i, r = 0; |
| 3943 | + bool in_ras_intr = amdgpu_ras_intr_triggered(); | |
| 3944 | + | |
| 3945 | + /* | |
| 3946 | + * Flush RAM to disk so that after reboot | |
| 3947 | + * the user can read log and see why the system rebooted. | |
| 3948 | + */ | |
| 3949 | + if (in_ras_intr && amdgpu_ras_get_context(adev)->reboot) { | |
| 3950 | + | |
| 3951 | + DRM_WARN("Emergency reboot."); | |
| 3952 | + | |
| 3953 | + ksys_sync_helper(); | |
| 3954 | + emergency_restart(); | |
| 3955 | + } | |
| 3777 | 3956 | |
| 3778 | 3957 | need_full_reset = job_signaled = false; |
| 3779 | 3958 | INIT_LIST_HEAD(&device_list); |
| 3780 | 3959 | |
| 3781 | − dev_info(adev->dev, "GPU reset begin!\n"); | |
| 3960 | + dev_info(adev->dev, "GPU %s begin!\n", in_ras_intr ? "jobs stop":"reset"); | |
| 3782 | 3961 | |
| 3783 | 3962 | cancel_delayed_work_sync(&adev->delayed_init_work); |
| 3784 | 3963 | |
| @@ -3805,9 +3984,16 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 20 unmodified lines | ||
| 3805 | 3984 | return 0; |
| 3806 | 3985 | } |
| 3807 | 3986 | |
| 3987 | + /* Block kfd: SRIOV would do it separately */ | |
| 3988 | + if (!amdgpu_sriov_vf(adev)) | |
| 3989 | + amdgpu_amdkfd_pre_reset(adev); | |
| 3990 | + | |
| 3808 | 3991 | /* Build list of devices to reset */ |
| 3809 | 3992 | if (adev->gmc.xgmi.num_physical_nodes > 1) { |
| 3810 | 3993 | if (!hive) { |
| 3994 | + /*unlock kfd: SRIOV would do it separately */ | |
| 3995 | + if (!amdgpu_sriov_vf(adev)) | |
| 3996 | + amdgpu_amdkfd_post_reset(adev); | |
| 3811 | 3997 | amdgpu_device_unlock_adev(adev); |
| 3812 | 3998 | return -ENODEV; |
| 3813 | 3999 | } |
| @@ -3823,17 +4009,22 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 9 unmodified lines | ||
| 3823 | 4009 | device_list_handle = &device_list; |
| 3824 | 4010 | } |
| 3825 | 4011 | |
| 3826 | − /* | |
| 3827 | − * Mark these ASICs to be reseted as untracked first | |
| 3828 | − * And add them back after reset completed | |
| 3829 | − */ | |
| 3830 | − list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) | |
| 3831 | − amdgpu_unregister_gpu_instance(tmp_adev); | |
| 3832 | − | |
| 3833 | 4012 | /* block all schedulers and reset given job's ring */ |
| 3834 | 4013 | list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) { |
| 4014 | + if (tmp_adev != adev) { | |
| 4015 | + amdgpu_device_lock_adev(tmp_adev, false); | |
| 4016 | + if (!amdgpu_sriov_vf(tmp_adev)) | |
| 4017 | + amdgpu_amdkfd_pre_reset(tmp_adev); | |
| 4018 | + } | |
| 4019 | + | |
| 4020 | + /* | |
| 4021 | + * Mark these ASICs to be reseted as untracked first | |
| 4022 | + * And add them back after reset completed | |
| 4023 | + */ | |
| 4024 | + amdgpu_unregister_gpu_instance(tmp_adev); | |
| 4025 | + | |
| 3835 | 4026 | /* disable ras on ALL IPs */ |
| 3836 | − if (amdgpu_device_ip_need_full_reset(tmp_adev)) | |
| 4027 | + if (!in_ras_intr && amdgpu_device_ip_need_full_reset(tmp_adev)) | |
| 3837 | 4028 | amdgpu_ras_suspend(tmp_adev); |
| 3838 | 4029 | |
| 3839 | 4030 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| @@ -3843,10 +4034,16 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 3 unmodified lines | ||
| 3843 | 4034 | continue; |
| 3844 | 4035 | |
| 3845 | 4036 | drm_sched_stop(&ring->sched, job ? &job->base : NULL); |
| 4037 | + | |
| 4038 | + if (in_ras_intr) | |
| 4039 | + amdgpu_job_stop_all_jobs_on_sched(&ring->sched); | |
| 3846 | 4040 | } |
| 3847 | 4041 | } |
| 3848 | 4042 | |
| 3849 | 4043 | |
| 4044 | + if (in_ras_intr) | |
| 4045 | + goto skip_sched_resume; | |
| 4046 | + | |
| 3850 | 4047 | /* |
| 3851 | 4048 | * Must check guilty signal here since after this point all old |
| 3852 | 4049 | * HW fences are force signaled. |
| @@ -3857,9 +4054,6 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 4 unmodified lines | ||
| 3857 | 4054 | dma_fence_is_signaled(job->base.s_fence->parent)) |
| 3858 | 4055 | job_signaled = true; |
| 3859 | 4056 | |
| 3860 | − if (!amdgpu_device_ip_need_full_reset(adev)) | |
| 3861 | − device_list_handle = &device_list; | |
| 3862 | − | |
| 3863 | 4057 | if (job_signaled) { |
| 3864 | 4058 | dev_info(adev->dev, "Guilty job already signaled, skipping HW reset"); |
| 3865 | 4059 | goto skip_hw_reset; |
| @@ -3881,7 +4075,6 @@ retry: /* Rest of adevs pre asic reset from XGMI hive. */ · 15 unmodified lines | ||
| 3881 | 4075 | if (tmp_adev == adev) |
| 3882 | 4076 | continue; |
| 3883 | 4077 | |
| 3884 | − amdgpu_device_lock_adev(tmp_adev, false); | |
| 3885 | 4078 | r = amdgpu_device_pre_asic_reset(tmp_adev, |
| 3886 | 4079 | NULL, |
| 3887 | 4080 | &need_full_reset); |
| @@ -3909,6 +4102,7 @@ skip_hw_reset: · 21 unmodified lines | ||
| 3909 | 4102 | |
| 3910 | 4103 | /* Post ASIC reset for all devs .*/ |
| 3911 | 4104 | list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) { |
| 4105 | + | |
| 3912 | 4106 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { |
| 3913 | 4107 | struct amdgpu_ring *ring = tmp_adev->rings[i]; |
| 3914 | 4108 | |
| @@ -3930,12 +4124,18 @@ skip_hw_reset: · 15 unmodified lines | ||
| 3930 | 4124 | |
| 3931 | 4125 | if (r) { |
| 3932 | 4126 | /* bad news, how to tell it to userspace ? */ |
| 3933 | − dev_info(tmp_adev->dev, "GPU reset(%d) failed\n", atomic_read(&adev->gpu_reset_counter)); | |
| 4127 | + dev_info(tmp_adev->dev, "GPU reset(%d) failed\n", atomic_read(&tmp_adev->gpu_reset_counter)); | |
| 3934 | 4128 | amdgpu_vf_error_put(tmp_adev, AMDGIM_ERROR_VF_GPU_RESET_FAIL, 0, r); |
| 3935 | 4129 | } else { |
| 3936 | − dev_info(tmp_adev->dev, "GPU reset(%d) succeeded!\n", atomic_read(&adev->gpu_reset_counter)); | |
| 4130 | + dev_info(tmp_adev->dev, "GPU reset(%d) succeeded!\n", atomic_read(&tmp_adev->gpu_reset_counter)); | |
| 3937 | 4131 | } |
| 4132 | + } | |
| 3938 | 4133 | |
| 4134 | +skip_sched_resume: | |
| 4135 | + list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) { | |
| 4136 | + /*unlock kfd: SRIOV would do it separately */ | |
| 4137 | + if (!in_ras_intr && !amdgpu_sriov_vf(tmp_adev)) | |
| 4138 | + amdgpu_amdkfd_post_reset(tmp_adev); | |
| 3939 | 4139 | amdgpu_device_unlock_adev(tmp_adev); |
| 3940 | 4140 | } |
| 3941 | 4141 | |
| @@ -134,20 +134,10 @@ static int hw_id_map[MAX_HWIP] = { · 133 unmodified lines | ||
| 134 | 134 | |
| 135 | 135 | static int amdgpu_discovery_read_binary(struct amdgpu_device *adev, uint8_t *binary) |
| 136 | 136 | { |
| 137 | − uint32_t *p = (uint32_t *)binary; | |
| 138 | 137 | uint64_t vram_size = (uint64_t)RREG32(mmRCC_CONFIG_MEMSIZE) << 20; |
| 139 | − uint64_t pos = vram_size - BINARY_MAX_SIZE; | |
| 140 | − unsigned long flags; | |
| 141 | − | |
| 142 | − while (pos < vram_size) { | |
| 143 | − spin_lock_irqsave(&adev->mmio_idx_lock, flags); | |
| 144 | − WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)pos) | 0x80000000); | |
| 145 | − WREG32_NO_KIQ(mmMM_INDEX_HI, pos >> 31); | |
| 146 | − *p++ = RREG32_NO_KIQ(mmMM_DATA); | |
| 147 | − spin_unlock_irqrestore(&adev->mmio_idx_lock, flags); | |
| 148 | − pos += 4; | |
| 149 | − } | |
| 138 | + uint64_t pos = vram_size - DISCOVERY_TMR_SIZE; | |
| 150 | 139 | |
| 140 | + amdgpu_device_vram_access(adev, pos, (uint32_t *)binary, DISCOVERY_TMR_SIZE, false); | |
| 151 | 141 | return 0; |
| 152 | 142 | } |
| 153 | 143 | |
| @@ -179,7 +169,7 @@ int amdgpu_discovery_init(struct amdgpu_device *adev) · 25 unmodified lines | ||
| 179 | 169 | uint16_t checksum; |
| 180 | 170 | int r; |
| 181 | 171 | |
| 182 | − adev->discovery = kzalloc(BINARY_MAX_SIZE, GFP_KERNEL); | |
| 172 | + adev->discovery = kzalloc(DISCOVERY_TMR_SIZE, GFP_KERNEL); | |
| 183 | 173 | if (!adev->discovery) |
| 184 | 174 | return -ENOMEM; |
| 185 | 175 | |
| @@ -333,7 +323,7 @@ int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev) · 147 unmodified lines | ||
| 333 | 323 | } |
| 334 | 324 | |
| 335 | 325 | int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, |
| 336 | − int *major, int *minor) | |
| 326 | + int *major, int *minor, int *revision) | |
| 337 | 327 | { |
| 338 | 328 | struct binary_header *bhdr; |
| 339 | 329 | struct ip_discovery_header *ihdr; |
| @@ -369,6 +359,8 @@ int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, · 29 unmodified lines | ||
| 369 | 359 | *major = ip->major; |
| 370 | 360 | if (minor) |
| 371 | 361 | *minor = ip->minor; |
| 362 | + if (revision) | |
| 363 | + *revision = ip->revision; | |
| 372 | 364 | return 0; |
| 373 | 365 | } |
| 374 | 366 | ip_offset += sizeof(*ip) + 4 * (ip->num_base_address - 1); |
| @@ -24,11 +24,13 @@ · 23 unmodified lines | ||
| 24 | 24 | #ifndef __AMDGPU_DISCOVERY__ |
| 25 | 25 | #define __AMDGPU_DISCOVERY__ |
| 26 | 26 | |
| 27 | +#define DISCOVERY_TMR_SIZE (64 << 10) | |
| 28 | + | |
| 27 | 29 | int amdgpu_discovery_init(struct amdgpu_device *adev); |
| 28 | 30 | void amdgpu_discovery_fini(struct amdgpu_device *adev); |
| 29 | 31 | int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev); |
| 30 | 32 | int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, |
| 31 | − int *major, int *minor); | |
| 33 | + int *major, int *minor, int *revision); | |
| 32 | 34 | int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev); |
| 33 | 35 | |
| 34 | 36 | #endif /* __AMDGPU_DISCOVERY__ */ |
| @@ -370,11 +370,13 @@ void amdgpu_display_print_display_setup(struct drm_device *dev) · 369 unmodified lines | ||
| 370 | 370 | struct amdgpu_connector *amdgpu_connector; |
| 371 | 371 | struct drm_encoder *encoder; |
| 372 | 372 | struct amdgpu_encoder *amdgpu_encoder; |
| 373 | + struct drm_connector_list_iter iter; | |
| 373 | 374 | uint32_t devices; |
| 374 | 375 | int i = 0; |
| 375 | 376 | |
| 377 | + drm_connector_list_iter_begin(dev, &iter); | |
| 376 | 378 | DRM_INFO("AMDGPU Display Connectors\n"); |
| 377 | − list_for_each_entry(connector, &dev->mode_config.connector_list, head) { | |
| 379 | + drm_for_each_connector_iter(connector, &iter) { | |
| 378 | 380 | amdgpu_connector = to_amdgpu_connector(connector); |
| 379 | 381 | DRM_INFO("Connector %d:\n", i); |
| 380 | 382 | DRM_INFO(" %s\n", connector->name); |
| @@ -438,6 +440,7 @@ void amdgpu_display_print_display_setup(struct drm_device *dev) · 57 unmodified lines | ||
| 438 | 440 | } |
| 439 | 441 | i++; |
| 440 | 442 | } |
| 443 | + drm_connector_list_iter_end(&iter); | |
| 441 | 444 | } |
| 442 | 445 | |
| 443 | 446 | /** |
| @@ -34,26 +34,11 @@ · 33 unmodified lines | ||
| 34 | 34 | #include "amdgpu.h" |
| 35 | 35 | #include "amdgpu_display.h" |
| 36 | 36 | #include "amdgpu_gem.h" |
| 37 | +#include "amdgpu_dma_buf.h" | |
| 37 | 38 | #include <drm/amdgpu_drm.h> |
| 38 | 39 | #include <linux/dma-buf.h> |
| 39 | 40 | #include <linux/dma-fence-array.h> |
| 40 | 41 | |
| 41 | −/** | |
| 42 | − * amdgpu_gem_prime_get_sg_table - &drm_driver.gem_prime_get_sg_table | |
| 43 | − * implementation | |
| 44 | − * @obj: GEM buffer object (BO) | |
| 45 | − * | |
| 46 | − * Returns: | |
| 47 | − * A scatter/gather table for the pinned pages of the BO's memory. | |
| 48 | − */ | |
| 49 | −struct sg_table *amdgpu_gem_prime_get_sg_table(struct drm_gem_object *obj) | |
| 50 | −{ | |
| 51 | − struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj); | |
| 52 | − int npages = bo->tbo.num_pages; | |
| 53 | − | |
| 54 | − return drm_prime_pages_to_sg(bo->tbo.ttm->pages, npages); | |
| 55 | −} | |
| 56 | − | |
| 57 | 42 | /** |
| 58 | 43 | * amdgpu_gem_prime_vmap - &dma_buf_ops.vmap implementation |
| 59 | 44 | * @obj: GEM BO |
| @@ -179,92 +164,126 @@ err_fences_put: · 119 unmodified lines | ||
| 179 | 164 | } |
| 180 | 165 | |
| 181 | 166 | /** |
| 182 | − * amdgpu_dma_buf_map_attach - &dma_buf_ops.attach implementation | |
| 183 | − * @dma_buf: Shared DMA buffer | |
| 167 | + * amdgpu_dma_buf_attach - &dma_buf_ops.attach implementation | |
| 168 | + * | |
| 169 | + * @dmabuf: DMA-buf where we attach to | |
| 170 | + * @attach: attachment to add | |
| 171 | + * | |
| 172 | + * Add the attachment as user to the exported DMA-buf. | |
| 173 | + */ | |
| 174 | +static int amdgpu_dma_buf_attach(struct dma_buf *dmabuf, | |
| 175 | + struct dma_buf_attachment *attach) | |
| 176 | +{ | |
| 177 | + struct drm_gem_object *obj = dmabuf->priv; | |
| 178 | + struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj); | |
| 179 | + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | |
| 180 | + int r; | |
| 181 | + | |
| 182 | + if (attach->dev->driver == adev->dev->driver) | |
| 183 | + return 0; | |
| 184 | + | |
| 185 | + r = amdgpu_bo_reserve(bo, false); | |
| 186 | + if (unlikely(r != 0)) | |
| 187 | + return r; | |
| 188 | + | |
| 189 | + /* | |
| 190 | + * We only create shared fences for internal use, but importers | |
| 191 | + * of the dmabuf rely on exclusive fences for implicitly | |
| 192 | + * tracking write hazards. As any of the current fences may | |
| 193 | + * correspond to a write, we need to convert all existing | |
| 194 | + * fences on the reservation object into a single exclusive | |
| 195 | + * fence. | |
| 196 | + */ | |
| 197 | + r = __dma_resv_make_exclusive(bo->tbo.base.resv); | |
| 198 | + if (r) | |
| 199 | + return r; | |
| 200 | + | |
| 201 | + bo->prime_shared_count++; | |
| 202 | + amdgpu_bo_unreserve(bo); | |
| 203 | + return 0; | |
| 204 | +} | |
| 205 | + | |
| 206 | +/** | |
| 207 | + * amdgpu_dma_buf_detach - &dma_buf_ops.detach implementation | |
| 208 | + * | |
| 209 | + * @dmabuf: DMA-buf where we remove the attachment from | |
| 210 | + * @attach: the attachment to remove | |
| 211 | + * | |
| 212 | + * Called when an attachment is removed from the DMA-buf. | |
| 213 | + */ | |
| 214 | +static void amdgpu_dma_buf_detach(struct dma_buf *dmabuf, | |
| 215 | + struct dma_buf_attachment *attach) | |
| 216 | +{ | |
| 217 | + struct drm_gem_object *obj = dmabuf->priv; | |
| 218 | + struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj); | |
| 219 | + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | |
| 220 | + | |
| 221 | + if (attach->dev->driver != adev->dev->driver && bo->prime_shared_count) | |
| 222 | + bo->prime_shared_count--; | |
| 223 | +} | |
| 224 | + | |
| 225 | +/** | |
| 226 | + * amdgpu_dma_buf_map - &dma_buf_ops.map_dma_buf implementation | |
| 184 | 227 | * @attach: DMA-buf attachment |
| 228 | + * @dir: DMA direction | |
| 185 | 229 | * |
| 186 | 230 | * Makes sure that the shared DMA buffer can be accessed by the target device. |
| 187 | 231 | * For now, simply pins it to the GTT domain, where it should be accessible by |
| 188 | 232 | * all DMA devices. |
| 189 | 233 | * |
| 190 | 234 | * Returns: |
| 191 | − * 0 on success or a negative error code on failure. | |
| 235 | + * sg_table filled with the DMA addresses to use or ERR_PRT with negative error | |
| 236 | + * code. | |
| 192 | 237 | */ |
| 193 | −static int amdgpu_dma_buf_map_attach(struct dma_buf *dma_buf, | |
| 194 | − struct dma_buf_attachment *attach) | |
| 238 | +static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach, | |
| 239 | + enum dma_data_direction dir) | |
| 195 | 240 | { |
| 241 | + struct dma_buf *dma_buf = attach->dmabuf; | |
| 196 | 242 | struct drm_gem_object *obj = dma_buf->priv; |
| 197 | 243 | struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj); |
| 198 | − struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | |
| 244 | + struct sg_table *sgt; | |
| 199 | 245 | long r; |
| 200 | 246 | |
| 201 | − r = drm_gem_map_attach(dma_buf, attach); | |
| 202 | − if (r) | |
| 203 | − return r; | |
| 204 | − | |
| 205 | − r = amdgpu_bo_reserve(bo, false); | |
| 206 | − if (unlikely(r != 0)) | |
| 207 | − goto error_detach; | |
| 208 | − | |
| 209 | − | |
| 210 | − if (attach->dev->driver != adev->dev->driver) { | |
| 211 | − /* | |
| 212 | − * We only create shared fences for internal use, but importers | |
| 213 | − * of the dmabuf rely on exclusive fences for implicitly | |
| 214 | − * tracking write hazards. As any of the current fences may | |
| 215 | − * correspond to a write, we need to convert all existing | |
| 216 | − * fences on the reservation object into a single exclusive | |
| 217 | − * fence. | |
| 218 | − */ | |
| 219 | − r = __dma_resv_make_exclusive(bo->tbo.base.resv); | |
| 220 | − if (r) | |
| 221 | − goto error_unreserve; | |
| 222 | − } | |
| 223 | − | |
| 224 | − /* pin buffer into GTT */ | |
| 225 | 247 | r = amdgpu_bo_pin(bo, AMDGPU_GEM_DOMAIN_GTT); |
| 226 | 248 | if (r) |
| 227 | − goto error_unreserve; | |
| 249 | + return ERR_PTR(r); | |
| 228 | 250 | |
| 229 | − if (attach->dev->driver != adev->dev->driver) | |
| 230 | − bo->prime_shared_count++; | |
| 251 | + sgt = drm_prime_pages_to_sg(bo->tbo.ttm->pages, bo->tbo.num_pages); | |
| 252 | + if (IS_ERR(sgt)) | |
| 253 | + return sgt; | |
| 231 | 254 | |
| 232 | −error_unreserve: | |
| 233 | − amdgpu_bo_unreserve(bo); | |
| 255 | + if (!dma_map_sg_attrs(attach->dev, sgt->sgl, sgt->nents, dir, | |
| 256 | + DMA_ATTR_SKIP_CPU_SYNC)) | |
| 257 | + goto error_free; | |
| 234 | 258 | |
| 235 | −error_detach: | |
| 236 | − if (r) | |
| 237 | − drm_gem_map_detach(dma_buf, attach); | |
| 238 | − return r; | |
| 259 | + return sgt; | |
| 260 | + | |
| 261 | +error_free: | |
| 262 | + sg_free_table(sgt); | |
| 263 | + kfree(sgt); | |
| 264 | + return ERR_PTR(-ENOMEM); | |
| 239 | 265 | } |
| 240 | 266 | |
| 241 | 267 | /** |
| 242 | − * amdgpu_dma_buf_map_detach - &dma_buf_ops.detach implementation | |
| 243 | − * @dma_buf: Shared DMA buffer | |
| 268 | + * amdgpu_dma_buf_unmap - &dma_buf_ops.unmap_dma_buf implementation | |
| 244 | 269 | * @attach: DMA-buf attachment |
| 270 | + * @sgt: sg_table to unmap | |
| 271 | + * @dir: DMA direction | |
| 245 | 272 | * |
| 246 | 273 | * This is called when a shared DMA buffer no longer needs to be accessible by |
| 247 | 274 | * another device. For now, simply unpins the buffer from GTT. |
| 248 | 275 | */ |
| 249 | −static void amdgpu_dma_buf_map_detach(struct dma_buf *dma_buf, | |
| 250 | − struct dma_buf_attachment *attach) | |
| 276 | +static void amdgpu_dma_buf_unmap(struct dma_buf_attachment *attach, | |
| 277 | + struct sg_table *sgt, | |
| 278 | + enum dma_data_direction dir) | |
| 251 | 279 | { |
| 252 | − struct drm_gem_object *obj = dma_buf->priv; | |
| 280 | + struct drm_gem_object *obj = attach->dmabuf->priv; | |
| 253 | 281 | struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj); |
| 254 | − struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | |
| 255 | − int ret = 0; | |
| 256 | − | |
| 257 | − ret = amdgpu_bo_reserve(bo, true); | |
| 258 | − if (unlikely(ret != 0)) | |
| 259 | − goto error; | |
| 260 | 282 | |
| 283 | + dma_unmap_sg(attach->dev, sgt->sgl, sgt->nents, dir); | |
| 284 | + sg_free_table(sgt); | |
| 285 | + kfree(sgt); | |
| 261 | 286 | amdgpu_bo_unpin(bo); |
| 262 | − if (attach->dev->driver != adev->dev->driver && bo->prime_shared_count) | |
| 263 | − bo->prime_shared_count--; | |
| 264 | − amdgpu_bo_unreserve(bo); | |
| 265 | − | |
| 266 | −error: | |
| 267 | − drm_gem_map_detach(dma_buf, attach); | |
| 268 | 287 | } |
| 269 | 288 | |
| 270 | 289 | /** |
| @@ -308,10 +327,11 @@ static int amdgpu_dma_buf_begin_cpu_access(struct dma_buf *dma_buf, · 37 unmodified lines | ||
| 308 | 327 | } |
| 309 | 328 | |
| 310 | 329 | const struct dma_buf_ops amdgpu_dmabuf_ops = { |
| 311 | − .attach = amdgpu_dma_buf_map_attach, | |
| 312 | − .detach = amdgpu_dma_buf_map_detach, | |
| 313 | − .map_dma_buf = drm_gem_map_dma_buf, | |
| 314 | − .unmap_dma_buf = drm_gem_unmap_dma_buf, | |
| 330 | + .dynamic_mapping = true, | |
| 331 | + .attach = amdgpu_dma_buf_attach, | |
| 332 | + .detach = amdgpu_dma_buf_detach, | |
| 333 | + .map_dma_buf = amdgpu_dma_buf_map, | |
| 334 | + .unmap_dma_buf = amdgpu_dma_buf_unmap, | |
| 315 | 335 | .release = drm_gem_dmabuf_release, |
| 316 | 336 | .begin_cpu_access = amdgpu_dma_buf_begin_cpu_access, |
| 317 | 337 | .mmap = drm_gem_dmabuf_mmap, |
| @@ -321,7 +341,6 @@ const struct dma_buf_ops amdgpu_dmabuf_ops = { · 3 unmodified lines | ||
| 321 | 341 | |
| 322 | 342 | /** |
| 323 | 343 | * amdgpu_gem_prime_export - &drm_driver.gem_prime_export implementation |
| 324 | − * @dev: DRM device | |
| 325 | 344 | * @gobj: GEM BO |
| 326 | 345 | * @flags: Flags such as DRM_CLOEXEC and DRM_RDWR. |
| 327 | 346 | * |
| @@ -350,31 +369,28 @@ struct dma_buf *amdgpu_gem_prime_export(struct drm_gem_object *gobj, · 22 unmodified lines | ||
| 350 | 369 | } |
| 351 | 370 | |
| 352 | 371 | /** |
| 353 | − * amdgpu_gem_prime_import_sg_table - &drm_driver.gem_prime_import_sg_table | |
| 354 | − * implementation | |
| 372 | + * amdgpu_dma_buf_create_obj - create BO for DMA-buf import | |
| 373 | + * | |
| 355 | 374 | * @dev: DRM device |
| 356 | − * @attach: DMA-buf attachment | |
| 357 | − * @sg: Scatter/gather table | |
| 375 | + * @dma_buf: DMA-buf | |
| 358 | 376 | * |
| 359 | − * Imports shared DMA buffer memory exported by another device. | |
| 377 | + * Creates an empty SG BO for DMA-buf import. | |
| 360 | 378 | * |
| 361 | 379 | * Returns: |
| 362 | 380 | * A new GEM BO of the given DRM device, representing the memory |
| 363 | 381 | * described by the given DMA-buf attachment and scatter/gather table. |
| 364 | 382 | */ |
| 365 | −struct drm_gem_object * | |
| 366 | −amdgpu_gem_prime_import_sg_table(struct drm_device *dev, | |
| 367 | − struct dma_buf_attachment *attach, | |
| 368 | − struct sg_table *sg) | |
| 383 | +static struct drm_gem_object * | |
| 384 | +amdgpu_dma_buf_create_obj(struct drm_device *dev, struct dma_buf *dma_buf) | |
| 369 | 385 | { |
| 370 | − struct dma_resv *resv = attach->dmabuf->resv; | |
| 386 | + struct dma_resv *resv = dma_buf->resv; | |
| 371 | 387 | struct amdgpu_device *adev = dev->dev_private; |
| 372 | 388 | struct amdgpu_bo *bo; |
| 373 | 389 | struct amdgpu_bo_param bp; |
| 374 | 390 | int ret; |
| 375 | 391 | |
| 376 | 392 | memset(&bp, 0, sizeof(bp)); |
| 377 | − bp.size = attach->dmabuf->size; | |
| 393 | + bp.size = dma_buf->size; | |
| 378 | 394 | bp.byte_align = PAGE_SIZE; |
| 379 | 395 | bp.domain = AMDGPU_GEM_DOMAIN_CPU; |
| 380 | 396 | bp.flags = 0; |
| @@ -385,11 +401,9 @@ amdgpu_gem_prime_import_sg_table(struct drm_device *dev, · 4 unmodified lines | ||
| 385 | 401 | if (ret) |
| 386 | 402 | goto error; |
| 387 | 403 | |
| 388 | − bo->tbo.sg = sg; | |
| 389 | − bo->tbo.ttm->sg = sg; | |
| 390 | 404 | bo->allowed_domains = AMDGPU_GEM_DOMAIN_GTT; |
| 391 | 405 | bo->preferred_domains = AMDGPU_GEM_DOMAIN_GTT; |
| 392 | − if (attach->dmabuf->ops != &amdgpu_dmabuf_ops) | |
| 406 | + if (dma_buf->ops != &amdgpu_dmabuf_ops) | |
| 393 | 407 | bo->prime_shared_count = 1; |
| 394 | 408 | |
| 395 | 409 | dma_resv_unlock(resv); |
| @@ -405,15 +419,15 @@ error: · 9 unmodified lines | ||
| 405 | 419 | * @dev: DRM device |
| 406 | 420 | * @dma_buf: Shared DMA buffer |
| 407 | 421 | * |
| 408 | − * The main work is done by the &drm_gem_prime_import helper, which in turn | |
| 409 | − * uses &amdgpu_gem_prime_import_sg_table. | |
| 422 | + * Import a dma_buf into a the driver and potentially create a new GEM object. | |
| 410 | 423 | * |
| 411 | 424 | * Returns: |
| 412 | 425 | * GEM BO representing the shared DMA buffer for the given device. |
| 413 | 426 | */ |
| 414 | 427 | struct drm_gem_object *amdgpu_gem_prime_import(struct drm_device *dev, |
| 415 | − struct dma_buf *dma_buf) | |
| 428 | + struct dma_buf *dma_buf) | |
| 416 | 429 | { |
| 430 | + struct dma_buf_attachment *attach; | |
| 417 | 431 | struct drm_gem_object *obj; |
| 418 | 432 | |
| 419 | 433 | if (dma_buf->ops == &amdgpu_dmabuf_ops) { |
| @@ -428,5 +442,17 @@ struct drm_gem_object *amdgpu_gem_prime_import(struct drm_device *dev, · 8 unmodified lines | ||
| 428 | 442 | } |
| 429 | 443 | } |
| 430 | 444 | |
| 431 | − return drm_gem_prime_import(dev, dma_buf); | |
| 445 | + obj = amdgpu_dma_buf_create_obj(dev, dma_buf); | |
| 446 | + if (IS_ERR(obj)) | |
| 447 | + return obj; | |
| 448 | + | |
| 449 | + attach = dma_buf_dynamic_attach(dma_buf, dev->dev, true); | |
| 450 | + if (IS_ERR(attach)) { | |
| 451 | + drm_gem_object_put(obj); | |
| 452 | + return ERR_CAST(attach); | |
| 453 | + } | |
| 454 | + | |
| 455 | + get_dma_buf(dma_buf); | |
| 456 | + obj->import_attach = attach; | |
| 457 | + return obj; | |
| 432 | 458 | } |
| @@ -25,11 +25,6 @@ · 24 unmodified lines | ||
| 25 | 25 | |
| 26 | 26 | #include <drm/drm_gem.h> |
| 27 | 27 | |
| 28 | −struct sg_table *amdgpu_gem_prime_get_sg_table(struct drm_gem_object *obj); | |
| 29 | −struct drm_gem_object * | |
| 30 | −amdgpu_gem_prime_import_sg_table(struct drm_device *dev, | |
| 31 | − struct dma_buf_attachment *attach, | |
| 32 | − struct sg_table *sg); | |
| 33 | 28 | struct dma_buf *amdgpu_gem_prime_export(struct drm_gem_object *gobj, |
| 34 | 29 | int flags); |
| 35 | 30 | struct drm_gem_object *amdgpu_gem_prime_import(struct drm_device *dev, |
| @@ -911,7 +911,8 @@ int amdgpu_dpm_get_sclk(struct amdgpu_device *adev, bool low) · 910 unmodified lines | ||
| 911 | 911 | if (is_support_sw_smu(adev)) { |
| 912 | 912 | ret = smu_get_dpm_freq_range(&adev->smu, SMU_GFXCLK, |
| 913 | 913 | low ? &clk_freq : NULL, |
| 914 | − !low ? &clk_freq : NULL); | |
| 914 | + !low ? &clk_freq : NULL, | |
| 915 | + true); | |
| 915 | 916 | if (ret) |
| 916 | 917 | return 0; |
| 917 | 918 | return clk_freq * 100; |
| @@ -928,7 +929,8 @@ int amdgpu_dpm_get_mclk(struct amdgpu_device *adev, bool low) · 10 unmodified lines | ||
| 928 | 929 | if (is_support_sw_smu(adev)) { |
| 929 | 930 | ret = smu_get_dpm_freq_range(&adev->smu, SMU_UCLK, |
| 930 | 931 | low ? &clk_freq : NULL, |
| 931 | − !low ? &clk_freq : NULL); | |
| 932 | + !low ? &clk_freq : NULL, | |
| 933 | + true); | |
| 932 | 934 | if (ret) |
| 933 | 935 | return 0; |
| 934 | 936 | return clk_freq * 100; |
| @@ -298,12 +298,6 @@ enum amdgpu_pcie_gen { · 297 unmodified lines | ||
| 298 | 298 | #define amdgpu_dpm_get_current_power_state(adev) \ |
| 299 | 299 | ((adev)->powerplay.pp_funcs->get_current_power_state((adev)->powerplay.pp_handle)) |
| 300 | 300 | |
| 301 | −#define amdgpu_smu_get_current_power_state(adev) \ | |
| 302 | − ((adev)->smu.ppt_funcs->get_current_power_state(&((adev)->smu))) | |
| 303 | − | |
| 304 | −#define amdgpu_smu_set_power_state(adev) \ | |
| 305 | − ((adev)->smu.ppt_funcs->set_power_state(&((adev)->smu))) | |
| 306 | − | |
| 307 | 301 | #define amdgpu_dpm_get_pp_num_states(adev, data) \ |
| 308 | 302 | ((adev)->powerplay.pp_funcs->get_pp_num_states((adev)->powerplay.pp_handle, data)) |
| 309 | 303 | |
| @@ -43,6 +43,8 @@ · 42 unmodified lines | ||
| 43 | 43 | |
| 44 | 44 | #include "amdgpu_amdkfd.h" |
| 45 | 45 | |
| 46 | +#include "amdgpu_ras.h" | |
| 47 | + | |
| 46 | 48 | /* |
| 47 | 49 | * KMS wrapper. |
| 48 | 50 | * - 3.0.0 - initial driver |
| @@ -82,13 +84,12 @@ · 33 unmodified lines | ||
| 82 | 84 | * - 3.33.0 - Fixes for GDS ENOMEM failures in AMDGPU_CS. |
| 83 | 85 | * - 3.34.0 - Non-DC can flip correctly between buffers with different pitches |
| 84 | 86 | * - 3.35.0 - Add drm_amdgpu_info_device::tcc_disabled_mask |
| 87 | + * - 3.36.0 - Allow reading more status registers on si/cik | |
| 85 | 88 | */ |
| 86 | 89 | #define KMS_DRIVER_MAJOR 3 |
| 87 | −#define KMS_DRIVER_MINOR 35 | |
| 90 | +#define KMS_DRIVER_MINOR 36 | |
| 88 | 91 | #define KMS_DRIVER_PATCHLEVEL 0 |
| 89 | 92 | |
| 90 | −#define AMDGPU_MAX_TIMEOUT_PARAM_LENTH 256 | |
| 91 | − | |
| 92 | 93 | int amdgpu_vram_limit = 0; |
| 93 | 94 | int amdgpu_vis_vram_limit = 0; |
| 94 | 95 | int amdgpu_gart_size = -1; /* auto */ |
| @@ -101,7 +102,7 @@ int amdgpu_disp_priority = 0; · 6 unmodified lines | ||
| 101 | 102 | int amdgpu_hw_i2c = 0; |
| 102 | 103 | int amdgpu_pcie_gen2 = -1; |
| 103 | 104 | int amdgpu_msi = -1; |
| 104 | −char amdgpu_lockup_timeout[AMDGPU_MAX_TIMEOUT_PARAM_LENTH]; | |
| 105 | +char amdgpu_lockup_timeout[AMDGPU_MAX_TIMEOUT_PARAM_LENGTH]; | |
| 105 | 106 | int amdgpu_dpm = -1; |
| 106 | 107 | int amdgpu_fw_load_type = -1; |
| 107 | 108 | int amdgpu_aspm = -1; |
| @@ -128,11 +129,7 @@ char *amdgpu_disable_cu = NULL; · 20 unmodified lines | ||
| 128 | 129 | char *amdgpu_virtual_display = NULL; |
| 129 | 130 | /* OverDrive(bit 14) disabled by default*/ |
| 130 | 131 | uint amdgpu_pp_feature_mask = 0xffffbfff; |
| 131 | −int amdgpu_ngg = 0; | |
| 132 | −int amdgpu_prim_buf_per_se = 0; | |
| 133 | −int amdgpu_pos_buf_per_se = 0; | |
| 134 | −int amdgpu_cntl_sb_buf_per_se = 0; | |
| 135 | −int amdgpu_param_buf_per_se = 0; | |
| 132 | +uint amdgpu_force_long_training = 0; | |
| 136 | 133 | int amdgpu_job_hang_limit = 0; |
| 137 | 134 | int amdgpu_lbpw = -1; |
| 138 | 135 | int amdgpu_compute_multipipe = -1; |
| @@ -146,12 +143,13 @@ int amdgpu_mcbp = 0; · 7 unmodified lines | ||
| 146 | 143 | int amdgpu_discovery = -1; |
| 147 | 144 | int amdgpu_mes = 0; |
| 148 | 145 | int amdgpu_noretry = 1; |
| 146 | +int amdgpu_force_asic_type = -1; | |
| 149 | 147 | |
| 150 | 148 | struct amdgpu_mgpu_info mgpu_info = { |
| 151 | 149 | .mutex = __MUTEX_INITIALIZER(mgpu_info.mutex), |
| 152 | 150 | }; |
| 153 | 151 | int amdgpu_ras_enable = -1; |
| 154 | −uint amdgpu_ras_mask = 0xfffffffb; | |
| 152 | +uint amdgpu_ras_mask = 0xffffffff; | |
| 155 | 153 | |
| 156 | 154 | /** |
| 157 | 155 | * DOC: vramlimit (int) |
| @@ -244,16 +242,21 @@ module_param_named(msi, amdgpu_msi, int, 0444); · 86 unmodified lines | ||
| 244 | 242 | * |
| 245 | 243 | * The format can be [Non-Compute] or [GFX,Compute,SDMA,Video]. That is there can be one or |
| 246 | 244 | * multiple values specified. 0 and negative values are invalidated. They will be adjusted |
| 247 | − * to default timeout. | |
| 248 | − * - With one value specified, the setting will apply to all non-compute jobs. | |
| 249 | − * - With multiple values specified, the first one will be for GFX. The second one is for Compute. | |
| 250 | − * And the third and fourth ones are for SDMA and Video. | |
| 245 | + * to the default timeout. | |
| 246 | + * | |
| 247 | + * - With one value specified, the setting will apply to all non-compute jobs. | |
| 248 | + * - With multiple values specified, the first one will be for GFX. | |
| 249 | + * The second one is for Compute. The third and fourth ones are | |
| 250 | + * for SDMA and Video. | |
| 251 | + * | |
| 251 | 252 | * By default(with no lockup_timeout settings), the timeout for all non-compute(GFX, SDMA and Video) |
| 252 | 253 | * jobs is 10000. And there is no timeout enforced on compute jobs. |
| 253 | 254 | */ |
| 254 | −MODULE_PARM_DESC(lockup_timeout, "GPU lockup timeout in ms (default: 10000 for non-compute jobs and infinity timeout for compute jobs." | |
| 255 | +MODULE_PARM_DESC(lockup_timeout, "GPU lockup timeout in ms (default: for bare metal 10000 for non-compute jobs and infinity timeout for compute jobs; " | |
| 256 | + "for passthrough or sriov, 10000 for all jobs." | |
| 255 | 257 | " 0: keep default value. negative: infinity timeout), " |
| 256 | − "format is [Non-Compute] or [GFX,Compute,SDMA,Video]"); | |
| 258 | + "format: for bare metal [Non-Compute] or [GFX,Compute,SDMA,Video]; " | |
| 259 | + "for passthrough or sriov [all jobs] or [GFX,Compute,SDMA,Video]."); | |
| 257 | 260 | module_param_string(lockup_timeout, amdgpu_lockup_timeout, sizeof(amdgpu_lockup_timeout), 0444); |
| 258 | 261 | |
| 259 | 262 | /** |
| @@ -391,6 +394,14 @@ module_param_named(sched_hw_submission, amdgpu_sched_hw_submission, int, 0444); · 131 unmodified lines | ||
| 391 | 394 | MODULE_PARM_DESC(ppfeaturemask, "all power features enabled (default))"); |
| 392 | 395 | module_param_named(ppfeaturemask, amdgpu_pp_feature_mask, uint, 0444); |
| 393 | 396 | |
| 397 | +/** | |
| 398 | + * DOC: forcelongtraining (uint) | |
| 399 | + * Force long memory training in resume. | |
| 400 | + * The default is zero, indicates short training in resume. | |
| 401 | + */ | |
| 402 | +MODULE_PARM_DESC(forcelongtraining, "force memory long training"); | |
| 403 | +module_param_named(forcelongtraining, amdgpu_force_long_training, uint, 0444); | |
| 404 | + | |
| 394 | 405 | /** |
| 395 | 406 | * DOC: pcie_gen_cap (uint) |
| 396 | 407 | * Override PCIE gen speed capabilities. See the CAIL flags in drivers/gpu/drm/amd/include/amd_pcie.h. |
| @@ -448,42 +459,6 @@ MODULE_PARM_DESC(virtual_display, · 51 unmodified lines | ||
| 448 | 459 | "Enable virtual display feature (the virtual_display will be set like xxxx:xx:xx.x,x;xxxx:xx:xx.x,x)"); |
| 449 | 460 | module_param_named(virtual_display, amdgpu_virtual_display, charp, 0444); |
| 450 | 461 | |
| 451 | −/** | |
| 452 | − * DOC: ngg (int) | |
| 453 | − * Set to enable Next Generation Graphics (1 = enable). The default is 0 (disabled). | |
| 454 | − */ | |
| 455 | −MODULE_PARM_DESC(ngg, "Next Generation Graphics (1 = enable, 0 = disable(default depending on gfx))"); | |
| 456 | −module_param_named(ngg, amdgpu_ngg, int, 0444); | |
| 457 | − | |
| 458 | −/** | |
| 459 | − * DOC: prim_buf_per_se (int) | |
| 460 | − * Override the size of Primitive Buffer per Shader Engine in Byte. The default is 0 (depending on gfx). | |
| 461 | − */ | |
| 462 | −MODULE_PARM_DESC(prim_buf_per_se, "the size of Primitive Buffer per Shader Engine (default depending on gfx)"); | |
| 463 | −module_param_named(prim_buf_per_se, amdgpu_prim_buf_per_se, int, 0444); | |
| 464 | − | |
| 465 | −/** | |
| 466 | − * DOC: pos_buf_per_se (int) | |
| 467 | − * Override the size of Position Buffer per Shader Engine in Byte. The default is 0 (depending on gfx). | |
| 468 | − */ | |
| 469 | −MODULE_PARM_DESC(pos_buf_per_se, "the size of Position Buffer per Shader Engine (default depending on gfx)"); | |
| 470 | −module_param_named(pos_buf_per_se, amdgpu_pos_buf_per_se, int, 0444); | |
| 471 | − | |
| 472 | −/** | |
| 473 | − * DOC: cntl_sb_buf_per_se (int) | |
| 474 | − * Override the size of Control Sideband per Shader Engine in Byte. The default is 0 (depending on gfx). | |
| 475 | − */ | |
| 476 | −MODULE_PARM_DESC(cntl_sb_buf_per_se, "the size of Control Sideband per Shader Engine (default depending on gfx)"); | |
| 477 | −module_param_named(cntl_sb_buf_per_se, amdgpu_cntl_sb_buf_per_se, int, 0444); | |
| 478 | − | |
| 479 | −/** | |
| 480 | − * DOC: param_buf_per_se (int) | |
| 481 | − * Override the size of Off-Chip Parameter Cache per Shader Engine in Byte. | |
| 482 | − * The default is 0 (depending on gfx). | |
| 483 | − */ | |
| 484 | −MODULE_PARM_DESC(param_buf_per_se, "the size of Off-Chip Parameter Cache per Shader Engine (default depending on gfx)"); | |
| 485 | −module_param_named(param_buf_per_se, amdgpu_param_buf_per_se, int, 0444); | |
| 486 | − | |
| 487 | 462 | /** |
| 488 | 463 | * DOC: job_hang_limit (int) |
| 489 | 464 | * Set how much time allow a job hang and not drop it. The default is 0. |
| @@ -616,6 +591,16 @@ MODULE_PARM_DESC(noretry, · 126 unmodified lines | ||
| 616 | 591 | "Disable retry faults (0 = retry enabled, 1 = retry disabled (default))"); |
| 617 | 592 | module_param_named(noretry, amdgpu_noretry, int, 0644); |
| 618 | 593 | |
| 594 | +/** | |
| 595 | + * DOC: force_asic_type (int) | |
| 596 | + * A non negative value used to specify the asic type for all supported GPUs. | |
| 597 | + */ | |
| 598 | +MODULE_PARM_DESC(force_asic_type, | |
| 599 | + "A non negative value used to specify the asic type for all supported GPUs"); | |
| 600 | +module_param_named(force_asic_type, amdgpu_force_asic_type, int, 0444); | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 619 | 604 | #ifdef CONFIG_HSA_AMD |
| 620 | 605 | /** |
| 621 | 606 | * DOC: sched_policy (int) |
| @@ -1016,12 +1001,14 @@ static const struct pci_device_id pciidlist[] = { · 394 unmodified lines | ||
| 1016 | 1001 | {0x1002, 0x7340, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_NAVI14|AMD_EXP_HW_SUPPORT}, |
| 1017 | 1002 | {0x1002, 0x7341, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_NAVI14|AMD_EXP_HW_SUPPORT}, |
| 1018 | 1003 | {0x1002, 0x7347, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_NAVI14|AMD_EXP_HW_SUPPORT}, |
| 1004 | + {0x1002, 0x734F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_NAVI14|AMD_EXP_HW_SUPPORT}, | |
| 1019 | 1005 | |
| 1020 | 1006 | /* Renoir */ |
| 1021 | 1007 | {0x1002, 0x1636, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_RENOIR|AMD_IS_APU|AMD_EXP_HW_SUPPORT}, |
| 1022 | 1008 | |
| 1023 | 1009 | /* Navi12 */ |
| 1024 | 1010 | {0x1002, 0x7360, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_NAVI12|AMD_EXP_HW_SUPPORT}, |
| 1011 | + {0x1002, 0x7362, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_NAVI12|AMD_EXP_HW_SUPPORT}, | |
| 1025 | 1012 | |
| 1026 | 1013 | {0, 0, 0} |
| 1027 | 1014 | }; |
| @@ -1048,6 +1035,41 @@ static int amdgpu_pci_probe(struct pci_dev *pdev, · 20 unmodified lines | ||
| 1048 | 1035 | return -ENODEV; |
| 1049 | 1036 | } |
| 1050 | 1037 | |
| 1038 | +#ifdef CONFIG_DRM_AMDGPU_SI | |
| 1039 | + if (!amdgpu_si_support) { | |
| 1040 | + switch (flags & AMD_ASIC_MASK) { | |
| 1041 | + case CHIP_TAHITI: | |
| 1042 | + case CHIP_PITCAIRN: | |
| 1043 | + case CHIP_VERDE: | |
| 1044 | + case CHIP_OLAND: | |
| 1045 | + case CHIP_HAINAN: | |
| 1046 | + dev_info(&pdev->dev, | |
| 1047 | + "SI support provided by radeon.\n"); | |
| 1048 | + dev_info(&pdev->dev, | |
| 1049 | + "Use radeon.si_support=0 amdgpu.si_support=1 to override.\n" | |
| 1050 | + ); | |
| 1051 | + return -ENODEV; | |
| 1052 | + } | |
| 1053 | + } | |
| 1054 | +#endif | |
| 1055 | +#ifdef CONFIG_DRM_AMDGPU_CIK | |
| 1056 | + if (!amdgpu_cik_support) { | |
| 1057 | + switch (flags & AMD_ASIC_MASK) { | |
| 1058 | + case CHIP_KAVERI: | |
| 1059 | + case CHIP_BONAIRE: | |
| 1060 | + case CHIP_HAWAII: | |
| 1061 | + case CHIP_KABINI: | |
| 1062 | + case CHIP_MULLINS: | |
| 1063 | + dev_info(&pdev->dev, | |
| 1064 | + "CIK support provided by radeon.\n"); | |
| 1065 | + dev_info(&pdev->dev, | |
| 1066 | + "Use radeon.cik_support=0 amdgpu.cik_support=1 to override.\n" | |
| 1067 | + ); | |
| 1068 | + return -ENODEV; | |
| 1069 | + } | |
| 1070 | + } | |
| 1071 | +#endif | |
| 1072 | + | |
| 1051 | 1073 | /* Get rid of things like offb */ |
| 1052 | 1074 | ret = drm_fb_helper_remove_conflicting_pci_framebuffers(pdev, "amdgpudrmfb"); |
| 1053 | 1075 | if (ret) |
| @@ -1092,7 +1114,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) · 38 unmodified lines | ||
| 1092 | 1114 | { |
| 1093 | 1115 | struct drm_device *dev = pci_get_drvdata(pdev); |
| 1094 | 1116 | |
| 1095 | − DRM_ERROR("Device removal is currently not supported outside of fbcon\n"); | |
| 1117 | +#ifdef MODULE | |
| 1118 | + if (THIS_MODULE->state != MODULE_STATE_GOING) | |
| 1119 | +#endif | |
| 1120 | + DRM_ERROR("Hotplug removal is not supported\n"); | |
| 1096 | 1121 | drm_dev_unplug(dev); |
| 1097 | 1122 | drm_dev_put(dev); |
| 1098 | 1123 | pci_disable_device(pdev); |
| @@ -1105,6 +1130,9 @@ amdgpu_pci_shutdown(struct pci_dev *pdev) · 6 unmodified lines | ||
| 1105 | 1130 | struct drm_device *dev = pci_get_drvdata(pdev); |
| 1106 | 1131 | struct amdgpu_device *adev = dev->dev_private; |
| 1107 | 1132 | |
| 1133 | + if (amdgpu_ras_intr_triggered()) | |
| 1134 | + return; | |
| 1135 | + | |
| 1108 | 1136 | /* if we are running in a VM, make sure the device |
| 1109 | 1137 | * torn down properly on reboot/shutdown. |
| 1110 | 1138 | * unfortunately we can't detect certain |
| @@ -1139,8 +1167,13 @@ static int amdgpu_pmops_resume(struct device *dev) · 28 unmodified lines | ||
| 1139 | 1167 | static int amdgpu_pmops_freeze(struct device *dev) |
| 1140 | 1168 | { |
| 1141 | 1169 | struct drm_device *drm_dev = dev_get_drvdata(dev); |
| 1170 | + struct amdgpu_device *adev = drm_dev->dev_private; | |
| 1171 | + int r; | |
| 1142 | 1172 | |
| 1143 | − return amdgpu_device_suspend(drm_dev, false, true); | |
| 1173 | + r = amdgpu_device_suspend(drm_dev, false, true); | |
| 1174 | + if (r) | |
| 1175 | + return r; | |
| 1176 | + return amdgpu_asic_reset(adev); | |
| 1144 | 1177 | } |
| 1145 | 1178 | |
| 1146 | 1179 | static int amdgpu_pmops_thaw(struct device *dev) |
| @@ -1312,66 +1345,6 @@ int amdgpu_file_to_fpriv(struct file *filp, struct amdgpu_fpriv **fpriv) · 165 unmodified lines | ||
| 1312 | 1345 | return 0; |
| 1313 | 1346 | } |
| 1314 | 1347 | |
| 1315 | −int amdgpu_device_get_job_timeout_settings(struct amdgpu_device *adev) | |
| 1316 | −{ | |
| 1317 | − char *input = amdgpu_lockup_timeout; | |
| 1318 | − char *timeout_setting = NULL; | |
| 1319 | − int index = 0; | |
| 1320 | − long timeout; | |
| 1321 | − int ret = 0; | |
| 1322 | − | |
| 1323 | − /* | |
| 1324 | − * By default timeout for non compute jobs is 10000. | |
| 1325 | − * And there is no timeout enforced on compute jobs. | |
| 1326 | − */ | |
| 1327 | − adev->gfx_timeout = msecs_to_jiffies(10000); | |
| 1328 | − adev->sdma_timeout = adev->video_timeout = adev->gfx_timeout; | |
| 1329 | − adev->compute_timeout = MAX_SCHEDULE_TIMEOUT; | |
| 1330 | − | |
| 1331 | − if (strnlen(input, AMDGPU_MAX_TIMEOUT_PARAM_LENTH)) { | |
| 1332 | − while ((timeout_setting = strsep(&input, ",")) && | |
| 1333 | − strnlen(timeout_setting, AMDGPU_MAX_TIMEOUT_PARAM_LENTH)) { | |
| 1334 | − ret = kstrtol(timeout_setting, 0, &timeout); | |
| 1335 | − if (ret) | |
| 1336 | − return ret; | |
| 1337 | − | |
| 1338 | − if (timeout == 0) { | |
| 1339 | − index++; | |
| 1340 | − continue; | |
| 1341 | − } else if (timeout < 0) { | |
| 1342 | − timeout = MAX_SCHEDULE_TIMEOUT; | |
| 1343 | − } else { | |
| 1344 | − timeout = msecs_to_jiffies(timeout); | |
| 1345 | − } | |
| 1346 | − | |
| 1347 | − switch (index++) { | |
| 1348 | − case 0: | |
| 1349 | − adev->gfx_timeout = timeout; | |
| 1350 | − break; | |
| 1351 | − case 1: | |
| 1352 | − adev->compute_timeout = timeout; | |
| 1353 | − break; | |
| 1354 | − case 2: | |
| 1355 | − adev->sdma_timeout = timeout; | |
| 1356 | − break; | |
| 1357 | − case 3: | |
| 1358 | − adev->video_timeout = timeout; | |
| 1359 | − break; | |
| 1360 | − default: | |
| 1361 | − break; | |
| 1362 | − } | |
| 1363 | − } | |
| 1364 | − /* | |
| 1365 | − * There is only one value specified and | |
| 1366 | − * it should apply to all non-compute jobs. | |
| 1367 | − */ | |
| 1368 | − if (index == 1) | |
| 1369 | − adev->sdma_timeout = adev->video_timeout = adev->gfx_timeout; | |
| 1370 | − } | |
| 1371 | − | |
| 1372 | − return ret; | |
| 1373 | −} | |
| 1374 | − | |
| 1375 | 1348 | static bool |
| 1376 | 1349 | amdgpu_get_crtc_scanout_position(struct drm_device *dev, unsigned int pipe, |
| 1377 | 1350 | bool in_vblank_irq, int *vpos, int *hpos, |
| @@ -1410,8 +1383,6 @@ static struct drm_driver kms_driver = { · 32 unmodified lines | ||
| 1410 | 1383 | .prime_fd_to_handle = drm_gem_prime_fd_to_handle, |
| 1411 | 1384 | .gem_prime_export = amdgpu_gem_prime_export, |
| 1412 | 1385 | .gem_prime_import = amdgpu_gem_prime_import, |
| 1413 | − .gem_prime_get_sg_table = amdgpu_gem_prime_get_sg_table, | |
| 1414 | − .gem_prime_import_sg_table = amdgpu_gem_prime_import_sg_table, | |
| 1415 | 1386 | .gem_prime_vmap = amdgpu_gem_prime_vmap, |
| 1416 | 1387 | .gem_prime_vunmap = amdgpu_gem_prime_vunmap, |
| 1417 | 1388 | .gem_prime_mmap = amdgpu_gem_prime_mmap, |
| @@ -37,12 +37,14 @@ amdgpu_link_encoder_connector(struct drm_device *dev) · 36 unmodified lines | ||
| 37 | 37 | { |
| 38 | 38 | struct amdgpu_device *adev = dev->dev_private; |
| 39 | 39 | struct drm_connector *connector; |
| 40 | + struct drm_connector_list_iter iter; | |
| 40 | 41 | struct amdgpu_connector *amdgpu_connector; |
| 41 | 42 | struct drm_encoder *encoder; |
| 42 | 43 | struct amdgpu_encoder *amdgpu_encoder; |
| 43 | 44 | |
| 45 | + drm_connector_list_iter_begin(dev, &iter); | |
| 44 | 46 | /* walk the list and link encoders to connectors */ |
| 45 | − list_for_each_entry(connector, &dev->mode_config.connector_list, head) { | |
| 47 | + drm_for_each_connector_iter(connector, &iter) { | |
| 46 | 48 | amdgpu_connector = to_amdgpu_connector(connector); |
| 47 | 49 | list_for_each_entry(encoder, &dev->mode_config.encoder_list, head) { |
| 48 | 50 | amdgpu_encoder = to_amdgpu_encoder(encoder); |
| @@ -55,6 +57,7 @@ amdgpu_link_encoder_connector(struct drm_device *dev) · 6 unmodified lines | ||
| 55 | 57 | } |
| 56 | 58 | } |
| 57 | 59 | } |
| 60 | + drm_connector_list_iter_end(&iter); | |
| 58 | 61 | } |
| 59 | 62 | |
| 60 | 63 | void amdgpu_encoder_set_active_device(struct drm_encoder *encoder) |
| @@ -62,8 +65,10 @@ void amdgpu_encoder_set_active_device(struct drm_encoder *encoder) · 1 unmodified line | ||
| 62 | 65 | struct drm_device *dev = encoder->dev; |
| 63 | 66 | struct amdgpu_encoder *amdgpu_encoder = to_amdgpu_encoder(encoder); |
| 64 | 67 | struct drm_connector *connector; |
| 68 | + struct drm_connector_list_iter iter; | |
| 65 | 69 | |
| 66 | − list_for_each_entry(connector, &dev->mode_config.connector_list, head) { | |
| 70 | + drm_connector_list_iter_begin(dev, &iter); | |
| 71 | + drm_for_each_connector_iter(connector, &iter) { | |
| 67 | 72 | if (connector->encoder == encoder) { |
| 68 | 73 | struct amdgpu_connector *amdgpu_connector = to_amdgpu_connector(connector); |
| 69 | 74 | amdgpu_encoder->active_device = amdgpu_encoder->devices & amdgpu_connector->devices; |
| @@ -72,6 +77,7 @@ void amdgpu_encoder_set_active_device(struct drm_encoder *encoder) · 2 unmodified lines | ||
| 72 | 77 | amdgpu_connector->devices, encoder->encoder_type); |
| 73 | 78 | } |
| 74 | 79 | } |
| 80 | + drm_connector_list_iter_end(&iter); | |
| 75 | 81 | } |
| 76 | 82 | |
| 77 | 83 | struct drm_connector * |
| @@ -79,15 +85,20 @@ amdgpu_get_connector_for_encoder(struct drm_encoder *encoder) · 1 unmodified line | ||
| 79 | 85 | { |
| 80 | 86 | struct drm_device *dev = encoder->dev; |
| 81 | 87 | struct amdgpu_encoder *amdgpu_encoder = to_amdgpu_encoder(encoder); |
| 82 | − struct drm_connector *connector; | |
| 88 | + struct drm_connector *connector, *found = NULL; | |
| 89 | + struct drm_connector_list_iter iter; | |
| 83 | 90 | struct amdgpu_connector *amdgpu_connector; |
| 84 | 91 | |
| 85 | − list_for_each_entry(connector, &dev->mode_config.connector_list, head) { | |
| 92 | + drm_connector_list_iter_begin(dev, &iter); | |
| 93 | + drm_for_each_connector_iter(connector, &iter) { | |
| 86 | 94 | amdgpu_connector = to_amdgpu_connector(connector); |
| 87 | − if (amdgpu_encoder->active_device & amdgpu_connector->devices) | |
| 88 | − return connector; | |
| 95 | + if (amdgpu_encoder->active_device & amdgpu_connector->devices) { | |
| 96 | + found = connector; | |
| 97 | + break; | |
| 98 | + } | |
| 89 | 99 | } |
| 90 | − return NULL; | |
| 100 | + drm_connector_list_iter_end(&iter); | |
| 101 | + return found; | |
| 91 | 102 | } |
| 92 | 103 | |
| 93 | 104 | struct drm_connector * |
| @@ -95,15 +106,20 @@ amdgpu_get_connector_for_encoder_init(struct drm_encoder *encoder) · 1 unmodified line | ||
| 95 | 106 | { |
| 96 | 107 | struct drm_device *dev = encoder->dev; |
| 97 | 108 | struct amdgpu_encoder *amdgpu_encoder = to_amdgpu_encoder(encoder); |
| 98 | − struct drm_connector *connector; | |
| 109 | + struct drm_connector *connector, *found = NULL; | |
| 110 | + struct drm_connector_list_iter iter; | |
| 99 | 111 | struct amdgpu_connector *amdgpu_connector; |
| 100 | 112 | |
| 101 | − list_for_each_entry(connector, &dev->mode_config.connector_list, head) { | |
| 113 | + drm_connector_list_iter_begin(dev, &iter); | |
| 114 | + drm_for_each_connector_iter(connector, &iter) { | |
| 102 | 115 | amdgpu_connector = to_amdgpu_connector(connector); |
| 103 | − if (amdgpu_encoder->devices & amdgpu_connector->devices) | |
| 104 | − return connector; | |
| 116 | + if (amdgpu_encoder->devices & amdgpu_connector->devices) { | |
| 117 | + found = connector; | |
| 118 | + break; | |
| 119 | + } | |
| 105 | 120 | } |
| 106 | − return NULL; | |
| 121 | + drm_connector_list_iter_end(&iter); | |
| 122 | + return found; | |
| 107 | 123 | } |
| 108 | 124 | |
| 109 | 125 | struct drm_encoder *amdgpu_get_external_encoder(struct drm_encoder *encoder) |
| @@ -462,18 +462,7 @@ int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, · 461 unmodified lines | ||
| 462 | 462 | timeout = adev->gfx_timeout; |
| 463 | 463 | break; |
| 464 | 464 | case AMDGPU_RING_TYPE_COMPUTE: |
| 465 | − /* | |
| 466 | − * For non-sriov case, no timeout enforce | |
| 467 | − * on compute ring by default. Unless user | |
| 468 | − * specifies a timeout for compute ring. | |
| 469 | − * | |
| 470 | − * For sriov case, always use the timeout | |
| 471 | − * as gfx ring | |
| 472 | − */ | |
| 473 | − if (!amdgpu_sriov_vf(ring->adev)) | |
| 474 | − timeout = adev->compute_timeout; | |
| 475 | − else | |
| 476 | − timeout = adev->gfx_timeout; | |
| 465 | + timeout = adev->compute_timeout; | |
| 477 | 466 | break; |
| 478 | 467 | case AMDGPU_RING_TYPE_SDMA: |
| 479 | 468 | timeout = adev->sdma_timeout; |
| @@ -71,7 +71,7 @@ · 70 unmodified lines | ||
| 71 | 71 | */ |
| 72 | 72 | static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) |
| 73 | 73 | { |
| 74 | − struct page *dummy_page = adev->mman.bdev.glob->dummy_read_page; | |
| 74 | + struct page *dummy_page = ttm_bo_glob.dummy_read_page; | |
| 75 | 75 | |
| 76 | 76 | if (adev->dummy_page_addr) |
| 77 | 77 | return 0; |
| @@ -175,7 +175,7 @@ void amdgpu_gem_object_close(struct drm_gem_object *obj, · 174 unmodified lines | ||
| 175 | 175 | |
| 176 | 176 | amdgpu_vm_get_pd_bo(vm, &list, &vm_pd); |
| 177 | 177 | |
| 178 | − r = ttm_eu_reserve_buffers(&ticket, &list, false, &duplicates, false); | |
| 178 | + r = ttm_eu_reserve_buffers(&ticket, &list, false, &duplicates); | |
| 179 | 179 | if (r) { |
| 180 | 180 | dev_err(adev->dev, "leaking bo va because " |
| 181 | 181 | "we fail to reserve bo (%d)\n", r); |
| @@ -527,13 +527,41 @@ static void amdgpu_gem_va_update_vm(struct amdgpu_device *adev, · 345 unmodified lines | ||
| 527 | 527 | goto error; |
| 528 | 528 | } |
| 529 | 529 | |
| 530 | − r = amdgpu_vm_update_directories(adev, vm); | |
| 530 | + r = amdgpu_vm_update_pdes(adev, vm, false); | |
| 531 | 531 | |
| 532 | 532 | error: |
| 533 | 533 | if (r && r != -ERESTARTSYS) |
| 534 | 534 | DRM_ERROR("Couldn't update BO_VA (%d)\n", r); |
| 535 | 535 | } |
| 536 | 536 | |
| 537 | +/** | |
| 538 | + * amdgpu_gem_va_map_flags - map GEM UAPI flags into hardware flags | |
| 539 | + * | |
| 540 | + * @adev: amdgpu_device pointer | |
| 541 | + * @flags: GEM UAPI flags | |
| 542 | + * | |
| 543 | + * Returns the GEM UAPI flags mapped into hardware for the ASIC. | |
| 544 | + */ | |
| 545 | +uint64_t amdgpu_gem_va_map_flags(struct amdgpu_device *adev, uint32_t flags) | |
| 546 | +{ | |
| 547 | + uint64_t pte_flag = 0; | |
| 548 | + | |
| 549 | + if (flags & AMDGPU_VM_PAGE_EXECUTABLE) | |
| 550 | + pte_flag |= AMDGPU_PTE_EXECUTABLE; | |
| 551 | + if (flags & AMDGPU_VM_PAGE_READABLE) | |
| 552 | + pte_flag |= AMDGPU_PTE_READABLE; | |
| 553 | + if (flags & AMDGPU_VM_PAGE_WRITEABLE) | |
| 554 | + pte_flag |= AMDGPU_PTE_WRITEABLE; | |
| 555 | + if (flags & AMDGPU_VM_PAGE_PRT) | |
| 556 | + pte_flag |= AMDGPU_PTE_PRT; | |
| 557 | + | |
| 558 | + if (adev->gmc.gmc_funcs->map_mtype) | |
| 559 | + pte_flag |= amdgpu_gmc_map_mtype(adev, | |
| 560 | + flags & AMDGPU_VM_MTYPE_MASK); | |
| 561 | + | |
| 562 | + return pte_flag; | |
| 563 | +} | |
| 564 | + | |
| 537 | 565 | int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, |
| 538 | 566 | struct drm_file *filp) |
| 539 | 567 | { |
| @@ -613,7 +641,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, · 73 unmodified lines | ||
| 613 | 641 | |
| 614 | 642 | amdgpu_vm_get_pd_bo(&fpriv->vm, &list, &vm_pd); |
| 615 | 643 | |
| 616 | − r = ttm_eu_reserve_buffers(&ticket, &list, true, &duplicates, false); | |
| 644 | + r = ttm_eu_reserve_buffers(&ticket, &list, true, &duplicates); | |
| 617 | 645 | if (r) |
| 618 | 646 | goto error_unref; |
| 619 | 647 | |
| @@ -631,7 +659,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, · 11 unmodified lines | ||
| 631 | 659 | |
| 632 | 660 | switch (args->operation) { |
| 633 | 661 | case AMDGPU_VA_OP_MAP: |
| 634 | − va_flags = amdgpu_gmc_get_pte_flags(adev, args->flags); | |
| 662 | + va_flags = amdgpu_gem_va_map_flags(adev, args->flags); | |
| 635 | 663 | r = amdgpu_vm_bo_map(adev, bo_va, args->va_address, |
| 636 | 664 | args->offset_in_bo, args->map_size, |
| 637 | 665 | va_flags); |
| @@ -646,7 +674,7 @@ int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, · 8 unmodified lines | ||
| 646 | 674 | args->map_size); |
| 647 | 675 | break; |
| 648 | 676 | case AMDGPU_VA_OP_REPLACE: |
| 649 | − va_flags = amdgpu_gmc_get_pte_flags(adev, args->flags); | |
| 677 | + va_flags = amdgpu_gem_va_map_flags(adev, args->flags); | |
| 650 | 678 | r = amdgpu_vm_bo_replace_map(adev, bo_va, args->va_address, |
| 651 | 679 | args->offset_in_bo, args->map_size, |
| 652 | 680 | va_flags); |
| @@ -67,6 +67,7 @@ int amdgpu_gem_mmap_ioctl(struct drm_device *dev, void *data, · 66 unmodified lines | ||
| 67 | 67 | struct drm_file *filp); |
| 68 | 68 | int amdgpu_gem_wait_idle_ioctl(struct drm_device *dev, void *data, |
| 69 | 69 | struct drm_file *filp); |
| 70 | +uint64_t amdgpu_gem_va_map_flags(struct amdgpu_device *adev, uint32_t flags); | |
| 70 | 71 | int amdgpu_gem_va_ioctl(struct drm_device *dev, void *data, |
| 71 | 72 | struct drm_file *filp); |
| 72 | 73 | int amdgpu_gem_op_ioctl(struct drm_device *dev, void *data, |
| @@ -26,6 +26,7 @@ · 25 unmodified lines | ||
| 26 | 26 | #include "amdgpu.h" |
| 27 | 27 | #include "amdgpu_gfx.h" |
| 28 | 28 | #include "amdgpu_rlc.h" |
| 29 | +#include "amdgpu_ras.h" | |
| 29 | 30 | |
| 30 | 31 | /* delay 0.1 second to enable gfx off feature */ |
| 31 | 32 | #define GFX_OFF_DELAY_ENABLE msecs_to_jiffies(100) |
| @@ -231,12 +232,10 @@ void amdgpu_gfx_compute_queue_acquire(struct amdgpu_device *adev) · 199 unmodified lines | ||
| 231 | 232 | |
| 232 | 233 | void amdgpu_gfx_graphics_queue_acquire(struct amdgpu_device *adev) |
| 233 | 234 | { |
| 234 | − int i, queue, pipe, me; | |
| 235 | + int i, queue, me; | |
| 235 | 236 | |
| 236 | 237 | for (i = 0; i < AMDGPU_MAX_GFX_QUEUES; ++i) { |
| 237 | 238 | queue = i % adev->gfx.me.num_queue_per_pipe; |
| 238 | − pipe = (i / adev->gfx.me.num_queue_per_pipe) | |
| 239 | − % adev->gfx.me.num_pipe_per_me; | |
| 240 | 239 | me = (i / adev->gfx.me.num_queue_per_pipe) |
| 241 | 240 | / adev->gfx.me.num_pipe_per_me; |
| 242 | 241 | |
| @@ -320,8 +319,7 @@ int amdgpu_gfx_kiq_init_ring(struct amdgpu_device *adev, · 77 unmodified lines | ||
| 320 | 319 | return r; |
| 321 | 320 | } |
| 322 | 321 | |
| 323 | −void amdgpu_gfx_kiq_free_ring(struct amdgpu_ring *ring, | |
| 324 | − struct amdgpu_irq_src *irq) | |
| 322 | +void amdgpu_gfx_kiq_free_ring(struct amdgpu_ring *ring) | |
| 325 | 323 | { |
| 326 | 324 | amdgpu_device_wb_free(ring->adev, ring->adev->virt.reg_val_offs); |
| 327 | 325 | amdgpu_ring_fini(ring); |
| @@ -569,3 +567,102 @@ void amdgpu_gfx_off_ctrl(struct amdgpu_device *adev, bool enable) · 241 unmodified lines | ||
| 569 | 567 | |
| 570 | 568 | mutex_unlock(&adev->gfx.gfx_off_mutex); |
| 571 | 569 | } |
| 570 | + | |
| 571 | +int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev) | |
| 572 | +{ | |
| 573 | + int r; | |
| 574 | + struct ras_fs_if fs_info = { | |
| 575 | + .sysfs_name = "gfx_err_count", | |
| 576 | + .debugfs_name = "gfx_err_inject", | |
| 577 | + }; | |
| 578 | + struct ras_ih_if ih_info = { | |
| 579 | + .cb = amdgpu_gfx_process_ras_data_cb, | |
| 580 | + }; | |
| 581 | + | |
| 582 | + if (!adev->gfx.ras_if) { | |
| 583 | + adev->gfx.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 584 | + if (!adev->gfx.ras_if) | |
| 585 | + return -ENOMEM; | |
| 586 | + adev->gfx.ras_if->block = AMDGPU_RAS_BLOCK__GFX; | |
| 587 | + adev->gfx.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 588 | + adev->gfx.ras_if->sub_block_index = 0; | |
| 589 | + strcpy(adev->gfx.ras_if->name, "gfx"); | |
| 590 | + } | |
| 591 | + fs_info.head = ih_info.head = *adev->gfx.ras_if; | |
| 592 | + | |
| 593 | + r = amdgpu_ras_late_init(adev, adev->gfx.ras_if, | |
| 594 | + &fs_info, &ih_info); | |
| 595 | + if (r) | |
| 596 | + goto free; | |
| 597 | + | |
| 598 | + if (amdgpu_ras_is_supported(adev, adev->gfx.ras_if->block)) { | |
| 599 | + r = amdgpu_irq_get(adev, &adev->gfx.cp_ecc_error_irq, 0); | |
| 600 | + if (r) | |
| 601 | + goto late_fini; | |
| 602 | + } else { | |
| 603 | + /* free gfx ras_if if ras is not supported */ | |
| 604 | + r = 0; | |
| 605 | + goto free; | |
| 606 | + } | |
| 607 | + | |
| 608 | + return 0; | |
| 609 | +late_fini: | |
| 610 | + amdgpu_ras_late_fini(adev, adev->gfx.ras_if, &ih_info); | |
| 611 | +free: | |
| 612 | + kfree(adev->gfx.ras_if); | |
| 613 | + adev->gfx.ras_if = NULL; | |
| 614 | + return r; | |
| 615 | +} | |
| 616 | + | |
| 617 | +void amdgpu_gfx_ras_fini(struct amdgpu_device *adev) | |
| 618 | +{ | |
| 619 | + if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX) && | |
| 620 | + adev->gfx.ras_if) { | |
| 621 | + struct ras_common_if *ras_if = adev->gfx.ras_if; | |
| 622 | + struct ras_ih_if ih_info = { | |
| 623 | + .head = *ras_if, | |
| 624 | + .cb = amdgpu_gfx_process_ras_data_cb, | |
| 625 | + }; | |
| 626 | + | |
| 627 | + amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 628 | + kfree(ras_if); | |
| 629 | + } | |
| 630 | +} | |
| 631 | + | |
| 632 | +int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, | |
| 633 | + void *err_data, | |
| 634 | + struct amdgpu_iv_entry *entry) | |
| 635 | +{ | |
| 636 | + /* TODO ue will trigger an interrupt. | |
| 637 | + * | |
| 638 | + * When “Full RAS” is enabled, the per-IP interrupt sources should | |
| 639 | + * be disabled and the driver should only look for the aggregated | |
| 640 | + * interrupt via sync flood | |
| 641 | + */ | |
| 642 | + if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) { | |
| 643 | + kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); | |
| 644 | + if (adev->gfx.funcs->query_ras_error_count) | |
| 645 | + adev->gfx.funcs->query_ras_error_count(adev, err_data); | |
| 646 | + amdgpu_ras_reset_gpu(adev, 0); | |
| 647 | + } | |
| 648 | + return AMDGPU_RAS_SUCCESS; | |
| 649 | +} | |
| 650 | + | |
| 651 | +int amdgpu_gfx_cp_ecc_error_irq(struct amdgpu_device *adev, | |
| 652 | + struct amdgpu_irq_src *source, | |
| 653 | + struct amdgpu_iv_entry *entry) | |
| 654 | +{ | |
| 655 | + struct ras_common_if *ras_if = adev->gfx.ras_if; | |
| 656 | + struct ras_dispatch_if ih_data = { | |
| 657 | + .entry = entry, | |
| 658 | + }; | |
| 659 | + | |
| 660 | + if (!ras_if) | |
| 661 | + return 0; | |
| 662 | + | |
| 663 | + ih_data.head = *ras_if; | |
| 664 | + | |
| 665 | + DRM_ERROR("CP ECC ERROR IRQ\n"); | |
| 666 | + amdgpu_ras_interrupt_dispatch(adev, &ih_data); | |
| 667 | + return 0; | |
| 668 | +} | |
| @@ -201,28 +201,6 @@ struct amdgpu_gfx_funcs { · 200 unmodified lines | ||
| 201 | 201 | int (*query_ras_error_count) (struct amdgpu_device *adev, void *ras_error_status); |
| 202 | 202 | }; |
| 203 | 203 | |
| 204 | −struct amdgpu_ngg_buf { | |
| 205 | − struct amdgpu_bo *bo; | |
| 206 | − uint64_t gpu_addr; | |
| 207 | − uint32_t size; | |
| 208 | − uint32_t bo_size; | |
| 209 | −}; | |
| 210 | − | |
| 211 | −enum { | |
| 212 | − NGG_PRIM = 0, | |
| 213 | − NGG_POS, | |
| 214 | − NGG_CNTL, | |
| 215 | − NGG_PARAM, | |
| 216 | − NGG_BUF_MAX | |
| 217 | −}; | |
| 218 | − | |
| 219 | −struct amdgpu_ngg { | |
| 220 | − struct amdgpu_ngg_buf buf[NGG_BUF_MAX]; | |
| 221 | − uint32_t gds_reserve_addr; | |
| 222 | − uint32_t gds_reserve_size; | |
| 223 | − bool init; | |
| 224 | −}; | |
| 225 | − | |
| 226 | 204 | struct sq_work { |
| 227 | 205 | struct work_struct work; |
| 228 | 206 | unsigned ih_data; |
| @@ -289,6 +267,7 @@ struct amdgpu_gfx { · 60 unmodified lines | ||
| 289 | 267 | uint32_t mec2_feature_version; |
| 290 | 268 | bool mec_fw_write_wait; |
| 291 | 269 | bool me_fw_write_wait; |
| 270 | + bool cp_fw_write_wait; | |
| 292 | 271 | struct amdgpu_ring gfx_ring[AMDGPU_MAX_GFX_RINGS]; |
| 293 | 272 | unsigned num_gfx_rings; |
| 294 | 273 | struct amdgpu_ring compute_ring[AMDGPU_MAX_COMPUTE_RINGS]; |
| @@ -311,9 +290,6 @@ struct amdgpu_gfx { · 16 unmodified lines | ||
| 311 | 290 | uint32_t grbm_soft_reset; |
| 312 | 291 | uint32_t srbm_soft_reset; |
| 313 | 292 | |
| 314 | − /* NGG */ | |
| 315 | − struct amdgpu_ngg ngg; | |
| 316 | − | |
| 317 | 293 | /* gfx off */ |
| 318 | 294 | bool gfx_off_state; /* true: enabled, false: disabled */ |
| 319 | 295 | struct mutex gfx_off_mutex; |
| @@ -355,8 +331,7 @@ int amdgpu_gfx_kiq_init_ring(struct amdgpu_device *adev, · 35 unmodified lines | ||
| 355 | 331 | struct amdgpu_ring *ring, |
| 356 | 332 | struct amdgpu_irq_src *irq); |
| 357 | 333 | |
| 358 | −void amdgpu_gfx_kiq_free_ring(struct amdgpu_ring *ring, | |
| 359 | − struct amdgpu_irq_src *irq); | |
| 334 | +void amdgpu_gfx_kiq_free_ring(struct amdgpu_ring *ring); | |
| 360 | 335 | |
| 361 | 336 | void amdgpu_gfx_kiq_fini(struct amdgpu_device *adev); |
| 362 | 337 | int amdgpu_gfx_kiq_init(struct amdgpu_device *adev, |
| @@ -384,5 +359,12 @@ void amdgpu_gfx_bit_to_me_queue(struct amdgpu_device *adev, int bit, · 21 unmodified lines | ||
| 384 | 359 | bool amdgpu_gfx_is_me_queue_enabled(struct amdgpu_device *adev, int me, |
| 385 | 360 | int pipe, int queue); |
| 386 | 361 | void amdgpu_gfx_off_ctrl(struct amdgpu_device *adev, bool enable); |
| 387 | − | |
| 362 | +int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev); | |
| 363 | +void amdgpu_gfx_ras_fini(struct amdgpu_device *adev); | |
| 364 | +int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, | |
| 365 | + void *err_data, | |
| 366 | + struct amdgpu_iv_entry *entry); | |
| 367 | +int amdgpu_gfx_cp_ecc_error_irq(struct amdgpu_device *adev, | |
| 368 | + struct amdgpu_irq_src *source, | |
| 369 | + struct amdgpu_iv_entry *entry); | |
| 388 | 370 | #endif |
| @@ -27,6 +27,8 @@ · 26 unmodified lines | ||
| 27 | 27 | #include <linux/io-64-nonatomic-lo-hi.h> |
| 28 | 28 | |
| 29 | 29 | #include "amdgpu.h" |
| 30 | +#include "amdgpu_ras.h" | |
| 31 | +#include "amdgpu_xgmi.h" | |
| 30 | 32 | |
| 31 | 33 | /** |
| 32 | 34 | * amdgpu_gmc_get_pde_for_bo - get the PDE for a BO |
| @@ -305,3 +307,29 @@ bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr, · 272 unmodified lines | ||
| 305 | 307 | gmc->fault_hash[hash].idx = gmc->last_fault++; |
| 306 | 308 | return false; |
| 307 | 309 | } |
| 310 | + | |
| 311 | +int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev) | |
| 312 | +{ | |
| 313 | + int r; | |
| 314 | + | |
| 315 | + if (adev->umc.funcs && adev->umc.funcs->ras_late_init) { | |
| 316 | + r = adev->umc.funcs->ras_late_init(adev); | |
| 317 | + if (r) | |
| 318 | + return r; | |
| 319 | + } | |
| 320 | + | |
| 321 | + if (adev->mmhub.funcs && adev->mmhub.funcs->ras_late_init) { | |
| 322 | + r = adev->mmhub.funcs->ras_late_init(adev); | |
| 323 | + if (r) | |
| 324 | + return r; | |
| 325 | + } | |
| 326 | + | |
| 327 | + return amdgpu_xgmi_ras_late_init(adev); | |
| 328 | +} | |
| 329 | + | |
| 330 | +void amdgpu_gmc_ras_fini(struct amdgpu_device *adev) | |
| 331 | +{ | |
| 332 | + amdgpu_umc_ras_fini(adev); | |
| 333 | + amdgpu_mmhub_ras_fini(adev); | |
| 334 | + amdgpu_xgmi_ras_fini(adev); | |
| 335 | +} | |
| @@ -99,12 +99,15 @@ struct amdgpu_gmc_funcs { · 98 unmodified lines | ||
| 99 | 99 | unsigned pasid); |
| 100 | 100 | /* enable/disable PRT support */ |
| 101 | 101 | void (*set_prt)(struct amdgpu_device *adev, bool enable); |
| 102 | − /* set pte flags based per asic */ | |
| 103 | − uint64_t (*get_vm_pte_flags)(struct amdgpu_device *adev, | |
| 104 | − uint32_t flags); | |
| 102 | + /* map mtype to hardware flags */ | |
| 103 | + uint64_t (*map_mtype)(struct amdgpu_device *adev, uint32_t flags); | |
| 105 | 104 | /* get the pde for a given mc addr */ |
| 106 | 105 | void (*get_vm_pde)(struct amdgpu_device *adev, int level, |
| 107 | 106 | u64 *dst, u64 *flags); |
| 107 | + /* get the pte flags to use for a BO VA mapping */ | |
| 108 | + void (*get_vm_pte)(struct amdgpu_device *adev, | |
| 109 | + struct amdgpu_bo_va_mapping *mapping, | |
| 110 | + uint64_t *flags); | |
| 108 | 111 | }; |
| 109 | 112 | |
| 110 | 113 | struct amdgpu_xgmi { |
| @@ -120,21 +123,52 @@ struct amdgpu_xgmi { · 9 unmodified lines | ||
| 120 | 123 | /* gpu list in the same hive */ |
| 121 | 124 | struct list_head head; |
| 122 | 125 | bool supported; |
| 126 | + struct ras_common_if *ras_if; | |
| 123 | 127 | }; |
| 124 | 128 | |
| 125 | 129 | struct amdgpu_gmc { |
| 130 | + /* FB's physical address in MMIO space (for CPU to | |
| 131 | + * map FB). This is different compared to the agp/ | |
| 132 | + * gart/vram_start/end field as the later is from | |
| 133 | + * GPU's view and aper_base is from CPU's view. | |
| 134 | + */ | |
| 126 | 135 | resource_size_t aper_size; |
| 127 | 136 | resource_size_t aper_base; |
| 128 | 137 | /* for some chips with <= 32MB we need to lie |
| 129 | 138 | * about vram size near mc fb location */ |
| 130 | 139 | u64 mc_vram_size; |
| 131 | 140 | u64 visible_vram_size; |
| 141 | + /* AGP aperture start and end in MC address space | |
| 142 | + * Driver find a hole in the MC address space | |
| 143 | + * to place AGP by setting MC_VM_AGP_BOT/TOP registers | |
| 144 | + * Under VMID0, logical address == MC address. AGP | |
| 145 | + * aperture maps to physical bus or IOVA addressed. | |
| 146 | + * AGP aperture is used to simulate FB in ZFB case. | |
| 147 | + * AGP aperture is also used for page table in system | |
| 148 | + * memory (mainly for APU). | |
| 149 | + * | |
| 150 | + */ | |
| 132 | 151 | u64 agp_size; |
| 133 | 152 | u64 agp_start; |
| 134 | 153 | u64 agp_end; |
| 154 | + /* GART aperture start and end in MC address space | |
| 155 | + * Driver find a hole in the MC address space | |
| 156 | + * to place GART by setting VM_CONTEXT0_PAGE_TABLE_START/END_ADDR | |
| 157 | + * registers | |
| 158 | + * Under VMID0, logical address inside GART aperture will | |
| 159 | + * be translated through gpuvm gart page table to access | |
| 160 | + * paged system memory | |
| 161 | + */ | |
| 135 | 162 | u64 gart_size; |
| 136 | 163 | u64 gart_start; |
| 137 | 164 | u64 gart_end; |
| 165 | + /* Frame buffer aperture of this GPU device. Different from | |
| 166 | + * fb_start (see below), this only covers the local GPU device. | |
| 167 | + * Driver get fb_start from MC_VM_FB_LOCATION_BASE (set by vbios) | |
| 168 | + * and calculate vram_start of this local device by adding an | |
| 169 | + * offset inside the XGMI hive. | |
| 170 | + * Under VMID0, logical address == MC address | |
| 171 | + */ | |
| 138 | 172 | u64 vram_start; |
| 139 | 173 | u64 vram_end; |
| 140 | 174 | /* FB region , it's same as local vram region in single GPU, in XGMI |
| @@ -153,6 +187,7 @@ struct amdgpu_gmc { · 12 unmodified lines | ||
| 153 | 187 | uint32_t fw_version; |
| 154 | 188 | struct amdgpu_irq_src vm_fault; |
| 155 | 189 | uint32_t vram_type; |
| 190 | + uint8_t vram_vendor; | |
| 156 | 191 | uint32_t srbm_soft_reset; |
| 157 | 192 | bool prt_warning; |
| 158 | 193 | uint64_t stolen_size; |
| @@ -177,15 +212,14 @@ struct amdgpu_gmc { · 18 unmodified lines | ||
| 177 | 212 | |
| 178 | 213 | struct amdgpu_xgmi xgmi; |
| 179 | 214 | struct amdgpu_irq_src ecc_irq; |
| 180 | − struct ras_common_if *umc_ras_if; | |
| 181 | − struct ras_common_if *mmhub_ras_if; | |
| 182 | 215 | }; |
| 183 | 216 | |
| 184 | 217 | #define amdgpu_gmc_flush_gpu_tlb(adev, vmid, vmhub, type) ((adev)->gmc.gmc_funcs->flush_gpu_tlb((adev), (vmid), (vmhub), (type))) |
| 185 | 218 | #define amdgpu_gmc_emit_flush_gpu_tlb(r, vmid, addr) (r)->adev->gmc.gmc_funcs->emit_flush_gpu_tlb((r), (vmid), (addr)) |
| 186 | 219 | #define amdgpu_gmc_emit_pasid_mapping(r, vmid, pasid) (r)->adev->gmc.gmc_funcs->emit_pasid_mapping((r), (vmid), (pasid)) |
| 220 | +#define amdgpu_gmc_map_mtype(adev, flags) (adev)->gmc.gmc_funcs->map_mtype((adev),(flags)) | |
| 187 | 221 | #define amdgpu_gmc_get_vm_pde(adev, level, dst, flags) (adev)->gmc.gmc_funcs->get_vm_pde((adev), (level), (dst), (flags)) |
| 188 | −#define amdgpu_gmc_get_pte_flags(adev, flags) (adev)->gmc.gmc_funcs->get_vm_pte_flags((adev),(flags)) | |
| 222 | +#define amdgpu_gmc_get_vm_pte(adev, mapping, flags) (adev)->gmc.gmc_funcs->get_vm_pte((adev), (mapping), (flags)) | |
| 189 | 223 | |
| 190 | 224 | /** |
| 191 | 225 | * amdgpu_gmc_vram_full_visible - Check if full VRAM is visible through the BAR |
| @@ -230,5 +264,7 @@ void amdgpu_gmc_agp_location(struct amdgpu_device *adev, · 38 unmodified lines | ||
| 230 | 264 | struct amdgpu_gmc *mc); |
| 231 | 265 | bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, uint64_t addr, |
| 232 | 266 | uint16_t pasid, uint64_t timestamp); |
| 267 | +int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev); | |
| 268 | +void amdgpu_gmc_ras_fini(struct amdgpu_device *adev); | |
| 233 | 269 | |
| 234 | 270 | #endif |
| @@ -282,7 +282,7 @@ static int amdgpu_vmid_grab_reserved(struct amdgpu_vm *vm, · 281 unmodified lines | ||
| 282 | 282 | !dma_fence_is_later(updates, (*id)->flushed_updates)) |
| 283 | 283 | updates = NULL; |
| 284 | 284 | |
| 285 | − if ((*id)->owner != vm->entity.fence_context || | |
| 285 | + if ((*id)->owner != vm->direct.fence_context || | |
| 286 | 286 | job->vm_pd_addr != (*id)->pd_gpu_addr || |
| 287 | 287 | updates || !(*id)->last_flush || |
| 288 | 288 | ((*id)->last_flush->context != fence_context && |
| @@ -349,7 +349,7 @@ static int amdgpu_vmid_grab_used(struct amdgpu_vm *vm, · 60 unmodified lines | ||
| 349 | 349 | struct dma_fence *flushed; |
| 350 | 350 | |
| 351 | 351 | /* Check all the prerequisites to using this VMID */ |
| 352 | − if ((*id)->owner != vm->entity.fence_context) | |
| 352 | + if ((*id)->owner != vm->direct.fence_context) | |
| 353 | 353 | continue; |
| 354 | 354 | |
| 355 | 355 | if ((*id)->pd_gpu_addr != job->vm_pd_addr) |
| @@ -449,7 +449,7 @@ int amdgpu_vmid_grab(struct amdgpu_vm *vm, struct amdgpu_ring *ring, · 93 unmodified lines | ||
| 449 | 449 | } |
| 450 | 450 | |
| 451 | 451 | id->pd_gpu_addr = job->vm_pd_addr; |
| 452 | − id->owner = vm->entity.fence_context; | |
| 452 | + id->owner = vm->direct.fence_context; | |
| 453 | 453 | |
| 454 | 454 | if (job->vm_needs_flush) { |
| 455 | 455 | dma_fence_put(id->last_flush); |
| @@ -87,10 +87,13 @@ static void amdgpu_hotplug_work_func(struct work_struct *work) · 86 unmodified lines | ||
| 87 | 87 | struct drm_device *dev = adev->ddev; |
| 88 | 88 | struct drm_mode_config *mode_config = &dev->mode_config; |
| 89 | 89 | struct drm_connector *connector; |
| 90 | + struct drm_connector_list_iter iter; | |
| 90 | 91 | |
| 91 | 92 | mutex_lock(&mode_config->mutex); |
| 92 | − list_for_each_entry(connector, &mode_config->connector_list, head) | |
| 93 | + drm_connector_list_iter_begin(dev, &iter); | |
| 94 | + drm_for_each_connector_iter(connector, &iter) | |
| 93 | 95 | amdgpu_connector_hotplug(connector); |
| 96 | + drm_connector_list_iter_end(&iter); | |
| 94 | 97 | mutex_unlock(&mode_config->mutex); |
| 95 | 98 | /* Just fire off a uevent and let userspace tell us what to do */ |
| 96 | 99 | drm_helper_hpd_irq_event(dev); |
| @@ -153,6 +156,20 @@ irqreturn_t amdgpu_irq_handler(int irq, void *arg) · 56 unmodified lines | ||
| 153 | 156 | ret = amdgpu_ih_process(adev, &adev->irq.ih); |
| 154 | 157 | if (ret == IRQ_HANDLED) |
| 155 | 158 | pm_runtime_mark_last_busy(dev->dev); |
| 159 | + | |
| 160 | + /* For the hardware that cannot enable bif ring for both ras_controller_irq | |
| 161 | + * and ras_err_evnet_athub_irq ih cookies, the driver has to poll status | |
| 162 | + * register to check whether the interrupt is triggered or not, and properly | |
| 163 | + * ack the interrupt if it is there | |
| 164 | + */ | |
| 165 | + if (adev->nbio.funcs && | |
| 166 | + adev->nbio.funcs->handle_ras_controller_intr_no_bifring) | |
| 167 | + adev->nbio.funcs->handle_ras_controller_intr_no_bifring(adev); | |
| 168 | + | |
| 169 | + if (adev->nbio.funcs && | |
| 170 | + adev->nbio.funcs->handle_ras_err_event_athub_intr_no_bifring) | |
| 171 | + adev->nbio.funcs->handle_ras_err_event_athub_intr_no_bifring(adev); | |
| 172 | + | |
| 156 | 173 | return ret; |
| 157 | 174 | } |
| 158 | 175 | |
| @@ -228,10 +245,19 @@ int amdgpu_irq_init(struct amdgpu_device *adev) · 69 unmodified lines | ||
| 228 | 245 | adev->irq.msi_enabled = false; |
| 229 | 246 | |
| 230 | 247 | if (amdgpu_msi_ok(adev)) { |
| 231 | − int ret = pci_enable_msi(adev->pdev); | |
| 232 | − if (!ret) { | |
| 248 | + int nvec = pci_msix_vec_count(adev->pdev); | |
| 249 | + unsigned int flags; | |
| 250 | + | |
| 251 | + if (nvec <= 0) { | |
| 252 | + flags = PCI_IRQ_MSI; | |
| 253 | + } else { | |
| 254 | + flags = PCI_IRQ_MSI | PCI_IRQ_MSIX; | |
| 255 | + } | |
| 256 | + /* we only need one vector */ | |
| 257 | + nvec = pci_alloc_irq_vectors(adev->pdev, 1, 1, flags); | |
| 258 | + if (nvec > 0) { | |
| 233 | 259 | adev->irq.msi_enabled = true; |
| 234 | − dev_dbg(adev->dev, "amdgpu: using MSI.\n"); | |
| 260 | + dev_dbg(adev->dev, "amdgpu: using MSI/MSI-X.\n"); | |
| 235 | 261 | } |
| 236 | 262 | } |
| 237 | 263 | |
| @@ -254,7 +280,8 @@ int amdgpu_irq_init(struct amdgpu_device *adev) · 16 unmodified lines | ||
| 254 | 280 | INIT_WORK(&adev->irq.ih2_work, amdgpu_irq_handle_ih2); |
| 255 | 281 | |
| 256 | 282 | adev->irq.installed = true; |
| 257 | − r = drm_irq_install(adev->ddev, adev->ddev->pdev->irq); | |
| 283 | + /* Use vector 0 for MSI-X */ | |
| 284 | + r = drm_irq_install(adev->ddev, pci_irq_vector(adev->pdev, 0)); | |
| 258 | 285 | if (r) { |
| 259 | 286 | adev->irq.installed = false; |
| 260 | 287 | if (!amdgpu_device_has_dc_support(adev)) |
| @@ -284,7 +311,7 @@ void amdgpu_irq_fini(struct amdgpu_device *adev) · 23 unmodified lines | ||
| 284 | 311 | drm_irq_uninstall(adev->ddev); |
| 285 | 312 | adev->irq.installed = false; |
| 286 | 313 | if (adev->irq.msi_enabled) |
| 287 | − pci_disable_msi(adev->pdev); | |
| 314 | + pci_free_irq_vectors(adev->pdev); | |
| 288 | 315 | if (!amdgpu_device_has_dc_support(adev)) |
| 289 | 316 | flush_work(&adev->hotplug_work); |
| 290 | 317 | } |
| @@ -369,7 +396,7 @@ int amdgpu_irq_add_id(struct amdgpu_device *adev, · 78 unmodified lines | ||
| 369 | 396 | * amdgpu_irq_dispatch - dispatch IRQ to IP blocks |
| 370 | 397 | * |
| 371 | 398 | * @adev: amdgpu device pointer |
| 372 | − * @entry: interrupt vector pointer | |
| 399 | + * @ih: interrupt ring instance | |
| 373 | 400 | * |
| 374 | 401 | * Dispatches IRQ to IP blocks. |
| 375 | 402 | */ |
| @@ -218,7 +218,7 @@ static struct dma_fence *amdgpu_job_run(struct drm_sched_job *sched_job) · 217 unmodified lines | ||
| 218 | 218 | struct amdgpu_ring *ring = to_amdgpu_ring(sched_job->sched); |
| 219 | 219 | struct dma_fence *fence = NULL, *finished; |
| 220 | 220 | struct amdgpu_job *job; |
| 221 | − int r; | |
| 221 | + int r = 0; | |
| 222 | 222 | |
| 223 | 223 | job = to_amdgpu_job(sched_job); |
| 224 | 224 | finished = &job->base.s_fence->finished; |
| @@ -243,9 +243,49 @@ static struct dma_fence *amdgpu_job_run(struct drm_sched_job *sched_job) · 18 unmodified lines | ||
| 243 | 243 | job->fence = dma_fence_get(fence); |
| 244 | 244 | |
| 245 | 245 | amdgpu_job_free_resources(job); |
| 246 | + | |
| 247 | + fence = r ? ERR_PTR(r) : fence; | |
| 246 | 248 | return fence; |
| 247 | 249 | } |
| 248 | 250 | |
| 251 | +#define to_drm_sched_job(sched_job) \ | |
| 252 | + container_of((sched_job), struct drm_sched_job, queue_node) | |
| 253 | + | |
| 254 | +void amdgpu_job_stop_all_jobs_on_sched(struct drm_gpu_scheduler *sched) | |
| 255 | +{ | |
| 256 | + struct drm_sched_job *s_job; | |
| 257 | + struct drm_sched_entity *s_entity = NULL; | |
| 258 | + int i; | |
| 259 | + | |
| 260 | + /* Signal all jobs not yet scheduled */ | |
| 261 | + for (i = DRM_SCHED_PRIORITY_MAX - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { | |
| 262 | + struct drm_sched_rq *rq = &sched->sched_rq[i]; | |
| 263 | + | |
| 264 | + if (!rq) | |
| 265 | + continue; | |
| 266 | + | |
| 267 | + spin_lock(&rq->lock); | |
| 268 | + list_for_each_entry(s_entity, &rq->entities, list) { | |
| 269 | + while ((s_job = to_drm_sched_job(spsc_queue_pop(&s_entity->job_queue)))) { | |
| 270 | + struct drm_sched_fence *s_fence = s_job->s_fence; | |
| 271 | + | |
| 272 | + dma_fence_signal(&s_fence->scheduled); | |
| 273 | + dma_fence_set_error(&s_fence->finished, -EHWPOISON); | |
| 274 | + dma_fence_signal(&s_fence->finished); | |
| 275 | + } | |
| 276 | + } | |
| 277 | + spin_unlock(&rq->lock); | |
| 278 | + } | |
| 279 | + | |
| 280 | + /* Signal all jobs already scheduled to HW */ | |
| 281 | + list_for_each_entry(s_job, &sched->ring_mirror_list, node) { | |
| 282 | + struct drm_sched_fence *s_fence = s_job->s_fence; | |
| 283 | + | |
| 284 | + dma_fence_set_error(&s_fence->finished, -EHWPOISON); | |
| 285 | + dma_fence_signal(&s_fence->finished); | |
| 286 | + } | |
| 287 | +} | |
| 288 | + | |
| 249 | 289 | const struct drm_sched_backend_ops amdgpu_sched_ops = { |
| 250 | 290 | .dependency = amdgpu_job_dependency, |
| 251 | 291 | .run_job = amdgpu_job_run, |
| @@ -76,4 +76,7 @@ int amdgpu_job_submit(struct amdgpu_job *job, struct drm_sched_entity *entity, · 75 unmodified lines | ||
| 76 | 76 | void *owner, struct dma_fence **f); |
| 77 | 77 | int amdgpu_job_submit_direct(struct amdgpu_job *job, struct amdgpu_ring *ring, |
| 78 | 78 | struct dma_fence **fence); |
| 79 | + | |
| 80 | +void amdgpu_job_stop_all_jobs_on_sched(struct drm_gpu_scheduler *sched); | |
| 81 | + | |
| 79 | 82 | #endif |
| @@ -144,41 +144,6 @@ int amdgpu_driver_load_kms(struct drm_device *dev, unsigned long flags) · 143 unmodified lines | ||
| 144 | 144 | struct amdgpu_device *adev; |
| 145 | 145 | int r, acpi_status; |
| 146 | 146 | |
| 147 | −#ifdef CONFIG_DRM_AMDGPU_SI | |
| 148 | − if (!amdgpu_si_support) { | |
| 149 | − switch (flags & AMD_ASIC_MASK) { | |
| 150 | − case CHIP_TAHITI: | |
| 151 | − case CHIP_PITCAIRN: | |
| 152 | − case CHIP_VERDE: | |
| 153 | − case CHIP_OLAND: | |
| 154 | − case CHIP_HAINAN: | |
| 155 | − dev_info(dev->dev, | |
| 156 | − "SI support provided by radeon.\n"); | |
| 157 | − dev_info(dev->dev, | |
| 158 | − "Use radeon.si_support=0 amdgpu.si_support=1 to override.\n" | |
| 159 | − ); | |
| 160 | − return -ENODEV; | |
| 161 | − } | |
| 162 | − } | |
| 163 | −#endif | |
| 164 | −#ifdef CONFIG_DRM_AMDGPU_CIK | |
| 165 | − if (!amdgpu_cik_support) { | |
| 166 | − switch (flags & AMD_ASIC_MASK) { | |
| 167 | − case CHIP_KAVERI: | |
| 168 | − case CHIP_BONAIRE: | |
| 169 | − case CHIP_HAWAII: | |
| 170 | − case CHIP_KABINI: | |
| 171 | − case CHIP_MULLINS: | |
| 172 | − dev_info(dev->dev, | |
| 173 | − "CIK support provided by radeon.\n"); | |
| 174 | − dev_info(dev->dev, | |
| 175 | − "Use radeon.cik_support=0 amdgpu.cik_support=1 to override.\n" | |
| 176 | − ); | |
| 177 | − return -ENODEV; | |
| 178 | − } | |
| 179 | − } | |
| 180 | −#endif | |
| 181 | − | |
| 182 | 147 | adev = kzalloc(sizeof(struct amdgpu_device), GFP_KERNEL); |
| 183 | 148 | if (adev == NULL) { |
| 184 | 149 | return -ENOMEM; |
| @@ -225,7 +190,6 @@ int amdgpu_driver_load_kms(struct drm_device *dev, unsigned long flags) · 40 unmodified lines | ||
| 225 | 190 | pm_runtime_put_autosuspend(dev->dev); |
| 226 | 191 | } |
| 227 | 192 | |
| 228 | − amdgpu_register_gpu_instance(adev); | |
| 229 | 193 | out: |
| 230 | 194 | if (r) { |
| 231 | 195 | /* balance pm_runtime_get_sync in amdgpu_driver_unload_kms */ |
| @@ -619,9 +583,12 @@ static int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file · 387 unmodified lines | ||
| 619 | 583 | struct drm_amdgpu_info_vram_gtt vram_gtt; |
| 620 | 584 | |
| 621 | 585 | vram_gtt.vram_size = adev->gmc.real_vram_size - |
| 622 | − atomic64_read(&adev->vram_pin_size); | |
| 623 | − vram_gtt.vram_cpu_accessible_size = adev->gmc.visible_vram_size - | |
| 624 | − atomic64_read(&adev->visible_pin_size); | |
| 586 | + atomic64_read(&adev->vram_pin_size) - | |
| 587 | + AMDGPU_VM_RESERVED_VRAM; | |
| 588 | + vram_gtt.vram_cpu_accessible_size = | |
| 589 | + min(adev->gmc.visible_vram_size - | |
| 590 | + atomic64_read(&adev->visible_pin_size), | |
| 591 | + vram_gtt.vram_size); | |
| 625 | 592 | vram_gtt.gtt_size = adev->mman.bdev.man[TTM_PL_TT].size; |
| 626 | 593 | vram_gtt.gtt_size *= PAGE_SIZE; |
| 627 | 594 | vram_gtt.gtt_size -= atomic64_read(&adev->gart_pin_size); |
| @@ -634,15 +601,18 @@ static int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file · 6 unmodified lines | ||
| 634 | 601 | memset(&mem, 0, sizeof(mem)); |
| 635 | 602 | mem.vram.total_heap_size = adev->gmc.real_vram_size; |
| 636 | 603 | mem.vram.usable_heap_size = adev->gmc.real_vram_size - |
| 637 | − atomic64_read(&adev->vram_pin_size); | |
| 604 | + atomic64_read(&adev->vram_pin_size) - | |
| 605 | + AMDGPU_VM_RESERVED_VRAM; | |
| 638 | 606 | mem.vram.heap_usage = |
| 639 | 607 | amdgpu_vram_mgr_usage(&adev->mman.bdev.man[TTM_PL_VRAM]); |
| 640 | 608 | mem.vram.max_allocation = mem.vram.usable_heap_size * 3 / 4; |
| 641 | 609 | |
| 642 | 610 | mem.cpu_accessible_vram.total_heap_size = |
| 643 | 611 | adev->gmc.visible_vram_size; |
| 644 | − mem.cpu_accessible_vram.usable_heap_size = adev->gmc.visible_vram_size - | |
| 645 | − atomic64_read(&adev->visible_pin_size); | |
| 612 | + mem.cpu_accessible_vram.usable_heap_size = | |
| 613 | + min(adev->gmc.visible_vram_size - | |
| 614 | + atomic64_read(&adev->visible_pin_size), | |
| 615 | + mem.vram.usable_heap_size); | |
| 646 | 616 | mem.cpu_accessible_vram.heap_usage = |
| 647 | 617 | amdgpu_vram_mgr_vis_usage(&adev->mman.bdev.man[TTM_PL_VRAM]); |
| 648 | 618 | mem.cpu_accessible_vram.max_allocation = |
| @@ -764,17 +734,6 @@ static int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file · 115 unmodified lines | ||
| 764 | 734 | dev_info.vce_harvest_config = adev->vce.harvest_config; |
| 765 | 735 | dev_info.gc_double_offchip_lds_buf = |
| 766 | 736 | adev->gfx.config.double_offchip_lds_buf; |
| 767 | − | |
| 768 | − if (amdgpu_ngg) { | |
| 769 | − dev_info.prim_buf_gpu_addr = adev->gfx.ngg.buf[NGG_PRIM].gpu_addr; | |
| 770 | − dev_info.prim_buf_size = adev->gfx.ngg.buf[NGG_PRIM].size; | |
| 771 | − dev_info.pos_buf_gpu_addr = adev->gfx.ngg.buf[NGG_POS].gpu_addr; | |
| 772 | − dev_info.pos_buf_size = adev->gfx.ngg.buf[NGG_POS].size; | |
| 773 | − dev_info.cntl_sb_buf_gpu_addr = adev->gfx.ngg.buf[NGG_CNTL].gpu_addr; | |
| 774 | − dev_info.cntl_sb_buf_size = adev->gfx.ngg.buf[NGG_CNTL].size; | |
| 775 | − dev_info.param_buf_gpu_addr = adev->gfx.ngg.buf[NGG_PARAM].gpu_addr; | |
| 776 | − dev_info.param_buf_size = adev->gfx.ngg.buf[NGG_PARAM].size; | |
| 777 | − } | |
| 778 | 737 | dev_info.wave_front_size = adev->gfx.cu_info.wave_front_size; |
| 779 | 738 | dev_info.num_shader_visible_vgprs = adev->gfx.config.max_gprs; |
| 780 | 739 | dev_info.num_cu_per_sh = adev->gfx.config.max_cu_per_sh; |
| @@ -1003,6 +962,12 @@ int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv) · 222 unmodified lines | ||
| 1003 | 962 | /* Ensure IB tests are run on ring */ |
| 1004 | 963 | flush_delayed_work(&adev->delayed_init_work); |
| 1005 | 964 | |
| 965 | + | |
| 966 | + if (amdgpu_ras_intr_triggered()) { | |
| 967 | + DRM_ERROR("RAS Intr triggered, device disabled!!"); | |
| 968 | + return -EHWPOISON; | |
| 969 | + } | |
| 970 | + | |
| 1006 | 971 | file_priv->driver_priv = NULL; |
| 1007 | 972 | |
| 1008 | 973 | r = pm_runtime_get_sync(dev->dev); |
| @@ -0,0 +1,70 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2019 Advanced Micro Devices, Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + * | |
| 22 | + */ | |
| 23 | + | |
| 24 | +#include "amdgpu.h" | |
| 25 | +#include "amdgpu_ras.h" | |
| 26 | + | |
| 27 | +int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev) | |
| 28 | +{ | |
| 29 | + int r; | |
| 30 | + struct ras_ih_if ih_info = { | |
| 31 | + .cb = NULL, | |
| 32 | + }; | |
| 33 | + struct ras_fs_if fs_info = { | |
| 34 | + .sysfs_name = "mmhub_err_count", | |
| 35 | + .debugfs_name = "mmhub_err_inject", | |
| 36 | + }; | |
| 37 | + | |
| 38 | + if (!adev->mmhub.ras_if) { | |
| 39 | + adev->mmhub.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 40 | + if (!adev->mmhub.ras_if) | |
| 41 | + return -ENOMEM; | |
| 42 | + adev->mmhub.ras_if->block = AMDGPU_RAS_BLOCK__MMHUB; | |
| 43 | + adev->mmhub.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 44 | + adev->mmhub.ras_if->sub_block_index = 0; | |
| 45 | + strcpy(adev->mmhub.ras_if->name, "mmhub"); | |
| 46 | + } | |
| 47 | + ih_info.head = fs_info.head = *adev->mmhub.ras_if; | |
| 48 | + r = amdgpu_ras_late_init(adev, adev->mmhub.ras_if, | |
| 49 | + &fs_info, &ih_info); | |
| 50 | + if (r || !amdgpu_ras_is_supported(adev, adev->mmhub.ras_if->block)) { | |
| 51 | + kfree(adev->mmhub.ras_if); | |
| 52 | + adev->mmhub.ras_if = NULL; | |
| 53 | + } | |
| 54 | + | |
| 55 | + return r; | |
| 56 | +} | |
| 57 | + | |
| 58 | +void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev) | |
| 59 | +{ | |
| 60 | + if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__MMHUB) && | |
| 61 | + adev->mmhub.ras_if) { | |
| 62 | + struct ras_common_if *ras_if = adev->mmhub.ras_if; | |
| 63 | + struct ras_ih_if ih_info = { | |
| 64 | + .cb = NULL, | |
| 65 | + }; | |
| 66 | + | |
| 67 | + amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 68 | + kfree(ras_if); | |
| 69 | + } | |
| 70 | +} | |
| @@ -23,9 +23,17 @@ · 22 unmodified lines | ||
| 23 | 23 | |
| 24 | 24 | struct amdgpu_mmhub_funcs { |
| 25 | 25 | void (*ras_init)(struct amdgpu_device *adev); |
| 26 | + int (*ras_late_init)(struct amdgpu_device *adev); | |
| 26 | 27 | void (*query_ras_error_count)(struct amdgpu_device *adev, |
| 27 | 28 | void *ras_error_status); |
| 28 | 29 | }; |
| 29 | 30 | |
| 31 | +struct amdgpu_mmhub { | |
| 32 | + struct ras_common_if *ras_if; | |
| 33 | + const struct amdgpu_mmhub_funcs *funcs; | |
| 34 | +}; | |
| 35 | + | |
| 36 | +int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev); | |
| 37 | +void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev); | |
| 30 | 38 | #endif |
| 31 | 39 | |
| @@ -136,6 +136,7 @@ void amdgpu_mn_unlock(struct amdgpu_mn *mn) · 135 unmodified lines | ||
| 136 | 136 | * amdgpu_mn_read_lock - take the read side lock for this notifier |
| 137 | 137 | * |
| 138 | 138 | * @amn: our notifier |
| 139 | + * @blockable: is the notifier blockable | |
| 139 | 140 | */ |
| 140 | 141 | static int amdgpu_mn_read_lock(struct amdgpu_mn *amn, bool blockable) |
| 141 | 142 | { |
| @@ -0,0 +1,84 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright (C) 2019 Advanced Micro Devices, Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included | |
| 12 | + * in all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS | |
| 15 | + * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN | |
| 18 | + * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN | |
| 19 | + * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. | |
| 20 | + */ | |
| 21 | + | |
| 22 | +#include "amdgpu.h" | |
| 23 | +#include "amdgpu_ras.h" | |
| 24 | + | |
| 25 | +int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev) | |
| 26 | +{ | |
| 27 | + int r; | |
| 28 | + struct ras_ih_if ih_info = { | |
| 29 | + .cb = NULL, | |
| 30 | + }; | |
| 31 | + struct ras_fs_if fs_info = { | |
| 32 | + .sysfs_name = "pcie_bif_err_count", | |
| 33 | + .debugfs_name = "pcie_bif_err_inject", | |
| 34 | + }; | |
| 35 | + | |
| 36 | + if (!adev->nbio.ras_if) { | |
| 37 | + adev->nbio.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 38 | + if (!adev->nbio.ras_if) | |
| 39 | + return -ENOMEM; | |
| 40 | + adev->nbio.ras_if->block = AMDGPU_RAS_BLOCK__PCIE_BIF; | |
| 41 | + adev->nbio.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 42 | + adev->nbio.ras_if->sub_block_index = 0; | |
| 43 | + strcpy(adev->nbio.ras_if->name, "pcie_bif"); | |
| 44 | + } | |
| 45 | + ih_info.head = fs_info.head = *adev->nbio.ras_if; | |
| 46 | + r = amdgpu_ras_late_init(adev, adev->nbio.ras_if, | |
| 47 | + &fs_info, &ih_info); | |
| 48 | + if (r) | |
| 49 | + goto free; | |
| 50 | + | |
| 51 | + if (amdgpu_ras_is_supported(adev, adev->nbio.ras_if->block)) { | |
| 52 | + r = amdgpu_irq_get(adev, &adev->nbio.ras_controller_irq, 0); | |
| 53 | + if (r) | |
| 54 | + goto late_fini; | |
| 55 | + r = amdgpu_irq_get(adev, &adev->nbio.ras_err_event_athub_irq, 0); | |
| 56 | + if (r) | |
| 57 | + goto late_fini; | |
| 58 | + } else { | |
| 59 | + r = 0; | |
| 60 | + goto free; | |
| 61 | + } | |
| 62 | + | |
| 63 | + return 0; | |
| 64 | +late_fini: | |
| 65 | + amdgpu_ras_late_fini(adev, adev->nbio.ras_if, &ih_info); | |
| 66 | +free: | |
| 67 | + kfree(adev->nbio.ras_if); | |
| 68 | + adev->nbio.ras_if = NULL; | |
| 69 | + return r; | |
| 70 | +} | |
| 71 | + | |
| 72 | +void amdgpu_nbio_ras_fini(struct amdgpu_device *adev) | |
| 73 | +{ | |
| 74 | + if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF) && | |
| 75 | + adev->nbio.ras_if) { | |
| 76 | + struct ras_common_if *ras_if = adev->nbio.ras_if; | |
| 77 | + struct ras_ih_if ih_info = { | |
| 78 | + .cb = NULL, | |
| 79 | + }; | |
| 80 | + | |
| 81 | + amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 82 | + kfree(ras_if); | |
| 83 | + } | |
| 84 | +} | |
| @@ -0,0 +1,101 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2019 Advanced Micro Devices, Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + * | |
| 22 | + */ | |
| 23 | +#ifndef __AMDGPU_NBIO_H__ | |
| 24 | +#define __AMDGPU_NBIO_H__ | |
| 25 | + | |
| 26 | +/* | |
| 27 | + * amdgpu nbio functions | |
| 28 | + */ | |
| 29 | +struct nbio_hdp_flush_reg { | |
| 30 | + u32 ref_and_mask_cp0; | |
| 31 | + u32 ref_and_mask_cp1; | |
| 32 | + u32 ref_and_mask_cp2; | |
| 33 | + u32 ref_and_mask_cp3; | |
| 34 | + u32 ref_and_mask_cp4; | |
| 35 | + u32 ref_and_mask_cp5; | |
| 36 | + u32 ref_and_mask_cp6; | |
| 37 | + u32 ref_and_mask_cp7; | |
| 38 | + u32 ref_and_mask_cp8; | |
| 39 | + u32 ref_and_mask_cp9; | |
| 40 | + u32 ref_and_mask_sdma0; | |
| 41 | + u32 ref_and_mask_sdma1; | |
| 42 | + u32 ref_and_mask_sdma2; | |
| 43 | + u32 ref_and_mask_sdma3; | |
| 44 | + u32 ref_and_mask_sdma4; | |
| 45 | + u32 ref_and_mask_sdma5; | |
| 46 | + u32 ref_and_mask_sdma6; | |
| 47 | + u32 ref_and_mask_sdma7; | |
| 48 | +}; | |
| 49 | + | |
| 50 | +struct amdgpu_nbio_funcs { | |
| 51 | + const struct nbio_hdp_flush_reg *hdp_flush_reg; | |
| 52 | + u32 (*get_hdp_flush_req_offset)(struct amdgpu_device *adev); | |
| 53 | + u32 (*get_hdp_flush_done_offset)(struct amdgpu_device *adev); | |
| 54 | + u32 (*get_pcie_index_offset)(struct amdgpu_device *adev); | |
| 55 | + u32 (*get_pcie_data_offset)(struct amdgpu_device *adev); | |
| 56 | + u32 (*get_rev_id)(struct amdgpu_device *adev); | |
| 57 | + void (*mc_access_enable)(struct amdgpu_device *adev, bool enable); | |
| 58 | + void (*hdp_flush)(struct amdgpu_device *adev, struct amdgpu_ring *ring); | |
| 59 | + u32 (*get_memsize)(struct amdgpu_device *adev); | |
| 60 | + void (*sdma_doorbell_range)(struct amdgpu_device *adev, int instance, | |
| 61 | + bool use_doorbell, int doorbell_index, int doorbell_size); | |
| 62 | + void (*vcn_doorbell_range)(struct amdgpu_device *adev, bool use_doorbell, | |
| 63 | + int doorbell_index, int instance); | |
| 64 | + void (*enable_doorbell_aperture)(struct amdgpu_device *adev, | |
| 65 | + bool enable); | |
| 66 | + void (*enable_doorbell_selfring_aperture)(struct amdgpu_device *adev, | |
| 67 | + bool enable); | |
| 68 | + void (*ih_doorbell_range)(struct amdgpu_device *adev, | |
| 69 | + bool use_doorbell, int doorbell_index); | |
| 70 | + void (*enable_doorbell_interrupt)(struct amdgpu_device *adev, | |
| 71 | + bool enable); | |
| 72 | + void (*update_medium_grain_clock_gating)(struct amdgpu_device *adev, | |
| 73 | + bool enable); | |
| 74 | + void (*update_medium_grain_light_sleep)(struct amdgpu_device *adev, | |
| 75 | + bool enable); | |
| 76 | + void (*get_clockgating_state)(struct amdgpu_device *adev, | |
| 77 | + u32 *flags); | |
| 78 | + void (*ih_control)(struct amdgpu_device *adev); | |
| 79 | + void (*init_registers)(struct amdgpu_device *adev); | |
| 80 | + void (*detect_hw_virt)(struct amdgpu_device *adev); | |
| 81 | + void (*remap_hdp_registers)(struct amdgpu_device *adev); | |
| 82 | + void (*handle_ras_controller_intr_no_bifring)(struct amdgpu_device *adev); | |
| 83 | + void (*handle_ras_err_event_athub_intr_no_bifring)(struct amdgpu_device *adev); | |
| 84 | + int (*init_ras_controller_interrupt)(struct amdgpu_device *adev); | |
| 85 | + int (*init_ras_err_event_athub_interrupt)(struct amdgpu_device *adev); | |
| 86 | + void (*query_ras_error_count)(struct amdgpu_device *adev, | |
| 87 | + void *ras_error_status); | |
| 88 | + int (*ras_late_init)(struct amdgpu_device *adev); | |
| 89 | +}; | |
| 90 | + | |
| 91 | +struct amdgpu_nbio { | |
| 92 | + const struct nbio_hdp_flush_reg *hdp_flush_reg; | |
| 93 | + struct amdgpu_irq_src ras_controller_irq; | |
| 94 | + struct amdgpu_irq_src ras_err_event_athub_irq; | |
| 95 | + struct ras_common_if *ras_if; | |
| 96 | + const struct amdgpu_nbio_funcs *funcs; | |
| 97 | +}; | |
| 98 | + | |
| 99 | +int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev); | |
| 100 | +void amdgpu_nbio_ras_fini(struct amdgpu_device *adev); | |
| 101 | +#endif | |
| @@ -342,6 +342,70 @@ int amdgpu_bo_create_kernel(struct amdgpu_device *adev, · 341 unmodified lines | ||
| 342 | 342 | return 0; |
| 343 | 343 | } |
| 344 | 344 | |
| 345 | +/** | |
| 346 | + * amdgpu_bo_create_kernel_at - create BO for kernel use at specific location | |
| 347 | + * | |
| 348 | + * @adev: amdgpu device object | |
| 349 | + * @offset: offset of the BO | |
| 350 | + * @size: size of the BO | |
| 351 | + * @domain: where to place it | |
| 352 | + * @bo_ptr: used to initialize BOs in structures | |
| 353 | + * @cpu_addr: optional CPU address mapping | |
| 354 | + * | |
| 355 | + * Creates a kernel BO at a specific offset in the address space of the domain. | |
| 356 | + * | |
| 357 | + * Returns: | |
| 358 | + * 0 on success, negative error code otherwise. | |
| 359 | + */ | |
| 360 | +int amdgpu_bo_create_kernel_at(struct amdgpu_device *adev, | |
| 361 | + uint64_t offset, uint64_t size, uint32_t domain, | |
| 362 | + struct amdgpu_bo **bo_ptr, void **cpu_addr) | |
| 363 | +{ | |
| 364 | + struct ttm_operation_ctx ctx = { false, false }; | |
| 365 | + unsigned int i; | |
| 366 | + int r; | |
| 367 | + | |
| 368 | + offset &= PAGE_MASK; | |
| 369 | + size = ALIGN(size, PAGE_SIZE); | |
| 370 | + | |
| 371 | + r = amdgpu_bo_create_reserved(adev, size, PAGE_SIZE, domain, bo_ptr, | |
| 372 | + NULL, cpu_addr); | |
| 373 | + if (r) | |
| 374 | + return r; | |
| 375 | + | |
| 376 | + /* | |
| 377 | + * Remove the original mem node and create a new one at the request | |
| 378 | + * position. | |
| 379 | + */ | |
| 380 | + if (cpu_addr) | |
| 381 | + amdgpu_bo_kunmap(*bo_ptr); | |
| 382 | + | |
| 383 | + ttm_bo_mem_put(&(*bo_ptr)->tbo, &(*bo_ptr)->tbo.mem); | |
| 384 | + | |
| 385 | + for (i = 0; i < (*bo_ptr)->placement.num_placement; ++i) { | |
| 386 | + (*bo_ptr)->placements[i].fpfn = offset >> PAGE_SHIFT; | |
| 387 | + (*bo_ptr)->placements[i].lpfn = (offset + size) >> PAGE_SHIFT; | |
| 388 | + } | |
| 389 | + r = ttm_bo_mem_space(&(*bo_ptr)->tbo, &(*bo_ptr)->placement, | |
| 390 | + &(*bo_ptr)->tbo.mem, &ctx); | |
| 391 | + if (r) | |
| 392 | + goto error; | |
| 393 | + | |
| 394 | + if (cpu_addr) { | |
| 395 | + r = amdgpu_bo_kmap(*bo_ptr, cpu_addr); | |
| 396 | + if (r) | |
| 397 | + goto error; | |
| 398 | + } | |
| 399 | + | |
| 400 | + amdgpu_bo_unreserve(*bo_ptr); | |
| 401 | + return 0; | |
| 402 | + | |
| 403 | +error: | |
| 404 | + amdgpu_bo_unreserve(*bo_ptr); | |
| 405 | + amdgpu_bo_unref(bo_ptr); | |
| 406 | + return r; | |
| 407 | +} | |
| 408 | + | |
| 345 | 409 | /** |
| 346 | 410 | * amdgpu_bo_free_kernel - free BO for kernel use |
| 347 | 411 | * |
| @@ -451,9 +515,10 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, · 103 unmodified lines | ||
| 451 | 515 | { |
| 452 | 516 | struct ttm_operation_ctx ctx = { |
| 453 | 517 | .interruptible = (bp->type != ttm_bo_type_kernel), |
| 454 | − .no_wait_gpu = false, | |
| 518 | + .no_wait_gpu = bp->no_wait_gpu, | |
| 455 | 519 | .resv = bp->resv, |
| 456 | − .flags = TTM_OPT_FLAG_ALLOW_RES_EVICT | |
| 520 | + .flags = bp->type != ttm_bo_type_kernel ? | |
| 521 | + TTM_OPT_FLAG_ALLOW_RES_EVICT : 0 | |
| 457 | 522 | }; |
| 458 | 523 | struct amdgpu_bo *bo; |
| 459 | 524 | unsigned long page_align, size = bp->size; |
| @@ -1058,7 +1123,10 @@ void amdgpu_bo_fini(struct amdgpu_device *adev) · 598 unmodified lines | ||
| 1058 | 1123 | int amdgpu_bo_fbdev_mmap(struct amdgpu_bo *bo, |
| 1059 | 1124 | struct vm_area_struct *vma) |
| 1060 | 1125 | { |
| 1061 | − return ttm_fbdev_mmap(vma, &bo->tbo); | |
| 1126 | + if (vma->vm_pgoff != 0) | |
| 1127 | + return -EACCES; | |
| 1128 | + | |
| 1129 | + return ttm_bo_mmap_obj(vma, &bo->tbo); | |
| 1062 | 1130 | } |
| 1063 | 1131 | |
| 1064 | 1132 | /** |
| @@ -41,6 +41,7 @@ struct amdgpu_bo_param { · 40 unmodified lines | ||
| 41 | 41 | u32 preferred_domain; |
| 42 | 42 | u64 flags; |
| 43 | 43 | enum ttm_bo_type type; |
| 44 | + bool no_wait_gpu; | |
| 44 | 45 | struct dma_resv *resv; |
| 45 | 46 | }; |
| 46 | 47 | |
| @@ -237,6 +238,9 @@ int amdgpu_bo_create_kernel(struct amdgpu_device *adev, · 190 unmodified lines | ||
| 237 | 238 | unsigned long size, int align, |
| 238 | 239 | u32 domain, struct amdgpu_bo **bo_ptr, |
| 239 | 240 | u64 *gpu_addr, void **cpu_addr); |
| 241 | +int amdgpu_bo_create_kernel_at(struct amdgpu_device *adev, | |
| 242 | + uint64_t offset, uint64_t size, uint32_t domain, | |
| 243 | + struct amdgpu_bo **bo_ptr, void **cpu_addr); | |
| 240 | 244 | void amdgpu_bo_free_kernel(struct amdgpu_bo **bo, u64 *gpu_addr, |
| 241 | 245 | void **cpu_addr); |
| 242 | 246 | int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr); |
| @@ -161,7 +161,7 @@ static ssize_t amdgpu_get_dpm_state(struct device *dev, · 160 unmodified lines | ||
| 161 | 161 | |
| 162 | 162 | if (is_support_sw_smu(adev)) { |
| 163 | 163 | if (adev->smu.ppt_funcs->get_current_power_state) |
| 164 | − pm = amdgpu_smu_get_current_power_state(adev); | |
| 164 | + pm = smu_get_current_power_state(&adev->smu); | |
| 165 | 165 | else |
| 166 | 166 | pm = adev->pm.dpm.user_state; |
| 167 | 167 | } else if (adev->powerplay.pp_funcs->get_current_power_state) { |
| @@ -805,8 +805,7 @@ static ssize_t amdgpu_get_pp_feature_status(struct device *dev, · 637 unmodified lines | ||
| 805 | 805 | } |
| 806 | 806 | |
| 807 | 807 | /** |
| 808 | − * DOC: pp_dpm_sclk pp_dpm_mclk pp_dpm_socclk pp_dpm_fclk pp_dpm_dcefclk | |
| 809 | − * pp_dpm_pcie | |
| 808 | + * DOC: pp_dpm_sclk pp_dpm_mclk pp_dpm_socclk pp_dpm_fclk pp_dpm_dcefclk pp_dpm_pcie | |
| 810 | 809 | * |
| 811 | 810 | * The amdgpu driver provides a sysfs API for adjusting what power levels |
| 812 | 811 | * are enabled for a given power state. The files pp_dpm_sclk, pp_dpm_mclk, |
| @@ -822,9 +821,15 @@ static ssize_t amdgpu_get_pp_feature_status(struct device *dev, · 9 unmodified lines | ||
| 822 | 821 | * |
| 823 | 822 | * To manually adjust these states, first select manual using |
| 824 | 823 | * power_dpm_force_performance_level. |
| 825 | − * Secondly,Enter a new value for each level by inputing a string that | |
| 824 | + * Secondly, enter a new value for each level by inputing a string that | |
| 826 | 825 | * contains " echo xx xx xx > pp_dpm_sclk/mclk/pcie" |
| 827 | − * E.g., echo 4 5 6 to > pp_dpm_sclk will enable sclk levels 4, 5, and 6. | |
| 826 | + * E.g., | |
| 827 | + * | |
| 828 | + * .. code-block:: bash | |
| 829 | + * | |
| 830 | + * echo "4 5 6" > pp_dpm_sclk | |
| 831 | + * | |
| 832 | + * will enable sclk levels 4, 5, and 6. | |
| 828 | 833 | * |
| 829 | 834 | * NOTE: change to the dcefclk max dpm level is not supported now |
| 830 | 835 | */ |
| @@ -902,7 +907,7 @@ static ssize_t amdgpu_set_pp_dpm_sclk(struct device *dev, · 71 unmodified lines | ||
| 902 | 907 | return ret; |
| 903 | 908 | |
| 904 | 909 | if (is_support_sw_smu(adev)) |
| 905 | − ret = smu_force_clk_levels(&adev->smu, SMU_SCLK, mask); | |
| 910 | + ret = smu_force_clk_levels(&adev->smu, SMU_SCLK, mask, true); | |
| 906 | 911 | else if (adev->powerplay.pp_funcs->force_clock_level) |
| 907 | 912 | ret = amdgpu_dpm_force_clock_level(adev, PP_SCLK, mask); |
| 908 | 913 | |
| @@ -949,7 +954,7 @@ static ssize_t amdgpu_set_pp_dpm_mclk(struct device *dev, · 40 unmodified lines | ||
| 949 | 954 | return ret; |
| 950 | 955 | |
| 951 | 956 | if (is_support_sw_smu(adev)) |
| 952 | − ret = smu_force_clk_levels(&adev->smu, SMU_MCLK, mask); | |
| 957 | + ret = smu_force_clk_levels(&adev->smu, SMU_MCLK, mask, true); | |
| 953 | 958 | else if (adev->powerplay.pp_funcs->force_clock_level) |
| 954 | 959 | ret = amdgpu_dpm_force_clock_level(adev, PP_MCLK, mask); |
| 955 | 960 | |
| @@ -989,7 +994,7 @@ static ssize_t amdgpu_set_pp_dpm_socclk(struct device *dev, · 33 unmodified lines | ||
| 989 | 994 | return ret; |
| 990 | 995 | |
| 991 | 996 | if (is_support_sw_smu(adev)) |
| 992 | − ret = smu_force_clk_levels(&adev->smu, SMU_SOCCLK, mask); | |
| 997 | + ret = smu_force_clk_levels(&adev->smu, SMU_SOCCLK, mask, true); | |
| 993 | 998 | else if (adev->powerplay.pp_funcs->force_clock_level) |
| 994 | 999 | ret = amdgpu_dpm_force_clock_level(adev, PP_SOCCLK, mask); |
| 995 | 1000 | |
| @@ -1029,7 +1034,7 @@ static ssize_t amdgpu_set_pp_dpm_fclk(struct device *dev, · 33 unmodified lines | ||
| 1029 | 1034 | return ret; |
| 1030 | 1035 | |
| 1031 | 1036 | if (is_support_sw_smu(adev)) |
| 1032 | − ret = smu_force_clk_levels(&adev->smu, SMU_FCLK, mask); | |
| 1037 | + ret = smu_force_clk_levels(&adev->smu, SMU_FCLK, mask, true); | |
| 1033 | 1038 | else if (adev->powerplay.pp_funcs->force_clock_level) |
| 1034 | 1039 | ret = amdgpu_dpm_force_clock_level(adev, PP_FCLK, mask); |
| 1035 | 1040 | |
| @@ -1069,7 +1074,7 @@ static ssize_t amdgpu_set_pp_dpm_dcefclk(struct device *dev, · 33 unmodified lines | ||
| 1069 | 1074 | return ret; |
| 1070 | 1075 | |
| 1071 | 1076 | if (is_support_sw_smu(adev)) |
| 1072 | − ret = smu_force_clk_levels(&adev->smu, SMU_DCEFCLK, mask); | |
| 1077 | + ret = smu_force_clk_levels(&adev->smu, SMU_DCEFCLK, mask, true); | |
| 1073 | 1078 | else if (adev->powerplay.pp_funcs->force_clock_level) |
| 1074 | 1079 | ret = amdgpu_dpm_force_clock_level(adev, PP_DCEFCLK, mask); |
| 1075 | 1080 | |
| @@ -1109,7 +1114,7 @@ static ssize_t amdgpu_set_pp_dpm_pcie(struct device *dev, · 33 unmodified lines | ||
| 1109 | 1114 | return ret; |
| 1110 | 1115 | |
| 1111 | 1116 | if (is_support_sw_smu(adev)) |
| 1112 | − ret = smu_force_clk_levels(&adev->smu, SMU_PCIE, mask); | |
| 1117 | + ret = smu_force_clk_levels(&adev->smu, SMU_PCIE, mask, true); | |
| 1113 | 1118 | else if (adev->powerplay.pp_funcs->force_clock_level) |
| 1114 | 1119 | ret = amdgpu_dpm_force_clock_level(adev, PP_PCIE, mask); |
| 1115 | 1120 | |
| @@ -1301,7 +1306,7 @@ static ssize_t amdgpu_set_pp_power_profile_mode(struct device *dev, · 185 unmodified lines | ||
| 1301 | 1306 | } |
| 1302 | 1307 | parameter[parameter_size] = profile_mode; |
| 1303 | 1308 | if (is_support_sw_smu(adev)) |
| 1304 | − ret = smu_set_power_profile_mode(&adev->smu, parameter, parameter_size); | |
| 1309 | + ret = smu_set_power_profile_mode(&adev->smu, parameter, parameter_size, true); | |
| 1305 | 1310 | else if (adev->powerplay.pp_funcs->set_power_profile_mode) |
| 1306 | 1311 | ret = amdgpu_dpm_set_power_profile_mode(adev, parameter, parameter_size); |
| 1307 | 1312 | if (!ret) |
| @@ -2010,7 +2015,7 @@ static ssize_t amdgpu_hwmon_show_power_cap_max(struct device *dev, · 702 unmodified lines | ||
| 2010 | 2015 | uint32_t limit = 0; |
| 2011 | 2016 | |
| 2012 | 2017 | if (is_support_sw_smu(adev)) { |
| 2013 | − smu_get_power_limit(&adev->smu, &limit, true); | |
| 2018 | + smu_get_power_limit(&adev->smu, &limit, true, true); | |
| 2014 | 2019 | return snprintf(buf, PAGE_SIZE, "%u\n", limit * 1000000); |
| 2015 | 2020 | } else if (adev->powerplay.pp_funcs && adev->powerplay.pp_funcs->get_power_limit) { |
| 2016 | 2021 | adev->powerplay.pp_funcs->get_power_limit(adev->powerplay.pp_handle, &limit, true); |
| @@ -2028,7 +2033,7 @@ static ssize_t amdgpu_hwmon_show_power_cap(struct device *dev, · 11 unmodified lines | ||
| 2028 | 2033 | uint32_t limit = 0; |
| 2029 | 2034 | |
| 2030 | 2035 | if (is_support_sw_smu(adev)) { |
| 2031 | − smu_get_power_limit(&adev->smu, &limit, false); | |
| 2036 | + smu_get_power_limit(&adev->smu, &limit, false, true); | |
| 2032 | 2037 | return snprintf(buf, PAGE_SIZE, "%u\n", limit * 1000000); |
| 2033 | 2038 | } else if (adev->powerplay.pp_funcs && adev->powerplay.pp_funcs->get_power_limit) { |
| 2034 | 2039 | adev->powerplay.pp_funcs->get_power_limit(adev->powerplay.pp_handle, &limit, false); |
| @@ -2196,9 +2201,9 @@ static ssize_t amdgpu_hwmon_show_mclk_label(struct device *dev, · 161 unmodified lines | ||
| 2196 | 2201 | * |
| 2197 | 2202 | * - fan1_input: fan speed in RPM |
| 2198 | 2203 | * |
| 2199 | − * - fan[1-*]_target: Desired fan speed Unit: revolution/min (RPM) | |
| 2204 | + * - fan[1-\*]_target: Desired fan speed Unit: revolution/min (RPM) | |
| 2200 | 2205 | * |
| 2201 | − * - fan[1-*]_enable: Enable or disable the sensors.1: Enable 0: Disable | |
| 2206 | + * - fan[1-\*]_enable: Enable or disable the sensors.1: Enable 0: Disable | |
| 2202 | 2207 | * |
| 2203 | 2208 | * hwmon interfaces for GPU clocks: |
| 2204 | 2209 | * |
| @@ -2825,6 +2830,19 @@ int amdgpu_pm_sysfs_init(struct amdgpu_device *adev) · 620 unmodified lines | ||
| 2825 | 2830 | DRM_ERROR("failed to create device file pp_dpm_sclk\n"); |
| 2826 | 2831 | return ret; |
| 2827 | 2832 | } |
| 2833 | + | |
| 2834 | + /* Arcturus does not support standalone mclk/socclk/fclk level setting */ | |
| 2835 | + if (adev->asic_type == CHIP_ARCTURUS) { | |
| 2836 | + dev_attr_pp_dpm_mclk.attr.mode &= ~S_IWUGO; | |
| 2837 | + dev_attr_pp_dpm_mclk.store = NULL; | |
| 2838 | + | |
| 2839 | + dev_attr_pp_dpm_socclk.attr.mode &= ~S_IWUGO; | |
| 2840 | + dev_attr_pp_dpm_socclk.store = NULL; | |
| 2841 | + | |
| 2842 | + dev_attr_pp_dpm_fclk.attr.mode &= ~S_IWUGO; | |
| 2843 | + dev_attr_pp_dpm_fclk.store = NULL; | |
| 2844 | + } | |
| 2845 | + | |
| 2828 | 2846 | ret = device_create_file(adev->dev, &dev_attr_pp_dpm_mclk); |
| 2829 | 2847 | if (ret) { |
| 2830 | 2848 | DRM_ERROR("failed to create device file pp_dpm_mclk\n"); |
| @@ -3008,7 +3026,8 @@ void amdgpu_pm_compute_clocks(struct amdgpu_device *adev) · 177 unmodified lines | ||
| 3008 | 3026 | struct smu_dpm_context *smu_dpm = &adev->smu.smu_dpm; |
| 3009 | 3027 | smu_handle_task(&adev->smu, |
| 3010 | 3028 | smu_dpm->dpm_level, |
| 3011 | − AMD_PP_TASK_DISPLAY_CONFIG_CHANGE); | |
| 3029 | + AMD_PP_TASK_DISPLAY_CONFIG_CHANGE, | |
| 3030 | + true); | |
| 3012 | 3031 | } else { |
| 3013 | 3032 | if (adev->powerplay.pp_funcs->dispatch_tasks) { |
| 3014 | 3033 | if (!amdgpu_device_has_dc_support(adev)) { |
| @@ -34,6 +34,8 @@ · 33 unmodified lines | ||
| 34 | 34 | #include "psp_v11_0.h" |
| 35 | 35 | #include "psp_v12_0.h" |
| 36 | 36 | |
| 37 | +#include "amdgpu_ras.h" | |
| 38 | + | |
| 37 | 39 | static void psp_set_funcs(struct amdgpu_device *adev); |
| 38 | 40 | |
| 39 | 41 | static int psp_early_init(void *handle) |
| @@ -88,6 +90,17 @@ static int psp_sw_init(void *handle) · 48 unmodified lines | ||
| 88 | 90 | return ret; |
| 89 | 91 | } |
| 90 | 92 | |
| 93 | + ret = psp_mem_training_init(psp); | |
| 94 | + if (ret) { | |
| 95 | + DRM_ERROR("Failed to initialize memory training!\n"); | |
| 96 | + return ret; | |
| 97 | + } | |
| 98 | + ret = psp_mem_training(psp, PSP_MEM_TRAIN_COLD_BOOT); | |
| 99 | + if (ret) { | |
| 100 | + DRM_ERROR("Failed to process memory training!\n"); | |
| 101 | + return ret; | |
| 102 | + } | |
| 103 | + | |
| 91 | 104 | return 0; |
| 92 | 105 | } |
| 93 | 106 | |
| @@ -95,6 +108,7 @@ static int psp_sw_fini(void *handle) · 1 unmodified line | ||
| 95 | 108 | { |
| 96 | 109 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 97 | 110 | |
| 111 | + psp_mem_training_fini(&adev->psp); | |
| 98 | 112 | release_firmware(adev->psp.sos_fw); |
| 99 | 113 | adev->psp.sos_fw = NULL; |
| 100 | 114 | release_firmware(adev->psp.asd_fw); |
| @@ -151,10 +165,19 @@ psp_cmd_submit_buf(struct psp_context *psp, · 50 unmodified lines | ||
| 151 | 165 | return ret; |
| 152 | 166 | } |
| 153 | 167 | |
| 168 | + amdgpu_asic_invalidate_hdp(psp->adev, NULL); | |
| 154 | 169 | while (*((unsigned int *)psp->fence_buf) != index) { |
| 155 | 170 | if (--timeout == 0) |
| 156 | 171 | break; |
| 172 | + /* | |
| 173 | + * Shouldn't wait for timeout when err_event_athub occurs, | |
| 174 | + * because gpu reset thread triggered and lock resource should | |
| 175 | + * be released for psp resume sequence. | |
| 176 | + */ | |
| 177 | + if (amdgpu_ras_intr_triggered()) | |
| 178 | + break; | |
| 157 | 179 | msleep(1); |
| 180 | + amdgpu_asic_invalidate_hdp(psp->adev, NULL); | |
| 158 | 181 | } |
| 159 | 182 | |
| 160 | 183 | /* In some cases, psp response status is not 0 even there is no |
| @@ -168,8 +191,9 @@ psp_cmd_submit_buf(struct psp_context *psp, · 7 unmodified lines | ||
| 168 | 191 | if (ucode) |
| 169 | 192 | DRM_WARN("failed to load ucode id (%d) ", |
| 170 | 193 | ucode->ucode_id); |
| 171 | − DRM_WARN("psp command failed and response status is (0x%X)\n", | |
| 172 | − psp->cmd_buf_mem->resp.status & GFX_CMD_STATUS_MASK); | |
| 194 | + DRM_DEBUG_DRIVER("psp command (0x%X) failed and response status is (0x%X)\n", | |
| 195 | + psp->cmd_buf_mem->cmd_id, | |
| 196 | + psp->cmd_buf_mem->resp.status & GFX_CMD_STATUS_MASK); | |
| 173 | 197 | if (!timeout) { |
| 174 | 198 | mutex_unlock(&psp->mutex); |
| 175 | 199 | return -EINVAL; |
| @@ -253,7 +277,8 @@ static int psp_tmr_init(struct psp_context *psp) · 77 unmodified lines | ||
| 253 | 277 | |
| 254 | 278 | /* For ASICs support RLC autoload, psp will parse the toc |
| 255 | 279 | * and calculate the total size of TMR needed */ |
| 256 | − if (psp->toc_start_addr && | |
| 280 | + if (!amdgpu_sriov_vf(psp->adev) && | |
| 281 | + psp->toc_start_addr && | |
| 257 | 282 | psp->toc_bin_size && |
| 258 | 283 | psp->fw_pri_buf) { |
| 259 | 284 | ret = psp_load_toc(psp, &tmr_size); |
| @@ -287,15 +312,9 @@ static int psp_tmr_load(struct psp_context *psp) · 27 unmodified lines | ||
| 287 | 312 | |
| 288 | 313 | ret = psp_cmd_submit_buf(psp, NULL, cmd, |
| 289 | 314 | psp->fence_buf_mc_addr); |
| 290 | − if (ret) | |
| 291 | − goto failed; | |
| 292 | 315 | |
| 293 | 316 | kfree(cmd); |
| 294 | 317 | |
| 295 | − return 0; | |
| 296 | − | |
| 297 | −failed: | |
| 298 | − kfree(cmd); | |
| 299 | 318 | return ret; |
| 300 | 319 | } |
| 301 | 320 | |
| @@ -772,6 +791,324 @@ static int psp_ras_initialize(struct psp_context *psp) · 470 unmodified lines | ||
| 772 | 791 | } |
| 773 | 792 | // ras end |
| 774 | 793 | |
| 794 | +// HDCP start | |
| 795 | +static void psp_prep_hdcp_ta_load_cmd_buf(struct psp_gfx_cmd_resp *cmd, | |
| 796 | + uint64_t hdcp_ta_mc, | |
| 797 | + uint64_t hdcp_mc_shared, | |
| 798 | + uint32_t hdcp_ta_size, | |
| 799 | + uint32_t shared_size) | |
| 800 | +{ | |
| 801 | + cmd->cmd_id = GFX_CMD_ID_LOAD_TA; | |
| 802 | + cmd->cmd.cmd_load_ta.app_phy_addr_lo = lower_32_bits(hdcp_ta_mc); | |
| 803 | + cmd->cmd.cmd_load_ta.app_phy_addr_hi = upper_32_bits(hdcp_ta_mc); | |
| 804 | + cmd->cmd.cmd_load_ta.app_len = hdcp_ta_size; | |
| 805 | + | |
| 806 | + cmd->cmd.cmd_load_ta.cmd_buf_phy_addr_lo = | |
| 807 | + lower_32_bits(hdcp_mc_shared); | |
| 808 | + cmd->cmd.cmd_load_ta.cmd_buf_phy_addr_hi = | |
| 809 | + upper_32_bits(hdcp_mc_shared); | |
| 810 | + cmd->cmd.cmd_load_ta.cmd_buf_len = shared_size; | |
| 811 | +} | |
| 812 | + | |
| 813 | +static int psp_hdcp_init_shared_buf(struct psp_context *psp) | |
| 814 | +{ | |
| 815 | + int ret; | |
| 816 | + | |
| 817 | + /* | |
| 818 | + * Allocate 16k memory aligned to 4k from Frame Buffer (local | |
| 819 | + * physical) for hdcp ta <-> Driver | |
| 820 | + */ | |
| 821 | + ret = amdgpu_bo_create_kernel(psp->adev, PSP_HDCP_SHARED_MEM_SIZE, | |
| 822 | + PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM, | |
| 823 | + &psp->hdcp_context.hdcp_shared_bo, | |
| 824 | + &psp->hdcp_context.hdcp_shared_mc_addr, | |
| 825 | + &psp->hdcp_context.hdcp_shared_buf); | |
| 826 | + | |
| 827 | + return ret; | |
| 828 | +} | |
| 829 | + | |
| 830 | +static int psp_hdcp_load(struct psp_context *psp) | |
| 831 | +{ | |
| 832 | + int ret; | |
| 833 | + struct psp_gfx_cmd_resp *cmd; | |
| 834 | + | |
| 835 | + /* | |
| 836 | + * TODO: bypass the loading in sriov for now | |
| 837 | + */ | |
| 838 | + if (amdgpu_sriov_vf(psp->adev)) | |
| 839 | + return 0; | |
| 840 | + | |
| 841 | + cmd = kzalloc(sizeof(struct psp_gfx_cmd_resp), GFP_KERNEL); | |
| 842 | + if (!cmd) | |
| 843 | + return -ENOMEM; | |
| 844 | + | |
| 845 | + memset(psp->fw_pri_buf, 0, PSP_1_MEG); | |
| 846 | + memcpy(psp->fw_pri_buf, psp->ta_hdcp_start_addr, | |
| 847 | + psp->ta_hdcp_ucode_size); | |
| 848 | + | |
| 849 | + psp_prep_hdcp_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, | |
| 850 | + psp->hdcp_context.hdcp_shared_mc_addr, | |
| 851 | + psp->ta_hdcp_ucode_size, | |
| 852 | + PSP_HDCP_SHARED_MEM_SIZE); | |
| 853 | + | |
| 854 | + ret = psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr); | |
| 855 | + | |
| 856 | + if (!ret) { | |
| 857 | + psp->hdcp_context.hdcp_initialized = 1; | |
| 858 | + psp->hdcp_context.session_id = cmd->resp.session_id; | |
| 859 | + } | |
| 860 | + | |
| 861 | + kfree(cmd); | |
| 862 | + | |
| 863 | + return ret; | |
| 864 | +} | |
| 865 | +static int psp_hdcp_initialize(struct psp_context *psp) | |
| 866 | +{ | |
| 867 | + int ret; | |
| 868 | + | |
| 869 | + if (!psp->hdcp_context.hdcp_initialized) { | |
| 870 | + ret = psp_hdcp_init_shared_buf(psp); | |
| 871 | + if (ret) | |
| 872 | + return ret; | |
| 873 | + } | |
| 874 | + | |
| 875 | + ret = psp_hdcp_load(psp); | |
| 876 | + if (ret) | |
| 877 | + return ret; | |
| 878 | + | |
| 879 | + return 0; | |
| 880 | +} | |
| 881 | +static void psp_prep_hdcp_ta_unload_cmd_buf(struct psp_gfx_cmd_resp *cmd, | |
| 882 | + uint32_t hdcp_session_id) | |
| 883 | +{ | |
| 884 | + cmd->cmd_id = GFX_CMD_ID_UNLOAD_TA; | |
| 885 | + cmd->cmd.cmd_unload_ta.session_id = hdcp_session_id; | |
| 886 | +} | |
| 887 | + | |
| 888 | +static int psp_hdcp_unload(struct psp_context *psp) | |
| 889 | +{ | |
| 890 | + int ret; | |
| 891 | + struct psp_gfx_cmd_resp *cmd; | |
| 892 | + | |
| 893 | + /* | |
| 894 | + * TODO: bypass the unloading in sriov for now | |
| 895 | + */ | |
| 896 | + if (amdgpu_sriov_vf(psp->adev)) | |
| 897 | + return 0; | |
| 898 | + | |
| 899 | + cmd = kzalloc(sizeof(struct psp_gfx_cmd_resp), GFP_KERNEL); | |
| 900 | + if (!cmd) | |
| 901 | + return -ENOMEM; | |
| 902 | + | |
| 903 | + psp_prep_hdcp_ta_unload_cmd_buf(cmd, psp->hdcp_context.session_id); | |
| 904 | + | |
| 905 | + ret = psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr); | |
| 906 | + | |
| 907 | + kfree(cmd); | |
| 908 | + | |
| 909 | + return ret; | |
| 910 | +} | |
| 911 | + | |
| 912 | +static void psp_prep_hdcp_ta_invoke_cmd_buf(struct psp_gfx_cmd_resp *cmd, | |
| 913 | + uint32_t ta_cmd_id, | |
| 914 | + uint32_t hdcp_session_id) | |
| 915 | +{ | |
| 916 | + cmd->cmd_id = GFX_CMD_ID_INVOKE_CMD; | |
| 917 | + cmd->cmd.cmd_invoke_cmd.session_id = hdcp_session_id; | |
| 918 | + cmd->cmd.cmd_invoke_cmd.ta_cmd_id = ta_cmd_id; | |
| 919 | + /* Note: cmd_invoke_cmd.buf is not used for now */ | |
| 920 | +} | |
| 921 | + | |
| 922 | +int psp_hdcp_invoke(struct psp_context *psp, uint32_t ta_cmd_id) | |
| 923 | +{ | |
| 924 | + int ret; | |
| 925 | + struct psp_gfx_cmd_resp *cmd; | |
| 926 | + | |
| 927 | + /* | |
| 928 | + * TODO: bypass the loading in sriov for now | |
| 929 | + */ | |
| 930 | + if (amdgpu_sriov_vf(psp->adev)) | |
| 931 | + return 0; | |
| 932 | + | |
| 933 | + cmd = kzalloc(sizeof(struct psp_gfx_cmd_resp), GFP_KERNEL); | |
| 934 | + if (!cmd) | |
| 935 | + return -ENOMEM; | |
| 936 | + | |
| 937 | + psp_prep_hdcp_ta_invoke_cmd_buf(cmd, ta_cmd_id, | |
| 938 | + psp->hdcp_context.session_id); | |
| 939 | + | |
| 940 | + ret = psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr); | |
| 941 | + | |
| 942 | + kfree(cmd); | |
| 943 | + | |
| 944 | + return ret; | |
| 945 | +} | |
| 946 | + | |
| 947 | +static int psp_hdcp_terminate(struct psp_context *psp) | |
| 948 | +{ | |
| 949 | + int ret; | |
| 950 | + | |
| 951 | + if (!psp->hdcp_context.hdcp_initialized) | |
| 952 | + return 0; | |
| 953 | + | |
| 954 | + ret = psp_hdcp_unload(psp); | |
| 955 | + if (ret) | |
| 956 | + return ret; | |
| 957 | + | |
| 958 | + psp->hdcp_context.hdcp_initialized = 0; | |
| 959 | + | |
| 960 | + /* free hdcp shared memory */ | |
| 961 | + amdgpu_bo_free_kernel(&psp->hdcp_context.hdcp_shared_bo, | |
| 962 | + &psp->hdcp_context.hdcp_shared_mc_addr, | |
| 963 | + &psp->hdcp_context.hdcp_shared_buf); | |
| 964 | + | |
| 965 | + return 0; | |
| 966 | +} | |
| 967 | +// HDCP end | |
| 968 | + | |
| 969 | +// DTM start | |
| 970 | +static void psp_prep_dtm_ta_load_cmd_buf(struct psp_gfx_cmd_resp *cmd, | |
| 971 | + uint64_t dtm_ta_mc, | |
| 972 | + uint64_t dtm_mc_shared, | |
| 973 | + uint32_t dtm_ta_size, | |
| 974 | + uint32_t shared_size) | |
| 975 | +{ | |
| 976 | + cmd->cmd_id = GFX_CMD_ID_LOAD_TA; | |
| 977 | + cmd->cmd.cmd_load_ta.app_phy_addr_lo = lower_32_bits(dtm_ta_mc); | |
| 978 | + cmd->cmd.cmd_load_ta.app_phy_addr_hi = upper_32_bits(dtm_ta_mc); | |
| 979 | + cmd->cmd.cmd_load_ta.app_len = dtm_ta_size; | |
| 980 | + | |
| 981 | + cmd->cmd.cmd_load_ta.cmd_buf_phy_addr_lo = lower_32_bits(dtm_mc_shared); | |
| 982 | + cmd->cmd.cmd_load_ta.cmd_buf_phy_addr_hi = upper_32_bits(dtm_mc_shared); | |
| 983 | + cmd->cmd.cmd_load_ta.cmd_buf_len = shared_size; | |
| 984 | +} | |
| 985 | + | |
| 986 | +static int psp_dtm_init_shared_buf(struct psp_context *psp) | |
| 987 | +{ | |
| 988 | + int ret; | |
| 989 | + | |
| 990 | + /* | |
| 991 | + * Allocate 16k memory aligned to 4k from Frame Buffer (local | |
| 992 | + * physical) for dtm ta <-> Driver | |
| 993 | + */ | |
| 994 | + ret = amdgpu_bo_create_kernel(psp->adev, PSP_DTM_SHARED_MEM_SIZE, | |
| 995 | + PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM, | |
| 996 | + &psp->dtm_context.dtm_shared_bo, | |
| 997 | + &psp->dtm_context.dtm_shared_mc_addr, | |
| 998 | + &psp->dtm_context.dtm_shared_buf); | |
| 999 | + | |
| 1000 | + return ret; | |
| 1001 | +} | |
| 1002 | + | |
| 1003 | +static int psp_dtm_load(struct psp_context *psp) | |
| 1004 | +{ | |
| 1005 | + int ret; | |
| 1006 | + struct psp_gfx_cmd_resp *cmd; | |
| 1007 | + | |
| 1008 | + /* | |
| 1009 | + * TODO: bypass the loading in sriov for now | |
| 1010 | + */ | |
| 1011 | + if (amdgpu_sriov_vf(psp->adev)) | |
| 1012 | + return 0; | |
| 1013 | + | |
| 1014 | + cmd = kzalloc(sizeof(struct psp_gfx_cmd_resp), GFP_KERNEL); | |
| 1015 | + if (!cmd) | |
| 1016 | + return -ENOMEM; | |
| 1017 | + | |
| 1018 | + memset(psp->fw_pri_buf, 0, PSP_1_MEG); | |
| 1019 | + memcpy(psp->fw_pri_buf, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); | |
| 1020 | + | |
| 1021 | + psp_prep_dtm_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, | |
| 1022 | + psp->dtm_context.dtm_shared_mc_addr, | |
| 1023 | + psp->ta_dtm_ucode_size, | |
| 1024 | + PSP_DTM_SHARED_MEM_SIZE); | |
| 1025 | + | |
| 1026 | + ret = psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr); | |
| 1027 | + | |
| 1028 | + if (!ret) { | |
| 1029 | + psp->dtm_context.dtm_initialized = 1; | |
| 1030 | + psp->dtm_context.session_id = cmd->resp.session_id; | |
| 1031 | + } | |
| 1032 | + | |
| 1033 | + kfree(cmd); | |
| 1034 | + | |
| 1035 | + return ret; | |
| 1036 | +} | |
| 1037 | + | |
| 1038 | +static int psp_dtm_initialize(struct psp_context *psp) | |
| 1039 | +{ | |
| 1040 | + int ret; | |
| 1041 | + | |
| 1042 | + if (!psp->dtm_context.dtm_initialized) { | |
| 1043 | + ret = psp_dtm_init_shared_buf(psp); | |
| 1044 | + if (ret) | |
| 1045 | + return ret; | |
| 1046 | + } | |
| 1047 | + | |
| 1048 | + ret = psp_dtm_load(psp); | |
| 1049 | + if (ret) | |
| 1050 | + return ret; | |
| 1051 | + | |
| 1052 | + return 0; | |
| 1053 | +} | |
| 1054 | + | |
| 1055 | +static void psp_prep_dtm_ta_invoke_cmd_buf(struct psp_gfx_cmd_resp *cmd, | |
| 1056 | + uint32_t ta_cmd_id, | |
| 1057 | + uint32_t dtm_session_id) | |
| 1058 | +{ | |
| 1059 | + cmd->cmd_id = GFX_CMD_ID_INVOKE_CMD; | |
| 1060 | + cmd->cmd.cmd_invoke_cmd.session_id = dtm_session_id; | |
| 1061 | + cmd->cmd.cmd_invoke_cmd.ta_cmd_id = ta_cmd_id; | |
| 1062 | + /* Note: cmd_invoke_cmd.buf is not used for now */ | |
| 1063 | +} | |
| 1064 | + | |
| 1065 | +int psp_dtm_invoke(struct psp_context *psp, uint32_t ta_cmd_id) | |
| 1066 | +{ | |
| 1067 | + int ret; | |
| 1068 | + struct psp_gfx_cmd_resp *cmd; | |
| 1069 | + | |
| 1070 | + /* | |
| 1071 | + * TODO: bypass the loading in sriov for now | |
| 1072 | + */ | |
| 1073 | + if (amdgpu_sriov_vf(psp->adev)) | |
| 1074 | + return 0; | |
| 1075 | + | |
| 1076 | + cmd = kzalloc(sizeof(struct psp_gfx_cmd_resp), GFP_KERNEL); | |
| 1077 | + if (!cmd) | |
| 1078 | + return -ENOMEM; | |
| 1079 | + | |
| 1080 | + psp_prep_dtm_ta_invoke_cmd_buf(cmd, ta_cmd_id, | |
| 1081 | + psp->dtm_context.session_id); | |
| 1082 | + | |
| 1083 | + ret = psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr); | |
| 1084 | + | |
| 1085 | + kfree(cmd); | |
| 1086 | + | |
| 1087 | + return ret; | |
| 1088 | +} | |
| 1089 | + | |
| 1090 | +static int psp_dtm_terminate(struct psp_context *psp) | |
| 1091 | +{ | |
| 1092 | + int ret; | |
| 1093 | + | |
| 1094 | + if (!psp->dtm_context.dtm_initialized) | |
| 1095 | + return 0; | |
| 1096 | + | |
| 1097 | + ret = psp_hdcp_unload(psp); | |
| 1098 | + if (ret) | |
| 1099 | + return ret; | |
| 1100 | + | |
| 1101 | + psp->dtm_context.dtm_initialized = 0; | |
| 1102 | + | |
| 1103 | + /* free hdcp shared memory */ | |
| 1104 | + amdgpu_bo_free_kernel(&psp->dtm_context.dtm_shared_bo, | |
| 1105 | + &psp->dtm_context.dtm_shared_mc_addr, | |
| 1106 | + &psp->dtm_context.dtm_shared_buf); | |
| 1107 | + | |
| 1108 | + return 0; | |
| 1109 | +} | |
| 1110 | +// DTM end | |
| 1111 | + | |
| 775 | 1112 | static int psp_hw_start(struct psp_context *psp) |
| 776 | 1113 | { |
| 777 | 1114 | struct amdgpu_device *adev = psp->adev; |
| @@ -845,6 +1182,16 @@ static int psp_hw_start(struct psp_context *psp) · 67 unmodified lines | ||
| 845 | 1182 | if (ret) |
| 846 | 1183 | dev_err(psp->adev->dev, |
| 847 | 1184 | "RAS: Failed to initialize RAS\n"); |
| 1185 | + | |
| 1186 | + ret = psp_hdcp_initialize(psp); | |
| 1187 | + if (ret) | |
| 1188 | + dev_err(psp->adev->dev, | |
| 1189 | + "HDCP: Failed to initialize HDCP\n"); | |
| 1190 | + | |
| 1191 | + ret = psp_dtm_initialize(psp); | |
| 1192 | + if (ret) | |
| 1193 | + dev_err(psp->adev->dev, | |
| 1194 | + "DTM: Failed to initialize DTM\n"); | |
| 848 | 1195 | } |
| 849 | 1196 | |
| 850 | 1197 | return 0; |
| @@ -950,21 +1297,7 @@ static void psp_print_fw_hdr(struct psp_context *psp, · 99 unmodified lines | ||
| 950 | 1297 | struct amdgpu_firmware_info *ucode) |
| 951 | 1298 | { |
| 952 | 1299 | struct amdgpu_device *adev = psp->adev; |
| 953 | − const struct sdma_firmware_header_v1_0 *sdma_hdr = | |
| 954 | − (const struct sdma_firmware_header_v1_0 *) | |
| 955 | − adev->sdma.instance[ucode->ucode_id - AMDGPU_UCODE_ID_SDMA0].fw->data; | |
| 956 | − const struct gfx_firmware_header_v1_0 *ce_hdr = | |
| 957 | − (const struct gfx_firmware_header_v1_0 *)adev->gfx.ce_fw->data; | |
| 958 | − const struct gfx_firmware_header_v1_0 *pfp_hdr = | |
| 959 | − (const struct gfx_firmware_header_v1_0 *)adev->gfx.pfp_fw->data; | |
| 960 | − const struct gfx_firmware_header_v1_0 *me_hdr = | |
| 961 | − (const struct gfx_firmware_header_v1_0 *)adev->gfx.me_fw->data; | |
| 962 | − const struct gfx_firmware_header_v1_0 *mec_hdr = | |
| 963 | − (const struct gfx_firmware_header_v1_0 *)adev->gfx.mec_fw->data; | |
| 964 | − const struct rlc_firmware_header_v2_0 *rlc_hdr = | |
| 965 | − (const struct rlc_firmware_header_v2_0 *)adev->gfx.rlc_fw->data; | |
| 966 | − const struct smc_firmware_header_v1_0 *smc_hdr = | |
| 967 | − (const struct smc_firmware_header_v1_0 *)adev->pm.fw->data; | |
| 1300 | + struct common_firmware_header *hdr; | |
| 968 | 1301 | |
| 969 | 1302 | switch (ucode->ucode_id) { |
| 970 | 1303 | case AMDGPU_UCODE_ID_SDMA0: |
| @@ -975,25 +1308,33 @@ static void psp_print_fw_hdr(struct psp_context *psp, · 4 unmodified lines | ||
| 975 | 1308 | case AMDGPU_UCODE_ID_SDMA5: |
| 976 | 1309 | case AMDGPU_UCODE_ID_SDMA6: |
| 977 | 1310 | case AMDGPU_UCODE_ID_SDMA7: |
| 978 | − amdgpu_ucode_print_sdma_hdr(&sdma_hdr->header); | |
| 1311 | + hdr = (struct common_firmware_header *) | |
| 1312 | + adev->sdma.instance[ucode->ucode_id - AMDGPU_UCODE_ID_SDMA0].fw->data; | |
| 1313 | + amdgpu_ucode_print_sdma_hdr(hdr); | |
| 979 | 1314 | break; |
| 980 | 1315 | case AMDGPU_UCODE_ID_CP_CE: |
| 981 | − amdgpu_ucode_print_gfx_hdr(&ce_hdr->header); | |
| 1316 | + hdr = (struct common_firmware_header *)adev->gfx.ce_fw->data; | |
| 1317 | + amdgpu_ucode_print_gfx_hdr(hdr); | |
| 982 | 1318 | break; |
| 983 | 1319 | case AMDGPU_UCODE_ID_CP_PFP: |
| 984 | − amdgpu_ucode_print_gfx_hdr(&pfp_hdr->header); | |
| 1320 | + hdr = (struct common_firmware_header *)adev->gfx.pfp_fw->data; | |
| 1321 | + amdgpu_ucode_print_gfx_hdr(hdr); | |
| 985 | 1322 | break; |
| 986 | 1323 | case AMDGPU_UCODE_ID_CP_ME: |
| 987 | − amdgpu_ucode_print_gfx_hdr(&me_hdr->header); | |
| 1324 | + hdr = (struct common_firmware_header *)adev->gfx.me_fw->data; | |
| 1325 | + amdgpu_ucode_print_gfx_hdr(hdr); | |
| 988 | 1326 | break; |
| 989 | 1327 | case AMDGPU_UCODE_ID_CP_MEC1: |
| 990 | − amdgpu_ucode_print_gfx_hdr(&mec_hdr->header); | |
| 1328 | + hdr = (struct common_firmware_header *)adev->gfx.mec_fw->data; | |
| 1329 | + amdgpu_ucode_print_gfx_hdr(hdr); | |
| 991 | 1330 | break; |
| 992 | 1331 | case AMDGPU_UCODE_ID_RLC_G: |
| 993 | − amdgpu_ucode_print_rlc_hdr(&rlc_hdr->header); | |
| 1332 | + hdr = (struct common_firmware_header *)adev->gfx.rlc_fw->data; | |
| 1333 | + amdgpu_ucode_print_rlc_hdr(hdr); | |
| 994 | 1334 | break; |
| 995 | 1335 | case AMDGPU_UCODE_ID_SMC: |
| 996 | − amdgpu_ucode_print_smc_hdr(&smc_hdr->header); | |
| 1336 | + hdr = (struct common_firmware_header *)adev->pm.fw->data; | |
| 1337 | + amdgpu_ucode_print_smc_hdr(hdr); | |
| 997 | 1338 | break; |
| 998 | 1339 | default: |
| 999 | 1340 | break; |
| @@ -1079,10 +1420,6 @@ out: · 79 unmodified lines | ||
| 1079 | 1420 | ucode->ucode_id == AMDGPU_UCODE_ID_CP_MEC2_JT)) |
| 1080 | 1421 | /* skip mec JT when autoload is enabled */ |
| 1081 | 1422 | continue; |
| 1082 | − /* Renoir only needs to load mec jump table one time */ | |
| 1083 | − if (adev->asic_type == CHIP_RENOIR && | |
| 1084 | − ucode->ucode_id == AMDGPU_UCODE_ID_CP_MEC2_JT) | |
| 1085 | − continue; | |
| 1086 | 1423 | |
| 1087 | 1424 | psp_print_fw_hdr(psp, ucode); |
| 1088 | 1425 | |
| @@ -1091,7 +1428,8 @@ out: · 2 unmodified lines | ||
| 1091 | 1428 | return ret; |
| 1092 | 1429 | |
| 1093 | 1430 | /* Start rlc autoload after psp recieved all the gfx firmware */ |
| 1094 | − if (ucode->ucode_id == AMDGPU_UCODE_ID_RLC_RESTORE_LIST_SRM_MEM) { | |
| 1431 | + if (psp->autoload_supported && ucode->ucode_id == | |
| 1432 | + AMDGPU_UCODE_ID_RLC_RESTORE_LIST_SRM_MEM) { | |
| 1095 | 1433 | ret = psp_rlc_autoload(psp); |
| 1096 | 1434 | if (ret) { |
| 1097 | 1435 | DRM_ERROR("Failed to start rlc autoload\n"); |
| @@ -1216,8 +1554,11 @@ static int psp_hw_fini(void *handle) · 118 unmodified lines | ||
| 1216 | 1554 | psp->xgmi_context.initialized == 1) |
| 1217 | 1555 | psp_xgmi_terminate(psp); |
| 1218 | 1556 | |
| 1219 | − if (psp->adev->psp.ta_fw) | |
| 1557 | + if (psp->adev->psp.ta_fw) { | |
| 1220 | 1558 | psp_ras_terminate(psp); |
| 1559 | + psp_dtm_terminate(psp); | |
| 1560 | + psp_hdcp_terminate(psp); | |
| 1561 | + } | |
| 1221 | 1562 | |
| 1222 | 1563 | psp_ring_destroy(psp, PSP_RING_TYPE__KM); |
| 1223 | 1564 | |
| @@ -1259,6 +1600,16 @@ static int psp_suspend(void *handle) · 35 unmodified lines | ||
| 1259 | 1600 | DRM_ERROR("Failed to terminate ras ta\n"); |
| 1260 | 1601 | return ret; |
| 1261 | 1602 | } |
| 1603 | + ret = psp_hdcp_terminate(psp); | |
| 1604 | + if (ret) { | |
| 1605 | + DRM_ERROR("Failed to terminate hdcp ta\n"); | |
| 1606 | + return ret; | |
| 1607 | + } | |
| 1608 | + ret = psp_dtm_terminate(psp); | |
| 1609 | + if (ret) { | |
| 1610 | + DRM_ERROR("Failed to terminate dtm ta\n"); | |
| 1611 | + return ret; | |
| 1612 | + } | |
| 1262 | 1613 | } |
| 1263 | 1614 | |
| 1264 | 1615 | ret = psp_ring_stop(psp, PSP_RING_TYPE__KM); |
| @@ -1278,6 +1629,12 @@ static int psp_resume(void *handle) · 13 unmodified lines | ||
| 1278 | 1629 | |
| 1279 | 1630 | DRM_INFO("PSP is resuming...\n"); |
| 1280 | 1631 | |
| 1632 | + ret = psp_mem_training(psp, PSP_MEM_TRAIN_RESUME); | |
| 1633 | + if (ret) { | |
| 1634 | + DRM_ERROR("Failed to process memory training!\n"); | |
| 1635 | + return ret; | |
| 1636 | + } | |
| 1637 | + | |
| 1281 | 1638 | mutex_lock(&adev->firmware.mutex); |
| 1282 | 1639 | |
| 1283 | 1640 | ret = psp_hw_start(psp); |
| @@ -1317,9 +1674,6 @@ int psp_rlc_autoload_start(struct psp_context *psp) · 33 unmodified lines | ||
| 1317 | 1674 | int ret; |
| 1318 | 1675 | struct psp_gfx_cmd_resp *cmd; |
| 1319 | 1676 | |
| 1320 | − if (amdgpu_sriov_vf(psp->adev)) | |
| 1321 | − return 0; | |
| 1322 | − | |
| 1323 | 1677 | cmd = kzalloc(sizeof(struct psp_gfx_cmd_resp), GFP_KERNEL); |
| 1324 | 1678 | if (!cmd) |
| 1325 | 1679 | return -ENOMEM; |
| @@ -37,6 +37,9 @@ · 36 unmodified lines | ||
| 37 | 37 | #define PSP_RAS_SHARED_MEM_SIZE 0x4000 |
| 38 | 38 | #define PSP_1_MEG 0x100000 |
| 39 | 39 | #define PSP_TMR_SIZE 0x400000 |
| 40 | +#define PSP_HDCP_SHARED_MEM_SIZE 0x4000 | |
| 41 | +#define PSP_DTM_SHARED_MEM_SIZE 0x4000 | |
| 42 | +#define PSP_SHARED_MEM_SIZE 0x4000 | |
| 40 | 43 | |
| 41 | 44 | struct psp_context; |
| 42 | 45 | struct psp_xgmi_node_info; |
| @@ -46,6 +49,8 @@ enum psp_bootloader_cmd { · 3 unmodified lines | ||
| 46 | 49 | PSP_BL__LOAD_SYSDRV = 0x10000, |
| 47 | 50 | PSP_BL__LOAD_SOSDRV = 0x20000, |
| 48 | 51 | PSP_BL__LOAD_KEY_DATABASE = 0x80000, |
| 52 | + PSP_BL__DRAM_LONG_TRAIN = 0x100000, | |
| 53 | + PSP_BL__DRAM_SHORT_TRAIN = 0x200000, | |
| 49 | 54 | }; |
| 50 | 55 | |
| 51 | 56 | enum psp_ring_type |
| @@ -108,6 +113,9 @@ struct psp_funcs · 56 unmodified lines | ||
| 108 | 113 | struct ta_ras_trigger_error_input *info); |
| 109 | 114 | int (*ras_cure_posion)(struct psp_context *psp, uint64_t *mode_ptr); |
| 110 | 115 | int (*rlc_autoload_start)(struct psp_context *psp); |
| 116 | + int (*mem_training_init)(struct psp_context *psp); | |
| 117 | + void (*mem_training_fini)(struct psp_context *psp); | |
| 118 | + int (*mem_training)(struct psp_context *psp, uint32_t ops); | |
| 111 | 119 | }; |
| 112 | 120 | |
| 113 | 121 | #define AMDGPU_XGMI_MAX_CONNECTED_NODES 64 |
| @@ -142,6 +150,65 @@ struct psp_ras_context { · 28 unmodified lines | ||
| 142 | 150 | struct amdgpu_ras *ras; |
| 143 | 151 | }; |
| 144 | 152 | |
| 153 | +struct psp_hdcp_context { | |
| 154 | + bool hdcp_initialized; | |
| 155 | + uint32_t session_id; | |
| 156 | + struct amdgpu_bo *hdcp_shared_bo; | |
| 157 | + uint64_t hdcp_shared_mc_addr; | |
| 158 | + void *hdcp_shared_buf; | |
| 159 | +}; | |
| 160 | + | |
| 161 | +struct psp_dtm_context { | |
| 162 | + bool dtm_initialized; | |
| 163 | + uint32_t session_id; | |
| 164 | + struct amdgpu_bo *dtm_shared_bo; | |
| 165 | + uint64_t dtm_shared_mc_addr; | |
| 166 | + void *dtm_shared_buf; | |
| 167 | +}; | |
| 168 | + | |
| 169 | +#define MEM_TRAIN_SYSTEM_SIGNATURE 0x54534942 | |
| 170 | +#define GDDR6_MEM_TRAINING_DATA_SIZE_IN_BYTES 0x1000 | |
| 171 | +#define GDDR6_MEM_TRAINING_OFFSET 0x8000 | |
| 172 | + | |
| 173 | +enum psp_memory_training_init_flag { | |
| 174 | + PSP_MEM_TRAIN_NOT_SUPPORT = 0x0, | |
| 175 | + PSP_MEM_TRAIN_SUPPORT = 0x1, | |
| 176 | + PSP_MEM_TRAIN_INIT_FAILED = 0x2, | |
| 177 | + PSP_MEM_TRAIN_RESERVE_SUCCESS = 0x4, | |
| 178 | + PSP_MEM_TRAIN_INIT_SUCCESS = 0x8, | |
| 179 | +}; | |
| 180 | + | |
| 181 | +enum psp_memory_training_ops { | |
| 182 | + PSP_MEM_TRAIN_SEND_LONG_MSG = 0x1, | |
| 183 | + PSP_MEM_TRAIN_SAVE = 0x2, | |
| 184 | + PSP_MEM_TRAIN_RESTORE = 0x4, | |
| 185 | + PSP_MEM_TRAIN_SEND_SHORT_MSG = 0x8, | |
| 186 | + PSP_MEM_TRAIN_COLD_BOOT = PSP_MEM_TRAIN_SEND_LONG_MSG, | |
| 187 | + PSP_MEM_TRAIN_RESUME = PSP_MEM_TRAIN_SEND_SHORT_MSG, | |
| 188 | +}; | |
| 189 | + | |
| 190 | +struct psp_memory_training_context { | |
| 191 | + /*training data size*/ | |
| 192 | + u64 train_data_size; | |
| 193 | + /* | |
| 194 | + * sys_cache | |
| 195 | + * cpu virtual address | |
| 196 | + * system memory buffer that used to store the training data. | |
| 197 | + */ | |
| 198 | + void *sys_cache; | |
| 199 | + | |
| 200 | + /*vram offset of the p2c training data*/ | |
| 201 | + u64 p2c_train_data_offset; | |
| 202 | + struct amdgpu_bo *p2c_bo; | |
| 203 | + | |
| 204 | + /*vram offset of the c2p training data*/ | |
| 205 | + u64 c2p_train_data_offset; | |
| 206 | + struct amdgpu_bo *c2p_bo; | |
| 207 | + | |
| 208 | + enum psp_memory_training_init_flag init; | |
| 209 | + u32 training_cnt; | |
| 210 | +}; | |
| 211 | + | |
| 145 | 212 | struct psp_context |
| 146 | 213 | { |
| 147 | 214 | struct amdgpu_device *adev; |
| @@ -206,9 +273,21 @@ struct psp_context · 58 unmodified lines | ||
| 206 | 273 | uint32_t ta_ras_ucode_version; |
| 207 | 274 | uint32_t ta_ras_ucode_size; |
| 208 | 275 | uint8_t *ta_ras_start_addr; |
| 276 | + | |
| 277 | + uint32_t ta_hdcp_ucode_version; | |
| 278 | + uint32_t ta_hdcp_ucode_size; | |
| 279 | + uint8_t *ta_hdcp_start_addr; | |
| 280 | + | |
| 281 | + uint32_t ta_dtm_ucode_version; | |
| 282 | + uint32_t ta_dtm_ucode_size; | |
| 283 | + uint8_t *ta_dtm_start_addr; | |
| 284 | + | |
| 209 | 285 | struct psp_xgmi_context xgmi_context; |
| 210 | 286 | struct psp_ras_context ras; |
| 287 | + struct psp_hdcp_context hdcp_context; | |
| 288 | + struct psp_dtm_context dtm_context; | |
| 211 | 289 | struct mutex mutex; |
| 290 | + struct psp_memory_training_context mem_train_ctx; | |
| 212 | 291 | }; |
| 213 | 292 | |
| 214 | 293 | struct amdgpu_psp_funcs { |
| @@ -251,6 +330,12 @@ struct amdgpu_psp_funcs { · 36 unmodified lines | ||
| 251 | 330 | (psp)->funcs->xgmi_set_topology_info((psp), (num_device), (topology)) : -EINVAL) |
| 252 | 331 | #define psp_rlc_autoload(psp) \ |
| 253 | 332 | ((psp)->funcs->rlc_autoload_start ? (psp)->funcs->rlc_autoload_start((psp)) : 0) |
| 333 | +#define psp_mem_training_init(psp) \ | |
| 334 | + ((psp)->funcs->mem_training_init ? (psp)->funcs->mem_training_init((psp)) : 0) | |
| 335 | +#define psp_mem_training_fini(psp) \ | |
| 336 | + ((psp)->funcs->mem_training_fini ? (psp)->funcs->mem_training_fini((psp)) : 0) | |
| 337 | +#define psp_mem_training(psp, ops) \ | |
| 338 | + ((psp)->funcs->mem_training ? (psp)->funcs->mem_training((psp), (ops)) : 0) | |
| 254 | 339 | |
| 255 | 340 | #define amdgpu_psp_check_fw_loading_status(adev, i) (adev)->firmware.funcs->check_fw_loading_status((adev), (i)) |
| 256 | 341 | |
| @@ -279,6 +364,8 @@ int psp_xgmi_invoke(struct psp_context *psp, uint32_t ta_cmd_id); · 22 unmodified lines | ||
| 279 | 364 | int psp_ras_invoke(struct psp_context *psp, uint32_t ta_cmd_id); |
| 280 | 365 | int psp_ras_enable_features(struct psp_context *psp, |
| 281 | 366 | union ta_ras_cmd_input *info, bool enable); |
| 367 | +int psp_hdcp_invoke(struct psp_context *psp, uint32_t ta_cmd_id); | |
| 368 | +int psp_dtm_invoke(struct psp_context *psp, uint32_t ta_cmd_id); | |
| 282 | 369 | |
| 283 | 370 | int psp_rlc_autoload_start(struct psp_context *psp); |
| 284 | 371 | |
| @@ -25,10 +25,13 @@ · 24 unmodified lines | ||
| 25 | 25 | #include <linux/list.h> |
| 26 | 26 | #include <linux/module.h> |
| 27 | 27 | #include <linux/uaccess.h> |
| 28 | +#include <linux/reboot.h> | |
| 29 | +#include <linux/syscalls.h> | |
| 28 | 30 | |
| 29 | 31 | #include "amdgpu.h" |
| 30 | 32 | #include "amdgpu_ras.h" |
| 31 | 33 | #include "amdgpu_atomfirmware.h" |
| 34 | +#include "ivsrcid/nbio/irqsrcs_nbif_7_4.h" | |
| 32 | 35 | |
| 33 | 36 | const char *ras_error_string[] = { |
| 34 | 37 | "none", |
| @@ -65,11 +68,16 @@ const char *ras_block_string[] = { · 30 unmodified lines | ||
| 65 | 68 | /* inject address is 52 bits */ |
| 66 | 69 | #define RAS_UMC_INJECT_ADDR_LIMIT (0x1ULL << 52) |
| 67 | 70 | |
| 68 | −static int amdgpu_ras_reserve_vram(struct amdgpu_device *adev, | |
| 69 | − uint64_t offset, uint64_t size, | |
| 70 | − struct amdgpu_bo **bo_ptr); | |
| 71 | −static int amdgpu_ras_release_vram(struct amdgpu_device *adev, | |
| 72 | − struct amdgpu_bo **bo_ptr); | |
| 71 | +enum amdgpu_ras_retire_page_reservation { | |
| 72 | + AMDGPU_RAS_RETIRE_PAGE_RESERVED, | |
| 73 | + AMDGPU_RAS_RETIRE_PAGE_PENDING, | |
| 74 | + AMDGPU_RAS_RETIRE_PAGE_FAULT, | |
| 75 | +}; | |
| 76 | + | |
| 77 | +atomic_t amdgpu_ras_in_intr = ATOMIC_INIT(0); | |
| 78 | + | |
| 79 | +static bool amdgpu_ras_check_bad_page(struct amdgpu_device *adev, | |
| 80 | + uint64_t addr); | |
| 73 | 81 | |
| 74 | 82 | static ssize_t amdgpu_ras_debugfs_read(struct file *f, char __user *buf, |
| 75 | 83 | size_t size, loff_t *pos) |
| @@ -189,6 +197,10 @@ static int amdgpu_ras_debugfs_ctrl_parse_data(struct file *f, · 113 unmodified lines | ||
| 189 | 197 | |
| 190 | 198 | return 0; |
| 191 | 199 | } |
| 200 | + | |
| 201 | +static struct ras_manager *amdgpu_ras_find_obj(struct amdgpu_device *adev, | |
| 202 | + struct ras_common_if *head); | |
| 203 | + | |
| 192 | 204 | /** |
| 193 | 205 | * DOC: AMDGPU RAS debugfs control interface |
| 194 | 206 | * |
| @@ -208,31 +220,44 @@ static int amdgpu_ras_debugfs_ctrl_parse_data(struct file *f, · 13 unmodified lines | ||
| 208 | 220 | * As their names indicate, inject operation will write the |
| 209 | 221 | * value to the address. |
| 210 | 222 | * |
| 211 | − * Second member: struct ras_debug_if::op. | |
| 223 | + * The second member: struct ras_debug_if::op. | |
| 212 | 224 | * It has three kinds of operations. |
| 213 | − * 0: disable RAS on the block. Take ::head as its data. | |
| 214 | − * 1: enable RAS on the block. Take ::head as its data. | |
| 215 | − * 2: inject errors on the block. Take ::inject as its data. | |
| 225 | + * | |
| 226 | + * - 0: disable RAS on the block. Take ::head as its data. | |
| 227 | + * - 1: enable RAS on the block. Take ::head as its data. | |
| 228 | + * - 2: inject errors on the block. Take ::inject as its data. | |
| 216 | 229 | * |
| 217 | 230 | * How to use the interface? |
| 218 | − * programs: | |
| 219 | − * copy the struct ras_debug_if in your codes and initialize it. | |
| 220 | − * write the struct to the control node. | |
| 221 | 231 | * |
| 222 | − * bash: | |
| 223 | − * echo op block [error [sub_blcok address value]] > .../ras/ras_ctrl | |
| 224 | − * op: disable, enable, inject | |
| 225 | − * disable: only block is needed | |
| 226 | − * enable: block and error are needed | |
| 227 | − * inject: error, address, value are needed | |
| 228 | − * block: umc, smda, gfx, ......... | |
| 229 | − * see ras_block_string[] for details | |
| 230 | − * error: ue, ce | |
| 231 | − * ue: multi_uncorrectable | |
| 232 | − * ce: single_correctable | |
| 233 | − * sub_block: sub block index, pass 0 if there is no sub block | |
| 232 | + * Programs | |
| 233 | + * | |
| 234 | + * Copy the struct ras_debug_if in your codes and initialize it. | |
| 235 | + * Write the struct to the control node. | |
| 236 | + * | |
| 237 | + * Shells | |
| 238 | + * | |
| 239 | + * .. code-block:: bash | |
| 240 | + * | |
| 241 | + * echo op block [error [sub_block address value]] > .../ras/ras_ctrl | |
| 242 | + * | |
| 243 | + * Parameters: | |
| 244 | + * | |
| 245 | + * op: disable, enable, inject | |
| 246 | + * disable: only block is needed | |
| 247 | + * enable: block and error are needed | |
| 248 | + * inject: error, address, value are needed | |
| 249 | + * block: umc, sdma, gfx, ......... | |
| 250 | + * see ras_block_string[] for details | |
| 251 | + * error: ue, ce | |
| 252 | + * ue: multi_uncorrectable | |
| 253 | + * ce: single_correctable | |
| 254 | + * sub_block: | |
| 255 | + * sub block index, pass 0 if there is no sub block | |
| 256 | + * | |
| 257 | + * here are some examples for bash commands: | |
| 258 | + * | |
| 259 | + * .. code-block:: bash | |
| 234 | 260 | * |
| 235 | − * here are some examples for bash commands, | |
| 236 | 261 | * echo inject umc ue 0x0 0x0 0x0 > /sys/kernel/debug/dri/0/ras/ras_ctrl |
| 237 | 262 | * echo inject umc ce 0 0 0 > /sys/kernel/debug/dri/0/ras/ras_ctrl |
| 238 | 263 | * echo disable umc > /sys/kernel/debug/dri/0/ras/ras_ctrl |
| @@ -245,8 +270,11 @@ static int amdgpu_ras_debugfs_ctrl_parse_data(struct file *f, · 6 unmodified lines | ||
| 245 | 270 | * For inject, please check corresponding err count at |
| 246 | 271 | * /sys/class/drm/card[0/1/2...]/device/ras/[gfx/sdma/...]_err_count |
| 247 | 272 | * |
| 248 | − * NOTE: operation is only allowed on blocks which are supported. | |
| 249 | − * Please check ras mask at /sys/module/amdgpu/parameters/ras_mask | |
| 273 | + * .. note:: | |
| 274 | + * Operations are only allowed on blocks which are supported. | |
| 275 | + * Please check ras mask at /sys/module/amdgpu/parameters/ras_mask | |
| 276 | + * to see which blocks support RAS on a particular asic. | |
| 277 | + * | |
| 250 | 278 | */ |
| 251 | 279 | static ssize_t amdgpu_ras_debugfs_ctrl_write(struct file *f, const char __user *buf, |
| 252 | 280 | size_t size, loff_t *pos) |
| @@ -276,6 +304,14 @@ static ssize_t amdgpu_ras_debugfs_ctrl_write(struct file *f, const char __user * · 23 unmodified lines | ||
| 276 | 304 | break; |
| 277 | 305 | } |
| 278 | 306 | |
| 307 | + /* umc ce/ue error injection for a bad page is not allowed */ | |
| 308 | + if ((data.head.block == AMDGPU_RAS_BLOCK__UMC) && | |
| 309 | + amdgpu_ras_check_bad_page(adev, data.inject.address)) { | |
| 310 | + DRM_WARN("RAS WARN: 0x%llx has been marked as bad before error injection!\n", | |
| 311 | + data.inject.address); | |
| 312 | + break; | |
| 313 | + } | |
| 314 | + | |
| 279 | 315 | /* data.inject.address is offset instead of absolute gpu address */ |
| 280 | 316 | ret = amdgpu_ras_error_inject(adev, &data.inject); |
| 281 | 317 | break; |
| @@ -290,6 +326,33 @@ static ssize_t amdgpu_ras_debugfs_ctrl_write(struct file *f, const char __user * · 8 unmodified lines | ||
| 290 | 326 | return size; |
| 291 | 327 | } |
| 292 | 328 | |
| 329 | +/** | |
| 330 | + * DOC: AMDGPU RAS debugfs EEPROM table reset interface | |
| 331 | + * | |
| 332 | + * Some boards contain an EEPROM which is used to persistently store a list of | |
| 333 | + * bad pages which experiences ECC errors in vram. This interface provides | |
| 334 | + * a way to reset the EEPROM, e.g., after testing error injection. | |
| 335 | + * | |
| 336 | + * Usage: | |
| 337 | + * | |
| 338 | + * .. code-block:: bash | |
| 339 | + * | |
| 340 | + * echo 1 > ../ras/ras_eeprom_reset | |
| 341 | + * | |
| 342 | + * will reset EEPROM table to 0 entries. | |
| 343 | + * | |
| 344 | + */ | |
| 345 | +static ssize_t amdgpu_ras_debugfs_eeprom_write(struct file *f, const char __user *buf, | |
| 346 | + size_t size, loff_t *pos) | |
| 347 | +{ | |
| 348 | + struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(f)->i_private; | |
| 349 | + int ret; | |
| 350 | + | |
| 351 | + ret = amdgpu_ras_eeprom_reset_table(&adev->psp.ras.ras->eeprom_control); | |
| 352 | + | |
| 353 | + return ret == 1 ? size : -EIO; | |
| 354 | +} | |
| 355 | + | |
| 293 | 356 | static const struct file_operations amdgpu_ras_debugfs_ctrl_ops = { |
| 294 | 357 | .owner = THIS_MODULE, |
| 295 | 358 | .read = NULL, |
| @@ -297,6 +360,34 @@ static const struct file_operations amdgpu_ras_debugfs_ctrl_ops = { · 1 unmodified line | ||
| 297 | 360 | .llseek = default_llseek |
| 298 | 361 | }; |
| 299 | 362 | |
| 363 | +static const struct file_operations amdgpu_ras_debugfs_eeprom_ops = { | |
| 364 | + .owner = THIS_MODULE, | |
| 365 | + .read = NULL, | |
| 366 | + .write = amdgpu_ras_debugfs_eeprom_write, | |
| 367 | + .llseek = default_llseek | |
| 368 | +}; | |
| 369 | + | |
| 370 | +/** | |
| 371 | + * DOC: AMDGPU RAS sysfs Error Count Interface | |
| 372 | + * | |
| 373 | + * It allows the user to read the error count for each IP block on the gpu through | |
| 374 | + * /sys/class/drm/card[0/1/2...]/device/ras/[gfx/sdma/...]_err_count | |
| 375 | + * | |
| 376 | + * It outputs the multiple lines which report the uncorrected (ue) and corrected | |
| 377 | + * (ce) error counts. | |
| 378 | + * | |
| 379 | + * The format of one line is below, | |
| 380 | + * | |
| 381 | + * [ce|ue]: count | |
| 382 | + * | |
| 383 | + * Example: | |
| 384 | + * | |
| 385 | + * .. code-block:: bash | |
| 386 | + * | |
| 387 | + * ue: 0 | |
| 388 | + * ce: 1 | |
| 389 | + * | |
| 390 | + */ | |
| 300 | 391 | static ssize_t amdgpu_ras_sysfs_read(struct device *dev, |
| 301 | 392 | struct device_attribute *attr, char *buf) |
| 302 | 393 | { |
| @@ -475,15 +566,17 @@ int amdgpu_ras_feature_enable(struct amdgpu_device *adev, · 172 unmodified lines | ||
| 475 | 566 | if (!(!!enable ^ !!amdgpu_ras_is_feature_enabled(adev, head))) |
| 476 | 567 | return 0; |
| 477 | 568 | |
| 478 | − ret = psp_ras_enable_features(&adev->psp, &info, enable); | |
| 479 | − if (ret) { | |
| 480 | − DRM_ERROR("RAS ERROR: %s %s feature failed ret %d\n", | |
| 481 | − enable ? "enable":"disable", | |
| 482 | − ras_block_str(head->block), | |
| 483 | − ret); | |
| 484 | − if (ret == TA_RAS_STATUS__RESET_NEEDED) | |
| 485 | − return -EAGAIN; | |
| 486 | − return -EINVAL; | |
| 569 | + if (!amdgpu_ras_intr_triggered()) { | |
| 570 | + ret = psp_ras_enable_features(&adev->psp, &info, enable); | |
| 571 | + if (ret) { | |
| 572 | + DRM_ERROR("RAS ERROR: %s %s feature failed ret %d\n", | |
| 573 | + enable ? "enable":"disable", | |
| 574 | + ras_block_str(head->block), | |
| 575 | + ret); | |
| 576 | + if (ret == TA_RAS_STATUS__RESET_NEEDED) | |
| 577 | + return -EAGAIN; | |
| 578 | + return -EINVAL; | |
| 579 | + } | |
| 487 | 580 | } |
| 488 | 581 | |
| 489 | 582 | /* setup the obj */ |
| @@ -615,8 +708,12 @@ int amdgpu_ras_error_query(struct amdgpu_device *adev, · 125 unmodified lines | ||
| 615 | 708 | adev->gfx.funcs->query_ras_error_count(adev, &err_data); |
| 616 | 709 | break; |
| 617 | 710 | case AMDGPU_RAS_BLOCK__MMHUB: |
| 618 | − if (adev->mmhub_funcs->query_ras_error_count) | |
| 619 | − adev->mmhub_funcs->query_ras_error_count(adev, &err_data); | |
| 711 | + if (adev->mmhub.funcs->query_ras_error_count) | |
| 712 | + adev->mmhub.funcs->query_ras_error_count(adev, &err_data); | |
| 713 | + break; | |
| 714 | + case AMDGPU_RAS_BLOCK__PCIE_BIF: | |
| 715 | + if (adev->nbio.funcs->query_ras_error_count) | |
| 716 | + adev->nbio.funcs->query_ras_error_count(adev, &err_data); | |
| 620 | 717 | break; |
| 621 | 718 | default: |
| 622 | 719 | break; |
| @@ -628,12 +725,14 @@ int amdgpu_ras_error_query(struct amdgpu_device *adev, · 5 unmodified lines | ||
| 628 | 725 | info->ue_count = obj->err_data.ue_count; |
| 629 | 726 | info->ce_count = obj->err_data.ce_count; |
| 630 | 727 | |
| 631 | − if (err_data.ce_count) | |
| 728 | + if (err_data.ce_count) { | |
| 632 | 729 | dev_info(adev->dev, "%ld correctable errors detected in %s block\n", |
| 633 | 730 | obj->err_data.ce_count, ras_block_str(info->head.block)); |
| 634 | − if (err_data.ue_count) | |
| 731 | + } | |
| 732 | + if (err_data.ue_count) { | |
| 635 | 733 | dev_info(adev->dev, "%ld uncorrectable errors detected in %s block\n", |
| 636 | 734 | obj->err_data.ue_count, ras_block_str(info->head.block)); |
| 735 | + } | |
| 637 | 736 | |
| 638 | 737 | return 0; |
| 639 | 738 | } |
| @@ -664,6 +763,8 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 24 unmodified lines | ||
| 664 | 763 | break; |
| 665 | 764 | case AMDGPU_RAS_BLOCK__UMC: |
| 666 | 765 | case AMDGPU_RAS_BLOCK__MMHUB: |
| 766 | + case AMDGPU_RAS_BLOCK__XGMI_WAFL: | |
| 767 | + case AMDGPU_RAS_BLOCK__PCIE_BIF: | |
| 667 | 768 | ret = psp_ras_trigger_error(&adev->psp, &block_info); |
| 668 | 769 | break; |
| 669 | 770 | default: |
| @@ -723,18 +824,18 @@ static int amdgpu_ras_badpages_read(struct amdgpu_device *adev, · 53 unmodified lines | ||
| 723 | 824 | static char *amdgpu_ras_badpage_flags_str(unsigned int flags) |
| 724 | 825 | { |
| 725 | 826 | switch (flags) { |
| 726 | − case 0: | |
| 827 | + case AMDGPU_RAS_RETIRE_PAGE_RESERVED: | |
| 727 | 828 | return "R"; |
| 728 | − case 1: | |
| 829 | + case AMDGPU_RAS_RETIRE_PAGE_PENDING: | |
| 729 | 830 | return "P"; |
| 730 | − case 2: | |
| 831 | + case AMDGPU_RAS_RETIRE_PAGE_FAULT: | |
| 731 | 832 | default: |
| 732 | 833 | return "F"; |
| 733 | 834 | }; |
| 734 | 835 | } |
| 735 | 836 | |
| 736 | −/* | |
| 737 | − * DOC: ras sysfs gpu_vram_bad_pages interface | |
| 837 | +/** | |
| 838 | + * DOC: AMDGPU RAS sysfs gpu_vram_bad_pages Interface | |
| 738 | 839 | * |
| 739 | 840 | * It allows user to read the bad pages of vram on the gpu through |
| 740 | 841 | * /sys/class/drm/card[0/1/2...]/device/ras/gpu_vram_bad_pages |
| @@ -746,14 +847,21 @@ static char *amdgpu_ras_badpage_flags_str(unsigned int flags) · 5 unmodified lines | ||
| 746 | 847 | * |
| 747 | 848 | * gpu pfn and gpu page size are printed in hex format. |
| 748 | 849 | * flags can be one of below character, |
| 850 | + * | |
| 749 | 851 | * R: reserved, this gpu page is reserved and not able to use. |
| 852 | + * | |
| 750 | 853 | * P: pending for reserve, this gpu page is marked as bad, will be reserved |
| 751 | − * in next window of page_reserve. | |
| 854 | + * in next window of page_reserve. | |
| 855 | + * | |
| 752 | 856 | * F: unable to reserve. this gpu page can't be reserved due to some reasons. |
| 753 | 857 | * |
| 754 | − * examples: | |
| 755 | − * 0x00000001 : 0x00001000 : R | |
| 756 | − * 0x00000002 : 0x00001000 : P | |
| 858 | + * Examples: | |
| 859 | + * | |
| 860 | + * .. code-block:: bash | |
| 861 | + * | |
| 862 | + * 0x00000001 : 0x00001000 : R | |
| 863 | + * 0x00000002 : 0x00001000 : P | |
| 864 | + * | |
| 757 | 865 | */ |
| 758 | 866 | |
| 759 | 867 | static ssize_t amdgpu_ras_sysfs_badpages_read(struct file *f, |
| @@ -927,6 +1035,24 @@ static int amdgpu_ras_sysfs_remove_all(struct amdgpu_device *adev) · 167 unmodified lines | ||
| 927 | 1035 | } |
| 928 | 1036 | /* sysfs end */ |
| 929 | 1037 | |
| 1038 | +/** | |
| 1039 | + * DOC: AMDGPU RAS Reboot Behavior for Unrecoverable Errors | |
| 1040 | + * | |
| 1041 | + * Normally when there is an uncorrectable error, the driver will reset | |
| 1042 | + * the GPU to recover. However, in the event of an unrecoverable error, | |
| 1043 | + * the driver provides an interface to reboot the system automatically | |
| 1044 | + * in that event. | |
| 1045 | + * | |
| 1046 | + * The following file in debugfs provides that interface: | |
| 1047 | + * /sys/kernel/debug/dri/[0/1/2...]/ras/auto_reboot | |
| 1048 | + * | |
| 1049 | + * Usage: | |
| 1050 | + * | |
| 1051 | + * .. code-block:: bash | |
| 1052 | + * | |
| 1053 | + * echo true > .../ras/auto_reboot | |
| 1054 | + * | |
| 1055 | + */ | |
| 930 | 1056 | /* debugfs begin */ |
| 931 | 1057 | static void amdgpu_ras_debugfs_create_ctrl_node(struct amdgpu_device *adev) |
| 932 | 1058 | { |
| @@ -934,8 +1060,21 @@ static void amdgpu_ras_debugfs_create_ctrl_node(struct amdgpu_device *adev) · 1 unmodified line | ||
| 934 | 1060 | struct drm_minor *minor = adev->ddev->primary; |
| 935 | 1061 | |
| 936 | 1062 | con->dir = debugfs_create_dir("ras", minor->debugfs_root); |
| 937 | − con->ent = debugfs_create_file("ras_ctrl", S_IWUGO | S_IRUGO, con->dir, | |
| 938 | − adev, &amdgpu_ras_debugfs_ctrl_ops); | |
| 1063 | + debugfs_create_file("ras_ctrl", S_IWUGO | S_IRUGO, con->dir, | |
| 1064 | + adev, &amdgpu_ras_debugfs_ctrl_ops); | |
| 1065 | + debugfs_create_file("ras_eeprom_reset", S_IWUGO | S_IRUGO, con->dir, | |
| 1066 | + adev, &amdgpu_ras_debugfs_eeprom_ops); | |
| 1067 | + | |
| 1068 | + /* | |
| 1069 | + * After one uncorrectable error happens, usually GPU recovery will | |
| 1070 | + * be scheduled. But due to the known problem in GPU recovery failing | |
| 1071 | + * to bring GPU back, below interface provides one direct way to | |
| 1072 | + * user to reboot system automatically in such case within | |
| 1073 | + * ERREVENT_ATHUB_INTERRUPT generated. Normal GPU recovery routine | |
| 1074 | + * will never be called. | |
| 1075 | + */ | |
| 1076 | + debugfs_create_bool("auto_reboot", S_IWUGO | S_IRUGO, con->dir, | |
| 1077 | + &con->reboot); | |
| 939 | 1078 | } |
| 940 | 1079 | |
| 941 | 1080 | void amdgpu_ras_debugfs_create(struct amdgpu_device *adev, |
| @@ -980,10 +1119,8 @@ static void amdgpu_ras_debugfs_remove_all(struct amdgpu_device *adev) · 38 unmodified lines | ||
| 980 | 1119 | amdgpu_ras_debugfs_remove(adev, &obj->head); |
| 981 | 1120 | } |
| 982 | 1121 | |
| 983 | − debugfs_remove(con->ent); | |
| 984 | − debugfs_remove(con->dir); | |
| 1122 | + debugfs_remove_recursive(con->dir); | |
| 985 | 1123 | con->dir = NULL; |
| 986 | − con->ent = NULL; | |
| 987 | 1124 | } |
| 988 | 1125 | /* debugfs end */ |
| 989 | 1126 | |
| @@ -1188,15 +1325,15 @@ static int amdgpu_ras_badpages_read(struct amdgpu_device *adev, · 198 unmodified lines | ||
| 1188 | 1325 | |
| 1189 | 1326 | for (; i < data->count; i++) { |
| 1190 | 1327 | (*bps)[i] = (struct ras_badpage){ |
| 1191 | − .bp = data->bps[i].bp, | |
| 1328 | + .bp = data->bps[i].retired_page, | |
| 1192 | 1329 | .size = AMDGPU_GPU_PAGE_SIZE, |
| 1193 | − .flags = 0, | |
| 1330 | + .flags = AMDGPU_RAS_RETIRE_PAGE_RESERVED, | |
| 1194 | 1331 | }; |
| 1195 | 1332 | |
| 1196 | 1333 | if (data->last_reserved <= i) |
| 1197 | − (*bps)[i].flags = 1; | |
| 1198 | − else if (data->bps[i].bo == NULL) | |
| 1199 | − (*bps)[i].flags = 2; | |
| 1334 | + (*bps)[i].flags = AMDGPU_RAS_RETIRE_PAGE_PENDING; | |
| 1335 | + else if (data->bps_bo[i] == NULL) | |
| 1336 | + (*bps)[i].flags = AMDGPU_RAS_RETIRE_PAGE_FAULT; | |
| 1200 | 1337 | } |
| 1201 | 1338 | |
| 1202 | 1339 | *count = data->count; |
| @@ -1214,105 +1351,46 @@ static void amdgpu_ras_do_recovery(struct work_struct *work) · 11 unmodified lines | ||
| 1214 | 1351 | atomic_set(&ras->in_recovery, 0); |
| 1215 | 1352 | } |
| 1216 | 1353 | |
| 1217 | −static int amdgpu_ras_release_vram(struct amdgpu_device *adev, | |
| 1218 | − struct amdgpu_bo **bo_ptr) | |
| 1219 | −{ | |
| 1220 | − /* no need to free it actually. */ | |
| 1221 | − amdgpu_bo_free_kernel(bo_ptr, NULL, NULL); | |
| 1222 | − return 0; | |
| 1223 | −} | |
| 1224 | − | |
| 1225 | −/* reserve vram with size@offset */ | |
| 1226 | −static int amdgpu_ras_reserve_vram(struct amdgpu_device *adev, | |
| 1227 | − uint64_t offset, uint64_t size, | |
| 1228 | − struct amdgpu_bo **bo_ptr) | |
| 1229 | −{ | |
| 1230 | − struct ttm_operation_ctx ctx = { false, false }; | |
| 1231 | − struct amdgpu_bo_param bp; | |
| 1232 | − int r = 0; | |
| 1233 | − int i; | |
| 1234 | − struct amdgpu_bo *bo; | |
| 1235 | − | |
| 1236 | − if (bo_ptr) | |
| 1237 | − *bo_ptr = NULL; | |
| 1238 | − memset(&bp, 0, sizeof(bp)); | |
| 1239 | − bp.size = size; | |
| 1240 | − bp.byte_align = PAGE_SIZE; | |
| 1241 | − bp.domain = AMDGPU_GEM_DOMAIN_VRAM; | |
| 1242 | − bp.flags = AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS | | |
| 1243 | − AMDGPU_GEM_CREATE_NO_CPU_ACCESS; | |
| 1244 | − bp.type = ttm_bo_type_kernel; | |
| 1245 | − bp.resv = NULL; | |
| 1246 | − | |
| 1247 | − r = amdgpu_bo_create(adev, &bp, &bo); | |
| 1248 | − if (r) | |
| 1249 | − return -EINVAL; | |
| 1250 | − | |
| 1251 | − r = amdgpu_bo_reserve(bo, false); | |
| 1252 | − if (r) | |
| 1253 | − goto error_reserve; | |
| 1254 | − | |
| 1255 | − offset = ALIGN(offset, PAGE_SIZE); | |
| 1256 | − for (i = 0; i < bo->placement.num_placement; ++i) { | |
| 1257 | − bo->placements[i].fpfn = offset >> PAGE_SHIFT; | |
| 1258 | − bo->placements[i].lpfn = (offset + size) >> PAGE_SHIFT; | |
| 1259 | − } | |
| 1260 | − | |
| 1261 | − ttm_bo_mem_put(&bo->tbo, &bo->tbo.mem); | |
| 1262 | − r = ttm_bo_mem_space(&bo->tbo, &bo->placement, &bo->tbo.mem, &ctx); | |
| 1263 | − if (r) | |
| 1264 | − goto error_pin; | |
| 1265 | − | |
| 1266 | − r = amdgpu_bo_pin_restricted(bo, | |
| 1267 | − AMDGPU_GEM_DOMAIN_VRAM, | |
| 1268 | − offset, | |
| 1269 | − offset + size); | |
| 1270 | − if (r) | |
| 1271 | − goto error_pin; | |
| 1272 | − | |
| 1273 | − if (bo_ptr) | |
| 1274 | − *bo_ptr = bo; | |
| 1275 | − | |
| 1276 | − amdgpu_bo_unreserve(bo); | |
| 1277 | − return r; | |
| 1278 | − | |
| 1279 | −error_pin: | |
| 1280 | − amdgpu_bo_unreserve(bo); | |
| 1281 | −error_reserve: | |
| 1282 | − amdgpu_bo_unref(&bo); | |
| 1283 | − return r; | |
| 1284 | −} | |
| 1285 | − | |
| 1286 | 1354 | /* alloc/realloc bps array */ |
| 1287 | 1355 | static int amdgpu_ras_realloc_eh_data_space(struct amdgpu_device *adev, |
| 1288 | 1356 | struct ras_err_handler_data *data, int pages) |
| 1289 | 1357 | { |
| 1290 | 1358 | unsigned int old_space = data->count + data->space_left; |
| 1291 | 1359 | unsigned int new_space = old_space + pages; |
| 1292 | − unsigned int align_space = ALIGN(new_space, 1024); | |
| 1293 | − void *tmp = kmalloc(align_space * sizeof(*data->bps), GFP_KERNEL); | |
| 1294 | − | |
| 1295 | − if (!tmp) | |
| 1360 | + unsigned int align_space = ALIGN(new_space, 512); | |
| 1361 | + void *bps = kmalloc(align_space * sizeof(*data->bps), GFP_KERNEL); | |
| 1362 | + struct amdgpu_bo **bps_bo = | |
| 1363 | + kmalloc(align_space * sizeof(*data->bps_bo), GFP_KERNEL); | |
| 1364 | + | |
| 1365 | + if (!bps || !bps_bo) { | |
| 1366 | + kfree(bps); | |
| 1367 | + kfree(bps_bo); | |
| 1296 | 1368 | return -ENOMEM; |
| 1369 | + } | |
| 1297 | 1370 | |
| 1298 | 1371 | if (data->bps) { |
| 1299 | − memcpy(tmp, data->bps, | |
| 1372 | + memcpy(bps, data->bps, | |
| 1300 | 1373 | data->count * sizeof(*data->bps)); |
| 1301 | 1374 | kfree(data->bps); |
| 1302 | 1375 | } |
| 1376 | + if (data->bps_bo) { | |
| 1377 | + memcpy(bps_bo, data->bps_bo, | |
| 1378 | + data->count * sizeof(*data->bps_bo)); | |
| 1379 | + kfree(data->bps_bo); | |
| 1380 | + } | |
| 1303 | 1381 | |
| 1304 | − data->bps = tmp; | |
| 1382 | + data->bps = bps; | |
| 1383 | + data->bps_bo = bps_bo; | |
| 1305 | 1384 | data->space_left += align_space - old_space; |
| 1306 | 1385 | return 0; |
| 1307 | 1386 | } |
| 1308 | 1387 | |
| 1309 | 1388 | /* it deal with vram only. */ |
| 1310 | 1389 | int amdgpu_ras_add_bad_pages(struct amdgpu_device *adev, |
| 1311 | − unsigned long *bps, int pages) | |
| 1390 | + struct eeprom_table_record *bps, int pages) | |
| 1312 | 1391 | { |
| 1313 | 1392 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); |
| 1314 | 1393 | struct ras_err_handler_data *data; |
| 1315 | − int i = pages; | |
| 1316 | 1394 | int ret = 0; |
| 1317 | 1395 | |
| 1318 | 1396 | if (!con || !con->eh_data || !bps || pages <= 0) |
| @@ -1329,24 +1407,120 @@ int amdgpu_ras_add_bad_pages(struct amdgpu_device *adev, · 10 unmodified lines | ||
| 1329 | 1407 | goto out; |
| 1330 | 1408 | } |
| 1331 | 1409 | |
| 1332 | − while (i--) | |
| 1333 | − data->bps[data->count++].bp = bps[i]; | |
| 1334 | − | |
| 1410 | + memcpy(&data->bps[data->count], bps, pages * sizeof(*data->bps)); | |
| 1411 | + data->count += pages; | |
| 1335 | 1412 | data->space_left -= pages; |
| 1413 | + | |
| 1336 | 1414 | out: |
| 1337 | 1415 | mutex_unlock(&con->recovery_lock); |
| 1338 | 1416 | |
| 1339 | 1417 | return ret; |
| 1340 | 1418 | } |
| 1341 | 1419 | |
| 1420 | +/* | |
| 1421 | + * write error record array to eeprom, the function should be | |
| 1422 | + * protected by recovery_lock | |
| 1423 | + */ | |
| 1424 | +static int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev) | |
| 1425 | +{ | |
| 1426 | + struct amdgpu_ras *con = amdgpu_ras_get_context(adev); | |
| 1427 | + struct ras_err_handler_data *data; | |
| 1428 | + struct amdgpu_ras_eeprom_control *control; | |
| 1429 | + int save_count; | |
| 1430 | + | |
| 1431 | + if (!con || !con->eh_data) | |
| 1432 | + return 0; | |
| 1433 | + | |
| 1434 | + control = &con->eeprom_control; | |
| 1435 | + data = con->eh_data; | |
| 1436 | + save_count = data->count - control->num_recs; | |
| 1437 | + /* only new entries are saved */ | |
| 1438 | + if (save_count > 0) | |
| 1439 | + if (amdgpu_ras_eeprom_process_recods(control, | |
| 1440 | + &data->bps[control->num_recs], | |
| 1441 | + true, | |
| 1442 | + save_count)) { | |
| 1443 | + DRM_ERROR("Failed to save EEPROM table data!"); | |
| 1444 | + return -EIO; | |
| 1445 | + } | |
| 1446 | + | |
| 1447 | + return 0; | |
| 1448 | +} | |
| 1449 | + | |
| 1450 | +/* | |
| 1451 | + * read error record array in eeprom and reserve enough space for | |
| 1452 | + * storing new bad pages | |
| 1453 | + */ | |
| 1454 | +static int amdgpu_ras_load_bad_pages(struct amdgpu_device *adev) | |
| 1455 | +{ | |
| 1456 | + struct amdgpu_ras_eeprom_control *control = | |
| 1457 | + &adev->psp.ras.ras->eeprom_control; | |
| 1458 | + struct eeprom_table_record *bps = NULL; | |
| 1459 | + int ret = 0; | |
| 1460 | + | |
| 1461 | + /* no bad page record, skip eeprom access */ | |
| 1462 | + if (!control->num_recs) | |
| 1463 | + return ret; | |
| 1464 | + | |
| 1465 | + bps = kcalloc(control->num_recs, sizeof(*bps), GFP_KERNEL); | |
| 1466 | + if (!bps) | |
| 1467 | + return -ENOMEM; | |
| 1468 | + | |
| 1469 | + if (amdgpu_ras_eeprom_process_recods(control, bps, false, | |
| 1470 | + control->num_recs)) { | |
| 1471 | + DRM_ERROR("Failed to load EEPROM table records!"); | |
| 1472 | + ret = -EIO; | |
| 1473 | + goto out; | |
| 1474 | + } | |
| 1475 | + | |
| 1476 | + ret = amdgpu_ras_add_bad_pages(adev, bps, control->num_recs); | |
| 1477 | + | |
| 1478 | +out: | |
| 1479 | + kfree(bps); | |
| 1480 | + return ret; | |
| 1481 | +} | |
| 1482 | + | |
| 1483 | +/* | |
| 1484 | + * check if an address belongs to bad page | |
| 1485 | + * | |
| 1486 | + * Note: this check is only for umc block | |
| 1487 | + */ | |
| 1488 | +static bool amdgpu_ras_check_bad_page(struct amdgpu_device *adev, | |
| 1489 | + uint64_t addr) | |
| 1490 | +{ | |
| 1491 | + struct amdgpu_ras *con = amdgpu_ras_get_context(adev); | |
| 1492 | + struct ras_err_handler_data *data; | |
| 1493 | + int i; | |
| 1494 | + bool ret = false; | |
| 1495 | + | |
| 1496 | + if (!con || !con->eh_data) | |
| 1497 | + return ret; | |
| 1498 | + | |
| 1499 | + mutex_lock(&con->recovery_lock); | |
| 1500 | + data = con->eh_data; | |
| 1501 | + if (!data) | |
| 1502 | + goto out; | |
| 1503 | + | |
| 1504 | + addr >>= AMDGPU_GPU_PAGE_SHIFT; | |
| 1505 | + for (i = 0; i < data->count; i++) | |
| 1506 | + if (addr == data->bps[i].retired_page) { | |
| 1507 | + ret = true; | |
| 1508 | + goto out; | |
| 1509 | + } | |
| 1510 | + | |
| 1511 | +out: | |
| 1512 | + mutex_unlock(&con->recovery_lock); | |
| 1513 | + return ret; | |
| 1514 | +} | |
| 1515 | + | |
| 1342 | 1516 | /* called in gpu recovery/init */ |
| 1343 | 1517 | int amdgpu_ras_reserve_bad_pages(struct amdgpu_device *adev) |
| 1344 | 1518 | { |
| 1345 | 1519 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); |
| 1346 | 1520 | struct ras_err_handler_data *data; |
| 1347 | 1521 | uint64_t bp; |
| 1348 | − struct amdgpu_bo *bo; | |
| 1349 | − int i; | |
| 1522 | + struct amdgpu_bo *bo = NULL; | |
| 1523 | + int i, ret = 0; | |
| 1350 | 1524 | |
| 1351 | 1525 | if (!con || !con->eh_data) |
| 1352 | 1526 | return 0; |
| @@ -1357,18 +1531,29 @@ int amdgpu_ras_reserve_bad_pages(struct amdgpu_device *adev) · 4 unmodified lines | ||
| 1357 | 1531 | goto out; |
| 1358 | 1532 | /* reserve vram at driver post stage. */ |
| 1359 | 1533 | for (i = data->last_reserved; i < data->count; i++) { |
| 1360 | − bp = data->bps[i].bp; | |
| 1534 | + bp = data->bps[i].retired_page; | |
| 1361 | 1535 | |
| 1362 | − if (amdgpu_ras_reserve_vram(adev, bp << PAGE_SHIFT, | |
| 1363 | − PAGE_SIZE, &bo)) | |
| 1364 | − DRM_ERROR("RAS ERROR: reserve vram %llx fail\n", bp); | |
| 1536 | + /* There are two cases of reserve error should be ignored: | |
| 1537 | + * 1) a ras bad page has been allocated (used by someone); | |
| 1538 | + * 2) a ras bad page has been reserved (duplicate error injection | |
| 1539 | + * for one page); | |
| 1540 | + */ | |
| 1541 | + if (amdgpu_bo_create_kernel_at(adev, bp << AMDGPU_GPU_PAGE_SHIFT, | |
| 1542 | + AMDGPU_GPU_PAGE_SIZE, | |
| 1543 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 1544 | + &bo, NULL)) | |
| 1545 | + DRM_WARN("RAS WARN: reserve vram for retired page %llx fail\n", bp); | |
| 1365 | 1546 | |
| 1366 | − data->bps[i].bo = bo; | |
| 1547 | + data->bps_bo[i] = bo; | |
| 1367 | 1548 | data->last_reserved = i + 1; |
| 1549 | + bo = NULL; | |
| 1368 | 1550 | } |
| 1551 | + | |
| 1552 | + /* continue to save bad pages to eeprom even reesrve_vram fails */ | |
| 1553 | + ret = amdgpu_ras_save_bad_pages(adev); | |
| 1369 | 1554 | out: |
| 1370 | 1555 | mutex_unlock(&con->recovery_lock); |
| 1371 | − return 0; | |
| 1556 | + return ret; | |
| 1372 | 1557 | } |
| 1373 | 1558 | |
| 1374 | 1559 | /* called when driver unload */ |
| @@ -1388,11 +1573,11 @@ static int amdgpu_ras_release_bad_pages(struct amdgpu_device *adev) · 13 unmodified lines | ||
| 1388 | 1573 | goto out; |
| 1389 | 1574 | |
| 1390 | 1575 | for (i = data->last_reserved - 1; i >= 0; i--) { |
| 1391 | − bo = data->bps[i].bo; | |
| 1576 | + bo = data->bps_bo[i]; | |
| 1392 | 1577 | |
| 1393 | − amdgpu_ras_release_vram(adev, &bo); | |
| 1578 | + amdgpu_bo_free_kernel(&bo, NULL, NULL); | |
| 1394 | 1579 | |
| 1395 | − data->bps[i].bo = bo; | |
| 1580 | + data->bps_bo[i] = bo; | |
| 1396 | 1581 | data->last_reserved = i; |
| 1397 | 1582 | } |
| 1398 | 1583 | out: |
| @@ -1400,41 +1585,54 @@ out: · 1 unmodified line | ||
| 1400 | 1585 | return 0; |
| 1401 | 1586 | } |
| 1402 | 1587 | |
| 1403 | −static int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev) | |
| 1404 | −{ | |
| 1405 | − /* TODO | |
| 1406 | − * write the array to eeprom when SMU disabled. | |
| 1407 | − */ | |
| 1408 | − return 0; | |
| 1409 | −} | |
| 1410 | − | |
| 1411 | −static int amdgpu_ras_load_bad_pages(struct amdgpu_device *adev) | |
| 1412 | −{ | |
| 1413 | − /* TODO | |
| 1414 | − * read the array to eeprom when SMU disabled. | |
| 1415 | − */ | |
| 1416 | − return 0; | |
| 1417 | −} | |
| 1418 | − | |
| 1419 | −static int amdgpu_ras_recovery_init(struct amdgpu_device *adev) | |
| 1588 | +int amdgpu_ras_recovery_init(struct amdgpu_device *adev) | |
| 1420 | 1589 | { |
| 1421 | 1590 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); |
| 1422 | − struct ras_err_handler_data **data = &con->eh_data; | |
| 1591 | + struct ras_err_handler_data **data; | |
| 1592 | + int ret; | |
| 1423 | 1593 | |
| 1424 | − *data = kmalloc(sizeof(**data), | |
| 1425 | − GFP_KERNEL|__GFP_ZERO); | |
| 1426 | − if (!*data) | |
| 1427 | − return -ENOMEM; | |
| 1594 | + if (con) | |
| 1595 | + data = &con->eh_data; | |
| 1596 | + else | |
| 1597 | + return 0; | |
| 1598 | + | |
| 1599 | + *data = kmalloc(sizeof(**data), GFP_KERNEL | __GFP_ZERO); | |
| 1600 | + if (!*data) { | |
| 1601 | + ret = -ENOMEM; | |
| 1602 | + goto out; | |
| 1603 | + } | |
| 1428 | 1604 | |
| 1429 | 1605 | mutex_init(&con->recovery_lock); |
| 1430 | 1606 | INIT_WORK(&con->recovery_work, amdgpu_ras_do_recovery); |
| 1431 | 1607 | atomic_set(&con->in_recovery, 0); |
| 1432 | 1608 | con->adev = adev; |
| 1433 | 1609 | |
| 1434 | − amdgpu_ras_load_bad_pages(adev); | |
| 1435 | − amdgpu_ras_reserve_bad_pages(adev); | |
| 1610 | + ret = amdgpu_ras_eeprom_init(&con->eeprom_control); | |
| 1611 | + if (ret) | |
| 1612 | + goto free; | |
| 1613 | + | |
| 1614 | + if (con->eeprom_control.num_recs) { | |
| 1615 | + ret = amdgpu_ras_load_bad_pages(adev); | |
| 1616 | + if (ret) | |
| 1617 | + goto free; | |
| 1618 | + ret = amdgpu_ras_reserve_bad_pages(adev); | |
| 1619 | + if (ret) | |
| 1620 | + goto release; | |
| 1621 | + } | |
| 1436 | 1622 | |
| 1437 | 1623 | return 0; |
| 1624 | + | |
| 1625 | +release: | |
| 1626 | + amdgpu_ras_release_bad_pages(adev); | |
| 1627 | +free: | |
| 1628 | + kfree((*data)->bps); | |
| 1629 | + kfree((*data)->bps_bo); | |
| 1630 | + kfree(*data); | |
| 1631 | + con->eh_data = NULL; | |
| 1632 | +out: | |
| 1633 | + DRM_WARN("Failed to initialize ras recovery!\n"); | |
| 1634 | + | |
| 1635 | + return ret; | |
| 1438 | 1636 | } |
| 1439 | 1637 | |
| 1440 | 1638 | static int amdgpu_ras_recovery_fini(struct amdgpu_device *adev) |
| @@ -1442,13 +1640,17 @@ static int amdgpu_ras_recovery_fini(struct amdgpu_device *adev) · 1 unmodified line | ||
| 1442 | 1640 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); |
| 1443 | 1641 | struct ras_err_handler_data *data = con->eh_data; |
| 1444 | 1642 | |
| 1643 | + /* recovery_init failed to init it, fini is useless */ | |
| 1644 | + if (!data) | |
| 1645 | + return 0; | |
| 1646 | + | |
| 1445 | 1647 | cancel_work_sync(&con->recovery_work); |
| 1446 | − amdgpu_ras_save_bad_pages(adev); | |
| 1447 | 1648 | amdgpu_ras_release_bad_pages(adev); |
| 1448 | 1649 | |
| 1449 | 1650 | mutex_lock(&con->recovery_lock); |
| 1450 | 1651 | con->eh_data = NULL; |
| 1451 | 1652 | kfree(data->bps); |
| 1653 | + kfree(data->bps_bo); | |
| 1452 | 1654 | kfree(data); |
| 1453 | 1655 | mutex_unlock(&con->recovery_lock); |
| 1454 | 1656 | |
| @@ -1500,6 +1702,7 @@ static void amdgpu_ras_check_supported(struct amdgpu_device *adev, · 45 unmodified lines | ||
| 1500 | 1702 | int amdgpu_ras_init(struct amdgpu_device *adev) |
| 1501 | 1703 | { |
| 1502 | 1704 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); |
| 1705 | + int r; | |
| 1503 | 1706 | |
| 1504 | 1707 | if (con) |
| 1505 | 1708 | return 0; |
| @@ -1527,31 +1730,106 @@ int amdgpu_ras_init(struct amdgpu_device *adev) · 21 unmodified lines | ||
| 1527 | 1730 | /* Might need get this flag from vbios. */ |
| 1528 | 1731 | con->flags = RAS_DEFAULT_FLAGS; |
| 1529 | 1732 | |
| 1530 | − if (amdgpu_ras_recovery_init(adev)) | |
| 1531 | − goto recovery_out; | |
| 1733 | + if (adev->nbio.funcs->init_ras_controller_interrupt) { | |
| 1734 | + r = adev->nbio.funcs->init_ras_controller_interrupt(adev); | |
| 1735 | + if (r) | |
| 1736 | + return r; | |
| 1737 | + } | |
| 1738 | + | |
| 1739 | + if (adev->nbio.funcs->init_ras_err_event_athub_interrupt) { | |
| 1740 | + r = adev->nbio.funcs->init_ras_err_event_athub_interrupt(adev); | |
| 1741 | + if (r) | |
| 1742 | + return r; | |
| 1743 | + } | |
| 1532 | 1744 | |
| 1533 | 1745 | amdgpu_ras_mask &= AMDGPU_RAS_BLOCK_MASK; |
| 1534 | 1746 | |
| 1535 | 1747 | if (amdgpu_ras_fs_init(adev)) |
| 1536 | 1748 | goto fs_out; |
| 1537 | 1749 | |
| 1538 | − /* ras init for each ras block */ | |
| 1539 | − if (adev->umc.funcs->ras_init) | |
| 1540 | − adev->umc.funcs->ras_init(adev); | |
| 1541 | − | |
| 1542 | 1750 | DRM_INFO("RAS INFO: ras initialized successfully, " |
| 1543 | 1751 | "hardware ability[%x] ras_mask[%x]\n", |
| 1544 | 1752 | con->hw_supported, con->supported); |
| 1545 | 1753 | return 0; |
| 1546 | 1754 | fs_out: |
| 1547 | − amdgpu_ras_recovery_fini(adev); | |
| 1548 | −recovery_out: | |
| 1549 | 1755 | amdgpu_ras_set_context(adev, NULL); |
| 1550 | 1756 | kfree(con); |
| 1551 | 1757 | |
| 1552 | 1758 | return -EINVAL; |
| 1553 | 1759 | } |
| 1554 | 1760 | |
| 1761 | +/* helper function to handle common stuff in ip late init phase */ | |
| 1762 | +int amdgpu_ras_late_init(struct amdgpu_device *adev, | |
| 1763 | + struct ras_common_if *ras_block, | |
| 1764 | + struct ras_fs_if *fs_info, | |
| 1765 | + struct ras_ih_if *ih_info) | |
| 1766 | +{ | |
| 1767 | + int r; | |
| 1768 | + | |
| 1769 | + /* disable RAS feature per IP block if it is not supported */ | |
| 1770 | + if (!amdgpu_ras_is_supported(adev, ras_block->block)) { | |
| 1771 | + amdgpu_ras_feature_enable_on_boot(adev, ras_block, 0); | |
| 1772 | + return 0; | |
| 1773 | + } | |
| 1774 | + | |
| 1775 | + r = amdgpu_ras_feature_enable_on_boot(adev, ras_block, 1); | |
| 1776 | + if (r) { | |
| 1777 | + if (r == -EAGAIN) { | |
| 1778 | + /* request gpu reset. will run again */ | |
| 1779 | + amdgpu_ras_request_reset_on_boot(adev, | |
| 1780 | + ras_block->block); | |
| 1781 | + return 0; | |
| 1782 | + } else if (adev->in_suspend || adev->in_gpu_reset) { | |
| 1783 | + /* in resume phase, if fail to enable ras, | |
| 1784 | + * clean up all ras fs nodes, and disable ras */ | |
| 1785 | + goto cleanup; | |
| 1786 | + } else | |
| 1787 | + return r; | |
| 1788 | + } | |
| 1789 | + | |
| 1790 | + /* in resume phase, no need to create ras fs node */ | |
| 1791 | + if (adev->in_suspend || adev->in_gpu_reset) | |
| 1792 | + return 0; | |
| 1793 | + | |
| 1794 | + if (ih_info->cb) { | |
| 1795 | + r = amdgpu_ras_interrupt_add_handler(adev, ih_info); | |
| 1796 | + if (r) | |
| 1797 | + goto interrupt; | |
| 1798 | + } | |
| 1799 | + | |
| 1800 | + amdgpu_ras_debugfs_create(adev, fs_info); | |
| 1801 | + | |
| 1802 | + r = amdgpu_ras_sysfs_create(adev, fs_info); | |
| 1803 | + if (r) | |
| 1804 | + goto sysfs; | |
| 1805 | + | |
| 1806 | + return 0; | |
| 1807 | +cleanup: | |
| 1808 | + amdgpu_ras_sysfs_remove(adev, ras_block); | |
| 1809 | +sysfs: | |
| 1810 | + amdgpu_ras_debugfs_remove(adev, ras_block); | |
| 1811 | + if (ih_info->cb) | |
| 1812 | + amdgpu_ras_interrupt_remove_handler(adev, ih_info); | |
| 1813 | +interrupt: | |
| 1814 | + amdgpu_ras_feature_enable(adev, ras_block, 0); | |
| 1815 | + return r; | |
| 1816 | +} | |
| 1817 | + | |
| 1818 | +/* helper function to remove ras fs node and interrupt handler */ | |
| 1819 | +void amdgpu_ras_late_fini(struct amdgpu_device *adev, | |
| 1820 | + struct ras_common_if *ras_block, | |
| 1821 | + struct ras_ih_if *ih_info) | |
| 1822 | +{ | |
| 1823 | + if (!ras_block || !ih_info) | |
| 1824 | + return; | |
| 1825 | + | |
| 1826 | + amdgpu_ras_sysfs_remove(adev, ras_block); | |
| 1827 | + amdgpu_ras_debugfs_remove(adev, ras_block); | |
| 1828 | + if (ih_info->cb) | |
| 1829 | + amdgpu_ras_interrupt_remove_handler(adev, ih_info); | |
| 1830 | + amdgpu_ras_feature_enable(adev, ras_block, 0); | |
| 1831 | +} | |
| 1832 | + | |
| 1555 | 1833 | /* do some init work after IP late init as dependence. |
| 1556 | 1834 | * and it runs in resume/gpu reset/booting up cases. |
| 1557 | 1835 | */ |
| @@ -1645,3 +1923,18 @@ int amdgpu_ras_fini(struct amdgpu_device *adev) · 87 unmodified lines | ||
| 1645 | 1923 | |
| 1646 | 1924 | return 0; |
| 1647 | 1925 | } |
| 1926 | + | |
| 1927 | +void amdgpu_ras_global_ras_isr(struct amdgpu_device *adev) | |
| 1928 | +{ | |
| 1929 | + uint32_t hw_supported, supported; | |
| 1930 | + | |
| 1931 | + amdgpu_ras_check_supported(adev, &hw_supported, &supported); | |
| 1932 | + if (!hw_supported) | |
| 1933 | + return; | |
| 1934 | + | |
| 1935 | + if (atomic_cmpxchg(&amdgpu_ras_in_intr, 0, 1) == 0) { | |
| 1936 | + DRM_WARN("RAS event of type ERREVENT_ATHUB_INTERRUPT detected!\n"); | |
| 1937 | + | |
| 1938 | + amdgpu_ras_reset_gpu(adev, false); | |
| 1939 | + } | |
| 1940 | +} | |
| @@ -317,8 +317,6 @@ struct amdgpu_ras { · 316 unmodified lines | ||
| 317 | 317 | struct list_head head; |
| 318 | 318 | /* debugfs */ |
| 319 | 319 | struct dentry *dir; |
| 320 | − /* debugfs ctrl */ | |
| 321 | − struct dentry *ent; | |
| 322 | 320 | /* sysfs */ |
| 323 | 321 | struct device_attribute features_attr; |
| 324 | 322 | struct bin_attribute badpages_attr; |
| @@ -334,7 +332,7 @@ struct amdgpu_ras { · 9 unmodified lines | ||
| 334 | 332 | struct mutex recovery_lock; |
| 335 | 333 | |
| 336 | 334 | uint32_t flags; |
| 337 | − | |
| 335 | + bool reboot; | |
| 338 | 336 | struct amdgpu_ras_eeprom_control eeprom_control; |
| 339 | 337 | }; |
| 340 | 338 | |
| @@ -347,15 +345,14 @@ struct ras_err_data { · 6 unmodified lines | ||
| 347 | 345 | unsigned long ue_count; |
| 348 | 346 | unsigned long ce_count; |
| 349 | 347 | unsigned long err_addr_cnt; |
| 350 | − uint64_t *err_addr; | |
| 348 | + struct eeprom_table_record *err_addr; | |
| 351 | 349 | }; |
| 352 | 350 | |
| 353 | 351 | struct ras_err_handler_data { |
| 354 | − /* point to bad pages array */ | |
| 355 | − struct { | |
| 356 | − unsigned long bp; | |
| 357 | − struct amdgpu_bo *bo; | |
| 358 | − } *bps; | |
| 352 | + /* point to bad page records array */ | |
| 353 | + struct eeprom_table_record *bps; | |
| 354 | + /* point to reserved bo array */ | |
| 355 | + struct amdgpu_bo **bps_bo; | |
| 359 | 356 | /* the count of entries */ |
| 360 | 357 | int count; |
| 361 | 358 | /* the space can place new entries */ |
| @@ -365,7 +362,7 @@ struct ras_err_handler_data { · 3 unmodified lines | ||
| 365 | 362 | }; |
| 366 | 363 | |
| 367 | 364 | typedef int (*ras_ih_cb)(struct amdgpu_device *adev, |
| 368 | − struct ras_err_data *err_data, | |
| 365 | + void *err_data, | |
| 369 | 366 | struct amdgpu_iv_entry *entry); |
| 370 | 367 | |
| 371 | 368 | struct ras_ih_data { |
| @@ -481,6 +478,7 @@ static inline int amdgpu_ras_is_supported(struct amdgpu_device *adev, · 109 unmodified lines | ||
| 481 | 478 | return ras && (ras->supported & (1 << block)); |
| 482 | 479 | } |
| 483 | 480 | |
| 481 | +int amdgpu_ras_recovery_init(struct amdgpu_device *adev); | |
| 484 | 482 | int amdgpu_ras_request_reset_on_boot(struct amdgpu_device *adev, |
| 485 | 483 | unsigned int block); |
| 486 | 484 | |
| @@ -492,7 +490,7 @@ unsigned long amdgpu_ras_query_error_count(struct amdgpu_device *adev, · 5 unmodified lines | ||
| 492 | 490 | |
| 493 | 491 | /* error handling functions */ |
| 494 | 492 | int amdgpu_ras_add_bad_pages(struct amdgpu_device *adev, |
| 495 | − unsigned long *bps, int pages); | |
| 493 | + struct eeprom_table_record *bps, int pages); | |
| 496 | 494 | |
| 497 | 495 | int amdgpu_ras_reserve_bad_pages(struct amdgpu_device *adev); |
| 498 | 496 | |
| @@ -501,6 +499,12 @@ static inline int amdgpu_ras_reset_gpu(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 501 | 499 | { |
| 502 | 500 | struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); |
| 503 | 501 | |
| 502 | + /* save bad page to eeprom before gpu reset, | |
| 503 | + * i2c may be unstable in gpu reset | |
| 504 | + */ | |
| 505 | + if (in_task()) | |
| 506 | + amdgpu_ras_reserve_bad_pages(adev); | |
| 507 | + | |
| 504 | 508 | if (atomic_cmpxchg(&ras->in_recovery, 0, 1) == 0) |
| 505 | 509 | schedule_work(&ras->recovery_work); |
| 506 | 510 | return 0; |
| @@ -566,6 +570,13 @@ amdgpu_ras_error_to_ta(enum amdgpu_ras_error_type error) { · 59 unmodified lines | ||
| 566 | 570 | int amdgpu_ras_init(struct amdgpu_device *adev); |
| 567 | 571 | int amdgpu_ras_fini(struct amdgpu_device *adev); |
| 568 | 572 | int amdgpu_ras_pre_fini(struct amdgpu_device *adev); |
| 573 | +int amdgpu_ras_late_init(struct amdgpu_device *adev, | |
| 574 | + struct ras_common_if *ras_block, | |
| 575 | + struct ras_fs_if *fs_info, | |
| 576 | + struct ras_ih_if *ih_info); | |
| 577 | +void amdgpu_ras_late_fini(struct amdgpu_device *adev, | |
| 578 | + struct ras_common_if *ras_block, | |
| 579 | + struct ras_ih_if *ih_info); | |
| 569 | 580 | |
| 570 | 581 | int amdgpu_ras_feature_enable(struct amdgpu_device *adev, |
| 571 | 582 | struct ras_common_if *head, bool enable); |
| @@ -599,4 +610,14 @@ int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, · 27 unmodified lines | ||
| 599 | 610 | |
| 600 | 611 | int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, |
| 601 | 612 | struct ras_dispatch_if *info); |
| 613 | + | |
| 614 | +extern atomic_t amdgpu_ras_in_intr; | |
| 615 | + | |
| 616 | +static inline bool amdgpu_ras_intr_triggered(void) | |
| 617 | +{ | |
| 618 | + return !!atomic_read(&amdgpu_ras_in_intr); | |
| 619 | +} | |
| 620 | + | |
| 621 | +void amdgpu_ras_global_ras_isr(struct amdgpu_device *adev); | |
| 622 | + | |
| 602 | 623 | #endif |
| @@ -100,7 +100,101 @@ static int __update_table_header(struct amdgpu_ras_eeprom_control *control, · 99 unmodified lines | ||
| 100 | 100 | return ret; |
| 101 | 101 | } |
| 102 | 102 | |
| 103 | −static uint32_t __calc_hdr_byte_sum(struct amdgpu_ras_eeprom_control *control); | |
| 103 | + | |
| 104 | + | |
| 105 | +static uint32_t __calc_hdr_byte_sum(struct amdgpu_ras_eeprom_control *control) | |
| 106 | +{ | |
| 107 | + int i; | |
| 108 | + uint32_t tbl_sum = 0; | |
| 109 | + | |
| 110 | + /* Header checksum, skip checksum field in the calculation */ | |
| 111 | + for (i = 0; i < sizeof(control->tbl_hdr) - sizeof(control->tbl_hdr.checksum); i++) | |
| 112 | + tbl_sum += *(((unsigned char *)&control->tbl_hdr) + i); | |
| 113 | + | |
| 114 | + return tbl_sum; | |
| 115 | +} | |
| 116 | + | |
| 117 | +static uint32_t __calc_recs_byte_sum(struct eeprom_table_record *records, | |
| 118 | + int num) | |
| 119 | +{ | |
| 120 | + int i, j; | |
| 121 | + uint32_t tbl_sum = 0; | |
| 122 | + | |
| 123 | + /* Records checksum */ | |
| 124 | + for (i = 0; i < num; i++) { | |
| 125 | + struct eeprom_table_record *record = &records[i]; | |
| 126 | + | |
| 127 | + for (j = 0; j < sizeof(*record); j++) { | |
| 128 | + tbl_sum += *(((unsigned char *)record) + j); | |
| 129 | + } | |
| 130 | + } | |
| 131 | + | |
| 132 | + return tbl_sum; | |
| 133 | +} | |
| 134 | + | |
| 135 | +static inline uint32_t __calc_tbl_byte_sum(struct amdgpu_ras_eeprom_control *control, | |
| 136 | + struct eeprom_table_record *records, int num) | |
| 137 | +{ | |
| 138 | + return __calc_hdr_byte_sum(control) + __calc_recs_byte_sum(records, num); | |
| 139 | +} | |
| 140 | + | |
| 141 | +/* Checksum = 256 -((sum of all table entries) mod 256) */ | |
| 142 | +static void __update_tbl_checksum(struct amdgpu_ras_eeprom_control *control, | |
| 143 | + struct eeprom_table_record *records, int num, | |
| 144 | + uint32_t old_hdr_byte_sum) | |
| 145 | +{ | |
| 146 | + /* | |
| 147 | + * This will update the table sum with new records. | |
| 148 | + * | |
| 149 | + * TODO: What happens when the EEPROM table is to be wrapped around | |
| 150 | + * and old records from start will get overridden. | |
| 151 | + */ | |
| 152 | + | |
| 153 | + /* need to recalculate updated header byte sum */ | |
| 154 | + control->tbl_byte_sum -= old_hdr_byte_sum; | |
| 155 | + control->tbl_byte_sum += __calc_tbl_byte_sum(control, records, num); | |
| 156 | + | |
| 157 | + control->tbl_hdr.checksum = 256 - (control->tbl_byte_sum % 256); | |
| 158 | +} | |
| 159 | + | |
| 160 | +/* table sum mod 256 + checksum must equals 256 */ | |
| 161 | +static bool __validate_tbl_checksum(struct amdgpu_ras_eeprom_control *control, | |
| 162 | + struct eeprom_table_record *records, int num) | |
| 163 | +{ | |
| 164 | + control->tbl_byte_sum = __calc_tbl_byte_sum(control, records, num); | |
| 165 | + | |
| 166 | + if (control->tbl_hdr.checksum + (control->tbl_byte_sum % 256) != 256) { | |
| 167 | + DRM_WARN("Checksum mismatch, checksum: %u ", control->tbl_hdr.checksum); | |
| 168 | + return false; | |
| 169 | + } | |
| 170 | + | |
| 171 | + return true; | |
| 172 | +} | |
| 173 | + | |
| 174 | +int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control) | |
| 175 | +{ | |
| 176 | + unsigned char buff[EEPROM_ADDRESS_SIZE + EEPROM_TABLE_HEADER_SIZE] = { 0 }; | |
| 177 | + struct amdgpu_ras_eeprom_table_header *hdr = &control->tbl_hdr; | |
| 178 | + int ret = 0; | |
| 179 | + | |
| 180 | + mutex_lock(&control->tbl_mutex); | |
| 181 | + | |
| 182 | + hdr->header = EEPROM_TABLE_HDR_VAL; | |
| 183 | + hdr->version = EEPROM_TABLE_VER; | |
| 184 | + hdr->first_rec_offset = EEPROM_RECORD_START; | |
| 185 | + hdr->tbl_size = EEPROM_TABLE_HEADER_SIZE; | |
| 186 | + | |
| 187 | + control->tbl_byte_sum = 0; | |
| 188 | + __update_tbl_checksum(control, NULL, 0, 0); | |
| 189 | + control->next_addr = EEPROM_RECORD_START; | |
| 190 | + | |
| 191 | + ret = __update_table_header(control, buff); | |
| 192 | + | |
| 193 | + mutex_unlock(&control->tbl_mutex); | |
| 194 | + | |
| 195 | + return ret; | |
| 196 | + | |
| 197 | +} | |
| 104 | 198 | |
| 105 | 199 | int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control) |
| 106 | 200 | { |
| @@ -122,6 +216,10 @@ int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control) · 15 unmodified lines | ||
| 122 | 216 | ret = smu_v11_0_i2c_eeprom_control_init(&control->eeprom_accessor); |
| 123 | 217 | break; |
| 124 | 218 | |
| 219 | + case CHIP_ARCTURUS: | |
| 220 | + ret = smu_i2c_eeprom_init(&adev->smu, &control->eeprom_accessor); | |
| 221 | + break; | |
| 222 | + | |
| 125 | 223 | default: |
| 126 | 224 | return 0; |
| 127 | 225 | } |
| @@ -143,25 +241,18 @@ int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control) · 15 unmodified lines | ||
| 143 | 241 | if (hdr->header == EEPROM_TABLE_HDR_VAL) { |
| 144 | 242 | control->num_recs = (hdr->tbl_size - EEPROM_TABLE_HEADER_SIZE) / |
| 145 | 243 | EEPROM_TABLE_RECORD_SIZE; |
| 244 | + control->tbl_byte_sum = __calc_hdr_byte_sum(control); | |
| 245 | + control->next_addr = EEPROM_RECORD_START; | |
| 246 | + | |
| 146 | 247 | DRM_DEBUG_DRIVER("Found existing EEPROM table with %d records", |
| 147 | 248 | control->num_recs); |
| 148 | 249 | |
| 149 | 250 | } else { |
| 150 | 251 | DRM_INFO("Creating new EEPROM table"); |
| 151 | 252 | |
| 152 | − hdr->header = EEPROM_TABLE_HDR_VAL; | |
| 153 | − hdr->version = EEPROM_TABLE_VER; | |
| 154 | − hdr->first_rec_offset = EEPROM_RECORD_START; | |
| 155 | − hdr->tbl_size = EEPROM_TABLE_HEADER_SIZE; | |
| 156 | − | |
| 157 | − adev->psp.ras.ras->eeprom_control.tbl_byte_sum = | |
| 158 | − __calc_hdr_byte_sum(&adev->psp.ras.ras->eeprom_control); | |
| 159 | − ret = __update_table_header(control, buff); | |
| 253 | + ret = amdgpu_ras_eeprom_reset_table(control); | |
| 160 | 254 | } |
| 161 | 255 | |
| 162 | − /* Start inserting records from here */ | |
| 163 | − adev->psp.ras.ras->eeprom_control.next_addr = EEPROM_RECORD_START; | |
| 164 | − | |
| 165 | 256 | return ret == 1 ? 0 : -EIO; |
| 166 | 257 | } |
| 167 | 258 | |
| @@ -173,6 +264,9 @@ void amdgpu_ras_eeprom_fini(struct amdgpu_ras_eeprom_control *control) · 5 unmodified lines | ||
| 173 | 264 | case CHIP_VEGA20: |
| 174 | 265 | smu_v11_0_i2c_eeprom_control_fini(&control->eeprom_accessor); |
| 175 | 266 | break; |
| 267 | + case CHIP_ARCTURUS: | |
| 268 | + smu_i2c_eeprom_fini(&adev->smu, &control->eeprom_accessor); | |
| 269 | + break; | |
| 176 | 270 | |
| 177 | 271 | default: |
| 178 | 272 | return; |
| @@ -226,8 +320,8 @@ static void __decode_table_record_from_buff(struct amdgpu_ras_eeprom_control *co · 47 unmodified lines | ||
| 226 | 320 | record->offset = (le64_to_cpu(tmp) & 0xffffffffffff); |
| 227 | 321 | i += 6; |
| 228 | 322 | |
| 229 | − buff[i++] = record->mem_channel; | |
| 230 | − buff[i++] = record->mcumc_id; | |
| 323 | + record->mem_channel = buff[i++]; | |
| 324 | + record->mcumc_id = buff[i++]; | |
| 231 | 325 | |
| 232 | 326 | memcpy(&tmp, buff + i, 6); |
| 233 | 327 | record->retired_page = (le64_to_cpu(tmp) & 0xffffffffffff); |
| @@ -266,87 +360,18 @@ static uint32_t __correct_eeprom_dest_address(uint32_t curr_address) · 32 unmodified lines | ||
| 266 | 360 | return curr_address; |
| 267 | 361 | } |
| 268 | 362 | |
| 269 | − | |
| 270 | −static uint32_t __calc_hdr_byte_sum(struct amdgpu_ras_eeprom_control *control) | |
| 271 | −{ | |
| 272 | − int i; | |
| 273 | − uint32_t tbl_sum = 0; | |
| 274 | − | |
| 275 | − /* Header checksum, skip checksum field in the calculation */ | |
| 276 | − for (i = 0; i < sizeof(control->tbl_hdr) - sizeof(control->tbl_hdr.checksum); i++) | |
| 277 | − tbl_sum += *(((unsigned char *)&control->tbl_hdr) + i); | |
| 278 | − | |
| 279 | − return tbl_sum; | |
| 280 | −} | |
| 281 | − | |
| 282 | −static uint32_t __calc_recs_byte_sum(struct eeprom_table_record *records, | |
| 283 | − int num) | |
| 284 | −{ | |
| 285 | − int i, j; | |
| 286 | − uint32_t tbl_sum = 0; | |
| 287 | − | |
| 288 | − /* Records checksum */ | |
| 289 | − for (i = 0; i < num; i++) { | |
| 290 | − struct eeprom_table_record *record = &records[i]; | |
| 291 | − | |
| 292 | − for (j = 0; j < sizeof(*record); j++) { | |
| 293 | − tbl_sum += *(((unsigned char *)record) + j); | |
| 294 | − } | |
| 295 | − } | |
| 296 | − | |
| 297 | − return tbl_sum; | |
| 298 | −} | |
| 299 | − | |
| 300 | −static inline uint32_t __calc_tbl_byte_sum(struct amdgpu_ras_eeprom_control *control, | |
| 301 | − struct eeprom_table_record *records, int num) | |
| 302 | −{ | |
| 303 | − return __calc_hdr_byte_sum(control) + __calc_recs_byte_sum(records, num); | |
| 304 | −} | |
| 305 | − | |
| 306 | −/* Checksum = 256 -((sum of all table entries) mod 256) */ | |
| 307 | −static void __update_tbl_checksum(struct amdgpu_ras_eeprom_control *control, | |
| 308 | − struct eeprom_table_record *records, int num, | |
| 309 | − uint32_t old_hdr_byte_sum) | |
| 310 | −{ | |
| 311 | − /* | |
| 312 | − * This will update the table sum with new records. | |
| 313 | − * | |
| 314 | − * TODO: What happens when the EEPROM table is to be wrapped around | |
| 315 | − * and old records from start will get overridden. | |
| 316 | − */ | |
| 317 | − | |
| 318 | − /* need to recalculate updated header byte sum */ | |
| 319 | − control->tbl_byte_sum -= old_hdr_byte_sum; | |
| 320 | − control->tbl_byte_sum += __calc_tbl_byte_sum(control, records, num); | |
| 321 | − | |
| 322 | − control->tbl_hdr.checksum = 256 - (control->tbl_byte_sum % 256); | |
| 323 | −} | |
| 324 | − | |
| 325 | −/* table sum mod 256 + checksum must equals 256 */ | |
| 326 | −static bool __validate_tbl_checksum(struct amdgpu_ras_eeprom_control *control, | |
| 327 | − struct eeprom_table_record *records, int num) | |
| 328 | −{ | |
| 329 | − control->tbl_byte_sum = __calc_tbl_byte_sum(control, records, num); | |
| 330 | − | |
| 331 | − if (control->tbl_hdr.checksum + (control->tbl_byte_sum % 256) != 256) { | |
| 332 | − DRM_WARN("Checksum mismatch, checksum: %u ", control->tbl_hdr.checksum); | |
| 333 | − return false; | |
| 334 | − } | |
| 335 | − | |
| 336 | − return true; | |
| 337 | −} | |
| 338 | − | |
| 339 | 363 | int amdgpu_ras_eeprom_process_recods(struct amdgpu_ras_eeprom_control *control, |
| 340 | 364 | struct eeprom_table_record *records, |
| 341 | 365 | bool write, |
| 342 | 366 | int num) |
| 343 | 367 | { |
| 344 | 368 | int i, ret = 0; |
| 345 | − struct i2c_msg *msgs; | |
| 346 | − unsigned char *buffs; | |
| 369 | + struct i2c_msg *msgs, *msg; | |
| 370 | + unsigned char *buffs, *buff; | |
| 371 | + struct eeprom_table_record *record; | |
| 347 | 372 | struct amdgpu_device *adev = to_amdgpu_device(control); |
| 348 | 373 | |
| 349 | − if (adev->asic_type != CHIP_VEGA20) | |
| 374 | + if (adev->asic_type != CHIP_VEGA20 && adev->asic_type != CHIP_ARCTURUS) | |
| 350 | 375 | return 0; |
| 351 | 376 | |
| 352 | 377 | buffs = kcalloc(num, EEPROM_ADDRESS_SIZE + EEPROM_TABLE_RECORD_SIZE, |
| @@ -373,9 +398,9 @@ int amdgpu_ras_eeprom_process_recods(struct amdgpu_ras_eeprom_control *control, · 20 unmodified lines | ||
| 373 | 398 | * 256b |
| 374 | 399 | */ |
| 375 | 400 | for (i = 0; i < num; i++) { |
| 376 | − unsigned char *buff = &buffs[i * (EEPROM_ADDRESS_SIZE + EEPROM_TABLE_RECORD_SIZE)]; | |
| 377 | − struct eeprom_table_record *record = &records[i]; | |
| 378 | − struct i2c_msg *msg = &msgs[i]; | |
| 401 | + buff = &buffs[i * (EEPROM_ADDRESS_SIZE + EEPROM_TABLE_RECORD_SIZE)]; | |
| 402 | + record = &records[i]; | |
| 403 | + msg = &msgs[i]; | |
| 379 | 404 | |
| 380 | 405 | control->next_addr = __correct_eeprom_dest_address(control->next_addr); |
| 381 | 406 | |
| @@ -415,8 +440,8 @@ int amdgpu_ras_eeprom_process_recods(struct amdgpu_ras_eeprom_control *control, · 33 unmodified lines | ||
| 415 | 440 | |
| 416 | 441 | if (!write) { |
| 417 | 442 | for (i = 0; i < num; i++) { |
| 418 | − unsigned char *buff = &buffs[i*(EEPROM_ADDRESS_SIZE + EEPROM_TABLE_RECORD_SIZE)]; | |
| 419 | − struct eeprom_table_record *record = &records[i]; | |
| 443 | + buff = &buffs[i*(EEPROM_ADDRESS_SIZE + EEPROM_TABLE_RECORD_SIZE)]; | |
| 444 | + record = &records[i]; | |
| 420 | 445 | |
| 421 | 446 | __decode_table_record_from_buff(control, record, buff + EEPROM_ADDRESS_SIZE); |
| 422 | 447 | } |
| @@ -79,6 +79,7 @@ struct eeprom_table_record { · 78 unmodified lines | ||
| 79 | 79 | |
| 80 | 80 | int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control); |
| 81 | 81 | void amdgpu_ras_eeprom_fini(struct amdgpu_ras_eeprom_control *control); |
| 82 | +int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control); | |
| 82 | 83 | |
| 83 | 84 | int amdgpu_ras_eeprom_process_recods(struct amdgpu_ras_eeprom_control *control, |
| 84 | 85 | struct eeprom_table_record *records, |
| @@ -23,6 +23,7 @@ · 22 unmodified lines | ||
| 23 | 23 | |
| 24 | 24 | #include "amdgpu.h" |
| 25 | 25 | #include "amdgpu_sdma.h" |
| 26 | +#include "amdgpu_ras.h" | |
| 26 | 27 | |
| 27 | 28 | #define AMDGPU_CSA_SDMA_SIZE 64 |
| 28 | 29 | /* SDMA CSA reside in the 3rd page of CSA */ |
| @@ -83,3 +84,101 @@ uint64_t amdgpu_sdma_get_csa_mc_addr(struct amdgpu_ring *ring, · 54 unmodified lines | ||
| 83 | 84 | |
| 84 | 85 | return csa_mc_addr; |
| 85 | 86 | } |
| 87 | + | |
| 88 | +int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev, | |
| 89 | + void *ras_ih_info) | |
| 90 | +{ | |
| 91 | + int r, i; | |
| 92 | + struct ras_ih_if *ih_info = (struct ras_ih_if *)ras_ih_info; | |
| 93 | + struct ras_fs_if fs_info = { | |
| 94 | + .sysfs_name = "sdma_err_count", | |
| 95 | + .debugfs_name = "sdma_err_inject", | |
| 96 | + }; | |
| 97 | + | |
| 98 | + if (!ih_info) | |
| 99 | + return -EINVAL; | |
| 100 | + | |
| 101 | + if (!adev->sdma.ras_if) { | |
| 102 | + adev->sdma.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 103 | + if (!adev->sdma.ras_if) | |
| 104 | + return -ENOMEM; | |
| 105 | + adev->sdma.ras_if->block = AMDGPU_RAS_BLOCK__SDMA; | |
| 106 | + adev->sdma.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 107 | + adev->sdma.ras_if->sub_block_index = 0; | |
| 108 | + strcpy(adev->sdma.ras_if->name, "sdma"); | |
| 109 | + } | |
| 110 | + fs_info.head = ih_info->head = *adev->sdma.ras_if; | |
| 111 | + | |
| 112 | + r = amdgpu_ras_late_init(adev, adev->sdma.ras_if, | |
| 113 | + &fs_info, ih_info); | |
| 114 | + if (r) | |
| 115 | + goto free; | |
| 116 | + | |
| 117 | + if (amdgpu_ras_is_supported(adev, adev->sdma.ras_if->block)) { | |
| 118 | + for (i = 0; i < adev->sdma.num_instances; i++) { | |
| 119 | + r = amdgpu_irq_get(adev, &adev->sdma.ecc_irq, | |
| 120 | + AMDGPU_SDMA_IRQ_INSTANCE0 + i); | |
| 121 | + if (r) | |
| 122 | + goto late_fini; | |
| 123 | + } | |
| 124 | + } else { | |
| 125 | + r = 0; | |
| 126 | + goto free; | |
| 127 | + } | |
| 128 | + | |
| 129 | + return 0; | |
| 130 | + | |
| 131 | +late_fini: | |
| 132 | + amdgpu_ras_late_fini(adev, adev->sdma.ras_if, ih_info); | |
| 133 | +free: | |
| 134 | + kfree(adev->sdma.ras_if); | |
| 135 | + adev->sdma.ras_if = NULL; | |
| 136 | + return r; | |
| 137 | +} | |
| 138 | + | |
| 139 | +void amdgpu_sdma_ras_fini(struct amdgpu_device *adev) | |
| 140 | +{ | |
| 141 | + if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__SDMA) && | |
| 142 | + adev->sdma.ras_if) { | |
| 143 | + struct ras_common_if *ras_if = adev->sdma.ras_if; | |
| 144 | + struct ras_ih_if ih_info = { | |
| 145 | + .head = *ras_if, | |
| 146 | + /* the cb member will not be used by | |
| 147 | + * amdgpu_ras_interrupt_remove_handler, init it only | |
| 148 | + * to cheat the check in ras_late_fini | |
| 149 | + */ | |
| 150 | + .cb = amdgpu_sdma_process_ras_data_cb, | |
| 151 | + }; | |
| 152 | + | |
| 153 | + amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 154 | + kfree(ras_if); | |
| 155 | + } | |
| 156 | +} | |
| 157 | + | |
| 158 | +int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev, | |
| 159 | + void *err_data, | |
| 160 | + struct amdgpu_iv_entry *entry) | |
| 161 | +{ | |
| 162 | + kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); | |
| 163 | + amdgpu_ras_reset_gpu(adev, 0); | |
| 164 | + | |
| 165 | + return AMDGPU_RAS_SUCCESS; | |
| 166 | +} | |
| 167 | + | |
| 168 | +int amdgpu_sdma_process_ecc_irq(struct amdgpu_device *adev, | |
| 169 | + struct amdgpu_irq_src *source, | |
| 170 | + struct amdgpu_iv_entry *entry) | |
| 171 | +{ | |
| 172 | + struct ras_common_if *ras_if = adev->sdma.ras_if; | |
| 173 | + struct ras_dispatch_if ih_data = { | |
| 174 | + .entry = entry, | |
| 175 | + }; | |
| 176 | + | |
| 177 | + if (!ras_if) | |
| 178 | + return 0; | |
| 179 | + | |
| 180 | + ih_data.head = *ras_if; | |
| 181 | + | |
| 182 | + amdgpu_ras_interrupt_dispatch(adev, &ih_data); | |
| 183 | + return 0; | |
| 184 | +} | |
| @@ -104,4 +104,13 @@ struct amdgpu_sdma_instance * · 103 unmodified lines | ||
| 104 | 104 | amdgpu_sdma_get_instance_from_ring(struct amdgpu_ring *ring); |
| 105 | 105 | int amdgpu_sdma_get_index_from_ring(struct amdgpu_ring *ring, uint32_t *index); |
| 106 | 106 | uint64_t amdgpu_sdma_get_csa_mc_addr(struct amdgpu_ring *ring, unsigned vmid); |
| 107 | +int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev, | |
| 108 | + void *ras_ih_info); | |
| 109 | +void amdgpu_sdma_ras_fini(struct amdgpu_device *adev); | |
| 110 | +int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev, | |
| 111 | + void *err_data, | |
| 112 | + struct amdgpu_iv_entry *entry); | |
| 113 | +int amdgpu_sdma_process_ecc_irq(struct amdgpu_device *adev, | |
| 114 | + struct amdgpu_irq_src *source, | |
| 115 | + struct amdgpu_iv_entry *entry); | |
| 107 | 116 | #endif |
| @@ -138,6 +138,7 @@ static void amdgpu_do_test_moves(struct amdgpu_device *adev) · 137 unmodified lines | ||
| 138 | 138 | } |
| 139 | 139 | |
| 140 | 140 | dma_fence_put(fence); |
| 141 | + fence = NULL; | |
| 141 | 142 | |
| 142 | 143 | r = amdgpu_bo_kmap(vram_obj, &vram_map); |
| 143 | 144 | if (r) { |
| @@ -183,6 +184,7 @@ static void amdgpu_do_test_moves(struct amdgpu_device *adev) · 39 unmodified lines | ||
| 183 | 184 | } |
| 184 | 185 | |
| 185 | 186 | dma_fence_put(fence); |
| 187 | + fence = NULL; | |
| 186 | 188 | |
| 187 | 189 | r = amdgpu_bo_kmap(gtt_obj[i], >t_map); |
| 188 | 190 | if (r) { |
| @@ -170,7 +170,7 @@ TRACE_EVENT(amdgpu_cs_ioctl, · 169 unmodified lines | ||
| 170 | 170 | __field(unsigned int, context) |
| 171 | 171 | __field(unsigned int, seqno) |
| 172 | 172 | __field(struct dma_fence *, fence) |
| 173 | − __field(char *, ring_name) | |
| 173 | + __string(ring, to_amdgpu_ring(job->base.sched)->name) | |
| 174 | 174 | __field(u32, num_ibs) |
| 175 | 175 | ), |
| 176 | 176 | |
| @@ -179,12 +179,12 @@ TRACE_EVENT(amdgpu_cs_ioctl, · 2 unmodified lines | ||
| 179 | 179 | __assign_str(timeline, AMDGPU_JOB_GET_TIMELINE_NAME(job)) |
| 180 | 180 | __entry->context = job->base.s_fence->finished.context; |
| 181 | 181 | __entry->seqno = job->base.s_fence->finished.seqno; |
| 182 | − __entry->ring_name = to_amdgpu_ring(job->base.sched)->name; | |
| 182 | + __assign_str(ring, to_amdgpu_ring(job->base.sched)->name) | |
| 183 | 183 | __entry->num_ibs = job->num_ibs; |
| 184 | 184 | ), |
| 185 | 185 | TP_printk("sched_job=%llu, timeline=%s, context=%u, seqno=%u, ring_name=%s, num_ibs=%u", |
| 186 | 186 | __entry->sched_job_id, __get_str(timeline), __entry->context, |
| 187 | − __entry->seqno, __entry->ring_name, __entry->num_ibs) | |
| 187 | + __entry->seqno, __get_str(ring), __entry->num_ibs) | |
| 188 | 188 | ); |
| 189 | 189 | |
| 190 | 190 | TRACE_EVENT(amdgpu_sched_run_job, |
| @@ -195,7 +195,7 @@ TRACE_EVENT(amdgpu_sched_run_job, · 4 unmodified lines | ||
| 195 | 195 | __string(timeline, AMDGPU_JOB_GET_TIMELINE_NAME(job)) |
| 196 | 196 | __field(unsigned int, context) |
| 197 | 197 | __field(unsigned int, seqno) |
| 198 | − __field(char *, ring_name) | |
| 198 | + __string(ring, to_amdgpu_ring(job->base.sched)->name) | |
| 199 | 199 | __field(u32, num_ibs) |
| 200 | 200 | ), |
| 201 | 201 | |
| @@ -204,12 +204,12 @@ TRACE_EVENT(amdgpu_sched_run_job, · 2 unmodified lines | ||
| 204 | 204 | __assign_str(timeline, AMDGPU_JOB_GET_TIMELINE_NAME(job)) |
| 205 | 205 | __entry->context = job->base.s_fence->finished.context; |
| 206 | 206 | __entry->seqno = job->base.s_fence->finished.seqno; |
| 207 | − __entry->ring_name = to_amdgpu_ring(job->base.sched)->name; | |
| 207 | + __assign_str(ring, to_amdgpu_ring(job->base.sched)->name) | |
| 208 | 208 | __entry->num_ibs = job->num_ibs; |
| 209 | 209 | ), |
| 210 | 210 | TP_printk("sched_job=%llu, timeline=%s, context=%u, seqno=%u, ring_name=%s, num_ibs=%u", |
| 211 | 211 | __entry->sched_job_id, __get_str(timeline), __entry->context, |
| 212 | − __entry->seqno, __entry->ring_name, __entry->num_ibs) | |
| 212 | + __entry->seqno, __get_str(ring), __entry->num_ibs) | |
| 213 | 213 | ); |
| 214 | 214 | |
| 215 | 215 | |
| @@ -323,14 +323,15 @@ DEFINE_EVENT(amdgpu_vm_mapping, amdgpu_vm_bo_cs, · 107 unmodified lines | ||
| 323 | 323 | |
| 324 | 324 | TRACE_EVENT(amdgpu_vm_set_ptes, |
| 325 | 325 | TP_PROTO(uint64_t pe, uint64_t addr, unsigned count, |
| 326 | − uint32_t incr, uint64_t flags), | |
| 327 | − TP_ARGS(pe, addr, count, incr, flags), | |
| 326 | + uint32_t incr, uint64_t flags, bool direct), | |
| 327 | + TP_ARGS(pe, addr, count, incr, flags, direct), | |
| 328 | 328 | TP_STRUCT__entry( |
| 329 | 329 | __field(u64, pe) |
| 330 | 330 | __field(u64, addr) |
| 331 | 331 | __field(u32, count) |
| 332 | 332 | __field(u32, incr) |
| 333 | 333 | __field(u64, flags) |
| 334 | + __field(bool, direct) | |
| 334 | 335 | ), |
| 335 | 336 | |
| 336 | 337 | TP_fast_assign( |
| @@ -339,28 +340,32 @@ TRACE_EVENT(amdgpu_vm_set_ptes, · 2 unmodified lines | ||
| 339 | 340 | __entry->count = count; |
| 340 | 341 | __entry->incr = incr; |
| 341 | 342 | __entry->flags = flags; |
| 343 | + __entry->direct = direct; | |
| 342 | 344 | ), |
| 343 | − TP_printk("pe=%010Lx, addr=%010Lx, incr=%u, flags=%llx, count=%u", | |
| 344 | − __entry->pe, __entry->addr, __entry->incr, | |
| 345 | − __entry->flags, __entry->count) | |
| 345 | + TP_printk("pe=%010Lx, addr=%010Lx, incr=%u, flags=%llx, count=%u, " | |
| 346 | + "direct=%d", __entry->pe, __entry->addr, __entry->incr, | |
| 347 | + __entry->flags, __entry->count, __entry->direct) | |
| 346 | 348 | ); |
| 347 | 349 | |
| 348 | 350 | TRACE_EVENT(amdgpu_vm_copy_ptes, |
| 349 | − TP_PROTO(uint64_t pe, uint64_t src, unsigned count), | |
| 350 | − TP_ARGS(pe, src, count), | |
| 351 | + TP_PROTO(uint64_t pe, uint64_t src, unsigned count, bool direct), | |
| 352 | + TP_ARGS(pe, src, count, direct), | |
| 351 | 353 | TP_STRUCT__entry( |
| 352 | 354 | __field(u64, pe) |
| 353 | 355 | __field(u64, src) |
| 354 | 356 | __field(u32, count) |
| 357 | + __field(bool, direct) | |
| 355 | 358 | ), |
| 356 | 359 | |
| 357 | 360 | TP_fast_assign( |
| 358 | 361 | __entry->pe = pe; |
| 359 | 362 | __entry->src = src; |
| 360 | 363 | __entry->count = count; |
| 364 | + __entry->direct = direct; | |
| 361 | 365 | ), |
| 362 | − TP_printk("pe=%010Lx, src=%010Lx, count=%u", | |
| 363 | − __entry->pe, __entry->src, __entry->count) | |
| 366 | + TP_printk("pe=%010Lx, src=%010Lx, count=%u, direct=%d", | |
| 367 | + __entry->pe, __entry->src, __entry->count, | |
| 368 | + __entry->direct) | |
| 364 | 369 | ); |
| 365 | 370 | |
| 366 | 371 | TRACE_EVENT(amdgpu_vm_flush, |
| @@ -468,7 +473,7 @@ TRACE_EVENT(amdgpu_ib_pipe_sync, · 101 unmodified lines | ||
| 468 | 473 | TP_PROTO(struct amdgpu_job *sched_job, struct dma_fence *fence), |
| 469 | 474 | TP_ARGS(sched_job, fence), |
| 470 | 475 | TP_STRUCT__entry( |
| 471 | − __field(const char *,name) | |
| 476 | + __string(ring, sched_job->base.sched->name); | |
| 472 | 477 | __field(uint64_t, id) |
| 473 | 478 | __field(struct dma_fence *, fence) |
| 474 | 479 | __field(uint64_t, ctx) |
| @@ -476,14 +481,14 @@ TRACE_EVENT(amdgpu_ib_pipe_sync, · 1 unmodified line | ||
| 476 | 481 | ), |
| 477 | 482 | |
| 478 | 483 | TP_fast_assign( |
| 479 | − __entry->name = sched_job->base.sched->name; | |
| 484 | + __assign_str(ring, sched_job->base.sched->name) | |
| 480 | 485 | __entry->id = sched_job->base.id; |
| 481 | 486 | __entry->fence = fence; |
| 482 | 487 | __entry->ctx = fence->context; |
| 483 | 488 | __entry->seqno = fence->seqno; |
| 484 | 489 | ), |
| 485 | 490 | TP_printk("job ring=%s, id=%llu, need pipe sync to fence=%p, context=%llu, seq=%u", |
| 486 | − __entry->name, __entry->id, | |
| 491 | + __get_str(ring), __entry->id, | |
| 487 | 492 | __entry->fence, __entry->ctx, |
| 488 | 493 | __entry->seqno) |
| 489 | 494 | ); |
| @@ -39,6 +39,7 @@ · 38 unmodified lines | ||
| 39 | 39 | #include <linux/slab.h> |
| 40 | 40 | #include <linux/swap.h> |
| 41 | 41 | #include <linux/swiotlb.h> |
| 42 | +#include <linux/dma-buf.h> | |
| 42 | 43 | |
| 43 | 44 | #include <drm/ttm/ttm_bo_api.h> |
| 44 | 45 | #include <drm/ttm/ttm_bo_driver.h> |
| @@ -54,6 +55,7 @@ · 9 unmodified lines | ||
| 54 | 55 | #include "amdgpu_trace.h" |
| 55 | 56 | #include "amdgpu_amdkfd.h" |
| 56 | 57 | #include "amdgpu_sdma.h" |
| 58 | +#include "amdgpu_ras.h" | |
| 57 | 59 | #include "bif/bif_4_1_d.h" |
| 58 | 60 | |
| 59 | 61 | static int amdgpu_map_buffer(struct ttm_buffer_object *bo, |
| @@ -484,15 +486,12 @@ static int amdgpu_move_vram_ram(struct ttm_buffer_object *bo, bool evict, · 424 unmodified lines | ||
| 484 | 486 | struct ttm_operation_ctx *ctx, |
| 485 | 487 | struct ttm_mem_reg *new_mem) |
| 486 | 488 | { |
| 487 | − struct amdgpu_device *adev; | |
| 488 | 489 | struct ttm_mem_reg *old_mem = &bo->mem; |
| 489 | 490 | struct ttm_mem_reg tmp_mem; |
| 490 | 491 | struct ttm_place placements; |
| 491 | 492 | struct ttm_placement placement; |
| 492 | 493 | int r; |
| 493 | 494 | |
| 494 | − adev = amdgpu_ttm_adev(bo->bdev); | |
| 495 | − | |
| 496 | 495 | /* create space/pages for new_mem in GTT space */ |
| 497 | 496 | tmp_mem = *new_mem; |
| 498 | 497 | tmp_mem.mm_node = NULL; |
| @@ -543,15 +542,12 @@ static int amdgpu_move_ram_vram(struct ttm_buffer_object *bo, bool evict, · 44 unmodified lines | ||
| 543 | 542 | struct ttm_operation_ctx *ctx, |
| 544 | 543 | struct ttm_mem_reg *new_mem) |
| 545 | 544 | { |
| 546 | − struct amdgpu_device *adev; | |
| 547 | 545 | struct ttm_mem_reg *old_mem = &bo->mem; |
| 548 | 546 | struct ttm_mem_reg tmp_mem; |
| 549 | 547 | struct ttm_placement placement; |
| 550 | 548 | struct ttm_place placements; |
| 551 | 549 | int r; |
| 552 | 550 | |
| 553 | − adev = amdgpu_ttm_adev(bo->bdev); | |
| 554 | − | |
| 555 | 551 | /* make space in GTT for old_mem buffer */ |
| 556 | 552 | tmp_mem = *new_mem; |
| 557 | 553 | tmp_mem.mm_node = NULL; |
| @@ -763,6 +759,7 @@ static unsigned long amdgpu_ttm_io_mem_pfn(struct ttm_buffer_object *bo, · 205 unmodified lines | ||
| 763 | 759 | */ |
| 764 | 760 | struct amdgpu_ttm_tt { |
| 765 | 761 | struct ttm_dma_tt ttm; |
| 762 | + struct drm_gem_object *gobj; | |
| 766 | 763 | u64 offset; |
| 767 | 764 | uint64_t userptr; |
| 768 | 765 | struct task_struct *usertask; |
| @@ -1217,16 +1214,14 @@ static struct ttm_backend_func amdgpu_backend_func = { · 448 unmodified lines | ||
| 1217 | 1214 | static struct ttm_tt *amdgpu_ttm_tt_create(struct ttm_buffer_object *bo, |
| 1218 | 1215 | uint32_t page_flags) |
| 1219 | 1216 | { |
| 1220 | − struct amdgpu_device *adev; | |
| 1221 | 1217 | struct amdgpu_ttm_tt *gtt; |
| 1222 | 1218 | |
| 1223 | − adev = amdgpu_ttm_adev(bo->bdev); | |
| 1224 | − | |
| 1225 | 1219 | gtt = kzalloc(sizeof(struct amdgpu_ttm_tt), GFP_KERNEL); |
| 1226 | 1220 | if (gtt == NULL) { |
| 1227 | 1221 | return NULL; |
| 1228 | 1222 | } |
| 1229 | 1223 | gtt->ttm.ttm.func = &amdgpu_backend_func; |
| 1224 | + gtt->gobj = &bo->base; | |
| 1230 | 1225 | |
| 1231 | 1226 | /* allocate space for the uninitialized page entries */ |
| 1232 | 1227 | if (ttm_sg_tt_init(>t->ttm, bo, page_flags)) { |
| @@ -1247,7 +1242,6 @@ static int amdgpu_ttm_tt_populate(struct ttm_tt *ttm, · 14 unmodified lines | ||
| 1247 | 1242 | { |
| 1248 | 1243 | struct amdgpu_device *adev = amdgpu_ttm_adev(ttm->bdev); |
| 1249 | 1244 | struct amdgpu_ttm_tt *gtt = (void *)ttm; |
| 1250 | − bool slave = !!(ttm->page_flags & TTM_PAGE_FLAG_SG); | |
| 1251 | 1245 | |
| 1252 | 1246 | /* user pages are bound by amdgpu_ttm_tt_pin_userptr() */ |
| 1253 | 1247 | if (gtt && gtt->userptr) { |
| @@ -1260,7 +1254,19 @@ static int amdgpu_ttm_tt_populate(struct ttm_tt *ttm, · 6 unmodified lines | ||
| 1260 | 1254 | return 0; |
| 1261 | 1255 | } |
| 1262 | 1256 | |
| 1263 | − if (slave && ttm->sg) { | |
| 1257 | + if (ttm->page_flags & TTM_PAGE_FLAG_SG) { | |
| 1258 | + if (!ttm->sg) { | |
| 1259 | + struct dma_buf_attachment *attach; | |
| 1260 | + struct sg_table *sgt; | |
| 1261 | + | |
| 1262 | + attach = gtt->gobj->import_attach; | |
| 1263 | + sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL); | |
| 1264 | + if (IS_ERR(sgt)) | |
| 1265 | + return PTR_ERR(sgt); | |
| 1266 | + | |
| 1267 | + ttm->sg = sgt; | |
| 1268 | + } | |
| 1269 | + | |
| 1264 | 1270 | drm_prime_sg_to_page_addr_arrays(ttm->sg, ttm->pages, |
| 1265 | 1271 | gtt->ttm.dma_address, |
| 1266 | 1272 | ttm->num_pages); |
| @@ -1287,9 +1293,8 @@ static int amdgpu_ttm_tt_populate(struct ttm_tt *ttm, · 20 unmodified lines | ||
| 1287 | 1293 | */ |
| 1288 | 1294 | static void amdgpu_ttm_tt_unpopulate(struct ttm_tt *ttm) |
| 1289 | 1295 | { |
| 1290 | − struct amdgpu_device *adev; | |
| 1291 | 1296 | struct amdgpu_ttm_tt *gtt = (void *)ttm; |
| 1292 | − bool slave = !!(ttm->page_flags & TTM_PAGE_FLAG_SG); | |
| 1297 | + struct amdgpu_device *adev; | |
| 1293 | 1298 | |
| 1294 | 1299 | if (gtt && gtt->userptr) { |
| 1295 | 1300 | amdgpu_ttm_tt_set_user_pages(ttm, NULL); |
| @@ -1298,7 +1303,16 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_tt *ttm) · 2 unmodified lines | ||
| 1298 | 1303 | return; |
| 1299 | 1304 | } |
| 1300 | 1305 | |
| 1301 | − if (slave) | |
| 1306 | + if (ttm->sg && gtt->gobj->import_attach) { | |
| 1307 | + struct dma_buf_attachment *attach; | |
| 1308 | + | |
| 1309 | + attach = gtt->gobj->import_attach; | |
| 1310 | + dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL); | |
| 1311 | + ttm->sg = NULL; | |
| 1312 | + return; | |
| 1313 | + } | |
| 1314 | + | |
| 1315 | + if (ttm->page_flags & TTM_PAGE_FLAG_SG) | |
| 1302 | 1316 | return; |
| 1303 | 1317 | |
| 1304 | 1318 | adev = amdgpu_ttm_adev(ttm->bdev); |
| @@ -1634,81 +1648,105 @@ static void amdgpu_ttm_fw_reserve_vram_fini(struct amdgpu_device *adev) · 329 unmodified lines | ||
| 1634 | 1648 | */ |
| 1635 | 1649 | static int amdgpu_ttm_fw_reserve_vram_init(struct amdgpu_device *adev) |
| 1636 | 1650 | { |
| 1637 | − struct ttm_operation_ctx ctx = { false, false }; | |
| 1638 | − struct amdgpu_bo_param bp; | |
| 1639 | − int r = 0; | |
| 1640 | − int i; | |
| 1641 | − u64 vram_size = adev->gmc.visible_vram_size; | |
| 1642 | − u64 offset = adev->fw_vram_usage.start_offset; | |
| 1643 | − u64 size = adev->fw_vram_usage.size; | |
| 1644 | − struct amdgpu_bo *bo; | |
| 1645 | − | |
| 1646 | − memset(&bp, 0, sizeof(bp)); | |
| 1647 | − bp.size = adev->fw_vram_usage.size; | |
| 1648 | − bp.byte_align = PAGE_SIZE; | |
| 1649 | − bp.domain = AMDGPU_GEM_DOMAIN_VRAM; | |
| 1650 | − bp.flags = AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED | | |
| 1651 | − AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS; | |
| 1652 | − bp.type = ttm_bo_type_kernel; | |
| 1653 | − bp.resv = NULL; | |
| 1651 | + uint64_t vram_size = adev->gmc.visible_vram_size; | |
| 1652 | + | |
| 1654 | 1653 | adev->fw_vram_usage.va = NULL; |
| 1655 | 1654 | adev->fw_vram_usage.reserved_bo = NULL; |
| 1656 | 1655 | |
| 1657 | − if (adev->fw_vram_usage.size > 0 && | |
| 1658 | − adev->fw_vram_usage.size <= vram_size) { | |
| 1656 | + if (adev->fw_vram_usage.size == 0 || | |
| 1657 | + adev->fw_vram_usage.size > vram_size) | |
| 1658 | + return 0; | |
| 1659 | 1659 | |
| 1660 | − r = amdgpu_bo_create(adev, &bp, | |
| 1661 | − &adev->fw_vram_usage.reserved_bo); | |
| 1662 | − if (r) | |
| 1663 | − goto error_create; | |
| 1660 | + return amdgpu_bo_create_kernel_at(adev, | |
| 1661 | + adev->fw_vram_usage.start_offset, | |
| 1662 | + adev->fw_vram_usage.size, | |
| 1663 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 1664 | + &adev->fw_vram_usage.reserved_bo, | |
| 1665 | + &adev->fw_vram_usage.va); | |
| 1666 | +} | |
| 1664 | 1667 | |
| 1665 | − r = amdgpu_bo_reserve(adev->fw_vram_usage.reserved_bo, false); | |
| 1666 | − if (r) | |
| 1667 | − goto error_reserve; | |
| 1668 | +/* | |
| 1669 | + * Memoy training reservation functions | |
| 1670 | + */ | |
| 1668 | 1671 | |
| 1669 | − /* remove the original mem node and create a new one at the | |
| 1670 | − * request position | |
| 1671 | − */ | |
| 1672 | − bo = adev->fw_vram_usage.reserved_bo; | |
| 1673 | − offset = ALIGN(offset, PAGE_SIZE); | |
| 1674 | − for (i = 0; i < bo->placement.num_placement; ++i) { | |
| 1675 | − bo->placements[i].fpfn = offset >> PAGE_SHIFT; | |
| 1676 | − bo->placements[i].lpfn = (offset + size) >> PAGE_SHIFT; | |
| 1677 | − } | |
| 1672 | +/** | |
| 1673 | + * amdgpu_ttm_training_reserve_vram_fini - free memory training reserved vram | |
| 1674 | + * | |
| 1675 | + * @adev: amdgpu_device pointer | |
| 1676 | + * | |
| 1677 | + * free memory training reserved vram if it has been reserved. | |
| 1678 | + */ | |
| 1679 | +static int amdgpu_ttm_training_reserve_vram_fini(struct amdgpu_device *adev) | |
| 1680 | +{ | |
| 1681 | + struct psp_memory_training_context *ctx = &adev->psp.mem_train_ctx; | |
| 1678 | 1682 | |
| 1679 | − ttm_bo_mem_put(&bo->tbo, &bo->tbo.mem); | |
| 1680 | − r = ttm_bo_mem_space(&bo->tbo, &bo->placement, | |
| 1681 | − &bo->tbo.mem, &ctx); | |
| 1682 | − if (r) | |
| 1683 | − goto error_pin; | |
| 1683 | + ctx->init = PSP_MEM_TRAIN_NOT_SUPPORT; | |
| 1684 | + amdgpu_bo_free_kernel(&ctx->c2p_bo, NULL, NULL); | |
| 1685 | + ctx->c2p_bo = NULL; | |
| 1684 | 1686 | |
| 1685 | − r = amdgpu_bo_pin_restricted(adev->fw_vram_usage.reserved_bo, | |
| 1686 | − AMDGPU_GEM_DOMAIN_VRAM, | |
| 1687 | − adev->fw_vram_usage.start_offset, | |
| 1688 | − (adev->fw_vram_usage.start_offset + | |
| 1689 | − adev->fw_vram_usage.size)); | |
| 1690 | − if (r) | |
| 1691 | − goto error_pin; | |
| 1692 | − r = amdgpu_bo_kmap(adev->fw_vram_usage.reserved_bo, | |
| 1693 | − &adev->fw_vram_usage.va); | |
| 1694 | − if (r) | |
| 1695 | − goto error_kmap; | |
| 1687 | + amdgpu_bo_free_kernel(&ctx->p2c_bo, NULL, NULL); | |
| 1688 | + ctx->p2c_bo = NULL; | |
| 1696 | 1689 | |
| 1697 | − amdgpu_bo_unreserve(adev->fw_vram_usage.reserved_bo); | |
| 1690 | + return 0; | |
| 1691 | +} | |
| 1692 | + | |
| 1693 | +/** | |
| 1694 | + * amdgpu_ttm_training_reserve_vram_init - create bo vram reservation from memory training | |
| 1695 | + * | |
| 1696 | + * @adev: amdgpu_device pointer | |
| 1697 | + * | |
| 1698 | + * create bo vram reservation from memory training. | |
| 1699 | + */ | |
| 1700 | +static int amdgpu_ttm_training_reserve_vram_init(struct amdgpu_device *adev) | |
| 1701 | +{ | |
| 1702 | + int ret; | |
| 1703 | + struct psp_memory_training_context *ctx = &adev->psp.mem_train_ctx; | |
| 1704 | + | |
| 1705 | + memset(ctx, 0, sizeof(*ctx)); | |
| 1706 | + if (!adev->fw_vram_usage.mem_train_support) { | |
| 1707 | + DRM_DEBUG("memory training does not support!\n"); | |
| 1708 | + return 0; | |
| 1698 | 1709 | } |
| 1699 | − return r; | |
| 1700 | 1710 | |
| 1701 | −error_kmap: | |
| 1702 | − amdgpu_bo_unpin(adev->fw_vram_usage.reserved_bo); | |
| 1703 | −error_pin: | |
| 1704 | − amdgpu_bo_unreserve(adev->fw_vram_usage.reserved_bo); | |
| 1705 | −error_reserve: | |
| 1706 | − amdgpu_bo_unref(&adev->fw_vram_usage.reserved_bo); | |
| 1707 | −error_create: | |
| 1708 | − adev->fw_vram_usage.va = NULL; | |
| 1709 | − adev->fw_vram_usage.reserved_bo = NULL; | |
| 1710 | − return r; | |
| 1711 | + ctx->c2p_train_data_offset = adev->fw_vram_usage.mem_train_fb_loc; | |
| 1712 | + ctx->p2c_train_data_offset = (adev->gmc.mc_vram_size - GDDR6_MEM_TRAINING_OFFSET); | |
| 1713 | + ctx->train_data_size = GDDR6_MEM_TRAINING_DATA_SIZE_IN_BYTES; | |
| 1714 | + | |
| 1715 | + DRM_DEBUG("train_data_size:%llx,p2c_train_data_offset:%llx,c2p_train_data_offset:%llx.\n", | |
| 1716 | + ctx->train_data_size, | |
| 1717 | + ctx->p2c_train_data_offset, | |
| 1718 | + ctx->c2p_train_data_offset); | |
| 1719 | + | |
| 1720 | + ret = amdgpu_bo_create_kernel_at(adev, | |
| 1721 | + ctx->p2c_train_data_offset, | |
| 1722 | + ctx->train_data_size, | |
| 1723 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 1724 | + &ctx->p2c_bo, | |
| 1725 | + NULL); | |
| 1726 | + if (ret) { | |
| 1727 | + DRM_ERROR("alloc p2c_bo failed(%d)!\n", ret); | |
| 1728 | + goto Err_out; | |
| 1729 | + } | |
| 1730 | + | |
| 1731 | + ret = amdgpu_bo_create_kernel_at(adev, | |
| 1732 | + ctx->c2p_train_data_offset, | |
| 1733 | + ctx->train_data_size, | |
| 1734 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 1735 | + &ctx->c2p_bo, | |
| 1736 | + NULL); | |
| 1737 | + if (ret) { | |
| 1738 | + DRM_ERROR("alloc c2p_bo failed(%d)!\n", ret); | |
| 1739 | + goto Err_out; | |
| 1740 | + } | |
| 1741 | + | |
| 1742 | + ctx->init = PSP_MEM_TRAIN_RESERVE_SUCCESS; | |
| 1743 | + return 0; | |
| 1744 | + | |
| 1745 | +Err_out: | |
| 1746 | + amdgpu_ttm_training_reserve_vram_fini(adev); | |
| 1747 | + return ret; | |
| 1711 | 1748 | } |
| 1749 | + | |
| 1712 | 1750 | /** |
| 1713 | 1751 | * amdgpu_ttm_init - Init the memory management (ttm) as well as various |
| 1714 | 1752 | * gtt/vram related fields. |
| @@ -1772,6 +1810,14 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) · 57 unmodified lines | ||
| 1772 | 1810 | return r; |
| 1773 | 1811 | } |
| 1774 | 1812 | |
| 1813 | + /* | |
| 1814 | + *The reserved vram for memory training must be pinned to the specified | |
| 1815 | + *place on the VRAM, so reserve it early. | |
| 1816 | + */ | |
| 1817 | + r = amdgpu_ttm_training_reserve_vram_init(adev); | |
| 1818 | + if (r) | |
| 1819 | + return r; | |
| 1820 | + | |
| 1775 | 1821 | /* allocate memory as required for VGA |
| 1776 | 1822 | * This is used for VGA emulation and pre-OS scanout buffers to |
| 1777 | 1823 | * avoid display artifacts while transitioning between pre-OS |
| @@ -1782,6 +1828,20 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) · 4 unmodified lines | ||
| 1782 | 1828 | NULL, &stolen_vga_buf); |
| 1783 | 1829 | if (r) |
| 1784 | 1830 | return r; |
| 1831 | + | |
| 1832 | + /* | |
| 1833 | + * reserve one TMR (64K) memory at the top of VRAM which holds | |
| 1834 | + * IP Discovery data and is protected by PSP. | |
| 1835 | + */ | |
| 1836 | + r = amdgpu_bo_create_kernel_at(adev, | |
| 1837 | + adev->gmc.real_vram_size - DISCOVERY_TMR_SIZE, | |
| 1838 | + DISCOVERY_TMR_SIZE, | |
| 1839 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 1840 | + &adev->discovery_memory, | |
| 1841 | + NULL); | |
| 1842 | + if (r) | |
| 1843 | + return r; | |
| 1844 | + | |
| 1785 | 1845 | DRM_INFO("amdgpu: %uM of VRAM memory ready\n", |
| 1786 | 1846 | (unsigned) (adev->gmc.real_vram_size / (1024 * 1024))); |
| 1787 | 1847 | |
| @@ -1857,7 +1917,11 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) · 69 unmodified lines | ||
| 1857 | 1917 | return; |
| 1858 | 1918 | |
| 1859 | 1919 | amdgpu_ttm_debugfs_fini(adev); |
| 1920 | + amdgpu_ttm_training_reserve_vram_fini(adev); | |
| 1921 | + /* return the IP Discovery TMR memory back to VRAM */ | |
| 1922 | + amdgpu_bo_free_kernel(&adev->discovery_memory, NULL, NULL); | |
| 1860 | 1923 | amdgpu_ttm_fw_reserve_vram_fini(adev); |
| 1924 | + | |
| 1861 | 1925 | if (adev->mman.aper_base_kaddr) |
| 1862 | 1926 | iounmap(adev->mman.aper_base_kaddr); |
| 1863 | 1927 | adev->mman.aper_base_kaddr = NULL; |
| @@ -1953,10 +2017,7 @@ static int amdgpu_map_buffer(struct ttm_buffer_object *bo, · 89 unmodified lines | ||
| 1953 | 2017 | *addr += (u64)window * AMDGPU_GTT_MAX_TRANSFER_SIZE * |
| 1954 | 2018 | AMDGPU_GPU_PAGE_SIZE; |
| 1955 | 2019 | |
| 1956 | − num_dw = adev->mman.buffer_funcs->copy_num_dw; | |
| 1957 | − while (num_dw & 0x7) | |
| 1958 | − num_dw++; | |
| 1959 | − | |
| 2020 | + num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8); | |
| 1960 | 2021 | num_bytes = num_pages * 8; |
| 1961 | 2022 | |
| 1962 | 2023 | r = amdgpu_job_alloc_with_ib(adev, num_dw * 4 + num_bytes, &job); |
| @@ -2016,11 +2077,7 @@ int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset, · 53 unmodified lines | ||
| 2016 | 2077 | |
| 2017 | 2078 | max_bytes = adev->mman.buffer_funcs->copy_max_bytes; |
| 2018 | 2079 | num_loops = DIV_ROUND_UP(byte_count, max_bytes); |
| 2019 | − num_dw = num_loops * adev->mman.buffer_funcs->copy_num_dw; | |
| 2020 | − | |
| 2021 | − /* for IB padding */ | |
| 2022 | − while (num_dw & 0x7) | |
| 2023 | − num_dw++; | |
| 2080 | + num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->copy_num_dw, 8); | |
| 2024 | 2081 | |
| 2025 | 2082 | r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, &job); |
| 2026 | 2083 | if (r) |
| @@ -360,6 +360,7 @@ amdgpu_ucode_get_load_type(struct amdgpu_device *adev, int load_type) · 359 unmodified lines | ||
| 360 | 360 | case CHIP_RAVEN: |
| 361 | 361 | case CHIP_VEGA12: |
| 362 | 362 | case CHIP_VEGA20: |
| 363 | + case CHIP_ARCTURUS: | |
| 363 | 364 | case CHIP_RENOIR: |
| 364 | 365 | case CHIP_NAVI10: |
| 365 | 366 | case CHIP_NAVI14: |
| @@ -368,8 +369,6 @@ amdgpu_ucode_get_load_type(struct amdgpu_device *adev, int load_type) · 2 unmodified lines | ||
| 368 | 369 | return AMDGPU_FW_LOAD_DIRECT; |
| 369 | 370 | else |
| 370 | 371 | return AMDGPU_FW_LOAD_PSP; |
| 371 | − case CHIP_ARCTURUS: | |
| 372 | − return AMDGPU_FW_LOAD_DIRECT; | |
| 373 | 372 | |
| 374 | 373 | default: |
| 375 | 374 | DRM_ERROR("Unknown firmware load type\n"); |
| @@ -108,6 +108,12 @@ struct ta_firmware_header_v1_0 { · 107 unmodified lines | ||
| 108 | 108 | uint32_t ta_ras_ucode_version; |
| 109 | 109 | uint32_t ta_ras_offset_bytes; |
| 110 | 110 | uint32_t ta_ras_size_bytes; |
| 111 | + uint32_t ta_hdcp_ucode_version; | |
| 112 | + uint32_t ta_hdcp_offset_bytes; | |
| 113 | + uint32_t ta_hdcp_size_bytes; | |
| 114 | + uint32_t ta_dtm_ucode_version; | |
| 115 | + uint32_t ta_dtm_offset_bytes; | |
| 116 | + uint32_t ta_dtm_size_bytes; | |
| 111 | 117 | }; |
| 112 | 118 | |
| 113 | 119 | /* version_major=1, version_minor=0 */ |
| @@ -0,0 +1,158 @@ | ||
| 1 | +/* | |
| 2 | + * Copyright 2019 Advanced Micro Devices, Inc. | |
| 3 | + * | |
| 4 | + * Permission is hereby granted, free of charge, to any person obtaining a | |
| 5 | + * copy of this software and associated documentation files (the "Software"), | |
| 6 | + * to deal in the Software without restriction, including without limitation | |
| 7 | + * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 8 | + * and/or sell copies of the Software, and to permit persons to whom the | |
| 9 | + * Software is furnished to do so, subject to the following conditions: | |
| 10 | + * | |
| 11 | + * The above copyright notice and this permission notice shall be included in | |
| 12 | + * all copies or substantial portions of the Software. | |
| 13 | + * | |
| 14 | + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 15 | + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 16 | + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 17 | + * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 18 | + * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 19 | + * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 20 | + * OTHER DEALINGS IN THE SOFTWARE. | |
| 21 | + * | |
| 22 | + */ | |
| 23 | + | |
| 24 | +#include "amdgpu_ras.h" | |
| 25 | + | |
| 26 | +int amdgpu_umc_ras_late_init(struct amdgpu_device *adev) | |
| 27 | +{ | |
| 28 | + int r; | |
| 29 | + struct ras_fs_if fs_info = { | |
| 30 | + .sysfs_name = "umc_err_count", | |
| 31 | + .debugfs_name = "umc_err_inject", | |
| 32 | + }; | |
| 33 | + struct ras_ih_if ih_info = { | |
| 34 | + .cb = amdgpu_umc_process_ras_data_cb, | |
| 35 | + }; | |
| 36 | + | |
| 37 | + if (!adev->umc.ras_if) { | |
| 38 | + adev->umc.ras_if = | |
| 39 | + kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 40 | + if (!adev->umc.ras_if) | |
| 41 | + return -ENOMEM; | |
| 42 | + adev->umc.ras_if->block = AMDGPU_RAS_BLOCK__UMC; | |
| 43 | + adev->umc.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 44 | + adev->umc.ras_if->sub_block_index = 0; | |
| 45 | + strcpy(adev->umc.ras_if->name, "umc"); | |
| 46 | + } | |
| 47 | + ih_info.head = fs_info.head = *adev->umc.ras_if; | |
| 48 | + | |
| 49 | + r = amdgpu_ras_late_init(adev, adev->umc.ras_if, | |
| 50 | + &fs_info, &ih_info); | |
| 51 | + if (r) | |
| 52 | + goto free; | |
| 53 | + | |
| 54 | + if (amdgpu_ras_is_supported(adev, adev->umc.ras_if->block)) { | |
| 55 | + r = amdgpu_irq_get(adev, &adev->gmc.ecc_irq, 0); | |
| 56 | + if (r) | |
| 57 | + goto late_fini; | |
| 58 | + } else { | |
| 59 | + r = 0; | |
| 60 | + goto free; | |
| 61 | + } | |
| 62 | + | |
| 63 | + /* ras init of specific umc version */ | |
| 64 | + if (adev->umc.funcs && adev->umc.funcs->err_cnt_init) | |
| 65 | + adev->umc.funcs->err_cnt_init(adev); | |
| 66 | + | |
| 67 | + return 0; | |
| 68 | + | |
| 69 | +late_fini: | |
| 70 | + amdgpu_ras_late_fini(adev, adev->umc.ras_if, &ih_info); | |
| 71 | +free: | |
| 72 | + kfree(adev->umc.ras_if); | |
| 73 | + adev->umc.ras_if = NULL; | |
| 74 | + return r; | |
| 75 | +} | |
| 76 | + | |
| 77 | +void amdgpu_umc_ras_fini(struct amdgpu_device *adev) | |
| 78 | +{ | |
| 79 | + if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__UMC) && | |
| 80 | + adev->umc.ras_if) { | |
| 81 | + struct ras_common_if *ras_if = adev->umc.ras_if; | |
| 82 | + struct ras_ih_if ih_info = { | |
| 83 | + .head = *ras_if, | |
| 84 | + .cb = amdgpu_umc_process_ras_data_cb, | |
| 85 | + }; | |
| 86 | + | |
| 87 | + amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 88 | + kfree(ras_if); | |
| 89 | + } | |
| 90 | +} | |
| 91 | + | |
| 92 | +int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | |
| 93 | + void *ras_error_status, | |
| 94 | + struct amdgpu_iv_entry *entry) | |
| 95 | +{ | |
| 96 | + struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | |
| 97 | + | |
| 98 | + /* When “Full RAS” is enabled, the per-IP interrupt sources should | |
| 99 | + * be disabled and the driver should only look for the aggregated | |
| 100 | + * interrupt via sync flood | |
| 101 | + */ | |
| 102 | + if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) | |
| 103 | + return AMDGPU_RAS_SUCCESS; | |
| 104 | + | |
| 105 | + kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); | |
| 106 | + if (adev->umc.funcs && | |
| 107 | + adev->umc.funcs->query_ras_error_count) | |
| 108 | + adev->umc.funcs->query_ras_error_count(adev, ras_error_status); | |
| 109 | + | |
| 110 | + if (adev->umc.funcs && | |
| 111 | + adev->umc.funcs->query_ras_error_address && | |
| 112 | + adev->umc.max_ras_err_cnt_per_query) { | |
| 113 | + err_data->err_addr = | |
| 114 | + kcalloc(adev->umc.max_ras_err_cnt_per_query, | |
| 115 | + sizeof(struct eeprom_table_record), GFP_KERNEL); | |
| 116 | + /* still call query_ras_error_address to clear error status | |
| 117 | + * even NOMEM error is encountered | |
| 118 | + */ | |
| 119 | + if(!err_data->err_addr) | |
| 120 | + DRM_WARN("Failed to alloc memory for umc error address record!\n"); | |
| 121 | + | |
| 122 | + /* umc query_ras_error_address is also responsible for clearing | |
| 123 | + * error status | |
| 124 | + */ | |
| 125 | + adev->umc.funcs->query_ras_error_address(adev, ras_error_status); | |
| 126 | + } | |
| 127 | + | |
| 128 | + /* only uncorrectable error needs gpu reset */ | |
| 129 | + if (err_data->ue_count) { | |
| 130 | + if (err_data->err_addr_cnt && | |
| 131 | + amdgpu_ras_add_bad_pages(adev, err_data->err_addr, | |
| 132 | + err_data->err_addr_cnt)) | |
| 133 | + DRM_WARN("Failed to add ras bad page!\n"); | |
| 134 | + | |
| 135 | + amdgpu_ras_reset_gpu(adev, 0); | |
| 136 | + } | |
| 137 | + | |
| 138 | + kfree(err_data->err_addr); | |
| 139 | + return AMDGPU_RAS_SUCCESS; | |
| 140 | +} | |
| 141 | + | |
| 142 | +int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, | |
| 143 | + struct amdgpu_irq_src *source, | |
| 144 | + struct amdgpu_iv_entry *entry) | |
| 145 | +{ | |
| 146 | + struct ras_common_if *ras_if = adev->umc.ras_if; | |
| 147 | + struct ras_dispatch_if ih_data = { | |
| 148 | + .entry = entry, | |
| 149 | + }; | |
| 150 | + | |
| 151 | + if (!ras_if) | |
| 152 | + return 0; | |
| 153 | + | |
| 154 | + ih_data.head = *ras_if; | |
| 155 | + | |
| 156 | + amdgpu_ras_interrupt_dispatch(adev, &ih_data); | |
| 157 | + return 0; | |
| 158 | +} | |
| @@ -54,7 +54,8 @@ · 53 unmodified lines | ||
| 54 | 54 | adev->umc.funcs->disable_umc_index_mode(adev); |
| 55 | 55 | |
| 56 | 56 | struct amdgpu_umc_funcs { |
| 57 | − void (*ras_init)(struct amdgpu_device *adev); | |
| 57 | + void (*err_cnt_init)(struct amdgpu_device *adev); | |
| 58 | + int (*ras_late_init)(struct amdgpu_device *adev); | |
| 58 | 59 | void (*query_ras_error_count)(struct amdgpu_device *adev, |
| 59 | 60 | void *ras_error_status); |
| 60 | 61 | void (*query_ras_error_address)(struct amdgpu_device *adev, |
| @@ -62,6 +63,7 @@ struct amdgpu_umc_funcs { · 1 unmodified line | ||
| 62 | 63 | void (*enable_umc_index_mode)(struct amdgpu_device *adev, |
| 63 | 64 | uint32_t umc_instance); |
| 64 | 65 | void (*disable_umc_index_mode)(struct amdgpu_device *adev); |
| 66 | + void (*init_registers)(struct amdgpu_device *adev); | |
| 65 | 67 | }; |
| 66 | 68 | |
| 67 | 69 | struct amdgpu_umc { |
| @@ -75,8 +77,17 @@ struct amdgpu_umc { · 7 unmodified lines | ||
| 75 | 77 | uint32_t channel_offs; |
| 76 | 78 | /* channel index table of interleaved memory */ |
| 77 | 79 | const uint32_t *channel_idx_tbl; |
| 80 | + struct ras_common_if *ras_if; | |
| 78 | 81 | |
| 79 | 82 | const struct amdgpu_umc_funcs *funcs; |
| 80 | 83 | }; |
| 81 | 84 | |
| 85 | +int amdgpu_umc_ras_late_init(struct amdgpu_device *adev); | |
| 86 | +void amdgpu_umc_ras_fini(struct amdgpu_device *adev); | |
| 87 | +int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | |
| 88 | + void *ras_error_status, | |
| 89 | + struct amdgpu_iv_entry *entry); | |
| 90 | +int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, | |
| 91 | + struct amdgpu_irq_src *source, | |
| 92 | + struct amdgpu_iv_entry *entry); | |
| 82 | 93 | #endif |
| @@ -39,6 +39,8 @@ · 38 unmodified lines | ||
| 39 | 39 | #include "cikd.h" |
| 40 | 40 | #include "uvd/uvd_4_2_d.h" |
| 41 | 41 | |
| 42 | +#include "amdgpu_ras.h" | |
| 43 | + | |
| 42 | 44 | /* 1 second timeout */ |
| 43 | 45 | #define UVD_IDLE_TIMEOUT msecs_to_jiffies(1000) |
| 44 | 46 | |
| @@ -372,7 +374,13 @@ int amdgpu_uvd_suspend(struct amdgpu_device *adev) · 327 unmodified lines | ||
| 372 | 374 | if (!adev->uvd.inst[j].saved_bo) |
| 373 | 375 | return -ENOMEM; |
| 374 | 376 | |
| 375 | − memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); | |
| 377 | + /* re-write 0 since err_event_athub will corrupt VCPU buffer */ | |
| 378 | + if (amdgpu_ras_intr_triggered()) { | |
| 379 | + DRM_WARN("UVD VCPU state may lost due to RAS ERREVENT_ATHUB_INTERRUPT\n"); | |
| 380 | + memset(adev->uvd.inst[j].saved_bo, 0, size); | |
| 381 | + } else { | |
| 382 | + memcpy_fromio(adev->uvd.inst[j].saved_bo, ptr, size); | |
| 383 | + } | |
| 376 | 384 | } |
| 377 | 385 | return 0; |
| 378 | 386 | } |
| @@ -80,6 +80,11 @@ MODULE_FIRMWARE(FIRMWARE_VEGA12); · 79 unmodified lines | ||
| 80 | 80 | MODULE_FIRMWARE(FIRMWARE_VEGA20); |
| 81 | 81 | |
| 82 | 82 | static void amdgpu_vce_idle_work_handler(struct work_struct *work); |
| 83 | +static int amdgpu_vce_get_create_msg(struct amdgpu_ring *ring, uint32_t handle, | |
| 84 | + struct amdgpu_bo *bo, | |
| 85 | + struct dma_fence **fence); | |
| 86 | +static int amdgpu_vce_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, | |
| 87 | + bool direct, struct dma_fence **fence); | |
| 83 | 88 | |
| 84 | 89 | /** |
| 85 | 90 | * amdgpu_vce_init - allocate memory, load vce firmware |
| @@ -428,14 +433,15 @@ void amdgpu_vce_free_handles(struct amdgpu_device *adev, struct drm_file *filp) · 342 unmodified lines | ||
| 428 | 433 | * |
| 429 | 434 | * Open up a stream for HW test |
| 430 | 435 | */ |
| 431 | −int amdgpu_vce_get_create_msg(struct amdgpu_ring *ring, uint32_t handle, | |
| 432 | − struct dma_fence **fence) | |
| 436 | +static int amdgpu_vce_get_create_msg(struct amdgpu_ring *ring, uint32_t handle, | |
| 437 | + struct amdgpu_bo *bo, | |
| 438 | + struct dma_fence **fence) | |
| 433 | 439 | { |
| 434 | 440 | const unsigned ib_size_dw = 1024; |
| 435 | 441 | struct amdgpu_job *job; |
| 436 | 442 | struct amdgpu_ib *ib; |
| 437 | 443 | struct dma_fence *f = NULL; |
| 438 | − uint64_t dummy; | |
| 444 | + uint64_t addr; | |
| 439 | 445 | int i, r; |
| 440 | 446 | |
| 441 | 447 | r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4, &job); |
| @@ -444,7 +450,7 @@ int amdgpu_vce_get_create_msg(struct amdgpu_ring *ring, uint32_t handle, · 2 unmodified lines | ||
| 444 | 450 | |
| 445 | 451 | ib = &job->ibs[0]; |
| 446 | 452 | |
| 447 | − dummy = ib->gpu_addr + 1024; | |
| 453 | + addr = amdgpu_bo_gpu_offset(bo); | |
| 448 | 454 | |
| 449 | 455 | /* stitch together an VCE create msg */ |
| 450 | 456 | ib->length_dw = 0; |
| @@ -476,8 +482,8 @@ int amdgpu_vce_get_create_msg(struct amdgpu_ring *ring, uint32_t handle, · 25 unmodified lines | ||
| 476 | 482 | |
| 477 | 483 | ib->ptr[ib->length_dw++] = 0x00000014; /* len */ |
| 478 | 484 | ib->ptr[ib->length_dw++] = 0x05000005; /* feedback buffer */ |
| 479 | − ib->ptr[ib->length_dw++] = upper_32_bits(dummy); | |
| 480 | − ib->ptr[ib->length_dw++] = dummy; | |
| 485 | + ib->ptr[ib->length_dw++] = upper_32_bits(addr); | |
| 486 | + ib->ptr[ib->length_dw++] = addr; | |
| 481 | 487 | ib->ptr[ib->length_dw++] = 0x00000001; |
| 482 | 488 | |
| 483 | 489 | for (i = ib->length_dw; i < ib_size_dw; ++i) |
| @@ -507,8 +513,8 @@ err: · 23 unmodified lines | ||
| 507 | 513 | * |
| 508 | 514 | * Close up a stream for HW test or if userspace failed to do so |
| 509 | 515 | */ |
| 510 | −int amdgpu_vce_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, | |
| 511 | − bool direct, struct dma_fence **fence) | |
| 516 | +static int amdgpu_vce_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, | |
| 517 | + bool direct, struct dma_fence **fence) | |
| 512 | 518 | { |
| 513 | 519 | const unsigned ib_size_dw = 1024; |
| 514 | 520 | struct amdgpu_job *job; |
| @@ -1110,13 +1116,20 @@ int amdgpu_vce_ring_test_ring(struct amdgpu_ring *ring) · 595 unmodified lines | ||
| 1110 | 1116 | int amdgpu_vce_ring_test_ib(struct amdgpu_ring *ring, long timeout) |
| 1111 | 1117 | { |
| 1112 | 1118 | struct dma_fence *fence = NULL; |
| 1119 | + struct amdgpu_bo *bo = NULL; | |
| 1113 | 1120 | long r; |
| 1114 | 1121 | |
| 1115 | 1122 | /* skip vce ring1/2 ib test for now, since it's not reliable */ |
| 1116 | 1123 | if (ring != &ring->adev->vce.ring[0]) |
| 1117 | 1124 | return 0; |
| 1118 | 1125 | |
| 1119 | − r = amdgpu_vce_get_create_msg(ring, 1, NULL); | |
| 1126 | + r = amdgpu_bo_create_reserved(ring->adev, 512, PAGE_SIZE, | |
| 1127 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 1128 | + &bo, NULL, NULL); | |
| 1129 | + if (r) | |
| 1130 | + return r; | |
| 1131 | + | |
| 1132 | + r = amdgpu_vce_get_create_msg(ring, 1, bo, NULL); | |
| 1120 | 1133 | if (r) |
| 1121 | 1134 | goto error; |
| 1122 | 1135 | |
| @@ -1132,5 +1145,7 @@ int amdgpu_vce_ring_test_ib(struct amdgpu_ring *ring, long timeout) · 9 unmodified lines | ||
| 1132 | 1145 | |
| 1133 | 1146 | error: |
| 1134 | 1147 | dma_fence_put(fence); |
| 1148 | + amdgpu_bo_unreserve(bo); | |
| 1149 | + amdgpu_bo_unref(&bo); | |
| 1135 | 1150 | return r; |
| 1136 | 1151 | } |
| @@ -58,10 +58,6 @@ int amdgpu_vce_sw_fini(struct amdgpu_device *adev); · 57 unmodified lines | ||
| 58 | 58 | int amdgpu_vce_entity_init(struct amdgpu_device *adev); |
| 59 | 59 | int amdgpu_vce_suspend(struct amdgpu_device *adev); |
| 60 | 60 | int amdgpu_vce_resume(struct amdgpu_device *adev); |
| 61 | −int amdgpu_vce_get_create_msg(struct amdgpu_ring *ring, uint32_t handle, | |
| 62 | − struct dma_fence **fence); | |
| 63 | −int amdgpu_vce_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, | |
| 64 | − bool direct, struct dma_fence **fence); | |
| 65 | 61 | void amdgpu_vce_free_handles(struct amdgpu_device *adev, struct drm_file *filp); |
| 66 | 62 | int amdgpu_vce_ring_parse_cs(struct amdgpu_cs_parser *p, uint32_t ib_idx); |
| 67 | 63 | int amdgpu_vce_ring_parse_cs_vm(struct amdgpu_cs_parser *p, uint32_t ib_idx); |
| @@ -569,13 +569,14 @@ int amdgpu_vcn_enc_ring_test_ring(struct amdgpu_ring *ring) · 568 unmodified lines | ||
| 569 | 569 | } |
| 570 | 570 | |
| 571 | 571 | static int amdgpu_vcn_enc_get_create_msg(struct amdgpu_ring *ring, uint32_t handle, |
| 572 | − struct dma_fence **fence) | |
| 572 | + struct amdgpu_bo *bo, | |
| 573 | + struct dma_fence **fence) | |
| 573 | 574 | { |
| 574 | 575 | const unsigned ib_size_dw = 16; |
| 575 | 576 | struct amdgpu_job *job; |
| 576 | 577 | struct amdgpu_ib *ib; |
| 577 | 578 | struct dma_fence *f = NULL; |
| 578 | − uint64_t dummy; | |
| 579 | + uint64_t addr; | |
| 579 | 580 | int i, r; |
| 580 | 581 | |
| 581 | 582 | r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4, &job); |
| @@ -583,14 +584,14 @@ static int amdgpu_vcn_enc_get_create_msg(struct amdgpu_ring *ring, uint32_t hand · 1 unmodified line | ||
| 583 | 584 | return r; |
| 584 | 585 | |
| 585 | 586 | ib = &job->ibs[0]; |
| 586 | − dummy = ib->gpu_addr + 1024; | |
| 587 | + addr = amdgpu_bo_gpu_offset(bo); | |
| 587 | 588 | |
| 588 | 589 | ib->length_dw = 0; |
| 589 | 590 | ib->ptr[ib->length_dw++] = 0x00000018; |
| 590 | 591 | ib->ptr[ib->length_dw++] = 0x00000001; /* session info */ |
| 591 | 592 | ib->ptr[ib->length_dw++] = handle; |
| 592 | − ib->ptr[ib->length_dw++] = upper_32_bits(dummy); | |
| 593 | − ib->ptr[ib->length_dw++] = dummy; | |
| 593 | + ib->ptr[ib->length_dw++] = upper_32_bits(addr); | |
| 594 | + ib->ptr[ib->length_dw++] = addr; | |
| 594 | 595 | ib->ptr[ib->length_dw++] = 0x0000000b; |
| 595 | 596 | |
| 596 | 597 | ib->ptr[ib->length_dw++] = 0x00000014; |
| @@ -621,13 +622,14 @@ err: · 24 unmodified lines | ||
| 621 | 622 | } |
| 622 | 623 | |
| 623 | 624 | static int amdgpu_vcn_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t handle, |
| 624 | − struct dma_fence **fence) | |
| 625 | + struct amdgpu_bo *bo, | |
| 626 | + struct dma_fence **fence) | |
| 625 | 627 | { |
| 626 | 628 | const unsigned ib_size_dw = 16; |
| 627 | 629 | struct amdgpu_job *job; |
| 628 | 630 | struct amdgpu_ib *ib; |
| 629 | 631 | struct dma_fence *f = NULL; |
| 630 | − uint64_t dummy; | |
| 632 | + uint64_t addr; | |
| 631 | 633 | int i, r; |
| 632 | 634 | |
| 633 | 635 | r = amdgpu_job_alloc_with_ib(ring->adev, ib_size_dw * 4, &job); |
| @@ -635,14 +637,14 @@ static int amdgpu_vcn_enc_get_destroy_msg(struct amdgpu_ring *ring, uint32_t han · 1 unmodified line | ||
| 635 | 637 | return r; |
| 636 | 638 | |
| 637 | 639 | ib = &job->ibs[0]; |
| 638 | − dummy = ib->gpu_addr + 1024; | |
| 640 | + addr = amdgpu_bo_gpu_offset(bo); | |
| 639 | 641 | |
| 640 | 642 | ib->length_dw = 0; |
| 641 | 643 | ib->ptr[ib->length_dw++] = 0x00000018; |
| 642 | 644 | ib->ptr[ib->length_dw++] = 0x00000001; |
| 643 | 645 | ib->ptr[ib->length_dw++] = handle; |
| 644 | − ib->ptr[ib->length_dw++] = upper_32_bits(dummy); | |
| 645 | − ib->ptr[ib->length_dw++] = dummy; | |
| 646 | + ib->ptr[ib->length_dw++] = upper_32_bits(addr); | |
| 647 | + ib->ptr[ib->length_dw++] = addr; | |
| 646 | 648 | ib->ptr[ib->length_dw++] = 0x0000000b; |
| 647 | 649 | |
| 648 | 650 | ib->ptr[ib->length_dw++] = 0x00000014; |
| @@ -675,13 +677,20 @@ err: · 26 unmodified lines | ||
| 675 | 677 | int amdgpu_vcn_enc_ring_test_ib(struct amdgpu_ring *ring, long timeout) |
| 676 | 678 | { |
| 677 | 679 | struct dma_fence *fence = NULL; |
| 680 | + struct amdgpu_bo *bo = NULL; | |
| 678 | 681 | long r; |
| 679 | 682 | |
| 680 | − r = amdgpu_vcn_enc_get_create_msg(ring, 1, NULL); | |
| 683 | + r = amdgpu_bo_create_reserved(ring->adev, 128 * 1024, PAGE_SIZE, | |
| 684 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 685 | + &bo, NULL, NULL); | |
| 686 | + if (r) | |
| 687 | + return r; | |
| 688 | + | |
| 689 | + r = amdgpu_vcn_enc_get_create_msg(ring, 1, bo, NULL); | |
| 681 | 690 | if (r) |
| 682 | 691 | goto error; |
| 683 | 692 | |
| 684 | − r = amdgpu_vcn_enc_get_destroy_msg(ring, 1, &fence); | |
| 693 | + r = amdgpu_vcn_enc_get_destroy_msg(ring, 1, bo, &fence); | |
| 685 | 694 | if (r) |
| 686 | 695 | goto error; |
| 687 | 696 | |
| @@ -693,6 +702,8 @@ int amdgpu_vcn_enc_ring_test_ib(struct amdgpu_ring *ring, long timeout) · 5 unmodified lines | ||
| 693 | 702 | |
| 694 | 703 | error: |
| 695 | 704 | dma_fence_put(fence); |
| 705 | + amdgpu_bo_unreserve(bo); | |
| 706 | + amdgpu_bo_unref(&bo); | |
| 696 | 707 | return r; |
| 697 | 708 | } |
| 698 | 709 | |
| @@ -130,7 +130,8 @@ static unsigned amdgpu_vm_num_entries(struct amdgpu_device *adev, · 129 unmodified lines | ||
| 130 | 130 | |
| 131 | 131 | if (level == adev->vm_manager.root_level) |
| 132 | 132 | /* For the root directory */ |
| 133 | − return round_up(adev->vm_manager.max_pfn, 1ULL << shift) >> shift; | |
| 133 | + return round_up(adev->vm_manager.max_pfn, 1ULL << shift) | |
| 134 | + >> shift; | |
| 134 | 135 | else if (level != AMDGPU_VM_PTB) |
| 135 | 136 | /* Everything in between */ |
| 136 | 137 | return 512; |
| @@ -341,7 +342,7 @@ static struct amdgpu_vm_pt *amdgpu_vm_pt_parent(struct amdgpu_vm_pt *pt) · 204 unmodified lines | ||
| 341 | 342 | return container_of(parent->vm_bo, struct amdgpu_vm_pt, base); |
| 342 | 343 | } |
| 343 | 344 | |
| 344 | −/** | |
| 345 | +/* | |
| 345 | 346 | * amdgpu_vm_pt_cursor - state for for_each_amdgpu_vm_pt |
| 346 | 347 | */ |
| 347 | 348 | struct amdgpu_vm_pt_cursor { |
| @@ -482,6 +483,7 @@ static void amdgpu_vm_pt_next(struct amdgpu_device *adev, · 134 unmodified lines | ||
| 482 | 483 | * |
| 483 | 484 | * @adev: amdgpu_device structure |
| 484 | 485 | * @vm: amdgpu_vm structure |
| 486 | + * @start: optional cursor to start with | |
| 485 | 487 | * @cursor: state to initialize |
| 486 | 488 | * |
| 487 | 489 | * Starts a deep first traversal of the PD/PT tree. |
| @@ -535,7 +537,7 @@ static void amdgpu_vm_pt_next_dfs(struct amdgpu_device *adev, · 47 unmodified lines | ||
| 535 | 537 | amdgpu_vm_pt_ancestor(cursor); |
| 536 | 538 | } |
| 537 | 539 | |
| 538 | −/** | |
| 540 | +/* | |
| 539 | 541 | * for_each_amdgpu_vm_pt_dfs_safe - safe deep first search of all PDs/PTs |
| 540 | 542 | */ |
| 541 | 543 | #define for_each_amdgpu_vm_pt_dfs_safe(adev, vm, start, cursor, entry) \ |
| @@ -566,6 +568,14 @@ void amdgpu_vm_get_pd_bo(struct amdgpu_vm *vm, · 24 unmodified lines | ||
| 566 | 568 | list_add(&entry->tv.head, validated); |
| 567 | 569 | } |
| 568 | 570 | |
| 571 | +/** | |
| 572 | + * amdgpu_vm_del_from_lru_notify - update bulk_moveable flag | |
| 573 | + * | |
| 574 | + * @bo: BO which was removed from the LRU | |
| 575 | + * | |
| 576 | + * Make sure the bulk_moveable flag is updated when a BO is removed from the | |
| 577 | + * LRU. | |
| 578 | + */ | |
| 569 | 579 | void amdgpu_vm_del_from_lru_notify(struct ttm_buffer_object *bo) |
| 570 | 580 | { |
| 571 | 581 | struct amdgpu_bo *abo; |
| @@ -600,19 +610,18 @@ void amdgpu_vm_del_from_lru_notify(struct ttm_buffer_object *bo) · 28 unmodified lines | ||
| 600 | 610 | void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev, |
| 601 | 611 | struct amdgpu_vm *vm) |
| 602 | 612 | { |
| 603 | − struct ttm_bo_global *glob = adev->mman.bdev.glob; | |
| 604 | 613 | struct amdgpu_vm_bo_base *bo_base; |
| 605 | 614 | |
| 606 | 615 | if (vm->bulk_moveable) { |
| 607 | − spin_lock(&glob->lru_lock); | |
| 616 | + spin_lock(&ttm_bo_glob.lru_lock); | |
| 608 | 617 | ttm_bo_bulk_move_lru_tail(&vm->lru_bulk_move); |
| 609 | − spin_unlock(&glob->lru_lock); | |
| 618 | + spin_unlock(&ttm_bo_glob.lru_lock); | |
| 610 | 619 | return; |
| 611 | 620 | } |
| 612 | 621 | |
| 613 | 622 | memset(&vm->lru_bulk_move, 0, sizeof(vm->lru_bulk_move)); |
| 614 | 623 | |
| 615 | − spin_lock(&glob->lru_lock); | |
| 624 | + spin_lock(&ttm_bo_glob.lru_lock); | |
| 616 | 625 | list_for_each_entry(bo_base, &vm->idle, vm_status) { |
| 617 | 626 | struct amdgpu_bo *bo = bo_base->bo; |
| 618 | 627 | |
| @@ -624,7 +633,7 @@ void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev, · 5 unmodified lines | ||
| 624 | 633 | ttm_bo_move_to_lru_tail(&bo->shadow->tbo, |
| 625 | 634 | &vm->lru_bulk_move); |
| 626 | 635 | } |
| 627 | − spin_unlock(&glob->lru_lock); | |
| 636 | + spin_unlock(&ttm_bo_glob.lru_lock); | |
| 628 | 637 | |
| 629 | 638 | vm->bulk_moveable = true; |
| 630 | 639 | } |
| @@ -693,6 +702,7 @@ bool amdgpu_vm_ready(struct amdgpu_vm *vm) · 62 unmodified lines | ||
| 693 | 702 | * @adev: amdgpu_device pointer |
| 694 | 703 | * @vm: VM to clear BO from |
| 695 | 704 | * @bo: BO to clear |
| 705 | + * @direct: use a direct update | |
| 696 | 706 | * |
| 697 | 707 | * Root PD needs to be reserved when calling this. |
| 698 | 708 | * |
| @@ -701,7 +711,8 @@ bool amdgpu_vm_ready(struct amdgpu_vm *vm) · 2 unmodified lines | ||
| 701 | 711 | */ |
| 702 | 712 | static int amdgpu_vm_clear_bo(struct amdgpu_device *adev, |
| 703 | 713 | struct amdgpu_vm *vm, |
| 704 | − struct amdgpu_bo *bo) | |
| 714 | + struct amdgpu_bo *bo, | |
| 715 | + bool direct) | |
| 705 | 716 | { |
| 706 | 717 | struct ttm_operation_ctx ctx = { true, false }; |
| 707 | 718 | unsigned level = adev->vm_manager.root_level; |
| @@ -760,6 +771,7 @@ static int amdgpu_vm_clear_bo(struct amdgpu_device *adev, · 52 unmodified lines | ||
| 760 | 771 | memset(¶ms, 0, sizeof(params)); |
| 761 | 772 | params.adev = adev; |
| 762 | 773 | params.vm = vm; |
| 774 | + params.direct = direct; | |
| 763 | 775 | |
| 764 | 776 | r = vm->update_funcs->prepare(¶ms, AMDGPU_FENCE_OWNER_KFD, NULL); |
| 765 | 777 | if (r) |
| @@ -813,10 +825,13 @@ static int amdgpu_vm_clear_bo(struct amdgpu_device *adev, · 47 unmodified lines | ||
| 813 | 825 | * |
| 814 | 826 | * @adev: amdgpu_device pointer |
| 815 | 827 | * @vm: requesting vm |
| 828 | + * @level: the page table level | |
| 829 | + * @direct: use a direct update | |
| 816 | 830 | * @bp: resulting BO allocation parameters |
| 817 | 831 | */ |
| 818 | 832 | static void amdgpu_vm_bo_param(struct amdgpu_device *adev, struct amdgpu_vm *vm, |
| 819 | − int level, struct amdgpu_bo_param *bp) | |
| 833 | + int level, bool direct, | |
| 834 | + struct amdgpu_bo_param *bp) | |
| 820 | 835 | { |
| 821 | 836 | memset(bp, 0, sizeof(*bp)); |
| 822 | 837 | |
| @@ -831,6 +846,7 @@ static void amdgpu_vm_bo_param(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 8 unmodified lines | ||
| 831 | 846 | else if (!vm->root.base.bo || vm->root.base.bo->shadow) |
| 832 | 847 | bp->flags |= AMDGPU_GEM_CREATE_SHADOW; |
| 833 | 848 | bp->type = ttm_bo_type_kernel; |
| 849 | + bp->no_wait_gpu = direct; | |
| 834 | 850 | if (vm->root.base.bo) |
| 835 | 851 | bp->resv = vm->root.base.bo->tbo.base.resv; |
| 836 | 852 | } |
| @@ -841,6 +857,7 @@ static void amdgpu_vm_bo_param(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 4 unmodified lines | ||
| 841 | 857 | * @adev: amdgpu_device pointer |
| 842 | 858 | * @vm: VM to allocate page tables for |
| 843 | 859 | * @cursor: Which page table to allocate |
| 860 | + * @direct: use a direct update | |
| 844 | 861 | * |
| 845 | 862 | * Make sure a specific page table or directory is allocated. |
| 846 | 863 | * |
| @@ -850,7 +867,8 @@ static void amdgpu_vm_bo_param(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 3 unmodified lines | ||
| 850 | 867 | */ |
| 851 | 868 | static int amdgpu_vm_alloc_pts(struct amdgpu_device *adev, |
| 852 | 869 | struct amdgpu_vm *vm, |
| 853 | − struct amdgpu_vm_pt_cursor *cursor) | |
| 870 | + struct amdgpu_vm_pt_cursor *cursor, | |
| 871 | + bool direct) | |
| 854 | 872 | { |
| 855 | 873 | struct amdgpu_vm_pt *entry = cursor->entry; |
| 856 | 874 | struct amdgpu_bo_param bp; |
| @@ -871,7 +889,7 @@ static int amdgpu_vm_alloc_pts(struct amdgpu_device *adev, · 14 unmodified lines | ||
| 871 | 889 | if (entry->base.bo) |
| 872 | 890 | return 0; |
| 873 | 891 | |
| 874 | − amdgpu_vm_bo_param(adev, vm, cursor->level, &bp); | |
| 892 | + amdgpu_vm_bo_param(adev, vm, cursor->level, direct, &bp); | |
| 875 | 893 | |
| 876 | 894 | r = amdgpu_bo_create(adev, &bp, &pt); |
| 877 | 895 | if (r) |
| @@ -883,7 +901,7 @@ static int amdgpu_vm_alloc_pts(struct amdgpu_device *adev, · 5 unmodified lines | ||
| 883 | 901 | pt->parent = amdgpu_bo_ref(cursor->parent->base.bo); |
| 884 | 902 | amdgpu_vm_bo_base_init(&entry->base, vm, pt); |
| 885 | 903 | |
| 886 | − r = amdgpu_vm_clear_bo(adev, vm, pt); | |
| 904 | + r = amdgpu_vm_clear_bo(adev, vm, pt, direct); | |
| 887 | 905 | if (r) |
| 888 | 906 | goto error_free_pt; |
| 889 | 907 | |
| @@ -1020,7 +1038,8 @@ bool amdgpu_vm_need_pipeline_sync(struct amdgpu_ring *ring, · 130 unmodified lines | ||
| 1020 | 1038 | * Returns: |
| 1021 | 1039 | * 0 on success, errno otherwise. |
| 1022 | 1040 | */ |
| 1023 | −int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job, bool need_pipe_sync) | |
| 1041 | +int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job, | |
| 1042 | + bool need_pipe_sync) | |
| 1024 | 1043 | { |
| 1025 | 1044 | struct amdgpu_device *adev = ring->adev; |
| 1026 | 1045 | unsigned vmhub = ring->funcs->vmhub; |
| @@ -1034,10 +1053,8 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job, bool need_ · 7 unmodified lines | ||
| 1034 | 1053 | id->oa_base != job->oa_base || |
| 1035 | 1054 | id->oa_size != job->oa_size); |
| 1036 | 1055 | bool vm_flush_needed = job->vm_needs_flush; |
| 1037 | − bool pasid_mapping_needed = id->pasid != job->pasid || | |
| 1038 | − !id->pasid_mapping || | |
| 1039 | − !dma_fence_is_signaled(id->pasid_mapping); | |
| 1040 | 1056 | struct dma_fence *fence = NULL; |
| 1057 | + bool pasid_mapping_needed = false; | |
| 1041 | 1058 | unsigned patch_offset = 0; |
| 1042 | 1059 | int r; |
| 1043 | 1060 | |
| @@ -1047,6 +1064,12 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job, bool need_ · 3 unmodified lines | ||
| 1047 | 1064 | pasid_mapping_needed = true; |
| 1048 | 1065 | } |
| 1049 | 1066 | |
| 1067 | + mutex_lock(&id_mgr->lock); | |
| 1068 | + if (id->pasid != job->pasid || !id->pasid_mapping || | |
| 1069 | + !dma_fence_is_signaled(id->pasid_mapping)) | |
| 1070 | + pasid_mapping_needed = true; | |
| 1071 | + mutex_unlock(&id_mgr->lock); | |
| 1072 | + | |
| 1050 | 1073 | gds_switch_needed &= !!ring->funcs->emit_gds_switch; |
| 1051 | 1074 | vm_flush_needed &= !!ring->funcs->emit_vm_flush && |
| 1052 | 1075 | job->vm_pd_addr != AMDGPU_BO_INVALID_OFFSET; |
| @@ -1086,9 +1109,11 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job, bool need_ · 33 unmodified lines | ||
| 1086 | 1109 | } |
| 1087 | 1110 | |
| 1088 | 1111 | if (pasid_mapping_needed) { |
| 1112 | + mutex_lock(&id_mgr->lock); | |
| 1089 | 1113 | id->pasid = job->pasid; |
| 1090 | 1114 | dma_fence_put(id->pasid_mapping); |
| 1091 | 1115 | id->pasid_mapping = dma_fence_get(fence); |
| 1116 | + mutex_unlock(&id_mgr->lock); | |
| 1092 | 1117 | } |
| 1093 | 1118 | dma_fence_put(fence); |
| 1094 | 1119 | |
| @@ -1172,10 +1197,10 @@ uint64_t amdgpu_vm_map_gart(const dma_addr_t *pages_addr, uint64_t addr) · 77 unmodified lines | ||
| 1172 | 1197 | return result; |
| 1173 | 1198 | } |
| 1174 | 1199 | |
| 1175 | −/* | |
| 1200 | +/** | |
| 1176 | 1201 | * amdgpu_vm_update_pde - update a single level in the hierarchy |
| 1177 | 1202 | * |
| 1178 | − * @param: parameters for the update | |
| 1203 | + * @params: parameters for the update | |
| 1179 | 1204 | * @vm: requested vm |
| 1180 | 1205 | * @entry: entry to update |
| 1181 | 1206 | * |
| @@ -1199,7 +1224,7 @@ static int amdgpu_vm_update_pde(struct amdgpu_vm_update_params *params, · 17 unmodified lines | ||
| 1199 | 1224 | return vm->update_funcs->update(params, bo, pde, pt, 1, 0, flags); |
| 1200 | 1225 | } |
| 1201 | 1226 | |
| 1202 | −/* | |
| 1227 | +/** | |
| 1203 | 1228 | * amdgpu_vm_invalidate_pds - mark all PDs as invalid |
| 1204 | 1229 | * |
| 1205 | 1230 | * @adev: amdgpu_device pointer |
| @@ -1218,19 +1243,20 @@ static void amdgpu_vm_invalidate_pds(struct amdgpu_device *adev, · 12 unmodified lines | ||
| 1218 | 1243 | amdgpu_vm_bo_relocated(&entry->base); |
| 1219 | 1244 | } |
| 1220 | 1245 | |
| 1221 | −/* | |
| 1222 | − * amdgpu_vm_update_directories - make sure that all directories are valid | |
| 1246 | +/** | |
| 1247 | + * amdgpu_vm_update_pdes - make sure that all directories are valid | |
| 1223 | 1248 | * |
| 1224 | 1249 | * @adev: amdgpu_device pointer |
| 1225 | 1250 | * @vm: requested vm |
| 1251 | + * @direct: submit directly to the paging queue | |
| 1226 | 1252 | * |
| 1227 | 1253 | * Makes sure all directories are up to date. |
| 1228 | 1254 | * |
| 1229 | 1255 | * Returns: |
| 1230 | 1256 | * 0 for success, error for failure. |
| 1231 | 1257 | */ |
| 1232 | −int amdgpu_vm_update_directories(struct amdgpu_device *adev, | |
| 1233 | − struct amdgpu_vm *vm) | |
| 1258 | +int amdgpu_vm_update_pdes(struct amdgpu_device *adev, | |
| 1259 | + struct amdgpu_vm *vm, bool direct) | |
| 1234 | 1260 | { |
| 1235 | 1261 | struct amdgpu_vm_update_params params; |
| 1236 | 1262 | int r; |
| @@ -1241,6 +1267,7 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev, · 4 unmodified lines | ||
| 1241 | 1267 | memset(¶ms, 0, sizeof(params)); |
| 1242 | 1268 | params.adev = adev; |
| 1243 | 1269 | params.vm = vm; |
| 1270 | + params.direct = direct; | |
| 1244 | 1271 | |
| 1245 | 1272 | r = vm->update_funcs->prepare(¶ms, AMDGPU_FENCE_OWNER_VM, NULL); |
| 1246 | 1273 | if (r) |
| @@ -1268,7 +1295,7 @@ error: · 21 unmodified lines | ||
| 1268 | 1295 | return r; |
| 1269 | 1296 | } |
| 1270 | 1297 | |
| 1271 | −/** | |
| 1298 | +/* | |
| 1272 | 1299 | * amdgpu_vm_update_flags - figure out flags for PTE updates |
| 1273 | 1300 | * |
| 1274 | 1301 | * Make sure to set the right flags for the PTEs at the desired level. |
| @@ -1391,7 +1418,11 @@ static int amdgpu_vm_update_ptes(struct amdgpu_vm_update_params *params, · 116 unmodified lines | ||
| 1391 | 1418 | uint64_t incr, entry_end, pe_start; |
| 1392 | 1419 | struct amdgpu_bo *pt; |
| 1393 | 1420 | |
| 1394 | − r = amdgpu_vm_alloc_pts(params->adev, params->vm, &cursor); | |
| 1421 | + /* make sure that the page tables covering the address range are | |
| 1422 | + * actually allocated | |
| 1423 | + */ | |
| 1424 | + r = amdgpu_vm_alloc_pts(params->adev, params->vm, &cursor, | |
| 1425 | + params->direct); | |
| 1395 | 1426 | if (r) |
| 1396 | 1427 | return r; |
| 1397 | 1428 | |
| @@ -1463,7 +1494,12 @@ static int amdgpu_vm_update_ptes(struct amdgpu_vm_update_params *params, · 65 unmodified lines | ||
| 1463 | 1494 | } while (frag_start < entry_end); |
| 1464 | 1495 | |
| 1465 | 1496 | if (amdgpu_vm_pt_descendant(adev, &cursor)) { |
| 1466 | − /* Free all child entries */ | |
| 1497 | + /* Free all child entries. | |
| 1498 | + * Update the tables with the flags and addresses and free up subsequent | |
| 1499 | + * tables in the case of huge pages or freed up areas. | |
| 1500 | + * This is the maximum you can free, because all other page tables are not | |
| 1501 | + * completely covered by the range and so potentially still in use. | |
| 1502 | + */ | |
| 1467 | 1503 | while (cursor.pfn < frag_start) { |
| 1468 | 1504 | amdgpu_vm_free_pts(adev, params->vm, &cursor); |
| 1469 | 1505 | amdgpu_vm_pt_next(adev, &cursor); |
| @@ -1482,13 +1518,14 @@ static int amdgpu_vm_update_ptes(struct amdgpu_vm_update_params *params, · 12 unmodified lines | ||
| 1482 | 1518 | * amdgpu_vm_bo_update_mapping - update a mapping in the vm page table |
| 1483 | 1519 | * |
| 1484 | 1520 | * @adev: amdgpu_device pointer |
| 1485 | − * @exclusive: fence we need to sync to | |
| 1486 | − * @pages_addr: DMA addresses to use for mapping | |
| 1487 | 1521 | * @vm: requested vm |
| 1522 | + * @direct: direct submission in a page fault | |
| 1523 | + * @exclusive: fence we need to sync to | |
| 1488 | 1524 | * @start: start of mapped range |
| 1489 | 1525 | * @last: last mapped entry |
| 1490 | 1526 | * @flags: flags for the entries |
| 1491 | 1527 | * @addr: addr to set the area to |
| 1528 | + * @pages_addr: DMA addresses to use for mapping | |
| 1492 | 1529 | * @fence: optional resulting fence |
| 1493 | 1530 | * |
| 1494 | 1531 | * Fill in the page table entries between @start and @last. |
| @@ -1497,11 +1534,11 @@ static int amdgpu_vm_update_ptes(struct amdgpu_vm_update_params *params, · 2 unmodified lines | ||
| 1497 | 1534 | * 0 for success, -EINVAL for failure. |
| 1498 | 1535 | */ |
| 1499 | 1536 | static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, |
| 1537 | + struct amdgpu_vm *vm, bool direct, | |
| 1500 | 1538 | struct dma_fence *exclusive, |
| 1501 | − dma_addr_t *pages_addr, | |
| 1502 | − struct amdgpu_vm *vm, | |
| 1503 | 1539 | uint64_t start, uint64_t last, |
| 1504 | 1540 | uint64_t flags, uint64_t addr, |
| 1541 | + dma_addr_t *pages_addr, | |
| 1505 | 1542 | struct dma_fence **fence) |
| 1506 | 1543 | { |
| 1507 | 1544 | struct amdgpu_vm_update_params params; |
| @@ -1511,6 +1548,7 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, · 3 unmodified lines | ||
| 1511 | 1548 | memset(¶ms, 0, sizeof(params)); |
| 1512 | 1549 | params.adev = adev; |
| 1513 | 1550 | params.vm = vm; |
| 1551 | + params.direct = direct; | |
| 1514 | 1552 | params.pages_addr = pages_addr; |
| 1515 | 1553 | |
| 1516 | 1554 | /* sync to everything except eviction fences on unmapping */ |
| @@ -1569,27 +1607,8 @@ static int amdgpu_vm_bo_split_mapping(struct amdgpu_device *adev, · 52 unmodified lines | ||
| 1569 | 1607 | if (!(mapping->flags & AMDGPU_PTE_WRITEABLE)) |
| 1570 | 1608 | flags &= ~AMDGPU_PTE_WRITEABLE; |
| 1571 | 1609 | |
| 1572 | − flags &= ~AMDGPU_PTE_EXECUTABLE; | |
| 1573 | − flags |= mapping->flags & AMDGPU_PTE_EXECUTABLE; | |
381 more lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 145 lines aren’t shown here.
This diff is large: its 145 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 146 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 229 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 1638 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 191 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 574 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 381 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 256 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 253 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 380 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 235 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 419 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 170 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 334 lines aren’t shown here.
This diff is large: its 307 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 735 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 347 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 306 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 300 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 402 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 493 lines aren’t shown here.
This diff is large: its 534 lines aren’t shown here.
This diff is large: its 170 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 152 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 144 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 146 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 170 lines aren’t shown here.
This diff is large: its 250 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 131 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 185 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 754 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 704 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 223 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 471 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 643 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 168 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 325 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 427 lines aren’t shown here.
This diff is large: its 443 lines aren’t shown here.
This diff is large: its 532 lines aren’t shown here.
This diff is large: its 308 lines aren’t shown here.
This diff is large: its 306 lines aren’t shown here.
This diff is large: its 164 lines aren’t shown here.
This diff is large: its 140 lines aren’t shown here.
This diff is large: its 329 lines aren’t shown here.
This diff is large: its 273 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 290 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 154 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 99 lines aren’t shown here.
This diff is large: its 183 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 1562 lines aren’t shown here.
This diff is large: its 681 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 196 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 197 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 223 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 232 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 114 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 480 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 313 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 538 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 733 lines aren’t shown here.
This diff is large: its 330 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 262 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 220 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 168 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 132 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 174 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 248 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 1607 lines aren’t shown here.
This diff is large: its 183 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 254 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 188 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 750 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 305 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 120 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 154 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 206 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 224 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 461 lines aren’t shown here.
This diff is large: its 442 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 289 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 148 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 274 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 212 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 261 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 102 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 171 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 877 lines aren’t shown here.
This diff is large: its 178 lines aren’t shown here.
This diff is large: its 340 lines aren’t shown here.
This diff is large: its 510 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 139 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 3237 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 264 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 487 lines aren’t shown here.
This diff is large: its 116 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 123 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 150 lines aren’t shown here.
This diff is large: its 142 lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 359 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 177 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 391 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 261 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 118 lines aren’t shown here.
This diff is large: its 146 lines aren’t shown here.
This diff is large: its 134 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 148 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 145 lines aren’t shown here.
This diff is large: its 191 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 149 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 216 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 258 lines aren’t shown here.
This diff is large: its 140 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 158 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 355 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 167 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 82 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 369 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 89 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 119 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 169 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 215 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 153 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 213 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 358 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 209 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 228 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 280 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 124 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 261 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 1262 lines aren’t shown here.
This diff is large: its 852 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 370 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 419 lines aren’t shown here.
This diff is large: its 371 lines aren’t shown here.
This diff is large: its 902 lines aren’t shown here.
This diff is large: its 502 lines aren’t shown here.
This diff is large: its 552 lines aren’t shown here.
This diff is large: its 372 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 160 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 151 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 201 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 164 lines aren’t shown here.
This diff is large: its 188 lines aren’t shown here.
This diff is large: its 229 lines aren’t shown here.
This diff is large: its 555 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 194 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 77 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 5 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 117 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 100 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 238 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 327 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 795 lines aren’t shown here.
This diff is large: its 184 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 205 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 214 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 68 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 243 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 122 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 60 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 84 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 136 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 85 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 244 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 155 lines aren’t shown here.
This diff is large: its 289 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 92 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 74 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 59 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 56 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 109 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 96 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 182 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 65 lines aren’t shown here.
This diff is large: its 163 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 42 lines aren’t shown here.
This diff is large: its 51 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 827 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 22 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 267 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 135 lines aren’t shown here.
This diff is large: its 108 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 78 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 113 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 186 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 133 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 98 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 104 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 79 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 110 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 75 lines aren’t shown here.
This diff is large: its 103 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 271 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 49 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 55 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 86 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 40 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 48 lines aren’t shown here.
This diff is large: its 128 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 165 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 47 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 107 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 111 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 70 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 81 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 64 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 76 lines aren’t shown here.
This diff is large: its 182 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 97 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 6 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 101 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 27 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 38 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 46 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 72 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 14 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
File mode changed from 100644 to 100755.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 115 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 2 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
File renamed without changes.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 105 lines aren’t shown here.
This diff is large: its 125 lines aren’t shown here.