To catch up with recent rounds of pull requests and get some drm-misc dependencies so we can merge linux/string_helpers related changes.
Unified · Split
| @@ -80,6 +80,9 @@ Chris Chiu <chris.chiu@canonical.com> <chiu@endlessos.org> · 79 unmodified lines | ||
| 80 | 80 | Christian Borntraeger <borntraeger@linux.ibm.com> <borntraeger@de.ibm.com> |
| 81 | 81 | Christian Borntraeger <borntraeger@linux.ibm.com> <cborntra@de.ibm.com> |
| 82 | 82 | Christian Borntraeger <borntraeger@linux.ibm.com> <borntrae@de.ibm.com> |
| 83 | +Christian Brauner <brauner@kernel.org> <christian@brauner.io> | |
| 84 | +Christian Brauner <brauner@kernel.org> <christian.brauner@canonical.com> | |
| 85 | +Christian Brauner <brauner@kernel.org> <christian.brauner@ubuntu.com> | |
| 83 | 86 | Christophe Ricard <christophe.ricard@gmail.com> |
| 84 | 87 | Christoph Hellwig <hch@lst.de> |
| 85 | 88 | Colin Ian King <colin.king@intel.com> <colin.king@canonical.com> |
| @@ -330,6 +333,9 @@ Rémi Denis-Courmont <rdenis@simphalempin.com> · 244 unmodified lines | ||
| 330 | 333 | Ricardo Ribalda <ribalda@kernel.org> <ricardo@ribalda.com> |
| 331 | 334 | Ricardo Ribalda <ribalda@kernel.org> Ricardo Ribalda Delgado <ribalda@kernel.org> |
| 332 | 335 | Ricardo Ribalda <ribalda@kernel.org> <ricardo.ribalda@gmail.com> |
| 336 | +Roman Gushchin <roman.gushchin@linux.dev> <guro@fb.com> | |
| 337 | +Roman Gushchin <roman.gushchin@linux.dev> <guroan@gmail.com> | |
| 338 | +Roman Gushchin <roman.gushchin@linux.dev> <klamm@yandex-team.ru> | |
| 333 | 339 | Ross Zwisler <zwisler@kernel.org> <ross.zwisler@linux.intel.com> |
| 334 | 340 | Rudolf Marek <R.Marek@sh.cvut.cz> |
| 335 | 341 | Rui Saraiva <rmps@joel.ist.utl.pt> |
| @@ -468,6 +468,7 @@ Description: · 467 unmodified lines | ||
| 468 | 468 | auto: Charge normally, respect thresholds |
| 469 | 469 | inhibit-charge: Do not charge while AC is attached |
| 470 | 470 | force-discharge: Force discharge while AC is attached |
| 471 | + ================ ==================================== | |
| 471 | 472 | |
| 472 | 473 | What: /sys/class/power_supply/<supply_name>/technology |
| 473 | 474 | Date: May 2007 |
| @@ -1,4 +1,4 @@ | ||
| 1 | −What: /sys/bus/platform/drivers/aspeed-uart-routing/*/uart* | |
| 1 | +What: /sys/bus/platform/drivers/aspeed-uart-routing/\*/uart\* | |
| 2 | 2 | Date: September 2021 |
| 3 | 3 | Contact: Oskar Senft <osk@google.com> |
| 4 | 4 | Chia-Wei Wang <chiawei_wang@aspeedtech.com> |
| @@ -9,7 +9,7 @@ Description: Selects the RX source of the UARTx device. · 4 unmodified lines | ||
| 9 | 9 | depends on the selected file. |
| 10 | 10 | |
| 11 | 11 | e.g. |
| 12 | − cat /sys/bus/platform/drivers/aspeed-uart-routing/*.uart_routing/uart1 | |
| 12 | + cat /sys/bus/platform/drivers/aspeed-uart-routing/\*.uart_routing/uart1 | |
| 13 | 13 | [io1] io2 io3 io4 uart2 uart3 uart4 io6 |
| 14 | 14 | |
| 15 | 15 | In this case, UART1 gets its input from IO1 (physical serial port 1). |
| @@ -17,7 +17,7 @@ Description: Selects the RX source of the UARTx device. · 1 unmodified line | ||
| 17 | 17 | Users: OpenBMC. Proposed changes should be mailed to |
| 18 | 18 | openbmc@lists.ozlabs.org |
| 19 | 19 | |
| 20 | −What: /sys/bus/platform/drivers/aspeed-uart-routing/*/io* | |
| 20 | +What: /sys/bus/platform/drivers/aspeed-uart-routing/\*/io\* | |
| 21 | 21 | Date: September 2021 |
| 22 | 22 | Contact: Oskar Senft <osk@google.com> |
| 23 | 23 | Chia-Wei Wang <chiawei_wang@aspeedtech.com> |
| @@ -100,6 +100,8 @@ stable kernels. · 99 unmodified lines | ||
| 100 | 100 | +----------------+-----------------+-----------------+-----------------------------+ |
| 101 | 101 | | ARM | Cortex-A510 | #2051678 | ARM64_ERRATUM_2051678 | |
| 102 | 102 | +----------------+-----------------+-----------------+-----------------------------+ |
| 103 | +| ARM | Cortex-A510 | #2077057 | ARM64_ERRATUM_2077057 | | |
| 104 | ++----------------+-----------------+-----------------+-----------------------------+ | |
| 103 | 105 | | ARM | Cortex-A710 | #2119858 | ARM64_ERRATUM_2119858 | |
| 104 | 106 | +----------------+-----------------+-----------------+-----------------------------+ |
| 105 | 107 | | ARM | Cortex-A710 | #2054223 | ARM64_ERRATUM_2054223 | |
| @@ -130,3 +130,11 @@ accesses to DMA buffers in both privileged "supervisor" and unprivileged · 129 unmodified lines | ||
| 130 | 130 | subsystem that the buffer is fully accessible at the elevated privilege |
| 131 | 131 | level (and ideally inaccessible or at least read-only at the |
| 132 | 132 | lesser-privileged levels). |
| 133 | + | |
| 134 | +DMA_ATTR_OVERWRITE | |
| 135 | +------------------ | |
| 136 | + | |
| 137 | +This is a hint to the DMA-mapping subsystem that the device is expected to | |
| 138 | +overwrite the entire mapped size, thus the caller does not require any of the | |
| 139 | +previous buffer contents to be preserved. This allows bounce-buffering | |
| 140 | +implementations to optimise DMA_FROM_DEVICE transfers. | |
| @@ -75,6 +75,9 @@ And optionally · 74 unmodified lines | ||
| 75 | 75 | .resume - A pointer to a per-policy resume function which is called |
| 76 | 76 | with interrupts disabled and _before_ the governor is started again. |
| 77 | 77 | |
| 78 | + .ready - A pointer to a per-policy ready function which is called after | |
| 79 | + the policy is fully initialized. | |
| 80 | + | |
| 78 | 81 | .attr - A pointer to a NULL-terminated list of "struct freq_attr" which |
| 79 | 82 | allow to export values to sysfs. |
| 80 | 83 | |
| @@ -7,6 +7,14 @@ directory. These are intended to be small tests to exercise individual code · 6 unmodified lines | ||
| 7 | 7 | paths in the kernel. Tests are intended to be run after building, installing |
| 8 | 8 | and booting a kernel. |
| 9 | 9 | |
| 10 | +Kselftest from mainline can be run on older stable kernels. Running tests | |
| 11 | +from mainline offers the best coverage. Several test rings run mainline | |
| 12 | +kselftest suite on stable releases. The reason is that when a new test | |
| 13 | +gets added to test existing code to regression test a bug, we should be | |
| 14 | +able to run that test on an older kernel. Hence, it is important to keep | |
| 15 | +code that can still test an older kernel and make sure it skips the test | |
| 16 | +gracefully on newer releases. | |
| 17 | + | |
| 10 | 18 | You can find additional information on Kselftest framework, how to |
| 11 | 19 | write new tests using the framework on Kselftest wiki: |
| 12 | 20 | |
| @@ -242,7 +242,7 @@ example: · 241 unmodified lines | ||
| 242 | 242 | |
| 243 | 243 | int rectangle_area(struct shape *this) |
| 244 | 244 | { |
| 245 | − struct rectangle *self = container_of(this, struct shape, parent); | |
| 245 | + struct rectangle *self = container_of(this, struct rectangle, parent); | |
| 246 | 246 | |
| 247 | 247 | return self->length * self->width; |
| 248 | 248 | }; |
| @@ -119,6 +119,9 @@ Boards (incomplete list of examples): · 118 unmodified lines | ||
| 119 | 119 | - OMAP3 BeagleBoard : Low cost community board |
| 120 | 120 | compatible = "ti,omap3-beagle", "ti,omap3430", "ti,omap3" |
| 121 | 121 | |
| 122 | +- OMAP3 BeagleBoard A to B4 : Early BeagleBoard revisions A to B4 with a timer quirk | |
| 123 | + compatible = "ti,omap3-beagle-ab4", "ti,omap3-beagle", "ti,omap3430", "ti,omap3" | |
| 124 | + | |
| 122 | 125 | - OMAP3 Tobi with Overo : Commercial expansion board with daughter board |
| 123 | 126 | compatible = "gumstix,omap3-overo-tobi", "gumstix,omap3-overo", "ti,omap3430", "ti,omap3" |
| 124 | 127 | |
| @@ -83,6 +83,9 @@ properties: · 82 unmodified lines | ||
| 83 | 83 | type: boolean |
| 84 | 84 | description: let the driver enable audio HDMI codec function or not. |
| 85 | 85 | |
| 86 | + aux-bus: | |
| 87 | + $ref: /schemas/display/dp-aux-bus.yaml# | |
| 88 | + | |
| 86 | 89 | ports: |
| 87 | 90 | $ref: /schemas/graph.yaml#/properties/ports |
| 88 | 91 | |
| @@ -167,5 +170,19 @@ examples: · 78 unmodified lines | ||
| 167 | 170 | }; |
| 168 | 171 | }; |
| 169 | 172 | }; |
| 173 | + | |
| 174 | + aux-bus { | |
| 175 | + panel { | |
| 176 | + compatible = "innolux,n125hce-gn1"; | |
| 177 | + power-supply = <&pp3300_disp_x>; | |
| 178 | + backlight = <&backlight_lcd0>; | |
| 179 | + | |
| 180 | + port { | |
| 181 | + panel_in: endpoint { | |
| 182 | + remote-endpoint = <&anx7625_out>; | |
| 183 | + }; | |
| 184 | + }; | |
| 185 | + }; | |
| 186 | + }; | |
| 170 | 187 | }; |
| 171 | 188 | }; |
| @@ -0,0 +1,82 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/bridge/ingenic,jz4780-hdmi.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Bindings for Ingenic JZ4780 HDMI Transmitter | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - H. Nikolaus Schaller <hns@goldelico.com> | |
| 11 | + | |
| 12 | +description: | | |
| 13 | + The HDMI Transmitter in the Ingenic JZ4780 is a Synopsys DesignWare HDMI 1.4 | |
| 14 | + TX controller IP with accompanying PHY IP. | |
| 15 | + | |
| 16 | +allOf: | |
| 17 | + - $ref: synopsys,dw-hdmi.yaml# | |
| 18 | + | |
| 19 | +properties: | |
| 20 | + compatible: | |
| 21 | + const: ingenic,jz4780-dw-hdmi | |
| 22 | + | |
| 23 | + reg-io-width: | |
| 24 | + const: 4 | |
| 25 | + | |
| 26 | + clocks: | |
| 27 | + maxItems: 2 | |
| 28 | + | |
| 29 | + ports: | |
| 30 | + $ref: /schemas/graph.yaml#/properties/ports | |
| 31 | + | |
| 32 | + properties: | |
| 33 | + port@0: | |
| 34 | + $ref: /schemas/graph.yaml#/properties/port | |
| 35 | + description: Input from LCD controller output. | |
| 36 | + | |
| 37 | + port@1: | |
| 38 | + $ref: /schemas/graph.yaml#/properties/port | |
| 39 | + description: Link to the HDMI connector. | |
| 40 | + | |
| 41 | +required: | |
| 42 | + - compatible | |
| 43 | + - clocks | |
| 44 | + - clock-names | |
| 45 | + - ports | |
| 46 | + - reg-io-width | |
| 47 | + | |
| 48 | +unevaluatedProperties: false | |
| 49 | + | |
| 50 | +examples: | |
| 51 | + - | | |
| 52 | + #include <dt-bindings/clock/ingenic,jz4780-cgu.h> | |
| 53 | + | |
| 54 | + hdmi: hdmi@10180000 { | |
| 55 | + compatible = "ingenic,jz4780-dw-hdmi"; | |
| 56 | + reg = <0x10180000 0x8000>; | |
| 57 | + reg-io-width = <4>; | |
| 58 | + ddc-i2c-bus = <&i2c4>; | |
| 59 | + interrupt-parent = <&intc>; | |
| 60 | + interrupts = <3>; | |
| 61 | + clocks = <&cgu JZ4780_CLK_AHB0>, <&cgu JZ4780_CLK_HDMI>; | |
| 62 | + clock-names = "iahb", "isfr"; | |
| 63 | + | |
| 64 | + ports { | |
| 65 | + #address-cells = <1>; | |
| 66 | + #size-cells = <0>; | |
| 67 | + hdmi_in: port@0 { | |
| 68 | + reg = <0>; | |
| 69 | + dw_hdmi_in: endpoint { | |
| 70 | + remote-endpoint = <&jz4780_lcd_out>; | |
| 71 | + }; | |
| 72 | + }; | |
| 73 | + hdmi_out: port@1 { | |
| 74 | + reg = <1>; | |
| 75 | + dw_hdmi_out: endpoint { | |
| 76 | + remote-endpoint = <&hdmi_con>; | |
| 77 | + }; | |
| 78 | + }; | |
| 79 | + }; | |
| 80 | + }; | |
| 81 | + | |
| 82 | +... | |
| @@ -68,7 +68,7 @@ properties: · 67 unmodified lines | ||
| 68 | 68 | - vesa-24 |
| 69 | 69 | description: | |
| 70 | 70 | The color signals mapping order. See details in |
| 71 | − Documentation/devicetree/bindings/display/panel/lvds.yaml | |
| 71 | + Documentation/devicetree/bindings/display/lvds.yaml | |
| 72 | 72 | |
| 73 | 73 | port@1: |
| 74 | 74 | $ref: /schemas/graph.yaml#/properties/port |
| @@ -1,10 +1,10 @@ | ||
| 1 | 1 | # SPDX-License-Identifier: GPL-2.0 |
| 2 | 2 | %YAML 1.2 |
| 3 | 3 | --- |
| 4 | −$id: http://devicetree.org/schemas/display/panel/lvds.yaml# | |
| 4 | +$id: http://devicetree.org/schemas/display/lvds.yaml# | |
| 5 | 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# |
| 6 | 6 | |
| 7 | −title: LVDS Display Panel | |
| 7 | +title: LVDS Display Common Properties | |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | 10 | - Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com> |
| @@ -13,8 +13,8 @@ maintainers: · 2 unmodified lines | ||
| 13 | 13 | description: |+ |
| 14 | 14 | LVDS is a physical layer specification defined in ANSI/TIA/EIA-644-A. Multiple |
| 15 | 15 | incompatible data link layers have been used over time to transmit image data |
| 16 | − to LVDS panels. This bindings supports display panels compatible with the | |
| 17 | − following specifications. | |
| 16 | + to LVDS devices. This bindings supports devices compatible with the following | |
| 17 | + specifications. | |
| 18 | 18 | |
| 19 | 19 | [JEIDA] "Digital Interface Standards for Monitor", JEIDA-59-1999, February |
| 20 | 20 | 1999 (Version 1.0), Japan Electronic Industry Development Association (JEIDA) |
| @@ -26,18 +26,7 @@ description: |+ · 5 unmodified lines | ||
| 26 | 26 | Device compatible with those specifications have been marketed under the |
| 27 | 27 | FPD-Link and FlatLink brands. |
| 28 | 28 | |
| 29 | −allOf: | |
| 30 | − - $ref: panel-common.yaml# | |
| 31 | − | |
| 32 | 29 | properties: |
| 33 | − compatible: | |
| 34 | − contains: | |
| 35 | − const: panel-lvds | |
| 36 | − description: | |
| 37 | − Shall contain "panel-lvds" in addition to a mandatory panel-specific | |
| 38 | − compatible string defined in individual panel bindings. The "panel-lvds" | |
| 39 | − value shall never be used on its own. | |
| 40 | − | |
| 41 | 30 | data-mapping: |
| 42 | 31 | enum: |
| 43 | 32 | - jeida-18 |
| @@ -96,22 +85,6 @@ properties: · 52 unmodified lines | ||
| 96 | 85 | If set, reverse the bit order described in the data mappings below on all |
| 97 | 86 | data lanes, transmitting bits for slots 6 to 0 instead of 0 to 6. |
| 98 | 87 | |
| 99 | − port: true | |
| 100 | − ports: true | |
| 101 | − | |
| 102 | −required: | |
| 103 | − - compatible | |
| 104 | − - data-mapping | |
| 105 | − - width-mm | |
| 106 | − - height-mm | |
| 107 | − - panel-timing | |
| 108 | − | |
| 109 | −oneOf: | |
| 110 | − - required: | |
| 111 | − - port | |
| 112 | − - required: | |
| 113 | − - ports | |
| 114 | − | |
| 115 | 88 | additionalProperties: true |
| 116 | 89 | |
| 117 | 90 | ... |
| @@ -0,0 +1,77 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,aal.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display adaptive ambient light processor | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display adaptive ambient light processor, namely AAL, | |
| 15 | + is responsible for backlight power saving and sunlight visibility improving. | |
| 16 | + AAL device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt8173-disp-aal | |
| 26 | + - items: | |
| 27 | + - enum: | |
| 28 | + - mediatek,mt2712-disp-aal | |
| 29 | + - mediatek,mt8183-disp-aal | |
| 30 | + - mediatek,mt8192-disp-aal | |
| 31 | + - mediatek,mt8195-disp-aal | |
| 32 | + - enum: | |
| 33 | + - mediatek,mt8173-disp-aal | |
| 34 | + | |
| 35 | + reg: | |
| 36 | + maxItems: 1 | |
| 37 | + | |
| 38 | + interrupts: | |
| 39 | + maxItems: 1 | |
| 40 | + | |
| 41 | + power-domains: | |
| 42 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 43 | + the power controller specified by phandle. See | |
| 44 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 45 | + | |
| 46 | + clocks: | |
| 47 | + items: | |
| 48 | + - description: AAL Clock | |
| 49 | + | |
| 50 | + mediatek,gce-client-reg: | |
| 51 | + description: The register of client driver can be configured by gce with | |
| 52 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 53 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 54 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 55 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 56 | + maxItems: 1 | |
| 57 | + | |
| 58 | +required: | |
| 59 | + - compatible | |
| 60 | + - reg | |
| 61 | + - interrupts | |
| 62 | + - power-domains | |
| 63 | + - clocks | |
| 64 | + | |
| 65 | +additionalProperties: false | |
| 66 | + | |
| 67 | +examples: | |
| 68 | + - | | |
| 69 | + | |
| 70 | + aal@14015000 { | |
| 71 | + compatible = "mediatek,mt8173-disp-aal"; | |
| 72 | + reg = <0 0x14015000 0 0x1000>; | |
| 73 | + interrupts = <GIC_SPI 189 IRQ_TYPE_LEVEL_LOW>; | |
| 74 | + power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 75 | + clocks = <&mmsys CLK_MM_DISP_AAL>; | |
| 76 | + mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x5000 0x1000>; | |
| 77 | + }; | |
| @@ -0,0 +1,76 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,ccorr.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display color correction | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display color correction, namely CCORR, reproduces correct color | |
| 15 | + on panels with different color gamut. | |
| 16 | + CCORR device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt8183-disp-ccorr | |
| 26 | + - items: | |
| 27 | + - const: mediatek,mt8192-disp-ccorr | |
| 28 | + - items: | |
| 29 | + - enum: | |
| 30 | + - mediatek,mt8195-disp-ccorr | |
| 31 | + - enum: | |
| 32 | + - mediatek,mt8192-disp-ccorr | |
| 33 | + | |
| 34 | + reg: | |
| 35 | + maxItems: 1 | |
| 36 | + | |
| 37 | + interrupts: | |
| 38 | + maxItems: 1 | |
| 39 | + | |
| 40 | + power-domains: | |
| 41 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 42 | + the power controller specified by phandle. See | |
| 43 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 44 | + | |
| 45 | + clocks: | |
| 46 | + items: | |
| 47 | + - description: CCORR Clock | |
| 48 | + | |
| 49 | + mediatek,gce-client-reg: | |
| 50 | + description: The register of client driver can be configured by gce with | |
| 51 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 52 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 53 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 54 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 55 | + maxItems: 1 | |
| 56 | + | |
| 57 | +required: | |
| 58 | + - compatible | |
| 59 | + - reg | |
| 60 | + - interrupts | |
| 61 | + - power-domains | |
| 62 | + - clocks | |
| 63 | + | |
| 64 | +additionalProperties: false | |
| 65 | + | |
| 66 | +examples: | |
| 67 | + - | | |
| 68 | + | |
| 69 | + ccorr0: ccorr@1400f000 { | |
| 70 | + compatible = "mediatek,mt8183-disp-ccorr"; | |
| 71 | + reg = <0 0x1400f000 0 0x1000>; | |
| 72 | + interrupts = <GIC_SPI 232 IRQ_TYPE_LEVEL_LOW>; | |
| 73 | + power-domains = <&spm MT8183_POWER_DOMAIN_DISP>; | |
| 74 | + clocks = <&mmsys CLK_MM_DISP_CCORR0>; | |
| 75 | + mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0xf000 0x1000>; | |
| 76 | + }; | |
| @@ -0,0 +1,86 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,color.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display color processor | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display color processor, namely COLOR, provides hue, luma and | |
| 15 | + saturation adjustments to get better picture quality and to have one panel | |
| 16 | + resemble the other in their output characteristics. | |
| 17 | + COLOR device node must be siblings to the central MMSYS_CONFIG node. | |
| 18 | + For a description of the MMSYS_CONFIG binding, see | |
| 19 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 20 | + for details. | |
| 21 | + | |
| 22 | +properties: | |
| 23 | + compatible: | |
| 24 | + oneOf: | |
| 25 | + - items: | |
| 26 | + - const: mediatek,mt2701-disp-color | |
| 27 | + - items: | |
| 28 | + - const: mediatek,mt8167-disp-color | |
| 29 | + - items: | |
| 30 | + - const: mediatek,mt8173-disp-color | |
| 31 | + - items: | |
| 32 | + - enum: | |
| 33 | + - mediatek,mt7623-disp-color | |
| 34 | + - mediatek,mt2712-disp-color | |
| 35 | + - enum: | |
| 36 | + - mediatek,mt2701-disp-color | |
| 37 | + - items: | |
| 38 | + - enum: | |
| 39 | + - mediatek,mt8183-disp-color | |
| 40 | + - mediatek,mt8192-disp-color | |
| 41 | + - mediatek,mt8195-disp-color | |
| 42 | + - enum: | |
| 43 | + - mediatek,mt8173-disp-color | |
| 44 | + reg: | |
| 45 | + maxItems: 1 | |
| 46 | + | |
| 47 | + interrupts: | |
| 48 | + maxItems: 1 | |
| 49 | + | |
| 50 | + power-domains: | |
| 51 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 52 | + the power controller specified by phandle. See | |
| 53 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 54 | + | |
| 55 | + clocks: | |
| 56 | + items: | |
| 57 | + - description: COLOR Clock | |
| 58 | + | |
| 59 | + mediatek,gce-client-reg: | |
| 60 | + description: The register of client driver can be configured by gce with | |
| 61 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 62 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 63 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 64 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 65 | + maxItems: 1 | |
| 66 | + | |
| 67 | +required: | |
| 68 | + - compatible | |
| 69 | + - reg | |
| 70 | + - interrupts | |
| 71 | + - power-domains | |
| 72 | + - clocks | |
| 73 | + | |
| 74 | +additionalProperties: false | |
| 75 | + | |
| 76 | +examples: | |
| 77 | + - | | |
| 78 | + | |
| 79 | + color0: color@14013000 { | |
| 80 | + compatible = "mediatek,mt8173-disp-color"; | |
| 81 | + reg = <0 0x14013000 0 0x1000>; | |
| 82 | + interrupts = <GIC_SPI 187 IRQ_TYPE_LEVEL_LOW>; | |
| 83 | + power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 84 | + clocks = <&mmsys CLK_MM_DISP_COLOR0>; | |
| 85 | + mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x3000 0x1000>; | |
| 86 | + }; | |
| @@ -1,219 +0,0 @@ | ||
| 1 | −Mediatek display subsystem | |
| 2 | −========================== | |
| 3 | − | |
| 4 | −The Mediatek display subsystem consists of various DISP function blocks in the | |
| 5 | −MMSYS register space. The connections between them can be configured by output | |
| 6 | −and input selectors in the MMSYS_CONFIG register space. Pixel clock and start | |
| 7 | −of frame signal are distributed to the other function blocks by a DISP_MUTEX | |
| 8 | −function block. | |
| 9 | − | |
| 10 | −All DISP device tree nodes must be siblings to the central MMSYS_CONFIG node. | |
| 11 | −For a description of the MMSYS_CONFIG binding, see | |
| 12 | −Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml. | |
| 13 | − | |
| 14 | −DISP function blocks | |
| 15 | −==================== | |
| 16 | − | |
| 17 | −A display stream starts at a source function block that reads pixel data from | |
| 18 | −memory and ends with a sink function block that drives pixels on a display | |
| 19 | −interface, or writes pixels back to memory. All DISP function blocks have | |
| 20 | −their own register space, interrupt, and clock gate. The blocks that can | |
| 21 | −access memory additionally have to list the IOMMU and local arbiter they are | |
| 22 | −connected to. | |
| 23 | − | |
| 24 | −For a description of the display interface sink function blocks, see | |
| 25 | −Documentation/devicetree/bindings/display/mediatek/mediatek,dsi.txt and | |
| 26 | −Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.yaml. | |
| 27 | − | |
| 28 | −Required properties (all function blocks): | |
| 29 | −- compatible: "mediatek,<chip>-disp-<function>", one of | |
| 30 | − "mediatek,<chip>-disp-ovl" - overlay (4 layers, blending, csc) | |
| 31 | − "mediatek,<chip>-disp-ovl-2l" - overlay (2 layers, blending, csc) | |
| 32 | − "mediatek,<chip>-disp-rdma" - read DMA / line buffer | |
| 33 | − "mediatek,<chip>-disp-wdma" - write DMA | |
| 34 | − "mediatek,<chip>-disp-ccorr" - color correction | |
| 35 | − "mediatek,<chip>-disp-color" - color processor | |
| 36 | − "mediatek,<chip>-disp-dither" - dither | |
| 37 | − "mediatek,<chip>-disp-aal" - adaptive ambient light controller | |
| 38 | − "mediatek,<chip>-disp-gamma" - gamma correction | |
| 39 | − "mediatek,<chip>-disp-merge" - merge streams from two RDMA sources | |
| 40 | − "mediatek,<chip>-disp-postmask" - control round corner for display frame | |
| 41 | − "mediatek,<chip>-disp-split" - split stream to two encoders | |
| 42 | − "mediatek,<chip>-disp-ufoe" - data compression engine | |
| 43 | − "mediatek,<chip>-dsi" - DSI controller, see mediatek,dsi.txt | |
| 44 | − "mediatek,<chip>-dpi" - DPI controller, see mediatek,dpi.txt | |
| 45 | − "mediatek,<chip>-disp-mutex" - display mutex | |
| 46 | − "mediatek,<chip>-disp-od" - overdrive | |
| 47 | − the supported chips are mt2701, mt7623, mt2712, mt8167, mt8173, mt8183 and mt8192. | |
| 48 | −- reg: Physical base address and length of the function block register space | |
| 49 | −- interrupts: The interrupt signal from the function block (required, except for | |
| 50 | − merge and split function blocks). | |
| 51 | −- clocks: device clocks | |
| 52 | − See Documentation/devicetree/bindings/clock/clock-bindings.txt for details. | |
| 53 | − For most function blocks this is just a single clock input. Only the DSI and | |
| 54 | − DPI controller nodes have multiple clock inputs. These are documented in | |
| 55 | − mediatek,dsi.txt and mediatek,dpi.txt, respectively. | |
| 56 | − An exception is that the mt8183 mutex is always free running with no clocks property. | |
| 57 | − | |
| 58 | −Required properties (DMA function blocks): | |
| 59 | −- compatible: Should be one of | |
| 60 | − "mediatek,<chip>-disp-ovl" | |
| 61 | − "mediatek,<chip>-disp-rdma" | |
| 62 | − "mediatek,<chip>-disp-wdma" | |
| 63 | − the supported chips are mt2701, mt8167 and mt8173. | |
| 64 | −- larb: Should contain a phandle pointing to the local arbiter device as defined | |
| 65 | − in Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml | |
| 66 | −- iommus: Should point to the respective IOMMU block with master port as | |
| 67 | − argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml | |
| 68 | − for details. | |
| 69 | − | |
| 70 | −Optional properties (RDMA function blocks): | |
| 71 | −- mediatek,rdma-fifo-size: rdma fifo size may be different even in same SOC, add this | |
| 72 | − property to the corresponding rdma | |
| 73 | − the value is the Max value which defined in hardware data sheet. | |
| 74 | − mediatek,rdma-fifo-size of mt8173-rdma0 is 8K | |
| 75 | − mediatek,rdma-fifo-size of mt8183-rdma0 is 5K | |
| 76 | − mediatek,rdma-fifo-size of mt8183-rdma1 is 2K | |
| 77 | − | |
| 78 | −Examples: | |
| 79 | − | |
| 80 | −mmsys: clock-controller@14000000 { | |
| 81 | − compatible = "mediatek,mt8173-mmsys", "syscon"; | |
| 82 | − reg = <0 0x14000000 0 0x1000>; | |
| 83 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 84 | − #clock-cells = <1>; | |
| 85 | −}; | |
| 86 | − | |
| 87 | −ovl0: ovl@1400c000 { | |
| 88 | − compatible = "mediatek,mt8173-disp-ovl"; | |
| 89 | − reg = <0 0x1400c000 0 0x1000>; | |
| 90 | − interrupts = <GIC_SPI 180 IRQ_TYPE_LEVEL_LOW>; | |
| 91 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 92 | − clocks = <&mmsys CLK_MM_DISP_OVL0>; | |
| 93 | − iommus = <&iommu M4U_PORT_DISP_OVL0>; | |
| 94 | − mediatek,larb = <&larb0>; | |
| 95 | −}; | |
| 96 | − | |
| 97 | −ovl1: ovl@1400d000 { | |
| 98 | − compatible = "mediatek,mt8173-disp-ovl"; | |
| 99 | − reg = <0 0x1400d000 0 0x1000>; | |
| 100 | − interrupts = <GIC_SPI 181 IRQ_TYPE_LEVEL_LOW>; | |
| 101 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 102 | − clocks = <&mmsys CLK_MM_DISP_OVL1>; | |
| 103 | − iommus = <&iommu M4U_PORT_DISP_OVL1>; | |
| 104 | − mediatek,larb = <&larb4>; | |
| 105 | −}; | |
| 106 | − | |
| 107 | −rdma0: rdma@1400e000 { | |
| 108 | − compatible = "mediatek,mt8173-disp-rdma"; | |
| 109 | − reg = <0 0x1400e000 0 0x1000>; | |
| 110 | − interrupts = <GIC_SPI 182 IRQ_TYPE_LEVEL_LOW>; | |
| 111 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 112 | − clocks = <&mmsys CLK_MM_DISP_RDMA0>; | |
| 113 | − iommus = <&iommu M4U_PORT_DISP_RDMA0>; | |
| 114 | − mediatek,larb = <&larb0>; | |
| 115 | − mediatek,rdma-fifosize = <8192>; | |
| 116 | −}; | |
| 117 | − | |
| 118 | −rdma1: rdma@1400f000 { | |
| 119 | − compatible = "mediatek,mt8173-disp-rdma"; | |
| 120 | − reg = <0 0x1400f000 0 0x1000>; | |
| 121 | − interrupts = <GIC_SPI 183 IRQ_TYPE_LEVEL_LOW>; | |
| 122 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 123 | − clocks = <&mmsys CLK_MM_DISP_RDMA1>; | |
| 124 | − iommus = <&iommu M4U_PORT_DISP_RDMA1>; | |
| 125 | − mediatek,larb = <&larb4>; | |
| 126 | −}; | |
| 127 | − | |
| 128 | −rdma2: rdma@14010000 { | |
| 129 | − compatible = "mediatek,mt8173-disp-rdma"; | |
| 130 | − reg = <0 0x14010000 0 0x1000>; | |
| 131 | − interrupts = <GIC_SPI 184 IRQ_TYPE_LEVEL_LOW>; | |
| 132 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 133 | − clocks = <&mmsys CLK_MM_DISP_RDMA2>; | |
| 134 | − iommus = <&iommu M4U_PORT_DISP_RDMA2>; | |
| 135 | − mediatek,larb = <&larb4>; | |
| 136 | −}; | |
| 137 | − | |
| 138 | −wdma0: wdma@14011000 { | |
| 139 | − compatible = "mediatek,mt8173-disp-wdma"; | |
| 140 | − reg = <0 0x14011000 0 0x1000>; | |
| 141 | − interrupts = <GIC_SPI 185 IRQ_TYPE_LEVEL_LOW>; | |
| 142 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 143 | − clocks = <&mmsys CLK_MM_DISP_WDMA0>; | |
| 144 | − iommus = <&iommu M4U_PORT_DISP_WDMA0>; | |
| 145 | − mediatek,larb = <&larb0>; | |
| 146 | −}; | |
| 147 | − | |
| 148 | −wdma1: wdma@14012000 { | |
| 149 | − compatible = "mediatek,mt8173-disp-wdma"; | |
| 150 | − reg = <0 0x14012000 0 0x1000>; | |
| 151 | − interrupts = <GIC_SPI 186 IRQ_TYPE_LEVEL_LOW>; | |
| 152 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 153 | − clocks = <&mmsys CLK_MM_DISP_WDMA1>; | |
| 154 | − iommus = <&iommu M4U_PORT_DISP_WDMA1>; | |
| 155 | − mediatek,larb = <&larb4>; | |
| 156 | −}; | |
| 157 | − | |
| 158 | −color0: color@14013000 { | |
| 159 | − compatible = "mediatek,mt8173-disp-color"; | |
| 160 | − reg = <0 0x14013000 0 0x1000>; | |
| 161 | − interrupts = <GIC_SPI 187 IRQ_TYPE_LEVEL_LOW>; | |
| 162 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 163 | − clocks = <&mmsys CLK_MM_DISP_COLOR0>; | |
| 164 | −}; | |
| 165 | − | |
| 166 | −color1: color@14014000 { | |
| 167 | − compatible = "mediatek,mt8173-disp-color"; | |
| 168 | − reg = <0 0x14014000 0 0x1000>; | |
| 169 | − interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_LOW>; | |
| 170 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 171 | − clocks = <&mmsys CLK_MM_DISP_COLOR1>; | |
| 172 | −}; | |
| 173 | − | |
| 174 | −aal@14015000 { | |
| 175 | − compatible = "mediatek,mt8173-disp-aal"; | |
| 176 | − reg = <0 0x14015000 0 0x1000>; | |
| 177 | − interrupts = <GIC_SPI 189 IRQ_TYPE_LEVEL_LOW>; | |
| 178 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 179 | − clocks = <&mmsys CLK_MM_DISP_AAL>; | |
| 180 | −}; | |
| 181 | − | |
| 182 | −gamma@14016000 { | |
| 183 | − compatible = "mediatek,mt8173-disp-gamma"; | |
| 184 | − reg = <0 0x14016000 0 0x1000>; | |
| 185 | − interrupts = <GIC_SPI 190 IRQ_TYPE_LEVEL_LOW>; | |
| 186 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 187 | − clocks = <&mmsys CLK_MM_DISP_GAMMA>; | |
| 188 | −}; | |
| 189 | − | |
| 190 | −ufoe@1401a000 { | |
| 191 | − compatible = "mediatek,mt8173-disp-ufoe"; | |
| 192 | − reg = <0 0x1401a000 0 0x1000>; | |
| 193 | − interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_LOW>; | |
| 194 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 195 | − clocks = <&mmsys CLK_MM_DISP_UFOE>; | |
| 196 | −}; | |
| 197 | − | |
| 198 | −dsi0: dsi@1401b000 { | |
| 199 | − /* See mediatek,dsi.txt for details */ | |
| 200 | −}; | |
| 201 | − | |
| 202 | −dpi0: dpi@1401d000 { | |
| 203 | − /* See mediatek,dpi.txt for details */ | |
| 204 | −}; | |
| 205 | − | |
| 206 | −mutex: mutex@14020000 { | |
| 207 | − compatible = "mediatek,mt8173-disp-mutex"; | |
| 208 | − reg = <0 0x14020000 0 0x1000>; | |
| 209 | − interrupts = <GIC_SPI 169 IRQ_TYPE_LEVEL_LOW>; | |
| 210 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 211 | − clocks = <&mmsys CLK_MM_MUTEX_32K>; | |
| 212 | −}; | |
| 213 | − | |
| 214 | −od@14023000 { | |
| 215 | − compatible = "mediatek,mt8173-disp-od"; | |
| 216 | − reg = <0 0x14023000 0 0x1000>; | |
| 217 | − power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 218 | − clocks = <&mmsys CLK_MM_DISP_OD>; | |
| 219 | −}; | |
| @@ -0,0 +1,76 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,dither.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display dither processor | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display dither processor, namely DITHER, works by approximating | |
| 15 | + unavailable colors with available colors and by mixing and matching available | |
| 16 | + colors to mimic unavailable ones. | |
| 17 | + DITHER device node must be siblings to the central MMSYS_CONFIG node. | |
| 18 | + For a description of the MMSYS_CONFIG binding, see | |
| 19 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 20 | + for details. | |
| 21 | + | |
| 22 | +properties: | |
| 23 | + compatible: | |
| 24 | + oneOf: | |
| 25 | + - items: | |
| 26 | + - const: mediatek,mt8183-disp-dither | |
| 27 | + - items: | |
| 28 | + - enum: | |
| 29 | + - mediatek,mt8192-disp-dither | |
| 30 | + - mediatek,mt8195-disp-dither | |
| 31 | + - enum: | |
| 32 | + - mediatek,mt8183-disp-dither | |
| 33 | + | |
| 34 | + reg: | |
| 35 | + maxItems: 1 | |
| 36 | + | |
| 37 | + interrupts: | |
| 38 | + maxItems: 1 | |
| 39 | + | |
| 40 | + power-domains: | |
| 41 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 42 | + the power controller specified by phandle. See | |
| 43 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 44 | + | |
| 45 | + clocks: | |
| 46 | + items: | |
| 47 | + - description: DITHER Clock | |
| 48 | + | |
| 49 | + mediatek,gce-client-reg: | |
| 50 | + description: The register of client driver can be configured by gce with | |
| 51 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 52 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 53 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 54 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 55 | + maxItems: 1 | |
| 56 | + | |
| 57 | +required: | |
| 58 | + - compatible | |
| 59 | + - reg | |
| 60 | + - interrupts | |
| 61 | + - power-domains | |
| 62 | + - clocks | |
| 63 | + | |
| 64 | +additionalProperties: false | |
| 65 | + | |
| 66 | +examples: | |
| 67 | + - | | |
| 68 | + | |
| 69 | + dither0: dither@14012000 { | |
| 70 | + compatible = "mediatek,mt8183-disp-dither"; | |
| 71 | + reg = <0 0x14012000 0 0x1000>; | |
| 72 | + interrupts = <GIC_SPI 235 IRQ_TYPE_LEVEL_LOW>; | |
| 73 | + power-domains = <&spm MT8183_POWER_DOMAIN_DISP>; | |
| 74 | + clocks = <&mmsys CLK_MM_DISP_DITHER0>; | |
| 75 | + mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x2000 0x1000>; | |
| 76 | + }; | |
| @@ -0,0 +1,71 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,dsc.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: mediatek display DSC controller | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + The DSC standard is a specification of the algorithms used for | |
| 15 | + compressing and decompressing image display streams, including | |
| 16 | + the specification of the syntax and semantics of the compressed | |
| 17 | + video bit stream. DSC is designed for real-time systems with | |
| 18 | + real-time compression, transmission, decompression and Display. | |
| 19 | + | |
| 20 | +properties: | |
| 21 | + compatible: | |
| 22 | + oneOf: | |
| 23 | + - items: | |
| 24 | + - const: mediatek,mt8195-disp-dsc | |
| 25 | + | |
| 26 | + reg: | |
| 27 | + maxItems: 1 | |
| 28 | + | |
| 29 | + interrupts: | |
| 30 | + maxItems: 1 | |
| 31 | + | |
| 32 | + clocks: | |
| 33 | + items: | |
| 34 | + - description: DSC Wrapper Clock | |
| 35 | + | |
| 36 | + power-domains: | |
| 37 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 38 | + the power controller specified by phandle. See | |
| 39 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 40 | + | |
| 41 | + mediatek,gce-client-reg: | |
| 42 | + description: | |
| 43 | + The register of client driver can be configured by gce with 4 arguments | |
| 44 | + defined in this property, such as phandle of gce, subsys id, | |
| 45 | + register offset and size. | |
| 46 | + Each subsys id is mapping to a base address of display function blocks | |
| 47 | + register which is defined in the gce header | |
| 48 | + include/dt-bindings/gce/<chip>-gce.h. | |
| 49 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 50 | + maxItems: 1 | |
| 51 | + | |
| 52 | +required: | |
| 53 | + - compatible | |
| 54 | + - reg | |
| 55 | + - interrupts | |
| 56 | + - power-domains | |
| 57 | + - clocks | |
| 58 | + | |
| 59 | +additionalProperties: false | |
| 60 | + | |
| 61 | +examples: | |
| 62 | + - | | |
| 63 | + | |
| 64 | + dsc0: disp_dsc_wrap@1c009000 { | |
| 65 | + compatible = "mediatek,mt8195-disp-dsc"; | |
| 66 | + reg = <0 0x1c009000 0 0x1000>; | |
| 67 | + interrupts = <GIC_SPI 645 IRQ_TYPE_LEVEL_HIGH 0>; | |
| 68 | + power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS0>; | |
| 69 | + clocks = <&vdosys0 CLK_VDO0_DSC_WRAP0>; | |
| 70 | + mediatek,gce-client-reg = <&gce1 SUBSYS_1c00XXXX 0x9000 0x1000>; | |
| 71 | + }; | |
| @@ -0,0 +1,147 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,ethdr.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek Ethdr Device Tree Bindings | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + ETHDR is designed for HDR video and graphics conversion in the external display path. | |
| 15 | + It handles multiple HDR input types and performs tone mapping, color space/color | |
| 16 | + format conversion, and then combine different layers, output the required HDR or | |
| 17 | + SDR signal to the subsequent display path. This engine is composed of two video | |
| 18 | + frontends, two graphic frontends, one video backend and a mixer. ETHDR has two | |
| 19 | + DMA function blocks, DS and ADL. These two function blocks read the pre-programmed | |
| 20 | + registers from DRAM and set them to HW in the v-blanking period. | |
| 21 | + | |
| 22 | +properties: | |
| 23 | + compatible: | |
| 24 | + items: | |
| 25 | + - const: mediatek,mt8195-disp-ethdr | |
| 26 | + reg: | |
| 27 | + maxItems: 7 | |
| 28 | + reg-names: | |
| 29 | + items: | |
| 30 | + - const: mixer | |
| 31 | + - const: vdo_fe0 | |
| 32 | + - const: vdo_fe1 | |
| 33 | + - const: gfx_fe0 | |
| 34 | + - const: gfx_fe1 | |
| 35 | + - const: vdo_be | |
| 36 | + - const: adl_ds | |
| 37 | + interrupts: | |
| 38 | + minItems: 1 | |
| 39 | + iommus: | |
| 40 | + description: The compatible property is DMA function blocks. | |
| 41 | + Should point to the respective IOMMU block with master port as argument, | |
| 42 | + see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for | |
| 43 | + details. | |
| 44 | + minItems: 1 | |
| 45 | + maxItems: 2 | |
| 46 | + clocks: | |
| 47 | + items: | |
| 48 | + - description: mixer clock | |
| 49 | + - description: video frontend 0 clock | |
| 50 | + - description: video frontend 1 clock | |
| 51 | + - description: graphic frontend 0 clock | |
| 52 | + - description: graphic frontend 1 clock | |
| 53 | + - description: video backend clock | |
| 54 | + - description: autodownload and menuload clock | |
| 55 | + - description: video frontend 0 async clock | |
| 56 | + - description: video frontend 1 async clock | |
| 57 | + - description: graphic frontend 0 async clock | |
| 58 | + - description: graphic frontend 1 async clock | |
| 59 | + - description: video backend async clock | |
| 60 | + - description: ethdr top clock | |
| 61 | + clock-names: | |
| 62 | + items: | |
| 63 | + - const: mixer | |
| 64 | + - const: vdo_fe0 | |
| 65 | + - const: vdo_fe1 | |
| 66 | + - const: gfx_fe0 | |
| 67 | + - const: gfx_fe1 | |
| 68 | + - const: vdo_be | |
| 69 | + - const: adl_ds | |
| 70 | + - const: vdo_fe0_async | |
| 71 | + - const: vdo_fe1_async | |
| 72 | + - const: gfx_fe0_async | |
| 73 | + - const: gfx_fe1_async | |
| 74 | + - const: vdo_be_async | |
| 75 | + - const: ethdr_top | |
| 76 | + power-domains: | |
| 77 | + maxItems: 1 | |
| 78 | + resets: | |
| 79 | + maxItems: 5 | |
| 80 | + mediatek,gce-client-reg: | |
| 81 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 82 | + description: The register of display function block to be set by gce. | |
| 83 | + There are 4 arguments in this property, gce node, subsys id, offset and | |
| 84 | + register size. The subsys id is defined in the gce header of each chips | |
| 85 | + include/include/dt-bindings/gce/<chip>-gce.h, mapping to the register of | |
| 86 | + display function block. | |
| 87 | + | |
| 88 | +required: | |
| 89 | + - compatible | |
| 90 | + - reg | |
| 91 | + - clocks | |
| 92 | + - clock-names | |
| 93 | + - interrupts | |
| 94 | + - power-domains | |
| 95 | + | |
| 96 | +additionalProperties: false | |
| 97 | + | |
| 98 | +examples: | |
| 99 | + - | | |
| 100 | + | |
| 101 | + disp_ethdr@1c114000 { | |
| 102 | + compatible = "mediatek,mt8195-disp-ethdr"; | |
| 103 | + reg = <0 0x1c114000 0 0x1000>, | |
| 104 | + <0 0x1c115000 0 0x1000>, | |
| 105 | + <0 0x1c117000 0 0x1000>, | |
| 106 | + <0 0x1c119000 0 0x1000>, | |
| 107 | + <0 0x1c11A000 0 0x1000>, | |
| 108 | + <0 0x1c11B000 0 0x1000>, | |
| 109 | + <0 0x1c11C000 0 0x1000>; | |
| 110 | + reg-names = "mixer", "vdo_fe0", "vdo_fe1", "gfx_fe0", "gfx_fe1", | |
| 111 | + "vdo_be", "adl_ds"; | |
| 112 | + mediatek,gce-client-reg = <&gce0 SUBSYS_1c11XXXX 0x4000 0x1000>, | |
| 113 | + <&gce0 SUBSYS_1c11XXXX 0x5000 0x1000>, | |
| 114 | + <&gce0 SUBSYS_1c11XXXX 0x7000 0x1000>, | |
| 115 | + <&gce0 SUBSYS_1c11XXXX 0x9000 0x1000>, | |
| 116 | + <&gce0 SUBSYS_1c11XXXX 0xA000 0x1000>, | |
| 117 | + <&gce0 SUBSYS_1c11XXXX 0xB000 0x1000>, | |
| 118 | + <&gce0 SUBSYS_1c11XXXX 0xC000 0x1000>; | |
| 119 | + clocks = <&vdosys1 CLK_VDO1_DISP_MIXER>, | |
| 120 | + <&vdosys1 CLK_VDO1_HDR_VDO_FE0>, | |
| 121 | + <&vdosys1 CLK_VDO1_HDR_VDO_FE1>, | |
| 122 | + <&vdosys1 CLK_VDO1_HDR_GFX_FE0>, | |
| 123 | + <&vdosys1 CLK_VDO1_HDR_GFX_FE1>, | |
| 124 | + <&vdosys1 CLK_VDO1_HDR_VDO_BE>, | |
| 125 | + <&vdosys1 CLK_VDO1_26M_SLOW>, | |
| 126 | + <&vdosys1 CLK_VDO1_HDR_VDO_FE0_DL_ASYNC>, | |
| 127 | + <&vdosys1 CLK_VDO1_HDR_VDO_FE1_DL_ASYNC>, | |
| 128 | + <&vdosys1 CLK_VDO1_HDR_GFX_FE0_DL_ASYNC>, | |
| 129 | + <&vdosys1 CLK_VDO1_HDR_GFX_FE1_DL_ASYNC>, | |
| 130 | + <&vdosys1 CLK_VDO1_HDR_VDO_BE_DL_ASYNC>, | |
| 131 | + <&topckgen CLK_TOP_ETHDR_SEL>; | |
| 132 | + clock-names = "mixer", "vdo_fe0", "vdo_fe1", "gfx_fe0", "gfx_fe1", | |
| 133 | + "vdo_be", "adl_ds", "vdo_fe0_async", "vdo_fe1_async", | |
| 134 | + "gfx_fe0_async", "gfx_fe1_async","vdo_be_async", | |
| 135 | + "ethdr_top"; | |
| 136 | + power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>; | |
| 137 | + iommus = <&iommu_vpp M4U_PORT_L3_HDR_DS>, | |
| 138 | + <&iommu_vpp M4U_PORT_L3_HDR_ADL>; | |
| 139 | + interrupts = <GIC_SPI 517 IRQ_TYPE_LEVEL_HIGH 0>; /* disp mixer */ | |
| 140 | + resets = <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_VDO_FE0_DL_ASYNC>, | |
| 141 | + <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_VDO_FE1_DL_ASYNC>, | |
| 142 | + <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_GFX_FE0_DL_ASYNC>, | |
| 143 | + <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_GFX_FE1_DL_ASYNC>, | |
| 144 | + <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_VDO_BE_DL_ASYNC>; | |
| 145 | + }; | |
| 146 | + | |
| 147 | +... | |
| @@ -0,0 +1,77 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,gamma.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display gamma correction | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display gamma correction, namely GAMMA, provides a nonlinear | |
| 15 | + operation used to adjust luminance in display system. | |
| 16 | + GAMMA device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt8173-disp-gamma | |
| 26 | + - items: | |
| 27 | + - const: mediatek,mt8183-disp-gamma | |
| 28 | + - items: | |
| 29 | + - enum: | |
| 30 | + - mediatek,mt8192-disp-gamma | |
| 31 | + - mediatek,mt8195-disp-gamma | |
| 32 | + - enum: | |
| 33 | + - mediatek,mt8183-disp-gamma | |
| 34 | + | |
| 35 | + reg: | |
| 36 | + maxItems: 1 | |
| 37 | + | |
| 38 | + interrupts: | |
| 39 | + maxItems: 1 | |
| 40 | + | |
| 41 | + power-domains: | |
| 42 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 43 | + the power controller specified by phandle. See | |
| 44 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 45 | + | |
| 46 | + clocks: | |
| 47 | + items: | |
| 48 | + - description: GAMMA Clock | |
| 49 | + | |
| 50 | + mediatek,gce-client-reg: | |
| 51 | + description: The register of client driver can be configured by gce with | |
| 52 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 53 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 54 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 55 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 56 | + maxItems: 1 | |
| 57 | + | |
| 58 | +required: | |
| 59 | + - compatible | |
| 60 | + - reg | |
| 61 | + - interrupts | |
| 62 | + - power-domains | |
| 63 | + - clocks | |
| 64 | + | |
| 65 | +additionalProperties: false | |
| 66 | + | |
| 67 | +examples: | |
| 68 | + - | | |
| 69 | + | |
| 70 | + gamma@14016000 { | |
| 71 | + compatible = "mediatek,mt8173-disp-gamma"; | |
| 72 | + reg = <0 0x14016000 0 0x1000>; | |
| 73 | + interrupts = <GIC_SPI 190 IRQ_TYPE_LEVEL_LOW>; | |
| 74 | + power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 75 | + clocks = <&mmsys CLK_MM_DISP_GAMMA>; | |
| 76 | + mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x6000 0x1000>; | |
| 77 | + }; | |
| @@ -0,0 +1,110 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,merge.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display merge | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display merge, namely MERGE, is used to merge two slice-per-line | |
| 15 | + inputs into one side-by-side output. | |
| 16 | + MERGE device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt8173-disp-merge | |
| 26 | + - items: | |
| 27 | + - const: mediatek,mt8195-disp-merge | |
| 28 | + | |
| 29 | + reg: | |
| 30 | + maxItems: 1 | |
| 31 | + | |
| 32 | + interrupts: | |
| 33 | + maxItems: 1 | |
| 34 | + | |
| 35 | + power-domains: | |
| 36 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 37 | + the power controller specified by phandle. See | |
| 38 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 39 | + | |
| 40 | + clocks: | |
| 41 | + maxItems: 2 | |
| 42 | + items: | |
| 43 | + - description: MERGE Clock | |
| 44 | + - description: MERGE Async Clock | |
| 45 | + Controlling the synchronous process between MERGE and other display | |
| 46 | + function blocks cross clock domain. | |
| 47 | + | |
| 48 | + clock-names: | |
| 49 | + maxItems: 2 | |
| 50 | + items: | |
| 51 | + - const: merge | |
| 52 | + - const: merge_async | |
| 53 | + | |
| 54 | + mediatek,merge-fifo-en: | |
| 55 | + description: | |
| 56 | + The setting of merge fifo is mainly provided for the display latency | |
| 57 | + buffer to ensure that the back-end panel display data will not be | |
| 58 | + underrun, a little more data is needed in the fifo. | |
| 59 | + According to the merge fifo settings, when the water level is detected | |
| 60 | + to be insufficient, it will trigger RDMA sending ultra and preulra | |
| 61 | + command to SMI to speed up the data rate. | |
| 62 | + type: boolean | |
| 63 | + | |
| 64 | + mediatek,merge-mute: | |
| 65 | + description: Support mute function. Mute the content of merge output. | |
| 66 | + type: boolean | |
| 67 | + | |
| 68 | + mediatek,gce-client-reg: | |
| 69 | + description: The register of client driver can be configured by gce with | |
| 70 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 71 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 72 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 73 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 74 | + maxItems: 1 | |
| 75 | + | |
| 76 | + resets: | |
| 77 | + description: reset controller | |
| 78 | + See Documentation/devicetree/bindings/reset/reset.txt for details. | |
| 79 | + maxItems: 1 | |
| 80 | + | |
| 81 | +required: | |
| 82 | + - compatible | |
| 83 | + - reg | |
| 84 | + - power-domains | |
| 85 | + - clocks | |
| 86 | + | |
| 87 | +additionalProperties: false | |
| 88 | + | |
| 89 | +examples: | |
| 90 | + - | | |
| 91 | + | |
| 92 | + merge@14017000 { | |
| 93 | + compatible = "mediatek,mt8173-disp-merge"; | |
| 94 | + reg = <0 0x14017000 0 0x1000>; | |
| 95 | + power-domains = <&spm MT8173_POWER_DOMAIN_MM>; | |
| 96 | + clocks = <&mmsys CLK_MM_DISP_MERGE>; | |
| 97 | + }; | |
| 98 | + | |
| 99 | + merge5: disp_vpp_merge5@1c110000 { | |
| 100 | + compatible = "mediatek,mt8195-disp-merge"; | |
| 101 | + reg = <0 0x1c110000 0 0x1000>; | |
| 102 | + interrupts = <GIC_SPI 507 IRQ_TYPE_LEVEL_HIGH 0>; | |
| 103 | + clocks = <&vdosys1 CLK_VDO1_VPP_MERGE4>, | |
| 104 | + <&vdosys1 CLK_VDO1_MERGE4_DL_ASYNC>; | |
| 105 | + clock-names = "merge","merge_async"; | |
| 106 | + power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>; | |
| 107 | + mediatek,gce-client-reg = <&gce1 SUBSYS_1c11XXXX 0x0000 0x1000>; | |
| 108 | + mediatek,merge-fifo-en = <1>; | |
| 109 | + resets = <&vdosys1 MT8195_VDOSYS1_SW0_RST_B_MERGE4_DL_ASYNC>; | |
| 110 | + }; | |
| @@ -0,0 +1,83 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,mutex.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek mutex | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek mutex, namely MUTEX, is used to send the triggers signals called | |
| 15 | + Start Of Frame (SOF) / End Of Frame (EOF) to each sub-modules on the display | |
| 16 | + data path or MDP data path. | |
| 17 | + In some SoC, such as mt2701, MUTEX could be a hardware mutex which protects | |
| 18 | + the shadow register. | |
| 19 | + MUTEX device node must be siblings to the central MMSYS_CONFIG node. | |
| 20 | + For a description of the MMSYS_CONFIG binding, see | |
| 21 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 22 | + for details. | |
| 23 | + | |
| 24 | +properties: | |
| 25 | + compatible: | |
| 26 | + oneOf: | |
| 27 | + - items: | |
| 28 | + - const: mediatek,mt2701-disp-mutex | |
| 29 | + - items: | |
| 30 | + - const: mediatek,mt2712-disp-mutex | |
| 31 | + - items: | |
| 32 | + - const: mediatek,mt8167-disp-mutex | |
| 33 | + - items: | |
| 34 | + - const: mediatek,mt8173-disp-mutex | |
| 35 | + - items: | |
| 36 | + - const: mediatek,mt8183-disp-mutex | |
| 37 | + - items: | |
| 38 | + - const: mediatek,mt8192-disp-mutex | |
| 39 | + - items: | |
| 40 | + - const: mediatek,mt8195-disp-mutex | |
| 41 | + reg: | |
| 42 | + maxItems: 1 | |
| 43 | + | |
| 44 | + interrupts: | |
| 45 | + maxItems: 1 | |
| 46 | + | |
| 47 | + power-domains: | |
| 48 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 49 | + the power controller specified by phandle. See | |
| 50 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 51 | + | |
| 52 | + clocks: | |
| 53 | + items: | |
| 54 | + - description: MUTEX Clock | |
| 55 | + | |
| 56 | + mediatek,gce-events: | |
| 57 | + description: | |
| 58 | + The event id which is mapping to the specific hardware event signal | |
| 59 | + to gce. The event id is defined in the gce header | |
| 60 | + include/dt-bindings/gce/<chip>-gce.h of each chips. | |
| 61 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 62 | + | |
| 63 | +required: | |
| 64 | + - compatible | |
| 65 | + - reg | |
| 66 | + - interrupts | |
| 67 | + - power-domains | |
| 68 | + - clocks | |
| 69 | + | |
| 70 | +additionalProperties: false | |
| 71 | + | |
| 72 | +examples: | |
| 73 | + - | | |
| 74 | + | |
| 75 | + mutex: mutex@14020000 { | |
| 76 | + compatible = "mediatek,mt8173-disp-mutex"; | |
| 77 | + reg = <0 0x14020000 0 0x1000>; | |
| 78 | + interrupts = <GIC_SPI 169 IRQ_TYPE_LEVEL_LOW>; | |
| 79 | + power-domains = <&spm MT8173_POWER_DOMAIN_MM>; | |
| 80 | + clocks = <&mmsys CLK_MM_MUTEX_32K>; | |
| 81 | + mediatek,gce-events = <CMDQ_EVENT_MUTEX0_STREAM_EOF>, | |
| 82 | + <CMDQ_EVENT_MUTEX1_STREAM_EOF>; | |
| 83 | + }; | |
| @@ -0,0 +1,53 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,od.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display overdirve | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display overdrive, namely OD, increases the transition values | |
| 15 | + of pixels between consecutive frames to make LCD rotate faster. | |
| 16 | + OD device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt2712-disp-od | |
| 26 | + - items: | |
| 27 | + - const: mediatek,mt8173-disp-od | |
| 28 | + | |
| 29 | + reg: | |
| 30 | + maxItems: 1 | |
| 31 | + | |
| 32 | + interrupts: | |
| 33 | + maxItems: 1 | |
| 34 | + | |
| 35 | + clocks: | |
| 36 | + items: | |
| 37 | + - description: OD Clock | |
| 38 | + | |
| 39 | +required: | |
| 40 | + - compatible | |
| 41 | + - reg | |
| 42 | + - clocks | |
| 43 | + | |
| 44 | +additionalProperties: false | |
| 45 | + | |
| 46 | +examples: | |
| 47 | + - | | |
| 48 | + | |
| 49 | + od@14023000 { | |
| 50 | + compatible = "mediatek,mt8173-disp-od"; | |
| 51 | + reg = <0 0x14023000 0 0x1000>; | |
| 52 | + clocks = <&mmsys CLK_MM_DISP_OD>; | |
| 53 | + }; | |
| @@ -0,0 +1,88 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,ovl-2l.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display overlay 2 layer | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display overlay 2 layer, namely OVL-2L, provides 2 more layer | |
| 15 | + for OVL. | |
| 16 | + OVL-2L device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt8183-disp-ovl-2l | |
| 26 | + - items: | |
| 27 | + - const: mediatek,mt8192-disp-ovl-2l | |
| 28 | + | |
| 29 | + reg: | |
| 30 | + maxItems: 1 | |
| 31 | + | |
| 32 | + interrupts: | |
| 33 | + maxItems: 1 | |
| 34 | + | |
| 35 | + power-domains: | |
| 36 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 37 | + the power controller specified by phandle. See | |
| 38 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 39 | + | |
| 40 | + clocks: | |
| 41 | + items: | |
| 42 | + - description: OVL-2L Clock | |
| 43 | + | |
| 44 | + iommus: | |
| 45 | + description: | |
| 46 | + This property should point to the respective IOMMU block with master port as argument, | |
| 47 | + see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for details. | |
| 48 | + | |
| 49 | + mediatek,larb: | |
| 50 | + description: | |
| 51 | + This property should contain a phandle pointing to the local arbiter devices defined in | |
| 52 | + Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml. | |
| 53 | + It must sort according to the local arbiter index, like larb0, larb1, larb2... | |
| 54 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 55 | + minItems: 1 | |
| 56 | + maxItems: 32 | |
| 57 | + | |
| 58 | + mediatek,gce-client-reg: | |
| 59 | + description: The register of client driver can be configured by gce with | |
| 60 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 61 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 62 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 63 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 64 | + maxItems: 1 | |
| 65 | + | |
| 66 | +required: | |
| 67 | + - compatible | |
| 68 | + - reg | |
| 69 | + - interrupts | |
| 70 | + - power-domains | |
| 71 | + - clocks | |
| 72 | + - iommus | |
| 73 | + | |
| 74 | +additionalProperties: false | |
| 75 | + | |
| 76 | +examples: | |
| 77 | + - | | |
| 78 | + | |
| 79 | + ovl_2l0: ovl@14009000 { | |
| 80 | + compatible = "mediatek,mt8183-disp-ovl-2l"; | |
| 81 | + reg = <0 0x14009000 0 0x1000>; | |
| 82 | + interrupts = <GIC_SPI 226 IRQ_TYPE_LEVEL_LOW>; | |
| 83 | + power-domains = <&spm MT8183_POWER_DOMAIN_DISP>; | |
| 84 | + clocks = <&mmsys CLK_MM_DISP_OVL0_2L>; | |
| 85 | + iommus = <&iommu M4U_PORT_DISP_2L_OVL0_LARB0>; | |
| 86 | + mediatek,larb = <&larb0>; | |
| 87 | + mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0x9000 0x1000>; | |
| 88 | + }; | |
| @@ -0,0 +1,103 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,ovl.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display overlay | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display overlay, namely OVL, can do alpha blending from | |
| 15 | + the memory. | |
| 16 | + OVL device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt2701-disp-ovl | |
| 26 | + - items: | |
| 27 | + - const: mediatek,mt8173-disp-ovl | |
| 28 | + - items: | |
| 29 | + - const: mediatek,mt8183-disp-ovl | |
| 30 | + - items: | |
| 31 | + - const: mediatek,mt8192-disp-ovl | |
| 32 | + - items: | |
| 33 | + - enum: | |
| 34 | + - mediatek,mt7623-disp-ovl | |
| 35 | + - mediatek,mt2712-disp-ovl | |
| 36 | + - enum: | |
| 37 | + - mediatek,mt2701-disp-ovl | |
| 38 | + - items: | |
| 39 | + - enum: | |
| 40 | + - mediatek,mt8195-disp-ovl | |
| 41 | + - enum: | |
| 42 | + - mediatek,mt8183-disp-ovl | |
| 43 | + | |
| 44 | + reg: | |
| 45 | + maxItems: 1 | |
| 46 | + | |
| 47 | + interrupts: | |
| 48 | + maxItems: 1 | |
| 49 | + | |
| 50 | + power-domains: | |
| 51 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 52 | + the power controller specified by phandle. See | |
| 53 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 54 | + | |
| 55 | + clocks: | |
| 56 | + items: | |
| 57 | + - description: OVL Clock | |
| 58 | + | |
| 59 | + iommus: | |
| 60 | + description: | |
| 61 | + This property should point to the respective IOMMU block with master port as argument, | |
| 62 | + see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for details. | |
| 63 | + | |
| 64 | + mediatek,larb: | |
| 65 | + description: | |
| 66 | + This property should contain a phandle pointing to the local arbiter devices defined in | |
| 67 | + Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml. | |
| 68 | + It must sort according to the local arbiter index, like larb0, larb1, larb2... | |
| 69 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 70 | + minItems: 1 | |
| 71 | + maxItems: 32 | |
| 72 | + | |
| 73 | + mediatek,gce-client-reg: | |
| 74 | + description: The register of client driver can be configured by gce with | |
| 75 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 76 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 77 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 78 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 79 | + maxItems: 1 | |
| 80 | + | |
| 81 | +required: | |
| 82 | + - compatible | |
| 83 | + - reg | |
| 84 | + - interrupts | |
| 85 | + - power-domains | |
| 86 | + - clocks | |
| 87 | + - iommu | |
| 88 | + | |
| 89 | +additionalProperties: false | |
| 90 | + | |
| 91 | +examples: | |
| 92 | + - | | |
| 93 | + | |
| 94 | + ovl0: ovl@1400c000 { | |
| 95 | + compatible = "mediatek,mt8173-disp-ovl"; | |
| 96 | + reg = <0 0x1400c000 0 0x1000>; | |
| 97 | + interrupts = <GIC_SPI 180 IRQ_TYPE_LEVEL_LOW>; | |
| 98 | + power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 99 | + clocks = <&mmsys CLK_MM_DISP_OVL0>; | |
| 100 | + iommus = <&iommu M4U_PORT_DISP_OVL0>; | |
| 101 | + mediatek,larb = <&larb0>; | |
| 102 | + mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0xc000 0x1000>; | |
| 103 | + }; | |
| @@ -0,0 +1,69 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,postmask.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display postmask | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display postmask, namely POSTMASK, provides round corner pattern | |
| 15 | + generation. | |
| 16 | + POSTMASK device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt8192-disp-postmask | |
| 26 | + | |
| 27 | + reg: | |
| 28 | + maxItems: 1 | |
| 29 | + | |
| 30 | + interrupts: | |
| 31 | + maxItems: 1 | |
| 32 | + | |
| 33 | + power-domains: | |
| 34 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 35 | + the power controller specified by phandle. See | |
| 36 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 37 | + | |
| 38 | + clocks: | |
| 39 | + items: | |
| 40 | + - description: POSTMASK Clock | |
| 41 | + | |
| 42 | + mediatek,gce-client-reg: | |
| 43 | + description: The register of client driver can be configured by gce with | |
| 44 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 45 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 46 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 47 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 48 | + maxItems: 1 | |
| 49 | + | |
| 50 | +required: | |
| 51 | + - compatible | |
| 52 | + - reg | |
| 53 | + - interrupts | |
| 54 | + - power-domains | |
| 55 | + - clocks | |
| 56 | + | |
| 57 | +additionalProperties: false | |
| 58 | + | |
| 59 | +examples: | |
| 60 | + - | | |
| 61 | + | |
| 62 | + postmask0: postmask@1400d000 { | |
| 63 | + compatible = "mediatek,mt8192-disp-postmask"; | |
| 64 | + reg = <0 0x1400d000 0 0x1000>; | |
| 65 | + interrupts = <GIC_SPI 262 IRQ_TYPE_LEVEL_HIGH 0>; | |
| 66 | + power-domains = <&scpsys MT8192_POWER_DOMAIN_DISP>; | |
| 67 | + clocks = <&mmsys CLK_MM_DISP_POSTMASK0>; | |
| 68 | + mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0xd000 0x1000>; | |
| 69 | + }; | |
| @@ -0,0 +1,117 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,rdma.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek Read Direct Memory Access | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek Read Direct Memory Access(RDMA) component used to read the | |
| 15 | + data into DMA. It provides real time data to the back-end panel | |
| 16 | + driver, such as DSI, DPI and DP_INTF. | |
| 17 | + It contains one line buffer to store the sufficient pixel data. | |
| 18 | + RDMA device node must be siblings to the central MMSYS_CONFIG node. | |
| 19 | + For a description of the MMSYS_CONFIG binding, see | |
| 20 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 21 | + for details. | |
| 22 | + | |
| 23 | +properties: | |
| 24 | + compatible: | |
| 25 | + oneOf: | |
| 26 | + - items: | |
| 27 | + - const: mediatek,mt2701-disp-rdma | |
| 28 | + - items: | |
| 29 | + - const: mediatek,mt8173-disp-rdma | |
| 30 | + - items: | |
| 31 | + - const: mediatek,mt8183-disp-rdma | |
| 32 | + - items: | |
| 33 | + - const: mediatek,mt8195-disp-rdma | |
| 34 | + - items: | |
| 35 | + - enum: | |
| 36 | + - mediatek,mt7623-disp-rdma | |
| 37 | + - mediatek,mt2712-disp-rdma | |
| 38 | + - enum: | |
| 39 | + - mediatek,mt2701-disp-rdma | |
| 40 | + - items: | |
| 41 | + - enum: | |
| 42 | + - mediatek,mt8192-disp-rdma | |
| 43 | + - enum: | |
| 44 | + - mediatek,mt8183-disp-rdma | |
| 45 | + | |
| 46 | + reg: | |
| 47 | + maxItems: 1 | |
| 48 | + | |
| 49 | + interrupts: | |
| 50 | + maxItems: 1 | |
| 51 | + | |
| 52 | + power-domains: | |
| 53 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 54 | + the power controller specified by phandle. See | |
| 55 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 56 | + | |
| 57 | + clocks: | |
| 58 | + items: | |
| 59 | + - description: RDMA Clock | |
| 60 | + | |
| 61 | + iommus: | |
| 62 | + description: | |
| 63 | + This property should point to the respective IOMMU block with master port as argument, | |
| 64 | + see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for details. | |
| 65 | + | |
| 66 | + mediatek,larb: | |
| 67 | + description: | |
| 68 | + This property should contain a phandle pointing to the local arbiter devices defined in | |
| 69 | + Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml. | |
| 70 | + It must sort according to the local arbiter index, like larb0, larb1, larb2... | |
| 71 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 72 | + minItems: 1 | |
| 73 | + maxItems: 32 | |
| 74 | + | |
| 75 | + mediatek,rdma-fifo-size: | |
| 76 | + description: | |
| 77 | + rdma fifo size may be different even in same SOC, add this property to the | |
| 78 | + corresponding rdma. | |
| 79 | + The value below is the Max value which defined in hardware data sheet | |
| 80 | + mediatek,rdma-fifo-size of mt8173-rdma0 is 8K | |
| 81 | + mediatek,rdma-fifo-size of mt8183-rdma0 is 5K | |
| 82 | + mediatek,rdma-fifo-size of mt8183-rdma1 is 2K | |
| 83 | + $ref: /schemas/types.yaml#/definitions/uint32 | |
| 84 | + enum: [8192, 5120, 2048] | |
| 85 | + | |
| 86 | + mediatek,gce-client-reg: | |
| 87 | + description: The register of client driver can be configured by gce with | |
| 88 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 89 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 90 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 91 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 92 | + maxItems: 1 | |
| 93 | + | |
| 94 | +required: | |
| 95 | + - compatible | |
| 96 | + - reg | |
| 97 | + - interrupts | |
| 98 | + - power-domains | |
| 99 | + - clocks | |
| 100 | + - iommus | |
| 101 | + | |
| 102 | +additionalProperties: false | |
| 103 | + | |
| 104 | +examples: | |
| 105 | + - | | |
| 106 | + | |
| 107 | + rdma0: rdma@1400e000 { | |
| 108 | + compatible = "mediatek,mt8173-disp-rdma"; | |
| 109 | + reg = <0 0x1400e000 0 0x1000>; | |
| 110 | + interrupts = <GIC_SPI 182 IRQ_TYPE_LEVEL_LOW>; | |
| 111 | + power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 112 | + clocks = <&mmsys CLK_MM_DISP_RDMA0>; | |
| 113 | + iommus = <&iommu M4U_PORT_DISP_RDMA0>; | |
| 114 | + mediatek,larb = <&larb0>; | |
| 115 | + mediatek,rdma-fifosize = <8192>; | |
| 116 | + mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0xe000 0x1000>; | |
| 117 | + }; | |
| @@ -0,0 +1,58 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,split.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display split | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display split, namely SPLIT, is used to split stream to two | |
| 15 | + encoders. | |
| 16 | + SPLIT device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt8173-disp-split | |
| 26 | + | |
| 27 | + reg: | |
| 28 | + maxItems: 1 | |
| 29 | + | |
| 30 | + interrupts: | |
| 31 | + maxItems: 1 | |
| 32 | + | |
| 33 | + power-domains: | |
| 34 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 35 | + the power controller specified by phandle. See | |
| 36 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 37 | + | |
| 38 | + clocks: | |
| 39 | + items: | |
| 40 | + - description: SPLIT Clock | |
| 41 | + | |
| 42 | +required: | |
| 43 | + - compatible | |
| 44 | + - reg | |
| 45 | + - power-domains | |
| 46 | + - clocks | |
| 47 | + | |
| 48 | +additionalProperties: false | |
| 49 | + | |
| 50 | +examples: | |
| 51 | + - | | |
| 52 | + | |
| 53 | + split0: split@14018000 { | |
| 54 | + compatible = "mediatek,mt8173-disp-split"; | |
| 55 | + reg = <0 0x14018000 0 0x1000>; | |
| 56 | + power-domains = <&spm MT8173_POWER_DOMAIN_MM>; | |
| 57 | + clocks = <&mmsys CLK_MM_DISP_SPLIT0>; | |
| 58 | + }; | |
| @@ -0,0 +1,61 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,ufoe.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek display UFOe | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek display UFOe stands for Unified Frame Optimization engine. | |
| 15 | + UFOe can cut the data rate for DSI port which may lead to reduce power | |
| 16 | + consumption. | |
| 17 | + UFOe device node must be siblings to the central MMSYS_CONFIG node. | |
| 18 | + For a description of the MMSYS_CONFIG binding, see | |
| 19 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 20 | + for details. | |
| 21 | + | |
| 22 | +properties: | |
| 23 | + compatible: | |
| 24 | + oneOf: | |
| 25 | + - items: | |
| 26 | + - const: mediatek,mt8173-disp-ufoe | |
| 27 | + | |
| 28 | + reg: | |
| 29 | + maxItems: 1 | |
| 30 | + | |
| 31 | + interrupts: | |
| 32 | + maxItems: 1 | |
| 33 | + | |
| 34 | + power-domains: | |
| 35 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 36 | + the power controller specified by phandle. See | |
| 37 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 38 | + | |
| 39 | + clocks: | |
| 40 | + items: | |
| 41 | + - description: UFOe Clock | |
| 42 | + | |
| 43 | +required: | |
| 44 | + - compatible | |
| 45 | + - reg | |
| 46 | + - interrupts | |
| 47 | + - power-domains | |
| 48 | + - clocks | |
| 49 | + | |
| 50 | +additionalProperties: false | |
| 51 | + | |
| 52 | +examples: | |
| 53 | + - | | |
| 54 | + | |
| 55 | + ufoe@1401a000 { | |
| 56 | + compatible = "mediatek,mt8173-disp-ufoe"; | |
| 57 | + reg = <0 0x1401a000 0 0x1000>; | |
| 58 | + interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_LOW>; | |
| 59 | + power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 60 | + clocks = <&mmsys CLK_MM_DISP_UFOE>; | |
| 61 | + }; | |
| @@ -0,0 +1,86 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/mediatek/mediatek,wdma.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Mediatek Write Direct Memory Access | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Chun-Kuang Hu <chunkuang.hu@kernel.org> | |
| 11 | + - Philipp Zabel <p.zabel@pengutronix.de> | |
| 12 | + | |
| 13 | +description: | | |
| 14 | + Mediatek Write Direct Memory Access(WDMA) component used to write | |
| 15 | + the data into DMA. | |
| 16 | + WDMA device node must be siblings to the central MMSYS_CONFIG node. | |
| 17 | + For a description of the MMSYS_CONFIG binding, see | |
| 18 | + Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | |
| 19 | + for details. | |
| 20 | + | |
| 21 | +properties: | |
| 22 | + compatible: | |
| 23 | + oneOf: | |
| 24 | + - items: | |
| 25 | + - const: mediatek,mt8173-disp-wdma | |
| 26 | + | |
| 27 | + reg: | |
| 28 | + maxItems: 1 | |
| 29 | + | |
| 30 | + interrupts: | |
| 31 | + maxItems: 1 | |
| 32 | + | |
| 33 | + power-domains: | |
| 34 | + description: A phandle and PM domain specifier as defined by bindings of | |
| 35 | + the power controller specified by phandle. See | |
| 36 | + Documentation/devicetree/bindings/power/power-domain.yaml for details. | |
| 37 | + | |
| 38 | + clocks: | |
| 39 | + items: | |
| 40 | + - description: WDMA Clock | |
| 41 | + | |
| 42 | + iommus: | |
| 43 | + description: | |
| 44 | + This property should point to the respective IOMMU block with master port as argument, | |
| 45 | + see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for details. | |
| 46 | + | |
| 47 | + mediatek,larb: | |
| 48 | + description: | |
| 49 | + This property should contain a phandle pointing to the local arbiter devices defined in | |
| 50 | + Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml. | |
| 51 | + It must sort according to the local arbiter index, like larb0, larb1, larb2... | |
| 52 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 53 | + minItems: 1 | |
| 54 | + maxItems: 32 | |
| 55 | + | |
| 56 | + mediatek,gce-client-reg: | |
| 57 | + description: The register of client driver can be configured by gce with | |
| 58 | + 4 arguments defined in this property, such as phandle of gce, subsys id, | |
| 59 | + register offset and size. Each GCE subsys id is mapping to a client | |
| 60 | + defined in the header include/dt-bindings/gce/<chip>-gce.h. | |
| 61 | + $ref: /schemas/types.yaml#/definitions/phandle-array | |
| 62 | + maxItems: 1 | |
| 63 | + | |
| 64 | +required: | |
| 65 | + - compatible | |
| 66 | + - reg | |
| 67 | + - interrupts | |
| 68 | + - power-domains | |
| 69 | + - clocks | |
| 70 | + - iommus | |
| 71 | + | |
| 72 | +additionalProperties: false | |
| 73 | + | |
| 74 | +examples: | |
| 75 | + - | | |
| 76 | + | |
| 77 | + wdma0: wdma@14011000 { | |
| 78 | + compatible = "mediatek,mt8173-disp-wdma"; | |
| 79 | + reg = <0 0x14011000 0 0x1000>; | |
| 80 | + interrupts = <GIC_SPI 185 IRQ_TYPE_LEVEL_LOW>; | |
| 81 | + power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | |
| 82 | + clocks = <&mmsys CLK_MM_DISP_WDMA0>; | |
| 83 | + iommus = <&iommu M4U_PORT_DISP_WDMA0>; | |
| 84 | + mediatek,larb = <&larb0>; | |
| 85 | + mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x1000 0x1000>; | |
| 86 | + }; | |
| @@ -11,13 +11,23 @@ maintainers: · 10 unmodified lines | ||
| 11 | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | |
| 13 | 13 | allOf: |
| 14 | − - $ref: lvds.yaml# | |
| 14 | + - $ref: panel-common.yaml# | |
| 15 | + - $ref: /schemas/display/lvds.yaml/# | |
| 16 | + | |
| 17 | +select: | |
| 18 | + properties: | |
| 19 | + compatible: | |
| 20 | + contains: | |
| 21 | + const: advantech,idk-1110wr | |
| 22 | + | |
| 23 | + required: | |
| 24 | + - compatible | |
| 15 | 25 | |
| 16 | 26 | properties: |
| 17 | 27 | compatible: |
| 18 | 28 | items: |
| 19 | 29 | - const: advantech,idk-1110wr |
| 20 | − - {} # panel-lvds, but not listed here to avoid false select | |
| 30 | + - const: panel-lvds | |
| 21 | 31 | |
| 22 | 32 | data-mapping: |
| 23 | 33 | const: jeida-24 |
| @@ -35,6 +45,11 @@ additionalProperties: false · 11 unmodified lines | ||
| 35 | 45 | |
| 36 | 46 | required: |
| 37 | 47 | - compatible |
| 48 | + - data-mapping | |
| 49 | + - width-mm | |
| 50 | + - height-mm | |
| 51 | + - panel-timing | |
| 52 | + - port | |
| 38 | 53 | |
| 39 | 54 | examples: |
| 40 | 55 | - |+ |
| @@ -11,15 +11,26 @@ maintainers: · 10 unmodified lines | ||
| 11 | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | |
| 13 | 13 | allOf: |
| 14 | − - $ref: lvds.yaml# | |
| 14 | + - $ref: panel-common.yaml# | |
| 15 | + - $ref: /schemas/display/lvds.yaml/# | |
| 16 | + | |
| 17 | +select: | |
| 18 | + properties: | |
| 19 | + compatible: | |
| 20 | + contains: | |
| 21 | + const: innolux,ee101ia-01d | |
| 22 | + | |
| 23 | + required: | |
| 24 | + - compatible | |
| 15 | 25 | |
| 16 | 26 | properties: |
| 17 | 27 | compatible: |
| 18 | 28 | items: |
| 19 | 29 | - const: innolux,ee101ia-01d |
| 20 | − - {} # panel-lvds, but not listed here to avoid false select | |
| 30 | + - const: panel-lvds | |
| 21 | 31 | |
| 22 | 32 | backlight: true |
| 33 | + data-mapping: true | |
| 23 | 34 | enable-gpios: true |
| 24 | 35 | power-supply: true |
| 25 | 36 | width-mm: true |
| @@ -27,5 +38,13 @@ properties: · 1 unmodified line | ||
| 27 | 38 | panel-timing: true |
| 28 | 39 | port: true |
| 29 | 40 | |
| 41 | +required: | |
| 42 | + - compatible | |
| 43 | + - data-mapping | |
| 44 | + - width-mm | |
| 45 | + - height-mm | |
| 46 | + - panel-timing | |
| 47 | + - port | |
| 48 | + | |
| 30 | 49 | additionalProperties: false |
| 31 | 50 | ... |
| @@ -11,13 +11,23 @@ maintainers: · 10 unmodified lines | ||
| 11 | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | |
| 13 | 13 | allOf: |
| 14 | − - $ref: lvds.yaml# | |
| 14 | + - $ref: panel-common.yaml# | |
| 15 | + - $ref: /schemas/display/lvds.yaml/# | |
| 16 | + | |
| 17 | +select: | |
| 18 | + properties: | |
| 19 | + compatible: | |
| 20 | + contains: | |
| 21 | + const: mitsubishi,aa104xd12 | |
| 22 | + | |
| 23 | + required: | |
| 24 | + - compatible | |
| 15 | 25 | |
| 16 | 26 | properties: |
| 17 | 27 | compatible: |
| 18 | 28 | items: |
| 19 | 29 | - const: mitsubishi,aa104xd12 |
| 20 | − - {} # panel-lvds, but not listed here to avoid false select | |
| 30 | + - const: panel-lvds | |
| 21 | 31 | |
| 22 | 32 | vcc-supply: |
| 23 | 33 | description: Reference to the regulator powering the panel VCC pins. |
| @@ -39,6 +49,11 @@ additionalProperties: false · 15 unmodified lines | ||
| 39 | 49 | required: |
| 40 | 50 | - compatible |
| 41 | 51 | - vcc-supply |
| 52 | + - data-mapping | |
| 53 | + - width-mm | |
| 54 | + - height-mm | |
| 55 | + - panel-timing | |
| 56 | + - port | |
| 42 | 57 | |
| 43 | 58 | examples: |
| 44 | 59 | - |+ |
| @@ -11,13 +11,23 @@ maintainers: · 10 unmodified lines | ||
| 11 | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | |
| 13 | 13 | allOf: |
| 14 | − - $ref: lvds.yaml# | |
| 14 | + - $ref: panel-common.yaml# | |
| 15 | + - $ref: /schemas/display/lvds.yaml/# | |
| 16 | + | |
| 17 | +select: | |
| 18 | + properties: | |
| 19 | + compatible: | |
| 20 | + contains: | |
| 21 | + const: mitsubishi,aa121td01 | |
| 22 | + | |
| 23 | + required: | |
| 24 | + - compatible | |
| 15 | 25 | |
| 16 | 26 | properties: |
| 17 | 27 | compatible: |
| 18 | 28 | items: |
| 19 | 29 | - const: mitsubishi,aa121td01 |
| 20 | − - {} # panel-lvds, but not listed here to avoid false select | |
| 30 | + - const: panel-lvds | |
| 21 | 31 | |
| 22 | 32 | vcc-supply: |
| 23 | 33 | description: Reference to the regulator powering the panel VCC pins. |
| @@ -39,6 +49,11 @@ additionalProperties: false · 15 unmodified lines | ||
| 39 | 49 | required: |
| 40 | 50 | - compatible |
| 41 | 51 | - vcc-supply |
| 52 | + - data-mapping | |
| 53 | + - width-mm | |
| 54 | + - height-mm | |
| 55 | + - panel-timing | |
| 56 | + - port | |
| 42 | 57 | |
| 43 | 58 | examples: |
| 44 | 59 | - |+ |
| @@ -0,0 +1,57 @@ | ||
| 1 | +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | |
| 2 | +%YAML 1.2 | |
| 3 | +--- | |
| 4 | +$id: http://devicetree.org/schemas/display/panel/panel-lvds.yaml# | |
| 5 | +$schema: http://devicetree.org/meta-schemas/core.yaml# | |
| 6 | + | |
| 7 | +title: Generic LVDS Display Panel Device Tree Bindings | |
| 8 | + | |
| 9 | +maintainers: | |
| 10 | + - Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> | |
| 11 | + - Thierry Reding <thierry.reding@gmail.com> | |
| 12 | + | |
| 13 | +allOf: | |
| 14 | + - $ref: panel-common.yaml# | |
| 15 | + - $ref: /schemas/display/lvds.yaml/# | |
| 16 | + | |
| 17 | +select: | |
| 18 | + properties: | |
| 19 | + compatible: | |
| 20 | + contains: | |
| 21 | + const: panel-lvds | |
| 22 | + | |
| 23 | + not: | |
| 24 | + properties: | |
| 25 | + compatible: | |
| 26 | + contains: | |
| 27 | + enum: | |
| 28 | + - advantech,idk-1110wr | |
| 29 | + - advantech,idk-2121wr | |
| 30 | + - innolux,ee101ia-01d | |
| 31 | + - mitsubishi,aa104xd12 | |
| 32 | + - mitsubishi,aa121td01 | |
| 33 | + - sgd,gktw70sdae4se | |
| 34 | + | |
| 35 | + required: | |
| 36 | + - compatible | |
| 37 | + | |
| 38 | +properties: | |
| 39 | + compatible: | |
| 40 | + items: | |
| 41 | + - enum: | |
| 42 | + - auo,b101ew05 | |
| 43 | + - tbs,a711-panel | |
| 44 | + | |
| 45 | + - const: panel-lvds | |
| 46 | + | |
| 47 | +unevaluatedProperties: false | |
| 48 | + | |
| 49 | +required: | |
| 50 | + - compatible | |
| 51 | + - data-mapping | |
| 52 | + - width-mm | |
| 53 | + - height-mm | |
| 54 | + - panel-timing | |
| 55 | + - port | |
| 56 | + | |
| 57 | +... | |
| @@ -284,6 +284,8 @@ properties: · 283 unmodified lines | ||
| 284 | 284 | - sharp,lq101k1ly04 |
| 285 | 285 | # Sharp 12.3" (2400x1600 pixels) TFT LCD panel |
| 286 | 286 | - sharp,lq123p1jx31 |
| 287 | + # Sharp 14" (1920x1080 pixels) TFT LCD panel | |
| 288 | + - sharp,lq140m1jw46 | |
| 287 | 289 | # Sharp LS020B1DD01D 2.0" HQVGA TFT LCD panel |
| 288 | 290 | - sharp,ls020b1dd01d |
| 289 | 291 | # Shelly SCA07010-BFN-LNN 7.0" WVGA TFT LCD panel |
| @@ -11,13 +11,23 @@ maintainers: · 10 unmodified lines | ||
| 11 | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | |
| 13 | 13 | allOf: |
| 14 | − - $ref: lvds.yaml# | |
| 14 | + - $ref: panel-common.yaml# | |
| 15 | + - $ref: /schemas/display/lvds.yaml/# | |
| 16 | + | |
| 17 | +select: | |
| 18 | + properties: | |
| 19 | + compatible: | |
| 20 | + contains: | |
| 21 | + const: sgd,gktw70sdae4se | |
| 22 | + | |
| 23 | + required: | |
| 24 | + - compatible | |
| 15 | 25 | |
| 16 | 26 | properties: |
| 17 | 27 | compatible: |
| 18 | 28 | items: |
| 19 | 29 | - const: sgd,gktw70sdae4se |
| 20 | − - {} # panel-lvds, but not listed here to avoid false select | |
| 30 | + - const: panel-lvds | |
| 21 | 31 | |
| 22 | 32 | data-mapping: |
| 23 | 33 | const: jeida-18 |
| @@ -35,6 +45,11 @@ additionalProperties: false · 11 unmodified lines | ||
| 35 | 45 | |
| 36 | 46 | required: |
| 37 | 47 | - compatible |
| 48 | + - port | |
| 49 | + - data-mapping | |
| 50 | + - width-mm | |
| 51 | + - height-mm | |
| 52 | + - panel-timing | |
| 38 | 53 | |
| 39 | 54 | examples: |
| 40 | 55 | - |+ |
| @@ -8,6 +8,7 @@ title: Solomon SSD1307 OLED Controller Framebuffer · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | 10 | - Maxime Ripard <mripard@kernel.org> |
| 11 | + - Javier Martinez Canillas <javierm@redhat.com> | |
| 11 | 12 | |
| 12 | 13 | properties: |
| 13 | 14 | compatible: |
| @@ -7,7 +7,6 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: SiFive GPIO controller |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Yash Shah <yash.shah@sifive.com> | |
| 11 | 10 | - Paul Walmsley <paul.walmsley@sifive.com> |
| 12 | 11 | |
| 13 | 12 | properties: |
| @@ -159,6 +159,21 @@ allOf: · 158 unmodified lines | ||
| 159 | 159 | power-domains: |
| 160 | 160 | maxItems: 1 |
| 161 | 161 | sram-supply: false |
| 162 | + - if: | |
| 163 | + properties: | |
| 164 | + compatible: | |
| 165 | + contains: | |
| 166 | + const: rockchip,rk3568-mali | |
| 167 | + then: | |
| 168 | + properties: | |
| 169 | + clocks: | |
| 170 | + minItems: 2 | |
| 171 | + clock-names: | |
| 172 | + items: | |
| 173 | + - const: gpu | |
| 174 | + - const: bus | |
| 175 | + required: | |
| 176 | + - clock-names | |
| 162 | 177 | |
| 163 | 178 | examples: |
| 164 | 179 | - | |
| @@ -35,6 +35,10 @@ description: · 34 unmodified lines | ||
| 35 | 35 | contains a specific memory layout, which is documented in chapter 8 of the |
| 36 | 36 | SiFive U5 Coreplex Series Manual <https://static.dev.sifive.com/U54-MC-RVCoreIP.pdf>. |
| 37 | 37 | |
| 38 | + The thead,c900-plic is different from sifive,plic-1.0.0 in opensbi, the | |
| 39 | + T-HEAD PLIC implementation requires setting a delegation bit to allow access | |
| 40 | + from S-mode. So add thead,c900-plic to distinguish them. | |
| 41 | + | |
| 38 | 42 | maintainers: |
| 39 | 43 | - Sagar Kadam <sagar.kadam@sifive.com> |
| 40 | 44 | - Paul Walmsley <paul.walmsley@sifive.com> |
| @@ -42,12 +46,17 @@ maintainers: · 1 unmodified line | ||
| 42 | 46 | |
| 43 | 47 | properties: |
| 44 | 48 | compatible: |
| 45 | − items: | |
| 46 | − - enum: | |
| 47 | − - sifive,fu540-c000-plic | |
| 48 | − - starfive,jh7100-plic | |
| 49 | − - canaan,k210-plic | |
| 50 | − - const: sifive,plic-1.0.0 | |
| 49 | + oneOf: | |
| 50 | + - items: | |
| 51 | + - enum: | |
| 52 | + - sifive,fu540-c000-plic | |
| 53 | + - starfive,jh7100-plic | |
| 54 | + - canaan,k210-plic | |
| 55 | + - const: sifive,plic-1.0.0 | |
| 56 | + - items: | |
| 57 | + - enum: | |
| 58 | + - allwinner,sun20i-d1-plic | |
| 59 | + - const: thead,c900-plic | |
| 51 | 60 | |
| 52 | 61 | reg: |
| 53 | 62 | maxItems: 1 |
| @@ -20,7 +20,7 @@ description: | · 19 unmodified lines | ||
| 20 | 20 | |
| 21 | 21 | maintainers: |
| 22 | 22 | - Kishon Vijay Abraham I <kishon@ti.com> |
| 23 | − - Roger Quadros <rogerq@ti.com | |
| 23 | + - Roger Quadros <rogerq@kernel.org> | |
| 24 | 24 | |
| 25 | 25 | properties: |
| 26 | 26 | compatible: |
| @@ -107,6 +107,10 @@ properties: · 106 unmodified lines | ||
| 107 | 107 | - const: imem |
| 108 | 108 | - const: config |
| 109 | 109 | |
| 110 | + qcom,qmp: | |
| 111 | + $ref: /schemas/types.yaml#/definitions/phandle | |
| 112 | + description: phandle to the AOSS side-channel message RAM | |
| 113 | + | |
| 110 | 114 | qcom,smem-states: |
| 111 | 115 | $ref: /schemas/types.yaml#/definitions/phandle-array |
| 112 | 116 | description: State bits used in by the AP to signal the modem. |
| @@ -222,6 +226,8 @@ examples: · 109 unmodified lines | ||
| 222 | 226 | "imem", |
| 223 | 227 | "config"; |
| 224 | 228 | |
| 229 | + qcom,qmp = <&aoss_qmp>; | |
| 230 | + | |
| 225 | 231 | qcom,smem-states = <&ipa_smp2p_out 0>, |
| 226 | 232 | <&ipa_smp2p_out 1>; |
| 227 | 233 | qcom,smem-state-names = "ipa-clock-enabled-valid", |
| @@ -8,7 +8,7 @@ title: OMAP USB2 PHY · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | 10 | - Kishon Vijay Abraham I <kishon@ti.com> |
| 11 | − - Roger Quadros <rogerq@ti.com> | |
| 11 | + - Roger Quadros <rogerq@kernel.org> | |
| 12 | 12 | |
| 13 | 13 | properties: |
| 14 | 14 | compatible: |
| @@ -8,7 +8,6 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 7 unmodified lines | ||
| 8 | 8 | title: SiFive PWM controller |
| 9 | 9 | |
| 10 | 10 | maintainers: |
| 11 | − - Yash Shah <yash.shah@sifive.com> | |
| 12 | 11 | - Sagar Kadam <sagar.kadam@sifive.com> |
| 13 | 12 | - Paul Walmsley <paul.walmsley@sifive.com> |
| 14 | 13 | |
| @@ -9,7 +9,6 @@ title: SiFive L2 Cache Controller · 8 unmodified lines | ||
| 9 | 9 | |
| 10 | 10 | maintainers: |
| 11 | 11 | - Sagar Kadam <sagar.kadam@sifive.com> |
| 12 | − - Yash Shah <yash.shah@sifive.com> | |
| 13 | 12 | - Paul Walmsley <paul.walmsley@sifive.com> |
| 14 | 13 | |
| 15 | 14 | description: |
| @@ -8,6 +8,7 @@ title: Audio codec controlled by ChromeOS EC · 7 unmodified lines | ||
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | 10 | - Cheng-Yi Chiang <cychiang@chromium.org> |
| 11 | + - Tzung-Bi Shih <tzungbi@google.com> | |
| 11 | 12 | |
| 12 | 13 | description: | |
| 13 | 14 | Google's ChromeOS EC codec is a digital mic codec provided by the |
| @@ -23,8 +23,9 @@ properties: · 22 unmodified lines | ||
| 23 | 23 | minItems: 1 |
| 24 | 24 | maxItems: 256 |
| 25 | 25 | items: |
| 26 | − minimum: 0 | |
| 27 | − maximum: 256 | |
| 26 | + items: | |
| 27 | + - minimum: 0 | |
| 28 | + maximum: 256 | |
| 28 | 29 | description: |
| 29 | 30 | Chip select used by the device. |
| 30 | 31 | |
| @@ -7,7 +7,7 @@ $schema: "http://devicetree.org/meta-schemas/core.yaml#" · 6 unmodified lines | ||
| 7 | 7 | title: Bindings for the TI wrapper module for the Cadence USBSS-DRD controller |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Roger Quadros <rogerq@ti.com> | |
| 10 | + - Roger Quadros <rogerq@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | properties: |
| 13 | 13 | compatible: |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | ||
| 7 | 7 | title: TI Keystone Soc USB Controller |
| 8 | 8 | |
| 9 | 9 | maintainers: |
| 10 | − - Roger Quadros <rogerq@ti.com> | |
| 10 | + - Roger Quadros <rogerq@kernel.org> | |
| 11 | 11 | |
| 12 | 12 | properties: |
| 13 | 13 | compatible: |
| @@ -462,6 +462,10 @@ operation table looks like the following:: · 461 unmodified lines | ||
| 462 | 462 | struct iov_iter *iter, |
| 463 | 463 | netfs_io_terminated_t term_func, |
| 464 | 464 | void *term_func_priv); |
| 465 | + | |
| 466 | + int (*query_occupancy)(struct netfs_cache_resources *cres, | |
| 467 | + loff_t start, size_t len, size_t granularity, | |
| 468 | + loff_t *_data_start, size_t *_data_len); | |
| 465 | 469 | }; |
| 466 | 470 | |
| 467 | 471 | With a termination handler function pointer:: |
| @@ -536,6 +540,18 @@ The methods defined in the table are: · 68 unmodified lines | ||
| 536 | 540 | indicating whether the termination is definitely happening in the caller's |
| 537 | 541 | context. |
| 538 | 542 | |
| 543 | + * ``query_occupancy()`` | |
| 544 | + | |
| 545 | + [Required] Called to find out where the next piece of data is within a | |
| 546 | + particular region of the cache. The start and length of the region to be | |
| 547 | + queried are passed in, along with the granularity to which the answer needs | |
| 548 | + to be aligned. The function passes back the start and length of the data, | |
| 549 | + if any, available within that region. Note that there may be a hole at the | |
| 550 | + front. | |
| 551 | + | |
| 552 | + It returns 0 if some data was found, -ENODATA if there was no usable data | |
| 553 | + within the region or -ENOBUFS if there is no caching on this file. | |
| 554 | + | |
| 539 | 555 | Note that these methods are passed a pointer to the cache resource structure, |
| 540 | 556 | not the read request structure as they could be used in other situations where |
| 541 | 557 | there isn't a read request structure as well, such as writing dirty data to the |
| @@ -300,30 +300,6 @@ Contact: Daniel Vetter, Noralf Tronnes · 299 unmodified lines | ||
| 300 | 300 | |
| 301 | 301 | Level: Advanced |
| 302 | 302 | |
| 303 | −Garbage collect fbdev scrolling acceleration | |
| 304 | −-------------------------------------------- | |
| 305 | − | |
| 306 | −Scroll acceleration has been disabled in fbcon. Now it works as the old | |
| 307 | −SCROLL_REDRAW mode. A ton of code was removed in fbcon.c and the hook bmove was | |
| 308 | −removed from fbcon_ops. | |
| 309 | −Remaining tasks: | |
| 310 | − | |
| 311 | −- a bunch of the hooks in fbcon_ops could be removed or simplified by calling | |
| 312 | − directly instead of the function table (with a switch on p->rotate) | |
| 313 | − | |
| 314 | −- fb_copyarea is unused after this, and can be deleted from all drivers | |
| 315 | − | |
| 316 | −- after that, fb_copyarea can be deleted from fb_ops in include/linux/fb.h as | |
| 317 | − well as cfb_copyarea | |
| 318 | − | |
| 319 | −Note that not all acceleration code can be deleted, since clearing and cursor | |
| 320 | −support is still accelerated, which might be good candidates for further | |
| 321 | −deletion projects. | |
| 322 | − | |
| 323 | −Contact: Daniel Vetter | |
| 324 | − | |
| 325 | −Level: Intermediate | |
| 326 | − | |
| 327 | 303 | idr_init_base() |
| 328 | 304 | --------------- |
| 329 | 305 | |
| @@ -2,7 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | |
| 3 | 3 | Set the histogram bucket size (default *1*). |
| 4 | 4 | |
| 5 | −**-e**, **--entries** *N* | |
| 5 | +**-E**, **--entries** *N* | |
| 6 | 6 | |
| 7 | 7 | Set the number of entries of the histogram (default 256). |
| 8 | 8 | |
| @@ -1,7 +1,7 @@ | ||
| 1 | 1 | The **rtla osnoise** tool is an interface for the *osnoise* tracer. The |
| 2 | 2 | *osnoise* tracer dispatches a kernel thread per-cpu. These threads read the |
| 3 | 3 | time in a loop while with preemption, softirq and IRQs enabled, thus |
| 4 | −allowing all the sources of operating systme noise during its execution. | |
| 4 | +allowing all the sources of operating system noise during its execution. | |
| 5 | 5 | The *osnoise*'s tracer threads take note of the delta between each time |
| 6 | 6 | read, along with an interference counter of all sources of interference. |
| 7 | 7 | At the end of each period, the *osnoise* tracer displays a summary of |
| @@ -36,7 +36,7 @@ default). The reason for reducing the runtime is to avoid starving the · 35 unmodified lines | ||
| 36 | 36 | **rtla** tool. The tool is also set to run for *one minute*. The output |
| 37 | 37 | histogram is set to group outputs in buckets of *10us* and *25* entries:: |
| 38 | 38 | |
| 39 | − [root@f34 ~/]# rtla osnoise hist -P F:1 -c 0-11 -r 900000 -d 1M -b 10 -e 25 | |
| 39 | + [root@f34 ~/]# rtla osnoise hist -P F:1 -c 0-11 -r 900000 -d 1M -b 10 -E 25 | |
| 40 | 40 | # RTLA osnoise histogram |
| 41 | 41 | # Time unit is microseconds (us) |
| 42 | 42 | # Duration: 0 00:01:00 |
| @@ -84,6 +84,8 @@ CPUfreq核心层注册一个cpufreq_driver结构体。 · 83 unmodified lines | ||
| 84 | 84 | .resume - 一个指向per-policy恢复函数的指针,该函数在关中断且在调节器再一次启动前被 |
| 85 | 85 | 调用。 |
| 86 | 86 | |
| 87 | + .ready - 一个指向per-policy准备函数的指针,该函数在策略完全初始化之后被调用。 | |
| 88 | + | |
| 87 | 89 | .attr - 一个指向NULL结尾的"struct freq_attr"列表的指针,该列表允许导出值到 |
| 88 | 90 | sysfs。 |
| 89 | 91 | |
| @@ -115,6 +115,7 @@ Code Seq# Include File Comments · 114 unmodified lines | ||
| 115 | 115 | 'B' 00-1F linux/cciss_ioctl.h conflict! |
| 116 | 116 | 'B' 00-0F include/linux/pmu.h conflict! |
| 117 | 117 | 'B' C0-FF advanced bbus <mailto:maassen@uni-freiburg.de> |
| 118 | +'B' 00-0F xen/xenbus_dev.h conflict! | |
| 118 | 119 | 'C' all linux/soundcard.h conflict! |
| 119 | 120 | 'C' 01-2F linux/capi.h conflict! |
| 120 | 121 | 'C' F0-FF drivers/net/wan/cosa.h conflict! |
| @@ -134,6 +135,7 @@ Code Seq# Include File Comments · 13 unmodified lines | ||
| 134 | 135 | 'F' 80-8F linux/arcfb.h conflict! |
| 135 | 136 | 'F' DD video/sstfb.h conflict! |
| 136 | 137 | 'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict! |
| 138 | +'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict! | |
| 137 | 139 | 'H' 00-7F linux/hiddev.h conflict! |
| 138 | 140 | 'H' 00-0F linux/hidraw.h conflict! |
| 139 | 141 | 'H' 01 linux/mei.h conflict! |
| @@ -176,6 +178,7 @@ Code Seq# Include File Comments · 36 unmodified lines | ||
| 176 | 178 | 'P' 60-6F sound/sscape_ioctl.h conflict! |
| 177 | 179 | 'P' 00-0F drivers/usb/class/usblp.c conflict! |
| 178 | 180 | 'P' 01-09 drivers/misc/pci_endpoint_test.c conflict! |
| 181 | +'P' 00-0F xen/privcmd.h conflict! | |
| 179 | 182 | 'Q' all linux/soundcard.h |
| 180 | 183 | 'R' 00-1F linux/random.h conflict! |
| 181 | 184 | 'R' 01 linux/rfkill.h conflict! |
| @@ -1394,7 +1394,7 @@ documentation when it pops into existence). · 1393 unmodified lines | ||
| 1394 | 1394 | ------------------- |
| 1395 | 1395 | |
| 1396 | 1396 | :Capability: KVM_CAP_ENABLE_CAP |
| 1397 | −:Architectures: mips, ppc, s390 | |
| 1397 | +:Architectures: mips, ppc, s390, x86 | |
| 1398 | 1398 | :Type: vcpu ioctl |
| 1399 | 1399 | :Parameters: struct kvm_enable_cap (in) |
| 1400 | 1400 | :Returns: 0 on success; -1 on error |
| @@ -6997,6 +6997,20 @@ indicated by the fd to the VM this is called on. · 5596 unmodified lines | ||
| 6997 | 6997 | This is intended to support intra-host migration of VMs between userspace VMMs, |
| 6998 | 6998 | upgrading the VMM process without interrupting the guest. |
| 6999 | 6999 | |
| 7000 | +7.30 KVM_CAP_PPC_AIL_MODE_3 | |
| 7001 | +------------------------------- | |
| 7002 | + | |
| 7003 | +:Capability: KVM_CAP_PPC_AIL_MODE_3 | |
| 7004 | +:Architectures: ppc | |
| 7005 | +:Type: vm | |
| 7006 | + | |
| 7007 | +This capability indicates that the kernel supports the mode 3 setting for the | |
| 7008 | +"Address Translation Mode on Interrupt" aka "Alternate Interrupt Location" | |
| 7009 | +resource that is controlled with the H_SET_MODE hypercall. | |
| 7010 | + | |
| 7011 | +This capability allows a guest kernel to use a better-performance mode for | |
| 7012 | +handling interrupts and system calls. | |
| 7013 | + | |
| 7000 | 7014 | 8. Other capabilities. |
| 7001 | 7015 | ====================== |
| 7002 | 7016 | |
| @@ -1620,6 +1620,7 @@ M: Olof Johansson <olof@lixom.net> · 1619 unmodified lines | ||
| 1620 | 1620 | M: soc@kernel.org |
| 1621 | 1621 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 1622 | 1622 | S: Maintained |
| 1623 | +C: irc://irc.libera.chat/armlinux | |
| 1623 | 1624 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git |
| 1624 | 1625 | F: arch/arm/boot/dts/Makefile |
| 1625 | 1626 | F: arch/arm64/boot/dts/Makefile |
| @@ -1627,6 +1628,7 @@ F: arch/arm64/boot/dts/Makefile · 1 unmodified line | ||
| 1627 | 1628 | ARM SUB-ARCHITECTURES |
| 1628 | 1629 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 1629 | 1630 | S: Maintained |
| 1631 | +C: irc://irc.libera.chat/armlinux | |
| 1630 | 1632 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git |
| 1631 | 1633 | F: arch/arm/mach-*/ |
| 1632 | 1634 | F: arch/arm/plat-*/ |
| @@ -1780,6 +1782,7 @@ F: drivers/irqchip/irq-apple-aic.c · 147 unmodified lines | ||
| 1780 | 1782 | F: drivers/mailbox/apple-mailbox.c |
| 1781 | 1783 | F: drivers/pinctrl/pinctrl-apple-gpio.c |
| 1782 | 1784 | F: drivers/soc/apple/* |
| 1785 | +F: drivers/watchdog/apple_wdt.c | |
| 1783 | 1786 | F: include/dt-bindings/interrupt-controller/apple-aic.h |
| 1784 | 1787 | F: include/dt-bindings/pinctrl/apple.h |
| 1785 | 1788 | F: include/linux/apple-mailbox.h |
| @@ -2570,10 +2573,13 @@ N: rockchip · 784 unmodified lines | ||
| 2570 | 2573 | |
| 2571 | 2574 | ARM/SAMSUNG S3C, S5P AND EXYNOS ARM ARCHITECTURES |
| 2572 | 2575 | M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com> |
| 2576 | +R: Alim Akhtar <alim.akhtar@samsung.com> | |
| 2573 | 2577 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 2574 | 2578 | L: linux-samsung-soc@vger.kernel.org |
| 2575 | 2579 | S: Maintained |
| 2580 | +C: irc://irc.libera.chat/linux-exynos | |
| 2576 | 2581 | Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/ |
| 2582 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git | |
| 2577 | 2583 | F: Documentation/arm/samsung/ |
| 2578 | 2584 | F: Documentation/devicetree/bindings/arm/samsung/ |
| 2579 | 2585 | F: Documentation/devicetree/bindings/power/pd-samsung.yaml |
| @@ -3133,11 +3139,9 @@ W: https://wireless.wiki.kernel.org/en/users/Drivers/ath5k · 553 unmodified lines | ||
| 3133 | 3139 | F: drivers/net/wireless/ath/ath5k/ |
| 3134 | 3140 | |
| 3135 | 3141 | ATHEROS ATH6KL WIRELESS DRIVER |
| 3136 | −M: Kalle Valo <kvalo@kernel.org> | |
| 3137 | 3142 | L: linux-wireless@vger.kernel.org |
| 3138 | −S: Supported | |
| 3143 | +S: Orphan | |
| 3139 | 3144 | W: https://wireless.wiki.kernel.org/en/users/Drivers/ath6kl |
| 3140 | −T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git | |
| 3141 | 3145 | F: drivers/net/wireless/ath/ath6kl/ |
| 3142 | 3146 | |
| 3143 | 3147 | ATI_REMOTE2 DRIVER |
| @@ -4157,9 +4161,8 @@ N: csky · 1013 unmodified lines | ||
| 4157 | 4161 | K: csky |
| 4158 | 4162 | |
| 4159 | 4163 | CA8210 IEEE-802.15.4 RADIO DRIVER |
| 4160 | −M: Harry Morris <h.morris@cascoda.com> | |
| 4161 | 4164 | L: linux-wpan@vger.kernel.org |
| 4162 | −S: Maintained | |
| 4165 | +S: Orphan | |
| 4163 | 4166 | W: https://github.com/Cascoda/ca8210-linux.git |
| 4164 | 4167 | F: Documentation/devicetree/bindings/net/ieee802154/ca8210.txt |
| 4165 | 4168 | F: drivers/net/ieee802154/ca8210.c |
| @@ -4544,6 +4547,7 @@ F: drivers/platform/chrome/ · 378 unmodified lines | ||
| 4544 | 4547 | |
| 4545 | 4548 | CHROMEOS EC CODEC DRIVER |
| 4546 | 4549 | M: Cheng-Yi Chiang <cychiang@chromium.org> |
| 4550 | +M: Tzung-Bi Shih <tzungbi@google.com> | |
| 4547 | 4551 | R: Guenter Roeck <groeck@chromium.org> |
| 4548 | 4552 | S: Maintained |
| 4549 | 4553 | F: Documentation/devicetree/bindings/sound/google,cros-ec-codec.yaml |
| @@ -4909,7 +4913,8 @@ F: kernel/cgroup/cpuset.c · 359 unmodified lines | ||
| 4909 | 4913 | CONTROL GROUP - MEMORY RESOURCE CONTROLLER (MEMCG) |
| 4910 | 4914 | M: Johannes Weiner <hannes@cmpxchg.org> |
| 4911 | 4915 | M: Michal Hocko <mhocko@kernel.org> |
| 4912 | −M: Vladimir Davydov <vdavydov.dev@gmail.com> | |
| 4916 | +M: Roman Gushchin <roman.gushchin@linux.dev> | |
| 4917 | +M: Shakeel Butt <shakeelb@google.com> | |
| 4913 | 4918 | L: cgroups@vger.kernel.org |
| 4914 | 4919 | L: linux-mm@kvack.org |
| 4915 | 4920 | S: Maintained |
| @@ -5773,7 +5778,7 @@ F: tools/testing/selftests/dma/ · 857 unmodified lines | ||
| 5773 | 5778 | |
| 5774 | 5779 | DMA-BUF HEAPS FRAMEWORK |
| 5775 | 5780 | M: Sumit Semwal <sumit.semwal@linaro.org> |
| 5776 | −R: Benjamin Gaignard <benjamin.gaignard@linaro.org> | |
| 5781 | +R: Benjamin Gaignard <benjamin.gaignard@collabora.com> | |
| 5777 | 5782 | R: Liam Mark <lmark@codeaurora.org> |
| 5778 | 5783 | R: Laura Abbott <labbott@redhat.com> |
| 5779 | 5784 | R: Brian Starkey <Brian.Starkey@arm.com> |
| @@ -6077,7 +6082,8 @@ L: dri-devel@lists.freedesktop.org · 297 unmodified lines | ||
| 6077 | 6082 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| 6078 | 6083 | S: Maintained |
| 6079 | 6084 | F: drivers/gpu/drm/panel/panel-lvds.c |
| 6080 | −F: Documentation/devicetree/bindings/display/panel/lvds.yaml | |
| 6085 | +F: Documentation/devicetree/bindings/display/lvds.yaml | |
| 6086 | +F: Documentation/devicetree/bindings/display/panel/panel-lvds.yaml | |
| 6081 | 6087 | |
| 6082 | 6088 | DRM DRIVER FOR MANTIX MLAF057WE51 PANELS |
| 6083 | 6089 | M: Guido Günther <agx@sigxcpu.org> |
| @@ -6126,6 +6132,13 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 42 unmodified lines | ||
| 6126 | 6132 | F: Documentation/devicetree/bindings/display/panel/novatek,nt35510.yaml |
| 6127 | 6133 | F: drivers/gpu/drm/panel/panel-novatek-nt35510.c |
| 6128 | 6134 | |
| 6135 | +DRM DRIVER FOR NOVATEK NT35560 PANELS | |
| 6136 | +M: Linus Walleij <linus.walleij@linaro.org> | |
| 6137 | +S: Maintained | |
| 6138 | +T: git git://anongit.freedesktop.org/drm/drm-misc | |
| 6139 | +F: Documentation/devicetree/bindings/display/panel/sony,acx424akp.yaml | |
| 6140 | +F: drivers/gpu/drm/panel/panel-novatek-nt35560.c | |
| 6141 | + | |
| 6129 | 6142 | DRM DRIVER FOR NOVATEK NT36672A PANELS |
| 6130 | 6143 | M: Sumit Semwal <sumit.semwal@linaro.org> |
| 6131 | 6144 | S: Maintained |
| @@ -6162,6 +6175,13 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 30 unmodified lines | ||
| 6162 | 6175 | F: Documentation/devicetree/bindings/display/repaper.txt |
| 6163 | 6176 | F: drivers/gpu/drm/tiny/repaper.c |
| 6164 | 6177 | |
| 6178 | +DRM DRIVER FOR SOLOMON SSD130X OLED DISPLAYS | |
| 6179 | +M: Javier Martinez Canillas <javierm@redhat.com> | |
| 6180 | +S: Maintained | |
| 6181 | +T: git git://anongit.freedesktop.org/drm/drm-misc | |
| 6182 | +F: Documentation/devicetree/bindings/display/solomon,ssd1307fb.yaml | |
| 6183 | +F: drivers/gpu/drm/solomon/ssd130x* | |
| 6184 | + | |
| 6165 | 6185 | DRM DRIVER FOR QEMU'S CIRRUS DEVICE |
| 6166 | 6186 | M: Dave Airlie <airlied@redhat.com> |
| 6167 | 6187 | M: Gerd Hoffmann <kraxel@redhat.com> |
| @@ -6250,12 +6270,6 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 82 unmodified lines | ||
| 6250 | 6270 | F: Documentation/devicetree/bindings/display/sitronix,st7735r.yaml |
| 6251 | 6271 | F: drivers/gpu/drm/tiny/st7735r.c |
| 6252 | 6272 | |
| 6253 | −DRM DRIVER FOR SONY ACX424AKP PANELS | |
| 6254 | −M: Linus Walleij <linus.walleij@linaro.org> | |
| 6255 | −S: Maintained | |
| 6256 | −T: git git://anongit.freedesktop.org/drm/drm-misc | |
| 6257 | −F: drivers/gpu/drm/panel/panel-sony-acx424akp.c | |
| 6258 | − | |
| 6259 | 6273 | DRM DRIVER FOR ST-ERICSSON MCDE |
| 6260 | 6274 | M: Linus Walleij <linus.walleij@linaro.org> |
| 6261 | 6275 | S: Maintained |
| @@ -6503,7 +6517,7 @@ F: Documentation/devicetree/bindings/display/rockchip/ · 241 unmodified lines | ||
| 6503 | 6517 | F: drivers/gpu/drm/rockchip/ |
| 6504 | 6518 | |
| 6505 | 6519 | DRM DRIVERS FOR STI |
| 6506 | −M: Benjamin Gaignard <benjamin.gaignard@linaro.org> | |
| 6520 | +M: Alain Volmat <alain.volmat@foss.st.com> | |
| 6507 | 6521 | L: dri-devel@lists.freedesktop.org |
| 6508 | 6522 | S: Maintained |
| 6509 | 6523 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| @@ -6512,8 +6526,8 @@ F: drivers/gpu/drm/sti · 2 unmodified lines | ||
| 6512 | 6526 | |
| 6513 | 6527 | DRM DRIVERS FOR STM |
| 6514 | 6528 | M: Yannick Fertre <yannick.fertre@foss.st.com> |
| 6529 | +M: Raphael Gallais-Pou <raphael.gallais-pou@foss.st.com> | |
| 6515 | 6530 | M: Philippe Cornu <philippe.cornu@foss.st.com> |
| 6516 | −M: Benjamin Gaignard <benjamin.gaignard@linaro.org> | |
| 6517 | 6531 | L: dri-devel@lists.freedesktop.org |
| 6518 | 6532 | S: Maintained |
| 6519 | 6533 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| @@ -7007,12 +7021,6 @@ L: linux-edac@vger.kernel.org · 487 unmodified lines | ||
| 7007 | 7021 | S: Maintained |
| 7008 | 7022 | F: drivers/edac/sb_edac.c |
| 7009 | 7023 | |
| 7010 | −EDAC-SIFIVE | |
| 7011 | −M: Yash Shah <yash.shah@sifive.com> | |
| 7012 | −L: linux-edac@vger.kernel.org | |
| 7013 | −S: Supported | |
| 7014 | −F: drivers/edac/sifive_edac.c | |
| 7015 | − | |
| 7016 | 7024 | EDAC-SKYLAKE |
| 7017 | 7025 | M: Tony Luck <tony.luck@intel.com> |
| 7018 | 7026 | L: linux-edac@vger.kernel.org |
| @@ -7183,7 +7191,7 @@ F: drivers/net/can/usb/etas_es58x/ · 164 unmodified lines | ||
| 7183 | 7191 | |
| 7184 | 7192 | ETHERNET BRIDGE |
| 7185 | 7193 | M: Roopa Prabhu <roopa@nvidia.com> |
| 7186 | −M: Nikolay Aleksandrov <nikolay@nvidia.com> | |
| 7194 | +M: Nikolay Aleksandrov <razor@blackwall.org> | |
| 7187 | 7195 | L: bridge@lists.linux-foundation.org (moderated for non-subscribers) |
| 7188 | 7196 | L: netdev@vger.kernel.org |
| 7189 | 7197 | S: Maintained |
| @@ -7576,6 +7584,12 @@ S: Maintained · 386 unmodified lines | ||
| 7576 | 7584 | W: http://floatingpoint.sourceforge.net/emulator/index.html |
| 7577 | 7585 | F: arch/x86/math-emu/ |
| 7578 | 7586 | |
| 7587 | +FRAMEBUFFER CORE | |
| 7588 | +M: Daniel Vetter <daniel@ffwll.ch> | |
| 7589 | +F: drivers/video/fbdev/core/ | |
| 7590 | +S: Odd Fixes | |
| 7591 | +T: git git://anongit.freedesktop.org/drm/drm-misc | |
| 7592 | + | |
| 7579 | 7593 | FRAMEBUFFER LAYER |
| 7580 | 7594 | M: Helge Deller <deller@gmx.de> |
| 7581 | 7595 | L: linux-fbdev@vger.kernel.org |
| @@ -9253,6 +9267,15 @@ S: Maintained · 1671 unmodified lines | ||
| 9253 | 9267 | W: https://github.com/o2genum/ideapad-slidebar |
| 9254 | 9268 | F: drivers/input/misc/ideapad_slidebar.c |
| 9255 | 9269 | |
| 9270 | +IDMAPPED MOUNTS | |
| 9271 | +M: Christian Brauner <brauner@kernel.org> | |
| 9272 | +L: linux-fsdevel@vger.kernel.org | |
| 9273 | +S: Maintained | |
| 9274 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux.git | |
| 9275 | +F: Documentation/filesystems/idmappings.rst | |
| 9276 | +F: tools/testing/selftests/mount_setattr/ | |
| 9277 | +F: include/linux/mnt_idmapping.h | |
| 9278 | + | |
| 9256 | 9279 | IDT VersaClock 5 CLOCK DRIVER |
| 9257 | 9280 | M: Luca Ceresoli <luca@lucaceresoli.net> |
| 9258 | 9281 | S: Maintained |
| @@ -10887,6 +10910,12 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git · 1628 unmodified lines | ||
| 10887 | 10910 | F: drivers/ata/pata_arasan_cf.c |
| 10888 | 10911 | F: include/linux/pata_arasan_cf_data.h |
| 10889 | 10912 | |
| 10913 | +LIBATA PATA DRIVERS | |
| 10914 | +R: Sergey Shtylyov <s.shtylyov@omp.ru> | |
| 10915 | +L: linux-ide@vger.kernel.org | |
| 10916 | +F: drivers/ata/ata_*.c | |
| 10917 | +F: drivers/ata/pata_*.c | |
| 10918 | + | |
| 10890 | 10919 | LIBATA PATA FARADAY FTIDE010 AND GEMINI SATA BRIDGE DRIVERS |
| 10891 | 10920 | M: Linus Walleij <linus.walleij@linaro.org> |
| 10892 | 10921 | L: linux-ide@vger.kernel.org |
| @@ -12407,7 +12436,7 @@ F: include/uapi/linux/membarrier.h · 1514 unmodified lines | ||
| 12407 | 12436 | F: kernel/sched/membarrier.c |
| 12408 | 12437 | |
| 12409 | 12438 | MEMBLOCK |
| 12410 | −M: Mike Rapoport <rppt@linux.ibm.com> | |
| 12439 | +M: Mike Rapoport <rppt@kernel.org> | |
| 12411 | 12440 | L: linux-mm@kvack.org |
| 12412 | 12441 | S: Maintained |
| 12413 | 12442 | F: Documentation/core-api/boot-time-mm.rst |
| @@ -13305,8 +13334,8 @@ W: http://www.iptables.org/ · 891 unmodified lines | ||
| 13305 | 13334 | W: http://www.nftables.org/ |
| 13306 | 13335 | Q: http://patchwork.ozlabs.org/project/netfilter-devel/list/ |
| 13307 | 13336 | C: irc://irc.libera.chat/netfilter |
| 13308 | −T: git git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf.git | |
| 13309 | −T: git git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next.git | |
| 13337 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf.git | |
| 13338 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf-next.git | |
| 13310 | 13339 | F: include/linux/netfilter* |
| 13311 | 13340 | F: include/linux/netfilter/ |
| 13312 | 13341 | F: include/net/netfilter/ |
| @@ -13573,7 +13602,7 @@ F: tools/testing/selftests/nci/ · 260 unmodified lines | ||
| 13573 | 13602 | |
| 13574 | 13603 | NFS, SUNRPC, AND LOCKD CLIENTS |
| 13575 | 13604 | M: Trond Myklebust <trond.myklebust@hammerspace.com> |
| 13576 | −M: Anna Schumaker <anna.schumaker@netapp.com> | |
| 13605 | +M: Anna Schumaker <anna@kernel.org> | |
| 13577 | 13606 | L: linux-nfs@vger.kernel.org |
| 13578 | 13607 | S: Maintained |
| 13579 | 13608 | W: http://client.linux-nfs.org |
| @@ -14391,6 +14420,7 @@ M: Rob Herring <robh+dt@kernel.org> · 811 unmodified lines | ||
| 14391 | 14420 | M: Frank Rowand <frowand.list@gmail.com> |
| 14392 | 14421 | L: devicetree@vger.kernel.org |
| 14393 | 14422 | S: Maintained |
| 14423 | +C: irc://irc.libera.chat/devicetree | |
| 14394 | 14424 | W: http://www.devicetree.org/ |
| 14395 | 14425 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git |
| 14396 | 14426 | F: Documentation/ABI/testing/sysfs-firmware-ofw |
| @@ -14402,6 +14432,7 @@ OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS · 5 unmodified lines | ||
| 14402 | 14432 | M: Rob Herring <robh+dt@kernel.org> |
| 14403 | 14433 | L: devicetree@vger.kernel.org |
| 14404 | 14434 | S: Maintained |
| 14435 | +C: irc://irc.libera.chat/devicetree | |
| 14405 | 14436 | Q: http://patchwork.ozlabs.org/project/devicetree-bindings/list/ |
| 14406 | 14437 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git |
| 14407 | 14438 | F: Documentation/devicetree/ |
| @@ -15135,7 +15166,7 @@ M: Ingo Molnar <mingo@redhat.com> · 727 unmodified lines | ||
| 15135 | 15166 | M: Arnaldo Carvalho de Melo <acme@kernel.org> |
| 15136 | 15167 | R: Mark Rutland <mark.rutland@arm.com> |
| 15137 | 15168 | R: Alexander Shishkin <alexander.shishkin@linux.intel.com> |
| 15138 | −R: Jiri Olsa <jolsa@redhat.com> | |
| 15169 | +R: Jiri Olsa <jolsa@kernel.org> | |
| 15139 | 15170 | R: Namhyung Kim <namhyung@kernel.org> |
| 15140 | 15171 | L: linux-perf-users@vger.kernel.org |
| 15141 | 15172 | L: linux-kernel@vger.kernel.org |
| @@ -15292,9 +15323,11 @@ PIN CONTROLLER - SAMSUNG · 150 unmodified lines | ||
| 15292 | 15323 | M: Tomasz Figa <tomasz.figa@gmail.com> |
| 15293 | 15324 | M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com> |
| 15294 | 15325 | M: Sylwester Nawrocki <s.nawrocki@samsung.com> |
| 15326 | +R: Alim Akhtar <alim.akhtar@samsung.com> | |
| 15295 | 15327 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 15296 | 15328 | L: linux-samsung-soc@vger.kernel.org |
| 15297 | 15329 | S: Maintained |
| 15330 | +C: irc://irc.libera.chat/linux-exynos | |
| 15298 | 15331 | Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/ |
| 15299 | 15332 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/samsung.git |
| 15300 | 15333 | F: Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt |
| @@ -15551,6 +15584,7 @@ M: Iurii Zaikin <yzaikin@google.com> · 250 unmodified lines | ||
| 15551 | 15584 | L: linux-kernel@vger.kernel.org |
| 15552 | 15585 | L: linux-fsdevel@vger.kernel.org |
| 15553 | 15586 | S: Maintained |
| 15587 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git sysctl-next | |
| 15554 | 15588 | F: fs/proc/proc_sysctl.c |
| 15555 | 15589 | F: include/linux/sysctl.h |
| 15556 | 15590 | F: kernel/sysctl-test.c |
| @@ -15898,6 +15932,7 @@ S: Supported · 341 unmodified lines | ||
| 15898 | 15932 | W: https://wireless.wiki.kernel.org/en/users/Drivers/ath10k |
| 15899 | 15933 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git |
| 15900 | 15934 | F: drivers/net/wireless/ath/ath10k/ |
| 15935 | +F: Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt | |
| 15901 | 15936 | |
| 15902 | 15937 | QUALCOMM ATHEROS ATH11K WIRELESS DRIVER |
| 15903 | 15938 | M: Kalle Valo <kvalo@kernel.org> |
| @@ -15905,11 +15940,12 @@ L: ath11k@lists.infradead.org · 1 unmodified line | ||
| 15905 | 15940 | S: Supported |
| 15906 | 15941 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git |
| 15907 | 15942 | F: drivers/net/wireless/ath/ath11k/ |
| 15943 | +F: Documentation/devicetree/bindings/net/wireless/qcom,ath11k.txt | |
| 15908 | 15944 | |
| 15909 | 15945 | QUALCOMM ATHEROS ATH9K WIRELESS DRIVER |
| 15910 | −M: ath9k-devel@qca.qualcomm.com | |
| 15946 | +M: Toke Høiland-Jørgensen <toke@toke.dk> | |
| 15911 | 15947 | L: linux-wireless@vger.kernel.org |
| 15912 | −S: Supported | |
| 15948 | +S: Maintained | |
| 15913 | 15949 | W: https://wireless.wiki.kernel.org/en/users/Drivers/ath9k |
| 15914 | 15950 | F: Documentation/devicetree/bindings/net/wireless/qca,ath9k.yaml |
| 15915 | 15951 | F: drivers/net/wireless/ath/ath9k/ |
| @@ -15984,14 +16020,6 @@ F: Documentation/devicetree/bindings/misc/qcom,fastrpc.txt · 68 unmodified lines | ||
| 15984 | 16020 | F: drivers/misc/fastrpc.c |
| 15985 | 16021 | F: include/uapi/misc/fastrpc.h |
| 15986 | 16022 | |
| 15987 | −QUALCOMM GENERIC INTERFACE I2C DRIVER | |
| 15988 | −M: Akash Asthana <akashast@codeaurora.org> | |
| 15989 | −M: Mukesh Savaliya <msavaliy@codeaurora.org> | |
| 15990 | −L: linux-i2c@vger.kernel.org | |
| 15991 | −L: linux-arm-msm@vger.kernel.org | |
| 15992 | −S: Supported | |
| 15993 | −F: drivers/i2c/busses/i2c-qcom-geni.c | |
| 15994 | − | |
| 15995 | 16023 | QUALCOMM HEXAGON ARCHITECTURE |
| 15996 | 16024 | M: Brian Cain <bcain@codeaurora.org> |
| 15997 | 16025 | L: linux-hexagon@vger.kernel.org |
| @@ -16063,8 +16091,8 @@ F: Documentation/devicetree/bindings/mtd/qcom,nandc.yaml · 65 unmodified lines | ||
| 16063 | 16091 | F: drivers/mtd/nand/raw/qcom_nandc.c |
| 16064 | 16092 | |
| 16065 | 16093 | QUALCOMM RMNET DRIVER |
| 16066 | −M: Subash Abhinov Kasiviswanathan <subashab@codeaurora.org> | |
| 16067 | −M: Sean Tranchetti <stranche@codeaurora.org> | |
| 16094 | +M: Subash Abhinov Kasiviswanathan <quic_subashab@quicinc.com> | |
| 16095 | +M: Sean Tranchetti <quic_stranche@quicinc.com> | |
| 16068 | 16096 | L: netdev@vger.kernel.org |
| 16069 | 16097 | S: Maintained |
| 16070 | 16098 | F: Documentation/networking/device_drivers/cellular/qualcomm/rmnet.rst |
| @@ -16090,11 +16118,10 @@ F: Documentation/devicetree/bindings/media/*venus* · 19 unmodified lines | ||
| 16090 | 16118 | F: drivers/media/platform/qcom/venus/ |
| 16091 | 16119 | |
| 16092 | 16120 | QUALCOMM WCN36XX WIRELESS DRIVER |
| 16093 | −M: Kalle Valo <kvalo@kernel.org> | |
| 16121 | +M: Loic Poulain <loic.poulain@linaro.org> | |
| 16094 | 16122 | L: wcn36xx@lists.infradead.org |
| 16095 | 16123 | S: Supported |
| 16096 | 16124 | W: https://wireless.wiki.kernel.org/en/users/Drivers/wcn36xx |
| 16097 | −T: git git://github.com/KrasnikovEugene/wcn36xx.git | |
| 16098 | 16125 | F: drivers/net/wireless/ath/wcn36xx/ |
| 16099 | 16126 | |
| 16100 | 16127 | QUANTENNA QTNFMAC WIRELESS DRIVER |
| @@ -16357,6 +16384,7 @@ F: drivers/watchdog/realtek_otto_wdt.c · 256 unmodified lines | ||
| 16357 | 16384 | |
| 16358 | 16385 | REALTEK RTL83xx SMI DSA ROUTER CHIPS |
| 16359 | 16386 | M: Linus Walleij <linus.walleij@linaro.org> |
| 16387 | +M: Alvin Šipraga <alsi@bang-olufsen.dk> | |
| 16360 | 16388 | S: Maintained |
| 16361 | 16389 | F: Documentation/devicetree/bindings/net/dsa/realtek-smi.txt |
| 16362 | 16390 | F: drivers/net/dsa/realtek-smi* |
| @@ -16476,6 +16504,14 @@ F: Documentation/devicetree/bindings/i2c/renesas,rmobile-iic.yaml · 113 unmodified lines | ||
| 16476 | 16504 | F: drivers/i2c/busses/i2c-rcar.c |
| 16477 | 16505 | F: drivers/i2c/busses/i2c-sh_mobile.c |
| 16478 | 16506 | |
| 16507 | +RENESAS R-CAR SATA DRIVER | |
| 16508 | +R: Sergey Shtylyov <s.shtylyov@omp.ru> | |
| 16509 | +S: Supported | |
| 16510 | +L: linux-ide@vger.kernel.org | |
| 16511 | +L: linux-renesas-soc@vger.kernel.org | |
| 16512 | +F: Documentation/devicetree/bindings/ata/renesas,rcar-sata.yaml | |
| 16513 | +F: drivers/ata/sata_rcar.c | |
| 16514 | + | |
| 16479 | 16515 | RENESAS R-CAR THERMAL DRIVERS |
| 16480 | 16516 | M: Niklas Söderlund <niklas.soderlund@ragnatech.se> |
| 16481 | 16517 | L: linux-renesas-soc@vger.kernel.org |
| @@ -16811,8 +16847,8 @@ F: drivers/video/fbdev/savage/ · 329 unmodified lines | ||
| 16811 | 16847 | S390 |
| 16812 | 16848 | M: Heiko Carstens <hca@linux.ibm.com> |
| 16813 | 16849 | M: Vasily Gorbik <gor@linux.ibm.com> |
| 16814 | −M: Christian Borntraeger <borntraeger@linux.ibm.com> | |
| 16815 | −R: Alexander Gordeev <agordeev@linux.ibm.com> | |
| 16850 | +M: Alexander Gordeev <agordeev@linux.ibm.com> | |
| 16851 | +R: Christian Borntraeger <borntraeger@linux.ibm.com> | |
| 16816 | 16852 | R: Sven Schnelle <svens@linux.ibm.com> |
| 16817 | 16853 | L: linux-s390@vger.kernel.org |
| 16818 | 16854 | S: Supported |
| @@ -17083,6 +17119,7 @@ SAMSUNG SOC CLOCK DRIVERS · 264 unmodified lines | ||
| 17083 | 17119 | M: Sylwester Nawrocki <s.nawrocki@samsung.com> |
| 17084 | 17120 | M: Tomasz Figa <tomasz.figa@gmail.com> |
| 17085 | 17121 | M: Chanwoo Choi <cw00.choi@samsung.com> |
| 17122 | +R: Alim Akhtar <alim.akhtar@samsung.com> | |
| 17086 | 17123 | L: linux-samsung-soc@vger.kernel.org |
| 17087 | 17124 | S: Supported |
| 17088 | 17125 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/snawrocki/clk.git |
| @@ -17719,6 +17756,21 @@ S: Maintained · 630 unmodified lines | ||
| 17719 | 17756 | W: http://www.winischhofer.at/linuxsisusbvga.shtml |
| 17720 | 17757 | F: drivers/usb/misc/sisusbvga/ |
| 17721 | 17758 | |
| 17759 | +SL28 CPLD MFD DRIVER | |
| 17760 | +M: Michael Walle <michael@walle.cc> | |
| 17761 | +S: Maintained | |
| 17762 | +F: Documentation/devicetree/bindings/gpio/kontron,sl28cpld-gpio.yaml | |
| 17763 | +F: Documentation/devicetree/bindings/hwmon/kontron,sl28cpld-hwmon.yaml | |
| 17764 | +F: Documentation/devicetree/bindings/interrupt-controller/kontron,sl28cpld-intc.yaml | |
| 17765 | +F: Documentation/devicetree/bindings/mfd/kontron,sl28cpld.yaml | |
| 17766 | +F: Documentation/devicetree/bindings/pwm/kontron,sl28cpld-pwm.yaml | |
| 17767 | +F: Documentation/devicetree/bindings/watchdog/kontron,sl28cpld-wdt.yaml | |
| 17768 | +F: drivers/gpio/gpio-sl28cpld.c | |
| 17769 | +F: drivers/hwmon/sl28cpld-hwmon.c | |
| 17770 | +F: drivers/irqchip/irq-sl28cpld.c | |
| 17771 | +F: drivers/pwm/pwm-sl28cpld.c | |
| 17772 | +F: drivers/watchdog/sl28cpld_wdt.c | |
| 17773 | + | |
| 17722 | 17774 | SLAB ALLOCATOR |
| 17723 | 17775 | M: Christoph Lameter <cl@linux.com> |
| 17724 | 17776 | M: Pekka Enberg <penberg@kernel.org> |
| @@ -17726,8 +17778,10 @@ M: David Rientjes <rientjes@google.com> · 1 unmodified line | ||
| 17726 | 17778 | M: Joonsoo Kim <iamjoonsoo.kim@lge.com> |
| 17727 | 17779 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17728 | 17780 | M: Vlastimil Babka <vbabka@suse.cz> |
| 17781 | +R: Roman Gushchin <roman.gushchin@linux.dev> | |
| 17729 | 17782 | L: linux-mm@kvack.org |
| 17730 | 17783 | S: Maintained |
| 17784 | +T: git git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git | |
| 17731 | 17785 | F: include/linux/sl?b*.h |
| 17732 | 17786 | F: mm/sl?b* |
| 17733 | 17787 | |
| @@ -18435,7 +18489,7 @@ F: Documentation/devicetree/bindings/sound/st,sti-asoc-card.txt · 701 unmodified lines | ||
| 18435 | 18489 | F: sound/soc/sti/ |
| 18436 | 18490 | |
| 18437 | 18491 | STI CEC DRIVER |
| 18438 | −M: Benjamin Gaignard <benjamin.gaignard@linaro.org> | |
| 18492 | +M: Alain Volmat <alain.volmat@foss.st.com> | |
| 18439 | 18493 | S: Maintained |
| 18440 | 18494 | F: Documentation/devicetree/bindings/media/stih-cec.txt |
| 18441 | 18495 | F: drivers/media/cec/platform/sti/ |
| @@ -19589,6 +19643,14 @@ F: Documentation/trace/timerlat-tracer.rst · 1147 unmodified lines | ||
| 19589 | 19643 | F: Documentation/trace/hwlat_detector.rst |
| 19590 | 19644 | F: arch/*/kernel/trace.c |
| 19591 | 19645 | |
| 19646 | +Real-time Linux Analysis (RTLA) tools | |
| 19647 | +M: Daniel Bristot de Oliveira <bristot@kernel.org> | |
| 19648 | +M: Steven Rostedt <rostedt@goodmis.org> | |
| 19649 | +L: linux-trace-devel@vger.kernel.org | |
| 19650 | +S: Maintained | |
| 19651 | +F: Documentation/tools/rtla/ | |
| 19652 | +F: tools/tracing/rtla/ | |
| 19653 | + | |
| 19592 | 19654 | TRADITIONAL CHINESE DOCUMENTATION |
| 19593 | 19655 | M: Hu Haowen <src.res@email.cn> |
| 19594 | 19656 | L: linux-doc-tw-discuss@lists.sourceforge.net |
| @@ -2,8 +2,8 @@ · 1 unmodified line | ||
| 2 | 2 | VERSION = 5 |
| 3 | 3 | PATCHLEVEL = 17 |
| 4 | 4 | SUBLEVEL = 0 |
| 5 | −EXTRAVERSION = -rc2 | |
| 6 | −NAME = Gobble Gobble | |
| 5 | +EXTRAVERSION = -rc6 | |
| 6 | +NAME = Superb Owl | |
| 7 | 7 | |
| 8 | 8 | # *DOCUMENTATION* |
| 9 | 9 | # To see a list of typical targets execute "make help" |
| @@ -806,6 +806,7 @@ dtb-$(CONFIG_ARCH_OMAP3) += \ · 805 unmodified lines | ||
| 806 | 806 | logicpd-som-lv-37xx-devkit.dtb \ |
| 807 | 807 | omap3430-sdp.dtb \ |
| 808 | 808 | omap3-beagle.dtb \ |
| 809 | + omap3-beagle-ab4.dtb \ | |
| 809 | 810 | omap3-beagle-xm.dtb \ |
| 810 | 811 | omap3-beagle-xm-ab.dtb \ |
| 811 | 812 | omap3-cm-t3517.dtb \ |
| @@ -55,7 +55,7 @@ · 54 unmodified lines | ||
| 55 | 55 | 2 1 0 0 /* # 0: INACTIVE, 1: TX, 2: RX */ |
| 56 | 56 | >; |
| 57 | 57 | tx-num-evt = <16>; |
| 58 | − rt-num-evt = <16>; | |
| 58 | + rx-num-evt = <16>; | |
| 59 | 59 | status = "okay"; |
| 60 | 60 | }; |
| 61 | 61 | |
| @@ -160,7 +160,7 @@ · 159 unmodified lines | ||
| 160 | 160 | target-module@48210000 { |
| 161 | 161 | compatible = "ti,sysc-omap4-simple", "ti,sysc"; |
| 162 | 162 | power-domains = <&prm_mpu>; |
| 163 | − clocks = <&mpu_clkctrl DRA7_MPU_CLKCTRL 0>; | |
| 163 | + clocks = <&mpu_clkctrl DRA7_MPU_MPU_CLKCTRL 0>; | |
| 164 | 164 | clock-names = "fck"; |
| 165 | 165 | #address-cells = <1>; |
| 166 | 166 | #size-cells = <1>; |
| @@ -875,10 +875,10 @@ · 708 unmodified lines | ||
| 875 | 875 | <0x58000014 4>; |
| 876 | 876 | reg-names = "rev", "syss"; |
| 877 | 877 | ti,syss-mask = <1>; |
| 878 | − clocks = <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 0>, | |
| 879 | − <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 9>, | |
| 880 | − <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 10>, | |
| 881 | − <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 11>; | |
| 878 | + clocks = <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 0>, | |
| 879 | + <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 9>, | |
| 880 | + <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 10>, | |
| 881 | + <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 11>; | |
| 882 | 882 | clock-names = "fck", "hdmi_clk", "sys_clk", "tv_clk"; |
| 883 | 883 | #address-cells = <1>; |
| 884 | 884 | #size-cells = <1>; |
| @@ -912,7 +912,7 @@ · 27 unmodified lines | ||
| 912 | 912 | SYSC_OMAP2_SOFTRESET | |
| 913 | 913 | SYSC_OMAP2_AUTOIDLE)>; |
| 914 | 914 | ti,syss-mask = <1>; |
| 915 | − clocks = <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 8>; | |
| 915 | + clocks = <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 8>; | |
| 916 | 916 | clock-names = "fck"; |
| 917 | 917 | #address-cells = <1>; |
| 918 | 918 | #size-cells = <1>; |
| @@ -939,8 +939,8 @@ · 20 unmodified lines | ||
| 939 | 939 | <SYSC_IDLE_SMART>, |
| 940 | 940 | <SYSC_IDLE_SMART_WKUP>; |
| 941 | 941 | ti,sysc-mask = <(SYSC_OMAP4_SOFTRESET)>; |
| 942 | − clocks = <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 9>, | |
| 943 | − <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 8>; | |
| 942 | + clocks = <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 9>, | |
| 943 | + <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 8>; | |
| 944 | 944 | clock-names = "fck", "dss_clk"; |
| 945 | 945 | #address-cells = <1>; |
| 946 | 946 | #size-cells = <1>; |
| @@ -979,7 +979,7 @@ · 32 unmodified lines | ||
| 979 | 979 | compatible = "vivante,gc"; |
| 980 | 980 | reg = <0x0 0x700>; |
| 981 | 981 | interrupts = <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>; |
| 982 | − clocks = <&dss_clkctrl DRA7_BB2D_CLKCTRL 0>; | |
| 982 | + clocks = <&dss_clkctrl DRA7_DSS_BB2D_CLKCTRL 0>; | |
| 983 | 983 | clock-names = "core"; |
| 984 | 984 | }; |
| 985 | 985 | }; |
| @@ -1333,7 +1333,7 @@ · 347 unmodified lines | ||
| 1333 | 1333 | ti,no-reset-on-init; |
| 1334 | 1334 | ti,no-idle; |
| 1335 | 1335 | timer@0 { |
| 1336 | − assigned-clocks = <&wkupaon_clkctrl DRA7_TIMER1_CLKCTRL 24>; | |
| 1336 | + assigned-clocks = <&wkupaon_clkctrl DRA7_WKUPAON_TIMER1_CLKCTRL 24>; | |
| 1337 | 1337 | assigned-clock-parents = <&sys_32k_ck>; |
| 1338 | 1338 | }; |
| 1339 | 1339 | }; |
| @@ -79,7 +79,6 @@ · 78 unmodified lines | ||
| 79 | 79 | MX23_PAD_LCD_RESET__GPIO_1_18 |
| 80 | 80 | MX23_PAD_PWM3__GPIO_1_29 |
| 81 | 81 | MX23_PAD_PWM4__GPIO_1_30 |
| 82 | − MX23_PAD_SSP1_DETECT__SSP1_DETECT | |
| 83 | 82 | >; |
| 84 | 83 | fsl,drive-strength = <MXS_DRIVE_4mA>; |
| 85 | 84 | fsl,voltage = <MXS_VOLTAGE_HIGH>; |
| @@ -5,6 +5,8 @@ · 4 unmodified lines | ||
| 5 | 5 | * Author: Fabio Estevam <fabio.estevam@freescale.com> |
| 6 | 6 | */ |
| 7 | 7 | |
| 8 | +#include <dt-bindings/gpio/gpio.h> | |
| 9 | + | |
| 8 | 10 | / { |
| 9 | 11 | aliases { |
| 10 | 12 | backlight = &backlight; |
| @@ -226,6 +228,7 @@ · 215 unmodified lines | ||
| 226 | 228 | MX6QDL_PAD_SD3_DAT1__SD3_DATA1 0x17059 |
| 227 | 229 | MX6QDL_PAD_SD3_DAT2__SD3_DATA2 0x17059 |
| 228 | 230 | MX6QDL_PAD_SD3_DAT3__SD3_DATA3 0x17059 |
| 231 | + MX6QDL_PAD_SD3_DAT5__GPIO7_IO00 0x1b0b0 | |
| 229 | 232 | >; |
| 230 | 233 | }; |
| 231 | 234 | |
| @@ -304,7 +307,7 @@ · 72 unmodified lines | ||
| 304 | 307 | &usdhc3 { |
| 305 | 308 | pinctrl-names = "default"; |
| 306 | 309 | pinctrl-0 = <&pinctrl_usdhc3>; |
| 307 | − non-removable; | |
| 310 | + cd-gpios = <&gpio7 0 GPIO_ACTIVE_LOW>; | |
| 308 | 311 | status = "okay"; |
| 309 | 312 | }; |
| 310 | 313 | |
| @@ -259,7 +259,7 @@ · 258 unmodified lines | ||
| 259 | 259 | interrupts = <GIC_SPI 55 IRQ_TYPE_LEVEL_HIGH>; |
| 260 | 260 | clocks = <&pcc2 IMX7ULP_CLK_WDG1>; |
| 261 | 261 | assigned-clocks = <&pcc2 IMX7ULP_CLK_WDG1>; |
| 262 | − assigned-clocks-parents = <&scg1 IMX7ULP_CLK_FIRC_BUS_CLK>; | |
| 262 | + assigned-clock-parents = <&scg1 IMX7ULP_CLK_FIRC_BUS_CLK>; | |
| 263 | 263 | timeout-sec = <40>; |
| 264 | 264 | }; |
| 265 | 265 | |
| @@ -59,7 +59,7 @@ · 58 unmodified lines | ||
| 59 | 59 | }; |
| 60 | 60 | |
| 61 | 61 | uart_A: serial@84c0 { |
| 62 | − compatible = "amlogic,meson6-uart", "amlogic,meson-uart"; | |
| 62 | + compatible = "amlogic,meson6-uart"; | |
| 63 | 63 | reg = <0x84c0 0x18>; |
| 64 | 64 | interrupts = <GIC_SPI 26 IRQ_TYPE_EDGE_RISING>; |
| 65 | 65 | fifo-size = <128>; |
| @@ -67,7 +67,7 @@ · 1 unmodified line | ||
| 67 | 67 | }; |
| 68 | 68 | |
| 69 | 69 | uart_B: serial@84dc { |
| 70 | − compatible = "amlogic,meson6-uart", "amlogic,meson-uart"; | |
| 70 | + compatible = "amlogic,meson6-uart"; | |
| 71 | 71 | reg = <0x84dc 0x18>; |
| 72 | 72 | interrupts = <GIC_SPI 75 IRQ_TYPE_EDGE_RISING>; |
| 73 | 73 | status = "disabled"; |
| @@ -105,7 +105,7 @@ · 31 unmodified lines | ||
| 105 | 105 | }; |
| 106 | 106 | |
| 107 | 107 | uart_C: serial@8700 { |
| 108 | − compatible = "amlogic,meson6-uart", "amlogic,meson-uart"; | |
| 108 | + compatible = "amlogic,meson6-uart"; | |
| 109 | 109 | reg = <0x8700 0x18>; |
| 110 | 110 | interrupts = <GIC_SPI 93 IRQ_TYPE_EDGE_RISING>; |
| 111 | 111 | status = "disabled"; |
| @@ -228,7 +228,7 @@ · 116 unmodified lines | ||
| 228 | 228 | }; |
| 229 | 229 | |
| 230 | 230 | uart_AO: serial@4c0 { |
| 231 | − compatible = "amlogic,meson6-uart", "amlogic,meson-ao-uart", "amlogic,meson-uart"; | |
| 231 | + compatible = "amlogic,meson6-uart", "amlogic,meson-ao-uart"; | |
| 232 | 232 | reg = <0x4c0 0x18>; |
| 233 | 233 | interrupts = <GIC_SPI 90 IRQ_TYPE_EDGE_RISING>; |
| 234 | 234 | status = "disabled"; |
| @@ -736,27 +736,27 @@ · 735 unmodified lines | ||
| 736 | 736 | }; |
| 737 | 737 | |
| 738 | 738 | &uart_AO { |
| 739 | − compatible = "amlogic,meson8-uart", "amlogic,meson-uart"; | |
| 740 | − clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_CLK81>; | |
| 741 | − clock-names = "baud", "xtal", "pclk"; | |
| 739 | + compatible = "amlogic,meson8-uart", "amlogic,meson-ao-uart"; | |
| 740 | + clocks = <&xtal>, <&clkc CLKID_CLK81>, <&clkc CLKID_CLK81>; | |
| 741 | + clock-names = "xtal", "pclk", "baud"; | |
| 742 | 742 | }; |
| 743 | 743 | |
| 744 | 744 | &uart_A { |
| 745 | − compatible = "amlogic,meson8-uart", "amlogic,meson-uart"; | |
| 746 | − clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART0>; | |
| 747 | − clock-names = "baud", "xtal", "pclk"; | |
| 745 | + compatible = "amlogic,meson8-uart"; | |
| 746 | + clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; | |
| 747 | + clock-names = "xtal", "pclk", "baud"; | |
| 748 | 748 | }; |
| 749 | 749 | |
| 750 | 750 | &uart_B { |
| 751 | − compatible = "amlogic,meson8-uart", "amlogic,meson-uart"; | |
| 752 | − clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART1>; | |
| 753 | − clock-names = "baud", "xtal", "pclk"; | |
| 751 | + compatible = "amlogic,meson8-uart"; | |
| 752 | + clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; | |
| 753 | + clock-names = "xtal", "pclk", "baud"; | |
| 754 | 754 | }; |
| 755 | 755 | |
| 756 | 756 | &uart_C { |
| 757 | − compatible = "amlogic,meson8-uart", "amlogic,meson-uart"; | |
| 758 | − clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART2>; | |
| 759 | − clock-names = "baud", "xtal", "pclk"; | |
| 757 | + compatible = "amlogic,meson8-uart"; | |
| 758 | + clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; | |
| 759 | + clock-names = "xtal", "pclk", "baud"; | |
| 760 | 760 | }; |
| 761 | 761 | |
| 762 | 762 | &usb0 { |
| @@ -724,27 +724,27 @@ · 723 unmodified lines | ||
| 724 | 724 | }; |
| 725 | 725 | |
| 726 | 726 | &uart_AO { |
| 727 | − compatible = "amlogic,meson8b-uart", "amlogic,meson-uart"; | |
| 728 | − clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_CLK81>; | |
| 729 | − clock-names = "baud", "xtal", "pclk"; | |
| 727 | + compatible = "amlogic,meson8b-uart", "amlogic,meson-ao-uart"; | |
| 728 | + clocks = <&xtal>, <&clkc CLKID_CLK81>, <&clkc CLKID_CLK81>; | |
| 729 | + clock-names = "xtal", "pclk", "baud"; | |
| 730 | 730 | }; |
| 731 | 731 | |
| 732 | 732 | &uart_A { |
| 733 | − compatible = "amlogic,meson8b-uart", "amlogic,meson-uart"; | |
| 734 | − clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART0>; | |
| 735 | − clock-names = "baud", "xtal", "pclk"; | |
| 733 | + compatible = "amlogic,meson8b-uart"; | |
| 734 | + clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; | |
| 735 | + clock-names = "xtal", "pclk", "baud"; | |
| 736 | 736 | }; |
| 737 | 737 | |
| 738 | 738 | &uart_B { |
| 739 | − compatible = "amlogic,meson8b-uart", "amlogic,meson-uart"; | |
| 740 | − clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART1>; | |
| 741 | − clock-names = "baud", "xtal", "pclk"; | |
| 739 | + compatible = "amlogic,meson8b-uart"; | |
| 740 | + clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; | |
| 741 | + clock-names = "xtal", "pclk", "baud"; | |
| 742 | 742 | }; |
| 743 | 743 | |
| 744 | 744 | &uart_C { |
| 745 | − compatible = "amlogic,meson8b-uart", "amlogic,meson-uart"; | |
| 746 | − clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART2>; | |
| 747 | − clock-names = "baud", "xtal", "pclk"; | |
| 745 | + compatible = "amlogic,meson8b-uart"; | |
| 746 | + clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; | |
| 747 | + clock-names = "xtal", "pclk", "baud"; | |
| 748 | 748 | }; |
| 749 | 749 | |
| 750 | 750 | &usb0 { |
| @@ -0,0 +1,47 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0-only | |
| 2 | +/dts-v1/; | |
| 3 | + | |
| 4 | +#include "omap3-beagle.dts" | |
| 5 | + | |
| 6 | +/ { | |
| 7 | + model = "TI OMAP3 BeagleBoard A to B4"; | |
| 8 | + compatible = "ti,omap3-beagle-ab4", "ti,omap3-beagle", "ti,omap3430", "ti,omap3"; | |
| 9 | +}; | |
| 10 | + | |
| 11 | +/* | |
| 12 | + * Workaround for capacitor C70 issue, see "Boards revision A and < B5" | |
| 13 | + * section at https://elinux.org/BeagleBoard_Community | |
| 14 | + */ | |
| 15 | + | |
| 16 | +/* Unusable as clocksource because of unreliable oscillator */ | |
| 17 | +&counter32k { | |
| 18 | + status = "disabled"; | |
| 19 | +}; | |
| 20 | + | |
| 21 | +/* Unusable as clockevent because of unreliable oscillator, allow to idle */ | |
| 22 | +&timer1_target { | |
| 23 | + /delete-property/ti,no-reset-on-init; | |
| 24 | + /delete-property/ti,no-idle; | |
| 25 | + timer@0 { | |
| 26 | + /delete-property/ti,timer-alwon; | |
| 27 | + }; | |
| 28 | +}; | |
| 29 | + | |
| 30 | +/* Preferred always-on timer for clocksource */ | |
| 31 | +&timer12_target { | |
| 32 | + ti,no-reset-on-init; | |
| 33 | + ti,no-idle; | |
| 34 | + timer@0 { | |
| 35 | + /* Always clocked by secure_32k_fck */ | |
| 36 | + }; | |
| 37 | +}; | |
| 38 | + | |
| 39 | +/* Preferred timer for clockevent */ | |
| 40 | +&timer2_target { | |
| 41 | + ti,no-reset-on-init; | |
| 42 | + ti,no-idle; | |
| 43 | + timer@0 { | |
| 44 | + assigned-clocks = <&gpt2_fck>; | |
| 45 | + assigned-clock-parents = <&sys_ck>; | |
| 46 | + }; | |
| 47 | +}; | |
| @@ -304,39 +304,6 @@ · 303 unmodified lines | ||
| 304 | 304 | phys = <0 &hsusb2_phy>; |
| 305 | 305 | }; |
| 306 | 306 | |
| 307 | −/* Unusable as clocksource because of unreliable oscillator */ | |
| 308 | −&counter32k { | |
| 309 | − status = "disabled"; | |
| 310 | −}; | |
| 311 | − | |
| 312 | −/* Unusable as clockevent because if unreliable oscillator, allow to idle */ | |
| 313 | −&timer1_target { | |
| 314 | − /delete-property/ti,no-reset-on-init; | |
| 315 | − /delete-property/ti,no-idle; | |
| 316 | − timer@0 { | |
| 317 | − /delete-property/ti,timer-alwon; | |
| 318 | − }; | |
| 319 | −}; | |
| 320 | − | |
| 321 | −/* Preferred always-on timer for clocksource */ | |
| 322 | −&timer12_target { | |
| 323 | − ti,no-reset-on-init; | |
| 324 | − ti,no-idle; | |
| 325 | − timer@0 { | |
| 326 | − /* Always clocked by secure_32k_fck */ | |
| 327 | − }; | |
| 328 | −}; | |
| 329 | − | |
| 330 | −/* Preferred timer for clockevent */ | |
| 331 | −&timer2_target { | |
| 332 | − ti,no-reset-on-init; | |
| 333 | − ti,no-idle; | |
| 334 | − timer@0 { | |
| 335 | − assigned-clocks = <&gpt2_fck>; | |
| 336 | − assigned-clock-parents = <&sys_ck>; | |
| 337 | − }; | |
| 338 | −}; | |
| 339 | − | |
| 340 | 307 | &twl_gpio { |
| 341 | 308 | ti,use-leds; |
| 342 | 309 | /* pullups: BIT(1) */ |
| @@ -235,7 +235,6 @@ · 234 unmodified lines | ||
| 235 | 235 | #address-cells = <1>; |
| 236 | 236 | #size-cells = <0>; |
| 237 | 237 | reg = <0x41>; |
| 238 | − irq-over-gpio; | |
| 239 | 238 | irq-gpios = <&gpiopinctrl 29 0x4>; |
| 240 | 239 | id = <0>; |
| 241 | 240 | blocks = <0x5>; |
| @@ -185,10 +185,6 @@ · 184 unmodified lines | ||
| 185 | 185 | cap-sd-highspeed; |
| 186 | 186 | cap-mmc-highspeed; |
| 187 | 187 | /* All direction control is used */ |
| 188 | − st,sig-dir-cmd; | |
| 189 | − st,sig-dir-dat0; | |
| 190 | − st,sig-dir-dat2; | |
| 191 | − st,sig-dir-dat31; | |
| 192 | 188 | st,sig-pin-fbclk; |
| 193 | 189 | full-pwr-cycle; |
| 194 | 190 | vmmc-supply = <&ab8500_ldo_aux3_reg>; |
| @@ -13,12 +13,12 @@ · 12 unmodified lines | ||
| 13 | 13 | static int crypto_blake2s_update_arm(struct shash_desc *desc, |
| 14 | 14 | const u8 *in, unsigned int inlen) |
| 15 | 15 | { |
| 16 | − return crypto_blake2s_update(desc, in, inlen, blake2s_compress); | |
| 16 | + return crypto_blake2s_update(desc, in, inlen, false); | |
| 17 | 17 | } |
| 18 | 18 | |
| 19 | 19 | static int crypto_blake2s_final_arm(struct shash_desc *desc, u8 *out) |
| 20 | 20 | { |
| 21 | − return crypto_blake2s_final(desc, out, blake2s_compress); | |
| 21 | + return crypto_blake2s_final(desc, out, false); | |
| 22 | 22 | } |
| 23 | 23 | |
| 24 | 24 | #define BLAKE2S_ALG(name, driver_name, digest_size) \ |
| @@ -263,9 +263,9 @@ static int __init omapdss_init_of(void) · 262 unmodified lines | ||
| 263 | 263 | } |
| 264 | 264 | |
| 265 | 265 | r = of_platform_populate(node, NULL, NULL, &pdev->dev); |
| 266 | + put_device(&pdev->dev); | |
| 266 | 267 | if (r) { |
| 267 | 268 | pr_err("Unable to populate DSS submodule devices\n"); |
| 268 | − put_device(&pdev->dev); | |
| 269 | 269 | return r; |
| 270 | 270 | } |
| 271 | 271 | |
| @@ -752,8 +752,10 @@ static int __init _init_clkctrl_providers(void) · 751 unmodified lines | ||
| 752 | 752 | |
| 753 | 753 | for_each_matching_node(np, ti_clkctrl_match_table) { |
| 754 | 754 | ret = _setup_clkctrl_provider(np); |
| 755 | − if (ret) | |
| 755 | + if (ret) { | |
| 756 | + of_node_put(np); | |
| 756 | 757 | break; |
| 758 | + } | |
| 757 | 759 | } |
| 758 | 760 | |
| 759 | 761 | return ret; |
| @@ -2,6 +2,7 @@ · 1 unmodified line | ||
| 2 | 2 | menuconfig ARCH_INTEL_SOCFPGA |
| 3 | 3 | bool "Altera SOCFPGA family" |
| 4 | 4 | depends on ARCH_MULTI_V7 |
| 5 | + select ARCH_HAS_RESET_CONTROLLER | |
| 5 | 6 | select ARCH_SUPPORTS_BIG_ENDIAN |
| 6 | 7 | select ARM_AMBA |
| 7 | 8 | select ARM_GIC |
| @@ -18,6 +19,7 @@ menuconfig ARCH_INTEL_SOCFPGA · 10 unmodified lines | ||
| 18 | 19 | select PL310_ERRATA_727915 |
| 19 | 20 | select PL310_ERRATA_753970 if PL310 |
| 20 | 21 | select PL310_ERRATA_769419 |
| 22 | + select RESET_CONTROLLER | |
| 21 | 23 | |
| 22 | 24 | if ARCH_INTEL_SOCFPGA |
| 23 | 25 | config SOCFPGA_SUSPEND |
| @@ -672,6 +672,7 @@ config ARM64_WORKAROUND_TRBE_OVERWRITE_FILL_MODE · 671 unmodified lines | ||
| 672 | 672 | |
| 673 | 673 | config ARM64_ERRATUM_2051678 |
| 674 | 674 | bool "Cortex-A510: 2051678: disable Hardware Update of the page table dirty bit" |
| 675 | + default y | |
| 675 | 676 | help |
| 676 | 677 | This options adds the workaround for ARM Cortex-A510 erratum ARM64_ERRATUM_2051678. |
| 677 | 678 | Affected Coretex-A510 might not respect the ordering rules for |
| @@ -680,6 +681,22 @@ config ARM64_ERRATUM_2051678 · 2 unmodified lines | ||
| 680 | 681 | |
| 681 | 682 | If unsure, say Y. |
| 682 | 683 | |
| 684 | +config ARM64_ERRATUM_2077057 | |
| 685 | + bool "Cortex-A510: 2077057: workaround software-step corrupting SPSR_EL2" | |
| 686 | + help | |
| 687 | + This option adds the workaround for ARM Cortex-A510 erratum 2077057. | |
| 688 | + Affected Cortex-A510 may corrupt SPSR_EL2 when the a step exception is | |
| 689 | + expected, but a Pointer Authentication trap is taken instead. The | |
| 690 | + erratum causes SPSR_EL1 to be copied to SPSR_EL2, which could allow | |
| 691 | + EL1 to cause a return to EL2 with a guest controlled ELR_EL2. | |
| 692 | + | |
| 693 | + This can only happen when EL2 is stepping EL1. | |
| 694 | + | |
| 695 | + When these conditions occur, the SPSR_EL2 value is unchanged from the | |
| 696 | + previous guest entry, and can be restored from the in-memory copy. | |
| 697 | + | |
| 698 | + If unsure, say Y. | |
| 699 | + | |
| 683 | 700 | config ARM64_ERRATUM_2119858 |
| 684 | 701 | bool "Cortex-A710/X2: 2119858: workaround TRBE overwriting trace data in FILL mode" |
| 685 | 702 | default y |
| @@ -309,9 +309,6 @@ config ARCH_VISCONTI · 308 unmodified lines | ||
| 309 | 309 | help |
| 310 | 310 | This enables support for Toshiba Visconti SoCs Family. |
| 311 | 311 | |
| 312 | −config ARCH_VULCAN | |
| 313 | − def_bool n | |
| 314 | − | |
| 315 | 312 | config ARCH_XGENE |
| 316 | 313 | bool "AppliedMicro X-Gene SOC Family" |
| 317 | 314 | help |
| @@ -107,6 +107,12 @@ · 106 unmodified lines | ||
| 107 | 107 | no-map; |
| 108 | 108 | }; |
| 109 | 109 | |
| 110 | + /* 32 MiB reserved for ARM Trusted Firmware (BL32) */ | |
| 111 | + secmon_reserved_bl32: secmon@5300000 { | |
| 112 | + reg = <0x0 0x05300000 0x0 0x2000000>; | |
| 113 | + no-map; | |
| 114 | + }; | |
| 115 | + | |
| 110 | 116 | linux,cma { |
| 111 | 117 | compatible = "shared-dma-pool"; |
| 112 | 118 | reusable; |
| @@ -157,14 +157,6 @@ · 156 unmodified lines | ||
| 157 | 157 | regulator-always-on; |
| 158 | 158 | }; |
| 159 | 159 | |
| 160 | − reserved-memory { | |
| 161 | − /* TEE Reserved Memory */ | |
| 162 | − bl32_reserved: bl32@5000000 { | |
| 163 | − reg = <0x0 0x05300000 0x0 0x2000000>; | |
| 164 | − no-map; | |
| 165 | − }; | |
| 166 | − }; | |
| 167 | − | |
| 168 | 160 | sdio_pwrseq: sdio-pwrseq { |
| 169 | 161 | compatible = "mmc-pwrseq-simple"; |
| 170 | 162 | reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>; |
| @@ -17,7 +17,7 @@ · 16 unmodified lines | ||
| 17 | 17 | rtc1 = &vrtc; |
| 18 | 18 | }; |
| 19 | 19 | |
| 20 | − dioo2133: audio-amplifier-0 { | |
| 20 | + dio2133: audio-amplifier-0 { | |
| 21 | 21 | compatible = "simple-audio-amplifier"; |
| 22 | 22 | enable-gpios = <&gpio_ao GPIOAO_2 GPIO_ACTIVE_HIGH>; |
| 23 | 23 | VCC-supply = <&vcc_5v>; |
| @@ -219,7 +219,7 @@ · 195 unmodified lines | ||
| 219 | 219 | audio-widgets = "Line", "Lineout"; |
| 220 | 220 | audio-aux-devs = <&tdmout_b>, <&tdmout_c>, <&tdmin_a>, |
| 221 | 221 | <&tdmin_b>, <&tdmin_c>, <&tdmin_lb>, |
| 222 | − <&dioo2133>; | |
| 222 | + <&dio2133>; | |
| 223 | 223 | audio-routing = "TDMOUT_B IN 0", "FRDDR_A OUT 1", |
| 224 | 224 | "TDMOUT_B IN 1", "FRDDR_B OUT 1", |
| 225 | 225 | "TDMOUT_B IN 2", "FRDDR_C OUT 1", |
| @@ -49,6 +49,12 @@ · 48 unmodified lines | ||
| 49 | 49 | no-map; |
| 50 | 50 | }; |
| 51 | 51 | |
| 52 | + /* 32 MiB reserved for ARM Trusted Firmware (BL32) */ | |
| 53 | + secmon_reserved_bl32: secmon@5300000 { | |
| 54 | + reg = <0x0 0x05300000 0x0 0x2000000>; | |
| 55 | + no-map; | |
| 56 | + }; | |
| 57 | + | |
| 52 | 58 | linux,cma { |
| 53 | 59 | compatible = "shared-dma-pool"; |
| 54 | 60 | reusable; |
| @@ -123,7 +123,7 @@ · 122 unmodified lines | ||
| 123 | 123 | regulator-min-microvolt = <1800000>; |
| 124 | 124 | regulator-max-microvolt = <3300000>; |
| 125 | 125 | |
| 126 | − enable-gpio = <&gpio GPIOE_2 GPIO_ACTIVE_HIGH>; | |
| 126 | + enable-gpio = <&gpio_ao GPIOE_2 GPIO_ACTIVE_HIGH>; | |
| 127 | 127 | enable-active-high; |
| 128 | 128 | regulator-always-on; |
| 129 | 129 | |
| @@ -48,7 +48,7 @@ · 47 unmodified lines | ||
| 48 | 48 | regulator-max-microvolt = <3300000>; |
| 49 | 49 | vin-supply = <&vcc_5v>; |
| 50 | 50 | |
| 51 | − enable-gpio = <&gpio GPIOE_2 GPIO_ACTIVE_HIGH>; | |
| 51 | + enable-gpio = <&gpio_ao GPIOE_2 GPIO_OPEN_DRAIN>; | |
| 52 | 52 | enable-active-high; |
| 53 | 53 | regulator-always-on; |
| 54 | 54 | |
| @@ -203,14 +203,6 @@ · 202 unmodified lines | ||
| 203 | 203 | regulator-always-on; |
| 204 | 204 | }; |
| 205 | 205 | |
| 206 | − reserved-memory { | |
| 207 | − /* TEE Reserved Memory */ | |
| 208 | − bl32_reserved: bl32@5000000 { | |
| 209 | − reg = <0x0 0x05300000 0x0 0x2000000>; | |
| 210 | − no-map; | |
| 211 | − }; | |
| 212 | − }; | |
| 213 | − | |
| 214 | 206 | sdio_pwrseq: sdio-pwrseq { |
| 215 | 207 | compatible = "mmc-pwrseq-simple"; |
| 216 | 208 | reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>; |
| @@ -157,6 +157,10 @@ · 156 unmodified lines | ||
| 157 | 157 | }; |
| 158 | 158 | }; |
| 159 | 159 | |
| 160 | +&ftm_alarm0 { | |
| 161 | + status = "okay"; | |
| 162 | +}; | |
| 163 | + | |
| 160 | 164 | &gpio1 { |
| 161 | 165 | gpio-line-names = |
| 162 | 166 | "", "", "", "", "", "", "", "", |
| @@ -1115,8 +1115,8 @@ · 1114 unmodified lines | ||
| 1115 | 1115 | status = "okay"; |
| 1116 | 1116 | |
| 1117 | 1117 | ports { |
| 1118 | − port@1 { | |
| 1119 | − reg = <1>; | |
| 1118 | + port@0 { | |
| 1119 | + reg = <0>; | |
| 1120 | 1120 | |
| 1121 | 1121 | mipi1_sensor_ep: endpoint { |
| 1122 | 1122 | remote-endpoint = <&camera1_ep>; |
| @@ -554,7 +554,7 @@ · 553 unmodified lines | ||
| 554 | 554 | assigned-clock-rates = <0>, <0>, <0>, <594000000>; |
| 555 | 555 | status = "disabled"; |
| 556 | 556 | |
| 557 | − port@0 { | |
| 557 | + port { | |
| 558 | 558 | lcdif_mipi_dsi: endpoint { |
| 559 | 559 | remote-endpoint = <&mipi_dsi_lcdif_in>; |
| 560 | 560 | }; |
| @@ -1151,8 +1151,8 @@ · 590 unmodified lines | ||
| 1151 | 1151 | #address-cells = <1>; |
| 1152 | 1152 | #size-cells = <0>; |
| 1153 | 1153 | |
| 1154 | − port@0 { | |
| 1155 | − reg = <0>; | |
| 1154 | + port@1 { | |
| 1155 | + reg = <1>; | |
| 1156 | 1156 | |
| 1157 | 1157 | csi1_mipi_ep: endpoint { |
| 1158 | 1158 | remote-endpoint = <&csi1_ep>; |
| @@ -1203,8 +1203,8 @@ · 44 unmodified lines | ||
| 1203 | 1203 | #address-cells = <1>; |
| 1204 | 1204 | #size-cells = <0>; |
| 1205 | 1205 | |
| 1206 | − port@0 { | |
| 1207 | − reg = <0>; | |
| 1206 | + port@1 { | |
| 1207 | + reg = <1>; | |
| 1208 | 1208 | |
| 1209 | 1209 | csi2_mipi_ep: endpoint { |
| 1210 | 1210 | remote-endpoint = <&csi2_ep>; |
| @@ -91,7 +91,7 @@ · 90 unmodified lines | ||
| 91 | 91 | |
| 92 | 92 | sound { |
| 93 | 93 | compatible = "fsl,imx-audio-tlv320aic32x4"; |
| 94 | − model = "tqm-tlv320aic32"; | |
| 94 | + model = "imx-audio-tlv320aic32x4"; | |
| 95 | 95 | ssi-controller = <&sai3>; |
| 96 | 96 | audio-codec = <&tlv320aic3x04>; |
| 97 | 97 | }; |
| @@ -15,8 +15,18 @@ · 14 unmodified lines | ||
| 15 | 15 | model = "Texas Instruments J721S2 EVM"; |
| 16 | 16 | |
| 17 | 17 | chosen { |
| 18 | − stdout-path = "serial10:115200n8"; | |
| 19 | − bootargs = "console=ttyS10,115200n8 earlycon=ns16550a,mmio32,2880000"; | |
| 18 | + stdout-path = "serial2:115200n8"; | |
| 19 | + bootargs = "console=ttyS2,115200n8 earlycon=ns16550a,mmio32,2880000"; | |
| 20 | + }; | |
| 21 | + | |
| 22 | + aliases { | |
| 23 | + serial1 = &mcu_uart0; | |
| 24 | + serial2 = &main_uart8; | |
| 25 | + mmc0 = &main_sdhci0; | |
| 26 | + mmc1 = &main_sdhci1; | |
| 27 | + can0 = &main_mcan16; | |
| 28 | + can1 = &mcu_mcan0; | |
| 29 | + can2 = &mcu_mcan1; | |
| 20 | 30 | }; |
| 21 | 31 | |
| 22 | 32 | evm_12v0: fixedregulator-evm12v0 { |
| @@ -21,28 +21,6 @@ · 20 unmodified lines | ||
| 21 | 21 | #address-cells = <2>; |
| 22 | 22 | #size-cells = <2>; |
| 23 | 23 | |
| 24 | − aliases { | |
| 25 | − serial0 = &wkup_uart0; | |
| 26 | − serial1 = &mcu_uart0; | |
| 27 | − serial2 = &main_uart0; | |
| 28 | − serial3 = &main_uart1; | |
| 29 | − serial4 = &main_uart2; | |
| 30 | − serial5 = &main_uart3; | |
| 31 | − serial6 = &main_uart4; | |
| 32 | − serial7 = &main_uart5; | |
| 33 | − serial8 = &main_uart6; | |
| 34 | − serial9 = &main_uart7; | |
| 35 | − serial10 = &main_uart8; | |
| 36 | − serial11 = &main_uart9; | |
| 37 | − mmc0 = &main_sdhci0; | |
| 38 | − mmc1 = &main_sdhci1; | |
| 39 | − can0 = &main_mcan16; | |
| 40 | − can1 = &mcu_mcan0; | |
| 41 | − can2 = &mcu_mcan1; | |
| 42 | − can3 = &main_mcan3; | |
| 43 | − can4 = &main_mcan5; | |
| 44 | − }; | |
| 45 | − | |
| 46 | 24 | chosen { }; |
| 47 | 25 | |
| 48 | 26 | cpus { |
| @@ -106,7 +106,7 @@ · 105 unmodified lines | ||
| 106 | 106 | msr_s SYS_ICC_SRE_EL2, x0 |
| 107 | 107 | isb // Make sure SRE is now set |
| 108 | 108 | mrs_s x0, SYS_ICC_SRE_EL2 // Read SRE back, |
| 109 | − tbz x0, #0, 1f // and check that it sticks | |
| 109 | + tbz x0, #0, .Lskip_gicv3_\@ // and check that it sticks | |
| 110 | 110 | msr_s SYS_ICH_HCR_EL2, xzr // Reset ICC_HCR_EL2 to defaults |
| 111 | 111 | .Lskip_gicv3_\@: |
| 112 | 112 | .endm |
| @@ -600,6 +600,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = { · 599 unmodified lines | ||
| 600 | 600 | CAP_MIDR_RANGE_LIST(trbe_write_out_of_range_cpus), |
| 601 | 601 | }, |
| 602 | 602 | #endif |
| 603 | +#ifdef CONFIG_ARM64_ERRATUM_2077057 | |
| 604 | + { | |
| 605 | + .desc = "ARM erratum 2077057", | |
| 606 | + .capability = ARM64_WORKAROUND_2077057, | |
| 607 | + .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, | |
| 608 | + ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A510, 0, 0, 2), | |
| 609 | + }, | |
| 610 | +#endif | |
| 603 | 611 | #ifdef CONFIG_ARM64_ERRATUM_2064142 |
| 604 | 612 | { |
| 605 | 613 | .desc = "ARM erratum 2064142", |
| @@ -797,6 +797,24 @@ static bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu, int *ret) · 796 unmodified lines | ||
| 797 | 797 | xfer_to_guest_mode_work_pending(); |
| 798 | 798 | } |
| 799 | 799 | |
| 800 | +/* | |
| 801 | + * Actually run the vCPU, entering an RCU extended quiescent state (EQS) while | |
| 802 | + * the vCPU is running. | |
| 803 | + * | |
| 804 | + * This must be noinstr as instrumentation may make use of RCU, and this is not | |
| 805 | + * safe during the EQS. | |
| 806 | + */ | |
| 807 | +static int noinstr kvm_arm_vcpu_enter_exit(struct kvm_vcpu *vcpu) | |
| 808 | +{ | |
| 809 | + int ret; | |
| 810 | + | |
| 811 | + guest_state_enter_irqoff(); | |
| 812 | + ret = kvm_call_hyp_ret(__kvm_vcpu_run, vcpu); | |
| 813 | + guest_state_exit_irqoff(); | |
| 814 | + | |
| 815 | + return ret; | |
| 816 | +} | |
| 817 | + | |
| 800 | 818 | /** |
| 801 | 819 | * kvm_arch_vcpu_ioctl_run - the main VCPU run function to execute guest code |
| 802 | 820 | * @vcpu: The VCPU pointer |
| @@ -881,9 +899,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 78 unmodified lines | ||
| 881 | 899 | * Enter the guest |
| 882 | 900 | */ |
| 883 | 901 | trace_kvm_entry(*vcpu_pc(vcpu)); |
| 884 | − guest_enter_irqoff(); | |
| 902 | + guest_timing_enter_irqoff(); | |
| 885 | 903 | |
| 886 | − ret = kvm_call_hyp_ret(__kvm_vcpu_run, vcpu); | |
| 904 | + ret = kvm_arm_vcpu_enter_exit(vcpu); | |
| 887 | 905 | |
| 888 | 906 | vcpu->mode = OUTSIDE_GUEST_MODE; |
| 889 | 907 | vcpu->stat.exits++; |
| @@ -918,26 +936,23 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 28 unmodified lines | ||
| 918 | 936 | kvm_arch_vcpu_ctxsync_fp(vcpu); |
| 919 | 937 | |
| 920 | 938 | /* |
| 921 | − * We may have taken a host interrupt in HYP mode (ie | |
| 922 | − * while executing the guest). This interrupt is still | |
| 923 | − * pending, as we haven't serviced it yet! | |
| 939 | + * We must ensure that any pending interrupts are taken before | |
| 940 | + * we exit guest timing so that timer ticks are accounted as | |
| 941 | + * guest time. Transiently unmask interrupts so that any | |
| 942 | + * pending interrupts are taken. | |
| 924 | 943 | * |
| 925 | − * We're now back in SVC mode, with interrupts | |
| 926 | − * disabled. Enabling the interrupts now will have | |
| 927 | − * the effect of taking the interrupt again, in SVC | |
| 928 | − * mode this time. | |
| 944 | + * Per ARM DDI 0487G.b section D1.13.4, an ISB (or other | |
| 945 | + * context synchronization event) is necessary to ensure that | |
| 946 | + * pending interrupts are taken. | |
| 929 | 947 | */ |
| 930 | 948 | local_irq_enable(); |
| 949 | + isb(); | |
| 950 | + local_irq_disable(); | |
| 951 | + | |
| 952 | + guest_timing_exit_irqoff(); | |
| 953 | + | |
| 954 | + local_irq_enable(); | |
| 931 | 955 | |
| 932 | − /* | |
| 933 | − * We do local_irq_enable() before calling guest_exit() so | |
| 934 | − * that if a timer interrupt hits while running the guest we | |
| 935 | − * account that tick as being spent in the guest. We enable | |
| 936 | − * preemption after calling guest_exit() so that if we get | |
| 937 | − * preempted we make sure ticks after that is not counted as | |
| 938 | − * guest time. | |
| 939 | − */ | |
| 940 | − guest_exit(); | |
| 941 | 956 | trace_kvm_exit(ret, kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu)); |
| 942 | 957 | |
| 943 | 958 | /* Exit types that need handling before we can be preempted */ |
| @@ -228,6 +228,14 @@ int handle_exit(struct kvm_vcpu *vcpu, int exception_index) · 227 unmodified lines | ||
| 228 | 228 | { |
| 229 | 229 | struct kvm_run *run = vcpu->run; |
| 230 | 230 | |
| 231 | + if (ARM_SERROR_PENDING(exception_index)) { | |
| 232 | + /* | |
| 233 | + * The SError is handled by handle_exit_early(). If the guest | |
| 234 | + * survives it will re-execute the original instruction. | |
| 235 | + */ | |
| 236 | + return 1; | |
| 237 | + } | |
| 238 | + | |
| 231 | 239 | exception_index = ARM_EXCEPTION_CODE(exception_index); |
| 232 | 240 | |
| 233 | 241 | switch (exception_index) { |
| @@ -402,6 +402,24 @@ static inline bool kvm_hyp_handle_exit(struct kvm_vcpu *vcpu, u64 *exit_code) · 401 unmodified lines | ||
| 402 | 402 | return false; |
| 403 | 403 | } |
| 404 | 404 | |
| 405 | +static inline void synchronize_vcpu_pstate(struct kvm_vcpu *vcpu, u64 *exit_code) | |
| 406 | +{ | |
| 407 | + /* | |
| 408 | + * Check for the conditions of Cortex-A510's #2077057. When these occur | |
| 409 | + * SPSR_EL2 can't be trusted, but isn't needed either as it is | |
| 410 | + * unchanged from the value in vcpu_gp_regs(vcpu)->pstate. | |
| 411 | + * Are we single-stepping the guest, and took a PAC exception from the | |
| 412 | + * active-not-pending state? | |
| 413 | + */ | |
| 414 | + if (cpus_have_final_cap(ARM64_WORKAROUND_2077057) && | |
| 415 | + vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP && | |
| 416 | + *vcpu_cpsr(vcpu) & DBG_SPSR_SS && | |
| 417 | + ESR_ELx_EC(read_sysreg_el2(SYS_ESR)) == ESR_ELx_EC_PAC) | |
| 418 | + write_sysreg_el2(*vcpu_cpsr(vcpu), SYS_SPSR); | |
| 419 | + | |
| 420 | + vcpu->arch.ctxt.regs.pstate = read_sysreg_el2(SYS_SPSR); | |
| 421 | +} | |
| 422 | + | |
| 405 | 423 | /* |
| 406 | 424 | * Return true when we were able to fixup the guest exit and should return to |
| 407 | 425 | * the guest, false when we should restore the host state and return to the |
| @@ -413,7 +431,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) · 5 unmodified lines | ||
| 413 | 431 | * Save PSTATE early so that we can evaluate the vcpu mode |
| 414 | 432 | * early on. |
| 415 | 433 | */ |
| 416 | − vcpu->arch.ctxt.regs.pstate = read_sysreg_el2(SYS_SPSR); | |
| 434 | + synchronize_vcpu_pstate(vcpu, exit_code); | |
| 417 | 435 | |
| 418 | 436 | /* |
| 419 | 437 | * Check whether we want to repaint the state one way or |
| @@ -424,7 +442,8 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) · 4 unmodified lines | ||
| 424 | 442 | if (ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ) |
| 425 | 443 | vcpu->arch.fault.esr_el2 = read_sysreg_el2(SYS_ESR); |
| 426 | 444 | |
| 427 | − if (ARM_SERROR_PENDING(*exit_code)) { | |
| 445 | + if (ARM_SERROR_PENDING(*exit_code) && | |
| 446 | + ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ) { | |
| 428 | 447 | u8 esr_ec = kvm_vcpu_trap_get_class(vcpu); |
| 429 | 448 | |
| 430 | 449 | /* |
| @@ -248,6 +248,8 @@ unsigned long vgic_mmio_read_pending(struct kvm_vcpu *vcpu, · 247 unmodified lines | ||
| 248 | 248 | IRQCHIP_STATE_PENDING, |
| 249 | 249 | &val); |
| 250 | 250 | WARN_RATELIMIT(err, "IRQ %d", irq->host_irq); |
| 251 | + } else if (vgic_irq_is_mapped_level(irq)) { | |
| 252 | + val = vgic_get_phys_line_level(irq); | |
| 251 | 253 | } else { |
| 252 | 254 | val = irq_is_pending(irq); |
| 253 | 255 | } |
| @@ -55,9 +55,10 @@ WORKAROUND_1418040 · 54 unmodified lines | ||
| 55 | 55 | WORKAROUND_1463225 |
| 56 | 56 | WORKAROUND_1508412 |
| 57 | 57 | WORKAROUND_1542419 |
| 58 | −WORKAROUND_2064142 | |
| 59 | −WORKAROUND_2038923 | |
| 60 | 58 | WORKAROUND_1902691 |
| 59 | +WORKAROUND_2038923 | |
| 60 | +WORKAROUND_2064142 | |
| 61 | +WORKAROUND_2077057 | |
| 61 | 62 | WORKAROUND_TRBE_OVERWRITE_FILL_MODE |
| 62 | 63 | WORKAROUND_TSB_FLUSH_FAILURE |
| 63 | 64 | WORKAROUND_TRBE_WRITE_OUT_OF_RANGE |
| @@ -83,6 +83,8 @@ · 82 unmodified lines | ||
| 83 | 83 | label = "HDMI OUT"; |
| 84 | 84 | type = "a"; |
| 85 | 85 | |
| 86 | + ddc-en-gpios = <&gpa 25 GPIO_ACTIVE_HIGH>; | |
| 87 | + | |
| 86 | 88 | port { |
| 87 | 89 | hdmi_con: endpoint { |
| 88 | 90 | remote-endpoint = <&dw_hdmi_out>; |
| @@ -114,17 +116,6 @@ · 25 unmodified lines | ||
| 114 | 116 | gpio = <&gpf 14 GPIO_ACTIVE_LOW>; |
| 115 | 117 | enable-active-high; |
| 116 | 118 | }; |
| 117 | − | |
| 118 | − hdmi_power: fixedregulator@3 { | |
| 119 | − compatible = "regulator-fixed"; | |
| 120 | − | |
| 121 | − regulator-name = "hdmi_power"; | |
| 122 | − regulator-min-microvolt = <5000000>; | |
| 123 | − regulator-max-microvolt = <5000000>; | |
| 124 | − | |
| 125 | − gpio = <&gpa 25 0>; | |
| 126 | − enable-active-high; | |
| 127 | − }; | |
| 128 | 119 | }; |
| 129 | 120 | |
| 130 | 121 | &ext { |
| @@ -576,8 +567,6 @@ · 445 unmodified lines | ||
| 576 | 567 | pinctrl-names = "default"; |
| 577 | 568 | pinctrl-0 = <&pins_hdmi_ddc>; |
| 578 | 569 | |
| 579 | − hdmi-5v-supply = <&hdmi_power>; | |
| 580 | − | |
| 581 | 570 | ports { |
| 582 | 571 | #address-cells = <1>; |
| 583 | 572 | #size-cells = <0>; |
| @@ -74,7 +74,7 @@ · 73 unmodified lines | ||
| 74 | 74 | #define EXC(inst_reg,addr,handler) \ |
| 75 | 75 | 9: inst_reg, addr; \ |
| 76 | 76 | .section __ex_table,"a"; \ |
| 77 | − PTR 9b, handler; \ | |
| 77 | + PTR_WD 9b, handler; \ | |
| 78 | 78 | .previous |
| 79 | 79 | |
| 80 | 80 | /* |
| @@ -414,6 +414,24 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu, · 413 unmodified lines | ||
| 414 | 414 | return -ENOIOCTLCMD; |
| 415 | 415 | } |
| 416 | 416 | |
| 417 | +/* | |
| 418 | + * Actually run the vCPU, entering an RCU extended quiescent state (EQS) while | |
| 419 | + * the vCPU is running. | |
| 420 | + * | |
| 421 | + * This must be noinstr as instrumentation may make use of RCU, and this is not | |
| 422 | + * safe during the EQS. | |
| 423 | + */ | |
| 424 | +static int noinstr kvm_mips_vcpu_enter_exit(struct kvm_vcpu *vcpu) | |
| 425 | +{ | |
| 426 | + int ret; | |
| 427 | + | |
| 428 | + guest_state_enter_irqoff(); | |
| 429 | + ret = kvm_mips_callbacks->vcpu_run(vcpu); | |
| 430 | + guest_state_exit_irqoff(); | |
| 431 | + | |
| 432 | + return ret; | |
| 433 | +} | |
| 434 | + | |
| 417 | 435 | int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) |
| 418 | 436 | { |
| 419 | 437 | int r = -EINTR; |
| @@ -434,7 +452,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 14 unmodified lines | ||
| 434 | 452 | lose_fpu(1); |
| 435 | 453 | |
| 436 | 454 | local_irq_disable(); |
| 437 | − guest_enter_irqoff(); | |
| 455 | + guest_timing_enter_irqoff(); | |
| 438 | 456 | trace_kvm_enter(vcpu); |
| 439 | 457 | |
| 440 | 458 | /* |
| @@ -445,10 +463,23 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 4 unmodified lines | ||
| 445 | 463 | */ |
| 446 | 464 | smp_store_mb(vcpu->mode, IN_GUEST_MODE); |
| 447 | 465 | |
| 448 | − r = kvm_mips_callbacks->vcpu_run(vcpu); | |
| 466 | + r = kvm_mips_vcpu_enter_exit(vcpu); | |
| 467 | + | |
| 468 | + /* | |
| 469 | + * We must ensure that any pending interrupts are taken before | |
| 470 | + * we exit guest timing so that timer ticks are accounted as | |
| 471 | + * guest time. Transiently unmask interrupts so that any | |
| 472 | + * pending interrupts are taken. | |
| 473 | + * | |
| 474 | + * TODO: is there a barrier which ensures that pending interrupts are | |
| 475 | + * recognised? Currently this just hopes that the CPU takes any pending | |
| 476 | + * interrupts between the enable and disable. | |
| 477 | + */ | |
| 478 | + local_irq_enable(); | |
| 479 | + local_irq_disable(); | |
| 449 | 480 | |
| 450 | 481 | trace_kvm_out(vcpu); |
| 451 | − guest_exit_irqoff(); | |
| 482 | + guest_timing_exit_irqoff(); | |
| 452 | 483 | local_irq_enable(); |
| 453 | 484 | |
| 454 | 485 | out: |
| @@ -1168,7 +1199,7 @@ static void kvm_mips_set_c0_status(void) · 713 unmodified lines | ||
| 1168 | 1199 | /* |
| 1169 | 1200 | * Return value is in the form (errcode<<2 | RESUME_FLAG_HOST | RESUME_FLAG_NV) |
| 1170 | 1201 | */ |
| 1171 | −int kvm_mips_handle_exit(struct kvm_vcpu *vcpu) | |
| 1202 | +static int __kvm_mips_handle_exit(struct kvm_vcpu *vcpu) | |
| 1172 | 1203 | { |
| 1173 | 1204 | struct kvm_run *run = vcpu->run; |
| 1174 | 1205 | u32 cause = vcpu->arch.host_cp0_cause; |
| @@ -1357,6 +1388,17 @@ int kvm_mips_handle_exit(struct kvm_vcpu *vcpu) · 182 unmodified lines | ||
| 1357 | 1388 | return ret; |
| 1358 | 1389 | } |
| 1359 | 1390 | |
| 1391 | +int noinstr kvm_mips_handle_exit(struct kvm_vcpu *vcpu) | |
| 1392 | +{ | |
| 1393 | + int ret; | |
| 1394 | + | |
| 1395 | + guest_state_exit_irqoff(); | |
| 1396 | + ret = __kvm_mips_handle_exit(vcpu); | |
| 1397 | + guest_state_enter_irqoff(); | |
| 1398 | + | |
| 1399 | + return ret; | |
| 1400 | +} | |
| 1401 | + | |
| 1360 | 1402 | /* Enable FPU for guest and restore context */ |
| 1361 | 1403 | void kvm_own_fpu(struct kvm_vcpu *vcpu) |
| 1362 | 1404 | { |
| @@ -458,8 +458,8 @@ void kvm_vz_acquire_htimer(struct kvm_vcpu *vcpu) · 457 unmodified lines | ||
| 458 | 458 | /** |
| 459 | 459 | * _kvm_vz_save_htimer() - Switch to software emulation of guest timer. |
| 460 | 460 | * @vcpu: Virtual CPU. |
| 461 | − * @compare: Pointer to write compare value to. | |
| 462 | − * @cause: Pointer to write cause value to. | |
| 461 | + * @out_compare: Pointer to write compare value to. | |
| 462 | + * @out_cause: Pointer to write cause value to. | |
| 463 | 463 | * |
| 464 | 464 | * Save VZ guest timer state and switch to software emulation of guest CP0 |
| 465 | 465 | * timer. The hard timer must already be in use, so preemption should be |
| @@ -1541,11 +1541,14 @@ static int kvm_trap_vz_handle_guest_exit(struct kvm_vcpu *vcpu) · 1075 unmodified lines | ||
| 1541 | 1541 | } |
| 1542 | 1542 | |
| 1543 | 1543 | /** |
| 1544 | − * kvm_trap_vz_handle_cop_unusuable() - Guest used unusable coprocessor. | |
| 1544 | + * kvm_trap_vz_handle_cop_unusable() - Guest used unusable coprocessor. | |
| 1545 | 1545 | * @vcpu: Virtual CPU context. |
| 1546 | 1546 | * |
| 1547 | 1547 | * Handle when the guest attempts to use a coprocessor which hasn't been allowed |
| 1548 | 1548 | * by the root context. |
| 1549 | + * | |
| 1550 | + * Return: value indicating whether to resume the host or the guest | |
| 1551 | + * (RESUME_HOST or RESUME_GUEST) | |
| 1549 | 1552 | */ |
| 1550 | 1553 | static int kvm_trap_vz_handle_cop_unusable(struct kvm_vcpu *vcpu) |
| 1551 | 1554 | { |
| @@ -1592,6 +1595,9 @@ static int kvm_trap_vz_handle_cop_unusable(struct kvm_vcpu *vcpu) · 40 unmodified lines | ||
| 1592 | 1595 | * |
| 1593 | 1596 | * Handle when the guest attempts to use MSA when it is disabled in the root |
| 1594 | 1597 | * context. |
| 1598 | + * | |
| 1599 | + * Return: value indicating whether to resume the host or the guest | |
| 1600 | + * (RESUME_HOST or RESUME_GUEST) | |
| 1595 | 1601 | */ |
| 1596 | 1602 | static int kvm_trap_vz_handle_msa_disabled(struct kvm_vcpu *vcpu) |
| 1597 | 1603 | { |
| @@ -12,6 +12,14 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <asm/barrier.h> |
| 13 | 13 | #include <linux/atomic.h> |
| 14 | 14 | |
| 15 | +/* compiler build environment sanity checks: */ | |
| 16 | +#if !defined(CONFIG_64BIT) && defined(__LP64__) | |
| 17 | +#error "Please use 'ARCH=parisc' to build the 32-bit kernel." | |
| 18 | +#endif | |
| 19 | +#if defined(CONFIG_64BIT) && !defined(__LP64__) | |
| 20 | +#error "Please use 'ARCH=parisc64' to build the 64-bit kernel." | |
| 21 | +#endif | |
| 22 | + | |
| 15 | 23 | /* See http://marc.theaimsgroup.com/?t=108826637900003 for discussion |
| 16 | 24 | * on use of volatile and __*_bit() (set/clear/change): |
| 17 | 25 | * *_bit() want use of volatile. |
| @@ -89,8 +89,8 @@ struct exception_table_entry { · 88 unmodified lines | ||
| 89 | 89 | __asm__("1: " ldx " 0(" sr "%2),%0\n" \ |
| 90 | 90 | "9:\n" \ |
| 91 | 91 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ |
| 92 | − : "=r"(__gu_val), "=r"(__gu_err) \ | |
| 93 | − : "r"(ptr), "1"(__gu_err)); \ | |
| 92 | + : "=r"(__gu_val), "+r"(__gu_err) \ | |
| 93 | + : "r"(ptr)); \ | |
| 94 | 94 | \ |
| 95 | 95 | (val) = (__force __typeof__(*(ptr))) __gu_val; \ |
| 96 | 96 | } |
| @@ -123,8 +123,8 @@ struct exception_table_entry { · 26 unmodified lines | ||
| 123 | 123 | "9:\n" \ |
| 124 | 124 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ |
| 125 | 125 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(2b, 9b) \ |
| 126 | − : "=&r"(__gu_tmp.l), "=r"(__gu_err) \ | |
| 127 | − : "r"(ptr), "1"(__gu_err)); \ | |
| 126 | + : "=&r"(__gu_tmp.l), "+r"(__gu_err) \ | |
| 127 | + : "r"(ptr)); \ | |
| 128 | 128 | \ |
| 129 | 129 | (val) = __gu_tmp.t; \ |
| 130 | 130 | } |
| @@ -135,13 +135,12 @@ struct exception_table_entry { · 4 unmodified lines | ||
| 135 | 135 | #define __put_user_internal(sr, x, ptr) \ |
| 136 | 136 | ({ \ |
| 137 | 137 | ASM_EXCEPTIONTABLE_VAR(__pu_err); \ |
| 138 | − __typeof__(*(ptr)) __x = (__typeof__(*(ptr)))(x); \ | |
| 139 | 138 | \ |
| 140 | 139 | switch (sizeof(*(ptr))) { \ |
| 141 | − case 1: __put_user_asm(sr, "stb", __x, ptr); break; \ | |
| 142 | − case 2: __put_user_asm(sr, "sth", __x, ptr); break; \ | |
| 143 | − case 4: __put_user_asm(sr, "stw", __x, ptr); break; \ | |
| 144 | − case 8: STD_USER(sr, __x, ptr); break; \ | |
| 140 | + case 1: __put_user_asm(sr, "stb", x, ptr); break; \ | |
| 141 | + case 2: __put_user_asm(sr, "sth", x, ptr); break; \ | |
| 142 | + case 4: __put_user_asm(sr, "stw", x, ptr); break; \ | |
| 143 | + case 8: STD_USER(sr, x, ptr); break; \ | |
| 145 | 144 | default: BUILD_BUG(); \ |
| 146 | 145 | } \ |
| 147 | 146 | \ |
| @@ -150,7 +149,9 @@ struct exception_table_entry { · 2 unmodified lines | ||
| 150 | 149 | |
| 151 | 150 | #define __put_user(x, ptr) \ |
| 152 | 151 | ({ \ |
| 153 | − __put_user_internal("%%sr3,", x, ptr); \ | |
| 152 | + __typeof__(&*(ptr)) __ptr = ptr; \ | |
| 153 | + __typeof__(*(__ptr)) __x = (__typeof__(*(__ptr)))(x); \ | |
| 154 | + __put_user_internal("%%sr3,", __x, __ptr); \ | |
| 154 | 155 | }) |
| 155 | 156 | |
| 156 | 157 | #define __put_kernel_nofault(dst, src, type, err_label) \ |
| @@ -180,8 +181,8 @@ struct exception_table_entry { · 23 unmodified lines | ||
| 180 | 181 | "1: " stx " %2,0(" sr "%1)\n" \ |
| 181 | 182 | "9:\n" \ |
| 182 | 183 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ |
| 183 | − : "=r"(__pu_err) \ | |
| 184 | − : "r"(ptr), "r"(x), "0"(__pu_err)) | |
| 184 | + : "+r"(__pu_err) \ | |
| 185 | + : "r"(ptr), "r"(x)) | |
| 185 | 186 | |
| 186 | 187 | |
| 187 | 188 | #if !defined(CONFIG_64BIT) |
| @@ -193,8 +194,8 @@ struct exception_table_entry { · 5 unmodified lines | ||
| 193 | 194 | "9:\n" \ |
| 194 | 195 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ |
| 195 | 196 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(2b, 9b) \ |
| 196 | − : "=r"(__pu_err) \ | |
| 197 | − : "r"(ptr), "r"(__val), "0"(__pu_err)); \ | |
| 197 | + : "+r"(__pu_err) \ | |
| 198 | + : "r"(ptr), "r"(__val)); \ | |
| 198 | 199 | } while (0) |
| 199 | 200 | |
| 200 | 201 | #endif /* !defined(CONFIG_64BIT) */ |
| @@ -340,7 +340,7 @@ static int emulate_stw(struct pt_regs *regs, int frreg, int flop) · 339 unmodified lines | ||
| 340 | 340 | : "r" (val), "r" (regs->ior), "r" (regs->isr) |
| 341 | 341 | : "r19", "r20", "r21", "r22", "r1", FIXUP_BRANCH_CLOBBER ); |
| 342 | 342 | |
| 343 | − return 0; | |
| 343 | + return ret; | |
| 344 | 344 | } |
| 345 | 345 | static int emulate_std(struct pt_regs *regs, int frreg, int flop) |
| 346 | 346 | { |
| @@ -397,7 +397,7 @@ static int emulate_std(struct pt_regs *regs, int frreg, int flop) · 50 unmodified lines | ||
| 397 | 397 | __asm__ __volatile__ ( |
| 398 | 398 | " mtsp %4, %%sr1\n" |
| 399 | 399 | " zdep %2, 29, 2, %%r19\n" |
| 400 | −" dep %%r0, 31, 2, %2\n" | |
| 400 | +" dep %%r0, 31, 2, %3\n" | |
| 401 | 401 | " mtsar %%r19\n" |
| 402 | 402 | " zvdepi -2, 32, %%r19\n" |
| 403 | 403 | "1: ldw 0(%%sr1,%3),%%r20\n" |
| @@ -409,7 +409,7 @@ static int emulate_std(struct pt_regs *regs, int frreg, int flop) · 5 unmodified lines | ||
| 409 | 409 | " andcm %%r21, %%r19, %%r21\n" |
| 410 | 410 | " or %1, %%r20, %1\n" |
| 411 | 411 | " or %2, %%r21, %2\n" |
| 412 | −"3: stw %1,0(%%sr1,%1)\n" | |
| 412 | +"3: stw %1,0(%%sr1,%3)\n" | |
| 413 | 413 | "4: stw %%r1,4(%%sr1,%3)\n" |
| 414 | 414 | "5: stw %2,8(%%sr1,%3)\n" |
| 415 | 415 | " copy %%r0, %0\n" |
| @@ -596,7 +596,6 @@ void handle_unaligned(struct pt_regs *regs) · 180 unmodified lines | ||
| 596 | 596 | ret = ERR_NOTHANDLED; /* "undefined", but lets kill them. */ |
| 597 | 597 | break; |
| 598 | 598 | } |
| 599 | −#ifdef CONFIG_PA20 | |
| 600 | 599 | switch (regs->iir & OPCODE2_MASK) |
| 601 | 600 | { |
| 602 | 601 | case OPCODE_FLDD_L: |
| @@ -607,22 +606,23 @@ void handle_unaligned(struct pt_regs *regs) · 4 unmodified lines | ||
| 607 | 606 | flop=1; |
| 608 | 607 | ret = emulate_std(regs, R2(regs->iir),1); |
| 609 | 608 | break; |
| 609 | +#ifdef CONFIG_PA20 | |
| 610 | 610 | case OPCODE_LDD_L: |
| 611 | 611 | ret = emulate_ldd(regs, R2(regs->iir),0); |
| 612 | 612 | break; |
| 613 | 613 | case OPCODE_STD_L: |
| 614 | 614 | ret = emulate_std(regs, R2(regs->iir),0); |
| 615 | 615 | break; |
| 616 | − } | |
| 617 | 616 | #endif |
| 617 | + } | |
| 618 | 618 | switch (regs->iir & OPCODE3_MASK) |
| 619 | 619 | { |
| 620 | 620 | case OPCODE_FLDW_L: |
| 621 | 621 | flop=1; |
| 622 | − ret = emulate_ldw(regs, R2(regs->iir),0); | |
| 622 | + ret = emulate_ldw(regs, R2(regs->iir), 1); | |
| 623 | 623 | break; |
| 624 | 624 | case OPCODE_LDW_M: |
| 625 | − ret = emulate_ldw(regs, R2(regs->iir),1); | |
| 625 | + ret = emulate_ldw(regs, R2(regs->iir), 0); | |
| 626 | 626 | break; |
| 627 | 627 | |
| 628 | 628 | case OPCODE_FSTW_L: |
| @@ -346,6 +346,16 @@ u64 ioread64be(const void __iomem *addr) · 345 unmodified lines | ||
| 346 | 346 | return *((u64 *)addr); |
| 347 | 347 | } |
| 348 | 348 | |
| 349 | +u64 ioread64_lo_hi(const void __iomem *addr) | |
| 350 | +{ | |
| 351 | + u32 low, high; | |
| 352 | + | |
| 353 | + low = ioread32(addr); | |
| 354 | + high = ioread32(addr + sizeof(u32)); | |
| 355 | + | |
| 356 | + return low + ((u64)high << 32); | |
| 357 | +} | |
| 358 | + | |
| 349 | 359 | u64 ioread64_hi_lo(const void __iomem *addr) |
| 350 | 360 | { |
| 351 | 361 | u32 low, high; |
| @@ -419,6 +429,12 @@ void iowrite64be(u64 datum, void __iomem *addr) · 67 unmodified lines | ||
| 419 | 429 | } |
| 420 | 430 | } |
| 421 | 431 | |
| 432 | +void iowrite64_lo_hi(u64 val, void __iomem *addr) | |
| 433 | +{ | |
| 434 | + iowrite32(val, addr); | |
| 435 | + iowrite32(val >> 32, addr + sizeof(u32)); | |
| 436 | +} | |
| 437 | + | |
| 422 | 438 | void iowrite64_hi_lo(u64 val, void __iomem *addr) |
| 423 | 439 | { |
| 424 | 440 | iowrite32(val >> 32, addr + sizeof(u32)); |
| @@ -530,6 +546,7 @@ EXPORT_SYMBOL(ioread32); · 105 unmodified lines | ||
| 530 | 546 | EXPORT_SYMBOL(ioread32be); |
| 531 | 547 | EXPORT_SYMBOL(ioread64); |
| 532 | 548 | EXPORT_SYMBOL(ioread64be); |
| 549 | +EXPORT_SYMBOL(ioread64_lo_hi); | |
| 533 | 550 | EXPORT_SYMBOL(ioread64_hi_lo); |
| 534 | 551 | EXPORT_SYMBOL(iowrite8); |
| 535 | 552 | EXPORT_SYMBOL(iowrite16); |
| @@ -538,6 +555,7 @@ EXPORT_SYMBOL(iowrite32); · 2 unmodified lines | ||
| 538 | 555 | EXPORT_SYMBOL(iowrite32be); |
| 539 | 556 | EXPORT_SYMBOL(iowrite64); |
| 540 | 557 | EXPORT_SYMBOL(iowrite64be); |
| 558 | +EXPORT_SYMBOL(iowrite64_lo_hi); | |
| 541 | 559 | EXPORT_SYMBOL(iowrite64_hi_lo); |
| 542 | 560 | EXPORT_SYMBOL(ioread8_rep); |
| 543 | 561 | EXPORT_SYMBOL(ioread16_rep); |
| @@ -337,9 +337,9 @@ static void __init setup_bootmem(void) · 336 unmodified lines | ||
| 337 | 337 | |
| 338 | 338 | static bool kernel_set_to_readonly; |
| 339 | 339 | |
| 340 | −static void __init map_pages(unsigned long start_vaddr, | |
| 341 | − unsigned long start_paddr, unsigned long size, | |
| 342 | − pgprot_t pgprot, int force) | |
| 340 | +static void __ref map_pages(unsigned long start_vaddr, | |
| 341 | + unsigned long start_paddr, unsigned long size, | |
| 342 | + pgprot_t pgprot, int force) | |
| 343 | 343 | { |
| 344 | 344 | pmd_t *pmd; |
| 345 | 345 | pte_t *pg_table; |
| @@ -449,7 +449,7 @@ void __init set_kernel_text_rw(int enable_read_write) · 103 unmodified lines | ||
| 449 | 449 | flush_tlb_all(); |
| 450 | 450 | } |
| 451 | 451 | |
| 452 | −void __ref free_initmem(void) | |
| 452 | +void free_initmem(void) | |
| 453 | 453 | { |
| 454 | 454 | unsigned long init_begin = (unsigned long)__init_begin; |
| 455 | 455 | unsigned long init_end = (unsigned long)__init_end; |
| @@ -463,7 +463,6 @@ void __ref free_initmem(void) · 7 unmodified lines | ||
| 463 | 463 | /* The init text pages are marked R-X. We have to |
| 464 | 464 | * flush the icache and mark them RW- |
| 465 | 465 | * |
| 466 | − * This is tricky, because map_pages is in the init section. | |
| 467 | 466 | * Do a dummy remap of the data section first (the data |
| 468 | 467 | * section is already PAGE_KERNEL) to pull in the TLB entries |
| 469 | 468 | * for map_kernel */ |
| @@ -421,14 +421,14 @@ InstructionTLBMiss: · 420 unmodified lines | ||
| 421 | 421 | */ |
| 422 | 422 | /* Get PTE (linux-style) and check access */ |
| 423 | 423 | mfspr r3,SPRN_IMISS |
| 424 | −#ifdef CONFIG_MODULES | |
| 424 | +#if defined(CONFIG_MODULES) || defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) | |
| 425 | 425 | lis r1, TASK_SIZE@h /* check if kernel address */ |
| 426 | 426 | cmplw 0,r1,r3 |
| 427 | 427 | #endif |
| 428 | 428 | mfspr r2, SPRN_SDR1 |
| 429 | 429 | li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | _PAGE_USER |
| 430 | 430 | rlwinm r2, r2, 28, 0xfffff000 |
| 431 | −#ifdef CONFIG_MODULES | |
| 431 | +#if defined(CONFIG_MODULES) || defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) | |
| 432 | 432 | bgt- 112f |
| 433 | 433 | lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */ |
| 434 | 434 | li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC |
| @@ -3264,12 +3264,14 @@ void emulate_update_regs(struct pt_regs *regs, struct instruction_op *op) · 3263 unmodified lines | ||
| 3264 | 3264 | case BARRIER_EIEIO: |
| 3265 | 3265 | eieio(); |
| 3266 | 3266 | break; |
| 3267 | +#ifdef CONFIG_PPC64 | |
| 3267 | 3268 | case BARRIER_LWSYNC: |
| 3268 | 3269 | asm volatile("lwsync" : : : "memory"); |
| 3269 | 3270 | break; |
| 3270 | 3271 | case BARRIER_PTESYNC: |
| 3271 | 3272 | asm volatile("ptesync" : : : "memory"); |
| 3272 | 3273 | break; |
| 3274 | +#endif | |
| 3273 | 3275 | } |
| 3274 | 3276 | break; |
| 3275 | 3277 | |
| @@ -50,6 +50,12 @@ riscv-march-$(CONFIG_ARCH_RV32I) := rv32ima · 49 unmodified lines | ||
| 50 | 50 | riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima |
| 51 | 51 | riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd |
| 52 | 52 | riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c |
| 53 | + | |
| 54 | +# Newer binutils versions default to ISA spec version 20191213 which moves some | |
| 55 | +# instructions from the I extension to the Zicsr and Zifencei extensions. | |
| 56 | +toolchain-need-zicsr-zifencei := $(call cc-option-yn, -march=$(riscv-march-y)_zicsr_zifencei) | |
| 57 | +riscv-march-$(toolchain-need-zicsr-zifencei) := $(riscv-march-y)_zicsr_zifencei | |
| 58 | + | |
| 53 | 59 | KBUILD_CFLAGS += -march=$(subst fd,,$(riscv-march-y)) |
| 54 | 60 | KBUILD_AFLAGS += -march=$(riscv-march-y) |
| 55 | 61 | |
| @@ -23,7 +23,7 @@ CONFIG_SLOB=y · 22 unmodified lines | ||
| 23 | 23 | CONFIG_SOC_CANAAN=y |
| 24 | 24 | CONFIG_SMP=y |
| 25 | 25 | CONFIG_NR_CPUS=2 |
| 26 | −CONFIG_CMDLINE="earlycon console=ttySIF0 rootdelay=2 root=/dev/mmcblk0p1 ro" | |
| 26 | +CONFIG_CMDLINE="earlycon console=ttySIF0 root=/dev/mmcblk0p1 rootwait ro" | |
| 27 | 27 | CONFIG_CMDLINE_FORCE=y |
| 28 | 28 | # CONFIG_SECCOMP is not set |
| 29 | 29 | # CONFIG_STACKPROTECTOR is not set |
| @@ -51,6 +51,8 @@ obj-$(CONFIG_MODULE_SECTIONS) += module-sections.o · 50 unmodified lines | ||
| 51 | 51 | obj-$(CONFIG_FUNCTION_TRACER) += mcount.o ftrace.o |
| 52 | 52 | obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o |
| 53 | 53 | |
| 54 | +obj-$(CONFIG_TRACE_IRQFLAGS) += trace_irq.o | |
| 55 | + | |
| 54 | 56 | obj-$(CONFIG_RISCV_BASE_PMU) += perf_event.o |
| 55 | 57 | obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o |
| 56 | 58 | obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | ||
| 12 | 12 | #include <linux/sched/hotplug.h> |
| 13 | 13 | #include <asm/irq.h> |
| 14 | 14 | #include <asm/cpu_ops.h> |
| 15 | +#include <asm/numa.h> | |
| 15 | 16 | #include <asm/sbi.h> |
| 16 | 17 | |
| 17 | 18 | bool cpu_has_hotplug(unsigned int cpu) |
| @@ -40,6 +41,7 @@ int __cpu_disable(void) · 22 unmodified lines | ||
| 40 | 41 | return ret; |
| 41 | 42 | |
| 42 | 43 | remove_cpu_topology(cpu); |
| 44 | + numa_remove_cpu(cpu); | |
| 43 | 45 | set_cpu_online(cpu, false); |
| 44 | 46 | irq_migrate_all_off_this_cpu(); |
| 45 | 47 | |
| @@ -108,7 +108,7 @@ _save_context: · 107 unmodified lines | ||
| 108 | 108 | .option pop |
| 109 | 109 | |
| 110 | 110 | #ifdef CONFIG_TRACE_IRQFLAGS |
| 111 | − call trace_hardirqs_off | |
| 111 | + call __trace_hardirqs_off | |
| 112 | 112 | #endif |
| 113 | 113 | |
| 114 | 114 | #ifdef CONFIG_CONTEXT_TRACKING |
| @@ -143,7 +143,7 @@ skip_context_tracking: · 28 unmodified lines | ||
| 143 | 143 | li t0, EXC_BREAKPOINT |
| 144 | 144 | beq s4, t0, 1f |
| 145 | 145 | #ifdef CONFIG_TRACE_IRQFLAGS |
| 146 | − call trace_hardirqs_on | |
| 146 | + call __trace_hardirqs_on | |
| 147 | 147 | #endif |
| 148 | 148 | csrs CSR_STATUS, SR_IE |
| 149 | 149 | |
| @@ -234,7 +234,7 @@ ret_from_exception: · 84 unmodified lines | ||
| 234 | 234 | REG_L s0, PT_STATUS(sp) |
| 235 | 235 | csrc CSR_STATUS, SR_IE |
| 236 | 236 | #ifdef CONFIG_TRACE_IRQFLAGS |
| 237 | − call trace_hardirqs_off | |
| 237 | + call __trace_hardirqs_off | |
| 238 | 238 | #endif |
| 239 | 239 | #ifdef CONFIG_RISCV_M_MODE |
| 240 | 240 | /* the MPP value is too large to be used as an immediate arg for addi */ |
| @@ -270,10 +270,10 @@ restore_all: · 29 unmodified lines | ||
| 270 | 270 | REG_L s1, PT_STATUS(sp) |
| 271 | 271 | andi t0, s1, SR_PIE |
| 272 | 272 | beqz t0, 1f |
| 273 | − call trace_hardirqs_on | |
| 273 | + call __trace_hardirqs_on | |
| 274 | 274 | j 2f |
| 275 | 275 | 1: |
| 276 | − call trace_hardirqs_off | |
| 276 | + call __trace_hardirqs_off | |
| 277 | 277 | 2: |
| 278 | 278 | #endif |
| 279 | 279 | REG_L a0, PT_STATUS(sp) |
| @@ -22,14 +22,13 @@ · 21 unmodified lines | ||
| 22 | 22 | add \reg, \reg, t0 |
| 23 | 23 | .endm |
| 24 | 24 | .macro XIP_FIXUP_FLASH_OFFSET reg |
| 25 | − la t1, __data_loc | |
| 26 | − li t0, XIP_OFFSET_MASK | |
| 27 | − and t1, t1, t0 | |
| 28 | − li t1, XIP_OFFSET | |
| 29 | − sub t0, t0, t1 | |
| 30 | − sub \reg, \reg, t0 | |
| 25 | + la t0, __data_loc | |
| 26 | + REG_L t1, _xip_phys_offset | |
| 27 | + sub \reg, \reg, t1 | |
| 28 | + add \reg, \reg, t0 | |
| 31 | 29 | .endm |
| 32 | 30 | _xip_fixup: .dword CONFIG_PHYS_RAM_BASE - CONFIG_XIP_PHYS_ADDR - XIP_OFFSET |
| 31 | +_xip_phys_offset: .dword CONFIG_XIP_PHYS_ADDR + XIP_OFFSET | |
| 33 | 32 | #else |
| 34 | 33 | .macro XIP_FIXUP_OFFSET reg |
| 35 | 34 | .endm |
| @@ -5,6 +5,7 @@ · 4 unmodified lines | ||
| 5 | 5 | * Copyright (c) 2020 Western Digital Corporation or its affiliates. |
| 6 | 6 | */ |
| 7 | 7 | |
| 8 | +#include <linux/bits.h> | |
| 8 | 9 | #include <linux/init.h> |
| 9 | 10 | #include <linux/pm.h> |
| 10 | 11 | #include <linux/reboot.h> |
| @@ -85,7 +86,7 @@ static unsigned long __sbi_v01_cpumask_to_hartmask(const struct cpumask *cpu_mas · 74 unmodified lines | ||
| 85 | 86 | pr_warn("Unable to send any request to hartid > BITS_PER_LONG for SBI v0.1\n"); |
| 86 | 87 | break; |
| 87 | 88 | } |
| 88 | − hmask |= 1 << hartid; | |
| 89 | + hmask |= BIT(hartid); | |
| 89 | 90 | } |
| 90 | 91 | |
| 91 | 92 | return hmask; |
| @@ -160,7 +161,7 @@ static int __sbi_send_ipi_v01(const struct cpumask *cpu_mask) · 68 unmodified lines | ||
| 160 | 161 | { |
| 161 | 162 | unsigned long hart_mask; |
| 162 | 163 | |
| 163 | − if (!cpu_mask) | |
| 164 | + if (!cpu_mask || cpumask_empty(cpu_mask)) | |
| 164 | 165 | cpu_mask = cpu_online_mask; |
| 165 | 166 | hart_mask = __sbi_v01_cpumask_to_hartmask(cpu_mask); |
| 166 | 167 | |
| @@ -176,7 +177,7 @@ static int __sbi_rfence_v01(int fid, const struct cpumask *cpu_mask, · 9 unmodified lines | ||
| 176 | 177 | int result = 0; |
| 177 | 178 | unsigned long hart_mask; |
| 178 | 179 | |
| 179 | − if (!cpu_mask) | |
| 180 | + if (!cpu_mask || cpumask_empty(cpu_mask)) | |
| 180 | 181 | cpu_mask = cpu_online_mask; |
| 181 | 182 | hart_mask = __sbi_v01_cpumask_to_hartmask(cpu_mask); |
| 182 | 183 | |
| @@ -249,26 +250,37 @@ static void __sbi_set_timer_v02(uint64_t stime_value) · 66 unmodified lines | ||
| 249 | 250 | |
| 250 | 251 | static int __sbi_send_ipi_v02(const struct cpumask *cpu_mask) |
| 251 | 252 | { |
| 252 | − unsigned long hartid, cpuid, hmask = 0, hbase = 0; | |
| 253 | + unsigned long hartid, cpuid, hmask = 0, hbase = 0, htop = 0; | |
| 253 | 254 | struct sbiret ret = {0}; |
| 254 | 255 | int result; |
| 255 | 256 | |
| 256 | − if (!cpu_mask) | |
| 257 | + if (!cpu_mask || cpumask_empty(cpu_mask)) | |
| 257 | 258 | cpu_mask = cpu_online_mask; |
| 258 | 259 | |
| 259 | 260 | for_each_cpu(cpuid, cpu_mask) { |
| 260 | 261 | hartid = cpuid_to_hartid_map(cpuid); |
| 261 | − if (hmask && ((hbase + BITS_PER_LONG) <= hartid)) { | |
| 262 | − ret = sbi_ecall(SBI_EXT_IPI, SBI_EXT_IPI_SEND_IPI, | |
| 263 | − hmask, hbase, 0, 0, 0, 0); | |
| 264 | − if (ret.error) | |
| 265 | − goto ecall_failed; | |
| 266 | − hmask = 0; | |
| 267 | − hbase = 0; | |
| 262 | + if (hmask) { | |
| 263 | + if (hartid + BITS_PER_LONG <= htop || | |
| 264 | + hbase + BITS_PER_LONG <= hartid) { | |
| 265 | + ret = sbi_ecall(SBI_EXT_IPI, | |
| 266 | + SBI_EXT_IPI_SEND_IPI, hmask, | |
| 267 | + hbase, 0, 0, 0, 0); | |
| 268 | + if (ret.error) | |
| 269 | + goto ecall_failed; | |
| 270 | + hmask = 0; | |
| 271 | + } else if (hartid < hbase) { | |
| 272 | + /* shift the mask to fit lower hartid */ | |
| 273 | + hmask <<= hbase - hartid; | |
| 274 | + hbase = hartid; | |
| 275 | + } | |
| 268 | 276 | } |
| 269 | − if (!hmask) | |
| 277 | + if (!hmask) { | |
| 270 | 278 | hbase = hartid; |
| 271 | − hmask |= 1UL << (hartid - hbase); | |
| 279 | + htop = hartid; | |
| 280 | + } else if (hartid > htop) { | |
| 281 | + htop = hartid; | |
| 282 | + } | |
| 283 | + hmask |= BIT(hartid - hbase); | |
| 272 | 284 | } |
| 273 | 285 | |
| 274 | 286 | if (hmask) { |
| @@ -344,25 +356,35 @@ static int __sbi_rfence_v02(int fid, const struct cpumask *cpu_mask, · 69 unmodified lines | ||
| 344 | 356 | unsigned long start, unsigned long size, |
| 345 | 357 | unsigned long arg4, unsigned long arg5) |
| 346 | 358 | { |
| 347 | − unsigned long hartid, cpuid, hmask = 0, hbase = 0; | |
| 359 | + unsigned long hartid, cpuid, hmask = 0, hbase = 0, htop = 0; | |
| 348 | 360 | int result; |
| 349 | 361 | |
| 350 | − if (!cpu_mask) | |
| 362 | + if (!cpu_mask || cpumask_empty(cpu_mask)) | |
| 351 | 363 | cpu_mask = cpu_online_mask; |
| 352 | 364 | |
| 353 | 365 | for_each_cpu(cpuid, cpu_mask) { |
| 354 | 366 | hartid = cpuid_to_hartid_map(cpuid); |
| 355 | − if (hmask && ((hbase + BITS_PER_LONG) <= hartid)) { | |
| 356 | − result = __sbi_rfence_v02_call(fid, hmask, hbase, | |
| 357 | − start, size, arg4, arg5); | |
| 358 | − if (result) | |
| 359 | − return result; | |
| 360 | − hmask = 0; | |
| 361 | − hbase = 0; | |
| 367 | + if (hmask) { | |
| 368 | + if (hartid + BITS_PER_LONG <= htop || | |
| 369 | + hbase + BITS_PER_LONG <= hartid) { | |
| 370 | + result = __sbi_rfence_v02_call(fid, hmask, | |
| 371 | + hbase, start, size, arg4, arg5); | |
| 372 | + if (result) | |
| 373 | + return result; | |
| 374 | + hmask = 0; | |
| 375 | + } else if (hartid < hbase) { | |
| 376 | + /* shift the mask to fit lower hartid */ | |
| 377 | + hmask <<= hbase - hartid; | |
| 378 | + hbase = hartid; | |
| 379 | + } | |
| 362 | 380 | } |
| 363 | − if (!hmask) | |
| 381 | + if (!hmask) { | |
| 364 | 382 | hbase = hartid; |
| 365 | − hmask |= 1UL << (hartid - hbase); | |
| 383 | + htop = hartid; | |
| 384 | + } else if (hartid > htop) { | |
| 385 | + htop = hartid; | |
| 386 | + } | |
| 387 | + hmask |= BIT(hartid - hbase); | |
| 366 | 388 | } |
| 367 | 389 | |
| 368 | 390 | if (hmask) { |
| @@ -22,15 +22,16 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs, · 21 unmodified lines | ||
| 22 | 22 | bool (*fn)(void *, unsigned long), void *arg) |
| 23 | 23 | { |
| 24 | 24 | unsigned long fp, sp, pc; |
| 25 | + int level = 0; | |
| 25 | 26 | |
| 26 | 27 | if (regs) { |
| 27 | 28 | fp = frame_pointer(regs); |
| 28 | 29 | sp = user_stack_pointer(regs); |
| 29 | 30 | pc = instruction_pointer(regs); |
| 30 | 31 | } else if (task == NULL || task == current) { |
| 31 | − fp = (unsigned long)__builtin_frame_address(1); | |
| 32 | − sp = (unsigned long)__builtin_frame_address(0); | |
| 33 | − pc = (unsigned long)__builtin_return_address(0); | |
| 32 | + fp = (unsigned long)__builtin_frame_address(0); | |
| 33 | + sp = sp_in_global; | |
| 34 | + pc = (unsigned long)walk_stackframe; | |
| 34 | 35 | } else { |
| 35 | 36 | /* task blocked in __switch_to */ |
| 36 | 37 | fp = task->thread.s[0]; |
| @@ -42,7 +43,7 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs, · 5 unmodified lines | ||
| 42 | 43 | unsigned long low, high; |
| 43 | 44 | struct stackframe *frame; |
| 44 | 45 | |
| 45 | − if (unlikely(!__kernel_text_address(pc) || !fn(arg, pc))) | |
| 46 | + if (unlikely(!__kernel_text_address(pc) || (level++ >= 1 && !fn(arg, pc)))) | |
| 46 | 47 | break; |
| 47 | 48 | |
| 48 | 49 | /* Validate frame pointer */ |
| @@ -0,0 +1,27 @@ | ||
| 1 | +// SPDX-License-Identifier: GPL-2.0 | |
| 2 | +/* | |
| 3 | + * Copyright (C) 2022 Changbin Du <changbin.du@gmail.com> | |
| 4 | + */ | |
| 5 | + | |
| 6 | +#include <linux/irqflags.h> | |
| 7 | +#include <linux/kprobes.h> | |
| 8 | +#include "trace_irq.h" | |
| 9 | + | |
| 10 | +/* | |
| 11 | + * trace_hardirqs_on/off require the caller to setup frame pointer properly. | |
| 12 | + * Otherwise, CALLER_ADDR1 might trigger an pagging exception in kernel. | |
| 13 | + * Here we add one extra level so they can be safely called by low | |
| 14 | + * level entry code which $fp is used for other purpose. | |
| 15 | + */ | |
| 16 | + | |
| 17 | +void __trace_hardirqs_on(void) | |
| 18 | +{ | |
| 19 | + trace_hardirqs_on(); | |
| 20 | +} | |
| 21 | +NOKPROBE_SYMBOL(__trace_hardirqs_on); | |
| 22 | + | |
| 23 | +void __trace_hardirqs_off(void) | |
| 24 | +{ | |
| 25 | + trace_hardirqs_off(); | |
| 26 | +} | |
| 27 | +NOKPROBE_SYMBOL(__trace_hardirqs_off); | |
| @@ -0,0 +1,11 @@ | ||
| 1 | +/* SPDX-License-Identifier: GPL-2.0 */ | |
| 2 | +/* | |
| 3 | + * Copyright (C) 2022 Changbin Du <changbin.du@gmail.com> | |
| 4 | + */ | |
| 5 | +#ifndef __TRACE_IRQ_H | |
| 6 | +#define __TRACE_IRQ_H | |
| 7 | + | |
| 8 | +void __trace_hardirqs_on(void); | |
| 9 | +void __trace_hardirqs_off(void); | |
| 10 | + | |
| 11 | +#endif /* __TRACE_IRQ_H */ | |
| @@ -90,6 +90,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) · 89 unmodified lines | ||
| 90 | 90 | int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) |
| 91 | 91 | { |
| 92 | 92 | struct kvm_cpu_context *cntx; |
| 93 | + struct kvm_vcpu_csr *reset_csr = &vcpu->arch.guest_reset_csr; | |
| 93 | 94 | |
| 94 | 95 | /* Mark this VCPU never ran */ |
| 95 | 96 | vcpu->arch.ran_atleast_once = false; |
| @@ -106,6 +107,9 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · 10 unmodified lines | ||
| 106 | 107 | cntx->hstatus |= HSTATUS_SPVP; |
| 107 | 108 | cntx->hstatus |= HSTATUS_SPV; |
| 108 | 109 | |
| 110 | + /* By default, make CY, TM, and IR counters accessible in VU mode */ | |
| 111 | + reset_csr->scounteren = 0x7; | |
| 112 | + | |
| 109 | 113 | /* Setup VCPU timer */ |
| 110 | 114 | kvm_riscv_vcpu_timer_init(vcpu); |
| 111 | 115 | |
| @@ -699,6 +703,20 @@ static void kvm_riscv_update_hvip(struct kvm_vcpu *vcpu) · 587 unmodified lines | ||
| 699 | 703 | csr_write(CSR_HVIP, csr->hvip); |
| 700 | 704 | } |
| 701 | 705 | |
| 706 | +/* | |
| 707 | + * Actually run the vCPU, entering an RCU extended quiescent state (EQS) while | |
| 708 | + * the vCPU is running. | |
| 709 | + * | |
| 710 | + * This must be noinstr as instrumentation may make use of RCU, and this is not | |
| 711 | + * safe during the EQS. | |
| 712 | + */ | |
| 713 | +static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu) | |
| 714 | +{ | |
| 715 | + guest_state_enter_irqoff(); | |
| 716 | + __kvm_riscv_switch_to(&vcpu->arch); | |
| 717 | + guest_state_exit_irqoff(); | |
| 718 | +} | |
| 719 | + | |
| 702 | 720 | int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) |
| 703 | 721 | { |
| 704 | 722 | int ret; |
| @@ -790,9 +808,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 85 unmodified lines | ||
| 790 | 808 | continue; |
| 791 | 809 | } |
| 792 | 810 | |
| 793 | − guest_enter_irqoff(); | |
| 811 | + guest_timing_enter_irqoff(); | |
| 794 | 812 | |
| 795 | − __kvm_riscv_switch_to(&vcpu->arch); | |
| 813 | + kvm_riscv_vcpu_enter_exit(vcpu); | |
| 796 | 814 | |
| 797 | 815 | vcpu->mode = OUTSIDE_GUEST_MODE; |
| 798 | 816 | vcpu->stat.exits++; |
| @@ -812,25 +830,21 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 13 unmodified lines | ||
| 812 | 830 | kvm_riscv_vcpu_sync_interrupts(vcpu); |
| 813 | 831 | |
| 814 | 832 | /* |
| 815 | − * We may have taken a host interrupt in VS/VU-mode (i.e. | |
| 816 | − * while executing the guest). This interrupt is still | |
| 817 | − * pending, as we haven't serviced it yet! | |
| 833 | + * We must ensure that any pending interrupts are taken before | |
| 834 | + * we exit guest timing so that timer ticks are accounted as | |
| 835 | + * guest time. Transiently unmask interrupts so that any | |
| 836 | + * pending interrupts are taken. | |
| 818 | 837 | * |
| 819 | − * We're now back in HS-mode with interrupts disabled | |
| 820 | − * so enabling the interrupts now will have the effect | |
| 821 | − * of taking the interrupt again, in HS-mode this time. | |
| 838 | + * There's no barrier which ensures that pending interrupts are | |
| 839 | + * recognised, so we just hope that the CPU takes any pending | |
| 840 | + * interrupts between the enable and disable. | |
| 822 | 841 | */ |
| 823 | 842 | local_irq_enable(); |
| 843 | + local_irq_disable(); | |
| 824 | 844 | |
| 825 | − /* | |
| 826 | − * We do local_irq_enable() before calling guest_exit() so | |
| 827 | − * that if a timer interrupt hits while running the guest | |
| 828 | − * we account that tick as being spent in the guest. We | |
| 829 | − * enable preemption after calling guest_exit() so that if | |
| 830 | − * we get preempted we make sure ticks after that is not | |
| 831 | − * counted as guest time. | |
| 832 | − */ | |
| 833 | − guest_exit(); | |
| 845 | + guest_timing_exit_irqoff(); | |
| 846 | + | |
| 847 | + local_irq_enable(); | |
| 834 | 848 | |
| 835 | 849 | preempt_enable(); |
| 836 | 850 | |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | #include <linux/errno.h> |
| 10 | 10 | #include <linux/err.h> |
| 11 | 11 | #include <linux/kvm_host.h> |
| 12 | +#include <linux/version.h> | |
| 12 | 13 | #include <asm/csr.h> |
| 13 | 14 | #include <asm/sbi.h> |
| 14 | 15 | #include <asm/kvm_vcpu_timer.h> |
| @@ -32,7 +33,7 @@ static int kvm_sbi_ext_base_handler(struct kvm_vcpu *vcpu, struct kvm_run *run, · 17 unmodified lines | ||
| 32 | 33 | *out_val = KVM_SBI_IMPID; |
| 33 | 34 | break; |
| 34 | 35 | case SBI_EXT_BASE_GET_IMP_VERSION: |
| 35 | − *out_val = 0; | |
| 36 | + *out_val = LINUX_VERSION_CODE; | |
| 36 | 37 | break; |
| 37 | 38 | case SBI_EXT_BASE_PROBE_EXT: |
| 38 | 39 | if ((cp->a0 >= SBI_EXT_EXPERIMENTAL_START && |
| @@ -33,7 +33,7 @@ static inline void regs_set_gpr(struct pt_regs *regs, unsigned int offset, · 32 unmodified lines | ||
| 33 | 33 | if (unlikely(offset > MAX_REG_OFFSET)) |
| 34 | 34 | return; |
| 35 | 35 | |
| 36 | − if (!offset) | |
| 36 | + if (offset) | |
| 37 | 37 | *(unsigned long *)((unsigned long)regs + offset) = val; |
| 38 | 38 | } |
| 39 | 39 | |
| @@ -43,8 +43,8 @@ static bool ex_handler_uaccess_err_zero(const struct exception_table_entry *ex, · 3 unmodified lines | ||
| 43 | 43 | int reg_err = FIELD_GET(EX_DATA_REG_ERR, ex->data); |
| 44 | 44 | int reg_zero = FIELD_GET(EX_DATA_REG_ZERO, ex->data); |
| 45 | 45 | |
| 46 | − regs_set_gpr(regs, reg_err, -EFAULT); | |
| 47 | − regs_set_gpr(regs, reg_zero, 0); | |
| 46 | + regs_set_gpr(regs, reg_err * sizeof(unsigned long), -EFAULT); | |
| 47 | + regs_set_gpr(regs, reg_zero * sizeof(unsigned long), 0); | |
| 48 | 48 | |
| 49 | 49 | regs->epc = get_ex_fixup(ex); |
| 50 | 50 | return true; |
| @@ -232,6 +232,7 @@ static pmd_t __maybe_unused early_dtb_pmd[PTRS_PER_PMD] __initdata __aligned(PAG · 231 unmodified lines | ||
| 232 | 232 | |
| 233 | 233 | #ifdef CONFIG_XIP_KERNEL |
| 234 | 234 | #define pt_ops (*(struct pt_alloc_ops *)XIP_FIXUP(&pt_ops)) |
| 235 | +#define riscv_pfn_base (*(unsigned long *)XIP_FIXUP(&riscv_pfn_base)) | |
| 235 | 236 | #define trampoline_pg_dir ((pgd_t *)XIP_FIXUP(trampoline_pg_dir)) |
| 236 | 237 | #define fixmap_pte ((pte_t *)XIP_FIXUP(fixmap_pte)) |
| 237 | 238 | #define early_pg_dir ((pgd_t *)XIP_FIXUP(early_pg_dir)) |
| @@ -522,6 +523,7 @@ static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size) · 284 unmodified lines | ||
| 522 | 523 | } |
| 523 | 524 | |
| 524 | 525 | #ifdef CONFIG_XIP_KERNEL |
| 526 | +#define phys_ram_base (*(phys_addr_t *)XIP_FIXUP(&phys_ram_base)) | |
| 525 | 527 | extern char _xiprom[], _exiprom[], __data_loc; |
| 526 | 528 | |
| 527 | 529 | /* called from head.S with MMU off */ |
| @@ -4667,6 +4667,8 @@ static long kvm_s390_guest_sida_op(struct kvm_vcpu *vcpu, · 4666 unmodified lines | ||
| 4667 | 4667 | return -EINVAL; |
| 4668 | 4668 | if (mop->size + mop->sida_offset > sida_size(vcpu->arch.sie_block)) |
| 4669 | 4669 | return -E2BIG; |
| 4670 | + if (!kvm_s390_pv_cpu_is_protected(vcpu)) | |
| 4671 | + return -EINVAL; | |
| 4670 | 4672 | |
| 4671 | 4673 | switch (mop->op) { |
| 4672 | 4674 | case KVM_S390_MEMOP_SIDA_READ: |
| @@ -5,9 +5,6 @@ · 4 unmodified lines | ||
| 5 | 5 | |
| 6 | 6 | #include "test_modules.h" |
| 7 | 7 | |
| 8 | −#define DECLARE_RETURN(i) int test_modules_return_ ## i(void) | |
| 9 | −REPEAT_10000(DECLARE_RETURN); | |
| 10 | − | |
| 11 | 8 | /* |
| 12 | 9 | * Test that modules with many relocations are loaded properly. |
| 13 | 10 | */ |
| @@ -47,4 +47,7 @@ · 46 unmodified lines | ||
| 47 | 47 | __REPEAT_10000_1(f, 8); \ |
| 48 | 48 | __REPEAT_10000_1(f, 9) |
| 49 | 49 | |
| 50 | +#define DECLARE_RETURN(i) int test_modules_return_ ## i(void) | |
| 51 | +REPEAT_10000(DECLARE_RETURN); | |
| 52 | + | |
| 50 | 53 | #endif |
| @@ -18,12 +18,12 @@ · 17 unmodified lines | ||
| 18 | 18 | static int crypto_blake2s_update_x86(struct shash_desc *desc, |
| 19 | 19 | const u8 *in, unsigned int inlen) |
| 20 | 20 | { |
| 21 | − return crypto_blake2s_update(desc, in, inlen, blake2s_compress); | |
| 21 | + return crypto_blake2s_update(desc, in, inlen, false); | |
| 22 | 22 | } |
| 23 | 23 | |
| 24 | 24 | static int crypto_blake2s_final_x86(struct shash_desc *desc, u8 *out) |
| 25 | 25 | { |
| 26 | − return crypto_blake2s_final(desc, out, blake2s_compress); | |
| 26 | + return crypto_blake2s_final(desc, out, false); | |
| 27 | 27 | } |
| 28 | 28 | |
| 29 | 29 | #define BLAKE2S_ALG(name, driver_name, digest_size) \ |
| @@ -4703,6 +4703,19 @@ static __initconst const struct x86_pmu intel_pmu = { · 4702 unmodified lines | ||
| 4703 | 4703 | .lbr_read = intel_pmu_lbr_read_64, |
| 4704 | 4704 | .lbr_save = intel_pmu_lbr_save, |
| 4705 | 4705 | .lbr_restore = intel_pmu_lbr_restore, |
| 4706 | + | |
| 4707 | + /* | |
| 4708 | + * SMM has access to all 4 rings and while traditionally SMM code only | |
| 4709 | + * ran in CPL0, 2021-era firmware is starting to make use of CPL3 in SMM. | |
| 4710 | + * | |
| 4711 | + * Since the EVENTSEL.{USR,OS} CPL filtering makes no distinction | |
| 4712 | + * between SMM or not, this results in what should be pure userspace | |
| 4713 | + * counters including SMM data. | |
| 4714 | + * | |
| 4715 | + * This is a clear privilege issue, therefore globally disable | |
| 4716 | + * counting SMM by default. | |
| 4717 | + */ | |
| 4718 | + .attr_freeze_on_smi = 1, | |
| 4706 | 4719 | }; |
| 4707 | 4720 | |
| 4708 | 4721 | static __init void intel_clovertown_quirk(void) |
| @@ -897,8 +897,9 @@ static void pt_handle_status(struct pt *pt) · 896 unmodified lines | ||
| 897 | 897 | * means we are already losing data; need to let the decoder |
| 898 | 898 | * know. |
| 899 | 899 | */ |
| 900 | − if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries) || | |
| 901 | − buf->output_off == pt_buffer_region_size(buf)) { | |
| 900 | + if (!buf->single && | |
| 901 | + (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries) || | |
| 902 | + buf->output_off == pt_buffer_region_size(buf))) { | |
| 902 | 903 | perf_aux_output_flag(&pt->handle, |
| 903 | 904 | PERF_AUX_FLAG_TRUNCATED); |
| 904 | 905 | advance++; |
| @@ -22,7 +22,7 @@ · 21 unmodified lines | ||
| 22 | 22 | |
| 23 | 23 | #ifdef CONFIG_DEBUG_BUGVERBOSE |
| 24 | 24 | |
| 25 | −#define _BUG_FLAGS(ins, flags) \ | |
| 25 | +#define _BUG_FLAGS(ins, flags, extra) \ | |
| 26 | 26 | do { \ |
| 27 | 27 | asm_inline volatile("1:\t" ins "\n" \ |
| 28 | 28 | ".pushsection __bug_table,\"aw\"\n" \ |
| @@ -31,7 +31,8 @@ do { \ · 2 unmodified lines | ||
| 31 | 31 | "\t.word %c1" "\t# bug_entry::line\n" \ |
| 32 | 32 | "\t.word %c2" "\t# bug_entry::flags\n" \ |
| 33 | 33 | "\t.org 2b+%c3\n" \ |
| 34 | − ".popsection" \ | |
| 34 | + ".popsection\n" \ | |
| 35 | + extra \ | |
| 35 | 36 | : : "i" (__FILE__), "i" (__LINE__), \ |
| 36 | 37 | "i" (flags), \ |
| 37 | 38 | "i" (sizeof(struct bug_entry))); \ |
| @@ -39,14 +40,15 @@ do { \ · 1 unmodified line | ||
| 39 | 40 | |
| 40 | 41 | #else /* !CONFIG_DEBUG_BUGVERBOSE */ |
| 41 | 42 | |
| 42 | −#define _BUG_FLAGS(ins, flags) \ | |
| 43 | +#define _BUG_FLAGS(ins, flags, extra) \ | |
| 43 | 44 | do { \ |
| 44 | 45 | asm_inline volatile("1:\t" ins "\n" \ |
| 45 | 46 | ".pushsection __bug_table,\"aw\"\n" \ |
| 46 | 47 | "2:\t" __BUG_REL(1b) "\t# bug_entry::bug_addr\n" \ |
| 47 | 48 | "\t.word %c0" "\t# bug_entry::flags\n" \ |
| 48 | 49 | "\t.org 2b+%c1\n" \ |
| 49 | − ".popsection" \ | |
| 50 | + ".popsection\n" \ | |
| 51 | + extra \ | |
| 50 | 52 | : : "i" (flags), \ |
| 51 | 53 | "i" (sizeof(struct bug_entry))); \ |
| 52 | 54 | } while (0) |
| @@ -55,7 +57,7 @@ do { \ · 2 unmodified lines | ||
| 55 | 57 | |
| 56 | 58 | #else |
| 57 | 59 | |
| 58 | −#define _BUG_FLAGS(ins, flags) asm volatile(ins) | |
| 60 | +#define _BUG_FLAGS(ins, flags, extra) asm volatile(ins) | |
| 59 | 61 | |
| 60 | 62 | #endif /* CONFIG_GENERIC_BUG */ |
| 61 | 63 | |
| @@ -63,8 +65,8 @@ do { \ · 1 unmodified line | ||
| 63 | 65 | #define BUG() \ |
| 64 | 66 | do { \ |
| 65 | 67 | instrumentation_begin(); \ |
| 66 | − _BUG_FLAGS(ASM_UD2, 0); \ | |
| 67 | − unreachable(); \ | |
| 68 | + _BUG_FLAGS(ASM_UD2, 0, ""); \ | |
| 69 | + __builtin_unreachable(); \ | |
| 68 | 70 | } while (0) |
| 69 | 71 | |
| 70 | 72 | /* |
| @@ -75,9 +77,9 @@ do { \ · 4 unmodified lines | ||
| 75 | 77 | */ |
| 76 | 78 | #define __WARN_FLAGS(flags) \ |
| 77 | 79 | do { \ |
| 80 | + __auto_type f = BUGFLAG_WARNING|(flags); \ | |
| 78 | 81 | instrumentation_begin(); \ |
| 79 | − _BUG_FLAGS(ASM_UD2, BUGFLAG_WARNING|(flags)); \ | |
| 80 | − annotate_reachable(); \ | |
| 82 | + _BUG_FLAGS(ASM_UD2, f, ASM_REACHABLE); \ | |
| 81 | 83 | instrumentation_end(); \ |
| 82 | 84 | } while (0) |
| 83 | 85 | |
| @@ -82,7 +82,7 @@ KVM_X86_OP_NULL(guest_apic_has_interrupt) · 81 unmodified lines | ||
| 82 | 82 | KVM_X86_OP(load_eoi_exitmap) |
| 83 | 83 | KVM_X86_OP(set_virtual_apic_mode) |
| 84 | 84 | KVM_X86_OP_NULL(set_apic_access_page_addr) |
| 85 | −KVM_X86_OP(deliver_posted_interrupt) | |
| 85 | +KVM_X86_OP(deliver_interrupt) | |
| 86 | 86 | KVM_X86_OP_NULL(sync_pir_to_irr) |
| 87 | 87 | KVM_X86_OP(set_tss_addr) |
| 88 | 88 | KVM_X86_OP(set_identity_map_addr) |
| @@ -703,7 +703,6 @@ struct kvm_vcpu_arch { · 702 unmodified lines | ||
| 703 | 703 | struct fpu_guest guest_fpu; |
| 704 | 704 | |
| 705 | 705 | u64 xcr0; |
| 706 | − u64 guest_supported_xcr0; | |
| 707 | 706 | |
| 708 | 707 | struct kvm_pio_request pio; |
| 709 | 708 | void *pio_data; |
| @@ -1410,7 +1409,8 @@ struct kvm_x86_ops { · 700 unmodified lines | ||
| 1410 | 1409 | void (*load_eoi_exitmap)(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap); |
| 1411 | 1410 | void (*set_virtual_apic_mode)(struct kvm_vcpu *vcpu); |
| 1412 | 1411 | void (*set_apic_access_page_addr)(struct kvm_vcpu *vcpu); |
| 1413 | − int (*deliver_posted_interrupt)(struct kvm_vcpu *vcpu, int vector); | |
| 1412 | + void (*deliver_interrupt)(struct kvm_lapic *apic, int delivery_mode, | |
| 1413 | + int trig_mode, int vector); | |
| 1414 | 1414 | int (*sync_pir_to_irr)(struct kvm_vcpu *vcpu); |
| 1415 | 1415 | int (*set_tss_addr)(struct kvm *kvm, unsigned int addr); |
| 1416 | 1416 | int (*set_identity_map_addr)(struct kvm *kvm, u64 ident_addr); |
| @@ -476,6 +476,7 @@ · 475 unmodified lines | ||
| 476 | 476 | #define MSR_AMD64_ICIBSEXTDCTL 0xc001103c |
| 477 | 477 | #define MSR_AMD64_IBSOPDATA4 0xc001103d |
| 478 | 478 | #define MSR_AMD64_IBS_REG_COUNT_MAX 8 /* includes MSR_AMD64_IBSBRTARGET */ |
| 479 | +#define MSR_AMD64_SVM_AVIC_DOORBELL 0xc001011b | |
| 479 | 480 | #define MSR_AMD64_VM_PAGE_FLUSH 0xc001011e |
| 480 | 481 | #define MSR_AMD64_SEV_ES_GHCB 0xc0010130 |
| 481 | 482 | #define MSR_AMD64_SEV 0xc0010131 |
| @@ -220,6 +220,42 @@ struct __attribute__ ((__packed__)) vmcb_control_area { · 219 unmodified lines | ||
| 220 | 220 | #define SVM_NESTED_CTL_SEV_ENABLE BIT(1) |
| 221 | 221 | #define SVM_NESTED_CTL_SEV_ES_ENABLE BIT(2) |
| 222 | 222 | |
| 223 | + | |
| 224 | +/* AVIC */ | |
| 225 | +#define AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK (0xFF) | |
| 226 | +#define AVIC_LOGICAL_ID_ENTRY_VALID_BIT 31 | |
| 227 | +#define AVIC_LOGICAL_ID_ENTRY_VALID_MASK (1 << 31) | |
| 228 | + | |
| 229 | +#define AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK (0xFFULL) | |
| 230 | +#define AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK (0xFFFFFFFFFFULL << 12) | |
| 231 | +#define AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK (1ULL << 62) | |
| 232 | +#define AVIC_PHYSICAL_ID_ENTRY_VALID_MASK (1ULL << 63) | |
| 233 | +#define AVIC_PHYSICAL_ID_TABLE_SIZE_MASK (0xFF) | |
| 234 | + | |
| 235 | +#define AVIC_DOORBELL_PHYSICAL_ID_MASK (0xFF) | |
| 236 | + | |
| 237 | +#define AVIC_UNACCEL_ACCESS_WRITE_MASK 1 | |
| 238 | +#define AVIC_UNACCEL_ACCESS_OFFSET_MASK 0xFF0 | |
| 239 | +#define AVIC_UNACCEL_ACCESS_VECTOR_MASK 0xFFFFFFFF | |
| 240 | + | |
| 241 | +enum avic_ipi_failure_cause { | |
| 242 | + AVIC_IPI_FAILURE_INVALID_INT_TYPE, | |
| 243 | + AVIC_IPI_FAILURE_TARGET_NOT_RUNNING, | |
| 244 | + AVIC_IPI_FAILURE_INVALID_TARGET, | |
| 245 | + AVIC_IPI_FAILURE_INVALID_BACKING_PAGE, | |
| 246 | +}; | |
| 247 | + | |
| 248 | + | |
| 249 | +/* | |
| 250 | + * 0xff is broadcast, so the max index allowed for physical APIC ID | |
| 251 | + * table is 0xfe. APIC IDs above 0xff are reserved. | |
| 252 | + */ | |
| 253 | +#define AVIC_MAX_PHYSICAL_ID_COUNT 0xff | |
| 254 | + | |
| 255 | +#define AVIC_HPA_MASK ~((0xFFFULL << 52) | 0xFFF) | |
| 256 | +#define VMCB_AVIC_APIC_BAR_MASK 0xFFFFFFFFFF000ULL | |
| 257 | + | |
| 258 | + | |
| 223 | 259 | struct vmcb_seg { |
| 224 | 260 | u16 selector; |
| 225 | 261 | u16 attrib; |
| @@ -100,6 +100,13 @@ · 99 unmodified lines | ||
| 100 | 100 | /* Memory mapped from other domains has valid IOMMU entries */ |
| 101 | 101 | #define XEN_HVM_CPUID_IOMMU_MAPPINGS (1u << 2) |
| 102 | 102 | #define XEN_HVM_CPUID_VCPU_ID_PRESENT (1u << 3) /* vcpu id is present in EBX */ |
| 103 | +#define XEN_HVM_CPUID_DOMID_PRESENT (1u << 4) /* domid is present in ECX */ | |
| 104 | +/* | |
| 105 | + * Bits 55:49 from the IO-APIC RTE and bits 11:5 from the MSI address can be | |
| 106 | + * used to store high bits for the Destination ID. This expands the Destination | |
| 107 | + * ID field from 8 to 15 bits, allowing to target APIC IDs up 32768. | |
| 108 | + */ | |
| 109 | +#define XEN_HVM_CPUID_EXT_DEST_ID (1u << 5) | |
| 103 | 110 | |
| 104 | 111 | /* |
| 105 | 112 | * Leaf 6 (0x40000x05) |
| @@ -43,20 +43,6 @@ static inline uint32_t xen_cpuid_base(void) · 42 unmodified lines | ||
| 43 | 43 | return hypervisor_cpuid_base("XenVMMXenVMM", 2); |
| 44 | 44 | } |
| 45 | 45 | |
| 46 | −#ifdef CONFIG_XEN | |
| 47 | −extern bool __init xen_hvm_need_lapic(void); | |
| 48 | − | |
| 49 | −static inline bool __init xen_x2apic_para_available(void) | |
| 50 | −{ | |
| 51 | − return xen_hvm_need_lapic(); | |
| 52 | −} | |
| 53 | −#else | |
| 54 | −static inline bool __init xen_x2apic_para_available(void) | |
| 55 | −{ | |
| 56 | − return (xen_cpuid_base() != 0); | |
| 57 | −} | |
| 58 | −#endif | |
| 59 | − | |
| 60 | 46 | struct pci_dev; |
| 61 | 47 | |
| 62 | 48 | #ifdef CONFIG_XEN_PV_DOM0 |
| @@ -410,6 +410,8 @@ void sgx_encl_release(struct kref *ref) · 409 unmodified lines | ||
| 410 | 410 | } |
| 411 | 411 | |
| 412 | 412 | kfree(entry); |
| 413 | + /* Invoke scheduler to prevent soft lockups. */ | |
| 414 | + cond_resched(); | |
| 413 | 415 | } |
| 414 | 416 | |
| 415 | 417 | xa_destroy(&encl->page_array); |
| @@ -344,10 +344,8 @@ static void sgx_reclaim_pages(void) · 343 unmodified lines | ||
| 344 | 344 | { |
| 345 | 345 | struct sgx_epc_page *chunk[SGX_NR_TO_SCAN]; |
| 346 | 346 | struct sgx_backing backing[SGX_NR_TO_SCAN]; |
| 347 | − struct sgx_epc_section *section; | |
| 348 | 347 | struct sgx_encl_page *encl_page; |
| 349 | 348 | struct sgx_epc_page *epc_page; |
| 350 | − struct sgx_numa_node *node; | |
| 351 | 349 | pgoff_t page_index; |
| 352 | 350 | int cnt = 0; |
| 353 | 351 | int ret; |
| @@ -418,13 +416,7 @@ skip: · 64 unmodified lines | ||
| 418 | 416 | kref_put(&encl_page->encl->refcount, sgx_encl_release); |
| 419 | 417 | epc_page->flags &= ~SGX_EPC_PAGE_RECLAIMER_TRACKED; |
| 420 | 418 | |
| 421 | − section = &sgx_epc_sections[epc_page->section]; | |
| 422 | − node = section->node; | |
| 423 | − | |
| 424 | − spin_lock(&node->lock); | |
| 425 | − list_add_tail(&epc_page->list, &node->free_page_list); | |
| 426 | − spin_unlock(&node->lock); | |
| 427 | − atomic_long_inc(&sgx_nr_free_pages); | |
| 419 | + sgx_free_epc_page(epc_page); | |
| 428 | 420 | } |
| 429 | 421 | } |
| 430 | 422 | |
| @@ -91,11 +91,9 @@ int xfpregs_set(struct task_struct *target, const struct user_regset *regset, · 90 unmodified lines | ||
| 91 | 91 | const void *kbuf, const void __user *ubuf) |
| 92 | 92 | { |
| 93 | 93 | struct fpu *fpu = &target->thread.fpu; |
| 94 | − struct user32_fxsr_struct newstate; | |
| 94 | + struct fxregs_state newstate; | |
| 95 | 95 | int ret; |
| 96 | 96 | |
| 97 | − BUILD_BUG_ON(sizeof(newstate) != sizeof(struct fxregs_state)); | |
| 98 | − | |
| 99 | 97 | if (!cpu_feature_enabled(X86_FEATURE_FXSR)) |
| 100 | 98 | return -ENODEV; |
| 101 | 99 | |
| @@ -116,9 +114,10 @@ int xfpregs_set(struct task_struct *target, const struct user_regset *regset, · 14 unmodified lines | ||
| 116 | 114 | /* Copy the state */ |
| 117 | 115 | memcpy(&fpu->fpstate->regs.fxsave, &newstate, sizeof(newstate)); |
| 118 | 116 | |
| 119 | − /* Clear xmm8..15 */ | |
| 117 | + /* Clear xmm8..15 for 32-bit callers */ | |
| 120 | 118 | BUILD_BUG_ON(sizeof(fpu->__fpstate.regs.fxsave.xmm_space) != 16 * 16); |
| 121 | − memset(&fpu->fpstate->regs.fxsave.xmm_space[8], 0, 8 * 16); | |
| 119 | + if (in_ia32_syscall()) | |
| 120 | + memset(&fpu->fpstate->regs.fxsave.xmm_space[8*4], 0, 8 * 16); | |
| 122 | 121 | |
| 123 | 122 | /* Mark FP and SSE as in use when XSAVE is enabled */ |
| 124 | 123 | if (use_xsave()) |
| @@ -1558,7 +1558,10 @@ static int fpstate_realloc(u64 xfeatures, unsigned int ksize, · 1557 unmodified lines | ||
| 1558 | 1558 | fpregs_restore_userregs(); |
| 1559 | 1559 | |
| 1560 | 1560 | newfps->xfeatures = curfps->xfeatures | xfeatures; |
| 1561 | − newfps->user_xfeatures = curfps->user_xfeatures | xfeatures; | |
| 1561 | + | |
| 1562 | + if (!guest_fpu) | |
| 1563 | + newfps->user_xfeatures = curfps->user_xfeatures | xfeatures; | |
| 1564 | + | |
| 1562 | 1565 | newfps->xfd = curfps->xfd & ~xfeatures; |
| 1563 | 1566 | |
| 1564 | 1567 | /* Do the final updates within the locked region */ |
| @@ -462,19 +462,22 @@ static bool pv_tlb_flush_supported(void) · 461 unmodified lines | ||
| 462 | 462 | { |
| 463 | 463 | return (kvm_para_has_feature(KVM_FEATURE_PV_TLB_FLUSH) && |
| 464 | 464 | !kvm_para_has_hint(KVM_HINTS_REALTIME) && |
| 465 | − kvm_para_has_feature(KVM_FEATURE_STEAL_TIME)); | |
| 465 | + kvm_para_has_feature(KVM_FEATURE_STEAL_TIME) && | |
| 466 | + (num_possible_cpus() != 1)); | |
| 466 | 467 | } |
| 467 | 468 | |
| 468 | 469 | static bool pv_ipi_supported(void) |
| 469 | 470 | { |
| 470 | − return kvm_para_has_feature(KVM_FEATURE_PV_SEND_IPI); | |
| 471 | + return (kvm_para_has_feature(KVM_FEATURE_PV_SEND_IPI) && | |
| 472 | + (num_possible_cpus() != 1)); | |
| 471 | 473 | } |
| 472 | 474 | |
| 473 | 475 | static bool pv_sched_yield_supported(void) |
| 474 | 476 | { |
| 475 | 477 | return (kvm_para_has_feature(KVM_FEATURE_PV_SCHED_YIELD) && |
| 476 | 478 | !kvm_para_has_hint(KVM_HINTS_REALTIME) && |
| 477 | − kvm_para_has_feature(KVM_FEATURE_STEAL_TIME)); | |
| 479 | + kvm_para_has_feature(KVM_FEATURE_STEAL_TIME) && | |
| 480 | + (num_possible_cpus() != 1)); | |
| 478 | 481 | } |
| 479 | 482 | |
| 480 | 483 | #define KVM_IPI_CLUSTER_SIZE (2 * BITS_PER_LONG) |
| @@ -1224,7 +1224,7 @@ static struct user_regset x86_64_regsets[] __ro_after_init = { · 1223 unmodified lines | ||
| 1224 | 1224 | }, |
| 1225 | 1225 | [REGSET_FP] = { |
| 1226 | 1226 | .core_note_type = NT_PRFPREG, |
| 1227 | − .n = sizeof(struct user_i387_struct) / sizeof(long), | |
| 1227 | + .n = sizeof(struct fxregs_state) / sizeof(long), | |
| 1228 | 1228 | .size = sizeof(long), .align = sizeof(long), |
| 1229 | 1229 | .active = regset_xregset_fpregs_active, .regset_get = xfpregs_get, .set = xfpregs_set |
| 1230 | 1230 | }, |
| @@ -1271,7 +1271,7 @@ static struct user_regset x86_32_regsets[] __ro_after_init = { · 40 unmodified lines | ||
| 1271 | 1271 | }, |
| 1272 | 1272 | [REGSET_XFP] = { |
| 1273 | 1273 | .core_note_type = NT_PRXFPREG, |
| 1274 | − .n = sizeof(struct user32_fxsr_struct) / sizeof(u32), | |
| 1274 | + .n = sizeof(struct fxregs_state) / sizeof(u32), | |
| 1275 | 1275 | .size = sizeof(u32), .align = sizeof(u32), |
| 1276 | 1276 | .active = regset_xregset_fpregs_active, .regset_get = xfpregs_get, .set = xfpregs_set |
| 1277 | 1277 | }, |
| @@ -1,5 +1,4 @@ | ||
| 1 | 1 | // SPDX-License-Identifier: GPL-2.0 |
| 2 | −#include <linux/dmi.h> | |
| 3 | 2 | #include <linux/ioport.h> |
| 4 | 3 | #include <asm/e820/api.h> |
| 5 | 4 | |
| @@ -24,31 +23,11 @@ static void resource_clip(struct resource *res, resource_size_t start, · 18 unmodified lines | ||
| 24 | 23 | res->start = end + 1; |
| 25 | 24 | } |
| 26 | 25 | |
| 27 | −/* | |
| 28 | − * Some BIOS-es contain a bug where they add addresses which map to | |
| 29 | − * system RAM in the PCI host bridge window returned by the ACPI _CRS | |
| 30 | − * method, see commit 4dc2287c1805 ("x86: avoid E820 regions when | |
| 31 | − * allocating address space"). To avoid this Linux by default excludes | |
| 32 | − * E820 reservations when allocating addresses since 2010. | |
| 33 | − * In 2019 some systems have shown-up with E820 reservations which cover | |
| 34 | − * the entire _CRS returned PCI host bridge window, causing all attempts | |
| 35 | − * to assign memory to PCI BARs to fail if Linux uses E820 reservations. | |
| 36 | − * | |
| 37 | − * Ideally Linux would fully stop using E820 reservations, but then | |
| 38 | − * the old systems this was added for will regress. | |
| 39 | − * Instead keep the old behavior for old systems, while ignoring the | |
| 40 | − * E820 reservations for any systems from now on. | |
| 41 | − */ | |
| 42 | 26 | static void remove_e820_regions(struct resource *avail) |
| 43 | 27 | { |
| 44 | − int i, year = dmi_get_bios_year(); | |
| 28 | + int i; | |
| 45 | 29 | struct e820_entry *entry; |
| 46 | 30 | |
| 47 | − if (year >= 2018) | |
| 48 | − return; | |
| 49 | − | |
| 50 | − pr_info_once("PCI: Removing E820 reservations from host bridge windows\n"); | |
| 51 | − | |
| 52 | 31 | for (i = 0; i < e820_table->nr_entries; i++) { |
| 53 | 32 | entry = &e820_table->entries[i]; |
| 54 | 33 | |
| @@ -282,6 +282,7 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · 281 unmodified lines | ||
| 282 | 282 | { |
| 283 | 283 | struct kvm_lapic *apic = vcpu->arch.apic; |
| 284 | 284 | struct kvm_cpuid_entry2 *best; |
| 285 | + u64 guest_supported_xcr0; | |
| 285 | 286 | |
| 286 | 287 | best = kvm_find_cpuid_entry(vcpu, 1, 0); |
| 287 | 288 | if (best && apic) { |
| @@ -293,9 +294,11 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · 5 unmodified lines | ||
| 293 | 294 | kvm_apic_set_version(vcpu); |
| 294 | 295 | } |
| 295 | 296 | |
| 296 | − vcpu->arch.guest_supported_xcr0 = | |
| 297 | + guest_supported_xcr0 = | |
| 297 | 298 | cpuid_get_supported_xcr0(vcpu->arch.cpuid_entries, vcpu->arch.cpuid_nent); |
| 298 | 299 | |
| 300 | + vcpu->arch.guest_fpu.fpstate->user_xfeatures = guest_supported_xcr0; | |
| 301 | + | |
| 299 | 302 | kvm_update_pv_runtime(vcpu); |
| 300 | 303 | |
| 301 | 304 | vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu); |
| @@ -554,12 +557,13 @@ void kvm_set_cpu_caps(void) · 252 unmodified lines | ||
| 554 | 557 | ); |
| 555 | 558 | |
| 556 | 559 | kvm_cpu_cap_mask(CPUID_7_0_EBX, |
| 557 | − F(FSGSBASE) | F(SGX) | F(BMI1) | F(HLE) | F(AVX2) | F(SMEP) | | |
| 558 | − F(BMI2) | F(ERMS) | F(INVPCID) | F(RTM) | 0 /*MPX*/ | F(RDSEED) | | |
| 559 | − F(ADX) | F(SMAP) | F(AVX512IFMA) | F(AVX512F) | F(AVX512PF) | | |
| 560 | − F(AVX512ER) | F(AVX512CD) | F(CLFLUSHOPT) | F(CLWB) | F(AVX512DQ) | | |
| 561 | − F(SHA_NI) | F(AVX512BW) | F(AVX512VL) | 0 /*INTEL_PT*/ | |
| 562 | − ); | |
| 560 | + F(FSGSBASE) | F(SGX) | F(BMI1) | F(HLE) | F(AVX2) | | |
| 561 | + F(FDP_EXCPTN_ONLY) | F(SMEP) | F(BMI2) | F(ERMS) | F(INVPCID) | | |
| 562 | + F(RTM) | F(ZERO_FCS_FDS) | 0 /*MPX*/ | F(AVX512F) | | |
| 563 | + F(AVX512DQ) | F(RDSEED) | F(ADX) | F(SMAP) | F(AVX512IFMA) | | |
| 564 | + F(CLFLUSHOPT) | F(CLWB) | 0 /*INTEL_PT*/ | F(AVX512PF) | | |
| 565 | + F(AVX512ER) | F(AVX512CD) | F(SHA_NI) | F(AVX512BW) | | |
| 566 | + F(AVX512VL)); | |
| 563 | 567 | |
| 564 | 568 | kvm_cpu_cap_mask(CPUID_7_ECX, |
| 565 | 569 | F(AVX512VBMI) | F(LA57) | F(PKU) | 0 /*OSPKE*/ | F(RDPID) | |
| @@ -1096,14 +1096,8 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode, · 1095 unmodified lines | ||
| 1096 | 1096 | apic->regs + APIC_TMR); |
| 1097 | 1097 | } |
| 1098 | 1098 | |
| 1099 | − if (static_call(kvm_x86_deliver_posted_interrupt)(vcpu, vector)) { | |
| 1100 | − kvm_lapic_set_irr(vector, apic); | |
| 1101 | − kvm_make_request(KVM_REQ_EVENT, vcpu); | |
| 1102 | − kvm_vcpu_kick(vcpu); | |
| 1103 | − } else { | |
| 1104 | − trace_kvm_apicv_accept_irq(vcpu->vcpu_id, delivery_mode, | |
| 1105 | − trig_mode, vector); | |
| 1106 | − } | |
| 1099 | + static_call(kvm_x86_deliver_interrupt)(apic, delivery_mode, | |
| 1100 | + trig_mode, vector); | |
| 1107 | 1101 | break; |
| 1108 | 1102 | |
| 1109 | 1103 | case APIC_DM_REMRD: |
| @@ -2312,7 +2306,12 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu) · 1202 unmodified lines | ||
| 2312 | 2306 | apic->irr_pending = true; |
| 2313 | 2307 | apic->isr_count = 1; |
| 2314 | 2308 | } else { |
| 2315 | − apic->irr_pending = (apic_search_irr(apic) != -1); | |
| 2309 | + /* | |
| 2310 | + * Don't clear irr_pending, searching the IRR can race with | |
| 2311 | + * updates from the CPU as APICv is still active from hardware's | |
| 2312 | + * perspective. The flag will be cleared as appropriate when | |
| 2313 | + * KVM injects the interrupt. | |
| 2314 | + */ | |
| 2316 | 2315 | apic->isr_count = count_vectors(apic->regs + APIC_ISR); |
| 2317 | 2316 | } |
| 2318 | 2317 | } |
| @@ -3889,12 +3889,23 @@ static void shadow_page_table_clear_flood(struct kvm_vcpu *vcpu, gva_t addr) · 3888 unmodified lines | ||
| 3889 | 3889 | walk_shadow_page_lockless_end(vcpu); |
| 3890 | 3890 | } |
| 3891 | 3891 | |
| 3892 | +static u32 alloc_apf_token(struct kvm_vcpu *vcpu) | |
| 3893 | +{ | |
| 3894 | + /* make sure the token value is not 0 */ | |
| 3895 | + u32 id = vcpu->arch.apf.id; | |
| 3896 | + | |
| 3897 | + if (id << 12 == 0) | |
| 3898 | + vcpu->arch.apf.id = 1; | |
| 3899 | + | |
| 3900 | + return (vcpu->arch.apf.id++ << 12) | vcpu->vcpu_id; | |
| 3901 | +} | |
| 3902 | + | |
| 3892 | 3903 | static bool kvm_arch_setup_async_pf(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, |
| 3893 | 3904 | gfn_t gfn) |
| 3894 | 3905 | { |
| 3895 | 3906 | struct kvm_arch_async_pf arch; |
| 3896 | 3907 | |
| 3897 | − arch.token = (vcpu->arch.apf.id++ << 12) | vcpu->vcpu_id; | |
| 3908 | + arch.token = alloc_apf_token(vcpu); | |
| 3898 | 3909 | arch.gfn = gfn; |
| 3899 | 3910 | arch.direct_map = vcpu->arch.mmu->direct_map; |
| 3900 | 3911 | arch.cr3 = vcpu->arch.mmu->get_guest_pgd(vcpu); |
| @@ -95,7 +95,7 @@ static void kvm_perf_overflow(struct perf_event *perf_event, · 94 unmodified lines | ||
| 95 | 95 | } |
| 96 | 96 | |
| 97 | 97 | static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, |
| 98 | − unsigned config, bool exclude_user, | |
| 98 | + u64 config, bool exclude_user, | |
| 99 | 99 | bool exclude_kernel, bool intr, |
| 100 | 100 | bool in_tx, bool in_tx_cp) |
| 101 | 101 | { |
| @@ -181,7 +181,8 @@ static int cmp_u64(const void *a, const void *b) · 79 unmodified lines | ||
| 181 | 181 | |
| 182 | 182 | void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel) |
| 183 | 183 | { |
| 184 | − unsigned config, type = PERF_TYPE_RAW; | |
| 184 | + u64 config; | |
| 185 | + u32 type = PERF_TYPE_RAW; | |
| 185 | 186 | struct kvm *kvm = pmc->vcpu->kvm; |
| 186 | 187 | struct kvm_pmu_event_filter *filter; |
| 187 | 188 | bool allow_event = true; |
| @@ -220,7 +221,7 @@ void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel) · 32 unmodified lines | ||
| 220 | 221 | } |
| 221 | 222 | |
| 222 | 223 | if (type == PERF_TYPE_RAW) |
| 223 | − config = eventsel & X86_RAW_EVENT_MASK; | |
| 224 | + config = eventsel & AMD64_RAW_EVENT_MASK; | |
| 224 | 225 | |
| 225 | 226 | if (pmc->current_config == eventsel && pmc_resume_counter(pmc)) |
| 226 | 227 | return; |
| @@ -27,20 +27,6 @@ · 26 unmodified lines | ||
| 27 | 27 | #include "irq.h" |
| 28 | 28 | #include "svm.h" |
| 29 | 29 | |
| 30 | −#define SVM_AVIC_DOORBELL 0xc001011b | |
| 31 | − | |
| 32 | −#define AVIC_HPA_MASK ~((0xFFFULL << 52) | 0xFFF) | |
| 33 | − | |
| 34 | −/* | |
| 35 | − * 0xff is broadcast, so the max index allowed for physical APIC ID | |
| 36 | − * table is 0xfe. APIC IDs above 0xff are reserved. | |
| 37 | − */ | |
| 38 | −#define AVIC_MAX_PHYSICAL_ID_COUNT 255 | |
| 39 | − | |
| 40 | −#define AVIC_UNACCEL_ACCESS_WRITE_MASK 1 | |
| 41 | −#define AVIC_UNACCEL_ACCESS_OFFSET_MASK 0xFF0 | |
| 42 | −#define AVIC_UNACCEL_ACCESS_VECTOR_MASK 0xFFFFFFFF | |
| 43 | − | |
| 44 | 30 | /* AVIC GATAG is encoded using VM and VCPU IDs */ |
| 45 | 31 | #define AVIC_VCPU_ID_BITS 8 |
| 46 | 32 | #define AVIC_VCPU_ID_MASK ((1 << AVIC_VCPU_ID_BITS) - 1) |
| @@ -73,12 +59,6 @@ struct amd_svm_iommu_ir { · 26 unmodified lines | ||
| 73 | 59 | void *data; /* Storing pointer to struct amd_ir_data */ |
| 74 | 60 | }; |
| 75 | 61 | |
| 76 | −enum avic_ipi_failure_cause { | |
| 77 | − AVIC_IPI_FAILURE_INVALID_INT_TYPE, | |
| 78 | − AVIC_IPI_FAILURE_TARGET_NOT_RUNNING, | |
| 79 | − AVIC_IPI_FAILURE_INVALID_TARGET, | |
| 80 | − AVIC_IPI_FAILURE_INVALID_BACKING_PAGE, | |
| 81 | −}; | |
| 82 | 62 | |
| 83 | 63 | /* Note: |
| 84 | 64 | * This function is called from IOMMU driver to notify |
| @@ -289,6 +269,22 @@ static int avic_init_backing_page(struct kvm_vcpu *vcpu) · 204 unmodified lines | ||
| 289 | 269 | return 0; |
| 290 | 270 | } |
| 291 | 271 | |
| 272 | +void avic_ring_doorbell(struct kvm_vcpu *vcpu) | |
| 273 | +{ | |
| 274 | + /* | |
| 275 | + * Note, the vCPU could get migrated to a different pCPU at any point, | |
| 276 | + * which could result in signalling the wrong/previous pCPU. But if | |
| 277 | + * that happens the vCPU is guaranteed to do a VMRUN (after being | |
| 278 | + * migrated) and thus will process pending interrupts, i.e. a doorbell | |
| 279 | + * is not needed (and the spurious one is harmless). | |
| 280 | + */ | |
| 281 | + int cpu = READ_ONCE(vcpu->cpu); | |
| 282 | + | |
| 283 | + if (cpu != get_cpu()) | |
| 284 | + wrmsrl(MSR_AMD64_SVM_AVIC_DOORBELL, kvm_cpu_get_apicid(cpu)); | |
| 285 | + put_cpu(); | |
| 286 | +} | |
| 287 | + | |
| 292 | 288 | static void avic_kick_target_vcpus(struct kvm *kvm, struct kvm_lapic *source, |
| 293 | 289 | u32 icrl, u32 icrh) |
| 294 | 290 | { |
| @@ -304,8 +300,13 @@ static void avic_kick_target_vcpus(struct kvm *kvm, struct kvm_lapic *source, · 9 unmodified lines | ||
| 304 | 300 | kvm_for_each_vcpu(i, vcpu, kvm) { |
| 305 | 301 | if (kvm_apic_match_dest(vcpu, source, icrl & APIC_SHORT_MASK, |
| 306 | 302 | GET_APIC_DEST_FIELD(icrh), |
| 307 | − icrl & APIC_DEST_MASK)) | |
| 308 | − kvm_vcpu_wake_up(vcpu); | |
| 303 | + icrl & APIC_DEST_MASK)) { | |
| 304 | + vcpu->arch.apic->irr_pending = true; | |
| 305 | + svm_complete_interrupt_delivery(vcpu, | |
| 306 | + icrl & APIC_MODE_MASK, | |
| 307 | + icrl & APIC_INT_LEVELTRIG, | |
| 308 | + icrl & APIC_VECTOR_MASK); | |
| 309 | + } | |
| 309 | 310 | } |
| 310 | 311 | } |
| 311 | 312 | |
| @@ -345,8 +346,6 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu) · 33 unmodified lines | ||
| 345 | 346 | avic_kick_target_vcpus(vcpu->kvm, apic, icrl, icrh); |
| 346 | 347 | break; |
| 347 | 348 | case AVIC_IPI_FAILURE_INVALID_TARGET: |
| 348 | − WARN_ONCE(1, "Invalid IPI target: index=%u, vcpu=%d, icr=%#0x:%#0x\n", | |
| 349 | − index, vcpu->vcpu_id, icrh, icrl); | |
| 350 | 349 | break; |
| 351 | 350 | case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE: |
| 352 | 351 | WARN_ONCE(1, "Invalid backing page\n"); |
| @@ -669,52 +668,6 @@ void svm_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) · 316 unmodified lines | ||
| 669 | 668 | return; |
| 670 | 669 | } |
| 671 | 670 | |
| 672 | −int svm_deliver_avic_intr(struct kvm_vcpu *vcpu, int vec) | |
| 673 | −{ | |
| 674 | − if (!vcpu->arch.apicv_active) | |
| 675 | − return -1; | |
| 676 | − | |
| 677 | − kvm_lapic_set_irr(vec, vcpu->arch.apic); | |
| 678 | − | |
| 679 | − /* | |
| 680 | − * Pairs with the smp_mb_*() after setting vcpu->guest_mode in | |
| 681 | − * vcpu_enter_guest() to ensure the write to the vIRR is ordered before | |
| 682 | − * the read of guest_mode, which guarantees that either VMRUN will see | |
| 683 | − * and process the new vIRR entry, or that the below code will signal | |
| 684 | − * the doorbell if the vCPU is already running in the guest. | |
| 685 | − */ | |
| 686 | − smp_mb__after_atomic(); | |
| 687 | − | |
| 688 | − /* | |
| 689 | − * Signal the doorbell to tell hardware to inject the IRQ if the vCPU | |
| 690 | − * is in the guest. If the vCPU is not in the guest, hardware will | |
| 691 | − * automatically process AVIC interrupts at VMRUN. | |
| 692 | − */ | |
| 693 | − if (vcpu->mode == IN_GUEST_MODE) { | |
| 694 | − int cpu = READ_ONCE(vcpu->cpu); | |
| 695 | − | |
| 696 | − /* | |
| 697 | − * Note, the vCPU could get migrated to a different pCPU at any | |
| 698 | − * point, which could result in signalling the wrong/previous | |
| 699 | − * pCPU. But if that happens the vCPU is guaranteed to do a | |
| 700 | − * VMRUN (after being migrated) and thus will process pending | |
| 701 | − * interrupts, i.e. a doorbell is not needed (and the spurious | |
| 702 | − * one is harmless). | |
| 703 | − */ | |
| 704 | − if (cpu != get_cpu()) | |
| 705 | − wrmsrl(SVM_AVIC_DOORBELL, kvm_cpu_get_apicid(cpu)); | |
| 706 | − put_cpu(); | |
| 707 | − } else { | |
| 708 | − /* | |
| 709 | − * Wake the vCPU if it was blocking. KVM will then detect the | |
| 710 | − * pending IRQ when checking if the vCPU has a wake event. | |
| 711 | − */ | |
| 712 | − kvm_vcpu_wake_up(vcpu); | |
| 713 | − } | |
| 714 | − | |
| 715 | − return 0; | |
| 716 | −} | |
| 717 | − | |
| 718 | 671 | bool svm_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu) |
| 719 | 672 | { |
| 720 | 673 | return false; |
| @@ -1457,18 +1457,6 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, · 1456 unmodified lines | ||
| 1457 | 1457 | !__nested_vmcb_check_save(vcpu, &save_cached)) |
| 1458 | 1458 | goto out_free; |
| 1459 | 1459 | |
| 1460 | − /* | |
| 1461 | − * While the nested guest CR3 is already checked and set by | |
| 1462 | − * KVM_SET_SREGS, it was set when nested state was yet loaded, | |
| 1463 | − * thus MMU might not be initialized correctly. | |
| 1464 | − * Set it again to fix this. | |
| 1465 | − */ | |
| 1466 | − | |
| 1467 | − ret = nested_svm_load_cr3(&svm->vcpu, vcpu->arch.cr3, | |
| 1468 | − nested_npt_enabled(svm), false); | |
| 1469 | − if (WARN_ON_ONCE(ret)) | |
| 1470 | − goto out_free; | |
| 1471 | − | |
| 1472 | 1460 | |
| 1473 | 1461 | /* |
| 1474 | 1462 | * All checks done, we can enter guest mode. Userspace provides |
| @@ -1494,6 +1482,20 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, · 19 unmodified lines | ||
| 1494 | 1482 | |
| 1495 | 1483 | svm_switch_vmcb(svm, &svm->nested.vmcb02); |
| 1496 | 1484 | nested_vmcb02_prepare_control(svm); |
| 1485 | + | |
| 1486 | + /* | |
| 1487 | + * While the nested guest CR3 is already checked and set by | |
| 1488 | + * KVM_SET_SREGS, it was set when nested state was yet loaded, | |
| 1489 | + * thus MMU might not be initialized correctly. | |
| 1490 | + * Set it again to fix this. | |
| 1491 | + */ | |
| 1492 | + | |
| 1493 | + ret = nested_svm_load_cr3(&svm->vcpu, vcpu->arch.cr3, | |
| 1494 | + nested_npt_enabled(svm), false); | |
| 1495 | + if (WARN_ON_ONCE(ret)) | |
| 1496 | + goto out_free; | |
| 1497 | + | |
| 1498 | + | |
| 1497 | 1499 | kvm_make_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu); |
| 1498 | 1500 | ret = 0; |
| 1499 | 1501 | out_free: |
| @@ -1585,6 +1585,7 @@ void svm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · 1584 unmodified lines | ||
| 1585 | 1585 | { |
| 1586 | 1586 | struct vcpu_svm *svm = to_svm(vcpu); |
| 1587 | 1587 | u64 hcr0 = cr0; |
| 1588 | + bool old_paging = is_paging(vcpu); | |
| 1588 | 1589 | |
| 1589 | 1590 | #ifdef CONFIG_X86_64 |
| 1590 | 1591 | if (vcpu->arch.efer & EFER_LME && !vcpu->arch.guest_state_protected) { |
| @@ -1601,8 +1602,11 @@ void svm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · 10 unmodified lines | ||
| 1601 | 1602 | #endif |
| 1602 | 1603 | vcpu->arch.cr0 = cr0; |
| 1603 | 1604 | |
| 1604 | − if (!npt_enabled) | |
| 1605 | + if (!npt_enabled) { | |
| 1605 | 1606 | hcr0 |= X86_CR0_PG | X86_CR0_WP; |
| 1607 | + if (old_paging != is_paging(vcpu)) | |
| 1608 | + svm_set_cr4(vcpu, kvm_read_cr4(vcpu)); | |
| 1609 | + } | |
| 1606 | 1610 | |
| 1607 | 1611 | /* |
| 1608 | 1612 | * re-enable caching here because the QEMU bios |
| @@ -1646,8 +1650,12 @@ void svm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) · 37 unmodified lines | ||
| 1646 | 1650 | svm_flush_tlb(vcpu); |
| 1647 | 1651 | |
| 1648 | 1652 | vcpu->arch.cr4 = cr4; |
| 1649 | − if (!npt_enabled) | |
| 1653 | + if (!npt_enabled) { | |
| 1650 | 1654 | cr4 |= X86_CR4_PAE; |
| 1655 | + | |
| 1656 | + if (!is_paging(vcpu)) | |
| 1657 | + cr4 &= ~(X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_PKE); | |
| 1658 | + } | |
| 1651 | 1659 | cr4 |= host_cr4_mce; |
| 1652 | 1660 | to_svm(vcpu)->vmcb->save.cr4 = cr4; |
| 1653 | 1661 | vmcb_mark_dirty(to_svm(vcpu)->vmcb, VMCB_CR); |
| @@ -2685,8 +2693,23 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · 1031 unmodified lines | ||
| 2685 | 2693 | u64 data = msr->data; |
| 2686 | 2694 | switch (ecx) { |
| 2687 | 2695 | case MSR_AMD64_TSC_RATIO: |
| 2688 | − if (!msr->host_initiated && !svm->tsc_scaling_enabled) | |
| 2689 | − return 1; | |
| 2696 | + | |
| 2697 | + if (!svm->tsc_scaling_enabled) { | |
| 2698 | + | |
| 2699 | + if (!msr->host_initiated) | |
| 2700 | + return 1; | |
| 2701 | + /* | |
| 2702 | + * In case TSC scaling is not enabled, always | |
| 2703 | + * leave this MSR at the default value. | |
| 2704 | + * | |
| 2705 | + * Due to bug in qemu 6.2.0, it would try to set | |
| 2706 | + * this msr to 0 if tsc scaling is not enabled. | |
| 2707 | + * Ignore this value as well. | |
| 2708 | + */ | |
| 2709 | + if (data != 0 && data != svm->tsc_ratio_msr) | |
| 2710 | + return 1; | |
| 2711 | + break; | |
| 2712 | + } | |
| 2690 | 2713 | |
| 2691 | 2714 | if (data & TSC_RATIO_RSVD) |
| 2692 | 2715 | return 1; |
| @@ -3291,6 +3314,55 @@ static void svm_set_irq(struct kvm_vcpu *vcpu) · 598 unmodified lines | ||
| 3291 | 3314 | SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_INTR; |
| 3292 | 3315 | } |
| 3293 | 3316 | |
| 3317 | +void svm_complete_interrupt_delivery(struct kvm_vcpu *vcpu, int delivery_mode, | |
| 3318 | + int trig_mode, int vector) | |
| 3319 | +{ | |
| 3320 | + /* | |
| 3321 | + * vcpu->arch.apicv_active must be read after vcpu->mode. | |
| 3322 | + * Pairs with smp_store_release in vcpu_enter_guest. | |
| 3323 | + */ | |
| 3324 | + bool in_guest_mode = (smp_load_acquire(&vcpu->mode) == IN_GUEST_MODE); | |
| 3325 | + | |
| 3326 | + if (!READ_ONCE(vcpu->arch.apicv_active)) { | |
| 3327 | + /* Process the interrupt via inject_pending_event */ | |
| 3328 | + kvm_make_request(KVM_REQ_EVENT, vcpu); | |
| 3329 | + kvm_vcpu_kick(vcpu); | |
| 3330 | + return; | |
| 3331 | + } | |
| 3332 | + | |
| 3333 | + trace_kvm_apicv_accept_irq(vcpu->vcpu_id, delivery_mode, trig_mode, vector); | |
| 3334 | + if (in_guest_mode) { | |
| 3335 | + /* | |
| 3336 | + * Signal the doorbell to tell hardware to inject the IRQ. If | |
| 3337 | + * the vCPU exits the guest before the doorbell chimes, hardware | |
| 3338 | + * will automatically process AVIC interrupts at the next VMRUN. | |
| 3339 | + */ | |
| 3340 | + avic_ring_doorbell(vcpu); | |
| 3341 | + } else { | |
| 3342 | + /* | |
| 3343 | + * Wake the vCPU if it was blocking. KVM will then detect the | |
| 3344 | + * pending IRQ when checking if the vCPU has a wake event. | |
| 3345 | + */ | |
| 3346 | + kvm_vcpu_wake_up(vcpu); | |
| 3347 | + } | |
| 3348 | +} | |
| 3349 | + | |
| 3350 | +static void svm_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, | |
| 3351 | + int trig_mode, int vector) | |
| 3352 | +{ | |
| 3353 | + kvm_lapic_set_irr(vector, apic); | |
| 3354 | + | |
| 3355 | + /* | |
| 3356 | + * Pairs with the smp_mb_*() after setting vcpu->guest_mode in | |
| 3357 | + * vcpu_enter_guest() to ensure the write to the vIRR is ordered before | |
| 3358 | + * the read of guest_mode. This guarantees that either VMRUN will see | |
| 3359 | + * and process the new vIRR entry, or that svm_complete_interrupt_delivery | |
| 3360 | + * will signal the doorbell if the CPU has already entered the guest. | |
| 3361 | + */ | |
| 3362 | + smp_mb__after_atomic(); | |
| 3363 | + svm_complete_interrupt_delivery(apic->vcpu, delivery_mode, trig_mode, vector); | |
| 3364 | +} | |
| 3365 | + | |
| 3294 | 3366 | static void svm_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) |
| 3295 | 3367 | { |
| 3296 | 3368 | struct vcpu_svm *svm = to_svm(vcpu); |
| @@ -3338,11 +3410,13 @@ static int svm_nmi_allowed(struct kvm_vcpu *vcpu, bool for_injection) · 41 unmodified lines | ||
| 3338 | 3410 | if (svm->nested.nested_run_pending) |
| 3339 | 3411 | return -EBUSY; |
| 3340 | 3412 | |
| 3413 | + if (svm_nmi_blocked(vcpu)) | |
| 3414 | + return 0; | |
| 3415 | + | |
| 3341 | 3416 | /* An NMI must not be injected into L2 if it's supposed to VM-Exit. */ |
| 3342 | 3417 | if (for_injection && is_guest_mode(vcpu) && nested_exit_on_nmi(svm)) |
| 3343 | 3418 | return -EBUSY; |
| 3344 | − | |
| 3345 | − return !svm_nmi_blocked(vcpu); | |
| 3419 | + return 1; | |
| 3346 | 3420 | } |
| 3347 | 3421 | |
| 3348 | 3422 | static bool svm_get_nmi_mask(struct kvm_vcpu *vcpu) |
| @@ -3394,9 +3468,13 @@ bool svm_interrupt_blocked(struct kvm_vcpu *vcpu) · 45 unmodified lines | ||
| 3394 | 3468 | static int svm_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection) |
| 3395 | 3469 | { |
| 3396 | 3470 | struct vcpu_svm *svm = to_svm(vcpu); |
| 3471 | + | |
| 3397 | 3472 | if (svm->nested.nested_run_pending) |
| 3398 | 3473 | return -EBUSY; |
| 3399 | 3474 | |
| 3475 | + if (svm_interrupt_blocked(vcpu)) | |
| 3476 | + return 0; | |
| 3477 | + | |
| 3400 | 3478 | /* |
| 3401 | 3479 | * An IRQ must not be injected into L2 if it's supposed to VM-Exit, |
| 3402 | 3480 | * e.g. if the IRQ arrived asynchronously after checking nested events. |
| @@ -3404,7 +3482,7 @@ static int svm_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection) · 1 unmodified line | ||
| 3404 | 3482 | if (for_injection && is_guest_mode(vcpu) && nested_exit_on_intr(svm)) |
| 3405 | 3483 | return -EBUSY; |
| 3406 | 3484 | |
| 3407 | − return !svm_interrupt_blocked(vcpu); | |
| 3485 | + return 1; | |
| 3408 | 3486 | } |
| 3409 | 3487 | |
| 3410 | 3488 | static void svm_enable_irq_window(struct kvm_vcpu *vcpu) |
| @@ -3615,7 +3693,7 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu) · 204 unmodified lines | ||
| 3615 | 3693 | struct vcpu_svm *svm = to_svm(vcpu); |
| 3616 | 3694 | unsigned long vmcb_pa = svm->current_vmcb->pa; |
| 3617 | 3695 | |
| 3618 | − kvm_guest_enter_irqoff(); | |
| 3696 | + guest_state_enter_irqoff(); | |
| 3619 | 3697 | |
| 3620 | 3698 | if (sev_es_guest(vcpu->kvm)) { |
| 3621 | 3699 | __svm_sev_es_vcpu_run(vmcb_pa); |
| @@ -3635,7 +3713,7 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu) · 13 unmodified lines | ||
| 3635 | 3713 | vmload(__sme_page_pa(sd->save_area)); |
| 3636 | 3714 | } |
| 3637 | 3715 | |
| 3638 | − kvm_guest_exit_irqoff(); | |
| 3716 | + guest_state_exit_irqoff(); | |
| 3639 | 3717 | } |
| 3640 | 3718 | |
| 3641 | 3719 | static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) |
| @@ -4135,11 +4213,14 @@ static int svm_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection) · 493 unmodified lines | ||
| 4135 | 4213 | if (svm->nested.nested_run_pending) |
| 4136 | 4214 | return -EBUSY; |
| 4137 | 4215 | |
| 4216 | + if (svm_smi_blocked(vcpu)) | |
| 4217 | + return 0; | |
| 4218 | + | |
| 4138 | 4219 | /* An SMI must not be injected into L2 if it's supposed to VM-Exit. */ |
| 4139 | 4220 | if (for_injection && is_guest_mode(vcpu) && nested_exit_on_smi(svm)) |
| 4140 | 4221 | return -EBUSY; |
| 4141 | 4222 | |
| 4142 | − return !svm_smi_blocked(vcpu); | |
| 4223 | + return 1; | |
| 4143 | 4224 | } |
| 4144 | 4225 | |
| 4145 | 4226 | static int svm_enter_smm(struct kvm_vcpu *vcpu, char *smstate) |
| @@ -4233,11 +4314,18 @@ static int svm_leave_smm(struct kvm_vcpu *vcpu, const char *smstate) · 87 unmodified lines | ||
| 4233 | 4314 | * Enter the nested guest now |
| 4234 | 4315 | */ |
| 4235 | 4316 | |
| 4317 | + vmcb_mark_all_dirty(svm->vmcb01.ptr); | |
| 4318 | + | |
| 4236 | 4319 | vmcb12 = map.hva; |
| 4237 | 4320 | nested_copy_vmcb_control_to_cache(svm, &vmcb12->control); |
| 4238 | 4321 | nested_copy_vmcb_save_to_cache(svm, &vmcb12->save); |
| 4239 | 4322 | ret = enter_svm_guest_mode(vcpu, vmcb12_gpa, vmcb12, false); |
| 4240 | 4323 | |
| 4324 | + if (ret) | |
| 4325 | + goto unmap_save; | |
| 4326 | + | |
| 4327 | + svm->nested.nested_run_pending = 1; | |
| 4328 | + | |
| 4241 | 4329 | unmap_save: |
| 4242 | 4330 | kvm_vcpu_unmap(vcpu, &map_save, true); |
| 4243 | 4331 | unmap_map: |
| @@ -4545,7 +4633,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { · 301 unmodified lines | ||
| 4545 | 4633 | .pmu_ops = &amd_pmu_ops, |
| 4546 | 4634 | .nested_ops = &svm_nested_ops, |
| 4547 | 4635 | |
| 4548 | − .deliver_posted_interrupt = svm_deliver_avic_intr, | |
| 4636 | + .deliver_interrupt = svm_deliver_interrupt, | |
| 4549 | 4637 | .dy_apicv_has_pending_interrupt = svm_dy_apicv_has_pending_interrupt, |
| 4550 | 4638 | .update_pi_irte = svm_update_pi_irte, |
| 4551 | 4639 | .setup_mce = svm_setup_mce, |
| @@ -4622,6 +4710,7 @@ static __init void svm_set_cpu_caps(void) · 70 unmodified lines | ||
| 4622 | 4710 | /* CPUID 0x80000001 and 0x8000000A (SVM features) */ |
| 4623 | 4711 | if (nested) { |
| 4624 | 4712 | kvm_cpu_cap_set(X86_FEATURE_SVM); |
| 4713 | + kvm_cpu_cap_set(X86_FEATURE_VMCBCLEAN); | |
| 4625 | 4714 | |
| 4626 | 4715 | if (nrips) |
| 4627 | 4716 | kvm_cpu_cap_set(X86_FEATURE_NRIPS); |
| @@ -489,6 +489,8 @@ void svm_set_gif(struct vcpu_svm *svm, bool value); · 488 unmodified lines | ||
| 489 | 489 | int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code); |
| 490 | 490 | void set_msr_interception(struct kvm_vcpu *vcpu, u32 *msrpm, u32 msr, |
| 491 | 491 | int read, int write); |
| 492 | +void svm_complete_interrupt_delivery(struct kvm_vcpu *vcpu, int delivery_mode, | |
| 493 | + int trig_mode, int vec); | |
| 492 | 494 | |
| 493 | 495 | /* nested.c */ |
| 494 | 496 | |
| @@ -556,17 +558,6 @@ extern struct kvm_x86_nested_ops svm_nested_ops; · 61 unmodified lines | ||
| 556 | 558 | |
| 557 | 559 | /* avic.c */ |
| 558 | 560 | |
| 559 | −#define AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK (0xFF) | |
| 560 | −#define AVIC_LOGICAL_ID_ENTRY_VALID_BIT 31 | |
| 561 | −#define AVIC_LOGICAL_ID_ENTRY_VALID_MASK (1 << 31) | |
| 562 | − | |
| 563 | −#define AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK (0xFFULL) | |
| 564 | −#define AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK (0xFFFFFFFFFFULL << 12) | |
| 565 | −#define AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK (1ULL << 62) | |
| 566 | −#define AVIC_PHYSICAL_ID_ENTRY_VALID_MASK (1ULL << 63) | |
| 567 | − | |
| 568 | −#define VMCB_AVIC_APIC_BAR_MASK 0xFFFFFFFFFF000ULL | |
| 569 | − | |
| 570 | 561 | int avic_ga_log_notifier(u32 ga_tag); |
| 571 | 562 | void avic_vm_destroy(struct kvm *kvm); |
| 572 | 563 | int avic_vm_init(struct kvm *kvm); |
| @@ -583,12 +574,12 @@ bool svm_check_apicv_inhibit_reasons(ulong bit); · 10 unmodified lines | ||
| 583 | 574 | void svm_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap); |
| 584 | 575 | void svm_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr); |
| 585 | 576 | void svm_hwapic_isr_update(struct kvm_vcpu *vcpu, int max_isr); |
| 586 | −int svm_deliver_avic_intr(struct kvm_vcpu *vcpu, int vec); | |
| 587 | 577 | bool svm_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu); |
| 588 | 578 | int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq, |
| 589 | 579 | uint32_t guest_irq, bool set); |
| 590 | 580 | void avic_vcpu_blocking(struct kvm_vcpu *vcpu); |
| 591 | 581 | void avic_vcpu_unblocking(struct kvm_vcpu *vcpu); |
| 582 | +void avic_ring_doorbell(struct kvm_vcpu *vcpu); | |
| 592 | 583 | |
| 593 | 584 | /* sev.c */ |
| 594 | 585 | |
| @@ -4041,6 +4041,21 @@ static int vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector) · 4040 unmodified lines | ||
| 4041 | 4041 | return 0; |
| 4042 | 4042 | } |
| 4043 | 4043 | |
| 4044 | +static void vmx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, | |
| 4045 | + int trig_mode, int vector) | |
| 4046 | +{ | |
| 4047 | + struct kvm_vcpu *vcpu = apic->vcpu; | |
| 4048 | + | |
| 4049 | + if (vmx_deliver_posted_interrupt(vcpu, vector)) { | |
| 4050 | + kvm_lapic_set_irr(vector, apic); | |
| 4051 | + kvm_make_request(KVM_REQ_EVENT, vcpu); | |
| 4052 | + kvm_vcpu_kick(vcpu); | |
| 4053 | + } else { | |
| 4054 | + trace_kvm_apicv_accept_irq(vcpu->vcpu_id, delivery_mode, | |
| 4055 | + trig_mode, vector); | |
| 4056 | + } | |
| 4057 | +} | |
| 4058 | + | |
| 4044 | 4059 | /* |
| 4045 | 4060 | * Set up the vmcs's constant host-state fields, i.e., host-state fields that |
| 4046 | 4061 | * will not change in the lifetime of the guest. |
| @@ -6754,7 +6769,7 @@ static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) · 2707 unmodified lines | ||
| 6754 | 6769 | static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, |
| 6755 | 6770 | struct vcpu_vmx *vmx) |
| 6756 | 6771 | { |
| 6757 | − kvm_guest_enter_irqoff(); | |
| 6772 | + guest_state_enter_irqoff(); | |
| 6758 | 6773 | |
| 6759 | 6774 | /* L1D Flush includes CPU buffer clear to mitigate MDS */ |
| 6760 | 6775 | if (static_branch_unlikely(&vmx_l1d_should_flush)) |
| @@ -6770,7 +6785,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, · 9 unmodified lines | ||
| 6770 | 6785 | |
| 6771 | 6786 | vcpu->arch.cr2 = native_read_cr2(); |
| 6772 | 6787 | |
| 6773 | − kvm_guest_exit_irqoff(); | |
| 6788 | + guest_state_exit_irqoff(); | |
| 6774 | 6789 | } |
| 6775 | 6790 | |
| 6776 | 6791 | static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) |
| @@ -7644,6 +7659,7 @@ static int vmx_leave_smm(struct kvm_vcpu *vcpu, const char *smstate) · 867 unmodified lines | ||
| 7644 | 7659 | if (ret) |
| 7645 | 7660 | return ret; |
| 7646 | 7661 | |
| 7662 | + vmx->nested.nested_run_pending = 1; | |
| 7647 | 7663 | vmx->nested.smm.guest_mode = false; |
| 7648 | 7664 | } |
| 7649 | 7665 | return 0; |
| @@ -7768,7 +7784,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { · 118 unmodified lines | ||
| 7768 | 7784 | .hwapic_isr_update = vmx_hwapic_isr_update, |
| 7769 | 7785 | .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, |
| 7770 | 7786 | .sync_pir_to_irr = vmx_sync_pir_to_irr, |
| 7771 | − .deliver_posted_interrupt = vmx_deliver_posted_interrupt, | |
| 7787 | + .deliver_interrupt = vmx_deliver_interrupt, | |
| 7772 | 7788 | .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, |
| 7773 | 7789 | |
| 7774 | 7790 | .set_tss_addr = vmx_set_tss_addr, |
| @@ -90,6 +90,8 @@ · 89 unmodified lines | ||
| 90 | 90 | u64 __read_mostly kvm_mce_cap_supported = MCG_CTL_P | MCG_SER_P; |
| 91 | 91 | EXPORT_SYMBOL_GPL(kvm_mce_cap_supported); |
| 92 | 92 | |
| 93 | +#define ERR_PTR_USR(e) ((void __user *)ERR_PTR(e)) | |
| 94 | + | |
| 93 | 95 | #define emul_to_vcpu(ctxt) \ |
| 94 | 96 | ((struct kvm_vcpu *)(ctxt)->vcpu) |
| 95 | 97 | |
| @@ -982,6 +984,18 @@ void kvm_load_host_xsave_state(struct kvm_vcpu *vcpu) · 886 unmodified lines | ||
| 982 | 984 | } |
| 983 | 985 | EXPORT_SYMBOL_GPL(kvm_load_host_xsave_state); |
| 984 | 986 | |
| 987 | +static inline u64 kvm_guest_supported_xcr0(struct kvm_vcpu *vcpu) | |
| 988 | +{ | |
| 989 | + return vcpu->arch.guest_fpu.fpstate->user_xfeatures; | |
| 990 | +} | |
| 991 | + | |
| 992 | +#ifdef CONFIG_X86_64 | |
| 993 | +static inline u64 kvm_guest_supported_xfd(struct kvm_vcpu *vcpu) | |
| 994 | +{ | |
| 995 | + return kvm_guest_supported_xcr0(vcpu) & XFEATURE_MASK_USER_DYNAMIC; | |
| 996 | +} | |
| 997 | +#endif | |
| 998 | + | |
| 985 | 999 | static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) |
| 986 | 1000 | { |
| 987 | 1001 | u64 xcr0 = xcr; |
| @@ -1001,7 +1015,7 @@ static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) · 13 unmodified lines | ||
| 1001 | 1015 | * saving. However, xcr0 bit 0 is always set, even if the |
| 1002 | 1016 | * emulated CPU does not support XSAVE (see kvm_vcpu_reset()). |
| 1003 | 1017 | */ |
| 1004 | − valid_bits = vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FP; | |
| 1018 | + valid_bits = kvm_guest_supported_xcr0(vcpu) | XFEATURE_MASK_FP; | |
| 1005 | 1019 | if (xcr0 & ~valid_bits) |
| 1006 | 1020 | return 1; |
| 1007 | 1021 | |
| @@ -2349,10 +2363,12 @@ static u64 compute_guest_tsc(struct kvm_vcpu *vcpu, s64 kernel_ns) · 1341 unmodified lines | ||
| 2349 | 2363 | return tsc; |
| 2350 | 2364 | } |
| 2351 | 2365 | |
| 2366 | +#ifdef CONFIG_X86_64 | |
| 2352 | 2367 | static inline int gtod_is_based_on_tsc(int mode) |
| 2353 | 2368 | { |
| 2354 | 2369 | return mode == VDSO_CLOCKMODE_TSC || mode == VDSO_CLOCKMODE_HVCLOCK; |
| 2355 | 2370 | } |
| 2371 | +#endif | |
| 2356 | 2372 | |
| 2357 | 2373 | static void kvm_track_tsc_matching(struct kvm_vcpu *vcpu) |
| 2358 | 2374 | { |
| @@ -3704,8 +3720,7 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · 1345 unmodified lines | ||
| 3704 | 3720 | !guest_cpuid_has(vcpu, X86_FEATURE_XFD)) |
| 3705 | 3721 | return 1; |
| 3706 | 3722 | |
| 3707 | − if (data & ~(XFEATURE_MASK_USER_DYNAMIC & | |
| 3708 | − vcpu->arch.guest_supported_xcr0)) | |
| 3723 | + if (data & ~kvm_guest_supported_xfd(vcpu)) | |
| 3709 | 3724 | return 1; |
| 3710 | 3725 | |
| 3711 | 3726 | fpu_update_guest_xfd(&vcpu->arch.guest_fpu, data); |
| @@ -3715,8 +3730,7 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · 3 unmodified lines | ||
| 3715 | 3730 | !guest_cpuid_has(vcpu, X86_FEATURE_XFD)) |
| 3716 | 3731 | return 1; |
| 3717 | 3732 | |
| 3718 | − if (data & ~(XFEATURE_MASK_USER_DYNAMIC & | |
| 3719 | − vcpu->arch.guest_supported_xcr0)) | |
| 3733 | + if (data & ~kvm_guest_supported_xfd(vcpu)) | |
| 3720 | 3734 | return 1; |
| 3721 | 3735 | |
| 3722 | 3736 | vcpu->arch.guest_fpu.xfd_err = data; |
| @@ -4231,6 +4245,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 508 unmodified lines | ||
| 4231 | 4245 | case KVM_CAP_EXIT_ON_EMULATION_FAILURE: |
| 4232 | 4246 | case KVM_CAP_VCPU_ATTRIBUTES: |
| 4233 | 4247 | case KVM_CAP_SYS_ATTRIBUTES: |
| 4248 | + case KVM_CAP_ENABLE_CAP: | |
| 4234 | 4249 | r = 1; |
| 4235 | 4250 | break; |
| 4236 | 4251 | case KVM_CAP_EXIT_HYPERCALL: |
| @@ -4340,7 +4355,7 @@ static inline void __user *kvm_get_attr_addr(struct kvm_device_attr *attr) · 103 unmodified lines | ||
| 4340 | 4355 | void __user *uaddr = (void __user*)(unsigned long)attr->addr; |
| 4341 | 4356 | |
| 4342 | 4357 | if ((u64)(unsigned long)uaddr != attr->addr) |
| 4343 | − return ERR_PTR(-EFAULT); | |
| 4358 | + return ERR_PTR_USR(-EFAULT); | |
| 4344 | 4359 | return uaddr; |
| 4345 | 4360 | } |
| 4346 | 4361 | |
| @@ -8940,6 +8955,13 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, · 4593 unmodified lines | ||
| 8940 | 8955 | if (clock_type != KVM_CLOCK_PAIRING_WALLCLOCK) |
| 8941 | 8956 | return -KVM_EOPNOTSUPP; |
| 8942 | 8957 | |
| 8958 | + /* | |
| 8959 | + * When tsc is in permanent catchup mode guests won't be able to use | |
| 8960 | + * pvclock_read_retry loop to get consistent view of pvclock | |
| 8961 | + */ | |
| 8962 | + if (vcpu->arch.tsc_always_catchup) | |
| 8963 | + return -KVM_EOPNOTSUPP; | |
| 8964 | + | |
| 8943 | 8965 | if (!kvm_get_walltime_and_clockread(&ts, &cycle)) |
| 8944 | 8966 | return -KVM_EOPNOTSUPP; |
| 8945 | 8967 | |
| @@ -9981,7 +10003,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · 1035 unmodified lines | ||
| 9981 | 10003 | * result in virtual interrupt delivery. |
| 9982 | 10004 | */ |
| 9983 | 10005 | local_irq_disable(); |
| 9984 | − vcpu->mode = IN_GUEST_MODE; | |
| 10006 | + | |
| 10007 | + /* Store vcpu->apicv_active before vcpu->mode. */ | |
| 10008 | + smp_store_release(&vcpu->mode, IN_GUEST_MODE); | |
| 9985 | 10009 | |
| 9986 | 10010 | srcu_read_unlock(&vcpu->kvm->srcu, vcpu->srcu_idx); |
| 9987 | 10011 | |
| @@ -10041,6 +10065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · 53 unmodified lines | ||
| 10041 | 10065 | set_debugreg(0, 7); |
| 10042 | 10066 | } |
| 10043 | 10067 | |
| 10068 | + guest_timing_enter_irqoff(); | |
| 10069 | + | |
| 10044 | 10070 | for (;;) { |
| 10045 | 10071 | /* |
| 10046 | 10072 | * Assert that vCPU vs. VM APICv state is consistent. An APICv |
| @@ -10125,7 +10151,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · 78 unmodified lines | ||
| 10125 | 10151 | * of accounting via context tracking, but the loss of accuracy is |
| 10126 | 10152 | * acceptable for all known use cases. |
| 10127 | 10153 | */ |
| 10128 | − vtime_account_guest_exit(); | |
| 10154 | + guest_timing_exit_irqoff(); | |
| 10129 | 10155 | |
| 10130 | 10156 | if (lapic_in_kernel(vcpu)) { |
| 10131 | 10157 | s64 delta = vcpu->arch.apic->lapic_timer.advance_expire_delta; |
| @@ -11639,8 +11665,6 @@ void kvm_arch_sync_events(struct kvm *kvm) · 1507 unmodified lines | ||
| 11639 | 11665 | kvm_free_pit(kvm); |
| 11640 | 11666 | } |
| 11641 | 11667 | |
| 11642 | −#define ERR_PTR_USR(e) ((void __user *)ERR_PTR(e)) | |
| 11643 | − | |
| 11644 | 11668 | /** |
| 11645 | 11669 | * __x86_set_memory_region: Setup KVM internal memory slot |
| 11646 | 11670 | * |
| @@ -10,51 +10,6 @@ · 9 unmodified lines | ||
| 10 | 10 | |
| 11 | 11 | void kvm_spurious_fault(void); |
| 12 | 12 | |
| 13 | −static __always_inline void kvm_guest_enter_irqoff(void) | |
| 14 | −{ | |
| 15 | − /* | |
| 16 | − * VMENTER enables interrupts (host state), but the kernel state is | |
| 17 | − * interrupts disabled when this is invoked. Also tell RCU about | |
| 18 | − * it. This is the same logic as for exit_to_user_mode(). | |
| 19 | − * | |
| 20 | − * This ensures that e.g. latency analysis on the host observes | |
| 21 | − * guest mode as interrupt enabled. | |
| 22 | − * | |
| 23 | − * guest_enter_irqoff() informs context tracking about the | |
| 24 | − * transition to guest mode and if enabled adjusts RCU state | |
| 25 | − * accordingly. | |
| 26 | − */ | |
| 27 | − instrumentation_begin(); | |
| 28 | − trace_hardirqs_on_prepare(); | |
| 29 | − lockdep_hardirqs_on_prepare(CALLER_ADDR0); | |
| 30 | − instrumentation_end(); | |
| 31 | − | |
| 32 | − guest_enter_irqoff(); | |
| 33 | − lockdep_hardirqs_on(CALLER_ADDR0); | |
| 34 | −} | |
| 35 | − | |
| 36 | −static __always_inline void kvm_guest_exit_irqoff(void) | |
| 37 | −{ | |
| 38 | − /* | |
| 39 | − * VMEXIT disables interrupts (host state), but tracing and lockdep | |
| 40 | − * have them in state 'on' as recorded before entering guest mode. | |
| 41 | − * Same as enter_from_user_mode(). | |
| 42 | − * | |
| 43 | − * context_tracking_guest_exit() restores host context and reinstates | |
| 44 | − * RCU if enabled and required. | |
| 45 | − * | |
| 46 | − * This needs to be done immediately after VM-Exit, before any code | |
| 47 | − * that might contain tracepoints or call out to the greater world, | |
| 48 | − * e.g. before x86_spec_ctrl_restore_host(). | |
| 49 | − */ | |
| 50 | − lockdep_hardirqs_off(CALLER_ADDR0); | |
| 51 | − context_tracking_guest_exit(); | |
| 52 | − | |
| 53 | − instrumentation_begin(); | |
| 54 | − trace_hardirqs_off_finish(); | |
| 55 | − instrumentation_end(); | |
| 56 | −} | |
| 57 | − | |
| 58 | 13 | #define KVM_NESTED_VMENTER_CONSISTENCY_CHECK(consistency_check) \ |
| 59 | 14 | ({ \ |
| 60 | 15 | bool failed = (consistency_check); \ |
| @@ -133,32 +133,57 @@ static void kvm_xen_update_runstate(struct kvm_vcpu *v, int state) · 132 unmodified lines | ||
| 133 | 133 | void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, int state) |
| 134 | 134 | { |
| 135 | 135 | struct kvm_vcpu_xen *vx = &v->arch.xen; |
| 136 | + struct gfn_to_hva_cache *ghc = &vx->runstate_cache; | |
| 137 | + struct kvm_memslots *slots = kvm_memslots(v->kvm); | |
| 138 | + bool atomic = (state == RUNSTATE_runnable); | |
| 136 | 139 | uint64_t state_entry_time; |
| 137 | − unsigned int offset; | |
| 140 | + int __user *user_state; | |
| 141 | + uint64_t __user *user_times; | |
| 138 | 142 | |
| 139 | 143 | kvm_xen_update_runstate(v, state); |
| 140 | 144 | |
| 141 | 145 | if (!vx->runstate_set) |
| 142 | 146 | return; |
| 143 | 147 | |
| 144 | − BUILD_BUG_ON(sizeof(struct compat_vcpu_runstate_info) != 0x2c); | |
| 148 | + if (unlikely(slots->generation != ghc->generation || kvm_is_error_hva(ghc->hva)) && | |
| 149 | + kvm_gfn_to_hva_cache_init(v->kvm, ghc, ghc->gpa, ghc->len)) | |
| 150 | + return; | |
| 151 | + | |
| 152 | + /* We made sure it fits in a single page */ | |
| 153 | + BUG_ON(!ghc->memslot); | |
| 154 | + | |
| 155 | + if (atomic) | |
| 156 | + pagefault_disable(); | |
| 145 | 157 | |
| 146 | − offset = offsetof(struct compat_vcpu_runstate_info, state_entry_time); | |
| 147 | −#ifdef CONFIG_X86_64 | |
| 148 | 158 | /* |
| 149 | − * The only difference is alignment of uint64_t in 32-bit. | |
| 150 | − * So the first field 'state' is accessed directly using | |
| 151 | − * offsetof() (where its offset happens to be zero), while the | |
| 152 | − * remaining fields which are all uint64_t, start at 'offset' | |
| 153 | − * which we tweak here by adding 4. | |
| 159 | + * The only difference between 32-bit and 64-bit versions of the | |
| 160 | + * runstate struct us the alignment of uint64_t in 32-bit, which | |
| 161 | + * means that the 64-bit version has an additional 4 bytes of | |
| 162 | + * padding after the first field 'state'. | |
| 163 | + * | |
| 164 | + * So we use 'int __user *user_state' to point to the state field, | |
| 165 | + * and 'uint64_t __user *user_times' for runstate_entry_time. So | |
| 166 | + * the actual array of time[] in each state starts at user_times[1]. | |
| 154 | 167 | */ |
| 168 | + BUILD_BUG_ON(offsetof(struct vcpu_runstate_info, state) != 0); | |
| 169 | + BUILD_BUG_ON(offsetof(struct compat_vcpu_runstate_info, state) != 0); | |
| 170 | + user_state = (int __user *)ghc->hva; | |
| 171 | + | |
| 172 | + BUILD_BUG_ON(sizeof(struct compat_vcpu_runstate_info) != 0x2c); | |
| 173 | + | |
| 174 | + user_times = (uint64_t __user *)(ghc->hva + | |
| 175 | + offsetof(struct compat_vcpu_runstate_info, | |
| 176 | + state_entry_time)); | |
| 177 | +#ifdef CONFIG_X86_64 | |
| 155 | 178 | BUILD_BUG_ON(offsetof(struct vcpu_runstate_info, state_entry_time) != |
| 156 | 179 | offsetof(struct compat_vcpu_runstate_info, state_entry_time) + 4); |
| 157 | 180 | BUILD_BUG_ON(offsetof(struct vcpu_runstate_info, time) != |
| 158 | 181 | offsetof(struct compat_vcpu_runstate_info, time) + 4); |
| 159 | 182 | |
| 160 | 183 | if (v->kvm->arch.xen.long_mode) |
| 161 | − offset = offsetof(struct vcpu_runstate_info, state_entry_time); | |
| 184 | + user_times = (uint64_t __user *)(ghc->hva + | |
| 185 | + offsetof(struct vcpu_runstate_info, | |
| 186 | + state_entry_time)); | |
| 162 | 187 | #endif |
| 163 | 188 | /* |
| 164 | 189 | * First write the updated state_entry_time at the appropriate |
| @@ -172,10 +197,8 @@ void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, int state) · 7 unmodified lines | ||
| 172 | 197 | BUILD_BUG_ON(sizeof_field(struct compat_vcpu_runstate_info, state_entry_time) != |
| 173 | 198 | sizeof(state_entry_time)); |
| 174 | 199 | |
| 175 | − if (kvm_write_guest_offset_cached(v->kvm, &v->arch.xen.runstate_cache, | |
| 176 | − &state_entry_time, offset, | |
| 177 | − sizeof(state_entry_time))) | |
| 178 | − return; | |
| 200 | + if (__put_user(state_entry_time, user_times)) | |
| 201 | + goto out; | |
| 179 | 202 | smp_wmb(); |
| 180 | 203 | |
| 181 | 204 | /* |
| @@ -189,11 +212,8 @@ void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, int state) · 7 unmodified lines | ||
| 189 | 212 | BUILD_BUG_ON(sizeof_field(struct compat_vcpu_runstate_info, state) != |
| 190 | 213 | sizeof(vx->current_runstate)); |
| 191 | 214 | |
| 192 | − if (kvm_write_guest_offset_cached(v->kvm, &v->arch.xen.runstate_cache, | |
| 193 | − &vx->current_runstate, | |
| 194 | − offsetof(struct vcpu_runstate_info, state), | |
| 195 | − sizeof(vx->current_runstate))) | |
| 196 | − return; | |
| 215 | + if (__put_user(vx->current_runstate, user_state)) | |
| 216 | + goto out; | |
| 197 | 217 | |
| 198 | 218 | /* |
| 199 | 219 | * Write the actual runstate times immediately after the |
| @@ -208,24 +228,23 @@ void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, int state) · 8 unmodified lines | ||
| 208 | 228 | BUILD_BUG_ON(sizeof_field(struct vcpu_runstate_info, time) != |
| 209 | 229 | sizeof(vx->runstate_times)); |
| 210 | 230 | |
| 211 | − if (kvm_write_guest_offset_cached(v->kvm, &v->arch.xen.runstate_cache, | |
| 212 | − &vx->runstate_times[0], | |
| 213 | − offset + sizeof(u64), | |
| 214 | − sizeof(vx->runstate_times))) | |
| 215 | − return; | |
| 216 | − | |
| 231 | + if (__copy_to_user(user_times + 1, vx->runstate_times, sizeof(vx->runstate_times))) | |
| 232 | + goto out; | |
| 217 | 233 | smp_wmb(); |
| 218 | 234 | |
| 219 | 235 | /* |
| 220 | 236 | * Finally, clear the XEN_RUNSTATE_UPDATE bit in the guest's |
| 221 | 237 | * runstate_entry_time field. |
| 222 | 238 | */ |
| 223 | − | |
| 224 | 239 | state_entry_time &= ~XEN_RUNSTATE_UPDATE; |
| 225 | − if (kvm_write_guest_offset_cached(v->kvm, &v->arch.xen.runstate_cache, | |
| 226 | − &state_entry_time, offset, | |
| 227 | − sizeof(state_entry_time))) | |
| 228 | − return; | |
| 240 | + __put_user(state_entry_time, user_times); | |
| 241 | + smp_wmb(); | |
| 242 | + | |
| 243 | + out: | |
| 244 | + mark_page_dirty_in_slot(v->kvm, ghc->memslot, ghc->gpa >> PAGE_SHIFT); | |
| 245 | + | |
| 246 | + if (atomic) | |
| 247 | + pagefault_enable(); | |
| 229 | 248 | } |
| 230 | 249 | |
| 231 | 250 | int __kvm_xen_has_interrupt(struct kvm_vcpu *v) |
| @@ -443,6 +462,12 @@ int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) · 211 unmodified lines | ||
| 443 | 462 | break; |
| 444 | 463 | } |
| 445 | 464 | |
| 465 | + /* It must fit within a single page */ | |
| 466 | + if ((data->u.gpa & ~PAGE_MASK) + sizeof(struct vcpu_info) > PAGE_SIZE) { | |
| 467 | + r = -EINVAL; | |
| 468 | + break; | |
| 469 | + } | |
| 470 | + | |
| 446 | 471 | r = kvm_gfn_to_hva_cache_init(vcpu->kvm, |
| 447 | 472 | &vcpu->arch.xen.vcpu_info_cache, |
| 448 | 473 | data->u.gpa, |
| @@ -460,6 +485,12 @@ int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) · 11 unmodified lines | ||
| 460 | 485 | break; |
| 461 | 486 | } |
| 462 | 487 | |
| 488 | + /* It must fit within a single page */ | |
| 489 | + if ((data->u.gpa & ~PAGE_MASK) + sizeof(struct pvclock_vcpu_time_info) > PAGE_SIZE) { | |
| 490 | + r = -EINVAL; | |
| 491 | + break; | |
| 492 | + } | |
| 493 | + | |
| 463 | 494 | r = kvm_gfn_to_hva_cache_init(vcpu->kvm, |
| 464 | 495 | &vcpu->arch.xen.vcpu_time_info_cache, |
| 465 | 496 | data->u.gpa, |
| @@ -481,6 +512,12 @@ int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) · 15 unmodified lines | ||
| 481 | 512 | break; |
| 482 | 513 | } |
| 483 | 514 | |
| 515 | + /* It must fit within a single page */ | |
| 516 | + if ((data->u.gpa & ~PAGE_MASK) + sizeof(struct vcpu_runstate_info) > PAGE_SIZE) { | |
| 517 | + r = -EINVAL; | |
| 518 | + break; | |
| 519 | + } | |
| 520 | + | |
| 484 | 521 | r = kvm_gfn_to_hva_cache_init(vcpu->kvm, |
| 485 | 522 | &vcpu->arch.xen.runstate_cache, |
| 486 | 523 | data->u.gpa, |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | ||
| 9 | 9 | #include <xen/events.h> |
| 10 | 10 | #include <xen/interface/memory.h> |
| 11 | 11 | |
| 12 | +#include <asm/apic.h> | |
| 12 | 13 | #include <asm/cpu.h> |
| 13 | 14 | #include <asm/smp.h> |
| 14 | 15 | #include <asm/io_apic.h> |
| @@ -184,8 +185,7 @@ static int xen_cpu_dead_hvm(unsigned int cpu) · 169 unmodified lines | ||
| 184 | 185 | |
| 185 | 186 | if (xen_have_vector_callback && xen_feature(XENFEAT_hvm_safe_pvclock)) |
| 186 | 187 | xen_teardown_timer(cpu); |
| 187 | − | |
| 188 | − return 0; | |
| 188 | + return 0; | |
| 189 | 189 | } |
| 190 | 190 | |
| 191 | 191 | static bool no_vector_callback __initdata; |
| @@ -242,15 +242,14 @@ static __init int xen_parse_no_vector_callback(char *arg) · 50 unmodified lines | ||
| 242 | 242 | } |
| 243 | 243 | early_param("xen_no_vector_callback", xen_parse_no_vector_callback); |
| 244 | 244 | |
| 245 | −bool __init xen_hvm_need_lapic(void) | |
| 245 | +static __init bool xen_x2apic_available(void) | |
| 246 | 246 | { |
| 247 | − if (xen_pv_domain()) | |
| 248 | − return false; | |
| 249 | − if (!xen_hvm_domain()) | |
| 250 | − return false; | |
| 251 | − if (xen_feature(XENFEAT_hvm_pirqs) && xen_have_vector_callback) | |
| 252 | − return false; | |
| 253 | − return true; | |
| 247 | + return x2apic_supported(); | |
| 248 | +} | |
| 249 | + | |
| 250 | +static bool __init msi_ext_dest_id(void) | |
| 251 | +{ | |
| 252 | + return cpuid_eax(xen_cpuid_base() + 4) & XEN_HVM_CPUID_EXT_DEST_ID; | |
| 254 | 253 | } |
| 255 | 254 | |
| 256 | 255 | static __init void xen_hvm_guest_late_init(void) |
| @@ -312,9 +311,10 @@ struct hypervisor_x86 x86_hyper_xen_hvm __initdata = { · 55 unmodified lines | ||
| 312 | 311 | .detect = xen_platform_hvm, |
| 313 | 312 | .type = X86_HYPER_XEN_HVM, |
| 314 | 313 | .init.init_platform = xen_hvm_guest_init, |
| 315 | − .init.x2apic_available = xen_x2apic_para_available, | |
| 314 | + .init.x2apic_available = xen_x2apic_available, | |
| 316 | 315 | .init.init_mem_mapping = xen_hvm_init_mem_mapping, |
| 317 | 316 | .init.guest_late_init = xen_hvm_guest_late_init, |
| 317 | + .init.msi_ext_dest_id = msi_ext_dest_id, | |
| 318 | 318 | .runtime.pin_vcpu = xen_pin_vcpu, |
| 319 | 319 | .ignore_nopv = true, |
| 320 | 320 | }; |
| @@ -1341,10 +1341,6 @@ asmlinkage __visible void __init xen_start_kernel(void) · 1340 unmodified lines | ||
| 1341 | 1341 | |
| 1342 | 1342 | xen_acpi_sleep_register(); |
| 1343 | 1343 | |
| 1344 | − /* Avoid searching for BIOS MP tables */ | |
| 1345 | − x86_init.mpparse.find_smp_config = x86_init_noop; | |
| 1346 | − x86_init.mpparse.get_smp_config = x86_init_uint_noop; | |
| 1347 | − | |
| 1348 | 1344 | xen_boot_params_init_edd(); |
| 1349 | 1345 | |
| 1350 | 1346 | #ifdef CONFIG_ACPI |
| @@ -148,28 +148,12 @@ int xen_smp_intr_init_pv(unsigned int cpu) · 147 unmodified lines | ||
| 148 | 148 | return rc; |
| 149 | 149 | } |
| 150 | 150 | |
| 151 | −static void __init xen_fill_possible_map(void) | |
| 152 | −{ | |
| 153 | − int i, rc; | |
| 154 | − | |
| 155 | − if (xen_initial_domain()) | |
| 156 | − return; | |
| 157 | − | |
| 158 | − for (i = 0; i < nr_cpu_ids; i++) { | |
| 159 | − rc = HYPERVISOR_vcpu_op(VCPUOP_is_up, i, NULL); | |
| 160 | − if (rc >= 0) { | |
| 161 | − num_processors++; | |
| 162 | − set_cpu_possible(i, true); | |
| 163 | − } | |
| 164 | − } | |
| 165 | −} | |
| 166 | − | |
| 167 | −static void __init xen_filter_cpu_maps(void) | |
| 151 | +static void __init _get_smp_config(unsigned int early) | |
| 168 | 152 | { |
| 169 | 153 | int i, rc; |
| 170 | 154 | unsigned int subtract = 0; |
| 171 | 155 | |
| 172 | − if (!xen_initial_domain()) | |
| 156 | + if (early) | |
| 173 | 157 | return; |
| 174 | 158 | |
| 175 | 159 | num_processors = 0; |
| @@ -210,7 +194,6 @@ static void __init xen_pv_smp_prepare_boot_cpu(void) · 34 unmodified lines | ||
| 210 | 194 | * sure the old memory can be recycled. */ |
| 211 | 195 | make_lowmem_page_readwrite(xen_initial_gdt); |
| 212 | 196 | |
| 213 | − xen_filter_cpu_maps(); | |
| 214 | 197 | xen_setup_vcpu_info_placement(); |
| 215 | 198 | |
| 216 | 199 | /* |
| @@ -476,5 +459,8 @@ static const struct smp_ops xen_smp_ops __initconst = { · 259 unmodified lines | ||
| 476 | 459 | void __init xen_smp_init(void) |
| 477 | 460 | { |
| 478 | 461 | smp_ops = xen_smp_ops; |
| 479 | − xen_fill_possible_map(); | |
| 462 | + | |
| 463 | + /* Avoid searching for BIOS MP tables */ | |
| 464 | + x86_init.mpparse.find_smp_config = x86_init_noop; | |
| 465 | + x86_init.mpparse.get_smp_config = _get_smp_config; | |
| 480 | 466 | } |
| @@ -57,6 +57,14 @@ void __init xen_init_vga(const struct dom0_vga_console_info *info, size_t size) · 56 unmodified lines | ||
| 57 | 57 | screen_info->rsvd_size = info->u.vesa_lfb.rsvd_size; |
| 58 | 58 | screen_info->rsvd_pos = info->u.vesa_lfb.rsvd_pos; |
| 59 | 59 | |
| 60 | + if (size >= offsetof(struct dom0_vga_console_info, | |
| 61 | + u.vesa_lfb.ext_lfb_base) | |
| 62 | + + sizeof(info->u.vesa_lfb.ext_lfb_base) | |
| 63 | + && info->u.vesa_lfb.ext_lfb_base) { | |
| 64 | + screen_info->ext_lfb_base = info->u.vesa_lfb.ext_lfb_base; | |
| 65 | + screen_info->capabilities |= VIDEO_CAPABILITY_64BIT_BASE; | |
| 66 | + } | |
| 67 | + | |
| 60 | 68 | if (info->video_type == XEN_VGATYPE_EFI_LFB) { |
| 61 | 69 | screen_info->orig_video_isVGA = VIDEO_TYPE_EFI; |
| 62 | 70 | break; |
| @@ -66,14 +74,6 @@ void __init xen_init_vga(const struct dom0_vga_console_info *info, size_t size) · 3 unmodified lines | ||
| 66 | 74 | u.vesa_lfb.mode_attrs) |
| 67 | 75 | + sizeof(info->u.vesa_lfb.mode_attrs)) |
| 68 | 76 | screen_info->vesa_attributes = info->u.vesa_lfb.mode_attrs; |
| 69 | − | |
| 70 | − if (size >= offsetof(struct dom0_vga_console_info, | |
| 71 | − u.vesa_lfb.ext_lfb_base) | |
| 72 | − + sizeof(info->u.vesa_lfb.ext_lfb_base) | |
| 73 | − && info->u.vesa_lfb.ext_lfb_base) { | |
| 74 | − screen_info->ext_lfb_base = info->u.vesa_lfb.ext_lfb_base; | |
| 75 | − screen_info->capabilities |= VIDEO_CAPABILITY_64BIT_BASE; | |
| 76 | − } | |
| 77 | 77 | break; |
| 78 | 78 | } |
| 79 | 79 | } |
| @@ -7018,6 +7018,8 @@ static void bfq_exit_queue(struct elevator_queue *e) · 7017 unmodified lines | ||
| 7018 | 7018 | spin_unlock_irq(&bfqd->lock); |
| 7019 | 7019 | #endif |
| 7020 | 7020 | |
| 7021 | + wbt_enable_default(bfqd->queue); | |
| 7022 | + | |
| 7021 | 7023 | kfree(bfqd); |
| 7022 | 7024 | } |
| 7023 | 7025 | |
| @@ -373,7 +373,7 @@ void bio_integrity_advance(struct bio *bio, unsigned int bytes_done) · 372 unmodified lines | ||
| 373 | 373 | struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk); |
| 374 | 374 | unsigned bytes = bio_integrity_bytes(bi, bytes_done >> 9); |
| 375 | 375 | |
| 376 | − bip->bip_iter.bi_sector += bytes_done >> 9; | |
| 376 | + bip->bip_iter.bi_sector += bio_integrity_intervals(bi, bytes_done >> 9); | |
| 377 | 377 | bvec_iter_advance(bip->bip_vec, &bip->bip_iter, bytes); |
| 378 | 378 | } |
| 379 | 379 | |
| @@ -284,13 +284,6 @@ void blk_queue_start_drain(struct request_queue *q) · 283 unmodified lines | ||
| 284 | 284 | wake_up_all(&q->mq_freeze_wq); |
| 285 | 285 | } |
| 286 | 286 | |
| 287 | −void blk_set_queue_dying(struct request_queue *q) | |
| 288 | −{ | |
| 289 | − blk_queue_flag_set(QUEUE_FLAG_DYING, q); | |
| 290 | − blk_queue_start_drain(q); | |
| 291 | −} | |
| 292 | −EXPORT_SYMBOL_GPL(blk_set_queue_dying); | |
| 293 | − | |
| 294 | 287 | /** |
| 295 | 288 | * blk_cleanup_queue - shutdown a request queue |
| 296 | 289 | * @q: request queue to shutdown |
| @@ -308,7 +301,8 @@ void blk_cleanup_queue(struct request_queue *q) · 11 unmodified lines | ||
| 308 | 301 | WARN_ON_ONCE(blk_queue_registered(q)); |
| 309 | 302 | |
| 310 | 303 | /* mark @q DYING, no new request or merges will be allowed afterwards */ |
| 311 | − blk_set_queue_dying(q); | |
| 304 | + blk_queue_flag_set(QUEUE_FLAG_DYING, q); | |
| 305 | + blk_queue_start_drain(q); | |
| 312 | 306 | |
| 313 | 307 | blk_queue_flag_set(QUEUE_FLAG_NOMERGES, q); |
| 314 | 308 | blk_queue_flag_set(QUEUE_FLAG_NOXMERGES, q); |
| @@ -446,7 +446,7 @@ static struct bio *bio_copy_kern(struct request_queue *q, void *data, · 445 unmodified lines | ||
| 446 | 446 | if (bytes > len) |
| 447 | 447 | bytes = len; |
| 448 | 448 | |
| 449 | − page = alloc_page(GFP_NOIO | gfp_mask); | |
| 449 | + page = alloc_page(GFP_NOIO | __GFP_ZERO | gfp_mask); | |
| 450 | 450 | if (!page) |
| 451 | 451 | goto cleanup; |
| 452 | 452 | |
| @@ -736,6 +736,10 @@ static void blk_complete_request(struct request *req) · 735 unmodified lines | ||
| 736 | 736 | |
| 737 | 737 | /* Completion has already been traced */ |
| 738 | 738 | bio_clear_flag(bio, BIO_TRACE_COMPLETION); |
| 739 | + | |
| 740 | + if (req_op(req) == REQ_OP_ZONE_APPEND) | |
| 741 | + bio->bi_iter.bi_sector = req->__sector; | |
| 742 | + | |
| 739 | 743 | if (!is_flush) |
| 740 | 744 | bio_endio(bio); |
| 741 | 745 | bio = next; |
| @@ -525,8 +525,6 @@ void elv_unregister_queue(struct request_queue *q) · 524 unmodified lines | ||
| 525 | 525 | kobject_del(&e->kobj); |
| 526 | 526 | |
| 527 | 527 | e->registered = 0; |
| 528 | − /* Re-enable throttling in case elevator disabled it */ | |
| 529 | − wbt_enable_default(q); | |
| 530 | 528 | } |
| 531 | 529 | } |
| 532 | 530 | |
| @@ -289,6 +289,8 @@ static void blkdev_bio_end_io_async(struct bio *bio) · 288 unmodified lines | ||
| 289 | 289 | struct kiocb *iocb = dio->iocb; |
| 290 | 290 | ssize_t ret; |
| 291 | 291 | |
| 292 | + WRITE_ONCE(iocb->private, NULL); | |
| 293 | + | |
| 292 | 294 | if (likely(!bio->bi_status)) { |
| 293 | 295 | ret = dio->size; |
| 294 | 296 | iocb->ki_pos += ret; |
| @@ -566,34 +568,37 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to) · 271 unmodified lines | ||
| 566 | 568 | { |
| 567 | 569 | struct block_device *bdev = iocb->ki_filp->private_data; |
| 568 | 570 | loff_t size = bdev_nr_bytes(bdev); |
| 569 | − size_t count = iov_iter_count(to); | |
| 570 | 571 | loff_t pos = iocb->ki_pos; |
| 571 | 572 | size_t shorted = 0; |
| 572 | 573 | ssize_t ret = 0; |
| 574 | + size_t count; | |
| 573 | 575 | |
| 574 | − if (unlikely(pos + count > size)) { | |
| 576 | + if (unlikely(pos + iov_iter_count(to) > size)) { | |
| 575 | 577 | if (pos >= size) |
| 576 | 578 | return 0; |
| 577 | 579 | size -= pos; |
| 578 | − if (count > size) { | |
| 579 | − shorted = count - size; | |
| 580 | − iov_iter_truncate(to, size); | |
| 581 | − } | |
| 580 | + shorted = iov_iter_count(to) - size; | |
| 581 | + iov_iter_truncate(to, size); | |
| 582 | 582 | } |
| 583 | 583 | |
| 584 | + count = iov_iter_count(to); | |
| 585 | + if (!count) | |
| 586 | + goto reexpand; /* skip atime */ | |
| 587 | + | |
| 584 | 588 | if (iocb->ki_flags & IOCB_DIRECT) { |
| 585 | 589 | struct address_space *mapping = iocb->ki_filp->f_mapping; |
| 586 | 590 | |
| 587 | 591 | if (iocb->ki_flags & IOCB_NOWAIT) { |
| 588 | − if (filemap_range_needs_writeback(mapping, iocb->ki_pos, | |
| 589 | − iocb->ki_pos + count - 1)) | |
| 590 | − return -EAGAIN; | |
| 592 | + if (filemap_range_needs_writeback(mapping, pos, | |
| 593 | + pos + count - 1)) { | |
| 594 | + ret = -EAGAIN; | |
| 595 | + goto reexpand; | |
| 596 | + } | |
| 591 | 597 | } else { |
| 592 | − ret = filemap_write_and_wait_range(mapping, | |
| 593 | − iocb->ki_pos, | |
| 594 | − iocb->ki_pos + count - 1); | |
| 598 | + ret = filemap_write_and_wait_range(mapping, pos, | |
| 599 | + pos + count - 1); | |
| 595 | 600 | if (ret < 0) |
| 596 | − return ret; | |
| 601 | + goto reexpand; | |
| 597 | 602 | } |
| 598 | 603 | |
| 599 | 604 | file_accessed(iocb->ki_filp); |
| @@ -603,12 +608,14 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to) · 3 unmodified lines | ||
| 603 | 608 | iocb->ki_pos += ret; |
| 604 | 609 | count -= ret; |
| 605 | 610 | } |
| 611 | + iov_iter_revert(to, count - iov_iter_count(to)); | |
| 606 | 612 | if (ret < 0 || !count) |
| 607 | − return ret; | |
| 613 | + goto reexpand; | |
| 608 | 614 | } |
| 609 | 615 | |
| 610 | 616 | ret = filemap_read(iocb, to, ret); |
| 611 | 617 | |
| 618 | +reexpand: | |
| 612 | 619 | if (unlikely(shorted)) |
| 613 | 620 | iov_iter_reexpand(to, iov_iter_count(to) + shorted); |
| 614 | 621 | return ret; |
| @@ -548,6 +548,20 @@ out_free_ext_minor: · 547 unmodified lines | ||
| 548 | 548 | } |
| 549 | 549 | EXPORT_SYMBOL(device_add_disk); |
| 550 | 550 | |
| 551 | +/** | |
| 552 | + * blk_mark_disk_dead - mark a disk as dead | |
| 553 | + * @disk: disk to mark as dead | |
| 554 | + * | |
| 555 | + * Mark as disk as dead (e.g. surprise removed) and don't accept any new I/O | |
| 556 | + * to this disk. | |
| 557 | + */ | |
| 558 | +void blk_mark_disk_dead(struct gendisk *disk) | |
| 559 | +{ | |
| 560 | + set_bit(GD_DEAD, &disk->state); | |
| 561 | + blk_queue_start_drain(disk->queue); | |
| 562 | +} | |
| 563 | +EXPORT_SYMBOL_GPL(blk_mark_disk_dead); | |
| 564 | + | |
| 551 | 565 | /** |
| 552 | 566 | * del_gendisk - remove the gendisk |
| 553 | 567 | * @disk: the struct gendisk to remove |
| @@ -25,12 +25,9 @@ struct alg_type_list { · 24 unmodified lines | ||
| 25 | 25 | struct list_head list; |
| 26 | 26 | }; |
| 27 | 27 | |
| 28 | −static atomic_long_t alg_memory_allocated; | |
| 29 | − | |
| 30 | 28 | static struct proto alg_proto = { |
| 31 | 29 | .name = "ALG", |
| 32 | 30 | .owner = THIS_MODULE, |
| 33 | − .memory_allocated = &alg_memory_allocated, | |
| 34 | 31 | .obj_size = sizeof(struct alg_sock), |
| 35 | 32 | }; |
| 36 | 33 | |
| @@ -1324,3 +1324,4 @@ module_exit(crypto_algapi_exit); · 1323 unmodified lines | ||
| 1324 | 1324 | |
| 1325 | 1325 | MODULE_LICENSE("GPL"); |
| 1326 | 1326 | MODULE_DESCRIPTION("Cryptographic algorithms API"); |
| 1327 | +MODULE_SOFTDEP("pre: cryptomgr"); | |
| @@ -643,4 +643,3 @@ EXPORT_SYMBOL_GPL(crypto_req_done); · 642 unmodified lines | ||
| 643 | 643 | |
| 644 | 644 | MODULE_DESCRIPTION("Cryptographic core API"); |
| 645 | 645 | MODULE_LICENSE("GPL"); |
| 646 | −MODULE_SOFTDEP("pre: cryptomgr"); | |
| @@ -15,12 +15,12 @@ · 14 unmodified lines | ||
| 15 | 15 | static int crypto_blake2s_update_generic(struct shash_desc *desc, |
| 16 | 16 | const u8 *in, unsigned int inlen) |
| 17 | 17 | { |
| 18 | − return crypto_blake2s_update(desc, in, inlen, blake2s_compress_generic); | |
| 18 | + return crypto_blake2s_update(desc, in, inlen, true); | |
| 19 | 19 | } |
| 20 | 20 | |
| 21 | 21 | static int crypto_blake2s_final_generic(struct shash_desc *desc, u8 *out) |
| 22 | 22 | { |
| 23 | − return crypto_blake2s_final(desc, out, blake2s_compress_generic); | |
| 23 | + return crypto_blake2s_final(desc, out, true); | |
| 24 | 24 | } |
| 25 | 25 | |
| 26 | 26 | #define BLAKE2S_ALG(name, driver_name, digest_size) \ |
| @@ -44,6 +44,7 @@ static struct var_t vars[] = { · 43 unmodified lines | ||
| 44 | 44 | { CAPS_START, .u.s = {"[:dv ap 160] " } }, |
| 45 | 45 | { CAPS_STOP, .u.s = {"[:dv ap 100 ] " } }, |
| 46 | 46 | { RATE, .u.n = {"[:ra %d] ", 180, 75, 650, 0, 0, NULL } }, |
| 47 | + { PITCH, .u.n = {"[:dv ap %d] ", 122, 50, 350, 0, 0, NULL } }, | |
| 47 | 48 | { INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } }, |
| 48 | 49 | { VOL, .u.n = {"[:dv g5 %d] ", 86, 60, 86, 0, 0, NULL } }, |
| 49 | 50 | { PUNCT, .u.n = {"[:pu %c] ", 0, 0, 2, 0, 0, "nsa" } }, |
| @@ -11,6 +11,7 @@ menuconfig ACPI · 10 unmodified lines | ||
| 11 | 11 | depends on ARCH_SUPPORTS_ACPI |
| 12 | 12 | select PNP |
| 13 | 13 | select NLS |
| 14 | + select CRC32 | |
| 14 | 15 | default y if X86 |
| 15 | 16 | help |
| 16 | 17 | Advanced Configuration and Power Interface (ACPI) support for |
| @@ -1361,9 +1361,17 @@ static void __init arm_smmu_v3_pmcg_init_resources(struct resource *res, · 1360 unmodified lines | ||
| 1361 | 1361 | res[0].start = pmcg->page0_base_address; |
| 1362 | 1362 | res[0].end = pmcg->page0_base_address + SZ_4K - 1; |
| 1363 | 1363 | res[0].flags = IORESOURCE_MEM; |
| 1364 | − res[1].start = pmcg->page1_base_address; | |
| 1365 | − res[1].end = pmcg->page1_base_address + SZ_4K - 1; | |
| 1366 | − res[1].flags = IORESOURCE_MEM; | |
| 1364 | + /* | |
| 1365 | + * The initial version in DEN0049C lacked a way to describe register | |
| 1366 | + * page 1, which makes it broken for most PMCG implementations; in | |
| 1367 | + * that case, just let the driver fail gracefully if it expects to | |
| 1368 | + * find a second memory resource. | |
| 1369 | + */ | |
| 1370 | + if (node->revision > 0) { | |
| 1371 | + res[1].start = pmcg->page1_base_address; | |
| 1372 | + res[1].end = pmcg->page1_base_address + SZ_4K - 1; | |
| 1373 | + res[1].flags = IORESOURCE_MEM; | |
| 1374 | + } | |
| 1367 | 1375 | |
| 1368 | 1376 | if (pmcg->overflow_gsiv) |
| 1369 | 1377 | acpi_iort_register_irq(pmcg->overflow_gsiv, "overflow", |
| @@ -2065,6 +2065,16 @@ bool acpi_ec_dispatch_gpe(void) · 2064 unmodified lines | ||
| 2065 | 2065 | if (acpi_any_gpe_status_set(first_ec->gpe)) |
| 2066 | 2066 | return true; |
| 2067 | 2067 | |
| 2068 | + /* | |
| 2069 | + * Cancel the SCI wakeup and process all pending events in case there | |
| 2070 | + * are any wakeup ones in there. | |
| 2071 | + * | |
| 2072 | + * Note that if any non-EC GPEs are active at this point, the SCI will | |
| 2073 | + * retrigger after the rearming in acpi_s2idle_wake(), so no events | |
| 2074 | + * should be missed by canceling the wakeup here. | |
| 2075 | + */ | |
| 2076 | + pm_system_cancel_wakeup(); | |
| 2077 | + | |
| 2068 | 2078 | /* |
| 2069 | 2079 | * Dispatch the EC GPE in-band, but do not report wakeup in any case |
| 2070 | 2080 | * to allow the caller to process events properly after that. |
| @@ -96,6 +96,11 @@ static const struct dmi_system_id processor_power_dmi_table[] = { · 95 unmodified lines | ||
| 96 | 96 | DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK Computer Inc."), |
| 97 | 97 | DMI_MATCH(DMI_PRODUCT_NAME,"L8400B series Notebook PC")}, |
| 98 | 98 | (void *)1}, |
| 99 | + /* T40 can not handle C3 idle state */ | |
| 100 | + { set_max_cstate, "IBM ThinkPad T40", { | |
| 101 | + DMI_MATCH(DMI_SYS_VENDOR, "IBM"), | |
| 102 | + DMI_MATCH(DMI_PRODUCT_NAME, "23737CU")}, | |
| 103 | + (void *)2}, | |
| 99 | 104 | {}, |
| 100 | 105 | }; |
| 101 | 106 | |
| @@ -736,21 +736,15 @@ bool acpi_s2idle_wake(void) · 735 unmodified lines | ||
| 736 | 736 | return true; |
| 737 | 737 | } |
| 738 | 738 | |
| 739 | − /* Check non-EC GPE wakeups and dispatch the EC GPE. */ | |
| 739 | + /* | |
| 740 | + * Check non-EC GPE wakeups and if there are none, cancel the | |
| 741 | + * SCI-related wakeup and dispatch the EC GPE. | |
| 742 | + */ | |
| 740 | 743 | if (acpi_ec_dispatch_gpe()) { |
| 741 | 744 | pm_pr_dbg("ACPI non-EC GPE wakeup\n"); |
| 742 | 745 | return true; |
| 743 | 746 | } |
| 744 | 747 | |
| 745 | − /* | |
| 746 | − * Cancel the SCI wakeup and process all pending events in case | |
| 747 | − * there are any wakeup ones in there. | |
| 748 | − * | |
| 749 | − * Note that if any non-EC GPEs are active at this point, the | |
| 750 | − * SCI will retrigger after the rearming below, so no events | |
| 751 | − * should be missed by canceling the wakeup here. | |
| 752 | − */ | |
| 753 | − pm_system_cancel_wakeup(); | |
| 754 | 748 | acpi_os_wait_events_complete(); |
| 755 | 749 | |
| 756 | 750 | /* |
| @@ -764,6 +758,7 @@ bool acpi_s2idle_wake(void) · 7 unmodified lines | ||
| 764 | 758 | return true; |
| 765 | 759 | } |
| 766 | 760 | |
| 761 | + pm_wakeup_clear(acpi_sci_irq); | |
| 767 | 762 | rearm_wake_irq(acpi_sci_irq); |
| 768 | 763 | } |
| 769 | 764 | |
| @@ -400,7 +400,7 @@ int __init_or_acpilib acpi_table_parse_entries_array( · 399 unmodified lines | ||
| 400 | 400 | |
| 401 | 401 | acpi_get_table(id, instance, &table_header); |
| 402 | 402 | if (!table_header) { |
| 403 | − pr_warn("%4.4s not present\n", id); | |
| 403 | + pr_debug("%4.4s not present\n", id); | |
| 404 | 404 | return -ENODEV; |
| 405 | 405 | } |
| 406 | 406 | |
| @@ -424,15 +424,11 @@ static int lps0_device_attach(struct acpi_device *adev, · 423 unmodified lines | ||
| 424 | 424 | mem_sleep_current = PM_SUSPEND_TO_IDLE; |
| 425 | 425 | |
| 426 | 426 | /* |
| 427 | − * Some Intel based LPS0 systems, like ASUS Zenbook UX430UNR/i7-8550U don't | |
| 428 | − * use intel-hid or intel-vbtn but require the EC GPE to be enabled while | |
| 429 | − * suspended for certain wakeup devices to work, so mark it as wakeup-capable. | |
| 430 | − * | |
| 431 | − * Only enable on !AMD as enabling this universally causes problems for a number | |
| 432 | − * of AMD based systems. | |
| 427 | + * Some LPS0 systems, like ASUS Zenbook UX430UNR/i7-8550U, require the | |
| 428 | + * EC GPE to be enabled while suspended for certain wakeup devices to | |
| 429 | + * work, so mark it as wakeup-capable. | |
| 433 | 430 | */ |
| 434 | − if (!acpi_s2idle_vendor_amd()) | |
| 435 | − acpi_ec_mark_gpe_for_wake(); | |
| 431 | + acpi_ec_mark_gpe_for_wake(); | |
| 436 | 432 | |
| 437 | 433 | return 0; |
| 438 | 434 | } |
| @@ -2007,6 +2007,9 @@ static bool ata_log_supported(struct ata_device *dev, u8 log) · 2006 unmodified lines | ||
| 2007 | 2007 | { |
| 2008 | 2008 | struct ata_port *ap = dev->link->ap; |
| 2009 | 2009 | |
| 2010 | + if (dev->horkage & ATA_HORKAGE_NO_LOG_DIR) | |
| 2011 | + return false; | |
| 2012 | + | |
| 2010 | 2013 | if (ata_read_log_page(dev, ATA_LOG_DIRECTORY, 0, ap->sector_buf, 1)) |
| 2011 | 2014 | return false; |
| 2012 | 2015 | return get_unaligned_le16(&ap->sector_buf[log * 2]) ? true : false; |
| @@ -2445,23 +2448,21 @@ static void ata_dev_config_cpr(struct ata_device *dev) · 432 unmodified lines | ||
| 2445 | 2448 | struct ata_cpr_log *cpr_log = NULL; |
| 2446 | 2449 | u8 *desc, *buf = NULL; |
| 2447 | 2450 | |
| 2448 | − if (!ata_identify_page_supported(dev, | |
| 2449 | − ATA_LOG_CONCURRENT_POSITIONING_RANGES)) | |
| 2451 | + if (ata_id_major_version(dev->id) < 11 || | |
| 2452 | + !ata_log_supported(dev, ATA_LOG_CONCURRENT_POSITIONING_RANGES)) | |
| 2450 | 2453 | goto out; |
| 2451 | 2454 | |
| 2452 | 2455 | /* |
| 2453 | − * Read IDENTIFY DEVICE data log, page 0x47 | |
| 2454 | − * (concurrent positioning ranges). We can have at most 255 32B range | |
| 2455 | − * descriptors plus a 64B header. | |
| 2456 | + * Read the concurrent positioning ranges log (0x47). We can have at | |
| 2457 | + * most 255 32B range descriptors plus a 64B header. | |
| 2456 | 2458 | */ |
| 2457 | 2459 | buf_len = (64 + 255 * 32 + 511) & ~511; |
| 2458 | 2460 | buf = kzalloc(buf_len, GFP_KERNEL); |
| 2459 | 2461 | if (!buf) |
| 2460 | 2462 | goto out; |
| 2461 | 2463 | |
| 2462 | − err_mask = ata_read_log_page(dev, ATA_LOG_IDENTIFY_DEVICE, | |
| 2463 | − ATA_LOG_CONCURRENT_POSITIONING_RANGES, | |
| 2464 | − buf, buf_len >> 9); | |
| 2464 | + err_mask = ata_read_log_page(dev, ATA_LOG_CONCURRENT_POSITIONING_RANGES, | |
| 2465 | + 0, buf, buf_len >> 9); | |
| 2465 | 2466 | if (err_mask) |
| 2466 | 2467 | goto out; |
| 2467 | 2468 | |
| @@ -4028,6 +4029,7 @@ static const struct ata_blacklist_entry ata_device_blacklist [] = { · 1560 unmodified lines | ||
| 4028 | 4029 | |
| 4029 | 4030 | /* devices that don't properly handle TRIM commands */ |
| 4030 | 4031 | { "SuperSSpeed S238*", NULL, ATA_HORKAGE_NOTRIM, }, |
| 4032 | + { "M88V29*", NULL, ATA_HORKAGE_NOTRIM, }, | |
| 4031 | 4033 | |
| 4032 | 4034 | /* |
| 4033 | 4035 | * As defined, the DRAT (Deterministic Read After Trim) and RZAT |
| @@ -4073,6 +4075,13 @@ static const struct ata_blacklist_entry ata_device_blacklist [] = { · 39 unmodified lines | ||
| 4073 | 4075 | { "WDC WD3000JD-*", NULL, ATA_HORKAGE_WD_BROKEN_LPM }, |
| 4074 | 4076 | { "WDC WD3200JD-*", NULL, ATA_HORKAGE_WD_BROKEN_LPM }, |
| 4075 | 4077 | |
| 4078 | + /* | |
| 4079 | + * This sata dom device goes on a walkabout when the ATA_LOG_DIRECTORY | |
| 4080 | + * log page is accessed. Ensure we never ask for this log page with | |
| 4081 | + * these devices. | |
| 4082 | + */ | |
| 4083 | + { "SATADOM-ML 3ME", NULL, ATA_HORKAGE_NO_LOG_DIR }, | |
| 4084 | + | |
| 4076 | 4085 | /* End Marker */ |
| 4077 | 4086 | { } |
| 4078 | 4087 | }; |
| @@ -919,6 +919,20 @@ static int hpt37x_init_one(struct pci_dev *dev, const struct pci_device_id *id) · 918 unmodified lines | ||
| 919 | 919 | irqmask &= ~0x10; |
| 920 | 920 | pci_write_config_byte(dev, 0x5a, irqmask); |
| 921 | 921 | |
| 922 | + /* | |
| 923 | + * HPT371 chips physically have only one channel, the secondary one, | |
| 924 | + * but the primary channel registers do exist! Go figure... | |
| 925 | + * So, we manually disable the non-existing channel here | |
| 926 | + * (if the BIOS hasn't done this already). | |
| 927 | + */ | |
| 928 | + if (dev->device == PCI_DEVICE_ID_TTI_HPT371) { | |
| 929 | + u8 mcr1; | |
| 930 | + | |
| 931 | + pci_read_config_byte(dev, 0x50, &mcr1); | |
| 932 | + mcr1 &= ~0x04; | |
| 933 | + pci_write_config_byte(dev, 0x50, mcr1); | |
| 934 | + } | |
| 935 | + | |
| 922 | 936 | /* |
| 923 | 937 | * default to pci clock. make sure MA15/16 are set to output |
| 924 | 938 | * to prevent drives having problems with 40-pin cables. Needed |
| @@ -950,14 +964,14 @@ static int hpt37x_init_one(struct pci_dev *dev, const struct pci_device_id *id) · 25 unmodified lines | ||
| 950 | 964 | |
| 951 | 965 | if ((freq >> 12) != 0xABCDE) { |
| 952 | 966 | int i; |
| 953 | − u8 sr; | |
| 967 | + u16 sr; | |
| 954 | 968 | u32 total = 0; |
| 955 | 969 | |
| 956 | 970 | dev_warn(&dev->dev, "BIOS has not set timing clocks\n"); |
| 957 | 971 | |
| 958 | 972 | /* This is the process the HPT371 BIOS is reported to use */ |
| 959 | 973 | for (i = 0; i < 128; i++) { |
| 960 | − pci_read_config_byte(dev, 0x78, &sr); | |
| 974 | + pci_read_config_word(dev, 0x78, &sr); | |
| 961 | 975 | total += sr & 0x1FF; |
| 962 | 976 | udelay(15); |
| 963 | 977 | } |
| @@ -322,7 +322,7 @@ static void fsl_sata_set_irq_coalescing(struct ata_host *host, · 321 unmodified lines | ||
| 322 | 322 | static ssize_t fsl_sata_intr_coalescing_show(struct device *dev, |
| 323 | 323 | struct device_attribute *attr, char *buf) |
| 324 | 324 | { |
| 325 | − return sysfs_emit(buf, "%d %d\n", | |
| 325 | + return sysfs_emit(buf, "%u %u\n", | |
| 326 | 326 | intr_coalescing_count, intr_coalescing_ticks); |
| 327 | 327 | } |
| 328 | 328 | |
| @@ -332,10 +332,8 @@ static ssize_t fsl_sata_intr_coalescing_store(struct device *dev, · 3 unmodified lines | ||
| 332 | 332 | { |
| 333 | 333 | unsigned int coalescing_count, coalescing_ticks; |
| 334 | 334 | |
| 335 | − if (sscanf(buf, "%d%d", | |
| 336 | − &coalescing_count, | |
| 337 | − &coalescing_ticks) != 2) { | |
| 338 | − printk(KERN_ERR "fsl-sata: wrong parameter format.\n"); | |
| 335 | + if (sscanf(buf, "%u%u", &coalescing_count, &coalescing_ticks) != 2) { | |
| 336 | + dev_err(dev, "fsl-sata: wrong parameter format.\n"); | |
| 339 | 337 | return -EINVAL; |
| 340 | 338 | } |
| 341 | 339 | |
| @@ -359,7 +357,7 @@ static ssize_t fsl_sata_rx_watermark_show(struct device *dev, · 17 unmodified lines | ||
| 359 | 357 | rx_watermark &= 0x1f; |
| 360 | 358 | spin_unlock_irqrestore(&host->lock, flags); |
| 361 | 359 | |
| 362 | − return sysfs_emit(buf, "%d\n", rx_watermark); | |
| 360 | + return sysfs_emit(buf, "%u\n", rx_watermark); | |
| 363 | 361 | } |
| 364 | 362 | |
| 365 | 363 | static ssize_t fsl_sata_rx_watermark_store(struct device *dev, |
| @@ -373,8 +371,8 @@ static ssize_t fsl_sata_rx_watermark_store(struct device *dev, · 7 unmodified lines | ||
| 373 | 371 | void __iomem *csr_base = host_priv->csr_base; |
| 374 | 372 | u32 temp; |
| 375 | 373 | |
| 376 | − if (sscanf(buf, "%d", &rx_watermark) != 1) { | |
| 377 | − printk(KERN_ERR "fsl-sata: wrong parameter format.\n"); | |
| 374 | + if (kstrtouint(buf, 10, &rx_watermark) < 0) { | |
| 375 | + dev_err(dev, "fsl-sata: wrong parameter format.\n"); | |
| 378 | 376 | return -EINVAL; |
| 379 | 377 | } |
| 380 | 378 | |
| @@ -382,8 +380,8 @@ static ssize_t fsl_sata_rx_watermark_store(struct device *dev, · 1 unmodified line | ||
| 382 | 380 | temp = ioread32(csr_base + TRANSCFG); |
| 383 | 381 | temp &= 0xffffffe0; |
| 384 | 382 | iowrite32(temp | rx_watermark, csr_base + TRANSCFG); |
| 385 | − | |
| 386 | 383 | spin_unlock_irqrestore(&host->lock, flags); |
| 384 | + | |
| 387 | 385 | return strlen(buf); |
| 388 | 386 | } |
| 389 | 387 | |
| @@ -629,6 +629,9 @@ re_probe: · 628 unmodified lines | ||
| 629 | 629 | drv->remove(dev); |
| 630 | 630 | |
| 631 | 631 | devres_release_all(dev); |
| 632 | + arch_teardown_dma_ops(dev); | |
| 633 | + kfree(dev->dma_range_map); | |
| 634 | + dev->dma_range_map = NULL; | |
| 632 | 635 | driver_sysfs_remove(dev); |
| 633 | 636 | dev->driver = NULL; |
| 634 | 637 | dev_set_drvdata(dev, NULL); |
| @@ -1209,6 +1212,8 @@ static void __device_release_driver(struct device *dev, struct device *parent) · 574 unmodified lines | ||
| 1209 | 1212 | |
| 1210 | 1213 | devres_release_all(dev); |
| 1211 | 1214 | arch_teardown_dma_ops(dev); |
| 1215 | + kfree(dev->dma_range_map); | |
| 1216 | + dev->dma_range_map = NULL; | |
| 1212 | 1217 | dev->driver = NULL; |
| 1213 | 1218 | dev_set_drvdata(dev, NULL); |
| 1214 | 1219 | if (dev->pm_domain && dev->pm_domain->dismiss) |
| @@ -34,7 +34,8 @@ suspend_state_t pm_suspend_target_state; · 33 unmodified lines | ||
| 34 | 34 | bool events_check_enabled __read_mostly; |
| 35 | 35 | |
| 36 | 36 | /* First wakeup IRQ seen by the kernel in the last cycle. */ |
| 37 | −unsigned int pm_wakeup_irq __read_mostly; | |
| 37 | +static unsigned int wakeup_irq[2] __read_mostly; | |
| 38 | +static DEFINE_RAW_SPINLOCK(wakeup_irq_lock); | |
| 38 | 39 | |
| 39 | 40 | /* If greater than 0 and the system is suspending, terminate the suspend. */ |
| 40 | 41 | static atomic_t pm_abort_suspend __read_mostly; |
| @@ -942,19 +943,45 @@ void pm_system_cancel_wakeup(void) · 901 unmodified lines | ||
| 942 | 943 | atomic_dec_if_positive(&pm_abort_suspend); |
| 943 | 944 | } |
| 944 | 945 | |
| 945 | −void pm_wakeup_clear(bool reset) | |
| 946 | +void pm_wakeup_clear(unsigned int irq_number) | |
| 946 | 947 | { |
| 947 | − pm_wakeup_irq = 0; | |
| 948 | − if (reset) | |
| 948 | + raw_spin_lock_irq(&wakeup_irq_lock); | |
| 949 | + | |
| 950 | + if (irq_number && wakeup_irq[0] == irq_number) | |
| 951 | + wakeup_irq[0] = wakeup_irq[1]; | |
| 952 | + else | |
| 953 | + wakeup_irq[0] = 0; | |
| 954 | + | |
| 955 | + wakeup_irq[1] = 0; | |
| 956 | + | |
| 957 | + raw_spin_unlock_irq(&wakeup_irq_lock); | |
| 958 | + | |
| 959 | + if (!irq_number) | |
| 949 | 960 | atomic_set(&pm_abort_suspend, 0); |
| 950 | 961 | } |
| 951 | 962 | |
| 952 | 963 | void pm_system_irq_wakeup(unsigned int irq_number) |
| 953 | 964 | { |
| 954 | − if (pm_wakeup_irq == 0) { | |
| 955 | − pm_wakeup_irq = irq_number; | |
| 965 | + unsigned long flags; | |
| 966 | + | |
| 967 | + raw_spin_lock_irqsave(&wakeup_irq_lock, flags); | |
| 968 | + | |
| 969 | + if (wakeup_irq[0] == 0) | |
| 970 | + wakeup_irq[0] = irq_number; | |
| 971 | + else if (wakeup_irq[1] == 0) | |
| 972 | + wakeup_irq[1] = irq_number; | |
| 973 | + else | |
| 974 | + irq_number = 0; | |
| 975 | + | |
| 976 | + raw_spin_unlock_irqrestore(&wakeup_irq_lock, flags); | |
| 977 | + | |
| 978 | + if (irq_number) | |
| 956 | 979 | pm_system_wakeup(); |
| 957 | − } | |
| 980 | +} | |
| 981 | + | |
| 982 | +unsigned int pm_wakeup_irq(void) | |
| 983 | +{ | |
| 984 | + return wakeup_irq[0]; | |
| 958 | 985 | } |
| 959 | 986 | |
| 960 | 987 | /** |
| @@ -189,11 +189,9 @@ static void regmap_irq_sync_unlock(struct irq_data *data) · 188 unmodified lines | ||
| 189 | 189 | ret = regmap_write(map, reg, d->mask_buf[i]); |
| 190 | 190 | if (d->chip->clear_ack) { |
| 191 | 191 | if (d->chip->ack_invert && !ret) |
| 192 | − ret = regmap_write(map, reg, | |
| 193 | − d->mask_buf[i]); | |
| 192 | + ret = regmap_write(map, reg, UINT_MAX); | |
| 194 | 193 | else if (!ret) |
| 195 | − ret = regmap_write(map, reg, | |
| 196 | − ~d->mask_buf[i]); | |
| 194 | + ret = regmap_write(map, reg, 0); | |
| 197 | 195 | } |
| 198 | 196 | if (ret != 0) |
| 199 | 197 | dev_err(d->map->dev, "Failed to ack 0x%x: %d\n", |
| @@ -556,11 +554,9 @@ static irqreturn_t regmap_irq_thread(int irq, void *d) · 356 unmodified lines | ||
| 556 | 554 | data->status_buf[i]); |
| 557 | 555 | if (chip->clear_ack) { |
| 558 | 556 | if (chip->ack_invert && !ret) |
| 559 | − ret = regmap_write(map, reg, | |
| 560 | − data->status_buf[i]); | |
| 557 | + ret = regmap_write(map, reg, UINT_MAX); | |
| 561 | 558 | else if (!ret) |
| 562 | − ret = regmap_write(map, reg, | |
| 563 | − ~data->status_buf[i]); | |
| 559 | + ret = regmap_write(map, reg, 0); | |
| 564 | 560 | } |
| 565 | 561 | if (ret != 0) |
| 566 | 562 | dev_err(map->dev, "Failed to ack 0x%x: %d\n", |
| @@ -817,13 +813,9 @@ int regmap_add_irq_chip_fwnode(struct fwnode_handle *fwnode, · 250 unmodified lines | ||
| 817 | 813 | d->status_buf[i] & d->mask_buf[i]); |
| 818 | 814 | if (chip->clear_ack) { |
| 819 | 815 | if (chip->ack_invert && !ret) |
| 820 | − ret = regmap_write(map, reg, | |
| 821 | − (d->status_buf[i] & | |
| 822 | − d->mask_buf[i])); | |
| 816 | + ret = regmap_write(map, reg, UINT_MAX); | |
| 823 | 817 | else if (!ret) |
| 824 | − ret = regmap_write(map, reg, | |
| 825 | − ~(d->status_buf[i] & | |
| 826 | − d->mask_buf[i])); | |
| 818 | + ret = regmap_write(map, reg, 0); | |
| 827 | 819 | } |
| 828 | 820 | if (ret != 0) { |
| 829 | 821 | dev_err(map->dev, "Failed to ack 0x%x: %d\n", |
| @@ -79,6 +79,7 @@ · 78 unmodified lines | ||
| 79 | 79 | #include <linux/ioprio.h> |
| 80 | 80 | #include <linux/blk-cgroup.h> |
| 81 | 81 | #include <linux/sched/mm.h> |
| 82 | +#include <linux/statfs.h> | |
| 82 | 83 | |
| 83 | 84 | #include "loop.h" |
| 84 | 85 | |
| @@ -774,8 +775,13 @@ static void loop_config_discard(struct loop_device *lo) · 689 unmodified lines | ||
| 774 | 775 | granularity = 0; |
| 775 | 776 | |
| 776 | 777 | } else { |
| 778 | + struct kstatfs sbuf; | |
| 779 | + | |
| 777 | 780 | max_discard_sectors = UINT_MAX >> 9; |
| 778 | − granularity = inode->i_sb->s_blocksize; | |
| 781 | + if (!vfs_statfs(&file->f_path, &sbuf)) | |
| 782 | + granularity = sbuf.f_bsize; | |
| 783 | + else | |
| 784 | + max_discard_sectors = 0; | |
| 779 | 785 | } |
| 780 | 786 | |
| 781 | 787 | if (max_discard_sectors) { |
| @@ -1082,7 +1088,7 @@ out_putf: · 300 unmodified lines | ||
| 1082 | 1088 | return error; |
| 1083 | 1089 | } |
| 1084 | 1090 | |
| 1085 | −static void __loop_clr_fd(struct loop_device *lo) | |
| 1091 | +static void __loop_clr_fd(struct loop_device *lo, bool release) | |
| 1086 | 1092 | { |
| 1087 | 1093 | struct file *filp; |
| 1088 | 1094 | gfp_t gfp = lo->old_gfp_mask; |
| @@ -1144,6 +1150,8 @@ static void __loop_clr_fd(struct loop_device *lo) · 55 unmodified lines | ||
| 1144 | 1150 | /* let user-space know about this change */ |
| 1145 | 1151 | kobject_uevent(&disk_to_dev(lo->lo_disk)->kobj, KOBJ_CHANGE); |
| 1146 | 1152 | mapping_set_gfp_mask(filp->f_mapping, gfp); |
| 1153 | + /* This is safe: open() is still holding a reference. */ | |
| 1154 | + module_put(THIS_MODULE); | |
| 1147 | 1155 | blk_mq_unfreeze_queue(lo->lo_queue); |
| 1148 | 1156 | |
| 1149 | 1157 | disk_force_media_change(lo->lo_disk, DISK_EVENT_MEDIA_CHANGE); |
| @@ -1151,52 +1159,44 @@ static void __loop_clr_fd(struct loop_device *lo) · 1 unmodified line | ||
| 1151 | 1159 | if (lo->lo_flags & LO_FLAGS_PARTSCAN) { |
| 1152 | 1160 | int err; |
| 1153 | 1161 | |
| 1154 | − mutex_lock(&lo->lo_disk->open_mutex); | |
| 1162 | + /* | |
| 1163 | + * open_mutex has been held already in release path, so don't | |
| 1164 | + * acquire it if this function is called in such case. | |
| 1165 | + * | |
| 1166 | + * If the reread partition isn't from release path, lo_refcnt | |
| 1167 | + * must be at least one and it can only become zero when the | |
| 1168 | + * current holder is released. | |
| 1169 | + */ | |
| 1170 | + if (!release) | |
| 1171 | + mutex_lock(&lo->lo_disk->open_mutex); | |
| 1155 | 1172 | err = bdev_disk_changed(lo->lo_disk, false); |
| 1156 | − mutex_unlock(&lo->lo_disk->open_mutex); | |
| 1173 | + if (!release) | |
| 1174 | + mutex_unlock(&lo->lo_disk->open_mutex); | |
| 1157 | 1175 | if (err) |
| 1158 | 1176 | pr_warn("%s: partition scan of loop%d failed (rc=%d)\n", |
| 1159 | 1177 | __func__, lo->lo_number, err); |
| 1160 | 1178 | /* Device is gone, no point in returning error */ |
| 1161 | 1179 | } |
| 1162 | 1180 | |
| 1181 | + /* | |
| 1182 | + * lo->lo_state is set to Lo_unbound here after above partscan has | |
| 1183 | + * finished. There cannot be anybody else entering __loop_clr_fd() as | |
| 1184 | + * Lo_rundown state protects us from all the other places trying to | |
| 1185 | + * change the 'lo' device. | |
| 1186 | + */ | |
| 1163 | 1187 | lo->lo_flags = 0; |
| 1164 | 1188 | if (!part_shift) |
| 1165 | 1189 | lo->lo_disk->flags |= GENHD_FL_NO_PART; |
| 1166 | − | |
| 1167 | − fput(filp); | |
| 1168 | −} | |
| 1169 | − | |
| 1170 | −static void loop_rundown_completed(struct loop_device *lo) | |
| 1171 | −{ | |
| 1172 | 1190 | mutex_lock(&lo->lo_mutex); |
| 1173 | 1191 | lo->lo_state = Lo_unbound; |
| 1174 | 1192 | mutex_unlock(&lo->lo_mutex); |
| 1175 | − module_put(THIS_MODULE); | |
| 1176 | −} | |
| 1177 | − | |
| 1178 | −static void loop_rundown_workfn(struct work_struct *work) | |
| 1179 | −{ | |
| 1180 | − struct loop_device *lo = container_of(work, struct loop_device, | |
| 1181 | − rundown_work); | |
| 1182 | − struct block_device *bdev = lo->lo_device; | |
| 1183 | − struct gendisk *disk = lo->lo_disk; | |
| 1184 | − | |
| 1185 | − __loop_clr_fd(lo); | |
| 1186 | − kobject_put(&bdev->bd_device.kobj); | |
| 1187 | − module_put(disk->fops->owner); | |
| 1188 | − loop_rundown_completed(lo); | |
| 1189 | −} | |
| 1190 | − | |
| 1191 | −static void loop_schedule_rundown(struct loop_device *lo) | |
| 1192 | −{ | |
| 1193 | − struct block_device *bdev = lo->lo_device; | |
| 1194 | − struct gendisk *disk = lo->lo_disk; | |
| 1195 | 1193 | |
| 1196 | − __module_get(disk->fops->owner); | |
| 1197 | − kobject_get(&bdev->bd_device.kobj); | |
| 1198 | − INIT_WORK(&lo->rundown_work, loop_rundown_workfn); | |
| 1199 | − queue_work(system_long_wq, &lo->rundown_work); | |
| 1194 | + /* | |
| 1195 | + * Need not hold lo_mutex to fput backing file. Calling fput holding | |
| 1196 | + * lo_mutex triggers a circular lock dependency possibility warning as | |
| 1197 | + * fput can take open_mutex which is usually taken before lo_mutex. | |
| 1198 | + */ | |
| 1199 | + fput(filp); | |
| 1200 | 1200 | } |
| 1201 | 1201 | |
| 1202 | 1202 | static int loop_clr_fd(struct loop_device *lo) |
| @@ -1228,8 +1228,7 @@ static int loop_clr_fd(struct loop_device *lo) · 25 unmodified lines | ||
| 1228 | 1228 | lo->lo_state = Lo_rundown; |
| 1229 | 1229 | mutex_unlock(&lo->lo_mutex); |
| 1230 | 1230 | |
| 1231 | − __loop_clr_fd(lo); | |
| 1232 | − loop_rundown_completed(lo); | |
| 1231 | + __loop_clr_fd(lo, false); | |
| 1233 | 1232 | return 0; |
| 1234 | 1233 | } |
| 1235 | 1234 | |
| @@ -1754,7 +1753,7 @@ static void lo_release(struct gendisk *disk, fmode_t mode) · 518 unmodified lines | ||
| 1754 | 1753 | * In autoclear mode, stop the loop thread |
| 1755 | 1754 | * and remove configuration after last close. |
| 1756 | 1755 | */ |
| 1757 | − loop_schedule_rundown(lo); | |
| 1756 | + __loop_clr_fd(lo, true); | |
| 1758 | 1757 | return; |
| 1759 | 1758 | } else if (lo->lo_state == Lo_bound) { |
| 1760 | 1759 | /* |
| @@ -56,7 +56,6 @@ struct loop_device { · 55 unmodified lines | ||
| 56 | 56 | struct gendisk *lo_disk; |
| 57 | 57 | struct mutex lo_mutex; |
| 58 | 58 | bool idr_visible; |
| 59 | − struct work_struct rundown_work; | |
| 60 | 59 | }; |
| 61 | 60 | |
| 62 | 61 | struct loop_cmd { |
| @@ -4112,7 +4112,7 @@ static void mtip_pci_remove(struct pci_dev *pdev) · 4111 unmodified lines | ||
| 4112 | 4112 | "Completion workers still active!\n"); |
| 4113 | 4113 | } |
| 4114 | 4114 | |
| 4115 | − blk_set_queue_dying(dd->queue); | |
| 4115 | + blk_mark_disk_dead(dd->disk); | |
| 4116 | 4116 | set_bit(MTIP_DDF_REMOVE_PENDING_BIT, &dd->dd_flag); |
| 4117 | 4117 | |
| 4118 | 4118 | /* Clean up the block layer. */ |
| @@ -7185,7 +7185,7 @@ static ssize_t do_rbd_remove(struct bus_type *bus, · 7184 unmodified lines | ||
| 7185 | 7185 | * IO to complete/fail. |
| 7186 | 7186 | */ |
| 7187 | 7187 | blk_mq_freeze_queue(rbd_dev->disk->queue); |
| 7188 | − blk_set_queue_dying(rbd_dev->disk->queue); | |
| 7188 | + blk_mark_disk_dead(rbd_dev->disk); | |
| 7189 | 7189 | } |
| 7190 | 7190 | |
| 7191 | 7191 | del_gendisk(rbd_dev->disk); |
| @@ -2126,7 +2126,7 @@ static void blkfront_closing(struct blkfront_info *info) · 2125 unmodified lines | ||
| 2126 | 2126 | |
| 2127 | 2127 | /* No more blkif_request(). */ |
| 2128 | 2128 | blk_mq_stop_hw_queues(info->rq); |
| 2129 | − blk_set_queue_dying(info->rq); | |
| 2129 | + blk_mark_disk_dead(info->gd); | |
| 2130 | 2130 | set_capacity(info->gd, 0); |
| 2131 | 2131 | |
| 2132 | 2132 | for_each_rinfo(info, rinfo, i) { |
| @@ -366,6 +366,7 @@ static const struct mhi_pci_dev_info mhi_foxconn_sdx55_info = { · 365 unmodified lines | ||
| 366 | 366 | .config = &modem_foxconn_sdx55_config, |
| 367 | 367 | .bar_num = MHI_PCI_DEFAULT_BAR_NUM, |
| 368 | 368 | .dma_data_width = 32, |
| 369 | + .mru_default = 32768, | |
| 369 | 370 | .sideband_wake = false, |
| 370 | 371 | }; |
| 371 | 372 | |
| @@ -401,6 +402,7 @@ static const struct mhi_pci_dev_info mhi_mv31_info = { · 29 unmodified lines | ||
| 401 | 402 | .config = &modem_mv31_config, |
| 402 | 403 | .bar_num = MHI_PCI_DEFAULT_BAR_NUM, |
| 403 | 404 | .dma_data_width = 32, |
| 405 | + .mru_default = 32768, | |
| 404 | 406 | }; |
| 405 | 407 | |
| 406 | 408 | static const struct mhi_channel_config mhi_sierra_em919x_channels[] = { |
| @@ -762,7 +762,7 @@ static bool crng_init_try_arch(struct crng_state *crng) · 761 unmodified lines | ||
| 762 | 762 | return arch_init; |
| 763 | 763 | } |
| 764 | 764 | |
| 765 | −static bool __init crng_init_try_arch_early(struct crng_state *crng) | |
| 765 | +static bool __init crng_init_try_arch_early(void) | |
| 766 | 766 | { |
| 767 | 767 | int i; |
| 768 | 768 | bool arch_init = true; |
| @@ -774,7 +774,7 @@ static bool __init crng_init_try_arch_early(struct crng_state *crng) · 5 unmodified lines | ||
| 774 | 774 | rv = random_get_entropy(); |
| 775 | 775 | arch_init = false; |
| 776 | 776 | } |
| 777 | − crng->state[i] ^= rv; | |
| 777 | + primary_crng.state[i] ^= rv; | |
| 778 | 778 | } |
| 779 | 779 | |
| 780 | 780 | return arch_init; |
| @@ -788,22 +788,20 @@ static void crng_initialize_secondary(struct crng_state *crng) · 7 unmodified lines | ||
| 788 | 788 | crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1; |
| 789 | 789 | } |
| 790 | 790 | |
| 791 | −static void __init crng_initialize_primary(struct crng_state *crng) | |
| 791 | +static void __init crng_initialize_primary(void) | |
| 792 | 792 | { |
| 793 | − _extract_entropy(&crng->state[4], sizeof(u32) * 12); | |
| 794 | − if (crng_init_try_arch_early(crng) && trust_cpu && crng_init < 2) { | |
| 793 | + _extract_entropy(&primary_crng.state[4], sizeof(u32) * 12); | |
| 794 | + if (crng_init_try_arch_early() && trust_cpu && crng_init < 2) { | |
| 795 | 795 | invalidate_batched_entropy(); |
| 796 | 796 | numa_crng_init(); |
| 797 | 797 | crng_init = 2; |
| 798 | 798 | pr_notice("crng init done (trusting CPU's manufacturer)\n"); |
| 799 | 799 | } |
| 800 | − crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1; | |
| 800 | + primary_crng.init_time = jiffies - CRNG_RESEED_INTERVAL - 1; | |
| 801 | 801 | } |
| 802 | 802 | |
| 803 | −static void crng_finalize_init(struct crng_state *crng) | |
| 803 | +static void crng_finalize_init(void) | |
| 804 | 804 | { |
| 805 | − if (crng != &primary_crng || crng_init >= 2) | |
| 806 | − return; | |
| 807 | 805 | if (!system_wq) { |
| 808 | 806 | /* We can't call numa_crng_init until we have workqueues, |
| 809 | 807 | * so mark this for processing later. */ |
| @@ -814,6 +812,7 @@ static void crng_finalize_init(struct crng_state *crng) · 4 unmodified lines | ||
| 814 | 812 | invalidate_batched_entropy(); |
| 815 | 813 | numa_crng_init(); |
| 816 | 814 | crng_init = 2; |
| 815 | + crng_need_final_init = false; | |
| 817 | 816 | process_random_ready_list(); |
| 818 | 817 | wake_up_interruptible(&crng_init_wait); |
| 819 | 818 | kill_fasync(&fasync, SIGIO, POLL_IN); |
| @@ -980,7 +979,8 @@ static void crng_reseed(struct crng_state *crng, bool use_input_pool) · 160 unmodified lines | ||
| 980 | 979 | memzero_explicit(&buf, sizeof(buf)); |
| 981 | 980 | WRITE_ONCE(crng->init_time, jiffies); |
| 982 | 981 | spin_unlock_irqrestore(&crng->lock, flags); |
| 983 | − crng_finalize_init(crng); | |
| 982 | + if (crng == &primary_crng && crng_init < 2) | |
| 983 | + crng_finalize_init(); | |
| 984 | 984 | } |
| 985 | 985 | |
| 986 | 986 | static void _extract_crng(struct crng_state *crng, u8 out[CHACHA_BLOCK_SIZE]) |
| @@ -1697,8 +1697,8 @@ int __init rand_initialize(void) · 710 unmodified lines | ||
| 1697 | 1697 | { |
| 1698 | 1698 | init_std_data(); |
| 1699 | 1699 | if (crng_need_final_init) |
| 1700 | − crng_finalize_init(&primary_crng); | |
| 1701 | − crng_initialize_primary(&primary_crng); | |
| 1700 | + crng_finalize_init(); | |
| 1701 | + crng_initialize_primary(); | |
| 1702 | 1702 | crng_global_init_time = jiffies; |
| 1703 | 1703 | if (ratelimit_disable) { |
| 1704 | 1704 | urandom_warning.interval = 0; |
| @@ -1856,7 +1856,10 @@ static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg) · 151 unmodified lines | ||
| 1856 | 1856 | */ |
| 1857 | 1857 | if (!capable(CAP_SYS_ADMIN)) |
| 1858 | 1858 | return -EPERM; |
| 1859 | − input_pool.entropy_count = 0; | |
| 1859 | + if (xchg(&input_pool.entropy_count, 0) && random_write_wakeup_bits) { | |
| 1860 | + wake_up_interruptible(&random_write_wait); | |
| 1861 | + kill_fasync(&fasync, SIGIO, POLL_OUT); | |
| 1862 | + } | |
| 1860 | 1863 | return 0; |
| 1861 | 1864 | case RNDRESEEDCRNG: |
| 1862 | 1865 | if (!capable(CAP_SYS_ADMIN)) |
| @@ -2205,13 +2208,15 @@ void add_hwgenerator_randomness(const char *buffer, size_t count, · 342 unmodified lines | ||
| 2205 | 2208 | return; |
| 2206 | 2209 | } |
| 2207 | 2210 | |
| 2208 | − /* Suspend writing if we're above the trickle threshold. | |
| 2211 | + /* Throttle writing if we're above the trickle threshold. | |
| 2209 | 2212 | * We'll be woken up again once below random_write_wakeup_thresh, |
| 2210 | − * or when the calling thread is about to terminate. | |
| 2213 | + * when the calling thread is about to terminate, or once | |
| 2214 | + * CRNG_RESEED_INTERVAL has lapsed. | |
| 2211 | 2215 | */ |
| 2212 | − wait_event_interruptible(random_write_wait, | |
| 2216 | + wait_event_interruptible_timeout(random_write_wait, | |
| 2213 | 2217 | !system_wq || kthread_should_stop() || |
| 2214 | − POOL_ENTROPY_BITS() <= random_write_wakeup_bits); | |
| 2218 | + POOL_ENTROPY_BITS() <= random_write_wakeup_bits, | |
| 2219 | + CRNG_RESEED_INTERVAL); | |
| 2215 | 2220 | mix_pool_bytes(buffer, count); |
| 2216 | 2221 | credit_entropy_bits(entropy); |
| 2217 | 2222 | } |
| @@ -139,11 +139,10 @@ static const struct ingenic_cgu_clk_info jz4725b_cgu_clocks[] = { · 138 unmodified lines | ||
| 139 | 139 | }, |
| 140 | 140 | |
| 141 | 141 | [JZ4725B_CLK_I2S] = { |
| 142 | − "i2s", CGU_CLK_MUX | CGU_CLK_DIV | CGU_CLK_GATE, | |
| 142 | + "i2s", CGU_CLK_MUX | CGU_CLK_DIV, | |
| 143 | 143 | .parents = { JZ4725B_CLK_EXT, JZ4725B_CLK_PLL_HALF, -1, -1 }, |
| 144 | 144 | .mux = { CGU_REG_CPCCR, 31, 1 }, |
| 145 | 145 | .div = { CGU_REG_I2SCDR, 0, 1, 9, -1, -1, -1 }, |
| 146 | − .gate = { CGU_REG_CLKGR, 6 }, | |
| 147 | 146 | }, |
| 148 | 147 | |
| 149 | 148 | [JZ4725B_CLK_SPI] = { |
| @@ -108,42 +108,6 @@ static const struct clk_parent_data gcc_xo_gpll0_gpll4[] = { · 107 unmodified lines | ||
| 108 | 108 | { .hw = &gpll4.clkr.hw }, |
| 109 | 109 | }; |
| 110 | 110 | |
| 111 | −static struct clk_rcg2 system_noc_clk_src = { | |
| 112 | − .cmd_rcgr = 0x0120, | |
| 113 | − .hid_width = 5, | |
| 114 | − .parent_map = gcc_xo_gpll0_map, | |
| 115 | − .clkr.hw.init = &(struct clk_init_data){ | |
| 116 | − .name = "system_noc_clk_src", | |
| 117 | − .parent_data = gcc_xo_gpll0, | |
| 118 | − .num_parents = ARRAY_SIZE(gcc_xo_gpll0), | |
| 119 | − .ops = &clk_rcg2_ops, | |
| 120 | − }, | |
| 121 | −}; | |
| 122 | − | |
| 123 | −static struct clk_rcg2 config_noc_clk_src = { | |
| 124 | − .cmd_rcgr = 0x0150, | |
| 125 | − .hid_width = 5, | |
| 126 | − .parent_map = gcc_xo_gpll0_map, | |
| 127 | − .clkr.hw.init = &(struct clk_init_data){ | |
| 128 | − .name = "config_noc_clk_src", | |
| 129 | − .parent_data = gcc_xo_gpll0, | |
| 130 | − .num_parents = ARRAY_SIZE(gcc_xo_gpll0), | |
| 131 | − .ops = &clk_rcg2_ops, | |
| 132 | − }, | |
| 133 | −}; | |
| 134 | − | |
| 135 | −static struct clk_rcg2 periph_noc_clk_src = { | |
| 136 | − .cmd_rcgr = 0x0190, | |
| 137 | − .hid_width = 5, | |
| 138 | − .parent_map = gcc_xo_gpll0_map, | |
| 139 | − .clkr.hw.init = &(struct clk_init_data){ | |
| 140 | − .name = "periph_noc_clk_src", | |
| 141 | − .parent_data = gcc_xo_gpll0, | |
| 142 | − .num_parents = ARRAY_SIZE(gcc_xo_gpll0), | |
| 143 | − .ops = &clk_rcg2_ops, | |
| 144 | − }, | |
| 145 | −}; | |
| 146 | − | |
| 147 | 111 | static struct freq_tbl ftbl_ufs_axi_clk_src[] = { |
| 148 | 112 | F(50000000, P_GPLL0, 12, 0, 0), |
| 149 | 113 | F(100000000, P_GPLL0, 6, 0, 0), |
| @@ -1150,8 +1114,6 @@ static struct clk_branch gcc_blsp1_ahb_clk = { · 1000 unmodified lines | ||
| 1150 | 1114 | .enable_mask = BIT(17), |
| 1151 | 1115 | .hw.init = &(struct clk_init_data){ |
| 1152 | 1116 | .name = "gcc_blsp1_ahb_clk", |
| 1153 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 1154 | − .num_parents = 1, | |
| 1155 | 1117 | .ops = &clk_branch2_ops, |
| 1156 | 1118 | }, |
| 1157 | 1119 | }, |
| @@ -1435,8 +1397,6 @@ static struct clk_branch gcc_blsp2_ahb_clk = { · 277 unmodified lines | ||
| 1435 | 1397 | .enable_mask = BIT(15), |
| 1436 | 1398 | .hw.init = &(struct clk_init_data){ |
| 1437 | 1399 | .name = "gcc_blsp2_ahb_clk", |
| 1438 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 1439 | − .num_parents = 1, | |
| 1440 | 1400 | .ops = &clk_branch2_ops, |
| 1441 | 1401 | }, |
| 1442 | 1402 | }, |
| @@ -1764,8 +1724,6 @@ static struct clk_branch gcc_lpass_q6_axi_clk = { · 321 unmodified lines | ||
| 1764 | 1724 | .enable_mask = BIT(0), |
| 1765 | 1725 | .hw.init = &(struct clk_init_data){ |
| 1766 | 1726 | .name = "gcc_lpass_q6_axi_clk", |
| 1767 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 1768 | − .num_parents = 1, | |
| 1769 | 1727 | .ops = &clk_branch2_ops, |
| 1770 | 1728 | }, |
| 1771 | 1729 | }, |
| @@ -1778,8 +1736,6 @@ static struct clk_branch gcc_mss_q6_bimc_axi_clk = { · 6 unmodified lines | ||
| 1778 | 1736 | .enable_mask = BIT(0), |
| 1779 | 1737 | .hw.init = &(struct clk_init_data){ |
| 1780 | 1738 | .name = "gcc_mss_q6_bimc_axi_clk", |
| 1781 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 1782 | − .num_parents = 1, | |
| 1783 | 1739 | .ops = &clk_branch2_ops, |
| 1784 | 1740 | }, |
| 1785 | 1741 | }, |
| @@ -1807,9 +1763,6 @@ static struct clk_branch gcc_pcie_0_cfg_ahb_clk = { · 21 unmodified lines | ||
| 1807 | 1763 | .enable_mask = BIT(0), |
| 1808 | 1764 | .hw.init = &(struct clk_init_data){ |
| 1809 | 1765 | .name = "gcc_pcie_0_cfg_ahb_clk", |
| 1810 | − .parent_hws = (const struct clk_hw *[]){ &config_noc_clk_src.clkr.hw }, | |
| 1811 | − .num_parents = 1, | |
| 1812 | − .flags = CLK_SET_RATE_PARENT, | |
| 1813 | 1766 | .ops = &clk_branch2_ops, |
| 1814 | 1767 | }, |
| 1815 | 1768 | }, |
| @@ -1822,9 +1775,6 @@ static struct clk_branch gcc_pcie_0_mstr_axi_clk = { · 6 unmodified lines | ||
| 1822 | 1775 | .enable_mask = BIT(0), |
| 1823 | 1776 | .hw.init = &(struct clk_init_data){ |
| 1824 | 1777 | .name = "gcc_pcie_0_mstr_axi_clk", |
| 1825 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 1826 | − .num_parents = 1, | |
| 1827 | − .flags = CLK_SET_RATE_PARENT, | |
| 1828 | 1778 | .ops = &clk_branch2_ops, |
| 1829 | 1779 | }, |
| 1830 | 1780 | }, |
| @@ -1854,9 +1804,6 @@ static struct clk_branch gcc_pcie_0_slv_axi_clk = { · 23 unmodified lines | ||
| 1854 | 1804 | .enable_mask = BIT(0), |
| 1855 | 1805 | .hw.init = &(struct clk_init_data){ |
| 1856 | 1806 | .name = "gcc_pcie_0_slv_axi_clk", |
| 1857 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 1858 | − .num_parents = 1, | |
| 1859 | − .flags = CLK_SET_RATE_PARENT, | |
| 1860 | 1807 | .ops = &clk_branch2_ops, |
| 1861 | 1808 | }, |
| 1862 | 1809 | }, |
| @@ -1884,9 +1831,6 @@ static struct clk_branch gcc_pcie_1_cfg_ahb_clk = { · 21 unmodified lines | ||
| 1884 | 1831 | .enable_mask = BIT(0), |
| 1885 | 1832 | .hw.init = &(struct clk_init_data){ |
| 1886 | 1833 | .name = "gcc_pcie_1_cfg_ahb_clk", |
| 1887 | − .parent_hws = (const struct clk_hw *[]){ &config_noc_clk_src.clkr.hw }, | |
| 1888 | − .num_parents = 1, | |
| 1889 | − .flags = CLK_SET_RATE_PARENT, | |
| 1890 | 1834 | .ops = &clk_branch2_ops, |
| 1891 | 1835 | }, |
| 1892 | 1836 | }, |
| @@ -1899,9 +1843,6 @@ static struct clk_branch gcc_pcie_1_mstr_axi_clk = { · 6 unmodified lines | ||
| 1899 | 1843 | .enable_mask = BIT(0), |
| 1900 | 1844 | .hw.init = &(struct clk_init_data){ |
| 1901 | 1845 | .name = "gcc_pcie_1_mstr_axi_clk", |
| 1902 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 1903 | − .num_parents = 1, | |
| 1904 | − .flags = CLK_SET_RATE_PARENT, | |
| 1905 | 1846 | .ops = &clk_branch2_ops, |
| 1906 | 1847 | }, |
| 1907 | 1848 | }, |
| @@ -1930,9 +1871,6 @@ static struct clk_branch gcc_pcie_1_slv_axi_clk = { · 22 unmodified lines | ||
| 1930 | 1871 | .enable_mask = BIT(0), |
| 1931 | 1872 | .hw.init = &(struct clk_init_data){ |
| 1932 | 1873 | .name = "gcc_pcie_1_slv_axi_clk", |
| 1933 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 1934 | − .num_parents = 1, | |
| 1935 | − .flags = CLK_SET_RATE_PARENT, | |
| 1936 | 1874 | .ops = &clk_branch2_ops, |
| 1937 | 1875 | }, |
| 1938 | 1876 | }, |
| @@ -1960,8 +1898,6 @@ static struct clk_branch gcc_pdm_ahb_clk = { · 21 unmodified lines | ||
| 1960 | 1898 | .enable_mask = BIT(0), |
| 1961 | 1899 | .hw.init = &(struct clk_init_data){ |
| 1962 | 1900 | .name = "gcc_pdm_ahb_clk", |
| 1963 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 1964 | − .num_parents = 1, | |
| 1965 | 1901 | .ops = &clk_branch2_ops, |
| 1966 | 1902 | }, |
| 1967 | 1903 | }, |
| @@ -1989,9 +1925,6 @@ static struct clk_branch gcc_sdcc1_ahb_clk = { · 21 unmodified lines | ||
| 1989 | 1925 | .enable_mask = BIT(0), |
| 1990 | 1926 | .hw.init = &(struct clk_init_data){ |
| 1991 | 1927 | .name = "gcc_sdcc1_ahb_clk", |
| 1992 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 1993 | − .num_parents = 1, | |
| 1994 | − .flags = CLK_SET_RATE_PARENT, | |
| 1995 | 1928 | .ops = &clk_branch2_ops, |
| 1996 | 1929 | }, |
| 1997 | 1930 | }, |
| @@ -2004,9 +1937,6 @@ static struct clk_branch gcc_sdcc2_ahb_clk = { · 6 unmodified lines | ||
| 2004 | 1937 | .enable_mask = BIT(0), |
| 2005 | 1938 | .hw.init = &(struct clk_init_data){ |
| 2006 | 1939 | .name = "gcc_sdcc2_ahb_clk", |
| 2007 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 2008 | − .num_parents = 1, | |
| 2009 | − .flags = CLK_SET_RATE_PARENT, | |
| 2010 | 1940 | .ops = &clk_branch2_ops, |
| 2011 | 1941 | }, |
| 2012 | 1942 | }, |
| @@ -2034,9 +1964,6 @@ static struct clk_branch gcc_sdcc3_ahb_clk = { · 21 unmodified lines | ||
| 2034 | 1964 | .enable_mask = BIT(0), |
| 2035 | 1965 | .hw.init = &(struct clk_init_data){ |
| 2036 | 1966 | .name = "gcc_sdcc3_ahb_clk", |
| 2037 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 2038 | − .num_parents = 1, | |
| 2039 | − .flags = CLK_SET_RATE_PARENT, | |
| 2040 | 1967 | .ops = &clk_branch2_ops, |
| 2041 | 1968 | }, |
| 2042 | 1969 | }, |
| @@ -2064,9 +1991,6 @@ static struct clk_branch gcc_sdcc4_ahb_clk = { · 21 unmodified lines | ||
| 2064 | 1991 | .enable_mask = BIT(0), |
| 2065 | 1992 | .hw.init = &(struct clk_init_data){ |
| 2066 | 1993 | .name = "gcc_sdcc4_ahb_clk", |
| 2067 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 2068 | − .num_parents = 1, | |
| 2069 | − .flags = CLK_SET_RATE_PARENT, | |
| 2070 | 1994 | .ops = &clk_branch2_ops, |
| 2071 | 1995 | }, |
| 2072 | 1996 | }, |
| @@ -2124,8 +2048,6 @@ static struct clk_branch gcc_tsif_ahb_clk = { · 51 unmodified lines | ||
| 2124 | 2048 | .enable_mask = BIT(0), |
| 2125 | 2049 | .hw.init = &(struct clk_init_data){ |
| 2126 | 2050 | .name = "gcc_tsif_ahb_clk", |
| 2127 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 2128 | − .num_parents = 1, | |
| 2129 | 2051 | .ops = &clk_branch2_ops, |
| 2130 | 2052 | }, |
| 2131 | 2053 | }, |
| @@ -2153,8 +2075,6 @@ static struct clk_branch gcc_ufs_ahb_clk = { · 21 unmodified lines | ||
| 2153 | 2075 | .enable_mask = BIT(0), |
| 2154 | 2076 | .hw.init = &(struct clk_init_data){ |
| 2155 | 2077 | .name = "gcc_ufs_ahb_clk", |
| 2156 | − .parent_hws = (const struct clk_hw *[]){ &config_noc_clk_src.clkr.hw }, | |
| 2157 | − .num_parents = 1, | |
| 2158 | 2078 | .ops = &clk_branch2_ops, |
| 2159 | 2079 | }, |
| 2160 | 2080 | }, |
| @@ -2198,8 +2118,6 @@ static struct clk_branch gcc_ufs_rx_symbol_0_clk = { · 37 unmodified lines | ||
| 2198 | 2118 | .enable_mask = BIT(0), |
| 2199 | 2119 | .hw.init = &(struct clk_init_data){ |
| 2200 | 2120 | .name = "gcc_ufs_rx_symbol_0_clk", |
| 2201 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 2202 | − .num_parents = 1, | |
| 2203 | 2121 | .ops = &clk_branch2_ops, |
| 2204 | 2122 | }, |
| 2205 | 2123 | }, |
| @@ -2213,8 +2131,6 @@ static struct clk_branch gcc_ufs_rx_symbol_1_clk = { · 7 unmodified lines | ||
| 2213 | 2131 | .enable_mask = BIT(0), |
| 2214 | 2132 | .hw.init = &(struct clk_init_data){ |
| 2215 | 2133 | .name = "gcc_ufs_rx_symbol_1_clk", |
| 2216 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 2217 | − .num_parents = 1, | |
| 2218 | 2134 | .ops = &clk_branch2_ops, |
| 2219 | 2135 | }, |
| 2220 | 2136 | }, |
| @@ -2243,8 +2159,6 @@ static struct clk_branch gcc_ufs_tx_symbol_0_clk = { · 22 unmodified lines | ||
| 2243 | 2159 | .enable_mask = BIT(0), |
| 2244 | 2160 | .hw.init = &(struct clk_init_data){ |
| 2245 | 2161 | .name = "gcc_ufs_tx_symbol_0_clk", |
| 2246 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 2247 | − .num_parents = 1, | |
| 2248 | 2162 | .ops = &clk_branch2_ops, |
| 2249 | 2163 | }, |
| 2250 | 2164 | }, |
| @@ -2258,8 +2172,6 @@ static struct clk_branch gcc_ufs_tx_symbol_1_clk = { · 7 unmodified lines | ||
| 2258 | 2172 | .enable_mask = BIT(0), |
| 2259 | 2173 | .hw.init = &(struct clk_init_data){ |
| 2260 | 2174 | .name = "gcc_ufs_tx_symbol_1_clk", |
| 2261 | − .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | |
| 2262 | − .num_parents = 1, | |
| 2263 | 2175 | .ops = &clk_branch2_ops, |
| 2264 | 2176 | }, |
| 2265 | 2177 | }, |
| @@ -2364,8 +2276,6 @@ static struct clk_branch gcc_usb_hs_ahb_clk = { · 98 unmodified lines | ||
| 2364 | 2276 | .enable_mask = BIT(0), |
| 2365 | 2277 | .hw.init = &(struct clk_init_data){ |
| 2366 | 2278 | .name = "gcc_usb_hs_ahb_clk", |
| 2367 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 2368 | − .num_parents = 1, | |
| 2369 | 2279 | .ops = &clk_branch2_ops, |
| 2370 | 2280 | }, |
| 2371 | 2281 | }, |
| @@ -2488,8 +2398,6 @@ static struct clk_branch gcc_boot_rom_ahb_clk = { · 116 unmodified lines | ||
| 2488 | 2398 | .enable_mask = BIT(10), |
| 2489 | 2399 | .hw.init = &(struct clk_init_data){ |
| 2490 | 2400 | .name = "gcc_boot_rom_ahb_clk", |
| 2491 | − .parent_hws = (const struct clk_hw *[]){ &config_noc_clk_src.clkr.hw }, | |
| 2492 | − .num_parents = 1, | |
| 2493 | 2401 | .ops = &clk_branch2_ops, |
| 2494 | 2402 | }, |
| 2495 | 2403 | }, |
| @@ -2503,8 +2411,6 @@ static struct clk_branch gcc_prng_ahb_clk = { · 7 unmodified lines | ||
| 2503 | 2411 | .enable_mask = BIT(13), |
| 2504 | 2412 | .hw.init = &(struct clk_init_data){ |
| 2505 | 2413 | .name = "gcc_prng_ahb_clk", |
| 2506 | − .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | |
| 2507 | − .num_parents = 1, | |
| 2508 | 2414 | .ops = &clk_branch2_ops, |
| 2509 | 2415 | }, |
| 2510 | 2416 | }, |
| @@ -2547,9 +2453,6 @@ static struct clk_regmap *gcc_msm8994_clocks[] = { · 36 unmodified lines | ||
| 2547 | 2453 | [GPLL0] = &gpll0.clkr, |
| 2548 | 2454 | [GPLL4_EARLY] = &gpll4_early.clkr, |
| 2549 | 2455 | [GPLL4] = &gpll4.clkr, |
| 2550 | − [CONFIG_NOC_CLK_SRC] = &config_noc_clk_src.clkr, | |
| 2551 | − [PERIPH_NOC_CLK_SRC] = &periph_noc_clk_src.clkr, | |
| 2552 | − [SYSTEM_NOC_CLK_SRC] = &system_noc_clk_src.clkr, | |
| 2553 | 2456 | [UFS_AXI_CLK_SRC] = &ufs_axi_clk_src.clkr, |
| 2554 | 2457 | [USB30_MASTER_CLK_SRC] = &usb30_master_clk_src.clkr, |
| 2555 | 2458 | [BLSP1_QUP1_I2C_APPS_CLK_SRC] = &blsp1_qup1_i2c_apps_clk_src.clkr, |
| @@ -2696,6 +2599,15 @@ static struct clk_regmap *gcc_msm8994_clocks[] = { · 140 unmodified lines | ||
| 2696 | 2599 | [USB_SS_PHY_LDO] = &usb_ss_phy_ldo.clkr, |
| 2697 | 2600 | [GCC_BOOT_ROM_AHB_CLK] = &gcc_boot_rom_ahb_clk.clkr, |
| 2698 | 2601 | [GCC_PRNG_AHB_CLK] = &gcc_prng_ahb_clk.clkr, |
| 2602 | + | |
| 2603 | + /* | |
| 2604 | + * The following clocks should NOT be managed by this driver, but they once were | |
| 2605 | + * mistakengly added. Now they are only here to indicate that they are not defined | |
| 2606 | + * on purpose, even though the names will stay in the header file (for ABI sanity). | |
| 2607 | + */ | |
| 2608 | + [CONFIG_NOC_CLK_SRC] = NULL, | |
| 2609 | + [PERIPH_NOC_CLK_SRC] = NULL, | |
| 2610 | + [SYSTEM_NOC_CLK_SRC] = NULL, | |
| 2699 | 2611 | }; |
| 2700 | 2612 | |
| 2701 | 2613 | static struct gdsc *gcc_msm8994_gdscs[] = { |
| @@ -241,7 +241,7 @@ static void __init dmtimer_systimer_assign_alwon(void) · 240 unmodified lines | ||
| 241 | 241 | bool quirk_unreliable_oscillator = false; |
| 242 | 242 | |
| 243 | 243 | /* Quirk unreliable 32 KiHz oscillator with incomplete dts */ |
| 244 | − if (of_machine_is_compatible("ti,omap3-beagle") || | |
| 244 | + if (of_machine_is_compatible("ti,omap3-beagle-ab4") || | |
| 245 | 245 | of_machine_is_compatible("timll,omap3-devkit8000")) { |
| 246 | 246 | quirk_unreliable_oscillator = true; |
| 247 | 247 | counter_32k = -ENODEV; |
| @@ -1518,6 +1518,10 @@ static int cpufreq_online(unsigned int cpu) · 1517 unmodified lines | ||
| 1518 | 1518 | |
| 1519 | 1519 | kobject_uevent(&policy->kobj, KOBJ_ADD); |
| 1520 | 1520 | |
| 1521 | + /* Callback for handling stuff after policy is ready */ | |
| 1522 | + if (cpufreq_driver->ready) | |
| 1523 | + cpufreq_driver->ready(policy); | |
| 1524 | + | |
| 1521 | 1525 | if (cpufreq_thermal_control_enabled(cpufreq_driver)) |
| 1522 | 1526 | policy->cdev = of_cpufreq_cooling_register(policy); |
| 1523 | 1527 | |
| @@ -388,7 +388,7 @@ static int qcom_cpufreq_hw_lmh_init(struct cpufreq_policy *policy, int index) · 387 unmodified lines | ||
| 388 | 388 | |
| 389 | 389 | snprintf(data->irq_name, sizeof(data->irq_name), "dcvsh-irq-%u", policy->cpu); |
| 390 | 390 | ret = request_threaded_irq(data->throttle_irq, NULL, qcom_lmh_dcvs_handle_irq, |
| 391 | − IRQF_ONESHOT, data->irq_name, data); | |
| 391 | + IRQF_ONESHOT | IRQF_NO_AUTOEN, data->irq_name, data); | |
| 392 | 392 | if (ret) { |
| 393 | 393 | dev_err(&pdev->dev, "Error registering %s: %d\n", data->irq_name, ret); |
| 394 | 394 | return 0; |
| @@ -542,6 +542,14 @@ static int qcom_cpufreq_hw_cpu_exit(struct cpufreq_policy *policy) · 147 unmodified lines | ||
| 542 | 542 | return 0; |
| 543 | 543 | } |
| 544 | 544 | |
| 545 | +static void qcom_cpufreq_ready(struct cpufreq_policy *policy) | |
| 546 | +{ | |
| 547 | + struct qcom_cpufreq_data *data = policy->driver_data; | |
| 548 | + | |
| 549 | + if (data->throttle_irq >= 0) | |
| 550 | + enable_irq(data->throttle_irq); | |
| 551 | +} | |
| 552 | + | |
| 545 | 553 | static struct freq_attr *qcom_cpufreq_hw_attr[] = { |
| 546 | 554 | &cpufreq_freq_attr_scaling_available_freqs, |
| 547 | 555 | &cpufreq_freq_attr_scaling_boost_freqs, |
| @@ -561,6 +569,7 @@ static struct cpufreq_driver cpufreq_qcom_hw_driver = { · 13 unmodified lines | ||
| 561 | 569 | .fast_switch = qcom_cpufreq_hw_fast_switch, |
| 562 | 570 | .name = "qcom-cpufreq-hw", |
| 563 | 571 | .attr = qcom_cpufreq_hw_attr, |
| 572 | + .ready = qcom_cpufreq_ready, | |
| 564 | 573 | }; |
| 565 | 574 | |
| 566 | 575 | static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev) |
| @@ -1753,7 +1753,6 @@ void otx2_cpt_print_uc_dbg_info(struct otx2_cptpf_dev *cptpf) · 1752 unmodified lines | ||
| 1753 | 1753 | char engs_info[2 * OTX2_CPT_NAME_LENGTH]; |
| 1754 | 1754 | struct otx2_cpt_eng_grp_info *grp; |
| 1755 | 1755 | struct otx2_cpt_engs_rsvd *engs; |
| 1756 | − u32 mask[4]; | |
| 1757 | 1756 | int i, j; |
| 1758 | 1757 | |
| 1759 | 1758 | pr_debug("Engine groups global info"); |
| @@ -1785,6 +1784,8 @@ void otx2_cpt_print_uc_dbg_info(struct otx2_cptpf_dev *cptpf) · 25 unmodified lines | ||
| 1785 | 1784 | for (j = 0; j < OTX2_CPT_MAX_ETYPES_PER_GRP; j++) { |
| 1786 | 1785 | engs = &grp->engs[j]; |
| 1787 | 1786 | if (engs->type) { |
| 1787 | + u32 mask[5] = { }; | |
| 1788 | + | |
| 1788 | 1789 | get_engs_info(grp, engs_info, |
| 1789 | 1790 | 2 * OTX2_CPT_NAME_LENGTH, j); |
| 1790 | 1791 | pr_debug("Slot%d: %s", j, engs_info); |
| @@ -176,6 +176,20 @@ struct dma_fence_array *dma_fence_array_create(int num_fences, · 175 unmodified lines | ||
| 176 | 176 | |
| 177 | 177 | array->base.error = PENDING_ERROR; |
| 178 | 178 | |
| 179 | + /* | |
| 180 | + * dma_fence_array objects should never contain any other fence | |
| 181 | + * containers or otherwise we run into recursion and potential kernel | |
| 182 | + * stack overflow on operations on the dma_fence_array. | |
| 183 | + * | |
| 184 | + * The correct way of handling this is to flatten out the array by the | |
| 185 | + * caller instead. | |
| 186 | + * | |
| 187 | + * Enforce this here by checking that we don't create a dma_fence_array | |
| 188 | + * with any container inside. | |
| 189 | + */ | |
| 190 | + while (num_fences--) | |
| 191 | + WARN_ON(dma_fence_is_container(fences[num_fences])); | |
| 192 | + | |
| 179 | 193 | return array; |
| 180 | 194 | } |
| 181 | 195 | EXPORT_SYMBOL(dma_fence_array_create); |
| @@ -148,8 +148,7 @@ static bool dma_fence_chain_enable_signaling(struct dma_fence *fence) · 147 unmodified lines | ||
| 148 | 148 | |
| 149 | 149 | dma_fence_get(&head->base); |
| 150 | 150 | dma_fence_chain_for_each(fence, &head->base) { |
| 151 | − struct dma_fence_chain *chain = to_dma_fence_chain(fence); | |
| 152 | − struct dma_fence *f = chain ? chain->fence : fence; | |
| 151 | + struct dma_fence *f = dma_fence_chain_contained(fence); | |
| 153 | 152 | |
| 154 | 153 | dma_fence_get(f); |
| 155 | 154 | if (!dma_fence_add_callback(f, &head->cb, dma_fence_chain_cb)) { |
| @@ -165,8 +164,7 @@ static bool dma_fence_chain_enable_signaling(struct dma_fence *fence) · 9 unmodified lines | ||
| 165 | 164 | static bool dma_fence_chain_signaled(struct dma_fence *fence) |
| 166 | 165 | { |
| 167 | 166 | dma_fence_chain_for_each(fence, fence) { |
| 168 | − struct dma_fence_chain *chain = to_dma_fence_chain(fence); | |
| 169 | − struct dma_fence *f = chain ? chain->fence : fence; | |
| 167 | + struct dma_fence *f = dma_fence_chain_contained(fence); | |
| 170 | 168 | |
| 171 | 169 | if (!dma_fence_is_signaled(f)) { |
| 172 | 170 | dma_fence_put(fence); |
| @@ -254,5 +252,14 @@ void dma_fence_chain_init(struct dma_fence_chain *chain, · 81 unmodified lines | ||
| 254 | 252 | |
| 255 | 253 | dma_fence_init(&chain->base, &dma_fence_chain_ops, |
| 256 | 254 | &chain->lock, context, seqno); |
| 255 | + | |
| 256 | + /* | |
| 257 | + * Chaining dma_fence_chain container together is only allowed through | |
| 258 | + * the prev fence and not through the contained fence. | |
| 259 | + * | |
| 260 | + * The correct way of handling this is to flatten out the fence | |
| 261 | + * structure into a dma_fence_array by the caller instead. | |
| 262 | + */ | |
| 263 | + WARN_ON(dma_fence_is_chain(fence)); | |
| 257 | 264 | } |
| 258 | 265 | EXPORT_SYMBOL(dma_fence_chain_init); |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | ||
| 14 | 14 | #include <linux/xarray.h> |
| 15 | 15 | #include <linux/list.h> |
| 16 | 16 | #include <linux/slab.h> |
| 17 | +#include <linux/nospec.h> | |
| 17 | 18 | #include <linux/uaccess.h> |
| 18 | 19 | #include <linux/syscalls.h> |
| 19 | 20 | #include <linux/dma-heap.h> |
| @@ -135,6 +136,7 @@ static long dma_heap_ioctl(struct file *file, unsigned int ucmd, · 115 unmodified lines | ||
| 135 | 136 | if (nr >= ARRAY_SIZE(dma_heap_ioctl_cmds)) |
| 136 | 137 | return -EINVAL; |
| 137 | 138 | |
| 139 | + nr = array_index_nospec(nr, ARRAY_SIZE(dma_heap_ioctl_cmds)); | |
| 138 | 140 | /* Get the kernel ioctl cmd that matches */ |
| 139 | 141 | kcmd = dma_heap_ioctl_cmds[nr]; |
| 140 | 142 | |
| @@ -256,6 +256,11 @@ void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence) · 255 unmodified lines | ||
| 256 | 256 | |
| 257 | 257 | dma_resv_assert_held(obj); |
| 258 | 258 | |
| 259 | + /* Drivers should not add containers here, instead add each fence | |
| 260 | + * individually. | |
| 261 | + */ | |
| 262 | + WARN_ON(dma_fence_is_container(fence)); | |
| 263 | + | |
| 259 | 264 | fobj = dma_resv_shared_list(obj); |
| 260 | 265 | count = fobj->shared_count; |
| 261 | 266 | |
| @@ -323,12 +328,8 @@ void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence) · 61 unmodified lines | ||
| 323 | 328 | } |
| 324 | 329 | EXPORT_SYMBOL(dma_resv_add_excl_fence); |
| 325 | 330 | |
| 326 | −/** | |
| 327 | − * dma_resv_iter_restart_unlocked - restart the unlocked iterator | |
| 328 | − * @cursor: The dma_resv_iter object to restart | |
| 329 | − * | |
| 330 | − * Restart the unlocked iteration by initializing the cursor object. | |
| 331 | − */ | |
| 331 | +/* Restart the iterator by initializing all the necessary fields, but not the | |
| 332 | + * relation to the dma_resv object. */ | |
| 332 | 333 | static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor) |
| 333 | 334 | { |
| 334 | 335 | cursor->seq = read_seqcount_begin(&cursor->obj->seq); |
| @@ -344,14 +345,7 @@ static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor) · 9 unmodified lines | ||
| 344 | 345 | cursor->is_restarted = true; |
| 345 | 346 | } |
| 346 | 347 | |
| 347 | −/** | |
| 348 | − * dma_resv_iter_walk_unlocked - walk over fences in a dma_resv obj | |
| 349 | − * @cursor: cursor to record the current position | |
| 350 | − * | |
| 351 | − * Return all the fences in the dma_resv object which are not yet signaled. | |
| 352 | − * The returned fence has an extra local reference so will stay alive. | |
| 353 | − * If a concurrent modify is detected the whole iteration is started over again. | |
| 354 | − */ | |
| 348 | +/* Walk to the next not signaled fence and grab a reference to it */ | |
| 355 | 349 | static void dma_resv_iter_walk_unlocked(struct dma_resv_iter *cursor) |
| 356 | 350 | { |
| 357 | 351 | struct dma_resv *obj = cursor->obj; |
| @@ -387,6 +381,12 @@ static void dma_resv_iter_walk_unlocked(struct dma_resv_iter *cursor) · 29 unmodified lines | ||
| 387 | 381 | * dma_resv_iter_first_unlocked - first fence in an unlocked dma_resv obj. |
| 388 | 382 | * @cursor: the cursor with the current position |
| 389 | 383 | * |
| 384 | + * Subsequent fences are iterated with dma_resv_iter_next_unlocked(). | |
| 385 | + * | |
| 386 | + * Beware that the iterator can be restarted. Code which accumulates statistics | |
| 387 | + * or similar needs to check for this with dma_resv_iter_is_restarted(). For | |
| 388 | + * this reason prefer the locked dma_resv_iter_first() whenver possible. | |
| 389 | + * | |
| 390 | 390 | * Returns the first fence from an unlocked dma_resv obj. |
| 391 | 391 | */ |
| 392 | 392 | struct dma_fence *dma_resv_iter_first_unlocked(struct dma_resv_iter *cursor) |
| @@ -406,6 +406,10 @@ EXPORT_SYMBOL(dma_resv_iter_first_unlocked); · 13 unmodified lines | ||
| 406 | 406 | * dma_resv_iter_next_unlocked - next fence in an unlocked dma_resv obj. |
| 407 | 407 | * @cursor: the cursor with the current position |
| 408 | 408 | * |
| 409 | + * Beware that the iterator can be restarted. Code which accumulates statistics | |
| 410 | + * or similar needs to check for this with dma_resv_iter_is_restarted(). For | |
| 411 | + * this reason prefer the locked dma_resv_iter_next() whenver possible. | |
| 412 | + * | |
| 409 | 413 | * Returns the next fence from an unlocked dma_resv obj. |
| 410 | 414 | */ |
| 411 | 415 | struct dma_fence *dma_resv_iter_next_unlocked(struct dma_resv_iter *cursor) |
| @@ -431,6 +435,8 @@ EXPORT_SYMBOL(dma_resv_iter_next_unlocked); · 19 unmodified lines | ||
| 431 | 435 | * dma_resv_iter_first - first fence from a locked dma_resv object |
| 432 | 436 | * @cursor: cursor to record the current position |
| 433 | 437 | * |
| 438 | + * Subsequent fences are iterated with dma_resv_iter_next_unlocked(). | |
| 439 | + * | |
| 434 | 440 | * Return the first fence in the dma_resv object while holding the |
| 435 | 441 | * &dma_resv.lock. |
| 436 | 442 | */ |
| @@ -1681,8 +1681,10 @@ static void at_xdmac_tasklet(struct tasklet_struct *t) · 1680 unmodified lines | ||
| 1681 | 1681 | __func__, atchan->irq_status); |
| 1682 | 1682 | |
| 1683 | 1683 | if (!(atchan->irq_status & AT_XDMAC_CIS_LIS) && |
| 1684 | − !(atchan->irq_status & error_mask)) | |
| 1684 | + !(atchan->irq_status & error_mask)) { | |
| 1685 | + spin_unlock_irq(&atchan->lock); | |
| 1685 | 1686 | return; |
| 1687 | + } | |
| 1686 | 1688 | |
| 1687 | 1689 | if (atchan->irq_status & error_mask) |
| 1688 | 1690 | at_xdmac_handle_error(atchan); |
| @@ -207,7 +207,7 @@ int pt_core_init(struct pt_device *pt) · 206 unmodified lines | ||
| 207 | 207 | if (!cmd_q->qbase) { |
| 208 | 208 | dev_err(dev, "unable to allocate command queue\n"); |
| 209 | 209 | ret = -ENOMEM; |
| 210 | − goto e_dma_alloc; | |
| 210 | + goto e_destroy_pool; | |
| 211 | 211 | } |
| 212 | 212 | |
| 213 | 213 | cmd_q->qidx = 0; |
| @@ -229,8 +229,10 @@ int pt_core_init(struct pt_device *pt) · 15 unmodified lines | ||
| 229 | 229 | |
| 230 | 230 | /* Request an irq */ |
| 231 | 231 | ret = request_irq(pt->pt_irq, pt_core_irq_handler, 0, dev_name(pt->dev), pt); |
| 232 | − if (ret) | |
| 233 | − goto e_pool; | |
| 232 | + if (ret) { | |
| 233 | + dev_err(dev, "unable to allocate an IRQ\n"); | |
| 234 | + goto e_free_dma; | |
| 235 | + } | |
| 234 | 236 | |
| 235 | 237 | /* Update the device registers with queue information. */ |
| 236 | 238 | cmd_q->qcontrol &= ~CMD_Q_SIZE; |
| @@ -250,21 +252,20 @@ int pt_core_init(struct pt_device *pt) · 13 unmodified lines | ||
| 250 | 252 | /* Register the DMA engine support */ |
| 251 | 253 | ret = pt_dmaengine_register(pt); |
| 252 | 254 | if (ret) |
| 253 | − goto e_dmaengine; | |
| 255 | + goto e_free_irq; | |
| 254 | 256 | |
| 255 | 257 | /* Set up debugfs entries */ |
| 256 | 258 | ptdma_debugfs_setup(pt); |
| 257 | 259 | |
| 258 | 260 | return 0; |
| 259 | 261 | |
| 260 | −e_dmaengine: | |
| 262 | +e_free_irq: | |
| 261 | 263 | free_irq(pt->pt_irq, pt); |
| 262 | 264 | |
| 263 | −e_dma_alloc: | |
| 265 | +e_free_dma: | |
| 264 | 266 | dma_free_coherent(dev, cmd_q->qsize, cmd_q->qbase, cmd_q->qbase_dma); |
| 265 | 267 | |
| 266 | −e_pool: | |
| 267 | − dev_err(dev, "unable to allocate an IRQ\n"); | |
| 268 | +e_destroy_pool: | |
| 268 | 269 | dma_pool_destroy(pt->cmd_q.dma_pool); |
| 269 | 270 | |
| 270 | 271 | return ret; |
| @@ -1868,8 +1868,13 @@ static int rcar_dmac_probe(struct platform_device *pdev) · 1867 unmodified lines | ||
| 1868 | 1868 | |
| 1869 | 1869 | dmac->dev = &pdev->dev; |
| 1870 | 1870 | platform_set_drvdata(pdev, dmac); |
| 1871 | − dma_set_max_seg_size(dmac->dev, RCAR_DMATCR_MASK); | |
| 1872 | − dma_set_mask_and_coherent(dmac->dev, DMA_BIT_MASK(40)); | |
| 1871 | + ret = dma_set_max_seg_size(dmac->dev, RCAR_DMATCR_MASK); | |
| 1872 | + if (ret) | |
| 1873 | + return ret; | |
| 1874 | + | |
| 1875 | + ret = dma_set_mask_and_coherent(dmac->dev, DMA_BIT_MASK(40)); | |
| 1876 | + if (ret) | |
| 1877 | + return ret; | |
| 1873 | 1878 | |
| 1874 | 1879 | ret = rcar_dmac_parse_of(&pdev->dev, dmac); |
| 1875 | 1880 | if (ret < 0) |
| @@ -115,8 +115,10 @@ static dma_cookie_t shdma_tx_submit(struct dma_async_tx_descriptor *tx) · 114 unmodified lines | ||
| 115 | 115 | ret = pm_runtime_get(schan->dev); |
| 116 | 116 | |
| 117 | 117 | spin_unlock_irq(&schan->chan_lock); |
| 118 | − if (ret < 0) | |
| 118 | + if (ret < 0) { | |
| 119 | 119 | dev_err(schan->dev, "%s(): GET = %d\n", __func__, ret); |
| 120 | + pm_runtime_put(schan->dev); | |
| 121 | + } | |
| 120 | 122 | |
| 121 | 123 | pm_runtime_barrier(schan->dev); |
| 122 | 124 | |
| @@ -292,10 +292,12 @@ static int stm32_dmamux_probe(struct platform_device *pdev) · 291 unmodified lines | ||
| 292 | 292 | ret = of_dma_router_register(node, stm32_dmamux_route_allocate, |
| 293 | 293 | &stm32_dmamux->dmarouter); |
| 294 | 294 | if (ret) |
| 295 | − goto err_clk; | |
| 295 | + goto pm_disable; | |
| 296 | 296 | |
| 297 | 297 | return 0; |
| 298 | 298 | |
| 299 | +pm_disable: | |
| 300 | + pm_runtime_disable(&pdev->dev); | |
| 299 | 301 | err_clk: |
| 300 | 302 | clk_disable_unprepare(stm32_dmamux->clk); |
| 301 | 303 | |
| @@ -350,7 +350,7 @@ static int altr_sdram_probe(struct platform_device *pdev) · 349 unmodified lines | ||
| 350 | 350 | if (irq < 0) { |
| 351 | 351 | edac_printk(KERN_ERR, EDAC_MC, |
| 352 | 352 | "No irq %d in DT\n", irq); |
| 353 | − return -ENODEV; | |
| 353 | + return irq; | |
| 354 | 354 | } |
| 355 | 355 | |
| 356 | 356 | /* Arria10 has a 2nd IRQ */ |
| @@ -215,7 +215,7 @@ void *edac_align_ptr(void **p, unsigned int size, int n_elems) · 214 unmodified lines | ||
| 215 | 215 | else |
| 216 | 216 | return (char *)ptr; |
| 217 | 217 | |
| 218 | − r = (unsigned long)p % align; | |
| 218 | + r = (unsigned long)ptr % align; | |
| 219 | 219 | |
| 220 | 220 | if (r == 0) |
| 221 | 221 | return (char *)ptr; |
| @@ -1919,7 +1919,7 @@ static int xgene_edac_probe(struct platform_device *pdev) · 1918 unmodified lines | ||
| 1919 | 1919 | irq = platform_get_irq_optional(pdev, i); |
| 1920 | 1920 | if (irq < 0) { |
| 1921 | 1921 | dev_err(&pdev->dev, "No IRQ resource\n"); |
| 1922 | − rc = -EINVAL; | |
| 1922 | + rc = irq; | |
| 1923 | 1923 | goto out_err; |
| 1924 | 1924 | } |
| 1925 | 1925 | rc = devm_request_irq(&pdev->dev, irq, |
| @@ -278,7 +278,8 @@ static int gpio_fwd_get(struct gpio_chip *chip, unsigned int offset) · 277 unmodified lines | ||
| 278 | 278 | { |
| 279 | 279 | struct gpiochip_fwd *fwd = gpiochip_get_data(chip); |
| 280 | 280 | |
| 281 | − return gpiod_get_value(fwd->descs[offset]); | |
| 281 | + return chip->can_sleep ? gpiod_get_value_cansleep(fwd->descs[offset]) | |
| 282 | + : gpiod_get_value(fwd->descs[offset]); | |
| 282 | 283 | } |
| 283 | 284 | |
| 284 | 285 | static int gpio_fwd_get_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, |
| @@ -293,7 +294,10 @@ static int gpio_fwd_get_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, · 8 unmodified lines | ||
| 293 | 294 | for_each_set_bit(i, mask, fwd->chip.ngpio) |
| 294 | 295 | descs[j++] = fwd->descs[i]; |
| 295 | 296 | |
| 296 | − error = gpiod_get_array_value(j, descs, NULL, values); | |
| 297 | + if (fwd->chip.can_sleep) | |
| 298 | + error = gpiod_get_array_value_cansleep(j, descs, NULL, values); | |
| 299 | + else | |
| 300 | + error = gpiod_get_array_value(j, descs, NULL, values); | |
| 297 | 301 | if (error) |
| 298 | 302 | return error; |
| 299 | 303 | |
| @@ -328,7 +332,10 @@ static void gpio_fwd_set(struct gpio_chip *chip, unsigned int offset, int value) · 28 unmodified lines | ||
| 328 | 332 | { |
| 329 | 333 | struct gpiochip_fwd *fwd = gpiochip_get_data(chip); |
| 330 | 334 | |
| 331 | − gpiod_set_value(fwd->descs[offset], value); | |
| 335 | + if (chip->can_sleep) | |
| 336 | + gpiod_set_value_cansleep(fwd->descs[offset], value); | |
| 337 | + else | |
| 338 | + gpiod_set_value(fwd->descs[offset], value); | |
| 332 | 339 | } |
| 333 | 340 | |
| 334 | 341 | static void gpio_fwd_set_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, |
| @@ -343,7 +350,10 @@ static void gpio_fwd_set_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, · 8 unmodified lines | ||
| 343 | 350 | descs[j++] = fwd->descs[i]; |
| 344 | 351 | } |
| 345 | 352 | |
| 346 | − gpiod_set_array_value(j, descs, NULL, values); | |
| 353 | + if (fwd->chip.can_sleep) | |
| 354 | + gpiod_set_array_value_cansleep(j, descs, NULL, values); | |
| 355 | + else | |
| 356 | + gpiod_set_array_value(j, descs, NULL, values); | |
| 347 | 357 | } |
| 348 | 358 | |
| 349 | 359 | static void gpio_fwd_set_multiple_locked(struct gpio_chip *chip, |
| @@ -410,10 +410,8 @@ static int rockchip_irq_set_type(struct irq_data *d, unsigned int type) · 409 unmodified lines | ||
| 410 | 410 | level = rockchip_gpio_readl(bank, bank->gpio_regs->int_type); |
| 411 | 411 | polarity = rockchip_gpio_readl(bank, bank->gpio_regs->int_polarity); |
| 412 | 412 | |
| 413 | − switch (type) { | |
| 414 | − case IRQ_TYPE_EDGE_BOTH: | |
| 413 | + if (type == IRQ_TYPE_EDGE_BOTH) { | |
| 415 | 414 | if (bank->gpio_type == GPIO_TYPE_V2) { |
| 416 | − bank->toggle_edge_mode &= ~mask; | |
| 417 | 415 | rockchip_gpio_writel_bit(bank, d->hwirq, 1, |
| 418 | 416 | bank->gpio_regs->int_bothedge); |
| 419 | 417 | goto out; |
| @@ -431,30 +429,34 @@ static int rockchip_irq_set_type(struct irq_data *d, unsigned int type) · 11 unmodified lines | ||
| 431 | 429 | else |
| 432 | 430 | polarity |= mask; |
| 433 | 431 | } |
| 434 | − break; | |
| 435 | − case IRQ_TYPE_EDGE_RISING: | |
| 436 | − bank->toggle_edge_mode &= ~mask; | |
| 437 | − level |= mask; | |
| 438 | − polarity |= mask; | |
| 439 | − break; | |
| 440 | − case IRQ_TYPE_EDGE_FALLING: | |
| 441 | − bank->toggle_edge_mode &= ~mask; | |
| 442 | − level |= mask; | |
| 443 | − polarity &= ~mask; | |
| 444 | − break; | |
| 445 | − case IRQ_TYPE_LEVEL_HIGH: | |
| 446 | − bank->toggle_edge_mode &= ~mask; | |
| 447 | − level &= ~mask; | |
| 448 | − polarity |= mask; | |
| 449 | − break; | |
| 450 | − case IRQ_TYPE_LEVEL_LOW: | |
| 451 | − bank->toggle_edge_mode &= ~mask; | |
| 452 | − level &= ~mask; | |
| 453 | − polarity &= ~mask; | |
| 454 | − break; | |
| 455 | − default: | |
| 456 | − ret = -EINVAL; | |
| 457 | − goto out; | |
| 432 | + } else { | |
| 433 | + if (bank->gpio_type == GPIO_TYPE_V2) { | |
| 434 | + rockchip_gpio_writel_bit(bank, d->hwirq, 0, | |
| 435 | + bank->gpio_regs->int_bothedge); | |
| 436 | + } else { | |
| 437 | + bank->toggle_edge_mode &= ~mask; | |
| 438 | + } | |
| 439 | + switch (type) { | |
| 440 | + case IRQ_TYPE_EDGE_RISING: | |
| 441 | + level |= mask; | |
| 442 | + polarity |= mask; | |
| 443 | + break; | |
| 444 | + case IRQ_TYPE_EDGE_FALLING: | |
| 445 | + level |= mask; | |
| 446 | + polarity &= ~mask; | |
| 447 | + break; | |
| 448 | + case IRQ_TYPE_LEVEL_HIGH: | |
| 449 | + level &= ~mask; | |
| 450 | + polarity |= mask; | |
| 451 | + break; | |
| 452 | + case IRQ_TYPE_LEVEL_LOW: | |
| 453 | + level &= ~mask; | |
| 454 | + polarity &= ~mask; | |
| 455 | + break; | |
| 456 | + default: | |
| 457 | + ret = -EINVAL; | |
| 458 | + goto out; | |
| 459 | + } | |
| 458 | 460 | } |
| 459 | 461 | |
| 460 | 462 | rockchip_gpio_writel(bank, level, bank->gpio_regs->int_type); |
| @@ -223,7 +223,7 @@ static int sifive_gpio_probe(struct platform_device *pdev) · 222 unmodified lines | ||
| 223 | 223 | NULL, |
| 224 | 224 | chip->base + SIFIVE_GPIO_OUTPUT_EN, |
| 225 | 225 | chip->base + SIFIVE_GPIO_INPUT_EN, |
| 226 | − 0); | |
| 226 | + BGPIOF_READ_OUTPUT_REG_SET); | |
| 227 | 227 | if (ret) { |
| 228 | 228 | dev_err(dev, "unable to init generic GPIO\n"); |
| 229 | 229 | return ret; |
| @@ -570,6 +570,11 @@ static struct gpio_sim_bank *to_gpio_sim_bank(struct config_item *item) · 569 unmodified lines | ||
| 570 | 570 | return container_of(group, struct gpio_sim_bank, group); |
| 571 | 571 | } |
| 572 | 572 | |
| 573 | +static bool gpio_sim_bank_has_label(struct gpio_sim_bank *bank) | |
| 574 | +{ | |
| 575 | + return bank->label && *bank->label; | |
| 576 | +} | |
| 577 | + | |
| 573 | 578 | static struct gpio_sim_device * |
| 574 | 579 | gpio_sim_bank_get_device(struct gpio_sim_bank *bank) |
| 575 | 580 | { |
| @@ -770,9 +775,15 @@ static int gpio_sim_add_hogs(struct gpio_sim_device *dev) · 194 unmodified lines | ||
| 770 | 775 | * point the device doesn't exist yet and so dev_name() |
| 771 | 776 | * is not available. |
| 772 | 777 | */ |
| 773 | − hog->chip_label = kasprintf(GFP_KERNEL, | |
| 774 | − "gpio-sim.%u-%s", dev->id, | |
| 775 | − fwnode_get_name(bank->swnode)); | |
| 778 | + if (gpio_sim_bank_has_label(bank)) | |
| 779 | + hog->chip_label = kstrdup(bank->label, | |
| 780 | + GFP_KERNEL); | |
| 781 | + else | |
| 782 | + hog->chip_label = kasprintf(GFP_KERNEL, | |
| 783 | + "gpio-sim.%u-%s", | |
| 784 | + dev->id, | |
| 785 | + fwnode_get_name( | |
| 786 | + bank->swnode)); | |
| 776 | 787 | if (!hog->chip_label) { |
| 777 | 788 | gpio_sim_remove_hogs(dev); |
| 778 | 789 | return -ENOMEM; |
| @@ -816,7 +827,7 @@ gpio_sim_make_bank_swnode(struct gpio_sim_bank *bank, · 37 unmodified lines | ||
| 816 | 827 | |
| 817 | 828 | properties[prop_idx++] = PROPERTY_ENTRY_U32("ngpios", bank->num_lines); |
| 818 | 829 | |
| 819 | − if (bank->label && (strlen(bank->label) > 0)) | |
| 830 | + if (gpio_sim_bank_has_label(bank)) | |
| 820 | 831 | properties[prop_idx++] = PROPERTY_ENTRY_STRING("gpio-sim,label", |
| 821 | 832 | bank->label); |
| 822 | 833 | |
| @@ -343,9 +343,12 @@ static int tegra186_gpio_of_xlate(struct gpio_chip *chip, · 342 unmodified lines | ||
| 343 | 343 | return offset + pin; |
| 344 | 344 | } |
| 345 | 345 | |
| 346 | +#define to_tegra_gpio(x) container_of((x), struct tegra_gpio, gpio) | |
| 347 | + | |
| 346 | 348 | static void tegra186_irq_ack(struct irq_data *data) |
| 347 | 349 | { |
| 348 | − struct tegra_gpio *gpio = irq_data_get_irq_chip_data(data); | |
| 350 | + struct gpio_chip *gc = irq_data_get_irq_chip_data(data); | |
| 351 | + struct tegra_gpio *gpio = to_tegra_gpio(gc); | |
| 349 | 352 | void __iomem *base; |
| 350 | 353 | |
| 351 | 354 | base = tegra186_gpio_get_base(gpio, data->hwirq); |
| @@ -357,7 +360,8 @@ static void tegra186_irq_ack(struct irq_data *data) · 5 unmodified lines | ||
| 357 | 360 | |
| 358 | 361 | static void tegra186_irq_mask(struct irq_data *data) |
| 359 | 362 | { |
| 360 | − struct tegra_gpio *gpio = irq_data_get_irq_chip_data(data); | |
| 363 | + struct gpio_chip *gc = irq_data_get_irq_chip_data(data); | |
| 364 | + struct tegra_gpio *gpio = to_tegra_gpio(gc); | |
| 361 | 365 | void __iomem *base; |
| 362 | 366 | u32 value; |
| 363 | 367 | |
| @@ -372,7 +376,8 @@ static void tegra186_irq_mask(struct irq_data *data) · 8 unmodified lines | ||
| 372 | 376 | |
| 373 | 377 | static void tegra186_irq_unmask(struct irq_data *data) |
| 374 | 378 | { |
| 375 | − struct tegra_gpio *gpio = irq_data_get_irq_chip_data(data); | |
| 379 | + struct gpio_chip *gc = irq_data_get_irq_chip_data(data); | |
| 380 | + struct tegra_gpio *gpio = to_tegra_gpio(gc); | |
| 376 | 381 | void __iomem *base; |
| 377 | 382 | u32 value; |
| 378 | 383 | |
| @@ -387,7 +392,8 @@ static void tegra186_irq_unmask(struct irq_data *data) · 8 unmodified lines | ||
| 387 | 392 | |
| 388 | 393 | static int tegra186_irq_set_type(struct irq_data *data, unsigned int type) |
| 389 | 394 | { |
| 390 | − struct tegra_gpio *gpio = irq_data_get_irq_chip_data(data); | |
| 395 | + struct gpio_chip *gc = irq_data_get_irq_chip_data(data); | |
| 396 | + struct tegra_gpio *gpio = to_tegra_gpio(gc); | |
| 391 | 397 | void __iomem *base; |
| 392 | 398 | u32 value; |
| 393 | 399 | |
| @@ -330,7 +330,7 @@ static int linehandle_create(struct gpio_device *gdev, void __user *ip) · 329 unmodified lines | ||
| 330 | 330 | goto out_free_lh; |
| 331 | 331 | } |
| 332 | 332 | |
| 333 | − ret = gpiod_request(desc, lh->label); | |
| 333 | + ret = gpiod_request_user(desc, lh->label); | |
| 334 | 334 | if (ret) |
| 335 | 335 | goto out_free_lh; |
| 336 | 336 | lh->descs[i] = desc; |
| @@ -1378,7 +1378,7 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip) · 1041 unmodified lines | ||
| 1378 | 1378 | goto out_free_linereq; |
| 1379 | 1379 | } |
| 1380 | 1380 | |
| 1381 | − ret = gpiod_request(desc, lr->label); | |
| 1381 | + ret = gpiod_request_user(desc, lr->label); | |
| 1382 | 1382 | if (ret) |
| 1383 | 1383 | goto out_free_linereq; |
| 1384 | 1384 | |
| @@ -1764,7 +1764,7 @@ static int lineevent_create(struct gpio_device *gdev, void __user *ip) · 379 unmodified lines | ||
| 1764 | 1764 | } |
| 1765 | 1765 | } |
| 1766 | 1766 | |
| 1767 | − ret = gpiod_request(desc, le->label); | |
| 1767 | + ret = gpiod_request_user(desc, le->label); | |
| 1768 | 1768 | if (ret) |
| 1769 | 1769 | goto out_free_le; |
| 1770 | 1770 | le->desc = desc; |
| @@ -475,12 +475,9 @@ static ssize_t export_store(struct class *class, · 474 unmodified lines | ||
| 475 | 475 | * they may be undone on its behalf too. |
| 476 | 476 | */ |
| 477 | 477 | |
| 478 | − status = gpiod_request(desc, "sysfs"); | |
| 479 | − if (status) { | |
| 480 | − if (status == -EPROBE_DEFER) | |
| 481 | − status = -ENODEV; | |
| 478 | + status = gpiod_request_user(desc, "sysfs"); | |
| 479 | + if (status) | |
| 482 | 480 | goto done; |
| 483 | − } | |
| 484 | 481 | |
| 485 | 482 | status = gpiod_set_transitory(desc, false); |
| 486 | 483 | if (!status) { |
| @@ -3147,6 +3147,16 @@ int gpiod_to_irq(const struct gpio_desc *desc) · 3146 unmodified lines | ||
| 3147 | 3147 | |
| 3148 | 3148 | return retirq; |
| 3149 | 3149 | } |
| 3150 | +#ifdef CONFIG_GPIOLIB_IRQCHIP | |
| 3151 | + if (gc->irq.chip) { | |
| 3152 | + /* | |
| 3153 | + * Avoid race condition with other code, which tries to lookup | |
| 3154 | + * an IRQ before the irqchip has been properly registered, | |
| 3155 | + * i.e. while gpiochip is still being brought up. | |
| 3156 | + */ | |
| 3157 | + return -EPROBE_DEFER; | |
| 3158 | + } | |
| 3159 | +#endif | |
| 3150 | 3160 | return -ENXIO; |
| 3151 | 3161 | } |
| 3152 | 3162 | EXPORT_SYMBOL_GPL(gpiod_to_irq); |
| @@ -135,6 +135,18 @@ struct gpio_desc { · 134 unmodified lines | ||
| 135 | 135 | |
| 136 | 136 | int gpiod_request(struct gpio_desc *desc, const char *label); |
| 137 | 137 | void gpiod_free(struct gpio_desc *desc); |
| 138 | + | |
| 139 | +static inline int gpiod_request_user(struct gpio_desc *desc, const char *label) | |
| 140 | +{ | |
| 141 | + int ret; | |
| 142 | + | |
| 143 | + ret = gpiod_request(desc, label); | |
| 144 | + if (ret == -EPROBE_DEFER) | |
| 145 | + ret = -ENODEV; | |
| 146 | + | |
| 147 | + return ret; | |
| 148 | +} | |
| 149 | + | |
| 138 | 150 | int gpiod_configure_flags(struct gpio_desc *desc, const char *con_id, |
| 139 | 151 | unsigned long lflags, enum gpiod_flags dflags); |
| 140 | 152 | int gpio_set_debounce_timeout(struct gpio_desc *desc, unsigned int debounce); |
| @@ -71,6 +71,7 @@ config DRM_DEBUG_SELFTEST · 70 unmodified lines | ||
| 71 | 71 | select DRM_DP_HELPER |
| 72 | 72 | select DRM_LIB_RANDOM |
| 73 | 73 | select DRM_KMS_HELPER |
| 74 | + select DRM_BUDDY | |
| 74 | 75 | select DRM_EXPORT_FOR_TESTS if m |
| 75 | 76 | default n |
| 76 | 77 | help |
| @@ -403,6 +404,8 @@ source "drivers/gpu/drm/xlnx/Kconfig" · 326 unmodified lines | ||
| 403 | 404 | |
| 404 | 405 | source "drivers/gpu/drm/gud/Kconfig" |
| 405 | 406 | |
| 407 | +source "drivers/gpu/drm/solomon/Kconfig" | |
| 408 | + | |
| 406 | 409 | source "drivers/gpu/drm/sprd/Kconfig" |
| 407 | 410 | |
| 408 | 411 | config DRM_HYPERV |
| @@ -132,4 +132,5 @@ obj-$(CONFIG_DRM_TIDSS) += tidss/ · 131 unmodified lines | ||
| 132 | 132 | obj-y += xlnx/ |
| 133 | 133 | obj-y += gud/ |
| 134 | 134 | obj-$(CONFIG_DRM_HYPERV) += hyperv/ |
| 135 | +obj-y += solomon/ | |
| 135 | 136 | obj-$(CONFIG_DRM_SPRD) += sprd/ |
| @@ -46,7 +46,7 @@ amdgpu-y += amdgpu_device.o amdgpu_kms.o \ · 45 unmodified lines | ||
| 46 | 46 | atom.o amdgpu_fence.o amdgpu_ttm.o amdgpu_object.o amdgpu_gart.o \ |
| 47 | 47 | amdgpu_encoders.o amdgpu_display.o amdgpu_i2c.o \ |
| 48 | 48 | amdgpu_gem.o amdgpu_ring.o \ |
| 49 | − amdgpu_cs.o amdgpu_bios.o amdgpu_benchmark.o amdgpu_test.o \ | |
| 49 | + amdgpu_cs.o amdgpu_bios.o amdgpu_benchmark.o \ | |
| 50 | 50 | atombios_dp.o amdgpu_afmt.o amdgpu_trace_points.o \ |
| 51 | 51 | atombios_encoders.o amdgpu_sa.o atombios_i2c.o \ |
| 52 | 52 | amdgpu_dma_buf.o amdgpu_vm.o amdgpu_ib.o amdgpu_pll.o \ |
| @@ -260,7 +260,7 @@ static int aldebaran_mode2_restore_ip(struct amdgpu_device *adev) · 259 unmodified lines | ||
| 260 | 260 | adev->gfx.rlc.funcs->resume(adev); |
| 261 | 261 | |
| 262 | 262 | /* Wait for FW reset event complete */ |
| 263 | − r = smu_wait_for_event(adev, SMU_EVENT_RESET_COMPLETE, 0); | |
| 263 | + r = amdgpu_dpm_wait_for_event(adev, SMU_EVENT_RESET_COMPLETE, 0); | |
| 264 | 264 | if (r) { |
| 265 | 265 | dev_err(adev->dev, |
| 266 | 266 | "Failed to get response from firmware after reset\n"); |
| @@ -99,7 +99,6 @@ · 98 unmodified lines | ||
| 99 | 99 | #include "amdgpu_gem.h" |
| 100 | 100 | #include "amdgpu_doorbell.h" |
| 101 | 101 | #include "amdgpu_amdkfd.h" |
| 102 | −#include "amdgpu_smu.h" | |
| 103 | 102 | #include "amdgpu_discovery.h" |
| 104 | 103 | #include "amdgpu_mes.h" |
| 105 | 104 | #include "amdgpu_umc.h" |
| @@ -109,6 +108,7 @@ · 3 unmodified lines | ||
| 109 | 108 | #include "amdgpu_smuio.h" |
| 110 | 109 | #include "amdgpu_fdinfo.h" |
| 111 | 110 | #include "amdgpu_mca.h" |
| 111 | +#include "amdgpu_ras.h" | |
| 112 | 112 | |
| 113 | 113 | #define MAX_GPU_INSTANCE 16 |
| 114 | 114 | |
| @@ -155,8 +155,6 @@ extern int amdgpu_vis_vram_limit; · 40 unmodified lines | ||
| 155 | 155 | extern int amdgpu_gart_size; |
| 156 | 156 | extern int amdgpu_gtt_size; |
| 157 | 157 | extern int amdgpu_moverate; |
| 158 | −extern int amdgpu_benchmarking; | |
| 159 | −extern int amdgpu_testing; | |
| 160 | 158 | extern int amdgpu_audio; |
| 161 | 159 | extern int amdgpu_disp_priority; |
| 162 | 160 | extern int amdgpu_hw_i2c; |
| @@ -197,7 +195,6 @@ extern int amdgpu_emu_mode; · 34 unmodified lines | ||
| 197 | 195 | extern uint amdgpu_smu_memory_pool_size; |
| 198 | 196 | extern int amdgpu_smu_pptable_id; |
| 199 | 197 | extern uint amdgpu_dc_feature_mask; |
| 200 | −extern uint amdgpu_freesync_vid_mode; | |
| 201 | 198 | extern uint amdgpu_dc_debug_mask; |
| 202 | 199 | extern uint amdgpu_dm_abm_level; |
| 203 | 200 | extern int amdgpu_backlight; |
| @@ -214,6 +211,7 @@ extern int amdgpu_mes; · 10 unmodified lines | ||
| 214 | 211 | extern int amdgpu_noretry; |
| 215 | 212 | extern int amdgpu_force_asic_type; |
| 216 | 213 | extern int amdgpu_smartshift_bias; |
| 214 | +extern int amdgpu_use_xgmi_p2p; | |
| 217 | 215 | #ifdef CONFIG_HSA_AMD |
| 218 | 216 | extern int sched_policy; |
| 219 | 217 | extern bool debug_evictions; |
| @@ -373,7 +371,8 @@ int amdgpu_device_ip_block_add(struct amdgpu_device *adev, · 153 unmodified lines | ||
| 373 | 371 | */ |
| 374 | 372 | bool amdgpu_get_bios(struct amdgpu_device *adev); |
| 375 | 373 | bool amdgpu_read_bios(struct amdgpu_device *adev); |
| 376 | − | |
| 374 | +bool amdgpu_soc15_read_bios_from_rom(struct amdgpu_device *adev, | |
| 375 | + u8 *bios, u32 length_bytes); | |
| 377 | 376 | /* |
| 378 | 377 | * Clocks |
| 379 | 378 | */ |
| @@ -586,13 +585,7 @@ void amdgpu_device_wb_free(struct amdgpu_device *adev, u32 wb); · 206 unmodified lines | ||
| 586 | 585 | /* |
| 587 | 586 | * Benchmarking |
| 588 | 587 | */ |
| 589 | −void amdgpu_benchmark(struct amdgpu_device *adev, int test_number); | |
| 590 | − | |
| 591 | − | |
| 592 | −/* | |
| 593 | − * Testing | |
| 594 | − */ | |
| 595 | −void amdgpu_test_moves(struct amdgpu_device *adev); | |
| 588 | +int amdgpu_benchmark(struct amdgpu_device *adev, int test_number); | |
| 596 | 589 | |
| 597 | 590 | /* |
| 598 | 591 | * ASIC specific register table accessible by UMD |
| @@ -771,6 +764,8 @@ struct amd_powerplay { · 172 unmodified lines | ||
| 771 | 764 | const struct amd_pm_funcs *pp_funcs; |
| 772 | 765 | }; |
| 773 | 766 | |
| 767 | +struct ip_discovery_top; | |
| 768 | + | |
| 774 | 769 | /* polaris10 kickers */ |
| 775 | 770 | #define ASICID_IS_P20(did, rid) (((did == 0x67DF) && \ |
| 776 | 771 | ((rid == 0xE3) || \ |
| @@ -813,6 +808,8 @@ struct amd_powerplay { · 36 unmodified lines | ||
| 813 | 808 | #define AMDGPU_RESET_MAGIC_NUM 64 |
| 814 | 809 | #define AMDGPU_MAX_DF_PERFMONS 4 |
| 815 | 810 | #define AMDGPU_PRODUCT_NAME_LEN 64 |
| 811 | +struct amdgpu_reset_domain; | |
| 812 | + | |
| 816 | 813 | struct amdgpu_device { |
| 817 | 814 | struct device *dev; |
| 818 | 815 | struct pci_dev *pdev; |
| @@ -950,12 +947,6 @@ struct amdgpu_device { · 131 unmodified lines | ||
| 950 | 947 | |
| 951 | 948 | /* powerplay */ |
| 952 | 949 | struct amd_powerplay powerplay; |
| 953 | − bool pp_force_state_enabled; | |
| 954 | − | |
| 955 | − /* smu */ | |
| 956 | − struct smu_context smu; | |
| 957 | − | |
| 958 | − /* dpm */ | |
| 959 | 950 | struct amdgpu_pm pm; |
| 960 | 951 | u32 cg_flags; |
| 961 | 952 | u32 pg_flags; |
| @@ -1054,9 +1045,7 @@ struct amdgpu_device { · 92 unmodified lines | ||
| 1054 | 1045 | bool in_s4; |
| 1055 | 1046 | bool in_s0ix; |
| 1056 | 1047 | |
| 1057 | − atomic_t in_gpu_reset; | |
| 1058 | 1048 | enum pp_mp1_state mp1_state; |
| 1059 | − struct rw_semaphore reset_sem; | |
| 1060 | 1049 | struct amdgpu_doorbell_index doorbell_index; |
| 1061 | 1050 | |
| 1062 | 1051 | struct mutex notifier_lock; |
| @@ -1100,6 +1089,18 @@ struct amdgpu_device { · 37 unmodified lines | ||
| 1100 | 1089 | uint32_t ip_versions[MAX_HWIP][HWIP_MAX_INSTANCE]; |
| 1101 | 1090 | |
| 1102 | 1091 | bool ram_is_direct_mapped; |
| 1092 | + | |
| 1093 | + struct list_head ras_list; | |
| 1094 | + | |
| 1095 | + struct ip_discovery_top *ip_top; | |
| 1096 | + | |
| 1097 | + struct amdgpu_reset_domain *reset_domain; | |
| 1098 | + | |
| 1099 | + struct mutex benchmark_mutex; | |
| 1100 | + | |
| 1101 | + /* reset dump register */ | |
| 1102 | + uint32_t *reset_dump_reg_list; | |
| 1103 | + int num_regs; | |
| 1103 | 1104 | }; |
| 1104 | 1105 | |
| 1105 | 1106 | static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) |
| @@ -1293,9 +1294,12 @@ bool amdgpu_device_has_job_running(struct amdgpu_device *adev); · 187 unmodified lines | ||
| 1293 | 1294 | bool amdgpu_device_should_recover_gpu(struct amdgpu_device *adev); |
| 1294 | 1295 | int amdgpu_device_gpu_recover(struct amdgpu_device *adev, |
| 1295 | 1296 | struct amdgpu_job* job); |
| 1297 | +int amdgpu_device_gpu_recover_imp(struct amdgpu_device *adev, | |
| 1298 | + struct amdgpu_job *job); | |
| 1296 | 1299 | void amdgpu_device_pci_config_reset(struct amdgpu_device *adev); |
| 1297 | 1300 | int amdgpu_device_pci_reset(struct amdgpu_device *adev); |
| 1298 | 1301 | bool amdgpu_device_need_post(struct amdgpu_device *adev); |
| 1302 | +bool amdgpu_device_should_use_aspm(struct amdgpu_device *adev); | |
| 1299 | 1303 | |
| 1300 | 1304 | void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes, |
| 1301 | 1305 | u64 num_vis_bytes); |
| @@ -1321,6 +1325,10 @@ void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, · 19 unmodified lines | ||
| 1321 | 1325 | struct amdgpu_ring *ring); |
| 1322 | 1326 | |
| 1323 | 1327 | void amdgpu_device_halt(struct amdgpu_device *adev); |
| 1328 | +u32 amdgpu_device_pcie_port_rreg(struct amdgpu_device *adev, | |
| 1329 | + u32 reg); | |
| 1330 | +void amdgpu_device_pcie_port_wreg(struct amdgpu_device *adev, | |
| 1331 | + u32 reg, u32 v); | |
| 1324 | 1332 | |
| 1325 | 1333 | /* atpx handler */ |
| 1326 | 1334 | #if defined(CONFIG_VGA_SWITCHEROO) |
| @@ -1408,12 +1416,10 @@ int amdgpu_acpi_smart_shift_update(struct drm_device *dev, enum amdgpu_ss ss_sta · 81 unmodified lines | ||
| 1408 | 1416 | int amdgpu_acpi_pcie_notify_device_ready(struct amdgpu_device *adev); |
| 1409 | 1417 | |
| 1410 | 1418 | void amdgpu_acpi_get_backlight_caps(struct amdgpu_dm_backlight_caps *caps); |
| 1411 | −bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev); | |
| 1412 | 1419 | void amdgpu_acpi_detect(void); |
| 1413 | 1420 | #else |
| 1414 | 1421 | static inline int amdgpu_acpi_init(struct amdgpu_device *adev) { return 0; } |
| 1415 | 1422 | static inline void amdgpu_acpi_fini(struct amdgpu_device *adev) { } |
| 1416 | −static inline bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev) { return false; } | |
| 1417 | 1423 | static inline void amdgpu_acpi_detect(void) { } |
| 1418 | 1424 | static inline bool amdgpu_acpi_is_power_shift_control_supported(void) { return false; } |
| 1419 | 1425 | static inline int amdgpu_acpi_power_shift_control(struct amdgpu_device *adev, |
| @@ -1422,6 +1428,14 @@ static inline int amdgpu_acpi_smart_shift_update(struct drm_device *dev, · 2 unmodified lines | ||
| 1422 | 1428 | enum amdgpu_ss ss_state) { return 0; } |
| 1423 | 1429 | #endif |
| 1424 | 1430 | |
| 1431 | +#if defined(CONFIG_ACPI) && defined(CONFIG_SUSPEND) | |
| 1432 | +bool amdgpu_acpi_is_s3_active(struct amdgpu_device *adev); | |
| 1433 | +bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev); | |
| 1434 | +#else | |
| 1435 | +static inline bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev) { return false; } | |
| 1436 | +static inline bool amdgpu_acpi_is_s3_active(struct amdgpu_device *adev) { return false; } | |
| 1437 | +#endif | |
| 1438 | + | |
| 1425 | 1439 | int amdgpu_cs_find_mapping(struct amdgpu_cs_parser *parser, |
| 1426 | 1440 | uint64_t addr, struct amdgpu_bo **bo, |
| 1427 | 1441 | struct amdgpu_bo_va_mapping **mapping); |
| @@ -1452,6 +1466,15 @@ int amdgpu_device_set_cg_state(struct amdgpu_device *adev, · 24 unmodified lines | ||
| 1452 | 1466 | int amdgpu_device_set_pg_state(struct amdgpu_device *adev, |
| 1453 | 1467 | enum amd_powergating_state state); |
| 1454 | 1468 | |
| 1469 | +static inline bool amdgpu_device_has_timeouts_enabled(struct amdgpu_device *adev) | |
| 1470 | +{ | |
| 1471 | + return amdgpu_gpu_recovery != 0 && | |
| 1472 | + adev->gfx_timeout != MAX_SCHEDULE_TIMEOUT && | |
| 1473 | + adev->compute_timeout != MAX_SCHEDULE_TIMEOUT && | |
| 1474 | + adev->sdma_timeout != MAX_SCHEDULE_TIMEOUT && | |
| 1475 | + adev->video_timeout != MAX_SCHEDULE_TIMEOUT; | |
| 1476 | +} | |
| 1477 | + | |
| 1455 | 1478 | #include "amdgpu_object.h" |
| 1456 | 1479 | |
| 1457 | 1480 | static inline bool amdgpu_is_tmz(struct amdgpu_device *adev) |
| @@ -1459,8 +1482,6 @@ static inline bool amdgpu_is_tmz(struct amdgpu_device *adev) · 1 unmodified line | ||
| 1459 | 1482 | return adev->gmc.tmz_enabled; |
| 1460 | 1483 | } |
| 1461 | 1484 | |
| 1462 | −static inline int amdgpu_in_reset(struct amdgpu_device *adev) | |
| 1463 | −{ | |
| 1464 | − return atomic_read(&adev->in_gpu_reset); | |
| 1465 | −} | |
| 1485 | +int amdgpu_in_reset(struct amdgpu_device *adev); | |
| 1486 | + | |
| 1466 | 1487 | #endif |
| @@ -1031,6 +1031,20 @@ void amdgpu_acpi_detect(void) · 1030 unmodified lines | ||
| 1031 | 1031 | } |
| 1032 | 1032 | } |
| 1033 | 1033 | |
| 1034 | +#if IS_ENABLED(CONFIG_SUSPEND) | |
| 1035 | +/** | |
| 1036 | + * amdgpu_acpi_is_s3_active | |
| 1037 | + * | |
| 1038 | + * @adev: amdgpu_device_pointer | |
| 1039 | + * | |
| 1040 | + * returns true if supported, false if not. | |
| 1041 | + */ | |
| 1042 | +bool amdgpu_acpi_is_s3_active(struct amdgpu_device *adev) | |
| 1043 | +{ | |
| 1044 | + return !(adev->flags & AMD_IS_APU) || | |
| 1045 | + (pm_suspend_target_state == PM_SUSPEND_MEM); | |
| 1046 | +} | |
| 1047 | + | |
| 1034 | 1048 | /** |
| 1035 | 1049 | * amdgpu_acpi_is_s0ix_active |
| 1036 | 1050 | * |
| @@ -1040,11 +1054,24 @@ void amdgpu_acpi_detect(void) · 3 unmodified lines | ||
| 1040 | 1054 | */ |
| 1041 | 1055 | bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev) |
| 1042 | 1056 | { |
| 1043 | −#if IS_ENABLED(CONFIG_AMD_PMC) && IS_ENABLED(CONFIG_SUSPEND) | |
| 1044 | − if (acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0) { | |
| 1045 | − if (adev->flags & AMD_IS_APU) | |
| 1046 | − return pm_suspend_target_state == PM_SUSPEND_TO_IDLE; | |
| 1057 | + if (!(adev->flags & AMD_IS_APU) || | |
| 1058 | + (pm_suspend_target_state != PM_SUSPEND_TO_IDLE)) | |
| 1059 | + return false; | |
| 1060 | + | |
| 1061 | + if (!(acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0)) { | |
| 1062 | + dev_warn_once(adev->dev, | |
| 1063 | + "Power consumption will be higher as BIOS has not been configured for suspend-to-idle.\n" | |
| 1064 | + "To use suspend-to-idle change the sleep mode in BIOS setup.\n"); | |
| 1065 | + return false; | |
| 1047 | 1066 | } |
| 1048 | −#endif | |
| 1067 | + | |
| 1068 | +#if !IS_ENABLED(CONFIG_AMD_PMC) | |
| 1069 | + dev_warn_once(adev->dev, | |
| 1070 | + "Power consumption will be higher as the kernel has not been compiled with CONFIG_AMD_PMC.\n"); | |
| 1049 | 1071 | return false; |
| 1072 | +#else | |
| 1073 | + return true; | |
| 1074 | +#endif /* CONFIG_AMD_PMC */ | |
| 1050 | 1075 | } |
| 1076 | + | |
| 1077 | +#endif /* CONFIG_SUSPEND */ | |
| @@ -131,6 +131,7 @@ struct amdkfd_process_info { · 130 unmodified lines | ||
| 131 | 131 | atomic_t evicted_bos; |
| 132 | 132 | struct delayed_work restore_userptr_work; |
| 133 | 133 | struct pid *pid; |
| 134 | + bool block_mmu_notifications; | |
| 134 | 135 | }; |
| 135 | 136 | |
| 136 | 137 | int amdgpu_amdkfd_init(void); |
| @@ -268,7 +269,7 @@ uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *drm_priv); · 131 unmodified lines | ||
| 268 | 269 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( |
| 269 | 270 | struct amdgpu_device *adev, uint64_t va, uint64_t size, |
| 270 | 271 | void *drm_priv, struct kgd_mem **mem, |
| 271 | − uint64_t *offset, uint32_t flags); | |
| 272 | + uint64_t *offset, uint32_t flags, bool criu_resume); | |
| 272 | 273 | int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( |
| 273 | 274 | struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv, |
| 274 | 275 | uint64_t *size); |
| @@ -297,6 +298,10 @@ int amdgpu_amdkfd_get_tile_config(struct amdgpu_device *adev, · 22 unmodified lines | ||
| 297 | 298 | struct tile_config *config); |
| 298 | 299 | void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev, |
| 299 | 300 | bool reset); |
| 301 | +bool amdgpu_amdkfd_bo_mapped_to_dev(struct amdgpu_device *adev, struct kgd_mem *mem); | |
| 302 | +void amdgpu_amdkfd_block_mmu_notifications(void *p); | |
| 303 | +int amdgpu_amdkfd_criu_resume(void *p); | |
| 304 | + | |
| 300 | 305 | #if IS_ENABLED(CONFIG_HSA_AMD) |
| 301 | 306 | void amdgpu_amdkfd_gpuvm_init_mem_limits(void); |
| 302 | 307 | void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev, |
| @@ -37,10 +37,7 @@ const struct kfd2kgd_calls aldebaran_kfd2kgd = { · 36 unmodified lines | ||
| 37 | 37 | .hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied, |
| 38 | 38 | .hqd_destroy = kgd_gfx_v9_hqd_destroy, |
| 39 | 39 | .hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy, |
| 40 | − .address_watch_disable = kgd_gfx_v9_address_watch_disable, | |
| 41 | − .address_watch_execute = kgd_gfx_v9_address_watch_execute, | |
| 42 | 40 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, |
| 43 | − .address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset, | |
| 44 | 41 | .get_atc_vmid_pasid_mapping_info = |
| 45 | 42 | kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, |
| 46 | 43 | .set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base, |
| @@ -289,10 +289,7 @@ const struct kfd2kgd_calls arcturus_kfd2kgd = { · 288 unmodified lines | ||
| 289 | 289 | .hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied, |
| 290 | 290 | .hqd_destroy = kgd_gfx_v9_hqd_destroy, |
| 291 | 291 | .hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy, |
| 292 | − .address_watch_disable = kgd_gfx_v9_address_watch_disable, | |
| 293 | − .address_watch_execute = kgd_gfx_v9_address_watch_execute, | |
| 294 | 292 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, |
| 295 | − .address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset, | |
| 296 | 293 | .get_atc_vmid_pasid_mapping_info = |
| 297 | 294 | kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, |
| 298 | 295 | .set_vm_context_page_table_base = |
| @@ -671,20 +671,6 @@ static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, · 670 unmodified lines | ||
| 671 | 671 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 672 | 672 | } |
| 673 | 673 | |
| 674 | −static int kgd_address_watch_disable(struct amdgpu_device *adev) | |
| 675 | −{ | |
| 676 | − return 0; | |
| 677 | −} | |
| 678 | − | |
| 679 | −static int kgd_address_watch_execute(struct amdgpu_device *adev, | |
| 680 | − unsigned int watch_point_id, | |
| 681 | − uint32_t cntl_val, | |
| 682 | − uint32_t addr_hi, | |
| 683 | − uint32_t addr_lo) | |
| 684 | −{ | |
| 685 | − return 0; | |
| 686 | −} | |
| 687 | − | |
| 688 | 674 | static int kgd_wave_control_execute(struct amdgpu_device *adev, |
| 689 | 675 | uint32_t gfx_index_val, |
| 690 | 676 | uint32_t sq_cmd) |
| @@ -709,13 +695,6 @@ static int kgd_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines | ||
| 709 | 695 | return 0; |
| 710 | 696 | } |
| 711 | 697 | |
| 712 | −static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev, | |
| 713 | − unsigned int watch_point_id, | |
| 714 | − unsigned int reg_offset) | |
| 715 | −{ | |
| 716 | − return 0; | |
| 717 | −} | |
| 718 | − | |
| 719 | 698 | static void set_vm_context_page_table_base(struct amdgpu_device *adev, |
| 720 | 699 | uint32_t vmid, uint64_t page_table_base) |
| 721 | 700 | { |
| @@ -767,10 +746,7 @@ const struct kfd2kgd_calls gfx_v10_kfd2kgd = { · 45 unmodified lines | ||
| 767 | 746 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, |
| 768 | 747 | .hqd_destroy = kgd_hqd_destroy, |
| 769 | 748 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, |
| 770 | − .address_watch_disable = kgd_address_watch_disable, | |
| 771 | − .address_watch_execute = kgd_address_watch_execute, | |
| 772 | 749 | .wave_control_execute = kgd_wave_control_execute, |
| 773 | − .address_watch_get_offset = kgd_address_watch_get_offset, | |
| 774 | 750 | .get_atc_vmid_pasid_mapping_info = |
| 775 | 751 | get_atc_vmid_pasid_mapping_info, |
| 776 | 752 | .set_vm_context_page_table_base = set_vm_context_page_table_base, |
| @@ -582,21 +582,6 @@ static int hqd_sdma_destroy_v10_3(struct amdgpu_device *adev, void *mqd, · 581 unmodified lines | ||
| 582 | 582 | return 0; |
| 583 | 583 | } |
| 584 | 584 | |
| 585 | − | |
| 586 | −static int address_watch_disable_v10_3(struct amdgpu_device *adev) | |
| 587 | −{ | |
| 588 | − return 0; | |
| 589 | −} | |
| 590 | − | |
| 591 | −static int address_watch_execute_v10_3(struct amdgpu_device *adev, | |
| 592 | − unsigned int watch_point_id, | |
| 593 | − uint32_t cntl_val, | |
| 594 | − uint32_t addr_hi, | |
| 595 | − uint32_t addr_lo) | |
| 596 | −{ | |
| 597 | − return 0; | |
| 598 | −} | |
| 599 | − | |
| 600 | 585 | static int wave_control_execute_v10_3(struct amdgpu_device *adev, |
| 601 | 586 | uint32_t gfx_index_val, |
| 602 | 587 | uint32_t sq_cmd) |
| @@ -621,13 +606,6 @@ static int wave_control_execute_v10_3(struct amdgpu_device *adev, · 18 unmodified lines | ||
| 621 | 606 | return 0; |
| 622 | 607 | } |
| 623 | 608 | |
| 624 | −static uint32_t address_watch_get_offset_v10_3(struct amdgpu_device *adev, | |
| 625 | − unsigned int watch_point_id, | |
| 626 | − unsigned int reg_offset) | |
| 627 | −{ | |
| 628 | − return 0; | |
| 629 | −} | |
| 630 | − | |
| 631 | 609 | static void set_vm_context_page_table_base_v10_3(struct amdgpu_device *adev, |
| 632 | 610 | uint32_t vmid, uint64_t page_table_base) |
| 633 | 611 | { |
| @@ -809,10 +787,7 @@ const struct kfd2kgd_calls gfx_v10_3_kfd2kgd = { · 175 unmodified lines | ||
| 809 | 787 | .hqd_sdma_is_occupied = hqd_sdma_is_occupied_v10_3, |
| 810 | 788 | .hqd_destroy = hqd_destroy_v10_3, |
| 811 | 789 | .hqd_sdma_destroy = hqd_sdma_destroy_v10_3, |
| 812 | − .address_watch_disable = address_watch_disable_v10_3, | |
| 813 | − .address_watch_execute = address_watch_execute_v10_3, | |
| 814 | 790 | .wave_control_execute = wave_control_execute_v10_3, |
| 815 | − .address_watch_get_offset = address_watch_get_offset_v10_3, | |
| 816 | 791 | .get_atc_vmid_pasid_mapping_info = NULL, |
| 817 | 792 | .set_vm_context_page_table_base = set_vm_context_page_table_base_v10_3, |
| 818 | 793 | .program_trap_handler_settings = program_trap_handler_settings_v10_3, |
| @@ -45,43 +45,6 @@ enum { · 44 unmodified lines | ||
| 45 | 45 | MAX_WATCH_ADDRESSES = 4 |
| 46 | 46 | }; |
| 47 | 47 | |
| 48 | −enum { | |
| 49 | − ADDRESS_WATCH_REG_ADDR_HI = 0, | |
| 50 | − ADDRESS_WATCH_REG_ADDR_LO, | |
| 51 | − ADDRESS_WATCH_REG_CNTL, | |
| 52 | − ADDRESS_WATCH_REG_MAX | |
| 53 | −}; | |
| 54 | − | |
| 55 | −/* not defined in the CI/KV reg file */ | |
| 56 | −enum { | |
| 57 | − ADDRESS_WATCH_REG_CNTL_ATC_BIT = 0x10000000UL, | |
| 58 | − ADDRESS_WATCH_REG_CNTL_DEFAULT_MASK = 0x00FFFFFF, | |
| 59 | − ADDRESS_WATCH_REG_ADDLOW_MASK_EXTENSION = 0x03000000, | |
| 60 | − /* extend the mask to 26 bits to match the low address field */ | |
| 61 | − ADDRESS_WATCH_REG_ADDLOW_SHIFT = 6, | |
| 62 | − ADDRESS_WATCH_REG_ADDHIGH_MASK = 0xFFFF | |
| 63 | −}; | |
| 64 | − | |
| 65 | −static const uint32_t watchRegs[MAX_WATCH_ADDRESSES * ADDRESS_WATCH_REG_MAX] = { | |
| 66 | − mmTCP_WATCH0_ADDR_H, mmTCP_WATCH0_ADDR_L, mmTCP_WATCH0_CNTL, | |
| 67 | − mmTCP_WATCH1_ADDR_H, mmTCP_WATCH1_ADDR_L, mmTCP_WATCH1_CNTL, | |
| 68 | − mmTCP_WATCH2_ADDR_H, mmTCP_WATCH2_ADDR_L, mmTCP_WATCH2_CNTL, | |
| 69 | − mmTCP_WATCH3_ADDR_H, mmTCP_WATCH3_ADDR_L, mmTCP_WATCH3_CNTL | |
| 70 | −}; | |
| 71 | − | |
| 72 | −union TCP_WATCH_CNTL_BITS { | |
| 73 | − struct { | |
| 74 | − uint32_t mask:24; | |
| 75 | − uint32_t vmid:4; | |
| 76 | − uint32_t atc:1; | |
| 77 | − uint32_t mode:2; | |
| 78 | − uint32_t valid:1; | |
| 79 | − } bitfields, bits; | |
| 80 | − uint32_t u32All; | |
| 81 | − signed int i32All; | |
| 82 | − float f32All; | |
| 83 | −}; | |
| 84 | − | |
| 85 | 48 | static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe, |
| 86 | 49 | uint32_t queue, uint32_t vmid) |
| 87 | 50 | { |
| @@ -529,55 +492,6 @@ static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd, · 441 unmodified lines | ||
| 529 | 492 | return 0; |
| 530 | 493 | } |
| 531 | 494 | |
| 532 | −static int kgd_address_watch_disable(struct amdgpu_device *adev) | |
| 533 | −{ | |
| 534 | − union TCP_WATCH_CNTL_BITS cntl; | |
| 535 | − unsigned int i; | |
| 536 | − | |
| 537 | − cntl.u32All = 0; | |
| 538 | − | |
| 539 | − cntl.bitfields.valid = 0; | |
| 540 | − cntl.bitfields.mask = ADDRESS_WATCH_REG_CNTL_DEFAULT_MASK; | |
| 541 | − cntl.bitfields.atc = 1; | |
| 542 | − | |
| 543 | − /* Turning off this address until we set all the registers */ | |
| 544 | − for (i = 0; i < MAX_WATCH_ADDRESSES; i++) | |
| 545 | − WREG32(watchRegs[i * ADDRESS_WATCH_REG_MAX + | |
| 546 | − ADDRESS_WATCH_REG_CNTL], cntl.u32All); | |
| 547 | − | |
| 548 | − return 0; | |
| 549 | −} | |
| 550 | − | |
| 551 | −static int kgd_address_watch_execute(struct amdgpu_device *adev, | |
| 552 | − unsigned int watch_point_id, | |
| 553 | − uint32_t cntl_val, | |
| 554 | − uint32_t addr_hi, | |
| 555 | − uint32_t addr_lo) | |
| 556 | −{ | |
| 557 | − union TCP_WATCH_CNTL_BITS cntl; | |
| 558 | − | |
| 559 | − cntl.u32All = cntl_val; | |
| 560 | − | |
| 561 | − /* Turning off this watch point until we set all the registers */ | |
| 562 | − cntl.bitfields.valid = 0; | |
| 563 | − WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + | |
| 564 | − ADDRESS_WATCH_REG_CNTL], cntl.u32All); | |
| 565 | − | |
| 566 | − WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + | |
| 567 | − ADDRESS_WATCH_REG_ADDR_HI], addr_hi); | |
| 568 | − | |
| 569 | − WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + | |
| 570 | − ADDRESS_WATCH_REG_ADDR_LO], addr_lo); | |
| 571 | − | |
| 572 | − /* Enable the watch point */ | |
| 573 | − cntl.bitfields.valid = 1; | |
| 574 | − | |
| 575 | − WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + | |
| 576 | − ADDRESS_WATCH_REG_CNTL], cntl.u32All); | |
| 577 | − | |
| 578 | − return 0; | |
| 579 | −} | |
| 580 | − | |
| 581 | 495 | static int kgd_wave_control_execute(struct amdgpu_device *adev, |
| 582 | 496 | uint32_t gfx_index_val, |
| 583 | 497 | uint32_t sq_cmd) |
| @@ -602,13 +516,6 @@ static int kgd_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines | ||
| 602 | 516 | return 0; |
| 603 | 517 | } |
| 604 | 518 | |
| 605 | −static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev, | |
| 606 | − unsigned int watch_point_id, | |
| 607 | − unsigned int reg_offset) | |
| 608 | −{ | |
| 609 | − return watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + reg_offset]; | |
| 610 | −} | |
| 611 | − | |
| 612 | 519 | static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, |
| 613 | 520 | uint8_t vmid, uint16_t *p_pasid) |
| 614 | 521 | { |
| @@ -665,10 +572,7 @@ const struct kfd2kgd_calls gfx_v7_kfd2kgd = { · 50 unmodified lines | ||
| 665 | 572 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, |
| 666 | 573 | .hqd_destroy = kgd_hqd_destroy, |
| 667 | 574 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, |
| 668 | − .address_watch_disable = kgd_address_watch_disable, | |
| 669 | − .address_watch_execute = kgd_address_watch_execute, | |
| 670 | 575 | .wave_control_execute = kgd_wave_control_execute, |
| 671 | − .address_watch_get_offset = kgd_address_watch_get_offset, | |
| 672 | 576 | .get_atc_vmid_pasid_mapping_info = get_atc_vmid_pasid_mapping_info, |
| 673 | 577 | .set_scratch_backing_va = set_scratch_backing_va, |
| 674 | 578 | .set_vm_context_page_table_base = set_vm_context_page_table_base, |
| @@ -538,20 +538,6 @@ static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, · 537 unmodified lines | ||
| 538 | 538 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 539 | 539 | } |
| 540 | 540 | |
| 541 | −static int kgd_address_watch_disable(struct amdgpu_device *adev) | |
| 542 | −{ | |
| 543 | − return 0; | |
| 544 | −} | |
| 545 | − | |
| 546 | −static int kgd_address_watch_execute(struct amdgpu_device *adev, | |
| 547 | − unsigned int watch_point_id, | |
| 548 | − uint32_t cntl_val, | |
| 549 | − uint32_t addr_hi, | |
| 550 | − uint32_t addr_lo) | |
| 551 | −{ | |
| 552 | − return 0; | |
| 553 | −} | |
| 554 | − | |
| 555 | 541 | static int kgd_wave_control_execute(struct amdgpu_device *adev, |
| 556 | 542 | uint32_t gfx_index_val, |
| 557 | 543 | uint32_t sq_cmd) |
| @@ -576,13 +562,6 @@ static int kgd_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines | ||
| 576 | 562 | return 0; |
| 577 | 563 | } |
| 578 | 564 | |
| 579 | −static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev, | |
| 580 | − unsigned int watch_point_id, | |
| 581 | − unsigned int reg_offset) | |
| 582 | −{ | |
| 583 | − return 0; | |
| 584 | −} | |
| 585 | − | |
| 586 | 565 | static void set_scratch_backing_va(struct amdgpu_device *adev, |
| 587 | 566 | uint64_t va, uint32_t vmid) |
| 588 | 567 | { |
| @@ -614,10 +593,7 @@ const struct kfd2kgd_calls gfx_v8_kfd2kgd = { · 25 unmodified lines | ||
| 614 | 593 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, |
| 615 | 594 | .hqd_destroy = kgd_hqd_destroy, |
| 616 | 595 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, |
| 617 | − .address_watch_disable = kgd_address_watch_disable, | |
| 618 | − .address_watch_execute = kgd_address_watch_execute, | |
| 619 | 596 | .wave_control_execute = kgd_wave_control_execute, |
| 620 | − .address_watch_get_offset = kgd_address_watch_get_offset, | |
| 621 | 597 | .get_atc_vmid_pasid_mapping_info = |
| 622 | 598 | get_atc_vmid_pasid_mapping_info, |
| 623 | 599 | .set_scratch_backing_va = set_scratch_backing_va, |
| @@ -622,20 +622,6 @@ bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, · 621 unmodified lines | ||
| 622 | 622 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 623 | 623 | } |
| 624 | 624 | |
| 625 | −int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev) | |
| 626 | −{ | |
| 627 | − return 0; | |
| 628 | −} | |
| 629 | − | |
| 630 | −int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev, | |
| 631 | − unsigned int watch_point_id, | |
| 632 | − uint32_t cntl_val, | |
| 633 | − uint32_t addr_hi, | |
| 634 | − uint32_t addr_lo) | |
| 635 | −{ | |
| 636 | − return 0; | |
| 637 | −} | |
| 638 | − | |
| 639 | 625 | int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, |
| 640 | 626 | uint32_t gfx_index_val, |
| 641 | 627 | uint32_t sq_cmd) |
| @@ -660,13 +646,6 @@ int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines | ||
| 660 | 646 | return 0; |
| 661 | 647 | } |
| 662 | 648 | |
| 663 | −uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev, | |
| 664 | − unsigned int watch_point_id, | |
| 665 | − unsigned int reg_offset) | |
| 666 | −{ | |
| 667 | − return 0; | |
| 668 | −} | |
| 669 | − | |
| 670 | 649 | void kgd_gfx_v9_set_vm_context_page_table_base(struct amdgpu_device *adev, |
| 671 | 650 | uint32_t vmid, uint64_t page_table_base) |
| 672 | 651 | { |
| @@ -736,7 +715,7 @@ static void get_wave_count(struct amdgpu_device *adev, int queue_idx, · 63 unmodified lines | ||
| 736 | 715 | * process whose pasid is provided as a parameter. The process could have ZERO |
| 737 | 716 | * or more queues running and submitting waves to compute units. |
| 738 | 717 | * |
| 739 | − * @kgd: Handle of device from which to get number of waves in flight | |
| 718 | + * @adev: Handle of device from which to get number of waves in flight | |
| 740 | 719 | * @pasid: Identifies the process for which this query call is invoked |
| 741 | 720 | * @pasid_wave_cnt: Output parameter updated with number of waves in flight that |
| 742 | 721 | * belong to process with given pasid |
| @@ -745,7 +724,7 @@ static void get_wave_count(struct amdgpu_device *adev, int queue_idx, · 2 unmodified lines | ||
| 745 | 724 | * |
| 746 | 725 | * Note: It's possible that the device has too many queues (oversubscription) |
| 747 | 726 | * in which case a VMID could be remapped to a different PASID. This could lead |
| 748 | − * to an iaccurate wave count. Following is a high-level sequence: | |
| 727 | + * to an inaccurate wave count. Following is a high-level sequence: | |
| 749 | 728 | * Time T1: vmid = getVmid(); vmid is associated with Pasid P1 |
| 750 | 729 | * Time T2: passId = getPasId(vmid); vmid is associated with Pasid P2 |
| 751 | 730 | * In the sequence above wave count obtained from time T1 will be incorrectly |
| @@ -888,10 +867,7 @@ const struct kfd2kgd_calls gfx_v9_kfd2kgd = { · 136 unmodified lines | ||
| 888 | 867 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, |
| 889 | 868 | .hqd_destroy = kgd_gfx_v9_hqd_destroy, |
| 890 | 869 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, |
| 891 | − .address_watch_disable = kgd_gfx_v9_address_watch_disable, | |
| 892 | − .address_watch_execute = kgd_gfx_v9_address_watch_execute, | |
| 893 | 870 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, |
| 894 | − .address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset, | |
| 895 | 871 | .get_atc_vmid_pasid_mapping_info = |
| 896 | 872 | kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, |
| 897 | 873 | .set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base, |
| @@ -46,19 +46,9 @@ int kgd_gfx_v9_hqd_destroy(struct amdgpu_device *adev, void *mqd, · 45 unmodified lines | ||
| 46 | 46 | enum kfd_preempt_type reset_type, |
| 47 | 47 | unsigned int utimeout, uint32_t pipe_id, |
| 48 | 48 | uint32_t queue_id); |
| 49 | −int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev); | |
| 50 | −int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev, | |
| 51 | − unsigned int watch_point_id, | |
| 52 | − uint32_t cntl_val, | |
| 53 | − uint32_t addr_hi, | |
| 54 | − uint32_t addr_lo); | |
| 55 | 49 | int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, |
| 56 | 50 | uint32_t gfx_index_val, |
| 57 | 51 | uint32_t sq_cmd); |
| 58 | −uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev, | |
| 59 | − unsigned int watch_point_id, | |
| 60 | − unsigned int reg_offset); | |
| 61 | − | |
| 62 | 52 | bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, |
| 63 | 53 | uint8_t vmid, uint16_t *p_pasid); |
| 64 | 54 | |
| @@ -121,7 +121,7 @@ static size_t amdgpu_amdkfd_acc_size(uint64_t size) · 120 unmodified lines | ||
| 121 | 121 | } |
| 122 | 122 | |
| 123 | 123 | /** |
| 124 | − * @amdgpu_amdkfd_reserve_mem_limit() - Decrease available memory by size | |
| 124 | + * amdgpu_amdkfd_reserve_mem_limit() - Decrease available memory by size | |
| 125 | 125 | * of buffer including any reserved for control structures |
| 126 | 126 | * |
| 127 | 127 | * @adev: Device to which allocated BO belongs to |
| @@ -778,7 +778,7 @@ unwind: · 650 unmodified lines | ||
| 778 | 778 | continue; |
| 779 | 779 | if (attachment[i]->bo_va) { |
| 780 | 780 | amdgpu_bo_reserve(bo[i], true); |
| 781 | − amdgpu_vm_bo_rmv(adev, attachment[i]->bo_va); | |
| 781 | + amdgpu_vm_bo_del(adev, attachment[i]->bo_va); | |
| 782 | 782 | amdgpu_bo_unreserve(bo[i]); |
| 783 | 783 | list_del(&attachment[i]->list); |
| 784 | 784 | } |
| @@ -795,7 +795,7 @@ static void kfd_mem_detach(struct kfd_mem_attachment *attachment) · 10 unmodified lines | ||
| 795 | 795 | |
| 796 | 796 | pr_debug("\t remove VA 0x%llx in entry %p\n", |
| 797 | 797 | attachment->va, attachment); |
| 798 | − amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); | |
| 798 | + amdgpu_vm_bo_del(attachment->adev, attachment->bo_va); | |
| 799 | 799 | drm_gem_object_put(&bo->tbo.base); |
| 800 | 800 | list_del(&attachment->list); |
| 801 | 801 | kfree(attachment); |
| @@ -842,7 +842,8 @@ static void remove_kgd_mem_from_kfd_bo_list(struct kgd_mem *mem, · 40 unmodified lines | ||
| 842 | 842 | * |
| 843 | 843 | * Returns 0 for success, negative errno for errors. |
| 844 | 844 | */ |
| 845 | −static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr) | |
| 845 | +static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr, | |
| 846 | + bool criu_resume) | |
| 846 | 847 | { |
| 847 | 848 | struct amdkfd_process_info *process_info = mem->process_info; |
| 848 | 849 | struct amdgpu_bo *bo = mem->bo; |
| @@ -864,6 +865,18 @@ static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr) · 15 unmodified lines | ||
| 864 | 865 | goto out; |
| 865 | 866 | } |
| 866 | 867 | |
| 868 | + if (criu_resume) { | |
| 869 | + /* | |
| 870 | + * During a CRIU restore operation, the userptr buffer objects | |
| 871 | + * will be validated in the restore_userptr_work worker at a | |
| 872 | + * later stage when it is scheduled by another ioctl called by | |
| 873 | + * CRIU master process for the target pid for restore. | |
| 874 | + */ | |
| 875 | + atomic_inc(&mem->invalid); | |
| 876 | + mutex_unlock(&process_info->lock); | |
| 877 | + return 0; | |
| 878 | + } | |
| 879 | + | |
| 867 | 880 | ret = amdgpu_ttm_tt_get_user_pages(bo, bo->tbo.ttm->pages); |
| 868 | 881 | if (ret) { |
| 869 | 882 | pr_err("%s: Failed to get user pages: %d\n", __func__, ret); |
| @@ -1452,10 +1465,39 @@ uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *drm_priv) · 582 unmodified lines | ||
| 1452 | 1465 | return avm->pd_phys_addr; |
| 1453 | 1466 | } |
| 1454 | 1467 | |
| 1468 | +void amdgpu_amdkfd_block_mmu_notifications(void *p) | |
| 1469 | +{ | |
| 1470 | + struct amdkfd_process_info *pinfo = (struct amdkfd_process_info *)p; | |
| 1471 | + | |
| 1472 | + mutex_lock(&pinfo->lock); | |
| 1473 | + WRITE_ONCE(pinfo->block_mmu_notifications, true); | |
| 1474 | + mutex_unlock(&pinfo->lock); | |
| 1475 | +} | |
| 1476 | + | |
| 1477 | +int amdgpu_amdkfd_criu_resume(void *p) | |
| 1478 | +{ | |
| 1479 | + int ret = 0; | |
| 1480 | + struct amdkfd_process_info *pinfo = (struct amdkfd_process_info *)p; | |
| 1481 | + | |
| 1482 | + mutex_lock(&pinfo->lock); | |
| 1483 | + pr_debug("scheduling work\n"); | |
| 1484 | + atomic_inc(&pinfo->evicted_bos); | |
| 1485 | + if (!READ_ONCE(pinfo->block_mmu_notifications)) { | |
| 1486 | + ret = -EINVAL; | |
| 1487 | + goto out_unlock; | |
| 1488 | + } | |
| 1489 | + WRITE_ONCE(pinfo->block_mmu_notifications, false); | |
| 1490 | + schedule_delayed_work(&pinfo->restore_userptr_work, 0); | |
| 1491 | + | |
| 1492 | +out_unlock: | |
| 1493 | + mutex_unlock(&pinfo->lock); | |
| 1494 | + return ret; | |
| 1495 | +} | |
| 1496 | + | |
| 1455 | 1497 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( |
| 1456 | 1498 | struct amdgpu_device *adev, uint64_t va, uint64_t size, |
| 1457 | 1499 | void *drm_priv, struct kgd_mem **mem, |
| 1458 | − uint64_t *offset, uint32_t flags) | |
| 1500 | + uint64_t *offset, uint32_t flags, bool criu_resume) | |
| 1459 | 1501 | { |
| 1460 | 1502 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); |
| 1461 | 1503 | enum ttm_bo_type bo_type = ttm_bo_type_device; |
| @@ -1558,7 +1600,8 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( · 96 unmodified lines | ||
| 1558 | 1600 | add_kgd_mem_to_kfd_bo_list(*mem, avm->process_info, user_addr); |
| 1559 | 1601 | |
| 1560 | 1602 | if (user_addr) { |
| 1561 | − ret = init_user_pages(*mem, user_addr); | |
| 1603 | + pr_debug("creating userptr BO for user_addr = %llu\n", user_addr); | |
| 1604 | + ret = init_user_pages(*mem, user_addr, criu_resume); | |
| 1562 | 1605 | if (ret) |
| 1563 | 1606 | goto allocate_init_user_pages_failed; |
| 1564 | 1607 | } else if (flags & (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL | |
| @@ -1813,12 +1856,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( · 248 unmodified lines | ||
| 1813 | 1856 | true); |
| 1814 | 1857 | ret = unreserve_bo_and_vms(&ctx, false, false); |
| 1815 | 1858 | |
| 1816 | − /* Only apply no TLB flush on Aldebaran to | |
| 1817 | − * workaround regressions on other Asics. | |
| 1818 | − */ | |
| 1819 | − if (table_freed && (adev->asic_type != CHIP_ALDEBARAN)) | |
| 1820 | − *table_freed = true; | |
| 1821 | − | |
| 1822 | 1859 | goto out; |
| 1823 | 1860 | |
| 1824 | 1861 | out_unreserve: |
| @@ -2068,6 +2105,10 @@ int amdgpu_amdkfd_evict_userptr(struct kgd_mem *mem, · 243 unmodified lines | ||
| 2068 | 2105 | int evicted_bos; |
| 2069 | 2106 | int r = 0; |
| 2070 | 2107 | |
| 2108 | + /* Do not process MMU notifications until stage-4 IOCTL is received */ | |
| 2109 | + if (READ_ONCE(process_info->block_mmu_notifications)) | |
| 2110 | + return 0; | |
| 2111 | + | |
| 2071 | 2112 | atomic_inc(&mem->invalid); |
| 2072 | 2113 | evicted_bos = atomic_inc_return(&process_info->evicted_bos); |
| 2073 | 2114 | if (evicted_bos == 1) { |
| @@ -2635,3 +2676,14 @@ int amdgpu_amdkfd_get_tile_config(struct amdgpu_device *adev, · 561 unmodified lines | ||
| 2635 | 2676 | |
| 2636 | 2677 | return 0; |
| 2637 | 2678 | } |
| 2679 | + | |
| 2680 | +bool amdgpu_amdkfd_bo_mapped_to_dev(struct amdgpu_device *adev, struct kgd_mem *mem) | |
| 2681 | +{ | |
| 2682 | + struct kfd_mem_attachment *entry; | |
| 2683 | + | |
| 2684 | + list_for_each_entry(entry, &mem->attachments, list) { | |
| 2685 | + if (entry->is_mapped && entry->adev == adev) | |
| 2686 | + return true; | |
| 2687 | + } | |
| 2688 | + return false; | |
| 2689 | +} | |
| @@ -1083,6 +1083,7 @@ int amdgpu_atombios_get_clock_dividers(struct amdgpu_device *adev, · 1082 unmodified lines | ||
| 1083 | 1083 | return 0; |
| 1084 | 1084 | } |
| 1085 | 1085 | |
| 1086 | +#ifdef CONFIG_DRM_AMDGPU_SI | |
| 1086 | 1087 | int amdgpu_atombios_get_memory_pll_dividers(struct amdgpu_device *adev, |
| 1087 | 1088 | u32 clock, |
| 1088 | 1089 | bool strobe_mode, |
| @@ -1503,6 +1504,7 @@ int amdgpu_atombios_init_mc_reg_table(struct amdgpu_device *adev, · 414 unmodified lines | ||
| 1503 | 1504 | } |
| 1504 | 1505 | return -EINVAL; |
| 1505 | 1506 | } |
| 1507 | +#endif | |
| 1506 | 1508 | |
| 1507 | 1509 | bool amdgpu_atombios_has_gpu_virtualization_table(struct amdgpu_device *adev) |
| 1508 | 1510 | { |
| @@ -160,6 +160,7 @@ int amdgpu_atombios_get_clock_dividers(struct amdgpu_device *adev, · 159 unmodified lines | ||
| 160 | 160 | bool strobe_mode, |
| 161 | 161 | struct atom_clock_dividers *dividers); |
| 162 | 162 | |
| 163 | +#ifdef CONFIG_DRM_AMDGPU_SI | |
| 163 | 164 | int amdgpu_atombios_get_memory_pll_dividers(struct amdgpu_device *adev, |
| 164 | 165 | u32 clock, |
| 165 | 166 | bool strobe_mode, |
| @@ -179,6 +180,17 @@ int amdgpu_atombios_get_voltage_table(struct amdgpu_device *adev, · 13 unmodified lines | ||
| 179 | 180 | int amdgpu_atombios_init_mc_reg_table(struct amdgpu_device *adev, |
| 180 | 181 | u8 module_index, |
| 181 | 182 | struct atom_mc_reg_table *reg_table); |
| 183 | +int amdgpu_atombios_get_max_vddc(struct amdgpu_device *adev, u8 voltage_type, | |
| 184 | + u16 voltage_id, u16 *voltage); | |
| 185 | +int amdgpu_atombios_get_leakage_vddc_based_on_leakage_idx(struct amdgpu_device *adev, | |
| 186 | + u16 *voltage, | |
| 187 | + u16 leakage_idx); | |
| 188 | +void amdgpu_atombios_get_default_voltages(struct amdgpu_device *adev, | |
| 189 | + u16 *vddc, u16 *vddci, u16 *mvdd); | |
| 190 | +int amdgpu_atombios_get_svi2_info(struct amdgpu_device *adev, | |
| 191 | + u8 voltage_type, | |
| 192 | + u8 *svd_gpio_id, u8 *svc_gpio_id); | |
| 193 | +#endif | |
| 182 | 194 | |
| 183 | 195 | bool amdgpu_atombios_has_gpu_virtualization_table(struct amdgpu_device *adev); |
| 184 | 196 | |
| @@ -190,21 +202,11 @@ void amdgpu_atombios_scratch_regs_set_backlight_level(struct amdgpu_device *adev · 5 unmodified lines | ||
| 190 | 202 | bool amdgpu_atombios_scratch_need_asic_init(struct amdgpu_device *adev); |
| 191 | 203 | |
| 192 | 204 | void amdgpu_atombios_copy_swap(u8 *dst, u8 *src, u8 num_bytes, bool to_le); |
| 193 | −int amdgpu_atombios_get_max_vddc(struct amdgpu_device *adev, u8 voltage_type, | |
| 194 | − u16 voltage_id, u16 *voltage); | |
| 195 | −int amdgpu_atombios_get_leakage_vddc_based_on_leakage_idx(struct amdgpu_device *adev, | |
| 196 | − u16 *voltage, | |
| 197 | − u16 leakage_idx); | |
| 198 | −void amdgpu_atombios_get_default_voltages(struct amdgpu_device *adev, | |
| 199 | − u16 *vddc, u16 *vddci, u16 *mvdd); | |
| 200 | 205 | int amdgpu_atombios_get_clock_dividers(struct amdgpu_device *adev, |
| 201 | 206 | u8 clock_type, |
| 202 | 207 | u32 clock, |
| 203 | 208 | bool strobe_mode, |
| 204 | 209 | struct atom_clock_dividers *dividers); |
| 205 | −int amdgpu_atombios_get_svi2_info(struct amdgpu_device *adev, | |
| 206 | − u8 voltage_type, | |
| 207 | − u8 *svd_gpio_id, u8 *svc_gpio_id); | |
| 208 | 210 | |
| 209 | 211 | int amdgpu_atombios_get_data_table(struct amdgpu_device *adev, |
| 210 | 212 | uint32_t table, |
| @@ -29,14 +29,13 @@ · 28 unmodified lines | ||
| 29 | 29 | #define AMDGPU_BENCHMARK_COMMON_MODES_N 17 |
| 30 | 30 | |
| 31 | 31 | static int amdgpu_benchmark_do_move(struct amdgpu_device *adev, unsigned size, |
| 32 | − uint64_t saddr, uint64_t daddr, int n) | |
| 32 | + uint64_t saddr, uint64_t daddr, int n, s64 *time_ms) | |
| 33 | 33 | { |
| 34 | − unsigned long start_jiffies; | |
| 35 | − unsigned long end_jiffies; | |
| 34 | + ktime_t stime, etime; | |
| 36 | 35 | struct dma_fence *fence; |
| 37 | 36 | int i, r; |
| 38 | 37 | |
| 39 | − start_jiffies = jiffies; | |
| 38 | + stime = ktime_get(); | |
| 40 | 39 | for (i = 0; i < n; i++) { |
| 41 | 40 | struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; |
| 42 | 41 | r = amdgpu_copy_buffer(ring, saddr, daddr, size, NULL, &fence, |
| @@ -48,120 +47,81 @@ static int amdgpu_benchmark_do_move(struct amdgpu_device *adev, unsigned size, · 5 unmodified lines | ||
| 48 | 47 | if (r) |
| 49 | 48 | goto exit_do_move; |
| 50 | 49 | } |
| 51 | − end_jiffies = jiffies; | |
| 52 | − r = jiffies_to_msecs(end_jiffies - start_jiffies); | |
| 53 | 50 | |
| 54 | 51 | exit_do_move: |
| 52 | + etime = ktime_get(); | |
| 53 | + *time_ms = ktime_ms_delta(etime, stime); | |
| 54 | + | |
| 55 | 55 | return r; |
| 56 | 56 | } |
| 57 | 57 | |
| 58 | 58 | |
| 59 | −static void amdgpu_benchmark_log_results(int n, unsigned size, | |
| 60 | − unsigned int time, | |
| 59 | +static void amdgpu_benchmark_log_results(struct amdgpu_device *adev, | |
| 60 | + int n, unsigned size, | |
| 61 | + s64 time_ms, | |
| 61 | 62 | unsigned sdomain, unsigned ddomain, |
| 62 | 63 | char *kind) |
| 63 | 64 | { |
| 64 | − unsigned int throughput = (n * (size >> 10)) / time; | |
| 65 | − DRM_INFO("amdgpu: %s %u bo moves of %u kB from" | |
| 66 | − " %d to %d in %u ms, throughput: %u Mb/s or %u MB/s\n", | |
| 67 | − kind, n, size >> 10, sdomain, ddomain, time, | |
| 65 | + s64 throughput = (n * (size >> 10)); | |
| 66 | + | |
| 67 | + throughput = div64_s64(throughput, time_ms); | |
| 68 | + | |
| 69 | + dev_info(adev->dev, "amdgpu: %s %u bo moves of %u kB from" | |
| 70 | + " %d to %d in %lld ms, throughput: %lld Mb/s or %lld MB/s\n", | |
| 71 | + kind, n, size >> 10, sdomain, ddomain, time_ms, | |
| 68 | 72 | throughput * 8, throughput); |
| 69 | 73 | } |
| 70 | 74 | |
| 71 | −static void amdgpu_benchmark_move(struct amdgpu_device *adev, unsigned size, | |
| 72 | − unsigned sdomain, unsigned ddomain) | |
| 75 | +static int amdgpu_benchmark_move(struct amdgpu_device *adev, unsigned size, | |
| 76 | + unsigned sdomain, unsigned ddomain) | |
| 73 | 77 | { |
| 74 | 78 | struct amdgpu_bo *dobj = NULL; |
| 75 | 79 | struct amdgpu_bo *sobj = NULL; |
| 76 | − struct amdgpu_bo_param bp; | |
| 77 | 80 | uint64_t saddr, daddr; |
| 81 | + s64 time_ms; | |
| 78 | 82 | int r, n; |
| 79 | − int time; | |
| 80 | − | |
| 81 | − memset(&bp, 0, sizeof(bp)); | |
| 82 | − bp.size = size; | |
| 83 | − bp.byte_align = PAGE_SIZE; | |
| 84 | − bp.domain = sdomain; | |
| 85 | − bp.flags = 0; | |
| 86 | − bp.type = ttm_bo_type_kernel; | |
| 87 | − bp.resv = NULL; | |
| 88 | − bp.bo_ptr_size = sizeof(struct amdgpu_bo); | |
| 89 | 83 | |
| 90 | 84 | n = AMDGPU_BENCHMARK_ITERATIONS; |
| 91 | − r = amdgpu_bo_create(adev, &bp, &sobj); | |
| 92 | − if (r) { | |
| 93 | − goto out_cleanup; | |
| 94 | − } | |
| 95 | − r = amdgpu_bo_reserve(sobj, false); | |
| 96 | − if (unlikely(r != 0)) | |
| 97 | − goto out_cleanup; | |
| 98 | − r = amdgpu_bo_pin(sobj, sdomain); | |
| 99 | − if (r) { | |
| 100 | − amdgpu_bo_unreserve(sobj); | |
| 101 | − goto out_cleanup; | |
| 102 | − } | |
| 103 | − r = amdgpu_ttm_alloc_gart(&sobj->tbo); | |
| 104 | − amdgpu_bo_unreserve(sobj); | |
| 105 | − if (r) { | |
| 106 | − goto out_cleanup; | |
| 107 | − } | |
| 108 | − saddr = amdgpu_bo_gpu_offset(sobj); | |
| 109 | − bp.domain = ddomain; | |
| 110 | − r = amdgpu_bo_create(adev, &bp, &dobj); | |
| 111 | − if (r) { | |
| 112 | − goto out_cleanup; | |
| 113 | − } | |
| 114 | − r = amdgpu_bo_reserve(dobj, false); | |
| 115 | − if (unlikely(r != 0)) | |
| 116 | − goto out_cleanup; | |
| 117 | − r = amdgpu_bo_pin(dobj, ddomain); | |
| 118 | − if (r) { | |
| 119 | − amdgpu_bo_unreserve(sobj); | |
| 85 | + | |
| 86 | + r = amdgpu_bo_create_kernel(adev, size, | |
| 87 | + PAGE_SIZE, sdomain, | |
| 88 | + &sobj, | |
| 89 | + &saddr, | |
| 90 | + NULL); | |
| 91 | + if (r) | |
| 120 | 92 | goto out_cleanup; |
| 121 | − } | |
| 122 | − r = amdgpu_ttm_alloc_gart(&dobj->tbo); | |
| 123 | − amdgpu_bo_unreserve(dobj); | |
| 124 | − if (r) { | |
| 93 | + r = amdgpu_bo_create_kernel(adev, size, | |
| 94 | + PAGE_SIZE, ddomain, | |
| 95 | + &dobj, | |
| 96 | + &daddr, | |
| 97 | + NULL); | |
| 98 | + if (r) | |
| 125 | 99 | goto out_cleanup; |
| 126 | − } | |
| 127 | − daddr = amdgpu_bo_gpu_offset(dobj); | |
| 128 | 100 | |
| 129 | 101 | if (adev->mman.buffer_funcs) { |
| 130 | − time = amdgpu_benchmark_do_move(adev, size, saddr, daddr, n); | |
| 131 | − if (time < 0) | |
| 102 | + r = amdgpu_benchmark_do_move(adev, size, saddr, daddr, n, &time_ms); | |
| 103 | + if (r) | |
| 132 | 104 | goto out_cleanup; |
| 133 | − if (time > 0) | |
| 134 | − amdgpu_benchmark_log_results(n, size, time, | |
| 105 | + else | |
| 106 | + amdgpu_benchmark_log_results(adev, n, size, time_ms, | |
| 135 | 107 | sdomain, ddomain, "dma"); |
| 136 | 108 | } |
| 137 | 109 | |
| 138 | 110 | out_cleanup: |
| 139 | 111 | /* Check error value now. The value can be overwritten when clean up.*/ |
| 140 | − if (r) { | |
| 141 | − DRM_ERROR("Error while benchmarking BO move.\n"); | |
| 142 | − } | |
| 112 | + if (r < 0) | |
| 113 | + dev_info(adev->dev, "Error while benchmarking BO move.\n"); | |
| 143 | 114 | |
| 144 | − if (sobj) { | |
| 145 | − r = amdgpu_bo_reserve(sobj, true); | |
| 146 | − if (likely(r == 0)) { | |
| 147 | − amdgpu_bo_unpin(sobj); | |
| 148 | − amdgpu_bo_unreserve(sobj); | |
| 149 | − } | |
| 150 | − amdgpu_bo_unref(&sobj); | |
| 151 | − } | |
| 152 | − if (dobj) { | |
| 153 | − r = amdgpu_bo_reserve(dobj, true); | |
| 154 | − if (likely(r == 0)) { | |
| 155 | − amdgpu_bo_unpin(dobj); | |
| 156 | − amdgpu_bo_unreserve(dobj); | |
| 157 | − } | |
| 158 | − amdgpu_bo_unref(&dobj); | |
| 159 | − } | |
| 115 | + if (sobj) | |
| 116 | + amdgpu_bo_free_kernel(&sobj, &saddr, NULL); | |
| 117 | + if (dobj) | |
| 118 | + amdgpu_bo_free_kernel(&dobj, &daddr, NULL); | |
| 119 | + return r; | |
| 160 | 120 | } |
| 161 | 121 | |
| 162 | −void amdgpu_benchmark(struct amdgpu_device *adev, int test_number) | |
| 122 | +int amdgpu_benchmark(struct amdgpu_device *adev, int test_number) | |
| 163 | 123 | { |
| 164 | − int i; | |
| 124 | + int i, r; | |
| 165 | 125 | static const int common_modes[AMDGPU_BENCHMARK_COMMON_MODES_N] = { |
| 166 | 126 | 640 * 480 * 4, |
| 167 | 127 | 720 * 480 * 4, |
| @@ -182,63 +142,119 @@ void amdgpu_benchmark(struct amdgpu_device *adev, int test_number) · 14 unmodified lines | ||
| 182 | 142 | 1920 * 1200 * 4 |
| 183 | 143 | }; |
| 184 | 144 | |
| 145 | + mutex_lock(&adev->benchmark_mutex); | |
| 185 | 146 | switch (test_number) { |
| 186 | 147 | case 1: |
| 148 | + dev_info(adev->dev, | |
| 149 | + "benchmark test: %d (simple test, VRAM to GTT and GTT to VRAM)\n", | |
| 150 | + test_number); | |
| 187 | 151 | /* simple test, VRAM to GTT and GTT to VRAM */ |
| 188 | − amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_GTT, | |
| 189 | − AMDGPU_GEM_DOMAIN_VRAM); | |
| 190 | − amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_VRAM, | |
| 191 | − AMDGPU_GEM_DOMAIN_GTT); | |
| 152 | + r = amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_GTT, | |
| 153 | + AMDGPU_GEM_DOMAIN_VRAM); | |
| 154 | + if (r) | |
| 155 | + goto done; | |
| 156 | + r = amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_VRAM, | |
| 157 | + AMDGPU_GEM_DOMAIN_GTT); | |
| 158 | + if (r) | |
| 159 | + goto done; | |
| 192 | 160 | break; |
| 193 | 161 | case 2: |
| 162 | + dev_info(adev->dev, | |
| 163 | + "benchmark test: %d (simple test, VRAM to VRAM)\n", | |
| 164 | + test_number); | |
| 194 | 165 | /* simple test, VRAM to VRAM */ |
| 195 | − amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_VRAM, | |
| 196 | − AMDGPU_GEM_DOMAIN_VRAM); | |
| 166 | + r = amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_VRAM, | |
| 167 | + AMDGPU_GEM_DOMAIN_VRAM); | |
| 168 | + if (r) | |
| 169 | + goto done; | |
| 197 | 170 | break; |
| 198 | 171 | case 3: |
| 172 | + dev_info(adev->dev, | |
| 173 | + "benchmark test: %d (GTT to VRAM, buffer size sweep, powers of 2)\n", | |
| 174 | + test_number); | |
| 199 | 175 | /* GTT to VRAM, buffer size sweep, powers of 2 */ |
| 200 | − for (i = 1; i <= 16384; i <<= 1) | |
| 201 | − amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, | |
| 202 | − AMDGPU_GEM_DOMAIN_GTT, | |
| 203 | − AMDGPU_GEM_DOMAIN_VRAM); | |
| 176 | + for (i = 1; i <= 16384; i <<= 1) { | |
| 177 | + r = amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, | |
| 178 | + AMDGPU_GEM_DOMAIN_GTT, | |
| 179 | + AMDGPU_GEM_DOMAIN_VRAM); | |
| 180 | + if (r) | |
| 181 | + goto done; | |
| 182 | + } | |
| 204 | 183 | break; |
| 205 | 184 | case 4: |
| 185 | + dev_info(adev->dev, | |
| 186 | + "benchmark test: %d (VRAM to GTT, buffer size sweep, powers of 2)\n", | |
| 187 | + test_number); | |
| 206 | 188 | /* VRAM to GTT, buffer size sweep, powers of 2 */ |
| 207 | − for (i = 1; i <= 16384; i <<= 1) | |
| 208 | − amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, | |
| 209 | − AMDGPU_GEM_DOMAIN_VRAM, | |
| 210 | − AMDGPU_GEM_DOMAIN_GTT); | |
| 189 | + for (i = 1; i <= 16384; i <<= 1) { | |
| 190 | + r = amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, | |
| 191 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 192 | + AMDGPU_GEM_DOMAIN_GTT); | |
| 193 | + if (r) | |
| 194 | + goto done; | |
| 195 | + } | |
| 211 | 196 | break; |
| 212 | 197 | case 5: |
| 198 | + dev_info(adev->dev, | |
| 199 | + "benchmark test: %d (VRAM to VRAM, buffer size sweep, powers of 2)\n", | |
| 200 | + test_number); | |
| 213 | 201 | /* VRAM to VRAM, buffer size sweep, powers of 2 */ |
| 214 | − for (i = 1; i <= 16384; i <<= 1) | |
| 215 | − amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, | |
| 216 | − AMDGPU_GEM_DOMAIN_VRAM, | |
| 217 | − AMDGPU_GEM_DOMAIN_VRAM); | |
| 202 | + for (i = 1; i <= 16384; i <<= 1) { | |
| 203 | + r = amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, | |
| 204 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 205 | + AMDGPU_GEM_DOMAIN_VRAM); | |
| 206 | + if (r) | |
| 207 | + goto done; | |
| 208 | + } | |
| 218 | 209 | break; |
| 219 | 210 | case 6: |
| 211 | + dev_info(adev->dev, | |
| 212 | + "benchmark test: %d (GTT to VRAM, buffer size sweep, common modes)\n", | |
| 213 | + test_number); | |
| 220 | 214 | /* GTT to VRAM, buffer size sweep, common modes */ |
| 221 | − for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) | |
| 222 | − amdgpu_benchmark_move(adev, common_modes[i], | |
| 223 | − AMDGPU_GEM_DOMAIN_GTT, | |
| 224 | − AMDGPU_GEM_DOMAIN_VRAM); | |
| 215 | + for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) { | |
| 216 | + r = amdgpu_benchmark_move(adev, common_modes[i], | |
| 217 | + AMDGPU_GEM_DOMAIN_GTT, | |
| 218 | + AMDGPU_GEM_DOMAIN_VRAM); | |
| 219 | + if (r) | |
| 220 | + goto done; | |
| 221 | + } | |
| 225 | 222 | break; |
| 226 | 223 | case 7: |
| 224 | + dev_info(adev->dev, | |
| 225 | + "benchmark test: %d (VRAM to GTT, buffer size sweep, common modes)\n", | |
| 226 | + test_number); | |
| 227 | 227 | /* VRAM to GTT, buffer size sweep, common modes */ |
| 228 | − for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) | |
| 229 | − amdgpu_benchmark_move(adev, common_modes[i], | |
| 230 | − AMDGPU_GEM_DOMAIN_VRAM, | |
| 231 | − AMDGPU_GEM_DOMAIN_GTT); | |
| 228 | + for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) { | |
| 229 | + r = amdgpu_benchmark_move(adev, common_modes[i], | |
| 230 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 231 | + AMDGPU_GEM_DOMAIN_GTT); | |
| 232 | + if (r) | |
| 233 | + goto done; | |
| 234 | + } | |
| 232 | 235 | break; |
| 233 | 236 | case 8: |
| 237 | + dev_info(adev->dev, | |
| 238 | + "benchmark test: %d (VRAM to VRAM, buffer size sweep, common modes)\n", | |
| 239 | + test_number); | |
| 234 | 240 | /* VRAM to VRAM, buffer size sweep, common modes */ |
| 235 | − for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) | |
| 236 | − amdgpu_benchmark_move(adev, common_modes[i], | |
| 241 | + for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) { | |
| 242 | + r = amdgpu_benchmark_move(adev, common_modes[i], | |
| 237 | 243 | AMDGPU_GEM_DOMAIN_VRAM, |
| 238 | 244 | AMDGPU_GEM_DOMAIN_VRAM); |
| 245 | + if (r) | |
| 246 | + goto done; | |
| 247 | + } | |
| 239 | 248 | break; |
| 240 | 249 | |
| 241 | 250 | default: |
| 242 | − DRM_ERROR("Unknown benchmark\n"); | |
| 251 | + dev_info(adev->dev, "Unknown benchmark %d\n", test_number); | |
| 252 | + r = -EINVAL; | |
| 253 | + break; | |
| 243 | 254 | } |
| 255 | + | |
| 256 | +done: | |
| 257 | + mutex_unlock(&adev->benchmark_mutex); | |
| 258 | + | |
| 259 | + return r; | |
| 244 | 260 | } |
| @@ -464,3 +464,41 @@ success: · 463 unmodified lines | ||
| 464 | 464 | adev->is_atom_fw = (adev->asic_type >= CHIP_VEGA10) ? true : false; |
| 465 | 465 | return true; |
| 466 | 466 | } |
| 467 | + | |
| 468 | +/* helper function for soc15 and onwards to read bios from rom */ | |
| 469 | +bool amdgpu_soc15_read_bios_from_rom(struct amdgpu_device *adev, | |
| 470 | + u8 *bios, u32 length_bytes) | |
| 471 | +{ | |
| 472 | + u32 *dw_ptr; | |
| 473 | + u32 i, length_dw; | |
| 474 | + u32 rom_index_offset; | |
| 475 | + u32 rom_data_offset; | |
| 476 | + | |
| 477 | + if (bios == NULL) | |
| 478 | + return false; | |
| 479 | + if (length_bytes == 0) | |
| 480 | + return false; | |
| 481 | + /* APU vbios image is part of sbios image */ | |
| 482 | + if (adev->flags & AMD_IS_APU) | |
| 483 | + return false; | |
| 484 | + if (!adev->smuio.funcs || | |
| 485 | + !adev->smuio.funcs->get_rom_index_offset || | |
| 486 | + !adev->smuio.funcs->get_rom_data_offset) | |
| 487 | + return false; | |
| 488 | + | |
| 489 | + dw_ptr = (u32 *)bios; | |
| 490 | + length_dw = ALIGN(length_bytes, 4) / 4; | |
| 491 | + | |
| 492 | + rom_index_offset = | |
| 493 | + adev->smuio.funcs->get_rom_index_offset(adev); | |
| 494 | + rom_data_offset = | |
| 495 | + adev->smuio.funcs->get_rom_data_offset(adev); | |
| 496 | + | |
| 497 | + /* set rom index to 0 */ | |
| 498 | + WREG32(rom_index_offset, 0); | |
| 499 | + /* read out the rom data */ | |
| 500 | + for (i = 0; i < length_dw; i++) | |
| 501 | + dw_ptr[i] = RREG32(rom_data_offset); | |
| 502 | + | |
| 503 | + return true; | |
| 504 | +} | |
| @@ -127,8 +127,6 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs · 126 unmodified lines | ||
| 127 | 127 | goto free_chunk; |
| 128 | 128 | } |
| 129 | 129 | |
| 130 | − mutex_lock(&p->ctx->lock); | |
| 131 | − | |
| 132 | 130 | /* skip guilty context job */ |
| 133 | 131 | if (atomic_read(&p->ctx->guilty) == 1) { |
| 134 | 132 | ret = -ECANCELED; |
| @@ -314,7 +312,7 @@ static void amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev, · 179 unmodified lines | ||
| 314 | 312 | } |
| 315 | 313 | |
| 316 | 314 | total_vram = adev->gmc.real_vram_size - atomic64_read(&adev->vram_pin_size); |
| 317 | − used_vram = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr); | |
| 315 | + used_vram = ttm_resource_manager_usage(&adev->mman.vram_mgr.manager); | |
| 318 | 316 | free_vram = used_vram >= total_vram ? 0 : total_vram - used_vram; |
| 319 | 317 | |
| 320 | 318 | spin_lock(&adev->mm_stats.lock); |
| @@ -585,6 +583,16 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, · 264 unmodified lines | ||
| 585 | 583 | } |
| 586 | 584 | } |
| 587 | 585 | |
| 586 | + /* Move fence waiting after getting reservation lock of | |
| 587 | + * PD root. Then there is no need on a ctx mutex lock. | |
| 588 | + */ | |
| 589 | + r = amdgpu_ctx_wait_prev_fence(p->ctx, p->entity); | |
| 590 | + if (unlikely(r != 0)) { | |
| 591 | + if (r != -ERESTARTSYS) | |
| 592 | + DRM_ERROR("amdgpu_ctx_wait_prev_fence failed.\n"); | |
| 593 | + goto error_validate; | |
| 594 | + } | |
| 595 | + | |
| 588 | 596 | amdgpu_cs_get_threshold_for_moves(p->adev, &p->bytes_moved_threshold, |
| 589 | 597 | &p->bytes_moved_vis_threshold); |
| 590 | 598 | p->bytes_moved = 0; |
| @@ -700,7 +708,6 @@ static void amdgpu_cs_parser_fini(struct amdgpu_cs_parser *parser, int error, · 109 unmodified lines | ||
| 700 | 708 | dma_fence_put(parser->fence); |
| 701 | 709 | |
| 702 | 710 | if (parser->ctx) { |
| 703 | − mutex_unlock(&parser->ctx->lock); | |
| 704 | 711 | amdgpu_ctx_put(parser->ctx); |
| 705 | 712 | } |
| 706 | 713 | if (parser->bo_list) |
| @@ -944,7 +951,7 @@ static int amdgpu_cs_ib_fill(struct amdgpu_device *adev, · 237 unmodified lines | ||
| 944 | 951 | if (parser->job->uf_addr && ring->funcs->no_user_fence) |
| 945 | 952 | return -EINVAL; |
| 946 | 953 | |
| 947 | − return amdgpu_ctx_wait_prev_fence(parser->ctx, parser->entity); | |
| 954 | + return 0; | |
| 948 | 955 | } |
| 949 | 956 | |
| 950 | 957 | static int amdgpu_cs_process_fence_dep(struct amdgpu_cs_parser *p, |
| @@ -1360,7 +1367,6 @@ int amdgpu_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 409 unmodified lines | ||
| 1360 | 1367 | goto out; |
| 1361 | 1368 | |
| 1362 | 1369 | r = amdgpu_cs_submit(&parser, cs); |
| 1363 | − | |
| 1364 | 1370 | out: |
| 1365 | 1371 | amdgpu_cs_parser_fini(&parser, r, reserved_buffers); |
| 1366 | 1372 | |
| @@ -1506,6 +1512,7 @@ int amdgpu_cs_fence_to_handle_ioctl(struct drm_device *dev, void *data, · 139 unmodified lines | ||
| 1506 | 1512 | return 0; |
| 1507 | 1513 | |
| 1508 | 1514 | default: |
| 1515 | + dma_fence_put(fence); | |
| 1509 | 1516 | return -EINVAL; |
| 1510 | 1517 | } |
| 1511 | 1518 | } |
| @@ -98,7 +98,7 @@ int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 97 unmodified lines | ||
| 98 | 98 | |
| 99 | 99 | if (r) { |
| 100 | 100 | DRM_ERROR("failed to do bo_map on static CSA, err=%d\n", r); |
| 101 | − amdgpu_vm_bo_rmv(adev, *bo_va); | |
| 101 | + amdgpu_vm_bo_del(adev, *bo_va); | |
| 102 | 102 | ttm_eu_backoff_reservation(&ticket, &list); |
| 103 | 103 | return r; |
| 104 | 104 | } |
| @@ -230,13 +230,13 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, · 229 unmodified lines | ||
| 230 | 230 | |
| 231 | 231 | kref_init(&ctx->refcount); |
| 232 | 232 | spin_lock_init(&ctx->ring_lock); |
| 233 | − mutex_init(&ctx->lock); | |
| 234 | 233 | |
| 235 | 234 | ctx->reset_counter = atomic_read(&adev->gpu_reset_counter); |
| 236 | 235 | ctx->reset_counter_query = ctx->reset_counter; |
| 237 | 236 | ctx->vram_lost_counter = atomic_read(&adev->vram_lost_counter); |
| 238 | 237 | ctx->init_priority = priority; |
| 239 | 238 | ctx->override_priority = AMDGPU_CTX_PRIORITY_UNSET; |
| 239 | + ctx->stable_pstate = AMDGPU_CTX_STABLE_PSTATE_NONE; | |
| 240 | 240 | |
| 241 | 241 | return 0; |
| 242 | 242 | } |
| @@ -255,6 +255,86 @@ static void amdgpu_ctx_fini_entity(struct amdgpu_ctx_entity *entity) · 12 unmodified lines | ||
| 255 | 255 | kfree(entity); |
| 256 | 256 | } |
| 257 | 257 | |
| 258 | +static int amdgpu_ctx_get_stable_pstate(struct amdgpu_ctx *ctx, | |
| 259 | + u32 *stable_pstate) | |
| 260 | +{ | |
| 261 | + struct amdgpu_device *adev = ctx->adev; | |
| 262 | + enum amd_dpm_forced_level current_level; | |
| 263 | + | |
| 264 | + if (!ctx) | |
| 265 | + return -EINVAL; | |
| 266 | + | |
| 267 | + current_level = amdgpu_dpm_get_performance_level(adev); | |
| 268 | + | |
| 269 | + switch (current_level) { | |
| 270 | + case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD: | |
| 271 | + *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_STANDARD; | |
| 272 | + break; | |
| 273 | + case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK: | |
| 274 | + *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_MIN_SCLK; | |
| 275 | + break; | |
| 276 | + case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK: | |
| 277 | + *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_MIN_MCLK; | |
| 278 | + break; | |
| 279 | + case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK: | |
| 280 | + *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_PEAK; | |
| 281 | + break; | |
| 282 | + default: | |
| 283 | + *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_NONE; | |
| 284 | + break; | |
| 285 | + } | |
| 286 | + return 0; | |
| 287 | +} | |
| 288 | + | |
| 289 | +static int amdgpu_ctx_set_stable_pstate(struct amdgpu_ctx *ctx, | |
| 290 | + u32 stable_pstate) | |
| 291 | +{ | |
| 292 | + struct amdgpu_device *adev = ctx->adev; | |
| 293 | + enum amd_dpm_forced_level level; | |
| 294 | + int r; | |
| 295 | + | |
| 296 | + if (!ctx) | |
| 297 | + return -EINVAL; | |
| 298 | + | |
| 299 | + mutex_lock(&adev->pm.stable_pstate_ctx_lock); | |
| 300 | + if (adev->pm.stable_pstate_ctx && adev->pm.stable_pstate_ctx != ctx) { | |
| 301 | + r = -EBUSY; | |
| 302 | + goto done; | |
| 303 | + } | |
| 304 | + | |
| 305 | + switch (stable_pstate) { | |
| 306 | + case AMDGPU_CTX_STABLE_PSTATE_NONE: | |
| 307 | + level = AMD_DPM_FORCED_LEVEL_AUTO; | |
| 308 | + break; | |
| 309 | + case AMDGPU_CTX_STABLE_PSTATE_STANDARD: | |
| 310 | + level = AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD; | |
| 311 | + break; | |
| 312 | + case AMDGPU_CTX_STABLE_PSTATE_MIN_SCLK: | |
| 313 | + level = AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK; | |
| 314 | + break; | |
| 315 | + case AMDGPU_CTX_STABLE_PSTATE_MIN_MCLK: | |
| 316 | + level = AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK; | |
| 317 | + break; | |
| 318 | + case AMDGPU_CTX_STABLE_PSTATE_PEAK: | |
| 319 | + level = AMD_DPM_FORCED_LEVEL_PROFILE_PEAK; | |
| 320 | + break; | |
| 321 | + default: | |
| 322 | + r = -EINVAL; | |
| 323 | + goto done; | |
| 324 | + } | |
| 325 | + | |
| 326 | + r = amdgpu_dpm_force_performance_level(adev, level); | |
| 327 | + | |
| 328 | + if (level == AMD_DPM_FORCED_LEVEL_AUTO) | |
| 329 | + adev->pm.stable_pstate_ctx = NULL; | |
| 330 | + else | |
| 331 | + adev->pm.stable_pstate_ctx = ctx; | |
| 332 | +done: | |
| 333 | + mutex_unlock(&adev->pm.stable_pstate_ctx_lock); | |
| 334 | + | |
| 335 | + return r; | |
| 336 | +} | |
| 337 | + | |
| 258 | 338 | static void amdgpu_ctx_fini(struct kref *ref) |
| 259 | 339 | { |
| 260 | 340 | struct amdgpu_ctx *ctx = container_of(ref, struct amdgpu_ctx, refcount); |
| @@ -270,8 +350,7 @@ static void amdgpu_ctx_fini(struct kref *ref) · 9 unmodified lines | ||
| 270 | 350 | ctx->entities[i][j] = NULL; |
| 271 | 351 | } |
| 272 | 352 | } |
| 273 | − | |
| 274 | − mutex_destroy(&ctx->lock); | |
| 353 | + amdgpu_ctx_set_stable_pstate(ctx, AMDGPU_CTX_STABLE_PSTATE_NONE); | |
| 275 | 354 | kfree(ctx); |
| 276 | 355 | } |
| 277 | 356 | |
| @@ -467,11 +546,41 @@ static int amdgpu_ctx_query2(struct amdgpu_device *adev, · 189 unmodified lines | ||
| 467 | 546 | return 0; |
| 468 | 547 | } |
| 469 | 548 | |
| 549 | + | |
| 550 | + | |
| 551 | +static int amdgpu_ctx_stable_pstate(struct amdgpu_device *adev, | |
| 552 | + struct amdgpu_fpriv *fpriv, uint32_t id, | |
| 553 | + bool set, u32 *stable_pstate) | |
| 554 | +{ | |
| 555 | + struct amdgpu_ctx *ctx; | |
| 556 | + struct amdgpu_ctx_mgr *mgr; | |
| 557 | + int r; | |
| 558 | + | |
| 559 | + if (!fpriv) | |
| 560 | + return -EINVAL; | |
| 561 | + | |
| 562 | + mgr = &fpriv->ctx_mgr; | |
| 563 | + mutex_lock(&mgr->lock); | |
| 564 | + ctx = idr_find(&mgr->ctx_handles, id); | |
| 565 | + if (!ctx) { | |
| 566 | + mutex_unlock(&mgr->lock); | |
| 567 | + return -EINVAL; | |
| 568 | + } | |
| 569 | + | |
| 570 | + if (set) | |
| 571 | + r = amdgpu_ctx_set_stable_pstate(ctx, *stable_pstate); | |
| 572 | + else | |
| 573 | + r = amdgpu_ctx_get_stable_pstate(ctx, stable_pstate); | |
| 574 | + | |
| 575 | + mutex_unlock(&mgr->lock); | |
| 576 | + return r; | |
| 577 | +} | |
| 578 | + | |
| 470 | 579 | int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, |
| 471 | 580 | struct drm_file *filp) |
| 472 | 581 | { |
| 473 | 582 | int r; |
| 474 | − uint32_t id; | |
| 583 | + uint32_t id, stable_pstate; | |
| 475 | 584 | int32_t priority; |
| 476 | 585 | |
| 477 | 586 | union drm_amdgpu_ctx *args = data; |
| @@ -500,6 +609,21 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, · 22 unmodified lines | ||
| 500 | 609 | case AMDGPU_CTX_OP_QUERY_STATE2: |
| 501 | 610 | r = amdgpu_ctx_query2(adev, fpriv, id, &args->out); |
| 502 | 611 | break; |
| 612 | + case AMDGPU_CTX_OP_GET_STABLE_PSTATE: | |
| 613 | + if (args->in.flags) | |
| 614 | + return -EINVAL; | |
| 615 | + r = amdgpu_ctx_stable_pstate(adev, fpriv, id, false, &stable_pstate); | |
| 616 | + if (!r) | |
| 617 | + args->out.pstate.flags = stable_pstate; | |
| 618 | + break; | |
| 619 | + case AMDGPU_CTX_OP_SET_STABLE_PSTATE: | |
| 620 | + if (args->in.flags & ~AMDGPU_CTX_STABLE_PSTATE_FLAGS_MASK) | |
| 621 | + return -EINVAL; | |
| 622 | + stable_pstate = args->in.flags & AMDGPU_CTX_STABLE_PSTATE_FLAGS_MASK; | |
| 623 | + if (stable_pstate > AMDGPU_CTX_STABLE_PSTATE_PEAK) | |
| 624 | + return -EINVAL; | |
| 625 | + r = amdgpu_ctx_stable_pstate(adev, fpriv, id, true, &stable_pstate); | |
| 626 | + break; | |
| 503 | 627 | default: |
| 504 | 628 | return -EINVAL; |
| 505 | 629 | } |
| @@ -49,10 +49,10 @@ struct amdgpu_ctx { · 48 unmodified lines | ||
| 49 | 49 | bool preamble_presented; |
| 50 | 50 | int32_t init_priority; |
| 51 | 51 | int32_t override_priority; |
| 52 | − struct mutex lock; | |
| 53 | 52 | atomic_t guilty; |
| 54 | 53 | unsigned long ras_counter_ce; |
| 55 | 54 | unsigned long ras_counter_ue; |
| 55 | + uint32_t stable_pstate; | |
| 56 | 56 | }; |
| 57 | 57 | |
| 58 | 58 | struct amdgpu_ctx_mgr { |
| @@ -37,6 +37,8 @@ · 36 unmodified lines | ||
| 37 | 37 | #include "amdgpu_fw_attestation.h" |
| 38 | 38 | #include "amdgpu_umr.h" |
| 39 | 39 | |
| 40 | +#include "amdgpu_reset.h" | |
| 41 | + | |
| 40 | 42 | #if defined(CONFIG_DEBUG_FS) |
| 41 | 43 | |
| 42 | 44 | /** |
| @@ -728,7 +730,7 @@ static ssize_t amdgpu_debugfs_gca_config_read(struct file *f, char __user *buf, · 685 unmodified lines | ||
| 728 | 730 | return -ENOMEM; |
| 729 | 731 | |
| 730 | 732 | /* version, increment each time something is added */ |
| 731 | − config[no_regs++] = 3; | |
| 733 | + config[no_regs++] = 4; | |
| 732 | 734 | config[no_regs++] = adev->gfx.config.max_shader_engines; |
| 733 | 735 | config[no_regs++] = adev->gfx.config.max_tile_pipes; |
| 734 | 736 | config[no_regs++] = adev->gfx.config.max_cu_per_sh; |
| @@ -768,6 +770,9 @@ static ssize_t amdgpu_debugfs_gca_config_read(struct file *f, char __user *buf, · 33 unmodified lines | ||
| 768 | 770 | config[no_regs++] = adev->pdev->subsystem_device; |
| 769 | 771 | config[no_regs++] = adev->pdev->subsystem_vendor; |
| 770 | 772 | |
| 773 | + /* rev==4 APU flag */ | |
| 774 | + config[no_regs++] = adev->flags & AMD_IS_APU ? 1 : 0; | |
| 775 | + | |
| 771 | 776 | while (size && (*pos < no_regs * 4)) { |
| 772 | 777 | uint32_t value; |
| 773 | 778 | |
| @@ -1120,8 +1125,10 @@ static ssize_t amdgpu_debugfs_gfxoff_read(struct file *f, char __user *buf, · 346 unmodified lines | ||
| 1120 | 1125 | return -EINVAL; |
| 1121 | 1126 | |
| 1122 | 1127 | r = pm_runtime_get_sync(adev_to_drm(adev)->dev); |
| 1123 | − if (r < 0) | |
| 1128 | + if (r < 0) { | |
| 1129 | + pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); | |
| 1124 | 1130 | return r; |
| 1131 | + } | |
| 1125 | 1132 | |
| 1126 | 1133 | while (size) { |
| 1127 | 1134 | uint32_t value; |
| @@ -1279,7 +1286,7 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · 151 unmodified lines | ||
| 1279 | 1286 | } |
| 1280 | 1287 | |
| 1281 | 1288 | /* Avoid accidently unparking the sched thread during GPU reset */ |
| 1282 | − r = down_write_killable(&adev->reset_sem); | |
| 1289 | + r = down_write_killable(&adev->reset_domain->sem); | |
| 1283 | 1290 | if (r) |
| 1284 | 1291 | return r; |
| 1285 | 1292 | |
| @@ -1308,7 +1315,7 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · 22 unmodified lines | ||
| 1308 | 1315 | kthread_unpark(ring->sched.thread); |
| 1309 | 1316 | } |
| 1310 | 1317 | |
| 1311 | − up_write(&adev->reset_sem); | |
| 1318 | + up_write(&adev->reset_domain->sem); | |
| 1312 | 1319 | |
| 1313 | 1320 | pm_runtime_mark_last_busy(dev->dev); |
| 1314 | 1321 | pm_runtime_put_autosuspend(dev->dev); |
| @@ -1357,6 +1364,25 @@ static int amdgpu_debugfs_evict_gtt(void *data, u64 *val) · 42 unmodified lines | ||
| 1357 | 1364 | return 0; |
| 1358 | 1365 | } |
| 1359 | 1366 | |
| 1367 | +static int amdgpu_debugfs_benchmark(void *data, u64 val) | |
| 1368 | +{ | |
| 1369 | + struct amdgpu_device *adev = (struct amdgpu_device *)data; | |
| 1370 | + struct drm_device *dev = adev_to_drm(adev); | |
| 1371 | + int r; | |
| 1372 | + | |
| 1373 | + r = pm_runtime_get_sync(dev->dev); | |
| 1374 | + if (r < 0) { | |
| 1375 | + pm_runtime_put_autosuspend(dev->dev); | |
| 1376 | + return r; | |
| 1377 | + } | |
| 1378 | + | |
| 1379 | + r = amdgpu_benchmark(adev, val); | |
| 1380 | + | |
| 1381 | + pm_runtime_mark_last_busy(dev->dev); | |
| 1382 | + pm_runtime_put_autosuspend(dev->dev); | |
| 1383 | + | |
| 1384 | + return r; | |
| 1385 | +} | |
| 1360 | 1386 | |
| 1361 | 1387 | static int amdgpu_debugfs_vm_info_show(struct seq_file *m, void *unused) |
| 1362 | 1388 | { |
| @@ -1393,6 +1419,8 @@ DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_evict_vram_fops, amdgpu_debugfs_evict_vram, · 30 unmodified lines | ||
| 1393 | 1419 | NULL, "%lld\n"); |
| 1394 | 1420 | DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_evict_gtt_fops, amdgpu_debugfs_evict_gtt, |
| 1395 | 1421 | NULL, "%lld\n"); |
| 1422 | +DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_benchmark_fops, NULL, amdgpu_debugfs_benchmark, | |
| 1423 | + "%lld\n"); | |
| 1396 | 1424 | |
| 1397 | 1425 | static void amdgpu_ib_preempt_fences_swap(struct amdgpu_ring *ring, |
| 1398 | 1426 | struct dma_fence **fences) |
| @@ -1517,7 +1545,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) · 118 unmodified lines | ||
| 1517 | 1545 | return -ENOMEM; |
| 1518 | 1546 | |
| 1519 | 1547 | /* Avoid accidently unparking the sched thread during GPU reset */ |
| 1520 | − r = down_read_killable(&adev->reset_sem); | |
| 1548 | + r = down_read_killable(&adev->reset_domain->sem); | |
| 1521 | 1549 | if (r) |
| 1522 | 1550 | goto pro_end; |
| 1523 | 1551 | |
| @@ -1560,7 +1588,7 @@ failure: · 36 unmodified lines | ||
| 1560 | 1588 | /* restart the scheduler */ |
| 1561 | 1589 | kthread_unpark(ring->sched.thread); |
| 1562 | 1590 | |
| 1563 | − up_read(&adev->reset_sem); | |
| 1591 | + up_read(&adev->reset_domain->sem); | |
| 1564 | 1592 | |
| 1565 | 1593 | ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, resched); |
| 1566 | 1594 | |
| @@ -1585,22 +1613,25 @@ static int amdgpu_debugfs_sclk_set(void *data, u64 val) · 18 unmodified lines | ||
| 1585 | 1613 | return ret; |
| 1586 | 1614 | } |
| 1587 | 1615 | |
| 1588 | − if (is_support_sw_smu(adev)) { | |
| 1589 | − ret = smu_get_dpm_freq_range(&adev->smu, SMU_SCLK, &min_freq, &max_freq); | |
| 1590 | − if (ret || val > max_freq || val < min_freq) | |
| 1591 | − return -EINVAL; | |
| 1592 | − ret = smu_set_soft_freq_range(&adev->smu, SMU_SCLK, (uint32_t)val, (uint32_t)val); | |
| 1593 | − } else { | |
| 1594 | − return 0; | |
| 1616 | + ret = amdgpu_dpm_get_dpm_freq_range(adev, PP_SCLK, &min_freq, &max_freq); | |
| 1617 | + if (ret == -EOPNOTSUPP) { | |
| 1618 | + ret = 0; | |
| 1619 | + goto out; | |
| 1595 | 1620 | } |
| 1621 | + if (ret || val > max_freq || val < min_freq) { | |
| 1622 | + ret = -EINVAL; | |
| 1623 | + goto out; | |
| 1624 | + } | |
| 1625 | + | |
| 1626 | + ret = amdgpu_dpm_set_soft_freq_range(adev, PP_SCLK, (uint32_t)val, (uint32_t)val); | |
| 1627 | + if (ret) | |
| 1628 | + ret = -EINVAL; | |
| 1596 | 1629 | |
| 1630 | +out: | |
| 1597 | 1631 | pm_runtime_mark_last_busy(adev_to_drm(adev)->dev); |
| 1598 | 1632 | pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); |
| 1599 | 1633 | |
| 1600 | − if (ret) | |
| 1601 | − return -EINVAL; | |
| 1602 | − | |
| 1603 | − return 0; | |
| 1634 | + return ret; | |
| 1604 | 1635 | } |
| 1605 | 1636 | |
| 1606 | 1637 | DEFINE_DEBUGFS_ATTRIBUTE(fops_ib_preempt, NULL, |
| @@ -1609,6 +1640,86 @@ DEFINE_DEBUGFS_ATTRIBUTE(fops_ib_preempt, NULL, · 2 unmodified lines | ||
| 1609 | 1640 | DEFINE_DEBUGFS_ATTRIBUTE(fops_sclk_set, NULL, |
| 1610 | 1641 | amdgpu_debugfs_sclk_set, "%llu\n"); |
| 1611 | 1642 | |
| 1643 | +static ssize_t amdgpu_reset_dump_register_list_read(struct file *f, | |
| 1644 | + char __user *buf, size_t size, loff_t *pos) | |
| 1645 | +{ | |
| 1646 | + struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(f)->i_private; | |
| 1647 | + char reg_offset[12]; | |
| 1648 | + int i, ret, len = 0; | |
| 1649 | + | |
| 1650 | + if (*pos) | |
| 1651 | + return 0; | |
| 1652 | + | |
| 1653 | + memset(reg_offset, 0, 12); | |
| 1654 | + ret = down_read_killable(&adev->reset_domain->sem); | |
| 1655 | + if (ret) | |
| 1656 | + return ret; | |
| 1657 | + | |
| 1658 | + for (i = 0; i < adev->num_regs; i++) { | |
| 1659 | + sprintf(reg_offset, "0x%x\n", adev->reset_dump_reg_list[i]); | |
| 1660 | + up_read(&adev->reset_domain->sem); | |
| 1661 | + if (copy_to_user(buf + len, reg_offset, strlen(reg_offset))) | |
| 1662 | + return -EFAULT; | |
| 1663 | + | |
| 1664 | + len += strlen(reg_offset); | |
| 1665 | + ret = down_read_killable(&adev->reset_domain->sem); | |
| 1666 | + if (ret) | |
| 1667 | + return ret; | |
| 1668 | + } | |
| 1669 | + | |
| 1670 | + up_read(&adev->reset_domain->sem); | |
| 1671 | + *pos += len; | |
| 1672 | + | |
| 1673 | + return len; | |
| 1674 | +} | |
| 1675 | + | |
| 1676 | +static ssize_t amdgpu_reset_dump_register_list_write(struct file *f, | |
| 1677 | + const char __user *buf, size_t size, loff_t *pos) | |
| 1678 | +{ | |
| 1679 | + struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(f)->i_private; | |
| 1680 | + char reg_offset[11]; | |
| 1681 | + uint32_t *tmp; | |
| 1682 | + int ret, i = 0, len = 0; | |
| 1683 | + | |
| 1684 | + do { | |
| 1685 | + memset(reg_offset, 0, 11); | |
| 1686 | + if (copy_from_user(reg_offset, buf + len, | |
| 1687 | + min(10, ((int)size-len)))) { | |
| 1688 | + ret = -EFAULT; | |
| 1689 | + goto error_free; | |
| 1690 | + } | |
| 1691 | + | |
| 1692 | + tmp = krealloc_array(tmp, i + 1, sizeof(uint32_t), GFP_KERNEL); | |
| 1693 | + if (sscanf(reg_offset, "%X %n", &tmp[i], &ret) != 1) { | |
| 1694 | + ret = -EINVAL; | |
| 1695 | + goto error_free; | |
| 1696 | + } | |
| 1697 | + | |
| 1698 | + len += ret; | |
| 1699 | + i++; | |
| 1700 | + } while (len < size); | |
| 1701 | + | |
| 1702 | + ret = down_write_killable(&adev->reset_domain->sem); | |
| 1703 | + if (ret) | |
| 1704 | + goto error_free; | |
| 1705 | + | |
| 1706 | + swap(adev->reset_dump_reg_list, tmp); | |
| 1707 | + adev->num_regs = i; | |
| 1708 | + up_write(&adev->reset_domain->sem); | |
| 1709 | + ret = size; | |
| 1710 | + | |
| 1711 | +error_free: | |
| 1712 | + kfree(tmp); | |
| 1713 | + return ret; | |
| 1714 | +} | |
| 1715 | + | |
| 1716 | +static const struct file_operations amdgpu_reset_dump_register_list = { | |
| 1717 | + .owner = THIS_MODULE, | |
| 1718 | + .read = amdgpu_reset_dump_register_list_read, | |
| 1719 | + .write = amdgpu_reset_dump_register_list_write, | |
| 1720 | + .llseek = default_llseek | |
| 1721 | +}; | |
| 1722 | + | |
| 1612 | 1723 | int amdgpu_debugfs_init(struct amdgpu_device *adev) |
| 1613 | 1724 | { |
| 1614 | 1725 | struct dentry *root = adev_to_drm(adev)->primary->debugfs_root; |
| @@ -1675,6 +1786,10 @@ int amdgpu_debugfs_init(struct amdgpu_device *adev) · 60 unmodified lines | ||
| 1675 | 1786 | &amdgpu_debugfs_test_ib_fops); |
| 1676 | 1787 | debugfs_create_file("amdgpu_vm_info", 0444, root, adev, |
| 1677 | 1788 | &amdgpu_debugfs_vm_info_fops); |
| 1789 | + debugfs_create_file("amdgpu_benchmark", 0200, root, adev, | |
| 1790 | + &amdgpu_benchmark_fops); | |
| 1791 | + debugfs_create_file("amdgpu_reset_dump_register_list", 0644, root, adev, | |
| 1792 | + &amdgpu_reset_dump_register_list); | |
| 1678 | 1793 | |
| 1679 | 1794 | adev->debugfs_vbios_blob.data = adev->bios; |
| 1680 | 1795 | adev->debugfs_vbios_blob.size = adev->bios_size; |
| @@ -31,6 +31,7 @@ · 30 unmodified lines | ||
| 31 | 31 | #include <linux/console.h> |
| 32 | 32 | #include <linux/slab.h> |
| 33 | 33 | #include <linux/iommu.h> |
| 34 | +#include <linux/pci.h> | |
| 34 | 35 | |
| 35 | 36 | #include <drm/drm_atomic_helper.h> |
| 36 | 37 | #include <drm/drm_probe_helper.h> |
| @@ -55,7 +56,6 @@ · 18 unmodified lines | ||
| 55 | 56 | #include "soc15.h" |
| 56 | 57 | #include "nv.h" |
| 57 | 58 | #include "bif/bif_4_1_d.h" |
| 58 | −#include <linux/pci.h> | |
| 59 | 59 | #include <linux/firmware.h> |
| 60 | 60 | #include "amdgpu_vf_error.h" |
| 61 | 61 | |
| @@ -88,6 +88,8 @@ MODULE_FIRMWARE("amdgpu/vangogh_gpu_info.bin"); · 26 unmodified lines | ||
| 88 | 88 | MODULE_FIRMWARE("amdgpu/yellow_carp_gpu_info.bin"); |
| 89 | 89 | |
| 90 | 90 | #define AMDGPU_RESUME_MS 2000 |
| 91 | +#define AMDGPU_MAX_RETRY_LIMIT 2 | |
| 92 | +#define AMDGPU_RETRY_SRIOV_RESET(r) ((r) == -EBUSY || (r) == -ETIMEDOUT || (r) == -EINVAL) | |
| 91 | 93 | |
| 92 | 94 | const char *amdgpu_asic_name[] = { |
| 93 | 95 | "TAHITI", |
| @@ -424,10 +426,10 @@ bool amdgpu_device_skip_hw_access(struct amdgpu_device *adev) · 330 unmodified lines | ||
| 424 | 426 | * the lock. |
| 425 | 427 | */ |
| 426 | 428 | if (in_task()) { |
| 427 | − if (down_read_trylock(&adev->reset_sem)) | |
| 428 | − up_read(&adev->reset_sem); | |
| 429 | + if (down_read_trylock(&adev->reset_domain->sem)) | |
| 430 | + up_read(&adev->reset_domain->sem); | |
| 429 | 431 | else |
| 430 | − lockdep_assert_held(&adev->reset_sem); | |
| 432 | + lockdep_assert_held(&adev->reset_domain->sem); | |
| 431 | 433 | } |
| 432 | 434 | #endif |
| 433 | 435 | return false; |
| @@ -453,9 +455,9 @@ uint32_t amdgpu_device_rreg(struct amdgpu_device *adev, · 19 unmodified lines | ||
| 453 | 455 | if ((reg * 4) < adev->rmmio_size) { |
| 454 | 456 | if (!(acc_flags & AMDGPU_REGS_NO_KIQ) && |
| 455 | 457 | amdgpu_sriov_runtime(adev) && |
| 456 | − down_read_trylock(&adev->reset_sem)) { | |
| 458 | + down_read_trylock(&adev->reset_domain->sem)) { | |
| 457 | 459 | ret = amdgpu_kiq_rreg(adev, reg); |
| 458 | − up_read(&adev->reset_sem); | |
| 460 | + up_read(&adev->reset_domain->sem); | |
| 459 | 461 | } else { |
| 460 | 462 | ret = readl(((void __iomem *)adev->rmmio) + (reg * 4)); |
| 461 | 463 | } |
| @@ -538,9 +540,9 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, · 76 unmodified lines | ||
| 538 | 540 | if ((reg * 4) < adev->rmmio_size) { |
| 539 | 541 | if (!(acc_flags & AMDGPU_REGS_NO_KIQ) && |
| 540 | 542 | amdgpu_sriov_runtime(adev) && |
| 541 | − down_read_trylock(&adev->reset_sem)) { | |
| 543 | + down_read_trylock(&adev->reset_domain->sem)) { | |
| 542 | 544 | amdgpu_kiq_wreg(adev, reg, v); |
| 543 | − up_read(&adev->reset_sem); | |
| 545 | + up_read(&adev->reset_domain->sem); | |
| 544 | 546 | } else { |
| 545 | 547 | writel(v, ((void __iomem *)adev->rmmio) + (reg * 4)); |
| 546 | 548 | } |
| @@ -554,7 +556,11 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, · 7 unmodified lines | ||
| 554 | 556 | /** |
| 555 | 557 | * amdgpu_mm_wreg_mmio_rlc - write register either with direct/indirect mmio or with RLC path if in range |
| 556 | 558 | * |
| 557 | − * this function is invoked only the debugfs register access | |
| 559 | + * @adev: amdgpu_device pointer | |
| 560 | + * @reg: mmio/rlc register | |
| 561 | + * @v: value to write | |
| 562 | + * | |
| 563 | + * this function is invoked only for the debugfs register access | |
| 558 | 564 | */ |
| 559 | 565 | void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, |
| 560 | 566 | uint32_t reg, uint32_t v) |
| @@ -566,7 +572,7 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, · 5 unmodified lines | ||
| 566 | 572 | adev->gfx.rlc.funcs && |
| 567 | 573 | adev->gfx.rlc.funcs->is_rlcg_access_range) { |
| 568 | 574 | if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg)) |
| 569 | − return adev->gfx.rlc.funcs->sriov_wreg(adev, reg, v, 0, 0); | |
| 575 | + return amdgpu_sriov_wreg(adev, reg, v, 0, 0); | |
| 570 | 576 | } else if ((reg * 4) >= adev->rmmio_size) { |
| 571 | 577 | adev->pcie_wreg(adev, reg * 4, v); |
| 572 | 578 | } else { |
| @@ -1312,6 +1318,31 @@ bool amdgpu_device_need_post(struct amdgpu_device *adev) · 739 unmodified lines | ||
| 1312 | 1318 | return true; |
| 1313 | 1319 | } |
| 1314 | 1320 | |
| 1321 | +/** | |
| 1322 | + * amdgpu_device_should_use_aspm - check if the device should program ASPM | |
| 1323 | + * | |
| 1324 | + * @adev: amdgpu_device pointer | |
| 1325 | + * | |
| 1326 | + * Confirm whether the module parameter and pcie bridge agree that ASPM should | |
| 1327 | + * be set for this device. | |
| 1328 | + * | |
| 1329 | + * Returns true if it should be used or false if not. | |
| 1330 | + */ | |
| 1331 | +bool amdgpu_device_should_use_aspm(struct amdgpu_device *adev) | |
| 1332 | +{ | |
| 1333 | + switch (amdgpu_aspm) { | |
| 1334 | + case -1: | |
| 1335 | + break; | |
| 1336 | + case 0: | |
| 1337 | + return false; | |
| 1338 | + case 1: | |
| 1339 | + return true; | |
| 1340 | + default: | |
| 1341 | + return false; | |
| 1342 | + } | |
| 1343 | + return pcie_aspm_enabled(adev->pdev); | |
| 1344 | +} | |
| 1345 | + | |
| 1315 | 1346 | /* if we get transitioned to only one device, take VGA back */ |
| 1316 | 1347 | /** |
| 1317 | 1348 | * amdgpu_device_vga_set_decode - enable/disable vga decode |
| @@ -1446,7 +1477,8 @@ static int amdgpu_device_init_apu_flags(struct amdgpu_device *adev) · 128 unmodified lines | ||
| 1446 | 1477 | case CHIP_YELLOW_CARP: |
| 1447 | 1478 | break; |
| 1448 | 1479 | case CHIP_CYAN_SKILLFISH: |
| 1449 | − if (adev->pdev->device == 0x13FE) | |
| 1480 | + if ((adev->pdev->device == 0x13FE) || | |
| 1481 | + (adev->pdev->device == 0x143F)) | |
| 1450 | 1482 | adev->apu_flags |= AMD_APU_IS_CYAN_SKILLFISH2; |
| 1451 | 1483 | break; |
| 1452 | 1484 | default: |
| @@ -1507,6 +1539,11 @@ static int amdgpu_device_check_arguments(struct amdgpu_device *adev) · 54 unmodified lines | ||
| 1507 | 1539 | amdgpu_sched_hw_submission = roundup_pow_of_two(amdgpu_sched_hw_submission); |
| 1508 | 1540 | } |
| 1509 | 1541 | |
| 1542 | + if (amdgpu_reset_method < -1 || amdgpu_reset_method > 4) { | |
| 1543 | + dev_warn(adev->dev, "invalid option for reset method, reverting to default\n"); | |
| 1544 | + amdgpu_reset_method = -1; | |
| 1545 | + } | |
| 1546 | + | |
| 1510 | 1547 | amdgpu_device_check_smu_prv_buffer_size(adev); |
| 1511 | 1548 | |
| 1512 | 1549 | amdgpu_device_check_vm_size(adev); |
| @@ -2073,6 +2110,8 @@ out: · 560 unmodified lines | ||
| 2073 | 2110 | */ |
| 2074 | 2111 | static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) |
| 2075 | 2112 | { |
| 2113 | + struct drm_device *dev = adev_to_drm(adev); | |
| 2114 | + struct pci_dev *parent; | |
| 2076 | 2115 | int i, r; |
| 2077 | 2116 | |
| 2078 | 2117 | amdgpu_device_enable_virtual_display(adev); |
| @@ -2137,6 +2176,16 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) · 58 unmodified lines | ||
| 2137 | 2176 | break; |
| 2138 | 2177 | } |
| 2139 | 2178 | |
| 2179 | + if (amdgpu_has_atpx() && | |
| 2180 | + (amdgpu_is_atpx_hybrid() || | |
| 2181 | + amdgpu_has_atpx_dgpu_power_cntl()) && | |
| 2182 | + ((adev->flags & AMD_IS_APU) == 0) && | |
| 2183 | + !pci_is_thunderbolt_attached(to_pci_dev(dev->dev))) | |
| 2184 | + adev->flags |= AMD_IS_PX; | |
| 2185 | + | |
| 2186 | + parent = pci_upstream_bridge(adev->pdev); | |
| 2187 | + adev->has_pr3 = parent ? pci_pr3_present(parent) : false; | |
| 2188 | + | |
| 2140 | 2189 | amdgpu_amdkfd_device_probe(adev); |
| 2141 | 2190 | |
| 2142 | 2191 | adev->pm.pp_feature = amdgpu_pp_feature_mask; |
| @@ -2287,6 +2336,49 @@ static int amdgpu_device_fw_loading(struct amdgpu_device *adev) · 144 unmodified lines | ||
| 2287 | 2336 | return r; |
| 2288 | 2337 | } |
| 2289 | 2338 | |
| 2339 | +static int amdgpu_device_init_schedulers(struct amdgpu_device *adev) | |
| 2340 | +{ | |
| 2341 | + long timeout; | |
| 2342 | + int r, i; | |
| 2343 | + | |
| 2344 | + for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | |
| 2345 | + struct amdgpu_ring *ring = adev->rings[i]; | |
| 2346 | + | |
| 2347 | + /* No need to setup the GPU scheduler for rings that don't need it */ | |
| 2348 | + if (!ring || ring->no_scheduler) | |
| 2349 | + continue; | |
| 2350 | + | |
| 2351 | + switch (ring->funcs->type) { | |
| 2352 | + case AMDGPU_RING_TYPE_GFX: | |
| 2353 | + timeout = adev->gfx_timeout; | |
| 2354 | + break; | |
| 2355 | + case AMDGPU_RING_TYPE_COMPUTE: | |
| 2356 | + timeout = adev->compute_timeout; | |
| 2357 | + break; | |
| 2358 | + case AMDGPU_RING_TYPE_SDMA: | |
| 2359 | + timeout = adev->sdma_timeout; | |
| 2360 | + break; | |
| 2361 | + default: | |
| 2362 | + timeout = adev->video_timeout; | |
| 2363 | + break; | |
| 2364 | + } | |
| 2365 | + | |
| 2366 | + r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, | |
| 2367 | + ring->num_hw_submission, amdgpu_job_hang_limit, | |
| 2368 | + timeout, adev->reset_domain->wq, | |
| 2369 | + ring->sched_score, ring->name, | |
| 2370 | + adev->dev); | |
| 2371 | + if (r) { | |
| 2372 | + DRM_ERROR("Failed to create scheduler on ring %s.\n", | |
| 2373 | + ring->name); | |
| 2374 | + return r; | |
| 2375 | + } | |
| 2376 | + } | |
| 2377 | + | |
| 2378 | + return 0; | |
| 2379 | +} | |
| 2380 | + | |
| 2381 | + | |
| 2290 | 2382 | /** |
| 2291 | 2383 | * amdgpu_device_ip_init - run init for hardware IPs |
| 2292 | 2384 | * |
| @@ -2398,8 +2490,28 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev) · 105 unmodified lines | ||
| 2398 | 2490 | if (r) |
| 2399 | 2491 | goto init_failed; |
| 2400 | 2492 | |
| 2401 | − if (adev->gmc.xgmi.num_physical_nodes > 1) | |
| 2402 | − amdgpu_xgmi_add_device(adev); | |
| 2493 | + /** | |
| 2494 | + * In case of XGMI grab extra reference for reset domain for this device | |
| 2495 | + */ | |
| 2496 | + if (adev->gmc.xgmi.num_physical_nodes > 1) { | |
| 2497 | + if (amdgpu_xgmi_add_device(adev) == 0) { | |
| 2498 | + struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev); | |
| 2499 | + | |
| 2500 | + if (!hive->reset_domain || | |
| 2501 | + !amdgpu_reset_get_reset_domain(hive->reset_domain)) { | |
| 2502 | + r = -ENOENT; | |
| 2503 | + goto init_failed; | |
| 2504 | + } | |
| 2505 | + | |
| 2506 | + /* Drop the early temporary reset domain we created for device */ | |
| 2507 | + amdgpu_reset_put_reset_domain(adev->reset_domain); | |
| 2508 | + adev->reset_domain = hive->reset_domain; | |
| 2509 | + } | |
| 2510 | + } | |
| 2511 | + | |
| 2512 | + r = amdgpu_device_init_schedulers(adev); | |
| 2513 | + if (r) | |
| 2514 | + goto init_failed; | |
| 2403 | 2515 | |
| 2404 | 2516 | /* Don't init kfd if whole hive need to be reset during init */ |
| 2405 | 2517 | if (!adev->gmc.xgmi.pending_reset) |
| @@ -2610,6 +2722,12 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) · 204 unmodified lines | ||
| 2610 | 2722 | adev->ip_blocks[i].status.late_initialized = true; |
| 2611 | 2723 | } |
| 2612 | 2724 | |
| 2725 | + r = amdgpu_ras_late_init(adev); | |
| 2726 | + if (r) { | |
| 2727 | + DRM_ERROR("amdgpu_ras_late_init failed %d", r); | |
| 2728 | + return r; | |
| 2729 | + } | |
| 2730 | + | |
| 2613 | 2731 | amdgpu_ras_set_error_query_ready(adev, true); |
| 2614 | 2732 | |
| 2615 | 2733 | amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE); |
| @@ -2624,7 +2742,7 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) · 8 unmodified lines | ||
| 2624 | 2742 | /* For passthrough configuration on arcturus and aldebaran, enable special handling SBR */ |
| 2625 | 2743 | if (amdgpu_passthrough(adev) && ((adev->asic_type == CHIP_ARCTURUS && adev->gmc.xgmi.num_physical_nodes > 1)|| |
| 2626 | 2744 | adev->asic_type == CHIP_ALDEBARAN )) |
| 2627 | − smu_handle_passthrough_sbr(&adev->smu, true); | |
| 2745 | + amdgpu_dpm_handle_passthrough_sbr(adev, true); | |
| 2628 | 2746 | |
| 2629 | 2747 | if (adev->gmc.xgmi.num_physical_nodes > 1) { |
| 2630 | 2748 | mutex_lock(&mgpu_info.mutex); |
| @@ -2708,11 +2826,11 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) · 77 unmodified lines | ||
| 2708 | 2826 | } |
| 2709 | 2827 | } |
| 2710 | 2828 | |
| 2711 | − amdgpu_amdkfd_suspend(adev, false); | |
| 2712 | − | |
| 2713 | 2829 | amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); |
| 2714 | 2830 | amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); |
| 2715 | 2831 | |
| 2832 | + amdgpu_amdkfd_suspend(adev, false); | |
| 2833 | + | |
| 2716 | 2834 | /* Workaroud for ASICs need to disable SMC first */ |
| 2717 | 2835 | amdgpu_device_smu_fini_early(adev); |
| 2718 | 2836 | |
| @@ -2881,7 +2999,7 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev) · 162 unmodified lines | ||
| 2881 | 2999 | int i, r; |
| 2882 | 3000 | |
| 2883 | 3001 | if (adev->in_s0ix) |
| 2884 | − amdgpu_gfx_state_change_set(adev, sGpuChangeState_D3Entry); | |
| 3002 | + amdgpu_dpm_gfx_state_change(adev, sGpuChangeState_D3Entry); | |
| 2885 | 3003 | |
| 2886 | 3004 | for (i = adev->num_ip_blocks - 1; i >= 0; i--) { |
| 2887 | 3005 | if (!adev->ip_blocks[i].status.valid) |
| @@ -3307,9 +3425,9 @@ static void amdgpu_device_xgmi_reset_func(struct work_struct *__work) · 419 unmodified lines | ||
| 3307 | 3425 | if (adev->asic_reset_res) |
| 3308 | 3426 | goto fail; |
| 3309 | 3427 | |
| 3310 | − if (adev->mmhub.ras_funcs && | |
| 3311 | − adev->mmhub.ras_funcs->reset_ras_error_count) | |
| 3312 | − adev->mmhub.ras_funcs->reset_ras_error_count(adev); | |
| 3428 | + if (adev->mmhub.ras && adev->mmhub.ras->ras_block.hw_ops && | |
| 3429 | + adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count) | |
| 3430 | + adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count(adev); | |
| 3313 | 3431 | } else { |
| 3314 | 3432 | |
| 3315 | 3433 | task_barrier_full(&hive->tb); |
| @@ -3493,12 +3611,12 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 177 unmodified lines | ||
| 3493 | 3611 | mutex_init(&adev->mn_lock); |
| 3494 | 3612 | mutex_init(&adev->virt.vf_errors.lock); |
| 3495 | 3613 | hash_init(adev->mn_hash); |
| 3496 | − atomic_set(&adev->in_gpu_reset, 0); | |
| 3497 | − init_rwsem(&adev->reset_sem); | |
| 3498 | 3614 | mutex_init(&adev->psp.mutex); |
| 3499 | 3615 | mutex_init(&adev->notifier_lock); |
| 3616 | + mutex_init(&adev->pm.stable_pstate_ctx_lock); | |
| 3617 | + mutex_init(&adev->benchmark_mutex); | |
| 3500 | 3618 | |
| 3501 | − amdgpu_device_init_apu_flags(adev); | |
| 3619 | + amdgpu_device_init_apu_flags(adev); | |
| 3502 | 3620 | |
| 3503 | 3621 | r = amdgpu_device_check_arguments(adev); |
| 3504 | 3622 | if (r) |
| @@ -3519,6 +3637,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 14 unmodified lines | ||
| 3519 | 3637 | |
| 3520 | 3638 | INIT_LIST_HEAD(&adev->reset_list); |
| 3521 | 3639 | |
| 3640 | + INIT_LIST_HEAD(&adev->ras_list); | |
| 3641 | + | |
| 3522 | 3642 | INIT_DELAYED_WORK(&adev->delayed_init_work, |
| 3523 | 3643 | amdgpu_device_delayed_init_work_handler); |
| 3524 | 3644 | INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work, |
| @@ -3577,6 +3697,15 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 52 unmodified lines | ||
| 3577 | 3697 | return r; |
| 3578 | 3698 | } |
| 3579 | 3699 | |
| 3700 | + /* | |
| 3701 | + * Reset domain needs to be present early, before XGMI hive discovered | |
| 3702 | + * (if any) and intitialized to use reset sem and in_gpu reset flag | |
| 3703 | + * early on during init. | |
| 3704 | + */ | |
| 3705 | + adev->reset_domain = amdgpu_reset_create_reset_domain(SINGLE_DEVICE ,"amdgpu-reset-dev"); | |
| 3706 | + if (!adev->reset_domain) | |
| 3707 | + return -ENOMEM; | |
| 3708 | + | |
| 3580 | 3709 | /* early init functions */ |
| 3581 | 3710 | r = amdgpu_device_ip_early_init(adev); |
| 3582 | 3711 | if (r) |
| @@ -3749,19 +3878,6 @@ fence_driver_init: · 166 unmodified lines | ||
| 3749 | 3878 | } else |
| 3750 | 3879 | adev->ucode_sysfs_en = true; |
| 3751 | 3880 | |
| 3752 | − if ((amdgpu_testing & 1)) { | |
| 3753 | − if (adev->accel_working) | |
| 3754 | − amdgpu_test_moves(adev); | |
| 3755 | − else | |
| 3756 | − DRM_INFO("amdgpu: acceleration disabled, skipping move tests\n"); | |
| 3757 | − } | |
| 3758 | − if (amdgpu_benchmarking) { | |
| 3759 | − if (adev->accel_working) | |
| 3760 | − amdgpu_benchmark(adev, amdgpu_benchmarking); | |
| 3761 | − else | |
| 3762 | − DRM_INFO("amdgpu: acceleration disabled, skipping benchmarks\n"); | |
| 3763 | − } | |
| 3764 | − | |
| 3765 | 3881 | /* |
| 3766 | 3882 | * Register gpu instance before amdgpu_device_enable_mgpu_fan_boost. |
| 3767 | 3883 | * Otherwise the mgpu fan boost feature will be skipped due to the |
| @@ -3953,6 +4069,9 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev) · 185 unmodified lines | ||
| 3953 | 4069 | if (adev->mman.discovery_bin) |
| 3954 | 4070 | amdgpu_discovery_fini(adev); |
| 3955 | 4071 | |
| 4072 | + amdgpu_reset_put_reset_domain(adev->reset_domain); | |
| 4073 | + adev->reset_domain = NULL; | |
| 4074 | + | |
| 3956 | 4075 | kfree(adev->pci_state); |
| 3957 | 4076 | |
| 3958 | 4077 | } |
| @@ -4044,7 +4163,7 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon) · 85 unmodified lines | ||
| 4044 | 4163 | return 0; |
| 4045 | 4164 | |
| 4046 | 4165 | if (adev->in_s0ix) |
| 4047 | − amdgpu_gfx_state_change_set(adev, sGpuChangeState_D0Entry); | |
| 4166 | + amdgpu_dpm_gfx_state_change(adev, sGpuChangeState_D0Entry); | |
| 4048 | 4167 | |
| 4049 | 4168 | /* post card */ |
| 4050 | 4169 | if (amdgpu_device_need_post(adev)) { |
| @@ -4347,7 +4466,9 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, · 296 unmodified lines | ||
| 4347 | 4466 | { |
| 4348 | 4467 | int r; |
| 4349 | 4468 | struct amdgpu_hive_info *hive = NULL; |
| 4469 | + int retry_limit = 0; | |
| 4350 | 4470 | |
| 4471 | +retry: | |
| 4351 | 4472 | amdgpu_amdkfd_pre_reset(adev); |
| 4352 | 4473 | |
| 4353 | 4474 | amdgpu_amdkfd_pre_reset(adev); |
| @@ -4396,6 +4517,14 @@ error: · 42 unmodified lines | ||
| 4396 | 4517 | } |
| 4397 | 4518 | amdgpu_virt_release_full_gpu(adev, true); |
| 4398 | 4519 | |
| 4520 | + if (AMDGPU_RETRY_SRIOV_RESET(r)) { | |
| 4521 | + if (retry_limit < AMDGPU_MAX_RETRY_LIMIT) { | |
| 4522 | + retry_limit++; | |
| 4523 | + goto retry; | |
| 4524 | + } else | |
| 4525 | + DRM_ERROR("GPU reset retry is beyond the retry limit\n"); | |
| 4526 | + } | |
| 4527 | + | |
| 4399 | 4528 | return r; |
| 4400 | 4529 | } |
| 4401 | 4530 | |
| @@ -4587,6 +4716,22 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, · 185 unmodified lines | ||
| 4587 | 4716 | return r; |
| 4588 | 4717 | } |
| 4589 | 4718 | |
| 4719 | +static int amdgpu_reset_reg_dumps(struct amdgpu_device *adev) | |
| 4720 | +{ | |
| 4721 | + uint32_t reg_value; | |
| 4722 | + int i; | |
| 4723 | + | |
| 4724 | + lockdep_assert_held(&adev->reset_domain->sem); | |
| 4725 | + dump_stack(); | |
| 4726 | + | |
| 4727 | + for (i = 0; i < adev->num_regs; i++) { | |
| 4728 | + reg_value = RREG32(adev->reset_dump_reg_list[i]); | |
| 4729 | + trace_amdgpu_reset_reg_dumps(adev->reset_dump_reg_list[i], reg_value); | |
| 4730 | + } | |
| 4731 | + | |
| 4732 | + return 0; | |
| 4733 | +} | |
| 4734 | + | |
| 4590 | 4735 | int amdgpu_do_asic_reset(struct list_head *device_list_handle, |
| 4591 | 4736 | struct amdgpu_reset_context *reset_context) |
| 4592 | 4737 | { |
| @@ -4597,6 +4742,7 @@ int amdgpu_do_asic_reset(struct list_head *device_list_handle, · 4 unmodified lines | ||
| 4597 | 4742 | /* Try reset handler method first */ |
| 4598 | 4743 | tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device, |
| 4599 | 4744 | reset_list); |
| 4745 | + amdgpu_reset_reg_dumps(tmp_adev); | |
| 4600 | 4746 | r = amdgpu_reset_perform_reset(tmp_adev, reset_context); |
| 4601 | 4747 | /* If reset handler not implemented, continue; otherwise return */ |
| 4602 | 4748 | if (r == -ENOSYS) |
| @@ -4645,9 +4791,9 @@ int amdgpu_do_asic_reset(struct list_head *device_list_handle, · 42 unmodified lines | ||
| 4645 | 4791 | |
| 4646 | 4792 | if (!r && amdgpu_ras_intr_triggered()) { |
| 4647 | 4793 | list_for_each_entry(tmp_adev, device_list_handle, reset_list) { |
| 4648 | − if (tmp_adev->mmhub.ras_funcs && | |
| 4649 | − tmp_adev->mmhub.ras_funcs->reset_ras_error_count) | |
| 4650 | − tmp_adev->mmhub.ras_funcs->reset_ras_error_count(tmp_adev); | |
| 4794 | + if (tmp_adev->mmhub.ras && tmp_adev->mmhub.ras->ras_block.hw_ops && | |
| 4795 | + tmp_adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count) | |
| 4796 | + tmp_adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count(tmp_adev); | |
| 4651 | 4797 | } |
| 4652 | 4798 | |
| 4653 | 4799 | amdgpu_ras_intr_cleared(); |
| @@ -4754,17 +4900,8 @@ end: · 100 unmodified lines | ||
| 4754 | 4900 | return r; |
| 4755 | 4901 | } |
| 4756 | 4902 | |
| 4757 | −static bool amdgpu_device_lock_adev(struct amdgpu_device *adev, | |
| 4758 | − struct amdgpu_hive_info *hive) | |
| 4903 | +static void amdgpu_device_set_mp1_state(struct amdgpu_device *adev) | |
| 4759 | 4904 | { |
| 4760 | − if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0) | |
| 4761 | − return false; | |
| 4762 | − | |
| 4763 | − if (hive) { | |
| 4764 | − down_write_nest_lock(&adev->reset_sem, &hive->hive_lock); | |
| 4765 | − } else { | |
| 4766 | − down_write(&adev->reset_sem); | |
| 4767 | − } | |
| 4768 | 4905 | |
| 4769 | 4906 | switch (amdgpu_asic_reset_method(adev)) { |
| 4770 | 4907 | case AMD_RESET_METHOD_MODE1: |
| @@ -4777,56 +4914,12 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev, · 6 unmodified lines | ||
| 4777 | 4914 | adev->mp1_state = PP_MP1_STATE_NONE; |
| 4778 | 4915 | break; |
| 4779 | 4916 | } |
| 4780 | − | |
| 4781 | − return true; | |
| 4782 | 4917 | } |
| 4783 | 4918 | |
| 4784 | −static void amdgpu_device_unlock_adev(struct amdgpu_device *adev) | |
| 4919 | +static void amdgpu_device_unset_mp1_state(struct amdgpu_device *adev) | |
| 4785 | 4920 | { |
| 4786 | 4921 | amdgpu_vf_error_trans_all(adev); |
| 4787 | 4922 | adev->mp1_state = PP_MP1_STATE_NONE; |
| 4788 | − atomic_set(&adev->in_gpu_reset, 0); | |
| 4789 | − up_write(&adev->reset_sem); | |
| 4790 | −} | |
| 4791 | − | |
| 4792 | −/* | |
| 4793 | − * to lockup a list of amdgpu devices in a hive safely, if not a hive | |
| 4794 | − * with multiple nodes, it will be similar as amdgpu_device_lock_adev. | |
| 4795 | − * | |
| 4796 | − * unlock won't require roll back. | |
| 4797 | − */ | |
| 4798 | −static int amdgpu_device_lock_hive_adev(struct amdgpu_device *adev, struct amdgpu_hive_info *hive) | |
| 4799 | −{ | |
| 4800 | − struct amdgpu_device *tmp_adev = NULL; | |
| 4801 | − | |
| 4802 | − if (!amdgpu_sriov_vf(adev) && (adev->gmc.xgmi.num_physical_nodes > 1)) { | |
| 4803 | − if (!hive) { | |
| 4804 | − dev_err(adev->dev, "Hive is NULL while device has multiple xgmi nodes"); | |
| 4805 | − return -ENODEV; | |
| 4806 | − } | |
| 4807 | − list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) { | |
| 4808 | − if (!amdgpu_device_lock_adev(tmp_adev, hive)) | |
| 4809 | − goto roll_back; | |
| 4810 | − } | |
| 4811 | − } else if (!amdgpu_device_lock_adev(adev, hive)) | |
| 4812 | − return -EAGAIN; | |
| 4813 | − | |
| 4814 | − return 0; | |
| 4815 | −roll_back: | |
| 4816 | − if (!list_is_first(&tmp_adev->gmc.xgmi.head, &hive->device_list)) { | |
| 4817 | − /* | |
| 4818 | − * if the lockup iteration break in the middle of a hive, | |
| 4819 | − * it may means there may has a race issue, | |
| 4820 | − * or a hive device locked up independently. | |
| 4821 | − * we may be in trouble and may not, so will try to roll back | |
| 4822 | − * the lock and give out a warnning. | |
| 4823 | − */ | |
| 4824 | − dev_warn(tmp_adev->dev, "Hive lock iteration broke in the middle. Rolling back to unlock"); | |
| 4825 | − list_for_each_entry_continue_reverse(tmp_adev, &hive->device_list, gmc.xgmi.head) { | |
| 4826 | − amdgpu_device_unlock_adev(tmp_adev); | |
| 4827 | − } | |
| 4828 | − } | |
| 4829 | − return -EAGAIN; | |
| 4830 | 4923 | } |
| 4831 | 4924 | |
| 4832 | 4925 | static void amdgpu_device_resume_display_audio(struct amdgpu_device *adev) |
| @@ -4960,7 +5053,7 @@ retry: · 127 unmodified lines | ||
| 4960 | 5053 | } |
| 4961 | 5054 | |
| 4962 | 5055 | /** |
| 4963 | − * amdgpu_device_gpu_recover - reset the asic and recover scheduler | |
| 5056 | + * amdgpu_device_gpu_recover_imp - reset the asic and recover scheduler | |
| 4964 | 5057 | * |
| 4965 | 5058 | * @adev: amdgpu_device pointer |
| 4966 | 5059 | * @job: which job trigger hang |
| @@ -4970,7 +5063,7 @@ retry: · 3 unmodified lines | ||
| 4970 | 5063 | * Returns 0 for success or an error on failure. |
| 4971 | 5064 | */ |
| 4972 | 5065 | |
| 4973 | −int amdgpu_device_gpu_recover(struct amdgpu_device *adev, | |
| 5066 | +int amdgpu_device_gpu_recover_imp(struct amdgpu_device *adev, | |
| 4974 | 5067 | struct amdgpu_job *job) |
| 4975 | 5068 | { |
| 4976 | 5069 | struct list_head device_list, *device_list_handle = NULL; |
| @@ -5004,26 +5097,10 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 27 unmodified lines | ||
| 5004 | 5097 | dev_info(adev->dev, "GPU %s begin!\n", |
| 5005 | 5098 | need_emergency_restart ? "jobs stop":"reset"); |
| 5006 | 5099 | |
| 5007 | − /* | |
| 5008 | − * Here we trylock to avoid chain of resets executing from | |
| 5009 | − * either trigger by jobs on different adevs in XGMI hive or jobs on | |
| 5010 | − * different schedulers for same device while this TO handler is running. | |
| 5011 | − * We always reset all schedulers for device and all devices for XGMI | |
| 5012 | − * hive so that should take care of them too. | |
| 5013 | − */ | |
| 5014 | 5100 | if (!amdgpu_sriov_vf(adev)) |
| 5015 | 5101 | hive = amdgpu_get_xgmi_hive(adev); |
| 5016 | − if (hive) { | |
| 5017 | − if (atomic_cmpxchg(&hive->in_reset, 0, 1) != 0) { | |
| 5018 | − DRM_INFO("Bailing on TDR for s_job:%llx, hive: %llx as another already in progress", | |
| 5019 | − job ? job->base.id : -1, hive->hive_id); | |
| 5020 | − amdgpu_put_xgmi_hive(hive); | |
| 5021 | − if (job && job->vm) | |
| 5022 | − drm_sched_increase_karma(&job->base); | |
| 5023 | − return 0; | |
| 5024 | − } | |
| 5102 | + if (hive) | |
| 5025 | 5103 | mutex_lock(&hive->hive_lock); |
| 5026 | − } | |
| 5027 | 5104 | |
| 5028 | 5105 | reset_context.method = AMD_RESET_METHOD_NONE; |
| 5029 | 5106 | reset_context.reset_req_dev = adev; |
| @@ -5031,22 +5108,6 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 1 unmodified line | ||
| 5031 | 5108 | reset_context.hive = hive; |
| 5032 | 5109 | clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); |
| 5033 | 5110 | |
| 5034 | − /* | |
| 5035 | − * lock the device before we try to operate the linked list | |
| 5036 | − * if didn't get the device lock, don't touch the linked list since | |
| 5037 | − * others may iterating it. | |
| 5038 | − */ | |
| 5039 | − r = amdgpu_device_lock_hive_adev(adev, hive); | |
| 5040 | − if (r) { | |
| 5041 | − dev_info(adev->dev, "Bailing on TDR for s_job:%llx, as another already in progress", | |
| 5042 | − job ? job->base.id : -1); | |
| 5043 | − | |
| 5044 | − /* even we skipped this reset, still need to set the job to guilty */ | |
| 5045 | − if (job && job->vm) | |
| 5046 | − drm_sched_increase_karma(&job->base); | |
| 5047 | − goto skip_recovery; | |
| 5048 | − } | |
| 5049 | − | |
| 5050 | 5111 | /* |
| 5051 | 5112 | * Build list of devices to reset. |
| 5052 | 5113 | * In case we are in XGMI hive mode, resort the device list |
| @@ -5064,8 +5125,16 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 11 unmodified lines | ||
| 5064 | 5125 | device_list_handle = &device_list; |
| 5065 | 5126 | } |
| 5066 | 5127 | |
| 5128 | + /* We need to lock reset domain only once both for XGMI and single device */ | |
| 5129 | + tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device, | |
| 5130 | + reset_list); | |
| 5131 | + amdgpu_device_lock_reset_domain(tmp_adev->reset_domain); | |
| 5132 | + | |
| 5067 | 5133 | /* block all schedulers and reset given job's ring */ |
| 5068 | 5134 | list_for_each_entry(tmp_adev, device_list_handle, reset_list) { |
| 5135 | + | |
| 5136 | + amdgpu_device_set_mp1_state(tmp_adev); | |
| 5137 | + | |
| 5069 | 5138 | /* |
| 5070 | 5139 | * Try to put the audio codec into suspend state |
| 5071 | 5140 | * before gpu reset started. |
| @@ -5187,6 +5256,9 @@ skip_hw_reset: · 115 unmodified lines | ||
| 5187 | 5256 | drm_helper_resume_force_mode(adev_to_drm(tmp_adev)); |
| 5188 | 5257 | } |
| 5189 | 5258 | |
| 5259 | + if (tmp_adev->asic_reset_res) | |
| 5260 | + r = tmp_adev->asic_reset_res; | |
| 5261 | + | |
| 5190 | 5262 | tmp_adev->asic_reset_res = 0; |
| 5191 | 5263 | |
| 5192 | 5264 | if (r) { |
| @@ -5214,21 +5286,55 @@ skip_sched_resume: · 21 unmodified lines | ||
| 5214 | 5286 | |
| 5215 | 5287 | if (audio_suspended) |
| 5216 | 5288 | amdgpu_device_resume_display_audio(tmp_adev); |
| 5217 | − amdgpu_device_unlock_adev(tmp_adev); | |
| 5289 | + | |
| 5290 | + amdgpu_device_unset_mp1_state(tmp_adev); | |
| 5218 | 5291 | } |
| 5219 | 5292 | |
| 5220 | −skip_recovery: | |
| 5293 | + tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device, | |
| 5294 | + reset_list); | |
| 5295 | + amdgpu_device_unlock_reset_domain(tmp_adev->reset_domain); | |
| 5296 | + | |
| 5221 | 5297 | if (hive) { |
| 5222 | − atomic_set(&hive->in_reset, 0); | |
| 5223 | 5298 | mutex_unlock(&hive->hive_lock); |
| 5224 | 5299 | amdgpu_put_xgmi_hive(hive); |
| 5225 | 5300 | } |
| 5226 | 5301 | |
| 5227 | − if (r && r != -EAGAIN) | |
| 5302 | + if (r) | |
| 5228 | 5303 | dev_info(adev->dev, "GPU reset end with ret = %d\n", r); |
| 5229 | 5304 | return r; |
| 5230 | 5305 | } |
| 5231 | 5306 | |
| 5307 | +struct amdgpu_recover_work_struct { | |
| 5308 | + struct work_struct base; | |
| 5309 | + struct amdgpu_device *adev; | |
| 5310 | + struct amdgpu_job *job; | |
| 5311 | + int ret; | |
| 5312 | +}; | |
| 5313 | + | |
| 5314 | +static void amdgpu_device_queue_gpu_recover_work(struct work_struct *work) | |
| 5315 | +{ | |
| 5316 | + struct amdgpu_recover_work_struct *recover_work = container_of(work, struct amdgpu_recover_work_struct, base); | |
| 5317 | + | |
| 5318 | + recover_work->ret = amdgpu_device_gpu_recover_imp(recover_work->adev, recover_work->job); | |
| 5319 | +} | |
| 5320 | +/* | |
| 5321 | + * Serialize gpu recover into reset domain single threaded wq | |
| 5322 | + */ | |
| 5323 | +int amdgpu_device_gpu_recover(struct amdgpu_device *adev, | |
| 5324 | + struct amdgpu_job *job) | |
| 5325 | +{ | |
| 5326 | + struct amdgpu_recover_work_struct work = {.adev = adev, .job = job}; | |
| 5327 | + | |
| 5328 | + INIT_WORK(&work.base, amdgpu_device_queue_gpu_recover_work); | |
| 5329 | + | |
| 5330 | + if (!amdgpu_reset_domain_schedule(adev->reset_domain, &work.base)) | |
| 5331 | + return -EAGAIN; | |
| 5332 | + | |
| 5333 | + flush_work(&work.base); | |
| 5334 | + | |
| 5335 | + return work.ret; | |
| 5336 | +} | |
| 5337 | + | |
| 5232 | 5338 | /** |
| 5233 | 5339 | * amdgpu_device_get_pcie_info - fence pcie info about the PCIE slot |
| 5234 | 5340 | * |
| @@ -5416,20 +5522,6 @@ int amdgpu_device_baco_exit(struct drm_device *dev) · 181 unmodified lines | ||
| 5416 | 5522 | return 0; |
| 5417 | 5523 | } |
| 5418 | 5524 | |
| 5419 | −static void amdgpu_cancel_all_tdr(struct amdgpu_device *adev) | |
| 5420 | −{ | |
| 5421 | − int i; | |
| 5422 | − | |
| 5423 | − for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | |
| 5424 | − struct amdgpu_ring *ring = adev->rings[i]; | |
| 5425 | − | |
| 5426 | − if (!ring || !ring->sched.thread) | |
| 5427 | − continue; | |
| 5428 | − | |
| 5429 | − cancel_delayed_work_sync(&ring->sched.work_tdr); | |
| 5430 | − } | |
| 5431 | −} | |
| 5432 | − | |
| 5433 | 5525 | /** |
| 5434 | 5526 | * amdgpu_pci_error_detected - Called when a PCI error is detected. |
| 5435 | 5527 | * @pdev: PCI device struct |
| @@ -5460,14 +5552,11 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_sta · 24 unmodified lines | ||
| 5460 | 5552 | /* Fatal error, prepare for slot reset */ |
| 5461 | 5553 | case pci_channel_io_frozen: |
| 5462 | 5554 | /* |
| 5463 | − * Cancel and wait for all TDRs in progress if failing to | |
| 5464 | − * set adev->in_gpu_reset in amdgpu_device_lock_adev | |
| 5465 | − * | |
| 5466 | − * Locking adev->reset_sem will prevent any external access | |
| 5555 | + * Locking adev->reset_domain->sem will prevent any external access | |
| 5467 | 5556 | * to GPU during PCI error recovery |
| 5468 | 5557 | */ |
| 5469 | − while (!amdgpu_device_lock_adev(adev, NULL)) | |
| 5470 | − amdgpu_cancel_all_tdr(adev); | |
| 5558 | + amdgpu_device_lock_reset_domain(adev->reset_domain); | |
| 5559 | + amdgpu_device_set_mp1_state(adev); | |
| 5471 | 5560 | |
| 5472 | 5561 | /* |
| 5473 | 5562 | * Block any work scheduling as we do for regular GPU reset |
| @@ -5574,7 +5663,8 @@ out: · 100 unmodified lines | ||
| 5574 | 5663 | DRM_INFO("PCIe error recovery succeeded\n"); |
| 5575 | 5664 | } else { |
| 5576 | 5665 | DRM_ERROR("PCIe error recovery failed, err:%d", r); |
| 5577 | − amdgpu_device_unlock_adev(adev); | |
| 5666 | + amdgpu_device_unset_mp1_state(adev); | |
| 5667 | + amdgpu_device_unlock_reset_domain(adev->reset_domain); | |
| 5578 | 5668 | } |
| 5579 | 5669 | |
| 5580 | 5670 | return r ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED; |
| @@ -5611,7 +5701,8 @@ void amdgpu_pci_resume(struct pci_dev *pdev) · 30 unmodified lines | ||
| 5611 | 5701 | drm_sched_start(&ring->sched, true); |
| 5612 | 5702 | } |
| 5613 | 5703 | |
| 5614 | − amdgpu_device_unlock_adev(adev); | |
| 5704 | + amdgpu_device_unset_mp1_state(adev); | |
| 5705 | + amdgpu_device_unlock_reset_domain(adev->reset_domain); | |
| 5615 | 5706 | } |
| 5616 | 5707 | |
| 5617 | 5708 | bool amdgpu_device_cache_pci_state(struct pci_dev *pdev) |
| @@ -5688,6 +5779,11 @@ void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, · 70 unmodified lines | ||
| 5688 | 5779 | amdgpu_asic_invalidate_hdp(adev, ring); |
| 5689 | 5780 | } |
| 5690 | 5781 | |
| 5782 | +int amdgpu_in_reset(struct amdgpu_device *adev) | |
| 5783 | +{ | |
| 5784 | + return atomic_read(&adev->reset_domain->in_gpu_reset); | |
| 5785 | + } | |
| 5786 | + | |
| 5691 | 5787 | /** |
| 5692 | 5788 | * amdgpu_device_halt() - bring hardware to some kind of halt state |
| 5693 | 5789 | * |
| @@ -5726,3 +5822,36 @@ void amdgpu_device_halt(struct amdgpu_device *adev) · 32 unmodified lines | ||
| 5726 | 5822 | pci_disable_device(pdev); |
| 5727 | 5823 | pci_wait_for_pending_transaction(pdev); |
| 5728 | 5824 | } |
| 5825 | + | |
| 5826 | +u32 amdgpu_device_pcie_port_rreg(struct amdgpu_device *adev, | |
| 5827 | + u32 reg) | |
| 5828 | +{ | |
| 5829 | + unsigned long flags, address, data; | |
| 5830 | + u32 r; | |
| 5831 | + | |
| 5832 | + address = adev->nbio.funcs->get_pcie_port_index_offset(adev); | |
| 5833 | + data = adev->nbio.funcs->get_pcie_port_data_offset(adev); | |
| 5834 | + | |
| 5835 | + spin_lock_irqsave(&adev->pcie_idx_lock, flags); | |
| 5836 | + WREG32(address, reg * 4); | |
| 5837 | + (void)RREG32(address); | |
| 5838 | + r = RREG32(data); | |
| 5839 | + spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); | |
| 5840 | + return r; | |
| 5841 | +} | |
| 5842 | + | |
| 5843 | +void amdgpu_device_pcie_port_wreg(struct amdgpu_device *adev, | |
| 5844 | + u32 reg, u32 v) | |
| 5845 | +{ | |
| 5846 | + unsigned long flags, address, data; | |
| 5847 | + | |
| 5848 | + address = adev->nbio.funcs->get_pcie_port_index_offset(adev); | |
| 5849 | + data = adev->nbio.funcs->get_pcie_port_data_offset(adev); | |
| 5850 | + | |
| 5851 | + spin_lock_irqsave(&adev->pcie_idx_lock, flags); | |
| 5852 | + WREG32(address, reg * 4); | |
| 5853 | + (void)RREG32(address); | |
| 5854 | + WREG32(data, v); | |
| 5855 | + (void)RREG32(data); | |
| 5856 | + spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); | |
| 5857 | +} | |
| @@ -360,8 +360,11 @@ out: · 359 unmodified lines | ||
| 360 | 360 | return r; |
| 361 | 361 | } |
| 362 | 362 | |
| 363 | +static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev); | |
| 364 | + | |
| 363 | 365 | void amdgpu_discovery_fini(struct amdgpu_device *adev) |
| 364 | 366 | { |
| 367 | + amdgpu_discovery_sysfs_fini(adev); | |
| 365 | 368 | kfree(adev->mman.discovery_bin); |
| 366 | 369 | adev->mman.discovery_bin = NULL; |
| 367 | 370 | } |
| @@ -382,6 +385,578 @@ static int amdgpu_discovery_validate_ip(const struct ip *ip) · 14 unmodified lines | ||
| 382 | 385 | return 0; |
| 383 | 386 | } |
| 384 | 387 | |
| 388 | +static void amdgpu_discovery_read_harvest_bit_per_ip(struct amdgpu_device *adev, | |
| 389 | + uint32_t *vcn_harvest_count) | |
| 390 | +{ | |
| 391 | + struct binary_header *bhdr; | |
| 392 | + struct ip_discovery_header *ihdr; | |
| 393 | + struct die_header *dhdr; | |
| 394 | + struct ip *ip; | |
| 395 | + uint16_t die_offset, ip_offset, num_dies, num_ips; | |
| 396 | + int i, j; | |
| 397 | + | |
| 398 | + bhdr = (struct binary_header *)adev->mman.discovery_bin; | |
| 399 | + ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin + | |
| 400 | + le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset)); | |
| 401 | + num_dies = le16_to_cpu(ihdr->num_dies); | |
| 402 | + | |
| 403 | + /* scan harvest bit of all IP data structures */ | |
| 404 | + for (i = 0; i < num_dies; i++) { | |
| 405 | + die_offset = le16_to_cpu(ihdr->die_info[i].die_offset); | |
| 406 | + dhdr = (struct die_header *)(adev->mman.discovery_bin + die_offset); | |
| 407 | + num_ips = le16_to_cpu(dhdr->num_ips); | |
| 408 | + ip_offset = die_offset + sizeof(*dhdr); | |
| 409 | + | |
| 410 | + for (j = 0; j < num_ips; j++) { | |
| 411 | + ip = (struct ip *)(adev->mman.discovery_bin + ip_offset); | |
| 412 | + | |
| 413 | + if (amdgpu_discovery_validate_ip(ip)) | |
| 414 | + goto next_ip; | |
| 415 | + | |
| 416 | + if (le16_to_cpu(ip->harvest) == 1) { | |
| 417 | + switch (le16_to_cpu(ip->hw_id)) { | |
| 418 | + case VCN_HWID: | |
| 419 | + (*vcn_harvest_count)++; | |
| 420 | + if (ip->number_instance == 0) | |
| 421 | + adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN0; | |
| 422 | + else | |
| 423 | + adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN1; | |
| 424 | + break; | |
| 425 | + case DMU_HWID: | |
| 426 | + adev->harvest_ip_mask |= AMD_HARVEST_IP_DMU_MASK; | |
| 427 | + break; | |
| 428 | + default: | |
| 429 | + break; | |
| 430 | + } | |
| 431 | + } | |
| 432 | +next_ip: | |
| 433 | + ip_offset += sizeof(*ip) + 4 * (ip->num_base_address - 1); | |
| 434 | + } | |
| 435 | + } | |
| 436 | +} | |
| 437 | + | |
| 438 | +static void amdgpu_discovery_read_from_harvest_table(struct amdgpu_device *adev, | |
| 439 | + uint32_t *vcn_harvest_count) | |
| 440 | +{ | |
| 441 | + struct binary_header *bhdr; | |
| 442 | + struct harvest_table *harvest_info; | |
| 443 | + int i; | |
| 444 | + | |
| 445 | + bhdr = (struct binary_header *)adev->mman.discovery_bin; | |
| 446 | + harvest_info = (struct harvest_table *)(adev->mman.discovery_bin + | |
| 447 | + le16_to_cpu(bhdr->table_list[HARVEST_INFO].offset)); | |
| 448 | + for (i = 0; i < 32; i++) { | |
| 449 | + if (le16_to_cpu(harvest_info->list[i].hw_id) == 0) | |
| 450 | + break; | |
| 451 | + | |
| 452 | + switch (le16_to_cpu(harvest_info->list[i].hw_id)) { | |
| 453 | + case VCN_HWID: | |
| 454 | + (*vcn_harvest_count)++; | |
| 455 | + if (harvest_info->list[i].number_instance == 0) | |
| 456 | + adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN0; | |
| 457 | + else | |
| 458 | + adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN1; | |
| 459 | + break; | |
| 460 | + case DMU_HWID: | |
| 461 | + adev->harvest_ip_mask |= AMD_HARVEST_IP_DMU_MASK; | |
| 462 | + break; | |
| 463 | + default: | |
| 464 | + break; | |
| 465 | + } | |
| 466 | + } | |
| 467 | +} | |
| 468 | + | |
| 469 | +/* ================================================== */ | |
| 470 | + | |
| 471 | +struct ip_hw_instance { | |
| 472 | + struct kobject kobj; /* ip_discovery/die/#die/#hw_id/#instance/<attrs...> */ | |
| 473 | + | |
| 474 | + int hw_id; | |
| 475 | + u8 num_instance; | |
| 476 | + u8 major, minor, revision; | |
| 477 | + u8 harvest; | |
| 478 | + | |
| 479 | + int num_base_addresses; | |
| 480 | + u32 base_addr[]; | |
| 481 | +}; | |
| 482 | + | |
| 483 | +struct ip_hw_id { | |
| 484 | + struct kset hw_id_kset; /* ip_discovery/die/#die/#hw_id/, contains ip_hw_instance */ | |
| 485 | + int hw_id; | |
| 486 | +}; | |
| 487 | + | |
| 488 | +struct ip_die_entry { | |
| 489 | + struct kset ip_kset; /* ip_discovery/die/#die/, contains ip_hw_id */ | |
| 490 | + u16 num_ips; | |
| 491 | +}; | |
| 492 | + | |
| 493 | +/* -------------------------------------------------- */ | |
| 494 | + | |
| 495 | +struct ip_hw_instance_attr { | |
| 496 | + struct attribute attr; | |
| 497 | + ssize_t (*show)(struct ip_hw_instance *ip_hw_instance, char *buf); | |
| 498 | +}; | |
| 499 | + | |
| 500 | +static ssize_t hw_id_show(struct ip_hw_instance *ip_hw_instance, char *buf) | |
| 501 | +{ | |
| 502 | + return sysfs_emit(buf, "%d\n", ip_hw_instance->hw_id); | |
| 503 | +} | |
| 504 | + | |
| 505 | +static ssize_t num_instance_show(struct ip_hw_instance *ip_hw_instance, char *buf) | |
| 506 | +{ | |
| 507 | + return sysfs_emit(buf, "%d\n", ip_hw_instance->num_instance); | |
| 508 | +} | |
| 509 | + | |
| 510 | +static ssize_t major_show(struct ip_hw_instance *ip_hw_instance, char *buf) | |
| 511 | +{ | |
| 512 | + return sysfs_emit(buf, "%d\n", ip_hw_instance->major); | |
| 513 | +} | |
| 514 | + | |
| 515 | +static ssize_t minor_show(struct ip_hw_instance *ip_hw_instance, char *buf) | |
| 516 | +{ | |
| 517 | + return sysfs_emit(buf, "%d\n", ip_hw_instance->minor); | |
| 518 | +} | |
| 519 | + | |
| 520 | +static ssize_t revision_show(struct ip_hw_instance *ip_hw_instance, char *buf) | |
| 521 | +{ | |
| 522 | + return sysfs_emit(buf, "%d\n", ip_hw_instance->revision); | |
| 523 | +} | |
| 524 | + | |
| 525 | +static ssize_t harvest_show(struct ip_hw_instance *ip_hw_instance, char *buf) | |
| 526 | +{ | |
| 527 | + return sysfs_emit(buf, "0x%01X\n", ip_hw_instance->harvest); | |
| 528 | +} | |
| 529 | + | |
| 530 | +static ssize_t num_base_addresses_show(struct ip_hw_instance *ip_hw_instance, char *buf) | |
| 531 | +{ | |
| 532 | + return sysfs_emit(buf, "%d\n", ip_hw_instance->num_base_addresses); | |
| 533 | +} | |
| 534 | + | |
| 535 | +static ssize_t base_addr_show(struct ip_hw_instance *ip_hw_instance, char *buf) | |
| 536 | +{ | |
| 537 | + ssize_t res, at; | |
| 538 | + int ii; | |
| 539 | + | |
| 540 | + for (res = at = ii = 0; ii < ip_hw_instance->num_base_addresses; ii++) { | |
| 541 | + /* Here we satisfy the condition that, at + size <= PAGE_SIZE. | |
| 542 | + */ | |
| 543 | + if (at + 12 > PAGE_SIZE) | |
| 544 | + break; | |
| 545 | + res = sysfs_emit_at(buf, at, "0x%08X\n", | |
| 546 | + ip_hw_instance->base_addr[ii]); | |
| 547 | + if (res <= 0) | |
| 548 | + break; | |
| 549 | + at += res; | |
| 550 | + } | |
| 551 | + | |
| 552 | + return res < 0 ? res : at; | |
| 553 | +} | |
| 554 | + | |
| 555 | +static struct ip_hw_instance_attr ip_hw_attr[] = { | |
| 556 | + __ATTR_RO(hw_id), | |
| 557 | + __ATTR_RO(num_instance), | |
| 558 | + __ATTR_RO(major), | |
| 559 | + __ATTR_RO(minor), | |
| 560 | + __ATTR_RO(revision), | |
| 561 | + __ATTR_RO(harvest), | |
| 562 | + __ATTR_RO(num_base_addresses), | |
| 563 | + __ATTR_RO(base_addr), | |
| 564 | +}; | |
| 565 | + | |
| 566 | +static struct attribute *ip_hw_instance_attrs[ARRAY_SIZE(ip_hw_attr) + 1]; | |
| 567 | +ATTRIBUTE_GROUPS(ip_hw_instance); | |
| 568 | + | |
| 569 | +#define to_ip_hw_instance(x) container_of(x, struct ip_hw_instance, kobj) | |
| 570 | +#define to_ip_hw_instance_attr(x) container_of(x, struct ip_hw_instance_attr, attr) | |
| 571 | + | |
| 572 | +static ssize_t ip_hw_instance_attr_show(struct kobject *kobj, | |
| 573 | + struct attribute *attr, | |
| 574 | + char *buf) | |
| 575 | +{ | |
| 576 | + struct ip_hw_instance *ip_hw_instance = to_ip_hw_instance(kobj); | |
| 577 | + struct ip_hw_instance_attr *ip_hw_attr = to_ip_hw_instance_attr(attr); | |
| 578 | + | |
| 579 | + if (!ip_hw_attr->show) | |
| 580 | + return -EIO; | |
| 581 | + | |
| 582 | + return ip_hw_attr->show(ip_hw_instance, buf); | |
| 583 | +} | |
| 584 | + | |
| 585 | +static const struct sysfs_ops ip_hw_instance_sysfs_ops = { | |
| 586 | + .show = ip_hw_instance_attr_show, | |
| 587 | +}; | |
| 588 | + | |
| 589 | +static void ip_hw_instance_release(struct kobject *kobj) | |
| 590 | +{ | |
| 591 | + struct ip_hw_instance *ip_hw_instance = to_ip_hw_instance(kobj); | |
| 592 | + | |
| 593 | + kfree(ip_hw_instance); | |
| 594 | +} | |
| 595 | + | |
| 596 | +static struct kobj_type ip_hw_instance_ktype = { | |
| 597 | + .release = ip_hw_instance_release, | |
| 598 | + .sysfs_ops = &ip_hw_instance_sysfs_ops, | |
| 599 | + .default_groups = ip_hw_instance_groups, | |
| 600 | +}; | |
| 601 | + | |
| 602 | +/* -------------------------------------------------- */ | |
| 603 | + | |
| 604 | +#define to_ip_hw_id(x) container_of(to_kset(x), struct ip_hw_id, hw_id_kset) | |
| 605 | + | |
| 606 | +static void ip_hw_id_release(struct kobject *kobj) | |
| 607 | +{ | |
| 608 | + struct ip_hw_id *ip_hw_id = to_ip_hw_id(kobj); | |
| 609 | + | |
| 610 | + if (!list_empty(&ip_hw_id->hw_id_kset.list)) | |
| 611 | + DRM_ERROR("ip_hw_id->hw_id_kset is not empty"); | |
| 612 | + kfree(ip_hw_id); | |
| 613 | +} | |
| 614 | + | |
| 615 | +static struct kobj_type ip_hw_id_ktype = { | |
| 616 | + .release = ip_hw_id_release, | |
| 617 | + .sysfs_ops = &kobj_sysfs_ops, | |
| 618 | +}; | |
| 619 | + | |
| 620 | +/* -------------------------------------------------- */ | |
| 621 | + | |
| 622 | +static void die_kobj_release(struct kobject *kobj); | |
| 623 | +static void ip_disc_release(struct kobject *kobj); | |
| 624 | + | |
| 625 | +struct ip_die_entry_attribute { | |
| 626 | + struct attribute attr; | |
| 627 | + ssize_t (*show)(struct ip_die_entry *ip_die_entry, char *buf); | |
| 628 | +}; | |
| 629 | + | |
| 630 | +#define to_ip_die_entry_attr(x) container_of(x, struct ip_die_entry_attribute, attr) | |
| 631 | + | |
| 632 | +static ssize_t num_ips_show(struct ip_die_entry *ip_die_entry, char *buf) | |
| 633 | +{ | |
| 634 | + return sysfs_emit(buf, "%d\n", ip_die_entry->num_ips); | |
| 635 | +} | |
| 636 | + | |
| 637 | +/* If there are more ip_die_entry attrs, other than the number of IPs, | |
| 638 | + * we can make this intro an array of attrs, and then initialize | |
| 639 | + * ip_die_entry_attrs in a loop. | |
| 640 | + */ | |
| 641 | +static struct ip_die_entry_attribute num_ips_attr = | |
| 642 | + __ATTR_RO(num_ips); | |
| 643 | + | |
| 644 | +static struct attribute *ip_die_entry_attrs[] = { | |
| 645 | + &num_ips_attr.attr, | |
| 646 | + NULL, | |
| 647 | +}; | |
| 648 | +ATTRIBUTE_GROUPS(ip_die_entry); /* ip_die_entry_groups */ | |
| 649 | + | |
| 650 | +#define to_ip_die_entry(x) container_of(to_kset(x), struct ip_die_entry, ip_kset) | |
| 651 | + | |
| 652 | +static ssize_t ip_die_entry_attr_show(struct kobject *kobj, | |
| 653 | + struct attribute *attr, | |
| 654 | + char *buf) | |
| 655 | +{ | |
| 656 | + struct ip_die_entry_attribute *ip_die_entry_attr = to_ip_die_entry_attr(attr); | |
| 657 | + struct ip_die_entry *ip_die_entry = to_ip_die_entry(kobj); | |
| 658 | + | |
| 659 | + if (!ip_die_entry_attr->show) | |
| 660 | + return -EIO; | |
| 661 | + | |
| 662 | + return ip_die_entry_attr->show(ip_die_entry, buf); | |
| 663 | +} | |
| 664 | + | |
| 665 | +static void ip_die_entry_release(struct kobject *kobj) | |
| 666 | +{ | |
| 667 | + struct ip_die_entry *ip_die_entry = to_ip_die_entry(kobj); | |
| 668 | + | |
| 669 | + if (!list_empty(&ip_die_entry->ip_kset.list)) | |
| 670 | + DRM_ERROR("ip_die_entry->ip_kset is not empty"); | |
| 671 | + kfree(ip_die_entry); | |
| 672 | +} | |
| 673 | + | |
| 674 | +static const struct sysfs_ops ip_die_entry_sysfs_ops = { | |
| 675 | + .show = ip_die_entry_attr_show, | |
| 676 | +}; | |
| 677 | + | |
| 678 | +static struct kobj_type ip_die_entry_ktype = { | |
| 679 | + .release = ip_die_entry_release, | |
| 680 | + .sysfs_ops = &ip_die_entry_sysfs_ops, | |
| 681 | + .default_groups = ip_die_entry_groups, | |
| 682 | +}; | |
| 683 | + | |
| 684 | +static struct kobj_type die_kobj_ktype = { | |
| 685 | + .release = die_kobj_release, | |
| 686 | + .sysfs_ops = &kobj_sysfs_ops, | |
| 687 | +}; | |
| 688 | + | |
| 689 | +static struct kobj_type ip_discovery_ktype = { | |
| 690 | + .release = ip_disc_release, | |
| 691 | + .sysfs_ops = &kobj_sysfs_ops, | |
| 692 | +}; | |
| 693 | + | |
| 694 | +struct ip_discovery_top { | |
| 695 | + struct kobject kobj; /* ip_discovery/ */ | |
| 696 | + struct kset die_kset; /* ip_discovery/die/, contains ip_die_entry */ | |
| 697 | + struct amdgpu_device *adev; | |
| 698 | +}; | |
| 699 | + | |
| 700 | +static void die_kobj_release(struct kobject *kobj) | |
| 701 | +{ | |
| 702 | + struct ip_discovery_top *ip_top = container_of(to_kset(kobj), | |
| 703 | + struct ip_discovery_top, | |
| 704 | + die_kset); | |
| 705 | + if (!list_empty(&ip_top->die_kset.list)) | |
| 706 | + DRM_ERROR("ip_top->die_kset is not empty"); | |
| 707 | +} | |
| 708 | + | |
| 709 | +static void ip_disc_release(struct kobject *kobj) | |
| 710 | +{ | |
| 711 | + struct ip_discovery_top *ip_top = container_of(kobj, struct ip_discovery_top, | |
| 712 | + kobj); | |
| 713 | + struct amdgpu_device *adev = ip_top->adev; | |
| 714 | + | |
| 715 | + adev->ip_top = NULL; | |
| 716 | + kfree(ip_top); | |
| 717 | +} | |
| 718 | + | |
| 719 | +static int amdgpu_discovery_sysfs_ips(struct amdgpu_device *adev, | |
| 720 | + struct ip_die_entry *ip_die_entry, | |
| 721 | + const size_t _ip_offset, const int num_ips) | |
| 722 | +{ | |
| 723 | + int ii, jj, kk, res; | |
| 724 | + | |
| 725 | + DRM_DEBUG("num_ips:%d", num_ips); | |
| 726 | + | |
| 727 | + /* Find all IPs of a given HW ID, and add their instance to | |
| 728 | + * #die/#hw_id/#instance/<attributes> | |
| 729 | + */ | |
| 730 | + for (ii = 0; ii < HW_ID_MAX; ii++) { | |
| 731 | + struct ip_hw_id *ip_hw_id = NULL; | |
| 732 | + size_t ip_offset = _ip_offset; | |
| 733 | + | |
| 734 | + for (jj = 0; jj < num_ips; jj++) { | |
| 735 | + struct ip *ip; | |
| 736 | + struct ip_hw_instance *ip_hw_instance; | |
| 737 | + | |
| 738 | + ip = (struct ip *)(adev->mman.discovery_bin + ip_offset); | |
| 739 | + if (amdgpu_discovery_validate_ip(ip) || | |
| 740 | + le16_to_cpu(ip->hw_id) != ii) | |
| 741 | + goto next_ip; | |
| 742 | + | |
| 743 | + DRM_DEBUG("match:%d @ ip_offset:%zu", ii, ip_offset); | |
| 744 | + | |
| 745 | + /* We have a hw_id match; register the hw | |
| 746 | + * block if not yet registered. | |
| 747 | + */ | |
| 748 | + if (!ip_hw_id) { | |
| 749 | + ip_hw_id = kzalloc(sizeof(*ip_hw_id), GFP_KERNEL); | |
| 750 | + if (!ip_hw_id) | |
| 751 | + return -ENOMEM; | |
| 752 | + ip_hw_id->hw_id = ii; | |
| 753 | + | |
| 754 | + kobject_set_name(&ip_hw_id->hw_id_kset.kobj, "%d", ii); | |
| 755 | + ip_hw_id->hw_id_kset.kobj.kset = &ip_die_entry->ip_kset; | |
| 756 | + ip_hw_id->hw_id_kset.kobj.ktype = &ip_hw_id_ktype; | |
| 757 | + res = kset_register(&ip_hw_id->hw_id_kset); | |
| 758 | + if (res) { | |
| 759 | + DRM_ERROR("Couldn't register ip_hw_id kset"); | |
| 760 | + kfree(ip_hw_id); | |
| 761 | + return res; | |
| 762 | + } | |
| 763 | + if (hw_id_names[ii]) { | |
| 764 | + res = sysfs_create_link(&ip_die_entry->ip_kset.kobj, | |
| 765 | + &ip_hw_id->hw_id_kset.kobj, | |
| 766 | + hw_id_names[ii]); | |
| 767 | + if (res) { | |
| 768 | + DRM_ERROR("Couldn't create IP link %s in IP Die:%s\n", | |
| 769 | + hw_id_names[ii], | |
| 770 | + kobject_name(&ip_die_entry->ip_kset.kobj)); | |
| 771 | + } | |
| 772 | + } | |
| 773 | + } | |
| 774 | + | |
| 775 | + /* Now register its instance. | |
| 776 | + */ | |
| 777 | + ip_hw_instance = kzalloc(struct_size(ip_hw_instance, | |
| 778 | + base_addr, | |
| 779 | + ip->num_base_address), | |
| 780 | + GFP_KERNEL); | |
| 781 | + if (!ip_hw_instance) { | |
| 782 | + DRM_ERROR("no memory for ip_hw_instance"); | |
| 783 | + return -ENOMEM; | |
| 784 | + } | |
| 785 | + ip_hw_instance->hw_id = le16_to_cpu(ip->hw_id); /* == ii */ | |
| 786 | + ip_hw_instance->num_instance = ip->number_instance; | |
| 787 | + ip_hw_instance->major = ip->major; | |
| 788 | + ip_hw_instance->minor = ip->minor; | |
| 789 | + ip_hw_instance->revision = ip->revision; | |
| 790 | + ip_hw_instance->harvest = ip->harvest; | |
| 791 | + ip_hw_instance->num_base_addresses = ip->num_base_address; | |
| 792 | + | |
| 793 | + for (kk = 0; kk < ip_hw_instance->num_base_addresses; kk++) | |
| 794 | + ip_hw_instance->base_addr[kk] = ip->base_address[kk]; | |
| 795 | + | |
| 796 | + kobject_init(&ip_hw_instance->kobj, &ip_hw_instance_ktype); | |
| 797 | + ip_hw_instance->kobj.kset = &ip_hw_id->hw_id_kset; | |
| 798 | + res = kobject_add(&ip_hw_instance->kobj, NULL, | |
| 799 | + "%d", ip_hw_instance->num_instance); | |
| 800 | +next_ip: | |
| 801 | + ip_offset += sizeof(*ip) + 4 * (ip->num_base_address - 1); | |
| 802 | + } | |
| 803 | + } | |
| 804 | + | |
| 805 | + return 0; | |
| 806 | +} | |
| 807 | + | |
| 808 | +static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev) | |
| 809 | +{ | |
| 810 | + struct binary_header *bhdr; | |
| 811 | + struct ip_discovery_header *ihdr; | |
| 812 | + struct die_header *dhdr; | |
| 813 | + struct kset *die_kset = &adev->ip_top->die_kset; | |
| 814 | + u16 num_dies, die_offset, num_ips; | |
| 815 | + size_t ip_offset; | |
| 816 | + int ii, res; | |
| 817 | + | |
| 818 | + bhdr = (struct binary_header *)adev->mman.discovery_bin; | |
| 819 | + ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin + | |
| 820 | + le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset)); | |
| 821 | + num_dies = le16_to_cpu(ihdr->num_dies); | |
| 822 | + | |
| 823 | + DRM_DEBUG("number of dies: %d\n", num_dies); | |
| 824 | + | |
| 825 | + for (ii = 0; ii < num_dies; ii++) { | |
| 826 | + struct ip_die_entry *ip_die_entry; | |
| 827 | + | |
| 828 | + die_offset = le16_to_cpu(ihdr->die_info[ii].die_offset); | |
| 829 | + dhdr = (struct die_header *)(adev->mman.discovery_bin + die_offset); | |
| 830 | + num_ips = le16_to_cpu(dhdr->num_ips); | |
| 831 | + ip_offset = die_offset + sizeof(*dhdr); | |
| 832 | + | |
| 833 | + /* Add the die to the kset. | |
| 834 | + * | |
| 835 | + * dhdr->die_id == ii, which was checked in | |
| 836 | + * amdgpu_discovery_reg_base_init(). | |
| 837 | + */ | |
| 838 | + | |
| 839 | + ip_die_entry = kzalloc(sizeof(*ip_die_entry), GFP_KERNEL); | |
| 840 | + if (!ip_die_entry) | |
| 841 | + return -ENOMEM; | |
| 842 | + | |
| 843 | + ip_die_entry->num_ips = num_ips; | |
| 844 | + | |
| 845 | + kobject_set_name(&ip_die_entry->ip_kset.kobj, "%d", le16_to_cpu(dhdr->die_id)); | |
| 846 | + ip_die_entry->ip_kset.kobj.kset = die_kset; | |
| 847 | + ip_die_entry->ip_kset.kobj.ktype = &ip_die_entry_ktype; | |
| 848 | + res = kset_register(&ip_die_entry->ip_kset); | |
| 849 | + if (res) { | |
| 850 | + DRM_ERROR("Couldn't register ip_die_entry kset"); | |
| 851 | + kfree(ip_die_entry); | |
| 852 | + return res; | |
| 853 | + } | |
| 854 | + | |
| 855 | + amdgpu_discovery_sysfs_ips(adev, ip_die_entry, ip_offset, num_ips); | |
| 856 | + } | |
| 857 | + | |
| 858 | + return 0; | |
| 859 | +} | |
| 860 | + | |
| 861 | +static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev) | |
| 862 | +{ | |
| 863 | + struct kset *die_kset; | |
| 864 | + int res, ii; | |
| 865 | + | |
| 866 | + adev->ip_top = kzalloc(sizeof(*adev->ip_top), GFP_KERNEL); | |
| 867 | + if (!adev->ip_top) | |
| 868 | + return -ENOMEM; | |
| 869 | + | |
| 870 | + adev->ip_top->adev = adev; | |
| 871 | + | |
| 872 | + res = kobject_init_and_add(&adev->ip_top->kobj, &ip_discovery_ktype, | |
| 873 | + &adev->dev->kobj, "ip_discovery"); | |
| 874 | + if (res) { | |
| 875 | + DRM_ERROR("Couldn't init and add ip_discovery/"); | |
| 876 | + goto Err; | |
| 877 | + } | |
| 878 | + | |
| 879 | + die_kset = &adev->ip_top->die_kset; | |
| 880 | + kobject_set_name(&die_kset->kobj, "%s", "die"); | |
| 881 | + die_kset->kobj.parent = &adev->ip_top->kobj; | |
| 882 | + die_kset->kobj.ktype = &die_kobj_ktype; | |
| 883 | + res = kset_register(&adev->ip_top->die_kset); | |
| 884 | + if (res) { | |
| 885 | + DRM_ERROR("Couldn't register die_kset"); | |
| 886 | + goto Err; | |
| 887 | + } | |
| 888 | + | |
| 889 | + for (ii = 0; ii < ARRAY_SIZE(ip_hw_attr); ii++) | |
| 890 | + ip_hw_instance_attrs[ii] = &ip_hw_attr[ii].attr; | |
| 891 | + ip_hw_instance_attrs[ii] = NULL; | |
| 892 | + | |
| 893 | + res = amdgpu_discovery_sysfs_recurse(adev); | |
| 894 | + | |
| 895 | + return res; | |
| 896 | +Err: | |
| 897 | + kobject_put(&adev->ip_top->kobj); | |
| 898 | + return res; | |
| 899 | +} | |
| 900 | + | |
| 901 | +/* -------------------------------------------------- */ | |
| 902 | + | |
| 903 | +#define list_to_kobj(el) container_of(el, struct kobject, entry) | |
| 904 | + | |
| 905 | +static void amdgpu_discovery_sysfs_ip_hw_free(struct ip_hw_id *ip_hw_id) | |
| 906 | +{ | |
| 907 | + struct list_head *el, *tmp; | |
| 908 | + struct kset *hw_id_kset; | |
| 909 | + | |
| 910 | + hw_id_kset = &ip_hw_id->hw_id_kset; | |
| 911 | + spin_lock(&hw_id_kset->list_lock); | |
| 912 | + list_for_each_prev_safe(el, tmp, &hw_id_kset->list) { | |
| 913 | + list_del_init(el); | |
| 914 | + spin_unlock(&hw_id_kset->list_lock); | |
| 915 | + /* kobject is embedded in ip_hw_instance */ | |
| 916 | + kobject_put(list_to_kobj(el)); | |
| 917 | + spin_lock(&hw_id_kset->list_lock); | |
| 918 | + } | |
| 919 | + spin_unlock(&hw_id_kset->list_lock); | |
| 920 | + kobject_put(&ip_hw_id->hw_id_kset.kobj); | |
| 921 | +} | |
| 922 | + | |
| 923 | +static void amdgpu_discovery_sysfs_die_free(struct ip_die_entry *ip_die_entry) | |
| 924 | +{ | |
| 925 | + struct list_head *el, *tmp; | |
| 926 | + struct kset *ip_kset; | |
| 927 | + | |
| 928 | + ip_kset = &ip_die_entry->ip_kset; | |
| 929 | + spin_lock(&ip_kset->list_lock); | |
| 930 | + list_for_each_prev_safe(el, tmp, &ip_kset->list) { | |
| 931 | + list_del_init(el); | |
| 932 | + spin_unlock(&ip_kset->list_lock); | |
| 933 | + amdgpu_discovery_sysfs_ip_hw_free(to_ip_hw_id(list_to_kobj(el))); | |
| 934 | + spin_lock(&ip_kset->list_lock); | |
| 935 | + } | |
| 936 | + spin_unlock(&ip_kset->list_lock); | |
| 937 | + kobject_put(&ip_die_entry->ip_kset.kobj); | |
| 938 | +} | |
| 939 | + | |
| 940 | +static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev) | |
| 941 | +{ | |
| 942 | + struct list_head *el, *tmp; | |
| 943 | + struct kset *die_kset; | |
| 944 | + | |
| 945 | + die_kset = &adev->ip_top->die_kset; | |
| 946 | + spin_lock(&die_kset->list_lock); | |
| 947 | + list_for_each_prev_safe(el, tmp, &die_kset->list) { | |
| 948 | + list_del_init(el); | |
| 949 | + spin_unlock(&die_kset->list_lock); | |
| 950 | + amdgpu_discovery_sysfs_die_free(to_ip_die_entry(list_to_kobj(el))); | |
| 951 | + spin_lock(&die_kset->list_lock); | |
| 952 | + } | |
| 953 | + spin_unlock(&die_kset->list_lock); | |
| 954 | + kobject_put(&adev->ip_top->die_kset.kobj); | |
| 955 | + kobject_put(&adev->ip_top->kobj); | |
| 956 | +} | |
| 957 | + | |
| 958 | +/* ================================================== */ | |
| 959 | + | |
| 385 | 960 | int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev) |
| 386 | 961 | { |
| 387 | 962 | struct binary_header *bhdr; |
| @@ -492,6 +1067,8 @@ next_ip: · 104 unmodified lines | ||
| 492 | 1067 | } |
| 493 | 1068 | } |
| 494 | 1069 | |
| 1070 | + amdgpu_discovery_sysfs_init(adev); | |
| 1071 | + | |
| 495 | 1072 | return 0; |
| 496 | 1073 | } |
| 497 | 1074 | |
| @@ -545,32 +1122,26 @@ int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, int n · 47 unmodified lines | ||
| 545 | 1122 | |
| 546 | 1123 | void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev) |
| 547 | 1124 | { |
| 548 | − struct binary_header *bhdr; | |
| 549 | − struct harvest_table *harvest_info; | |
| 550 | − int i, vcn_harvest_count = 0; | |
| 551 | − | |
| 552 | − bhdr = (struct binary_header *)adev->mman.discovery_bin; | |
| 553 | − harvest_info = (struct harvest_table *)(adev->mman.discovery_bin + | |
| 554 | − le16_to_cpu(bhdr->table_list[HARVEST_INFO].offset)); | |
| 555 | − | |
| 556 | − for (i = 0; i < 32; i++) { | |
| 557 | − if (le16_to_cpu(harvest_info->list[i].hw_id) == 0) | |
| 558 | − break; | |
| 1125 | + int vcn_harvest_count = 0; | |
| 559 | 1126 | |
| 560 | − switch (le16_to_cpu(harvest_info->list[i].hw_id)) { | |
| 561 | − case VCN_HWID: | |
| 562 | − vcn_harvest_count++; | |
| 563 | − if (harvest_info->list[i].number_instance == 0) | |
| 564 | − adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN0; | |
| 565 | − else | |
| 566 | − adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN1; | |
| 567 | − break; | |
| 568 | − case DMU_HWID: | |
| 569 | − adev->harvest_ip_mask |= AMD_HARVEST_IP_DMU_MASK; | |
| 570 | − break; | |
| 571 | − default: | |
| 572 | − break; | |
| 573 | − } | |
| 1127 | + /* | |
| 1128 | + * Harvest table does not fit Navi1x and legacy GPUs, | |
| 1129 | + * so read harvest bit per IP data structure to set | |
| 1130 | + * harvest configuration. | |
| 1131 | + */ | |
| 1132 | + if (adev->ip_versions[GC_HWIP][0] < IP_VERSION(10, 2, 0)) { | |
| 1133 | + if ((adev->pdev->device == 0x731E && | |
| 1134 | + (adev->pdev->revision == 0xC6 || | |
| 1135 | + adev->pdev->revision == 0xC7)) || | |
| 1136 | + (adev->pdev->device == 0x7340 && | |
| 1137 | + adev->pdev->revision == 0xC9) || | |
| 1138 | + (adev->pdev->device == 0x7360 && | |
| 1139 | + adev->pdev->revision == 0xC7)) | |
| 1140 | + amdgpu_discovery_read_harvest_bit_per_ip(adev, | |
| 1141 | + &vcn_harvest_count); | |
| 1142 | + } else { | |
| 1143 | + amdgpu_discovery_read_from_harvest_table(adev, | |
| 1144 | + &vcn_harvest_count); | |
| 574 | 1145 | } |
| 575 | 1146 | |
| 576 | 1147 | amdgpu_discovery_harvest_config_quirk(adev); |
| @@ -674,12 +1245,15 @@ static int amdgpu_discovery_set_common_ip_blocks(struct amdgpu_device *adev) · 97 unmodified lines | ||
| 674 | 1245 | case IP_VERSION(10, 1, 1): |
| 675 | 1246 | case IP_VERSION(10, 1, 2): |
| 676 | 1247 | case IP_VERSION(10, 1, 3): |
| 1248 | + case IP_VERSION(10, 1, 4): | |
| 677 | 1249 | case IP_VERSION(10, 3, 0): |
| 678 | 1250 | case IP_VERSION(10, 3, 1): |
| 679 | 1251 | case IP_VERSION(10, 3, 2): |
| 680 | 1252 | case IP_VERSION(10, 3, 3): |
| 681 | 1253 | case IP_VERSION(10, 3, 4): |
| 682 | 1254 | case IP_VERSION(10, 3, 5): |
| 1255 | + case IP_VERSION(10, 3, 6): | |
| 1256 | + case IP_VERSION(10, 3, 7): | |
| 683 | 1257 | amdgpu_device_ip_block_add(adev, &nv_common_ip_block); |
| 684 | 1258 | break; |
| 685 | 1259 | default: |
| @@ -709,12 +1283,15 @@ static int amdgpu_discovery_set_gmc_ip_blocks(struct amdgpu_device *adev) · 23 unmodified lines | ||
| 709 | 1283 | case IP_VERSION(10, 1, 1): |
| 710 | 1284 | case IP_VERSION(10, 1, 2): |
| 711 | 1285 | case IP_VERSION(10, 1, 3): |
| 1286 | + case IP_VERSION(10, 1, 4): | |
| 712 | 1287 | case IP_VERSION(10, 3, 0): |
| 713 | 1288 | case IP_VERSION(10, 3, 1): |
| 714 | 1289 | case IP_VERSION(10, 3, 2): |
| 715 | 1290 | case IP_VERSION(10, 3, 3): |
| 716 | 1291 | case IP_VERSION(10, 3, 4): |
| 717 | 1292 | case IP_VERSION(10, 3, 5): |
| 1293 | + case IP_VERSION(10, 3, 6): | |
| 1294 | + case IP_VERSION(10, 3, 7): | |
| 718 | 1295 | amdgpu_device_ip_block_add(adev, &gmc_v10_0_ip_block); |
| 719 | 1296 | break; |
| 720 | 1297 | default: |
| @@ -790,6 +1367,8 @@ static int amdgpu_discovery_set_psp_ip_blocks(struct amdgpu_device *adev) · 69 unmodified lines | ||
| 790 | 1367 | case IP_VERSION(13, 0, 1): |
| 791 | 1368 | case IP_VERSION(13, 0, 2): |
| 792 | 1369 | case IP_VERSION(13, 0, 3): |
| 1370 | + case IP_VERSION(13, 0, 5): | |
| 1371 | + case IP_VERSION(13, 0, 8): | |
| 793 | 1372 | amdgpu_device_ip_block_add(adev, &psp_v13_0_ip_block); |
| 794 | 1373 | break; |
| 795 | 1374 | default: |
| @@ -831,6 +1410,8 @@ static int amdgpu_discovery_set_smu_ip_blocks(struct amdgpu_device *adev) · 35 unmodified lines | ||
| 831 | 1410 | case IP_VERSION(13, 0, 1): |
| 832 | 1411 | case IP_VERSION(13, 0, 2): |
| 833 | 1412 | case IP_VERSION(13, 0, 3): |
| 1413 | + case IP_VERSION(13, 0, 5): | |
| 1414 | + case IP_VERSION(13, 0, 8): | |
| 834 | 1415 | amdgpu_device_ip_block_add(adev, &smu_v13_0_ip_block); |
| 835 | 1416 | break; |
| 836 | 1417 | default: |
| @@ -846,8 +1427,14 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev) · 9 unmodified lines | ||
| 846 | 1427 | { |
| 847 | 1428 | if (adev->enable_virtual_display || amdgpu_sriov_vf(adev)) { |
| 848 | 1429 | amdgpu_device_ip_block_add(adev, &amdgpu_vkms_ip_block); |
| 1430 | + return 0; | |
| 1431 | + } | |
| 1432 | + | |
| 1433 | + if (!amdgpu_device_has_dc_support(adev)) | |
| 1434 | + return 0; | |
| 1435 | + | |
| 849 | 1436 | #if defined(CONFIG_DRM_AMD_DC) |
| 850 | − } else if (adev->ip_versions[DCE_HWIP][0]) { | |
| 1437 | + if (adev->ip_versions[DCE_HWIP][0]) { | |
| 851 | 1438 | switch (adev->ip_versions[DCE_HWIP][0]) { |
| 852 | 1439 | case IP_VERSION(1, 0, 0): |
| 853 | 1440 | case IP_VERSION(1, 0, 1): |
| @@ -861,6 +1448,8 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev) · 7 unmodified lines | ||
| 861 | 1448 | case IP_VERSION(3, 0, 1): |
| 862 | 1449 | case IP_VERSION(3, 1, 2): |
| 863 | 1450 | case IP_VERSION(3, 1, 3): |
| 1451 | + case IP_VERSION(3, 1, 5): | |
| 1452 | + case IP_VERSION(3, 1, 6): | |
| 864 | 1453 | amdgpu_device_ip_block_add(adev, &dm_ip_block); |
| 865 | 1454 | break; |
| 866 | 1455 | default: |
| @@ -882,8 +1471,8 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev) · 15 unmodified lines | ||
| 882 | 1471 | adev->ip_versions[DCI_HWIP][0]); |
| 883 | 1472 | return -EINVAL; |
| 884 | 1473 | } |
| 885 | −#endif | |
| 886 | 1474 | } |
| 1475 | +#endif | |
| 887 | 1476 | return 0; |
| 888 | 1477 | } |
| 889 | 1478 | |
| @@ -904,12 +1493,15 @@ static int amdgpu_discovery_set_gc_ip_blocks(struct amdgpu_device *adev) · 14 unmodified lines | ||
| 904 | 1493 | case IP_VERSION(10, 1, 2): |
| 905 | 1494 | case IP_VERSION(10, 1, 1): |
| 906 | 1495 | case IP_VERSION(10, 1, 3): |
| 1496 | + case IP_VERSION(10, 1, 4): | |
| 907 | 1497 | case IP_VERSION(10, 3, 0): |
| 908 | 1498 | case IP_VERSION(10, 3, 2): |
| 909 | 1499 | case IP_VERSION(10, 3, 1): |
| 910 | 1500 | case IP_VERSION(10, 3, 4): |
| 911 | 1501 | case IP_VERSION(10, 3, 5): |
| 1502 | + case IP_VERSION(10, 3, 6): | |
| 912 | 1503 | case IP_VERSION(10, 3, 3): |
| 1504 | + case IP_VERSION(10, 3, 7): | |
| 913 | 1505 | amdgpu_device_ip_block_add(adev, &gfx_v10_0_ip_block); |
| 914 | 1506 | break; |
| 915 | 1507 | default: |
| @@ -944,8 +1536,10 @@ static int amdgpu_discovery_set_sdma_ip_blocks(struct amdgpu_device *adev) · 28 unmodified lines | ||
| 944 | 1536 | case IP_VERSION(5, 2, 2): |
| 945 | 1537 | case IP_VERSION(5, 2, 4): |
| 946 | 1538 | case IP_VERSION(5, 2, 5): |
| 1539 | + case IP_VERSION(5, 2, 6): | |
| 947 | 1540 | case IP_VERSION(5, 2, 3): |
| 948 | 1541 | case IP_VERSION(5, 2, 1): |
| 1542 | + case IP_VERSION(5, 2, 7): | |
| 949 | 1543 | amdgpu_device_ip_block_add(adev, &sdma_v5_2_ip_block); |
| 950 | 1544 | break; |
| 951 | 1545 | default: |
| @@ -1012,6 +1606,7 @@ static int amdgpu_discovery_set_mm_ip_blocks(struct amdgpu_device *adev) · 60 unmodified lines | ||
| 1012 | 1606 | case IP_VERSION(3, 0, 0): |
| 1013 | 1607 | case IP_VERSION(3, 0, 16): |
| 1014 | 1608 | case IP_VERSION(3, 1, 1): |
| 1609 | + case IP_VERSION(3, 1, 2): | |
| 1015 | 1610 | case IP_VERSION(3, 0, 2): |
| 1016 | 1611 | case IP_VERSION(3, 0, 192): |
| 1017 | 1612 | amdgpu_device_ip_block_add(adev, &vcn_v3_0_ip_block); |
| @@ -1038,12 +1633,14 @@ static int amdgpu_discovery_set_mes_ip_blocks(struct amdgpu_device *adev) · 20 unmodified lines | ||
| 1038 | 1633 | case IP_VERSION(10, 1, 1): |
| 1039 | 1634 | case IP_VERSION(10, 1, 2): |
| 1040 | 1635 | case IP_VERSION(10, 1, 3): |
| 1636 | + case IP_VERSION(10, 1, 4): | |
| 1041 | 1637 | case IP_VERSION(10, 3, 0): |
| 1042 | 1638 | case IP_VERSION(10, 3, 1): |
| 1043 | 1639 | case IP_VERSION(10, 3, 2): |
| 1044 | 1640 | case IP_VERSION(10, 3, 3): |
| 1045 | 1641 | case IP_VERSION(10, 3, 4): |
| 1046 | 1642 | case IP_VERSION(10, 3, 5): |
| 1643 | + case IP_VERSION(10, 3, 6): | |
| 1047 | 1644 | amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block); |
| 1048 | 1645 | break; |
| 1049 | 1646 | default: |
| @@ -1217,11 +1814,6 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 167 unmodified lines | ||
| 1217 | 1814 | return -EINVAL; |
| 1218 | 1815 | |
| 1219 | 1816 | amdgpu_discovery_harvest_ip(adev); |
| 1220 | − | |
| 1221 | − if (!adev->mman.discovery_bin) { | |
| 1222 | − DRM_ERROR("ip discovery uninitialized\n"); | |
| 1223 | − return -EINVAL; | |
| 1224 | − } | |
| 1225 | 1817 | break; |
| 1226 | 1818 | } |
| 1227 | 1819 | |
| @@ -1242,6 +1834,7 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 14 unmodified lines | ||
| 1242 | 1834 | case IP_VERSION(10, 1, 1): |
| 1243 | 1835 | case IP_VERSION(10, 1, 2): |
| 1244 | 1836 | case IP_VERSION(10, 1, 3): |
| 1837 | + case IP_VERSION(10, 1, 4): | |
| 1245 | 1838 | case IP_VERSION(10, 3, 0): |
| 1246 | 1839 | case IP_VERSION(10, 3, 2): |
| 1247 | 1840 | case IP_VERSION(10, 3, 4): |
| @@ -1254,10 +1847,32 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 6 unmodified lines | ||
| 1254 | 1847 | case IP_VERSION(10, 3, 3): |
| 1255 | 1848 | adev->family = AMDGPU_FAMILY_YC; |
| 1256 | 1849 | break; |
| 1850 | + case IP_VERSION(10, 3, 6): | |
| 1851 | + adev->family = AMDGPU_FAMILY_GC_10_3_6; | |
| 1852 | + break; | |
| 1853 | + case IP_VERSION(10, 3, 7): | |
| 1854 | + adev->family = AMDGPU_FAMILY_GC_10_3_7; | |
| 1855 | + break; | |
| 1257 | 1856 | default: |
| 1258 | 1857 | return -EINVAL; |
| 1259 | 1858 | } |
| 1260 | 1859 | |
| 1860 | + switch (adev->ip_versions[GC_HWIP][0]) { | |
| 1861 | + case IP_VERSION(9, 1, 0): | |
| 1862 | + case IP_VERSION(9, 2, 2): | |
| 1863 | + case IP_VERSION(9, 3, 0): | |
| 1864 | + case IP_VERSION(10, 1, 3): | |
| 1865 | + case IP_VERSION(10, 1, 4): | |
| 1866 | + case IP_VERSION(10, 3, 1): | |
| 1867 | + case IP_VERSION(10, 3, 3): | |
| 1868 | + case IP_VERSION(10, 3, 6): | |
| 1869 | + case IP_VERSION(10, 3, 7): | |
| 1870 | + adev->flags |= AMD_IS_APU; | |
| 1871 | + break; | |
| 1872 | + default: | |
| 1873 | + break; | |
| 1874 | + } | |
| 1875 | + | |
| 1261 | 1876 | if (adev->ip_versions[XGMI_HWIP][0] == IP_VERSION(4, 8, 0)) |
| 1262 | 1877 | adev->gmc.xgmi.supported = true; |
| 1263 | 1878 | |
| @@ -1285,7 +1900,9 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 21 unmodified lines | ||
| 1285 | 1900 | break; |
| 1286 | 1901 | case IP_VERSION(7, 2, 0): |
| 1287 | 1902 | case IP_VERSION(7, 2, 1): |
| 1903 | + case IP_VERSION(7, 3, 0): | |
| 1288 | 1904 | case IP_VERSION(7, 5, 0): |
| 1905 | + case IP_VERSION(7, 5, 1): | |
| 1289 | 1906 | adev->nbio.funcs = &nbio_v7_2_funcs; |
| 1290 | 1907 | adev->nbio.hdp_flush_reg = &nbio_v7_2_hdp_flush_reg; |
| 1291 | 1908 | break; |
| @@ -1368,6 +1985,8 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 76 unmodified lines | ||
| 1368 | 1985 | case IP_VERSION(11, 0, 11): |
| 1369 | 1986 | case IP_VERSION(11, 5, 0): |
| 1370 | 1987 | case IP_VERSION(13, 0, 1): |
| 1988 | + case IP_VERSION(13, 0, 9): | |
| 1989 | + case IP_VERSION(13, 0, 10): | |
| 1371 | 1990 | adev->smuio.funcs = &smuio_v11_0_6_funcs; |
| 1372 | 1991 | break; |
| 1373 | 1992 | case IP_VERSION(13, 0, 2): |
| @@ -506,28 +506,9 @@ uint32_t amdgpu_display_supported_domains(struct amdgpu_device *adev, · 505 unmodified lines | ||
| 506 | 506 | */ |
| 507 | 507 | if ((bo_flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC) && |
| 508 | 508 | amdgpu_bo_support_uswc(bo_flags) && |
| 509 | − amdgpu_device_asic_has_dc_support(adev->asic_type)) { | |
| 510 | − switch (adev->asic_type) { | |
| 511 | − case CHIP_CARRIZO: | |
| 512 | − case CHIP_STONEY: | |
| 513 | − domain |= AMDGPU_GEM_DOMAIN_GTT; | |
| 514 | − break; | |
| 515 | − case CHIP_RAVEN: | |
| 516 | − /* enable S/G on PCO and RV2 */ | |
| 517 | − if ((adev->apu_flags & AMD_APU_IS_RAVEN2) || | |
| 518 | − (adev->apu_flags & AMD_APU_IS_PICASSO)) | |
| 519 | − domain |= AMDGPU_GEM_DOMAIN_GTT; | |
| 520 | − break; | |
| 521 | − case CHIP_RENOIR: | |
| 522 | − case CHIP_VANGOGH: | |
| 523 | − case CHIP_YELLOW_CARP: | |
| 524 | − domain |= AMDGPU_GEM_DOMAIN_GTT; | |
| 525 | − break; | |
| 526 | − | |
| 527 | − default: | |
| 528 | − break; | |
| 529 | − } | |
| 530 | − } | |
| 509 | + amdgpu_device_asic_has_dc_support(adev->asic_type) && | |
| 510 | + adev->mode_info.gpu_vm_support) | |
| 511 | + domain |= AMDGPU_GEM_DOMAIN_GTT; | |
| 531 | 512 | #endif |
| 532 | 513 | |
| 533 | 514 | return domain; |
| @@ -956,7 +937,7 @@ static int amdgpu_display_verify_sizes(struct amdgpu_framebuffer *rfb) · 422 unmodified lines | ||
| 956 | 937 | int ret; |
| 957 | 938 | unsigned int i, block_width, block_height, block_size_log2; |
| 958 | 939 | |
| 959 | − if (!rfb->base.dev->mode_config.allow_fb_modifiers) | |
| 940 | + if (rfb->base.dev->mode_config.fb_modifiers_not_supported) | |
| 960 | 941 | return 0; |
| 961 | 942 | |
| 962 | 943 | for (i = 0; i < format_info->num_planes; ++i) { |
| @@ -1143,7 +1124,7 @@ int amdgpu_display_framebuffer_init(struct drm_device *dev, · 180 unmodified lines | ||
| 1143 | 1124 | if (ret) |
| 1144 | 1125 | return ret; |
| 1145 | 1126 | |
| 1146 | − if (!dev->mode_config.allow_fb_modifiers) { | |
| 1127 | + if (dev->mode_config.fb_modifiers_not_supported && !adev->enable_virtual_display) { | |
| 1147 | 1128 | drm_WARN_ONCE(dev, adev->family >= AMDGPU_FAMILY_AI, |
| 1148 | 1129 | "GFX9+ requires FB check based on format modifier\n"); |
| 1149 | 1130 | ret = check_tiling_flags_gfx6(rfb); |
| @@ -1151,7 +1132,7 @@ int amdgpu_display_framebuffer_init(struct drm_device *dev, · 1 unmodified line | ||
| 1151 | 1132 | return ret; |
| 1152 | 1133 | } |
| 1153 | 1134 | |
| 1154 | − if (dev->mode_config.allow_fb_modifiers && | |
| 1135 | + if (!dev->mode_config.fb_modifiers_not_supported && | |
| 1155 | 1136 | !(rfb->base.flags & DRM_MODE_FB_MODIFIERS)) { |
| 1156 | 1137 | ret = convert_tiling_flags_to_modifier(rfb); |
| 1157 | 1138 | if (ret) { |
| @@ -99,9 +99,10 @@ · 98 unmodified lines | ||
| 99 | 99 | * - 3.42.0 - Add 16bpc fixed point display support |
| 100 | 100 | * - 3.43.0 - Add device hot plug/unplug support |
| 101 | 101 | * - 3.44.0 - DCN3 supports DCC independent block settings: !64B && 128B, 64B && 128B |
| 102 | + * - 3.45.0 - Add context ioctl stable pstate interface | |
| 102 | 103 | */ |
| 103 | 104 | #define KMS_DRIVER_MAJOR 3 |
| 104 | −#define KMS_DRIVER_MINOR 44 | |
| 105 | +#define KMS_DRIVER_MINOR 45 | |
| 105 | 106 | #define KMS_DRIVER_PATCHLEVEL 0 |
| 106 | 107 | |
| 107 | 108 | int amdgpu_vram_limit; |
| @@ -109,8 +110,6 @@ int amdgpu_vis_vram_limit; · 1 unmodified line | ||
| 109 | 110 | int amdgpu_gart_size = -1; /* auto */ |
| 110 | 111 | int amdgpu_gtt_size = -1; /* auto */ |
| 111 | 112 | int amdgpu_moverate = -1; /* auto */ |
| 112 | −int amdgpu_benchmarking; | |
| 113 | −int amdgpu_testing; | |
| 114 | 113 | int amdgpu_audio = -1; |
| 115 | 114 | int amdgpu_disp_priority; |
| 116 | 115 | int amdgpu_hw_i2c; |
| @@ -174,10 +173,10 @@ int amdgpu_mes; · 57 unmodified lines | ||
| 174 | 173 | int amdgpu_noretry = -1; |
| 175 | 174 | int amdgpu_force_asic_type = -1; |
| 176 | 175 | int amdgpu_tmz = -1; /* auto */ |
| 177 | −uint amdgpu_freesync_vid_mode; | |
| 178 | 176 | int amdgpu_reset_method = -1; /* auto */ |
| 179 | 177 | int amdgpu_num_kcq = -1; |
| 180 | 178 | int amdgpu_smartshift_bias; |
| 179 | +int amdgpu_use_xgmi_p2p = 1; | |
| 181 | 180 | |
| 182 | 181 | static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work); |
| 183 | 182 | |
| @@ -231,20 +230,6 @@ module_param_named(gttsize, amdgpu_gtt_size, int, 0600); · 47 unmodified lines | ||
| 231 | 230 | MODULE_PARM_DESC(moverate, "Maximum buffer migration rate in MB/s. (32, 64, etc., -1=auto, 0=1=disabled)"); |
| 232 | 231 | module_param_named(moverate, amdgpu_moverate, int, 0600); |
| 233 | 232 | |
| 234 | −/** | |
| 235 | − * DOC: benchmark (int) | |
| 236 | − * Run benchmarks. The default is 0 (Skip benchmarks). | |
| 237 | − */ | |
| 238 | −MODULE_PARM_DESC(benchmark, "Run benchmark"); | |
| 239 | −module_param_named(benchmark, amdgpu_benchmarking, int, 0444); | |
| 240 | − | |
| 241 | −/** | |
| 242 | − * DOC: test (int) | |
| 243 | − * Test BO GTT->VRAM and VRAM->GTT GPU copies. The default is 0 (Skip test, only set 1 to run test). | |
| 244 | − */ | |
| 245 | −MODULE_PARM_DESC(test, "Run tests"); | |
| 246 | −module_param_named(test, amdgpu_testing, int, 0444); | |
| 247 | − | |
| 248 | 233 | /** |
| 249 | 234 | * DOC: audio (int) |
| 250 | 235 | * Set HDMI/DPAudio. Only affects non-DC display handling. The default is -1 (Enabled), set 0 to disabled it. |
| @@ -667,6 +652,13 @@ MODULE_PARM_DESC(force_asic_type, · 416 unmodified lines | ||
| 667 | 652 | "A non negative value used to specify the asic type for all supported GPUs"); |
| 668 | 653 | module_param_named(force_asic_type, amdgpu_force_asic_type, int, 0444); |
| 669 | 654 | |
| 655 | +/** | |
| 656 | + * DOC: use_xgmi_p2p (int) | |
| 657 | + * Enables/disables XGMI P2P interface (0 = disable, 1 = enable). | |
| 658 | + */ | |
| 659 | +MODULE_PARM_DESC(use_xgmi_p2p, | |
| 660 | + "Enable XGMI P2P interface (0 = disable; 1 = enable (default))"); | |
| 661 | +module_param_named(use_xgmi_p2p, amdgpu_use_xgmi_p2p, int, 0444); | |
| 670 | 662 | |
| 671 | 663 | |
| 672 | 664 | #ifdef CONFIG_HSA_AMD |
| @@ -740,7 +732,7 @@ MODULE_PARM_DESC(debug_largebar, · 67 unmodified lines | ||
| 740 | 732 | * systems with a broken CRAT table. |
| 741 | 733 | * |
| 742 | 734 | * Default is auto (according to asic type, iommu_v2, and crat table, to decide |
| 743 | − * whehter use CRAT) | |
| 735 | + * whether use CRAT) | |
| 744 | 736 | */ |
| 745 | 737 | int ignore_crat; |
| 746 | 738 | module_param(ignore_crat, int, 0444); |
| @@ -843,37 +835,11 @@ module_param_named(backlight, amdgpu_backlight, bint, 0444); · 96 unmodified lines | ||
| 843 | 835 | MODULE_PARM_DESC(tmz, "Enable TMZ feature (-1 = auto (default), 0 = off, 1 = on)"); |
| 844 | 836 | module_param_named(tmz, amdgpu_tmz, int, 0444); |
| 845 | 837 | |
| 846 | −/** | |
| 847 | − * DOC: freesync_video (uint) | |
| 848 | − * Enable the optimization to adjust front porch timing to achieve seamless | |
| 849 | − * mode change experience when setting a freesync supported mode for which full | |
| 850 | − * modeset is not needed. | |
| 851 | − * | |
| 852 | − * The Display Core will add a set of modes derived from the base FreeSync | |
| 853 | − * video mode into the corresponding connector's mode list based on commonly | |
| 854 | − * used refresh rates and VRR range of the connected display, when users enable | |
| 855 | − * this feature. From the userspace perspective, they can see a seamless mode | |
| 856 | − * change experience when the change between different refresh rates under the | |
| 857 | − * same resolution. Additionally, userspace applications such as Video playback | |
| 858 | − * can read this modeset list and change the refresh rate based on the video | |
| 859 | − * frame rate. Finally, the userspace can also derive an appropriate mode for a | |
| 860 | − * particular refresh rate based on the FreeSync Mode and add it to the | |
| 861 | − * connector's mode list. | |
| 862 | − * | |
| 863 | − * Note: This is an experimental feature. | |
| 864 | − * | |
| 865 | − * The default value: 0 (off). | |
| 866 | − */ | |
| 867 | −MODULE_PARM_DESC( | |
| 868 | − freesync_video, | |
| 869 | − "Enable freesync modesetting optimization feature (0 = off (default), 1 = on)"); | |
| 870 | −module_param_named(freesync_video, amdgpu_freesync_vid_mode, uint, 0444); | |
| 871 | − | |
| 872 | 838 | /** |
| 873 | 839 | * DOC: reset_method (int) |
| 874 | − * GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco, 5 = pci) | |
| 840 | + * GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco) | |
| 875 | 841 | */ |
| 876 | −MODULE_PARM_DESC(reset_method, "GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco/bamaco, 5 = pci)"); | |
| 842 | +MODULE_PARM_DESC(reset_method, "GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco/bamaco)"); | |
| 877 | 843 | module_param_named(reset_method, amdgpu_reset_method, int, 0444); |
| 878 | 844 | |
| 879 | 845 | /** |
| @@ -1942,13 +1908,14 @@ static const struct pci_device_id pciidlist[] = { · 1062 unmodified lines | ||
| 1942 | 1908 | {0x1002, 0x73FF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_DIMGREY_CAVEFISH}, |
| 1943 | 1909 | |
| 1944 | 1910 | /* Aldebaran */ |
| 1945 | − {0x1002, 0x7408, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT}, | |
| 1946 | − {0x1002, 0x740C, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT}, | |
| 1947 | − {0x1002, 0x740F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT}, | |
| 1948 | − {0x1002, 0x7410, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT}, | |
| 1911 | + {0x1002, 0x7408, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN}, | |
| 1912 | + {0x1002, 0x740C, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN}, | |
| 1913 | + {0x1002, 0x740F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN}, | |
| 1914 | + {0x1002, 0x7410, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN}, | |
| 1949 | 1915 | |
| 1950 | 1916 | /* CYAN_SKILLFISH */ |
| 1951 | 1917 | {0x1002, 0x13FE, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_CYAN_SKILLFISH|AMD_IS_APU}, |
| 1918 | + {0x1002, 0x143F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_CYAN_SKILLFISH|AMD_IS_APU}, | |
| 1952 | 1919 | |
| 1953 | 1920 | /* BEIGE_GOBY */ |
| 1954 | 1921 | {0x1002, 0x7420, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_BEIGE_GOBY}, |
| @@ -1994,6 +1961,28 @@ static bool amdgpu_is_fw_framebuffer(resource_size_t base, · 39 unmodified lines | ||
| 1994 | 1961 | return found; |
| 1995 | 1962 | } |
| 1996 | 1963 | |
| 1964 | +static void amdgpu_get_secondary_funcs(struct amdgpu_device *adev) | |
| 1965 | +{ | |
| 1966 | + struct pci_dev *p = NULL; | |
| 1967 | + int i; | |
| 1968 | + | |
| 1969 | + /* 0 - GPU | |
| 1970 | + * 1 - audio | |
| 1971 | + * 2 - USB | |
| 1972 | + * 3 - UCSI | |
| 1973 | + */ | |
| 1974 | + for (i = 1; i < 4; i++) { | |
| 1975 | + p = pci_get_domain_bus_and_slot(pci_domain_nr(adev->pdev->bus), | |
| 1976 | + adev->pdev->bus->number, i); | |
| 1977 | + if (p) { | |
| 1978 | + pm_runtime_get_sync(&p->dev); | |
| 1979 | + pm_runtime_mark_last_busy(&p->dev); | |
| 1980 | + pm_runtime_put_autosuspend(&p->dev); | |
| 1981 | + pci_dev_put(p); | |
| 1982 | + } | |
| 1983 | + } | |
| 1984 | +} | |
| 1985 | + | |
| 1997 | 1986 | static int amdgpu_pci_probe(struct pci_dev *pdev, |
| 1998 | 1987 | const struct pci_device_id *ent) |
| 1999 | 1988 | { |
| @@ -2011,6 +2000,9 @@ static int amdgpu_pci_probe(struct pci_dev *pdev, · 11 unmodified lines | ||
| 2011 | 2000 | return -ENODEV; |
| 2012 | 2001 | } |
| 2013 | 2002 | |
| 2003 | + if (amdgpu_aspm == -1 && !pcie_aspm_enabled(pdev)) | |
| 2004 | + amdgpu_aspm = 0; | |
| 2005 | + | |
| 2014 | 2006 | if (amdgpu_virtual_display || |
| 2015 | 2007 | amdgpu_device_asic_has_dc_support(flags & AMD_ASIC_MASK)) |
| 2016 | 2008 | supports_atomic = true; |
| @@ -2126,6 +2118,48 @@ retry_init: · 109 unmodified lines | ||
| 2126 | 2118 | if (ret) |
| 2127 | 2119 | DRM_ERROR("Creating debugfs files failed (%d).\n", ret); |
| 2128 | 2120 | |
| 2121 | + if (adev->runpm) { | |
| 2122 | + /* only need to skip on ATPX */ | |
| 2123 | + if (amdgpu_device_supports_px(ddev)) | |
| 2124 | + dev_pm_set_driver_flags(ddev->dev, DPM_FLAG_NO_DIRECT_COMPLETE); | |
| 2125 | + /* we want direct complete for BOCO */ | |
| 2126 | + if (amdgpu_device_supports_boco(ddev)) | |
| 2127 | + dev_pm_set_driver_flags(ddev->dev, DPM_FLAG_SMART_PREPARE | | |
| 2128 | + DPM_FLAG_SMART_SUSPEND | | |
| 2129 | + DPM_FLAG_MAY_SKIP_RESUME); | |
| 2130 | + pm_runtime_use_autosuspend(ddev->dev); | |
| 2131 | + pm_runtime_set_autosuspend_delay(ddev->dev, 5000); | |
| 2132 | + | |
| 2133 | + pm_runtime_allow(ddev->dev); | |
| 2134 | + | |
| 2135 | + pm_runtime_mark_last_busy(ddev->dev); | |
| 2136 | + pm_runtime_put_autosuspend(ddev->dev); | |
| 2137 | + | |
| 2138 | + /* | |
| 2139 | + * For runpm implemented via BACO, PMFW will handle the | |
| 2140 | + * timing for BACO in and out: | |
| 2141 | + * - put ASIC into BACO state only when both video and | |
| 2142 | + * audio functions are in D3 state. | |
| 2143 | + * - pull ASIC out of BACO state when either video or | |
| 2144 | + * audio function is in D0 state. | |
| 2145 | + * Also, at startup, PMFW assumes both functions are in | |
| 2146 | + * D0 state. | |
| 2147 | + * | |
| 2148 | + * So if snd driver was loaded prior to amdgpu driver | |
| 2149 | + * and audio function was put into D3 state, there will | |
| 2150 | + * be no PMFW-aware D-state transition(D0->D3) on runpm | |
| 2151 | + * suspend. Thus the BACO will be not correctly kicked in. | |
| 2152 | + * | |
| 2153 | + * Via amdgpu_get_secondary_funcs(), the audio dev is put | |
| 2154 | + * into D0 state. Then there will be a PMFW-aware D-state | |
| 2155 | + * transition(D0->D3) on runpm suspend. | |
| 2156 | + */ | |
| 2157 | + if (amdgpu_device_supports_baco(ddev) && | |
| 2158 | + !(adev->flags & AMD_IS_APU) && | |
| 2159 | + (adev->asic_type >= CHIP_NAVI10)) | |
| 2160 | + amdgpu_get_secondary_funcs(adev); | |
| 2161 | + } | |
| 2162 | + | |
| 2129 | 2163 | return 0; |
| 2130 | 2164 | |
| 2131 | 2165 | err_pci: |
| @@ -2137,8 +2171,15 @@ static void · 5 unmodified lines | ||
| 2137 | 2171 | amdgpu_pci_remove(struct pci_dev *pdev) |
| 2138 | 2172 | { |
| 2139 | 2173 | struct drm_device *dev = pci_get_drvdata(pdev); |
| 2174 | + struct amdgpu_device *adev = drm_to_adev(dev); | |
| 2140 | 2175 | |
| 2141 | 2176 | drm_dev_unplug(dev); |
| 2177 | + | |
| 2178 | + if (adev->runpm) { | |
| 2179 | + pm_runtime_get_sync(dev->dev); | |
| 2180 | + pm_runtime_forbid(dev->dev); | |
| 2181 | + } | |
| 2182 | + | |
| 2142 | 2183 | amdgpu_driver_unload_kms(dev); |
| 2143 | 2184 | |
| 2144 | 2185 | /* |
| @@ -2246,13 +2287,20 @@ static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work) · 101 unmodified lines | ||
| 2246 | 2287 | static int amdgpu_pmops_prepare(struct device *dev) |
| 2247 | 2288 | { |
| 2248 | 2289 | struct drm_device *drm_dev = dev_get_drvdata(dev); |
| 2290 | + struct amdgpu_device *adev = drm_to_adev(drm_dev); | |
| 2249 | 2291 | |
| 2250 | 2292 | /* Return a positive number here so |
| 2251 | 2293 | * DPM_FLAG_SMART_SUSPEND works properly |
| 2252 | 2294 | */ |
| 2253 | 2295 | if (amdgpu_device_supports_boco(drm_dev)) |
| 2254 | − return pm_runtime_suspended(dev) && | |
| 2255 | − pm_suspend_via_firmware(); | |
| 2296 | + return pm_runtime_suspended(dev); | |
| 2297 | + | |
| 2298 | + /* if we will not support s3 or s2i for the device | |
| 2299 | + * then skip suspend | |
| 2300 | + */ | |
| 2301 | + if (!amdgpu_acpi_is_s0ix_active(adev) && | |
| 2302 | + !amdgpu_acpi_is_s3_active(adev)) | |
| 2303 | + return 1; | |
| 2256 | 2304 | |
| 2257 | 2305 | return 0; |
| 2258 | 2306 | } |
| @@ -446,24 +446,18 @@ int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring, · 445 unmodified lines | ||
| 446 | 446 | * for the requested ring. |
| 447 | 447 | * |
| 448 | 448 | * @ring: ring to init the fence driver on |
| 449 | − * @num_hw_submission: number of entries on the hardware queue | |
| 450 | − * @sched_score: optional score atomic shared with other schedulers | |
| 451 | 449 | * |
| 452 | 450 | * Init the fence driver for the requested ring (all asics). |
| 453 | 451 | * Helper function for amdgpu_fence_driver_init(). |
| 454 | 452 | */ |
| 455 | −int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, | |
| 456 | − unsigned num_hw_submission, | |
| 457 | − atomic_t *sched_score) | |
| 453 | +int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring) | |
| 458 | 454 | { |
| 459 | 455 | struct amdgpu_device *adev = ring->adev; |
| 460 | − long timeout; | |
| 461 | − int r; | |
| 462 | 456 | |
| 463 | 457 | if (!adev) |
| 464 | 458 | return -EINVAL; |
| 465 | 459 | |
| 466 | − if (!is_power_of_2(num_hw_submission)) | |
| 460 | + if (!is_power_of_2(ring->num_hw_submission)) | |
| 467 | 461 | return -EINVAL; |
| 468 | 462 | |
| 469 | 463 | ring->fence_drv.cpu_addr = NULL; |
| @@ -474,41 +468,14 @@ int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, · 4 unmodified lines | ||
| 474 | 468 | |
| 475 | 469 | timer_setup(&ring->fence_drv.fallback_timer, amdgpu_fence_fallback, 0); |
| 476 | 470 | |
| 477 | − ring->fence_drv.num_fences_mask = num_hw_submission * 2 - 1; | |
| 471 | + ring->fence_drv.num_fences_mask = ring->num_hw_submission * 2 - 1; | |
| 478 | 472 | spin_lock_init(&ring->fence_drv.lock); |
| 479 | − ring->fence_drv.fences = kcalloc(num_hw_submission * 2, sizeof(void *), | |
| 473 | + ring->fence_drv.fences = kcalloc(ring->num_hw_submission * 2, sizeof(void *), | |
| 480 | 474 | GFP_KERNEL); |
| 475 | + | |
| 481 | 476 | if (!ring->fence_drv.fences) |
| 482 | 477 | return -ENOMEM; |
| 483 | 478 | |
| 484 | − /* No need to setup the GPU scheduler for rings that don't need it */ | |
| 485 | − if (ring->no_scheduler) | |
| 486 | − return 0; | |
| 487 | − | |
| 488 | − switch (ring->funcs->type) { | |
| 489 | − case AMDGPU_RING_TYPE_GFX: | |
| 490 | − timeout = adev->gfx_timeout; | |
| 491 | − break; | |
| 492 | − case AMDGPU_RING_TYPE_COMPUTE: | |
| 493 | − timeout = adev->compute_timeout; | |
| 494 | − break; | |
| 495 | − case AMDGPU_RING_TYPE_SDMA: | |
| 496 | − timeout = adev->sdma_timeout; | |
| 497 | − break; | |
| 498 | − default: | |
| 499 | − timeout = adev->video_timeout; | |
| 500 | − break; | |
| 501 | − } | |
| 502 | − | |
| 503 | − r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, | |
| 504 | − num_hw_submission, amdgpu_job_hang_limit, | |
| 505 | − timeout, NULL, sched_score, ring->name); | |
| 506 | − if (r) { | |
| 507 | − DRM_ERROR("Failed to create scheduler on ring %s.\n", | |
| 508 | − ring->name); | |
| 509 | − return r; | |
| 510 | − } | |
| 511 | − | |
| 512 | 479 | return 0; |
| 513 | 480 | } |
| 514 | 481 | |
| @@ -30,7 +30,6 @@ · 29 unmodified lines | ||
| 30 | 30 | #include "amdgpu_eeprom.h" |
| 31 | 31 | |
| 32 | 32 | #define FRU_EEPROM_MADDR 0x60000 |
| 33 | −#define I2C_PRODUCT_INFO_OFFSET 0xC0 | |
| 34 | 33 | |
| 35 | 34 | static bool is_fru_eeprom_supported(struct amdgpu_device *adev) |
| 36 | 35 | { |
| @@ -40,7 +39,13 @@ static bool is_fru_eeprom_supported(struct amdgpu_device *adev) · 3 unmodified lines | ||
| 40 | 39 | */ |
| 41 | 40 | struct atom_context *atom_ctx = adev->mode_info.atom_context; |
| 42 | 41 | |
| 43 | − /* VBIOS is of the format ###-DXXXYY-##. For SKU identification, | |
| 42 | + /* The i2c access is blocked on VF | |
| 43 | + * TODO: Need other way to get the info | |
| 44 | + */ | |
| 45 | + if (amdgpu_sriov_vf(adev)) | |
| 46 | + return false; | |
| 47 | + | |
| 48 | + /* VBIOS is of the format ###-DXXXYYYY-##. For SKU identification, | |
| 44 | 49 | * we can use just the "DXXX" portion. If there were more models, we |
| 45 | 50 | * could convert the 3 characters to a hex integer and use a switch |
| 46 | 51 | * for ease/speed/readability. For now, 2 string comparisons are |
| @@ -59,17 +64,24 @@ static bool is_fru_eeprom_supported(struct amdgpu_device *adev) · 12 unmodified lines | ||
| 59 | 64 | case CHIP_ALDEBARAN: |
| 60 | 65 | /* All Aldebaran SKUs have the FRU */ |
| 61 | 66 | return true; |
| 67 | + case CHIP_SIENNA_CICHLID: | |
| 68 | + if (strnstr(atom_ctx->vbios_version, "D603", | |
| 69 | + sizeof(atom_ctx->vbios_version))) | |
| 70 | + return true; | |
| 71 | + else | |
| 72 | + return false; | |
| 62 | 73 | default: |
| 63 | 74 | return false; |
| 64 | 75 | } |
| 65 | 76 | } |
| 66 | 77 | |
| 67 | 78 | static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr, |
| 68 | − unsigned char *buff) | |
| 79 | + unsigned char *buf, size_t buf_size) | |
| 69 | 80 | { |
| 70 | − int ret, size; | |
| 81 | + int ret; | |
| 82 | + u8 size; | |
| 71 | 83 | |
| 72 | − ret = amdgpu_eeprom_read(&adev->pm.smu_i2c, addrptr, buff, 1); | |
| 84 | + ret = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addrptr, buf, 1); | |
| 73 | 85 | if (ret < 1) { |
| 74 | 86 | DRM_WARN("FRU: Failed to get size field"); |
| 75 | 87 | return ret; |
| @@ -78,9 +90,11 @@ static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr, · 2 unmodified lines | ||
| 78 | 90 | /* The size returned by the i2c requires subtraction of 0xC0 since the |
| 79 | 91 | * size apparently always reports as 0xC0+actual size. |
| 80 | 92 | */ |
| 81 | − size = buff[0] - I2C_PRODUCT_INFO_OFFSET; | |
| 93 | + size = buf[0] & 0x3F; | |
| 94 | + size = min_t(size_t, size, buf_size); | |
| 82 | 95 | |
| 83 | − ret = amdgpu_eeprom_read(&adev->pm.smu_i2c, addrptr + 1, buff, size); | |
| 96 | + ret = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addrptr + 1, | |
| 97 | + buf, size); | |
| 84 | 98 | if (ret < 1) { |
| 85 | 99 | DRM_WARN("FRU: Failed to get data field"); |
| 86 | 100 | return ret; |
| @@ -91,19 +105,15 @@ static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr, · 4 unmodified lines | ||
| 91 | 105 | |
| 92 | 106 | int amdgpu_fru_get_product_info(struct amdgpu_device *adev) |
| 93 | 107 | { |
| 94 | − unsigned char buff[AMDGPU_PRODUCT_NAME_LEN+2]; | |
| 108 | + unsigned char buf[AMDGPU_PRODUCT_NAME_LEN]; | |
| 95 | 109 | u32 addrptr; |
| 96 | 110 | int size, len; |
| 97 | − int offset = 2; | |
| 98 | 111 | |
| 99 | 112 | if (!is_fru_eeprom_supported(adev)) |
| 100 | 113 | return 0; |
| 101 | 114 | |
| 102 | − if (adev->asic_type == CHIP_ALDEBARAN) | |
| 103 | − offset = 0; | |
| 104 | − | |
| 105 | 115 | /* If algo exists, it means that the i2c_adapter's initialized */ |
| 106 | − if (!adev->pm.smu_i2c.algo) { | |
| 116 | + if (!adev->pm.fru_eeprom_i2c_bus || !adev->pm.fru_eeprom_i2c_bus->algo) { | |
| 107 | 117 | DRM_WARN("Cannot access FRU, EEPROM accessor not initialized"); |
| 108 | 118 | return -ENODEV; |
| 109 | 119 | } |
| @@ -121,7 +131,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 11 unmodified lines | ||
| 121 | 131 | * and the language field, so just start from 0xb, manufacturer size |
| 122 | 132 | */ |
| 123 | 133 | addrptr = FRU_EEPROM_MADDR + 0xb; |
| 124 | − size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | |
| 134 | + size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); | |
| 125 | 135 | if (size < 1) { |
| 126 | 136 | DRM_ERROR("Failed to read FRU Manufacturer, ret:%d", size); |
| 127 | 137 | return -EINVAL; |
| @@ -131,7 +141,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 3 unmodified lines | ||
| 131 | 141 | * size field being 1 byte. This pattern continues below. |
| 132 | 142 | */ |
| 133 | 143 | addrptr += size + 1; |
| 134 | − size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | |
| 144 | + size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); | |
| 135 | 145 | if (size < 1) { |
| 136 | 146 | DRM_ERROR("Failed to read FRU product name, ret:%d", size); |
| 137 | 147 | return -EINVAL; |
| @@ -143,12 +153,11 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 5 unmodified lines | ||
| 143 | 153 | AMDGPU_PRODUCT_NAME_LEN); |
| 144 | 154 | len = AMDGPU_PRODUCT_NAME_LEN - 1; |
| 145 | 155 | } |
| 146 | − /* Start at 2 due to buff using fields 0 and 1 for the address */ | |
| 147 | − memcpy(adev->product_name, &buff[offset], len); | |
| 156 | + memcpy(adev->product_name, buf, len); | |
| 148 | 157 | adev->product_name[len] = '\0'; |
| 149 | 158 | |
| 150 | 159 | addrptr += size + 1; |
| 151 | − size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | |
| 160 | + size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); | |
| 152 | 161 | if (size < 1) { |
| 153 | 162 | DRM_ERROR("Failed to read FRU product number, ret:%d", size); |
| 154 | 163 | return -EINVAL; |
| @@ -162,11 +171,11 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 7 unmodified lines | ||
| 162 | 171 | DRM_WARN("FRU Product Number is larger than 16 characters. This is likely a mistake"); |
| 163 | 172 | len = sizeof(adev->product_number) - 1; |
| 164 | 173 | } |
| 165 | − memcpy(adev->product_number, &buff[offset], len); | |
| 174 | + memcpy(adev->product_number, buf, len); | |
| 166 | 175 | adev->product_number[len] = '\0'; |
| 167 | 176 | |
| 168 | 177 | addrptr += size + 1; |
| 169 | − size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | |
| 178 | + size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); | |
| 170 | 179 | |
| 171 | 180 | if (size < 1) { |
| 172 | 181 | DRM_ERROR("Failed to read FRU product version, ret:%d", size); |
| @@ -174,7 +183,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 1 unmodified line | ||
| 174 | 183 | } |
| 175 | 184 | |
| 176 | 185 | addrptr += size + 1; |
| 177 | − size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | |
| 186 | + size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); | |
| 178 | 187 | |
| 179 | 188 | if (size < 1) { |
| 180 | 189 | DRM_ERROR("Failed to read FRU serial number, ret:%d", size); |
| @@ -189,7 +198,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 8 unmodified lines | ||
| 189 | 198 | DRM_WARN("FRU Serial Number is larger than 16 characters. This is likely a mistake"); |
| 190 | 199 | len = sizeof(adev->serial) - 1; |
| 191 | 200 | } |
| 192 | − memcpy(adev->serial, &buff[offset], len); | |
| 201 | + memcpy(adev->serial, buf, len); | |
| 193 | 202 | adev->serial[len] = '\0'; |
| 194 | 203 | |
| 195 | 204 | return 0; |
| @@ -150,7 +150,7 @@ void amdgpu_gart_table_vram_free(struct amdgpu_device *adev) · 149 unmodified lines | ||
| 150 | 150 | * replaces them with the dummy page (all asics). |
| 151 | 151 | * Returns 0 for success, -EINVAL for failure. |
| 152 | 152 | */ |
| 153 | −int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, | |
| 153 | +void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, | |
| 154 | 154 | int pages) |
| 155 | 155 | { |
| 156 | 156 | unsigned t; |
| @@ -161,13 +161,11 @@ int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, · 4 unmodified lines | ||
| 161 | 161 | uint64_t flags = 0; |
| 162 | 162 | int idx; |
| 163 | 163 | |
| 164 | − if (!adev->gart.ready) { | |
| 165 | − WARN(1, "trying to unbind memory from uninitialized GART !\n"); | |
| 166 | − return -EINVAL; | |
| 167 | − } | |
| 164 | + if (!adev->gart.ptr) | |
| 165 | + return; | |
| 168 | 166 | |
| 169 | 167 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) |
| 170 | − return 0; | |
| 168 | + return; | |
| 171 | 169 | |
| 172 | 170 | t = offset / AMDGPU_GPU_PAGE_SIZE; |
| 173 | 171 | p = t / AMDGPU_GPU_PAGES_IN_CPU_PAGE; |
| @@ -188,7 +186,6 @@ int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, · 14 unmodified lines | ||
| 188 | 186 | amdgpu_gmc_flush_gpu_tlb(adev, 0, i, 0); |
| 189 | 187 | |
| 190 | 188 | drm_dev_exit(idx); |
| 191 | − return 0; | |
| 192 | 189 | } |
| 193 | 190 | |
| 194 | 191 | /** |
| @@ -204,7 +201,7 @@ int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, · 9 unmodified lines | ||
| 204 | 201 | * Map the dma_addresses into GART entries (all asics). |
| 205 | 202 | * Returns 0 for success, -EINVAL for failure. |
| 206 | 203 | */ |
| 207 | −int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, | |
| 204 | +void amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, | |
| 208 | 205 | int pages, dma_addr_t *dma_addr, uint64_t flags, |
| 209 | 206 | void *dst) |
| 210 | 207 | { |
| @@ -212,13 +209,8 @@ int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, · 1 unmodified line | ||
| 212 | 209 | unsigned i, j, t; |
| 213 | 210 | int idx; |
| 214 | 211 | |
| 215 | − if (!adev->gart.ready) { | |
| 216 | − WARN(1, "trying to bind memory to uninitialized GART !\n"); | |
| 217 | − return -EINVAL; | |
| 218 | − } | |
| 219 | − | |
| 220 | 212 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) |
| 221 | − return 0; | |
| 213 | + return; | |
| 222 | 214 | |
| 223 | 215 | t = offset / AMDGPU_GPU_PAGE_SIZE; |
| 224 | 216 | |
| @@ -230,7 +222,6 @@ int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, · 5 unmodified lines | ||
| 230 | 222 | } |
| 231 | 223 | } |
| 232 | 224 | drm_dev_exit(idx); |
| 233 | − return 0; | |
| 234 | 225 | } |
| 235 | 226 | |
| 236 | 227 | /** |
| @@ -246,20 +237,14 @@ int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, · 9 unmodified lines | ||
| 246 | 237 | * (all asics). |
| 247 | 238 | * Returns 0 for success, -EINVAL for failure. |
| 248 | 239 | */ |
| 249 | −int amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset, | |
| 240 | +void amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset, | |
| 250 | 241 | int pages, dma_addr_t *dma_addr, |
| 251 | 242 | uint64_t flags) |
| 252 | 243 | { |
| 253 | − if (!adev->gart.ready) { | |
| 254 | − WARN(1, "trying to bind memory to uninitialized GART !\n"); | |
| 255 | − return -EINVAL; | |
| 256 | − } | |
| 257 | − | |
| 258 | 244 | if (!adev->gart.ptr) |
| 259 | − return 0; | |
| 245 | + return; | |
| 260 | 246 | |
| 261 | − return amdgpu_gart_map(adev, offset, pages, dma_addr, flags, | |
| 262 | − adev->gart.ptr); | |
| 247 | + amdgpu_gart_map(adev, offset, pages, dma_addr, flags, adev->gart.ptr); | |
| 263 | 248 | } |
| 264 | 249 | |
| 265 | 250 | /** |
| @@ -274,6 +259,9 @@ void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev) · 8 unmodified lines | ||
| 274 | 259 | { |
| 275 | 260 | int i; |
| 276 | 261 | |
| 262 | + if (!adev->gart.ptr) | |
| 263 | + return; | |
| 264 | + | |
| 277 | 265 | mb(); |
| 278 | 266 | amdgpu_device_flush_hdp(adev, NULL); |
| 279 | 267 | for (i = 0; i < adev->num_vmhubs; i++) |
| @@ -46,7 +46,6 @@ struct amdgpu_gart { · 45 unmodified lines | ||
| 46 | 46 | unsigned num_gpu_pages; |
| 47 | 47 | unsigned num_cpu_pages; |
| 48 | 48 | unsigned table_size; |
| 49 | − bool ready; | |
| 50 | 49 | |
| 51 | 50 | /* Asic default pte flags */ |
| 52 | 51 | uint64_t gart_pte_flags; |
| @@ -58,12 +57,12 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); · 5 unmodified lines | ||
| 58 | 57 | void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); |
| 59 | 58 | int amdgpu_gart_init(struct amdgpu_device *adev); |
| 60 | 59 | void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); |
| 61 | −int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, | |
| 62 | − int pages); | |
| 63 | −int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, | |
| 64 | − int pages, dma_addr_t *dma_addr, uint64_t flags, | |
| 65 | − void *dst); | |
| 66 | −int amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset, | |
| 67 | − int pages, dma_addr_t *dma_addr, uint64_t flags); | |
| 60 | +void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, | |
| 61 | + int pages); | |
| 62 | +void amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, | |
| 63 | + int pages, dma_addr_t *dma_addr, uint64_t flags, | |
| 64 | + void *dst); | |
| 65 | +void amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset, | |
| 66 | + int pages, dma_addr_t *dma_addr, uint64_t flags); | |
| 68 | 67 | void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev); |
| 69 | 68 | #endif |
| @@ -222,7 +222,7 @@ static void amdgpu_gem_object_close(struct drm_gem_object *obj, · 221 unmodified lines | ||
| 222 | 222 | if (!bo_va || --bo_va->ref_count) |
| 223 | 223 | goto out_unlock; |
| 224 | 224 | |
| 225 | − amdgpu_vm_bo_rmv(adev, bo_va); | |
| 225 | + amdgpu_vm_bo_del(adev, bo_va); | |
| 226 | 226 | if (!amdgpu_vm_ready(vm)) |
| 227 | 227 | goto out_unlock; |
| 228 | 228 | |
| @@ -615,72 +615,40 @@ int amdgpu_get_gfx_off_status(struct amdgpu_device *adev, uint32_t *value) · 614 unmodified lines | ||
| 615 | 615 | |
| 616 | 616 | mutex_lock(&adev->gfx.gfx_off_mutex); |
| 617 | 617 | |
| 618 | − r = smu_get_status_gfxoff(adev, value); | |
| 618 | + r = amdgpu_dpm_get_status_gfxoff(adev, value); | |
| 619 | 619 | |
| 620 | 620 | mutex_unlock(&adev->gfx.gfx_off_mutex); |
| 621 | 621 | |
| 622 | 622 | return r; |
| 623 | 623 | } |
| 624 | 624 | |
| 625 | −int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev) | |
| 625 | +int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block) | |
| 626 | 626 | { |
| 627 | 627 | int r; |
| 628 | − struct ras_fs_if fs_info = { | |
| 629 | − .sysfs_name = "gfx_err_count", | |
| 630 | − }; | |
| 631 | − struct ras_ih_if ih_info = { | |
| 632 | − .cb = amdgpu_gfx_process_ras_data_cb, | |
| 633 | − }; | |
| 634 | − | |
| 635 | − if (!adev->gfx.ras_if) { | |
| 636 | − adev->gfx.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 637 | − if (!adev->gfx.ras_if) | |
| 638 | − return -ENOMEM; | |
| 639 | − adev->gfx.ras_if->block = AMDGPU_RAS_BLOCK__GFX; | |
| 640 | − adev->gfx.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 641 | − adev->gfx.ras_if->sub_block_index = 0; | |
| 642 | − } | |
| 643 | − fs_info.head = ih_info.head = *adev->gfx.ras_if; | |
| 644 | − r = amdgpu_ras_late_init(adev, adev->gfx.ras_if, | |
| 645 | − &fs_info, &ih_info); | |
| 628 | + r = amdgpu_ras_block_late_init(adev, ras_block); | |
| 646 | 629 | if (r) |
| 647 | − goto free; | |
| 630 | + return r; | |
| 648 | 631 | |
| 649 | − if (amdgpu_ras_is_supported(adev, adev->gfx.ras_if->block)) { | |
| 632 | + if (amdgpu_ras_is_supported(adev, ras_block->block)) { | |
| 650 | 633 | if (!amdgpu_persistent_edc_harvesting_supported(adev)) |
| 651 | 634 | amdgpu_ras_reset_error_status(adev, AMDGPU_RAS_BLOCK__GFX); |
| 652 | 635 | |
| 653 | 636 | r = amdgpu_irq_get(adev, &adev->gfx.cp_ecc_error_irq, 0); |
| 654 | 637 | if (r) |
| 655 | 638 | goto late_fini; |
| 656 | − } else { | |
| 657 | − /* free gfx ras_if if ras is not supported */ | |
| 658 | − r = 0; | |
| 659 | − goto free; | |
| 660 | 639 | } |
| 661 | 640 | |
| 662 | 641 | return 0; |
| 663 | 642 | late_fini: |
| 664 | − amdgpu_ras_late_fini(adev, adev->gfx.ras_if, &ih_info); | |
| 665 | −free: | |
| 666 | − kfree(adev->gfx.ras_if); | |
| 667 | − adev->gfx.ras_if = NULL; | |
| 643 | + amdgpu_ras_block_late_fini(adev, ras_block); | |
| 668 | 644 | return r; |
| 669 | 645 | } |
| 670 | 646 | |
| 671 | 647 | void amdgpu_gfx_ras_fini(struct amdgpu_device *adev) |
| 672 | 648 | { |
| 673 | 649 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX) && |
| 674 | − adev->gfx.ras_if) { | |
| 675 | − struct ras_common_if *ras_if = adev->gfx.ras_if; | |
| 676 | − struct ras_ih_if ih_info = { | |
| 677 | − .head = *ras_if, | |
| 678 | − .cb = amdgpu_gfx_process_ras_data_cb, | |
| 679 | − }; | |
| 680 | − | |
| 681 | − amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 682 | − kfree(ras_if); | |
| 683 | − } | |
| 650 | + adev->gfx.ras_if) | |
| 651 | + amdgpu_ras_block_late_fini(adev, adev->gfx.ras_if); | |
| 684 | 652 | } |
| 685 | 653 | |
| 686 | 654 | int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, |
| @@ -695,9 +663,9 @@ int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, · 8 unmodified lines | ||
| 695 | 663 | */ |
| 696 | 664 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) { |
| 697 | 665 | kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); |
| 698 | − if (adev->gfx.ras_funcs && | |
| 699 | − adev->gfx.ras_funcs->query_ras_error_count) | |
| 700 | − adev->gfx.ras_funcs->query_ras_error_count(adev, err_data); | |
| 666 | + if (adev->gfx.ras && adev->gfx.ras->ras_block.hw_ops && | |
| 667 | + adev->gfx.ras->ras_block.hw_ops->query_ras_error_count) | |
| 668 | + adev->gfx.ras->ras_block.hw_ops->query_ras_error_count(adev, err_data); | |
| 701 | 669 | amdgpu_ras_reset_gpu(adev); |
| 702 | 670 | } |
| 703 | 671 | return AMDGPU_RAS_SUCCESS; |
| @@ -852,19 +820,3 @@ int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev) · 148 unmodified lines | ||
| 852 | 820 | } |
| 853 | 821 | return amdgpu_num_kcq; |
| 854 | 822 | } |
| 855 | − | |
| 856 | −/* amdgpu_gfx_state_change_set - Handle gfx power state change set | |
| 857 | − * @adev: amdgpu_device pointer | |
| 858 | − * @state: gfx power state(1 -sGpuChangeState_D0Entry and 2 -sGpuChangeState_D3Entry) | |
| 859 | − * | |
| 860 | − */ | |
| 861 | − | |
| 862 | −void amdgpu_gfx_state_change_set(struct amdgpu_device *adev, enum gfx_change_state state) | |
| 863 | −{ | |
| 864 | − mutex_lock(&adev->pm.mutex); | |
| 865 | − if (adev->powerplay.pp_funcs && | |
| 866 | − adev->powerplay.pp_funcs->gfx_state_change_set) | |
| 867 | − ((adev)->powerplay.pp_funcs->gfx_state_change_set( | |
| 868 | − (adev)->powerplay.pp_handle, state)); | |
| 869 | − mutex_unlock(&adev->pm.mutex); | |
| 870 | −} | |
| @@ -31,6 +31,7 @@ · 30 unmodified lines | ||
| 31 | 31 | #include "amdgpu_ring.h" |
| 32 | 32 | #include "amdgpu_rlc.h" |
| 33 | 33 | #include "soc15.h" |
| 34 | +#include "amdgpu_ras.h" | |
| 34 | 35 | |
| 35 | 36 | /* GFX current status */ |
| 36 | 37 | #define AMDGPU_GFX_NORMAL_MODE 0x00000000L |
| @@ -47,12 +48,6 @@ enum amdgpu_gfx_pipe_priority { · 10 unmodified lines | ||
| 47 | 48 | AMDGPU_GFX_PIPE_PRIO_HIGH = AMDGPU_RING_PRIO_2 |
| 48 | 49 | }; |
| 49 | 50 | |
| 50 | −/* Argument for PPSMC_MSG_GpuChangeState */ | |
| 51 | −enum gfx_change_state { | |
| 52 | − sGpuChangeState_D0Entry = 1, | |
| 53 | − sGpuChangeState_D3Entry, | |
| 54 | −}; | |
| 55 | − | |
| 56 | 51 | #define AMDGPU_GFX_QUEUE_PRIORITY_MINIMUM 0 |
| 57 | 52 | #define AMDGPU_GFX_QUEUE_PRIORITY_MAXIMUM 15 |
| 58 | 53 | |
| @@ -204,16 +199,8 @@ struct amdgpu_cu_info { · 145 unmodified lines | ||
| 204 | 199 | uint32_t bitmap[4][4]; |
| 205 | 200 | }; |
| 206 | 201 | |
| 207 | −struct amdgpu_gfx_ras_funcs { | |
| 208 | − int (*ras_late_init)(struct amdgpu_device *adev); | |
| 209 | − void (*ras_fini)(struct amdgpu_device *adev); | |
| 210 | − int (*ras_error_inject)(struct amdgpu_device *adev, | |
| 211 | − void *inject_if); | |
| 212 | − int (*query_ras_error_count)(struct amdgpu_device *adev, | |
| 213 | − void *ras_error_status); | |
| 214 | − void (*reset_ras_error_count)(struct amdgpu_device *adev); | |
| 215 | − void (*query_ras_error_status)(struct amdgpu_device *adev); | |
| 216 | − void (*reset_ras_error_status)(struct amdgpu_device *adev); | |
| 202 | +struct amdgpu_gfx_ras { | |
| 203 | + struct amdgpu_ras_block_object ras_block; | |
| 217 | 204 | void (*enable_watchdog_timer)(struct amdgpu_device *adev); |
| 218 | 205 | }; |
| 219 | 206 | |
| @@ -337,7 +324,7 @@ struct amdgpu_gfx { · 117 unmodified lines | ||
| 337 | 324 | |
| 338 | 325 | /*ras */ |
| 339 | 326 | struct ras_common_if *ras_if; |
| 340 | − const struct amdgpu_gfx_ras_funcs *ras_funcs; | |
| 327 | + struct amdgpu_gfx_ras *ras; | |
| 341 | 328 | }; |
| 342 | 329 | |
| 343 | 330 | #define amdgpu_gfx_get_gpu_clock_counter(adev) (adev)->gfx.funcs->get_gpu_clock_counter((adev)) |
| @@ -399,7 +386,7 @@ bool amdgpu_gfx_is_me_queue_enabled(struct amdgpu_device *adev, int me, · 55 unmodified lines | ||
| 399 | 386 | int pipe, int queue); |
| 400 | 387 | void amdgpu_gfx_off_ctrl(struct amdgpu_device *adev, bool enable); |
| 401 | 388 | int amdgpu_get_gfx_off_status(struct amdgpu_device *adev, uint32_t *value); |
| 402 | −int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev); | |
| 389 | +int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block); | |
| 403 | 390 | void amdgpu_gfx_ras_fini(struct amdgpu_device *adev); |
| 404 | 391 | int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, |
| 405 | 392 | void *err_data, |
| @@ -410,5 +397,4 @@ int amdgpu_gfx_cp_ecc_error_irq(struct amdgpu_device *adev, · 4 unmodified lines | ||
| 410 | 397 | uint32_t amdgpu_kiq_rreg(struct amdgpu_device *adev, uint32_t reg); |
| 411 | 398 | void amdgpu_kiq_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v); |
| 412 | 399 | int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev); |
| 413 | −void amdgpu_gfx_state_change_set(struct amdgpu_device *adev, enum gfx_change_state state); | |
| 414 | 400 | #endif |
| @@ -436,82 +436,35 @@ void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr, · 435 unmodified lines | ||
| 436 | 436 | } while (fault->timestamp < tmp); |
| 437 | 437 | } |
| 438 | 438 | |
| 439 | −int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev) | |
| 439 | +int amdgpu_gmc_ras_early_init(struct amdgpu_device *adev) | |
| 440 | 440 | { |
| 441 | − int r; | |
| 442 | − | |
| 443 | − if (adev->umc.ras_funcs && | |
| 444 | − adev->umc.ras_funcs->ras_late_init) { | |
| 445 | − r = adev->umc.ras_funcs->ras_late_init(adev); | |
| 446 | − if (r) | |
| 447 | − return r; | |
| 448 | − } | |
| 449 | − | |
| 450 | − if (adev->mmhub.ras_funcs && | |
| 451 | − adev->mmhub.ras_funcs->ras_late_init) { | |
| 452 | − r = adev->mmhub.ras_funcs->ras_late_init(adev); | |
| 453 | − if (r) | |
| 454 | − return r; | |
| 455 | − } | |
| 456 | − | |
| 457 | − if (!adev->gmc.xgmi.connected_to_cpu) | |
| 458 | − adev->gmc.xgmi.ras_funcs = &xgmi_ras_funcs; | |
| 459 | − | |
| 460 | − if (adev->gmc.xgmi.ras_funcs && | |
| 461 | − adev->gmc.xgmi.ras_funcs->ras_late_init) { | |
| 462 | − r = adev->gmc.xgmi.ras_funcs->ras_late_init(adev); | |
| 463 | − if (r) | |
| 464 | − return r; | |
| 465 | − } | |
| 466 | − | |
| 467 | − if (adev->hdp.ras_funcs && | |
| 468 | − adev->hdp.ras_funcs->ras_late_init) { | |
| 469 | − r = adev->hdp.ras_funcs->ras_late_init(adev); | |
| 470 | − if (r) | |
| 471 | − return r; | |
| 472 | − } | |
| 473 | − | |
| 474 | − if (adev->mca.mp0.ras_funcs && | |
| 475 | − adev->mca.mp0.ras_funcs->ras_late_init) { | |
| 476 | − r = adev->mca.mp0.ras_funcs->ras_late_init(adev); | |
| 477 | − if (r) | |
| 478 | − return r; | |
| 479 | − } | |
| 480 | − | |
| 481 | − if (adev->mca.mp1.ras_funcs && | |
| 482 | − adev->mca.mp1.ras_funcs->ras_late_init) { | |
| 483 | − r = adev->mca.mp1.ras_funcs->ras_late_init(adev); | |
| 484 | − if (r) | |
| 485 | − return r; | |
| 441 | + if (!adev->gmc.xgmi.connected_to_cpu) { | |
| 442 | + adev->gmc.xgmi.ras = &xgmi_ras; | |
| 443 | + amdgpu_ras_register_ras_block(adev, &adev->gmc.xgmi.ras->ras_block); | |
| 444 | + adev->gmc.xgmi.ras_if = &adev->gmc.xgmi.ras->ras_block.ras_comm; | |
| 486 | 445 | } |
| 487 | 446 | |
| 488 | − if (adev->mca.mpio.ras_funcs && | |
| 489 | − adev->mca.mpio.ras_funcs->ras_late_init) { | |
| 490 | − r = adev->mca.mpio.ras_funcs->ras_late_init(adev); | |
| 491 | − if (r) | |
| 492 | − return r; | |
| 493 | − } | |
| 447 | + return 0; | |
| 448 | +} | |
| 494 | 449 | |
| 450 | +int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev) | |
| 451 | +{ | |
| 495 | 452 | return 0; |
| 496 | 453 | } |
| 497 | 454 | |
| 498 | 455 | void amdgpu_gmc_ras_fini(struct amdgpu_device *adev) |
| 499 | 456 | { |
| 500 | − if (adev->umc.ras_funcs && | |
| 501 | − adev->umc.ras_funcs->ras_fini) | |
| 502 | − adev->umc.ras_funcs->ras_fini(adev); | |
| 457 | + if (adev->umc.ras && adev->umc.ras->ras_block.ras_fini) | |
| 458 | + adev->umc.ras->ras_block.ras_fini(adev); | |
| 503 | 459 | |
| 504 | − if (adev->mmhub.ras_funcs && | |
| 505 | − adev->mmhub.ras_funcs->ras_fini) | |
| 506 | − adev->mmhub.ras_funcs->ras_fini(adev); | |
| 460 | + if (adev->mmhub.ras && adev->mmhub.ras->ras_block.ras_fini) | |
| 461 | + adev->mmhub.ras->ras_block.ras_fini(adev); | |
| 507 | 462 | |
| 508 | − if (adev->gmc.xgmi.ras_funcs && | |
| 509 | − adev->gmc.xgmi.ras_funcs->ras_fini) | |
| 510 | − adev->gmc.xgmi.ras_funcs->ras_fini(adev); | |
| 463 | + if (adev->gmc.xgmi.ras && adev->gmc.xgmi.ras->ras_block.ras_fini) | |
| 464 | + adev->gmc.xgmi.ras->ras_block.ras_fini(adev); | |
| 511 | 465 | |
| 512 | − if (adev->hdp.ras_funcs && | |
| 513 | − adev->hdp.ras_funcs->ras_fini) | |
| 514 | − adev->hdp.ras_funcs->ras_fini(adev); | |
| 466 | + if (adev->hdp.ras && adev->hdp.ras->ras_block.ras_fini) | |
| 467 | + adev->hdp.ras->ras_block.ras_fini(adev); | |
| 515 | 468 | } |
| 516 | 469 | |
| 517 | 470 | /* |
| @@ -584,6 +537,7 @@ void amdgpu_gmc_tmz_set(struct amdgpu_device *adev) · 66 unmodified lines | ||
| 584 | 537 | case CHIP_NAVI12: |
| 585 | 538 | case CHIP_VANGOGH: |
| 586 | 539 | case CHIP_YELLOW_CARP: |
| 540 | + case CHIP_IP_DISCOVERY: | |
| 587 | 541 | /* Don't enable it by default yet. |
| 588 | 542 | */ |
| 589 | 543 | if (amdgpu_tmz < 1) { |
| @@ -831,3 +785,49 @@ void amdgpu_gmc_get_reserved_allocation(struct amdgpu_device *adev) · 241 unmodified lines | ||
| 831 | 785 | break; |
| 832 | 786 | } |
| 833 | 787 | } |
| 788 | + | |
| 789 | +int amdgpu_gmc_vram_checking(struct amdgpu_device *adev) | |
| 790 | +{ | |
| 791 | + struct amdgpu_bo *vram_bo = NULL; | |
| 792 | + uint64_t vram_gpu = 0; | |
| 793 | + void *vram_ptr = NULL; | |
| 794 | + | |
| 795 | + int ret, size = 0x100000; | |
| 796 | + uint8_t cptr[10]; | |
| 797 | + | |
| 798 | + ret = amdgpu_bo_create_kernel(adev, size, PAGE_SIZE, | |
| 799 | + AMDGPU_GEM_DOMAIN_VRAM, | |
| 800 | + &vram_bo, | |
| 801 | + &vram_gpu, | |
| 802 | + &vram_ptr); | |
| 803 | + if (ret) | |
| 804 | + return ret; | |
| 805 | + | |
| 806 | + memset(vram_ptr, 0x86, size); | |
| 807 | + memset(cptr, 0x86, 10); | |
| 808 | + | |
| 809 | + /** | |
| 810 | + * Check the start, the mid, and the end of the memory if the content of | |
| 811 | + * each byte is the pattern "0x86". If yes, we suppose the vram bo is | |
| 812 | + * workable. | |
| 813 | + * | |
| 814 | + * Note: If check the each byte of whole 1M bo, it will cost too many | |
| 815 | + * seconds, so here, we just pick up three parts for emulation. | |
| 816 | + */ | |
| 817 | + ret = memcmp(vram_ptr, cptr, 10); | |
| 818 | + if (ret) | |
| 819 | + return ret; | |
| 820 | + | |
| 821 | + ret = memcmp(vram_ptr + (size / 2), cptr, 10); | |
| 822 | + if (ret) | |
| 823 | + return ret; | |
| 824 | + | |
| 825 | + ret = memcmp(vram_ptr + size - 10, cptr, 10); | |
| 826 | + if (ret) | |
| 827 | + return ret; | |
| 828 | + | |
| 829 | + amdgpu_bo_free_kernel(&vram_bo, &vram_gpu, | |
| 830 | + &vram_ptr); | |
| 831 | + | |
| 832 | + return 0; | |
| 833 | +} | |
| @@ -29,6 +29,7 @@ · 28 unmodified lines | ||
| 29 | 29 | #include <linux/types.h> |
| 30 | 30 | |
| 31 | 31 | #include "amdgpu_irq.h" |
| 32 | +#include "amdgpu_ras.h" | |
| 32 | 33 | |
| 33 | 34 | /* VA hole for 48bit addresses on Vega10 */ |
| 34 | 35 | #define AMDGPU_GMC_HOLE_START 0x0000800000000000ULL |
| @@ -135,12 +136,8 @@ struct amdgpu_gmc_funcs { · 100 unmodified lines | ||
| 135 | 136 | unsigned int (*get_vbios_fb_size)(struct amdgpu_device *adev); |
| 136 | 137 | }; |
| 137 | 138 | |
| 138 | −struct amdgpu_xgmi_ras_funcs { | |
| 139 | − int (*ras_late_init)(struct amdgpu_device *adev); | |
| 140 | − void (*ras_fini)(struct amdgpu_device *adev); | |
| 141 | − int (*query_ras_error_count)(struct amdgpu_device *adev, | |
| 142 | − void *ras_error_status); | |
| 143 | − void (*reset_ras_error_count)(struct amdgpu_device *adev); | |
| 139 | +struct amdgpu_xgmi_ras { | |
| 140 | + struct amdgpu_ras_block_object ras_block; | |
| 144 | 141 | }; |
| 145 | 142 | |
| 146 | 143 | struct amdgpu_xgmi { |
| @@ -159,7 +156,7 @@ struct amdgpu_xgmi { · 12 unmodified lines | ||
| 159 | 156 | struct ras_common_if *ras_if; |
| 160 | 157 | bool connected_to_cpu; |
| 161 | 158 | bool pending_reset; |
| 162 | − const struct amdgpu_xgmi_ras_funcs *ras_funcs; | |
| 159 | + struct amdgpu_xgmi_ras *ras; | |
| 163 | 160 | }; |
| 164 | 161 | |
| 165 | 162 | struct amdgpu_gmc { |
| @@ -321,6 +318,7 @@ bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, · 155 unmodified lines | ||
| 321 | 318 | uint16_t pasid, uint64_t timestamp); |
| 322 | 319 | void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr, |
| 323 | 320 | uint16_t pasid); |
| 321 | +int amdgpu_gmc_ras_early_init(struct amdgpu_device *adev); | |
| 324 | 322 | int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev); |
| 325 | 323 | void amdgpu_gmc_ras_fini(struct amdgpu_device *adev); |
| 326 | 324 | int amdgpu_gmc_allocate_vm_inv_eng(struct amdgpu_device *adev); |
| @@ -339,4 +337,5 @@ void amdgpu_gmc_init_pdb0(struct amdgpu_device *adev); · 12 unmodified lines | ||
| 339 | 337 | uint64_t amdgpu_gmc_vram_mc2pa(struct amdgpu_device *adev, uint64_t mc_addr); |
| 340 | 338 | uint64_t amdgpu_gmc_vram_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo); |
| 341 | 339 | uint64_t amdgpu_gmc_vram_cpu_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo); |
| 340 | +int amdgpu_gmc_vram_checking(struct amdgpu_device *adev); | |
| 342 | 341 | #endif |
| @@ -60,7 +60,7 @@ static ssize_t amdgpu_mem_info_gtt_total_show(struct device *dev, · 59 unmodified lines | ||
| 60 | 60 | struct ttm_resource_manager *man; |
| 61 | 61 | |
| 62 | 62 | man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); |
| 63 | − return sysfs_emit(buf, "%llu\n", man->size * PAGE_SIZE); | |
| 63 | + return sysfs_emit(buf, "%llu\n", man->size); | |
| 64 | 64 | } |
| 65 | 65 | |
| 66 | 66 | /** |
| @@ -77,8 +77,9 @@ static ssize_t amdgpu_mem_info_gtt_used_show(struct device *dev, · 10 unmodified lines | ||
| 77 | 77 | { |
| 78 | 78 | struct drm_device *ddev = dev_get_drvdata(dev); |
| 79 | 79 | struct amdgpu_device *adev = drm_to_adev(ddev); |
| 80 | + struct ttm_resource_manager *man = &adev->mman.gtt_mgr.manager; | |
| 80 | 81 | |
| 81 | − return sysfs_emit(buf, "%llu\n", amdgpu_gtt_mgr_usage(&adev->mman.gtt_mgr)); | |
| 82 | + return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man)); | |
| 82 | 83 | } |
| 83 | 84 | |
| 84 | 85 | static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, |
| @@ -130,20 +131,17 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, · 45 unmodified lines | ||
| 130 | 131 | struct amdgpu_gtt_node *node; |
| 131 | 132 | int r; |
| 132 | 133 | |
| 133 | − if (!(place->flags & TTM_PL_FLAG_TEMPORARY) && | |
| 134 | − atomic64_add_return(num_pages, &mgr->used) > man->size) { | |
| 135 | − atomic64_sub(num_pages, &mgr->used); | |
| 136 | − return -ENOSPC; | |
| 137 | − } | |
| 138 | − | |
| 139 | 134 | node = kzalloc(struct_size(node, base.mm_nodes, 1), GFP_KERNEL); |
| 140 | − if (!node) { | |
| 141 | − r = -ENOMEM; | |
| 142 | − goto err_out; | |
| 143 | − } | |
| 135 | + if (!node) | |
| 136 | + return -ENOMEM; | |
| 144 | 137 | |
| 145 | 138 | node->tbo = tbo; |
| 146 | 139 | ttm_resource_init(tbo, place, &node->base.base); |
| 140 | + if (!(place->flags & TTM_PL_FLAG_TEMPORARY) && | |
| 141 | + ttm_resource_manager_usage(man) > man->size) { | |
| 142 | + r = -ENOSPC; | |
| 143 | + goto err_free; | |
| 144 | + } | |
| 147 | 145 | |
| 148 | 146 | if (place->lpfn) { |
| 149 | 147 | spin_lock(&mgr->lock); |
| @@ -169,11 +167,6 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, · 19 unmodified lines | ||
| 169 | 167 | err_free: |
| 170 | 168 | ttm_resource_fini(man, &node->base.base); |
| 171 | 169 | kfree(node); |
| 172 | − | |
| 173 | −err_out: | |
| 174 | − if (!(place->flags & TTM_PL_FLAG_TEMPORARY)) | |
| 175 | − atomic64_sub(num_pages, &mgr->used); | |
| 176 | − | |
| 177 | 170 | return r; |
| 178 | 171 | } |
| 179 | 172 | |
| @@ -196,25 +189,10 @@ static void amdgpu_gtt_mgr_del(struct ttm_resource_manager *man, · 16 unmodified lines | ||
| 196 | 189 | drm_mm_remove_node(&node->base.mm_nodes[0]); |
| 197 | 190 | spin_unlock(&mgr->lock); |
| 198 | 191 | |
| 199 | − if (!(res->placement & TTM_PL_FLAG_TEMPORARY)) | |
| 200 | − atomic64_sub(res->num_pages, &mgr->used); | |
| 201 | − | |
| 202 | 192 | ttm_resource_fini(man, res); |
| 203 | 193 | kfree(node); |
| 204 | 194 | } |
| 205 | 195 | |
| 206 | −/** | |
| 207 | − * amdgpu_gtt_mgr_usage - return usage of GTT domain | |
| 208 | − * | |
| 209 | − * @mgr: amdgpu_gtt_mgr pointer | |
| 210 | − * | |
| 211 | − * Return how many bytes are used in the GTT domain | |
| 212 | − */ | |
| 213 | −uint64_t amdgpu_gtt_mgr_usage(struct amdgpu_gtt_mgr *mgr) | |
| 214 | −{ | |
| 215 | − return atomic64_read(&mgr->used) * PAGE_SIZE; | |
| 216 | −} | |
| 217 | − | |
| 218 | 196 | /** |
| 219 | 197 | * amdgpu_gtt_mgr_recover - re-init gart |
| 220 | 198 | * |
| @@ -222,26 +200,21 @@ uint64_t amdgpu_gtt_mgr_usage(struct amdgpu_gtt_mgr *mgr) · 1 unmodified line | ||
| 222 | 200 | * |
| 223 | 201 | * Re-init the gart for each known BO in the GTT. |
| 224 | 202 | */ |
| 225 | −int amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr) | |
| 203 | +void amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr) | |
| 226 | 204 | { |
| 227 | 205 | struct amdgpu_gtt_node *node; |
| 228 | 206 | struct drm_mm_node *mm_node; |
| 229 | 207 | struct amdgpu_device *adev; |
| 230 | − int r = 0; | |
| 231 | 208 | |
| 232 | 209 | adev = container_of(mgr, typeof(*adev), mman.gtt_mgr); |
| 233 | 210 | spin_lock(&mgr->lock); |
| 234 | 211 | drm_mm_for_each_node(mm_node, &mgr->mm) { |
| 235 | 212 | node = container_of(mm_node, typeof(*node), base.mm_nodes[0]); |
| 236 | − r = amdgpu_ttm_recover_gart(node->tbo); | |
| 237 | − if (r) | |
| 238 | − break; | |
| 213 | + amdgpu_ttm_recover_gart(node->tbo); | |
| 239 | 214 | } |
| 240 | 215 | spin_unlock(&mgr->lock); |
| 241 | 216 | |
| 242 | 217 | amdgpu_gart_invalidate_tlb(adev); |
| 243 | − | |
| 244 | − return r; | |
| 245 | 218 | } |
| 246 | 219 | |
| 247 | 220 | /** |
| @@ -260,9 +233,6 @@ static void amdgpu_gtt_mgr_debug(struct ttm_resource_manager *man, · 12 unmodified lines | ||
| 260 | 233 | spin_lock(&mgr->lock); |
| 261 | 234 | drm_mm_print(&mgr->mm, printer); |
| 262 | 235 | spin_unlock(&mgr->lock); |
| 263 | − | |
| 264 | − drm_printf(printer, "man size:%llu pages, gtt used:%llu pages\n", | |
| 265 | − man->size, atomic64_read(&mgr->used)); | |
| 266 | 236 | } |
| 267 | 237 | |
| 268 | 238 | static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func = { |
| @@ -288,14 +258,12 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) · 19 unmodified lines | ||
| 288 | 258 | man->use_tt = true; |
| 289 | 259 | man->func = &amdgpu_gtt_mgr_func; |
| 290 | 260 | |
| 291 | − ttm_resource_manager_init(man, &adev->mman.bdev, | |
| 292 | − gtt_size >> PAGE_SHIFT); | |
| 261 | + ttm_resource_manager_init(man, &adev->mman.bdev, gtt_size); | |
| 293 | 262 | |
| 294 | 263 | start = AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS; |
| 295 | 264 | size = (adev->gmc.gart_size >> PAGE_SHIFT) - start; |
| 296 | 265 | drm_mm_init(&mgr->mm, start, size); |
| 297 | 266 | spin_lock_init(&mgr->lock); |
| 298 | − atomic64_set(&mgr->used, 0); | |
| 299 | 267 | |
| 300 | 268 | ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); |
| 301 | 269 | ttm_resource_manager_set_used(man, true); |
| @@ -24,45 +24,9 @@ · 23 unmodified lines | ||
| 24 | 24 | #include "amdgpu.h" |
| 25 | 25 | #include "amdgpu_ras.h" |
| 26 | 26 | |
| 27 | −int amdgpu_hdp_ras_late_init(struct amdgpu_device *adev) | |
| 28 | −{ | |
| 29 | − int r; | |
| 30 | − struct ras_ih_if ih_info = { | |
| 31 | − .cb = NULL, | |
| 32 | − }; | |
| 33 | − struct ras_fs_if fs_info = { | |
| 34 | − .sysfs_name = "hdp_err_count", | |
| 35 | − }; | |
| 36 | − | |
| 37 | − if (!adev->hdp.ras_if) { | |
| 38 | − adev->hdp.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 39 | − if (!adev->hdp.ras_if) | |
| 40 | − return -ENOMEM; | |
| 41 | − adev->hdp.ras_if->block = AMDGPU_RAS_BLOCK__HDP; | |
| 42 | − adev->hdp.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 43 | − adev->hdp.ras_if->sub_block_index = 0; | |
| 44 | − } | |
| 45 | − ih_info.head = fs_info.head = *adev->hdp.ras_if; | |
| 46 | − r = amdgpu_ras_late_init(adev, adev->hdp.ras_if, | |
| 47 | − &fs_info, &ih_info); | |
| 48 | − if (r || !amdgpu_ras_is_supported(adev, adev->hdp.ras_if->block)) { | |
| 49 | − kfree(adev->hdp.ras_if); | |
| 50 | − adev->hdp.ras_if = NULL; | |
| 51 | − } | |
| 52 | − | |
| 53 | − return r; | |
| 54 | −} | |
| 55 | − | |
| 56 | 27 | void amdgpu_hdp_ras_fini(struct amdgpu_device *adev) |
| 57 | 28 | { |
| 58 | 29 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__HDP) && |
| 59 | − adev->hdp.ras_if) { | |
| 60 | − struct ras_common_if *ras_if = adev->hdp.ras_if; | |
| 61 | − struct ras_ih_if ih_info = { | |
| 62 | − .cb = NULL, | |
| 63 | − }; | |
| 64 | − | |
| 65 | − amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 66 | − kfree(ras_if); | |
| 67 | − } | |
| 30 | + adev->hdp.ras_if) | |
| 31 | + amdgpu_ras_block_late_fini(adev, adev->hdp.ras_if); | |
| 68 | 32 | } |
| @@ -22,13 +22,10 @@ · 21 unmodified lines | ||
| 22 | 22 | */ |
| 23 | 23 | #ifndef __AMDGPU_HDP_H__ |
| 24 | 24 | #define __AMDGPU_HDP_H__ |
| 25 | +#include "amdgpu_ras.h" | |
| 25 | 26 | |
| 26 | −struct amdgpu_hdp_ras_funcs { | |
| 27 | − int (*ras_late_init)(struct amdgpu_device *adev); | |
| 28 | − void (*ras_fini)(struct amdgpu_device *adev); | |
| 29 | − void (*query_ras_error_count)(struct amdgpu_device *adev, | |
| 30 | − void *ras_error_status); | |
| 31 | − void (*reset_ras_error_count)(struct amdgpu_device *adev); | |
| 27 | +struct amdgpu_hdp_ras { | |
| 28 | + struct amdgpu_ras_block_object ras_block; | |
| 32 | 29 | }; |
| 33 | 30 | |
| 34 | 31 | struct amdgpu_hdp_funcs { |
| @@ -43,9 +40,9 @@ struct amdgpu_hdp_funcs { · 8 unmodified lines | ||
| 43 | 40 | struct amdgpu_hdp { |
| 44 | 41 | struct ras_common_if *ras_if; |
| 45 | 42 | const struct amdgpu_hdp_funcs *funcs; |
| 46 | − const struct amdgpu_hdp_ras_funcs *ras_funcs; | |
| 43 | + struct amdgpu_hdp_ras *ras; | |
| 47 | 44 | }; |
| 48 | 45 | |
| 49 | −int amdgpu_hdp_ras_late_init(struct amdgpu_device *adev); | |
| 46 | +int amdgpu_hdp_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block); | |
| 50 | 47 | void amdgpu_hdp_ras_fini(struct amdgpu_device *adev); |
| 51 | 48 | #endif /* __AMDGPU_HDP_H__ */ |
| @@ -199,13 +199,13 @@ static irqreturn_t amdgpu_irq_handler(int irq, void *arg) · 198 unmodified lines | ||
| 199 | 199 | * ack the interrupt if it is there |
| 200 | 200 | */ |
| 201 | 201 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF)) { |
| 202 | − if (adev->nbio.ras_funcs && | |
| 203 | − adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring) | |
| 204 | − adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring(adev); | |
| 202 | + if (adev->nbio.ras && | |
| 203 | + adev->nbio.ras->handle_ras_controller_intr_no_bifring) | |
| 204 | + adev->nbio.ras->handle_ras_controller_intr_no_bifring(adev); | |
| 205 | 205 | |
| 206 | − if (adev->nbio.ras_funcs && | |
| 207 | − adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring) | |
| 208 | − adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring(adev); | |
| 206 | + if (adev->nbio.ras && | |
| 207 | + adev->nbio.ras->handle_ras_err_event_athub_intr_no_bifring) | |
| 208 | + adev->nbio.ras->handle_ras_err_event_athub_intr_no_bifring(adev); | |
| 209 | 209 | } |
| 210 | 210 | |
| 211 | 211 | return ret; |
| @@ -37,6 +37,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) · 36 unmodified lines | ||
| 37 | 37 | struct amdgpu_task_info ti; |
| 38 | 38 | struct amdgpu_device *adev = ring->adev; |
| 39 | 39 | int idx; |
| 40 | + int r; | |
| 40 | 41 | |
| 41 | 42 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) { |
| 42 | 43 | DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s", |
| @@ -63,7 +64,9 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) · 20 unmodified lines | ||
| 63 | 64 | ti.process_name, ti.tgid, ti.task_name, ti.pid); |
| 64 | 65 | |
| 65 | 66 | if (amdgpu_device_should_recover_gpu(ring->adev)) { |
| 66 | − amdgpu_device_gpu_recover(ring->adev, job); | |
| 67 | + r = amdgpu_device_gpu_recover_imp(ring->adev, job); | |
| 68 | + if (r) | |
| 69 | + DRM_ERROR("GPU Recovery Failed: %d\n", r); | |
| 67 | 70 | } else { |
| 68 | 71 | drm_sched_suspend_timeout(&ring->sched); |
| 69 | 72 | if (amdgpu_sriov_vf(adev)) |
| @@ -87,11 +87,6 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) · 86 unmodified lines | ||
| 87 | 87 | if (adev->rmmio == NULL) |
| 88 | 88 | return; |
| 89 | 89 | |
| 90 | − if (adev->runpm) { | |
| 91 | − pm_runtime_get_sync(dev->dev); | |
| 92 | − pm_runtime_forbid(dev->dev); | |
| 93 | − } | |
| 94 | − | |
| 95 | 90 | if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DRV_UNLOAD)) |
| 96 | 91 | DRM_WARN("smart shift update failed\n"); |
| 97 | 92 | |
| @@ -124,22 +119,6 @@ void amdgpu_register_gpu_instance(struct amdgpu_device *adev) · 26 unmodified lines | ||
| 124 | 119 | mutex_unlock(&mgpu_info.mutex); |
| 125 | 120 | } |
| 126 | 121 | |
| 127 | −static void amdgpu_get_audio_func(struct amdgpu_device *adev) | |
| 128 | −{ | |
| 129 | − struct pci_dev *p = NULL; | |
| 130 | − | |
| 131 | − p = pci_get_domain_bus_and_slot(pci_domain_nr(adev->pdev->bus), | |
| 132 | − adev->pdev->bus->number, 1); | |
| 133 | − if (p) { | |
| 134 | − pm_runtime_get_sync(&p->dev); | |
| 135 | − | |
| 136 | − pm_runtime_mark_last_busy(&p->dev); | |
| 137 | − pm_runtime_put_autosuspend(&p->dev); | |
| 138 | − | |
| 139 | − pci_dev_put(p); | |
| 140 | − } | |
| 141 | −} | |
| 142 | − | |
| 143 | 122 | /** |
| 144 | 123 | * amdgpu_driver_load_kms - Main load function for KMS. |
| 145 | 124 | * |
| @@ -152,21 +131,10 @@ static void amdgpu_get_audio_func(struct amdgpu_device *adev) · 6 unmodified lines | ||
| 152 | 131 | int amdgpu_driver_load_kms(struct amdgpu_device *adev, unsigned long flags) |
| 153 | 132 | { |
| 154 | 133 | struct drm_device *dev; |
| 155 | − struct pci_dev *parent; | |
| 156 | 134 | int r, acpi_status; |
| 157 | 135 | |
| 158 | 136 | dev = adev_to_drm(adev); |
| 159 | 137 | |
| 160 | − if (amdgpu_has_atpx() && | |
| 161 | − (amdgpu_is_atpx_hybrid() || | |
| 162 | − amdgpu_has_atpx_dgpu_power_cntl()) && | |
| 163 | − ((flags & AMD_IS_APU) == 0) && | |
| 164 | − !pci_is_thunderbolt_attached(to_pci_dev(dev->dev))) | |
| 165 | − flags |= AMD_IS_PX; | |
| 166 | − | |
| 167 | − parent = pci_upstream_bridge(adev->pdev); | |
| 168 | − adev->has_pr3 = parent ? pci_pr3_present(parent) : false; | |
| 169 | − | |
| 170 | 138 | /* amdgpu_device_init should report only fatal error |
| 171 | 139 | * like memory allocation failure or iomapping failure, |
| 172 | 140 | * or memory manager initialization failure, it must |
| @@ -224,58 +192,12 @@ int amdgpu_driver_load_kms(struct amdgpu_device *adev, unsigned long flags) · 51 unmodified lines | ||
| 224 | 192 | if (acpi_status) |
| 225 | 193 | dev_dbg(dev->dev, "Error during ACPI methods call\n"); |
| 226 | 194 | |
| 227 | − if (adev->runpm) { | |
| 228 | − /* only need to skip on ATPX */ | |
| 229 | − if (amdgpu_device_supports_px(dev)) | |
| 230 | − dev_pm_set_driver_flags(dev->dev, DPM_FLAG_NO_DIRECT_COMPLETE); | |
| 231 | − /* we want direct complete for BOCO */ | |
| 232 | − if (amdgpu_device_supports_boco(dev)) | |
| 233 | − dev_pm_set_driver_flags(dev->dev, DPM_FLAG_SMART_PREPARE | | |
| 234 | − DPM_FLAG_SMART_SUSPEND | | |
| 235 | − DPM_FLAG_MAY_SKIP_RESUME); | |
| 236 | − pm_runtime_use_autosuspend(dev->dev); | |
| 237 | − pm_runtime_set_autosuspend_delay(dev->dev, 5000); | |
| 238 | − | |
| 239 | − pm_runtime_allow(dev->dev); | |
| 240 | − | |
| 241 | − pm_runtime_mark_last_busy(dev->dev); | |
| 242 | − pm_runtime_put_autosuspend(dev->dev); | |
| 243 | − | |
| 244 | − /* | |
| 245 | − * For runpm implemented via BACO, PMFW will handle the | |
| 246 | − * timing for BACO in and out: | |
| 247 | − * - put ASIC into BACO state only when both video and | |
| 248 | − * audio functions are in D3 state. | |
| 249 | − * - pull ASIC out of BACO state when either video or | |
| 250 | − * audio function is in D0 state. | |
| 251 | − * Also, at startup, PMFW assumes both functions are in | |
| 252 | − * D0 state. | |
| 253 | − * | |
| 254 | − * So if snd driver was loaded prior to amdgpu driver | |
| 255 | − * and audio function was put into D3 state, there will | |
| 256 | − * be no PMFW-aware D-state transition(D0->D3) on runpm | |
| 257 | − * suspend. Thus the BACO will be not correctly kicked in. | |
| 258 | − * | |
| 259 | − * Via amdgpu_get_audio_func(), the audio dev is put | |
| 260 | − * into D0 state. Then there will be a PMFW-aware D-state | |
| 261 | − * transition(D0->D3) on runpm suspend. | |
| 262 | − */ | |
| 263 | − if (amdgpu_device_supports_baco(dev) && | |
| 264 | − !(adev->flags & AMD_IS_APU) && | |
| 265 | − (adev->asic_type >= CHIP_NAVI10)) | |
| 266 | − amdgpu_get_audio_func(adev); | |
| 267 | − } | |
| 268 | − | |
| 269 | 195 | if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DRV_LOAD)) |
| 270 | 196 | DRM_WARN("smart shift update failed\n"); |
| 271 | 197 | |
| 272 | 198 | out: |
| 273 | − if (r) { | |
| 274 | − /* balance pm_runtime_get_sync in amdgpu_driver_unload_kms */ | |
| 275 | − if (adev->rmmio && adev->runpm) | |
| 276 | − pm_runtime_put_noidle(dev->dev); | |
| 199 | + if (r) | |
| 277 | 200 | amdgpu_driver_unload_kms(dev); |
| 278 | − } | |
| 279 | 201 | |
| 280 | 202 | return r; |
| 281 | 203 | } |
| @@ -406,6 +328,10 @@ static int amdgpu_firmware_info(struct drm_amdgpu_info_firmware *fw_info, · 124 unmodified lines | ||
| 406 | 328 | fw_info->ver = adev->psp.toc.fw_version; |
| 407 | 329 | fw_info->feature = adev->psp.toc.feature_version; |
| 408 | 330 | break; |
| 331 | + case AMDGPU_INFO_FW_CAP: | |
| 332 | + fw_info->ver = adev->psp.cap_fw_version; | |
| 333 | + fw_info->feature = adev->psp.cap_feature_version; | |
| 334 | + break; | |
| 409 | 335 | default: |
| 410 | 336 | return -EINVAL; |
| 411 | 337 | } |
| @@ -678,13 +604,13 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 266 unmodified lines | ||
| 678 | 604 | ui64 = atomic64_read(&adev->num_vram_cpu_page_faults); |
| 679 | 605 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; |
| 680 | 606 | case AMDGPU_INFO_VRAM_USAGE: |
| 681 | − ui64 = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr); | |
| 607 | + ui64 = ttm_resource_manager_usage(&adev->mman.vram_mgr.manager); | |
| 682 | 608 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; |
| 683 | 609 | case AMDGPU_INFO_VIS_VRAM_USAGE: |
| 684 | 610 | ui64 = amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr); |
| 685 | 611 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; |
| 686 | 612 | case AMDGPU_INFO_GTT_USAGE: |
| 687 | − ui64 = amdgpu_gtt_mgr_usage(&adev->mman.gtt_mgr); | |
| 613 | + ui64 = ttm_resource_manager_usage(&adev->mman.gtt_mgr.manager); | |
| 688 | 614 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; |
| 689 | 615 | case AMDGPU_INFO_GDS_CONFIG: { |
| 690 | 616 | struct drm_amdgpu_info_gds gds_info; |
| @@ -716,14 +642,17 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 25 unmodified lines | ||
| 716 | 642 | case AMDGPU_INFO_MEMORY: { |
| 717 | 643 | struct drm_amdgpu_memory_info mem; |
| 718 | 644 | struct ttm_resource_manager *gtt_man = |
| 719 | − ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); | |
| 645 | + &adev->mman.gtt_mgr.manager; | |
| 646 | + struct ttm_resource_manager *vram_man = | |
| 647 | + &adev->mman.vram_mgr.manager; | |
| 648 | + | |
| 720 | 649 | memset(&mem, 0, sizeof(mem)); |
| 721 | 650 | mem.vram.total_heap_size = adev->gmc.real_vram_size; |
| 722 | 651 | mem.vram.usable_heap_size = adev->gmc.real_vram_size - |
| 723 | 652 | atomic64_read(&adev->vram_pin_size) - |
| 724 | 653 | AMDGPU_VM_RESERVED_VRAM; |
| 725 | 654 | mem.vram.heap_usage = |
| 726 | − amdgpu_vram_mgr_usage(&adev->mman.vram_mgr); | |
| 655 | + ttm_resource_manager_usage(vram_man); | |
| 727 | 656 | mem.vram.max_allocation = mem.vram.usable_heap_size * 3 / 4; |
| 728 | 657 | |
| 729 | 658 | mem.cpu_accessible_vram.total_heap_size = |
| @@ -741,8 +670,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 11 unmodified lines | ||
| 741 | 670 | mem.gtt.total_heap_size *= PAGE_SIZE; |
| 742 | 671 | mem.gtt.usable_heap_size = mem.gtt.total_heap_size - |
| 743 | 672 | atomic64_read(&adev->gart_pin_size); |
| 744 | − mem.gtt.heap_usage = | |
| 745 | − amdgpu_gtt_mgr_usage(&adev->mman.gtt_mgr); | |
| 673 | + mem.gtt.heap_usage = ttm_resource_manager_usage(gtt_man); | |
| 746 | 674 | mem.gtt.max_allocation = mem.gtt.usable_heap_size * 3 / 4; |
| 747 | 675 | |
| 748 | 676 | return copy_to_user(out, &mem, |
| @@ -1268,18 +1196,20 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, · 519 unmodified lines | ||
| 1268 | 1196 | if (amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_VCE) != NULL) |
| 1269 | 1197 | amdgpu_vce_free_handles(adev, file_priv); |
| 1270 | 1198 | |
| 1271 | − amdgpu_vm_bo_rmv(adev, fpriv->prt_va); | |
| 1272 | − | |
| 1273 | 1199 | if (amdgpu_mcbp || amdgpu_sriov_vf(adev)) { |
| 1274 | 1200 | /* TODO: how to handle reserve failure */ |
| 1275 | 1201 | BUG_ON(amdgpu_bo_reserve(adev->virt.csa_obj, true)); |
| 1276 | − amdgpu_vm_bo_rmv(adev, fpriv->csa_va); | |
| 1202 | + amdgpu_vm_bo_del(adev, fpriv->csa_va); | |
| 1277 | 1203 | fpriv->csa_va = NULL; |
| 1278 | 1204 | amdgpu_bo_unreserve(adev->virt.csa_obj); |
| 1279 | 1205 | } |
| 1280 | 1206 | |
| 1281 | 1207 | pasid = fpriv->vm.pasid; |
| 1282 | 1208 | pd = amdgpu_bo_ref(fpriv->vm.root.bo); |
| 1209 | + if (!WARN_ON(amdgpu_bo_reserve(pd, true))) { | |
| 1210 | + amdgpu_vm_bo_del(adev, fpriv->prt_va); | |
| 1211 | + amdgpu_bo_unreserve(pd); | |
| 1212 | + } | |
| 1283 | 1213 | |
| 1284 | 1214 | amdgpu_ctx_mgr_fini(&fpriv->ctx_mgr); |
| 1285 | 1215 | amdgpu_vm_fini(adev, &fpriv->vm); |
| @@ -1427,8 +1357,7 @@ static int amdgpu_debugfs_firmware_info_show(struct seq_file *m, void *unused) · 141 unmodified lines | ||
| 1427 | 1357 | struct drm_amdgpu_info_firmware fw_info; |
| 1428 | 1358 | struct drm_amdgpu_query_fw query_fw; |
| 1429 | 1359 | struct atom_context *ctx = adev->mode_info.atom_context; |
| 1430 | − uint8_t smu_minor, smu_debug; | |
| 1431 | − uint16_t smu_major; | |
| 1360 | + uint8_t smu_program, smu_major, smu_minor, smu_debug; | |
| 1432 | 1361 | int ret, i; |
| 1433 | 1362 | |
| 1434 | 1363 | static const char *ta_fw_name[TA_FW_TYPE_MAX_INDEX] = { |
| @@ -1574,11 +1503,12 @@ static int amdgpu_debugfs_firmware_info_show(struct seq_file *m, void *unused) · 139 unmodified lines | ||
| 1574 | 1503 | ret = amdgpu_firmware_info(&fw_info, &query_fw, adev); |
| 1575 | 1504 | if (ret) |
| 1576 | 1505 | return ret; |
| 1577 | − smu_major = (fw_info.ver >> 16) & 0xffff; | |
| 1506 | + smu_program = (fw_info.ver >> 24) & 0xff; | |
| 1507 | + smu_major = (fw_info.ver >> 16) & 0xff; | |
| 1578 | 1508 | smu_minor = (fw_info.ver >> 8) & 0xff; |
| 1579 | 1509 | smu_debug = (fw_info.ver >> 0) & 0xff; |
| 1580 | − seq_printf(m, "SMC feature version: %u, firmware version: 0x%08x (%d.%d.%d)\n", | |
| 1581 | − fw_info.feature, fw_info.ver, smu_major, smu_minor, smu_debug); | |
| 1510 | + seq_printf(m, "SMC feature version: %u, program: %d, firmware version: 0x%08x (%d.%d.%d)\n", | |
| 1511 | + fw_info.feature, smu_program, fw_info.ver, smu_major, smu_minor, smu_debug); | |
| 1582 | 1512 | |
| 1583 | 1513 | /* SDMA */ |
| 1584 | 1514 | query_fw.fw_type = AMDGPU_INFO_FW_SDMA; |
| @@ -1623,6 +1553,16 @@ static int amdgpu_debugfs_firmware_info_show(struct seq_file *m, void *unused) · 38 unmodified lines | ||
| 1623 | 1553 | seq_printf(m, "TOC feature version: %u, firmware version: 0x%08x\n", |
| 1624 | 1554 | fw_info.feature, fw_info.ver); |
| 1625 | 1555 | |
| 1556 | + /* CAP */ | |
| 1557 | + if (adev->psp.cap_fw) { | |
| 1558 | + query_fw.fw_type = AMDGPU_INFO_FW_CAP; | |
| 1559 | + ret = amdgpu_firmware_info(&fw_info, &query_fw, adev); | |
| 1560 | + if (ret) | |
| 1561 | + return ret; | |
| 1562 | + seq_printf(m, "CAP feature version: %u, firmware version: 0x%08x\n", | |
| 1563 | + fw_info.feature, fw_info.ver); | |
| 1564 | + } | |
| 1565 | + | |
| 1626 | 1566 | seq_printf(m, "VBIOS version: %s\n", ctx->vbios_version); |
| 1627 | 1567 | |
| 1628 | 1568 | return 0; |
| @@ -71,47 +71,8 @@ void amdgpu_mca_query_ras_error_count(struct amdgpu_device *adev, · 70 unmodified lines | ||
| 71 | 71 | amdgpu_mca_reset_error_count(adev, mc_status_addr); |
| 72 | 72 | } |
| 73 | 73 | |
| 74 | −int amdgpu_mca_ras_late_init(struct amdgpu_device *adev, | |
| 75 | − struct amdgpu_mca_ras *mca_dev) | |
| 76 | −{ | |
| 77 | − int r; | |
| 78 | − struct ras_ih_if ih_info = { | |
| 79 | − .cb = NULL, | |
| 80 | − }; | |
| 81 | − struct ras_fs_if fs_info = { | |
| 82 | − .sysfs_name = mca_dev->ras_funcs->sysfs_name, | |
| 83 | − }; | |
| 84 | − | |
| 85 | − if (!mca_dev->ras_if) { | |
| 86 | − mca_dev->ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 87 | − if (!mca_dev->ras_if) | |
| 88 | − return -ENOMEM; | |
| 89 | − mca_dev->ras_if->block = mca_dev->ras_funcs->ras_block; | |
| 90 | − mca_dev->ras_if->sub_block_index = mca_dev->ras_funcs->ras_sub_block; | |
| 91 | − mca_dev->ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 92 | − } | |
| 93 | − ih_info.head = fs_info.head = *mca_dev->ras_if; | |
| 94 | − r = amdgpu_ras_late_init(adev, mca_dev->ras_if, | |
| 95 | − &fs_info, &ih_info); | |
| 96 | − if (r || !amdgpu_ras_is_supported(adev, mca_dev->ras_if->block)) { | |
| 97 | − kfree(mca_dev->ras_if); | |
| 98 | − mca_dev->ras_if = NULL; | |
| 99 | − } | |
| 100 | − | |
| 101 | − return r; | |
| 102 | −} | |
| 103 | − | |
| 104 | 74 | void amdgpu_mca_ras_fini(struct amdgpu_device *adev, |
| 105 | 75 | struct amdgpu_mca_ras *mca_dev) |
| 106 | 76 | { |
| 107 | − struct ras_ih_if ih_info = { | |
| 108 | − .cb = NULL, | |
| 109 | − }; | |
| 110 | − | |
| 111 | − if (!mca_dev->ras_if) | |
| 112 | − return; | |
| 113 | − | |
| 114 | − amdgpu_ras_late_fini(adev, mca_dev->ras_if, &ih_info); | |
| 115 | − kfree(mca_dev->ras_if); | |
| 116 | − mca_dev->ras_if = NULL; | |
| 77 | + amdgpu_ras_block_late_fini(adev, mca_dev->ras_if); | |
| 117 | 78 | } |
| @@ -21,21 +21,13 @@ · 20 unmodified lines | ||
| 21 | 21 | #ifndef __AMDGPU_MCA_H__ |
| 22 | 22 | #define __AMDGPU_MCA_H__ |
| 23 | 23 | |
| 24 | −struct amdgpu_mca_ras_funcs { | |
| 25 | − int (*ras_late_init)(struct amdgpu_device *adev); | |
| 26 | − void (*ras_fini)(struct amdgpu_device *adev); | |
| 27 | − void (*query_ras_error_count)(struct amdgpu_device *adev, | |
| 28 | − void *ras_error_status); | |
| 29 | − void (*query_ras_error_address)(struct amdgpu_device *adev, | |
| 30 | − void *ras_error_status); | |
| 31 | − uint32_t ras_block; | |
| 32 | − uint32_t ras_sub_block; | |
| 33 | − const char* sysfs_name; | |
| 24 | +struct amdgpu_mca_ras_block { | |
| 25 | + struct amdgpu_ras_block_object ras_block; | |
| 34 | 26 | }; |
| 35 | 27 | |
| 36 | 28 | struct amdgpu_mca_ras { |
| 37 | 29 | struct ras_common_if *ras_if; |
| 38 | − const struct amdgpu_mca_ras_funcs *ras_funcs; | |
| 30 | + struct amdgpu_mca_ras_block *ras; | |
| 39 | 31 | }; |
| 40 | 32 | |
| 41 | 33 | struct amdgpu_mca_funcs { |
| @@ -64,9 +56,6 @@ void amdgpu_mca_query_ras_error_count(struct amdgpu_device *adev, · 22 unmodified lines | ||
| 64 | 56 | uint64_t mc_status_addr, |
| 65 | 57 | void *ras_error_status); |
| 66 | 58 | |
| 67 | −int amdgpu_mca_ras_late_init(struct amdgpu_device *adev, | |
| 68 | − struct amdgpu_mca_ras *mca_dev); | |
| 69 | − | |
| 70 | 59 | void amdgpu_mca_ras_fini(struct amdgpu_device *adev, |
| 71 | 60 | struct amdgpu_mca_ras *mca_dev); |
| 72 | 61 | |
| @@ -24,45 +24,9 @@ · 23 unmodified lines | ||
| 24 | 24 | #include "amdgpu.h" |
| 25 | 25 | #include "amdgpu_ras.h" |
| 26 | 26 | |
| 27 | −int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev) | |
| 28 | −{ | |
| 29 | − int r; | |
| 30 | − struct ras_ih_if ih_info = { | |
| 31 | − .cb = NULL, | |
| 32 | − }; | |
| 33 | − struct ras_fs_if fs_info = { | |
| 34 | − .sysfs_name = "mmhub_err_count", | |
| 35 | − }; | |
| 36 | − | |
| 37 | − if (!adev->mmhub.ras_if) { | |
| 38 | − adev->mmhub.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 39 | − if (!adev->mmhub.ras_if) | |
| 40 | − return -ENOMEM; | |
| 41 | − adev->mmhub.ras_if->block = AMDGPU_RAS_BLOCK__MMHUB; | |
| 42 | − adev->mmhub.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 43 | − adev->mmhub.ras_if->sub_block_index = 0; | |
| 44 | − } | |
| 45 | − ih_info.head = fs_info.head = *adev->mmhub.ras_if; | |
| 46 | − r = amdgpu_ras_late_init(adev, adev->mmhub.ras_if, | |
| 47 | − &fs_info, &ih_info); | |
| 48 | − if (r || !amdgpu_ras_is_supported(adev, adev->mmhub.ras_if->block)) { | |
| 49 | − kfree(adev->mmhub.ras_if); | |
| 50 | − adev->mmhub.ras_if = NULL; | |
| 51 | − } | |
| 52 | − | |
| 53 | − return r; | |
| 54 | −} | |
| 55 | − | |
| 56 | 27 | void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev) |
| 57 | 28 | { |
| 58 | 29 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__MMHUB) && |
| 59 | − adev->mmhub.ras_if) { | |
| 60 | − struct ras_common_if *ras_if = adev->mmhub.ras_if; | |
| 61 | − struct ras_ih_if ih_info = { | |
| 62 | − .cb = NULL, | |
| 63 | − }; | |
| 64 | − | |
| 65 | − amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 66 | − kfree(ras_if); | |
| 67 | − } | |
| 30 | + adev->mmhub.ras_if) | |
| 31 | + amdgpu_ras_block_late_fini(adev, adev->mmhub.ras_if); | |
| 68 | 32 | } |
| @@ -21,14 +21,8 @@ · 20 unmodified lines | ||
| 21 | 21 | #ifndef __AMDGPU_MMHUB_H__ |
| 22 | 22 | #define __AMDGPU_MMHUB_H__ |
| 23 | 23 | |
| 24 | −struct amdgpu_mmhub_ras_funcs { | |
| 25 | − int (*ras_late_init)(struct amdgpu_device *adev); | |
| 26 | − void (*ras_fini)(struct amdgpu_device *adev); | |
| 27 | − void (*query_ras_error_count)(struct amdgpu_device *adev, | |
| 28 | − void *ras_error_status); | |
| 29 | − void (*query_ras_error_status)(struct amdgpu_device *adev); | |
| 30 | − void (*reset_ras_error_count)(struct amdgpu_device *adev); | |
| 31 | − void (*reset_ras_error_status)(struct amdgpu_device *adev); | |
| 24 | +struct amdgpu_mmhub_ras { | |
| 25 | + struct amdgpu_ras_block_object ras_block; | |
| 32 | 26 | }; |
| 33 | 27 | |
| 34 | 28 | struct amdgpu_mmhub_funcs { |
| @@ -50,10 +44,9 @@ struct amdgpu_mmhub_funcs { · 15 unmodified lines | ||
| 50 | 44 | struct amdgpu_mmhub { |
| 51 | 45 | struct ras_common_if *ras_if; |
| 52 | 46 | const struct amdgpu_mmhub_funcs *funcs; |
| 53 | − const struct amdgpu_mmhub_ras_funcs *ras_funcs; | |
| 47 | + struct amdgpu_mmhub_ras *ras; | |
| 54 | 48 | }; |
| 55 | 49 | |
| 56 | −int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev); | |
| 57 | 50 | void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev); |
| 58 | 51 | #endif |
| 59 | 52 | |
| @@ -341,6 +341,7 @@ struct amdgpu_mode_info { · 340 unmodified lines | ||
| 341 | 341 | int num_crtc; /* number of crtcs */ |
| 342 | 342 | int num_hpd; /* number of hpd pins */ |
| 343 | 343 | int num_dig; /* number of dig blocks */ |
| 344 | + bool gpu_vm_support; /* supports display from GTT */ | |
| 344 | 345 | int disp_priority; |
| 345 | 346 | const struct amdgpu_display_funcs *funcs; |
| 346 | 347 | const enum drm_plane_type *plane_type; |
| @@ -22,61 +22,31 @@ · 21 unmodified lines | ||
| 22 | 22 | #include "amdgpu.h" |
| 23 | 23 | #include "amdgpu_ras.h" |
| 24 | 24 | |
| 25 | −int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev) | |
| 25 | +int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block) | |
| 26 | 26 | { |
| 27 | 27 | int r; |
| 28 | − struct ras_ih_if ih_info = { | |
| 29 | − .cb = NULL, | |
| 30 | − }; | |
| 31 | − struct ras_fs_if fs_info = { | |
| 32 | − .sysfs_name = "pcie_bif_err_count", | |
| 33 | − }; | |
| 34 | − | |
| 35 | − if (!adev->nbio.ras_if) { | |
| 36 | − adev->nbio.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 37 | − if (!adev->nbio.ras_if) | |
| 38 | − return -ENOMEM; | |
| 39 | − adev->nbio.ras_if->block = AMDGPU_RAS_BLOCK__PCIE_BIF; | |
| 40 | − adev->nbio.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 41 | − adev->nbio.ras_if->sub_block_index = 0; | |
| 42 | − } | |
| 43 | − ih_info.head = fs_info.head = *adev->nbio.ras_if; | |
| 44 | − r = amdgpu_ras_late_init(adev, adev->nbio.ras_if, | |
| 45 | − &fs_info, &ih_info); | |
| 28 | + r = amdgpu_ras_block_late_init(adev, ras_block); | |
| 46 | 29 | if (r) |
| 47 | − goto free; | |
| 30 | + return r; | |
| 48 | 31 | |
| 49 | − if (amdgpu_ras_is_supported(adev, adev->nbio.ras_if->block)) { | |
| 32 | + if (amdgpu_ras_is_supported(adev, ras_block->block)) { | |
| 50 | 33 | r = amdgpu_irq_get(adev, &adev->nbio.ras_controller_irq, 0); |
| 51 | 34 | if (r) |
| 52 | 35 | goto late_fini; |
| 53 | 36 | r = amdgpu_irq_get(adev, &adev->nbio.ras_err_event_athub_irq, 0); |
| 54 | 37 | if (r) |
| 55 | 38 | goto late_fini; |
| 56 | − } else { | |
| 57 | − r = 0; | |
| 58 | − goto free; | |
| 59 | 39 | } |
| 60 | 40 | |
| 61 | 41 | return 0; |
| 62 | 42 | late_fini: |
| 63 | − amdgpu_ras_late_fini(adev, adev->nbio.ras_if, &ih_info); | |
| 64 | −free: | |
| 65 | − kfree(adev->nbio.ras_if); | |
| 66 | − adev->nbio.ras_if = NULL; | |
| 43 | + amdgpu_ras_block_late_fini(adev, ras_block); | |
| 67 | 44 | return r; |
| 68 | 45 | } |
| 69 | 46 | |
| 70 | 47 | void amdgpu_nbio_ras_fini(struct amdgpu_device *adev) |
| 71 | 48 | { |
| 72 | 49 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF) && |
| 73 | − adev->nbio.ras_if) { | |
| 74 | − struct ras_common_if *ras_if = adev->nbio.ras_if; | |
| 75 | − struct ras_ih_if ih_info = { | |
| 76 | − .cb = NULL, | |
| 77 | − }; | |
| 78 | − | |
| 79 | − amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 80 | − kfree(ras_if); | |
| 81 | − } | |
| 50 | + adev->nbio.ras_if) | |
| 51 | + amdgpu_ras_block_late_fini(adev, adev->nbio.ras_if); | |
| 82 | 52 | } |
| @@ -47,15 +47,12 @@ struct nbio_hdp_flush_reg { · 46 unmodified lines | ||
| 47 | 47 | u32 ref_and_mask_sdma7; |
| 48 | 48 | }; |
| 49 | 49 | |
| 50 | −struct amdgpu_nbio_ras_funcs { | |
| 50 | +struct amdgpu_nbio_ras { | |
| 51 | + struct amdgpu_ras_block_object ras_block; | |
| 51 | 52 | void (*handle_ras_controller_intr_no_bifring)(struct amdgpu_device *adev); |
| 52 | 53 | void (*handle_ras_err_event_athub_intr_no_bifring)(struct amdgpu_device *adev); |
| 53 | 54 | int (*init_ras_controller_interrupt)(struct amdgpu_device *adev); |
| 54 | 55 | int (*init_ras_err_event_athub_interrupt)(struct amdgpu_device *adev); |
| 55 | − void (*query_ras_error_count)(struct amdgpu_device *adev, | |
| 56 | − void *ras_error_status); | |
| 57 | − int (*ras_late_init)(struct amdgpu_device *adev); | |
| 58 | − void (*ras_fini)(struct amdgpu_device *adev); | |
| 59 | 56 | }; |
| 60 | 57 | |
| 61 | 58 | struct amdgpu_nbio_funcs { |
| @@ -104,9 +101,9 @@ struct amdgpu_nbio { · 42 unmodified lines | ||
| 104 | 101 | struct amdgpu_irq_src ras_err_event_athub_irq; |
| 105 | 102 | struct ras_common_if *ras_if; |
| 106 | 103 | const struct amdgpu_nbio_funcs *funcs; |
| 107 | − const struct amdgpu_nbio_ras_funcs *ras_funcs; | |
| 104 | + struct amdgpu_nbio_ras *ras; | |
| 108 | 105 | }; |
| 109 | 106 | |
| 110 | −int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev); | |
| 107 | +int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block); | |
| 111 | 108 | void amdgpu_nbio_ras_fini(struct amdgpu_device *adev); |
| 112 | 109 | #endif |
| @@ -451,7 +451,7 @@ static bool amdgpu_bo_validate_size(struct amdgpu_device *adev, · 450 unmodified lines | ||
| 451 | 451 | if (domain & AMDGPU_GEM_DOMAIN_GTT) { |
| 452 | 452 | man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); |
| 453 | 453 | |
| 454 | − if (size < (man->size << PAGE_SHIFT)) | |
| 454 | + if (size < man->size) | |
| 455 | 455 | return true; |
| 456 | 456 | else |
| 457 | 457 | goto fail; |
| @@ -460,7 +460,7 @@ static bool amdgpu_bo_validate_size(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 460 | 460 | if (domain & AMDGPU_GEM_DOMAIN_VRAM) { |
| 461 | 461 | man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM); |
| 462 | 462 | |
| 463 | − if (size < (man->size << PAGE_SHIFT)) | |
| 463 | + if (size < man->size) | |
| 464 | 464 | return true; |
| 465 | 465 | else |
| 466 | 466 | goto fail; |
| @@ -575,6 +575,9 @@ int amdgpu_bo_create(struct amdgpu_device *adev, · 108 unmodified lines | ||
| 575 | 575 | if (!amdgpu_bo_support_uswc(bo->flags)) |
| 576 | 576 | bo->flags &= ~AMDGPU_GEM_CREATE_CPU_GTT_USWC; |
| 577 | 577 | |
| 578 | + if (adev->ras_enabled) | |
| 579 | + bo->flags |= AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE; | |
| 580 | + | |
| 578 | 581 | bo->tbo.bdev = &adev->mman.bdev; |
| 579 | 582 | if (bp->domain & (AMDGPU_GEM_DOMAIN_GWS | AMDGPU_GEM_DOMAIN_OA | |
| 580 | 583 | AMDGPU_GEM_DOMAIN_GDS)) |
| @@ -1303,7 +1306,8 @@ void amdgpu_bo_release_notify(struct ttm_buffer_object *bo) · 722 unmodified lines | ||
| 1303 | 1306 | !(abo->flags & AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE)) |
| 1304 | 1307 | return; |
| 1305 | 1308 | |
| 1306 | − dma_resv_lock(bo->base.resv, NULL); | |
| 1309 | + if (WARN_ON_ONCE(!dma_resv_trylock(bo->base.resv))) | |
| 1310 | + return; | |
| 1307 | 1311 | |
| 1308 | 1312 | r = amdgpu_fill_buffer(abo, AMDGPU_POISON, bo->base.resv, &fence); |
| 1309 | 1313 | if (!WARN_ON(r)) { |
| @@ -25,12 +25,6 @@ · 24 unmodified lines | ||
| 25 | 25 | |
| 26 | 26 | #include "amdgpu.h" |
| 27 | 27 | |
| 28 | −static inline struct amdgpu_preempt_mgr * | |
| 29 | −to_preempt_mgr(struct ttm_resource_manager *man) | |
| 30 | −{ | |
| 31 | − return container_of(man, struct amdgpu_preempt_mgr, manager); | |
| 32 | −} | |
| 33 | − | |
| 34 | 28 | /** |
| 35 | 29 | * DOC: mem_info_preempt_used |
| 36 | 30 | * |
| @@ -45,10 +39,9 @@ static ssize_t mem_info_preempt_used_show(struct device *dev, · 8 unmodified lines | ||
| 45 | 39 | { |
| 46 | 40 | struct drm_device *ddev = dev_get_drvdata(dev); |
| 47 | 41 | struct amdgpu_device *adev = drm_to_adev(ddev); |
| 48 | − struct ttm_resource_manager *man; | |
| 42 | + struct ttm_resource_manager *man = &adev->mman.preempt_mgr; | |
| 49 | 43 | |
| 50 | − man = ttm_manager_type(&adev->mman.bdev, AMDGPU_PL_PREEMPT); | |
| 51 | − return sysfs_emit(buf, "%llu\n", amdgpu_preempt_mgr_usage(man)); | |
| 44 | + return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man)); | |
| 52 | 45 | } |
| 53 | 46 | |
| 54 | 47 | static DEVICE_ATTR_RO(mem_info_preempt_used); |
| @@ -68,16 +61,12 @@ static int amdgpu_preempt_mgr_new(struct ttm_resource_manager *man, · 13 unmodified lines | ||
| 68 | 61 | const struct ttm_place *place, |
| 69 | 62 | struct ttm_resource **res) |
| 70 | 63 | { |
| 71 | − struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man); | |
| 72 | − | |
| 73 | 64 | *res = kzalloc(sizeof(**res), GFP_KERNEL); |
| 74 | 65 | if (!*res) |
| 75 | 66 | return -ENOMEM; |
| 76 | 67 | |
| 77 | 68 | ttm_resource_init(tbo, place, *res); |
| 78 | 69 | (*res)->start = AMDGPU_BO_INVALID_OFFSET; |
| 79 | − | |
| 80 | − atomic64_add((*res)->num_pages, &mgr->used); | |
| 81 | 70 | return 0; |
| 82 | 71 | } |
| 83 | 72 | |
| @@ -92,49 +81,13 @@ static int amdgpu_preempt_mgr_new(struct ttm_resource_manager *man, · 8 unmodified lines | ||
| 92 | 81 | static void amdgpu_preempt_mgr_del(struct ttm_resource_manager *man, |
| 93 | 82 | struct ttm_resource *res) |
| 94 | 83 | { |
| 95 | − struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man); | |
| 96 | − | |
| 97 | − atomic64_sub(res->num_pages, &mgr->used); | |
| 98 | 84 | ttm_resource_fini(man, res); |
| 99 | 85 | kfree(res); |
| 100 | 86 | } |
| 101 | 87 | |
| 102 | −/** | |
| 103 | − * amdgpu_preempt_mgr_usage - return usage of PREEMPT domain | |
| 104 | − * | |
| 105 | − * @man: TTM memory type manager | |
| 106 | − * | |
| 107 | − * Return how many bytes are used in the GTT domain | |
| 108 | − */ | |
| 109 | −uint64_t amdgpu_preempt_mgr_usage(struct ttm_resource_manager *man) | |
| 110 | −{ | |
| 111 | − struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man); | |
| 112 | − s64 result = atomic64_read(&mgr->used); | |
| 113 | − | |
| 114 | − return (result > 0 ? result : 0) * PAGE_SIZE; | |
| 115 | −} | |
| 116 | − | |
| 117 | −/** | |
| 118 | − * amdgpu_preempt_mgr_debug - dump VRAM table | |
| 119 | − * | |
| 120 | − * @man: TTM memory type manager | |
| 121 | − * @printer: DRM printer to use | |
| 122 | − * | |
| 123 | − * Dump the table content using printk. | |
| 124 | − */ | |
| 125 | −static void amdgpu_preempt_mgr_debug(struct ttm_resource_manager *man, | |
| 126 | − struct drm_printer *printer) | |
| 127 | −{ | |
| 128 | − struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man); | |
| 129 | − | |
| 130 | − drm_printf(printer, "man size:%llu pages, preempt used:%lld pages\n", | |
| 131 | − man->size, (u64)atomic64_read(&mgr->used)); | |
| 132 | −} | |
| 133 | − | |
| 134 | 88 | static const struct ttm_resource_manager_func amdgpu_preempt_mgr_func = { |
| 135 | 89 | .alloc = amdgpu_preempt_mgr_new, |
| 136 | 90 | .free = amdgpu_preempt_mgr_del, |
| 137 | − .debug = amdgpu_preempt_mgr_debug | |
| 138 | 91 | }; |
| 139 | 92 | |
| 140 | 93 | /** |
| @@ -146,8 +99,7 @@ static const struct ttm_resource_manager_func amdgpu_preempt_mgr_func = { · 5 unmodified lines | ||
| 146 | 99 | */ |
| 147 | 100 | int amdgpu_preempt_mgr_init(struct amdgpu_device *adev) |
| 148 | 101 | { |
| 149 | − struct amdgpu_preempt_mgr *mgr = &adev->mman.preempt_mgr; | |
| 150 | − struct ttm_resource_manager *man = &mgr->manager; | |
| 102 | + struct ttm_resource_manager *man = &adev->mman.preempt_mgr; | |
| 151 | 103 | int ret; |
| 152 | 104 | |
| 153 | 105 | man->use_tt = true; |
| @@ -155,16 +107,13 @@ int amdgpu_preempt_mgr_init(struct amdgpu_device *adev) · 1 unmodified line | ||
| 155 | 107 | |
| 156 | 108 | ttm_resource_manager_init(man, &adev->mman.bdev, (1 << 30)); |
| 157 | 109 | |
| 158 | − atomic64_set(&mgr->used, 0); | |
| 159 | − | |
| 160 | 110 | ret = device_create_file(adev->dev, &dev_attr_mem_info_preempt_used); |
| 161 | 111 | if (ret) { |
| 162 | 112 | DRM_ERROR("Failed to create device file mem_info_preempt_used\n"); |
| 163 | 113 | return ret; |
| 164 | 114 | } |
| 165 | 115 | |
| 166 | − ttm_set_driver_manager(&adev->mman.bdev, AMDGPU_PL_PREEMPT, | |
| 167 | − &mgr->manager); | |
| 116 | + ttm_set_driver_manager(&adev->mman.bdev, AMDGPU_PL_PREEMPT, man); | |
| 168 | 117 | ttm_resource_manager_set_used(man, true); |
| 169 | 118 | return 0; |
| 170 | 119 | } |
| @@ -179,8 +128,7 @@ int amdgpu_preempt_mgr_init(struct amdgpu_device *adev) · 8 unmodified lines | ||
| 179 | 128 | */ |
| 180 | 129 | void amdgpu_preempt_mgr_fini(struct amdgpu_device *adev) |
| 181 | 130 | { |
| 182 | − struct amdgpu_preempt_mgr *mgr = &adev->mman.preempt_mgr; | |
| 183 | − struct ttm_resource_manager *man = &mgr->manager; | |
| 131 | + struct ttm_resource_manager *man = &adev->mman.preempt_mgr; | |
| 184 | 132 | int ret; |
| 185 | 133 | |
| 186 | 134 | ttm_resource_manager_set_used(man, false); |
| @@ -133,6 +133,8 @@ static int psp_early_init(void *handle) · 132 unmodified lines | ||
| 133 | 133 | break; |
| 134 | 134 | case IP_VERSION(13, 0, 1): |
| 135 | 135 | case IP_VERSION(13, 0, 3): |
| 136 | + case IP_VERSION(13, 0, 5): | |
| 137 | + case IP_VERSION(13, 0, 8): | |
| 136 | 138 | psp_v13_0_set_psp_funcs(psp); |
| 137 | 139 | psp->autoload_supported = true; |
| 138 | 140 | break; |
| @@ -259,6 +261,32 @@ static bool psp_get_runtime_db_entry(struct amdgpu_device *adev, · 120 unmodified lines | ||
| 259 | 261 | return ret; |
| 260 | 262 | } |
| 261 | 263 | |
| 264 | +static int psp_init_sriov_microcode(struct psp_context *psp) | |
| 265 | +{ | |
| 266 | + struct amdgpu_device *adev = psp->adev; | |
| 267 | + int ret = 0; | |
| 268 | + | |
| 269 | + switch (adev->ip_versions[MP0_HWIP][0]) { | |
| 270 | + case IP_VERSION(9, 0, 0): | |
| 271 | + ret = psp_init_cap_microcode(psp, "vega10"); | |
| 272 | + break; | |
| 273 | + case IP_VERSION(11, 0, 9): | |
| 274 | + ret = psp_init_cap_microcode(psp, "navi12"); | |
| 275 | + break; | |
| 276 | + case IP_VERSION(11, 0, 7): | |
| 277 | + ret = psp_init_cap_microcode(psp, "sienna_cichlid"); | |
| 278 | + break; | |
| 279 | + case IP_VERSION(13, 0, 2): | |
| 280 | + ret = psp_init_ta_microcode(psp, "aldebaran"); | |
| 281 | + break; | |
| 282 | + default: | |
| 283 | + BUG(); | |
| 284 | + break; | |
| 285 | + } | |
| 286 | + | |
| 287 | + return ret; | |
| 288 | +} | |
| 289 | + | |
| 262 | 290 | static int psp_sw_init(void *handle) |
| 263 | 291 | { |
| 264 | 292 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| @@ -273,19 +301,13 @@ static int psp_sw_init(void *handle) · 8 unmodified lines | ||
| 273 | 301 | ret = -ENOMEM; |
| 274 | 302 | } |
| 275 | 303 | |
| 276 | − if (!amdgpu_sriov_vf(adev)) { | |
| 304 | + if (amdgpu_sriov_vf(adev)) | |
| 305 | + ret = psp_init_sriov_microcode(psp); | |
| 306 | + else | |
| 277 | 307 | ret = psp_init_microcode(psp); |
| 278 | − if (ret) { | |
| 279 | − DRM_ERROR("Failed to load psp firmware!\n"); | |
| 280 | − return ret; | |
| 281 | − } | |
| 282 | − } else if (amdgpu_sriov_vf(adev) && | |
| 283 | − adev->ip_versions[MP0_HWIP][0] == IP_VERSION(13, 0, 2)) { | |
| 284 | − ret = psp_init_ta_microcode(psp, "aldebaran"); | |
| 285 | − if (ret) { | |
| 286 | − DRM_ERROR("Failed to initialize ta microcode!\n"); | |
| 287 | − return ret; | |
| 288 | − } | |
| 308 | + if (ret) { | |
| 309 | + DRM_ERROR("Failed to load psp firmware!\n"); | |
| 310 | + return ret; | |
| 289 | 311 | } |
| 290 | 312 | |
| 291 | 313 | memset(&boot_cfg_entry, 0, sizeof(boot_cfg_entry)); |
| @@ -353,6 +375,10 @@ static int psp_sw_fini(void *handle) · 61 unmodified lines | ||
| 353 | 375 | release_firmware(psp->ta_fw); |
| 354 | 376 | psp->ta_fw = NULL; |
| 355 | 377 | } |
| 378 | + if (adev->psp.cap_fw) { | |
| 379 | + release_firmware(psp->cap_fw); | |
| 380 | + psp->cap_fw = NULL; | |
| 381 | + } | |
| 356 | 382 | |
| 357 | 383 | if (adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 0) || |
| 358 | 384 | adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 7)) |
| @@ -491,7 +517,10 @@ psp_cmd_submit_buf(struct psp_context *psp, · 132 unmodified lines | ||
| 491 | 517 | DRM_WARN("psp gfx command %s(0x%X) failed and response status is (0x%X)\n", |
| 492 | 518 | psp_gfx_cmd_name(psp->cmd_buf_mem->cmd_id), psp->cmd_buf_mem->cmd_id, |
| 493 | 519 | psp->cmd_buf_mem->resp.status); |
| 494 | − if (!timeout) { | |
| 520 | + /* If we load CAP FW, PSP must return 0 under SRIOV | |
| 521 | + * also return failure in case of timeout | |
| 522 | + */ | |
| 523 | + if ((ucode && (ucode->ucode_id == AMDGPU_UCODE_ID_CAP)) || !timeout) { | |
| 495 | 524 | ret = -EINVAL; |
| 496 | 525 | goto exit; |
| 497 | 526 | } |
| @@ -914,19 +943,15 @@ static void psp_prep_ta_load_cmd_buf(struct psp_gfx_cmd_resp *cmd, · 416 unmodified lines | ||
| 914 | 943 | static int psp_ta_init_shared_buf(struct psp_context *psp, |
| 915 | 944 | struct ta_mem_context *mem_ctx) |
| 916 | 945 | { |
| 917 | − int ret; | |
| 918 | − | |
| 919 | 946 | /* |
| 920 | 947 | * Allocate 16k memory aligned to 4k from Frame Buffer (local |
| 921 | 948 | * physical) for ta to host memory |
| 922 | 949 | */ |
| 923 | − ret = amdgpu_bo_create_kernel(psp->adev, mem_ctx->shared_mem_size, | |
| 950 | + return amdgpu_bo_create_kernel(psp->adev, mem_ctx->shared_mem_size, | |
| 924 | 951 | PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM, |
| 925 | 952 | &mem_ctx->shared_bo, |
| 926 | 953 | &mem_ctx->shared_mc_addr, |
| 927 | 954 | &mem_ctx->shared_buf); |
| 928 | − | |
| 929 | − return ret; | |
| 930 | 955 | } |
| 931 | 956 | |
| 932 | 957 | static void psp_ta_free_shared_buf(struct ta_mem_context *mem_ctx) |
| @@ -1308,6 +1333,11 @@ static void psp_ras_ta_check_status(struct psp_context *psp) · 375 unmodified lines | ||
| 1308 | 1333 | break; |
| 1309 | 1334 | case TA_RAS_STATUS__SUCCESS: |
| 1310 | 1335 | break; |
| 1336 | + case TA_RAS_STATUS__TEE_ERROR_ACCESS_DENIED: | |
| 1337 | + if (ras_cmd->cmd_id == TA_RAS_COMMAND__TRIGGER_ERROR) | |
| 1338 | + dev_warn(psp->adev->dev, | |
| 1339 | + "RAS WARNING: Inject error to critical region is not allowed\n"); | |
| 1340 | + break; | |
| 1311 | 1341 | default: |
| 1312 | 1342 | dev_warn(psp->adev->dev, |
| 1313 | 1343 | "RAS WARNING: ras status = 0x%X\n", ras_cmd->ras_status); |
| @@ -1520,7 +1550,9 @@ int psp_ras_trigger_error(struct psp_context *psp, · 206 unmodified lines | ||
| 1520 | 1550 | if (amdgpu_ras_intr_triggered()) |
| 1521 | 1551 | return 0; |
| 1522 | 1552 | |
| 1523 | − if (ras_cmd->ras_status) | |
| 1553 | + if (ras_cmd->ras_status == TA_RAS_STATUS__TEE_ERROR_ACCESS_DENIED) | |
| 1554 | + return -EACCES; | |
| 1555 | + else if (ras_cmd->ras_status) | |
| 1524 | 1556 | return -EINVAL; |
| 1525 | 1557 | |
| 1526 | 1558 | return 0; |
| @@ -2051,6 +2083,9 @@ static int psp_get_fw_type(struct amdgpu_firmware_info *ucode, · 524 unmodified lines | ||
| 2051 | 2083 | enum psp_gfx_fw_type *type) |
| 2052 | 2084 | { |
| 2053 | 2085 | switch (ucode->ucode_id) { |
| 2086 | + case AMDGPU_UCODE_ID_CAP: | |
| 2087 | + *type = GFX_FW_TYPE_CAP; | |
| 2088 | + break; | |
| 2054 | 2089 | case AMDGPU_UCODE_ID_SDMA0: |
| 2055 | 2090 | *type = GFX_FW_TYPE_SDMA0; |
| 2056 | 2091 | break; |
| @@ -3217,6 +3252,58 @@ out: · 1160 unmodified lines | ||
| 3217 | 3252 | return err; |
| 3218 | 3253 | } |
| 3219 | 3254 | |
| 3255 | +int psp_init_cap_microcode(struct psp_context *psp, | |
| 3256 | + const char *chip_name) | |
| 3257 | +{ | |
| 3258 | + struct amdgpu_device *adev = psp->adev; | |
| 3259 | + char fw_name[PSP_FW_NAME_LEN]; | |
| 3260 | + const struct psp_firmware_header_v1_0 *cap_hdr_v1_0; | |
| 3261 | + struct amdgpu_firmware_info *info = NULL; | |
| 3262 | + int err = 0; | |
| 3263 | + | |
| 3264 | + if (!chip_name) { | |
| 3265 | + dev_err(adev->dev, "invalid chip name for cap microcode\n"); | |
| 3266 | + return -EINVAL; | |
| 3267 | + } | |
| 3268 | + | |
| 3269 | + if (!amdgpu_sriov_vf(adev)) { | |
| 3270 | + dev_err(adev->dev, "cap microcode should only be loaded under SRIOV\n"); | |
| 3271 | + return -EINVAL; | |
| 3272 | + } | |
| 3273 | + | |
| 3274 | + snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_cap.bin", chip_name); | |
| 3275 | + err = request_firmware(&adev->psp.cap_fw, fw_name, adev->dev); | |
| 3276 | + if (err) { | |
| 3277 | + dev_warn(adev->dev, "cap microcode does not exist, skip\n"); | |
| 3278 | + err = 0; | |
| 3279 | + goto out; | |
| 3280 | + } | |
| 3281 | + | |
| 3282 | + err = amdgpu_ucode_validate(adev->psp.cap_fw); | |
| 3283 | + if (err) { | |
| 3284 | + dev_err(adev->dev, "fail to initialize cap microcode\n"); | |
| 3285 | + goto out; | |
| 3286 | + } | |
| 3287 | + | |
| 3288 | + info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CAP]; | |
| 3289 | + info->ucode_id = AMDGPU_UCODE_ID_CAP; | |
| 3290 | + info->fw = adev->psp.cap_fw; | |
| 3291 | + cap_hdr_v1_0 = (const struct psp_firmware_header_v1_0 *) | |
| 3292 | + adev->psp.cap_fw->data; | |
| 3293 | + adev->firmware.fw_size += ALIGN( | |
| 3294 | + le32_to_cpu(cap_hdr_v1_0->header.ucode_size_bytes), PAGE_SIZE); | |
| 3295 | + adev->psp.cap_fw_version = le32_to_cpu(cap_hdr_v1_0->header.ucode_version); | |
| 3296 | + adev->psp.cap_feature_version = le32_to_cpu(cap_hdr_v1_0->sos.fw_version); | |
| 3297 | + adev->psp.cap_ucode_size = le32_to_cpu(cap_hdr_v1_0->header.ucode_size_bytes); | |
| 3298 | + | |
| 3299 | + return 0; | |
| 3300 | + | |
| 3301 | +out: | |
| 3302 | + release_firmware(adev->psp.cap_fw); | |
| 3303 | + adev->psp.cap_fw = NULL; | |
| 3304 | + return err; | |
| 3305 | +} | |
| 3306 | + | |
| 3220 | 3307 | static int psp_set_clockgating_state(void *handle, |
| 3221 | 3308 | enum amd_clockgating_state state) |
| 3222 | 3309 | { |
| @@ -306,6 +306,9 @@ struct psp_context · 305 unmodified lines | ||
| 306 | 306 | /* toc firmware */ |
| 307 | 307 | const struct firmware *toc_fw; |
| 308 | 308 | |
| 309 | + /* cap firmware */ | |
| 310 | + const struct firmware *cap_fw; | |
| 311 | + | |
| 309 | 312 | /* fence buffer */ |
| 310 | 313 | struct amdgpu_bo *fence_buf_bo; |
| 311 | 314 | uint64_t fence_buf_mc_addr; |
| @@ -327,6 +330,10 @@ struct psp_context · 15 unmodified lines | ||
| 327 | 330 | const struct firmware *ta_fw; |
| 328 | 331 | uint32_t ta_fw_version; |
| 329 | 332 | |
| 333 | + uint32_t cap_fw_version; | |
| 334 | + uint32_t cap_feature_version; | |
| 335 | + uint32_t cap_ucode_size; | |
| 336 | + | |
| 330 | 337 | struct ta_context asd_context; |
| 331 | 338 | struct psp_xgmi_context xgmi_context; |
| 332 | 339 | struct psp_ras_context ras_context; |
| @@ -440,6 +447,8 @@ int psp_init_sos_microcode(struct psp_context *psp, · 107 unmodified lines | ||
| 440 | 447 | const char *chip_name); |
| 441 | 448 | int psp_init_ta_microcode(struct psp_context *psp, |
| 442 | 449 | const char *chip_name); |
| 450 | +int psp_init_cap_microcode(struct psp_context *psp, | |
| 451 | + const char *chip_name); | |
| 443 | 452 | int psp_get_fw_attestation_records_addr(struct psp_context *psp, |
| 444 | 453 | uint64_t *output_ptr); |
| 445 | 454 | |
| @@ -75,6 +75,13 @@ const char *ras_mca_block_string[] = { · 74 unmodified lines | ||
| 75 | 75 | "mca_iohc", |
| 76 | 76 | }; |
| 77 | 77 | |
| 78 | +struct amdgpu_ras_block_list { | |
| 79 | + /* ras block link */ | |
| 80 | + struct list_head node; | |
| 81 | + | |
| 82 | + struct amdgpu_ras_block_object *ras_obj; | |
| 83 | +}; | |
| 84 | + | |
| 78 | 85 | const char *get_ras_block_str(struct ras_common_if *ras_block) |
| 79 | 86 | { |
| 80 | 87 | if (!ras_block) |
| @@ -89,6 +96,9 @@ const char *get_ras_block_str(struct ras_common_if *ras_block) · 8 unmodified lines | ||
| 89 | 96 | return ras_block_string[ras_block->block]; |
| 90 | 97 | } |
| 91 | 98 | |
| 99 | +#define ras_block_str(_BLOCK_) \ | |
| 100 | + (((_BLOCK_) < ARRAY_SIZE(ras_block_string)) ? ras_block_string[_BLOCK_] : "Out Of Range") | |
| 101 | + | |
| 92 | 102 | #define ras_err_str(i) (ras_error_string[ffs(i)]) |
| 93 | 103 | |
| 94 | 104 | #define RAS_DEFAULT_FLAGS (AMDGPU_RAS_FLAG_INIT_BY_VBIOS) |
| @@ -155,14 +165,9 @@ static int amdgpu_reserve_page_direct(struct amdgpu_device *adev, uint64_t addre · 60 unmodified lines | ||
| 155 | 165 | } |
| 156 | 166 | |
| 157 | 167 | memset(&err_rec, 0x0, sizeof(struct eeprom_table_record)); |
| 158 | − | |
| 159 | − err_rec.address = address; | |
| 160 | − err_rec.retired_page = address >> AMDGPU_GPU_PAGE_SHIFT; | |
| 161 | − err_rec.ts = (uint64_t)ktime_get_real_seconds(); | |
| 162 | − err_rec.err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE; | |
| 163 | − | |
| 164 | 168 | err_data.err_addr = &err_rec; |
| 165 | − err_data.err_addr_cnt = 1; | |
| 169 | + amdgpu_umc_fill_error_record(&err_data, address, | |
| 170 | + (address >> AMDGPU_GPU_PAGE_SHIFT), 0, 0); | |
| 166 | 171 | |
| 167 | 172 | if (amdgpu_bad_page_threshold != 0) { |
| 168 | 173 | amdgpu_ras_add_bad_pages(adev, err_data.err_addr, |
| @@ -452,7 +457,7 @@ static ssize_t amdgpu_ras_debugfs_ctrl_write(struct file *f, · 283 unmodified lines | ||
| 452 | 457 | } |
| 453 | 458 | |
| 454 | 459 | if (ret) |
| 455 | − return -EINVAL; | |
| 460 | + return ret; | |
| 456 | 461 | |
| 457 | 462 | return size; |
| 458 | 463 | } |
| @@ -866,30 +871,47 @@ static int amdgpu_ras_enable_all_features(struct amdgpu_device *adev, · 407 unmodified lines | ||
| 866 | 871 | } |
| 867 | 872 | /* feature ctl end */ |
| 868 | 873 | |
| 874 | +static int amdgpu_ras_block_match_default(struct amdgpu_ras_block_object *block_obj, | |
| 875 | + enum amdgpu_ras_block block) | |
| 876 | +{ | |
| 877 | + if (!block_obj) | |
| 878 | + return -EINVAL; | |
| 879 | + | |
| 880 | + if (block_obj->ras_comm.block == block) | |
| 881 | + return 0; | |
| 882 | + | |
| 883 | + return -EINVAL; | |
| 884 | +} | |
| 869 | 885 | |
| 870 | −static void amdgpu_ras_mca_query_error_status(struct amdgpu_device *adev, | |
| 871 | − struct ras_common_if *ras_block, | |
| 872 | − struct ras_err_data *err_data) | |
| 886 | +static struct amdgpu_ras_block_object *amdgpu_ras_get_ras_block(struct amdgpu_device *adev, | |
| 887 | + enum amdgpu_ras_block block, uint32_t sub_block_index) | |
| 873 | 888 | { |
| 874 | − switch (ras_block->sub_block_index) { | |
| 875 | − case AMDGPU_RAS_MCA_BLOCK__MP0: | |
| 876 | − if (adev->mca.mp0.ras_funcs && | |
| 877 | − adev->mca.mp0.ras_funcs->query_ras_error_count) | |
| 878 | − adev->mca.mp0.ras_funcs->query_ras_error_count(adev, &err_data); | |
| 879 | − break; | |
| 880 | − case AMDGPU_RAS_MCA_BLOCK__MP1: | |
| 881 | − if (adev->mca.mp1.ras_funcs && | |
| 882 | − adev->mca.mp1.ras_funcs->query_ras_error_count) | |
| 883 | − adev->mca.mp1.ras_funcs->query_ras_error_count(adev, &err_data); | |
| 884 | − break; | |
| 885 | − case AMDGPU_RAS_MCA_BLOCK__MPIO: | |
| 886 | − if (adev->mca.mpio.ras_funcs && | |
| 887 | − adev->mca.mpio.ras_funcs->query_ras_error_count) | |
| 888 | − adev->mca.mpio.ras_funcs->query_ras_error_count(adev, &err_data); | |
| 889 | − break; | |
| 890 | − default: | |
| 891 | − break; | |
| 889 | + struct amdgpu_ras_block_list *node, *tmp; | |
| 890 | + struct amdgpu_ras_block_object *obj; | |
| 891 | + | |
| 892 | + if (block >= AMDGPU_RAS_BLOCK__LAST) | |
| 893 | + return NULL; | |
| 894 | + | |
| 895 | + if (!amdgpu_ras_is_supported(adev, block)) | |
| 896 | + return NULL; | |
| 897 | + | |
| 898 | + list_for_each_entry_safe(node, tmp, &adev->ras_list, node) { | |
| 899 | + if (!node->ras_obj) { | |
| 900 | + dev_warn(adev->dev, "Warning: abnormal ras list node.\n"); | |
| 901 | + continue; | |
| 902 | + } | |
| 903 | + | |
| 904 | + obj = node->ras_obj; | |
| 905 | + if (obj->ras_block_match) { | |
| 906 | + if (obj->ras_block_match(obj, block, sub_block_index) == 0) | |
| 907 | + return obj; | |
| 908 | + } else { | |
| 909 | + if (amdgpu_ras_block_match_default(obj, block) == 0) | |
| 910 | + return obj; | |
| 911 | + } | |
| 892 | 912 | } |
| 913 | + | |
| 914 | + return NULL; | |
| 893 | 915 | } |
| 894 | 916 | |
| 895 | 917 | static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_data *err_data) |
| @@ -901,26 +923,26 @@ static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_d · 5 unmodified lines | ||
| 901 | 923 | * choosing right query method according to |
| 902 | 924 | * whether smu support query error information |
| 903 | 925 | */ |
| 904 | − ret = smu_get_ecc_info(&adev->smu, (void *)&(ras->umc_ecc)); | |
| 926 | + ret = amdgpu_dpm_get_ecc_info(adev, (void *)&(ras->umc_ecc)); | |
| 905 | 927 | if (ret == -EOPNOTSUPP) { |
| 906 | − if (adev->umc.ras_funcs && | |
| 907 | − adev->umc.ras_funcs->query_ras_error_count) | |
| 908 | − adev->umc.ras_funcs->query_ras_error_count(adev, err_data); | |
| 928 | + if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && | |
| 929 | + adev->umc.ras->ras_block.hw_ops->query_ras_error_count) | |
| 930 | + adev->umc.ras->ras_block.hw_ops->query_ras_error_count(adev, err_data); | |
| 909 | 931 | |
| 910 | 932 | /* umc query_ras_error_address is also responsible for clearing |
| 911 | 933 | * error status |
| 912 | 934 | */ |
| 913 | − if (adev->umc.ras_funcs && | |
| 914 | − adev->umc.ras_funcs->query_ras_error_address) | |
| 915 | − adev->umc.ras_funcs->query_ras_error_address(adev, err_data); | |
| 935 | + if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && | |
| 936 | + adev->umc.ras->ras_block.hw_ops->query_ras_error_address) | |
| 937 | + adev->umc.ras->ras_block.hw_ops->query_ras_error_address(adev, err_data); | |
| 916 | 938 | } else if (!ret) { |
| 917 | − if (adev->umc.ras_funcs && | |
| 918 | − adev->umc.ras_funcs->ecc_info_query_ras_error_count) | |
| 919 | − adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, err_data); | |
| 939 | + if (adev->umc.ras && | |
| 940 | + adev->umc.ras->ecc_info_query_ras_error_count) | |
| 941 | + adev->umc.ras->ecc_info_query_ras_error_count(adev, err_data); | |
| 920 | 942 | |
| 921 | − if (adev->umc.ras_funcs && | |
| 922 | − adev->umc.ras_funcs->ecc_info_query_ras_error_address) | |
| 923 | − adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, err_data); | |
| 943 | + if (adev->umc.ras && | |
| 944 | + adev->umc.ras->ecc_info_query_ras_error_address) | |
| 945 | + adev->umc.ras->ecc_info_query_ras_error_address(adev, err_data); | |
| 924 | 946 | } |
| 925 | 947 | } |
| 926 | 948 | |
| @@ -928,62 +950,32 @@ static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_d · 1 unmodified line | ||
| 928 | 950 | int amdgpu_ras_query_error_status(struct amdgpu_device *adev, |
| 929 | 951 | struct ras_query_if *info) |
| 930 | 952 | { |
| 953 | + struct amdgpu_ras_block_object *block_obj = NULL; | |
| 931 | 954 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head); |
| 932 | 955 | struct ras_err_data err_data = {0, 0, 0, NULL}; |
| 933 | − int i; | |
| 934 | 956 | |
| 935 | 957 | if (!obj) |
| 936 | 958 | return -EINVAL; |
| 937 | 959 | |
| 938 | − switch (info->head.block) { | |
| 939 | − case AMDGPU_RAS_BLOCK__UMC: | |
| 960 | + if (info->head.block == AMDGPU_RAS_BLOCK__UMC) { | |
| 940 | 961 | amdgpu_ras_get_ecc_info(adev, &err_data); |
| 941 | − break; | |
| 942 | − case AMDGPU_RAS_BLOCK__SDMA: | |
| 943 | − if (adev->sdma.funcs->query_ras_error_count) { | |
| 944 | − for (i = 0; i < adev->sdma.num_instances; i++) | |
| 945 | − adev->sdma.funcs->query_ras_error_count(adev, i, | |
| 946 | − &err_data); | |
| 962 | + } else { | |
| 963 | + block_obj = amdgpu_ras_get_ras_block(adev, info->head.block, 0); | |
| 964 | + if (!block_obj || !block_obj->hw_ops) { | |
| 965 | + dev_dbg_once(adev->dev, "%s doesn't config RAS function\n", | |
| 966 | + get_ras_block_str(&info->head)); | |
| 967 | + return -EINVAL; | |
| 947 | 968 | } |
| 948 | − break; | |
| 949 | − case AMDGPU_RAS_BLOCK__GFX: | |
| 950 | − if (adev->gfx.ras_funcs && | |
| 951 | − adev->gfx.ras_funcs->query_ras_error_count) | |
| 952 | − adev->gfx.ras_funcs->query_ras_error_count(adev, &err_data); | |
| 953 | − | |
| 954 | − if (adev->gfx.ras_funcs && | |
| 955 | − adev->gfx.ras_funcs->query_ras_error_status) | |
| 956 | − adev->gfx.ras_funcs->query_ras_error_status(adev); | |
| 957 | − break; | |
| 958 | − case AMDGPU_RAS_BLOCK__MMHUB: | |
| 959 | − if (adev->mmhub.ras_funcs && | |
| 960 | − adev->mmhub.ras_funcs->query_ras_error_count) | |
| 961 | − adev->mmhub.ras_funcs->query_ras_error_count(adev, &err_data); | |
| 962 | − | |
| 963 | − if (adev->mmhub.ras_funcs && | |
| 964 | − adev->mmhub.ras_funcs->query_ras_error_status) | |
| 965 | − adev->mmhub.ras_funcs->query_ras_error_status(adev); | |
| 966 | − break; | |
| 967 | − case AMDGPU_RAS_BLOCK__PCIE_BIF: | |
| 968 | − if (adev->nbio.ras_funcs && | |
| 969 | − adev->nbio.ras_funcs->query_ras_error_count) | |
| 970 | − adev->nbio.ras_funcs->query_ras_error_count(adev, &err_data); | |
| 971 | − break; | |
| 972 | − case AMDGPU_RAS_BLOCK__XGMI_WAFL: | |
| 973 | − if (adev->gmc.xgmi.ras_funcs && | |
| 974 | − adev->gmc.xgmi.ras_funcs->query_ras_error_count) | |
| 975 | − adev->gmc.xgmi.ras_funcs->query_ras_error_count(adev, &err_data); | |
| 976 | − break; | |
| 977 | − case AMDGPU_RAS_BLOCK__HDP: | |
| 978 | − if (adev->hdp.ras_funcs && | |
| 979 | − adev->hdp.ras_funcs->query_ras_error_count) | |
| 980 | − adev->hdp.ras_funcs->query_ras_error_count(adev, &err_data); | |
| 981 | − break; | |
| 982 | − case AMDGPU_RAS_BLOCK__MCA: | |
| 983 | − amdgpu_ras_mca_query_error_status(adev, &info->head, &err_data); | |
| 984 | − break; | |
| 985 | − default: | |
| 986 | − break; | |
| 969 | + | |
| 970 | + if (block_obj->hw_ops->query_ras_error_count) | |
| 971 | + block_obj->hw_ops->query_ras_error_count(adev, &err_data); | |
| 972 | + | |
| 973 | + if ((info->head.block == AMDGPU_RAS_BLOCK__SDMA) || | |
| 974 | + (info->head.block == AMDGPU_RAS_BLOCK__GFX) || | |
| 975 | + (info->head.block == AMDGPU_RAS_BLOCK__MMHUB)) { | |
| 976 | + if (block_obj->hw_ops->query_ras_error_status) | |
| 977 | + block_obj->hw_ops->query_ras_error_status(adev); | |
| 978 | + } | |
| 987 | 979 | } |
| 988 | 980 | |
| 989 | 981 | obj->err_data.ue_count += err_data.ue_count; |
| @@ -1040,68 +1032,27 @@ int amdgpu_ras_query_error_status(struct amdgpu_device *adev, · 50 unmodified lines | ||
| 1040 | 1032 | int amdgpu_ras_reset_error_status(struct amdgpu_device *adev, |
| 1041 | 1033 | enum amdgpu_ras_block block) |
| 1042 | 1034 | { |
| 1035 | + struct amdgpu_ras_block_object *block_obj = amdgpu_ras_get_ras_block(adev, block, 0); | |
| 1036 | + | |
| 1043 | 1037 | if (!amdgpu_ras_is_supported(adev, block)) |
| 1044 | 1038 | return -EINVAL; |
| 1045 | 1039 | |
| 1046 | − switch (block) { | |
| 1047 | − case AMDGPU_RAS_BLOCK__GFX: | |
| 1048 | − if (adev->gfx.ras_funcs && | |
| 1049 | − adev->gfx.ras_funcs->reset_ras_error_count) | |
| 1050 | − adev->gfx.ras_funcs->reset_ras_error_count(adev); | |
| 1051 | − | |
| 1052 | − if (adev->gfx.ras_funcs && | |
| 1053 | − adev->gfx.ras_funcs->reset_ras_error_status) | |
| 1054 | − adev->gfx.ras_funcs->reset_ras_error_status(adev); | |
| 1055 | − break; | |
| 1056 | − case AMDGPU_RAS_BLOCK__MMHUB: | |
| 1057 | − if (adev->mmhub.ras_funcs && | |
| 1058 | − adev->mmhub.ras_funcs->reset_ras_error_count) | |
| 1059 | − adev->mmhub.ras_funcs->reset_ras_error_count(adev); | |
| 1060 | − | |
| 1061 | − if (adev->mmhub.ras_funcs && | |
| 1062 | − adev->mmhub.ras_funcs->reset_ras_error_status) | |
| 1063 | − adev->mmhub.ras_funcs->reset_ras_error_status(adev); | |
| 1064 | − break; | |
| 1065 | − case AMDGPU_RAS_BLOCK__SDMA: | |
| 1066 | − if (adev->sdma.funcs->reset_ras_error_count) | |
| 1067 | − adev->sdma.funcs->reset_ras_error_count(adev); | |
| 1068 | − break; | |
| 1069 | − case AMDGPU_RAS_BLOCK__HDP: | |
| 1070 | − if (adev->hdp.ras_funcs && | |
| 1071 | − adev->hdp.ras_funcs->reset_ras_error_count) | |
| 1072 | − adev->hdp.ras_funcs->reset_ras_error_count(adev); | |
| 1073 | − break; | |
| 1074 | − default: | |
| 1075 | − break; | |
| 1040 | + if (!block_obj || !block_obj->hw_ops) { | |
| 1041 | + dev_dbg_once(adev->dev, "%s doesn't config RAS function\n", | |
| 1042 | + ras_block_str(block)); | |
| 1043 | + return -EINVAL; | |
| 1076 | 1044 | } |
| 1077 | 1045 | |
| 1078 | − return 0; | |
| 1079 | −} | |
| 1080 | − | |
| 1081 | −/* Trigger XGMI/WAFL error */ | |
| 1082 | −static int amdgpu_ras_error_inject_xgmi(struct amdgpu_device *adev, | |
| 1083 | − struct ta_ras_trigger_error_input *block_info) | |
| 1084 | −{ | |
| 1085 | − int ret; | |
| 1046 | + if (block_obj->hw_ops->reset_ras_error_count) | |
| 1047 | + block_obj->hw_ops->reset_ras_error_count(adev); | |
| 1086 | 1048 | |
| 1087 | − if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_DISALLOW)) | |
| 1088 | − dev_warn(adev->dev, "Failed to disallow df cstate"); | |
| 1089 | − | |
| 1090 | − if (amdgpu_dpm_allow_xgmi_power_down(adev, false)) | |
| 1091 | − dev_warn(adev->dev, "Failed to disallow XGMI power down"); | |
| 1092 | − | |
| 1093 | − ret = psp_ras_trigger_error(&adev->psp, block_info); | |
| 1094 | − | |
| 1095 | − if (amdgpu_ras_intr_triggered()) | |
| 1096 | − return ret; | |
| 1097 | − | |
| 1098 | − if (amdgpu_dpm_allow_xgmi_power_down(adev, true)) | |
| 1099 | − dev_warn(adev->dev, "Failed to allow XGMI power down"); | |
| 1100 | − | |
| 1101 | − if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_ALLOW)) | |
| 1102 | − dev_warn(adev->dev, "Failed to allow df cstate"); | |
| 1049 | + if ((block == AMDGPU_RAS_BLOCK__GFX) || | |
| 1050 | + (block == AMDGPU_RAS_BLOCK__MMHUB)) { | |
| 1051 | + if (block_obj->hw_ops->reset_ras_error_status) | |
| 1052 | + block_obj->hw_ops->reset_ras_error_status(adev); | |
| 1053 | + } | |
| 1103 | 1054 | |
| 1104 | − return ret; | |
| 1055 | + return 0; | |
| 1105 | 1056 | } |
| 1106 | 1057 | |
| 1107 | 1058 | /* wrapper of psp_ras_trigger_error */ |
| @@ -1116,11 +1067,20 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 8 unmodified lines | ||
| 1116 | 1067 | .address = info->address, |
| 1117 | 1068 | .value = info->value, |
| 1118 | 1069 | }; |
| 1119 | − int ret = 0; | |
| 1070 | + int ret = -EINVAL; | |
| 1071 | + struct amdgpu_ras_block_object *block_obj = amdgpu_ras_get_ras_block(adev, | |
| 1072 | + info->head.block, | |
| 1073 | + info->head.sub_block_index); | |
| 1120 | 1074 | |
| 1121 | 1075 | if (!obj) |
| 1122 | 1076 | return -EINVAL; |
| 1123 | 1077 | |
| 1078 | + if (!block_obj || !block_obj->hw_ops) { | |
| 1079 | + dev_dbg_once(adev->dev, "%s doesn't config RAS function\n", | |
| 1080 | + get_ras_block_str(&info->head)); | |
| 1081 | + return -EINVAL; | |
| 1082 | + } | |
| 1083 | + | |
| 1124 | 1084 | /* Calculate XGMI relative offset */ |
| 1125 | 1085 | if (adev->gmc.xgmi.num_physical_nodes > 1) { |
| 1126 | 1086 | block_info.address = |
| @@ -1128,28 +1088,15 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 1 unmodified line | ||
| 1128 | 1088 | block_info.address); |
| 1129 | 1089 | } |
| 1130 | 1090 | |
| 1131 | − switch (info->head.block) { | |
| 1132 | − case AMDGPU_RAS_BLOCK__GFX: | |
| 1133 | − if (adev->gfx.ras_funcs && | |
| 1134 | − adev->gfx.ras_funcs->ras_error_inject) | |
| 1135 | − ret = adev->gfx.ras_funcs->ras_error_inject(adev, info); | |
| 1136 | − else | |
| 1137 | − ret = -EINVAL; | |
| 1138 | − break; | |
| 1139 | − case AMDGPU_RAS_BLOCK__UMC: | |
| 1140 | − case AMDGPU_RAS_BLOCK__SDMA: | |
| 1141 | − case AMDGPU_RAS_BLOCK__MMHUB: | |
| 1142 | − case AMDGPU_RAS_BLOCK__PCIE_BIF: | |
| 1143 | − case AMDGPU_RAS_BLOCK__MCA: | |
| 1144 | − ret = psp_ras_trigger_error(&adev->psp, &block_info); | |
| 1145 | − break; | |
| 1146 | − case AMDGPU_RAS_BLOCK__XGMI_WAFL: | |
| 1147 | − ret = amdgpu_ras_error_inject_xgmi(adev, &block_info); | |
| 1148 | − break; | |
| 1149 | − default: | |
| 1150 | − dev_info(adev->dev, "%s error injection is not supported yet\n", | |
| 1151 | − get_ras_block_str(&info->head)); | |
| 1152 | − ret = -EINVAL; | |
| 1091 | + if (info->head.block == AMDGPU_RAS_BLOCK__GFX) { | |
| 1092 | + if (block_obj->hw_ops->ras_error_inject) | |
| 1093 | + ret = block_obj->hw_ops->ras_error_inject(adev, info); | |
| 1094 | + } else { | |
| 1095 | + /* If defined special ras_error_inject(e.g: xgmi), implement special ras_error_inject */ | |
| 1096 | + if (block_obj->hw_ops->ras_error_inject) | |
| 1097 | + ret = block_obj->hw_ops->ras_error_inject(adev, &block_info); | |
| 1098 | + else /*If not defined .ras_error_inject, use default ras_error_inject*/ | |
| 1099 | + ret = psp_ras_trigger_error(&adev->psp, &block_info); | |
| 1153 | 1100 | } |
| 1154 | 1101 | |
| 1155 | 1102 | if (ret) |
| @@ -1329,18 +1276,17 @@ static int amdgpu_ras_sysfs_remove_feature_node(struct amdgpu_device *adev) · 173 unmodified lines | ||
| 1329 | 1276 | } |
| 1330 | 1277 | |
| 1331 | 1278 | int amdgpu_ras_sysfs_create(struct amdgpu_device *adev, |
| 1332 | − struct ras_fs_if *head) | |
| 1279 | + struct ras_common_if *head) | |
| 1333 | 1280 | { |
| 1334 | − struct ras_manager *obj = amdgpu_ras_find_obj(adev, &head->head); | |
| 1281 | + struct ras_manager *obj = amdgpu_ras_find_obj(adev, head); | |
| 1335 | 1282 | |
| 1336 | 1283 | if (!obj || obj->attr_inuse) |
| 1337 | 1284 | return -EINVAL; |
| 1338 | 1285 | |
| 1339 | 1286 | get_obj(obj); |
| 1340 | 1287 | |
| 1341 | − memcpy(obj->fs_data.sysfs_name, | |
| 1342 | − head->sysfs_name, | |
| 1343 | − sizeof(obj->fs_data.sysfs_name)); | |
| 1288 | + snprintf(obj->fs_data.sysfs_name, sizeof(obj->fs_data.sysfs_name), | |
| 1289 | + "%s_err_count", head->name); | |
| 1344 | 1290 | |
| 1345 | 1291 | obj->sysfs_attr = (struct device_attribute){ |
| 1346 | 1292 | .attr = { |
| @@ -1647,9 +1593,9 @@ int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, · 300 unmodified lines | ||
| 1647 | 1593 | } |
| 1648 | 1594 | |
| 1649 | 1595 | int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, |
| 1650 | − struct ras_ih_if *info) | |
| 1596 | + struct ras_common_if *head) | |
| 1651 | 1597 | { |
| 1652 | − struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head); | |
| 1598 | + struct ras_manager *obj = amdgpu_ras_find_obj(adev, head); | |
| 1653 | 1599 | struct ras_ih_data *data; |
| 1654 | 1600 | |
| 1655 | 1601 | if (!obj) |
| @@ -1669,24 +1615,27 @@ int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, · 13 unmodified lines | ||
| 1669 | 1615 | } |
| 1670 | 1616 | |
| 1671 | 1617 | int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev, |
| 1672 | − struct ras_ih_if *info) | |
| 1618 | + struct ras_common_if *head) | |
| 1673 | 1619 | { |
| 1674 | − struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head); | |
| 1620 | + struct ras_manager *obj = amdgpu_ras_find_obj(adev, head); | |
| 1675 | 1621 | struct ras_ih_data *data; |
| 1622 | + struct amdgpu_ras_block_object *ras_obj; | |
| 1676 | 1623 | |
| 1677 | 1624 | if (!obj) { |
| 1678 | 1625 | /* in case we registe the IH before enable ras feature */ |
| 1679 | − obj = amdgpu_ras_create_obj(adev, &info->head); | |
| 1626 | + obj = amdgpu_ras_create_obj(adev, head); | |
| 1680 | 1627 | if (!obj) |
| 1681 | 1628 | return -EINVAL; |
| 1682 | 1629 | } else |
| 1683 | 1630 | get_obj(obj); |
| 1684 | 1631 | |
| 1632 | + ras_obj = container_of(head, struct amdgpu_ras_block_object, ras_comm); | |
| 1633 | + | |
| 1685 | 1634 | data = &obj->ih_data; |
| 1686 | 1635 | /* add the callback.etc */ |
| 1687 | 1636 | *data = (struct ras_ih_data) { |
| 1688 | 1637 | .inuse = 0, |
| 1689 | − .cb = info->cb, | |
| 1638 | + .cb = ras_obj->ras_cb, | |
| 1690 | 1639 | .element_size = sizeof(struct amdgpu_iv_entry), |
| 1691 | 1640 | .rptr = 0, |
| 1692 | 1641 | .wptr = 0, |
| @@ -1715,10 +1664,7 @@ static int amdgpu_ras_interrupt_remove_all(struct amdgpu_device *adev) · 22 unmodified lines | ||
| 1715 | 1664 | struct ras_manager *obj, *tmp; |
| 1716 | 1665 | |
| 1717 | 1666 | list_for_each_entry_safe(obj, tmp, &con->head, node) { |
| 1718 | − struct ras_ih_if info = { | |
| 1719 | − .head = obj->head, | |
| 1720 | − }; | |
| 1721 | − amdgpu_ras_interrupt_remove_handler(adev, &info); | |
| 1667 | + amdgpu_ras_interrupt_remove_handler(adev, &obj->head); | |
| 1722 | 1668 | } |
| 1723 | 1669 | |
| 1724 | 1670 | return 0; |
| @@ -1766,24 +1712,28 @@ static void amdgpu_ras_log_on_err_counter(struct amdgpu_device *adev) · 41 unmodified lines | ||
| 1766 | 1712 | static void amdgpu_ras_error_status_query(struct amdgpu_device *adev, |
| 1767 | 1713 | struct ras_query_if *info) |
| 1768 | 1714 | { |
| 1715 | + struct amdgpu_ras_block_object *block_obj; | |
| 1769 | 1716 | /* |
| 1770 | 1717 | * Only two block need to query read/write |
| 1771 | 1718 | * RspStatus at current state |
| 1772 | 1719 | */ |
| 1773 | − switch (info->head.block) { | |
| 1774 | − case AMDGPU_RAS_BLOCK__GFX: | |
| 1775 | − if (adev->gfx.ras_funcs && | |
| 1776 | − adev->gfx.ras_funcs->query_ras_error_status) | |
| 1777 | − adev->gfx.ras_funcs->query_ras_error_status(adev); | |
| 1778 | − break; | |
| 1779 | − case AMDGPU_RAS_BLOCK__MMHUB: | |
| 1780 | − if (adev->mmhub.ras_funcs && | |
| 1781 | − adev->mmhub.ras_funcs->query_ras_error_status) | |
| 1782 | − adev->mmhub.ras_funcs->query_ras_error_status(adev); | |
| 1783 | − break; | |
| 1784 | − default: | |
| 1785 | − break; | |
| 1720 | + if ((info->head.block != AMDGPU_RAS_BLOCK__GFX) && | |
| 1721 | + (info->head.block != AMDGPU_RAS_BLOCK__MMHUB)) | |
| 1722 | + return; | |
| 1723 | + | |
| 1724 | + block_obj = amdgpu_ras_get_ras_block(adev, | |
| 1725 | + info->head.block, | |
| 1726 | + info->head.sub_block_index); | |
| 1727 | + | |
| 1728 | + if (!block_obj || !block_obj->hw_ops) { | |
| 1729 | + dev_dbg_once(adev->dev, "%s doesn't config RAS function\n", | |
| 1730 | + get_ras_block_str(&info->head)); | |
| 1731 | + return; | |
| 1786 | 1732 | } |
| 1733 | + | |
| 1734 | + if (block_obj->hw_ops->query_ras_error_status) | |
| 1735 | + block_obj->hw_ops->query_ras_error_status(adev); | |
| 1736 | + | |
| 1787 | 1737 | } |
| 1788 | 1738 | |
| 1789 | 1739 | static void amdgpu_ras_query_err_status(struct amdgpu_device *adev) |
| @@ -2141,8 +2091,7 @@ int amdgpu_ras_recovery_init(struct amdgpu_device *adev) · 351 unmodified lines | ||
| 2141 | 2091 | if (ret) |
| 2142 | 2092 | goto free; |
| 2143 | 2093 | |
| 2144 | − if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num) | |
| 2145 | − adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs); | |
| 2094 | + amdgpu_dpm_send_hbm_bad_pages_num(adev, con->eeprom_control.ras_num_recs); | |
| 2146 | 2095 | } |
| 2147 | 2096 | |
| 2148 | 2097 | #ifdef CONFIG_X86_MCE_AMD |
| @@ -2348,24 +2297,27 @@ int amdgpu_ras_init(struct amdgpu_device *adev) · 199 unmodified lines | ||
| 2348 | 2297 | case CHIP_VEGA20: |
| 2349 | 2298 | case CHIP_ARCTURUS: |
| 2350 | 2299 | case CHIP_ALDEBARAN: |
| 2351 | − if (!adev->gmc.xgmi.connected_to_cpu) | |
| 2352 | − adev->nbio.ras_funcs = &nbio_v7_4_ras_funcs; | |
| 2300 | + if (!adev->gmc.xgmi.connected_to_cpu) { | |
| 2301 | + adev->nbio.ras = &nbio_v7_4_ras; | |
| 2302 | + amdgpu_ras_register_ras_block(adev, &adev->nbio.ras->ras_block); | |
| 2303 | + adev->nbio.ras_if = &adev->nbio.ras->ras_block.ras_comm; | |
| 2304 | + } | |
| 2353 | 2305 | break; |
| 2354 | 2306 | default: |
| 2355 | 2307 | /* nbio ras is not available */ |
| 2356 | 2308 | break; |
| 2357 | 2309 | } |
| 2358 | 2310 | |
| 2359 | − if (adev->nbio.ras_funcs && | |
| 2360 | − adev->nbio.ras_funcs->init_ras_controller_interrupt) { | |
| 2361 | − r = adev->nbio.ras_funcs->init_ras_controller_interrupt(adev); | |
| 2311 | + if (adev->nbio.ras && | |
| 2312 | + adev->nbio.ras->init_ras_controller_interrupt) { | |
| 2313 | + r = adev->nbio.ras->init_ras_controller_interrupt(adev); | |
| 2362 | 2314 | if (r) |
| 2363 | 2315 | goto release_con; |
| 2364 | 2316 | } |
| 2365 | 2317 | |
| 2366 | − if (adev->nbio.ras_funcs && | |
| 2367 | − adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt) { | |
| 2368 | − r = adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt(adev); | |
| 2318 | + if (adev->nbio.ras && | |
| 2319 | + adev->nbio.ras->init_ras_err_event_athub_interrupt) { | |
| 2320 | + r = adev->nbio.ras->init_ras_err_event_athub_interrupt(adev); | |
| 2369 | 2321 | if (r) |
| 2370 | 2322 | goto release_con; |
| 2371 | 2323 | } |
| @@ -2377,12 +2329,12 @@ int amdgpu_ras_init(struct amdgpu_device *adev) · 5 unmodified lines | ||
| 2377 | 2329 | } |
| 2378 | 2330 | else if (adev->df.funcs && |
| 2379 | 2331 | adev->df.funcs->query_ras_poison_mode && |
| 2380 | − adev->umc.ras_funcs && | |
| 2381 | − adev->umc.ras_funcs->query_ras_poison_mode) { | |
| 2332 | + adev->umc.ras && | |
| 2333 | + adev->umc.ras->query_ras_poison_mode) { | |
| 2382 | 2334 | df_poison = |
| 2383 | 2335 | adev->df.funcs->query_ras_poison_mode(adev); |
| 2384 | 2336 | umc_poison = |
| 2385 | − adev->umc.ras_funcs->query_ras_poison_mode(adev); | |
| 2337 | + adev->umc.ras->query_ras_poison_mode(adev); | |
| 2386 | 2338 | /* Only poison is set in both DF and UMC, we can support it */ |
| 2387 | 2339 | if (df_poison && umc_poison) |
| 2388 | 2340 | con->poison_supported = true; |
| @@ -2445,11 +2397,10 @@ bool amdgpu_ras_is_poison_mode_supported(struct amdgpu_device *adev) · 56 unmodified lines | ||
| 2445 | 2397 | } |
| 2446 | 2398 | |
| 2447 | 2399 | /* helper function to handle common stuff in ip late init phase */ |
| 2448 | −int amdgpu_ras_late_init(struct amdgpu_device *adev, | |
| 2449 | − struct ras_common_if *ras_block, | |
| 2450 | − struct ras_fs_if *fs_info, | |
| 2451 | − struct ras_ih_if *ih_info) | |
| 2400 | +int amdgpu_ras_block_late_init(struct amdgpu_device *adev, | |
| 2401 | + struct ras_common_if *ras_block) | |
| 2452 | 2402 | { |
| 2403 | + struct amdgpu_ras_block_object *ras_obj = NULL; | |
| 2453 | 2404 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); |
| 2454 | 2405 | unsigned long ue_count, ce_count; |
| 2455 | 2406 | int r; |
| @@ -2477,15 +2428,16 @@ int amdgpu_ras_late_init(struct amdgpu_device *adev, · 21 unmodified lines | ||
| 2477 | 2428 | if (adev->in_suspend || amdgpu_in_reset(adev)) |
| 2478 | 2429 | return 0; |
| 2479 | 2430 | |
| 2480 | − if (ih_info->cb) { | |
| 2481 | − r = amdgpu_ras_interrupt_add_handler(adev, ih_info); | |
| 2431 | + ras_obj = container_of(ras_block, struct amdgpu_ras_block_object, ras_comm); | |
| 2432 | + if (ras_obj->ras_cb) { | |
| 2433 | + r = amdgpu_ras_interrupt_add_handler(adev, ras_block); | |
| 2482 | 2434 | if (r) |
| 2483 | − goto interrupt; | |
| 2435 | + goto cleanup; | |
| 2484 | 2436 | } |
| 2485 | 2437 | |
| 2486 | − r = amdgpu_ras_sysfs_create(adev, fs_info); | |
| 2438 | + r = amdgpu_ras_sysfs_create(adev, ras_block); | |
| 2487 | 2439 | if (r) |
| 2488 | − goto sysfs; | |
| 2440 | + goto interrupt; | |
| 2489 | 2441 | |
| 2490 | 2442 | /* Those are the cached values at init. |
| 2491 | 2443 | */ |
| @@ -2495,27 +2447,34 @@ int amdgpu_ras_late_init(struct amdgpu_device *adev, · 3 unmodified lines | ||
| 2495 | 2447 | } |
| 2496 | 2448 | |
| 2497 | 2449 | return 0; |
| 2498 | −cleanup: | |
| 2499 | − amdgpu_ras_sysfs_remove(adev, ras_block); | |
| 2500 | −sysfs: | |
| 2501 | − if (ih_info->cb) | |
| 2502 | − amdgpu_ras_interrupt_remove_handler(adev, ih_info); | |
| 2450 | + | |
| 2503 | 2451 | interrupt: |
| 2452 | + if (ras_obj->ras_cb) | |
| 2453 | + amdgpu_ras_interrupt_remove_handler(adev, ras_block); | |
| 2454 | +cleanup: | |
| 2504 | 2455 | amdgpu_ras_feature_enable(adev, ras_block, 0); |
| 2505 | 2456 | return r; |
| 2506 | 2457 | } |
| 2507 | 2458 | |
| 2459 | +static int amdgpu_ras_block_late_init_default(struct amdgpu_device *adev, | |
| 2460 | + struct ras_common_if *ras_block) | |
| 2461 | +{ | |
| 2462 | + return amdgpu_ras_block_late_init(adev, ras_block); | |
| 2463 | +} | |
| 2464 | + | |
| 2508 | 2465 | /* helper function to remove ras fs node and interrupt handler */ |
| 2509 | −void amdgpu_ras_late_fini(struct amdgpu_device *adev, | |
| 2510 | − struct ras_common_if *ras_block, | |
| 2511 | − struct ras_ih_if *ih_info) | |
| 2466 | +void amdgpu_ras_block_late_fini(struct amdgpu_device *adev, | |
| 2467 | + struct ras_common_if *ras_block) | |
| 2512 | 2468 | { |
| 2513 | − if (!ras_block || !ih_info) | |
| 2469 | + struct amdgpu_ras_block_object *ras_obj; | |
| 2470 | + if (!ras_block) | |
| 2514 | 2471 | return; |
| 2515 | 2472 | |
| 2516 | 2473 | amdgpu_ras_sysfs_remove(adev, ras_block); |
| 2517 | − if (ih_info->cb) | |
| 2518 | − amdgpu_ras_interrupt_remove_handler(adev, ih_info); | |
| 2474 | + | |
| 2475 | + ras_obj = container_of(ras_block, struct amdgpu_ras_block_object, ras_comm); | |
| 2476 | + if (ras_obj->ras_cb) | |
| 2477 | + amdgpu_ras_interrupt_remove_handler(adev, ras_block); | |
| 2519 | 2478 | } |
| 2520 | 2479 | |
| 2521 | 2480 | /* do some init work after IP late init as dependence. |
| @@ -2568,6 +2527,33 @@ void amdgpu_ras_suspend(struct amdgpu_device *adev) · 46 unmodified lines | ||
| 2568 | 2527 | amdgpu_ras_disable_all_features(adev, 1); |
| 2569 | 2528 | } |
| 2570 | 2529 | |
| 2530 | +int amdgpu_ras_late_init(struct amdgpu_device *adev) | |
| 2531 | +{ | |
| 2532 | + struct amdgpu_ras_block_list *node, *tmp; | |
| 2533 | + struct amdgpu_ras_block_object *obj; | |
| 2534 | + int r; | |
| 2535 | + | |
| 2536 | + list_for_each_entry_safe(node, tmp, &adev->ras_list, node) { | |
| 2537 | + if (!node->ras_obj) { | |
| 2538 | + dev_warn(adev->dev, "Warning: abnormal ras list node.\n"); | |
| 2539 | + continue; | |
| 2540 | + } | |
| 2541 | + | |
| 2542 | + obj = node->ras_obj; | |
| 2543 | + if (obj->ras_late_init) { | |
| 2544 | + r = obj->ras_late_init(adev, &obj->ras_comm); | |
| 2545 | + if (r) { | |
| 2546 | + dev_err(adev->dev, "%s failed to execute ras_late_init! ret:%d\n", | |
| 2547 | + obj->ras_comm.name, r); | |
| 2548 | + return r; | |
| 2549 | + } | |
| 2550 | + } else | |
| 2551 | + amdgpu_ras_block_late_init_default(adev, &obj->ras_comm); | |
| 2552 | + } | |
| 2553 | + | |
| 2554 | + return 0; | |
| 2555 | +} | |
| 2556 | + | |
| 2571 | 2557 | /* do some fini work before IP fini as dependence */ |
| 2572 | 2558 | int amdgpu_ras_pre_fini(struct amdgpu_device *adev) |
| 2573 | 2559 | { |
| @@ -2585,6 +2571,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) · 11 unmodified lines | ||
| 2585 | 2571 | |
| 2586 | 2572 | int amdgpu_ras_fini(struct amdgpu_device *adev) |
| 2587 | 2573 | { |
| 2574 | + struct amdgpu_ras_block_list *ras_node, *tmp; | |
| 2588 | 2575 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); |
| 2589 | 2576 | |
| 2590 | 2577 | if (!adev->ras_enabled || !con) |
| @@ -2603,6 +2590,12 @@ int amdgpu_ras_fini(struct amdgpu_device *adev) · 12 unmodified lines | ||
| 2603 | 2590 | amdgpu_ras_set_context(adev, NULL); |
| 2604 | 2591 | kfree(con); |
| 2605 | 2592 | |
| 2593 | + /* Clear ras blocks from ras_list and free ras block list node */ | |
| 2594 | + list_for_each_entry_safe(ras_node, tmp, &adev->ras_list, node) { | |
| 2595 | + list_del(&ras_node->node); | |
| 2596 | + kfree(ras_node); | |
| 2597 | + } | |
| 2598 | + | |
| 2606 | 2599 | return 0; |
| 2607 | 2600 | } |
| 2608 | 2601 | |
| @@ -2717,8 +2710,6 @@ static int amdgpu_bad_page_notifier(struct notifier_block *nb, · 108 unmodified lines | ||
| 2717 | 2710 | dev_info(adev->dev, "Uncorrectable error detected in UMC inst: %d, chan_idx: %d", |
| 2718 | 2711 | umc_inst, ch_inst); |
| 2719 | 2712 | |
| 2720 | − memset(&err_rec, 0x0, sizeof(struct eeprom_table_record)); | |
| 2721 | − | |
| 2722 | 2713 | /* |
| 2723 | 2714 | * Translate UMC channel address to Physical address |
| 2724 | 2715 | */ |
| @@ -2730,16 +2721,10 @@ static int amdgpu_bad_page_notifier(struct notifier_block *nb, · 5 unmodified lines | ||
| 2730 | 2721 | ADDR_OF_256B_BLOCK(channel_index) | |
| 2731 | 2722 | OFFSET_IN_256B_BLOCK(m->addr); |
| 2732 | 2723 | |
| 2733 | − err_rec.address = m->addr; | |
| 2734 | − err_rec.retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT; | |
| 2735 | − err_rec.ts = (uint64_t)ktime_get_real_seconds(); | |
| 2736 | − err_rec.err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE; | |
| 2737 | − err_rec.cu = 0; | |
| 2738 | − err_rec.mem_channel = channel_index; | |
| 2739 | − err_rec.mcumc_id = umc_inst; | |
| 2740 | − | |
| 2724 | + memset(&err_rec, 0x0, sizeof(struct eeprom_table_record)); | |
| 2741 | 2725 | err_data.err_addr = &err_rec; |
| 2742 | − err_data.err_addr_cnt = 1; | |
| 2726 | + amdgpu_umc_fill_error_record(&err_data, m->addr, | |
| 2727 | + retired_page, channel_index, umc_inst); | |
| 2743 | 2728 | |
| 2744 | 2729 | if (amdgpu_bad_page_threshold != 0) { |
| 2745 | 2730 | amdgpu_ras_add_bad_pages(adev, err_data.err_addr, |
| @@ -2777,3 +2762,63 @@ static void amdgpu_register_bad_pages_mca_notifier(struct amdgpu_device *adev) · 31 unmodified lines | ||
| 2777 | 2762 | } |
| 2778 | 2763 | } |
| 2779 | 2764 | #endif |
| 2765 | + | |
| 2766 | +struct amdgpu_ras *amdgpu_ras_get_context(struct amdgpu_device *adev) | |
| 2767 | +{ | |
| 2768 | + if (!adev) | |
| 2769 | + return NULL; | |
| 2770 | + | |
| 2771 | + return adev->psp.ras_context.ras; | |
| 2772 | +} | |
| 2773 | + | |
| 2774 | +int amdgpu_ras_set_context(struct amdgpu_device *adev, struct amdgpu_ras *ras_con) | |
| 2775 | +{ | |
| 2776 | + if (!adev) | |
| 2777 | + return -EINVAL; | |
| 2778 | + | |
| 2779 | + adev->psp.ras_context.ras = ras_con; | |
| 2780 | + return 0; | |
| 2781 | +} | |
| 2782 | + | |
| 2783 | +/* check if ras is supported on block, say, sdma, gfx */ | |
| 2784 | +int amdgpu_ras_is_supported(struct amdgpu_device *adev, | |
| 2785 | + unsigned int block) | |
| 2786 | +{ | |
| 2787 | + struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | |
| 2788 | + | |
| 2789 | + if (block >= AMDGPU_RAS_BLOCK_COUNT) | |
| 2790 | + return 0; | |
| 2791 | + return ras && (adev->ras_enabled & (1 << block)); | |
| 2792 | +} | |
| 2793 | + | |
| 2794 | +int amdgpu_ras_reset_gpu(struct amdgpu_device *adev) | |
| 2795 | +{ | |
| 2796 | + struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | |
| 2797 | + | |
| 2798 | + if (atomic_cmpxchg(&ras->in_recovery, 0, 1) == 0) | |
| 2799 | + schedule_work(&ras->recovery_work); | |
| 2800 | + return 0; | |
| 2801 | +} | |
| 2802 | + | |
| 2803 | + | |
| 2804 | +/* Register each ip ras block into amdgpu ras */ | |
| 2805 | +int amdgpu_ras_register_ras_block(struct amdgpu_device *adev, | |
| 2806 | + struct amdgpu_ras_block_object *ras_block_obj) | |
| 2807 | +{ | |
| 2808 | + struct amdgpu_ras_block_list *ras_node; | |
| 2809 | + if (!adev || !ras_block_obj) | |
| 2810 | + return -EINVAL; | |
| 2811 | + | |
| 2812 | + if (!amdgpu_ras_asic_supported(adev)) | |
| 2813 | + return 0; | |
| 2814 | + | |
| 2815 | + ras_node = kzalloc(sizeof(*ras_node), GFP_KERNEL); | |
| 2816 | + if (!ras_node) | |
| 2817 | + return -ENOMEM; | |
| 2818 | + | |
| 2819 | + INIT_LIST_HEAD(&ras_node->node); | |
| 2820 | + ras_node->ras_obj = ras_block_obj; | |
| 2821 | + list_add_tail(&ras_node->node, &adev->ras_list); | |
| 2822 | + | |
| 2823 | + return 0; | |
| 2824 | +} | |
| @@ -26,11 +26,11 @@ · 25 unmodified lines | ||
| 26 | 26 | |
| 27 | 27 | #include <linux/debugfs.h> |
| 28 | 28 | #include <linux/list.h> |
| 29 | −#include "amdgpu.h" | |
| 30 | −#include "amdgpu_psp.h" | |
| 31 | 29 | #include "ta_ras_if.h" |
| 32 | 30 | #include "amdgpu_ras_eeprom.h" |
| 33 | 31 | |
| 32 | +struct amdgpu_iv_entry; | |
| 33 | + | |
| 34 | 34 | #define AMDGPU_RAS_FLAG_INIT_BY_VBIOS (0x1 << 0) |
| 35 | 35 | |
| 36 | 36 | enum amdgpu_ras_block { |
| @@ -484,6 +484,27 @@ struct ras_debug_if { · 447 unmodified lines | ||
| 484 | 484 | }; |
| 485 | 485 | int op; |
| 486 | 486 | }; |
| 487 | + | |
| 488 | +struct amdgpu_ras_block_object { | |
| 489 | + struct ras_common_if ras_comm; | |
| 490 | + | |
| 491 | + int (*ras_block_match)(struct amdgpu_ras_block_object *block_obj, | |
| 492 | + enum amdgpu_ras_block block, uint32_t sub_block_index); | |
| 493 | + int (*ras_late_init)(struct amdgpu_device *adev, struct ras_common_if *ras_block); | |
| 494 | + void (*ras_fini)(struct amdgpu_device *adev); | |
| 495 | + ras_ih_cb ras_cb; | |
| 496 | + const struct amdgpu_ras_block_hw_ops *hw_ops; | |
| 497 | +}; | |
| 498 | + | |
| 499 | +struct amdgpu_ras_block_hw_ops { | |
| 500 | + int (*ras_error_inject)(struct amdgpu_device *adev, void *inject_if); | |
| 501 | + void (*query_ras_error_count)(struct amdgpu_device *adev, void *ras_error_status); | |
| 502 | + void (*query_ras_error_status)(struct amdgpu_device *adev); | |
| 503 | + void (*query_ras_error_address)(struct amdgpu_device *adev, void *ras_error_status); | |
| 504 | + void (*reset_ras_error_count)(struct amdgpu_device *adev); | |
| 505 | + void (*reset_ras_error_status)(struct amdgpu_device *adev); | |
| 506 | +}; | |
| 507 | + | |
| 487 | 508 | /* work flow |
| 488 | 509 | * vbios |
| 489 | 510 | * 1: ras feature enable (enabled by default) |
| @@ -498,19 +519,6 @@ struct ras_debug_if { · 8 unmodified lines | ||
| 498 | 519 | * 8: feature disable |
| 499 | 520 | */ |
| 500 | 521 | |
| 501 | −#define amdgpu_ras_get_context(adev) ((adev)->psp.ras_context.ras) | |
| 502 | −#define amdgpu_ras_set_context(adev, ras_con) ((adev)->psp.ras_context.ras = (ras_con)) | |
| 503 | − | |
| 504 | −/* check if ras is supported on block, say, sdma, gfx */ | |
| 505 | −static inline int amdgpu_ras_is_supported(struct amdgpu_device *adev, | |
| 506 | − unsigned int block) | |
| 507 | −{ | |
| 508 | − struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | |
| 509 | − | |
| 510 | − if (block >= AMDGPU_RAS_BLOCK_COUNT) | |
| 511 | − return 0; | |
| 512 | − return ras && (adev->ras_enabled & (1 << block)); | |
| 513 | −} | |
| 514 | 522 | |
| 515 | 523 | int amdgpu_ras_recovery_init(struct amdgpu_device *adev); |
| 516 | 524 | |
| @@ -527,15 +535,6 @@ int amdgpu_ras_add_bad_pages(struct amdgpu_device *adev, · 10 unmodified lines | ||
| 527 | 535 | |
| 528 | 536 | int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev); |
| 529 | 537 | |
| 530 | −static inline int amdgpu_ras_reset_gpu(struct amdgpu_device *adev) | |
| 531 | −{ | |
| 532 | − struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | |
| 533 | − | |
| 534 | − if (atomic_cmpxchg(&ras->in_recovery, 0, 1) == 0) | |
| 535 | − schedule_work(&ras->recovery_work); | |
| 536 | − return 0; | |
| 537 | −} | |
| 538 | − | |
| 539 | 538 | static inline enum ta_ras_block |
| 540 | 539 | amdgpu_ras_block_to_ta(enum amdgpu_ras_block block) { |
| 541 | 540 | switch (block) { |
| @@ -596,15 +595,15 @@ amdgpu_ras_error_to_ta(enum amdgpu_ras_error_type error) { · 54 unmodified lines | ||
| 596 | 595 | |
| 597 | 596 | /* called in ip_init and ip_fini */ |
| 598 | 597 | int amdgpu_ras_init(struct amdgpu_device *adev); |
| 598 | +int amdgpu_ras_late_init(struct amdgpu_device *adev); | |
| 599 | 599 | int amdgpu_ras_fini(struct amdgpu_device *adev); |
| 600 | 600 | int amdgpu_ras_pre_fini(struct amdgpu_device *adev); |
| 601 | −int amdgpu_ras_late_init(struct amdgpu_device *adev, | |
| 602 | − struct ras_common_if *ras_block, | |
| 603 | − struct ras_fs_if *fs_info, | |
| 604 | − struct ras_ih_if *ih_info); | |
| 605 | −void amdgpu_ras_late_fini(struct amdgpu_device *adev, | |
| 606 | − struct ras_common_if *ras_block, | |
| 607 | − struct ras_ih_if *ih_info); | |
| 601 | + | |
| 602 | +int amdgpu_ras_block_late_init(struct amdgpu_device *adev, | |
| 603 | + struct ras_common_if *ras_block); | |
| 604 | + | |
| 605 | +void amdgpu_ras_block_late_fini(struct amdgpu_device *adev, | |
| 606 | + struct ras_common_if *ras_block); | |
| 608 | 607 | |
| 609 | 608 | int amdgpu_ras_feature_enable(struct amdgpu_device *adev, |
| 610 | 609 | struct ras_common_if *head, bool enable); |
| @@ -613,7 +612,7 @@ int amdgpu_ras_feature_enable_on_boot(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 613 | 612 | struct ras_common_if *head, bool enable); |
| 614 | 613 | |
| 615 | 614 | int amdgpu_ras_sysfs_create(struct amdgpu_device *adev, |
| 616 | − struct ras_fs_if *head); | |
| 615 | + struct ras_common_if *head); | |
| 617 | 616 | |
| 618 | 617 | int amdgpu_ras_sysfs_remove(struct amdgpu_device *adev, |
| 619 | 618 | struct ras_common_if *head); |
| @@ -630,10 +629,10 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 10 unmodified lines | ||
| 630 | 629 | struct ras_inject_if *info); |
| 631 | 630 | |
| 632 | 631 | int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev, |
| 633 | − struct ras_ih_if *info); | |
| 632 | + struct ras_common_if *head); | |
| 634 | 633 | |
| 635 | 634 | int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, |
| 636 | − struct ras_ih_if *info); | |
| 635 | + struct ras_common_if *head); | |
| 637 | 636 | |
| 638 | 637 | int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, |
| 639 | 638 | struct ras_dispatch_if *info); |
| @@ -667,4 +666,14 @@ const char *get_ras_block_str(struct ras_common_if *ras_block); · 27 unmodified lines | ||
| 667 | 666 | |
| 668 | 667 | bool amdgpu_ras_is_poison_mode_supported(struct amdgpu_device *adev); |
| 669 | 668 | |
| 669 | +int amdgpu_ras_is_supported(struct amdgpu_device *adev, unsigned int block); | |
| 670 | + | |
| 671 | +int amdgpu_ras_reset_gpu(struct amdgpu_device *adev); | |
| 672 | + | |
| 673 | +struct amdgpu_ras* amdgpu_ras_get_context(struct amdgpu_device *adev); | |
| 674 | + | |
| 675 | +int amdgpu_ras_set_context(struct amdgpu_device *adev, struct amdgpu_ras *ras_con); | |
| 676 | + | |
| 677 | +int amdgpu_ras_register_ras_block(struct amdgpu_device *adev, | |
| 678 | + struct amdgpu_ras_block_object *ras_block_obj); | |
| 670 | 679 | #endif |
| @@ -31,6 +31,8 @@ · 30 unmodified lines | ||
| 31 | 31 | #include <linux/debugfs.h> |
| 32 | 32 | #include <linux/uaccess.h> |
| 33 | 33 | |
| 34 | +#include "amdgpu_reset.h" | |
| 35 | + | |
| 34 | 36 | #define EEPROM_I2C_MADDR_VEGA20 0x0 |
| 35 | 37 | #define EEPROM_I2C_MADDR_ARCTURUS 0x40000 |
| 36 | 38 | #define EEPROM_I2C_MADDR_ARCTURUS_D342 0x0 |
| @@ -193,12 +195,12 @@ static int __write_table_header(struct amdgpu_ras_eeprom_control *control) · 156 unmodified lines | ||
| 193 | 195 | __encode_table_header_to_buf(&control->tbl_hdr, buf); |
| 194 | 196 | |
| 195 | 197 | /* i2c may be unstable in gpu reset */ |
| 196 | − down_read(&adev->reset_sem); | |
| 197 | − res = amdgpu_eeprom_write(&adev->pm.smu_i2c, | |
| 198 | + down_read(&adev->reset_domain->sem); | |
| 199 | + res = amdgpu_eeprom_write(adev->pm.ras_eeprom_i2c_bus, | |
| 198 | 200 | control->i2c_address + |
| 199 | 201 | control->ras_header_offset, |
| 200 | 202 | buf, RAS_TABLE_HEADER_SIZE); |
| 201 | − up_read(&adev->reset_sem); | |
| 203 | + up_read(&adev->reset_domain->sem); | |
| 202 | 204 | |
| 203 | 205 | if (res < 0) { |
| 204 | 206 | DRM_ERROR("Failed to write EEPROM table header:%d", res); |
| @@ -263,6 +265,7 @@ static int amdgpu_ras_eeprom_correct_header_tag( · 58 unmodified lines | ||
| 263 | 265 | */ |
| 264 | 266 | int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control) |
| 265 | 267 | { |
| 268 | + struct amdgpu_device *adev = to_amdgpu_device(control); | |
| 266 | 269 | struct amdgpu_ras_eeprom_table_header *hdr = &control->tbl_hdr; |
| 267 | 270 | u8 csum; |
| 268 | 271 | int res; |
| @@ -282,6 +285,8 @@ int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control) · 13 unmodified lines | ||
| 282 | 285 | control->ras_num_recs = 0; |
| 283 | 286 | control->ras_fri = 0; |
| 284 | 287 | |
| 288 | + amdgpu_dpm_send_hbm_bad_pages_num(adev, control->ras_num_recs); | |
| 289 | + | |
| 285 | 290 | amdgpu_ras_debugfs_set_ret_size(control); |
| 286 | 291 | |
| 287 | 292 | mutex_unlock(&control->ras_tbl_mutex); |
| @@ -387,13 +392,13 @@ static int __amdgpu_ras_eeprom_write(struct amdgpu_ras_eeprom_control *control, · 99 unmodified lines | ||
| 387 | 392 | int res; |
| 388 | 393 | |
| 389 | 394 | /* i2c may be unstable in gpu reset */ |
| 390 | − down_read(&adev->reset_sem); | |
| 395 | + down_read(&adev->reset_domain->sem); | |
| 391 | 396 | buf_size = num * RAS_TABLE_RECORD_SIZE; |
| 392 | − res = amdgpu_eeprom_write(&adev->pm.smu_i2c, | |
| 397 | + res = amdgpu_eeprom_write(adev->pm.ras_eeprom_i2c_bus, | |
| 393 | 398 | control->i2c_address + |
| 394 | 399 | RAS_INDEX_TO_OFFSET(control, fri), |
| 395 | 400 | buf, buf_size); |
| 396 | − up_read(&adev->reset_sem); | |
| 401 | + up_read(&adev->reset_domain->sem); | |
| 397 | 402 | if (res < 0) { |
| 398 | 403 | DRM_ERROR("Writing %d EEPROM table records error:%d", |
| 399 | 404 | num, res); |
| @@ -547,12 +552,12 @@ amdgpu_ras_eeprom_update_header(struct amdgpu_ras_eeprom_control *control) · 147 unmodified lines | ||
| 547 | 552 | goto Out; |
| 548 | 553 | } |
| 549 | 554 | |
| 550 | − down_read(&adev->reset_sem); | |
| 551 | − res = amdgpu_eeprom_read(&adev->pm.smu_i2c, | |
| 555 | + down_read(&adev->reset_domain->sem); | |
| 556 | + res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus, | |
| 552 | 557 | control->i2c_address + |
| 553 | 558 | control->ras_record_offset, |
| 554 | 559 | buf, buf_size); |
| 555 | − up_read(&adev->reset_sem); | |
| 560 | + up_read(&adev->reset_domain->sem); | |
| 556 | 561 | if (res < 0) { |
| 557 | 562 | DRM_ERROR("EEPROM failed reading records:%d\n", |
| 558 | 563 | res); |
| @@ -642,13 +647,13 @@ static int __amdgpu_ras_eeprom_read(struct amdgpu_ras_eeprom_control *control, · 83 unmodified lines | ||
| 642 | 647 | int res; |
| 643 | 648 | |
| 644 | 649 | /* i2c may be unstable in gpu reset */ |
| 645 | − down_read(&adev->reset_sem); | |
| 650 | + down_read(&adev->reset_domain->sem); | |
| 646 | 651 | buf_size = num * RAS_TABLE_RECORD_SIZE; |
| 647 | − res = amdgpu_eeprom_read(&adev->pm.smu_i2c, | |
| 652 | + res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus, | |
| 648 | 653 | control->i2c_address + |
| 649 | 654 | RAS_INDEX_TO_OFFSET(control, fri), |
| 650 | 655 | buf, buf_size); |
| 651 | − up_read(&adev->reset_sem); | |
| 656 | + up_read(&adev->reset_domain->sem); | |
| 652 | 657 | if (res < 0) { |
| 653 | 658 | DRM_ERROR("Reading %d EEPROM table records error:%d", |
| 654 | 659 | num, res); |
| @@ -1009,7 +1014,7 @@ static int __verify_ras_table_checksum(struct amdgpu_ras_eeprom_control *control · 354 unmodified lines | ||
| 1009 | 1014 | return -ENOMEM; |
| 1010 | 1015 | } |
| 1011 | 1016 | |
| 1012 | − res = amdgpu_eeprom_read(&adev->pm.smu_i2c, | |
| 1017 | + res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus, | |
| 1013 | 1018 | control->i2c_address + |
| 1014 | 1019 | control->ras_header_offset, |
| 1015 | 1020 | buf, buf_size); |
| @@ -1045,7 +1050,7 @@ int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control, · 29 unmodified lines | ||
| 1045 | 1050 | return 0; |
| 1046 | 1051 | |
| 1047 | 1052 | /* Verify i2c adapter is initialized */ |
| 1048 | − if (!adev->pm.smu_i2c.algo) | |
| 1053 | + if (!adev->pm.ras_eeprom_i2c_bus || !adev->pm.ras_eeprom_i2c_bus->algo) | |
| 1049 | 1054 | return -ENOENT; |
| 1050 | 1055 | |
| 1051 | 1056 | if (!__get_eeprom_i2c_addr(adev, control)) |
| @@ -1057,7 +1062,7 @@ int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control, · 5 unmodified lines | ||
| 1057 | 1062 | mutex_init(&control->ras_tbl_mutex); |
| 1058 | 1063 | |
| 1059 | 1064 | /* Read the table header from EEPROM address */ |
| 1060 | − res = amdgpu_eeprom_read(&adev->pm.smu_i2c, | |
| 1065 | + res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus, | |
| 1061 | 1066 | control->i2c_address + control->ras_header_offset, |
| 1062 | 1067 | buf, RAS_TABLE_HEADER_SIZE); |
| 1063 | 1068 | if (res < RAS_TABLE_HEADER_SIZE) { |
| @@ -96,3 +96,59 @@ int amdgpu_reset_perform_reset(struct amdgpu_device *adev, · 95 unmodified lines | ||
| 96 | 96 | return reset_handler->restore_hwcontext(adev->reset_cntl, |
| 97 | 97 | reset_context); |
| 98 | 98 | } |
| 99 | + | |
| 100 | + | |
| 101 | +void amdgpu_reset_destroy_reset_domain(struct kref *ref) | |
| 102 | +{ | |
| 103 | + struct amdgpu_reset_domain *reset_domain = container_of(ref, | |
| 104 | + struct amdgpu_reset_domain, | |
| 105 | + refcount); | |
| 106 | + if (reset_domain->wq) | |
| 107 | + destroy_workqueue(reset_domain->wq); | |
| 108 | + | |
| 109 | + kvfree(reset_domain); | |
| 110 | +} | |
| 111 | + | |
| 112 | +struct amdgpu_reset_domain *amdgpu_reset_create_reset_domain(enum amdgpu_reset_domain_type type, | |
| 113 | + char *wq_name) | |
| 114 | +{ | |
| 115 | + struct amdgpu_reset_domain *reset_domain; | |
| 116 | + | |
| 117 | + reset_domain = kvzalloc(sizeof(struct amdgpu_reset_domain), GFP_KERNEL); | |
| 118 | + if (!reset_domain) { | |
| 119 | + DRM_ERROR("Failed to allocate amdgpu_reset_domain!"); | |
| 120 | + return NULL; | |
| 121 | + } | |
| 122 | + | |
| 123 | + reset_domain->type = type; | |
| 124 | + kref_init(&reset_domain->refcount); | |
| 125 | + | |
| 126 | + reset_domain->wq = create_singlethread_workqueue(wq_name); | |
| 127 | + if (!reset_domain->wq) { | |
| 128 | + DRM_ERROR("Failed to allocate wq for amdgpu_reset_domain!"); | |
| 129 | + amdgpu_reset_put_reset_domain(reset_domain); | |
| 130 | + return NULL; | |
| 131 | + | |
| 132 | + } | |
| 133 | + | |
| 134 | + atomic_set(&reset_domain->in_gpu_reset, 0); | |
| 135 | + init_rwsem(&reset_domain->sem); | |
| 136 | + | |
| 137 | + return reset_domain; | |
| 138 | +} | |
| 139 | + | |
| 140 | +void amdgpu_device_lock_reset_domain(struct amdgpu_reset_domain *reset_domain) | |
| 141 | +{ | |
| 142 | + atomic_set(&reset_domain->in_gpu_reset, 1); | |
| 143 | + down_write(&reset_domain->sem); | |
| 144 | +} | |
| 145 | + | |
| 146 | + | |
| 147 | +void amdgpu_device_unlock_reset_domain(struct amdgpu_reset_domain *reset_domain) | |
| 148 | +{ | |
| 149 | + atomic_set(&reset_domain->in_gpu_reset, 0); | |
| 150 | + up_write(&reset_domain->sem); | |
| 151 | +} | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| @@ -70,6 +70,21 @@ struct amdgpu_reset_control { · 69 unmodified lines | ||
| 70 | 70 | void (*async_reset)(struct work_struct *work); |
| 71 | 71 | }; |
| 72 | 72 | |
| 73 | + | |
| 74 | +enum amdgpu_reset_domain_type { | |
| 75 | + SINGLE_DEVICE, | |
| 76 | + XGMI_HIVE | |
| 77 | +}; | |
| 78 | + | |
| 79 | +struct amdgpu_reset_domain { | |
| 80 | + struct kref refcount; | |
| 81 | + struct workqueue_struct *wq; | |
| 82 | + enum amdgpu_reset_domain_type type; | |
| 83 | + struct rw_semaphore sem; | |
| 84 | + atomic_t in_gpu_reset; | |
| 85 | +}; | |
| 86 | + | |
| 87 | + | |
| 73 | 88 | int amdgpu_reset_init(struct amdgpu_device *adev); |
| 74 | 89 | int amdgpu_reset_fini(struct amdgpu_device *adev); |
| 75 | 90 | |
| @@ -82,4 +97,29 @@ int amdgpu_reset_perform_reset(struct amdgpu_device *adev, · 6 unmodified lines | ||
| 82 | 97 | int amdgpu_reset_add_handler(struct amdgpu_reset_control *reset_ctl, |
| 83 | 98 | struct amdgpu_reset_handler *handler); |
| 84 | 99 | |
| 100 | +struct amdgpu_reset_domain *amdgpu_reset_create_reset_domain(enum amdgpu_reset_domain_type type, | |
| 101 | + char *wq_name); | |
| 102 | + | |
| 103 | +void amdgpu_reset_destroy_reset_domain(struct kref *ref); | |
| 104 | + | |
| 105 | +static inline bool amdgpu_reset_get_reset_domain(struct amdgpu_reset_domain *domain) | |
| 106 | +{ | |
| 107 | + return kref_get_unless_zero(&domain->refcount) != 0; | |
| 108 | +} | |
| 109 | + | |
| 110 | +static inline void amdgpu_reset_put_reset_domain(struct amdgpu_reset_domain *domain) | |
| 111 | +{ | |
| 112 | + kref_put(&domain->refcount, amdgpu_reset_destroy_reset_domain); | |
| 113 | +} | |
| 114 | + | |
| 115 | +static inline bool amdgpu_reset_domain_schedule(struct amdgpu_reset_domain *domain, | |
| 116 | + struct work_struct *work) | |
| 117 | +{ | |
| 118 | + return queue_work(domain->wq, work); | |
| 119 | +} | |
| 120 | + | |
| 121 | +void amdgpu_device_lock_reset_domain(struct amdgpu_reset_domain *reset_domain); | |
| 122 | + | |
| 123 | +void amdgpu_device_unlock_reset_domain(struct amdgpu_reset_domain *reset_domain); | |
| 124 | + | |
| 85 | 125 | #endif |
| @@ -191,8 +191,9 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, · 190 unmodified lines | ||
| 191 | 191 | ring->adev = adev; |
| 192 | 192 | ring->idx = adev->num_rings++; |
| 193 | 193 | adev->rings[ring->idx] = ring; |
| 194 | − r = amdgpu_fence_driver_init_ring(ring, sched_hw_submission, | |
| 195 | − sched_score); | |
| 194 | + ring->num_hw_submission = sched_hw_submission; | |
| 195 | + ring->sched_score = sched_score; | |
| 196 | + r = amdgpu_fence_driver_init_ring(ring); | |
| 196 | 197 | if (r) |
| 197 | 198 | return r; |
| 198 | 199 | } |
| @@ -114,9 +114,7 @@ struct amdgpu_fence_driver { · 113 unmodified lines | ||
| 114 | 114 | void amdgpu_fence_driver_clear_job_fences(struct amdgpu_ring *ring); |
| 115 | 115 | void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring); |
| 116 | 116 | |
| 117 | −int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, | |
| 118 | − unsigned num_hw_submission, | |
| 119 | − atomic_t *sched_score); | |
| 117 | +int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring); | |
| 120 | 118 | int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring, |
| 121 | 119 | struct amdgpu_irq_src *irq_src, |
| 122 | 120 | unsigned irq_type); |
| @@ -251,6 +249,8 @@ struct amdgpu_ring { · 128 unmodified lines | ||
| 251 | 249 | bool has_compute_vm_bug; |
| 252 | 250 | bool no_scheduler; |
| 253 | 251 | int hw_prio; |
| 252 | + unsigned num_hw_submission; | |
| 253 | + atomic_t *sched_score; | |
| 254 | 254 | }; |
| 255 | 255 | |
| 256 | 256 | #define amdgpu_ring_parse_cs(r, p, ib) ((r)->funcs->parse_cs((p), (ib))) |
| @@ -127,11 +127,19 @@ struct amdgpu_rlc_funcs { · 126 unmodified lines | ||
| 127 | 127 | void (*reset)(struct amdgpu_device *adev); |
| 128 | 128 | void (*start)(struct amdgpu_device *adev); |
| 129 | 129 | void (*update_spm_vmid)(struct amdgpu_device *adev, unsigned vmid); |
| 130 | − void (*sriov_wreg)(struct amdgpu_device *adev, u32 offset, u32 v, u32 acc_flags, u32 hwip); | |
| 131 | − u32 (*sriov_rreg)(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip); | |
| 132 | 130 | bool (*is_rlcg_access_range)(struct amdgpu_device *adev, uint32_t reg); |
| 133 | 131 | }; |
| 134 | 132 | |
| 133 | +struct amdgpu_rlcg_reg_access_ctrl { | |
| 134 | + uint32_t scratch_reg0; | |
| 135 | + uint32_t scratch_reg1; | |
| 136 | + uint32_t scratch_reg2; | |
| 137 | + uint32_t scratch_reg3; | |
| 138 | + uint32_t grbm_cntl; | |
| 139 | + uint32_t grbm_idx; | |
| 140 | + uint32_t spare_int; | |
| 141 | +}; | |
| 142 | + | |
| 135 | 143 | struct amdgpu_rlc { |
| 136 | 144 | /* for power gating */ |
| 137 | 145 | struct amdgpu_bo *save_restore_obj; |
| @@ -191,6 +199,10 @@ struct amdgpu_rlc { · 53 unmodified lines | ||
| 191 | 199 | struct amdgpu_bo *rlc_toc_bo; |
| 192 | 200 | uint64_t rlc_toc_gpu_addr; |
| 193 | 201 | void *rlc_toc_buf; |
| 202 | + | |
| 203 | + bool rlcg_reg_access_supported; | |
| 204 | + /* registers for rlcg indirect reg access */ | |
| 205 | + struct amdgpu_rlcg_reg_access_ctrl reg_access_ctrl; | |
| 194 | 206 | }; |
| 195 | 207 | |
| 196 | 208 | void amdgpu_gfx_rlc_enter_safe_mode(struct amdgpu_device *adev); |
| @@ -87,71 +87,35 @@ uint64_t amdgpu_sdma_get_csa_mc_addr(struct amdgpu_ring *ring, · 86 unmodified lines | ||
| 87 | 87 | } |
| 88 | 88 | |
| 89 | 89 | int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev, |
| 90 | − void *ras_ih_info) | |
| 90 | + struct ras_common_if *ras_block) | |
| 91 | 91 | { |
| 92 | 92 | int r, i; |
| 93 | − struct ras_ih_if *ih_info = (struct ras_ih_if *)ras_ih_info; | |
| 94 | − struct ras_fs_if fs_info = { | |
| 95 | − .sysfs_name = "sdma_err_count", | |
| 96 | − }; | |
| 97 | − | |
| 98 | − if (!ih_info) | |
| 99 | − return -EINVAL; | |
| 100 | 93 | |
| 101 | − if (!adev->sdma.ras_if) { | |
| 102 | − adev->sdma.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 103 | − if (!adev->sdma.ras_if) | |
| 104 | − return -ENOMEM; | |
| 105 | − adev->sdma.ras_if->block = AMDGPU_RAS_BLOCK__SDMA; | |
| 106 | − adev->sdma.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 107 | − adev->sdma.ras_if->sub_block_index = 0; | |
| 108 | − } | |
| 109 | − fs_info.head = ih_info->head = *adev->sdma.ras_if; | |
| 110 | − | |
| 111 | − r = amdgpu_ras_late_init(adev, adev->sdma.ras_if, | |
| 112 | − &fs_info, ih_info); | |
| 94 | + r = amdgpu_ras_block_late_init(adev, ras_block); | |
| 113 | 95 | if (r) |
| 114 | − goto free; | |
| 96 | + return r; | |
| 115 | 97 | |
| 116 | − if (amdgpu_ras_is_supported(adev, adev->sdma.ras_if->block)) { | |
| 98 | + if (amdgpu_ras_is_supported(adev, ras_block->block)) { | |
| 117 | 99 | for (i = 0; i < adev->sdma.num_instances; i++) { |
| 118 | 100 | r = amdgpu_irq_get(adev, &adev->sdma.ecc_irq, |
| 119 | 101 | AMDGPU_SDMA_IRQ_INSTANCE0 + i); |
| 120 | 102 | if (r) |
| 121 | 103 | goto late_fini; |
| 122 | 104 | } |
| 123 | − } else { | |
| 124 | − r = 0; | |
| 125 | − goto free; | |
| 126 | 105 | } |
| 127 | 106 | |
| 128 | 107 | return 0; |
| 129 | 108 | |
| 130 | 109 | late_fini: |
| 131 | − amdgpu_ras_late_fini(adev, adev->sdma.ras_if, ih_info); | |
| 132 | −free: | |
| 133 | − kfree(adev->sdma.ras_if); | |
| 134 | − adev->sdma.ras_if = NULL; | |
| 110 | + amdgpu_ras_block_late_fini(adev, ras_block); | |
| 135 | 111 | return r; |
| 136 | 112 | } |
| 137 | 113 | |
| 138 | 114 | void amdgpu_sdma_ras_fini(struct amdgpu_device *adev) |
| 139 | 115 | { |
| 140 | 116 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__SDMA) && |
| 141 | − adev->sdma.ras_if) { | |
| 142 | − struct ras_common_if *ras_if = adev->sdma.ras_if; | |
| 143 | − struct ras_ih_if ih_info = { | |
| 144 | − .head = *ras_if, | |
| 145 | − /* the cb member will not be used by | |
| 146 | − * amdgpu_ras_interrupt_remove_handler, init it only | |
| 147 | − * to cheat the check in ras_late_fini | |
| 148 | − */ | |
| 149 | − .cb = amdgpu_sdma_process_ras_data_cb, | |
| 150 | − }; | |
| 151 | − | |
| 152 | − amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 153 | − kfree(ras_if); | |
| 154 | − } | |
| 117 | + adev->sdma.ras_if) | |
| 118 | + amdgpu_ras_block_late_fini(adev, adev->sdma.ras_if); | |
| 155 | 119 | } |
| 156 | 120 | |
| 157 | 121 | int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev, |
| @@ -23,6 +23,7 @@ · 22 unmodified lines | ||
| 23 | 23 | |
| 24 | 24 | #ifndef __AMDGPU_SDMA_H__ |
| 25 | 25 | #define __AMDGPU_SDMA_H__ |
| 26 | +#include "amdgpu_ras.h" | |
| 26 | 27 | |
| 27 | 28 | /* max number of IP instances */ |
| 28 | 29 | #define AMDGPU_MAX_SDMA_INSTANCES 8 |
| @@ -50,13 +51,8 @@ struct amdgpu_sdma_instance { · 21 unmodified lines | ||
| 50 | 51 | bool burst_nop; |
| 51 | 52 | }; |
| 52 | 53 | |
| 53 | −struct amdgpu_sdma_ras_funcs { | |
| 54 | − int (*ras_late_init)(struct amdgpu_device *adev, | |
| 55 | − void *ras_ih_info); | |
| 56 | − void (*ras_fini)(struct amdgpu_device *adev); | |
| 57 | − int (*query_ras_error_count)(struct amdgpu_device *adev, | |
| 58 | − uint32_t instance, void *ras_error_status); | |
| 59 | − void (*reset_ras_error_count)(struct amdgpu_device *adev); | |
| 54 | +struct amdgpu_sdma_ras { | |
| 55 | + struct amdgpu_ras_block_object ras_block; | |
| 60 | 56 | }; |
| 61 | 57 | |
| 62 | 58 | struct amdgpu_sdma { |
| @@ -73,7 +69,7 @@ struct amdgpu_sdma { · 10 unmodified lines | ||
| 73 | 69 | uint32_t srbm_soft_reset; |
| 74 | 70 | bool has_page_queue; |
| 75 | 71 | struct ras_common_if *ras_if; |
| 76 | − const struct amdgpu_sdma_ras_funcs *funcs; | |
| 72 | + struct amdgpu_sdma_ras *ras; | |
| 77 | 73 | }; |
| 78 | 74 | |
| 79 | 75 | /* |
| @@ -121,7 +117,7 @@ amdgpu_sdma_get_instance_from_ring(struct amdgpu_ring *ring); · 41 unmodified lines | ||
| 121 | 117 | int amdgpu_sdma_get_index_from_ring(struct amdgpu_ring *ring, uint32_t *index); |
| 122 | 118 | uint64_t amdgpu_sdma_get_csa_mc_addr(struct amdgpu_ring *ring, unsigned vmid); |
| 123 | 119 | int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev, |
| 124 | − void *ras_ih_info); | |
| 120 | + struct ras_common_if *ras_block); | |
| 125 | 121 | void amdgpu_sdma_ras_fini(struct amdgpu_device *adev); |
| 126 | 122 | int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev, |
| 127 | 123 | void *err_data, |
| @@ -261,10 +261,9 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct amdgpu_sync *sync, · 260 unmodified lines | ||
| 261 | 261 | |
| 262 | 262 | dma_resv_for_each_fence(&cursor, resv, true, f) { |
| 263 | 263 | dma_fence_chain_for_each(f, f) { |
| 264 | − struct dma_fence_chain *chain = to_dma_fence_chain(f); | |
| 264 | + struct dma_fence *tmp = dma_fence_chain_contained(f); | |
| 265 | 265 | |
| 266 | − if (amdgpu_sync_test_fence(adev, mode, owner, chain ? | |
| 267 | − chain->fence : f)) { | |
| 266 | + if (amdgpu_sync_test_fence(adev, mode, owner, tmp)) { | |
| 268 | 267 | r = amdgpu_sync_fence(sync, f); |
| 269 | 268 | dma_fence_put(f); |
| 270 | 269 | if (r) |
| @@ -1,250 +0,0 @@ | ||
| 1 | −// SPDX-License-Identifier: GPL-2.0 OR MIT | |
| 2 | −/* | |
| 3 | − * Copyright 2009 VMware, Inc. | |
| 4 | − * | |
| 5 | − * Permission is hereby granted, free of charge, to any person obtaining a | |
| 6 | − * copy of this software and associated documentation files (the "Software"), | |
| 7 | − * to deal in the Software without restriction, including without limitation | |
| 8 | − * the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| 9 | − * and/or sell copies of the Software, and to permit persons to whom the | |
| 10 | − * Software is furnished to do so, subject to the following conditions: | |
| 11 | − * | |
| 12 | − * The above copyright notice and this permission notice shall be included in | |
| 13 | − * all copies or substantial portions of the Software. | |
| 14 | − * | |
| 15 | − * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | |
| 16 | − * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | |
| 17 | − * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | |
| 18 | − * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | |
| 19 | − * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | |
| 20 | − * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | |
| 21 | − * OTHER DEALINGS IN THE SOFTWARE. | |
| 22 | − * | |
| 23 | − * Authors: Michel Dänzer | |
| 24 | − */ | |
| 25 | − | |
| 26 | −#include <drm/amdgpu_drm.h> | |
| 27 | −#include "amdgpu.h" | |
| 28 | −#include "amdgpu_uvd.h" | |
| 29 | −#include "amdgpu_vce.h" | |
| 30 | − | |
| 31 | −/* Test BO GTT->VRAM and VRAM->GTT GPU copies across the whole GTT aperture */ | |
| 32 | −static void amdgpu_do_test_moves(struct amdgpu_device *adev) | |
| 33 | −{ | |
| 34 | − struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; | |
| 35 | − struct amdgpu_bo *vram_obj = NULL; | |
| 36 | − struct amdgpu_bo **gtt_obj = NULL; | |
| 37 | − struct amdgpu_bo_param bp; | |
| 38 | − uint64_t gart_addr, vram_addr; | |
| 39 | − unsigned n, size; | |
| 40 | − int i, r; | |
| 41 | − | |
| 42 | − size = 1024 * 1024; | |
| 43 | − | |
| 44 | − /* Number of tests = | |
| 45 | − * (Total GTT - gart_pin_size - (2 transfer windows for buffer moves)) / test size | |
| 46 | − */ | |
| 47 | − n = adev->gmc.gart_size - atomic64_read(&adev->gart_pin_size); | |
| 48 | − n -= AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS * | |
| 49 | − AMDGPU_GPU_PAGE_SIZE; | |
| 50 | − n /= size; | |
| 51 | − | |
| 52 | − gtt_obj = kcalloc(n, sizeof(*gtt_obj), GFP_KERNEL); | |
| 53 | − if (!gtt_obj) { | |
| 54 | − DRM_ERROR("Failed to allocate %d pointers\n", n); | |
| 55 | − r = 1; | |
| 56 | − goto out_cleanup; | |
| 57 | − } | |
| 58 | − memset(&bp, 0, sizeof(bp)); | |
| 59 | − bp.size = size; | |
| 60 | − bp.byte_align = PAGE_SIZE; | |
| 61 | − bp.domain = AMDGPU_GEM_DOMAIN_VRAM; | |
| 62 | − bp.flags = 0; | |
| 63 | − bp.type = ttm_bo_type_kernel; | |
| 64 | − bp.resv = NULL; | |
| 65 | − bp.bo_ptr_size = sizeof(struct amdgpu_bo); | |
| 66 | − | |
| 67 | − r = amdgpu_bo_create(adev, &bp, &vram_obj); | |
| 68 | − if (r) { | |
| 69 | − DRM_ERROR("Failed to create VRAM object\n"); | |
| 70 | − goto out_cleanup; | |
| 71 | − } | |
| 72 | − r = amdgpu_bo_reserve(vram_obj, false); | |
| 73 | − if (unlikely(r != 0)) | |
| 74 | − goto out_unref; | |
| 75 | − r = amdgpu_bo_pin(vram_obj, AMDGPU_GEM_DOMAIN_VRAM); | |
| 76 | − if (r) { | |
| 77 | − DRM_ERROR("Failed to pin VRAM object\n"); | |
| 78 | − goto out_unres; | |
| 79 | − } | |
| 80 | − vram_addr = amdgpu_bo_gpu_offset(vram_obj); | |
| 81 | − for (i = 0; i < n; i++) { | |
| 82 | − void *gtt_map, *vram_map; | |
| 83 | − void **gart_start, **gart_end; | |
| 84 | − void **vram_start, **vram_end; | |
| 85 | − struct dma_fence *fence = NULL; | |
| 86 | − | |
| 87 | − bp.domain = AMDGPU_GEM_DOMAIN_GTT; | |
| 88 | − r = amdgpu_bo_create(adev, &bp, gtt_obj + i); | |
| 89 | − if (r) { | |
| 90 | − DRM_ERROR("Failed to create GTT object %d\n", i); | |
| 91 | − goto out_lclean; | |
| 92 | − } | |
| 93 | − | |
| 94 | − r = amdgpu_bo_reserve(gtt_obj[i], false); | |
| 95 | − if (unlikely(r != 0)) | |
| 96 | − goto out_lclean_unref; | |
| 97 | − r = amdgpu_bo_pin(gtt_obj[i], AMDGPU_GEM_DOMAIN_GTT); | |
| 98 | − if (r) { | |
| 99 | − DRM_ERROR("Failed to pin GTT object %d\n", i); | |
| 100 | − goto out_lclean_unres; | |
| 101 | − } | |
| 102 | − r = amdgpu_ttm_alloc_gart(>t_obj[i]->tbo); | |
| 103 | − if (r) { | |
| 104 | − DRM_ERROR("%p bind failed\n", gtt_obj[i]); | |
| 105 | − goto out_lclean_unpin; | |
| 106 | − } | |
| 107 | − gart_addr = amdgpu_bo_gpu_offset(gtt_obj[i]); | |
| 108 | − | |
| 109 | − r = amdgpu_bo_kmap(gtt_obj[i], >t_map); | |
| 110 | − if (r) { | |
| 111 | − DRM_ERROR("Failed to map GTT object %d\n", i); | |
| 112 | − goto out_lclean_unpin; | |
| 113 | − } | |
| 114 | − | |
| 115 | − for (gart_start = gtt_map, gart_end = gtt_map + size; | |
| 116 | − gart_start < gart_end; | |
| 117 | − gart_start++) | |
| 118 | − *gart_start = gart_start; | |
| 119 | − | |
| 120 | − amdgpu_bo_kunmap(gtt_obj[i]); | |
| 121 | − | |
| 122 | − r = amdgpu_copy_buffer(ring, gart_addr, vram_addr, | |
| 123 | − size, NULL, &fence, false, false, false); | |
| 124 | − | |
| 125 | − if (r) { | |
| 126 | − DRM_ERROR("Failed GTT->VRAM copy %d\n", i); | |
| 127 | − goto out_lclean_unpin; | |
| 128 | − } | |
| 129 | − | |
| 130 | − r = dma_fence_wait(fence, false); | |
| 131 | − if (r) { | |
| 132 | − DRM_ERROR("Failed to wait for GTT->VRAM fence %d\n", i); | |
| 133 | − goto out_lclean_unpin; | |
| 134 | − } | |
| 135 | − | |
| 136 | − dma_fence_put(fence); | |
| 137 | − fence = NULL; | |
| 138 | − | |
| 139 | − r = amdgpu_bo_kmap(vram_obj, &vram_map); | |
| 140 | − if (r) { | |
| 141 | − DRM_ERROR("Failed to map VRAM object after copy %d\n", i); | |
| 142 | − goto out_lclean_unpin; | |
| 143 | − } | |
| 144 | − | |
| 145 | − for (gart_start = gtt_map, gart_end = gtt_map + size, | |
| 146 | − vram_start = vram_map, vram_end = vram_map + size; | |
| 147 | − vram_start < vram_end; | |
| 148 | − gart_start++, vram_start++) { | |
| 149 | − if (*vram_start != gart_start) { | |
| 150 | − DRM_ERROR("Incorrect GTT->VRAM copy %d: Got 0x%p, " | |
| 151 | − "expected 0x%p (GTT/VRAM offset " | |
| 152 | − "0x%16llx/0x%16llx)\n", | |
| 153 | − i, *vram_start, gart_start, | |
| 154 | − (unsigned long long) | |
| 155 | − (gart_addr - adev->gmc.gart_start + | |
| 156 | − (void *)gart_start - gtt_map), | |
| 157 | − (unsigned long long) | |
| 158 | − (vram_addr - adev->gmc.vram_start + | |
| 159 | − (void *)gart_start - gtt_map)); | |
| 160 | − amdgpu_bo_kunmap(vram_obj); | |
| 161 | − goto out_lclean_unpin; | |
| 162 | − } | |
| 163 | − *vram_start = vram_start; | |
| 164 | − } | |
| 165 | − | |
| 166 | − amdgpu_bo_kunmap(vram_obj); | |
| 167 | − | |
| 168 | − r = amdgpu_copy_buffer(ring, vram_addr, gart_addr, | |
| 169 | − size, NULL, &fence, false, false, false); | |
| 170 | − | |
| 171 | − if (r) { | |
| 172 | − DRM_ERROR("Failed VRAM->GTT copy %d\n", i); | |
| 173 | − goto out_lclean_unpin; | |
| 174 | − } | |
| 175 | − | |
| 176 | − r = dma_fence_wait(fence, false); | |
| 177 | − if (r) { | |
| 178 | − DRM_ERROR("Failed to wait for VRAM->GTT fence %d\n", i); | |
| 179 | − goto out_lclean_unpin; | |
| 180 | − } | |
| 181 | − | |
| 182 | − dma_fence_put(fence); | |
| 183 | − fence = NULL; | |
| 184 | − | |
| 185 | − r = amdgpu_bo_kmap(gtt_obj[i], >t_map); | |
| 186 | − if (r) { | |
| 187 | − DRM_ERROR("Failed to map GTT object after copy %d\n", i); | |
| 188 | − goto out_lclean_unpin; | |
| 189 | − } | |
| 190 | − | |
| 191 | − for (gart_start = gtt_map, gart_end = gtt_map + size, | |
| 192 | − vram_start = vram_map, vram_end = vram_map + size; | |
| 193 | − gart_start < gart_end; | |
| 194 | − gart_start++, vram_start++) { | |
| 195 | − if (*gart_start != vram_start) { | |
| 196 | − DRM_ERROR("Incorrect VRAM->GTT copy %d: Got 0x%p, " | |
| 197 | − "expected 0x%p (VRAM/GTT offset " | |
| 198 | − "0x%16llx/0x%16llx)\n", | |
| 199 | − i, *gart_start, vram_start, | |
| 200 | − (unsigned long long) | |
| 201 | − (vram_addr - adev->gmc.vram_start + | |
| 202 | − (void *)vram_start - vram_map), | |
| 203 | − (unsigned long long) | |
| 204 | − (gart_addr - adev->gmc.gart_start + | |
| 205 | − (void *)vram_start - vram_map)); | |
| 206 | − amdgpu_bo_kunmap(gtt_obj[i]); | |
| 207 | − goto out_lclean_unpin; | |
| 208 | − } | |
| 209 | − } | |
| 210 | − | |
| 211 | − amdgpu_bo_kunmap(gtt_obj[i]); | |
| 212 | − | |
| 213 | − DRM_INFO("Tested GTT->VRAM and VRAM->GTT copy for GTT offset 0x%llx\n", | |
| 214 | − gart_addr - adev->gmc.gart_start); | |
| 215 | − continue; | |
| 216 | − | |
| 217 | −out_lclean_unpin: | |
| 218 | − amdgpu_bo_unpin(gtt_obj[i]); | |
| 219 | −out_lclean_unres: | |
| 220 | − amdgpu_bo_unreserve(gtt_obj[i]); | |
| 221 | −out_lclean_unref: | |
| 222 | − amdgpu_bo_unref(>t_obj[i]); | |
| 223 | −out_lclean: | |
| 224 | − for (--i; i >= 0; --i) { | |
| 225 | − amdgpu_bo_unpin(gtt_obj[i]); | |
| 226 | − amdgpu_bo_unreserve(gtt_obj[i]); | |
| 227 | − amdgpu_bo_unref(>t_obj[i]); | |
| 228 | − } | |
| 229 | − if (fence) | |
| 230 | − dma_fence_put(fence); | |
| 231 | − break; | |
| 232 | − } | |
| 233 | − | |
| 234 | − amdgpu_bo_unpin(vram_obj); | |
| 235 | −out_unres: | |
| 236 | − amdgpu_bo_unreserve(vram_obj); | |
| 237 | −out_unref: | |
| 238 | − amdgpu_bo_unref(&vram_obj); | |
| 239 | −out_cleanup: | |
| 240 | − kfree(gtt_obj); | |
| 241 | − if (r) { | |
| 242 | − pr_warn("Error while testing BO move\n"); | |
| 243 | − } | |
| 244 | −} | |
| 245 | − | |
| 246 | −void amdgpu_test_moves(struct amdgpu_device *adev) | |
| 247 | −{ | |
| 248 | − if (adev->mman.buffer_funcs) | |
| 249 | − amdgpu_do_test_moves(adev); | |
| 250 | −} | |
| @@ -358,11 +358,10 @@ TRACE_EVENT(amdgpu_vm_update_ptes, · 357 unmodified lines | ||
| 358 | 358 | } |
| 359 | 359 | ), |
| 360 | 360 | TP_printk("pid:%u vm_ctx:0x%llx start:0x%010llx end:0x%010llx," |
| 361 | − " flags:0x%llx, incr:%llu, dst:\n%s%s", __entry->pid, | |
| 361 | + " flags:0x%llx, incr:%llu, dst:\n%s", __entry->pid, | |
| 362 | 362 | __entry->vm_ctx, __entry->start, __entry->end, |
| 363 | 363 | __entry->flags, __entry->incr, __print_array( |
| 364 | − __get_dynamic_array(dst), min(__entry->nptes, 32u), 8), | |
| 365 | − __entry->nptes > 32 ? "..." : "") | |
| 364 | + __get_dynamic_array(dst), __entry->nptes, 8)) | |
| 366 | 365 | ); |
| 367 | 366 | |
| 368 | 367 | TRACE_EVENT(amdgpu_vm_set_ptes, |
| @@ -537,6 +536,22 @@ TRACE_EVENT(amdgpu_ib_pipe_sync, · 168 unmodified lines | ||
| 537 | 536 | __entry->seqno) |
| 538 | 537 | ); |
| 539 | 538 | |
| 539 | +TRACE_EVENT(amdgpu_reset_reg_dumps, | |
| 540 | + TP_PROTO(uint32_t address, uint32_t value), | |
| 541 | + TP_ARGS(address, value), | |
| 542 | + TP_STRUCT__entry( | |
| 543 | + __field(uint32_t, address) | |
| 544 | + __field(uint32_t, value) | |
| 545 | + ), | |
| 546 | + TP_fast_assign( | |
| 547 | + __entry->address = address; | |
| 548 | + __entry->value = value; | |
| 549 | + ), | |
| 550 | + TP_printk("amdgpu register dump 0x%x: 0x%x", | |
| 551 | + __entry->address, | |
| 552 | + __entry->value) | |
| 553 | +); | |
| 554 | + | |
| 540 | 555 | #undef AMDGPU_JOB_GET_TIMELINE_NAME |
| 541 | 556 | #endif |
| 542 | 557 | |
| @@ -50,6 +50,7 @@ · 49 unmodified lines | ||
| 50 | 50 | #include <drm/ttm/ttm_range_manager.h> |
| 51 | 51 | |
| 52 | 52 | #include <drm/amdgpu_drm.h> |
| 53 | +#include <drm/drm_drv.h> | |
| 53 | 54 | |
| 54 | 55 | #include "amdgpu.h" |
| 55 | 56 | #include "amdgpu_object.h" |
| @@ -170,10 +171,10 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, · 114 unmodified lines | ||
| 170 | 171 | * @bo: buffer object to map |
| 171 | 172 | * @mem: memory object to map |
| 172 | 173 | * @mm_cur: range to map |
| 173 | − * @num_pages: number of pages to map | |
| 174 | 174 | * @window: which GART window to use |
| 175 | 175 | * @ring: DMA ring to use for the copy |
| 176 | 176 | * @tmz: if we should setup a TMZ enabled mapping |
| 177 | + * @size: in number of bytes to map, out number of bytes mapped | |
| 177 | 178 | * @addr: resulting address inside the MC address space |
| 178 | 179 | * |
| 179 | 180 | * Setup one of the GART windows to access a specific piece of memory or return |
| @@ -182,15 +183,14 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, · 2 unmodified lines | ||
| 182 | 183 | static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, |
| 183 | 184 | struct ttm_resource *mem, |
| 184 | 185 | struct amdgpu_res_cursor *mm_cur, |
| 185 | − unsigned num_pages, unsigned window, | |
| 186 | − struct amdgpu_ring *ring, bool tmz, | |
| 187 | − uint64_t *addr) | |
| 186 | + unsigned window, struct amdgpu_ring *ring, | |
| 187 | + bool tmz, uint64_t *size, uint64_t *addr) | |
| 188 | 188 | { |
| 189 | 189 | struct amdgpu_device *adev = ring->adev; |
| 190 | − struct amdgpu_job *job; | |
| 191 | − unsigned num_dw, num_bytes; | |
| 192 | − struct dma_fence *fence; | |
| 190 | + unsigned offset, num_pages, num_dw, num_bytes; | |
| 193 | 191 | uint64_t src_addr, dst_addr; |
| 192 | + struct dma_fence *fence; | |
| 193 | + struct amdgpu_job *job; | |
| 194 | 194 | void *cpu_addr; |
| 195 | 195 | uint64_t flags; |
| 196 | 196 | unsigned int i; |
| @@ -198,7 +198,9 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 1 unmodified line | ||
| 198 | 198 | |
| 199 | 199 | BUG_ON(adev->mman.buffer_funcs->copy_max_bytes < |
| 200 | 200 | AMDGPU_GTT_MAX_TRANSFER_SIZE * 8); |
| 201 | − BUG_ON(mem->mem_type == AMDGPU_PL_PREEMPT); | |
| 201 | + | |
| 202 | + if (WARN_ON(mem->mem_type == AMDGPU_PL_PREEMPT)) | |
| 203 | + return -EINVAL; | |
| 202 | 204 | |
| 203 | 205 | /* Map only what can't be accessed directly */ |
| 204 | 206 | if (!tmz && mem->start != AMDGPU_BO_INVALID_OFFSET) { |
| @@ -207,10 +209,22 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 2 unmodified lines | ||
| 207 | 209 | return 0; |
| 208 | 210 | } |
| 209 | 211 | |
| 212 | + | |
| 213 | + /* | |
| 214 | + * If start begins at an offset inside the page, then adjust the size | |
| 215 | + * and addr accordingly | |
| 216 | + */ | |
| 217 | + offset = mm_cur->start & ~PAGE_MASK; | |
| 218 | + | |
| 219 | + num_pages = PFN_UP(*size + offset); | |
| 220 | + num_pages = min_t(uint32_t, num_pages, AMDGPU_GTT_MAX_TRANSFER_SIZE); | |
| 221 | + | |
| 222 | + *size = min(*size, (uint64_t)num_pages * PAGE_SIZE - offset); | |
| 223 | + | |
| 210 | 224 | *addr = adev->gmc.gart_start; |
| 211 | 225 | *addr += (u64)window * AMDGPU_GTT_MAX_TRANSFER_SIZE * |
| 212 | 226 | AMDGPU_GPU_PAGE_SIZE; |
| 213 | − *addr += mm_cur->start & ~PAGE_MASK; | |
| 227 | + *addr += offset; | |
| 214 | 228 | |
| 215 | 229 | num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8); |
| 216 | 230 | num_bytes = num_pages * 8 * AMDGPU_GPU_PAGES_IN_CPU_PAGE; |
| @@ -241,10 +255,7 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 24 unmodified lines | ||
| 241 | 255 | dma_addr_t *dma_addr; |
| 242 | 256 | |
| 243 | 257 | dma_addr = &bo->ttm->dma_address[mm_cur->start >> PAGE_SHIFT]; |
| 244 | − r = amdgpu_gart_map(adev, 0, num_pages, dma_addr, flags, | |
| 245 | − cpu_addr); | |
| 246 | − if (r) | |
| 247 | − goto error_free; | |
| 258 | + amdgpu_gart_map(adev, 0, num_pages, dma_addr, flags, cpu_addr); | |
| 248 | 259 | } else { |
| 249 | 260 | dma_addr_t dma_address; |
| 250 | 261 | |
| @@ -252,11 +263,8 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 1 unmodified line | ||
| 252 | 263 | dma_address += adev->vm_manager.vram_base_offset; |
| 253 | 264 | |
| 254 | 265 | for (i = 0; i < num_pages; ++i) { |
| 255 | − r = amdgpu_gart_map(adev, i << PAGE_SHIFT, 1, | |
| 256 | − &dma_address, flags, cpu_addr); | |
| 257 | − if (r) | |
| 258 | − goto error_free; | |
| 259 | − | |
| 266 | + amdgpu_gart_map(adev, i << PAGE_SHIFT, 1, &dma_address, | |
| 267 | + flags, cpu_addr); | |
| 260 | 268 | dma_address += PAGE_SIZE; |
| 261 | 269 | } |
| 262 | 270 | } |
| @@ -297,9 +305,6 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev, · 34 unmodified lines | ||
| 297 | 305 | struct dma_resv *resv, |
| 298 | 306 | struct dma_fence **f) |
| 299 | 307 | { |
| 300 | − const uint32_t GTT_MAX_BYTES = (AMDGPU_GTT_MAX_TRANSFER_SIZE * | |
| 301 | − AMDGPU_GPU_PAGE_SIZE); | |
| 302 | − | |
| 303 | 308 | struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; |
| 304 | 309 | struct amdgpu_res_cursor src_mm, dst_mm; |
| 305 | 310 | struct dma_fence *fence = NULL; |
| @@ -315,29 +320,20 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev, · 9 unmodified lines | ||
| 315 | 320 | |
| 316 | 321 | mutex_lock(&adev->mman.gtt_window_lock); |
| 317 | 322 | while (src_mm.remaining) { |
| 318 | − uint32_t src_page_offset = src_mm.start & ~PAGE_MASK; | |
| 319 | − uint32_t dst_page_offset = dst_mm.start & ~PAGE_MASK; | |
| 323 | + uint64_t from, to, cur_size; | |
| 320 | 324 | struct dma_fence *next; |
| 321 | − uint32_t cur_size; | |
| 322 | − uint64_t from, to; | |
| 323 | 325 | |
| 324 | − /* Copy size cannot exceed GTT_MAX_BYTES. So if src or dst | |
| 325 | − * begins at an offset, then adjust the size accordingly | |
| 326 | − */ | |
| 327 | − cur_size = max(src_page_offset, dst_page_offset); | |
| 328 | − cur_size = min(min3(src_mm.size, dst_mm.size, size), | |
| 329 | − (uint64_t)(GTT_MAX_BYTES - cur_size)); | |
| 326 | + /* Never copy more than 256MiB at once to avoid a timeout */ | |
| 327 | + cur_size = min3(src_mm.size, dst_mm.size, 256ULL << 20); | |
| 330 | 328 | |
| 331 | 329 | /* Map src to window 0 and dst to window 1. */ |
| 332 | 330 | r = amdgpu_ttm_map_buffer(src->bo, src->mem, &src_mm, |
| 333 | − PFN_UP(cur_size + src_page_offset), | |
| 334 | − 0, ring, tmz, &from); | |
| 331 | + 0, ring, tmz, &cur_size, &from); | |
| 335 | 332 | if (r) |
| 336 | 333 | goto error; |
| 337 | 334 | |
| 338 | 335 | r = amdgpu_ttm_map_buffer(dst->bo, dst->mem, &dst_mm, |
| 339 | − PFN_UP(cur_size + dst_page_offset), | |
| 340 | − 1, ring, tmz, &to); | |
| 336 | + 1, ring, tmz, &cur_size, &to); | |
| 341 | 337 | if (r) |
| 342 | 338 | goto error; |
| 343 | 339 | |
| @@ -396,8 +392,7 @@ static int amdgpu_move_blit(struct ttm_buffer_object *bo, · 52 unmodified lines | ||
| 396 | 392 | (abo->flags & AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE)) { |
| 397 | 393 | struct dma_fence *wipe_fence = NULL; |
| 398 | 394 | |
| 399 | − r = amdgpu_fill_buffer(ttm_to_amdgpu_bo(bo), AMDGPU_POISON, | |
| 400 | − NULL, &wipe_fence); | |
| 395 | + r = amdgpu_fill_buffer(abo, AMDGPU_POISON, NULL, &wipe_fence); | |
| 401 | 396 | if (r) { |
| 402 | 397 | goto error; |
| 403 | 398 | } else if (wipe_fence) { |
| @@ -821,14 +816,13 @@ static void amdgpu_ttm_tt_unpin_userptr(struct ttm_device *bdev, · 417 unmodified lines | ||
| 821 | 816 | #endif |
| 822 | 817 | } |
| 823 | 818 | |
| 824 | −static int amdgpu_ttm_gart_bind(struct amdgpu_device *adev, | |
| 825 | − struct ttm_buffer_object *tbo, | |
| 826 | − uint64_t flags) | |
| 819 | +static void amdgpu_ttm_gart_bind(struct amdgpu_device *adev, | |
| 820 | + struct ttm_buffer_object *tbo, | |
| 821 | + uint64_t flags) | |
| 827 | 822 | { |
| 828 | 823 | struct amdgpu_bo *abo = ttm_to_amdgpu_bo(tbo); |
| 829 | 824 | struct ttm_tt *ttm = tbo->ttm; |
| 830 | 825 | struct amdgpu_ttm_tt *gtt = (void *)ttm; |
| 831 | − int r; | |
| 832 | 826 | |
| 833 | 827 | if (amdgpu_bo_encrypted(abo)) |
| 834 | 828 | flags |= AMDGPU_PTE_TMZ; |
| @@ -836,10 +830,8 @@ static int amdgpu_ttm_gart_bind(struct amdgpu_device *adev, · 1 unmodified line | ||
| 836 | 830 | if (abo->flags & AMDGPU_GEM_CREATE_CP_MQD_GFX9) { |
| 837 | 831 | uint64_t page_idx = 1; |
| 838 | 832 | |
| 839 | − r = amdgpu_gart_bind(adev, gtt->offset, page_idx, | |
| 840 | − gtt->ttm.dma_address, flags); | |
| 841 | − if (r) | |
| 842 | − goto gart_bind_fail; | |
| 833 | + amdgpu_gart_bind(adev, gtt->offset, page_idx, | |
| 834 | + gtt->ttm.dma_address, flags); | |
| 843 | 835 | |
| 844 | 836 | /* The memory type of the first page defaults to UC. Now |
| 845 | 837 | * modify the memory type to NC from the second page of |
| @@ -848,21 +840,13 @@ static int amdgpu_ttm_gart_bind(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 848 | 840 | flags &= ~AMDGPU_PTE_MTYPE_VG10_MASK; |
| 849 | 841 | flags |= AMDGPU_PTE_MTYPE_VG10(AMDGPU_MTYPE_NC); |
| 850 | 842 | |
| 851 | − r = amdgpu_gart_bind(adev, | |
| 852 | − gtt->offset + (page_idx << PAGE_SHIFT), | |
| 853 | − ttm->num_pages - page_idx, | |
| 854 | − &(gtt->ttm.dma_address[page_idx]), flags); | |
| 843 | + amdgpu_gart_bind(adev, gtt->offset + (page_idx << PAGE_SHIFT), | |
| 844 | + ttm->num_pages - page_idx, | |
| 845 | + &(gtt->ttm.dma_address[page_idx]), flags); | |
| 855 | 846 | } else { |
| 856 | − r = amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages, | |
| 857 | − gtt->ttm.dma_address, flags); | |
| 847 | + amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages, | |
| 848 | + gtt->ttm.dma_address, flags); | |
| 858 | 849 | } |
| 859 | − | |
| 860 | −gart_bind_fail: | |
| 861 | − if (r) | |
| 862 | − DRM_ERROR("failed to bind %u pages at 0x%08llX\n", | |
| 863 | − ttm->num_pages, gtt->offset); | |
| 864 | − | |
| 865 | − return r; | |
| 866 | 850 | } |
| 867 | 851 | |
| 868 | 852 | /* |
| @@ -878,7 +862,7 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev, · 9 unmodified lines | ||
| 878 | 862 | struct amdgpu_device *adev = amdgpu_ttm_adev(bdev); |
| 879 | 863 | struct amdgpu_ttm_tt *gtt = (void*)ttm; |
| 880 | 864 | uint64_t flags; |
| 881 | − int r = 0; | |
| 865 | + int r; | |
| 882 | 866 | |
| 883 | 867 | if (!bo_mem) |
| 884 | 868 | return -EINVAL; |
| @@ -925,14 +909,10 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev, · 40 unmodified lines | ||
| 925 | 909 | |
| 926 | 910 | /* bind pages into GART page tables */ |
| 927 | 911 | gtt->offset = (u64)bo_mem->start << PAGE_SHIFT; |
| 928 | − r = amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages, | |
| 929 | − gtt->ttm.dma_address, flags); | |
| 930 | − | |
| 931 | − if (r) | |
| 932 | − DRM_ERROR("failed to bind %u pages at 0x%08llX\n", | |
| 933 | − ttm->num_pages, gtt->offset); | |
| 912 | + amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages, | |
| 913 | + gtt->ttm.dma_address, flags); | |
| 934 | 914 | gtt->bound = true; |
| 935 | − return r; | |
| 915 | + return 0; | |
| 936 | 916 | } |
| 937 | 917 | |
| 938 | 918 | /* |
| @@ -982,12 +962,7 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo) · 43 unmodified lines | ||
| 982 | 962 | |
| 983 | 963 | /* Bind pages */ |
| 984 | 964 | gtt->offset = (u64)tmp->start << PAGE_SHIFT; |
| 985 | − r = amdgpu_ttm_gart_bind(adev, bo, flags); | |
| 986 | − if (unlikely(r)) { | |
| 987 | − ttm_resource_free(bo, &tmp); | |
| 988 | − return r; | |
| 989 | − } | |
| 990 | − | |
| 965 | + amdgpu_ttm_gart_bind(adev, bo, flags); | |
| 991 | 966 | amdgpu_gart_invalidate_tlb(adev); |
| 992 | 967 | ttm_resource_free(bo, &bo->resource); |
| 993 | 968 | ttm_bo_assign_mem(bo, tmp); |
| @@ -1001,19 +976,16 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo) · 7 unmodified lines | ||
| 1001 | 976 | * Called by amdgpu_gtt_mgr_recover() from amdgpu_device_reset() to |
| 1002 | 977 | * rebind GTT pages during a GPU reset. |
| 1003 | 978 | */ |
| 1004 | −int amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo) | |
| 979 | +void amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo) | |
| 1005 | 980 | { |
| 1006 | 981 | struct amdgpu_device *adev = amdgpu_ttm_adev(tbo->bdev); |
| 1007 | 982 | uint64_t flags; |
| 1008 | − int r; | |
| 1009 | 983 | |
| 1010 | 984 | if (!tbo->ttm) |
| 1011 | − return 0; | |
| 985 | + return; | |
| 1012 | 986 | |
| 1013 | 987 | flags = amdgpu_ttm_tt_pte_flags(adev, tbo->ttm, tbo->resource); |
| 1014 | − r = amdgpu_ttm_gart_bind(adev, tbo, flags); | |
| 1015 | − | |
| 1016 | − return r; | |
| 988 | + amdgpu_ttm_gart_bind(adev, tbo, flags); | |
| 1017 | 989 | } |
| 1018 | 990 | |
| 1019 | 991 | /* |
| @@ -1027,7 +999,6 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev, · 7 unmodified lines | ||
| 1027 | 999 | { |
| 1028 | 1000 | struct amdgpu_device *adev = amdgpu_ttm_adev(bdev); |
| 1029 | 1001 | struct amdgpu_ttm_tt *gtt = (void *)ttm; |
| 1030 | − int r; | |
| 1031 | 1002 | |
| 1032 | 1003 | /* if the pages have userptr pinning then clear that first */ |
| 1033 | 1004 | if (gtt->userptr) { |
| @@ -1047,10 +1018,7 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev, · 13 unmodified lines | ||
| 1047 | 1018 | return; |
| 1048 | 1019 | |
| 1049 | 1020 | /* unbind shouldn't be done for GDS/GWS/OA in ttm_bo_clean_mm */ |
| 1050 | − r = amdgpu_gart_unbind(adev, gtt->offset, ttm->num_pages); | |
| 1051 | − if (r) | |
| 1052 | − DRM_ERROR("failed to unbind %u pages at 0x%08llX\n", | |
| 1053 | − gtt->ttm.num_pages, gtt->offset); | |
| 1021 | + amdgpu_gart_unbind(adev, gtt->offset, ttm->num_pages); | |
| 1054 | 1022 | gtt->bound = false; |
| 1055 | 1023 | } |
| 1056 | 1024 | |
| @@ -1168,6 +1136,26 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev, · 111 unmodified lines | ||
| 1168 | 1136 | return ttm_pool_free(&adev->mman.bdev.pool, ttm); |
| 1169 | 1137 | } |
| 1170 | 1138 | |
| 1139 | +/** | |
| 1140 | + * amdgpu_ttm_tt_get_userptr - Return the userptr GTT ttm_tt for the current | |
| 1141 | + * task | |
| 1142 | + * | |
| 1143 | + * @tbo: The ttm_buffer_object that contains the userptr | |
| 1144 | + * @user_addr: The returned value | |
| 1145 | + */ | |
| 1146 | +int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo, | |
| 1147 | + uint64_t *user_addr) | |
| 1148 | +{ | |
| 1149 | + struct amdgpu_ttm_tt *gtt; | |
| 1150 | + | |
| 1151 | + if (!tbo->ttm) | |
| 1152 | + return -EINVAL; | |
| 1153 | + | |
| 1154 | + gtt = (void *)tbo->ttm; | |
| 1155 | + *user_addr = gtt->userptr; | |
| 1156 | + return 0; | |
| 1157 | +} | |
| 1158 | + | |
| 1171 | 1159 | /** |
| 1172 | 1160 | * amdgpu_ttm_tt_set_userptr - Initialize userptr GTT ttm_tt for the current |
| 1173 | 1161 | * task |
| @@ -1433,6 +1421,63 @@ static void amdgpu_ttm_vram_mm_access(struct amdgpu_device *adev, loff_t pos, · 259 unmodified lines | ||
| 1433 | 1421 | } |
| 1434 | 1422 | } |
| 1435 | 1423 | |
| 1424 | +static int amdgpu_ttm_access_memory_sdma(struct ttm_buffer_object *bo, | |
| 1425 | + unsigned long offset, void *buf, int len, int write) | |
| 1426 | +{ | |
| 1427 | + struct amdgpu_bo *abo = ttm_to_amdgpu_bo(bo); | |
| 1428 | + struct amdgpu_device *adev = amdgpu_ttm_adev(abo->tbo.bdev); | |
| 1429 | + struct amdgpu_res_cursor src_mm; | |
| 1430 | + struct amdgpu_job *job; | |
| 1431 | + struct dma_fence *fence; | |
| 1432 | + uint64_t src_addr, dst_addr; | |
| 1433 | + unsigned int num_dw; | |
| 1434 | + int r, idx; | |
| 1435 | + | |
| 1436 | + if (len != PAGE_SIZE) | |
| 1437 | + return -EINVAL; | |
| 1438 | + | |
| 1439 | + if (!adev->mman.sdma_access_ptr) | |
| 1440 | + return -EACCES; | |
| 1441 | + | |
| 1442 | + if (!drm_dev_enter(adev_to_drm(adev), &idx)) | |
| 1443 | + return -ENODEV; | |
| 1444 | + | |
| 1445 | + if (write) | |
| 1446 | + memcpy(adev->mman.sdma_access_ptr, buf, len); | |
| 1447 | + | |
| 1448 | + num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8); | |
| 1449 | + r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, AMDGPU_IB_POOL_DELAYED, &job); | |
| 1450 | + if (r) | |
| 1451 | + goto out; | |
| 1452 | + | |
| 1453 | + amdgpu_res_first(abo->tbo.resource, offset, len, &src_mm); | |
| 1454 | + src_addr = amdgpu_ttm_domain_start(adev, bo->resource->mem_type) + src_mm.start; | |
| 1455 | + dst_addr = amdgpu_bo_gpu_offset(adev->mman.sdma_access_bo); | |
| 1456 | + if (write) | |
| 1457 | + swap(src_addr, dst_addr); | |
| 1458 | + | |
| 1459 | + amdgpu_emit_copy_buffer(adev, &job->ibs[0], src_addr, dst_addr, PAGE_SIZE, false); | |
| 1460 | + | |
| 1461 | + amdgpu_ring_pad_ib(adev->mman.buffer_funcs_ring, &job->ibs[0]); | |
| 1462 | + WARN_ON(job->ibs[0].length_dw > num_dw); | |
| 1463 | + | |
| 1464 | + r = amdgpu_job_submit(job, &adev->mman.entity, AMDGPU_FENCE_OWNER_UNDEFINED, &fence); | |
| 1465 | + if (r) { | |
| 1466 | + amdgpu_job_free(job); | |
| 1467 | + goto out; | |
| 1468 | + } | |
| 1469 | + | |
| 1470 | + if (!dma_fence_wait_timeout(fence, false, adev->sdma_timeout)) | |
| 1471 | + r = -ETIMEDOUT; | |
| 1472 | + dma_fence_put(fence); | |
| 1473 | + | |
| 1474 | + if (!(r || write)) | |
| 1475 | + memcpy(buf, adev->mman.sdma_access_ptr, len); | |
| 1476 | +out: | |
| 1477 | + drm_dev_exit(idx); | |
| 1478 | + return r; | |
| 1479 | +} | |
| 1480 | + | |
| 1436 | 1481 | /** |
| 1437 | 1482 | * amdgpu_ttm_access_memory - Read or Write memory that backs a buffer object. |
| 1438 | 1483 | * |
| @@ -1457,6 +1502,10 @@ static int amdgpu_ttm_access_memory(struct ttm_buffer_object *bo, · 18 unmodified lines | ||
| 1457 | 1502 | if (bo->resource->mem_type != TTM_PL_VRAM) |
| 1458 | 1503 | return -EIO; |
| 1459 | 1504 | |
| 1505 | + if (amdgpu_device_has_timeouts_enabled(adev) && | |
| 1506 | + !amdgpu_ttm_access_memory_sdma(bo, offset, buf, len, write)) | |
| 1507 | + return len; | |
| 1508 | + | |
| 1460 | 1509 | amdgpu_res_first(bo->resource, offset, len, &cursor); |
| 1461 | 1510 | while (cursor.remaining) { |
| 1462 | 1511 | size_t count, size = cursor.size; |
| @@ -1797,6 +1846,12 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) · 334 unmodified lines | ||
| 1797 | 1846 | return r; |
| 1798 | 1847 | } |
| 1799 | 1848 | |
| 1849 | + if (amdgpu_bo_create_kernel(adev, PAGE_SIZE, PAGE_SIZE, | |
| 1850 | + AMDGPU_GEM_DOMAIN_GTT, | |
| 1851 | + &adev->mman.sdma_access_bo, NULL, | |
| 1852 | + &adev->mman.sdma_access_ptr)) | |
| 1853 | + DRM_WARN("Debug VRAM access will use slowpath MM access\n"); | |
| 1854 | + | |
| 1800 | 1855 | return 0; |
| 1801 | 1856 | } |
| 1802 | 1857 | |
| @@ -1818,6 +1873,8 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) · 15 unmodified lines | ||
| 1818 | 1873 | if (adev->mman.stolen_reserved_size) |
| 1819 | 1874 | amdgpu_bo_free_kernel(&adev->mman.stolen_reserved_memory, |
| 1820 | 1875 | NULL, NULL); |
| 1876 | + amdgpu_bo_free_kernel(&adev->mman.sdma_access_bo, NULL, | |
| 1877 | + &adev->mman.sdma_access_ptr); | |
| 1821 | 1878 | amdgpu_ttm_fw_reserve_vram_fini(adev); |
| 1822 | 1879 | |
| 1823 | 1880 | if (drm_dev_enter(adev_to_drm(adev), &idx)) { |
| @@ -1884,27 +1941,59 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) · 60 unmodified lines | ||
| 1884 | 1941 | size = adev->gmc.real_vram_size; |
| 1885 | 1942 | else |
| 1886 | 1943 | size = adev->gmc.visible_vram_size; |
| 1887 | − man->size = size >> PAGE_SHIFT; | |
| 1944 | + man->size = size; | |
| 1888 | 1945 | adev->mman.buffer_funcs_enabled = enable; |
| 1889 | 1946 | } |
| 1890 | 1947 | |
| 1948 | +static int amdgpu_ttm_prepare_job(struct amdgpu_device *adev, | |
| 1949 | + bool direct_submit, | |
| 1950 | + unsigned int num_dw, | |
| 1951 | + struct dma_resv *resv, | |
| 1952 | + bool vm_needs_flush, | |
| 1953 | + struct amdgpu_job **job) | |
| 1954 | +{ | |
| 1955 | + enum amdgpu_ib_pool_type pool = direct_submit ? | |
| 1956 | + AMDGPU_IB_POOL_DIRECT : | |
| 1957 | + AMDGPU_IB_POOL_DELAYED; | |
| 1958 | + int r; | |
| 1959 | + | |
| 1960 | + r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, pool, job); | |
| 1961 | + if (r) | |
| 1962 | + return r; | |
| 1963 | + | |
| 1964 | + if (vm_needs_flush) { | |
| 1965 | + (*job)->vm_pd_addr = amdgpu_gmc_pd_addr(adev->gmc.pdb0_bo ? | |
| 1966 | + adev->gmc.pdb0_bo : | |
| 1967 | + adev->gart.bo); | |
| 1968 | + (*job)->vm_needs_flush = true; | |
| 1969 | + } | |
| 1970 | + if (resv) { | |
| 1971 | + r = amdgpu_sync_resv(adev, &(*job)->sync, resv, | |
| 1972 | + AMDGPU_SYNC_ALWAYS, | |
| 1973 | + AMDGPU_FENCE_OWNER_UNDEFINED); | |
| 1974 | + if (r) { | |
| 1975 | + DRM_ERROR("sync failed (%d).\n", r); | |
| 1976 | + amdgpu_job_free(*job); | |
| 1977 | + return r; | |
| 1978 | + } | |
| 1979 | + } | |
| 1980 | + return 0; | |
| 1981 | +} | |
| 1982 | + | |
| 1891 | 1983 | int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset, |
| 1892 | 1984 | uint64_t dst_offset, uint32_t byte_count, |
| 1893 | 1985 | struct dma_resv *resv, |
| 1894 | 1986 | struct dma_fence **fence, bool direct_submit, |
| 1895 | 1987 | bool vm_needs_flush, bool tmz) |
| 1896 | 1988 | { |
| 1897 | − enum amdgpu_ib_pool_type pool = direct_submit ? AMDGPU_IB_POOL_DIRECT : | |
| 1898 | − AMDGPU_IB_POOL_DELAYED; | |
| 1899 | 1989 | struct amdgpu_device *adev = ring->adev; |
| 1990 | + unsigned num_loops, num_dw; | |
| 1900 | 1991 | struct amdgpu_job *job; |
| 1901 | − | |
| 1902 | 1992 | uint32_t max_bytes; |
| 1903 | − unsigned num_loops, num_dw; | |
| 1904 | 1993 | unsigned i; |
| 1905 | 1994 | int r; |
| 1906 | 1995 | |
| 1907 | − if (direct_submit && !ring->sched.ready) { | |
| 1996 | + if (!direct_submit && !ring->sched.ready) { | |
| 1908 | 1997 | DRM_ERROR("Trying to move memory with ring turned off.\n"); |
| 1909 | 1998 | return -EINVAL; |
| 1910 | 1999 | } |
| @@ -1912,26 +2001,11 @@ int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset, · 1 unmodified line | ||
| 1912 | 2001 | max_bytes = adev->mman.buffer_funcs->copy_max_bytes; |
| 1913 | 2002 | num_loops = DIV_ROUND_UP(byte_count, max_bytes); |
| 1914 | 2003 | num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->copy_num_dw, 8); |
| 1915 | − | |
| 1916 | − r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, pool, &job); | |
| 2004 | + r = amdgpu_ttm_prepare_job(adev, direct_submit, num_dw, | |
| 2005 | + resv, vm_needs_flush, &job); | |
| 1917 | 2006 | if (r) |
| 1918 | 2007 | return r; |
| 1919 | 2008 | |
| 1920 | − if (vm_needs_flush) { | |
| 1921 | − job->vm_pd_addr = amdgpu_gmc_pd_addr(adev->gmc.pdb0_bo ? | |
| 1922 | − adev->gmc.pdb0_bo : adev->gart.bo); | |
| 1923 | − job->vm_needs_flush = true; | |
| 1924 | − } | |
| 1925 | − if (resv) { | |
| 1926 | − r = amdgpu_sync_resv(adev, &job->sync, resv, | |
| 1927 | − AMDGPU_SYNC_ALWAYS, | |
| 1928 | − AMDGPU_FENCE_OWNER_UNDEFINED); | |
| 1929 | − if (r) { | |
| 1930 | − DRM_ERROR("sync failed (%d).\n", r); | |
| 1931 | − goto error_free; | |
| 1932 | − } | |
| 1933 | − } | |
| 1934 | − | |
| 1935 | 2009 | for (i = 0; i < num_loops; i++) { |
| 1936 | 2010 | uint32_t cur_size_in_bytes = min(byte_count, max_bytes); |
| 1937 | 2011 | |
| @@ -1961,77 +2035,35 @@ error_free: · 23 unmodified lines | ||
| 1961 | 2035 | return r; |
| 1962 | 2036 | } |
| 1963 | 2037 | |
| 1964 | −int amdgpu_fill_buffer(struct amdgpu_bo *bo, | |
| 1965 | − uint32_t src_data, | |
| 1966 | − struct dma_resv *resv, | |
| 1967 | − struct dma_fence **fence) | |
| 2038 | +static int amdgpu_ttm_fill_mem(struct amdgpu_ring *ring, uint32_t src_data, | |
| 2039 | + uint64_t dst_addr, uint32_t byte_count, | |
| 2040 | + struct dma_resv *resv, | |
| 2041 | + struct dma_fence **fence, | |
| 2042 | + bool vm_needs_flush) | |
| 1968 | 2043 | { |
| 1969 | − struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | |
| 1970 | − uint32_t max_bytes = adev->mman.buffer_funcs->fill_max_bytes; | |
| 1971 | − struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; | |
| 1972 | − | |
| 1973 | − struct amdgpu_res_cursor cursor; | |
| 2044 | + struct amdgpu_device *adev = ring->adev; | |
| 1974 | 2045 | unsigned int num_loops, num_dw; |
| 1975 | − uint64_t num_bytes; | |
| 1976 | − | |
| 1977 | 2046 | struct amdgpu_job *job; |
| 2047 | + uint32_t max_bytes; | |
| 2048 | + unsigned int i; | |
| 1978 | 2049 | int r; |
| 1979 | 2050 | |
| 1980 | − if (!adev->mman.buffer_funcs_enabled) { | |
| 1981 | − DRM_ERROR("Trying to clear memory with ring turned off.\n"); | |
| 1982 | − return -EINVAL; | |
| 1983 | − } | |
| 1984 | − | |
| 1985 | − if (bo->tbo.resource->mem_type == AMDGPU_PL_PREEMPT) { | |
| 1986 | − DRM_ERROR("Trying to clear preemptible memory.\n"); | |
| 1987 | − return -EINVAL; | |
| 1988 | − } | |
| 1989 | − | |
| 1990 | − if (bo->tbo.resource->mem_type == TTM_PL_TT) { | |
| 1991 | − r = amdgpu_ttm_alloc_gart(&bo->tbo); | |
| 1992 | − if (r) | |
| 1993 | − return r; | |
| 1994 | − } | |
| 1995 | − | |
| 1996 | − num_bytes = bo->tbo.resource->num_pages << PAGE_SHIFT; | |
| 1997 | − num_loops = 0; | |
| 1998 | − | |
| 1999 | − amdgpu_res_first(bo->tbo.resource, 0, num_bytes, &cursor); | |
| 2000 | − while (cursor.remaining) { | |
| 2001 | − num_loops += DIV_ROUND_UP_ULL(cursor.size, max_bytes); | |
| 2002 | − amdgpu_res_next(&cursor, cursor.size); | |
| 2003 | − } | |
| 2004 | − num_dw = num_loops * adev->mman.buffer_funcs->fill_num_dw; | |
| 2005 | − | |
| 2006 | − /* for IB padding */ | |
| 2007 | − num_dw += 64; | |
| 2008 | − | |
| 2009 | − r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, AMDGPU_IB_POOL_DELAYED, | |
| 2010 | − &job); | |
| 2051 | + max_bytes = adev->mman.buffer_funcs->fill_max_bytes; | |
| 2052 | + num_loops = DIV_ROUND_UP_ULL(byte_count, max_bytes); | |
| 2053 | + num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->fill_num_dw, 8); | |
| 2054 | + r = amdgpu_ttm_prepare_job(adev, false, num_dw, resv, vm_needs_flush, | |
| 2055 | + &job); | |
| 2011 | 2056 | if (r) |
| 2012 | 2057 | return r; |
| 2013 | 2058 | |
| 2014 | − if (resv) { | |
| 2015 | − r = amdgpu_sync_resv(adev, &job->sync, resv, | |
| 2016 | − AMDGPU_SYNC_ALWAYS, | |
| 2017 | − AMDGPU_FENCE_OWNER_UNDEFINED); | |
| 2018 | − if (r) { | |
| 2019 | − DRM_ERROR("sync failed (%d).\n", r); | |
| 2020 | − goto error_free; | |
| 2021 | − } | |
| 2022 | − } | |
| 2023 | − | |
| 2024 | − amdgpu_res_first(bo->tbo.resource, 0, num_bytes, &cursor); | |
| 2025 | − while (cursor.remaining) { | |
| 2026 | − uint32_t cur_size = min_t(uint64_t, cursor.size, max_bytes); | |
| 2027 | − uint64_t dst_addr = cursor.start; | |
| 2059 | + for (i = 0; i < num_loops; i++) { | |
| 2060 | + uint32_t cur_size = min(byte_count, max_bytes); | |
| 2028 | 2061 | |
| 2029 | − dst_addr += amdgpu_ttm_domain_start(adev, | |
| 2030 | − bo->tbo.resource->mem_type); | |
| 2031 | 2062 | amdgpu_emit_fill_buffer(adev, &job->ibs[0], src_data, dst_addr, |
| 2032 | 2063 | cur_size); |
| 2033 | 2064 | |
| 2034 | − amdgpu_res_next(&cursor, cur_size); | |
| 2065 | + dst_addr += cur_size; | |
| 2066 | + byte_count -= cur_size; | |
| 2035 | 2067 | } |
| 2036 | 2068 | |
| 2037 | 2069 | amdgpu_ring_pad_ib(ring, &job->ibs[0]); |
| @@ -2048,6 +2080,55 @@ error_free: · 10 unmodified lines | ||
| 2048 | 2080 | return r; |
| 2049 | 2081 | } |
| 2050 | 2082 | |
| 2083 | +int amdgpu_fill_buffer(struct amdgpu_bo *bo, | |
| 2084 | + uint32_t src_data, | |
| 2085 | + struct dma_resv *resv, | |
| 2086 | + struct dma_fence **f) | |
| 2087 | +{ | |
| 2088 | + struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | |
| 2089 | + struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; | |
| 2090 | + struct dma_fence *fence = NULL; | |
| 2091 | + struct amdgpu_res_cursor dst; | |
| 2092 | + int r; | |
| 2093 | + | |
| 2094 | + if (!adev->mman.buffer_funcs_enabled) { | |
| 2095 | + DRM_ERROR("Trying to clear memory with ring turned off.\n"); | |
| 2096 | + return -EINVAL; | |
| 2097 | + } | |
| 2098 | + | |
| 2099 | + amdgpu_res_first(bo->tbo.resource, 0, amdgpu_bo_size(bo), &dst); | |
| 2100 | + | |
| 2101 | + mutex_lock(&adev->mman.gtt_window_lock); | |
| 2102 | + while (dst.remaining) { | |
| 2103 | + struct dma_fence *next; | |
| 2104 | + uint64_t cur_size, to; | |
| 2105 | + | |
| 2106 | + /* Never fill more than 256MiB at once to avoid timeouts */ | |
| 2107 | + cur_size = min(dst.size, 256ULL << 20); | |
| 2108 | + | |
| 2109 | + r = amdgpu_ttm_map_buffer(&bo->tbo, bo->tbo.resource, &dst, | |
| 2110 | + 1, ring, false, &cur_size, &to); | |
| 2111 | + if (r) | |
| 2112 | + goto error; | |
| 2113 | + | |
| 2114 | + r = amdgpu_ttm_fill_mem(ring, src_data, to, cur_size, resv, | |
| 2115 | + &next, true); | |
| 2116 | + if (r) | |
| 2117 | + goto error; | |
| 2118 | + | |
| 2119 | + dma_fence_put(fence); | |
| 2120 | + fence = next; | |
| 2121 | + | |
| 2122 | + amdgpu_res_next(&dst, cur_size); | |
| 2123 | + } | |
| 2124 | +error: | |
| 2125 | + mutex_unlock(&adev->mman.gtt_window_lock); | |
| 2126 | + if (f) | |
| 2127 | + *f = dma_fence_get(fence); | |
| 2128 | + dma_fence_put(fence); | |
| 2129 | + return r; | |
| 2130 | +} | |
| 2131 | + | |
| 2051 | 2132 | /** |
| 2052 | 2133 | * amdgpu_ttm_evict_resources - evict memory buffers |
| 2053 | 2134 | * @adev: amdgpu device object |
| @@ -44,7 +44,6 @@ struct amdgpu_vram_mgr { · 43 unmodified lines | ||
| 44 | 44 | spinlock_t lock; |
| 45 | 45 | struct list_head reservations_pending; |
| 46 | 46 | struct list_head reserved_pages; |
| 47 | − atomic64_t usage; | |
| 48 | 47 | atomic64_t vis_usage; |
| 49 | 48 | }; |
| 50 | 49 | |
| @@ -52,12 +51,6 @@ struct amdgpu_gtt_mgr { · 1 unmodified line | ||
| 52 | 51 | struct ttm_resource_manager manager; |
| 53 | 52 | struct drm_mm mm; |
| 54 | 53 | spinlock_t lock; |
| 55 | − atomic64_t used; | |
| 56 | −}; | |
| 57 | − | |
| 58 | −struct amdgpu_preempt_mgr { | |
| 59 | − struct ttm_resource_manager manager; | |
| 60 | − atomic64_t used; | |
| 61 | 54 | }; |
| 62 | 55 | |
| 63 | 56 | struct amdgpu_mman { |
| @@ -76,7 +69,7 @@ struct amdgpu_mman { · 12 unmodified lines | ||
| 76 | 69 | |
| 77 | 70 | struct amdgpu_vram_mgr vram_mgr; |
| 78 | 71 | struct amdgpu_gtt_mgr gtt_mgr; |
| 79 | − struct amdgpu_preempt_mgr preempt_mgr; | |
| 72 | + struct ttm_resource_manager preempt_mgr; | |
| 80 | 73 | |
| 81 | 74 | uint64_t stolen_vga_size; |
| 82 | 75 | struct amdgpu_bo *stolen_vga_memory; |
| @@ -98,6 +91,10 @@ struct amdgpu_mman { · 15 unmodified lines | ||
| 98 | 91 | u64 fw_vram_usage_size; |
| 99 | 92 | struct amdgpu_bo *fw_vram_usage_reserved_bo; |
| 100 | 93 | void *fw_vram_usage_va; |
| 94 | + | |
| 95 | + /* PAGE_SIZE'd BO for process memory r/w over SDMA. */ | |
| 96 | + struct amdgpu_bo *sdma_access_bo; | |
| 97 | + void *sdma_access_ptr; | |
| 101 | 98 | }; |
| 102 | 99 | |
| 103 | 100 | struct amdgpu_copy_mem { |
| @@ -114,8 +111,7 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev); · 10 unmodified lines | ||
| 114 | 111 | void amdgpu_vram_mgr_fini(struct amdgpu_device *adev); |
| 115 | 112 | |
| 116 | 113 | bool amdgpu_gtt_mgr_has_gart_addr(struct ttm_resource *mem); |
| 117 | −uint64_t amdgpu_gtt_mgr_usage(struct amdgpu_gtt_mgr *mgr); | |
| 118 | −int amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr); | |
| 114 | +void amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr); | |
| 119 | 115 | |
| 120 | 116 | uint64_t amdgpu_preempt_mgr_usage(struct ttm_resource_manager *man); |
| 121 | 117 | |
| @@ -129,7 +125,6 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev, · 7 unmodified lines | ||
| 129 | 125 | void amdgpu_vram_mgr_free_sgt(struct device *dev, |
| 130 | 126 | enum dma_data_direction dir, |
| 131 | 127 | struct sg_table *sgt); |
| 132 | −uint64_t amdgpu_vram_mgr_usage(struct amdgpu_vram_mgr *mgr); | |
| 133 | 128 | uint64_t amdgpu_vram_mgr_vis_usage(struct amdgpu_vram_mgr *mgr); |
| 134 | 129 | int amdgpu_vram_mgr_reserve_range(struct amdgpu_vram_mgr *mgr, |
| 135 | 130 | uint64_t start, uint64_t size); |
| @@ -158,7 +153,7 @@ int amdgpu_fill_buffer(struct amdgpu_bo *bo, · 22 unmodified lines | ||
| 158 | 153 | struct dma_fence **fence); |
| 159 | 154 | |
| 160 | 155 | int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo); |
| 161 | −int amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo); | |
| 156 | +void amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo); | |
| 162 | 157 | uint64_t amdgpu_ttm_domain_start(struct amdgpu_device *adev, uint32_t type); |
| 163 | 158 | |
| 164 | 159 | #if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR) |
| @@ -177,6 +172,8 @@ static inline bool amdgpu_ttm_tt_get_user_pages_done(struct ttm_tt *ttm) · 12 unmodified lines | ||
| 177 | 172 | #endif |
| 178 | 173 | |
| 179 | 174 | void amdgpu_ttm_tt_set_user_pages(struct ttm_tt *ttm, struct page **pages); |
| 175 | +int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo, | |
| 176 | + uint64_t *user_addr); | |
| 180 | 177 | int amdgpu_ttm_tt_set_userptr(struct ttm_buffer_object *bo, |
| 181 | 178 | uint64_t addr, uint32_t flags); |
| 182 | 179 | bool amdgpu_ttm_tt_has_userptr(struct ttm_tt *ttm); |
| @@ -378,6 +378,7 @@ enum AMDGPU_UCODE_ID { · 377 unmodified lines | ||
| 378 | 378 | AMDGPU_UCODE_ID_VCN0_RAM, |
| 379 | 379 | AMDGPU_UCODE_ID_VCN1_RAM, |
| 380 | 380 | AMDGPU_UCODE_ID_DMCUB, |
| 381 | + AMDGPU_UCODE_ID_CAP, | |
| 381 | 382 | AMDGPU_UCODE_ID_MAXIMUM, |
| 382 | 383 | }; |
| 383 | 384 | |
| @@ -21,7 +21,7 @@ · 20 unmodified lines | ||
| 21 | 21 | * |
| 22 | 22 | */ |
| 23 | 23 | |
| 24 | −#include "amdgpu_ras.h" | |
| 24 | +#include "amdgpu.h" | |
| 25 | 25 | |
| 26 | 26 | static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, |
| 27 | 27 | void *ras_error_status, |
| @@ -33,14 +33,14 @@ static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, · 5 unmodified lines | ||
| 33 | 33 | int ret = 0; |
| 34 | 34 | |
| 35 | 35 | kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); |
| 36 | − ret = smu_get_ecc_info(&adev->smu, (void *)&(con->umc_ecc)); | |
| 36 | + ret = amdgpu_dpm_get_ecc_info(adev, (void *)&(con->umc_ecc)); | |
| 37 | 37 | if (ret == -EOPNOTSUPP) { |
| 38 | − if (adev->umc.ras_funcs && | |
| 39 | − adev->umc.ras_funcs->query_ras_error_count) | |
| 40 | − adev->umc.ras_funcs->query_ras_error_count(adev, ras_error_status); | |
| 38 | + if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && | |
| 39 | + adev->umc.ras->ras_block.hw_ops->query_ras_error_count) | |
| 40 | + adev->umc.ras->ras_block.hw_ops->query_ras_error_count(adev, ras_error_status); | |
| 41 | 41 | |
| 42 | − if (adev->umc.ras_funcs && | |
| 43 | − adev->umc.ras_funcs->query_ras_error_address && | |
| 42 | + if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && | |
| 43 | + adev->umc.ras->ras_block.hw_ops->query_ras_error_address && | |
| 44 | 44 | adev->umc.max_ras_err_cnt_per_query) { |
| 45 | 45 | err_data->err_addr = |
| 46 | 46 | kcalloc(adev->umc.max_ras_err_cnt_per_query, |
| @@ -56,15 +56,15 @@ static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, · 9 unmodified lines | ||
| 56 | 56 | /* umc query_ras_error_address is also responsible for clearing |
| 57 | 57 | * error status |
| 58 | 58 | */ |
| 59 | − adev->umc.ras_funcs->query_ras_error_address(adev, ras_error_status); | |
| 59 | + adev->umc.ras->ras_block.hw_ops->query_ras_error_address(adev, ras_error_status); | |
| 60 | 60 | } |
| 61 | 61 | } else if (!ret) { |
| 62 | − if (adev->umc.ras_funcs && | |
| 63 | − adev->umc.ras_funcs->ecc_info_query_ras_error_count) | |
| 64 | − adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, ras_error_status); | |
| 62 | + if (adev->umc.ras && | |
| 63 | + adev->umc.ras->ecc_info_query_ras_error_count) | |
| 64 | + adev->umc.ras->ecc_info_query_ras_error_count(adev, ras_error_status); | |
| 65 | 65 | |
| 66 | − if (adev->umc.ras_funcs && | |
| 67 | − adev->umc.ras_funcs->ecc_info_query_ras_error_address && | |
| 66 | + if (adev->umc.ras && | |
| 67 | + adev->umc.ras->ecc_info_query_ras_error_address && | |
| 68 | 68 | adev->umc.max_ras_err_cnt_per_query) { |
| 69 | 69 | err_data->err_addr = |
| 70 | 70 | kcalloc(adev->umc.max_ras_err_cnt_per_query, |
| @@ -80,7 +80,7 @@ static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, · 9 unmodified lines | ||
| 80 | 80 | /* umc query_ras_error_address is also responsible for clearing |
| 81 | 81 | * error status |
| 82 | 82 | */ |
| 83 | − adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, ras_error_status); | |
| 83 | + adev->umc.ras->ecc_info_query_ras_error_address(adev, ras_error_status); | |
| 84 | 84 | } |
| 85 | 85 | } |
| 86 | 86 | |
| @@ -96,8 +96,7 @@ static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, · 9 unmodified lines | ||
| 96 | 96 | err_data->err_addr_cnt); |
| 97 | 97 | amdgpu_ras_save_bad_pages(adev); |
| 98 | 98 | |
| 99 | − if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num) | |
| 100 | − adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs); | |
| 99 | + amdgpu_dpm_send_hbm_bad_pages_num(adev, con->eeprom_control.ras_num_recs); | |
| 101 | 100 | } |
| 102 | 101 | |
| 103 | 102 | if (reset) |
| @@ -130,76 +129,44 @@ int amdgpu_umc_poison_handler(struct amdgpu_device *adev, · 26 unmodified lines | ||
| 130 | 129 | return ret; |
| 131 | 130 | } |
| 132 | 131 | |
| 133 | −static int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | |
| 132 | +int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | |
| 134 | 133 | void *ras_error_status, |
| 135 | 134 | struct amdgpu_iv_entry *entry) |
| 136 | 135 | { |
| 137 | 136 | return amdgpu_umc_do_page_retirement(adev, ras_error_status, entry, true); |
| 138 | 137 | } |
| 139 | 138 | |
| 140 | −int amdgpu_umc_ras_late_init(struct amdgpu_device *adev) | |
| 139 | +int amdgpu_umc_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block) | |
| 141 | 140 | { |
| 142 | 141 | int r; |
| 143 | − struct ras_fs_if fs_info = { | |
| 144 | − .sysfs_name = "umc_err_count", | |
| 145 | − }; | |
| 146 | − struct ras_ih_if ih_info = { | |
| 147 | − .cb = amdgpu_umc_process_ras_data_cb, | |
| 148 | − }; | |
| 149 | − | |
| 150 | − if (!adev->umc.ras_if) { | |
| 151 | − adev->umc.ras_if = | |
| 152 | − kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 153 | − if (!adev->umc.ras_if) | |
| 154 | − return -ENOMEM; | |
| 155 | − adev->umc.ras_if->block = AMDGPU_RAS_BLOCK__UMC; | |
| 156 | − adev->umc.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 157 | − adev->umc.ras_if->sub_block_index = 0; | |
| 158 | − } | |
| 159 | − ih_info.head = fs_info.head = *adev->umc.ras_if; | |
| 160 | 142 | |
| 161 | − r = amdgpu_ras_late_init(adev, adev->umc.ras_if, | |
| 162 | − &fs_info, &ih_info); | |
| 143 | + r = amdgpu_ras_block_late_init(adev, ras_block); | |
| 163 | 144 | if (r) |
| 164 | − goto free; | |
| 145 | + return r; | |
| 165 | 146 | |
| 166 | − if (amdgpu_ras_is_supported(adev, adev->umc.ras_if->block)) { | |
| 147 | + if (amdgpu_ras_is_supported(adev, ras_block->block)) { | |
| 167 | 148 | r = amdgpu_irq_get(adev, &adev->gmc.ecc_irq, 0); |
| 168 | 149 | if (r) |
| 169 | 150 | goto late_fini; |
| 170 | − } else { | |
| 171 | − r = 0; | |
| 172 | − goto free; | |
| 173 | 151 | } |
| 174 | 152 | |
| 175 | 153 | /* ras init of specific umc version */ |
| 176 | − if (adev->umc.ras_funcs && | |
| 177 | − adev->umc.ras_funcs->err_cnt_init) | |
| 178 | − adev->umc.ras_funcs->err_cnt_init(adev); | |
| 154 | + if (adev->umc.ras && | |
| 155 | + adev->umc.ras->err_cnt_init) | |
| 156 | + adev->umc.ras->err_cnt_init(adev); | |
| 179 | 157 | |
| 180 | 158 | return 0; |
| 181 | 159 | |
| 182 | 160 | late_fini: |
| 183 | − amdgpu_ras_late_fini(adev, adev->umc.ras_if, &ih_info); | |
| 184 | −free: | |
| 185 | − kfree(adev->umc.ras_if); | |
| 186 | − adev->umc.ras_if = NULL; | |
| 161 | + amdgpu_ras_block_late_fini(adev, ras_block); | |
| 187 | 162 | return r; |
| 188 | 163 | } |
| 189 | 164 | |
| 190 | 165 | void amdgpu_umc_ras_fini(struct amdgpu_device *adev) |
| 191 | 166 | { |
| 192 | 167 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__UMC) && |
| 193 | − adev->umc.ras_if) { | |
| 194 | − struct ras_common_if *ras_if = adev->umc.ras_if; | |
| 195 | − struct ras_ih_if ih_info = { | |
| 196 | − .head = *ras_if, | |
| 197 | − .cb = amdgpu_umc_process_ras_data_cb, | |
| 198 | − }; | |
| 199 | − | |
| 200 | − amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 201 | − kfree(ras_if); | |
| 202 | − } | |
| 168 | + adev->umc.ras_if) | |
| 169 | + amdgpu_ras_block_late_fini(adev, adev->umc.ras_if); | |
| 203 | 170 | } |
| 204 | 171 | |
| 205 | 172 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, |
| @@ -219,3 +186,24 @@ int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, · 13 unmodified lines | ||
| 219 | 186 | amdgpu_ras_interrupt_dispatch(adev, &ih_data); |
| 220 | 187 | return 0; |
| 221 | 188 | } |
| 189 | + | |
| 190 | +void amdgpu_umc_fill_error_record(struct ras_err_data *err_data, | |
| 191 | + uint64_t err_addr, | |
| 192 | + uint64_t retired_page, | |
| 193 | + uint32_t channel_index, | |
| 194 | + uint32_t umc_inst) | |
| 195 | +{ | |
| 196 | + struct eeprom_table_record *err_rec = | |
| 197 | + &err_data->err_addr[err_data->err_addr_cnt]; | |
| 198 | + | |
| 199 | + err_rec->address = err_addr; | |
| 200 | + /* page frame address is saved */ | |
| 201 | + err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT; | |
| 202 | + err_rec->ts = (uint64_t)ktime_get_real_seconds(); | |
| 203 | + err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE; | |
| 204 | + err_rec->cu = 0; | |
| 205 | + err_rec->mem_channel = channel_index; | |
| 206 | + err_rec->mcumc_id = umc_inst; | |
| 207 | + | |
| 208 | + err_data->err_addr_cnt++; | |
| 209 | +} | |
| @@ -20,6 +20,7 @@ · 19 unmodified lines | ||
| 20 | 20 | */ |
| 21 | 21 | #ifndef __AMDGPU_UMC_H__ |
| 22 | 22 | #define __AMDGPU_UMC_H__ |
| 23 | +#include "amdgpu_ras.h" | |
| 23 | 24 | |
| 24 | 25 | /* |
| 25 | 26 | * (addr / 256) * 4096, the higher 26 bits in ErrorAddr |
| @@ -40,14 +41,9 @@ · 14 unmodified lines | ||
| 40 | 41 | #define LOOP_UMC_CH_INST(ch_inst) for ((ch_inst) = 0; (ch_inst) < adev->umc.channel_inst_num; (ch_inst)++) |
| 41 | 42 | #define LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) LOOP_UMC_INST((umc_inst)) LOOP_UMC_CH_INST((ch_inst)) |
| 42 | 43 | |
| 43 | −struct amdgpu_umc_ras_funcs { | |
| 44 | +struct amdgpu_umc_ras { | |
| 45 | + struct amdgpu_ras_block_object ras_block; | |
| 44 | 46 | void (*err_cnt_init)(struct amdgpu_device *adev); |
| 45 | − int (*ras_late_init)(struct amdgpu_device *adev); | |
| 46 | − void (*ras_fini)(struct amdgpu_device *adev); | |
| 47 | − void (*query_ras_error_count)(struct amdgpu_device *adev, | |
| 48 | − void *ras_error_status); | |
| 49 | − void (*query_ras_error_address)(struct amdgpu_device *adev, | |
| 50 | − void *ras_error_status); | |
| 51 | 47 | bool (*query_ras_poison_mode)(struct amdgpu_device *adev); |
| 52 | 48 | void (*ecc_info_query_ras_error_count)(struct amdgpu_device *adev, |
| 53 | 49 | void *ras_error_status); |
| @@ -73,10 +69,10 @@ struct amdgpu_umc { · 19 unmodified lines | ||
| 73 | 69 | struct ras_common_if *ras_if; |
| 74 | 70 | |
| 75 | 71 | const struct amdgpu_umc_funcs *funcs; |
| 76 | − const struct amdgpu_umc_ras_funcs *ras_funcs; | |
| 72 | + struct amdgpu_umc_ras *ras; | |
| 77 | 73 | }; |
| 78 | 74 | |
| 79 | −int amdgpu_umc_ras_late_init(struct amdgpu_device *adev); | |
| 75 | +int amdgpu_umc_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block); | |
| 80 | 76 | void amdgpu_umc_ras_fini(struct amdgpu_device *adev); |
| 81 | 77 | int amdgpu_umc_poison_handler(struct amdgpu_device *adev, |
| 82 | 78 | void *ras_error_status, |
| @@ -84,4 +80,13 @@ int amdgpu_umc_poison_handler(struct amdgpu_device *adev, · 1 unmodified line | ||
| 84 | 80 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, |
| 85 | 81 | struct amdgpu_irq_src *source, |
| 86 | 82 | struct amdgpu_iv_entry *entry); |
| 83 | +void amdgpu_umc_fill_error_record(struct ras_err_data *err_data, | |
| 84 | + uint64_t err_addr, | |
| 85 | + uint64_t retired_page, | |
| 86 | + uint32_t channel_index, | |
| 87 | + uint32_t umc_inst); | |
| 88 | + | |
| 89 | +int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | |
| 90 | + void *ras_error_status, | |
| 91 | + struct amdgpu_iv_entry *entry); | |
| 87 | 92 | #endif |
| @@ -834,6 +834,7 @@ static int amdgpu_uvd_cs_msg(struct amdgpu_uvd_cs_ctx *ctx, · 833 unmodified lines | ||
| 834 | 834 | handle = msg[2]; |
| 835 | 835 | |
| 836 | 836 | if (handle == 0) { |
| 837 | + amdgpu_bo_kunmap(bo); | |
| 837 | 838 | DRM_ERROR("Invalid UVD handle!\n"); |
| 838 | 839 | return -EINVAL; |
| 839 | 840 | } |
| @@ -892,6 +893,7 @@ static int amdgpu_uvd_cs_msg(struct amdgpu_uvd_cs_ctx *ctx, · 52 unmodified lines | ||
| 892 | 893 | DRM_ERROR("Illegal UVD message type (%d)!\n", msg_type); |
| 893 | 894 | } |
| 894 | 895 | |
| 896 | + amdgpu_bo_kunmap(bo); | |
| 895 | 897 | return -EINVAL; |
| 896 | 898 | } |
| 897 | 899 | |
| @@ -51,6 +51,7 @@ · 50 unmodified lines | ||
| 51 | 51 | #define FIRMWARE_ALDEBARAN "amdgpu/aldebaran_vcn.bin" |
| 52 | 52 | #define FIRMWARE_BEIGE_GOBY "amdgpu/beige_goby_vcn.bin" |
| 53 | 53 | #define FIRMWARE_YELLOW_CARP "amdgpu/yellow_carp_vcn.bin" |
| 54 | +#define FIRMWARE_VCN_3_1_2 "amdgpu/vcn_3_1_2_vcn.bin" | |
| 54 | 55 | |
| 55 | 56 | MODULE_FIRMWARE(FIRMWARE_RAVEN); |
| 56 | 57 | MODULE_FIRMWARE(FIRMWARE_PICASSO); |
| @@ -68,6 +69,7 @@ MODULE_FIRMWARE(FIRMWARE_VANGOGH); · 11 unmodified lines | ||
| 68 | 69 | MODULE_FIRMWARE(FIRMWARE_DIMGREY_CAVEFISH); |
| 69 | 70 | MODULE_FIRMWARE(FIRMWARE_BEIGE_GOBY); |
| 70 | 71 | MODULE_FIRMWARE(FIRMWARE_YELLOW_CARP); |
| 72 | +MODULE_FIRMWARE(FIRMWARE_VCN_3_1_2); | |
| 71 | 73 | |
| 72 | 74 | static void amdgpu_vcn_idle_work_handler(struct work_struct *work); |
| 73 | 75 | |
| @@ -165,6 +167,12 @@ int amdgpu_vcn_sw_init(struct amdgpu_device *adev) · 91 unmodified lines | ||
| 165 | 167 | (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG)) |
| 166 | 168 | adev->vcn.indirect_sram = true; |
| 167 | 169 | break; |
| 170 | + case IP_VERSION(3, 1, 2): | |
| 171 | + fw_name = FIRMWARE_VCN_3_1_2; | |
| 172 | + if ((adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) && | |
| 173 | + (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG)) | |
| 174 | + adev->vcn.indirect_sram = true; | |
| 175 | + break; | |
| 168 | 176 | default: |
| 169 | 177 | return -EINVAL; |
| 170 | 178 | } |
| @@ -575,8 +575,10 @@ static int amdgpu_virt_write_vf2pf_data(struct amdgpu_device *adev) · 574 unmodified lines | ||
| 575 | 575 | vf2pf_info->driver_cert = 0; |
| 576 | 576 | vf2pf_info->os_info.all = 0; |
| 577 | 577 | |
| 578 | − vf2pf_info->fb_usage = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr) >> 20; | |
| 579 | − vf2pf_info->fb_vis_usage = amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr) >> 20; | |
| 578 | + vf2pf_info->fb_usage = | |
| 579 | + ttm_resource_manager_usage(&adev->mman.vram_mgr.manager) >> 20; | |
| 580 | + vf2pf_info->fb_vis_usage = | |
| 581 | + amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr) >> 20; | |
| 580 | 582 | vf2pf_info->fb_size = adev->gmc.real_vram_size >> 20; |
| 581 | 583 | vf2pf_info->fb_vis_size = adev->gmc.visible_vram_size >> 20; |
| 582 | 584 | |
| @@ -820,3 +822,148 @@ void amdgpu_virt_update_sriov_video_codec(struct amdgpu_device *adev, · 237 unmodified lines | ||
| 820 | 822 | } |
| 821 | 823 | } |
| 822 | 824 | } |
| 825 | + | |
| 826 | +static bool amdgpu_virt_get_rlcg_reg_access_flag(struct amdgpu_device *adev, | |
| 827 | + u32 acc_flags, u32 hwip, | |
| 828 | + bool write, u32 *rlcg_flag) | |
| 829 | +{ | |
| 830 | + bool ret = false; | |
| 831 | + | |
| 832 | + switch (hwip) { | |
| 833 | + case GC_HWIP: | |
| 834 | + if (amdgpu_sriov_reg_indirect_gc(adev)) { | |
| 835 | + *rlcg_flag = | |
| 836 | + write ? AMDGPU_RLCG_GC_WRITE : AMDGPU_RLCG_GC_READ; | |
| 837 | + ret = true; | |
| 838 | + /* only in new version, AMDGPU_REGS_NO_KIQ and | |
| 839 | + * AMDGPU_REGS_RLC are enabled simultaneously */ | |
| 840 | + } else if ((acc_flags & AMDGPU_REGS_RLC) && | |
| 841 | + !(acc_flags & AMDGPU_REGS_NO_KIQ) && write) { | |
| 842 | + *rlcg_flag = AMDGPU_RLCG_GC_WRITE_LEGACY; | |
| 843 | + ret = true; | |
| 844 | + } | |
| 845 | + break; | |
| 846 | + case MMHUB_HWIP: | |
| 847 | + if (amdgpu_sriov_reg_indirect_mmhub(adev) && | |
| 848 | + (acc_flags & AMDGPU_REGS_RLC) && write) { | |
| 849 | + *rlcg_flag = AMDGPU_RLCG_MMHUB_WRITE; | |
| 850 | + ret = true; | |
| 851 | + } | |
| 852 | + break; | |
| 853 | + default: | |
| 854 | + break; | |
| 855 | + } | |
| 856 | + return ret; | |
| 857 | +} | |
| 858 | + | |
| 859 | +static u32 amdgpu_virt_rlcg_reg_rw(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag) | |
| 860 | +{ | |
| 861 | + struct amdgpu_rlcg_reg_access_ctrl *reg_access_ctrl; | |
| 862 | + uint32_t timeout = 50000; | |
| 863 | + uint32_t i, tmp; | |
| 864 | + uint32_t ret = 0; | |
| 865 | + static void *scratch_reg0; | |
| 866 | + static void *scratch_reg1; | |
| 867 | + static void *scratch_reg2; | |
| 868 | + static void *scratch_reg3; | |
| 869 | + static void *spare_int; | |
| 870 | + | |
| 871 | + if (!adev->gfx.rlc.rlcg_reg_access_supported) { | |
| 872 | + dev_err(adev->dev, | |
| 873 | + "indirect registers access through rlcg is not available\n"); | |
| 874 | + return 0; | |
| 875 | + } | |
| 876 | + | |
| 877 | + reg_access_ctrl = &adev->gfx.rlc.reg_access_ctrl; | |
| 878 | + scratch_reg0 = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->scratch_reg0; | |
| 879 | + scratch_reg1 = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->scratch_reg1; | |
| 880 | + scratch_reg2 = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->scratch_reg2; | |
| 881 | + scratch_reg3 = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->scratch_reg3; | |
| 882 | + if (reg_access_ctrl->spare_int) | |
| 883 | + spare_int = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->spare_int; | |
| 884 | + | |
| 885 | + if (offset == reg_access_ctrl->grbm_cntl) { | |
| 886 | + /* if the target reg offset is grbm_cntl, write to scratch_reg2 */ | |
| 887 | + writel(v, scratch_reg2); | |
| 888 | + writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); | |
| 889 | + } else if (offset == reg_access_ctrl->grbm_idx) { | |
| 890 | + /* if the target reg offset is grbm_idx, write to scratch_reg3 */ | |
| 891 | + writel(v, scratch_reg3); | |
| 892 | + writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); | |
| 893 | + } else { | |
| 894 | + /* | |
| 895 | + * SCRATCH_REG0 = read/write value | |
| 896 | + * SCRATCH_REG1[30:28] = command | |
| 897 | + * SCRATCH_REG1[19:0] = address in dword | |
| 898 | + * SCRATCH_REG1[26:24] = Error reporting | |
| 899 | + */ | |
| 900 | + writel(v, scratch_reg0); | |
| 901 | + writel((offset | flag), scratch_reg1); | |
| 902 | + if (reg_access_ctrl->spare_int) | |
| 903 | + writel(1, spare_int); | |
| 904 | + | |
| 905 | + for (i = 0; i < timeout; i++) { | |
| 906 | + tmp = readl(scratch_reg1); | |
| 907 | + if (!(tmp & AMDGPU_RLCG_SCRATCH1_ADDRESS_MASK)) | |
| 908 | + break; | |
| 909 | + udelay(10); | |
| 910 | + } | |
| 911 | + | |
| 912 | + if (i >= timeout) { | |
| 913 | + if (amdgpu_sriov_rlcg_error_report_enabled(adev)) { | |
| 914 | + if (tmp & AMDGPU_RLCG_VFGATE_DISABLED) { | |
| 915 | + dev_err(adev->dev, | |
| 916 | + "vfgate is disabled, rlcg failed to program reg: 0x%05x\n", offset); | |
| 917 | + } else if (tmp & AMDGPU_RLCG_WRONG_OPERATION_TYPE) { | |
| 918 | + dev_err(adev->dev, | |
| 919 | + "wrong operation type, rlcg failed to program reg: 0x%05x\n", offset); | |
| 920 | + } else if (tmp & AMDGPU_RLCG_REG_NOT_IN_RANGE) { | |
| 921 | + dev_err(adev->dev, | |
| 922 | + "regiser is not in range, rlcg failed to program reg: 0x%05x\n", offset); | |
| 923 | + } else { | |
| 924 | + dev_err(adev->dev, | |
| 925 | + "unknown error type, rlcg failed to program reg: 0x%05x\n", offset); | |
| 926 | + } | |
| 927 | + } else { | |
| 928 | + dev_err(adev->dev, | |
| 929 | + "timeout: rlcg faled to program reg: 0x%05x\n", offset); | |
| 930 | + } | |
| 931 | + } | |
| 932 | + } | |
| 933 | + | |
| 934 | + ret = readl(scratch_reg0); | |
| 935 | + return ret; | |
| 936 | +} | |
| 937 | + | |
| 938 | +void amdgpu_sriov_wreg(struct amdgpu_device *adev, | |
| 939 | + u32 offset, u32 value, | |
| 940 | + u32 acc_flags, u32 hwip) | |
| 941 | +{ | |
| 942 | + u32 rlcg_flag; | |
| 943 | + | |
| 944 | + if (!amdgpu_sriov_runtime(adev) && | |
| 945 | + amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, true, &rlcg_flag)) { | |
| 946 | + amdgpu_virt_rlcg_reg_rw(adev, offset, value, rlcg_flag); | |
| 947 | + return; | |
| 948 | + } | |
| 949 | + | |
| 950 | + if (acc_flags & AMDGPU_REGS_NO_KIQ) | |
| 951 | + WREG32_NO_KIQ(offset, value); | |
| 952 | + else | |
| 953 | + WREG32(offset, value); | |
| 954 | +} | |
| 955 | + | |
| 956 | +u32 amdgpu_sriov_rreg(struct amdgpu_device *adev, | |
| 957 | + u32 offset, u32 acc_flags, u32 hwip) | |
| 958 | +{ | |
| 959 | + u32 rlcg_flag; | |
| 960 | + | |
| 961 | + if (!amdgpu_sriov_runtime(adev) && | |
| 962 | + amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag)) | |
| 963 | + return amdgpu_virt_rlcg_reg_rw(adev, offset, 0, rlcg_flag); | |
| 964 | + | |
| 965 | + if (acc_flags & AMDGPU_REGS_NO_KIQ) | |
| 966 | + return RREG32_NO_KIQ(offset); | |
| 967 | + else | |
| 968 | + return RREG32(offset); | |
| 969 | +} | |
| @@ -32,6 +32,19 @@ · 31 unmodified lines | ||
| 32 | 32 | #define AMDGPU_PASSTHROUGH_MODE (1 << 3) /* thw whole GPU is pass through for VM */ |
| 33 | 33 | #define AMDGPU_SRIOV_CAPS_RUNTIME (1 << 4) /* is out of full access mode */ |
| 34 | 34 | |
| 35 | +/* flags for indirect register access path supported by rlcg for sriov */ | |
| 36 | +#define AMDGPU_RLCG_GC_WRITE_LEGACY (0x8 << 28) | |
| 37 | +#define AMDGPU_RLCG_GC_WRITE (0x0 << 28) | |
| 38 | +#define AMDGPU_RLCG_GC_READ (0x1 << 28) | |
| 39 | +#define AMDGPU_RLCG_MMHUB_WRITE (0x2 << 28) | |
| 40 | + | |
| 41 | +/* error code for indirect register access path supported by rlcg for sriov */ | |
| 42 | +#define AMDGPU_RLCG_VFGATE_DISABLED 0x4000000 | |
| 43 | +#define AMDGPU_RLCG_WRONG_OPERATION_TYPE 0x2000000 | |
| 44 | +#define AMDGPU_RLCG_REG_NOT_IN_RANGE 0x1000000 | |
| 45 | + | |
| 46 | +#define AMDGPU_RLCG_SCRATCH1_ADDRESS_MASK 0xFFFFF | |
| 47 | + | |
| 35 | 48 | /* all asic after AI use this offset */ |
| 36 | 49 | #define mmRCC_IOV_FUNC_IDENTIFIER 0xDE5 |
| 37 | 50 | /* tonga/fiji use this offset */ |
| @@ -275,13 +288,18 @@ struct amdgpu_video_codec_info; · 237 unmodified lines | ||
| 275 | 288 | (amdgpu_sriov_vf((adev)) && \ |
| 276 | 289 | ((adev)->virt.reg_access & (AMDGIM_FEATURE_GC_REG_RLC_EN))) |
| 277 | 290 | |
| 291 | +#define amdgpu_sriov_rlcg_error_report_enabled(adev) \ | |
| 292 | + (amdgpu_sriov_reg_indirect_mmhub(adev) || amdgpu_sriov_reg_indirect_gc(adev)) | |
| 293 | + | |
| 278 | 294 | #define amdgpu_passthrough(adev) \ |
| 279 | 295 | ((adev)->virt.caps & AMDGPU_PASSTHROUGH_MODE) |
| 280 | 296 | |
| 281 | 297 | static inline bool is_virtual_machine(void) |
| 282 | 298 | { |
| 283 | −#ifdef CONFIG_X86 | |
| 299 | +#if defined(CONFIG_X86) | |
| 284 | 300 | return boot_cpu_has(X86_FEATURE_HYPERVISOR); |
| 301 | +#elif defined(CONFIG_ARM64) | |
| 302 | + return !is_kernel_in_hyp_mode(); | |
| 285 | 303 | #else |
| 286 | 304 | return false; |
| 287 | 305 | #endif |
| @@ -293,7 +311,6 @@ static inline bool is_virtual_machine(void) · 5 unmodified lines | ||
| 293 | 311 | ((!amdgpu_in_reset(adev)) && adev->virt.tdr_debug) |
| 294 | 312 | #define amdgpu_sriov_is_normal(adev) \ |
| 295 | 313 | ((!amdgpu_in_reset(adev)) && (!adev->virt.tdr_debug)) |
| 296 | − | |
| 297 | 314 | bool amdgpu_virt_mmio_blocked(struct amdgpu_device *adev); |
| 298 | 315 | void amdgpu_virt_init_setting(struct amdgpu_device *adev); |
| 299 | 316 | void amdgpu_virt_kiq_reg_write_reg_wait(struct amdgpu_device *adev, |
| @@ -321,4 +338,9 @@ enum amdgpu_sriov_vf_mode amdgpu_virt_get_sriov_vf_mode(struct amdgpu_device *ad · 21 unmodified lines | ||
| 321 | 338 | void amdgpu_virt_update_sriov_video_codec(struct amdgpu_device *adev, |
| 322 | 339 | struct amdgpu_video_codec_info *encode, uint32_t encode_array_size, |
| 323 | 340 | struct amdgpu_video_codec_info *decode, uint32_t decode_array_size); |
| 341 | +void amdgpu_sriov_wreg(struct amdgpu_device *adev, | |
| 342 | + u32 offset, u32 value, | |
| 343 | + u32 acc_flags, u32 hwip); | |
| 344 | +u32 amdgpu_sriov_rreg(struct amdgpu_device *adev, | |
| 345 | + u32 offset, u32 acc_flags, u32 hwip); | |
| 324 | 346 | #endif |
| @@ -391,7 +391,6 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev, · 390 unmodified lines | ||
| 391 | 391 | int index) |
| 392 | 392 | { |
| 393 | 393 | struct drm_plane *plane; |
| 394 | − uint64_t modifiers[] = {DRM_FORMAT_MOD_LINEAR, DRM_FORMAT_MOD_INVALID}; | |
| 395 | 394 | int ret; |
| 396 | 395 | |
| 397 | 396 | plane = kzalloc(sizeof(*plane), GFP_KERNEL); |
| @@ -402,7 +401,7 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev, · 4 unmodified lines | ||
| 402 | 401 | &amdgpu_vkms_plane_funcs, |
| 403 | 402 | amdgpu_vkms_formats, |
| 404 | 403 | ARRAY_SIZE(amdgpu_vkms_formats), |
| 405 | − modifiers, type, NULL); | |
| 404 | + NULL, type, NULL); | |
| 406 | 405 | if (ret) { |
| 407 | 406 | kfree(plane); |
| 408 | 407 | return ERR_PTR(ret); |
| @@ -375,6 +375,8 @@ static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, · 374 unmodified lines | ||
| 375 | 375 | if (bo->tbo.base.resv != vm->root.bo->tbo.base.resv) |
| 376 | 376 | return; |
| 377 | 377 | |
| 378 | + dma_resv_assert_held(vm->root.bo->tbo.base.resv); | |
| 379 | + | |
| 378 | 380 | vm->bulk_moveable = false; |
| 379 | 381 | if (bo->tbo.type == ttm_bo_type_kernel && bo->parent) |
| 380 | 382 | amdgpu_vm_bo_relocated(base); |
| @@ -768,11 +770,16 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 387 unmodified lines | ||
| 768 | 770 | * Check if all VM PDs/PTs are ready for updates |
| 769 | 771 | * |
| 770 | 772 | * Returns: |
| 771 | − * True if eviction list is empty. | |
| 773 | + * True if VM is not evicting. | |
| 772 | 774 | */ |
| 773 | 775 | bool amdgpu_vm_ready(struct amdgpu_vm *vm) |
| 774 | 776 | { |
| 775 | − return list_empty(&vm->evicted); | |
| 777 | + bool ret; | |
| 778 | + | |
| 779 | + amdgpu_vm_eviction_lock(vm); | |
| 780 | + ret = !vm->evicting; | |
| 781 | + amdgpu_vm_eviction_unlock(vm); | |
| 782 | + return ret; | |
| 776 | 783 | } |
| 777 | 784 | |
| 778 | 785 | /** |
| @@ -1634,7 +1641,7 @@ static int amdgpu_vm_update_ptes(struct amdgpu_vm_update_params *params, · 855 unmodified lines | ||
| 1634 | 1641 | nptes = max(nptes, 1u); |
| 1635 | 1642 | |
| 1636 | 1643 | trace_amdgpu_vm_update_ptes(params, frag_start, upd_end, |
| 1637 | − nptes, dst, incr, upd_flags, | |
| 1644 | + min(nptes, 32u), dst, incr, upd_flags, | |
| 1638 | 1645 | vm->task_info.pid, |
| 1639 | 1646 | vm->immediate.fence_context); |
| 1640 | 1647 | amdgpu_vm_update_flags(params, to_amdgpu_bo_vm(pt), |
| @@ -2257,6 +2264,7 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev, · 616 unmodified lines | ||
| 2257 | 2264 | if (!bo) |
| 2258 | 2265 | return bo_va; |
| 2259 | 2266 | |
| 2267 | + dma_resv_assert_held(bo->tbo.base.resv); | |
| 2260 | 2268 | if (amdgpu_dmabuf_is_xgmi_accessible(adev, bo)) { |
| 2261 | 2269 | bo_va->is_xgmi = true; |
| 2262 | 2270 | /* Power up XGMI if it can be potentially used */ |
| @@ -2634,7 +2642,7 @@ void amdgpu_vm_bo_trace_cs(struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket) · 371 unmodified lines | ||
| 2634 | 2642 | } |
| 2635 | 2643 | |
| 2636 | 2644 | /** |
| 2637 | − * amdgpu_vm_bo_rmv - remove a bo to a specific vm | |
| 2645 | + * amdgpu_vm_bo_del - remove a bo from a specific vm | |
| 2638 | 2646 | * |
| 2639 | 2647 | * @adev: amdgpu_device pointer |
| 2640 | 2648 | * @bo_va: requested bo_va |
| @@ -2643,7 +2651,7 @@ void amdgpu_vm_bo_trace_cs(struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket) · 2 unmodified lines | ||
| 2643 | 2651 | * |
| 2644 | 2652 | * Object have to be reserved! |
| 2645 | 2653 | */ |
| 2646 | −void amdgpu_vm_bo_rmv(struct amdgpu_device *adev, | |
| 2654 | +void amdgpu_vm_bo_del(struct amdgpu_device *adev, | |
| 2647 | 2655 | struct amdgpu_bo_va *bo_va) |
| 2648 | 2656 | { |
| 2649 | 2657 | struct amdgpu_bo_va_mapping *mapping, *next; |
| @@ -2651,7 +2659,10 @@ void amdgpu_vm_bo_rmv(struct amdgpu_device *adev, · 1 unmodified line | ||
| 2651 | 2659 | struct amdgpu_vm *vm = bo_va->base.vm; |
| 2652 | 2660 | struct amdgpu_vm_bo_base **base; |
| 2653 | 2661 | |
| 2662 | + dma_resv_assert_held(vm->root.bo->tbo.base.resv); | |
| 2663 | + | |
| 2654 | 2664 | if (bo) { |
| 2665 | + dma_resv_assert_held(bo->tbo.base.resv); | |
| 2655 | 2666 | if (bo->tbo.base.resv == vm->root.bo->tbo.base.resv) |
| 2656 | 2667 | vm->bulk_moveable = false; |
| 2657 | 2668 | |
| @@ -435,7 +435,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev, · 434 unmodified lines | ||
| 435 | 435 | struct amdgpu_bo_va_mapping *amdgpu_vm_bo_lookup_mapping(struct amdgpu_vm *vm, |
| 436 | 436 | uint64_t addr); |
| 437 | 437 | void amdgpu_vm_bo_trace_cs(struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket); |
| 438 | −void amdgpu_vm_bo_rmv(struct amdgpu_device *adev, | |
| 438 | +void amdgpu_vm_bo_del(struct amdgpu_device *adev, | |
| 439 | 439 | struct amdgpu_bo_va *bo_va); |
| 440 | 440 | void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint32_t min_vm_size, |
| 441 | 441 | uint32_t fragment_size_default, unsigned max_level, |
| @@ -96,9 +96,9 @@ static ssize_t amdgpu_mem_info_vram_used_show(struct device *dev, · 95 unmodified lines | ||
| 96 | 96 | { |
| 97 | 97 | struct drm_device *ddev = dev_get_drvdata(dev); |
| 98 | 98 | struct amdgpu_device *adev = drm_to_adev(ddev); |
| 99 | + struct ttm_resource_manager *man = &adev->mman.vram_mgr.manager; | |
| 99 | 100 | |
| 100 | − return sysfs_emit(buf, "%llu\n", | |
| 101 | − amdgpu_vram_mgr_usage(&adev->mman.vram_mgr)); | |
| 101 | + return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man)); | |
| 102 | 102 | } |
| 103 | 103 | |
| 104 | 104 | /** |
| @@ -253,7 +253,9 @@ static void amdgpu_vram_mgr_do_reserve(struct ttm_resource_manager *man) · 148 unmodified lines | ||
| 253 | 253 | |
| 254 | 254 | vis_usage = amdgpu_vram_mgr_vis_size(adev, &rsv->mm_node); |
| 255 | 255 | atomic64_add(vis_usage, &mgr->vis_usage); |
| 256 | − atomic64_add(rsv->mm_node.size << PAGE_SHIFT, &mgr->usage); | |
| 256 | + spin_lock(&man->bdev->lru_lock); | |
| 257 | + man->usage += rsv->mm_node.size << PAGE_SHIFT; | |
| 258 | + spin_unlock(&man->bdev->lru_lock); | |
| 257 | 259 | list_move(&rsv->node, &mgr->reserved_pages); |
| 258 | 260 | } |
| 259 | 261 | } |
| @@ -281,7 +283,7 @@ int amdgpu_vram_mgr_reserve_range(struct amdgpu_vram_mgr *mgr, · 21 unmodified lines | ||
| 281 | 283 | rsv->mm_node.size = size >> PAGE_SHIFT; |
| 282 | 284 | |
| 283 | 285 | spin_lock(&mgr->lock); |
| 284 | − list_add_tail(&mgr->reservations_pending, &rsv->node); | |
| 286 | + list_add_tail(&rsv->node, &mgr->reservations_pending); | |
| 285 | 287 | amdgpu_vram_mgr_do_reserve(&mgr->manager); |
| 286 | 288 | spin_unlock(&mgr->lock); |
| 287 | 289 | |
| @@ -378,19 +380,13 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, · 90 unmodified lines | ||
| 378 | 380 | |
| 379 | 381 | lpfn = place->lpfn; |
| 380 | 382 | if (!lpfn) |
| 381 | − lpfn = man->size; | |
| 383 | + lpfn = man->size >> PAGE_SHIFT; | |
| 382 | 384 | |
| 383 | 385 | max_bytes = adev->gmc.mc_vram_size; |
| 384 | 386 | if (tbo->type != ttm_bo_type_kernel) |
| 385 | 387 | max_bytes -= AMDGPU_VM_RESERVED_VRAM; |
| 386 | 388 | |
| 387 | − /* bail out quickly if there's likely not enough VRAM for this BO */ | |
| 388 | 389 | mem_bytes = tbo->base.size; |
| 389 | − if (atomic64_add_return(mem_bytes, &mgr->usage) > max_bytes) { | |
| 390 | − r = -ENOSPC; | |
| 391 | − goto error_sub; | |
| 392 | − } | |
| 393 | − | |
| 394 | 390 | if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { |
| 395 | 391 | pages_per_node = ~0ul; |
| 396 | 392 | num_nodes = 1; |
| @@ -408,13 +404,17 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, · 11 unmodified lines | ||
| 408 | 404 | |
| 409 | 405 | node = kvmalloc(struct_size(node, mm_nodes, num_nodes), |
| 410 | 406 | GFP_KERNEL | __GFP_ZERO); |
| 411 | − if (!node) { | |
| 412 | − r = -ENOMEM; | |
| 413 | − goto error_sub; | |
| 414 | − } | |
| 407 | + if (!node) | |
| 408 | + return -ENOMEM; | |
| 415 | 409 | |
| 416 | 410 | ttm_resource_init(tbo, place, &node->base); |
| 417 | 411 | |
| 412 | + /* bail out quickly if there's likely not enough VRAM for this BO */ | |
| 413 | + if (ttm_resource_manager_usage(man) > max_bytes) { | |
| 414 | + r = -ENOSPC; | |
| 415 | + goto error_fini; | |
| 416 | + } | |
| 417 | + | |
| 418 | 418 | mode = DRM_MM_INSERT_BEST; |
| 419 | 419 | if (place->flags & TTM_PL_FLAG_TOPDOWN) |
| 420 | 420 | mode = DRM_MM_INSERT_HIGH; |
| @@ -472,11 +472,10 @@ error_free: · 51 unmodified lines | ||
| 472 | 472 | while (i--) |
| 473 | 473 | drm_mm_remove_node(&node->mm_nodes[i]); |
| 474 | 474 | spin_unlock(&mgr->lock); |
| 475 | +error_fini: | |
| 475 | 476 | ttm_resource_fini(man, &node->base); |
| 476 | 477 | kvfree(node); |
| 477 | 478 | |
| 478 | −error_sub: | |
| 479 | − atomic64_sub(mem_bytes, &mgr->usage); | |
| 480 | 479 | return r; |
| 481 | 480 | } |
| 482 | 481 | |
| @@ -494,7 +493,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, · 11 unmodified lines | ||
| 494 | 493 | struct ttm_range_mgr_node *node = to_ttm_range_mgr_node(res); |
| 495 | 494 | struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); |
| 496 | 495 | struct amdgpu_device *adev = to_amdgpu_device(mgr); |
| 497 | − uint64_t usage = 0, vis_usage = 0; | |
| 496 | + uint64_t vis_usage = 0; | |
| 498 | 497 | unsigned i, pages; |
| 499 | 498 | |
| 500 | 499 | spin_lock(&mgr->lock); |
| @@ -503,13 +502,11 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, · 2 unmodified lines | ||
| 503 | 502 | struct drm_mm_node *mm = &node->mm_nodes[i]; |
| 504 | 503 | |
| 505 | 504 | drm_mm_remove_node(mm); |
| 506 | − usage += mm->size << PAGE_SHIFT; | |
| 507 | 505 | vis_usage += amdgpu_vram_mgr_vis_size(adev, mm); |
| 508 | 506 | } |
| 509 | 507 | amdgpu_vram_mgr_do_reserve(man); |
| 510 | 508 | spin_unlock(&mgr->lock); |
| 511 | 509 | |
| 512 | − atomic64_sub(usage, &mgr->usage); | |
| 513 | 510 | atomic64_sub(vis_usage, &mgr->vis_usage); |
| 514 | 511 | |
| 515 | 512 | ttm_resource_fini(man, res); |
| @@ -627,18 +624,6 @@ void amdgpu_vram_mgr_free_sgt(struct device *dev, · 111 unmodified lines | ||
| 627 | 624 | kfree(sgt); |
| 628 | 625 | } |
| 629 | 626 | |
| 630 | −/** | |
| 631 | − * amdgpu_vram_mgr_usage - how many bytes are used in this domain | |
| 632 | − * | |
| 633 | − * @mgr: amdgpu_vram_mgr pointer | |
| 634 | − * | |
| 635 | − * Returns how many bytes are used in this domain. | |
| 636 | − */ | |
| 637 | −uint64_t amdgpu_vram_mgr_usage(struct amdgpu_vram_mgr *mgr) | |
| 638 | −{ | |
| 639 | − return atomic64_read(&mgr->usage); | |
| 640 | −} | |
| 641 | − | |
| 642 | 627 | /** |
| 643 | 628 | * amdgpu_vram_mgr_vis_usage - how many bytes are used in the visible part |
| 644 | 629 | * |
| @@ -664,13 +649,12 @@ static void amdgpu_vram_mgr_debug(struct ttm_resource_manager *man, · 19 unmodified lines | ||
| 664 | 649 | { |
| 665 | 650 | struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); |
| 666 | 651 | |
| 652 | + drm_printf(printer, " vis usage:%llu\n", | |
| 653 | + amdgpu_vram_mgr_vis_usage(mgr)); | |
| 654 | + | |
| 667 | 655 | spin_lock(&mgr->lock); |
| 668 | 656 | drm_mm_print(&mgr->mm, printer); |
| 669 | 657 | spin_unlock(&mgr->lock); |
| 670 | − | |
| 671 | − drm_printf(printer, "man size:%llu pages, ram usage:%lluMB, vis usage:%lluMB\n", | |
| 672 | − man->size, amdgpu_vram_mgr_usage(mgr) >> 20, | |
| 673 | − amdgpu_vram_mgr_vis_usage(mgr) >> 20); | |
| 674 | 658 | } |
| 675 | 659 | |
| 676 | 660 | static const struct ttm_resource_manager_func amdgpu_vram_mgr_func = { |
| @@ -692,11 +676,11 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) · 15 unmodified lines | ||
| 692 | 676 | struct ttm_resource_manager *man = &mgr->manager; |
| 693 | 677 | |
| 694 | 678 | ttm_resource_manager_init(man, &adev->mman.bdev, |
| 695 | − adev->gmc.real_vram_size >> PAGE_SHIFT); | |
| 679 | + adev->gmc.real_vram_size); | |
| 696 | 680 | |
| 697 | 681 | man->func = &amdgpu_vram_mgr_func; |
| 698 | 682 | |
| 699 | − drm_mm_init(&mgr->mm, 0, man->size); | |
| 683 | + drm_mm_init(&mgr->mm, 0, man->size >> PAGE_SHIFT); | |
| 700 | 684 | spin_lock_init(&mgr->lock); |
| 701 | 685 | INIT_LIST_HEAD(&mgr->reservations_pending); |
| 702 | 686 | INIT_LIST_HEAD(&mgr->reserved_pages); |
| @@ -32,6 +32,8 @@ · 31 unmodified lines | ||
| 32 | 32 | #include "wafl/wafl2_4_0_0_smn.h" |
| 33 | 33 | #include "wafl/wafl2_4_0_0_sh_mask.h" |
| 34 | 34 | |
| 35 | +#include "amdgpu_reset.h" | |
| 36 | + | |
| 35 | 37 | #define smnPCS_XGMI23_PCS_ERROR_STATUS 0x11a01210 |
| 36 | 38 | #define smnPCS_XGMI3X16_PCS_ERROR_STATUS 0x11a0020c |
| 37 | 39 | #define smnPCS_GOPX1_PCS_ERROR_STATUS 0x12200210 |
| @@ -227,6 +229,9 @@ static void amdgpu_xgmi_hive_release(struct kobject *kobj) · 189 unmodified lines | ||
| 227 | 229 | struct amdgpu_hive_info *hive = container_of( |
| 228 | 230 | kobj, struct amdgpu_hive_info, kobj); |
| 229 | 231 | |
| 232 | + amdgpu_reset_put_reset_domain(hive->reset_domain); | |
| 233 | + hive->reset_domain = NULL; | |
| 234 | + | |
| 230 | 235 | mutex_destroy(&hive->hive_lock); |
| 231 | 236 | kfree(hive); |
| 232 | 237 | } |
| @@ -398,15 +403,35 @@ struct amdgpu_hive_info *amdgpu_get_xgmi_hive(struct amdgpu_device *adev) · 165 unmodified lines | ||
| 398 | 403 | goto pro_end; |
| 399 | 404 | } |
| 400 | 405 | |
| 406 | + /** | |
| 407 | + * Avoid recreating reset domain when hive is reconstructed for the case | |
| 408 | + * of reset the devices in the XGMI hive during probe for SRIOV | |
| 409 | + * See https://www.spinics.net/lists/amd-gfx/msg58836.html | |
| 410 | + */ | |
| 411 | + if (adev->reset_domain->type != XGMI_HIVE) { | |
| 412 | + hive->reset_domain = amdgpu_reset_create_reset_domain(XGMI_HIVE, "amdgpu-reset-hive"); | |
| 413 | + if (!hive->reset_domain) { | |
| 414 | + dev_err(adev->dev, "XGMI: failed initializing reset domain for xgmi hive\n"); | |
| 415 | + ret = -ENOMEM; | |
| 416 | + kobject_put(&hive->kobj); | |
| 417 | + kfree(hive); | |
| 418 | + hive = NULL; | |
| 419 | + goto pro_end; | |
| 420 | + } | |
| 421 | + } else { | |
| 422 | + amdgpu_reset_get_reset_domain(adev->reset_domain); | |
| 423 | + hive->reset_domain = adev->reset_domain; | |
| 424 | + } | |
| 425 | + | |
| 401 | 426 | hive->hive_id = adev->gmc.xgmi.hive_id; |
| 402 | 427 | INIT_LIST_HEAD(&hive->device_list); |
| 403 | 428 | INIT_LIST_HEAD(&hive->node); |
| 404 | 429 | mutex_init(&hive->hive_lock); |
| 405 | − atomic_set(&hive->in_reset, 0); | |
| 406 | 430 | atomic_set(&hive->number_devices, 0); |
| 407 | 431 | task_barrier_init(&hive->tb); |
| 408 | 432 | hive->pstate = AMDGPU_XGMI_PSTATE_UNKNOWN; |
| 409 | 433 | hive->hi_req_gpu = NULL; |
| 434 | + | |
| 410 | 435 | /* |
| 411 | 436 | * hive pstate on boot is high in vega20 so we have to go to low |
| 412 | 437 | * pstate on after boot. |
| @@ -732,53 +757,22 @@ int amdgpu_xgmi_remove_device(struct amdgpu_device *adev) · 319 unmodified lines | ||
| 732 | 757 | return psp_xgmi_terminate(&adev->psp); |
| 733 | 758 | } |
| 734 | 759 | |
| 735 | −static int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev) | |
| 760 | +static int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block) | |
| 736 | 761 | { |
| 737 | − int r; | |
| 738 | − struct ras_ih_if ih_info = { | |
| 739 | − .cb = NULL, | |
| 740 | − }; | |
| 741 | − struct ras_fs_if fs_info = { | |
| 742 | − .sysfs_name = "xgmi_wafl_err_count", | |
| 743 | − }; | |
| 744 | − | |
| 745 | 762 | if (!adev->gmc.xgmi.supported || |
| 746 | 763 | adev->gmc.xgmi.num_physical_nodes == 0) |
| 747 | 764 | return 0; |
| 748 | 765 | |
| 749 | − adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev); | |
| 750 | − | |
| 751 | − if (!adev->gmc.xgmi.ras_if) { | |
| 752 | − adev->gmc.xgmi.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | |
| 753 | − if (!adev->gmc.xgmi.ras_if) | |
| 754 | − return -ENOMEM; | |
| 755 | − adev->gmc.xgmi.ras_if->block = AMDGPU_RAS_BLOCK__XGMI_WAFL; | |
| 756 | − adev->gmc.xgmi.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 757 | − adev->gmc.xgmi.ras_if->sub_block_index = 0; | |
| 758 | − } | |
| 759 | − ih_info.head = fs_info.head = *adev->gmc.xgmi.ras_if; | |
| 760 | − r = amdgpu_ras_late_init(adev, adev->gmc.xgmi.ras_if, | |
| 761 | − &fs_info, &ih_info); | |
| 762 | − if (r || !amdgpu_ras_is_supported(adev, adev->gmc.xgmi.ras_if->block)) { | |
| 763 | − kfree(adev->gmc.xgmi.ras_if); | |
| 764 | − adev->gmc.xgmi.ras_if = NULL; | |
| 765 | − } | |
| 766 | + adev->gmc.xgmi.ras->ras_block.hw_ops->reset_ras_error_count(adev); | |
| 766 | 767 | |
| 767 | − return r; | |
| 768 | + return amdgpu_ras_block_late_init(adev, ras_block); | |
| 768 | 769 | } |
| 769 | 770 | |
| 770 | 771 | static void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev) |
| 771 | 772 | { |
| 772 | 773 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL) && |
| 773 | − adev->gmc.xgmi.ras_if) { | |
| 774 | − struct ras_common_if *ras_if = adev->gmc.xgmi.ras_if; | |
| 775 | − struct ras_ih_if ih_info = { | |
| 776 | − .cb = NULL, | |
| 777 | − }; | |
| 778 | − | |
| 779 | − amdgpu_ras_late_fini(adev, ras_if, &ih_info); | |
| 780 | − kfree(ras_if); | |
| 781 | − } | |
| 774 | + adev->gmc.xgmi.ras_if) | |
| 775 | + amdgpu_ras_block_late_fini(adev, adev->gmc.xgmi.ras_if); | |
| 782 | 776 | } |
| 783 | 777 | |
| 784 | 778 | uint64_t amdgpu_xgmi_get_relative_phy_addr(struct amdgpu_device *adev, |
| @@ -865,7 +859,7 @@ static int amdgpu_xgmi_query_pcs_error_status(struct amdgpu_device *adev, · 80 unmodified lines | ||
| 865 | 859 | return 0; |
| 866 | 860 | } |
| 867 | 861 | |
| 868 | −static int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev, | |
| 862 | +static void amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev, | |
| 869 | 863 | void *ras_error_status) |
| 870 | 864 | { |
| 871 | 865 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| @@ -874,7 +868,7 @@ static int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 874 | 868 | uint32_t ue_cnt = 0, ce_cnt = 0; |
| 875 | 869 | |
| 876 | 870 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL)) |
| 877 | − return -EINVAL; | |
| 871 | + return ; | |
| 878 | 872 | |
| 879 | 873 | err_data->ue_count = 0; |
| 880 | 874 | err_data->ce_count = 0; |
| @@ -940,17 +934,54 @@ static int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev, · 59 unmodified lines | ||
| 940 | 934 | break; |
| 941 | 935 | } |
| 942 | 936 | |
| 943 | − adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev); | |
| 937 | + adev->gmc.xgmi.ras->ras_block.hw_ops->reset_ras_error_count(adev); | |
| 944 | 938 | |
| 945 | 939 | err_data->ue_count += ue_cnt; |
| 946 | 940 | err_data->ce_count += ce_cnt; |
| 941 | +} | |
| 947 | 942 | |
| 948 | − return 0; | |
| 943 | +/* Trigger XGMI/WAFL error */ | |
| 944 | +static int amdgpu_ras_error_inject_xgmi(struct amdgpu_device *adev, void *inject_if) | |
| 945 | +{ | |
| 946 | + int ret = 0; | |
| 947 | + struct ta_ras_trigger_error_input *block_info = | |
| 948 | + (struct ta_ras_trigger_error_input *)inject_if; | |
| 949 | + | |
| 950 | + if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_DISALLOW)) | |
| 951 | + dev_warn(adev->dev, "Failed to disallow df cstate"); | |
| 952 | + | |
| 953 | + if (amdgpu_dpm_allow_xgmi_power_down(adev, false)) | |
| 954 | + dev_warn(adev->dev, "Failed to disallow XGMI power down"); | |
| 955 | + | |
| 956 | + ret = psp_ras_trigger_error(&adev->psp, block_info); | |
| 957 | + | |
| 958 | + if (amdgpu_ras_intr_triggered()) | |
| 959 | + return ret; | |
| 960 | + | |
| 961 | + if (amdgpu_dpm_allow_xgmi_power_down(adev, true)) | |
| 962 | + dev_warn(adev->dev, "Failed to allow XGMI power down"); | |
| 963 | + | |
| 964 | + if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_ALLOW)) | |
| 965 | + dev_warn(adev->dev, "Failed to allow df cstate"); | |
| 966 | + | |
| 967 | + return ret; | |
| 949 | 968 | } |
| 950 | 969 | |
| 951 | −const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs = { | |
| 952 | − .ras_late_init = amdgpu_xgmi_ras_late_init, | |
| 953 | − .ras_fini = amdgpu_xgmi_ras_fini, | |
| 970 | +struct amdgpu_ras_block_hw_ops xgmi_ras_hw_ops = { | |
| 954 | 971 | .query_ras_error_count = amdgpu_xgmi_query_ras_error_count, |
| 955 | 972 | .reset_ras_error_count = amdgpu_xgmi_reset_ras_error_count, |
| 973 | + .ras_error_inject = amdgpu_ras_error_inject_xgmi, | |
| 974 | +}; | |
| 975 | + | |
| 976 | +struct amdgpu_xgmi_ras xgmi_ras = { | |
| 977 | + .ras_block = { | |
| 978 | + .ras_comm = { | |
| 979 | + .name = "xgmi_wafl", | |
| 980 | + .block = AMDGPU_RAS_BLOCK__XGMI_WAFL, | |
| 981 | + .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | |
| 982 | + }, | |
| 983 | + .hw_ops = &xgmi_ras_hw_ops, | |
| 984 | + .ras_late_init = amdgpu_xgmi_ras_late_init, | |
| 985 | + .ras_fini = amdgpu_xgmi_ras_fini, | |
| 986 | + }, | |
| 956 | 987 | }; |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | ||
| 24 | 24 | |
| 25 | 25 | #include <drm/task_barrier.h> |
| 26 | 26 | #include "amdgpu_psp.h" |
| 27 | − | |
| 27 | +#include "amdgpu_ras.h" | |
| 28 | 28 | |
| 29 | 29 | struct amdgpu_hive_info { |
| 30 | 30 | struct kobject kobj; |
| @@ -33,7 +33,6 @@ struct amdgpu_hive_info { · 2 unmodified lines | ||
| 33 | 33 | struct list_head node; |
| 34 | 34 | atomic_t number_devices; |
| 35 | 35 | struct mutex hive_lock; |
| 36 | − atomic_t in_reset; | |
| 37 | 36 | int hi_req_count; |
| 38 | 37 | struct amdgpu_device *hi_req_gpu; |
| 39 | 38 | struct task_barrier tb; |
| @@ -42,6 +41,8 @@ struct amdgpu_hive_info { · 2 unmodified lines | ||
| 42 | 41 | AMDGPU_XGMI_PSTATE_MAX_VEGA20, |
| 43 | 42 | AMDGPU_XGMI_PSTATE_UNKNOWN |
| 44 | 43 | } pstate; |
| 44 | + | |
| 45 | + struct amdgpu_reset_domain *reset_domain; | |
| 45 | 46 | }; |
| 46 | 47 | |
| 47 | 48 | struct amdgpu_pcs_ras_field { |
| @@ -50,7 +51,7 @@ struct amdgpu_pcs_ras_field { · 2 unmodified lines | ||
| 50 | 51 | uint32_t pcs_err_shift; |
| 51 | 52 | }; |
| 52 | 53 | |
| 53 | −extern const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs; | |
| 54 | +extern struct amdgpu_xgmi_ras xgmi_ras; | |
| 54 | 55 | struct amdgpu_hive_info *amdgpu_get_xgmi_hive(struct amdgpu_device *adev); |
| 55 | 56 | void amdgpu_put_xgmi_hive(struct amdgpu_hive_info *hive); |
| 56 | 57 | int amdgpu_xgmi_update_topology(struct amdgpu_hive_info *hive, struct amdgpu_device *adev); |
| @@ -66,7 +67,8 @@ uint64_t amdgpu_xgmi_get_relative_phy_addr(struct amdgpu_device *adev, · 9 unmodified lines | ||
| 66 | 67 | static inline bool amdgpu_xgmi_same_hive(struct amdgpu_device *adev, |
| 67 | 68 | struct amdgpu_device *bo_adev) |
| 68 | 69 | { |
| 69 | − return (adev != bo_adev && | |
| 70 | + return (amdgpu_use_xgmi_p2p && | |
| 71 | + adev != bo_adev && | |
| 70 | 72 | adev->gmc.xgmi.hive_id && |
| 71 | 73 | adev->gmc.xgmi.hive_id == bo_adev->gmc.xgmi.hive_id); |
| 72 | 74 | } |
| @@ -68,12 +68,13 @@ int athub_v1_0_set_clockgating(struct amdgpu_device *adev, · 67 unmodified lines | ||
| 68 | 68 | if (amdgpu_sriov_vf(adev)) |
| 69 | 69 | return 0; |
| 70 | 70 | |
| 71 | − switch (adev->asic_type) { | |
| 72 | − case CHIP_VEGA10: | |
| 73 | − case CHIP_VEGA12: | |
| 74 | − case CHIP_VEGA20: | |
| 75 | − case CHIP_RAVEN: | |
| 76 | − case CHIP_RENOIR: | |
| 71 | + switch (adev->ip_versions[ATHUB_HWIP][0]) { | |
| 72 | + case IP_VERSION(9, 0, 0): | |
| 73 | + case IP_VERSION(9, 1, 0): | |
| 74 | + case IP_VERSION(9, 2, 0): | |
| 75 | + case IP_VERSION(9, 3, 0): | |
| 76 | + case IP_VERSION(9, 4, 0): | |
| 77 | + case IP_VERSION(1, 5, 0): | |
| 77 | 78 | athub_update_medium_grain_clock_gating(adev, |
| 78 | 79 | state == AMD_CG_STATE_GATE); |
| 79 | 80 | athub_update_medium_grain_light_sleep(adev, |
| @@ -78,6 +78,7 @@ int athub_v2_0_set_clockgating(struct amdgpu_device *adev, · 77 unmodified lines | ||
| 78 | 78 | return 0; |
| 79 | 79 | |
| 80 | 80 | switch (adev->ip_versions[ATHUB_HWIP][0]) { |
| 81 | + case IP_VERSION(1, 3, 1): | |
| 81 | 82 | case IP_VERSION(2, 0, 0): |
| 82 | 83 | case IP_VERSION(2, 0, 2): |
| 83 | 84 | athub_v2_0_update_medium_grain_clock_gating(adev, |
| @@ -74,6 +74,7 @@ int athub_v2_1_set_clockgating(struct amdgpu_device *adev, · 73 unmodified lines | ||
| 74 | 74 | case IP_VERSION(2, 1, 0): |
| 75 | 75 | case IP_VERSION(2, 1, 1): |
| 76 | 76 | case IP_VERSION(2, 1, 2): |
| 77 | + case IP_VERSION(2, 4, 0): | |
| 77 | 78 | athub_v2_1_update_medium_grain_clock_gating(adev, state == AMD_CG_STATE_GATE); |
| 78 | 79 | athub_v2_1_update_medium_grain_light_sleep(adev, state == AMD_CG_STATE_GATE); |
| 79 | 80 | break; |
| @@ -25,6 +25,8 @@ · 24 unmodified lines | ||
| 25 | 25 | #include <linux/module.h> |
| 26 | 26 | #include <linux/sched.h> |
| 27 | 27 | #include <linux/slab.h> |
| 28 | +#include <linux/string_helpers.h> | |
| 29 | + | |
| 28 | 30 | #include <asm/unaligned.h> |
| 29 | 31 | |
| 30 | 32 | #include <drm/drm_util.h> |
| @@ -740,7 +742,7 @@ static void atom_op_jump(atom_exec_context *ctx, int *ptr, int arg) · 709 unmodified lines | ||
| 740 | 742 | break; |
| 741 | 743 | } |
| 742 | 744 | if (arg != ATOM_COND_ALWAYS) |
| 743 | − SDEBUG(" taken: %s\n", execute ? "yes" : "no"); | |
| 745 | + SDEBUG(" taken: %s\n", str_yes_no(execute)); | |
| 744 | 746 | SDEBUG(" target: 0x%04X\n", target); |
| 745 | 747 | if (execute) { |
| 746 | 748 | if (ctx->last_jump == (ctx->start + target)) { |
| @@ -1719,7 +1719,7 @@ static void cik_program_aspm(struct amdgpu_device *adev) · 1718 unmodified lines | ||
| 1719 | 1719 | bool disable_l0s = false, disable_l1 = false, disable_plloff_in_l1 = false; |
| 1720 | 1720 | bool disable_clkreq = false; |
| 1721 | 1721 | |
| 1722 | − if (amdgpu_aspm == 0) | |
| 1722 | + if (!amdgpu_device_should_use_aspm(adev)) | |
| 1723 | 1723 | return; |
| 1724 | 1724 | |
| 1725 | 1725 | if (pci_is_root_bus(adev->pdev->bus)) |
| @@ -2532,7 +2532,7 @@ static void dce_v10_0_crtc_dpms(struct drm_crtc *crtc, int mode) · 2531 unmodified lines | ||
| 2532 | 2532 | break; |
| 2533 | 2533 | } |
| 2534 | 2534 | /* adjust pm to dpms */ |
| 2535 | − amdgpu_pm_compute_clocks(adev); | |
| 2535 | + amdgpu_dpm_compute_clocks(adev); | |
| 2536 | 2536 | } |
| 2537 | 2537 | |
| 2538 | 2538 | static void dce_v10_0_crtc_prepare(struct drm_crtc *crtc) |
| @@ -2798,6 +2798,8 @@ static int dce_v10_0_sw_init(void *handle) · 259 unmodified lines | ||
| 2798 | 2798 | adev_to_drm(adev)->mode_config.preferred_depth = 24; |
| 2799 | 2799 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; |
| 2800 | 2800 | |
| 2801 | + adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true; | |
| 2802 | + | |
| 2801 | 2803 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; |
| 2802 | 2804 | |
| 2803 | 2805 | r = amdgpu_display_modeset_create_props(adev); |
| @@ -2608,7 +2608,7 @@ static void dce_v11_0_crtc_dpms(struct drm_crtc *crtc, int mode) · 2607 unmodified lines | ||
| 2608 | 2608 | break; |
| 2609 | 2609 | } |
| 2610 | 2610 | /* adjust pm to dpms */ |
| 2611 | − amdgpu_pm_compute_clocks(adev); | |
| 2611 | + amdgpu_dpm_compute_clocks(adev); | |
| 2612 | 2612 | } |
| 2613 | 2613 | |
| 2614 | 2614 | static void dce_v11_0_crtc_prepare(struct drm_crtc *crtc) |
| @@ -2916,6 +2916,8 @@ static int dce_v11_0_sw_init(void *handle) · 301 unmodified lines | ||
| 2916 | 2916 | adev_to_drm(adev)->mode_config.preferred_depth = 24; |
| 2917 | 2917 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; |
| 2918 | 2918 | |
| 2919 | + adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true; | |
| 2920 | + | |
| 2919 | 2921 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; |
| 2920 | 2922 | |
| 2921 | 2923 | r = amdgpu_display_modeset_create_props(adev); |
| @@ -2424,7 +2424,7 @@ static void dce_v6_0_crtc_dpms(struct drm_crtc *crtc, int mode) · 2423 unmodified lines | ||
| 2424 | 2424 | break; |
| 2425 | 2425 | } |
| 2426 | 2426 | /* adjust pm to dpms */ |
| 2427 | − amdgpu_pm_compute_clocks(adev); | |
| 2427 | + amdgpu_dpm_compute_clocks(adev); | |
| 2428 | 2428 | } |
| 2429 | 2429 | |
| 2430 | 2430 | static void dce_v6_0_crtc_prepare(struct drm_crtc *crtc) |
| @@ -2674,6 +2674,7 @@ static int dce_v6_0_sw_init(void *handle) · 243 unmodified lines | ||
| 2674 | 2674 | adev_to_drm(adev)->mode_config.max_height = 16384; |
| 2675 | 2675 | adev_to_drm(adev)->mode_config.preferred_depth = 24; |
| 2676 | 2676 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; |
| 2677 | + adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true; | |
| 2677 | 2678 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; |
| 2678 | 2679 | |
| 2679 | 2680 | r = amdgpu_display_modeset_create_props(adev); |
| @@ -2433,7 +2433,7 @@ static void dce_v8_0_crtc_dpms(struct drm_crtc *crtc, int mode) · 2432 unmodified lines | ||
| 2433 | 2433 | break; |
| 2434 | 2434 | } |
| 2435 | 2435 | /* adjust pm to dpms */ |
| 2436 | − amdgpu_pm_compute_clocks(adev); | |
| 2436 | + amdgpu_dpm_compute_clocks(adev); | |
| 2437 | 2437 | } |
| 2438 | 2438 | |
| 2439 | 2439 | static void dce_v8_0_crtc_prepare(struct drm_crtc *crtc) |
| @@ -2695,6 +2695,8 @@ static int dce_v8_0_sw_init(void *handle) · 255 unmodified lines | ||
| 2695 | 2695 | adev_to_drm(adev)->mode_config.preferred_depth = 24; |
| 2696 | 2696 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; |
| 2697 | 2697 | |
| 2698 | + adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true; | |
| 2699 | + | |
| 2698 | 2700 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; |
| 2699 | 2701 | |
| 2700 | 2702 | r = amdgpu_display_modeset_create_props(adev); |
| @@ -458,7 +458,7 @@ static int df_v3_6_pmc_add_cntr(struct amdgpu_device *adev, · 457 unmodified lines | ||
| 458 | 458 | |
| 459 | 459 | #define DEFERRED_ARM_MASK (1 << 31) |
| 460 | 460 | static int df_v3_6_pmc_set_deferred(struct amdgpu_device *adev, |
| 461 | − int counter_idx, uint64_t config, | |
| 461 | + uint64_t config, int counter_idx, | |
| 462 | 462 | bool is_deferred) |
| 463 | 463 | { |
| 464 | 464 | |
| @@ -476,8 +476,8 @@ static int df_v3_6_pmc_set_deferred(struct amdgpu_device *adev, · 11 unmodified lines | ||
| 476 | 476 | } |
| 477 | 477 | |
| 478 | 478 | static bool df_v3_6_pmc_is_deferred(struct amdgpu_device *adev, |
| 479 | − int counter_idx, | |
| 480 | − uint64_t config) | |
| 479 | + uint64_t config, | |
| 480 | + int counter_idx) | |
| 481 | 481 | { |
| 482 | 482 | return (df_v3_6_pmc_has_counter(adev, config, counter_idx) && |
| 483 | 483 | (adev->df_perfmon_config_assign_mask[counter_idx] |
| @@ -56,10 +56,6 @@ · 55 unmodified lines | ||
| 56 | 56 | #define GFX10_NUM_GFX_RINGS_Sienna_Cichlid 1 |
| 57 | 57 | #define GFX10_MEC_HPD_SIZE 2048 |
| 58 | 58 | |
| 59 | −#define RLCG_VFGATE_DISABLED 0x4000000 | |
| 60 | −#define RLCG_WRONG_OPERATION_TYPE 0x2000000 | |
| 61 | −#define RLCG_NOT_IN_RANGE 0x1000000 | |
| 62 | − | |
| 63 | 59 | #define F32_CE_PROGRAM_RAM_SIZE 65536 |
| 64 | 60 | #define RLCG_UCODE_LOADING_START_ADDRESS 0x00002000L |
| 65 | 61 | |
| @@ -110,6 +106,12 @@ · 44 unmodified lines | ||
| 110 | 106 | #define mmGOLDEN_TSC_COUNT_UPPER_Vangogh_BASE_IDX 1 |
| 111 | 107 | #define mmGOLDEN_TSC_COUNT_LOWER_Vangogh 0x0026 |
| 112 | 108 | #define mmGOLDEN_TSC_COUNT_LOWER_Vangogh_BASE_IDX 1 |
| 109 | + | |
| 110 | +#define mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6 0x002d | |
| 111 | +#define mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6_BASE_IDX 1 | |
| 112 | +#define mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6 0x002e | |
| 113 | +#define mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6_BASE_IDX 1 | |
| 114 | + | |
| 113 | 115 | #define mmSPI_CONFIG_CNTL_1_Vangogh 0x2441 |
| 114 | 116 | #define mmSPI_CONFIG_CNTL_1_Vangogh_BASE_IDX 1 |
| 115 | 117 | #define mmVGT_TF_MEMORY_BASE_HI_Vangogh 0x2261 |
| @@ -180,14 +182,6 @@ · 64 unmodified lines | ||
| 180 | 182 | #define mmRLC_SPARE_INT_0_Sienna_Cichlid 0x4ca5 |
| 181 | 183 | #define mmRLC_SPARE_INT_0_Sienna_Cichlid_BASE_IDX 1 |
| 182 | 184 | |
| 183 | −#define GFX_RLCG_GC_WRITE_OLD (0x8 << 28) | |
| 184 | −#define GFX_RLCG_GC_WRITE (0x0 << 28) | |
| 185 | −#define GFX_RLCG_GC_READ (0x1 << 28) | |
| 186 | −#define GFX_RLCG_MMHUB_WRITE (0x2 << 28) | |
| 187 | − | |
| 188 | −#define RLCG_ERROR_REPORT_ENABLED(adev) \ | |
| 189 | − (amdgpu_sriov_reg_indirect_mmhub(adev) || amdgpu_sriov_reg_indirect_gc(adev)) | |
| 190 | − | |
| 191 | 185 | MODULE_FIRMWARE("amdgpu/navi10_ce.bin"); |
| 192 | 186 | MODULE_FIRMWARE("amdgpu/navi10_pfp.bin"); |
| 193 | 187 | MODULE_FIRMWARE("amdgpu/navi10_me.bin"); |
| @@ -270,6 +264,20 @@ MODULE_FIRMWARE("amdgpu/cyan_skillfish2_mec.bin"); · 76 unmodified lines | ||
| 270 | 264 | MODULE_FIRMWARE("amdgpu/cyan_skillfish2_mec2.bin"); |
| 271 | 265 | MODULE_FIRMWARE("amdgpu/cyan_skillfish2_rlc.bin"); |
| 272 | 266 | |
| 267 | +MODULE_FIRMWARE("amdgpu/gc_10_3_6_ce.bin"); | |
| 268 | +MODULE_FIRMWARE("amdgpu/gc_10_3_6_pfp.bin"); | |
| 269 | +MODULE_FIRMWARE("amdgpu/gc_10_3_6_me.bin"); | |
| 270 | +MODULE_FIRMWARE("amdgpu/gc_10_3_6_mec.bin"); | |
| 271 | +MODULE_FIRMWARE("amdgpu/gc_10_3_6_mec2.bin"); | |
| 272 | +MODULE_FIRMWARE("amdgpu/gc_10_3_6_rlc.bin"); | |
| 273 | + | |
| 274 | +MODULE_FIRMWARE("amdgpu/gc_10_3_7_ce.bin"); | |
| 275 | +MODULE_FIRMWARE("amdgpu/gc_10_3_7_pfp.bin"); | |
| 276 | +MODULE_FIRMWARE("amdgpu/gc_10_3_7_me.bin"); | |
| 277 | +MODULE_FIRMWARE("amdgpu/gc_10_3_7_mec.bin"); | |
| 278 | +MODULE_FIRMWARE("amdgpu/gc_10_3_7_mec2.bin"); | |
| 279 | +MODULE_FIRMWARE("amdgpu/gc_10_3_7_rlc.bin"); | |
| 280 | + | |
| 273 | 281 | static const struct soc15_reg_golden golden_settings_gc_10_1[] = |
| 274 | 282 | { |
| 275 | 283 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmCB_HW_CONTROL_4, 0xffffffff, 0x00400014), |
| @@ -1463,143 +1471,6 @@ static const struct soc15_reg_golden golden_settings_gc_10_1_2[] = · 1187 unmodified lines | ||
| 1463 | 1471 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00c00000) |
| 1464 | 1472 | }; |
| 1465 | 1473 | |
| 1466 | −static bool gfx_v10_get_rlcg_flag(struct amdgpu_device *adev, u32 acc_flags, u32 hwip, | |
| 1467 | − int write, u32 *rlcg_flag) | |
| 1468 | −{ | |
| 1469 | − switch (hwip) { | |
| 1470 | − case GC_HWIP: | |
| 1471 | − if (amdgpu_sriov_reg_indirect_gc(adev)) { | |
| 1472 | − *rlcg_flag = write ? GFX_RLCG_GC_WRITE : GFX_RLCG_GC_READ; | |
| 1473 | − | |
| 1474 | − return true; | |
| 1475 | − /* only in new version, AMDGPU_REGS_NO_KIQ and AMDGPU_REGS_RLC enabled simultaneously */ | |
| 1476 | − } else if ((acc_flags & AMDGPU_REGS_RLC) && !(acc_flags & AMDGPU_REGS_NO_KIQ)) { | |
| 1477 | − *rlcg_flag = GFX_RLCG_GC_WRITE_OLD; | |
| 1478 | − | |
| 1479 | − return true; | |
| 1480 | − } | |
| 1481 | − | |
| 1482 | − break; | |
| 1483 | − case MMHUB_HWIP: | |
| 1484 | − if (amdgpu_sriov_reg_indirect_mmhub(adev) && | |
| 1485 | − (acc_flags & AMDGPU_REGS_RLC) && write) { | |
| 1486 | − *rlcg_flag = GFX_RLCG_MMHUB_WRITE; | |
| 1487 | − return true; | |
| 1488 | − } | |
| 1489 | − | |
| 1490 | − break; | |
| 1491 | − default: | |
| 1492 | − DRM_DEBUG("Not program register by RLCG\n"); | |
| 1493 | − } | |
| 1494 | − | |
| 1495 | − return false; | |
| 1496 | −} | |
| 1497 | − | |
| 1498 | −static u32 gfx_v10_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag) | |
| 1499 | −{ | |
| 1500 | − static void *scratch_reg0; | |
| 1501 | − static void *scratch_reg1; | |
| 1502 | − static void *scratch_reg2; | |
| 1503 | − static void *scratch_reg3; | |
| 1504 | − static void *spare_int; | |
| 1505 | − static uint32_t grbm_cntl; | |
| 1506 | − static uint32_t grbm_idx; | |
| 1507 | − uint32_t i = 0; | |
| 1508 | − uint32_t retries = 50000; | |
| 1509 | − u32 ret = 0; | |
| 1510 | − u32 tmp; | |
| 1511 | − | |
| 1512 | − scratch_reg0 = adev->rmmio + | |
| 1513 | − (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0) * 4; | |
| 1514 | − scratch_reg1 = adev->rmmio + | |
| 1515 | − (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1) * 4; | |
| 1516 | − scratch_reg2 = adev->rmmio + | |
| 1517 | − (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG2) * 4; | |
| 1518 | − scratch_reg3 = adev->rmmio + | |
| 1519 | − (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3) * 4; | |
| 1520 | − | |
| 1521 | − if (adev->ip_versions[GC_HWIP][0] >= IP_VERSION(10, 3, 0)) { | |
| 1522 | − spare_int = adev->rmmio + | |
| 1523 | − (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_0_Sienna_Cichlid_BASE_IDX] | |
| 1524 | − + mmRLC_SPARE_INT_0_Sienna_Cichlid) * 4; | |
| 1525 | − } else { | |
| 1526 | − spare_int = adev->rmmio + | |
| 1527 | − (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT) * 4; | |
| 1528 | − } | |
| 1529 | − | |
| 1530 | − grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL; | |
| 1531 | − grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX; | |
| 1532 | − | |
| 1533 | − if (offset == grbm_cntl || offset == grbm_idx) { | |
| 1534 | − if (offset == grbm_cntl) | |
| 1535 | − writel(v, scratch_reg2); | |
| 1536 | − else if (offset == grbm_idx) | |
| 1537 | − writel(v, scratch_reg3); | |
| 1538 | − | |
| 1539 | − writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); | |
| 1540 | − } else { | |
| 1541 | − writel(v, scratch_reg0); | |
| 1542 | − writel(offset | flag, scratch_reg1); | |
| 1543 | − writel(1, spare_int); | |
| 1544 | − | |
| 1545 | − for (i = 0; i < retries; i++) { | |
| 1546 | − tmp = readl(scratch_reg1); | |
| 1547 | − if (!(tmp & flag)) | |
| 1548 | − break; | |
| 1549 | − | |
| 1550 | − udelay(10); | |
| 1551 | − } | |
| 1552 | − | |
| 1553 | − if (i >= retries) { | |
| 1554 | − if (RLCG_ERROR_REPORT_ENABLED(adev)) { | |
| 1555 | − if (tmp & RLCG_VFGATE_DISABLED) | |
| 1556 | − pr_err("The vfgate is disabled, program reg:0x%05x failed!\n", offset); | |
| 1557 | − else if (tmp & RLCG_WRONG_OPERATION_TYPE) | |
| 1558 | − pr_err("Wrong operation type, program reg:0x%05x failed!\n", offset); | |
| 1559 | − else if (tmp & RLCG_NOT_IN_RANGE) | |
| 1560 | − pr_err("The register is not in range, program reg:0x%05x failed!\n", offset); | |
| 1561 | − else | |
| 1562 | − pr_err("Unknown error type, program reg:0x%05x failed!\n", offset); | |
| 1563 | − } else | |
| 1564 | − pr_err("timeout: rlcg program reg:0x%05x failed!\n", offset); | |
| 1565 | − } | |
| 1566 | − } | |
| 1567 | − | |
| 1568 | − ret = readl(scratch_reg0); | |
| 1569 | − | |
| 1570 | − return ret; | |
| 1571 | −} | |
| 1572 | − | |
| 1573 | −static void gfx_v10_sriov_wreg(struct amdgpu_device *adev, u32 offset, u32 value, u32 acc_flags, u32 hwip) | |
| 1574 | −{ | |
| 1575 | − u32 rlcg_flag; | |
| 1576 | − | |
| 1577 | − if (!amdgpu_sriov_runtime(adev) && | |
| 1578 | − gfx_v10_get_rlcg_flag(adev, acc_flags, hwip, 1, &rlcg_flag)) { | |
| 1579 | − gfx_v10_rlcg_rw(adev, offset, value, rlcg_flag); | |
| 1580 | − return; | |
| 1581 | − } | |
| 1582 | − | |
| 1583 | − if (acc_flags & AMDGPU_REGS_NO_KIQ) | |
| 1584 | − WREG32_NO_KIQ(offset, value); | |
| 1585 | − else | |
| 1586 | − WREG32(offset, value); | |
| 1587 | −} | |
| 1588 | − | |
| 1589 | −static u32 gfx_v10_sriov_rreg(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip) | |
| 1590 | −{ | |
| 1591 | − u32 rlcg_flag; | |
| 1592 | − | |
| 1593 | − if (!amdgpu_sriov_runtime(adev) && | |
| 1594 | − gfx_v10_get_rlcg_flag(adev, acc_flags, hwip, 0, &rlcg_flag)) | |
| 1595 | − return gfx_v10_rlcg_rw(adev, offset, 0, rlcg_flag); | |
| 1596 | − | |
| 1597 | − if (acc_flags & AMDGPU_REGS_NO_KIQ) | |
| 1598 | − return RREG32_NO_KIQ(offset); | |
| 1599 | − else | |
| 1600 | − return RREG32(offset); | |
| 1601 | −} | |
| 1602 | − | |
| 1603 | 1474 | static const struct soc15_reg_golden golden_settings_gc_10_1_nv14[] = |
| 1604 | 1475 | { |
| 1605 | 1476 | /* Pending on emulation bring up */ |
| @@ -3557,6 +3428,57 @@ static const struct soc15_reg_golden golden_settings_gc_10_0_cyan_skillfish[] = · 1951 unmodified lines | ||
| 3557 | 3428 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0x00800000, 0x00800000) |
| 3558 | 3429 | }; |
| 3559 | 3430 | |
| 3431 | +static const struct soc15_reg_golden golden_settings_gc_10_3_6[] = | |
| 3432 | +{ | |
| 3433 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCGTT_SPI_CS_CLK_CTRL, 0xff7f0fff, 0x78000100), | |
| 3434 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCH_PIPE_STEER, 0x000000ff, 0x00000044), | |
| 3435 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCPF_GCR_CNTL, 0x0007ffff, 0x0000c200), | |
| 3436 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG3, 0xffffffff, 0x00000280), | |
| 3437 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG4, 0xffffffff, 0x00800000), | |
| 3438 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGB_ADDR_CONFIG, 0x0c1807ff, 0x00000042), | |
| 3439 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCR_GENERAL_CNTL, 0x1ff1ffff, 0x00000500), | |
| 3440 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL1_PIPE_STEER, 0x000000ff, 0x00000044), | |
| 3441 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_0, 0x77777777, 0x32103210), | |
| 3442 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_1, 0x77777777, 0x32103210), | |
| 3443 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2A_ADDR_MATCH_MASK, 0xffffffff, 0xfffffff3), | |
| 3444 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_ADDR_MATCH_MASK, 0xffffffff, 0xfffffff3), | |
| 3445 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CM_CTRL1, 0xff8fff0f, 0x580f1008), | |
| 3446 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CTRL3, 0xf7ffffff, 0x00f80988), | |
| 3447 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmLDS_CONFIG, 0x000001ff, 0x00000020), | |
| 3448 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_CL_ENHANCE, 0xf17fffff, 0x01200007), | |
| 3449 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_BINNER_TIMEOUT_COUNTER, 0xffffffff, 0x00000800), | |
| 3450 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_ENHANCE_2, 0xffffffbf, 0x00000820), | |
| 3451 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmSQG_CONFIG, 0x000017ff, 0x00001000), | |
| 3452 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmSX_DEBUG_1, 0xffffff7f, 0x00010020), | |
| 3453 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmTA_CNTL_AUX, 0xfff7ffff, 0x01030000), | |
| 3454 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00100000) | |
| 3455 | +}; | |
| 3456 | + | |
| 3457 | +static const struct soc15_reg_golden golden_settings_gc_10_3_7[] = { | |
| 3458 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCGTT_SPI_CS_CLK_CTRL, 0xff7f0fff, 0x78000100), | |
| 3459 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCH_PIPE_STEER, 0x000000ff, 0x000000e4), | |
| 3460 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmCPF_GCR_CNTL, 0x0007ffff, 0x0000c200), | |
| 3461 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG3, 0xffffffff, 0x00000280), | |
| 3462 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG4, 0xffffffff, 0x00800000), | |
| 3463 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGB_ADDR_CONFIG, 0x0c1807ff, 0x00000041), | |
| 3464 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCR_GENERAL_CNTL, 0x1ff1ffff, 0x00000500), | |
| 3465 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL1_PIPE_STEER, 0x000000ff, 0x000000e4), | |
| 3466 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_0, 0x77777777, 0x32103210), | |
| 3467 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_1, 0x77777777, 0x32103210), | |
| 3468 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2A_ADDR_MATCH_MASK, 0xffffffff, 0xffffffff), | |
| 3469 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_ADDR_MATCH_MASK, 0xffffffff, 0xffffffff), | |
| 3470 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CM_CTRL1, 0xff8fff0f, 0x580f1008), | |
| 3471 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CTRL3, 0xf7ffffff, 0x00f80988), | |
| 3472 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmLDS_CONFIG, 0x000001ff, 0x00000020), | |
| 3473 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_CL_ENHANCE, 0xf000003f, 0x01200007), | |
| 3474 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_BINNER_TIMEOUT_COUNTER, 0xffffffff, 0x00000800), | |
| 3475 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_ENHANCE_2, 0xffffffbf, 0x00000820), | |
| 3476 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmSQG_CONFIG, 0x000017ff, 0x00001000), | |
| 3477 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmSX_DEBUG_1, 0xffffff7f, 0x00010020), | |
| 3478 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmTA_CNTL_AUX, 0xfff7ffff, 0x01030000), | |
| 3479 | + SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00100000) | |
| 3480 | +}; | |
| 3481 | + | |
| 3560 | 3482 | #define DEFAULT_SH_MEM_CONFIG \ |
| 3561 | 3483 | ((SH_MEM_ADDRESS_MODE_64 << SH_MEM_CONFIG__ADDRESS_MODE__SHIFT) | \ |
| 3562 | 3484 | (SH_MEM_ALIGNMENT_MODE_UNALIGNED << SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT) | \ |
| @@ -3790,10 +3712,21 @@ static void gfx_v10_0_init_golden_registers(struct amdgpu_device *adev) · 227 unmodified lines | ||
| 3790 | 3712 | (const u32)ARRAY_SIZE(golden_settings_gc_10_3_5)); |
| 3791 | 3713 | break; |
| 3792 | 3714 | case IP_VERSION(10, 1, 3): |
| 3715 | + case IP_VERSION(10, 1, 4): | |
| 3793 | 3716 | soc15_program_register_sequence(adev, |
| 3794 | 3717 | golden_settings_gc_10_0_cyan_skillfish, |
| 3795 | 3718 | (const u32)ARRAY_SIZE(golden_settings_gc_10_0_cyan_skillfish)); |
| 3796 | 3719 | break; |
| 3720 | + case IP_VERSION(10, 3, 6): | |
| 3721 | + soc15_program_register_sequence(adev, | |
| 3722 | + golden_settings_gc_10_3_6, | |
| 3723 | + (const u32)ARRAY_SIZE(golden_settings_gc_10_3_6)); | |
| 3724 | + break; | |
| 3725 | + case IP_VERSION(10, 3, 7): | |
| 3726 | + soc15_program_register_sequence(adev, | |
| 3727 | + golden_settings_gc_10_3_7, | |
| 3728 | + (const u32)ARRAY_SIZE(golden_settings_gc_10_3_7)); | |
| 3729 | + break; | |
| 3797 | 3730 | default: |
| 3798 | 3731 | break; |
| 3799 | 3732 | } |
| @@ -3968,6 +3901,7 @@ static void gfx_v10_0_check_fw_write_wait(struct amdgpu_device *adev) · 168 unmodified lines | ||
| 3968 | 3901 | case IP_VERSION(10, 1, 2): |
| 3969 | 3902 | case IP_VERSION(10, 1, 1): |
| 3970 | 3903 | case IP_VERSION(10, 1, 3): |
| 3904 | + case IP_VERSION(10, 1, 4): | |
| 3971 | 3905 | if ((adev->gfx.me_fw_version >= 0x00000046) && |
| 3972 | 3906 | (adev->gfx.me_feature_version >= 27) && |
| 3973 | 3907 | (adev->gfx.pfp_fw_version >= 0x00000068) && |
| @@ -3981,7 +3915,9 @@ static void gfx_v10_0_check_fw_write_wait(struct amdgpu_device *adev) · 7 unmodified lines | ||
| 3981 | 3915 | case IP_VERSION(10, 3, 1): |
| 3982 | 3916 | case IP_VERSION(10, 3, 4): |
| 3983 | 3917 | case IP_VERSION(10, 3, 5): |
| 3918 | + case IP_VERSION(10, 3, 6): | |
| 3984 | 3919 | case IP_VERSION(10, 3, 3): |
| 3920 | + case IP_VERSION(10, 3, 7): | |
| 3985 | 3921 | adev->gfx.cp_fw_write_wait = true; |
| 3986 | 3922 | break; |
| 3987 | 3923 | default: |
| @@ -4102,12 +4038,19 @@ static int gfx_v10_0_init_microcode(struct amdgpu_device *adev) · 114 unmodified lines | ||
| 4102 | 4038 | case IP_VERSION(10, 3, 3): |
| 4103 | 4039 | chip_name = "yellow_carp"; |
| 4104 | 4040 | break; |
| 4041 | + case IP_VERSION(10, 3, 6): | |
| 4042 | + chip_name = "gc_10_3_6"; | |
| 4043 | + break; | |
| 4105 | 4044 | case IP_VERSION(10, 1, 3): |
| 4045 | + case IP_VERSION(10, 1, 4): | |
| 4106 | 4046 | if (adev->apu_flags & AMD_APU_IS_CYAN_SKILLFISH2) |
| 4107 | 4047 | chip_name = "cyan_skillfish2"; |
| 4108 | 4048 | else |
| 4109 | 4049 | chip_name = "cyan_skillfish"; |
| 4110 | 4050 | break; |
| 4051 | + case IP_VERSION(10, 3, 7): | |
| 4052 | + chip_name = "gc_10_3_7"; | |
| 4053 | + break; | |
| 4111 | 4054 | default: |
| 4112 | 4055 | BUG(); |
| 4113 | 4056 | } |
| @@ -4448,6 +4391,30 @@ static void gfx_v10_0_rlc_fini(struct amdgpu_device *adev) · 334 unmodified lines | ||
| 4448 | 4391 | (void **)&adev->gfx.rlc.cp_table_ptr); |
| 4449 | 4392 | } |
| 4450 | 4393 | |
| 4394 | +static void gfx_v10_0_init_rlcg_reg_access_ctrl(struct amdgpu_device *adev) | |
| 4395 | +{ | |
| 4396 | + struct amdgpu_rlcg_reg_access_ctrl *reg_access_ctrl; | |
| 4397 | + | |
| 4398 | + reg_access_ctrl = &adev->gfx.rlc.reg_access_ctrl; | |
| 4399 | + reg_access_ctrl->scratch_reg0 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG0); | |
| 4400 | + reg_access_ctrl->scratch_reg1 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG1); | |
| 4401 | + reg_access_ctrl->scratch_reg2 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG2); | |
| 4402 | + reg_access_ctrl->scratch_reg3 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG3); | |
| 4403 | + reg_access_ctrl->grbm_cntl = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_CNTL); | |
| 4404 | + reg_access_ctrl->grbm_idx = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX); | |
| 4405 | + switch (adev->ip_versions[GC_HWIP][0]) { | |
| 4406 | + case IP_VERSION(10, 3, 0): | |
| 4407 | + reg_access_ctrl->spare_int = | |
| 4408 | + SOC15_REG_OFFSET(GC, 0, mmRLC_SPARE_INT_0_Sienna_Cichlid); | |
| 4409 | + break; | |
| 4410 | + default: | |
| 4411 | + reg_access_ctrl->spare_int = | |
| 4412 | + SOC15_REG_OFFSET(GC, 0, mmRLC_SPARE_INT); | |
| 4413 | + break; | |
| 4414 | + } | |
| 4415 | + adev->gfx.rlc.rlcg_reg_access_supported = true; | |
| 4416 | +} | |
| 4417 | + | |
| 4451 | 4418 | static int gfx_v10_0_rlc_init(struct amdgpu_device *adev) |
| 4452 | 4419 | { |
| 4453 | 4420 | const struct cs_section_def *cs_data; |
| @@ -4468,6 +4435,7 @@ static int gfx_v10_0_rlc_init(struct amdgpu_device *adev) · 14 unmodified lines | ||
| 4468 | 4435 | if (adev->gfx.rlc.funcs->update_spm_vmid) |
| 4469 | 4436 | adev->gfx.rlc.funcs->update_spm_vmid(adev, 0xf); |
| 4470 | 4437 | |
| 4438 | + | |
| 4471 | 4439 | return 0; |
| 4472 | 4440 | } |
| 4473 | 4441 | |
| @@ -4678,7 +4646,9 @@ static void gfx_v10_0_gpu_early_init(struct amdgpu_device *adev) · 204 unmodified lines | ||
| 4678 | 4646 | case IP_VERSION(10, 3, 1): |
| 4679 | 4647 | case IP_VERSION(10, 3, 4): |
| 4680 | 4648 | case IP_VERSION(10, 3, 5): |
| 4649 | + case IP_VERSION(10, 3, 6): | |
| 4681 | 4650 | case IP_VERSION(10, 3, 3): |
| 4651 | + case IP_VERSION(10, 3, 7): | |
| 4682 | 4652 | adev->gfx.config.max_hw_contexts = 8; |
| 4683 | 4653 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 4684 | 4654 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -4689,6 +4659,7 @@ static void gfx_v10_0_gpu_early_init(struct amdgpu_device *adev) · 4 unmodified lines | ||
| 4689 | 4659 | 1 << REG_GET_FIELD(gb_addr_config, GB_ADDR_CONFIG, NUM_PKRS); |
| 4690 | 4660 | break; |
| 4691 | 4661 | case IP_VERSION(10, 1, 3): |
| 4662 | + case IP_VERSION(10, 1, 4): | |
| 4692 | 4663 | adev->gfx.config.max_hw_contexts = 8; |
| 4693 | 4664 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 4694 | 4665 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -4801,6 +4772,7 @@ static int gfx_v10_0_sw_init(void *handle) · 106 unmodified lines | ||
| 4801 | 4772 | case IP_VERSION(10, 1, 1): |
| 4802 | 4773 | case IP_VERSION(10, 1, 2): |
| 4803 | 4774 | case IP_VERSION(10, 1, 3): |
| 4775 | + case IP_VERSION(10, 1, 4): | |
| 4804 | 4776 | adev->gfx.me.num_me = 1; |
| 4805 | 4777 | adev->gfx.me.num_pipe_per_me = 1; |
| 4806 | 4778 | adev->gfx.me.num_queue_per_pipe = 1; |
| @@ -4813,7 +4785,9 @@ static int gfx_v10_0_sw_init(void *handle) · 6 unmodified lines | ||
| 4813 | 4785 | case IP_VERSION(10, 3, 1): |
| 4814 | 4786 | case IP_VERSION(10, 3, 4): |
| 4815 | 4787 | case IP_VERSION(10, 3, 5): |
| 4788 | + case IP_VERSION(10, 3, 6): | |
| 4816 | 4789 | case IP_VERSION(10, 3, 3): |
| 4790 | + case IP_VERSION(10, 3, 7): | |
| 4817 | 4791 | adev->gfx.me.num_me = 1; |
| 4818 | 4792 | adev->gfx.me.num_pipe_per_me = 1; |
| 4819 | 4793 | adev->gfx.me.num_queue_per_pipe = 1; |
| @@ -4865,10 +4839,14 @@ static int gfx_v10_0_sw_init(void *handle) · 45 unmodified lines | ||
| 4865 | 4839 | if (r) |
| 4866 | 4840 | return r; |
| 4867 | 4841 | |
| 4868 | − r = gfx_v10_0_rlc_init(adev); | |
| 4869 | − if (r) { | |
| 4870 | − DRM_ERROR("Failed to init rlc BOs!\n"); | |
| 4871 | − return r; | |
| 4842 | + if (adev->gfx.rlc.funcs) { | |
| 4843 | + if (adev->gfx.rlc.funcs->init) { | |
| 4844 | + r = adev->gfx.rlc.funcs->init(adev); | |
| 4845 | + if (r) { | |
| 4846 | + dev_err(adev->dev, "Failed to init rlc BOs!\n"); | |
| 4847 | + return r; | |
| 4848 | + } | |
| 4849 | + } | |
| 4872 | 4850 | } |
| 4873 | 4851 | |
| 4874 | 4852 | r = gfx_v10_0_mec_init(adev); |
| @@ -5047,7 +5025,8 @@ static void gfx_v10_0_setup_rb(struct amdgpu_device *adev) · 172 unmodified lines | ||
| 5047 | 5025 | for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) { |
| 5048 | 5026 | bitmap = i * adev->gfx.config.max_sh_per_se + j; |
| 5049 | 5027 | if (((adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 0)) || |
| 5050 | − (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3))) && | |
| 5028 | + (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) || | |
| 5029 | + (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 6))) && | |
| 5051 | 5030 | ((gfx_v10_3_get_disabled_sa(adev) >> bitmap) & 1)) |
| 5052 | 5031 | continue; |
| 5053 | 5032 | gfx_v10_0_select_se_sh(adev, i, j, 0xffffffff); |
| @@ -6321,7 +6300,9 @@ static void gfx_v10_0_cp_gfx_set_doorbell(struct amdgpu_device *adev, · 1267 unmodified lines | ||
| 6321 | 6300 | case IP_VERSION(10, 3, 1): |
| 6322 | 6301 | case IP_VERSION(10, 3, 4): |
| 6323 | 6302 | case IP_VERSION(10, 3, 5): |
| 6303 | + case IP_VERSION(10, 3, 6): | |
| 6324 | 6304 | case IP_VERSION(10, 3, 3): |
| 6305 | + case IP_VERSION(10, 3, 7): | |
| 6325 | 6306 | tmp = REG_SET_FIELD(0, CP_RB_DOORBELL_RANGE_LOWER, |
| 6326 | 6307 | DOORBELL_RANGE_LOWER_Sienna_Cichlid, ring->doorbell_index); |
| 6327 | 6308 | WREG32_SOC15(GC, 0, mmCP_RB_DOORBELL_RANGE_LOWER, tmp); |
| @@ -6458,7 +6439,9 @@ static void gfx_v10_0_cp_compute_enable(struct amdgpu_device *adev, bool enable) · 130 unmodified lines | ||
| 6458 | 6439 | case IP_VERSION(10, 3, 1): |
| 6459 | 6440 | case IP_VERSION(10, 3, 4): |
| 6460 | 6441 | case IP_VERSION(10, 3, 5): |
| 6442 | + case IP_VERSION(10, 3, 6): | |
| 6461 | 6443 | case IP_VERSION(10, 3, 3): |
| 6444 | + case IP_VERSION(10, 3, 7): | |
| 6462 | 6445 | WREG32_SOC15(GC, 0, mmCP_MEC_CNTL_Sienna_Cichlid, 0); |
| 6463 | 6446 | break; |
| 6464 | 6447 | default: |
| @@ -6472,7 +6455,9 @@ static void gfx_v10_0_cp_compute_enable(struct amdgpu_device *adev, bool enable) · 7 unmodified lines | ||
| 6472 | 6455 | case IP_VERSION(10, 3, 1): |
| 6473 | 6456 | case IP_VERSION(10, 3, 4): |
| 6474 | 6457 | case IP_VERSION(10, 3, 5): |
| 6458 | + case IP_VERSION(10, 3, 6): | |
| 6475 | 6459 | case IP_VERSION(10, 3, 3): |
| 6460 | + case IP_VERSION(10, 3, 7): | |
| 6476 | 6461 | WREG32_SOC15(GC, 0, mmCP_MEC_CNTL_Sienna_Cichlid, |
| 6477 | 6462 | (CP_MEC_CNTL__MEC_ME1_HALT_MASK | |
| 6478 | 6463 | CP_MEC_CNTL__MEC_ME2_HALT_MASK)); |
| @@ -6570,6 +6555,7 @@ static void gfx_v10_0_kiq_setting(struct amdgpu_ring *ring) · 91 unmodified lines | ||
| 6570 | 6555 | case IP_VERSION(10, 3, 1): |
| 6571 | 6556 | case IP_VERSION(10, 3, 4): |
| 6572 | 6557 | case IP_VERSION(10, 3, 5): |
| 6558 | + case IP_VERSION(10, 3, 6): | |
| 6573 | 6559 | case IP_VERSION(10, 3, 3): |
| 6574 | 6560 | tmp = RREG32_SOC15(GC, 0, mmRLC_CP_SCHEDULERS_Sienna_Cichlid); |
| 6575 | 6561 | tmp &= 0xffffff00; |
| @@ -7300,6 +7286,8 @@ static bool gfx_v10_0_check_grbm_cam_remapping(struct amdgpu_device *adev) · 724 unmodified lines | ||
| 7300 | 7286 | break; |
| 7301 | 7287 | case IP_VERSION(10, 3, 1): |
| 7302 | 7288 | case IP_VERSION(10, 3, 3): |
| 7289 | + case IP_VERSION(10, 3, 6): | |
| 7290 | + case IP_VERSION(10, 3, 7): | |
| 7303 | 7291 | return true; |
| 7304 | 7292 | default: |
| 7305 | 7293 | data = RREG32_SOC15(GC, 0, mmVGT_ESGS_RING_SIZE); |
| @@ -7334,7 +7322,9 @@ static void gfx_v10_0_setup_grbm_cam_remapping(struct amdgpu_device *adev) · 28 unmodified lines | ||
| 7334 | 7322 | case IP_VERSION(10, 3, 1): |
| 7335 | 7323 | case IP_VERSION(10, 3, 4): |
| 7336 | 7324 | case IP_VERSION(10, 3, 5): |
| 7325 | + case IP_VERSION(10, 3, 6): | |
| 7337 | 7326 | case IP_VERSION(10, 3, 3): |
| 7327 | + case IP_VERSION(10, 3, 7): | |
| 7338 | 7328 | /* mmVGT_TF_RING_SIZE_UMD -> mmVGT_TF_RING_SIZE */ |
| 7339 | 7329 | data = (SOC15_REG_OFFSET(GC, 0, mmVGT_TF_RING_SIZE_UMD) << |
| 7340 | 7330 | GRBM_CAM_DATA__CAM_ADDR__SHIFT) | |
| @@ -7654,6 +7644,7 @@ static int gfx_v10_0_soft_reset(void *handle) · 313 unmodified lines | ||
| 7654 | 7644 | case IP_VERSION(10, 3, 1): |
| 7655 | 7645 | case IP_VERSION(10, 3, 4): |
| 7656 | 7646 | case IP_VERSION(10, 3, 5): |
| 7647 | + case IP_VERSION(10, 3, 6): | |
| 7657 | 7648 | case IP_VERSION(10, 3, 3): |
| 7658 | 7649 | if (REG_GET_FIELD(tmp, GRBM_STATUS2, RLC_BUSY_Sienna_Cichlid)) |
| 7659 | 7650 | grbm_soft_reset = REG_SET_FIELD(grbm_soft_reset, |
| @@ -7721,6 +7712,21 @@ static uint64_t gfx_v10_0_get_gpu_clock_counter(struct amdgpu_device *adev) · 61 unmodified lines | ||
| 7721 | 7712 | preempt_enable(); |
| 7722 | 7713 | clock = clock_lo | (clock_hi << 32ULL); |
| 7723 | 7714 | break; |
| 7715 | + case IP_VERSION(10, 3, 6): | |
| 7716 | + preempt_disable(); | |
| 7717 | + clock_hi = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6); | |
| 7718 | + clock_lo = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6); | |
| 7719 | + hi_check = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6); | |
| 7720 | + /* The SMUIO TSC clock frequency is 100MHz, which sets 32-bit carry over | |
| 7721 | + * roughly every 42 seconds. | |
| 7722 | + */ | |
| 7723 | + if (hi_check != clock_hi) { | |
| 7724 | + clock_lo = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6); | |
| 7725 | + clock_hi = hi_check; | |
| 7726 | + } | |
| 7727 | + preempt_enable(); | |
| 7728 | + clock = clock_lo | (clock_hi << 32ULL); | |
| 7729 | + break; | |
| 7724 | 7730 | default: |
| 7725 | 7731 | preempt_disable(); |
| 7726 | 7732 | clock_hi = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER); |
| @@ -7778,6 +7784,7 @@ static int gfx_v10_0_early_init(void *handle) · 51 unmodified lines | ||
| 7778 | 7784 | case IP_VERSION(10, 1, 1): |
| 7779 | 7785 | case IP_VERSION(10, 1, 2): |
| 7780 | 7786 | case IP_VERSION(10, 1, 3): |
| 7787 | + case IP_VERSION(10, 1, 4): | |
| 7781 | 7788 | adev->gfx.num_gfx_rings = GFX10_NUM_GFX_RINGS_NV1X; |
| 7782 | 7789 | break; |
| 7783 | 7790 | case IP_VERSION(10, 3, 0): |
| @@ -7785,7 +7792,9 @@ static int gfx_v10_0_early_init(void *handle) · 1 unmodified line | ||
| 7785 | 7792 | case IP_VERSION(10, 3, 1): |
| 7786 | 7793 | case IP_VERSION(10, 3, 4): |
| 7787 | 7794 | case IP_VERSION(10, 3, 5): |
| 7795 | + case IP_VERSION(10, 3, 6): | |
| 7788 | 7796 | case IP_VERSION(10, 3, 3): |
| 7797 | + case IP_VERSION(10, 3, 7): | |
| 7789 | 7798 | adev->gfx.num_gfx_rings = GFX10_NUM_GFX_RINGS_Sienna_Cichlid; |
| 7790 | 7799 | break; |
| 7791 | 7800 | default: |
| @@ -7801,6 +7810,9 @@ static int gfx_v10_0_early_init(void *handle) · 9 unmodified lines | ||
| 7801 | 7810 | gfx_v10_0_set_gds_init(adev); |
| 7802 | 7811 | gfx_v10_0_set_rlc_funcs(adev); |
| 7803 | 7812 | |
| 7813 | + /* init rlcg reg access ctrl */ | |
| 7814 | + gfx_v10_0_init_rlcg_reg_access_ctrl(adev); | |
| 7815 | + | |
| 7804 | 7816 | return 0; |
| 7805 | 7817 | } |
| 7806 | 7818 | |
| @@ -7843,6 +7855,7 @@ static void gfx_v10_0_set_safe_mode(struct amdgpu_device *adev) · 36 unmodified lines | ||
| 7843 | 7855 | case IP_VERSION(10, 3, 1): |
| 7844 | 7856 | case IP_VERSION(10, 3, 4): |
| 7845 | 7857 | case IP_VERSION(10, 3, 5): |
| 7858 | + case IP_VERSION(10, 3, 6): | |
| 7846 | 7859 | case IP_VERSION(10, 3, 3): |
| 7847 | 7860 | WREG32_SOC15(GC, 0, mmRLC_SAFE_MODE_Sienna_Cichlid, data); |
| 7848 | 7861 | |
| @@ -7879,6 +7892,7 @@ static void gfx_v10_0_unset_safe_mode(struct amdgpu_device *adev) · 30 unmodified lines | ||
| 7879 | 7892 | case IP_VERSION(10, 3, 1): |
| 7880 | 7893 | case IP_VERSION(10, 3, 4): |
| 7881 | 7894 | case IP_VERSION(10, 3, 5): |
| 7895 | + case IP_VERSION(10, 3, 6): | |
| 7882 | 7896 | case IP_VERSION(10, 3, 3): |
| 7883 | 7897 | WREG32_SOC15(GC, 0, mmRLC_SAFE_MODE_Sienna_Cichlid, data); |
| 7884 | 7898 | break; |
| @@ -8333,6 +8347,7 @@ static void gfx_v10_cntl_power_gating(struct amdgpu_device *adev, bool enable) · 448 unmodified lines | ||
| 8333 | 8347 | switch (adev->ip_versions[GC_HWIP][0]) { |
| 8334 | 8348 | case IP_VERSION(10, 3, 1): |
| 8335 | 8349 | case IP_VERSION(10, 3, 3): |
| 8350 | + case IP_VERSION(10, 3, 6): | |
| 8336 | 8351 | data = 0x4E20 & RLC_PG_DELAY_3__CGCG_ACTIVE_BEFORE_CGPG_MASK_Vangogh; |
| 8337 | 8352 | WREG32_SOC15(GC, 0, mmRLC_PG_DELAY_3, data); |
| 8338 | 8353 | break; |
| @@ -8377,8 +8392,6 @@ static const struct amdgpu_rlc_funcs gfx_v10_0_rlc_funcs_sriov = { · 38 unmodified lines | ||
| 8377 | 8392 | .reset = gfx_v10_0_rlc_reset, |
| 8378 | 8393 | .start = gfx_v10_0_rlc_start, |
| 8379 | 8394 | .update_spm_vmid = gfx_v10_0_update_spm_vmid, |
| 8380 | − .sriov_wreg = gfx_v10_sriov_wreg, | |
| 8381 | − .sriov_rreg = gfx_v10_sriov_rreg, | |
| 8382 | 8395 | .is_rlcg_access_range = gfx_v10_0_is_rlcg_access_range, |
| 8383 | 8396 | }; |
| 8384 | 8397 | |
| @@ -8403,6 +8416,7 @@ static int gfx_v10_0_set_powergating_state(void *handle, · 18 unmodified lines | ||
| 8403 | 8416 | break; |
| 8404 | 8417 | case IP_VERSION(10, 3, 1): |
| 8405 | 8418 | case IP_VERSION(10, 3, 3): |
| 8419 | + case IP_VERSION(10, 3, 6): | |
| 8406 | 8420 | gfx_v10_cntl_pg(adev, enable); |
| 8407 | 8421 | amdgpu_gfx_off_ctrl(adev, enable); |
| 8408 | 8422 | break; |
| @@ -8429,6 +8443,7 @@ static int gfx_v10_0_set_clockgating_state(void *handle, · 20 unmodified lines | ||
| 8429 | 8443 | case IP_VERSION(10, 3, 1): |
| 8430 | 8444 | case IP_VERSION(10, 3, 4): |
| 8431 | 8445 | case IP_VERSION(10, 3, 5): |
| 8446 | + case IP_VERSION(10, 3, 6): | |
| 8432 | 8447 | case IP_VERSION(10, 3, 3): |
| 8433 | 8448 | gfx_v10_0_update_gfx_clock_gating(adev, |
| 8434 | 8449 | state == AMD_CG_STATE_GATE); |
| @@ -9537,11 +9552,14 @@ static void gfx_v10_0_set_rlc_funcs(struct amdgpu_device *adev) · 1102 unmodified lines | ||
| 9537 | 9552 | case IP_VERSION(10, 1, 10): |
| 9538 | 9553 | case IP_VERSION(10, 1, 1): |
| 9539 | 9554 | case IP_VERSION(10, 1, 3): |
| 9555 | + case IP_VERSION(10, 1, 4): | |
| 9540 | 9556 | case IP_VERSION(10, 3, 2): |
| 9541 | 9557 | case IP_VERSION(10, 3, 1): |
| 9542 | 9558 | case IP_VERSION(10, 3, 4): |
| 9543 | 9559 | case IP_VERSION(10, 3, 5): |
| 9560 | + case IP_VERSION(10, 3, 6): | |
| 9544 | 9561 | case IP_VERSION(10, 3, 3): |
| 9562 | + case IP_VERSION(10, 3, 7): | |
| 9545 | 9563 | adev->gfx.rlc.funcs = &gfx_v10_0_rlc_funcs; |
| 9546 | 9564 | break; |
| 9547 | 9565 | case IP_VERSION(10, 1, 2): |
| @@ -9634,7 +9652,9 @@ static int gfx_v10_0_get_cu_info(struct amdgpu_device *adev, · 86 unmodified lines | ||
| 9634 | 9652 | for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) { |
| 9635 | 9653 | bitmap = i * adev->gfx.config.max_sh_per_se + j; |
| 9636 | 9654 | if (((adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 0)) || |
| 9637 | − (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3))) && | |
| 9655 | + (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) || | |
| 9656 | + (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 6)) || | |
| 9657 | + (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 7))) && | |
| 9638 | 9658 | ((gfx_v10_3_get_disabled_sa(adev) >> bitmap) & 1)) |
| 9639 | 9659 | continue; |
| 9640 | 9660 | mask = 1; |
| @@ -63,13 +63,6 @@ · 62 unmodified lines | ||
| 63 | 63 | #define mmGCEA_PROBE_MAP 0x070c |
| 64 | 64 | #define mmGCEA_PROBE_MAP_BASE_IDX 0 |
| 65 | 65 | |
| 66 | −#define GFX9_RLCG_GC_WRITE_OLD (0x8 << 28) | |
| 67 | −#define GFX9_RLCG_GC_WRITE (0x0 << 28) | |
| 68 | −#define GFX9_RLCG_GC_READ (0x1 << 28) | |
| 69 | −#define GFX9_RLCG_VFGATE_DISABLED 0x4000000 | |
| 70 | −#define GFX9_RLCG_WRONG_OPERATION_TYPE 0x2000000 | |
| 71 | −#define GFX9_RLCG_NOT_IN_RANGE 0x1000000 | |
| 72 | − | |
| 73 | 66 | MODULE_FIRMWARE("amdgpu/vega10_ce.bin"); |
| 74 | 67 | MODULE_FIRMWARE("amdgpu/vega10_pfp.bin"); |
| 75 | 68 | MODULE_FIRMWARE("amdgpu/vega10_me.bin"); |
| @@ -746,128 +739,6 @@ static const u32 GFX_RLC_SRM_INDEX_CNTL_DATA_OFFSETS[] = · 670 unmodified lines | ||
| 746 | 739 | mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0, |
| 747 | 740 | }; |
| 748 | 741 | |
| 749 | −static u32 gfx_v9_0_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag) | |
| 750 | −{ | |
| 751 | − static void *scratch_reg0; | |
| 752 | − static void *scratch_reg1; | |
| 753 | − static void *scratch_reg2; | |
| 754 | − static void *scratch_reg3; | |
| 755 | − static void *spare_int; | |
| 756 | − static uint32_t grbm_cntl; | |
| 757 | − static uint32_t grbm_idx; | |
| 758 | − uint32_t i = 0; | |
| 759 | − uint32_t retries = 50000; | |
| 760 | − u32 ret = 0; | |
| 761 | − u32 tmp; | |
| 762 | − | |
| 763 | − scratch_reg0 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0)*4; | |
| 764 | − scratch_reg1 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1)*4; | |
| 765 | − scratch_reg2 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG2_BASE_IDX] + mmSCRATCH_REG2)*4; | |
| 766 | − scratch_reg3 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG3_BASE_IDX] + mmSCRATCH_REG3)*4; | |
| 767 | − spare_int = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT)*4; | |
| 768 | − | |
| 769 | − grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL; | |
| 770 | − grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX; | |
| 771 | − | |
| 772 | − if (offset == grbm_cntl || offset == grbm_idx) { | |
| 773 | − if (offset == grbm_cntl) | |
| 774 | − writel(v, scratch_reg2); | |
| 775 | − else if (offset == grbm_idx) | |
| 776 | − writel(v, scratch_reg3); | |
| 777 | − | |
| 778 | − writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); | |
| 779 | − } else { | |
| 780 | − /* | |
| 781 | − * SCRATCH_REG0 = read/write value | |
| 782 | − * SCRATCH_REG1[30:28] = command | |
| 783 | − * SCRATCH_REG1[19:0] = address in dword | |
| 784 | − * SCRATCH_REG1[26:24] = Error reporting | |
| 785 | − */ | |
| 786 | − writel(v, scratch_reg0); | |
| 787 | − writel(offset | flag, scratch_reg1); | |
| 788 | − writel(1, spare_int); | |
| 789 | − | |
| 790 | − for (i = 0; i < retries; i++) { | |
| 791 | − tmp = readl(scratch_reg1); | |
| 792 | − if (!(tmp & flag)) | |
| 793 | − break; | |
| 794 | − | |
| 795 | − udelay(10); | |
| 796 | − } | |
| 797 | − | |
| 798 | − if (i >= retries) { | |
| 799 | − if (amdgpu_sriov_reg_indirect_gc(adev)) { | |
| 800 | − if (tmp & GFX9_RLCG_VFGATE_DISABLED) | |
| 801 | − pr_err("The vfgate is disabled, program reg:0x%05x failed!\n", offset); | |
| 802 | − else if (tmp & GFX9_RLCG_WRONG_OPERATION_TYPE) | |
| 803 | − pr_err("Wrong operation type, program reg:0x%05x failed!\n", offset); | |
| 804 | − else if (tmp & GFX9_RLCG_NOT_IN_RANGE) | |
| 805 | − pr_err("The register is not in range, program reg:0x%05x failed!\n", offset); | |
| 806 | − else | |
| 807 | − pr_err("Unknown error type, program reg:0x%05x failed!\n", offset); | |
| 808 | − } else | |
| 809 | − pr_err("timeout: rlcg program reg:0x%05x failed!\n", offset); | |
| 810 | − } | |
| 811 | − } | |
| 812 | − | |
| 813 | − ret = readl(scratch_reg0); | |
| 814 | − | |
| 815 | − return ret; | |
| 816 | −} | |
| 817 | − | |
| 818 | −static bool gfx_v9_0_get_rlcg_flag(struct amdgpu_device *adev, u32 acc_flags, u32 hwip, | |
| 819 | − int write, u32 *rlcg_flag) | |
| 820 | −{ | |
| 821 | − | |
| 822 | − switch (hwip) { | |
| 823 | − case GC_HWIP: | |
| 824 | − if (amdgpu_sriov_reg_indirect_gc(adev)) { | |
| 825 | − *rlcg_flag = write ? GFX9_RLCG_GC_WRITE : GFX9_RLCG_GC_READ; | |
| 826 | − | |
| 827 | − return true; | |
| 828 | − /* only in new version, AMDGPU_REGS_NO_KIQ and AMDGPU_REGS_RLC enabled simultaneously */ | |
| 829 | − } else if ((acc_flags & AMDGPU_REGS_RLC) && !(acc_flags & AMDGPU_REGS_NO_KIQ) && write) { | |
| 830 | − *rlcg_flag = GFX9_RLCG_GC_WRITE_OLD; | |
| 831 | − return true; | |
| 832 | − } | |
| 833 | − | |
| 834 | − break; | |
| 835 | − default: | |
| 836 | − return false; | |
| 837 | − } | |
| 838 | − | |
| 839 | − return false; | |
| 840 | −} | |
| 841 | − | |
| 842 | −static u32 gfx_v9_0_sriov_rreg(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip) | |
| 843 | −{ | |
| 844 | − u32 rlcg_flag; | |
| 845 | − | |
| 846 | − if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 0, &rlcg_flag)) | |
| 847 | − return gfx_v9_0_rlcg_rw(adev, offset, 0, rlcg_flag); | |
| 848 | − | |
| 849 | − if (acc_flags & AMDGPU_REGS_NO_KIQ) | |
| 850 | − return RREG32_NO_KIQ(offset); | |
| 851 | − else | |
| 852 | − return RREG32(offset); | |
| 853 | −} | |
| 854 | − | |
| 855 | −static void gfx_v9_0_sriov_wreg(struct amdgpu_device *adev, u32 offset, | |
| 856 | − u32 value, u32 acc_flags, u32 hwip) | |
| 857 | −{ | |
| 858 | − u32 rlcg_flag; | |
| 859 | − | |
| 860 | − if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 1, &rlcg_flag)) { | |
| 861 | − gfx_v9_0_rlcg_rw(adev, offset, value, rlcg_flag); | |
| 862 | − return; | |
| 863 | − } | |
| 864 | − | |
| 865 | − if (acc_flags & AMDGPU_REGS_NO_KIQ) | |
| 866 | − WREG32_NO_KIQ(offset, value); | |
| 867 | − else | |
| 868 | − WREG32(offset, value); | |
| 869 | −} | |
| 870 | − | |
| 871 | 742 | #define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042 |
| 872 | 743 | #define VEGA12_GB_ADDR_CONFIG_GOLDEN 0x24104041 |
| 873 | 744 | #define RAVEN_GB_ADDR_CONFIG_GOLDEN 0x24000042 |
| @@ -882,7 +753,7 @@ static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev, · 8 unmodified lines | ||
| 882 | 753 | static uint64_t gfx_v9_0_get_gpu_clock_counter(struct amdgpu_device *adev); |
| 883 | 754 | static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring); |
| 884 | 755 | static u64 gfx_v9_0_ring_get_rptr_compute(struct amdgpu_ring *ring); |
| 885 | −static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, | |
| 756 | +static void gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, | |
| 886 | 757 | void *ras_error_status); |
| 887 | 758 | static int gfx_v9_0_ras_error_inject(struct amdgpu_device *adev, |
| 888 | 759 | void *inject_if); |
| @@ -2008,6 +1879,21 @@ static int gfx_v9_0_cp_jump_table_num(struct amdgpu_device *adev) · 1119 unmodified lines | ||
| 2008 | 1879 | return 4; |
| 2009 | 1880 | } |
| 2010 | 1881 | |
| 1882 | +static void gfx_v9_0_init_rlcg_reg_access_ctrl(struct amdgpu_device *adev) | |
| 1883 | +{ | |
| 1884 | + struct amdgpu_rlcg_reg_access_ctrl *reg_access_ctrl; | |
| 1885 | + | |
| 1886 | + reg_access_ctrl = &adev->gfx.rlc.reg_access_ctrl; | |
| 1887 | + reg_access_ctrl->scratch_reg0 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG0); | |
| 1888 | + reg_access_ctrl->scratch_reg1 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG1); | |
| 1889 | + reg_access_ctrl->scratch_reg2 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG2); | |
| 1890 | + reg_access_ctrl->scratch_reg3 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG3); | |
| 1891 | + reg_access_ctrl->grbm_cntl = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_CNTL); | |
| 1892 | + reg_access_ctrl->grbm_idx = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX); | |
| 1893 | + reg_access_ctrl->spare_int = SOC15_REG_OFFSET(GC, 0, mmRLC_SPARE_INT); | |
| 1894 | + adev->gfx.rlc.rlcg_reg_access_supported = true; | |
| 1895 | +} | |
| 1896 | + | |
| 2011 | 1897 | static int gfx_v9_0_rlc_init(struct amdgpu_device *adev) |
| 2012 | 1898 | { |
| 2013 | 1899 | const struct cs_section_def *cs_data; |
| @@ -2197,12 +2083,16 @@ static const struct amdgpu_gfx_funcs gfx_v9_0_gfx_funcs = { · 183 unmodified lines | ||
| 2197 | 2083 | .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q, |
| 2198 | 2084 | }; |
| 2199 | 2085 | |
| 2200 | −static const struct amdgpu_gfx_ras_funcs gfx_v9_0_ras_funcs = { | |
| 2201 | − .ras_late_init = amdgpu_gfx_ras_late_init, | |
| 2202 | − .ras_fini = amdgpu_gfx_ras_fini, | |
| 2203 | − .ras_error_inject = &gfx_v9_0_ras_error_inject, | |
| 2204 | − .query_ras_error_count = &gfx_v9_0_query_ras_error_count, | |
| 2205 | − .reset_ras_error_count = &gfx_v9_0_reset_ras_error_count, | |
| 2086 | +const struct amdgpu_ras_block_hw_ops gfx_v9_0_ras_ops = { | |
| 2087 | + .ras_error_inject = &gfx_v9_0_ras_error_inject, | |
| 2088 | + .query_ras_error_count = &gfx_v9_0_query_ras_error_count, | |
| 2089 | + .reset_ras_error_count = &gfx_v9_0_reset_ras_error_count, | |
| 2090 | +}; | |
| 2091 | + | |
| 2092 | +static struct amdgpu_gfx_ras gfx_v9_0_ras = { | |
| 2093 | + .ras_block = { | |
| 2094 | + .hw_ops = &gfx_v9_0_ras_ops, | |
| 2095 | + }, | |
| 2206 | 2096 | }; |
| 2207 | 2097 | |
| 2208 | 2098 | static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) |
| @@ -2231,7 +2121,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 22 unmodified lines | ||
| 2231 | 2121 | DRM_INFO("fix gfx.config for vega12\n"); |
| 2232 | 2122 | break; |
| 2233 | 2123 | case IP_VERSION(9, 4, 0): |
| 2234 | − adev->gfx.ras_funcs = &gfx_v9_0_ras_funcs; | |
| 2124 | + adev->gfx.ras = &gfx_v9_0_ras; | |
| 2235 | 2125 | adev->gfx.config.max_hw_contexts = 8; |
| 2236 | 2126 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 2237 | 2127 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -2258,7 +2148,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 20 unmodified lines | ||
| 2258 | 2148 | gb_addr_config = RAVEN_GB_ADDR_CONFIG_GOLDEN; |
| 2259 | 2149 | break; |
| 2260 | 2150 | case IP_VERSION(9, 4, 1): |
| 2261 | − adev->gfx.ras_funcs = &gfx_v9_4_ras_funcs; | |
| 2151 | + adev->gfx.ras = &gfx_v9_4_ras; | |
| 2262 | 2152 | adev->gfx.config.max_hw_contexts = 8; |
| 2263 | 2153 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 2264 | 2154 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -2279,7 +2169,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 14 unmodified lines | ||
| 2279 | 2169 | gb_addr_config |= 0x22010042; |
| 2280 | 2170 | break; |
| 2281 | 2171 | case IP_VERSION(9, 4, 2): |
| 2282 | − adev->gfx.ras_funcs = &gfx_v9_4_2_ras_funcs; | |
| 2172 | + adev->gfx.ras = &gfx_v9_4_2_ras; | |
| 2283 | 2173 | adev->gfx.config.max_hw_contexts = 8; |
| 2284 | 2174 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 2285 | 2175 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -2298,6 +2188,31 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 12 unmodified lines | ||
| 2298 | 2188 | break; |
| 2299 | 2189 | } |
| 2300 | 2190 | |
| 2191 | + if (adev->gfx.ras) { | |
| 2192 | + err = amdgpu_ras_register_ras_block(adev, &adev->gfx.ras->ras_block); | |
| 2193 | + if (err) { | |
| 2194 | + DRM_ERROR("Failed to register gfx ras block!\n"); | |
| 2195 | + return err; | |
| 2196 | + } | |
| 2197 | + | |
| 2198 | + strcpy(adev->gfx.ras->ras_block.ras_comm.name, "gfx"); | |
| 2199 | + adev->gfx.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__GFX; | |
| 2200 | + adev->gfx.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 2201 | + adev->gfx.ras_if = &adev->gfx.ras->ras_block.ras_comm; | |
| 2202 | + | |
| 2203 | + /* If not define special ras_late_init function, use gfx default ras_late_init */ | |
| 2204 | + if (!adev->gfx.ras->ras_block.ras_late_init) | |
| 2205 | + adev->gfx.ras->ras_block.ras_late_init = amdgpu_gfx_ras_late_init; | |
| 2206 | + | |
| 2207 | + /* If not define special ras_fini function, use gfx default ras_fini */ | |
| 2208 | + if (!adev->gfx.ras->ras_block.ras_fini) | |
| 2209 | + adev->gfx.ras->ras_block.ras_fini = amdgpu_gfx_ras_fini; | |
| 2210 | + | |
| 2211 | + /* If not defined special ras_cb function, use default ras_cb */ | |
| 2212 | + if (!adev->gfx.ras->ras_block.ras_cb) | |
| 2213 | + adev->gfx.ras->ras_block.ras_cb = amdgpu_gfx_process_ras_data_cb; | |
| 2214 | + } | |
| 2215 | + | |
| 2301 | 2216 | adev->gfx.config.gb_addr_config = gb_addr_config; |
| 2302 | 2217 | |
| 2303 | 2218 | adev->gfx.config.gb_addr_config_fields.num_pipes = 1 << |
| @@ -2434,10 +2349,14 @@ static int gfx_v9_0_sw_init(void *handle) · 130 unmodified lines | ||
| 2434 | 2349 | return r; |
| 2435 | 2350 | } |
| 2436 | 2351 | |
| 2437 | − r = adev->gfx.rlc.funcs->init(adev); | |
| 2438 | − if (r) { | |
| 2439 | − DRM_ERROR("Failed to init rlc BOs!\n"); | |
| 2440 | − return r; | |
| 2352 | + if (adev->gfx.rlc.funcs) { | |
| 2353 | + if (adev->gfx.rlc.funcs->init) { | |
| 2354 | + r = adev->gfx.rlc.funcs->init(adev); | |
| 2355 | + if (r) { | |
| 2356 | + dev_err(adev->dev, "Failed to init rlc BOs!\n"); | |
| 2357 | + return r; | |
| 2358 | + } | |
| 2359 | + } | |
| 2441 | 2360 | } |
| 2442 | 2361 | |
| 2443 | 2362 | r = gfx_v9_0_mec_init(adev); |
| @@ -2513,9 +2432,8 @@ static int gfx_v9_0_sw_fini(void *handle) · 69 unmodified lines | ||
| 2513 | 2432 | int i; |
| 2514 | 2433 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 2515 | 2434 | |
| 2516 | − if (adev->gfx.ras_funcs && | |
| 2517 | − adev->gfx.ras_funcs->ras_fini) | |
| 2518 | − adev->gfx.ras_funcs->ras_fini(adev); | |
| 2435 | + if (adev->gfx.ras && adev->gfx.ras->ras_block.ras_fini) | |
| 2436 | + adev->gfx.ras->ras_block.ras_fini(adev); | |
| 2519 | 2437 | |
| 2520 | 2438 | for (i = 0; i < adev->gfx.num_gfx_rings; i++) |
| 2521 | 2439 | amdgpu_ring_fini(&adev->gfx.gfx_ring[i]); |
| @@ -4840,6 +4758,9 @@ static int gfx_v9_0_early_init(void *handle) · 2318 unmodified lines | ||
| 4840 | 4758 | gfx_v9_0_set_gds_init(adev); |
| 4841 | 4759 | gfx_v9_0_set_rlc_funcs(adev); |
| 4842 | 4760 | |
| 4761 | + /* init rlcg reg access ctrl */ | |
| 4762 | + gfx_v9_0_init_rlcg_reg_access_ctrl(adev); | |
| 4763 | + | |
| 4843 | 4764 | return 0; |
| 4844 | 4765 | } |
| 4845 | 4766 | |
| @@ -4870,16 +4791,9 @@ static int gfx_v9_0_ecc_late_init(void *handle) · 24 unmodified lines | ||
| 4870 | 4791 | if (r) |
| 4871 | 4792 | return r; |
| 4872 | 4793 | |
| 4873 | − if (adev->gfx.ras_funcs && | |
| 4874 | − adev->gfx.ras_funcs->ras_late_init) { | |
| 4875 | − r = adev->gfx.ras_funcs->ras_late_init(adev); | |
| 4876 | − if (r) | |
| 4877 | − return r; | |
| 4878 | − } | |
| 4879 | − | |
| 4880 | − if (adev->gfx.ras_funcs && | |
| 4881 | − adev->gfx.ras_funcs->enable_watchdog_timer) | |
| 4882 | − adev->gfx.ras_funcs->enable_watchdog_timer(adev); | |
| 4794 | + if (adev->gfx.ras && | |
| 4795 | + adev->gfx.ras->enable_watchdog_timer) | |
| 4796 | + adev->gfx.ras->enable_watchdog_timer(adev); | |
| 4883 | 4797 | |
| 4884 | 4798 | return 0; |
| 4885 | 4799 | } |
| @@ -5250,8 +5164,6 @@ static const struct amdgpu_rlc_funcs gfx_v9_0_rlc_funcs = { · 364 unmodified lines | ||
| 5250 | 5164 | .reset = gfx_v9_0_rlc_reset, |
| 5251 | 5165 | .start = gfx_v9_0_rlc_start, |
| 5252 | 5166 | .update_spm_vmid = gfx_v9_0_update_spm_vmid, |
| 5253 | − .sriov_wreg = gfx_v9_0_sriov_wreg, | |
| 5254 | − .sriov_rreg = gfx_v9_0_sriov_rreg, | |
| 5255 | 5167 | .is_rlcg_access_range = gfx_v9_0_is_rlcg_access_range, |
| 5256 | 5168 | }; |
| 5257 | 5169 | |
| @@ -6819,7 +6731,7 @@ static void gfx_v9_0_reset_ras_error_count(struct amdgpu_device *adev) · 1561 unmodified lines | ||
| 6819 | 6731 | WREG32_SOC15(GC, 0, mmATC_L2_CACHE_4K_EDC_INDEX, 255); |
| 6820 | 6732 | } |
| 6821 | 6733 | |
| 6822 | −static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, | |
| 6734 | +static void gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, | |
| 6823 | 6735 | void *ras_error_status) |
| 6824 | 6736 | { |
| 6825 | 6737 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| @@ -6828,7 +6740,7 @@ static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 6828 | 6740 | uint32_t reg_value; |
| 6829 | 6741 | |
| 6830 | 6742 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) |
| 6831 | − return -EINVAL; | |
| 6743 | + return; | |
| 6832 | 6744 | |
| 6833 | 6745 | err_data->ue_count = 0; |
| 6834 | 6746 | err_data->ce_count = 0; |
| @@ -6857,8 +6769,6 @@ static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, · 22 unmodified lines | ||
| 6857 | 6769 | mutex_unlock(&adev->grbm_idx_mutex); |
| 6858 | 6770 | |
| 6859 | 6771 | gfx_v9_0_query_utc_edc_status(adev, err_data); |
| 6860 | − | |
| 6861 | − return 0; | |
| 6862 | 6772 | } |
| 6863 | 6773 | |
| 6864 | 6774 | static void gfx_v9_0_emit_mem_sync(struct amdgpu_ring *ring) |
| @@ -863,7 +863,7 @@ static int gfx_v9_4_ras_error_count(struct amdgpu_device *adev, · 862 unmodified lines | ||
| 863 | 863 | return 0; |
| 864 | 864 | } |
| 865 | 865 | |
| 866 | −static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, | |
| 866 | +static void gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, | |
| 867 | 867 | void *ras_error_status) |
| 868 | 868 | { |
| 869 | 869 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| @@ -872,7 +872,7 @@ static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 872 | 872 | uint32_t reg_value; |
| 873 | 873 | |
| 874 | 874 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) |
| 875 | − return -EINVAL; | |
| 875 | + return; | |
| 876 | 876 | |
| 877 | 877 | err_data->ue_count = 0; |
| 878 | 878 | err_data->ce_count = 0; |
| @@ -903,7 +903,6 @@ static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, · 24 unmodified lines | ||
| 903 | 903 | |
| 904 | 904 | gfx_v9_4_query_utc_edc_status(adev, err_data); |
| 905 | 905 | |
| 906 | − return 0; | |
| 907 | 906 | } |
| 908 | 907 | |
| 909 | 908 | static void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev) |
| @@ -1029,11 +1028,16 @@ static void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev) · 119 unmodified lines | ||
| 1029 | 1028 | mutex_unlock(&adev->grbm_idx_mutex); |
| 1030 | 1029 | } |
| 1031 | 1030 | |
| 1032 | −const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs = { | |
| 1033 | − .ras_late_init = amdgpu_gfx_ras_late_init, | |
| 1034 | − .ras_fini = amdgpu_gfx_ras_fini, | |
| 1035 | − .ras_error_inject = &gfx_v9_4_ras_error_inject, | |
| 1036 | − .query_ras_error_count = &gfx_v9_4_query_ras_error_count, | |
| 1037 | − .reset_ras_error_count = &gfx_v9_4_reset_ras_error_count, | |
| 1038 | − .query_ras_error_status = &gfx_v9_4_query_ras_error_status, | |
| 1031 | + | |
| 1032 | +const struct amdgpu_ras_block_hw_ops gfx_v9_4_ras_ops = { | |
| 1033 | + .ras_error_inject = &gfx_v9_4_ras_error_inject, | |
| 1034 | + .query_ras_error_count = &gfx_v9_4_query_ras_error_count, | |
| 1035 | + .reset_ras_error_count = &gfx_v9_4_reset_ras_error_count, | |
| 1036 | + .query_ras_error_status = &gfx_v9_4_query_ras_error_status, | |
| 1037 | +}; | |
| 1038 | + | |
| 1039 | +struct amdgpu_gfx_ras gfx_v9_4_ras = { | |
| 1040 | + .ras_block = { | |
| 1041 | + .hw_ops = &gfx_v9_4_ras_ops, | |
| 1042 | + }, | |
| 1039 | 1043 | }; |
| @@ -24,6 +24,6 @@ · 23 unmodified lines | ||
| 24 | 24 | #ifndef __GFX_V9_4_H__ |
| 25 | 25 | #define __GFX_V9_4_H__ |
| 26 | 26 | |
| 27 | −extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs; | |
| 27 | +extern struct amdgpu_gfx_ras gfx_v9_4_ras; | |
| 28 | 28 | |
| 29 | 29 | #endif /* __GFX_V9_4_H__ */ |
| @@ -1641,14 +1641,14 @@ static int gfx_v9_4_2_query_utc_edc_count(struct amdgpu_device *adev, · 1640 unmodified lines | ||
| 1641 | 1641 | return 0; |
| 1642 | 1642 | } |
| 1643 | 1643 | |
| 1644 | −static int gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev, | |
| 1644 | +static void gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev, | |
| 1645 | 1645 | void *ras_error_status) |
| 1646 | 1646 | { |
| 1647 | 1647 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| 1648 | 1648 | uint32_t sec_count = 0, ded_count = 0; |
| 1649 | 1649 | |
| 1650 | 1650 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) |
| 1651 | − return -EINVAL; | |
| 1651 | + return; | |
| 1652 | 1652 | |
| 1653 | 1653 | err_data->ue_count = 0; |
| 1654 | 1654 | err_data->ce_count = 0; |
| @@ -1661,7 +1661,6 @@ static int gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev, · 6 unmodified lines | ||
| 1661 | 1661 | err_data->ce_count += sec_count; |
| 1662 | 1662 | err_data->ue_count += ded_count; |
| 1663 | 1663 | |
| 1664 | − return 0; | |
| 1665 | 1664 | } |
| 1666 | 1665 | |
| 1667 | 1666 | static void gfx_v9_4_2_reset_utc_err_status(struct amdgpu_device *adev) |
| @@ -1931,13 +1930,17 @@ static void gfx_v9_4_2_reset_sq_timeout_status(struct amdgpu_device *adev) · 263 unmodified lines | ||
| 1931 | 1930 | mutex_unlock(&adev->grbm_idx_mutex); |
| 1932 | 1931 | } |
| 1933 | 1932 | |
| 1934 | −const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs = { | |
| 1935 | − .ras_late_init = amdgpu_gfx_ras_late_init, | |
| 1936 | − .ras_fini = amdgpu_gfx_ras_fini, | |
| 1937 | − .ras_error_inject = &gfx_v9_4_2_ras_error_inject, | |
| 1938 | − .query_ras_error_count = &gfx_v9_4_2_query_ras_error_count, | |
| 1939 | − .reset_ras_error_count = &gfx_v9_4_2_reset_ras_error_count, | |
| 1940 | − .query_ras_error_status = &gfx_v9_4_2_query_ras_error_status, | |
| 1941 | − .reset_ras_error_status = &gfx_v9_4_2_reset_ras_error_status, | |
| 1933 | +struct amdgpu_ras_block_hw_ops gfx_v9_4_2_ras_ops = { | |
| 1934 | + .ras_error_inject = &gfx_v9_4_2_ras_error_inject, | |
| 1935 | + .query_ras_error_count = &gfx_v9_4_2_query_ras_error_count, | |
| 1936 | + .reset_ras_error_count = &gfx_v9_4_2_reset_ras_error_count, | |
| 1937 | + .query_ras_error_status = &gfx_v9_4_2_query_ras_error_status, | |
| 1938 | + .reset_ras_error_status = &gfx_v9_4_2_reset_ras_error_status, | |
| 1939 | +}; | |
| 1940 | + | |
| 1941 | +struct amdgpu_gfx_ras gfx_v9_4_2_ras = { | |
| 1942 | + .ras_block = { | |
| 1943 | + .hw_ops = &gfx_v9_4_2_ras_ops, | |
| 1944 | + }, | |
| 1942 | 1945 | .enable_watchdog_timer = &gfx_v9_4_2_enable_watchdog_timer, |
| 1943 | 1946 | }; |
| @@ -31,6 +31,6 @@ void gfx_v9_4_2_init_golden_registers(struct amdgpu_device *adev, · 30 unmodified lines | ||
| 31 | 31 | void gfx_v9_4_2_set_power_brake_sequence(struct amdgpu_device *adev); |
| 32 | 32 | int gfx_v9_4_2_do_edc_gpr_workarounds(struct amdgpu_device *adev); |
| 33 | 33 | |
| 34 | −extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs; | |
| 34 | +extern struct amdgpu_gfx_ras gfx_v9_4_2_ras; | |
| 35 | 35 | |
| 36 | 36 | #endif /* __GFX_V9_4_2_H__ */ |
| @@ -543,7 +543,9 @@ static void gfxhub_v2_1_utcl2_harvest(struct amdgpu_device *adev) · 542 unmodified lines | ||
| 543 | 543 | adev->gfx.config.max_sh_per_se * |
| 544 | 544 | adev->gfx.config.max_shader_engines); |
| 545 | 545 | |
| 546 | − if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) { | |
| 546 | + switch (adev->ip_versions[GC_HWIP][0]) { | |
| 547 | + case IP_VERSION(10, 3, 1): | |
| 548 | + case IP_VERSION(10, 3, 3): | |
| 547 | 549 | /* Get SA disabled bitmap from eFuse setting */ |
| 548 | 550 | efuse_setting = RREG32_SOC15(GC, 0, mmCC_GC_SA_UNIT_DISABLE); |
| 549 | 551 | efuse_setting &= CC_GC_SA_UNIT_DISABLE__SA_DISABLE_MASK; |
| @@ -566,6 +568,9 @@ static void gfxhub_v2_1_utcl2_harvest(struct amdgpu_device *adev) · 16 unmodified lines | ||
| 566 | 568 | disabled_sa = tmp; |
| 567 | 569 | |
| 568 | 570 | WREG32_SOC15(GC, 0, mmGCUTCL2_HARVEST_BYPASS_GROUPS_YELLOW_CARP, disabled_sa); |
| 571 | + break; | |
| 572 | + default: | |
| 573 | + break; | |
| 569 | 574 | } |
| 570 | 575 | } |
| 571 | 576 | |
| @@ -48,6 +48,8 @@ · 47 unmodified lines | ||
| 48 | 48 | #include "athub_v2_0.h" |
| 49 | 49 | #include "athub_v2_1.h" |
| 50 | 50 | |
| 51 | +#include "amdgpu_reset.h" | |
| 52 | + | |
| 51 | 53 | #if 0 |
| 52 | 54 | static const struct soc15_reg_golden golden_settings_navi10_hdp[] = |
| 53 | 55 | { |
| @@ -328,7 +330,7 @@ static void gmc_v10_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 274 unmodified lines | ||
| 328 | 330 | */ |
| 329 | 331 | if (adev->gfx.kiq.ring.sched.ready && |
| 330 | 332 | (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev)) && |
| 331 | − down_read_trylock(&adev->reset_sem)) { | |
| 333 | + down_read_trylock(&adev->reset_domain->sem)) { | |
| 332 | 334 | struct amdgpu_vmhub *hub = &adev->vmhub[vmhub]; |
| 333 | 335 | const unsigned eng = 17; |
| 334 | 336 | u32 inv_req = hub->vmhub_funcs->get_invalidate_req(vmid, flush_type); |
| @@ -338,7 +340,7 @@ static void gmc_v10_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 3 unmodified lines | ||
| 338 | 340 | amdgpu_virt_kiq_reg_write_reg_wait(adev, req, ack, inv_req, |
| 339 | 341 | 1 << vmid); |
| 340 | 342 | |
| 341 | − up_read(&adev->reset_sem); | |
| 343 | + up_read(&adev->reset_domain->sem); | |
| 342 | 344 | return; |
| 343 | 345 | } |
| 344 | 346 | |
| @@ -664,11 +666,31 @@ static void gmc_v10_0_set_umc_funcs(struct amdgpu_device *adev) · 319 unmodified lines | ||
| 664 | 666 | adev->umc.umc_inst_num = UMC_V8_7_UMC_INSTANCE_NUM; |
| 665 | 667 | adev->umc.channel_offs = UMC_V8_7_PER_CHANNEL_OFFSET_SIENNA; |
| 666 | 668 | adev->umc.channel_idx_tbl = &umc_v8_7_channel_idx_tbl[0][0]; |
| 667 | − adev->umc.ras_funcs = &umc_v8_7_ras_funcs; | |
| 669 | + adev->umc.ras = &umc_v8_7_ras; | |
| 668 | 670 | break; |
| 669 | 671 | default: |
| 670 | 672 | break; |
| 671 | 673 | } |
| 674 | + if (adev->umc.ras) { | |
| 675 | + amdgpu_ras_register_ras_block(adev, &adev->umc.ras->ras_block); | |
| 676 | + | |
| 677 | + strcpy(adev->umc.ras->ras_block.ras_comm.name, "umc"); | |
| 678 | + adev->umc.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__UMC; | |
| 679 | + adev->umc.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 680 | + adev->umc.ras_if = &adev->umc.ras->ras_block.ras_comm; | |
| 681 | + | |
| 682 | + /* If don't define special ras_late_init function, use default ras_late_init */ | |
| 683 | + if (!adev->umc.ras->ras_block.ras_late_init) | |
| 684 | + adev->umc.ras->ras_block.ras_late_init = amdgpu_umc_ras_late_init; | |
| 685 | + | |
| 686 | + /* If don't define special ras_fini function, use default ras_fini */ | |
| 687 | + if (!adev->umc.ras->ras_block.ras_fini) | |
| 688 | + adev->umc.ras->ras_block.ras_fini = amdgpu_umc_ras_fini; | |
| 689 | + | |
| 690 | + /* If not defined special ras_cb function, use default ras_cb */ | |
| 691 | + if (!adev->umc.ras->ras_block.ras_cb) | |
| 692 | + adev->umc.ras->ras_block.ras_cb = amdgpu_umc_process_ras_data_cb; | |
| 693 | + } | |
| 672 | 694 | } |
| 673 | 695 | |
| 674 | 696 | |
| @@ -677,6 +699,7 @@ static void gmc_v10_0_set_mmhub_funcs(struct amdgpu_device *adev) · 2 unmodified lines | ||
| 677 | 699 | switch (adev->ip_versions[MMHUB_HWIP][0]) { |
| 678 | 700 | case IP_VERSION(2, 3, 0): |
| 679 | 701 | case IP_VERSION(2, 4, 0): |
| 702 | + case IP_VERSION(2, 4, 1): | |
| 680 | 703 | adev->mmhub.funcs = &mmhub_v2_3_funcs; |
| 681 | 704 | break; |
| 682 | 705 | default: |
| @@ -693,7 +716,9 @@ static void gmc_v10_0_set_gfxhub_funcs(struct amdgpu_device *adev) · 10 unmodified lines | ||
| 693 | 716 | case IP_VERSION(10, 3, 1): |
| 694 | 717 | case IP_VERSION(10, 3, 4): |
| 695 | 718 | case IP_VERSION(10, 3, 5): |
| 719 | + case IP_VERSION(10, 3, 6): | |
| 696 | 720 | case IP_VERSION(10, 3, 3): |
| 721 | + case IP_VERSION(10, 3, 7): | |
| 697 | 722 | adev->gfxhub.funcs = &gfxhub_v2_1_funcs; |
| 698 | 723 | break; |
| 699 | 724 | default: |
| @@ -705,6 +730,7 @@ static void gmc_v10_0_set_gfxhub_funcs(struct amdgpu_device *adev) · 5 unmodified lines | ||
| 705 | 730 | |
| 706 | 731 | static int gmc_v10_0_early_init(void *handle) |
| 707 | 732 | { |
| 733 | + int r; | |
| 708 | 734 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 709 | 735 | |
| 710 | 736 | gmc_v10_0_set_mmhub_funcs(adev); |
| @@ -720,6 +746,10 @@ static int gmc_v10_0_early_init(void *handle) · 9 unmodified lines | ||
| 720 | 746 | adev->gmc.private_aperture_end = |
| 721 | 747 | adev->gmc.private_aperture_start + (4ULL << 30) - 1; |
| 722 | 748 | |
| 749 | + r = amdgpu_gmc_ras_early_init(adev); | |
| 750 | + if (r) | |
| 751 | + return r; | |
| 752 | + | |
| 723 | 753 | return 0; |
| 724 | 754 | } |
| 725 | 755 | |
| @@ -862,12 +892,15 @@ static int gmc_v10_0_sw_init(void *handle) · 136 unmodified lines | ||
| 862 | 892 | case IP_VERSION(10, 1, 1): |
| 863 | 893 | case IP_VERSION(10, 1, 2): |
| 864 | 894 | case IP_VERSION(10, 1, 3): |
| 895 | + case IP_VERSION(10, 1, 4): | |
| 865 | 896 | case IP_VERSION(10, 3, 0): |
| 866 | 897 | case IP_VERSION(10, 3, 2): |
| 867 | 898 | case IP_VERSION(10, 3, 1): |
| 868 | 899 | case IP_VERSION(10, 3, 4): |
| 869 | 900 | case IP_VERSION(10, 3, 5): |
| 901 | + case IP_VERSION(10, 3, 6): | |
| 870 | 902 | case IP_VERSION(10, 3, 3): |
| 903 | + case IP_VERSION(10, 3, 7): | |
| 871 | 904 | adev->num_vmhubs = 2; |
| 872 | 905 | /* |
| 873 | 906 | * To fulfill 4-level page support, |
| @@ -986,14 +1019,7 @@ static int gmc_v10_0_gart_enable(struct amdgpu_device *adev) · 112 unmodified lines | ||
| 986 | 1019 | return -EINVAL; |
| 987 | 1020 | } |
| 988 | 1021 | |
| 989 | − if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev)) | |
| 990 | − goto skip_pin_bo; | |
| 991 | − | |
| 992 | − r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 993 | − if (r) | |
| 994 | − return r; | |
| 995 | − | |
| 996 | −skip_pin_bo: | |
| 1022 | + amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 997 | 1023 | r = adev->gfxhub.funcs->gart_enable(adev); |
| 998 | 1024 | if (r) |
| 999 | 1025 | return r; |
| @@ -1019,8 +1045,6 @@ skip_pin_bo: · 19 unmodified lines | ||
| 1019 | 1045 | (unsigned)(adev->gmc.gart_size >> 20), |
| 1020 | 1046 | (unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo)); |
| 1021 | 1047 | |
| 1022 | − adev->gart.ready = true; | |
| 1023 | − | |
| 1024 | 1048 | return 0; |
| 1025 | 1049 | } |
| 1026 | 1050 | |
| @@ -1043,6 +1067,12 @@ static int gmc_v10_0_hw_init(void *handle) · 16 unmodified lines | ||
| 1043 | 1067 | if (r) |
| 1044 | 1068 | return r; |
| 1045 | 1069 | |
| 1070 | + if (amdgpu_emu_mode == 1) { | |
| 1071 | + r = amdgpu_gmc_vram_checking(adev); | |
| 1072 | + if (r) | |
| 1073 | + return r; | |
| 1074 | + } | |
| 1075 | + | |
| 1046 | 1076 | if (adev->umc.funcs && adev->umc.funcs->init_registers) |
| 1047 | 1077 | adev->umc.funcs->init_registers(adev); |
| 1048 | 1078 | |
| @@ -1140,6 +1170,10 @@ static void gmc_v10_0_get_clockgating_state(void *handle, u32 *flags) · 91 unmodified lines | ||
| 1140 | 1170 | { |
| 1141 | 1171 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1142 | 1172 | |
| 1173 | + if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 1, 3) || | |
| 1174 | + adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 1, 4)) | |
| 1175 | + return; | |
| 1176 | + | |
| 1143 | 1177 | adev->mmhub.funcs->get_clockgating(adev, flags); |
| 1144 | 1178 | |
| 1145 | 1179 | if (adev->ip_versions[ATHUB_HWIP][0] >= IP_VERSION(2, 1, 0)) |
| @@ -469,16 +469,14 @@ static void gmc_v6_0_set_prt(struct amdgpu_device *adev, bool enable) · 468 unmodified lines | ||
| 469 | 469 | static int gmc_v6_0_gart_enable(struct amdgpu_device *adev) |
| 470 | 470 | { |
| 471 | 471 | uint64_t table_addr; |
| 472 | − int r, i; | |
| 473 | 472 | u32 field; |
| 473 | + int i; | |
| 474 | 474 | |
| 475 | 475 | if (adev->gart.bo == NULL) { |
| 476 | 476 | dev_err(adev->dev, "No VRAM object for PCIE GART.\n"); |
| 477 | 477 | return -EINVAL; |
| 478 | 478 | } |
| 479 | − r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 480 | − if (r) | |
| 481 | − return r; | |
| 479 | + amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 482 | 480 | |
| 483 | 481 | table_addr = amdgpu_bo_gpu_offset(adev->gart.bo); |
| 484 | 482 | |
| @@ -558,7 +556,6 @@ static int gmc_v6_0_gart_enable(struct amdgpu_device *adev) · 73 unmodified lines | ||
| 558 | 556 | dev_info(adev->dev, "PCIE GART of %uM enabled (table at 0x%016llX).\n", |
| 559 | 557 | (unsigned)(adev->gmc.gart_size >> 20), |
| 560 | 558 | (unsigned long long)table_addr); |
| 561 | − adev->gart.ready = true; | |
| 562 | 559 | return 0; |
| 563 | 560 | } |
| 564 | 561 | |
| @@ -922,7 +919,10 @@ static int gmc_v6_0_hw_init(void *handle) · 357 unmodified lines | ||
| 922 | 919 | if (r) |
| 923 | 920 | return r; |
| 924 | 921 | |
| 925 | − return r; | |
| 922 | + if (amdgpu_emu_mode == 1) | |
| 923 | + return amdgpu_gmc_vram_checking(adev); | |
| 924 | + else | |
| 925 | + return r; | |
| 926 | 926 | } |
| 927 | 927 | |
| 928 | 928 | static int gmc_v6_0_hw_fini(void *handle) |
| @@ -613,17 +613,14 @@ static void gmc_v7_0_set_prt(struct amdgpu_device *adev, bool enable) · 612 unmodified lines | ||
| 613 | 613 | static int gmc_v7_0_gart_enable(struct amdgpu_device *adev) |
| 614 | 614 | { |
| 615 | 615 | uint64_t table_addr; |
| 616 | − int r, i; | |
| 617 | 616 | u32 tmp, field; |
| 617 | + int i; | |
| 618 | 618 | |
| 619 | 619 | if (adev->gart.bo == NULL) { |
| 620 | 620 | dev_err(adev->dev, "No VRAM object for PCIE GART.\n"); |
| 621 | 621 | return -EINVAL; |
| 622 | 622 | } |
| 623 | − r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 624 | − if (r) | |
| 625 | − return r; | |
| 626 | − | |
| 623 | + amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 627 | 624 | table_addr = amdgpu_bo_gpu_offset(adev->gart.bo); |
| 628 | 625 | |
| 629 | 626 | /* Setup TLB control */ |
| @@ -712,7 +709,6 @@ static int gmc_v7_0_gart_enable(struct amdgpu_device *adev) · 82 unmodified lines | ||
| 712 | 709 | DRM_INFO("PCIE GART of %uM enabled (table at 0x%016llX).\n", |
| 713 | 710 | (unsigned)(adev->gmc.gart_size >> 20), |
| 714 | 711 | (unsigned long long)table_addr); |
| 715 | − adev->gart.ready = true; | |
| 716 | 712 | return 0; |
| 717 | 713 | } |
| 718 | 714 | |
| @@ -1111,7 +1107,10 @@ static int gmc_v7_0_hw_init(void *handle) · 392 unmodified lines | ||
| 1111 | 1107 | if (r) |
| 1112 | 1108 | return r; |
| 1113 | 1109 | |
| 1114 | − return r; | |
| 1110 | + if (amdgpu_emu_mode == 1) | |
| 1111 | + return amdgpu_gmc_vram_checking(adev); | |
| 1112 | + else | |
| 1113 | + return r; | |
| 1115 | 1114 | } |
| 1116 | 1115 | |
| 1117 | 1116 | static int gmc_v7_0_hw_fini(void *handle) |
| @@ -837,17 +837,14 @@ static void gmc_v8_0_set_prt(struct amdgpu_device *adev, bool enable) · 836 unmodified lines | ||
| 837 | 837 | static int gmc_v8_0_gart_enable(struct amdgpu_device *adev) |
| 838 | 838 | { |
| 839 | 839 | uint64_t table_addr; |
| 840 | − int r, i; | |
| 841 | 840 | u32 tmp, field; |
| 841 | + int i; | |
| 842 | 842 | |
| 843 | 843 | if (adev->gart.bo == NULL) { |
| 844 | 844 | dev_err(adev->dev, "No VRAM object for PCIE GART.\n"); |
| 845 | 845 | return -EINVAL; |
| 846 | 846 | } |
| 847 | − r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 848 | − if (r) | |
| 849 | − return r; | |
| 850 | − | |
| 847 | + amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 851 | 848 | table_addr = amdgpu_bo_gpu_offset(adev->gart.bo); |
| 852 | 849 | |
| 853 | 850 | /* Setup TLB control */ |
| @@ -953,7 +950,6 @@ static int gmc_v8_0_gart_enable(struct amdgpu_device *adev) · 99 unmodified lines | ||
| 953 | 950 | DRM_INFO("PCIE GART of %uM enabled (table at 0x%016llX).\n", |
| 954 | 951 | (unsigned)(adev->gmc.gart_size >> 20), |
| 955 | 952 | (unsigned long long)table_addr); |
| 956 | − adev->gart.ready = true; | |
| 957 | 953 | return 0; |
| 958 | 954 | } |
| 959 | 955 | |
| @@ -1242,7 +1238,10 @@ static int gmc_v8_0_hw_init(void *handle) · 282 unmodified lines | ||
| 1242 | 1238 | if (r) |
| 1243 | 1239 | return r; |
| 1244 | 1240 | |
| 1245 | − return r; | |
| 1241 | + if (amdgpu_emu_mode == 1) | |
| 1242 | + return amdgpu_gmc_vram_checking(adev); | |
| 1243 | + else | |
| 1244 | + return r; | |
| 1246 | 1245 | } |
| 1247 | 1246 | |
| 1248 | 1247 | static int gmc_v8_0_hw_fini(void *handle) |
| @@ -62,6 +62,8 @@ · 61 unmodified lines | ||
| 62 | 62 | #include "amdgpu_ras.h" |
| 63 | 63 | #include "amdgpu_xgmi.h" |
| 64 | 64 | |
| 65 | +#include "amdgpu_reset.h" | |
| 66 | + | |
| 65 | 67 | /* add these here since we already include dce12 headers and these are for DCN */ |
| 66 | 68 | #define mmHUBP0_DCSURF_PRI_VIEWPORT_DIMENSION 0x055d |
| 67 | 69 | #define mmHUBP0_DCSURF_PRI_VIEWPORT_DIMENSION_BASE_IDX 2 |
| @@ -787,13 +789,13 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 719 unmodified lines | ||
| 787 | 789 | */ |
| 788 | 790 | if (adev->gfx.kiq.ring.sched.ready && |
| 789 | 791 | (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev)) && |
| 790 | − down_read_trylock(&adev->reset_sem)) { | |
| 792 | + down_read_trylock(&adev->reset_domain->sem)) { | |
| 791 | 793 | uint32_t req = hub->vm_inv_eng0_req + hub->eng_distance * eng; |
| 792 | 794 | uint32_t ack = hub->vm_inv_eng0_ack + hub->eng_distance * eng; |
| 793 | 795 | |
| 794 | 796 | amdgpu_virt_kiq_reg_write_reg_wait(adev, req, ack, inv_req, |
| 795 | 797 | 1 << vmid); |
| 796 | − up_read(&adev->reset_sem); | |
| 798 | + up_read(&adev->reset_domain->sem); | |
| 797 | 799 | return; |
| 798 | 800 | } |
| 799 | 801 | |
| @@ -900,7 +902,7 @@ static int gmc_v9_0_flush_gpu_tlb_pasid(struct amdgpu_device *adev, · 100 unmodified lines | ||
| 900 | 902 | if (amdgpu_in_reset(adev)) |
| 901 | 903 | return -EIO; |
| 902 | 904 | |
| 903 | − if (ring->sched.ready && down_read_trylock(&adev->reset_sem)) { | |
| 905 | + if (ring->sched.ready && down_read_trylock(&adev->reset_domain->sem)) { | |
| 904 | 906 | /* Vega20+XGMI caches PTEs in TC and TLB. Add a |
| 905 | 907 | * heavy-weight TLB flush (type 2), which flushes |
| 906 | 908 | * both. Due to a race condition with concurrent |
| @@ -927,7 +929,7 @@ static int gmc_v9_0_flush_gpu_tlb_pasid(struct amdgpu_device *adev, · 20 unmodified lines | ||
| 927 | 929 | if (r) { |
| 928 | 930 | amdgpu_ring_undo(ring); |
| 929 | 931 | spin_unlock(&adev->gfx.kiq.ring_lock); |
| 930 | − up_read(&adev->reset_sem); | |
| 932 | + up_read(&adev->reset_domain->sem); | |
| 931 | 933 | return -ETIME; |
| 932 | 934 | } |
| 933 | 935 | |
| @@ -936,10 +938,10 @@ static int gmc_v9_0_flush_gpu_tlb_pasid(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 936 | 938 | r = amdgpu_fence_wait_polling(ring, seq, adev->usec_timeout); |
| 937 | 939 | if (r < 1) { |
| 938 | 940 | dev_err(adev->dev, "wait for kiq fence error: %ld.\n", r); |
| 939 | − up_read(&adev->reset_sem); | |
| 941 | + up_read(&adev->reset_domain->sem); | |
| 940 | 942 | return -ETIME; |
| 941 | 943 | } |
| 942 | − up_read(&adev->reset_sem); | |
| 944 | + up_read(&adev->reset_domain->sem); | |
| 943 | 945 | return 0; |
| 944 | 946 | } |
| 945 | 947 | |
| @@ -1202,7 +1204,7 @@ static void gmc_v9_0_set_umc_funcs(struct amdgpu_device *adev) · 256 unmodified lines | ||
| 1202 | 1204 | adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM; |
| 1203 | 1205 | adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_VG20; |
| 1204 | 1206 | adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0]; |
| 1205 | − adev->umc.ras_funcs = &umc_v6_1_ras_funcs; | |
| 1207 | + adev->umc.ras = &umc_v6_1_ras; | |
| 1206 | 1208 | break; |
| 1207 | 1209 | case IP_VERSION(6, 1, 2): |
| 1208 | 1210 | adev->umc.max_ras_err_cnt_per_query = UMC_V6_1_TOTAL_CHANNEL_NUM; |
| @@ -1210,15 +1212,16 @@ static void gmc_v9_0_set_umc_funcs(struct amdgpu_device *adev) · 1 unmodified line | ||
| 1210 | 1212 | adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM; |
| 1211 | 1213 | adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_ARCT; |
| 1212 | 1214 | adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0]; |
| 1213 | − adev->umc.ras_funcs = &umc_v6_1_ras_funcs; | |
| 1215 | + adev->umc.ras = &umc_v6_1_ras; | |
| 1214 | 1216 | break; |
| 1215 | 1217 | case IP_VERSION(6, 7, 0): |
| 1216 | − adev->umc.max_ras_err_cnt_per_query = UMC_V6_7_TOTAL_CHANNEL_NUM; | |
| 1218 | + adev->umc.max_ras_err_cnt_per_query = | |
| 1219 | + UMC_V6_7_TOTAL_CHANNEL_NUM * UMC_V6_7_BAD_PAGE_NUM_PER_CHANNEL; | |
| 1217 | 1220 | adev->umc.channel_inst_num = UMC_V6_7_CHANNEL_INSTANCE_NUM; |
| 1218 | 1221 | adev->umc.umc_inst_num = UMC_V6_7_UMC_INSTANCE_NUM; |
| 1219 | 1222 | adev->umc.channel_offs = UMC_V6_7_PER_CHANNEL_OFFSET; |
| 1220 | 1223 | if (!adev->gmc.xgmi.connected_to_cpu) |
| 1221 | − adev->umc.ras_funcs = &umc_v6_7_ras_funcs; | |
| 1224 | + adev->umc.ras = &umc_v6_7_ras; | |
| 1222 | 1225 | if (1 & adev->smuio.funcs->get_die_id(adev)) |
| 1223 | 1226 | adev->umc.channel_idx_tbl = &umc_v6_7_channel_idx_tbl_first[0][0]; |
| 1224 | 1227 | else |
| @@ -1227,6 +1230,27 @@ static void gmc_v9_0_set_umc_funcs(struct amdgpu_device *adev) · 2 unmodified lines | ||
| 1227 | 1230 | default: |
| 1228 | 1231 | break; |
| 1229 | 1232 | } |
| 1233 | + | |
| 1234 | + if (adev->umc.ras) { | |
| 1235 | + amdgpu_ras_register_ras_block(adev, &adev->umc.ras->ras_block); | |
| 1236 | + | |
| 1237 | + strcpy(adev->umc.ras->ras_block.ras_comm.name, "umc"); | |
| 1238 | + adev->umc.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__UMC; | |
| 1239 | + adev->umc.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 1240 | + adev->umc.ras_if = &adev->umc.ras->ras_block.ras_comm; | |
| 1241 | + | |
| 1242 | + /* If don't define special ras_late_init function, use default ras_late_init */ | |
| 1243 | + if (!adev->umc.ras->ras_block.ras_late_init) | |
| 1244 | + adev->umc.ras->ras_block.ras_late_init = amdgpu_umc_ras_late_init; | |
| 1245 | + | |
| 1246 | + /* If don't define special ras_fini function, use default ras_fini */ | |
| 1247 | + if (!adev->umc.ras->ras_block.ras_fini) | |
| 1248 | + adev->umc.ras->ras_block.ras_fini = amdgpu_umc_ras_fini; | |
| 1249 | + | |
| 1250 | + /* If not defined special ras_cb function, use default ras_cb */ | |
| 1251 | + if (!adev->umc.ras->ras_block.ras_cb) | |
| 1252 | + adev->umc.ras->ras_block.ras_cb = amdgpu_umc_process_ras_data_cb; | |
| 1253 | + } | |
| 1230 | 1254 | } |
| 1231 | 1255 | |
| 1232 | 1256 | static void gmc_v9_0_set_mmhub_funcs(struct amdgpu_device *adev) |
| @@ -1248,18 +1272,31 @@ static void gmc_v9_0_set_mmhub_ras_funcs(struct amdgpu_device *adev) · 15 unmodified lines | ||
| 1248 | 1272 | { |
| 1249 | 1273 | switch (adev->ip_versions[MMHUB_HWIP][0]) { |
| 1250 | 1274 | case IP_VERSION(9, 4, 0): |
| 1251 | − adev->mmhub.ras_funcs = &mmhub_v1_0_ras_funcs; | |
| 1275 | + adev->mmhub.ras = &mmhub_v1_0_ras; | |
| 1252 | 1276 | break; |
| 1253 | 1277 | case IP_VERSION(9, 4, 1): |
| 1254 | − adev->mmhub.ras_funcs = &mmhub_v9_4_ras_funcs; | |
| 1278 | + adev->mmhub.ras = &mmhub_v9_4_ras; | |
| 1255 | 1279 | break; |
| 1256 | 1280 | case IP_VERSION(9, 4, 2): |
| 1257 | − adev->mmhub.ras_funcs = &mmhub_v1_7_ras_funcs; | |
| 1281 | + adev->mmhub.ras = &mmhub_v1_7_ras; | |
| 1258 | 1282 | break; |
| 1259 | 1283 | default: |
| 1260 | 1284 | /* mmhub ras is not available */ |
| 1261 | 1285 | break; |
| 1262 | 1286 | } |
| 1287 | + | |
| 1288 | + if (adev->mmhub.ras) { | |
| 1289 | + amdgpu_ras_register_ras_block(adev, &adev->mmhub.ras->ras_block); | |
| 1290 | + | |
| 1291 | + strcpy(adev->mmhub.ras->ras_block.ras_comm.name, "mmhub"); | |
| 1292 | + adev->mmhub.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__MMHUB; | |
| 1293 | + adev->mmhub.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | |
| 1294 | + adev->mmhub.ras_if = &adev->mmhub.ras->ras_block.ras_comm; | |
| 1295 | + | |
| 1296 | + /* If don't define special ras_fini function, use default ras_fini */ | |
| 1297 | + if (!adev->mmhub.ras->ras_block.ras_fini) | |
| 1298 | + adev->mmhub.ras->ras_block.ras_fini = amdgpu_mmhub_ras_fini; | |
| 1299 | + } | |
| 1263 | 1300 | } |
| 1264 | 1301 | |
| 1265 | 1302 | static void gmc_v9_0_set_gfxhub_funcs(struct amdgpu_device *adev) |
| @@ -1269,7 +1306,9 @@ static void gmc_v9_0_set_gfxhub_funcs(struct amdgpu_device *adev) · 3 unmodified lines | ||
| 1269 | 1306 | |
| 1270 | 1307 | static void gmc_v9_0_set_hdp_ras_funcs(struct amdgpu_device *adev) |
| 1271 | 1308 | { |
| 1272 | − adev->hdp.ras_funcs = &hdp_v4_0_ras_funcs; | |
| 1309 | + adev->hdp.ras = &hdp_v4_0_ras; | |
| 1310 | + amdgpu_ras_register_ras_block(adev, &adev->hdp.ras->ras_block); | |
| 1311 | + adev->hdp.ras_if = &adev->hdp.ras->ras_block.ras_comm; | |
| 1273 | 1312 | } |
| 1274 | 1313 | |
| 1275 | 1314 | static void gmc_v9_0_set_mca_funcs(struct amdgpu_device *adev) |
| @@ -1287,6 +1326,7 @@ static void gmc_v9_0_set_mca_funcs(struct amdgpu_device *adev) · 11 unmodified lines | ||
| 1287 | 1326 | |
| 1288 | 1327 | static int gmc_v9_0_early_init(void *handle) |
| 1289 | 1328 | { |
| 1329 | + int r; | |
| 1290 | 1330 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1291 | 1331 | |
| 1292 | 1332 | /* ARCT and VEGA20 don't have XGMI defined in their IP discovery tables */ |
| @@ -1316,6 +1356,10 @@ static int gmc_v9_0_early_init(void *handle) · 23 unmodified lines | ||
| 1316 | 1356 | adev->gmc.private_aperture_end = |
| 1317 | 1357 | adev->gmc.private_aperture_start + (4ULL << 30) - 1; |
| 1318 | 1358 | |
| 1359 | + r = amdgpu_gmc_ras_early_init(adev); | |
| 1360 | + if (r) | |
| 1361 | + return r; | |
| 1362 | + | |
| 1319 | 1363 | return 0; |
| 1320 | 1364 | } |
| 1321 | 1365 | |
| @@ -1342,13 +1386,13 @@ static int gmc_v9_0_late_init(void *handle) · 20 unmodified lines | ||
| 1342 | 1386 | } |
| 1343 | 1387 | |
| 1344 | 1388 | if (!amdgpu_persistent_edc_harvesting_supported(adev)) { |
| 1345 | − if (adev->mmhub.ras_funcs && | |
| 1346 | − adev->mmhub.ras_funcs->reset_ras_error_count) | |
| 1347 | − adev->mmhub.ras_funcs->reset_ras_error_count(adev); | |
| 1389 | + if (adev->mmhub.ras && adev->mmhub.ras->ras_block.hw_ops && | |
| 1390 | + adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count) | |
| 1391 | + adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count(adev); | |
| 1348 | 1392 | |
| 1349 | − if (adev->hdp.ras_funcs && | |
| 1350 | − adev->hdp.ras_funcs->reset_ras_error_count) | |
| 1351 | − adev->hdp.ras_funcs->reset_ras_error_count(adev); | |
| 1393 | + if (adev->hdp.ras && adev->hdp.ras->ras_block.hw_ops && | |
| 1394 | + adev->hdp.ras->ras_block.hw_ops->reset_ras_error_count) | |
| 1395 | + adev->hdp.ras->ras_block.hw_ops->reset_ras_error_count(adev); | |
| 1352 | 1396 | } |
| 1353 | 1397 | |
| 1354 | 1398 | r = amdgpu_gmc_ras_late_init(adev); |
| @@ -1752,14 +1796,7 @@ static int gmc_v9_0_gart_enable(struct amdgpu_device *adev) · 397 unmodified lines | ||
| 1752 | 1796 | return -EINVAL; |
| 1753 | 1797 | } |
| 1754 | 1798 | |
| 1755 | − if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev)) | |
| 1756 | − goto skip_pin_bo; | |
| 1757 | − | |
| 1758 | − r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 1759 | − if (r) | |
| 1760 | − return r; | |
| 1761 | − | |
| 1762 | −skip_pin_bo: | |
| 1799 | + amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | |
| 1763 | 1800 | r = adev->gfxhub.funcs->gart_enable(adev); |
| 1764 | 1801 | if (r) |
| 1765 | 1802 | return r; |
| @@ -1776,7 +1813,6 @@ skip_pin_bo: · 10 unmodified lines | ||
| 1776 | 1813 | DRM_INFO("PTB located at 0x%016llX\n", |
| 1777 | 1814 | (unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo)); |
| 1778 | 1815 | |
| 1779 | − adev->gart.ready = true; | |
| 1780 | 1816 | return 0; |
| 1781 | 1817 | } |
| 1782 | 1818 | |
| @@ -1784,7 +1820,7 @@ static int gmc_v9_0_hw_init(void *handle) · 1 unmodified line | ||
| 1784 | 1820 | { |
| 1785 | 1821 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1786 | 1822 | bool value; |
| 1787 | − int i; | |
| 1823 | + int i, r; | |
| 1788 | 1824 | |
| 1789 | 1825 | /* The sequence of these two function calls matters.*/ |
| 1790 | 1826 | gmc_v9_0_init_golden_registers(adev); |
| @@ -1819,7 +1855,14 @@ static int gmc_v9_0_hw_init(void *handle) · 28 unmodified lines | ||
| 1819 | 1855 | if (adev->umc.funcs && adev->umc.funcs->init_registers) |
| 1820 | 1856 | adev->umc.funcs->init_registers(adev); |
| 1821 | 1857 | |
| 1822 | − return gmc_v9_0_gart_enable(adev); | |
| 1858 | + r = gmc_v9_0_gart_enable(adev); | |
| 1859 | + if (r) | |
| 1860 | + return r; | |
| 1861 | + | |
| 1862 | + if (amdgpu_emu_mode == 1) | |
| 1863 | + return amdgpu_gmc_vram_checking(adev); | |
| 1864 | + else | |
| 1865 | + return r; | |
| 1823 | 1866 | } |
| 1824 | 1867 | |
| 1825 | 1868 | /** |
| @@ -146,17 +146,30 @@ static void hdp_v4_0_init_registers(struct amdgpu_device *adev) · 145 unmodified lines | ||
| 146 | 146 | |
| 147 | 147 | WREG32_FIELD15(HDP, 0, HDP_MISC_CNTL, FLUSH_INVALIDATE_CACHE, 1); |
| 148 | 148 | |
| 149 | + if (adev->ip_versions[HDP_HWIP][0] == IP_VERSION(4, 4, 0)) | |
| 150 | + WREG32_FIELD15(HDP, 0, HDP_MISC_CNTL, READ_BUFFER_WATERMARK, 2); | |
| 151 | + | |
| 149 | 152 | WREG32_SOC15(HDP, 0, mmHDP_NONSURFACE_BASE, (adev->gmc.vram_start >> 8)); |
| 150 | 153 | WREG32_SOC15(HDP, 0, mmHDP_NONSURFACE_BASE_HI, (adev->gmc.vram_start >> 40)); |
| 151 | 154 | } |
| 152 | 155 | |
| 153 | −const struct amdgpu_hdp_ras_funcs hdp_v4_0_ras_funcs = { | |
| 154 | − .ras_late_init = amdgpu_hdp_ras_late_init, | |
| 155 | − .ras_fini = amdgpu_hdp_ras_fini, | |
| 156 | +struct amdgpu_ras_block_hw_ops hdp_v4_0_ras_hw_ops = { | |
| 156 | 157 | .query_ras_error_count = hdp_v4_0_query_ras_error_count, |
| 157 | 158 | .reset_ras_error_count = hdp_v4_0_reset_ras_error_count, |
| 158 | 159 | }; |
| 159 | 160 | |
| 161 | +struct amdgpu_hdp_ras hdp_v4_0_ras = { | |
| 162 | + .ras_block = { | |
| 163 | + .ras_comm = { | |
| 164 | + .name = "hdp", | |
| 165 | + .block = AMDGPU_RAS_BLOCK__HDP, | |
| 166 | + .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | |
| 167 | + }, | |
| 168 | + .hw_ops = &hdp_v4_0_ras_hw_ops, | |
| 169 | + .ras_fini = amdgpu_hdp_ras_fini, | |
| 170 | + }, | |
| 171 | +}; | |
| 172 | + | |
| 160 | 173 | const struct amdgpu_hdp_funcs hdp_v4_0_funcs = { |
| 161 | 174 | .flush_hdp = hdp_v4_0_flush_hdp, |
| 162 | 175 | .invalidate_hdp = hdp_v4_0_invalidate_hdp, |
| @@ -27,6 +27,6 @@ · 26 unmodified lines | ||
| 27 | 27 | #include "soc15_common.h" |
| 28 | 28 | |
| 29 | 29 | extern const struct amdgpu_hdp_funcs hdp_v4_0_funcs; |
| 30 | −extern const struct amdgpu_hdp_ras_funcs hdp_v4_0_ras_funcs; | |
| 30 | +extern struct amdgpu_hdp_ras hdp_v4_0_ras; | |
| 31 | 31 | |
| 32 | 32 | #endif |
| @@ -50,11 +50,16 @@ static int jpeg_v3_0_early_init(void *handle) · 49 unmodified lines | ||
| 50 | 50 | { |
| 51 | 51 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 52 | 52 | |
| 53 | − if (adev->asic_type != CHIP_YELLOW_CARP) { | |
| 54 | − u32 harvest = RREG32_SOC15(JPEG, 0, mmCC_UVD_HARVESTING); | |
| 53 | + u32 harvest; | |
| 55 | 54 | |
| 55 | + switch (adev->ip_versions[UVD_HWIP][0]) { | |
| 56 | + case IP_VERSION(3, 1, 1): | |
| 57 | + break; | |
| 58 | + default: | |
| 59 | + harvest = RREG32_SOC15(JPEG, 0, mmCC_UVD_HARVESTING); | |
| 56 | 60 | if (harvest & CC_UVD_HARVESTING__UVD_DISABLE_MASK) |
| 57 | 61 | return -ENOENT; |
| 62 | + break; | |
| 58 | 63 | } |
| 59 | 64 | |
| 60 | 65 | adev->jpeg.num_jpeg_inst = 1; |
| @@ -37,24 +37,42 @@ static void mca_v3_0_mp0_query_ras_error_count(struct amdgpu_device *adev, · 36 unmodified lines | ||
| 37 | 37 | ras_error_status); |
| 38 | 38 | } |
| 39 | 39 | |
| 40 | −static int mca_v3_0_mp0_ras_late_init(struct amdgpu_device *adev) | |
| 40 | +static void mca_v3_0_mp0_ras_fini(struct amdgpu_device *adev) | |
| 41 | 41 | { |
| 42 | − return amdgpu_mca_ras_late_init(adev, &adev->mca.mp0); | |
| 42 | + amdgpu_mca_ras_fini(adev, &adev->mca.mp0); | |
| 43 | 43 | } |
| 44 | 44 | |
| 45 | −static void mca_v3_0_mp0_ras_fini(struct amdgpu_device *adev) | |
| 45 | +static int mca_v3_0_ras_block_match(struct amdgpu_ras_block_object *block_obj, | |
| 46 | + enum amdgpu_ras_block block, uint32_t sub_block_index) | |
| 46 | 47 | { |
| 47 | − amdgpu_mca_ras_fini(adev, &adev->mca.mp0); | |
| 48 | + if (!block_obj) | |
| 49 | + return -EINVAL; | |
| 50 | + | |
| 51 | + if ((block_obj->ras_comm.block == block) && | |
| 52 | + (block_obj->ras_comm.sub_block_index == sub_block_index)) { | |
| 53 | + return 0; | |
| 54 | + } | |
| 55 | + | |
| 56 | + return -EINVAL; | |
| 48 | 57 | } |
| 49 | 58 | |
| 50 | −const struct amdgpu_mca_ras_funcs mca_v3_0_mp0_ras_funcs = { | |
| 51 | − .ras_late_init = mca_v3_0_mp0_ras_late_init, | |
| 52 | − .ras_fini = mca_v3_0_mp0_ras_fini, | |
| 59 | +const struct amdgpu_ras_block_hw_ops mca_v3_0_mp0_hw_ops = { | |
| 53 | 60 | .query_ras_error_count = mca_v3_0_mp0_query_ras_error_count, |
| 54 | 61 | .query_ras_error_address = NULL, |
| 55 | − .ras_block = AMDGPU_RAS_BLOCK__MCA, | |
| 56 | − .ras_sub_block = AMDGPU_RAS_MCA_BLOCK__MP0, | |
| 57 | − .sysfs_name = "mp0_err_count", | |
| 62 | +}; | |
| 63 | + | |
| 64 | +struct amdgpu_mca_ras_block mca_v3_0_mp0_ras = { | |
| 65 | + .ras_block = { | |
| 66 | + .ras_comm = { | |
| 67 | + .block = AMDGPU_RAS_BLOCK__MCA, | |
| 68 | + .sub_block_index = AMDGPU_RAS_MCA_BLOCK__MP0, | |
| 69 | + .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | |
| 70 | + .name = "mp0", | |
| 71 | + }, | |
| 72 | + .hw_ops = &mca_v3_0_mp0_hw_ops, | |
| 73 | + .ras_block_match = mca_v3_0_ras_block_match, | |
| 74 | + .ras_fini = mca_v3_0_mp0_ras_fini, | |
| 75 | + }, | |
| 58 | 76 | }; |
| 59 | 77 | |
| 60 | 78 | static void mca_v3_0_mp1_query_ras_error_count(struct amdgpu_device *adev, |
| @@ -65,24 +83,28 @@ static void mca_v3_0_mp1_query_ras_error_count(struct amdgpu_device *adev, · 4 unmodified lines | ||
| 65 | 83 | ras_error_status); |
| 66 | 84 | } |
| 67 | 85 | |
| 68 | −static int mca_v3_0_mp1_ras_late_init(struct amdgpu_device *adev) | |
| 69 | −{ | |
| 70 | − return amdgpu_mca_ras_late_init(adev, &adev->mca.mp1); | |
| 71 | −} | |
| 72 | − | |
| 73 | 86 | static void mca_v3_0_mp1_ras_fini(struct amdgpu_device *adev) |
| 74 | 87 | { |
| 75 | 88 | amdgpu_mca_ras_fini(adev, &adev->mca.mp1); |
| 76 | 89 | } |
| 77 | 90 | |
| 78 | −const struct amdgpu_mca_ras_funcs mca_v3_0_mp1_ras_funcs = { | |
| 79 | − .ras_late_init = mca_v3_0_mp1_ras_late_init, | |
| 80 | − .ras_fini = mca_v3_0_mp1_ras_fini, | |
| 91 | +const struct amdgpu_ras_block_hw_ops mca_v3_0_mp1_hw_ops = { | |
| 81 | 92 | .query_ras_error_count = mca_v3_0_mp1_query_ras_error_count, |
| 82 | 93 | .query_ras_error_address = NULL, |
| 83 | − .ras_block = AMDGPU_RAS_BLOCK__MCA, | |
| 84 | − .ras_sub_block = AMDGPU_RAS_MCA_BLOCK__MP1, | |
| 85 | − .sysfs_name = "mp1_err_count", | |
| 94 | +}; | |
| 95 | + | |
| 96 | +struct amdgpu_mca_ras_block mca_v3_0_mp1_ras = { | |
| 97 | + .ras_block = { | |
| 98 | + .ras_comm = { | |
| 99 | + .block = AMDGPU_RAS_BLOCK__MCA, | |
| 100 | + .sub_block_index = AMDGPU_RAS_MCA_BLOCK__MP1, | |
| 101 | + .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | |
| 102 | + .name = "mp1", | |
| 103 | + }, | |
| 104 | + .hw_ops = &mca_v3_0_mp1_hw_ops, | |
| 105 | + .ras_block_match = mca_v3_0_ras_block_match, | |
| 106 | + .ras_fini = mca_v3_0_mp1_ras_fini, | |
| 107 | + }, | |
| 86 | 108 | }; |
| 87 | 109 | |
| 88 | 110 | static void mca_v3_0_mpio_query_ras_error_count(struct amdgpu_device *adev, |
| @@ -93,24 +115,28 @@ static void mca_v3_0_mpio_query_ras_error_count(struct amdgpu_device *adev, · 4 unmodified lines | ||
| 93 | 115 | ras_error_status); |
| 94 | 116 | } |
| 95 | 117 | |
| 96 | −static int mca_v3_0_mpio_ras_late_init(struct amdgpu_device *adev) | |
| 97 | −{ | |
| 98 | − return amdgpu_mca_ras_late_init(adev, &adev->mca.mpio); | |
| 99 | −} | |
| 100 | − | |
| 101 | 118 | static void mca_v3_0_mpio_ras_fini(struct amdgpu_device *adev) |
| 102 | 119 | { |
| 103 | 120 | amdgpu_mca_ras_fini(adev, &adev->mca.mpio); |
| 104 | 121 | } |
| 105 | 122 | |
| 106 | −const struct amdgpu_mca_ras_funcs mca_v3_0_mpio_ras_funcs = { | |
| 107 | − .ras_late_init = mca_v3_0_mpio_ras_late_init, | |
| 108 | − .ras_fini = mca_v3_0_mpio_ras_fini, | |
| 123 | +const struct amdgpu_ras_block_hw_ops mca_v3_0_mpio_hw_ops = { | |
| 109 | 124 | .query_ras_error_count = mca_v3_0_mpio_query_ras_error_count, |
| 110 | 125 | .query_ras_error_address = NULL, |
| 111 | − .ras_block = AMDGPU_RAS_BLOCK__MCA, | |
| 112 | − .ras_sub_block = AMDGPU_RAS_MCA_BLOCK__MPIO, | |
| 113 | − .sysfs_name = "mpio_err_count", | |
| 126 | +}; | |
| 127 | + | |
| 128 | +struct amdgpu_mca_ras_block mca_v3_0_mpio_ras = { | |
| 129 | + .ras_block = { | |
| 130 | + .ras_comm = { | |
| 131 | + .block = AMDGPU_RAS_BLOCK__MCA, | |
| 132 | + .sub_block_index = AMDGPU_RAS_MCA_BLOCK__MPIO, | |
| 133 | + .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | |
| 134 | + .name = "mpio", | |
| 135 | + }, | |
| 136 | + .hw_ops = &mca_v3_0_mpio_hw_ops, | |
| 137 | + .ras_block_match = mca_v3_0_ras_block_match, | |
| 138 | + .ras_fini = mca_v3_0_mpio_ras_fini, | |
| 139 | + }, | |
| 114 | 140 | }; |
| 115 | 141 | |
| 116 | 142 | |
| @@ -118,9 +144,15 @@ static void mca_v3_0_init(struct amdgpu_device *adev) · 1 unmodified line | ||
| 118 | 144 | { |
| 119 | 145 | struct amdgpu_mca *mca = &adev->mca; |
| 120 | 146 | |
| 121 | − mca->mp0.ras_funcs = &mca_v3_0_mp0_ras_funcs; | |
| 122 | − mca->mp1.ras_funcs = &mca_v3_0_mp1_ras_funcs; | |
| 123 | − mca->mpio.ras_funcs = &mca_v3_0_mpio_ras_funcs; | |
| 147 | + mca->mp0.ras = &mca_v3_0_mp0_ras; | |
| 148 | + mca->mp1.ras = &mca_v3_0_mp1_ras; | |
| 149 | + mca->mpio.ras = &mca_v3_0_mpio_ras; | |
| 150 | + amdgpu_ras_register_ras_block(adev, &mca->mp0.ras->ras_block); | |
| 151 | + amdgpu_ras_register_ras_block(adev, &mca->mp1.ras->ras_block); | |
| 152 | + amdgpu_ras_register_ras_block(adev, &mca->mpio.ras->ras_block); | |
| 153 | + mca->mp0.ras_if = &mca->mp0.ras->ras_block.ras_comm; | |
| 154 | + mca->mp1.ras_if = &mca->mp1.ras->ras_block.ras_comm; | |
| 155 | + mca->mpio.ras_if = &mca->mpio.ras->ras_block.ras_comm; | |
| 124 | 156 | } |
| 125 | 157 | |
| 126 | 158 | const struct amdgpu_mca_funcs mca_v3_0_funcs = { |
| @@ -774,13 +774,17 @@ static void mmhub_v1_0_reset_ras_error_count(struct amdgpu_device *adev) · 773 unmodified lines | ||
| 774 | 774 | } |
| 775 | 775 | } |
| 776 | 776 | |
| 777 | −const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs = { | |
| 778 | − .ras_late_init = amdgpu_mmhub_ras_late_init, | |
| 779 | − .ras_fini = amdgpu_mmhub_ras_fini, | |
| 777 | +struct amdgpu_ras_block_hw_ops mmhub_v1_0_ras_hw_ops = { | |
| 780 | 778 | .query_ras_error_count = mmhub_v1_0_query_ras_error_count, |
| 781 | 779 | .reset_ras_error_count = mmhub_v1_0_reset_ras_error_count, |
| 782 | 780 | }; |
| 783 | 781 | |
| 782 | +struct amdgpu_mmhub_ras mmhub_v1_0_ras = { | |
| 783 | + .ras_block = { | |
| 784 | + .hw_ops = &mmhub_v1_0_ras_hw_ops, | |
| 785 | + }, | |
| 786 | +}; | |
| 787 | + | |
| 784 | 788 | const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs = { |
| 785 | 789 | .get_fb_location = mmhub_v1_0_get_fb_location, |
| 786 | 790 | .init = mmhub_v1_0_init, |
| @@ -24,6 +24,6 @@ · 23 unmodified lines | ||
| 24 | 24 | #define __MMHUB_V1_0_H__ |
| 25 | 25 | |
| 26 | 26 | extern const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs; |
| 27 | −extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs; | |
| 27 | +extern struct amdgpu_mmhub_ras mmhub_v1_0_ras; | |
| 28 | 28 | |
| 29 | 29 | #endif |
| @@ -1321,15 +1321,19 @@ static void mmhub_v1_7_reset_ras_error_status(struct amdgpu_device *adev) · 1320 unmodified lines | ||
| 1321 | 1321 | } |
| 1322 | 1322 | } |
| 1323 | 1323 | |
| 1324 | −const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs = { | |
| 1325 | − .ras_late_init = amdgpu_mmhub_ras_late_init, | |
| 1326 | − .ras_fini = amdgpu_mmhub_ras_fini, | |
| 1324 | +struct amdgpu_ras_block_hw_ops mmhub_v1_7_ras_hw_ops = { | |
| 1327 | 1325 | .query_ras_error_count = mmhub_v1_7_query_ras_error_count, |
| 1328 | 1326 | .reset_ras_error_count = mmhub_v1_7_reset_ras_error_count, |
| 1329 | 1327 | .query_ras_error_status = mmhub_v1_7_query_ras_error_status, |
| 1330 | 1328 | .reset_ras_error_status = mmhub_v1_7_reset_ras_error_status, |
| 1331 | 1329 | }; |
| 1332 | 1330 | |
| 1331 | +struct amdgpu_mmhub_ras mmhub_v1_7_ras = { | |
| 1332 | + .ras_block = { | |
| 1333 | + .hw_ops = &mmhub_v1_7_ras_hw_ops, | |
| 1334 | + }, | |
| 1335 | +}; | |
| 1336 | + | |
| 1333 | 1337 | const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs = { |
| 1334 | 1338 | .get_fb_location = mmhub_v1_7_get_fb_location, |
| 1335 | 1339 | .init = mmhub_v1_7_init, |
| @@ -24,6 +24,6 @@ · 23 unmodified lines | ||
| 24 | 24 | #define __MMHUB_V1_7_H__ |
| 25 | 25 | |
| 26 | 26 | extern const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs; |
| 27 | −extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs; | |
| 27 | +extern struct amdgpu_mmhub_ras mmhub_v1_7_ras; | |
| 28 | 28 | |
| 29 | 29 | #endif |
| @@ -93,6 +93,7 @@ mmhub_v2_3_print_l2_protection_fault_status(struct amdgpu_device *adev, · 92 unmodified lines | ||
| 93 | 93 | switch (adev->ip_versions[MMHUB_HWIP][0]) { |
| 94 | 94 | case IP_VERSION(2, 3, 0): |
| 95 | 95 | case IP_VERSION(2, 4, 0): |
| 96 | + case IP_VERSION(2, 4, 1): | |
| 96 | 97 | mmhub_cid = mmhub_client_ids_vangogh[cid][rw]; |
| 97 | 98 | break; |
| 98 | 99 | default: |
| @@ -1655,14 +1655,18 @@ static void mmhub_v9_4_query_ras_error_status(struct amdgpu_device *adev) · 1654 unmodified lines | ||
| 1655 | 1655 | } |
| 1656 | 1656 | } |
| 1657 | 1657 | |
| 1658 | −const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs = { | |
| 1659 | − .ras_late_init = amdgpu_mmhub_ras_late_init, | |
| 1660 | − .ras_fini = amdgpu_mmhub_ras_fini, | |
| 1658 | +const struct amdgpu_ras_block_hw_ops mmhub_v9_4_ras_hw_ops = { | |
| 1661 | 1659 | .query_ras_error_count = mmhub_v9_4_query_ras_error_count, |
| 1662 | 1660 | .reset_ras_error_count = mmhub_v9_4_reset_ras_error_count, |
| 1663 | 1661 | .query_ras_error_status = mmhub_v9_4_query_ras_error_status, |
| 1664 | 1662 | }; |
| 1665 | 1663 | |
| 1664 | +struct amdgpu_mmhub_ras mmhub_v9_4_ras = { | |
| 1665 | + .ras_block = { | |
| 1666 | + .hw_ops = &mmhub_v9_4_ras_hw_ops, | |
| 1667 | + }, | |
| 1668 | +}; | |
| 1669 | + | |
| 1666 | 1670 | const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs = { |
| 1667 | 1671 | .get_fb_location = mmhub_v9_4_get_fb_location, |
| 1668 | 1672 | .init = mmhub_v9_4_init, |
| @@ -24,6 +24,6 @@ · 23 unmodified lines | ||
| 24 | 24 | #define __MMHUB_V9_4_H__ |
| 25 | 25 | |
| 26 | 26 | extern const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs; |
| 27 | −extern const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs; | |
| 27 | +extern struct amdgpu_mmhub_ras mmhub_v9_4_ras; | |
| 28 | 28 | |
| 29 | 29 | #endif |
| @@ -32,6 +32,8 @@ · 31 unmodified lines | ||
| 32 | 32 | #include "soc15_common.h" |
| 33 | 33 | #include "mxgpu_ai.h" |
| 34 | 34 | |
| 35 | +#include "amdgpu_reset.h" | |
| 36 | + | |
| 35 | 37 | static void xgpu_ai_mailbox_send_ack(struct amdgpu_device *adev) |
| 36 | 38 | { |
| 37 | 39 | WREG8(AI_MAIBOX_CONTROL_RCV_OFFSET_BYTE, 2); |
| @@ -257,10 +259,10 @@ static void xgpu_ai_mailbox_flr_work(struct work_struct *work) · 219 unmodified lines | ||
| 257 | 259 | * otherwise the mailbox msg will be ruined/reseted by |
| 258 | 260 | * the VF FLR. |
| 259 | 261 | */ |
| 260 | − if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0) | |
| 262 | + if (atomic_cmpxchg(&adev->reset_domain->in_gpu_reset, 0, 1) != 0) | |
| 261 | 263 | return; |
| 262 | 264 | |
| 263 | − down_write(&adev->reset_sem); | |
| 265 | + down_write(&adev->reset_domain->sem); | |
| 264 | 266 | |
| 265 | 267 | amdgpu_virt_fini_data_exchange(adev); |
| 266 | 268 | |
| @@ -275,14 +277,14 @@ static void xgpu_ai_mailbox_flr_work(struct work_struct *work) · 8 unmodified lines | ||
| 275 | 277 | } while (timeout > 1); |
| 276 | 278 | |
| 277 | 279 | flr_done: |
| 278 | − atomic_set(&adev->in_gpu_reset, 0); | |
| 279 | − up_write(&adev->reset_sem); | |
| 280 | + atomic_set(&adev->reset_domain->in_gpu_reset, 0); | |
| 281 | + up_write(&adev->reset_domain->sem); | |
| 280 | 282 | |
| 281 | 283 | /* Trigger recovery for world switch failure if no TDR */ |
| 282 | 284 | if (amdgpu_device_should_recover_gpu(adev) |
| 283 | 285 | && (!amdgpu_device_has_job_running(adev) || |
| 284 | 286 | adev->sdma_timeout == MAX_SCHEDULE_TIMEOUT)) |
| 285 | − amdgpu_device_gpu_recover(adev, NULL); | |
| 287 | + amdgpu_device_gpu_recover_imp(adev, NULL); | |
| 286 | 288 | } |
| 287 | 289 | |
| 288 | 290 | static int xgpu_ai_set_mailbox_rcv_irq(struct amdgpu_device *adev, |
| @@ -307,8 +309,11 @@ static int xgpu_ai_mailbox_rcv_irq(struct amdgpu_device *adev, · 18 unmodified lines | ||
| 307 | 309 | |
| 308 | 310 | switch (event) { |
| 309 | 311 | case IDH_FLR_NOTIFICATION: |
| 310 | − if (amdgpu_sriov_runtime(adev)) | |
| 311 | − schedule_work(&adev->virt.flr_work); | |
| 312 | + if (amdgpu_sriov_runtime(adev) && !amdgpu_in_reset(adev)) | |
| 313 | + WARN_ONCE(!amdgpu_reset_domain_schedule(adev->reset_domain, | |
| 314 | + &adev->virt.flr_work), | |
| 315 | + "Failed to queue work! at %s", | |
| 316 | + __func__); | |
| 312 | 317 | break; |
| 313 | 318 | case IDH_QUERY_ALIVE: |
| 314 | 319 | xgpu_ai_mailbox_send_ack(adev); |
| @@ -31,6 +31,8 @@ · 30 unmodified lines | ||
| 31 | 31 | #include "soc15_common.h" |
| 32 | 32 | #include "mxgpu_nv.h" |
| 33 | 33 | |
| 34 | +#include "amdgpu_reset.h" | |
| 35 | + | |
| 34 | 36 | static void xgpu_nv_mailbox_send_ack(struct amdgpu_device *adev) |
| 35 | 37 | { |
| 36 | 38 | WREG8(NV_MAIBOX_CONTROL_RCV_OFFSET_BYTE, 2); |
| @@ -281,10 +283,10 @@ static void xgpu_nv_mailbox_flr_work(struct work_struct *work) · 244 unmodified lines | ||
| 281 | 283 | * otherwise the mailbox msg will be ruined/reseted by |
| 282 | 284 | * the VF FLR. |
| 283 | 285 | */ |
| 284 | − if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0) | |
| 286 | + if (atomic_cmpxchg(&adev->reset_domain->in_gpu_reset, 0, 1) != 0) | |
| 285 | 287 | return; |
| 286 | 288 | |
| 287 | − down_write(&adev->reset_sem); | |
| 289 | + down_write(&adev->reset_domain->sem); | |
| 288 | 290 | |
| 289 | 291 | amdgpu_virt_fini_data_exchange(adev); |
| 290 | 292 | |
| @@ -299,8 +301,8 @@ static void xgpu_nv_mailbox_flr_work(struct work_struct *work) · 8 unmodified lines | ||
| 299 | 301 | } while (timeout > 1); |
| 300 | 302 | |
| 301 | 303 | flr_done: |
| 302 | − atomic_set(&adev->in_gpu_reset, 0); | |
| 303 | − up_write(&adev->reset_sem); | |
| 304 | + atomic_set(&adev->reset_domain->in_gpu_reset, 0); | |
| 305 | + up_write(&adev->reset_domain->sem); | |
| 304 | 306 | |
| 305 | 307 | /* Trigger recovery for world switch failure if no TDR */ |
| 306 | 308 | if (amdgpu_device_should_recover_gpu(adev) |
| @@ -309,7 +311,7 @@ flr_done: · 2 unmodified lines | ||
| 309 | 311 | adev->gfx_timeout == MAX_SCHEDULE_TIMEOUT || |
| 310 | 312 | adev->compute_timeout == MAX_SCHEDULE_TIMEOUT || |
| 311 | 313 | adev->video_timeout == MAX_SCHEDULE_TIMEOUT)) |
| 312 | − amdgpu_device_gpu_recover(adev, NULL); | |
| 314 | + amdgpu_device_gpu_recover_imp(adev, NULL); | |
| 313 | 315 | } |
| 314 | 316 | |
| 315 | 317 | static int xgpu_nv_set_mailbox_rcv_irq(struct amdgpu_device *adev, |
| @@ -337,8 +339,11 @@ static int xgpu_nv_mailbox_rcv_irq(struct amdgpu_device *adev, · 21 unmodified lines | ||
| 337 | 339 | |
| 338 | 340 | switch (event) { |
| 339 | 341 | case IDH_FLR_NOTIFICATION: |
| 340 | − if (amdgpu_sriov_runtime(adev)) | |
| 341 | − schedule_work(&adev->virt.flr_work); | |
| 342 | + if (amdgpu_sriov_runtime(adev) && !amdgpu_in_reset(adev)) | |
| 343 | + WARN_ONCE(!amdgpu_reset_domain_schedule(adev->reset_domain, | |
| 344 | + &adev->virt.flr_work), | |
| 345 | + "Failed to queue work! at %s", | |
| 346 | + __func__); | |
| 342 | 347 | break; |
| 343 | 348 | /* READY_TO_ACCESS_GPU is fetched by kernel polling, IRQ can ignore |
| 344 | 349 | * it byfar since that polling thread will handle it, |
| @@ -42,6 +42,8 @@ · 41 unmodified lines | ||
| 42 | 42 | #include "smu/smu_7_1_3_d.h" |
| 43 | 43 | #include "mxgpu_vi.h" |
| 44 | 44 | |
| 45 | +#include "amdgpu_reset.h" | |
| 46 | + | |
| 45 | 47 | /* VI golden setting */ |
| 46 | 48 | static const u32 xgpu_fiji_mgcg_cgcg_init[] = { |
| 47 | 49 | mmRLC_CGTT_MGCG_OVERRIDE, 0xffffffff, 0xffffffff, |
| @@ -521,7 +523,7 @@ static void xgpu_vi_mailbox_flr_work(struct work_struct *work) · 473 unmodified lines | ||
| 521 | 523 | |
| 522 | 524 | /* Trigger recovery due to world switch failure */ |
| 523 | 525 | if (amdgpu_device_should_recover_gpu(adev)) |
| 524 | − amdgpu_device_gpu_recover(adev, NULL); | |
| 526 | + amdgpu_device_gpu_recover_imp(adev, NULL); | |
| 525 | 527 | } |
| 526 | 528 | |
| 527 | 529 | static int xgpu_vi_set_mailbox_rcv_irq(struct amdgpu_device *adev, |
| @@ -544,14 +546,17 @@ static int xgpu_vi_mailbox_rcv_irq(struct amdgpu_device *adev, · 16 unmodified lines | ||
| 544 | 546 | { |
| 545 | 547 | int r; |
| 546 | 548 | |
| 547 | − /* trigger gpu-reset by hypervisor only if TDR disbaled */ | |
| 549 | + /* trigger gpu-reset by hypervisor only if TDR disabled */ | |
| 548 | 550 | if (!amdgpu_gpu_recovery) { |
| 549 | 551 | /* see what event we get */ |
| 550 | 552 | r = xgpu_vi_mailbox_rcv_msg(adev, IDH_FLR_NOTIFICATION); |
| 551 | 553 | |
| 552 | 554 | /* only handle FLR_NOTIFY now */ |
| 553 | − if (!r) | |
| 554 | − schedule_work(&adev->virt.flr_work); | |
| 555 | + if (!r && !amdgpu_in_reset(adev)) | |
| 556 | + WARN_ONCE(!amdgpu_reset_domain_schedule(adev->reset_domain, | |
| 557 | + &adev->virt.flr_work), | |
| 558 | + "Failed to queue work! at %s", | |
| 559 | + __func__); | |
| 555 | 560 | } |
| 556 | 561 | |
| 557 | 562 | return 0; |
| @@ -59,10 +59,16 @@ static u32 nbio_v7_2_get_rev_id(struct amdgpu_device *adev) · 58 unmodified lines | ||
| 59 | 59 | { |
| 60 | 60 | u32 tmp; |
| 61 | 61 | |
| 62 | − if (adev->asic_type == CHIP_YELLOW_CARP) | |
| 62 | + switch (adev->ip_versions[NBIO_HWIP][0]) { | |
| 63 | + case IP_VERSION(7, 2, 1): | |
| 64 | + case IP_VERSION(7, 3, 0): | |
| 65 | + case IP_VERSION(7, 5, 0): | |
| 63 | 66 | tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_YC); |
| 64 | − else | |
| 67 | + break; | |
| 68 | + default: | |
| 65 | 69 | tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0); |
| 70 | + break; | |
| 71 | + } | |
| 66 | 72 | |
| 67 | 73 | tmp &= RCC_STRAP0_RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0_MASK; |
| 68 | 74 | tmp >>= RCC_STRAP0_RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0__SHIFT; |
| @@ -72,20 +78,26 @@ static u32 nbio_v7_2_get_rev_id(struct amdgpu_device *adev) · 3 unmodified lines | ||
| 72 | 78 | |
| 73 | 79 | static void nbio_v7_2_mc_access_enable(struct amdgpu_device *adev, bool enable) |
| 74 | 80 | { |
| 75 | − if (enable) | |
| 76 | − if (adev->asic_type == CHIP_YELLOW_CARP) | |
| 81 | + switch (adev->ip_versions[NBIO_HWIP][0]) { | |
| 82 | + case IP_VERSION(7, 2, 1): | |
| 83 | + case IP_VERSION(7, 3, 0): | |
| 84 | + case IP_VERSION(7, 5, 0): | |
| 85 | + if (enable) | |
| 77 | 86 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC, |
| 78 | 87 | BIF_BX0_BIF_FB_EN__FB_READ_EN_MASK | |
| 79 | 88 | BIF_BX0_BIF_FB_EN__FB_WRITE_EN_MASK); |
| 80 | 89 | else |
| 90 | + WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC, 0); | |
| 91 | + break; | |
| 92 | + default: | |
| 93 | + if (enable) | |
| 81 | 94 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN, |
| 82 | 95 | BIF_BX0_BIF_FB_EN__FB_READ_EN_MASK | |
| 83 | 96 | BIF_BX0_BIF_FB_EN__FB_WRITE_EN_MASK); |
| 84 | − else | |
| 85 | − if (adev->asic_type == CHIP_YELLOW_CARP) | |
| 86 | − WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC, 0); | |
| 87 | 97 | else |
| 88 | 98 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN, 0); |
| 99 | + break; | |
| 100 | + } | |
| 89 | 101 | } |
| 90 | 102 | |
| 91 | 103 | static u32 nbio_v7_2_get_memsize(struct amdgpu_device *adev) |
| @@ -250,7 +262,10 @@ static void nbio_v7_2_update_medium_grain_light_sleep(struct amdgpu_device *adev · 158 unmodified lines | ||
| 250 | 262 | { |
| 251 | 263 | uint32_t def, data; |
| 252 | 264 | |
| 253 | − if (adev->asic_type == CHIP_YELLOW_CARP) { | |
| 265 | + switch (adev->ip_versions[NBIO_HWIP][0]) { | |
| 266 | + case IP_VERSION(7, 2, 1): | |
| 267 | + case IP_VERSION(7, 3, 0): | |
| 268 | + case IP_VERSION(7, 5, 0): | |
| 254 | 269 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2)); |
| 255 | 270 | if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS)) |
| 256 | 271 | data |= PCIE_CNTL2__SLV_MEM_LS_EN_MASK; |
| @@ -260,8 +275,8 @@ static void nbio_v7_2_update_medium_grain_light_sleep(struct amdgpu_device *adev · 3 unmodified lines | ||
| 260 | 275 | if (def != data) |
| 261 | 276 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2), data); |
| 262 | 277 | |
| 263 | − data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_TX_POWER_CTRL_1)); | |
| 264 | − def = data; | |
| 278 | + def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, | |
| 279 | + regBIF1_PCIE_TX_POWER_CTRL_1)); | |
| 265 | 280 | if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS)) |
| 266 | 281 | data |= (BIF1_PCIE_TX_POWER_CTRL_1__MST_MEM_LS_EN_MASK | |
| 267 | 282 | BIF1_PCIE_TX_POWER_CTRL_1__REPLAY_MEM_LS_EN_MASK); |
| @@ -272,7 +287,8 @@ static void nbio_v7_2_update_medium_grain_light_sleep(struct amdgpu_device *adev · 4 unmodified lines | ||
| 272 | 287 | if (def != data) |
| 273 | 288 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_TX_POWER_CTRL_1), |
| 274 | 289 | data); |
| 275 | − } else { | |
| 290 | + break; | |
| 291 | + default: | |
| 276 | 292 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2)); |
| 277 | 293 | if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS)) |
| 278 | 294 | data |= (PCIE_CNTL2__SLV_MEM_LS_EN_MASK | |
| @@ -285,6 +301,7 @@ static void nbio_v7_2_update_medium_grain_light_sleep(struct amdgpu_device *adev · 6 unmodified lines | ||
| 285 | 301 | |
| 286 | 302 | if (def != data) |
| 287 | 303 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2), data); |
| 304 | + break; | |
| 288 | 305 | } |
| 289 | 306 | } |
| 290 | 307 | |
| @@ -352,7 +369,10 @@ const struct nbio_hdp_flush_reg nbio_v7_2_hdp_flush_reg = { · 61 unmodified lines | ||
| 352 | 369 | static void nbio_v7_2_init_registers(struct amdgpu_device *adev) |
| 353 | 370 | { |
| 354 | 371 | uint32_t def, data; |
| 355 | − if (adev->asic_type == CHIP_YELLOW_CARP) { | |
| 372 | + switch (adev->ip_versions[NBIO_HWIP][0]) { | |
| 373 | + case IP_VERSION(7, 2, 1): | |
| 374 | + case IP_VERSION(7, 3, 0): | |
| 375 | + case IP_VERSION(7, 5, 0): | |
| 356 | 376 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_MST_CTRL_3)); |
| 357 | 377 | data = REG_SET_FIELD(data, BIF1_PCIE_MST_CTRL_3, |
| 358 | 378 | CI_SWUS_MAX_READ_REQUEST_SIZE_MODE, 1); |
| @@ -361,7 +381,8 @@ static void nbio_v7_2_init_registers(struct amdgpu_device *adev) · 2 unmodified lines | ||
| 361 | 381 | |
| 362 | 382 | if (def != data) |
| 363 | 383 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_MST_CTRL_3), data); |
| 364 | − } else { | |
| 384 | + break; | |
| 385 | + default: | |
| 365 | 386 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CONFIG_CNTL)); |
| 366 | 387 | data = REG_SET_FIELD(data, PCIE_CONFIG_CNTL, |
| 367 | 388 | CI_SWUS_MAX_READ_REQUEST_SIZE_MODE, 1); |
| @@ -370,6 +391,7 @@ static void nbio_v7_2_init_registers(struct amdgpu_device *adev) · 2 unmodified lines | ||
| 370 | 391 | |
| 371 | 392 | if (def != data) |
| 372 | 393 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CONFIG_CNTL), data); |
| 394 | + break; | |
| 373 | 395 | } |
| 374 | 396 | |
| 375 | 397 | if (amdgpu_sriov_vf(adev)) |
| @@ -658,16 +658,28 @@ static void nbio_v7_4_enable_doorbell_interrupt(struct amdgpu_device *adev, · 657 unmodified lines | ||
| 658 | 658 | DOORBELL_INTERRUPT_DISABLE, enable ? 0 : 1); |
| 659 | 659 | } |
| 660 | 660 | |
| 661 | −const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs = { | |
| 661 | +const struct amdgpu_ras_block_hw_ops nbio_v7_4_ras_hw_ops = { | |
| 662 | + .query_ras_error_count = nbio_v7_4_query_ras_error_count, | |
| 663 | +}; | |
| 664 | + | |
| 665 | +struct amdgpu_nbio_ras nbio_v7_4_ras = { | |
| 666 | + .ras_block = { | |
| 667 | + .ras_comm = { | |
| 668 | + .name = "pcie_bif", | |
| 669 | + .block = AMDGPU_RAS_BLOCK__PCIE_BIF, | |
| 670 | + .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | |
| 671 | + }, | |
| 672 | + .hw_ops = &nbio_v7_4_ras_hw_ops, | |
| 673 | + .ras_late_init = amdgpu_nbio_ras_late_init, | |
| 674 | + .ras_fini = amdgpu_nbio_ras_fini, | |
| 675 | + }, | |
| 662 | 676 | .handle_ras_controller_intr_no_bifring = nbio_v7_4_handle_ras_controller_intr_no_bifring, |
| 663 | 677 | .handle_ras_err_event_athub_intr_no_bifring = nbio_v7_4_handle_ras_err_event_athub_intr_no_bifring, |
| 664 | 678 | .init_ras_controller_interrupt = nbio_v7_4_init_ras_controller_interrupt, |
| 665 | 679 | .init_ras_err_event_athub_interrupt = nbio_v7_4_init_ras_err_event_athub_interrupt, |
| 666 | − .query_ras_error_count = nbio_v7_4_query_ras_error_count, | |
| 667 | − .ras_late_init = amdgpu_nbio_ras_late_init, | |
| 668 | − .ras_fini = amdgpu_nbio_ras_fini, | |
| 669 | 680 | }; |
| 670 | 681 | |
| 682 | + | |
| 671 | 683 | static void nbio_v7_4_program_ltr(struct amdgpu_device *adev) |
| 672 | 684 | { |
| 673 | 685 | uint32_t def, data; |
| @@ -29,6 +29,6 @@ · 28 unmodified lines | ||
| 29 | 29 | extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg; |
| 30 | 30 | extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg_ald; |
| 31 | 31 | extern const struct amdgpu_nbio_funcs nbio_v7_4_funcs; |
| 32 | −extern const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs; | |
| 32 | +extern struct amdgpu_nbio_ras nbio_v7_4_ras; | |
| 33 | 33 | |
| 34 | 34 | #endif |
| @@ -204,6 +204,7 @@ static int nv_query_video_codecs(struct amdgpu_device *adev, bool encode, · 203 unmodified lines | ||
| 204 | 204 | *codecs = &sc_video_codecs_decode; |
| 205 | 205 | return 0; |
| 206 | 206 | case IP_VERSION(3, 1, 1): |
| 207 | + case IP_VERSION(3, 1, 2): | |
| 207 | 208 | if (encode) |
| 208 | 209 | *codecs = &nv_video_codecs_encode; |
| 209 | 210 | else |
| @@ -258,21 +259,6 @@ static u64 nv_pcie_rreg64(struct amdgpu_device *adev, u32 reg) · 48 unmodified lines | ||
| 258 | 259 | return amdgpu_device_indirect_rreg64(adev, address, data, reg); |
| 259 | 260 | } |
| 260 | 261 | |
| 261 | −static u32 nv_pcie_port_rreg(struct amdgpu_device *adev, u32 reg) | |
| 262 | −{ | |
| 263 | − unsigned long flags, address, data; | |
| 264 | − u32 r; | |
| 265 | − address = adev->nbio.funcs->get_pcie_port_index_offset(adev); | |
| 266 | − data = adev->nbio.funcs->get_pcie_port_data_offset(adev); | |
| 267 | − | |
| 268 | − spin_lock_irqsave(&adev->pcie_idx_lock, flags); | |
| 269 | − WREG32(address, reg * 4); | |
| 270 | − (void)RREG32(address); | |
| 271 | − r = RREG32(data); | |
| 272 | − spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); | |
| 273 | − return r; | |
| 274 | −} | |
| 275 | − | |
| 276 | 262 | static void nv_pcie_wreg64(struct amdgpu_device *adev, u32 reg, u64 v) |
| 277 | 263 | { |
| 278 | 264 | unsigned long address, data; |
| @@ -283,21 +269,6 @@ static void nv_pcie_wreg64(struct amdgpu_device *adev, u32 reg, u64 v) · 4 unmodified lines | ||
| 283 | 269 | amdgpu_device_indirect_wreg64(adev, address, data, reg, v); |
| 284 | 270 | } |
| 285 | 271 | |
| 286 | −static void nv_pcie_port_wreg(struct amdgpu_device *adev, u32 reg, u32 v) | |
| 287 | −{ | |
| 288 | − unsigned long flags, address, data; | |
| 289 | − | |
| 290 | − address = adev->nbio.funcs->get_pcie_port_index_offset(adev); | |
| 291 | − data = adev->nbio.funcs->get_pcie_port_data_offset(adev); | |
| 292 | − | |
| 293 | − spin_lock_irqsave(&adev->pcie_idx_lock, flags); | |
| 294 | − WREG32(address, reg * 4); | |
| 295 | − (void)RREG32(address); | |
| 296 | − WREG32(data, v); | |
| 297 | − (void)RREG32(data); | |
| 298 | − spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); | |
| 299 | −} | |
| 300 | − | |
| 301 | 272 | static u32 nv_didt_rreg(struct amdgpu_device *adev, u32 reg) |
| 302 | 273 | { |
| 303 | 274 | unsigned long flags, address, data; |
| @@ -360,38 +331,6 @@ static bool nv_read_disabled_bios(struct amdgpu_device *adev) · 56 unmodified lines | ||
| 360 | 331 | return false; |
| 361 | 332 | } |
| 362 | 333 | |
| 363 | −static bool nv_read_bios_from_rom(struct amdgpu_device *adev, | |
| 364 | − u8 *bios, u32 length_bytes) | |
| 365 | −{ | |
| 366 | − u32 *dw_ptr; | |
| 367 | − u32 i, length_dw; | |
| 368 | − u32 rom_index_offset, rom_data_offset; | |
| 369 | − | |
| 370 | − if (bios == NULL) | |
| 371 | − return false; | |
| 372 | − if (length_bytes == 0) | |
| 373 | − return false; | |
| 374 | − /* APU vbios image is part of sbios image */ | |
| 375 | − if (adev->flags & AMD_IS_APU) | |
| 376 | − return false; | |
| 377 | − | |
| 378 | − dw_ptr = (u32 *)bios; | |
| 379 | − length_dw = ALIGN(length_bytes, 4) / 4; | |
| 380 | − | |
| 381 | − rom_index_offset = | |
| 382 | − adev->smuio.funcs->get_rom_index_offset(adev); | |
| 383 | − rom_data_offset = | |
| 384 | − adev->smuio.funcs->get_rom_data_offset(adev); | |
| 385 | − | |
| 386 | − /* set rom index to 0 */ | |
| 387 | − WREG32(rom_index_offset, 0); | |
| 388 | − /* read out the rom data */ | |
| 389 | − for (i = 0; i < length_dw; i++) | |
| 390 | − dw_ptr[i] = RREG32(rom_data_offset); | |
| 391 | − | |
| 392 | − return true; | |
| 393 | −} | |
| 394 | − | |
| 395 | 334 | static struct soc15_allowed_register_entry nv_allowed_read_registers[] = { |
| 396 | 335 | { SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS)}, |
| 397 | 336 | { SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS2)}, |
| @@ -515,6 +454,8 @@ nv_asic_reset_method(struct amdgpu_device *adev) · 117 unmodified lines | ||
| 515 | 454 | case IP_VERSION(11, 5, 0): |
| 516 | 455 | case IP_VERSION(13, 0, 1): |
| 517 | 456 | case IP_VERSION(13, 0, 3): |
| 457 | + case IP_VERSION(13, 0, 5): | |
| 458 | + case IP_VERSION(13, 0, 8): | |
| 518 | 459 | return AMD_RESET_METHOD_MODE2; |
| 519 | 460 | case IP_VERSION(11, 0, 7): |
| 520 | 461 | case IP_VERSION(11, 0, 11): |
| @@ -584,7 +525,7 @@ static void nv_pcie_gen3_enable(struct amdgpu_device *adev) · 63 unmodified lines | ||
| 584 | 525 | |
| 585 | 526 | static void nv_program_aspm(struct amdgpu_device *adev) |
| 586 | 527 | { |
| 587 | − if (!amdgpu_aspm) | |
| 528 | + if (!amdgpu_device_should_use_aspm(adev)) | |
| 588 | 529 | return; |
| 589 | 530 | |
| 590 | 531 | if (!(adev->flags & AMD_IS_APU) && |
| @@ -699,7 +640,8 @@ static int nv_update_umd_stable_pstate(struct amdgpu_device *adev, · 108 unmodified lines | ||
| 699 | 640 | adev->gfx.funcs->update_perfmon_mgcg(adev, !enter); |
| 700 | 641 | |
| 701 | 642 | if (!(adev->flags & AMD_IS_APU) && |
| 702 | − (adev->nbio.funcs->enable_aspm)) | |
| 643 | + (adev->nbio.funcs->enable_aspm) && | |
| 644 | + amdgpu_device_should_use_aspm(adev)) | |
| 703 | 645 | adev->nbio.funcs->enable_aspm(adev, !enter); |
| 704 | 646 | |
| 705 | 647 | return 0; |
| @@ -708,7 +650,7 @@ static int nv_update_umd_stable_pstate(struct amdgpu_device *adev, · 2 unmodified lines | ||
| 708 | 650 | static const struct amdgpu_asic_funcs nv_asic_funcs = |
| 709 | 651 | { |
| 710 | 652 | .read_disabled_bios = &nv_read_disabled_bios, |
| 711 | − .read_bios_from_rom = &nv_read_bios_from_rom, | |
| 653 | + .read_bios_from_rom = &amdgpu_soc15_read_bios_from_rom, | |
| 712 | 654 | .read_register = &nv_read_register, |
| 713 | 655 | .reset = &nv_asic_reset, |
| 714 | 656 | .reset_method = &nv_asic_reset_method, |
| @@ -742,8 +684,8 @@ static int nv_common_early_init(void *handle) · 27 unmodified lines | ||
| 742 | 684 | adev->pcie_wreg = &nv_pcie_wreg; |
| 743 | 685 | adev->pcie_rreg64 = &nv_pcie_rreg64; |
| 744 | 686 | adev->pcie_wreg64 = &nv_pcie_wreg64; |
| 745 | − adev->pciep_rreg = &nv_pcie_port_rreg; | |
| 746 | − adev->pciep_wreg = &nv_pcie_port_wreg; | |
| 687 | + adev->pciep_rreg = amdgpu_device_pcie_port_rreg; | |
| 688 | + adev->pciep_wreg = amdgpu_device_pcie_port_wreg; | |
| 747 | 689 | |
| 748 | 690 | /* TODO: will add them during VCN v2 implementation */ |
| 749 | 691 | adev->uvd_ctx_rreg = NULL; |
| @@ -964,10 +906,52 @@ static int nv_common_early_init(void *handle) · 214 unmodified lines | ||
| 964 | 906 | adev->external_rev_id = adev->rev_id + 0x01; |
| 965 | 907 | break; |
| 966 | 908 | case IP_VERSION(10, 1, 3): |
| 909 | + case IP_VERSION(10, 1, 4): | |
| 967 | 910 | adev->cg_flags = 0; |
| 968 | 911 | adev->pg_flags = 0; |
| 969 | 912 | adev->external_rev_id = adev->rev_id + 0x82; |
| 970 | 913 | break; |
| 914 | + case IP_VERSION(10, 3, 6): | |
| 915 | + adev->cg_flags = AMD_CG_SUPPORT_GFX_MGCG | | |
| 916 | + AMD_CG_SUPPORT_GFX_MGLS | | |
| 917 | + AMD_CG_SUPPORT_GFX_CGCG | | |
| 918 | + AMD_CG_SUPPORT_GFX_CGLS | | |
| 919 | + AMD_CG_SUPPORT_GFX_3D_CGCG | | |
| 920 | + AMD_CG_SUPPORT_GFX_3D_CGLS | | |
| 921 | + AMD_CG_SUPPORT_GFX_RLC_LS | | |
| 922 | + AMD_CG_SUPPORT_GFX_CP_LS | | |
| 923 | + AMD_CG_SUPPORT_GFX_FGCG | | |
| 924 | + AMD_CG_SUPPORT_MC_MGCG | | |
| 925 | + AMD_CG_SUPPORT_MC_LS | | |
| 926 | + AMD_CG_SUPPORT_SDMA_LS | | |
| 927 | + AMD_CG_SUPPORT_HDP_MGCG | | |
| 928 | + AMD_CG_SUPPORT_HDP_LS | | |
| 929 | + AMD_CG_SUPPORT_ATHUB_MGCG | | |
| 930 | + AMD_CG_SUPPORT_ATHUB_LS | | |
| 931 | + AMD_CG_SUPPORT_IH_CG | | |
| 932 | + AMD_CG_SUPPORT_VCN_MGCG | | |
| 933 | + AMD_CG_SUPPORT_JPEG_MGCG; | |
| 934 | + adev->pg_flags = AMD_PG_SUPPORT_GFX_PG | | |
| 935 | + AMD_PG_SUPPORT_VCN | | |
| 936 | + AMD_PG_SUPPORT_VCN_DPG | | |
| 937 | + AMD_PG_SUPPORT_JPEG; | |
| 938 | + adev->external_rev_id = adev->rev_id + 0x01; | |
| 939 | + break; | |
| 940 | + case IP_VERSION(10, 3, 7): | |
| 941 | + adev->cg_flags = AMD_CG_SUPPORT_GFX_MGCG | | |
| 942 | + AMD_CG_SUPPORT_GFX_MGLS | | |
| 943 | + AMD_CG_SUPPORT_GFX_CGCG | | |
| 944 | + AMD_CG_SUPPORT_GFX_CGLS | | |
| 945 | + AMD_CG_SUPPORT_GFX_3D_CGCG | | |
| 946 | + AMD_CG_SUPPORT_GFX_3D_CGLS | | |
| 947 | + AMD_CG_SUPPORT_GFX_RLC_LS | | |
| 948 | + AMD_CG_SUPPORT_GFX_CP_LS | | |
| 949 | + AMD_CG_SUPPORT_GFX_FGCG; | |
| 950 | + adev->pg_flags = AMD_PG_SUPPORT_VCN | | |
| 951 | + AMD_PG_SUPPORT_VCN_DPG | | |
| 952 | + AMD_PG_SUPPORT_JPEG; | |
| 953 | + adev->external_rev_id = adev->rev_id + 0x01; | |
| 954 | + break; | |
| 971 | 955 | default: |
| 972 | 956 | /* FIXME: not supported yet */ |
| 973 | 957 | return -EINVAL; |
| @@ -258,6 +258,7 @@ enum psp_gfx_fw_type { · 257 unmodified lines | ||
| 258 | 258 | GFX_FW_TYPE_SDMA6 = 56, /* SDMA6 MI */ |
| 259 | 259 | GFX_FW_TYPE_SDMA7 = 57, /* SDMA7 MI */ |
| 260 | 260 | GFX_FW_TYPE_VCN1 = 58, /* VCN1 MI */ |
| 261 | + GFX_FW_TYPE_CAP = 62, /* CAP_FW */ | |
| 261 | 262 | GFX_FW_TYPE_REG_LIST = 67, /* REG_LIST MI */ |
| 262 | 263 | GFX_FW_TYPE_MAX |
| 263 | 264 | }; |
| @@ -53,11 +53,13 @@ MODULE_FIRMWARE("amdgpu/navi14_ta.bin"); · 52 unmodified lines | ||
| 53 | 53 | MODULE_FIRMWARE("amdgpu/navi12_sos.bin"); |
| 54 | 54 | MODULE_FIRMWARE("amdgpu/navi12_asd.bin"); |
| 55 | 55 | MODULE_FIRMWARE("amdgpu/navi12_ta.bin"); |
| 56 | +MODULE_FIRMWARE("amdgpu/navi12_cap.bin"); | |
| 56 | 57 | MODULE_FIRMWARE("amdgpu/arcturus_sos.bin"); |
| 57 | 58 | MODULE_FIRMWARE("amdgpu/arcturus_asd.bin"); |
| 58 | 59 | MODULE_FIRMWARE("amdgpu/arcturus_ta.bin"); |
| 59 | 60 | MODULE_FIRMWARE("amdgpu/sienna_cichlid_sos.bin"); |
| 60 | 61 | MODULE_FIRMWARE("amdgpu/sienna_cichlid_ta.bin"); |
| 62 | +MODULE_FIRMWARE("amdgpu/sienna_cichlid_cap.bin"); | |
| 61 | 63 | MODULE_FIRMWARE("amdgpu/navy_flounder_sos.bin"); |
| 62 | 64 | MODULE_FIRMWARE("amdgpu/navy_flounder_ta.bin"); |
| 63 | 65 | MODULE_FIRMWARE("amdgpu/vangogh_asd.bin"); |
| @@ -177,8 +179,6 @@ static int psp_v11_0_init_microcode(struct psp_context *psp) · 113 unmodified lines | ||
| 177 | 179 | err = psp_init_asd_microcode(psp, chip_name); |
| 178 | 180 | if (err) |
| 179 | 181 | return err; |
| 180 | − if (amdgpu_sriov_vf(adev)) | |
| 181 | − break; | |
| 182 | 182 | snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_ta.bin", chip_name); |
| 183 | 183 | err = request_firmware(&adev->psp.ta_fw, fw_name, adev->dev); |
| 184 | 184 | if (err) { |
| @@ -34,6 +34,12 @@ MODULE_FIRMWARE("amdgpu/aldebaran_ta.bin"); · 33 unmodified lines | ||
| 34 | 34 | MODULE_FIRMWARE("amdgpu/yellow_carp_asd.bin"); |
| 35 | 35 | MODULE_FIRMWARE("amdgpu/yellow_carp_toc.bin"); |
| 36 | 36 | MODULE_FIRMWARE("amdgpu/yellow_carp_ta.bin"); |
| 37 | +MODULE_FIRMWARE("amdgpu/psp_13_0_5_asd.bin"); | |
| 38 | +MODULE_FIRMWARE("amdgpu/psp_13_0_5_toc.bin"); | |
| 39 | +MODULE_FIRMWARE("amdgpu/psp_13_0_5_ta.bin"); | |
| 40 | +MODULE_FIRMWARE("amdgpu/psp_13_0_8_asd.bin"); | |
| 41 | +MODULE_FIRMWARE("amdgpu/psp_13_0_8_toc.bin"); | |
| 42 | +MODULE_FIRMWARE("amdgpu/psp_13_0_8_ta.bin"); | |
| 37 | 43 | |
| 38 | 44 | /* For large FW files the time to complete can be very long */ |
| 39 | 45 | #define USBC_PD_POLLING_LIMIT_S 240 |
| @@ -55,6 +61,12 @@ static int psp_v13_0_init_microcode(struct psp_context *psp) · 15 unmodified lines | ||
| 55 | 61 | case IP_VERSION(13, 0, 3): |
| 56 | 62 | chip_name = "yellow_carp"; |
| 57 | 63 | break; |
| 64 | + case IP_VERSION(13, 0, 5): | |
| 65 | + chip_name = "psp_13_0_5"; | |
| 66 | + break; | |
| 67 | + case IP_VERSION(13, 0, 8): | |
| 68 | + chip_name = "psp_13_0_8"; | |
| 69 | + break; | |
| 58 | 70 | default: |
| 59 | 71 | BUG(); |
| 60 | 72 | } |
| @@ -69,6 +81,8 @@ static int psp_v13_0_init_microcode(struct psp_context *psp) · 8 unmodified lines | ||
| 69 | 81 | break; |
| 70 | 82 | case IP_VERSION(13, 0, 1): |
| 71 | 83 | case IP_VERSION(13, 0, 3): |
| 84 | + case IP_VERSION(13, 0, 5): | |
| 85 | + case IP_VERSION(13, 0, 8): | |
| 72 | 86 | err = psp_init_asd_microcode(psp, chip_name); |
| 73 | 87 | if (err) |
| 74 | 88 | return err; |
| @@ -44,6 +44,7 @@ · 43 unmodified lines | ||
| 44 | 44 | |
| 45 | 45 | MODULE_FIRMWARE("amdgpu/vega10_sos.bin"); |
| 46 | 46 | MODULE_FIRMWARE("amdgpu/vega10_asd.bin"); |
| 47 | +MODULE_FIRMWARE("amdgpu/vega10_cap.bin"); | |
| 47 | 48 | MODULE_FIRMWARE("amdgpu/vega12_sos.bin"); |
| 48 | 49 | MODULE_FIRMWARE("amdgpu/vega12_asd.bin"); |
| 49 | 50 | |
| @@ -1885,22 +1885,16 @@ static int sdma_v4_0_process_ras_data_cb(struct amdgpu_device *adev, · 1884 unmodified lines | ||
| 1885 | 1885 | static int sdma_v4_0_late_init(void *handle) |
| 1886 | 1886 | { |
| 1887 | 1887 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1888 | − struct ras_ih_if ih_info = { | |
| 1889 | − .cb = sdma_v4_0_process_ras_data_cb, | |
| 1890 | − }; | |
| 1891 | 1888 | |
| 1892 | 1889 | sdma_v4_0_setup_ulv(adev); |
| 1893 | 1890 | |
| 1894 | 1891 | if (!amdgpu_persistent_edc_harvesting_supported(adev)) { |
| 1895 | − if (adev->sdma.funcs && | |
| 1896 | − adev->sdma.funcs->reset_ras_error_count) | |
| 1897 | − adev->sdma.funcs->reset_ras_error_count(adev); | |
| 1892 | + if (adev->sdma.ras && adev->sdma.ras->ras_block.hw_ops && | |
| 1893 | + adev->sdma.ras->ras_block.hw_ops->reset_ras_error_count) | |
| 1894 | + adev->sdma.ras->ras_block.hw_ops->reset_ras_error_count(adev); | |
| 1898 | 1895 | } |
| 1899 | 1896 | |
| 1900 | − if (adev->sdma.funcs && adev->sdma.funcs->ras_late_init) | |
| 1901 | − return adev->sdma.funcs->ras_late_init(adev, &ih_info); | |
| 1902 | − else | |
| 1903 | − return 0; | |
| 1897 | + return 0; | |
| 1904 | 1898 | } |
| 1905 | 1899 | |
| 1906 | 1900 | static int sdma_v4_0_sw_init(void *handle) |
| @@ -2001,8 +1995,9 @@ static int sdma_v4_0_sw_fini(void *handle) · 94 unmodified lines | ||
| 2001 | 1995 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 2002 | 1996 | int i; |
| 2003 | 1997 | |
| 2004 | − if (adev->sdma.funcs && adev->sdma.funcs->ras_fini) | |
| 2005 | − adev->sdma.funcs->ras_fini(adev); | |
| 1998 | + if (adev->sdma.ras && adev->sdma.ras->ras_block.hw_ops && | |
| 1999 | + adev->sdma.ras->ras_block.ras_fini) | |
| 2000 | + adev->sdma.ras->ras_block.ras_fini(adev); | |
| 2006 | 2001 | |
| 2007 | 2002 | for (i = 0; i < adev->sdma.num_instances; i++) { |
| 2008 | 2003 | amdgpu_ring_fini(&adev->sdma.instance[i].ring); |
| @@ -2057,6 +2052,10 @@ static int sdma_v4_0_suspend(void *handle) · 48 unmodified lines | ||
| 2057 | 2052 | { |
| 2058 | 2053 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 2059 | 2054 | |
| 2055 | + /* SMU saves SDMA state for us */ | |
| 2056 | + if (adev->in_s0ix) | |
| 2057 | + return 0; | |
| 2058 | + | |
| 2060 | 2059 | return sdma_v4_0_hw_fini(adev); |
| 2061 | 2060 | } |
| 2062 | 2061 | |
| @@ -2064,6 +2063,10 @@ static int sdma_v4_0_resume(void *handle) · 1 unmodified line | ||
| 2064 | 2063 | { |
| 2065 | 2064 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 2066 | 2065 | |
| 2066 | + /* SMU restores SDMA state for us */ | |
| 2067 | + if (adev->in_s0ix) | |
| 2068 | + return 0; | |
| 2069 | + | |
| 2067 | 2070 | return sdma_v4_0_hw_init(adev); |
| 2068 | 2071 | } |
| 2069 | 2072 | |
| @@ -2740,7 +2743,7 @@ static void sdma_v4_0_get_ras_error_count(uint32_t value, · 670 unmodified lines | ||
| 2740 | 2743 | } |
| 2741 | 2744 | } |
| 2742 | 2745 | |
| 2743 | −static int sdma_v4_0_query_ras_error_count(struct amdgpu_device *adev, | |
| 2746 | +static int sdma_v4_0_query_ras_error_count_by_instance(struct amdgpu_device *adev, | |
| 2744 | 2747 | uint32_t instance, void *ras_error_status) |
| 2745 | 2748 | { |
| 2746 | 2749 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| @@ -2762,6 +2765,18 @@ static int sdma_v4_0_query_ras_error_count(struct amdgpu_device *adev, · 15 unmodified lines | ||
| 2762 | 2765 | return 0; |
| 2763 | 2766 | }; |
| 2764 | 2767 | |
| 2768 | +static void sdma_v4_0_query_ras_error_count(struct amdgpu_device *adev, void *ras_error_status) | |
| 2769 | +{ | |
| 2770 | + int i = 0; | |
| 2771 | + | |
| 2772 | + for (i = 0; i < adev->sdma.num_instances; i++) { | |
| 2773 | + if (sdma_v4_0_query_ras_error_count_by_instance(adev, i, ras_error_status)) { | |
| 2774 | + dev_err(adev->dev, "Query ras error count failed in SDMA%d\n", i); | |
| 2775 | + return; | |
| 2776 | + } | |
| 2777 | + } | |
| 2778 | +} | |
| 2779 | + | |
| 2765 | 2780 | static void sdma_v4_0_reset_ras_error_count(struct amdgpu_device *adev) |
| 2766 | 2781 | { |
| 2767 | 2782 | int i; |
| @@ -2773,26 +2788,52 @@ static void sdma_v4_0_reset_ras_error_count(struct amdgpu_device *adev) · 5 unmodified lines | ||
| 2773 | 2788 | } |
| 2774 | 2789 | } |
| 2775 | 2790 | |
| 2776 | −static const struct amdgpu_sdma_ras_funcs sdma_v4_0_ras_funcs = { | |
| 2777 | − .ras_late_init = amdgpu_sdma_ras_late_init, | |
| 2778 | − .ras_fini = amdgpu_sdma_ras_fini, | |
| 2791 | +const struct amdgpu_ras_block_hw_ops sdma_v4_0_ras_hw_ops = { | |
| 2779 | 2792 | .query_ras_error_count = sdma_v4_0_query_ras_error_count, |
| 2780 | 2793 | .reset_ras_error_count = sdma_v4_0_reset_ras_error_count, |
| 2781 | 2794 | }; |
| 2782 | 2795 | |
| 2796 | +static struct amdgpu_sdma_ras sdma_v4_0_ras = { | |
| 2797 | + .ras_block = { | |
| 2798 | + .hw_ops = &sdma_v4_0_ras_hw_ops, | |
| 2799 | + .ras_cb = sdma_v4_0_process_ras_data_cb, | |
| 2800 | + }, | |
| 2801 | +}; | |
| 2802 | + | |
| 2783 | 2803 | static void sdma_v4_0_set_ras_funcs(struct amdgpu_device *adev) |
| 2784 | 2804 | { |
| 2785 | 2805 | switch (adev->ip_versions[SDMA0_HWIP][0]) { |
| 2786 | 2806 | case IP_VERSION(4, 2, 0): |
| 2787 | 2807 | case IP_VERSION(4, 2, 2): |
| 2788 | − adev->sdma.funcs = &sdma_v4_0_ras_funcs; | |
| 2808 | + adev->sdma.ras = &sdma_v4_0_ras; | |
| 2789 | 2809 | break; |
| 2790 | 2810 | case IP_VERSION(4, 4, 0): |
| 2791 | − adev->sdma.funcs = &sdma_v4_4_ras_funcs; | |
| 2811 | + adev->sdma.ras = &sdma_v4_4_ras; | |
| 2792 | 2812 | break; |
| 2793 | 2813 | default: |
| 2794 | 2814 | break; |
| 2795 | 2815 | } |
24 more lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 246 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 384 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 235 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 2085 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 846 lines aren’t shown here.
This diff is large: its 231 lines aren’t shown here.
This diff is large: its 159 lines aren’t shown here.
This diff is large: its 294 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 448 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 474 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 197 lines aren’t shown here.
This diff is large: its 242 lines aren’t shown here.
This diff is large: its 254 lines aren’t shown here.
This diff is large: its 231 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 350 lines aren’t shown here.
This diff is large: its 207 lines aren’t shown here.
This diff is large: its 422 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 601 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 425 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 337 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 369 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 487 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 618 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 315 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 716 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 313 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 166 lines aren’t shown here.
This diff is large: its 1592 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 2589 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 918 lines aren’t shown here.
This diff is large: its 280 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 164 lines aren’t shown here.
This diff is large: its 269 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 320 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 359 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 256 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 2305 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 2307 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This diff is large: its 59 lines aren’t shown here.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This diff is large: its 358 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 261 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 375 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 439 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 255 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 15192 lines aren’t shown here.
This diff is larger than a page shows: the files after these aren’t listed. Clone the repository to see all of it.