To catch up with recent rounds of pull requests and get some drm-misc dependencies so we can merge linux/string_helpers related changes.
Unified · Split
| @@ -80,6 +80,9 @@ Chris Chiu <chris.chiu@canonical.com> <chiu@endlessos.org> · 79 unmodified lines | |||
| 80 | Christian Borntraeger <borntraeger@linux.ibm.com> <borntraeger@de.ibm.com> | 80 | Christian Borntraeger <borntraeger@linux.ibm.com> <borntraeger@de.ibm.com> |
| 81 | Christian Borntraeger <borntraeger@linux.ibm.com> <cborntra@de.ibm.com> | 81 | Christian Borntraeger <borntraeger@linux.ibm.com> <cborntra@de.ibm.com> |
| 82 | Christian Borntraeger <borntraeger@linux.ibm.com> <borntrae@de.ibm.com> | 82 | Christian Borntraeger <borntraeger@linux.ibm.com> <borntrae@de.ibm.com> |
| 83 | Christian Brauner <brauner@kernel.org> <christian@brauner.io> | ||
| 84 | Christian Brauner <brauner@kernel.org> <christian.brauner@canonical.com> | ||
| 85 | Christian Brauner <brauner@kernel.org> <christian.brauner@ubuntu.com> | ||
| 83 | Christophe Ricard <christophe.ricard@gmail.com> | 86 | Christophe Ricard <christophe.ricard@gmail.com> |
| 84 | Christoph Hellwig <hch@lst.de> | 87 | Christoph Hellwig <hch@lst.de> |
| 85 | Colin Ian King <colin.king@intel.com> <colin.king@canonical.com> | 88 | Colin Ian King <colin.king@intel.com> <colin.king@canonical.com> |
| @@ -330,6 +333,9 @@ Rémi Denis-Courmont <rdenis@simphalempin.com> · 244 unmodified lines | |||
| 330 | Ricardo Ribalda <ribalda@kernel.org> <ricardo@ribalda.com> | 333 | Ricardo Ribalda <ribalda@kernel.org> <ricardo@ribalda.com> |
| 331 | Ricardo Ribalda <ribalda@kernel.org> Ricardo Ribalda Delgado <ribalda@kernel.org> | 334 | Ricardo Ribalda <ribalda@kernel.org> Ricardo Ribalda Delgado <ribalda@kernel.org> |
| 332 | Ricardo Ribalda <ribalda@kernel.org> <ricardo.ribalda@gmail.com> | 335 | Ricardo Ribalda <ribalda@kernel.org> <ricardo.ribalda@gmail.com> |
| 336 | Roman Gushchin <roman.gushchin@linux.dev> <guro@fb.com> | ||
| 337 | Roman Gushchin <roman.gushchin@linux.dev> <guroan@gmail.com> | ||
| 338 | Roman Gushchin <roman.gushchin@linux.dev> <klamm@yandex-team.ru> | ||
| 333 | Ross Zwisler <zwisler@kernel.org> <ross.zwisler@linux.intel.com> | 339 | Ross Zwisler <zwisler@kernel.org> <ross.zwisler@linux.intel.com> |
| 334 | Rudolf Marek <R.Marek@sh.cvut.cz> | 340 | Rudolf Marek <R.Marek@sh.cvut.cz> |
| 335 | Rui Saraiva <rmps@joel.ist.utl.pt> | 341 | Rui Saraiva <rmps@joel.ist.utl.pt> |
| @@ -468,6 +468,7 @@ Description: · 467 unmodified lines | |||
| 468 | auto: Charge normally, respect thresholds | 468 | auto: Charge normally, respect thresholds |
| 469 | inhibit-charge: Do not charge while AC is attached | 469 | inhibit-charge: Do not charge while AC is attached |
| 470 | force-discharge: Force discharge while AC is attached | 470 | force-discharge: Force discharge while AC is attached |
| 471 | ================ ==================================== | ||
| 471 | 472 | ||
| 472 | What: /sys/class/power_supply/<supply_name>/technology | 473 | What: /sys/class/power_supply/<supply_name>/technology |
| 473 | Date: May 2007 | 474 | Date: May 2007 |
| @@ -1,4 +1,4 @@ | |||
| 1 | What: /sys/bus/platform/drivers/aspeed-uart-routing/*/uart* | 1 | What: /sys/bus/platform/drivers/aspeed-uart-routing/\*/uart\* |
| 2 | Date: September 2021 | 2 | Date: September 2021 |
| 3 | Contact: Oskar Senft <osk@google.com> | 3 | Contact: Oskar Senft <osk@google.com> |
| 4 | Chia-Wei Wang <chiawei_wang@aspeedtech.com> | 4 | Chia-Wei Wang <chiawei_wang@aspeedtech.com> |
| @@ -9,7 +9,7 @@ Description: Selects the RX source of the UARTx device. · 4 unmodified lines | |||
| 9 | depends on the selected file. | 9 | depends on the selected file. |
| 10 | 10 | ||
| 11 | e.g. | 11 | e.g. |
| 12 | cat /sys/bus/platform/drivers/aspeed-uart-routing/*.uart_routing/uart1 | 12 | cat /sys/bus/platform/drivers/aspeed-uart-routing/\*.uart_routing/uart1 |
| 13 | [io1] io2 io3 io4 uart2 uart3 uart4 io6 | 13 | [io1] io2 io3 io4 uart2 uart3 uart4 io6 |
| 14 | 14 | ||
| 15 | In this case, UART1 gets its input from IO1 (physical serial port 1). | 15 | In this case, UART1 gets its input from IO1 (physical serial port 1). |
| @@ -17,7 +17,7 @@ Description: Selects the RX source of the UARTx device. · 1 unmodified line | |||
| 17 | Users: OpenBMC. Proposed changes should be mailed to | 17 | Users: OpenBMC. Proposed changes should be mailed to |
| 18 | openbmc@lists.ozlabs.org | 18 | openbmc@lists.ozlabs.org |
| 19 | 19 | ||
| 20 | What: /sys/bus/platform/drivers/aspeed-uart-routing/*/io* | 20 | What: /sys/bus/platform/drivers/aspeed-uart-routing/\*/io\* |
| 21 | Date: September 2021 | 21 | Date: September 2021 |
| 22 | Contact: Oskar Senft <osk@google.com> | 22 | Contact: Oskar Senft <osk@google.com> |
| 23 | Chia-Wei Wang <chiawei_wang@aspeedtech.com> | 23 | Chia-Wei Wang <chiawei_wang@aspeedtech.com> |
| @@ -100,6 +100,8 @@ stable kernels. · 99 unmodified lines | |||
| 100 | +----------------+-----------------+-----------------+-----------------------------+ | 100 | +----------------+-----------------+-----------------+-----------------------------+ |
| 101 | | ARM | Cortex-A510 | #2051678 | ARM64_ERRATUM_2051678 | | 101 | | ARM | Cortex-A510 | #2051678 | ARM64_ERRATUM_2051678 | |
| 102 | +----------------+-----------------+-----------------+-----------------------------+ | 102 | +----------------+-----------------+-----------------+-----------------------------+ |
| 103 | | ARM | Cortex-A510 | #2077057 | ARM64_ERRATUM_2077057 | | ||
| 104 | +----------------+-----------------+-----------------+-----------------------------+ | ||
| 103 | | ARM | Cortex-A710 | #2119858 | ARM64_ERRATUM_2119858 | | 105 | | ARM | Cortex-A710 | #2119858 | ARM64_ERRATUM_2119858 | |
| 104 | +----------------+-----------------+-----------------+-----------------------------+ | 106 | +----------------+-----------------+-----------------+-----------------------------+ |
| 105 | | ARM | Cortex-A710 | #2054223 | ARM64_ERRATUM_2054223 | | 107 | | ARM | Cortex-A710 | #2054223 | ARM64_ERRATUM_2054223 | |
| @@ -130,3 +130,11 @@ accesses to DMA buffers in both privileged "supervisor" and unprivileged · 129 unmodified lines | |||
| 130 | subsystem that the buffer is fully accessible at the elevated privilege | 130 | subsystem that the buffer is fully accessible at the elevated privilege |
| 131 | level (and ideally inaccessible or at least read-only at the | 131 | level (and ideally inaccessible or at least read-only at the |
| 132 | lesser-privileged levels). | 132 | lesser-privileged levels). |
| 133 | |||
| 134 | DMA_ATTR_OVERWRITE | ||
| 135 | ------------------ | ||
| 136 | |||
| 137 | This is a hint to the DMA-mapping subsystem that the device is expected to | ||
| 138 | overwrite the entire mapped size, thus the caller does not require any of the | ||
| 139 | previous buffer contents to be preserved. This allows bounce-buffering | ||
| 140 | implementations to optimise DMA_FROM_DEVICE transfers. | ||
| @@ -75,6 +75,9 @@ And optionally · 74 unmodified lines | |||
| 75 | .resume - A pointer to a per-policy resume function which is called | 75 | .resume - A pointer to a per-policy resume function which is called |
| 76 | with interrupts disabled and _before_ the governor is started again. | 76 | with interrupts disabled and _before_ the governor is started again. |
| 77 | 77 | ||
| 78 | .ready - A pointer to a per-policy ready function which is called after | ||
| 79 | the policy is fully initialized. | ||
| 80 | |||
| 78 | .attr - A pointer to a NULL-terminated list of "struct freq_attr" which | 81 | .attr - A pointer to a NULL-terminated list of "struct freq_attr" which |
| 79 | allow to export values to sysfs. | 82 | allow to export values to sysfs. |
| 80 | 83 | ||
| @@ -7,6 +7,14 @@ directory. These are intended to be small tests to exercise individual code · 6 unmodified lines | |||
| 7 | paths in the kernel. Tests are intended to be run after building, installing | 7 | paths in the kernel. Tests are intended to be run after building, installing |
| 8 | and booting a kernel. | 8 | and booting a kernel. |
| 9 | 9 | ||
| 10 | Kselftest from mainline can be run on older stable kernels. Running tests | ||
| 11 | from mainline offers the best coverage. Several test rings run mainline | ||
| 12 | kselftest suite on stable releases. The reason is that when a new test | ||
| 13 | gets added to test existing code to regression test a bug, we should be | ||
| 14 | able to run that test on an older kernel. Hence, it is important to keep | ||
| 15 | code that can still test an older kernel and make sure it skips the test | ||
| 16 | gracefully on newer releases. | ||
| 17 | |||
| 10 | You can find additional information on Kselftest framework, how to | 18 | You can find additional information on Kselftest framework, how to |
| 11 | write new tests using the framework on Kselftest wiki: | 19 | write new tests using the framework on Kselftest wiki: |
| 12 | 20 | ||
| @@ -242,7 +242,7 @@ example: · 241 unmodified lines | |||
| 242 | 242 | ||
| 243 | int rectangle_area(struct shape *this) | 243 | int rectangle_area(struct shape *this) |
| 244 | { | 244 | { |
| 245 | struct rectangle *self = container_of(this, struct shape, parent); | 245 | struct rectangle *self = container_of(this, struct rectangle, parent); |
| 246 | 246 | ||
| 247 | return self->length * self->width; | 247 | return self->length * self->width; |
| 248 | }; | 248 | }; |
| @@ -119,6 +119,9 @@ Boards (incomplete list of examples): · 118 unmodified lines | |||
| 119 | - OMAP3 BeagleBoard : Low cost community board | 119 | - OMAP3 BeagleBoard : Low cost community board |
| 120 | compatible = "ti,omap3-beagle", "ti,omap3430", "ti,omap3" | 120 | compatible = "ti,omap3-beagle", "ti,omap3430", "ti,omap3" |
| 121 | 121 | ||
| 122 | - OMAP3 BeagleBoard A to B4 : Early BeagleBoard revisions A to B4 with a timer quirk | ||
| 123 | compatible = "ti,omap3-beagle-ab4", "ti,omap3-beagle", "ti,omap3430", "ti,omap3" | ||
| 124 | |||
| 122 | - OMAP3 Tobi with Overo : Commercial expansion board with daughter board | 125 | - OMAP3 Tobi with Overo : Commercial expansion board with daughter board |
| 123 | compatible = "gumstix,omap3-overo-tobi", "gumstix,omap3-overo", "ti,omap3430", "ti,omap3" | 126 | compatible = "gumstix,omap3-overo-tobi", "gumstix,omap3-overo", "ti,omap3430", "ti,omap3" |
| 124 | 127 | ||
| @@ -83,6 +83,9 @@ properties: · 82 unmodified lines | |||
| 83 | type: boolean | 83 | type: boolean |
| 84 | description: let the driver enable audio HDMI codec function or not. | 84 | description: let the driver enable audio HDMI codec function or not. |
| 85 | 85 | ||
| 86 | aux-bus: | ||
| 87 | $ref: /schemas/display/dp-aux-bus.yaml# | ||
| 88 | |||
| 86 | ports: | 89 | ports: |
| 87 | $ref: /schemas/graph.yaml#/properties/ports | 90 | $ref: /schemas/graph.yaml#/properties/ports |
| 88 | 91 | ||
| @@ -167,5 +170,19 @@ examples: · 78 unmodified lines | |||
| 167 | }; | 170 | }; |
| 168 | }; | 171 | }; |
| 169 | }; | 172 | }; |
| 173 | |||
| 174 | aux-bus { | ||
| 175 | panel { | ||
| 176 | compatible = "innolux,n125hce-gn1"; | ||
| 177 | power-supply = <&pp3300_disp_x>; | ||
| 178 | backlight = <&backlight_lcd0>; | ||
| 179 | |||
| 180 | port { | ||
| 181 | panel_in: endpoint { | ||
| 182 | remote-endpoint = <&anx7625_out>; | ||
| 183 | }; | ||
| 184 | }; | ||
| 185 | }; | ||
| 186 | }; | ||
| 170 | }; | 187 | }; |
| 171 | }; | 188 | }; |
| @@ -0,0 +1,82 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/bridge/ingenic,jz4780-hdmi.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Bindings for Ingenic JZ4780 HDMI Transmitter | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - H. Nikolaus Schaller <hns@goldelico.com> | ||
| 11 | |||
| 12 | description: | | ||
| 13 | The HDMI Transmitter in the Ingenic JZ4780 is a Synopsys DesignWare HDMI 1.4 | ||
| 14 | TX controller IP with accompanying PHY IP. | ||
| 15 | |||
| 16 | allOf: | ||
| 17 | - $ref: synopsys,dw-hdmi.yaml# | ||
| 18 | |||
| 19 | properties: | ||
| 20 | compatible: | ||
| 21 | const: ingenic,jz4780-dw-hdmi | ||
| 22 | |||
| 23 | reg-io-width: | ||
| 24 | const: 4 | ||
| 25 | |||
| 26 | clocks: | ||
| 27 | maxItems: 2 | ||
| 28 | |||
| 29 | ports: | ||
| 30 | $ref: /schemas/graph.yaml#/properties/ports | ||
| 31 | |||
| 32 | properties: | ||
| 33 | port@0: | ||
| 34 | $ref: /schemas/graph.yaml#/properties/port | ||
| 35 | description: Input from LCD controller output. | ||
| 36 | |||
| 37 | port@1: | ||
| 38 | $ref: /schemas/graph.yaml#/properties/port | ||
| 39 | description: Link to the HDMI connector. | ||
| 40 | |||
| 41 | required: | ||
| 42 | - compatible | ||
| 43 | - clocks | ||
| 44 | - clock-names | ||
| 45 | - ports | ||
| 46 | - reg-io-width | ||
| 47 | |||
| 48 | unevaluatedProperties: false | ||
| 49 | |||
| 50 | examples: | ||
| 51 | - | | ||
| 52 | #include <dt-bindings/clock/ingenic,jz4780-cgu.h> | ||
| 53 | |||
| 54 | hdmi: hdmi@10180000 { | ||
| 55 | compatible = "ingenic,jz4780-dw-hdmi"; | ||
| 56 | reg = <0x10180000 0x8000>; | ||
| 57 | reg-io-width = <4>; | ||
| 58 | ddc-i2c-bus = <&i2c4>; | ||
| 59 | interrupt-parent = <&intc>; | ||
| 60 | interrupts = <3>; | ||
| 61 | clocks = <&cgu JZ4780_CLK_AHB0>, <&cgu JZ4780_CLK_HDMI>; | ||
| 62 | clock-names = "iahb", "isfr"; | ||
| 63 | |||
| 64 | ports { | ||
| 65 | #address-cells = <1>; | ||
| 66 | #size-cells = <0>; | ||
| 67 | hdmi_in: port@0 { | ||
| 68 | reg = <0>; | ||
| 69 | dw_hdmi_in: endpoint { | ||
| 70 | remote-endpoint = <&jz4780_lcd_out>; | ||
| 71 | }; | ||
| 72 | }; | ||
| 73 | hdmi_out: port@1 { | ||
| 74 | reg = <1>; | ||
| 75 | dw_hdmi_out: endpoint { | ||
| 76 | remote-endpoint = <&hdmi_con>; | ||
| 77 | }; | ||
| 78 | }; | ||
| 79 | }; | ||
| 80 | }; | ||
| 81 | |||
| 82 | ... | ||
| @@ -68,7 +68,7 @@ properties: · 67 unmodified lines | |||
| 68 | - vesa-24 | 68 | - vesa-24 |
| 69 | description: | | 69 | description: | |
| 70 | The color signals mapping order. See details in | 70 | The color signals mapping order. See details in |
| 71 | Documentation/devicetree/bindings/display/panel/lvds.yaml | 71 | Documentation/devicetree/bindings/display/lvds.yaml |
| 72 | 72 | ||
| 73 | port@1: | 73 | port@1: |
| 74 | $ref: /schemas/graph.yaml#/properties/port | 74 | $ref: /schemas/graph.yaml#/properties/port |
| @@ -1,10 +1,10 @@ | |||
| 1 | # SPDX-License-Identifier: GPL-2.0 | 1 | # SPDX-License-Identifier: GPL-2.0 |
| 2 | %YAML 1.2 | 2 | %YAML 1.2 |
| 3 | --- | 3 | --- |
| 4 | $id: http://devicetree.org/schemas/display/panel/lvds.yaml# | 4 | $id: http://devicetree.org/schemas/display/lvds.yaml# |
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# |
| 6 | 6 | ||
| 7 | title: LVDS Display Panel | 7 | title: LVDS Display Common Properties |
| 8 | 8 | ||
| 9 | maintainers: | 9 | maintainers: |
| 10 | - Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com> | 10 | - Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com> |
| @@ -13,8 +13,8 @@ maintainers: · 2 unmodified lines | |||
| 13 | description: |+ | 13 | description: |+ |
| 14 | LVDS is a physical layer specification defined in ANSI/TIA/EIA-644-A. Multiple | 14 | LVDS is a physical layer specification defined in ANSI/TIA/EIA-644-A. Multiple |
| 15 | incompatible data link layers have been used over time to transmit image data | 15 | incompatible data link layers have been used over time to transmit image data |
| 16 | to LVDS panels. This bindings supports display panels compatible with the | 16 | to LVDS devices. This bindings supports devices compatible with the following |
| 17 | following specifications. | 17 | specifications. |
| 18 | 18 | ||
| 19 | [JEIDA] "Digital Interface Standards for Monitor", JEIDA-59-1999, February | 19 | [JEIDA] "Digital Interface Standards for Monitor", JEIDA-59-1999, February |
| 20 | 1999 (Version 1.0), Japan Electronic Industry Development Association (JEIDA) | 20 | 1999 (Version 1.0), Japan Electronic Industry Development Association (JEIDA) |
| @@ -26,18 +26,7 @@ description: |+ · 5 unmodified lines | |||
| 26 | Device compatible with those specifications have been marketed under the | 26 | Device compatible with those specifications have been marketed under the |
| 27 | FPD-Link and FlatLink brands. | 27 | FPD-Link and FlatLink brands. |
| 28 | 28 | ||
| 29 | allOf: | ||
| 30 | - $ref: panel-common.yaml# | ||
| 31 | |||
| 32 | properties: | 29 | properties: |
| 33 | compatible: | ||
| 34 | contains: | ||
| 35 | const: panel-lvds | ||
| 36 | description: | ||
| 37 | Shall contain "panel-lvds" in addition to a mandatory panel-specific | ||
| 38 | compatible string defined in individual panel bindings. The "panel-lvds" | ||
| 39 | value shall never be used on its own. | ||
| 40 | |||
| 41 | data-mapping: | 30 | data-mapping: |
| 42 | enum: | 31 | enum: |
| 43 | - jeida-18 | 32 | - jeida-18 |
| @@ -96,22 +85,6 @@ properties: · 52 unmodified lines | |||
| 96 | If set, reverse the bit order described in the data mappings below on all | 85 | If set, reverse the bit order described in the data mappings below on all |
| 97 | data lanes, transmitting bits for slots 6 to 0 instead of 0 to 6. | 86 | data lanes, transmitting bits for slots 6 to 0 instead of 0 to 6. |
| 98 | 87 | ||
| 99 | port: true | ||
| 100 | ports: true | ||
| 101 | |||
| 102 | required: | ||
| 103 | - compatible | ||
| 104 | - data-mapping | ||
| 105 | - width-mm | ||
| 106 | - height-mm | ||
| 107 | - panel-timing | ||
| 108 | |||
| 109 | oneOf: | ||
| 110 | - required: | ||
| 111 | - port | ||
| 112 | - required: | ||
| 113 | - ports | ||
| 114 | |||
| 115 | additionalProperties: true | 88 | additionalProperties: true |
| 116 | 89 | ||
| 117 | ... | 90 | ... |
| @@ -0,0 +1,77 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,aal.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display adaptive ambient light processor | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display adaptive ambient light processor, namely AAL, | ||
| 15 | is responsible for backlight power saving and sunlight visibility improving. | ||
| 16 | AAL device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt8173-disp-aal | ||
| 26 | - items: | ||
| 27 | - enum: | ||
| 28 | - mediatek,mt2712-disp-aal | ||
| 29 | - mediatek,mt8183-disp-aal | ||
| 30 | - mediatek,mt8192-disp-aal | ||
| 31 | - mediatek,mt8195-disp-aal | ||
| 32 | - enum: | ||
| 33 | - mediatek,mt8173-disp-aal | ||
| 34 | |||
| 35 | reg: | ||
| 36 | maxItems: 1 | ||
| 37 | |||
| 38 | interrupts: | ||
| 39 | maxItems: 1 | ||
| 40 | |||
| 41 | power-domains: | ||
| 42 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 43 | the power controller specified by phandle. See | ||
| 44 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 45 | |||
| 46 | clocks: | ||
| 47 | items: | ||
| 48 | - description: AAL Clock | ||
| 49 | |||
| 50 | mediatek,gce-client-reg: | ||
| 51 | description: The register of client driver can be configured by gce with | ||
| 52 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 53 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 54 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 55 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 56 | maxItems: 1 | ||
| 57 | |||
| 58 | required: | ||
| 59 | - compatible | ||
| 60 | - reg | ||
| 61 | - interrupts | ||
| 62 | - power-domains | ||
| 63 | - clocks | ||
| 64 | |||
| 65 | additionalProperties: false | ||
| 66 | |||
| 67 | examples: | ||
| 68 | - | | ||
| 69 | |||
| 70 | aal@14015000 { | ||
| 71 | compatible = "mediatek,mt8173-disp-aal"; | ||
| 72 | reg = <0 0x14015000 0 0x1000>; | ||
| 73 | interrupts = <GIC_SPI 189 IRQ_TYPE_LEVEL_LOW>; | ||
| 74 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 75 | clocks = <&mmsys CLK_MM_DISP_AAL>; | ||
| 76 | mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x5000 0x1000>; | ||
| 77 | }; | ||
| @@ -0,0 +1,76 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,ccorr.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display color correction | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display color correction, namely CCORR, reproduces correct color | ||
| 15 | on panels with different color gamut. | ||
| 16 | CCORR device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt8183-disp-ccorr | ||
| 26 | - items: | ||
| 27 | - const: mediatek,mt8192-disp-ccorr | ||
| 28 | - items: | ||
| 29 | - enum: | ||
| 30 | - mediatek,mt8195-disp-ccorr | ||
| 31 | - enum: | ||
| 32 | - mediatek,mt8192-disp-ccorr | ||
| 33 | |||
| 34 | reg: | ||
| 35 | maxItems: 1 | ||
| 36 | |||
| 37 | interrupts: | ||
| 38 | maxItems: 1 | ||
| 39 | |||
| 40 | power-domains: | ||
| 41 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 42 | the power controller specified by phandle. See | ||
| 43 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 44 | |||
| 45 | clocks: | ||
| 46 | items: | ||
| 47 | - description: CCORR Clock | ||
| 48 | |||
| 49 | mediatek,gce-client-reg: | ||
| 50 | description: The register of client driver can be configured by gce with | ||
| 51 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 52 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 53 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 54 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 55 | maxItems: 1 | ||
| 56 | |||
| 57 | required: | ||
| 58 | - compatible | ||
| 59 | - reg | ||
| 60 | - interrupts | ||
| 61 | - power-domains | ||
| 62 | - clocks | ||
| 63 | |||
| 64 | additionalProperties: false | ||
| 65 | |||
| 66 | examples: | ||
| 67 | - | | ||
| 68 | |||
| 69 | ccorr0: ccorr@1400f000 { | ||
| 70 | compatible = "mediatek,mt8183-disp-ccorr"; | ||
| 71 | reg = <0 0x1400f000 0 0x1000>; | ||
| 72 | interrupts = <GIC_SPI 232 IRQ_TYPE_LEVEL_LOW>; | ||
| 73 | power-domains = <&spm MT8183_POWER_DOMAIN_DISP>; | ||
| 74 | clocks = <&mmsys CLK_MM_DISP_CCORR0>; | ||
| 75 | mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0xf000 0x1000>; | ||
| 76 | }; | ||
| @@ -0,0 +1,86 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,color.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display color processor | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display color processor, namely COLOR, provides hue, luma and | ||
| 15 | saturation adjustments to get better picture quality and to have one panel | ||
| 16 | resemble the other in their output characteristics. | ||
| 17 | COLOR device node must be siblings to the central MMSYS_CONFIG node. | ||
| 18 | For a description of the MMSYS_CONFIG binding, see | ||
| 19 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 20 | for details. | ||
| 21 | |||
| 22 | properties: | ||
| 23 | compatible: | ||
| 24 | oneOf: | ||
| 25 | - items: | ||
| 26 | - const: mediatek,mt2701-disp-color | ||
| 27 | - items: | ||
| 28 | - const: mediatek,mt8167-disp-color | ||
| 29 | - items: | ||
| 30 | - const: mediatek,mt8173-disp-color | ||
| 31 | - items: | ||
| 32 | - enum: | ||
| 33 | - mediatek,mt7623-disp-color | ||
| 34 | - mediatek,mt2712-disp-color | ||
| 35 | - enum: | ||
| 36 | - mediatek,mt2701-disp-color | ||
| 37 | - items: | ||
| 38 | - enum: | ||
| 39 | - mediatek,mt8183-disp-color | ||
| 40 | - mediatek,mt8192-disp-color | ||
| 41 | - mediatek,mt8195-disp-color | ||
| 42 | - enum: | ||
| 43 | - mediatek,mt8173-disp-color | ||
| 44 | reg: | ||
| 45 | maxItems: 1 | ||
| 46 | |||
| 47 | interrupts: | ||
| 48 | maxItems: 1 | ||
| 49 | |||
| 50 | power-domains: | ||
| 51 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 52 | the power controller specified by phandle. See | ||
| 53 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 54 | |||
| 55 | clocks: | ||
| 56 | items: | ||
| 57 | - description: COLOR Clock | ||
| 58 | |||
| 59 | mediatek,gce-client-reg: | ||
| 60 | description: The register of client driver can be configured by gce with | ||
| 61 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 62 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 63 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 64 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 65 | maxItems: 1 | ||
| 66 | |||
| 67 | required: | ||
| 68 | - compatible | ||
| 69 | - reg | ||
| 70 | - interrupts | ||
| 71 | - power-domains | ||
| 72 | - clocks | ||
| 73 | |||
| 74 | additionalProperties: false | ||
| 75 | |||
| 76 | examples: | ||
| 77 | - | | ||
| 78 | |||
| 79 | color0: color@14013000 { | ||
| 80 | compatible = "mediatek,mt8173-disp-color"; | ||
| 81 | reg = <0 0x14013000 0 0x1000>; | ||
| 82 | interrupts = <GIC_SPI 187 IRQ_TYPE_LEVEL_LOW>; | ||
| 83 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 84 | clocks = <&mmsys CLK_MM_DISP_COLOR0>; | ||
| 85 | mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x3000 0x1000>; | ||
| 86 | }; | ||
| @@ -1,219 +0,0 @@ | |||
| 1 | Mediatek display subsystem | ||
| 2 | ========================== | ||
| 3 | |||
| 4 | The Mediatek display subsystem consists of various DISP function blocks in the | ||
| 5 | MMSYS register space. The connections between them can be configured by output | ||
| 6 | and input selectors in the MMSYS_CONFIG register space. Pixel clock and start | ||
| 7 | of frame signal are distributed to the other function blocks by a DISP_MUTEX | ||
| 8 | function block. | ||
| 9 | |||
| 10 | All DISP device tree nodes must be siblings to the central MMSYS_CONFIG node. | ||
| 11 | For a description of the MMSYS_CONFIG binding, see | ||
| 12 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml. | ||
| 13 | |||
| 14 | DISP function blocks | ||
| 15 | ==================== | ||
| 16 | |||
| 17 | A display stream starts at a source function block that reads pixel data from | ||
| 18 | memory and ends with a sink function block that drives pixels on a display | ||
| 19 | interface, or writes pixels back to memory. All DISP function blocks have | ||
| 20 | their own register space, interrupt, and clock gate. The blocks that can | ||
| 21 | access memory additionally have to list the IOMMU and local arbiter they are | ||
| 22 | connected to. | ||
| 23 | |||
| 24 | For a description of the display interface sink function blocks, see | ||
| 25 | Documentation/devicetree/bindings/display/mediatek/mediatek,dsi.txt and | ||
| 26 | Documentation/devicetree/bindings/display/mediatek/mediatek,dpi.yaml. | ||
| 27 | |||
| 28 | Required properties (all function blocks): | ||
| 29 | - compatible: "mediatek,<chip>-disp-<function>", one of | ||
| 30 | "mediatek,<chip>-disp-ovl" - overlay (4 layers, blending, csc) | ||
| 31 | "mediatek,<chip>-disp-ovl-2l" - overlay (2 layers, blending, csc) | ||
| 32 | "mediatek,<chip>-disp-rdma" - read DMA / line buffer | ||
| 33 | "mediatek,<chip>-disp-wdma" - write DMA | ||
| 34 | "mediatek,<chip>-disp-ccorr" - color correction | ||
| 35 | "mediatek,<chip>-disp-color" - color processor | ||
| 36 | "mediatek,<chip>-disp-dither" - dither | ||
| 37 | "mediatek,<chip>-disp-aal" - adaptive ambient light controller | ||
| 38 | "mediatek,<chip>-disp-gamma" - gamma correction | ||
| 39 | "mediatek,<chip>-disp-merge" - merge streams from two RDMA sources | ||
| 40 | "mediatek,<chip>-disp-postmask" - control round corner for display frame | ||
| 41 | "mediatek,<chip>-disp-split" - split stream to two encoders | ||
| 42 | "mediatek,<chip>-disp-ufoe" - data compression engine | ||
| 43 | "mediatek,<chip>-dsi" - DSI controller, see mediatek,dsi.txt | ||
| 44 | "mediatek,<chip>-dpi" - DPI controller, see mediatek,dpi.txt | ||
| 45 | "mediatek,<chip>-disp-mutex" - display mutex | ||
| 46 | "mediatek,<chip>-disp-od" - overdrive | ||
| 47 | the supported chips are mt2701, mt7623, mt2712, mt8167, mt8173, mt8183 and mt8192. | ||
| 48 | - reg: Physical base address and length of the function block register space | ||
| 49 | - interrupts: The interrupt signal from the function block (required, except for | ||
| 50 | merge and split function blocks). | ||
| 51 | - clocks: device clocks | ||
| 52 | See Documentation/devicetree/bindings/clock/clock-bindings.txt for details. | ||
| 53 | For most function blocks this is just a single clock input. Only the DSI and | ||
| 54 | DPI controller nodes have multiple clock inputs. These are documented in | ||
| 55 | mediatek,dsi.txt and mediatek,dpi.txt, respectively. | ||
| 56 | An exception is that the mt8183 mutex is always free running with no clocks property. | ||
| 57 | |||
| 58 | Required properties (DMA function blocks): | ||
| 59 | - compatible: Should be one of | ||
| 60 | "mediatek,<chip>-disp-ovl" | ||
| 61 | "mediatek,<chip>-disp-rdma" | ||
| 62 | "mediatek,<chip>-disp-wdma" | ||
| 63 | the supported chips are mt2701, mt8167 and mt8173. | ||
| 64 | - larb: Should contain a phandle pointing to the local arbiter device as defined | ||
| 65 | in Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml | ||
| 66 | - iommus: Should point to the respective IOMMU block with master port as | ||
| 67 | argument, see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml | ||
| 68 | for details. | ||
| 69 | |||
| 70 | Optional properties (RDMA function blocks): | ||
| 71 | - mediatek,rdma-fifo-size: rdma fifo size may be different even in same SOC, add this | ||
| 72 | property to the corresponding rdma | ||
| 73 | the value is the Max value which defined in hardware data sheet. | ||
| 74 | mediatek,rdma-fifo-size of mt8173-rdma0 is 8K | ||
| 75 | mediatek,rdma-fifo-size of mt8183-rdma0 is 5K | ||
| 76 | mediatek,rdma-fifo-size of mt8183-rdma1 is 2K | ||
| 77 | |||
| 78 | Examples: | ||
| 79 | |||
| 80 | mmsys: clock-controller@14000000 { | ||
| 81 | compatible = "mediatek,mt8173-mmsys", "syscon"; | ||
| 82 | reg = <0 0x14000000 0 0x1000>; | ||
| 83 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 84 | #clock-cells = <1>; | ||
| 85 | }; | ||
| 86 | |||
| 87 | ovl0: ovl@1400c000 { | ||
| 88 | compatible = "mediatek,mt8173-disp-ovl"; | ||
| 89 | reg = <0 0x1400c000 0 0x1000>; | ||
| 90 | interrupts = <GIC_SPI 180 IRQ_TYPE_LEVEL_LOW>; | ||
| 91 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 92 | clocks = <&mmsys CLK_MM_DISP_OVL0>; | ||
| 93 | iommus = <&iommu M4U_PORT_DISP_OVL0>; | ||
| 94 | mediatek,larb = <&larb0>; | ||
| 95 | }; | ||
| 96 | |||
| 97 | ovl1: ovl@1400d000 { | ||
| 98 | compatible = "mediatek,mt8173-disp-ovl"; | ||
| 99 | reg = <0 0x1400d000 0 0x1000>; | ||
| 100 | interrupts = <GIC_SPI 181 IRQ_TYPE_LEVEL_LOW>; | ||
| 101 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 102 | clocks = <&mmsys CLK_MM_DISP_OVL1>; | ||
| 103 | iommus = <&iommu M4U_PORT_DISP_OVL1>; | ||
| 104 | mediatek,larb = <&larb4>; | ||
| 105 | }; | ||
| 106 | |||
| 107 | rdma0: rdma@1400e000 { | ||
| 108 | compatible = "mediatek,mt8173-disp-rdma"; | ||
| 109 | reg = <0 0x1400e000 0 0x1000>; | ||
| 110 | interrupts = <GIC_SPI 182 IRQ_TYPE_LEVEL_LOW>; | ||
| 111 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 112 | clocks = <&mmsys CLK_MM_DISP_RDMA0>; | ||
| 113 | iommus = <&iommu M4U_PORT_DISP_RDMA0>; | ||
| 114 | mediatek,larb = <&larb0>; | ||
| 115 | mediatek,rdma-fifosize = <8192>; | ||
| 116 | }; | ||
| 117 | |||
| 118 | rdma1: rdma@1400f000 { | ||
| 119 | compatible = "mediatek,mt8173-disp-rdma"; | ||
| 120 | reg = <0 0x1400f000 0 0x1000>; | ||
| 121 | interrupts = <GIC_SPI 183 IRQ_TYPE_LEVEL_LOW>; | ||
| 122 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 123 | clocks = <&mmsys CLK_MM_DISP_RDMA1>; | ||
| 124 | iommus = <&iommu M4U_PORT_DISP_RDMA1>; | ||
| 125 | mediatek,larb = <&larb4>; | ||
| 126 | }; | ||
| 127 | |||
| 128 | rdma2: rdma@14010000 { | ||
| 129 | compatible = "mediatek,mt8173-disp-rdma"; | ||
| 130 | reg = <0 0x14010000 0 0x1000>; | ||
| 131 | interrupts = <GIC_SPI 184 IRQ_TYPE_LEVEL_LOW>; | ||
| 132 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 133 | clocks = <&mmsys CLK_MM_DISP_RDMA2>; | ||
| 134 | iommus = <&iommu M4U_PORT_DISP_RDMA2>; | ||
| 135 | mediatek,larb = <&larb4>; | ||
| 136 | }; | ||
| 137 | |||
| 138 | wdma0: wdma@14011000 { | ||
| 139 | compatible = "mediatek,mt8173-disp-wdma"; | ||
| 140 | reg = <0 0x14011000 0 0x1000>; | ||
| 141 | interrupts = <GIC_SPI 185 IRQ_TYPE_LEVEL_LOW>; | ||
| 142 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 143 | clocks = <&mmsys CLK_MM_DISP_WDMA0>; | ||
| 144 | iommus = <&iommu M4U_PORT_DISP_WDMA0>; | ||
| 145 | mediatek,larb = <&larb0>; | ||
| 146 | }; | ||
| 147 | |||
| 148 | wdma1: wdma@14012000 { | ||
| 149 | compatible = "mediatek,mt8173-disp-wdma"; | ||
| 150 | reg = <0 0x14012000 0 0x1000>; | ||
| 151 | interrupts = <GIC_SPI 186 IRQ_TYPE_LEVEL_LOW>; | ||
| 152 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 153 | clocks = <&mmsys CLK_MM_DISP_WDMA1>; | ||
| 154 | iommus = <&iommu M4U_PORT_DISP_WDMA1>; | ||
| 155 | mediatek,larb = <&larb4>; | ||
| 156 | }; | ||
| 157 | |||
| 158 | color0: color@14013000 { | ||
| 159 | compatible = "mediatek,mt8173-disp-color"; | ||
| 160 | reg = <0 0x14013000 0 0x1000>; | ||
| 161 | interrupts = <GIC_SPI 187 IRQ_TYPE_LEVEL_LOW>; | ||
| 162 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 163 | clocks = <&mmsys CLK_MM_DISP_COLOR0>; | ||
| 164 | }; | ||
| 165 | |||
| 166 | color1: color@14014000 { | ||
| 167 | compatible = "mediatek,mt8173-disp-color"; | ||
| 168 | reg = <0 0x14014000 0 0x1000>; | ||
| 169 | interrupts = <GIC_SPI 188 IRQ_TYPE_LEVEL_LOW>; | ||
| 170 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 171 | clocks = <&mmsys CLK_MM_DISP_COLOR1>; | ||
| 172 | }; | ||
| 173 | |||
| 174 | aal@14015000 { | ||
| 175 | compatible = "mediatek,mt8173-disp-aal"; | ||
| 176 | reg = <0 0x14015000 0 0x1000>; | ||
| 177 | interrupts = <GIC_SPI 189 IRQ_TYPE_LEVEL_LOW>; | ||
| 178 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 179 | clocks = <&mmsys CLK_MM_DISP_AAL>; | ||
| 180 | }; | ||
| 181 | |||
| 182 | gamma@14016000 { | ||
| 183 | compatible = "mediatek,mt8173-disp-gamma"; | ||
| 184 | reg = <0 0x14016000 0 0x1000>; | ||
| 185 | interrupts = <GIC_SPI 190 IRQ_TYPE_LEVEL_LOW>; | ||
| 186 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 187 | clocks = <&mmsys CLK_MM_DISP_GAMMA>; | ||
| 188 | }; | ||
| 189 | |||
| 190 | ufoe@1401a000 { | ||
| 191 | compatible = "mediatek,mt8173-disp-ufoe"; | ||
| 192 | reg = <0 0x1401a000 0 0x1000>; | ||
| 193 | interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_LOW>; | ||
| 194 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 195 | clocks = <&mmsys CLK_MM_DISP_UFOE>; | ||
| 196 | }; | ||
| 197 | |||
| 198 | dsi0: dsi@1401b000 { | ||
| 199 | /* See mediatek,dsi.txt for details */ | ||
| 200 | }; | ||
| 201 | |||
| 202 | dpi0: dpi@1401d000 { | ||
| 203 | /* See mediatek,dpi.txt for details */ | ||
| 204 | }; | ||
| 205 | |||
| 206 | mutex: mutex@14020000 { | ||
| 207 | compatible = "mediatek,mt8173-disp-mutex"; | ||
| 208 | reg = <0 0x14020000 0 0x1000>; | ||
| 209 | interrupts = <GIC_SPI 169 IRQ_TYPE_LEVEL_LOW>; | ||
| 210 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 211 | clocks = <&mmsys CLK_MM_MUTEX_32K>; | ||
| 212 | }; | ||
| 213 | |||
| 214 | od@14023000 { | ||
| 215 | compatible = "mediatek,mt8173-disp-od"; | ||
| 216 | reg = <0 0x14023000 0 0x1000>; | ||
| 217 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 218 | clocks = <&mmsys CLK_MM_DISP_OD>; | ||
| 219 | }; | ||
| @@ -0,0 +1,76 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,dither.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display dither processor | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display dither processor, namely DITHER, works by approximating | ||
| 15 | unavailable colors with available colors and by mixing and matching available | ||
| 16 | colors to mimic unavailable ones. | ||
| 17 | DITHER device node must be siblings to the central MMSYS_CONFIG node. | ||
| 18 | For a description of the MMSYS_CONFIG binding, see | ||
| 19 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 20 | for details. | ||
| 21 | |||
| 22 | properties: | ||
| 23 | compatible: | ||
| 24 | oneOf: | ||
| 25 | - items: | ||
| 26 | - const: mediatek,mt8183-disp-dither | ||
| 27 | - items: | ||
| 28 | - enum: | ||
| 29 | - mediatek,mt8192-disp-dither | ||
| 30 | - mediatek,mt8195-disp-dither | ||
| 31 | - enum: | ||
| 32 | - mediatek,mt8183-disp-dither | ||
| 33 | |||
| 34 | reg: | ||
| 35 | maxItems: 1 | ||
| 36 | |||
| 37 | interrupts: | ||
| 38 | maxItems: 1 | ||
| 39 | |||
| 40 | power-domains: | ||
| 41 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 42 | the power controller specified by phandle. See | ||
| 43 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 44 | |||
| 45 | clocks: | ||
| 46 | items: | ||
| 47 | - description: DITHER Clock | ||
| 48 | |||
| 49 | mediatek,gce-client-reg: | ||
| 50 | description: The register of client driver can be configured by gce with | ||
| 51 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 52 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 53 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 54 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 55 | maxItems: 1 | ||
| 56 | |||
| 57 | required: | ||
| 58 | - compatible | ||
| 59 | - reg | ||
| 60 | - interrupts | ||
| 61 | - power-domains | ||
| 62 | - clocks | ||
| 63 | |||
| 64 | additionalProperties: false | ||
| 65 | |||
| 66 | examples: | ||
| 67 | - | | ||
| 68 | |||
| 69 | dither0: dither@14012000 { | ||
| 70 | compatible = "mediatek,mt8183-disp-dither"; | ||
| 71 | reg = <0 0x14012000 0 0x1000>; | ||
| 72 | interrupts = <GIC_SPI 235 IRQ_TYPE_LEVEL_LOW>; | ||
| 73 | power-domains = <&spm MT8183_POWER_DOMAIN_DISP>; | ||
| 74 | clocks = <&mmsys CLK_MM_DISP_DITHER0>; | ||
| 75 | mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x2000 0x1000>; | ||
| 76 | }; | ||
| @@ -0,0 +1,71 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,dsc.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: mediatek display DSC controller | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | The DSC standard is a specification of the algorithms used for | ||
| 15 | compressing and decompressing image display streams, including | ||
| 16 | the specification of the syntax and semantics of the compressed | ||
| 17 | video bit stream. DSC is designed for real-time systems with | ||
| 18 | real-time compression, transmission, decompression and Display. | ||
| 19 | |||
| 20 | properties: | ||
| 21 | compatible: | ||
| 22 | oneOf: | ||
| 23 | - items: | ||
| 24 | - const: mediatek,mt8195-disp-dsc | ||
| 25 | |||
| 26 | reg: | ||
| 27 | maxItems: 1 | ||
| 28 | |||
| 29 | interrupts: | ||
| 30 | maxItems: 1 | ||
| 31 | |||
| 32 | clocks: | ||
| 33 | items: | ||
| 34 | - description: DSC Wrapper Clock | ||
| 35 | |||
| 36 | power-domains: | ||
| 37 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 38 | the power controller specified by phandle. See | ||
| 39 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 40 | |||
| 41 | mediatek,gce-client-reg: | ||
| 42 | description: | ||
| 43 | The register of client driver can be configured by gce with 4 arguments | ||
| 44 | defined in this property, such as phandle of gce, subsys id, | ||
| 45 | register offset and size. | ||
| 46 | Each subsys id is mapping to a base address of display function blocks | ||
| 47 | register which is defined in the gce header | ||
| 48 | include/dt-bindings/gce/<chip>-gce.h. | ||
| 49 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 50 | maxItems: 1 | ||
| 51 | |||
| 52 | required: | ||
| 53 | - compatible | ||
| 54 | - reg | ||
| 55 | - interrupts | ||
| 56 | - power-domains | ||
| 57 | - clocks | ||
| 58 | |||
| 59 | additionalProperties: false | ||
| 60 | |||
| 61 | examples: | ||
| 62 | - | | ||
| 63 | |||
| 64 | dsc0: disp_dsc_wrap@1c009000 { | ||
| 65 | compatible = "mediatek,mt8195-disp-dsc"; | ||
| 66 | reg = <0 0x1c009000 0 0x1000>; | ||
| 67 | interrupts = <GIC_SPI 645 IRQ_TYPE_LEVEL_HIGH 0>; | ||
| 68 | power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS0>; | ||
| 69 | clocks = <&vdosys0 CLK_VDO0_DSC_WRAP0>; | ||
| 70 | mediatek,gce-client-reg = <&gce1 SUBSYS_1c00XXXX 0x9000 0x1000>; | ||
| 71 | }; | ||
| @@ -0,0 +1,147 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,ethdr.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek Ethdr Device Tree Bindings | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | ETHDR is designed for HDR video and graphics conversion in the external display path. | ||
| 15 | It handles multiple HDR input types and performs tone mapping, color space/color | ||
| 16 | format conversion, and then combine different layers, output the required HDR or | ||
| 17 | SDR signal to the subsequent display path. This engine is composed of two video | ||
| 18 | frontends, two graphic frontends, one video backend and a mixer. ETHDR has two | ||
| 19 | DMA function blocks, DS and ADL. These two function blocks read the pre-programmed | ||
| 20 | registers from DRAM and set them to HW in the v-blanking period. | ||
| 21 | |||
| 22 | properties: | ||
| 23 | compatible: | ||
| 24 | items: | ||
| 25 | - const: mediatek,mt8195-disp-ethdr | ||
| 26 | reg: | ||
| 27 | maxItems: 7 | ||
| 28 | reg-names: | ||
| 29 | items: | ||
| 30 | - const: mixer | ||
| 31 | - const: vdo_fe0 | ||
| 32 | - const: vdo_fe1 | ||
| 33 | - const: gfx_fe0 | ||
| 34 | - const: gfx_fe1 | ||
| 35 | - const: vdo_be | ||
| 36 | - const: adl_ds | ||
| 37 | interrupts: | ||
| 38 | minItems: 1 | ||
| 39 | iommus: | ||
| 40 | description: The compatible property is DMA function blocks. | ||
| 41 | Should point to the respective IOMMU block with master port as argument, | ||
| 42 | see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for | ||
| 43 | details. | ||
| 44 | minItems: 1 | ||
| 45 | maxItems: 2 | ||
| 46 | clocks: | ||
| 47 | items: | ||
| 48 | - description: mixer clock | ||
| 49 | - description: video frontend 0 clock | ||
| 50 | - description: video frontend 1 clock | ||
| 51 | - description: graphic frontend 0 clock | ||
| 52 | - description: graphic frontend 1 clock | ||
| 53 | - description: video backend clock | ||
| 54 | - description: autodownload and menuload clock | ||
| 55 | - description: video frontend 0 async clock | ||
| 56 | - description: video frontend 1 async clock | ||
| 57 | - description: graphic frontend 0 async clock | ||
| 58 | - description: graphic frontend 1 async clock | ||
| 59 | - description: video backend async clock | ||
| 60 | - description: ethdr top clock | ||
| 61 | clock-names: | ||
| 62 | items: | ||
| 63 | - const: mixer | ||
| 64 | - const: vdo_fe0 | ||
| 65 | - const: vdo_fe1 | ||
| 66 | - const: gfx_fe0 | ||
| 67 | - const: gfx_fe1 | ||
| 68 | - const: vdo_be | ||
| 69 | - const: adl_ds | ||
| 70 | - const: vdo_fe0_async | ||
| 71 | - const: vdo_fe1_async | ||
| 72 | - const: gfx_fe0_async | ||
| 73 | - const: gfx_fe1_async | ||
| 74 | - const: vdo_be_async | ||
| 75 | - const: ethdr_top | ||
| 76 | power-domains: | ||
| 77 | maxItems: 1 | ||
| 78 | resets: | ||
| 79 | maxItems: 5 | ||
| 80 | mediatek,gce-client-reg: | ||
| 81 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 82 | description: The register of display function block to be set by gce. | ||
| 83 | There are 4 arguments in this property, gce node, subsys id, offset and | ||
| 84 | register size. The subsys id is defined in the gce header of each chips | ||
| 85 | include/include/dt-bindings/gce/<chip>-gce.h, mapping to the register of | ||
| 86 | display function block. | ||
| 87 | |||
| 88 | required: | ||
| 89 | - compatible | ||
| 90 | - reg | ||
| 91 | - clocks | ||
| 92 | - clock-names | ||
| 93 | - interrupts | ||
| 94 | - power-domains | ||
| 95 | |||
| 96 | additionalProperties: false | ||
| 97 | |||
| 98 | examples: | ||
| 99 | - | | ||
| 100 | |||
| 101 | disp_ethdr@1c114000 { | ||
| 102 | compatible = "mediatek,mt8195-disp-ethdr"; | ||
| 103 | reg = <0 0x1c114000 0 0x1000>, | ||
| 104 | <0 0x1c115000 0 0x1000>, | ||
| 105 | <0 0x1c117000 0 0x1000>, | ||
| 106 | <0 0x1c119000 0 0x1000>, | ||
| 107 | <0 0x1c11A000 0 0x1000>, | ||
| 108 | <0 0x1c11B000 0 0x1000>, | ||
| 109 | <0 0x1c11C000 0 0x1000>; | ||
| 110 | reg-names = "mixer", "vdo_fe0", "vdo_fe1", "gfx_fe0", "gfx_fe1", | ||
| 111 | "vdo_be", "adl_ds"; | ||
| 112 | mediatek,gce-client-reg = <&gce0 SUBSYS_1c11XXXX 0x4000 0x1000>, | ||
| 113 | <&gce0 SUBSYS_1c11XXXX 0x5000 0x1000>, | ||
| 114 | <&gce0 SUBSYS_1c11XXXX 0x7000 0x1000>, | ||
| 115 | <&gce0 SUBSYS_1c11XXXX 0x9000 0x1000>, | ||
| 116 | <&gce0 SUBSYS_1c11XXXX 0xA000 0x1000>, | ||
| 117 | <&gce0 SUBSYS_1c11XXXX 0xB000 0x1000>, | ||
| 118 | <&gce0 SUBSYS_1c11XXXX 0xC000 0x1000>; | ||
| 119 | clocks = <&vdosys1 CLK_VDO1_DISP_MIXER>, | ||
| 120 | <&vdosys1 CLK_VDO1_HDR_VDO_FE0>, | ||
| 121 | <&vdosys1 CLK_VDO1_HDR_VDO_FE1>, | ||
| 122 | <&vdosys1 CLK_VDO1_HDR_GFX_FE0>, | ||
| 123 | <&vdosys1 CLK_VDO1_HDR_GFX_FE1>, | ||
| 124 | <&vdosys1 CLK_VDO1_HDR_VDO_BE>, | ||
| 125 | <&vdosys1 CLK_VDO1_26M_SLOW>, | ||
| 126 | <&vdosys1 CLK_VDO1_HDR_VDO_FE0_DL_ASYNC>, | ||
| 127 | <&vdosys1 CLK_VDO1_HDR_VDO_FE1_DL_ASYNC>, | ||
| 128 | <&vdosys1 CLK_VDO1_HDR_GFX_FE0_DL_ASYNC>, | ||
| 129 | <&vdosys1 CLK_VDO1_HDR_GFX_FE1_DL_ASYNC>, | ||
| 130 | <&vdosys1 CLK_VDO1_HDR_VDO_BE_DL_ASYNC>, | ||
| 131 | <&topckgen CLK_TOP_ETHDR_SEL>; | ||
| 132 | clock-names = "mixer", "vdo_fe0", "vdo_fe1", "gfx_fe0", "gfx_fe1", | ||
| 133 | "vdo_be", "adl_ds", "vdo_fe0_async", "vdo_fe1_async", | ||
| 134 | "gfx_fe0_async", "gfx_fe1_async","vdo_be_async", | ||
| 135 | "ethdr_top"; | ||
| 136 | power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>; | ||
| 137 | iommus = <&iommu_vpp M4U_PORT_L3_HDR_DS>, | ||
| 138 | <&iommu_vpp M4U_PORT_L3_HDR_ADL>; | ||
| 139 | interrupts = <GIC_SPI 517 IRQ_TYPE_LEVEL_HIGH 0>; /* disp mixer */ | ||
| 140 | resets = <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_VDO_FE0_DL_ASYNC>, | ||
| 141 | <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_VDO_FE1_DL_ASYNC>, | ||
| 142 | <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_GFX_FE0_DL_ASYNC>, | ||
| 143 | <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_GFX_FE1_DL_ASYNC>, | ||
| 144 | <&vdosys1 MT8195_VDOSYS1_SW1_RST_B_HDR_VDO_BE_DL_ASYNC>; | ||
| 145 | }; | ||
| 146 | |||
| 147 | ... | ||
| @@ -0,0 +1,77 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,gamma.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display gamma correction | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display gamma correction, namely GAMMA, provides a nonlinear | ||
| 15 | operation used to adjust luminance in display system. | ||
| 16 | GAMMA device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt8173-disp-gamma | ||
| 26 | - items: | ||
| 27 | - const: mediatek,mt8183-disp-gamma | ||
| 28 | - items: | ||
| 29 | - enum: | ||
| 30 | - mediatek,mt8192-disp-gamma | ||
| 31 | - mediatek,mt8195-disp-gamma | ||
| 32 | - enum: | ||
| 33 | - mediatek,mt8183-disp-gamma | ||
| 34 | |||
| 35 | reg: | ||
| 36 | maxItems: 1 | ||
| 37 | |||
| 38 | interrupts: | ||
| 39 | maxItems: 1 | ||
| 40 | |||
| 41 | power-domains: | ||
| 42 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 43 | the power controller specified by phandle. See | ||
| 44 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 45 | |||
| 46 | clocks: | ||
| 47 | items: | ||
| 48 | - description: GAMMA Clock | ||
| 49 | |||
| 50 | mediatek,gce-client-reg: | ||
| 51 | description: The register of client driver can be configured by gce with | ||
| 52 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 53 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 54 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 55 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 56 | maxItems: 1 | ||
| 57 | |||
| 58 | required: | ||
| 59 | - compatible | ||
| 60 | - reg | ||
| 61 | - interrupts | ||
| 62 | - power-domains | ||
| 63 | - clocks | ||
| 64 | |||
| 65 | additionalProperties: false | ||
| 66 | |||
| 67 | examples: | ||
| 68 | - | | ||
| 69 | |||
| 70 | gamma@14016000 { | ||
| 71 | compatible = "mediatek,mt8173-disp-gamma"; | ||
| 72 | reg = <0 0x14016000 0 0x1000>; | ||
| 73 | interrupts = <GIC_SPI 190 IRQ_TYPE_LEVEL_LOW>; | ||
| 74 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 75 | clocks = <&mmsys CLK_MM_DISP_GAMMA>; | ||
| 76 | mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x6000 0x1000>; | ||
| 77 | }; | ||
| @@ -0,0 +1,110 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,merge.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display merge | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display merge, namely MERGE, is used to merge two slice-per-line | ||
| 15 | inputs into one side-by-side output. | ||
| 16 | MERGE device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt8173-disp-merge | ||
| 26 | - items: | ||
| 27 | - const: mediatek,mt8195-disp-merge | ||
| 28 | |||
| 29 | reg: | ||
| 30 | maxItems: 1 | ||
| 31 | |||
| 32 | interrupts: | ||
| 33 | maxItems: 1 | ||
| 34 | |||
| 35 | power-domains: | ||
| 36 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 37 | the power controller specified by phandle. See | ||
| 38 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 39 | |||
| 40 | clocks: | ||
| 41 | maxItems: 2 | ||
| 42 | items: | ||
| 43 | - description: MERGE Clock | ||
| 44 | - description: MERGE Async Clock | ||
| 45 | Controlling the synchronous process between MERGE and other display | ||
| 46 | function blocks cross clock domain. | ||
| 47 | |||
| 48 | clock-names: | ||
| 49 | maxItems: 2 | ||
| 50 | items: | ||
| 51 | - const: merge | ||
| 52 | - const: merge_async | ||
| 53 | |||
| 54 | mediatek,merge-fifo-en: | ||
| 55 | description: | ||
| 56 | The setting of merge fifo is mainly provided for the display latency | ||
| 57 | buffer to ensure that the back-end panel display data will not be | ||
| 58 | underrun, a little more data is needed in the fifo. | ||
| 59 | According to the merge fifo settings, when the water level is detected | ||
| 60 | to be insufficient, it will trigger RDMA sending ultra and preulra | ||
| 61 | command to SMI to speed up the data rate. | ||
| 62 | type: boolean | ||
| 63 | |||
| 64 | mediatek,merge-mute: | ||
| 65 | description: Support mute function. Mute the content of merge output. | ||
| 66 | type: boolean | ||
| 67 | |||
| 68 | mediatek,gce-client-reg: | ||
| 69 | description: The register of client driver can be configured by gce with | ||
| 70 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 71 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 72 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 73 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 74 | maxItems: 1 | ||
| 75 | |||
| 76 | resets: | ||
| 77 | description: reset controller | ||
| 78 | See Documentation/devicetree/bindings/reset/reset.txt for details. | ||
| 79 | maxItems: 1 | ||
| 80 | |||
| 81 | required: | ||
| 82 | - compatible | ||
| 83 | - reg | ||
| 84 | - power-domains | ||
| 85 | - clocks | ||
| 86 | |||
| 87 | additionalProperties: false | ||
| 88 | |||
| 89 | examples: | ||
| 90 | - | | ||
| 91 | |||
| 92 | merge@14017000 { | ||
| 93 | compatible = "mediatek,mt8173-disp-merge"; | ||
| 94 | reg = <0 0x14017000 0 0x1000>; | ||
| 95 | power-domains = <&spm MT8173_POWER_DOMAIN_MM>; | ||
| 96 | clocks = <&mmsys CLK_MM_DISP_MERGE>; | ||
| 97 | }; | ||
| 98 | |||
| 99 | merge5: disp_vpp_merge5@1c110000 { | ||
| 100 | compatible = "mediatek,mt8195-disp-merge"; | ||
| 101 | reg = <0 0x1c110000 0 0x1000>; | ||
| 102 | interrupts = <GIC_SPI 507 IRQ_TYPE_LEVEL_HIGH 0>; | ||
| 103 | clocks = <&vdosys1 CLK_VDO1_VPP_MERGE4>, | ||
| 104 | <&vdosys1 CLK_VDO1_MERGE4_DL_ASYNC>; | ||
| 105 | clock-names = "merge","merge_async"; | ||
| 106 | power-domains = <&spm MT8195_POWER_DOMAIN_VDOSYS1>; | ||
| 107 | mediatek,gce-client-reg = <&gce1 SUBSYS_1c11XXXX 0x0000 0x1000>; | ||
| 108 | mediatek,merge-fifo-en = <1>; | ||
| 109 | resets = <&vdosys1 MT8195_VDOSYS1_SW0_RST_B_MERGE4_DL_ASYNC>; | ||
| 110 | }; | ||
| @@ -0,0 +1,83 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,mutex.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek mutex | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek mutex, namely MUTEX, is used to send the triggers signals called | ||
| 15 | Start Of Frame (SOF) / End Of Frame (EOF) to each sub-modules on the display | ||
| 16 | data path or MDP data path. | ||
| 17 | In some SoC, such as mt2701, MUTEX could be a hardware mutex which protects | ||
| 18 | the shadow register. | ||
| 19 | MUTEX device node must be siblings to the central MMSYS_CONFIG node. | ||
| 20 | For a description of the MMSYS_CONFIG binding, see | ||
| 21 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 22 | for details. | ||
| 23 | |||
| 24 | properties: | ||
| 25 | compatible: | ||
| 26 | oneOf: | ||
| 27 | - items: | ||
| 28 | - const: mediatek,mt2701-disp-mutex | ||
| 29 | - items: | ||
| 30 | - const: mediatek,mt2712-disp-mutex | ||
| 31 | - items: | ||
| 32 | - const: mediatek,mt8167-disp-mutex | ||
| 33 | - items: | ||
| 34 | - const: mediatek,mt8173-disp-mutex | ||
| 35 | - items: | ||
| 36 | - const: mediatek,mt8183-disp-mutex | ||
| 37 | - items: | ||
| 38 | - const: mediatek,mt8192-disp-mutex | ||
| 39 | - items: | ||
| 40 | - const: mediatek,mt8195-disp-mutex | ||
| 41 | reg: | ||
| 42 | maxItems: 1 | ||
| 43 | |||
| 44 | interrupts: | ||
| 45 | maxItems: 1 | ||
| 46 | |||
| 47 | power-domains: | ||
| 48 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 49 | the power controller specified by phandle. See | ||
| 50 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 51 | |||
| 52 | clocks: | ||
| 53 | items: | ||
| 54 | - description: MUTEX Clock | ||
| 55 | |||
| 56 | mediatek,gce-events: | ||
| 57 | description: | ||
| 58 | The event id which is mapping to the specific hardware event signal | ||
| 59 | to gce. The event id is defined in the gce header | ||
| 60 | include/dt-bindings/gce/<chip>-gce.h of each chips. | ||
| 61 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 62 | |||
| 63 | required: | ||
| 64 | - compatible | ||
| 65 | - reg | ||
| 66 | - interrupts | ||
| 67 | - power-domains | ||
| 68 | - clocks | ||
| 69 | |||
| 70 | additionalProperties: false | ||
| 71 | |||
| 72 | examples: | ||
| 73 | - | | ||
| 74 | |||
| 75 | mutex: mutex@14020000 { | ||
| 76 | compatible = "mediatek,mt8173-disp-mutex"; | ||
| 77 | reg = <0 0x14020000 0 0x1000>; | ||
| 78 | interrupts = <GIC_SPI 169 IRQ_TYPE_LEVEL_LOW>; | ||
| 79 | power-domains = <&spm MT8173_POWER_DOMAIN_MM>; | ||
| 80 | clocks = <&mmsys CLK_MM_MUTEX_32K>; | ||
| 81 | mediatek,gce-events = <CMDQ_EVENT_MUTEX0_STREAM_EOF>, | ||
| 82 | <CMDQ_EVENT_MUTEX1_STREAM_EOF>; | ||
| 83 | }; | ||
| @@ -0,0 +1,53 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,od.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display overdirve | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display overdrive, namely OD, increases the transition values | ||
| 15 | of pixels between consecutive frames to make LCD rotate faster. | ||
| 16 | OD device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt2712-disp-od | ||
| 26 | - items: | ||
| 27 | - const: mediatek,mt8173-disp-od | ||
| 28 | |||
| 29 | reg: | ||
| 30 | maxItems: 1 | ||
| 31 | |||
| 32 | interrupts: | ||
| 33 | maxItems: 1 | ||
| 34 | |||
| 35 | clocks: | ||
| 36 | items: | ||
| 37 | - description: OD Clock | ||
| 38 | |||
| 39 | required: | ||
| 40 | - compatible | ||
| 41 | - reg | ||
| 42 | - clocks | ||
| 43 | |||
| 44 | additionalProperties: false | ||
| 45 | |||
| 46 | examples: | ||
| 47 | - | | ||
| 48 | |||
| 49 | od@14023000 { | ||
| 50 | compatible = "mediatek,mt8173-disp-od"; | ||
| 51 | reg = <0 0x14023000 0 0x1000>; | ||
| 52 | clocks = <&mmsys CLK_MM_DISP_OD>; | ||
| 53 | }; | ||
| @@ -0,0 +1,88 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,ovl-2l.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display overlay 2 layer | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display overlay 2 layer, namely OVL-2L, provides 2 more layer | ||
| 15 | for OVL. | ||
| 16 | OVL-2L device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt8183-disp-ovl-2l | ||
| 26 | - items: | ||
| 27 | - const: mediatek,mt8192-disp-ovl-2l | ||
| 28 | |||
| 29 | reg: | ||
| 30 | maxItems: 1 | ||
| 31 | |||
| 32 | interrupts: | ||
| 33 | maxItems: 1 | ||
| 34 | |||
| 35 | power-domains: | ||
| 36 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 37 | the power controller specified by phandle. See | ||
| 38 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 39 | |||
| 40 | clocks: | ||
| 41 | items: | ||
| 42 | - description: OVL-2L Clock | ||
| 43 | |||
| 44 | iommus: | ||
| 45 | description: | ||
| 46 | This property should point to the respective IOMMU block with master port as argument, | ||
| 47 | see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for details. | ||
| 48 | |||
| 49 | mediatek,larb: | ||
| 50 | description: | ||
| 51 | This property should contain a phandle pointing to the local arbiter devices defined in | ||
| 52 | Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml. | ||
| 53 | It must sort according to the local arbiter index, like larb0, larb1, larb2... | ||
| 54 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 55 | minItems: 1 | ||
| 56 | maxItems: 32 | ||
| 57 | |||
| 58 | mediatek,gce-client-reg: | ||
| 59 | description: The register of client driver can be configured by gce with | ||
| 60 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 61 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 62 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 63 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 64 | maxItems: 1 | ||
| 65 | |||
| 66 | required: | ||
| 67 | - compatible | ||
| 68 | - reg | ||
| 69 | - interrupts | ||
| 70 | - power-domains | ||
| 71 | - clocks | ||
| 72 | - iommus | ||
| 73 | |||
| 74 | additionalProperties: false | ||
| 75 | |||
| 76 | examples: | ||
| 77 | - | | ||
| 78 | |||
| 79 | ovl_2l0: ovl@14009000 { | ||
| 80 | compatible = "mediatek,mt8183-disp-ovl-2l"; | ||
| 81 | reg = <0 0x14009000 0 0x1000>; | ||
| 82 | interrupts = <GIC_SPI 226 IRQ_TYPE_LEVEL_LOW>; | ||
| 83 | power-domains = <&spm MT8183_POWER_DOMAIN_DISP>; | ||
| 84 | clocks = <&mmsys CLK_MM_DISP_OVL0_2L>; | ||
| 85 | iommus = <&iommu M4U_PORT_DISP_2L_OVL0_LARB0>; | ||
| 86 | mediatek,larb = <&larb0>; | ||
| 87 | mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0x9000 0x1000>; | ||
| 88 | }; | ||
| @@ -0,0 +1,103 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,ovl.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display overlay | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display overlay, namely OVL, can do alpha blending from | ||
| 15 | the memory. | ||
| 16 | OVL device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt2701-disp-ovl | ||
| 26 | - items: | ||
| 27 | - const: mediatek,mt8173-disp-ovl | ||
| 28 | - items: | ||
| 29 | - const: mediatek,mt8183-disp-ovl | ||
| 30 | - items: | ||
| 31 | - const: mediatek,mt8192-disp-ovl | ||
| 32 | - items: | ||
| 33 | - enum: | ||
| 34 | - mediatek,mt7623-disp-ovl | ||
| 35 | - mediatek,mt2712-disp-ovl | ||
| 36 | - enum: | ||
| 37 | - mediatek,mt2701-disp-ovl | ||
| 38 | - items: | ||
| 39 | - enum: | ||
| 40 | - mediatek,mt8195-disp-ovl | ||
| 41 | - enum: | ||
| 42 | - mediatek,mt8183-disp-ovl | ||
| 43 | |||
| 44 | reg: | ||
| 45 | maxItems: 1 | ||
| 46 | |||
| 47 | interrupts: | ||
| 48 | maxItems: 1 | ||
| 49 | |||
| 50 | power-domains: | ||
| 51 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 52 | the power controller specified by phandle. See | ||
| 53 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 54 | |||
| 55 | clocks: | ||
| 56 | items: | ||
| 57 | - description: OVL Clock | ||
| 58 | |||
| 59 | iommus: | ||
| 60 | description: | ||
| 61 | This property should point to the respective IOMMU block with master port as argument, | ||
| 62 | see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for details. | ||
| 63 | |||
| 64 | mediatek,larb: | ||
| 65 | description: | ||
| 66 | This property should contain a phandle pointing to the local arbiter devices defined in | ||
| 67 | Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml. | ||
| 68 | It must sort according to the local arbiter index, like larb0, larb1, larb2... | ||
| 69 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 70 | minItems: 1 | ||
| 71 | maxItems: 32 | ||
| 72 | |||
| 73 | mediatek,gce-client-reg: | ||
| 74 | description: The register of client driver can be configured by gce with | ||
| 75 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 76 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 77 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 78 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 79 | maxItems: 1 | ||
| 80 | |||
| 81 | required: | ||
| 82 | - compatible | ||
| 83 | - reg | ||
| 84 | - interrupts | ||
| 85 | - power-domains | ||
| 86 | - clocks | ||
| 87 | - iommu | ||
| 88 | |||
| 89 | additionalProperties: false | ||
| 90 | |||
| 91 | examples: | ||
| 92 | - | | ||
| 93 | |||
| 94 | ovl0: ovl@1400c000 { | ||
| 95 | compatible = "mediatek,mt8173-disp-ovl"; | ||
| 96 | reg = <0 0x1400c000 0 0x1000>; | ||
| 97 | interrupts = <GIC_SPI 180 IRQ_TYPE_LEVEL_LOW>; | ||
| 98 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 99 | clocks = <&mmsys CLK_MM_DISP_OVL0>; | ||
| 100 | iommus = <&iommu M4U_PORT_DISP_OVL0>; | ||
| 101 | mediatek,larb = <&larb0>; | ||
| 102 | mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0xc000 0x1000>; | ||
| 103 | }; | ||
| @@ -0,0 +1,69 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,postmask.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display postmask | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display postmask, namely POSTMASK, provides round corner pattern | ||
| 15 | generation. | ||
| 16 | POSTMASK device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt8192-disp-postmask | ||
| 26 | |||
| 27 | reg: | ||
| 28 | maxItems: 1 | ||
| 29 | |||
| 30 | interrupts: | ||
| 31 | maxItems: 1 | ||
| 32 | |||
| 33 | power-domains: | ||
| 34 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 35 | the power controller specified by phandle. See | ||
| 36 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 37 | |||
| 38 | clocks: | ||
| 39 | items: | ||
| 40 | - description: POSTMASK Clock | ||
| 41 | |||
| 42 | mediatek,gce-client-reg: | ||
| 43 | description: The register of client driver can be configured by gce with | ||
| 44 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 45 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 46 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 47 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 48 | maxItems: 1 | ||
| 49 | |||
| 50 | required: | ||
| 51 | - compatible | ||
| 52 | - reg | ||
| 53 | - interrupts | ||
| 54 | - power-domains | ||
| 55 | - clocks | ||
| 56 | |||
| 57 | additionalProperties: false | ||
| 58 | |||
| 59 | examples: | ||
| 60 | - | | ||
| 61 | |||
| 62 | postmask0: postmask@1400d000 { | ||
| 63 | compatible = "mediatek,mt8192-disp-postmask"; | ||
| 64 | reg = <0 0x1400d000 0 0x1000>; | ||
| 65 | interrupts = <GIC_SPI 262 IRQ_TYPE_LEVEL_HIGH 0>; | ||
| 66 | power-domains = <&scpsys MT8192_POWER_DOMAIN_DISP>; | ||
| 67 | clocks = <&mmsys CLK_MM_DISP_POSTMASK0>; | ||
| 68 | mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0xd000 0x1000>; | ||
| 69 | }; | ||
| @@ -0,0 +1,117 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,rdma.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek Read Direct Memory Access | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek Read Direct Memory Access(RDMA) component used to read the | ||
| 15 | data into DMA. It provides real time data to the back-end panel | ||
| 16 | driver, such as DSI, DPI and DP_INTF. | ||
| 17 | It contains one line buffer to store the sufficient pixel data. | ||
| 18 | RDMA device node must be siblings to the central MMSYS_CONFIG node. | ||
| 19 | For a description of the MMSYS_CONFIG binding, see | ||
| 20 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 21 | for details. | ||
| 22 | |||
| 23 | properties: | ||
| 24 | compatible: | ||
| 25 | oneOf: | ||
| 26 | - items: | ||
| 27 | - const: mediatek,mt2701-disp-rdma | ||
| 28 | - items: | ||
| 29 | - const: mediatek,mt8173-disp-rdma | ||
| 30 | - items: | ||
| 31 | - const: mediatek,mt8183-disp-rdma | ||
| 32 | - items: | ||
| 33 | - const: mediatek,mt8195-disp-rdma | ||
| 34 | - items: | ||
| 35 | - enum: | ||
| 36 | - mediatek,mt7623-disp-rdma | ||
| 37 | - mediatek,mt2712-disp-rdma | ||
| 38 | - enum: | ||
| 39 | - mediatek,mt2701-disp-rdma | ||
| 40 | - items: | ||
| 41 | - enum: | ||
| 42 | - mediatek,mt8192-disp-rdma | ||
| 43 | - enum: | ||
| 44 | - mediatek,mt8183-disp-rdma | ||
| 45 | |||
| 46 | reg: | ||
| 47 | maxItems: 1 | ||
| 48 | |||
| 49 | interrupts: | ||
| 50 | maxItems: 1 | ||
| 51 | |||
| 52 | power-domains: | ||
| 53 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 54 | the power controller specified by phandle. See | ||
| 55 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 56 | |||
| 57 | clocks: | ||
| 58 | items: | ||
| 59 | - description: RDMA Clock | ||
| 60 | |||
| 61 | iommus: | ||
| 62 | description: | ||
| 63 | This property should point to the respective IOMMU block with master port as argument, | ||
| 64 | see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for details. | ||
| 65 | |||
| 66 | mediatek,larb: | ||
| 67 | description: | ||
| 68 | This property should contain a phandle pointing to the local arbiter devices defined in | ||
| 69 | Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml. | ||
| 70 | It must sort according to the local arbiter index, like larb0, larb1, larb2... | ||
| 71 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 72 | minItems: 1 | ||
| 73 | maxItems: 32 | ||
| 74 | |||
| 75 | mediatek,rdma-fifo-size: | ||
| 76 | description: | ||
| 77 | rdma fifo size may be different even in same SOC, add this property to the | ||
| 78 | corresponding rdma. | ||
| 79 | The value below is the Max value which defined in hardware data sheet | ||
| 80 | mediatek,rdma-fifo-size of mt8173-rdma0 is 8K | ||
| 81 | mediatek,rdma-fifo-size of mt8183-rdma0 is 5K | ||
| 82 | mediatek,rdma-fifo-size of mt8183-rdma1 is 2K | ||
| 83 | $ref: /schemas/types.yaml#/definitions/uint32 | ||
| 84 | enum: [8192, 5120, 2048] | ||
| 85 | |||
| 86 | mediatek,gce-client-reg: | ||
| 87 | description: The register of client driver can be configured by gce with | ||
| 88 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 89 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 90 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 91 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 92 | maxItems: 1 | ||
| 93 | |||
| 94 | required: | ||
| 95 | - compatible | ||
| 96 | - reg | ||
| 97 | - interrupts | ||
| 98 | - power-domains | ||
| 99 | - clocks | ||
| 100 | - iommus | ||
| 101 | |||
| 102 | additionalProperties: false | ||
| 103 | |||
| 104 | examples: | ||
| 105 | - | | ||
| 106 | |||
| 107 | rdma0: rdma@1400e000 { | ||
| 108 | compatible = "mediatek,mt8173-disp-rdma"; | ||
| 109 | reg = <0 0x1400e000 0 0x1000>; | ||
| 110 | interrupts = <GIC_SPI 182 IRQ_TYPE_LEVEL_LOW>; | ||
| 111 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 112 | clocks = <&mmsys CLK_MM_DISP_RDMA0>; | ||
| 113 | iommus = <&iommu M4U_PORT_DISP_RDMA0>; | ||
| 114 | mediatek,larb = <&larb0>; | ||
| 115 | mediatek,rdma-fifosize = <8192>; | ||
| 116 | mediatek,gce-client-reg = <&gce SUBSYS_1400XXXX 0xe000 0x1000>; | ||
| 117 | }; | ||
| @@ -0,0 +1,58 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,split.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display split | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display split, namely SPLIT, is used to split stream to two | ||
| 15 | encoders. | ||
| 16 | SPLIT device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt8173-disp-split | ||
| 26 | |||
| 27 | reg: | ||
| 28 | maxItems: 1 | ||
| 29 | |||
| 30 | interrupts: | ||
| 31 | maxItems: 1 | ||
| 32 | |||
| 33 | power-domains: | ||
| 34 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 35 | the power controller specified by phandle. See | ||
| 36 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 37 | |||
| 38 | clocks: | ||
| 39 | items: | ||
| 40 | - description: SPLIT Clock | ||
| 41 | |||
| 42 | required: | ||
| 43 | - compatible | ||
| 44 | - reg | ||
| 45 | - power-domains | ||
| 46 | - clocks | ||
| 47 | |||
| 48 | additionalProperties: false | ||
| 49 | |||
| 50 | examples: | ||
| 51 | - | | ||
| 52 | |||
| 53 | split0: split@14018000 { | ||
| 54 | compatible = "mediatek,mt8173-disp-split"; | ||
| 55 | reg = <0 0x14018000 0 0x1000>; | ||
| 56 | power-domains = <&spm MT8173_POWER_DOMAIN_MM>; | ||
| 57 | clocks = <&mmsys CLK_MM_DISP_SPLIT0>; | ||
| 58 | }; | ||
| @@ -0,0 +1,61 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,ufoe.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek display UFOe | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek display UFOe stands for Unified Frame Optimization engine. | ||
| 15 | UFOe can cut the data rate for DSI port which may lead to reduce power | ||
| 16 | consumption. | ||
| 17 | UFOe device node must be siblings to the central MMSYS_CONFIG node. | ||
| 18 | For a description of the MMSYS_CONFIG binding, see | ||
| 19 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 20 | for details. | ||
| 21 | |||
| 22 | properties: | ||
| 23 | compatible: | ||
| 24 | oneOf: | ||
| 25 | - items: | ||
| 26 | - const: mediatek,mt8173-disp-ufoe | ||
| 27 | |||
| 28 | reg: | ||
| 29 | maxItems: 1 | ||
| 30 | |||
| 31 | interrupts: | ||
| 32 | maxItems: 1 | ||
| 33 | |||
| 34 | power-domains: | ||
| 35 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 36 | the power controller specified by phandle. See | ||
| 37 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 38 | |||
| 39 | clocks: | ||
| 40 | items: | ||
| 41 | - description: UFOe Clock | ||
| 42 | |||
| 43 | required: | ||
| 44 | - compatible | ||
| 45 | - reg | ||
| 46 | - interrupts | ||
| 47 | - power-domains | ||
| 48 | - clocks | ||
| 49 | |||
| 50 | additionalProperties: false | ||
| 51 | |||
| 52 | examples: | ||
| 53 | - | | ||
| 54 | |||
| 55 | ufoe@1401a000 { | ||
| 56 | compatible = "mediatek,mt8173-disp-ufoe"; | ||
| 57 | reg = <0 0x1401a000 0 0x1000>; | ||
| 58 | interrupts = <GIC_SPI 191 IRQ_TYPE_LEVEL_LOW>; | ||
| 59 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 60 | clocks = <&mmsys CLK_MM_DISP_UFOE>; | ||
| 61 | }; | ||
| @@ -0,0 +1,86 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/mediatek/mediatek,wdma.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Mediatek Write Direct Memory Access | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Chun-Kuang Hu <chunkuang.hu@kernel.org> | ||
| 11 | - Philipp Zabel <p.zabel@pengutronix.de> | ||
| 12 | |||
| 13 | description: | | ||
| 14 | Mediatek Write Direct Memory Access(WDMA) component used to write | ||
| 15 | the data into DMA. | ||
| 16 | WDMA device node must be siblings to the central MMSYS_CONFIG node. | ||
| 17 | For a description of the MMSYS_CONFIG binding, see | ||
| 18 | Documentation/devicetree/bindings/arm/mediatek/mediatek,mmsys.yaml | ||
| 19 | for details. | ||
| 20 | |||
| 21 | properties: | ||
| 22 | compatible: | ||
| 23 | oneOf: | ||
| 24 | - items: | ||
| 25 | - const: mediatek,mt8173-disp-wdma | ||
| 26 | |||
| 27 | reg: | ||
| 28 | maxItems: 1 | ||
| 29 | |||
| 30 | interrupts: | ||
| 31 | maxItems: 1 | ||
| 32 | |||
| 33 | power-domains: | ||
| 34 | description: A phandle and PM domain specifier as defined by bindings of | ||
| 35 | the power controller specified by phandle. See | ||
| 36 | Documentation/devicetree/bindings/power/power-domain.yaml for details. | ||
| 37 | |||
| 38 | clocks: | ||
| 39 | items: | ||
| 40 | - description: WDMA Clock | ||
| 41 | |||
| 42 | iommus: | ||
| 43 | description: | ||
| 44 | This property should point to the respective IOMMU block with master port as argument, | ||
| 45 | see Documentation/devicetree/bindings/iommu/mediatek,iommu.yaml for details. | ||
| 46 | |||
| 47 | mediatek,larb: | ||
| 48 | description: | ||
| 49 | This property should contain a phandle pointing to the local arbiter devices defined in | ||
| 50 | Documentation/devicetree/bindings/memory-controllers/mediatek,smi-larb.yaml. | ||
| 51 | It must sort according to the local arbiter index, like larb0, larb1, larb2... | ||
| 52 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 53 | minItems: 1 | ||
| 54 | maxItems: 32 | ||
| 55 | |||
| 56 | mediatek,gce-client-reg: | ||
| 57 | description: The register of client driver can be configured by gce with | ||
| 58 | 4 arguments defined in this property, such as phandle of gce, subsys id, | ||
| 59 | register offset and size. Each GCE subsys id is mapping to a client | ||
| 60 | defined in the header include/dt-bindings/gce/<chip>-gce.h. | ||
| 61 | $ref: /schemas/types.yaml#/definitions/phandle-array | ||
| 62 | maxItems: 1 | ||
| 63 | |||
| 64 | required: | ||
| 65 | - compatible | ||
| 66 | - reg | ||
| 67 | - interrupts | ||
| 68 | - power-domains | ||
| 69 | - clocks | ||
| 70 | - iommus | ||
| 71 | |||
| 72 | additionalProperties: false | ||
| 73 | |||
| 74 | examples: | ||
| 75 | - | | ||
| 76 | |||
| 77 | wdma0: wdma@14011000 { | ||
| 78 | compatible = "mediatek,mt8173-disp-wdma"; | ||
| 79 | reg = <0 0x14011000 0 0x1000>; | ||
| 80 | interrupts = <GIC_SPI 185 IRQ_TYPE_LEVEL_LOW>; | ||
| 81 | power-domains = <&scpsys MT8173_POWER_DOMAIN_MM>; | ||
| 82 | clocks = <&mmsys CLK_MM_DISP_WDMA0>; | ||
| 83 | iommus = <&iommu M4U_PORT_DISP_WDMA0>; | ||
| 84 | mediatek,larb = <&larb0>; | ||
| 85 | mediatek,gce-client-reg = <&gce SUBSYS_1401XXXX 0x1000 0x1000>; | ||
| 86 | }; | ||
| @@ -11,13 +11,23 @@ maintainers: · 10 unmodified lines | |||
| 11 | - Thierry Reding <thierry.reding@gmail.com> | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | ||
| 13 | allOf: | 13 | allOf: |
| 14 | - $ref: lvds.yaml# | 14 | - $ref: panel-common.yaml# |
| 15 | - $ref: /schemas/display/lvds.yaml/# | ||
| 16 | |||
| 17 | select: | ||
| 18 | properties: | ||
| 19 | compatible: | ||
| 20 | contains: | ||
| 21 | const: advantech,idk-1110wr | ||
| 22 | |||
| 23 | required: | ||
| 24 | - compatible | ||
| 15 | 25 | ||
| 16 | properties: | 26 | properties: |
| 17 | compatible: | 27 | compatible: |
| 18 | items: | 28 | items: |
| 19 | - const: advantech,idk-1110wr | 29 | - const: advantech,idk-1110wr |
| 20 | - {} # panel-lvds, but not listed here to avoid false select | 30 | - const: panel-lvds |
| 21 | 31 | ||
| 22 | data-mapping: | 32 | data-mapping: |
| 23 | const: jeida-24 | 33 | const: jeida-24 |
| @@ -35,6 +45,11 @@ additionalProperties: false · 11 unmodified lines | |||
| 35 | 45 | ||
| 36 | required: | 46 | required: |
| 37 | - compatible | 47 | - compatible |
| 48 | - data-mapping | ||
| 49 | - width-mm | ||
| 50 | - height-mm | ||
| 51 | - panel-timing | ||
| 52 | - port | ||
| 38 | 53 | ||
| 39 | examples: | 54 | examples: |
| 40 | - |+ | 55 | - |+ |
| @@ -11,15 +11,26 @@ maintainers: · 10 unmodified lines | |||
| 11 | - Thierry Reding <thierry.reding@gmail.com> | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | ||
| 13 | allOf: | 13 | allOf: |
| 14 | - $ref: lvds.yaml# | 14 | - $ref: panel-common.yaml# |
| 15 | - $ref: /schemas/display/lvds.yaml/# | ||
| 16 | |||
| 17 | select: | ||
| 18 | properties: | ||
| 19 | compatible: | ||
| 20 | contains: | ||
| 21 | const: innolux,ee101ia-01d | ||
| 22 | |||
| 23 | required: | ||
| 24 | - compatible | ||
| 15 | 25 | ||
| 16 | properties: | 26 | properties: |
| 17 | compatible: | 27 | compatible: |
| 18 | items: | 28 | items: |
| 19 | - const: innolux,ee101ia-01d | 29 | - const: innolux,ee101ia-01d |
| 20 | - {} # panel-lvds, but not listed here to avoid false select | 30 | - const: panel-lvds |
| 21 | 31 | ||
| 22 | backlight: true | 32 | backlight: true |
| 33 | data-mapping: true | ||
| 23 | enable-gpios: true | 34 | enable-gpios: true |
| 24 | power-supply: true | 35 | power-supply: true |
| 25 | width-mm: true | 36 | width-mm: true |
| @@ -27,5 +38,13 @@ properties: · 1 unmodified line | |||
| 27 | panel-timing: true | 38 | panel-timing: true |
| 28 | port: true | 39 | port: true |
| 29 | 40 | ||
| 41 | required: | ||
| 42 | - compatible | ||
| 43 | - data-mapping | ||
| 44 | - width-mm | ||
| 45 | - height-mm | ||
| 46 | - panel-timing | ||
| 47 | - port | ||
| 48 | |||
| 30 | additionalProperties: false | 49 | additionalProperties: false |
| 31 | ... | 50 | ... |
| @@ -11,13 +11,23 @@ maintainers: · 10 unmodified lines | |||
| 11 | - Thierry Reding <thierry.reding@gmail.com> | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | ||
| 13 | allOf: | 13 | allOf: |
| 14 | - $ref: lvds.yaml# | 14 | - $ref: panel-common.yaml# |
| 15 | - $ref: /schemas/display/lvds.yaml/# | ||
| 16 | |||
| 17 | select: | ||
| 18 | properties: | ||
| 19 | compatible: | ||
| 20 | contains: | ||
| 21 | const: mitsubishi,aa104xd12 | ||
| 22 | |||
| 23 | required: | ||
| 24 | - compatible | ||
| 15 | 25 | ||
| 16 | properties: | 26 | properties: |
| 17 | compatible: | 27 | compatible: |
| 18 | items: | 28 | items: |
| 19 | - const: mitsubishi,aa104xd12 | 29 | - const: mitsubishi,aa104xd12 |
| 20 | - {} # panel-lvds, but not listed here to avoid false select | 30 | - const: panel-lvds |
| 21 | 31 | ||
| 22 | vcc-supply: | 32 | vcc-supply: |
| 23 | description: Reference to the regulator powering the panel VCC pins. | 33 | description: Reference to the regulator powering the panel VCC pins. |
| @@ -39,6 +49,11 @@ additionalProperties: false · 15 unmodified lines | |||
| 39 | required: | 49 | required: |
| 40 | - compatible | 50 | - compatible |
| 41 | - vcc-supply | 51 | - vcc-supply |
| 52 | - data-mapping | ||
| 53 | - width-mm | ||
| 54 | - height-mm | ||
| 55 | - panel-timing | ||
| 56 | - port | ||
| 42 | 57 | ||
| 43 | examples: | 58 | examples: |
| 44 | - |+ | 59 | - |+ |
| @@ -11,13 +11,23 @@ maintainers: · 10 unmodified lines | |||
| 11 | - Thierry Reding <thierry.reding@gmail.com> | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | ||
| 13 | allOf: | 13 | allOf: |
| 14 | - $ref: lvds.yaml# | 14 | - $ref: panel-common.yaml# |
| 15 | - $ref: /schemas/display/lvds.yaml/# | ||
| 16 | |||
| 17 | select: | ||
| 18 | properties: | ||
| 19 | compatible: | ||
| 20 | contains: | ||
| 21 | const: mitsubishi,aa121td01 | ||
| 22 | |||
| 23 | required: | ||
| 24 | - compatible | ||
| 15 | 25 | ||
| 16 | properties: | 26 | properties: |
| 17 | compatible: | 27 | compatible: |
| 18 | items: | 28 | items: |
| 19 | - const: mitsubishi,aa121td01 | 29 | - const: mitsubishi,aa121td01 |
| 20 | - {} # panel-lvds, but not listed here to avoid false select | 30 | - const: panel-lvds |
| 21 | 31 | ||
| 22 | vcc-supply: | 32 | vcc-supply: |
| 23 | description: Reference to the regulator powering the panel VCC pins. | 33 | description: Reference to the regulator powering the panel VCC pins. |
| @@ -39,6 +49,11 @@ additionalProperties: false · 15 unmodified lines | |||
| 39 | required: | 49 | required: |
| 40 | - compatible | 50 | - compatible |
| 41 | - vcc-supply | 51 | - vcc-supply |
| 52 | - data-mapping | ||
| 53 | - width-mm | ||
| 54 | - height-mm | ||
| 55 | - panel-timing | ||
| 56 | - port | ||
| 42 | 57 | ||
| 43 | examples: | 58 | examples: |
| 44 | - |+ | 59 | - |+ |
| @@ -0,0 +1,57 @@ | |||
| 1 | # SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) | ||
| 2 | %YAML 1.2 | ||
| 3 | --- | ||
| 4 | $id: http://devicetree.org/schemas/display/panel/panel-lvds.yaml# | ||
| 5 | $schema: http://devicetree.org/meta-schemas/core.yaml# | ||
| 6 | |||
| 7 | title: Generic LVDS Display Panel Device Tree Bindings | ||
| 8 | |||
| 9 | maintainers: | ||
| 10 | - Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> | ||
| 11 | - Thierry Reding <thierry.reding@gmail.com> | ||
| 12 | |||
| 13 | allOf: | ||
| 14 | - $ref: panel-common.yaml# | ||
| 15 | - $ref: /schemas/display/lvds.yaml/# | ||
| 16 | |||
| 17 | select: | ||
| 18 | properties: | ||
| 19 | compatible: | ||
| 20 | contains: | ||
| 21 | const: panel-lvds | ||
| 22 | |||
| 23 | not: | ||
| 24 | properties: | ||
| 25 | compatible: | ||
| 26 | contains: | ||
| 27 | enum: | ||
| 28 | - advantech,idk-1110wr | ||
| 29 | - advantech,idk-2121wr | ||
| 30 | - innolux,ee101ia-01d | ||
| 31 | - mitsubishi,aa104xd12 | ||
| 32 | - mitsubishi,aa121td01 | ||
| 33 | - sgd,gktw70sdae4se | ||
| 34 | |||
| 35 | required: | ||
| 36 | - compatible | ||
| 37 | |||
| 38 | properties: | ||
| 39 | compatible: | ||
| 40 | items: | ||
| 41 | - enum: | ||
| 42 | - auo,b101ew05 | ||
| 43 | - tbs,a711-panel | ||
| 44 | |||
| 45 | - const: panel-lvds | ||
| 46 | |||
| 47 | unevaluatedProperties: false | ||
| 48 | |||
| 49 | required: | ||
| 50 | - compatible | ||
| 51 | - data-mapping | ||
| 52 | - width-mm | ||
| 53 | - height-mm | ||
| 54 | - panel-timing | ||
| 55 | - port | ||
| 56 | |||
| 57 | ... | ||
| @@ -284,6 +284,8 @@ properties: · 283 unmodified lines | |||
| 284 | - sharp,lq101k1ly04 | 284 | - sharp,lq101k1ly04 |
| 285 | # Sharp 12.3" (2400x1600 pixels) TFT LCD panel | 285 | # Sharp 12.3" (2400x1600 pixels) TFT LCD panel |
| 286 | - sharp,lq123p1jx31 | 286 | - sharp,lq123p1jx31 |
| 287 | # Sharp 14" (1920x1080 pixels) TFT LCD panel | ||
| 288 | - sharp,lq140m1jw46 | ||
| 287 | # Sharp LS020B1DD01D 2.0" HQVGA TFT LCD panel | 289 | # Sharp LS020B1DD01D 2.0" HQVGA TFT LCD panel |
| 288 | - sharp,ls020b1dd01d | 290 | - sharp,ls020b1dd01d |
| 289 | # Shelly SCA07010-BFN-LNN 7.0" WVGA TFT LCD panel | 291 | # Shelly SCA07010-BFN-LNN 7.0" WVGA TFT LCD panel |
| @@ -11,13 +11,23 @@ maintainers: · 10 unmodified lines | |||
| 11 | - Thierry Reding <thierry.reding@gmail.com> | 11 | - Thierry Reding <thierry.reding@gmail.com> |
| 12 | 12 | ||
| 13 | allOf: | 13 | allOf: |
| 14 | - $ref: lvds.yaml# | 14 | - $ref: panel-common.yaml# |
| 15 | - $ref: /schemas/display/lvds.yaml/# | ||
| 16 | |||
| 17 | select: | ||
| 18 | properties: | ||
| 19 | compatible: | ||
| 20 | contains: | ||
| 21 | const: sgd,gktw70sdae4se | ||
| 22 | |||
| 23 | required: | ||
| 24 | - compatible | ||
| 15 | 25 | ||
| 16 | properties: | 26 | properties: |
| 17 | compatible: | 27 | compatible: |
| 18 | items: | 28 | items: |
| 19 | - const: sgd,gktw70sdae4se | 29 | - const: sgd,gktw70sdae4se |
| 20 | - {} # panel-lvds, but not listed here to avoid false select | 30 | - const: panel-lvds |
| 21 | 31 | ||
| 22 | data-mapping: | 32 | data-mapping: |
| 23 | const: jeida-18 | 33 | const: jeida-18 |
| @@ -35,6 +45,11 @@ additionalProperties: false · 11 unmodified lines | |||
| 35 | 45 | ||
| 36 | required: | 46 | required: |
| 37 | - compatible | 47 | - compatible |
| 48 | - port | ||
| 49 | - data-mapping | ||
| 50 | - width-mm | ||
| 51 | - height-mm | ||
| 52 | - panel-timing | ||
| 38 | 53 | ||
| 39 | examples: | 54 | examples: |
| 40 | - |+ | 55 | - |+ |
| @@ -8,6 +8,7 @@ title: Solomon SSD1307 OLED Controller Framebuffer · 7 unmodified lines | |||
| 8 | 8 | ||
| 9 | maintainers: | 9 | maintainers: |
| 10 | - Maxime Ripard <mripard@kernel.org> | 10 | - Maxime Ripard <mripard@kernel.org> |
| 11 | - Javier Martinez Canillas <javierm@redhat.com> | ||
| 11 | 12 | ||
| 12 | properties: | 13 | properties: |
| 13 | compatible: | 14 | compatible: |
| @@ -7,7 +7,6 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | |||
| 7 | title: SiFive GPIO controller | 7 | title: SiFive GPIO controller |
| 8 | 8 | ||
| 9 | maintainers: | 9 | maintainers: |
| 10 | - Yash Shah <yash.shah@sifive.com> | ||
| 11 | - Paul Walmsley <paul.walmsley@sifive.com> | 10 | - Paul Walmsley <paul.walmsley@sifive.com> |
| 12 | 11 | ||
| 13 | properties: | 12 | properties: |
| @@ -159,6 +159,21 @@ allOf: · 158 unmodified lines | |||
| 159 | power-domains: | 159 | power-domains: |
| 160 | maxItems: 1 | 160 | maxItems: 1 |
| 161 | sram-supply: false | 161 | sram-supply: false |
| 162 | - if: | ||
| 163 | properties: | ||
| 164 | compatible: | ||
| 165 | contains: | ||
| 166 | const: rockchip,rk3568-mali | ||
| 167 | then: | ||
| 168 | properties: | ||
| 169 | clocks: | ||
| 170 | minItems: 2 | ||
| 171 | clock-names: | ||
| 172 | items: | ||
| 173 | - const: gpu | ||
| 174 | - const: bus | ||
| 175 | required: | ||
| 176 | - clock-names | ||
| 162 | 177 | ||
| 163 | examples: | 178 | examples: |
| 164 | - | | 179 | - | |
| @@ -35,6 +35,10 @@ description: · 34 unmodified lines | |||
| 35 | contains a specific memory layout, which is documented in chapter 8 of the | 35 | contains a specific memory layout, which is documented in chapter 8 of the |
| 36 | SiFive U5 Coreplex Series Manual <https://static.dev.sifive.com/U54-MC-RVCoreIP.pdf>. | 36 | SiFive U5 Coreplex Series Manual <https://static.dev.sifive.com/U54-MC-RVCoreIP.pdf>. |
| 37 | 37 | ||
| 38 | The thead,c900-plic is different from sifive,plic-1.0.0 in opensbi, the | ||
| 39 | T-HEAD PLIC implementation requires setting a delegation bit to allow access | ||
| 40 | from S-mode. So add thead,c900-plic to distinguish them. | ||
| 41 | |||
| 38 | maintainers: | 42 | maintainers: |
| 39 | - Sagar Kadam <sagar.kadam@sifive.com> | 43 | - Sagar Kadam <sagar.kadam@sifive.com> |
| 40 | - Paul Walmsley <paul.walmsley@sifive.com> | 44 | - Paul Walmsley <paul.walmsley@sifive.com> |
| @@ -42,12 +46,17 @@ maintainers: · 1 unmodified line | |||
| 42 | 46 | ||
| 43 | properties: | 47 | properties: |
| 44 | compatible: | 48 | compatible: |
| 45 | items: | 49 | oneOf: |
| 46 | - enum: | 50 | - items: |
| 47 | - sifive,fu540-c000-plic | 51 | - enum: |
| 48 | - starfive,jh7100-plic | 52 | - sifive,fu540-c000-plic |
| 49 | - canaan,k210-plic | 53 | - starfive,jh7100-plic |
| 50 | - const: sifive,plic-1.0.0 | 54 | - canaan,k210-plic |
| 55 | - const: sifive,plic-1.0.0 | ||
| 56 | - items: | ||
| 57 | - enum: | ||
| 58 | - allwinner,sun20i-d1-plic | ||
| 59 | - const: thead,c900-plic | ||
| 51 | 60 | ||
| 52 | reg: | 61 | reg: |
| 53 | maxItems: 1 | 62 | maxItems: 1 |
| @@ -20,7 +20,7 @@ description: | · 19 unmodified lines | |||
| 20 | 20 | ||
| 21 | maintainers: | 21 | maintainers: |
| 22 | - Kishon Vijay Abraham I <kishon@ti.com> | 22 | - Kishon Vijay Abraham I <kishon@ti.com> |
| 23 | - Roger Quadros <rogerq@ti.com | 23 | - Roger Quadros <rogerq@kernel.org> |
| 24 | 24 | ||
| 25 | properties: | 25 | properties: |
| 26 | compatible: | 26 | compatible: |
| @@ -107,6 +107,10 @@ properties: · 106 unmodified lines | |||
| 107 | - const: imem | 107 | - const: imem |
| 108 | - const: config | 108 | - const: config |
| 109 | 109 | ||
| 110 | qcom,qmp: | ||
| 111 | $ref: /schemas/types.yaml#/definitions/phandle | ||
| 112 | description: phandle to the AOSS side-channel message RAM | ||
| 113 | |||
| 110 | qcom,smem-states: | 114 | qcom,smem-states: |
| 111 | $ref: /schemas/types.yaml#/definitions/phandle-array | 115 | $ref: /schemas/types.yaml#/definitions/phandle-array |
| 112 | description: State bits used in by the AP to signal the modem. | 116 | description: State bits used in by the AP to signal the modem. |
| @@ -222,6 +226,8 @@ examples: · 109 unmodified lines | |||
| 222 | "imem", | 226 | "imem", |
| 223 | "config"; | 227 | "config"; |
| 224 | 228 | ||
| 229 | qcom,qmp = <&aoss_qmp>; | ||
| 230 | |||
| 225 | qcom,smem-states = <&ipa_smp2p_out 0>, | 231 | qcom,smem-states = <&ipa_smp2p_out 0>, |
| 226 | <&ipa_smp2p_out 1>; | 232 | <&ipa_smp2p_out 1>; |
| 227 | qcom,smem-state-names = "ipa-clock-enabled-valid", | 233 | qcom,smem-state-names = "ipa-clock-enabled-valid", |
| @@ -8,7 +8,7 @@ title: OMAP USB2 PHY · 7 unmodified lines | |||
| 8 | 8 | ||
| 9 | maintainers: | 9 | maintainers: |
| 10 | - Kishon Vijay Abraham I <kishon@ti.com> | 10 | - Kishon Vijay Abraham I <kishon@ti.com> |
| 11 | - Roger Quadros <rogerq@ti.com> | 11 | - Roger Quadros <rogerq@kernel.org> |
| 12 | 12 | ||
| 13 | properties: | 13 | properties: |
| 14 | compatible: | 14 | compatible: |
| @@ -8,7 +8,6 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 7 unmodified lines | |||
| 8 | title: SiFive PWM controller | 8 | title: SiFive PWM controller |
| 9 | 9 | ||
| 10 | maintainers: | 10 | maintainers: |
| 11 | - Yash Shah <yash.shah@sifive.com> | ||
| 12 | - Sagar Kadam <sagar.kadam@sifive.com> | 11 | - Sagar Kadam <sagar.kadam@sifive.com> |
| 13 | - Paul Walmsley <paul.walmsley@sifive.com> | 12 | - Paul Walmsley <paul.walmsley@sifive.com> |
| 14 | 13 | ||
| @@ -9,7 +9,6 @@ title: SiFive L2 Cache Controller · 8 unmodified lines | |||
| 9 | 9 | ||
| 10 | maintainers: | 10 | maintainers: |
| 11 | - Sagar Kadam <sagar.kadam@sifive.com> | 11 | - Sagar Kadam <sagar.kadam@sifive.com> |
| 12 | - Yash Shah <yash.shah@sifive.com> | ||
| 13 | - Paul Walmsley <paul.walmsley@sifive.com> | 12 | - Paul Walmsley <paul.walmsley@sifive.com> |
| 14 | 13 | ||
| 15 | description: | 14 | description: |
| @@ -8,6 +8,7 @@ title: Audio codec controlled by ChromeOS EC · 7 unmodified lines | |||
| 8 | 8 | ||
| 9 | maintainers: | 9 | maintainers: |
| 10 | - Cheng-Yi Chiang <cychiang@chromium.org> | 10 | - Cheng-Yi Chiang <cychiang@chromium.org> |
| 11 | - Tzung-Bi Shih <tzungbi@google.com> | ||
| 11 | 12 | ||
| 12 | description: | | 13 | description: | |
| 13 | Google's ChromeOS EC codec is a digital mic codec provided by the | 14 | Google's ChromeOS EC codec is a digital mic codec provided by the |
| @@ -23,8 +23,9 @@ properties: · 22 unmodified lines | |||
| 23 | minItems: 1 | 23 | minItems: 1 |
| 24 | maxItems: 256 | 24 | maxItems: 256 |
| 25 | items: | 25 | items: |
| 26 | minimum: 0 | 26 | items: |
| 27 | maximum: 256 | 27 | - minimum: 0 |
| 28 | maximum: 256 | ||
| 28 | description: | 29 | description: |
| 29 | Chip select used by the device. | 30 | Chip select used by the device. |
| 30 | 31 | ||
| @@ -7,7 +7,7 @@ $schema: "http://devicetree.org/meta-schemas/core.yaml#" · 6 unmodified lines | |||
| 7 | title: Bindings for the TI wrapper module for the Cadence USBSS-DRD controller | 7 | title: Bindings for the TI wrapper module for the Cadence USBSS-DRD controller |
| 8 | 8 | ||
| 9 | maintainers: | 9 | maintainers: |
| 10 | - Roger Quadros <rogerq@ti.com> | 10 | - Roger Quadros <rogerq@kernel.org> |
| 11 | 11 | ||
| 12 | properties: | 12 | properties: |
| 13 | compatible: | 13 | compatible: |
| @@ -7,7 +7,7 @@ $schema: http://devicetree.org/meta-schemas/core.yaml# · 6 unmodified lines | |||
| 7 | title: TI Keystone Soc USB Controller | 7 | title: TI Keystone Soc USB Controller |
| 8 | 8 | ||
| 9 | maintainers: | 9 | maintainers: |
| 10 | - Roger Quadros <rogerq@ti.com> | 10 | - Roger Quadros <rogerq@kernel.org> |
| 11 | 11 | ||
| 12 | properties: | 12 | properties: |
| 13 | compatible: | 13 | compatible: |
| @@ -462,6 +462,10 @@ operation table looks like the following:: · 461 unmodified lines | |||
| 462 | struct iov_iter *iter, | 462 | struct iov_iter *iter, |
| 463 | netfs_io_terminated_t term_func, | 463 | netfs_io_terminated_t term_func, |
| 464 | void *term_func_priv); | 464 | void *term_func_priv); |
| 465 | |||
| 466 | int (*query_occupancy)(struct netfs_cache_resources *cres, | ||
| 467 | loff_t start, size_t len, size_t granularity, | ||
| 468 | loff_t *_data_start, size_t *_data_len); | ||
| 465 | }; | 469 | }; |
| 466 | 470 | ||
| 467 | With a termination handler function pointer:: | 471 | With a termination handler function pointer:: |
| @@ -536,6 +540,18 @@ The methods defined in the table are: · 68 unmodified lines | |||
| 536 | indicating whether the termination is definitely happening in the caller's | 540 | indicating whether the termination is definitely happening in the caller's |
| 537 | context. | 541 | context. |
| 538 | 542 | ||
| 543 | * ``query_occupancy()`` | ||
| 544 | |||
| 545 | [Required] Called to find out where the next piece of data is within a | ||
| 546 | particular region of the cache. The start and length of the region to be | ||
| 547 | queried are passed in, along with the granularity to which the answer needs | ||
| 548 | to be aligned. The function passes back the start and length of the data, | ||
| 549 | if any, available within that region. Note that there may be a hole at the | ||
| 550 | front. | ||
| 551 | |||
| 552 | It returns 0 if some data was found, -ENODATA if there was no usable data | ||
| 553 | within the region or -ENOBUFS if there is no caching on this file. | ||
| 554 | |||
| 539 | Note that these methods are passed a pointer to the cache resource structure, | 555 | Note that these methods are passed a pointer to the cache resource structure, |
| 540 | not the read request structure as they could be used in other situations where | 556 | not the read request structure as they could be used in other situations where |
| 541 | there isn't a read request structure as well, such as writing dirty data to the | 557 | there isn't a read request structure as well, such as writing dirty data to the |
| @@ -300,30 +300,6 @@ Contact: Daniel Vetter, Noralf Tronnes · 299 unmodified lines | |||
| 300 | 300 | ||
| 301 | Level: Advanced | 301 | Level: Advanced |
| 302 | 302 | ||
| 303 | Garbage collect fbdev scrolling acceleration | ||
| 304 | -------------------------------------------- | ||
| 305 | |||
| 306 | Scroll acceleration has been disabled in fbcon. Now it works as the old | ||
| 307 | SCROLL_REDRAW mode. A ton of code was removed in fbcon.c and the hook bmove was | ||
| 308 | removed from fbcon_ops. | ||
| 309 | Remaining tasks: | ||
| 310 | |||
| 311 | - a bunch of the hooks in fbcon_ops could be removed or simplified by calling | ||
| 312 | directly instead of the function table (with a switch on p->rotate) | ||
| 313 | |||
| 314 | - fb_copyarea is unused after this, and can be deleted from all drivers | ||
| 315 | |||
| 316 | - after that, fb_copyarea can be deleted from fb_ops in include/linux/fb.h as | ||
| 317 | well as cfb_copyarea | ||
| 318 | |||
| 319 | Note that not all acceleration code can be deleted, since clearing and cursor | ||
| 320 | support is still accelerated, which might be good candidates for further | ||
| 321 | deletion projects. | ||
| 322 | |||
| 323 | Contact: Daniel Vetter | ||
| 324 | |||
| 325 | Level: Intermediate | ||
| 326 | |||
| 327 | idr_init_base() | 303 | idr_init_base() |
| 328 | --------------- | 304 | --------------- |
| 329 | 305 | ||
| @@ -2,7 +2,7 @@ · 1 unmodified line | |||
| 2 | 2 | ||
| 3 | Set the histogram bucket size (default *1*). | 3 | Set the histogram bucket size (default *1*). |
| 4 | 4 | ||
| 5 | **-e**, **--entries** *N* | 5 | **-E**, **--entries** *N* |
| 6 | 6 | ||
| 7 | Set the number of entries of the histogram (default 256). | 7 | Set the number of entries of the histogram (default 256). |
| 8 | 8 | ||
| @@ -1,7 +1,7 @@ | |||
| 1 | The **rtla osnoise** tool is an interface for the *osnoise* tracer. The | 1 | The **rtla osnoise** tool is an interface for the *osnoise* tracer. The |
| 2 | *osnoise* tracer dispatches a kernel thread per-cpu. These threads read the | 2 | *osnoise* tracer dispatches a kernel thread per-cpu. These threads read the |
| 3 | time in a loop while with preemption, softirq and IRQs enabled, thus | 3 | time in a loop while with preemption, softirq and IRQs enabled, thus |
| 4 | allowing all the sources of operating systme noise during its execution. | 4 | allowing all the sources of operating system noise during its execution. |
| 5 | The *osnoise*'s tracer threads take note of the delta between each time | 5 | The *osnoise*'s tracer threads take note of the delta between each time |
| 6 | read, along with an interference counter of all sources of interference. | 6 | read, along with an interference counter of all sources of interference. |
| 7 | At the end of each period, the *osnoise* tracer displays a summary of | 7 | At the end of each period, the *osnoise* tracer displays a summary of |
| @@ -36,7 +36,7 @@ default). The reason for reducing the runtime is to avoid starving the · 35 unmodified lines | |||
| 36 | **rtla** tool. The tool is also set to run for *one minute*. The output | 36 | **rtla** tool. The tool is also set to run for *one minute*. The output |
| 37 | histogram is set to group outputs in buckets of *10us* and *25* entries:: | 37 | histogram is set to group outputs in buckets of *10us* and *25* entries:: |
| 38 | 38 | ||
| 39 | [root@f34 ~/]# rtla osnoise hist -P F:1 -c 0-11 -r 900000 -d 1M -b 10 -e 25 | 39 | [root@f34 ~/]# rtla osnoise hist -P F:1 -c 0-11 -r 900000 -d 1M -b 10 -E 25 |
| 40 | # RTLA osnoise histogram | 40 | # RTLA osnoise histogram |
| 41 | # Time unit is microseconds (us) | 41 | # Time unit is microseconds (us) |
| 42 | # Duration: 0 00:01:00 | 42 | # Duration: 0 00:01:00 |
| @@ -84,6 +84,8 @@ CPUfreq核心层注册一个cpufreq_driver结构体。 · 83 unmodified lines | |||
| 84 | .resume - 一个指向per-policy恢复函数的指针,该函数在关中断且在调节器再一次启动前被 | 84 | .resume - 一个指向per-policy恢复函数的指针,该函数在关中断且在调节器再一次启动前被 |
| 85 | 调用。 | 85 | 调用。 |
| 86 | 86 | ||
| 87 | .ready - 一个指向per-policy准备函数的指针,该函数在策略完全初始化之后被调用。 | ||
| 88 | |||
| 87 | .attr - 一个指向NULL结尾的"struct freq_attr"列表的指针,该列表允许导出值到 | 89 | .attr - 一个指向NULL结尾的"struct freq_attr"列表的指针,该列表允许导出值到 |
| 88 | sysfs。 | 90 | sysfs。 |
| 89 | 91 | ||
| @@ -115,6 +115,7 @@ Code Seq# Include File Comments · 114 unmodified lines | |||
| 115 | 'B' 00-1F linux/cciss_ioctl.h conflict! | 115 | 'B' 00-1F linux/cciss_ioctl.h conflict! |
| 116 | 'B' 00-0F include/linux/pmu.h conflict! | 116 | 'B' 00-0F include/linux/pmu.h conflict! |
| 117 | 'B' C0-FF advanced bbus <mailto:maassen@uni-freiburg.de> | 117 | 'B' C0-FF advanced bbus <mailto:maassen@uni-freiburg.de> |
| 118 | 'B' 00-0F xen/xenbus_dev.h conflict! | ||
| 118 | 'C' all linux/soundcard.h conflict! | 119 | 'C' all linux/soundcard.h conflict! |
| 119 | 'C' 01-2F linux/capi.h conflict! | 120 | 'C' 01-2F linux/capi.h conflict! |
| 120 | 'C' F0-FF drivers/net/wan/cosa.h conflict! | 121 | 'C' F0-FF drivers/net/wan/cosa.h conflict! |
| @@ -134,6 +135,7 @@ Code Seq# Include File Comments · 13 unmodified lines | |||
| 134 | 'F' 80-8F linux/arcfb.h conflict! | 135 | 'F' 80-8F linux/arcfb.h conflict! |
| 135 | 'F' DD video/sstfb.h conflict! | 136 | 'F' DD video/sstfb.h conflict! |
| 136 | 'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict! | 137 | 'G' 00-3F drivers/misc/sgi-gru/grulib.h conflict! |
| 138 | 'G' 00-0F xen/gntalloc.h, xen/gntdev.h conflict! | ||
| 137 | 'H' 00-7F linux/hiddev.h conflict! | 139 | 'H' 00-7F linux/hiddev.h conflict! |
| 138 | 'H' 00-0F linux/hidraw.h conflict! | 140 | 'H' 00-0F linux/hidraw.h conflict! |
| 139 | 'H' 01 linux/mei.h conflict! | 141 | 'H' 01 linux/mei.h conflict! |
| @@ -176,6 +178,7 @@ Code Seq# Include File Comments · 36 unmodified lines | |||
| 176 | 'P' 60-6F sound/sscape_ioctl.h conflict! | 178 | 'P' 60-6F sound/sscape_ioctl.h conflict! |
| 177 | 'P' 00-0F drivers/usb/class/usblp.c conflict! | 179 | 'P' 00-0F drivers/usb/class/usblp.c conflict! |
| 178 | 'P' 01-09 drivers/misc/pci_endpoint_test.c conflict! | 180 | 'P' 01-09 drivers/misc/pci_endpoint_test.c conflict! |
| 181 | 'P' 00-0F xen/privcmd.h conflict! | ||
| 179 | 'Q' all linux/soundcard.h | 182 | 'Q' all linux/soundcard.h |
| 180 | 'R' 00-1F linux/random.h conflict! | 183 | 'R' 00-1F linux/random.h conflict! |
| 181 | 'R' 01 linux/rfkill.h conflict! | 184 | 'R' 01 linux/rfkill.h conflict! |
| @@ -1394,7 +1394,7 @@ documentation when it pops into existence). · 1393 unmodified lines | |||
| 1394 | ------------------- | 1394 | ------------------- |
| 1395 | 1395 | ||
| 1396 | :Capability: KVM_CAP_ENABLE_CAP | 1396 | :Capability: KVM_CAP_ENABLE_CAP |
| 1397 | :Architectures: mips, ppc, s390 | 1397 | :Architectures: mips, ppc, s390, x86 |
| 1398 | :Type: vcpu ioctl | 1398 | :Type: vcpu ioctl |
| 1399 | :Parameters: struct kvm_enable_cap (in) | 1399 | :Parameters: struct kvm_enable_cap (in) |
| 1400 | :Returns: 0 on success; -1 on error | 1400 | :Returns: 0 on success; -1 on error |
| @@ -6997,6 +6997,20 @@ indicated by the fd to the VM this is called on. · 5596 unmodified lines | |||
| 6997 | This is intended to support intra-host migration of VMs between userspace VMMs, | 6997 | This is intended to support intra-host migration of VMs between userspace VMMs, |
| 6998 | upgrading the VMM process without interrupting the guest. | 6998 | upgrading the VMM process without interrupting the guest. |
| 6999 | 6999 | ||
| 7000 | 7.30 KVM_CAP_PPC_AIL_MODE_3 | ||
| 7001 | ------------------------------- | ||
| 7002 | |||
| 7003 | :Capability: KVM_CAP_PPC_AIL_MODE_3 | ||
| 7004 | :Architectures: ppc | ||
| 7005 | :Type: vm | ||
| 7006 | |||
| 7007 | This capability indicates that the kernel supports the mode 3 setting for the | ||
| 7008 | "Address Translation Mode on Interrupt" aka "Alternate Interrupt Location" | ||
| 7009 | resource that is controlled with the H_SET_MODE hypercall. | ||
| 7010 | |||
| 7011 | This capability allows a guest kernel to use a better-performance mode for | ||
| 7012 | handling interrupts and system calls. | ||
| 7013 | |||
| 7000 | 8. Other capabilities. | 7014 | 8. Other capabilities. |
| 7001 | ====================== | 7015 | ====================== |
| 7002 | 7016 | ||
| @@ -1620,6 +1620,7 @@ M: Olof Johansson <olof@lixom.net> · 1619 unmodified lines | |||
| 1620 | M: soc@kernel.org | 1620 | M: soc@kernel.org |
| 1621 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | 1621 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 1622 | S: Maintained | 1622 | S: Maintained |
| 1623 | C: irc://irc.libera.chat/armlinux | ||
| 1623 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git | 1624 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git |
| 1624 | F: arch/arm/boot/dts/Makefile | 1625 | F: arch/arm/boot/dts/Makefile |
| 1625 | F: arch/arm64/boot/dts/Makefile | 1626 | F: arch/arm64/boot/dts/Makefile |
| @@ -1627,6 +1628,7 @@ F: arch/arm64/boot/dts/Makefile · 1 unmodified line | |||
| 1627 | ARM SUB-ARCHITECTURES | 1628 | ARM SUB-ARCHITECTURES |
| 1628 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | 1629 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 1629 | S: Maintained | 1630 | S: Maintained |
| 1631 | C: irc://irc.libera.chat/armlinux | ||
| 1630 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git | 1632 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git |
| 1631 | F: arch/arm/mach-*/ | 1633 | F: arch/arm/mach-*/ |
| 1632 | F: arch/arm/plat-*/ | 1634 | F: arch/arm/plat-*/ |
| @@ -1780,6 +1782,7 @@ F: drivers/irqchip/irq-apple-aic.c · 147 unmodified lines | |||
| 1780 | F: drivers/mailbox/apple-mailbox.c | 1782 | F: drivers/mailbox/apple-mailbox.c |
| 1781 | F: drivers/pinctrl/pinctrl-apple-gpio.c | 1783 | F: drivers/pinctrl/pinctrl-apple-gpio.c |
| 1782 | F: drivers/soc/apple/* | 1784 | F: drivers/soc/apple/* |
| 1785 | F: drivers/watchdog/apple_wdt.c | ||
| 1783 | F: include/dt-bindings/interrupt-controller/apple-aic.h | 1786 | F: include/dt-bindings/interrupt-controller/apple-aic.h |
| 1784 | F: include/dt-bindings/pinctrl/apple.h | 1787 | F: include/dt-bindings/pinctrl/apple.h |
| 1785 | F: include/linux/apple-mailbox.h | 1788 | F: include/linux/apple-mailbox.h |
| @@ -2570,10 +2573,13 @@ N: rockchip · 784 unmodified lines | |||
| 2570 | 2573 | ||
| 2571 | ARM/SAMSUNG S3C, S5P AND EXYNOS ARM ARCHITECTURES | 2574 | ARM/SAMSUNG S3C, S5P AND EXYNOS ARM ARCHITECTURES |
| 2572 | M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com> | 2575 | M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com> |
| 2576 | R: Alim Akhtar <alim.akhtar@samsung.com> | ||
| 2573 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | 2577 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 2574 | L: linux-samsung-soc@vger.kernel.org | 2578 | L: linux-samsung-soc@vger.kernel.org |
| 2575 | S: Maintained | 2579 | S: Maintained |
| 2580 | C: irc://irc.libera.chat/linux-exynos | ||
| 2576 | Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/ | 2581 | Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/ |
| 2582 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux.git | ||
| 2577 | F: Documentation/arm/samsung/ | 2583 | F: Documentation/arm/samsung/ |
| 2578 | F: Documentation/devicetree/bindings/arm/samsung/ | 2584 | F: Documentation/devicetree/bindings/arm/samsung/ |
| 2579 | F: Documentation/devicetree/bindings/power/pd-samsung.yaml | 2585 | F: Documentation/devicetree/bindings/power/pd-samsung.yaml |
| @@ -3133,11 +3139,9 @@ W: https://wireless.wiki.kernel.org/en/users/Drivers/ath5k · 553 unmodified lines | |||
| 3133 | F: drivers/net/wireless/ath/ath5k/ | 3139 | F: drivers/net/wireless/ath/ath5k/ |
| 3134 | 3140 | ||
| 3135 | ATHEROS ATH6KL WIRELESS DRIVER | 3141 | ATHEROS ATH6KL WIRELESS DRIVER |
| 3136 | M: Kalle Valo <kvalo@kernel.org> | ||
| 3137 | L: linux-wireless@vger.kernel.org | 3142 | L: linux-wireless@vger.kernel.org |
| 3138 | S: Supported | 3143 | S: Orphan |
| 3139 | W: https://wireless.wiki.kernel.org/en/users/Drivers/ath6kl | 3144 | W: https://wireless.wiki.kernel.org/en/users/Drivers/ath6kl |
| 3140 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git | ||
| 3141 | F: drivers/net/wireless/ath/ath6kl/ | 3145 | F: drivers/net/wireless/ath/ath6kl/ |
| 3142 | 3146 | ||
| 3143 | ATI_REMOTE2 DRIVER | 3147 | ATI_REMOTE2 DRIVER |
| @@ -4157,9 +4161,8 @@ N: csky · 1013 unmodified lines | |||
| 4157 | K: csky | 4161 | K: csky |
| 4158 | 4162 | ||
| 4159 | CA8210 IEEE-802.15.4 RADIO DRIVER | 4163 | CA8210 IEEE-802.15.4 RADIO DRIVER |
| 4160 | M: Harry Morris <h.morris@cascoda.com> | ||
| 4161 | L: linux-wpan@vger.kernel.org | 4164 | L: linux-wpan@vger.kernel.org |
| 4162 | S: Maintained | 4165 | S: Orphan |
| 4163 | W: https://github.com/Cascoda/ca8210-linux.git | 4166 | W: https://github.com/Cascoda/ca8210-linux.git |
| 4164 | F: Documentation/devicetree/bindings/net/ieee802154/ca8210.txt | 4167 | F: Documentation/devicetree/bindings/net/ieee802154/ca8210.txt |
| 4165 | F: drivers/net/ieee802154/ca8210.c | 4168 | F: drivers/net/ieee802154/ca8210.c |
| @@ -4544,6 +4547,7 @@ F: drivers/platform/chrome/ · 378 unmodified lines | |||
| 4544 | 4547 | ||
| 4545 | CHROMEOS EC CODEC DRIVER | 4548 | CHROMEOS EC CODEC DRIVER |
| 4546 | M: Cheng-Yi Chiang <cychiang@chromium.org> | 4549 | M: Cheng-Yi Chiang <cychiang@chromium.org> |
| 4550 | M: Tzung-Bi Shih <tzungbi@google.com> | ||
| 4547 | R: Guenter Roeck <groeck@chromium.org> | 4551 | R: Guenter Roeck <groeck@chromium.org> |
| 4548 | S: Maintained | 4552 | S: Maintained |
| 4549 | F: Documentation/devicetree/bindings/sound/google,cros-ec-codec.yaml | 4553 | F: Documentation/devicetree/bindings/sound/google,cros-ec-codec.yaml |
| @@ -4909,7 +4913,8 @@ F: kernel/cgroup/cpuset.c · 359 unmodified lines | |||
| 4909 | CONTROL GROUP - MEMORY RESOURCE CONTROLLER (MEMCG) | 4913 | CONTROL GROUP - MEMORY RESOURCE CONTROLLER (MEMCG) |
| 4910 | M: Johannes Weiner <hannes@cmpxchg.org> | 4914 | M: Johannes Weiner <hannes@cmpxchg.org> |
| 4911 | M: Michal Hocko <mhocko@kernel.org> | 4915 | M: Michal Hocko <mhocko@kernel.org> |
| 4912 | M: Vladimir Davydov <vdavydov.dev@gmail.com> | 4916 | M: Roman Gushchin <roman.gushchin@linux.dev> |
| 4917 | M: Shakeel Butt <shakeelb@google.com> | ||
| 4913 | L: cgroups@vger.kernel.org | 4918 | L: cgroups@vger.kernel.org |
| 4914 | L: linux-mm@kvack.org | 4919 | L: linux-mm@kvack.org |
| 4915 | S: Maintained | 4920 | S: Maintained |
| @@ -5773,7 +5778,7 @@ F: tools/testing/selftests/dma/ · 857 unmodified lines | |||
| 5773 | 5778 | ||
| 5774 | DMA-BUF HEAPS FRAMEWORK | 5779 | DMA-BUF HEAPS FRAMEWORK |
| 5775 | M: Sumit Semwal <sumit.semwal@linaro.org> | 5780 | M: Sumit Semwal <sumit.semwal@linaro.org> |
| 5776 | R: Benjamin Gaignard <benjamin.gaignard@linaro.org> | 5781 | R: Benjamin Gaignard <benjamin.gaignard@collabora.com> |
| 5777 | R: Liam Mark <lmark@codeaurora.org> | 5782 | R: Liam Mark <lmark@codeaurora.org> |
| 5778 | R: Laura Abbott <labbott@redhat.com> | 5783 | R: Laura Abbott <labbott@redhat.com> |
| 5779 | R: Brian Starkey <Brian.Starkey@arm.com> | 5784 | R: Brian Starkey <Brian.Starkey@arm.com> |
| @@ -6077,7 +6082,8 @@ L: dri-devel@lists.freedesktop.org · 297 unmodified lines | |||
| 6077 | T: git git://anongit.freedesktop.org/drm/drm-misc | 6082 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| 6078 | S: Maintained | 6083 | S: Maintained |
| 6079 | F: drivers/gpu/drm/panel/panel-lvds.c | 6084 | F: drivers/gpu/drm/panel/panel-lvds.c |
| 6080 | F: Documentation/devicetree/bindings/display/panel/lvds.yaml | 6085 | F: Documentation/devicetree/bindings/display/lvds.yaml |
| 6086 | F: Documentation/devicetree/bindings/display/panel/panel-lvds.yaml | ||
| 6081 | 6087 | ||
| 6082 | DRM DRIVER FOR MANTIX MLAF057WE51 PANELS | 6088 | DRM DRIVER FOR MANTIX MLAF057WE51 PANELS |
| 6083 | M: Guido Günther <agx@sigxcpu.org> | 6089 | M: Guido Günther <agx@sigxcpu.org> |
| @@ -6126,6 +6132,13 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 42 unmodified lines | |||
| 6126 | F: Documentation/devicetree/bindings/display/panel/novatek,nt35510.yaml | 6132 | F: Documentation/devicetree/bindings/display/panel/novatek,nt35510.yaml |
| 6127 | F: drivers/gpu/drm/panel/panel-novatek-nt35510.c | 6133 | F: drivers/gpu/drm/panel/panel-novatek-nt35510.c |
| 6128 | 6134 | ||
| 6135 | DRM DRIVER FOR NOVATEK NT35560 PANELS | ||
| 6136 | M: Linus Walleij <linus.walleij@linaro.org> | ||
| 6137 | S: Maintained | ||
| 6138 | T: git git://anongit.freedesktop.org/drm/drm-misc | ||
| 6139 | F: Documentation/devicetree/bindings/display/panel/sony,acx424akp.yaml | ||
| 6140 | F: drivers/gpu/drm/panel/panel-novatek-nt35560.c | ||
| 6141 | |||
| 6129 | DRM DRIVER FOR NOVATEK NT36672A PANELS | 6142 | DRM DRIVER FOR NOVATEK NT36672A PANELS |
| 6130 | M: Sumit Semwal <sumit.semwal@linaro.org> | 6143 | M: Sumit Semwal <sumit.semwal@linaro.org> |
| 6131 | S: Maintained | 6144 | S: Maintained |
| @@ -6162,6 +6175,13 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 30 unmodified lines | |||
| 6162 | F: Documentation/devicetree/bindings/display/repaper.txt | 6175 | F: Documentation/devicetree/bindings/display/repaper.txt |
| 6163 | F: drivers/gpu/drm/tiny/repaper.c | 6176 | F: drivers/gpu/drm/tiny/repaper.c |
| 6164 | 6177 | ||
| 6178 | DRM DRIVER FOR SOLOMON SSD130X OLED DISPLAYS | ||
| 6179 | M: Javier Martinez Canillas <javierm@redhat.com> | ||
| 6180 | S: Maintained | ||
| 6181 | T: git git://anongit.freedesktop.org/drm/drm-misc | ||
| 6182 | F: Documentation/devicetree/bindings/display/solomon,ssd1307fb.yaml | ||
| 6183 | F: drivers/gpu/drm/solomon/ssd130x* | ||
| 6184 | |||
| 6165 | DRM DRIVER FOR QEMU'S CIRRUS DEVICE | 6185 | DRM DRIVER FOR QEMU'S CIRRUS DEVICE |
| 6166 | M: Dave Airlie <airlied@redhat.com> | 6186 | M: Dave Airlie <airlied@redhat.com> |
| 6167 | M: Gerd Hoffmann <kraxel@redhat.com> | 6187 | M: Gerd Hoffmann <kraxel@redhat.com> |
| @@ -6250,12 +6270,6 @@ T: git git://anongit.freedesktop.org/drm/drm-misc · 82 unmodified lines | |||
| 6250 | F: Documentation/devicetree/bindings/display/sitronix,st7735r.yaml | 6270 | F: Documentation/devicetree/bindings/display/sitronix,st7735r.yaml |
| 6251 | F: drivers/gpu/drm/tiny/st7735r.c | 6271 | F: drivers/gpu/drm/tiny/st7735r.c |
| 6252 | 6272 | ||
| 6253 | DRM DRIVER FOR SONY ACX424AKP PANELS | ||
| 6254 | M: Linus Walleij <linus.walleij@linaro.org> | ||
| 6255 | S: Maintained | ||
| 6256 | T: git git://anongit.freedesktop.org/drm/drm-misc | ||
| 6257 | F: drivers/gpu/drm/panel/panel-sony-acx424akp.c | ||
| 6258 | |||
| 6259 | DRM DRIVER FOR ST-ERICSSON MCDE | 6273 | DRM DRIVER FOR ST-ERICSSON MCDE |
| 6260 | M: Linus Walleij <linus.walleij@linaro.org> | 6274 | M: Linus Walleij <linus.walleij@linaro.org> |
| 6261 | S: Maintained | 6275 | S: Maintained |
| @@ -6503,7 +6517,7 @@ F: Documentation/devicetree/bindings/display/rockchip/ · 241 unmodified lines | |||
| 6503 | F: drivers/gpu/drm/rockchip/ | 6517 | F: drivers/gpu/drm/rockchip/ |
| 6504 | 6518 | ||
| 6505 | DRM DRIVERS FOR STI | 6519 | DRM DRIVERS FOR STI |
| 6506 | M: Benjamin Gaignard <benjamin.gaignard@linaro.org> | 6520 | M: Alain Volmat <alain.volmat@foss.st.com> |
| 6507 | L: dri-devel@lists.freedesktop.org | 6521 | L: dri-devel@lists.freedesktop.org |
| 6508 | S: Maintained | 6522 | S: Maintained |
| 6509 | T: git git://anongit.freedesktop.org/drm/drm-misc | 6523 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| @@ -6512,8 +6526,8 @@ F: drivers/gpu/drm/sti · 2 unmodified lines | |||
| 6512 | 6526 | ||
| 6513 | DRM DRIVERS FOR STM | 6527 | DRM DRIVERS FOR STM |
| 6514 | M: Yannick Fertre <yannick.fertre@foss.st.com> | 6528 | M: Yannick Fertre <yannick.fertre@foss.st.com> |
| 6529 | M: Raphael Gallais-Pou <raphael.gallais-pou@foss.st.com> | ||
| 6515 | M: Philippe Cornu <philippe.cornu@foss.st.com> | 6530 | M: Philippe Cornu <philippe.cornu@foss.st.com> |
| 6516 | M: Benjamin Gaignard <benjamin.gaignard@linaro.org> | ||
| 6517 | L: dri-devel@lists.freedesktop.org | 6531 | L: dri-devel@lists.freedesktop.org |
| 6518 | S: Maintained | 6532 | S: Maintained |
| 6519 | T: git git://anongit.freedesktop.org/drm/drm-misc | 6533 | T: git git://anongit.freedesktop.org/drm/drm-misc |
| @@ -7007,12 +7021,6 @@ L: linux-edac@vger.kernel.org · 487 unmodified lines | |||
| 7007 | S: Maintained | 7021 | S: Maintained |
| 7008 | F: drivers/edac/sb_edac.c | 7022 | F: drivers/edac/sb_edac.c |
| 7009 | 7023 | ||
| 7010 | EDAC-SIFIVE | ||
| 7011 | M: Yash Shah <yash.shah@sifive.com> | ||
| 7012 | L: linux-edac@vger.kernel.org | ||
| 7013 | S: Supported | ||
| 7014 | F: drivers/edac/sifive_edac.c | ||
| 7015 | |||
| 7016 | EDAC-SKYLAKE | 7024 | EDAC-SKYLAKE |
| 7017 | M: Tony Luck <tony.luck@intel.com> | 7025 | M: Tony Luck <tony.luck@intel.com> |
| 7018 | L: linux-edac@vger.kernel.org | 7026 | L: linux-edac@vger.kernel.org |
| @@ -7183,7 +7191,7 @@ F: drivers/net/can/usb/etas_es58x/ · 164 unmodified lines | |||
| 7183 | 7191 | ||
| 7184 | ETHERNET BRIDGE | 7192 | ETHERNET BRIDGE |
| 7185 | M: Roopa Prabhu <roopa@nvidia.com> | 7193 | M: Roopa Prabhu <roopa@nvidia.com> |
| 7186 | M: Nikolay Aleksandrov <nikolay@nvidia.com> | 7194 | M: Nikolay Aleksandrov <razor@blackwall.org> |
| 7187 | L: bridge@lists.linux-foundation.org (moderated for non-subscribers) | 7195 | L: bridge@lists.linux-foundation.org (moderated for non-subscribers) |
| 7188 | L: netdev@vger.kernel.org | 7196 | L: netdev@vger.kernel.org |
| 7189 | S: Maintained | 7197 | S: Maintained |
| @@ -7576,6 +7584,12 @@ S: Maintained · 386 unmodified lines | |||
| 7576 | W: http://floatingpoint.sourceforge.net/emulator/index.html | 7584 | W: http://floatingpoint.sourceforge.net/emulator/index.html |
| 7577 | F: arch/x86/math-emu/ | 7585 | F: arch/x86/math-emu/ |
| 7578 | 7586 | ||
| 7587 | FRAMEBUFFER CORE | ||
| 7588 | M: Daniel Vetter <daniel@ffwll.ch> | ||
| 7589 | F: drivers/video/fbdev/core/ | ||
| 7590 | S: Odd Fixes | ||
| 7591 | T: git git://anongit.freedesktop.org/drm/drm-misc | ||
| 7592 | |||
| 7579 | FRAMEBUFFER LAYER | 7593 | FRAMEBUFFER LAYER |
| 7580 | M: Helge Deller <deller@gmx.de> | 7594 | M: Helge Deller <deller@gmx.de> |
| 7581 | L: linux-fbdev@vger.kernel.org | 7595 | L: linux-fbdev@vger.kernel.org |
| @@ -9253,6 +9267,15 @@ S: Maintained · 1671 unmodified lines | |||
| 9253 | W: https://github.com/o2genum/ideapad-slidebar | 9267 | W: https://github.com/o2genum/ideapad-slidebar |
| 9254 | F: drivers/input/misc/ideapad_slidebar.c | 9268 | F: drivers/input/misc/ideapad_slidebar.c |
| 9255 | 9269 | ||
| 9270 | IDMAPPED MOUNTS | ||
| 9271 | M: Christian Brauner <brauner@kernel.org> | ||
| 9272 | L: linux-fsdevel@vger.kernel.org | ||
| 9273 | S: Maintained | ||
| 9274 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux.git | ||
| 9275 | F: Documentation/filesystems/idmappings.rst | ||
| 9276 | F: tools/testing/selftests/mount_setattr/ | ||
| 9277 | F: include/linux/mnt_idmapping.h | ||
| 9278 | |||
| 9256 | IDT VersaClock 5 CLOCK DRIVER | 9279 | IDT VersaClock 5 CLOCK DRIVER |
| 9257 | M: Luca Ceresoli <luca@lucaceresoli.net> | 9280 | M: Luca Ceresoli <luca@lucaceresoli.net> |
| 9258 | S: Maintained | 9281 | S: Maintained |
| @@ -10887,6 +10910,12 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git · 1628 unmodified lines | |||
| 10887 | F: drivers/ata/pata_arasan_cf.c | 10910 | F: drivers/ata/pata_arasan_cf.c |
| 10888 | F: include/linux/pata_arasan_cf_data.h | 10911 | F: include/linux/pata_arasan_cf_data.h |
| 10889 | 10912 | ||
| 10913 | LIBATA PATA DRIVERS | ||
| 10914 | R: Sergey Shtylyov <s.shtylyov@omp.ru> | ||
| 10915 | L: linux-ide@vger.kernel.org | ||
| 10916 | F: drivers/ata/ata_*.c | ||
| 10917 | F: drivers/ata/pata_*.c | ||
| 10918 | |||
| 10890 | LIBATA PATA FARADAY FTIDE010 AND GEMINI SATA BRIDGE DRIVERS | 10919 | LIBATA PATA FARADAY FTIDE010 AND GEMINI SATA BRIDGE DRIVERS |
| 10891 | M: Linus Walleij <linus.walleij@linaro.org> | 10920 | M: Linus Walleij <linus.walleij@linaro.org> |
| 10892 | L: linux-ide@vger.kernel.org | 10921 | L: linux-ide@vger.kernel.org |
| @@ -12407,7 +12436,7 @@ F: include/uapi/linux/membarrier.h · 1514 unmodified lines | |||
| 12407 | F: kernel/sched/membarrier.c | 12436 | F: kernel/sched/membarrier.c |
| 12408 | 12437 | ||
| 12409 | MEMBLOCK | 12438 | MEMBLOCK |
| 12410 | M: Mike Rapoport <rppt@linux.ibm.com> | 12439 | M: Mike Rapoport <rppt@kernel.org> |
| 12411 | L: linux-mm@kvack.org | 12440 | L: linux-mm@kvack.org |
| 12412 | S: Maintained | 12441 | S: Maintained |
| 12413 | F: Documentation/core-api/boot-time-mm.rst | 12442 | F: Documentation/core-api/boot-time-mm.rst |
| @@ -13305,8 +13334,8 @@ W: http://www.iptables.org/ · 891 unmodified lines | |||
| 13305 | W: http://www.nftables.org/ | 13334 | W: http://www.nftables.org/ |
| 13306 | Q: http://patchwork.ozlabs.org/project/netfilter-devel/list/ | 13335 | Q: http://patchwork.ozlabs.org/project/netfilter-devel/list/ |
| 13307 | C: irc://irc.libera.chat/netfilter | 13336 | C: irc://irc.libera.chat/netfilter |
| 13308 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf.git | 13337 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf.git |
| 13309 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next.git | 13338 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/netfilter/nf-next.git |
| 13310 | F: include/linux/netfilter* | 13339 | F: include/linux/netfilter* |
| 13311 | F: include/linux/netfilter/ | 13340 | F: include/linux/netfilter/ |
| 13312 | F: include/net/netfilter/ | 13341 | F: include/net/netfilter/ |
| @@ -13573,7 +13602,7 @@ F: tools/testing/selftests/nci/ · 260 unmodified lines | |||
| 13573 | 13602 | ||
| 13574 | NFS, SUNRPC, AND LOCKD CLIENTS | 13603 | NFS, SUNRPC, AND LOCKD CLIENTS |
| 13575 | M: Trond Myklebust <trond.myklebust@hammerspace.com> | 13604 | M: Trond Myklebust <trond.myklebust@hammerspace.com> |
| 13576 | M: Anna Schumaker <anna.schumaker@netapp.com> | 13605 | M: Anna Schumaker <anna@kernel.org> |
| 13577 | L: linux-nfs@vger.kernel.org | 13606 | L: linux-nfs@vger.kernel.org |
| 13578 | S: Maintained | 13607 | S: Maintained |
| 13579 | W: http://client.linux-nfs.org | 13608 | W: http://client.linux-nfs.org |
| @@ -14391,6 +14420,7 @@ M: Rob Herring <robh+dt@kernel.org> · 811 unmodified lines | |||
| 14391 | M: Frank Rowand <frowand.list@gmail.com> | 14420 | M: Frank Rowand <frowand.list@gmail.com> |
| 14392 | L: devicetree@vger.kernel.org | 14421 | L: devicetree@vger.kernel.org |
| 14393 | S: Maintained | 14422 | S: Maintained |
| 14423 | C: irc://irc.libera.chat/devicetree | ||
| 14394 | W: http://www.devicetree.org/ | 14424 | W: http://www.devicetree.org/ |
| 14395 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git | 14425 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git |
| 14396 | F: Documentation/ABI/testing/sysfs-firmware-ofw | 14426 | F: Documentation/ABI/testing/sysfs-firmware-ofw |
| @@ -14402,6 +14432,7 @@ OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS · 5 unmodified lines | |||
| 14402 | M: Rob Herring <robh+dt@kernel.org> | 14432 | M: Rob Herring <robh+dt@kernel.org> |
| 14403 | L: devicetree@vger.kernel.org | 14433 | L: devicetree@vger.kernel.org |
| 14404 | S: Maintained | 14434 | S: Maintained |
| 14435 | C: irc://irc.libera.chat/devicetree | ||
| 14405 | Q: http://patchwork.ozlabs.org/project/devicetree-bindings/list/ | 14436 | Q: http://patchwork.ozlabs.org/project/devicetree-bindings/list/ |
| 14406 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git | 14437 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git |
| 14407 | F: Documentation/devicetree/ | 14438 | F: Documentation/devicetree/ |
| @@ -15135,7 +15166,7 @@ M: Ingo Molnar <mingo@redhat.com> · 727 unmodified lines | |||
| 15135 | M: Arnaldo Carvalho de Melo <acme@kernel.org> | 15166 | M: Arnaldo Carvalho de Melo <acme@kernel.org> |
| 15136 | R: Mark Rutland <mark.rutland@arm.com> | 15167 | R: Mark Rutland <mark.rutland@arm.com> |
| 15137 | R: Alexander Shishkin <alexander.shishkin@linux.intel.com> | 15168 | R: Alexander Shishkin <alexander.shishkin@linux.intel.com> |
| 15138 | R: Jiri Olsa <jolsa@redhat.com> | 15169 | R: Jiri Olsa <jolsa@kernel.org> |
| 15139 | R: Namhyung Kim <namhyung@kernel.org> | 15170 | R: Namhyung Kim <namhyung@kernel.org> |
| 15140 | L: linux-perf-users@vger.kernel.org | 15171 | L: linux-perf-users@vger.kernel.org |
| 15141 | L: linux-kernel@vger.kernel.org | 15172 | L: linux-kernel@vger.kernel.org |
| @@ -15292,9 +15323,11 @@ PIN CONTROLLER - SAMSUNG · 150 unmodified lines | |||
| 15292 | M: Tomasz Figa <tomasz.figa@gmail.com> | 15323 | M: Tomasz Figa <tomasz.figa@gmail.com> |
| 15293 | M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com> | 15324 | M: Krzysztof Kozlowski <krzysztof.kozlowski@canonical.com> |
| 15294 | M: Sylwester Nawrocki <s.nawrocki@samsung.com> | 15325 | M: Sylwester Nawrocki <s.nawrocki@samsung.com> |
| 15326 | R: Alim Akhtar <alim.akhtar@samsung.com> | ||
| 15295 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) | 15327 | L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) |
| 15296 | L: linux-samsung-soc@vger.kernel.org | 15328 | L: linux-samsung-soc@vger.kernel.org |
| 15297 | S: Maintained | 15329 | S: Maintained |
| 15330 | C: irc://irc.libera.chat/linux-exynos | ||
| 15298 | Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/ | 15331 | Q: https://patchwork.kernel.org/project/linux-samsung-soc/list/ |
| 15299 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/samsung.git | 15332 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/pinctrl/samsung.git |
| 15300 | F: Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt | 15333 | F: Documentation/devicetree/bindings/pinctrl/samsung-pinctrl.txt |
| @@ -15551,6 +15584,7 @@ M: Iurii Zaikin <yzaikin@google.com> · 250 unmodified lines | |||
| 15551 | L: linux-kernel@vger.kernel.org | 15584 | L: linux-kernel@vger.kernel.org |
| 15552 | L: linux-fsdevel@vger.kernel.org | 15585 | L: linux-fsdevel@vger.kernel.org |
| 15553 | S: Maintained | 15586 | S: Maintained |
| 15587 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git sysctl-next | ||
| 15554 | F: fs/proc/proc_sysctl.c | 15588 | F: fs/proc/proc_sysctl.c |
| 15555 | F: include/linux/sysctl.h | 15589 | F: include/linux/sysctl.h |
| 15556 | F: kernel/sysctl-test.c | 15590 | F: kernel/sysctl-test.c |
| @@ -15898,6 +15932,7 @@ S: Supported · 341 unmodified lines | |||
| 15898 | W: https://wireless.wiki.kernel.org/en/users/Drivers/ath10k | 15932 | W: https://wireless.wiki.kernel.org/en/users/Drivers/ath10k |
| 15899 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git | 15933 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git |
| 15900 | F: drivers/net/wireless/ath/ath10k/ | 15934 | F: drivers/net/wireless/ath/ath10k/ |
| 15935 | F: Documentation/devicetree/bindings/net/wireless/qcom,ath10k.txt | ||
| 15901 | 15936 | ||
| 15902 | QUALCOMM ATHEROS ATH11K WIRELESS DRIVER | 15937 | QUALCOMM ATHEROS ATH11K WIRELESS DRIVER |
| 15903 | M: Kalle Valo <kvalo@kernel.org> | 15938 | M: Kalle Valo <kvalo@kernel.org> |
| @@ -15905,11 +15940,12 @@ L: ath11k@lists.infradead.org · 1 unmodified line | |||
| 15905 | S: Supported | 15940 | S: Supported |
| 15906 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git | 15941 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/ath.git |
| 15907 | F: drivers/net/wireless/ath/ath11k/ | 15942 | F: drivers/net/wireless/ath/ath11k/ |
| 15943 | F: Documentation/devicetree/bindings/net/wireless/qcom,ath11k.txt | ||
| 15908 | 15944 | ||
| 15909 | QUALCOMM ATHEROS ATH9K WIRELESS DRIVER | 15945 | QUALCOMM ATHEROS ATH9K WIRELESS DRIVER |
| 15910 | M: ath9k-devel@qca.qualcomm.com | 15946 | M: Toke Høiland-Jørgensen <toke@toke.dk> |
| 15911 | L: linux-wireless@vger.kernel.org | 15947 | L: linux-wireless@vger.kernel.org |
| 15912 | S: Supported | 15948 | S: Maintained |
| 15913 | W: https://wireless.wiki.kernel.org/en/users/Drivers/ath9k | 15949 | W: https://wireless.wiki.kernel.org/en/users/Drivers/ath9k |
| 15914 | F: Documentation/devicetree/bindings/net/wireless/qca,ath9k.yaml | 15950 | F: Documentation/devicetree/bindings/net/wireless/qca,ath9k.yaml |
| 15915 | F: drivers/net/wireless/ath/ath9k/ | 15951 | F: drivers/net/wireless/ath/ath9k/ |
| @@ -15984,14 +16020,6 @@ F: Documentation/devicetree/bindings/misc/qcom,fastrpc.txt · 68 unmodified lines | |||
| 15984 | F: drivers/misc/fastrpc.c | 16020 | F: drivers/misc/fastrpc.c |
| 15985 | F: include/uapi/misc/fastrpc.h | 16021 | F: include/uapi/misc/fastrpc.h |
| 15986 | 16022 | ||
| 15987 | QUALCOMM GENERIC INTERFACE I2C DRIVER | ||
| 15988 | M: Akash Asthana <akashast@codeaurora.org> | ||
| 15989 | M: Mukesh Savaliya <msavaliy@codeaurora.org> | ||
| 15990 | L: linux-i2c@vger.kernel.org | ||
| 15991 | L: linux-arm-msm@vger.kernel.org | ||
| 15992 | S: Supported | ||
| 15993 | F: drivers/i2c/busses/i2c-qcom-geni.c | ||
| 15994 | |||
| 15995 | QUALCOMM HEXAGON ARCHITECTURE | 16023 | QUALCOMM HEXAGON ARCHITECTURE |
| 15996 | M: Brian Cain <bcain@codeaurora.org> | 16024 | M: Brian Cain <bcain@codeaurora.org> |
| 15997 | L: linux-hexagon@vger.kernel.org | 16025 | L: linux-hexagon@vger.kernel.org |
| @@ -16063,8 +16091,8 @@ F: Documentation/devicetree/bindings/mtd/qcom,nandc.yaml · 65 unmodified lines | |||
| 16063 | F: drivers/mtd/nand/raw/qcom_nandc.c | 16091 | F: drivers/mtd/nand/raw/qcom_nandc.c |
| 16064 | 16092 | ||
| 16065 | QUALCOMM RMNET DRIVER | 16093 | QUALCOMM RMNET DRIVER |
| 16066 | M: Subash Abhinov Kasiviswanathan <subashab@codeaurora.org> | 16094 | M: Subash Abhinov Kasiviswanathan <quic_subashab@quicinc.com> |
| 16067 | M: Sean Tranchetti <stranche@codeaurora.org> | 16095 | M: Sean Tranchetti <quic_stranche@quicinc.com> |
| 16068 | L: netdev@vger.kernel.org | 16096 | L: netdev@vger.kernel.org |
| 16069 | S: Maintained | 16097 | S: Maintained |
| 16070 | F: Documentation/networking/device_drivers/cellular/qualcomm/rmnet.rst | 16098 | F: Documentation/networking/device_drivers/cellular/qualcomm/rmnet.rst |
| @@ -16090,11 +16118,10 @@ F: Documentation/devicetree/bindings/media/*venus* · 19 unmodified lines | |||
| 16090 | F: drivers/media/platform/qcom/venus/ | 16118 | F: drivers/media/platform/qcom/venus/ |
| 16091 | 16119 | ||
| 16092 | QUALCOMM WCN36XX WIRELESS DRIVER | 16120 | QUALCOMM WCN36XX WIRELESS DRIVER |
| 16093 | M: Kalle Valo <kvalo@kernel.org> | 16121 | M: Loic Poulain <loic.poulain@linaro.org> |
| 16094 | L: wcn36xx@lists.infradead.org | 16122 | L: wcn36xx@lists.infradead.org |
| 16095 | S: Supported | 16123 | S: Supported |
| 16096 | W: https://wireless.wiki.kernel.org/en/users/Drivers/wcn36xx | 16124 | W: https://wireless.wiki.kernel.org/en/users/Drivers/wcn36xx |
| 16097 | T: git git://github.com/KrasnikovEugene/wcn36xx.git | ||
| 16098 | F: drivers/net/wireless/ath/wcn36xx/ | 16125 | F: drivers/net/wireless/ath/wcn36xx/ |
| 16099 | 16126 | ||
| 16100 | QUANTENNA QTNFMAC WIRELESS DRIVER | 16127 | QUANTENNA QTNFMAC WIRELESS DRIVER |
| @@ -16357,6 +16384,7 @@ F: drivers/watchdog/realtek_otto_wdt.c · 256 unmodified lines | |||
| 16357 | 16384 | ||
| 16358 | REALTEK RTL83xx SMI DSA ROUTER CHIPS | 16385 | REALTEK RTL83xx SMI DSA ROUTER CHIPS |
| 16359 | M: Linus Walleij <linus.walleij@linaro.org> | 16386 | M: Linus Walleij <linus.walleij@linaro.org> |
| 16387 | M: Alvin Šipraga <alsi@bang-olufsen.dk> | ||
| 16360 | S: Maintained | 16388 | S: Maintained |
| 16361 | F: Documentation/devicetree/bindings/net/dsa/realtek-smi.txt | 16389 | F: Documentation/devicetree/bindings/net/dsa/realtek-smi.txt |
| 16362 | F: drivers/net/dsa/realtek-smi* | 16390 | F: drivers/net/dsa/realtek-smi* |
| @@ -16476,6 +16504,14 @@ F: Documentation/devicetree/bindings/i2c/renesas,rmobile-iic.yaml · 113 unmodified lines | |||
| 16476 | F: drivers/i2c/busses/i2c-rcar.c | 16504 | F: drivers/i2c/busses/i2c-rcar.c |
| 16477 | F: drivers/i2c/busses/i2c-sh_mobile.c | 16505 | F: drivers/i2c/busses/i2c-sh_mobile.c |
| 16478 | 16506 | ||
| 16507 | RENESAS R-CAR SATA DRIVER | ||
| 16508 | R: Sergey Shtylyov <s.shtylyov@omp.ru> | ||
| 16509 | S: Supported | ||
| 16510 | L: linux-ide@vger.kernel.org | ||
| 16511 | L: linux-renesas-soc@vger.kernel.org | ||
| 16512 | F: Documentation/devicetree/bindings/ata/renesas,rcar-sata.yaml | ||
| 16513 | F: drivers/ata/sata_rcar.c | ||
| 16514 | |||
| 16479 | RENESAS R-CAR THERMAL DRIVERS | 16515 | RENESAS R-CAR THERMAL DRIVERS |
| 16480 | M: Niklas Söderlund <niklas.soderlund@ragnatech.se> | 16516 | M: Niklas Söderlund <niklas.soderlund@ragnatech.se> |
| 16481 | L: linux-renesas-soc@vger.kernel.org | 16517 | L: linux-renesas-soc@vger.kernel.org |
| @@ -16811,8 +16847,8 @@ F: drivers/video/fbdev/savage/ · 329 unmodified lines | |||
| 16811 | S390 | 16847 | S390 |
| 16812 | M: Heiko Carstens <hca@linux.ibm.com> | 16848 | M: Heiko Carstens <hca@linux.ibm.com> |
| 16813 | M: Vasily Gorbik <gor@linux.ibm.com> | 16849 | M: Vasily Gorbik <gor@linux.ibm.com> |
| 16814 | M: Christian Borntraeger <borntraeger@linux.ibm.com> | 16850 | M: Alexander Gordeev <agordeev@linux.ibm.com> |
| 16815 | R: Alexander Gordeev <agordeev@linux.ibm.com> | 16851 | R: Christian Borntraeger <borntraeger@linux.ibm.com> |
| 16816 | R: Sven Schnelle <svens@linux.ibm.com> | 16852 | R: Sven Schnelle <svens@linux.ibm.com> |
| 16817 | L: linux-s390@vger.kernel.org | 16853 | L: linux-s390@vger.kernel.org |
| 16818 | S: Supported | 16854 | S: Supported |
| @@ -17083,6 +17119,7 @@ SAMSUNG SOC CLOCK DRIVERS · 264 unmodified lines | |||
| 17083 | M: Sylwester Nawrocki <s.nawrocki@samsung.com> | 17119 | M: Sylwester Nawrocki <s.nawrocki@samsung.com> |
| 17084 | M: Tomasz Figa <tomasz.figa@gmail.com> | 17120 | M: Tomasz Figa <tomasz.figa@gmail.com> |
| 17085 | M: Chanwoo Choi <cw00.choi@samsung.com> | 17121 | M: Chanwoo Choi <cw00.choi@samsung.com> |
| 17122 | R: Alim Akhtar <alim.akhtar@samsung.com> | ||
| 17086 | L: linux-samsung-soc@vger.kernel.org | 17123 | L: linux-samsung-soc@vger.kernel.org |
| 17087 | S: Supported | 17124 | S: Supported |
| 17088 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/snawrocki/clk.git | 17125 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/snawrocki/clk.git |
| @@ -17719,6 +17756,21 @@ S: Maintained · 630 unmodified lines | |||
| 17719 | W: http://www.winischhofer.at/linuxsisusbvga.shtml | 17756 | W: http://www.winischhofer.at/linuxsisusbvga.shtml |
| 17720 | F: drivers/usb/misc/sisusbvga/ | 17757 | F: drivers/usb/misc/sisusbvga/ |
| 17721 | 17758 | ||
| 17759 | SL28 CPLD MFD DRIVER | ||
| 17760 | M: Michael Walle <michael@walle.cc> | ||
| 17761 | S: Maintained | ||
| 17762 | F: Documentation/devicetree/bindings/gpio/kontron,sl28cpld-gpio.yaml | ||
| 17763 | F: Documentation/devicetree/bindings/hwmon/kontron,sl28cpld-hwmon.yaml | ||
| 17764 | F: Documentation/devicetree/bindings/interrupt-controller/kontron,sl28cpld-intc.yaml | ||
| 17765 | F: Documentation/devicetree/bindings/mfd/kontron,sl28cpld.yaml | ||
| 17766 | F: Documentation/devicetree/bindings/pwm/kontron,sl28cpld-pwm.yaml | ||
| 17767 | F: Documentation/devicetree/bindings/watchdog/kontron,sl28cpld-wdt.yaml | ||
| 17768 | F: drivers/gpio/gpio-sl28cpld.c | ||
| 17769 | F: drivers/hwmon/sl28cpld-hwmon.c | ||
| 17770 | F: drivers/irqchip/irq-sl28cpld.c | ||
| 17771 | F: drivers/pwm/pwm-sl28cpld.c | ||
| 17772 | F: drivers/watchdog/sl28cpld_wdt.c | ||
| 17773 | |||
| 17722 | SLAB ALLOCATOR | 17774 | SLAB ALLOCATOR |
| 17723 | M: Christoph Lameter <cl@linux.com> | 17775 | M: Christoph Lameter <cl@linux.com> |
| 17724 | M: Pekka Enberg <penberg@kernel.org> | 17776 | M: Pekka Enberg <penberg@kernel.org> |
| @@ -17726,8 +17778,10 @@ M: David Rientjes <rientjes@google.com> · 1 unmodified line | |||
| 17726 | M: Joonsoo Kim <iamjoonsoo.kim@lge.com> | 17778 | M: Joonsoo Kim <iamjoonsoo.kim@lge.com> |
| 17727 | M: Andrew Morton <akpm@linux-foundation.org> | 17779 | M: Andrew Morton <akpm@linux-foundation.org> |
| 17728 | M: Vlastimil Babka <vbabka@suse.cz> | 17780 | M: Vlastimil Babka <vbabka@suse.cz> |
| 17781 | R: Roman Gushchin <roman.gushchin@linux.dev> | ||
| 17729 | L: linux-mm@kvack.org | 17782 | L: linux-mm@kvack.org |
| 17730 | S: Maintained | 17783 | S: Maintained |
| 17784 | T: git git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git | ||
| 17731 | F: include/linux/sl?b*.h | 17785 | F: include/linux/sl?b*.h |
| 17732 | F: mm/sl?b* | 17786 | F: mm/sl?b* |
| 17733 | 17787 | ||
| @@ -18435,7 +18489,7 @@ F: Documentation/devicetree/bindings/sound/st,sti-asoc-card.txt · 701 unmodified lines | |||
| 18435 | F: sound/soc/sti/ | 18489 | F: sound/soc/sti/ |
| 18436 | 18490 | ||
| 18437 | STI CEC DRIVER | 18491 | STI CEC DRIVER |
| 18438 | M: Benjamin Gaignard <benjamin.gaignard@linaro.org> | 18492 | M: Alain Volmat <alain.volmat@foss.st.com> |
| 18439 | S: Maintained | 18493 | S: Maintained |
| 18440 | F: Documentation/devicetree/bindings/media/stih-cec.txt | 18494 | F: Documentation/devicetree/bindings/media/stih-cec.txt |
| 18441 | F: drivers/media/cec/platform/sti/ | 18495 | F: drivers/media/cec/platform/sti/ |
| @@ -19589,6 +19643,14 @@ F: Documentation/trace/timerlat-tracer.rst · 1147 unmodified lines | |||
| 19589 | F: Documentation/trace/hwlat_detector.rst | 19643 | F: Documentation/trace/hwlat_detector.rst |
| 19590 | F: arch/*/kernel/trace.c | 19644 | F: arch/*/kernel/trace.c |
| 19591 | 19645 | ||
| 19646 | Real-time Linux Analysis (RTLA) tools | ||
| 19647 | M: Daniel Bristot de Oliveira <bristot@kernel.org> | ||
| 19648 | M: Steven Rostedt <rostedt@goodmis.org> | ||
| 19649 | L: linux-trace-devel@vger.kernel.org | ||
| 19650 | S: Maintained | ||
| 19651 | F: Documentation/tools/rtla/ | ||
| 19652 | F: tools/tracing/rtla/ | ||
| 19653 | |||
| 19592 | TRADITIONAL CHINESE DOCUMENTATION | 19654 | TRADITIONAL CHINESE DOCUMENTATION |
| 19593 | M: Hu Haowen <src.res@email.cn> | 19655 | M: Hu Haowen <src.res@email.cn> |
| 19594 | L: linux-doc-tw-discuss@lists.sourceforge.net | 19656 | L: linux-doc-tw-discuss@lists.sourceforge.net |
| @@ -2,8 +2,8 @@ · 1 unmodified line | |||
| 2 | VERSION = 5 | 2 | VERSION = 5 |
| 3 | PATCHLEVEL = 17 | 3 | PATCHLEVEL = 17 |
| 4 | SUBLEVEL = 0 | 4 | SUBLEVEL = 0 |
| 5 | EXTRAVERSION = -rc2 | 5 | EXTRAVERSION = -rc6 |
| 6 | NAME = Gobble Gobble | 6 | NAME = Superb Owl |
| 7 | 7 | ||
| 8 | # *DOCUMENTATION* | 8 | # *DOCUMENTATION* |
| 9 | # To see a list of typical targets execute "make help" | 9 | # To see a list of typical targets execute "make help" |
| @@ -806,6 +806,7 @@ dtb-$(CONFIG_ARCH_OMAP3) += \ · 805 unmodified lines | |||
| 806 | logicpd-som-lv-37xx-devkit.dtb \ | 806 | logicpd-som-lv-37xx-devkit.dtb \ |
| 807 | omap3430-sdp.dtb \ | 807 | omap3430-sdp.dtb \ |
| 808 | omap3-beagle.dtb \ | 808 | omap3-beagle.dtb \ |
| 809 | omap3-beagle-ab4.dtb \ | ||
| 809 | omap3-beagle-xm.dtb \ | 810 | omap3-beagle-xm.dtb \ |
| 810 | omap3-beagle-xm-ab.dtb \ | 811 | omap3-beagle-xm-ab.dtb \ |
| 811 | omap3-cm-t3517.dtb \ | 812 | omap3-cm-t3517.dtb \ |
| @@ -55,7 +55,7 @@ · 54 unmodified lines | |||
| 55 | 2 1 0 0 /* # 0: INACTIVE, 1: TX, 2: RX */ | 55 | 2 1 0 0 /* # 0: INACTIVE, 1: TX, 2: RX */ |
| 56 | >; | 56 | >; |
| 57 | tx-num-evt = <16>; | 57 | tx-num-evt = <16>; |
| 58 | rt-num-evt = <16>; | 58 | rx-num-evt = <16>; |
| 59 | status = "okay"; | 59 | status = "okay"; |
| 60 | }; | 60 | }; |
| 61 | 61 | ||
| @@ -160,7 +160,7 @@ · 159 unmodified lines | |||
| 160 | target-module@48210000 { | 160 | target-module@48210000 { |
| 161 | compatible = "ti,sysc-omap4-simple", "ti,sysc"; | 161 | compatible = "ti,sysc-omap4-simple", "ti,sysc"; |
| 162 | power-domains = <&prm_mpu>; | 162 | power-domains = <&prm_mpu>; |
| 163 | clocks = <&mpu_clkctrl DRA7_MPU_CLKCTRL 0>; | 163 | clocks = <&mpu_clkctrl DRA7_MPU_MPU_CLKCTRL 0>; |
| 164 | clock-names = "fck"; | 164 | clock-names = "fck"; |
| 165 | #address-cells = <1>; | 165 | #address-cells = <1>; |
| 166 | #size-cells = <1>; | 166 | #size-cells = <1>; |
| @@ -875,10 +875,10 @@ · 708 unmodified lines | |||
| 875 | <0x58000014 4>; | 875 | <0x58000014 4>; |
| 876 | reg-names = "rev", "syss"; | 876 | reg-names = "rev", "syss"; |
| 877 | ti,syss-mask = <1>; | 877 | ti,syss-mask = <1>; |
| 878 | clocks = <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 0>, | 878 | clocks = <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 0>, |
| 879 | <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 9>, | 879 | <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 9>, |
| 880 | <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 10>, | 880 | <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 10>, |
| 881 | <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 11>; | 881 | <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 11>; |
| 882 | clock-names = "fck", "hdmi_clk", "sys_clk", "tv_clk"; | 882 | clock-names = "fck", "hdmi_clk", "sys_clk", "tv_clk"; |
| 883 | #address-cells = <1>; | 883 | #address-cells = <1>; |
| 884 | #size-cells = <1>; | 884 | #size-cells = <1>; |
| @@ -912,7 +912,7 @@ · 27 unmodified lines | |||
| 912 | SYSC_OMAP2_SOFTRESET | | 912 | SYSC_OMAP2_SOFTRESET | |
| 913 | SYSC_OMAP2_AUTOIDLE)>; | 913 | SYSC_OMAP2_AUTOIDLE)>; |
| 914 | ti,syss-mask = <1>; | 914 | ti,syss-mask = <1>; |
| 915 | clocks = <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 8>; | 915 | clocks = <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 8>; |
| 916 | clock-names = "fck"; | 916 | clock-names = "fck"; |
| 917 | #address-cells = <1>; | 917 | #address-cells = <1>; |
| 918 | #size-cells = <1>; | 918 | #size-cells = <1>; |
| @@ -939,8 +939,8 @@ · 20 unmodified lines | |||
| 939 | <SYSC_IDLE_SMART>, | 939 | <SYSC_IDLE_SMART>, |
| 940 | <SYSC_IDLE_SMART_WKUP>; | 940 | <SYSC_IDLE_SMART_WKUP>; |
| 941 | ti,sysc-mask = <(SYSC_OMAP4_SOFTRESET)>; | 941 | ti,sysc-mask = <(SYSC_OMAP4_SOFTRESET)>; |
| 942 | clocks = <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 9>, | 942 | clocks = <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 9>, |
| 943 | <&dss_clkctrl DRA7_DSS_CORE_CLKCTRL 8>; | 943 | <&dss_clkctrl DRA7_DSS_DSS_CORE_CLKCTRL 8>; |
| 944 | clock-names = "fck", "dss_clk"; | 944 | clock-names = "fck", "dss_clk"; |
| 945 | #address-cells = <1>; | 945 | #address-cells = <1>; |
| 946 | #size-cells = <1>; | 946 | #size-cells = <1>; |
| @@ -979,7 +979,7 @@ · 32 unmodified lines | |||
| 979 | compatible = "vivante,gc"; | 979 | compatible = "vivante,gc"; |
| 980 | reg = <0x0 0x700>; | 980 | reg = <0x0 0x700>; |
| 981 | interrupts = <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>; | 981 | interrupts = <GIC_SPI 120 IRQ_TYPE_LEVEL_HIGH>; |
| 982 | clocks = <&dss_clkctrl DRA7_BB2D_CLKCTRL 0>; | 982 | clocks = <&dss_clkctrl DRA7_DSS_BB2D_CLKCTRL 0>; |
| 983 | clock-names = "core"; | 983 | clock-names = "core"; |
| 984 | }; | 984 | }; |
| 985 | }; | 985 | }; |
| @@ -1333,7 +1333,7 @@ · 347 unmodified lines | |||
| 1333 | ti,no-reset-on-init; | 1333 | ti,no-reset-on-init; |
| 1334 | ti,no-idle; | 1334 | ti,no-idle; |
| 1335 | timer@0 { | 1335 | timer@0 { |
| 1336 | assigned-clocks = <&wkupaon_clkctrl DRA7_TIMER1_CLKCTRL 24>; | 1336 | assigned-clocks = <&wkupaon_clkctrl DRA7_WKUPAON_TIMER1_CLKCTRL 24>; |
| 1337 | assigned-clock-parents = <&sys_32k_ck>; | 1337 | assigned-clock-parents = <&sys_32k_ck>; |
| 1338 | }; | 1338 | }; |
| 1339 | }; | 1339 | }; |
| @@ -79,7 +79,6 @@ · 78 unmodified lines | |||
| 79 | MX23_PAD_LCD_RESET__GPIO_1_18 | 79 | MX23_PAD_LCD_RESET__GPIO_1_18 |
| 80 | MX23_PAD_PWM3__GPIO_1_29 | 80 | MX23_PAD_PWM3__GPIO_1_29 |
| 81 | MX23_PAD_PWM4__GPIO_1_30 | 81 | MX23_PAD_PWM4__GPIO_1_30 |
| 82 | MX23_PAD_SSP1_DETECT__SSP1_DETECT | ||
| 83 | >; | 82 | >; |
| 84 | fsl,drive-strength = <MXS_DRIVE_4mA>; | 83 | fsl,drive-strength = <MXS_DRIVE_4mA>; |
| 85 | fsl,voltage = <MXS_VOLTAGE_HIGH>; | 84 | fsl,voltage = <MXS_VOLTAGE_HIGH>; |
| @@ -5,6 +5,8 @@ · 4 unmodified lines | |||
| 5 | * Author: Fabio Estevam <fabio.estevam@freescale.com> | 5 | * Author: Fabio Estevam <fabio.estevam@freescale.com> |
| 6 | */ | 6 | */ |
| 7 | 7 | ||
| 8 | #include <dt-bindings/gpio/gpio.h> | ||
| 9 | |||
| 8 | / { | 10 | / { |
| 9 | aliases { | 11 | aliases { |
| 10 | backlight = &backlight; | 12 | backlight = &backlight; |
| @@ -226,6 +228,7 @@ · 215 unmodified lines | |||
| 226 | MX6QDL_PAD_SD3_DAT1__SD3_DATA1 0x17059 | 228 | MX6QDL_PAD_SD3_DAT1__SD3_DATA1 0x17059 |
| 227 | MX6QDL_PAD_SD3_DAT2__SD3_DATA2 0x17059 | 229 | MX6QDL_PAD_SD3_DAT2__SD3_DATA2 0x17059 |
| 228 | MX6QDL_PAD_SD3_DAT3__SD3_DATA3 0x17059 | 230 | MX6QDL_PAD_SD3_DAT3__SD3_DATA3 0x17059 |
| 231 | MX6QDL_PAD_SD3_DAT5__GPIO7_IO00 0x1b0b0 | ||
| 229 | >; | 232 | >; |
| 230 | }; | 233 | }; |
| 231 | 234 | ||
| @@ -304,7 +307,7 @@ · 72 unmodified lines | |||
| 304 | &usdhc3 { | 307 | &usdhc3 { |
| 305 | pinctrl-names = "default"; | 308 | pinctrl-names = "default"; |
| 306 | pinctrl-0 = <&pinctrl_usdhc3>; | 309 | pinctrl-0 = <&pinctrl_usdhc3>; |
| 307 | non-removable; | 310 | cd-gpios = <&gpio7 0 GPIO_ACTIVE_LOW>; |
| 308 | status = "okay"; | 311 | status = "okay"; |
| 309 | }; | 312 | }; |
| 310 | 313 | ||
| @@ -259,7 +259,7 @@ · 258 unmodified lines | |||
| 259 | interrupts = <GIC_SPI 55 IRQ_TYPE_LEVEL_HIGH>; | 259 | interrupts = <GIC_SPI 55 IRQ_TYPE_LEVEL_HIGH>; |
| 260 | clocks = <&pcc2 IMX7ULP_CLK_WDG1>; | 260 | clocks = <&pcc2 IMX7ULP_CLK_WDG1>; |
| 261 | assigned-clocks = <&pcc2 IMX7ULP_CLK_WDG1>; | 261 | assigned-clocks = <&pcc2 IMX7ULP_CLK_WDG1>; |
| 262 | assigned-clocks-parents = <&scg1 IMX7ULP_CLK_FIRC_BUS_CLK>; | 262 | assigned-clock-parents = <&scg1 IMX7ULP_CLK_FIRC_BUS_CLK>; |
| 263 | timeout-sec = <40>; | 263 | timeout-sec = <40>; |
| 264 | }; | 264 | }; |
| 265 | 265 | ||
| @@ -59,7 +59,7 @@ · 58 unmodified lines | |||
| 59 | }; | 59 | }; |
| 60 | 60 | ||
| 61 | uart_A: serial@84c0 { | 61 | uart_A: serial@84c0 { |
| 62 | compatible = "amlogic,meson6-uart", "amlogic,meson-uart"; | 62 | compatible = "amlogic,meson6-uart"; |
| 63 | reg = <0x84c0 0x18>; | 63 | reg = <0x84c0 0x18>; |
| 64 | interrupts = <GIC_SPI 26 IRQ_TYPE_EDGE_RISING>; | 64 | interrupts = <GIC_SPI 26 IRQ_TYPE_EDGE_RISING>; |
| 65 | fifo-size = <128>; | 65 | fifo-size = <128>; |
| @@ -67,7 +67,7 @@ · 1 unmodified line | |||
| 67 | }; | 67 | }; |
| 68 | 68 | ||
| 69 | uart_B: serial@84dc { | 69 | uart_B: serial@84dc { |
| 70 | compatible = "amlogic,meson6-uart", "amlogic,meson-uart"; | 70 | compatible = "amlogic,meson6-uart"; |
| 71 | reg = <0x84dc 0x18>; | 71 | reg = <0x84dc 0x18>; |
| 72 | interrupts = <GIC_SPI 75 IRQ_TYPE_EDGE_RISING>; | 72 | interrupts = <GIC_SPI 75 IRQ_TYPE_EDGE_RISING>; |
| 73 | status = "disabled"; | 73 | status = "disabled"; |
| @@ -105,7 +105,7 @@ · 31 unmodified lines | |||
| 105 | }; | 105 | }; |
| 106 | 106 | ||
| 107 | uart_C: serial@8700 { | 107 | uart_C: serial@8700 { |
| 108 | compatible = "amlogic,meson6-uart", "amlogic,meson-uart"; | 108 | compatible = "amlogic,meson6-uart"; |
| 109 | reg = <0x8700 0x18>; | 109 | reg = <0x8700 0x18>; |
| 110 | interrupts = <GIC_SPI 93 IRQ_TYPE_EDGE_RISING>; | 110 | interrupts = <GIC_SPI 93 IRQ_TYPE_EDGE_RISING>; |
| 111 | status = "disabled"; | 111 | status = "disabled"; |
| @@ -228,7 +228,7 @@ · 116 unmodified lines | |||
| 228 | }; | 228 | }; |
| 229 | 229 | ||
| 230 | uart_AO: serial@4c0 { | 230 | uart_AO: serial@4c0 { |
| 231 | compatible = "amlogic,meson6-uart", "amlogic,meson-ao-uart", "amlogic,meson-uart"; | 231 | compatible = "amlogic,meson6-uart", "amlogic,meson-ao-uart"; |
| 232 | reg = <0x4c0 0x18>; | 232 | reg = <0x4c0 0x18>; |
| 233 | interrupts = <GIC_SPI 90 IRQ_TYPE_EDGE_RISING>; | 233 | interrupts = <GIC_SPI 90 IRQ_TYPE_EDGE_RISING>; |
| 234 | status = "disabled"; | 234 | status = "disabled"; |
| @@ -736,27 +736,27 @@ · 735 unmodified lines | |||
| 736 | }; | 736 | }; |
| 737 | 737 | ||
| 738 | &uart_AO { | 738 | &uart_AO { |
| 739 | compatible = "amlogic,meson8-uart", "amlogic,meson-uart"; | 739 | compatible = "amlogic,meson8-uart", "amlogic,meson-ao-uart"; |
| 740 | clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_CLK81>; | 740 | clocks = <&xtal>, <&clkc CLKID_CLK81>, <&clkc CLKID_CLK81>; |
| 741 | clock-names = "baud", "xtal", "pclk"; | 741 | clock-names = "xtal", "pclk", "baud"; |
| 742 | }; | 742 | }; |
| 743 | 743 | ||
| 744 | &uart_A { | 744 | &uart_A { |
| 745 | compatible = "amlogic,meson8-uart", "amlogic,meson-uart"; | 745 | compatible = "amlogic,meson8-uart"; |
| 746 | clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART0>; | 746 | clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; |
| 747 | clock-names = "baud", "xtal", "pclk"; | 747 | clock-names = "xtal", "pclk", "baud"; |
| 748 | }; | 748 | }; |
| 749 | 749 | ||
| 750 | &uart_B { | 750 | &uart_B { |
| 751 | compatible = "amlogic,meson8-uart", "amlogic,meson-uart"; | 751 | compatible = "amlogic,meson8-uart"; |
| 752 | clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART1>; | 752 | clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; |
| 753 | clock-names = "baud", "xtal", "pclk"; | 753 | clock-names = "xtal", "pclk", "baud"; |
| 754 | }; | 754 | }; |
| 755 | 755 | ||
| 756 | &uart_C { | 756 | &uart_C { |
| 757 | compatible = "amlogic,meson8-uart", "amlogic,meson-uart"; | 757 | compatible = "amlogic,meson8-uart"; |
| 758 | clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART2>; | 758 | clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; |
| 759 | clock-names = "baud", "xtal", "pclk"; | 759 | clock-names = "xtal", "pclk", "baud"; |
| 760 | }; | 760 | }; |
| 761 | 761 | ||
| 762 | &usb0 { | 762 | &usb0 { |
| @@ -724,27 +724,27 @@ · 723 unmodified lines | |||
| 724 | }; | 724 | }; |
| 725 | 725 | ||
| 726 | &uart_AO { | 726 | &uart_AO { |
| 727 | compatible = "amlogic,meson8b-uart", "amlogic,meson-uart"; | 727 | compatible = "amlogic,meson8b-uart", "amlogic,meson-ao-uart"; |
| 728 | clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_CLK81>; | 728 | clocks = <&xtal>, <&clkc CLKID_CLK81>, <&clkc CLKID_CLK81>; |
| 729 | clock-names = "baud", "xtal", "pclk"; | 729 | clock-names = "xtal", "pclk", "baud"; |
| 730 | }; | 730 | }; |
| 731 | 731 | ||
| 732 | &uart_A { | 732 | &uart_A { |
| 733 | compatible = "amlogic,meson8b-uart", "amlogic,meson-uart"; | 733 | compatible = "amlogic,meson8b-uart"; |
| 734 | clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART0>; | 734 | clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; |
| 735 | clock-names = "baud", "xtal", "pclk"; | 735 | clock-names = "xtal", "pclk", "baud"; |
| 736 | }; | 736 | }; |
| 737 | 737 | ||
| 738 | &uart_B { | 738 | &uart_B { |
| 739 | compatible = "amlogic,meson8b-uart", "amlogic,meson-uart"; | 739 | compatible = "amlogic,meson8b-uart"; |
| 740 | clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART1>; | 740 | clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; |
| 741 | clock-names = "baud", "xtal", "pclk"; | 741 | clock-names = "xtal", "pclk", "baud"; |
| 742 | }; | 742 | }; |
| 743 | 743 | ||
| 744 | &uart_C { | 744 | &uart_C { |
| 745 | compatible = "amlogic,meson8b-uart", "amlogic,meson-uart"; | 745 | compatible = "amlogic,meson8b-uart"; |
| 746 | clocks = <&clkc CLKID_CLK81>, <&xtal>, <&clkc CLKID_UART2>; | 746 | clocks = <&xtal>, <&clkc CLKID_UART0>, <&clkc CLKID_CLK81>; |
| 747 | clock-names = "baud", "xtal", "pclk"; | 747 | clock-names = "xtal", "pclk", "baud"; |
| 748 | }; | 748 | }; |
| 749 | 749 | ||
| 750 | &usb0 { | 750 | &usb0 { |
| @@ -0,0 +1,47 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0-only | ||
| 2 | /dts-v1/; | ||
| 3 | |||
| 4 | #include "omap3-beagle.dts" | ||
| 5 | |||
| 6 | / { | ||
| 7 | model = "TI OMAP3 BeagleBoard A to B4"; | ||
| 8 | compatible = "ti,omap3-beagle-ab4", "ti,omap3-beagle", "ti,omap3430", "ti,omap3"; | ||
| 9 | }; | ||
| 10 | |||
| 11 | /* | ||
| 12 | * Workaround for capacitor C70 issue, see "Boards revision A and < B5" | ||
| 13 | * section at https://elinux.org/BeagleBoard_Community | ||
| 14 | */ | ||
| 15 | |||
| 16 | /* Unusable as clocksource because of unreliable oscillator */ | ||
| 17 | &counter32k { | ||
| 18 | status = "disabled"; | ||
| 19 | }; | ||
| 20 | |||
| 21 | /* Unusable as clockevent because of unreliable oscillator, allow to idle */ | ||
| 22 | &timer1_target { | ||
| 23 | /delete-property/ti,no-reset-on-init; | ||
| 24 | /delete-property/ti,no-idle; | ||
| 25 | timer@0 { | ||
| 26 | /delete-property/ti,timer-alwon; | ||
| 27 | }; | ||
| 28 | }; | ||
| 29 | |||
| 30 | /* Preferred always-on timer for clocksource */ | ||
| 31 | &timer12_target { | ||
| 32 | ti,no-reset-on-init; | ||
| 33 | ti,no-idle; | ||
| 34 | timer@0 { | ||
| 35 | /* Always clocked by secure_32k_fck */ | ||
| 36 | }; | ||
| 37 | }; | ||
| 38 | |||
| 39 | /* Preferred timer for clockevent */ | ||
| 40 | &timer2_target { | ||
| 41 | ti,no-reset-on-init; | ||
| 42 | ti,no-idle; | ||
| 43 | timer@0 { | ||
| 44 | assigned-clocks = <&gpt2_fck>; | ||
| 45 | assigned-clock-parents = <&sys_ck>; | ||
| 46 | }; | ||
| 47 | }; | ||
| @@ -304,39 +304,6 @@ · 303 unmodified lines | |||
| 304 | phys = <0 &hsusb2_phy>; | 304 | phys = <0 &hsusb2_phy>; |
| 305 | }; | 305 | }; |
| 306 | 306 | ||
| 307 | /* Unusable as clocksource because of unreliable oscillator */ | ||
| 308 | &counter32k { | ||
| 309 | status = "disabled"; | ||
| 310 | }; | ||
| 311 | |||
| 312 | /* Unusable as clockevent because if unreliable oscillator, allow to idle */ | ||
| 313 | &timer1_target { | ||
| 314 | /delete-property/ti,no-reset-on-init; | ||
| 315 | /delete-property/ti,no-idle; | ||
| 316 | timer@0 { | ||
| 317 | /delete-property/ti,timer-alwon; | ||
| 318 | }; | ||
| 319 | }; | ||
| 320 | |||
| 321 | /* Preferred always-on timer for clocksource */ | ||
| 322 | &timer12_target { | ||
| 323 | ti,no-reset-on-init; | ||
| 324 | ti,no-idle; | ||
| 325 | timer@0 { | ||
| 326 | /* Always clocked by secure_32k_fck */ | ||
| 327 | }; | ||
| 328 | }; | ||
| 329 | |||
| 330 | /* Preferred timer for clockevent */ | ||
| 331 | &timer2_target { | ||
| 332 | ti,no-reset-on-init; | ||
| 333 | ti,no-idle; | ||
| 334 | timer@0 { | ||
| 335 | assigned-clocks = <&gpt2_fck>; | ||
| 336 | assigned-clock-parents = <&sys_ck>; | ||
| 337 | }; | ||
| 338 | }; | ||
| 339 | |||
| 340 | &twl_gpio { | 307 | &twl_gpio { |
| 341 | ti,use-leds; | 308 | ti,use-leds; |
| 342 | /* pullups: BIT(1) */ | 309 | /* pullups: BIT(1) */ |
| @@ -235,7 +235,6 @@ · 234 unmodified lines | |||
| 235 | #address-cells = <1>; | 235 | #address-cells = <1>; |
| 236 | #size-cells = <0>; | 236 | #size-cells = <0>; |
| 237 | reg = <0x41>; | 237 | reg = <0x41>; |
| 238 | irq-over-gpio; | ||
| 239 | irq-gpios = <&gpiopinctrl 29 0x4>; | 238 | irq-gpios = <&gpiopinctrl 29 0x4>; |
| 240 | id = <0>; | 239 | id = <0>; |
| 241 | blocks = <0x5>; | 240 | blocks = <0x5>; |
| @@ -185,10 +185,6 @@ · 184 unmodified lines | |||
| 185 | cap-sd-highspeed; | 185 | cap-sd-highspeed; |
| 186 | cap-mmc-highspeed; | 186 | cap-mmc-highspeed; |
| 187 | /* All direction control is used */ | 187 | /* All direction control is used */ |
| 188 | st,sig-dir-cmd; | ||
| 189 | st,sig-dir-dat0; | ||
| 190 | st,sig-dir-dat2; | ||
| 191 | st,sig-dir-dat31; | ||
| 192 | st,sig-pin-fbclk; | 188 | st,sig-pin-fbclk; |
| 193 | full-pwr-cycle; | 189 | full-pwr-cycle; |
| 194 | vmmc-supply = <&ab8500_ldo_aux3_reg>; | 190 | vmmc-supply = <&ab8500_ldo_aux3_reg>; |
| @@ -13,12 +13,12 @@ · 12 unmodified lines | |||
| 13 | static int crypto_blake2s_update_arm(struct shash_desc *desc, | 13 | static int crypto_blake2s_update_arm(struct shash_desc *desc, |
| 14 | const u8 *in, unsigned int inlen) | 14 | const u8 *in, unsigned int inlen) |
| 15 | { | 15 | { |
| 16 | return crypto_blake2s_update(desc, in, inlen, blake2s_compress); | 16 | return crypto_blake2s_update(desc, in, inlen, false); |
| 17 | } | 17 | } |
| 18 | 18 | ||
| 19 | static int crypto_blake2s_final_arm(struct shash_desc *desc, u8 *out) | 19 | static int crypto_blake2s_final_arm(struct shash_desc *desc, u8 *out) |
| 20 | { | 20 | { |
| 21 | return crypto_blake2s_final(desc, out, blake2s_compress); | 21 | return crypto_blake2s_final(desc, out, false); |
| 22 | } | 22 | } |
| 23 | 23 | ||
| 24 | #define BLAKE2S_ALG(name, driver_name, digest_size) \ | 24 | #define BLAKE2S_ALG(name, driver_name, digest_size) \ |
| @@ -263,9 +263,9 @@ static int __init omapdss_init_of(void) · 262 unmodified lines | |||
| 263 | } | 263 | } |
| 264 | 264 | ||
| 265 | r = of_platform_populate(node, NULL, NULL, &pdev->dev); | 265 | r = of_platform_populate(node, NULL, NULL, &pdev->dev); |
| 266 | put_device(&pdev->dev); | ||
| 266 | if (r) { | 267 | if (r) { |
| 267 | pr_err("Unable to populate DSS submodule devices\n"); | 268 | pr_err("Unable to populate DSS submodule devices\n"); |
| 268 | put_device(&pdev->dev); | ||
| 269 | return r; | 269 | return r; |
| 270 | } | 270 | } |
| 271 | 271 | ||
| @@ -752,8 +752,10 @@ static int __init _init_clkctrl_providers(void) · 751 unmodified lines | |||
| 752 | 752 | ||
| 753 | for_each_matching_node(np, ti_clkctrl_match_table) { | 753 | for_each_matching_node(np, ti_clkctrl_match_table) { |
| 754 | ret = _setup_clkctrl_provider(np); | 754 | ret = _setup_clkctrl_provider(np); |
| 755 | if (ret) | 755 | if (ret) { |
| 756 | of_node_put(np); | ||
| 756 | break; | 757 | break; |
| 758 | } | ||
| 757 | } | 759 | } |
| 758 | 760 | ||
| 759 | return ret; | 761 | return ret; |
| @@ -2,6 +2,7 @@ · 1 unmodified line | |||
| 2 | menuconfig ARCH_INTEL_SOCFPGA | 2 | menuconfig ARCH_INTEL_SOCFPGA |
| 3 | bool "Altera SOCFPGA family" | 3 | bool "Altera SOCFPGA family" |
| 4 | depends on ARCH_MULTI_V7 | 4 | depends on ARCH_MULTI_V7 |
| 5 | select ARCH_HAS_RESET_CONTROLLER | ||
| 5 | select ARCH_SUPPORTS_BIG_ENDIAN | 6 | select ARCH_SUPPORTS_BIG_ENDIAN |
| 6 | select ARM_AMBA | 7 | select ARM_AMBA |
| 7 | select ARM_GIC | 8 | select ARM_GIC |
| @@ -18,6 +19,7 @@ menuconfig ARCH_INTEL_SOCFPGA · 10 unmodified lines | |||
| 18 | select PL310_ERRATA_727915 | 19 | select PL310_ERRATA_727915 |
| 19 | select PL310_ERRATA_753970 if PL310 | 20 | select PL310_ERRATA_753970 if PL310 |
| 20 | select PL310_ERRATA_769419 | 21 | select PL310_ERRATA_769419 |
| 22 | select RESET_CONTROLLER | ||
| 21 | 23 | ||
| 22 | if ARCH_INTEL_SOCFPGA | 24 | if ARCH_INTEL_SOCFPGA |
| 23 | config SOCFPGA_SUSPEND | 25 | config SOCFPGA_SUSPEND |
| @@ -672,6 +672,7 @@ config ARM64_WORKAROUND_TRBE_OVERWRITE_FILL_MODE · 671 unmodified lines | |||
| 672 | 672 | ||
| 673 | config ARM64_ERRATUM_2051678 | 673 | config ARM64_ERRATUM_2051678 |
| 674 | bool "Cortex-A510: 2051678: disable Hardware Update of the page table dirty bit" | 674 | bool "Cortex-A510: 2051678: disable Hardware Update of the page table dirty bit" |
| 675 | default y | ||
| 675 | help | 676 | help |
| 676 | This options adds the workaround for ARM Cortex-A510 erratum ARM64_ERRATUM_2051678. | 677 | This options adds the workaround for ARM Cortex-A510 erratum ARM64_ERRATUM_2051678. |
| 677 | Affected Coretex-A510 might not respect the ordering rules for | 678 | Affected Coretex-A510 might not respect the ordering rules for |
| @@ -680,6 +681,22 @@ config ARM64_ERRATUM_2051678 · 2 unmodified lines | |||
| 680 | 681 | ||
| 681 | If unsure, say Y. | 682 | If unsure, say Y. |
| 682 | 683 | ||
| 684 | config ARM64_ERRATUM_2077057 | ||
| 685 | bool "Cortex-A510: 2077057: workaround software-step corrupting SPSR_EL2" | ||
| 686 | help | ||
| 687 | This option adds the workaround for ARM Cortex-A510 erratum 2077057. | ||
| 688 | Affected Cortex-A510 may corrupt SPSR_EL2 when the a step exception is | ||
| 689 | expected, but a Pointer Authentication trap is taken instead. The | ||
| 690 | erratum causes SPSR_EL1 to be copied to SPSR_EL2, which could allow | ||
| 691 | EL1 to cause a return to EL2 with a guest controlled ELR_EL2. | ||
| 692 | |||
| 693 | This can only happen when EL2 is stepping EL1. | ||
| 694 | |||
| 695 | When these conditions occur, the SPSR_EL2 value is unchanged from the | ||
| 696 | previous guest entry, and can be restored from the in-memory copy. | ||
| 697 | |||
| 698 | If unsure, say Y. | ||
| 699 | |||
| 683 | config ARM64_ERRATUM_2119858 | 700 | config ARM64_ERRATUM_2119858 |
| 684 | bool "Cortex-A710/X2: 2119858: workaround TRBE overwriting trace data in FILL mode" | 701 | bool "Cortex-A710/X2: 2119858: workaround TRBE overwriting trace data in FILL mode" |
| 685 | default y | 702 | default y |
| @@ -309,9 +309,6 @@ config ARCH_VISCONTI · 308 unmodified lines | |||
| 309 | help | 309 | help |
| 310 | This enables support for Toshiba Visconti SoCs Family. | 310 | This enables support for Toshiba Visconti SoCs Family. |
| 311 | 311 | ||
| 312 | config ARCH_VULCAN | ||
| 313 | def_bool n | ||
| 314 | |||
| 315 | config ARCH_XGENE | 312 | config ARCH_XGENE |
| 316 | bool "AppliedMicro X-Gene SOC Family" | 313 | bool "AppliedMicro X-Gene SOC Family" |
| 317 | help | 314 | help |
| @@ -107,6 +107,12 @@ · 106 unmodified lines | |||
| 107 | no-map; | 107 | no-map; |
| 108 | }; | 108 | }; |
| 109 | 109 | ||
| 110 | /* 32 MiB reserved for ARM Trusted Firmware (BL32) */ | ||
| 111 | secmon_reserved_bl32: secmon@5300000 { | ||
| 112 | reg = <0x0 0x05300000 0x0 0x2000000>; | ||
| 113 | no-map; | ||
| 114 | }; | ||
| 115 | |||
| 110 | linux,cma { | 116 | linux,cma { |
| 111 | compatible = "shared-dma-pool"; | 117 | compatible = "shared-dma-pool"; |
| 112 | reusable; | 118 | reusable; |
| @@ -157,14 +157,6 @@ · 156 unmodified lines | |||
| 157 | regulator-always-on; | 157 | regulator-always-on; |
| 158 | }; | 158 | }; |
| 159 | 159 | ||
| 160 | reserved-memory { | ||
| 161 | /* TEE Reserved Memory */ | ||
| 162 | bl32_reserved: bl32@5000000 { | ||
| 163 | reg = <0x0 0x05300000 0x0 0x2000000>; | ||
| 164 | no-map; | ||
| 165 | }; | ||
| 166 | }; | ||
| 167 | |||
| 168 | sdio_pwrseq: sdio-pwrseq { | 160 | sdio_pwrseq: sdio-pwrseq { |
| 169 | compatible = "mmc-pwrseq-simple"; | 161 | compatible = "mmc-pwrseq-simple"; |
| 170 | reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>; | 162 | reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>; |
| @@ -17,7 +17,7 @@ · 16 unmodified lines | |||
| 17 | rtc1 = &vrtc; | 17 | rtc1 = &vrtc; |
| 18 | }; | 18 | }; |
| 19 | 19 | ||
| 20 | dioo2133: audio-amplifier-0 { | 20 | dio2133: audio-amplifier-0 { |
| 21 | compatible = "simple-audio-amplifier"; | 21 | compatible = "simple-audio-amplifier"; |
| 22 | enable-gpios = <&gpio_ao GPIOAO_2 GPIO_ACTIVE_HIGH>; | 22 | enable-gpios = <&gpio_ao GPIOAO_2 GPIO_ACTIVE_HIGH>; |
| 23 | VCC-supply = <&vcc_5v>; | 23 | VCC-supply = <&vcc_5v>; |
| @@ -219,7 +219,7 @@ · 195 unmodified lines | |||
| 219 | audio-widgets = "Line", "Lineout"; | 219 | audio-widgets = "Line", "Lineout"; |
| 220 | audio-aux-devs = <&tdmout_b>, <&tdmout_c>, <&tdmin_a>, | 220 | audio-aux-devs = <&tdmout_b>, <&tdmout_c>, <&tdmin_a>, |
| 221 | <&tdmin_b>, <&tdmin_c>, <&tdmin_lb>, | 221 | <&tdmin_b>, <&tdmin_c>, <&tdmin_lb>, |
| 222 | <&dioo2133>; | 222 | <&dio2133>; |
| 223 | audio-routing = "TDMOUT_B IN 0", "FRDDR_A OUT 1", | 223 | audio-routing = "TDMOUT_B IN 0", "FRDDR_A OUT 1", |
| 224 | "TDMOUT_B IN 1", "FRDDR_B OUT 1", | 224 | "TDMOUT_B IN 1", "FRDDR_B OUT 1", |
| 225 | "TDMOUT_B IN 2", "FRDDR_C OUT 1", | 225 | "TDMOUT_B IN 2", "FRDDR_C OUT 1", |
| @@ -49,6 +49,12 @@ · 48 unmodified lines | |||
| 49 | no-map; | 49 | no-map; |
| 50 | }; | 50 | }; |
| 51 | 51 | ||
| 52 | /* 32 MiB reserved for ARM Trusted Firmware (BL32) */ | ||
| 53 | secmon_reserved_bl32: secmon@5300000 { | ||
| 54 | reg = <0x0 0x05300000 0x0 0x2000000>; | ||
| 55 | no-map; | ||
| 56 | }; | ||
| 57 | |||
| 52 | linux,cma { | 58 | linux,cma { |
| 53 | compatible = "shared-dma-pool"; | 59 | compatible = "shared-dma-pool"; |
| 54 | reusable; | 60 | reusable; |
| @@ -123,7 +123,7 @@ · 122 unmodified lines | |||
| 123 | regulator-min-microvolt = <1800000>; | 123 | regulator-min-microvolt = <1800000>; |
| 124 | regulator-max-microvolt = <3300000>; | 124 | regulator-max-microvolt = <3300000>; |
| 125 | 125 | ||
| 126 | enable-gpio = <&gpio GPIOE_2 GPIO_ACTIVE_HIGH>; | 126 | enable-gpio = <&gpio_ao GPIOE_2 GPIO_ACTIVE_HIGH>; |
| 127 | enable-active-high; | 127 | enable-active-high; |
| 128 | regulator-always-on; | 128 | regulator-always-on; |
| 129 | 129 | ||
| @@ -48,7 +48,7 @@ · 47 unmodified lines | |||
| 48 | regulator-max-microvolt = <3300000>; | 48 | regulator-max-microvolt = <3300000>; |
| 49 | vin-supply = <&vcc_5v>; | 49 | vin-supply = <&vcc_5v>; |
| 50 | 50 | ||
| 51 | enable-gpio = <&gpio GPIOE_2 GPIO_ACTIVE_HIGH>; | 51 | enable-gpio = <&gpio_ao GPIOE_2 GPIO_OPEN_DRAIN>; |
| 52 | enable-active-high; | 52 | enable-active-high; |
| 53 | regulator-always-on; | 53 | regulator-always-on; |
| 54 | 54 | ||
| @@ -203,14 +203,6 @@ · 202 unmodified lines | |||
| 203 | regulator-always-on; | 203 | regulator-always-on; |
| 204 | }; | 204 | }; |
| 205 | 205 | ||
| 206 | reserved-memory { | ||
| 207 | /* TEE Reserved Memory */ | ||
| 208 | bl32_reserved: bl32@5000000 { | ||
| 209 | reg = <0x0 0x05300000 0x0 0x2000000>; | ||
| 210 | no-map; | ||
| 211 | }; | ||
| 212 | }; | ||
| 213 | |||
| 214 | sdio_pwrseq: sdio-pwrseq { | 206 | sdio_pwrseq: sdio-pwrseq { |
| 215 | compatible = "mmc-pwrseq-simple"; | 207 | compatible = "mmc-pwrseq-simple"; |
| 216 | reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>; | 208 | reset-gpios = <&gpio GPIOX_6 GPIO_ACTIVE_LOW>; |
| @@ -157,6 +157,10 @@ · 156 unmodified lines | |||
| 157 | }; | 157 | }; |
| 158 | }; | 158 | }; |
| 159 | 159 | ||
| 160 | &ftm_alarm0 { | ||
| 161 | status = "okay"; | ||
| 162 | }; | ||
| 163 | |||
| 160 | &gpio1 { | 164 | &gpio1 { |
| 161 | gpio-line-names = | 165 | gpio-line-names = |
| 162 | "", "", "", "", "", "", "", "", | 166 | "", "", "", "", "", "", "", "", |
| @@ -1115,8 +1115,8 @@ · 1114 unmodified lines | |||
| 1115 | status = "okay"; | 1115 | status = "okay"; |
| 1116 | 1116 | ||
| 1117 | ports { | 1117 | ports { |
| 1118 | port@1 { | 1118 | port@0 { |
| 1119 | reg = <1>; | 1119 | reg = <0>; |
| 1120 | 1120 | ||
| 1121 | mipi1_sensor_ep: endpoint { | 1121 | mipi1_sensor_ep: endpoint { |
| 1122 | remote-endpoint = <&camera1_ep>; | 1122 | remote-endpoint = <&camera1_ep>; |
| @@ -554,7 +554,7 @@ · 553 unmodified lines | |||
| 554 | assigned-clock-rates = <0>, <0>, <0>, <594000000>; | 554 | assigned-clock-rates = <0>, <0>, <0>, <594000000>; |
| 555 | status = "disabled"; | 555 | status = "disabled"; |
| 556 | 556 | ||
| 557 | port@0 { | 557 | port { |
| 558 | lcdif_mipi_dsi: endpoint { | 558 | lcdif_mipi_dsi: endpoint { |
| 559 | remote-endpoint = <&mipi_dsi_lcdif_in>; | 559 | remote-endpoint = <&mipi_dsi_lcdif_in>; |
| 560 | }; | 560 | }; |
| @@ -1151,8 +1151,8 @@ · 590 unmodified lines | |||
| 1151 | #address-cells = <1>; | 1151 | #address-cells = <1>; |
| 1152 | #size-cells = <0>; | 1152 | #size-cells = <0>; |
| 1153 | 1153 | ||
| 1154 | port@0 { | 1154 | port@1 { |
| 1155 | reg = <0>; | 1155 | reg = <1>; |
| 1156 | 1156 | ||
| 1157 | csi1_mipi_ep: endpoint { | 1157 | csi1_mipi_ep: endpoint { |
| 1158 | remote-endpoint = <&csi1_ep>; | 1158 | remote-endpoint = <&csi1_ep>; |
| @@ -1203,8 +1203,8 @@ · 44 unmodified lines | |||
| 1203 | #address-cells = <1>; | 1203 | #address-cells = <1>; |
| 1204 | #size-cells = <0>; | 1204 | #size-cells = <0>; |
| 1205 | 1205 | ||
| 1206 | port@0 { | 1206 | port@1 { |
| 1207 | reg = <0>; | 1207 | reg = <1>; |
| 1208 | 1208 | ||
| 1209 | csi2_mipi_ep: endpoint { | 1209 | csi2_mipi_ep: endpoint { |
| 1210 | remote-endpoint = <&csi2_ep>; | 1210 | remote-endpoint = <&csi2_ep>; |
| @@ -91,7 +91,7 @@ · 90 unmodified lines | |||
| 91 | 91 | ||
| 92 | sound { | 92 | sound { |
| 93 | compatible = "fsl,imx-audio-tlv320aic32x4"; | 93 | compatible = "fsl,imx-audio-tlv320aic32x4"; |
| 94 | model = "tqm-tlv320aic32"; | 94 | model = "imx-audio-tlv320aic32x4"; |
| 95 | ssi-controller = <&sai3>; | 95 | ssi-controller = <&sai3>; |
| 96 | audio-codec = <&tlv320aic3x04>; | 96 | audio-codec = <&tlv320aic3x04>; |
| 97 | }; | 97 | }; |
| @@ -15,8 +15,18 @@ · 14 unmodified lines | |||
| 15 | model = "Texas Instruments J721S2 EVM"; | 15 | model = "Texas Instruments J721S2 EVM"; |
| 16 | 16 | ||
| 17 | chosen { | 17 | chosen { |
| 18 | stdout-path = "serial10:115200n8"; | 18 | stdout-path = "serial2:115200n8"; |
| 19 | bootargs = "console=ttyS10,115200n8 earlycon=ns16550a,mmio32,2880000"; | 19 | bootargs = "console=ttyS2,115200n8 earlycon=ns16550a,mmio32,2880000"; |
| 20 | }; | ||
| 21 | |||
| 22 | aliases { | ||
| 23 | serial1 = &mcu_uart0; | ||
| 24 | serial2 = &main_uart8; | ||
| 25 | mmc0 = &main_sdhci0; | ||
| 26 | mmc1 = &main_sdhci1; | ||
| 27 | can0 = &main_mcan16; | ||
| 28 | can1 = &mcu_mcan0; | ||
| 29 | can2 = &mcu_mcan1; | ||
| 20 | }; | 30 | }; |
| 21 | 31 | ||
| 22 | evm_12v0: fixedregulator-evm12v0 { | 32 | evm_12v0: fixedregulator-evm12v0 { |
| @@ -21,28 +21,6 @@ · 20 unmodified lines | |||
| 21 | #address-cells = <2>; | 21 | #address-cells = <2>; |
| 22 | #size-cells = <2>; | 22 | #size-cells = <2>; |
| 23 | 23 | ||
| 24 | aliases { | ||
| 25 | serial0 = &wkup_uart0; | ||
| 26 | serial1 = &mcu_uart0; | ||
| 27 | serial2 = &main_uart0; | ||
| 28 | serial3 = &main_uart1; | ||
| 29 | serial4 = &main_uart2; | ||
| 30 | serial5 = &main_uart3; | ||
| 31 | serial6 = &main_uart4; | ||
| 32 | serial7 = &main_uart5; | ||
| 33 | serial8 = &main_uart6; | ||
| 34 | serial9 = &main_uart7; | ||
| 35 | serial10 = &main_uart8; | ||
| 36 | serial11 = &main_uart9; | ||
| 37 | mmc0 = &main_sdhci0; | ||
| 38 | mmc1 = &main_sdhci1; | ||
| 39 | can0 = &main_mcan16; | ||
| 40 | can1 = &mcu_mcan0; | ||
| 41 | can2 = &mcu_mcan1; | ||
| 42 | can3 = &main_mcan3; | ||
| 43 | can4 = &main_mcan5; | ||
| 44 | }; | ||
| 45 | |||
| 46 | chosen { }; | 24 | chosen { }; |
| 47 | 25 | ||
| 48 | cpus { | 26 | cpus { |
| @@ -106,7 +106,7 @@ · 105 unmodified lines | |||
| 106 | msr_s SYS_ICC_SRE_EL2, x0 | 106 | msr_s SYS_ICC_SRE_EL2, x0 |
| 107 | isb // Make sure SRE is now set | 107 | isb // Make sure SRE is now set |
| 108 | mrs_s x0, SYS_ICC_SRE_EL2 // Read SRE back, | 108 | mrs_s x0, SYS_ICC_SRE_EL2 // Read SRE back, |
| 109 | tbz x0, #0, 1f // and check that it sticks | 109 | tbz x0, #0, .Lskip_gicv3_\@ // and check that it sticks |
| 110 | msr_s SYS_ICH_HCR_EL2, xzr // Reset ICC_HCR_EL2 to defaults | 110 | msr_s SYS_ICH_HCR_EL2, xzr // Reset ICC_HCR_EL2 to defaults |
| 111 | .Lskip_gicv3_\@: | 111 | .Lskip_gicv3_\@: |
| 112 | .endm | 112 | .endm |
| @@ -600,6 +600,14 @@ const struct arm64_cpu_capabilities arm64_errata[] = { · 599 unmodified lines | |||
| 600 | CAP_MIDR_RANGE_LIST(trbe_write_out_of_range_cpus), | 600 | CAP_MIDR_RANGE_LIST(trbe_write_out_of_range_cpus), |
| 601 | }, | 601 | }, |
| 602 | #endif | 602 | #endif |
| 603 | #ifdef CONFIG_ARM64_ERRATUM_2077057 | ||
| 604 | { | ||
| 605 | .desc = "ARM erratum 2077057", | ||
| 606 | .capability = ARM64_WORKAROUND_2077057, | ||
| 607 | .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, | ||
| 608 | ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A510, 0, 0, 2), | ||
| 609 | }, | ||
| 610 | #endif | ||
| 603 | #ifdef CONFIG_ARM64_ERRATUM_2064142 | 611 | #ifdef CONFIG_ARM64_ERRATUM_2064142 |
| 604 | { | 612 | { |
| 605 | .desc = "ARM erratum 2064142", | 613 | .desc = "ARM erratum 2064142", |
| @@ -797,6 +797,24 @@ static bool kvm_vcpu_exit_request(struct kvm_vcpu *vcpu, int *ret) · 796 unmodified lines | |||
| 797 | xfer_to_guest_mode_work_pending(); | 797 | xfer_to_guest_mode_work_pending(); |
| 798 | } | 798 | } |
| 799 | 799 | ||
| 800 | /* | ||
| 801 | * Actually run the vCPU, entering an RCU extended quiescent state (EQS) while | ||
| 802 | * the vCPU is running. | ||
| 803 | * | ||
| 804 | * This must be noinstr as instrumentation may make use of RCU, and this is not | ||
| 805 | * safe during the EQS. | ||
| 806 | */ | ||
| 807 | static int noinstr kvm_arm_vcpu_enter_exit(struct kvm_vcpu *vcpu) | ||
| 808 | { | ||
| 809 | int ret; | ||
| 810 | |||
| 811 | guest_state_enter_irqoff(); | ||
| 812 | ret = kvm_call_hyp_ret(__kvm_vcpu_run, vcpu); | ||
| 813 | guest_state_exit_irqoff(); | ||
| 814 | |||
| 815 | return ret; | ||
| 816 | } | ||
| 817 | |||
| 800 | /** | 818 | /** |
| 801 | * kvm_arch_vcpu_ioctl_run - the main VCPU run function to execute guest code | 819 | * kvm_arch_vcpu_ioctl_run - the main VCPU run function to execute guest code |
| 802 | * @vcpu: The VCPU pointer | 820 | * @vcpu: The VCPU pointer |
| @@ -881,9 +899,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 78 unmodified lines | |||
| 881 | * Enter the guest | 899 | * Enter the guest |
| 882 | */ | 900 | */ |
| 883 | trace_kvm_entry(*vcpu_pc(vcpu)); | 901 | trace_kvm_entry(*vcpu_pc(vcpu)); |
| 884 | guest_enter_irqoff(); | 902 | guest_timing_enter_irqoff(); |
| 885 | 903 | ||
| 886 | ret = kvm_call_hyp_ret(__kvm_vcpu_run, vcpu); | 904 | ret = kvm_arm_vcpu_enter_exit(vcpu); |
| 887 | 905 | ||
| 888 | vcpu->mode = OUTSIDE_GUEST_MODE; | 906 | vcpu->mode = OUTSIDE_GUEST_MODE; |
| 889 | vcpu->stat.exits++; | 907 | vcpu->stat.exits++; |
| @@ -918,26 +936,23 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 28 unmodified lines | |||
| 918 | kvm_arch_vcpu_ctxsync_fp(vcpu); | 936 | kvm_arch_vcpu_ctxsync_fp(vcpu); |
| 919 | 937 | ||
| 920 | /* | 938 | /* |
| 921 | * We may have taken a host interrupt in HYP mode (ie | 939 | * We must ensure that any pending interrupts are taken before |
| 922 | * while executing the guest). This interrupt is still | 940 | * we exit guest timing so that timer ticks are accounted as |
| 923 | * pending, as we haven't serviced it yet! | 941 | * guest time. Transiently unmask interrupts so that any |
| 942 | * pending interrupts are taken. | ||
| 924 | * | 943 | * |
| 925 | * We're now back in SVC mode, with interrupts | 944 | * Per ARM DDI 0487G.b section D1.13.4, an ISB (or other |
| 926 | * disabled. Enabling the interrupts now will have | 945 | * context synchronization event) is necessary to ensure that |
| 927 | * the effect of taking the interrupt again, in SVC | 946 | * pending interrupts are taken. |
| 928 | * mode this time. | ||
| 929 | */ | 947 | */ |
| 930 | local_irq_enable(); | 948 | local_irq_enable(); |
| 949 | isb(); | ||
| 950 | local_irq_disable(); | ||
| 951 | |||
| 952 | guest_timing_exit_irqoff(); | ||
| 953 | |||
| 954 | local_irq_enable(); | ||
| 931 | 955 | ||
| 932 | /* | ||
| 933 | * We do local_irq_enable() before calling guest_exit() so | ||
| 934 | * that if a timer interrupt hits while running the guest we | ||
| 935 | * account that tick as being spent in the guest. We enable | ||
| 936 | * preemption after calling guest_exit() so that if we get | ||
| 937 | * preempted we make sure ticks after that is not counted as | ||
| 938 | * guest time. | ||
| 939 | */ | ||
| 940 | guest_exit(); | ||
| 941 | trace_kvm_exit(ret, kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu)); | 956 | trace_kvm_exit(ret, kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu)); |
| 942 | 957 | ||
| 943 | /* Exit types that need handling before we can be preempted */ | 958 | /* Exit types that need handling before we can be preempted */ |
| @@ -228,6 +228,14 @@ int handle_exit(struct kvm_vcpu *vcpu, int exception_index) · 227 unmodified lines | |||
| 228 | { | 228 | { |
| 229 | struct kvm_run *run = vcpu->run; | 229 | struct kvm_run *run = vcpu->run; |
| 230 | 230 | ||
| 231 | if (ARM_SERROR_PENDING(exception_index)) { | ||
| 232 | /* | ||
| 233 | * The SError is handled by handle_exit_early(). If the guest | ||
| 234 | * survives it will re-execute the original instruction. | ||
| 235 | */ | ||
| 236 | return 1; | ||
| 237 | } | ||
| 238 | |||
| 231 | exception_index = ARM_EXCEPTION_CODE(exception_index); | 239 | exception_index = ARM_EXCEPTION_CODE(exception_index); |
| 232 | 240 | ||
| 233 | switch (exception_index) { | 241 | switch (exception_index) { |
| @@ -402,6 +402,24 @@ static inline bool kvm_hyp_handle_exit(struct kvm_vcpu *vcpu, u64 *exit_code) · 401 unmodified lines | |||
| 402 | return false; | 402 | return false; |
| 403 | } | 403 | } |
| 404 | 404 | ||
| 405 | static inline void synchronize_vcpu_pstate(struct kvm_vcpu *vcpu, u64 *exit_code) | ||
| 406 | { | ||
| 407 | /* | ||
| 408 | * Check for the conditions of Cortex-A510's #2077057. When these occur | ||
| 409 | * SPSR_EL2 can't be trusted, but isn't needed either as it is | ||
| 410 | * unchanged from the value in vcpu_gp_regs(vcpu)->pstate. | ||
| 411 | * Are we single-stepping the guest, and took a PAC exception from the | ||
| 412 | * active-not-pending state? | ||
| 413 | */ | ||
| 414 | if (cpus_have_final_cap(ARM64_WORKAROUND_2077057) && | ||
| 415 | vcpu->guest_debug & KVM_GUESTDBG_SINGLESTEP && | ||
| 416 | *vcpu_cpsr(vcpu) & DBG_SPSR_SS && | ||
| 417 | ESR_ELx_EC(read_sysreg_el2(SYS_ESR)) == ESR_ELx_EC_PAC) | ||
| 418 | write_sysreg_el2(*vcpu_cpsr(vcpu), SYS_SPSR); | ||
| 419 | |||
| 420 | vcpu->arch.ctxt.regs.pstate = read_sysreg_el2(SYS_SPSR); | ||
| 421 | } | ||
| 422 | |||
| 405 | /* | 423 | /* |
| 406 | * Return true when we were able to fixup the guest exit and should return to | 424 | * Return true when we were able to fixup the guest exit and should return to |
| 407 | * the guest, false when we should restore the host state and return to the | 425 | * the guest, false when we should restore the host state and return to the |
| @@ -413,7 +431,7 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) · 5 unmodified lines | |||
| 413 | * Save PSTATE early so that we can evaluate the vcpu mode | 431 | * Save PSTATE early so that we can evaluate the vcpu mode |
| 414 | * early on. | 432 | * early on. |
| 415 | */ | 433 | */ |
| 416 | vcpu->arch.ctxt.regs.pstate = read_sysreg_el2(SYS_SPSR); | 434 | synchronize_vcpu_pstate(vcpu, exit_code); |
| 417 | 435 | ||
| 418 | /* | 436 | /* |
| 419 | * Check whether we want to repaint the state one way or | 437 | * Check whether we want to repaint the state one way or |
| @@ -424,7 +442,8 @@ static inline bool fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code) · 4 unmodified lines | |||
| 424 | if (ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ) | 442 | if (ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ) |
| 425 | vcpu->arch.fault.esr_el2 = read_sysreg_el2(SYS_ESR); | 443 | vcpu->arch.fault.esr_el2 = read_sysreg_el2(SYS_ESR); |
| 426 | 444 | ||
| 427 | if (ARM_SERROR_PENDING(*exit_code)) { | 445 | if (ARM_SERROR_PENDING(*exit_code) && |
| 446 | ARM_EXCEPTION_CODE(*exit_code) != ARM_EXCEPTION_IRQ) { | ||
| 428 | u8 esr_ec = kvm_vcpu_trap_get_class(vcpu); | 447 | u8 esr_ec = kvm_vcpu_trap_get_class(vcpu); |
| 429 | 448 | ||
| 430 | /* | 449 | /* |
| @@ -248,6 +248,8 @@ unsigned long vgic_mmio_read_pending(struct kvm_vcpu *vcpu, · 247 unmodified lines | |||
| 248 | IRQCHIP_STATE_PENDING, | 248 | IRQCHIP_STATE_PENDING, |
| 249 | &val); | 249 | &val); |
| 250 | WARN_RATELIMIT(err, "IRQ %d", irq->host_irq); | 250 | WARN_RATELIMIT(err, "IRQ %d", irq->host_irq); |
| 251 | } else if (vgic_irq_is_mapped_level(irq)) { | ||
| 252 | val = vgic_get_phys_line_level(irq); | ||
| 251 | } else { | 253 | } else { |
| 252 | val = irq_is_pending(irq); | 254 | val = irq_is_pending(irq); |
| 253 | } | 255 | } |
| @@ -55,9 +55,10 @@ WORKAROUND_1418040 · 54 unmodified lines | |||
| 55 | WORKAROUND_1463225 | 55 | WORKAROUND_1463225 |
| 56 | WORKAROUND_1508412 | 56 | WORKAROUND_1508412 |
| 57 | WORKAROUND_1542419 | 57 | WORKAROUND_1542419 |
| 58 | WORKAROUND_2064142 | ||
| 59 | WORKAROUND_2038923 | ||
| 60 | WORKAROUND_1902691 | 58 | WORKAROUND_1902691 |
| 59 | WORKAROUND_2038923 | ||
| 60 | WORKAROUND_2064142 | ||
| 61 | WORKAROUND_2077057 | ||
| 61 | WORKAROUND_TRBE_OVERWRITE_FILL_MODE | 62 | WORKAROUND_TRBE_OVERWRITE_FILL_MODE |
| 62 | WORKAROUND_TSB_FLUSH_FAILURE | 63 | WORKAROUND_TSB_FLUSH_FAILURE |
| 63 | WORKAROUND_TRBE_WRITE_OUT_OF_RANGE | 64 | WORKAROUND_TRBE_WRITE_OUT_OF_RANGE |
| @@ -83,6 +83,8 @@ · 82 unmodified lines | |||
| 83 | label = "HDMI OUT"; | 83 | label = "HDMI OUT"; |
| 84 | type = "a"; | 84 | type = "a"; |
| 85 | 85 | ||
| 86 | ddc-en-gpios = <&gpa 25 GPIO_ACTIVE_HIGH>; | ||
| 87 | |||
| 86 | port { | 88 | port { |
| 87 | hdmi_con: endpoint { | 89 | hdmi_con: endpoint { |
| 88 | remote-endpoint = <&dw_hdmi_out>; | 90 | remote-endpoint = <&dw_hdmi_out>; |
| @@ -114,17 +116,6 @@ · 25 unmodified lines | |||
| 114 | gpio = <&gpf 14 GPIO_ACTIVE_LOW>; | 116 | gpio = <&gpf 14 GPIO_ACTIVE_LOW>; |
| 115 | enable-active-high; | 117 | enable-active-high; |
| 116 | }; | 118 | }; |
| 117 | |||
| 118 | hdmi_power: fixedregulator@3 { | ||
| 119 | compatible = "regulator-fixed"; | ||
| 120 | |||
| 121 | regulator-name = "hdmi_power"; | ||
| 122 | regulator-min-microvolt = <5000000>; | ||
| 123 | regulator-max-microvolt = <5000000>; | ||
| 124 | |||
| 125 | gpio = <&gpa 25 0>; | ||
| 126 | enable-active-high; | ||
| 127 | }; | ||
| 128 | }; | 119 | }; |
| 129 | 120 | ||
| 130 | &ext { | 121 | &ext { |
| @@ -576,8 +567,6 @@ · 445 unmodified lines | |||
| 576 | pinctrl-names = "default"; | 567 | pinctrl-names = "default"; |
| 577 | pinctrl-0 = <&pins_hdmi_ddc>; | 568 | pinctrl-0 = <&pins_hdmi_ddc>; |
| 578 | 569 | ||
| 579 | hdmi-5v-supply = <&hdmi_power>; | ||
| 580 | |||
| 581 | ports { | 570 | ports { |
| 582 | #address-cells = <1>; | 571 | #address-cells = <1>; |
| 583 | #size-cells = <0>; | 572 | #size-cells = <0>; |
| @@ -74,7 +74,7 @@ · 73 unmodified lines | |||
| 74 | #define EXC(inst_reg,addr,handler) \ | 74 | #define EXC(inst_reg,addr,handler) \ |
| 75 | 9: inst_reg, addr; \ | 75 | 9: inst_reg, addr; \ |
| 76 | .section __ex_table,"a"; \ | 76 | .section __ex_table,"a"; \ |
| 77 | PTR 9b, handler; \ | 77 | PTR_WD 9b, handler; \ |
| 78 | .previous | 78 | .previous |
| 79 | 79 | ||
| 80 | /* | 80 | /* |
| @@ -414,6 +414,24 @@ int kvm_arch_vcpu_ioctl_set_guest_debug(struct kvm_vcpu *vcpu, · 413 unmodified lines | |||
| 414 | return -ENOIOCTLCMD; | 414 | return -ENOIOCTLCMD; |
| 415 | } | 415 | } |
| 416 | 416 | ||
| 417 | /* | ||
| 418 | * Actually run the vCPU, entering an RCU extended quiescent state (EQS) while | ||
| 419 | * the vCPU is running. | ||
| 420 | * | ||
| 421 | * This must be noinstr as instrumentation may make use of RCU, and this is not | ||
| 422 | * safe during the EQS. | ||
| 423 | */ | ||
| 424 | static int noinstr kvm_mips_vcpu_enter_exit(struct kvm_vcpu *vcpu) | ||
| 425 | { | ||
| 426 | int ret; | ||
| 427 | |||
| 428 | guest_state_enter_irqoff(); | ||
| 429 | ret = kvm_mips_callbacks->vcpu_run(vcpu); | ||
| 430 | guest_state_exit_irqoff(); | ||
| 431 | |||
| 432 | return ret; | ||
| 433 | } | ||
| 434 | |||
| 417 | int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) | 435 | int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) |
| 418 | { | 436 | { |
| 419 | int r = -EINTR; | 437 | int r = -EINTR; |
| @@ -434,7 +452,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 14 unmodified lines | |||
| 434 | lose_fpu(1); | 452 | lose_fpu(1); |
| 435 | 453 | ||
| 436 | local_irq_disable(); | 454 | local_irq_disable(); |
| 437 | guest_enter_irqoff(); | 455 | guest_timing_enter_irqoff(); |
| 438 | trace_kvm_enter(vcpu); | 456 | trace_kvm_enter(vcpu); |
| 439 | 457 | ||
| 440 | /* | 458 | /* |
| @@ -445,10 +463,23 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 4 unmodified lines | |||
| 445 | */ | 463 | */ |
| 446 | smp_store_mb(vcpu->mode, IN_GUEST_MODE); | 464 | smp_store_mb(vcpu->mode, IN_GUEST_MODE); |
| 447 | 465 | ||
| 448 | r = kvm_mips_callbacks->vcpu_run(vcpu); | 466 | r = kvm_mips_vcpu_enter_exit(vcpu); |
| 467 | |||
| 468 | /* | ||
| 469 | * We must ensure that any pending interrupts are taken before | ||
| 470 | * we exit guest timing so that timer ticks are accounted as | ||
| 471 | * guest time. Transiently unmask interrupts so that any | ||
| 472 | * pending interrupts are taken. | ||
| 473 | * | ||
| 474 | * TODO: is there a barrier which ensures that pending interrupts are | ||
| 475 | * recognised? Currently this just hopes that the CPU takes any pending | ||
| 476 | * interrupts between the enable and disable. | ||
| 477 | */ | ||
| 478 | local_irq_enable(); | ||
| 479 | local_irq_disable(); | ||
| 449 | 480 | ||
| 450 | trace_kvm_out(vcpu); | 481 | trace_kvm_out(vcpu); |
| 451 | guest_exit_irqoff(); | 482 | guest_timing_exit_irqoff(); |
| 452 | local_irq_enable(); | 483 | local_irq_enable(); |
| 453 | 484 | ||
| 454 | out: | 485 | out: |
| @@ -1168,7 +1199,7 @@ static void kvm_mips_set_c0_status(void) · 713 unmodified lines | |||
| 1168 | /* | 1199 | /* |
| 1169 | * Return value is in the form (errcode<<2 | RESUME_FLAG_HOST | RESUME_FLAG_NV) | 1200 | * Return value is in the form (errcode<<2 | RESUME_FLAG_HOST | RESUME_FLAG_NV) |
| 1170 | */ | 1201 | */ |
| 1171 | int kvm_mips_handle_exit(struct kvm_vcpu *vcpu) | 1202 | static int __kvm_mips_handle_exit(struct kvm_vcpu *vcpu) |
| 1172 | { | 1203 | { |
| 1173 | struct kvm_run *run = vcpu->run; | 1204 | struct kvm_run *run = vcpu->run; |
| 1174 | u32 cause = vcpu->arch.host_cp0_cause; | 1205 | u32 cause = vcpu->arch.host_cp0_cause; |
| @@ -1357,6 +1388,17 @@ int kvm_mips_handle_exit(struct kvm_vcpu *vcpu) · 182 unmodified lines | |||
| 1357 | return ret; | 1388 | return ret; |
| 1358 | } | 1389 | } |
| 1359 | 1390 | ||
| 1391 | int noinstr kvm_mips_handle_exit(struct kvm_vcpu *vcpu) | ||
| 1392 | { | ||
| 1393 | int ret; | ||
| 1394 | |||
| 1395 | guest_state_exit_irqoff(); | ||
| 1396 | ret = __kvm_mips_handle_exit(vcpu); | ||
| 1397 | guest_state_enter_irqoff(); | ||
| 1398 | |||
| 1399 | return ret; | ||
| 1400 | } | ||
| 1401 | |||
| 1360 | /* Enable FPU for guest and restore context */ | 1402 | /* Enable FPU for guest and restore context */ |
| 1361 | void kvm_own_fpu(struct kvm_vcpu *vcpu) | 1403 | void kvm_own_fpu(struct kvm_vcpu *vcpu) |
| 1362 | { | 1404 | { |
| @@ -458,8 +458,8 @@ void kvm_vz_acquire_htimer(struct kvm_vcpu *vcpu) · 457 unmodified lines | |||
| 458 | /** | 458 | /** |
| 459 | * _kvm_vz_save_htimer() - Switch to software emulation of guest timer. | 459 | * _kvm_vz_save_htimer() - Switch to software emulation of guest timer. |
| 460 | * @vcpu: Virtual CPU. | 460 | * @vcpu: Virtual CPU. |
| 461 | * @compare: Pointer to write compare value to. | 461 | * @out_compare: Pointer to write compare value to. |
| 462 | * @cause: Pointer to write cause value to. | 462 | * @out_cause: Pointer to write cause value to. |
| 463 | * | 463 | * |
| 464 | * Save VZ guest timer state and switch to software emulation of guest CP0 | 464 | * Save VZ guest timer state and switch to software emulation of guest CP0 |
| 465 | * timer. The hard timer must already be in use, so preemption should be | 465 | * timer. The hard timer must already be in use, so preemption should be |
| @@ -1541,11 +1541,14 @@ static int kvm_trap_vz_handle_guest_exit(struct kvm_vcpu *vcpu) · 1075 unmodified lines | |||
| 1541 | } | 1541 | } |
| 1542 | 1542 | ||
| 1543 | /** | 1543 | /** |
| 1544 | * kvm_trap_vz_handle_cop_unusuable() - Guest used unusable coprocessor. | 1544 | * kvm_trap_vz_handle_cop_unusable() - Guest used unusable coprocessor. |
| 1545 | * @vcpu: Virtual CPU context. | 1545 | * @vcpu: Virtual CPU context. |
| 1546 | * | 1546 | * |
| 1547 | * Handle when the guest attempts to use a coprocessor which hasn't been allowed | 1547 | * Handle when the guest attempts to use a coprocessor which hasn't been allowed |
| 1548 | * by the root context. | 1548 | * by the root context. |
| 1549 | * | ||
| 1550 | * Return: value indicating whether to resume the host or the guest | ||
| 1551 | * (RESUME_HOST or RESUME_GUEST) | ||
| 1549 | */ | 1552 | */ |
| 1550 | static int kvm_trap_vz_handle_cop_unusable(struct kvm_vcpu *vcpu) | 1553 | static int kvm_trap_vz_handle_cop_unusable(struct kvm_vcpu *vcpu) |
| 1551 | { | 1554 | { |
| @@ -1592,6 +1595,9 @@ static int kvm_trap_vz_handle_cop_unusable(struct kvm_vcpu *vcpu) · 40 unmodified lines | |||
| 1592 | * | 1595 | * |
| 1593 | * Handle when the guest attempts to use MSA when it is disabled in the root | 1596 | * Handle when the guest attempts to use MSA when it is disabled in the root |
| 1594 | * context. | 1597 | * context. |
| 1598 | * | ||
| 1599 | * Return: value indicating whether to resume the host or the guest | ||
| 1600 | * (RESUME_HOST or RESUME_GUEST) | ||
| 1595 | */ | 1601 | */ |
| 1596 | static int kvm_trap_vz_handle_msa_disabled(struct kvm_vcpu *vcpu) | 1602 | static int kvm_trap_vz_handle_msa_disabled(struct kvm_vcpu *vcpu) |
| 1597 | { | 1603 | { |
| @@ -12,6 +12,14 @@ · 11 unmodified lines | |||
| 12 | #include <asm/barrier.h> | 12 | #include <asm/barrier.h> |
| 13 | #include <linux/atomic.h> | 13 | #include <linux/atomic.h> |
| 14 | 14 | ||
| 15 | /* compiler build environment sanity checks: */ | ||
| 16 | #if !defined(CONFIG_64BIT) && defined(__LP64__) | ||
| 17 | #error "Please use 'ARCH=parisc' to build the 32-bit kernel." | ||
| 18 | #endif | ||
| 19 | #if defined(CONFIG_64BIT) && !defined(__LP64__) | ||
| 20 | #error "Please use 'ARCH=parisc64' to build the 64-bit kernel." | ||
| 21 | #endif | ||
| 22 | |||
| 15 | /* See http://marc.theaimsgroup.com/?t=108826637900003 for discussion | 23 | /* See http://marc.theaimsgroup.com/?t=108826637900003 for discussion |
| 16 | * on use of volatile and __*_bit() (set/clear/change): | 24 | * on use of volatile and __*_bit() (set/clear/change): |
| 17 | * *_bit() want use of volatile. | 25 | * *_bit() want use of volatile. |
| @@ -89,8 +89,8 @@ struct exception_table_entry { · 88 unmodified lines | |||
| 89 | __asm__("1: " ldx " 0(" sr "%2),%0\n" \ | 89 | __asm__("1: " ldx " 0(" sr "%2),%0\n" \ |
| 90 | "9:\n" \ | 90 | "9:\n" \ |
| 91 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ | 91 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ |
| 92 | : "=r"(__gu_val), "=r"(__gu_err) \ | 92 | : "=r"(__gu_val), "+r"(__gu_err) \ |
| 93 | : "r"(ptr), "1"(__gu_err)); \ | 93 | : "r"(ptr)); \ |
| 94 | \ | 94 | \ |
| 95 | (val) = (__force __typeof__(*(ptr))) __gu_val; \ | 95 | (val) = (__force __typeof__(*(ptr))) __gu_val; \ |
| 96 | } | 96 | } |
| @@ -123,8 +123,8 @@ struct exception_table_entry { · 26 unmodified lines | |||
| 123 | "9:\n" \ | 123 | "9:\n" \ |
| 124 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ | 124 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ |
| 125 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(2b, 9b) \ | 125 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(2b, 9b) \ |
| 126 | : "=&r"(__gu_tmp.l), "=r"(__gu_err) \ | 126 | : "=&r"(__gu_tmp.l), "+r"(__gu_err) \ |
| 127 | : "r"(ptr), "1"(__gu_err)); \ | 127 | : "r"(ptr)); \ |
| 128 | \ | 128 | \ |
| 129 | (val) = __gu_tmp.t; \ | 129 | (val) = __gu_tmp.t; \ |
| 130 | } | 130 | } |
| @@ -135,13 +135,12 @@ struct exception_table_entry { · 4 unmodified lines | |||
| 135 | #define __put_user_internal(sr, x, ptr) \ | 135 | #define __put_user_internal(sr, x, ptr) \ |
| 136 | ({ \ | 136 | ({ \ |
| 137 | ASM_EXCEPTIONTABLE_VAR(__pu_err); \ | 137 | ASM_EXCEPTIONTABLE_VAR(__pu_err); \ |
| 138 | __typeof__(*(ptr)) __x = (__typeof__(*(ptr)))(x); \ | ||
| 139 | \ | 138 | \ |
| 140 | switch (sizeof(*(ptr))) { \ | 139 | switch (sizeof(*(ptr))) { \ |
| 141 | case 1: __put_user_asm(sr, "stb", __x, ptr); break; \ | 140 | case 1: __put_user_asm(sr, "stb", x, ptr); break; \ |
| 142 | case 2: __put_user_asm(sr, "sth", __x, ptr); break; \ | 141 | case 2: __put_user_asm(sr, "sth", x, ptr); break; \ |
| 143 | case 4: __put_user_asm(sr, "stw", __x, ptr); break; \ | 142 | case 4: __put_user_asm(sr, "stw", x, ptr); break; \ |
| 144 | case 8: STD_USER(sr, __x, ptr); break; \ | 143 | case 8: STD_USER(sr, x, ptr); break; \ |
| 145 | default: BUILD_BUG(); \ | 144 | default: BUILD_BUG(); \ |
| 146 | } \ | 145 | } \ |
| 147 | \ | 146 | \ |
| @@ -150,7 +149,9 @@ struct exception_table_entry { · 2 unmodified lines | |||
| 150 | 149 | ||
| 151 | #define __put_user(x, ptr) \ | 150 | #define __put_user(x, ptr) \ |
| 152 | ({ \ | 151 | ({ \ |
| 153 | __put_user_internal("%%sr3,", x, ptr); \ | 152 | __typeof__(&*(ptr)) __ptr = ptr; \ |
| 153 | __typeof__(*(__ptr)) __x = (__typeof__(*(__ptr)))(x); \ | ||
| 154 | __put_user_internal("%%sr3,", __x, __ptr); \ | ||
| 154 | }) | 155 | }) |
| 155 | 156 | ||
| 156 | #define __put_kernel_nofault(dst, src, type, err_label) \ | 157 | #define __put_kernel_nofault(dst, src, type, err_label) \ |
| @@ -180,8 +181,8 @@ struct exception_table_entry { · 23 unmodified lines | |||
| 180 | "1: " stx " %2,0(" sr "%1)\n" \ | 181 | "1: " stx " %2,0(" sr "%1)\n" \ |
| 181 | "9:\n" \ | 182 | "9:\n" \ |
| 182 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ | 183 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ |
| 183 | : "=r"(__pu_err) \ | 184 | : "+r"(__pu_err) \ |
| 184 | : "r"(ptr), "r"(x), "0"(__pu_err)) | 185 | : "r"(ptr), "r"(x)) |
| 185 | 186 | ||
| 186 | 187 | ||
| 187 | #if !defined(CONFIG_64BIT) | 188 | #if !defined(CONFIG_64BIT) |
| @@ -193,8 +194,8 @@ struct exception_table_entry { · 5 unmodified lines | |||
| 193 | "9:\n" \ | 194 | "9:\n" \ |
| 194 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ | 195 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(1b, 9b) \ |
| 195 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(2b, 9b) \ | 196 | ASM_EXCEPTIONTABLE_ENTRY_EFAULT(2b, 9b) \ |
| 196 | : "=r"(__pu_err) \ | 197 | : "+r"(__pu_err) \ |
| 197 | : "r"(ptr), "r"(__val), "0"(__pu_err)); \ | 198 | : "r"(ptr), "r"(__val)); \ |
| 198 | } while (0) | 199 | } while (0) |
| 199 | 200 | ||
| 200 | #endif /* !defined(CONFIG_64BIT) */ | 201 | #endif /* !defined(CONFIG_64BIT) */ |
| @@ -340,7 +340,7 @@ static int emulate_stw(struct pt_regs *regs, int frreg, int flop) · 339 unmodified lines | |||
| 340 | : "r" (val), "r" (regs->ior), "r" (regs->isr) | 340 | : "r" (val), "r" (regs->ior), "r" (regs->isr) |
| 341 | : "r19", "r20", "r21", "r22", "r1", FIXUP_BRANCH_CLOBBER ); | 341 | : "r19", "r20", "r21", "r22", "r1", FIXUP_BRANCH_CLOBBER ); |
| 342 | 342 | ||
| 343 | return 0; | 343 | return ret; |
| 344 | } | 344 | } |
| 345 | static int emulate_std(struct pt_regs *regs, int frreg, int flop) | 345 | static int emulate_std(struct pt_regs *regs, int frreg, int flop) |
| 346 | { | 346 | { |
| @@ -397,7 +397,7 @@ static int emulate_std(struct pt_regs *regs, int frreg, int flop) · 50 unmodified lines | |||
| 397 | __asm__ __volatile__ ( | 397 | __asm__ __volatile__ ( |
| 398 | " mtsp %4, %%sr1\n" | 398 | " mtsp %4, %%sr1\n" |
| 399 | " zdep %2, 29, 2, %%r19\n" | 399 | " zdep %2, 29, 2, %%r19\n" |
| 400 | " dep %%r0, 31, 2, %2\n" | 400 | " dep %%r0, 31, 2, %3\n" |
| 401 | " mtsar %%r19\n" | 401 | " mtsar %%r19\n" |
| 402 | " zvdepi -2, 32, %%r19\n" | 402 | " zvdepi -2, 32, %%r19\n" |
| 403 | "1: ldw 0(%%sr1,%3),%%r20\n" | 403 | "1: ldw 0(%%sr1,%3),%%r20\n" |
| @@ -409,7 +409,7 @@ static int emulate_std(struct pt_regs *regs, int frreg, int flop) · 5 unmodified lines | |||
| 409 | " andcm %%r21, %%r19, %%r21\n" | 409 | " andcm %%r21, %%r19, %%r21\n" |
| 410 | " or %1, %%r20, %1\n" | 410 | " or %1, %%r20, %1\n" |
| 411 | " or %2, %%r21, %2\n" | 411 | " or %2, %%r21, %2\n" |
| 412 | "3: stw %1,0(%%sr1,%1)\n" | 412 | "3: stw %1,0(%%sr1,%3)\n" |
| 413 | "4: stw %%r1,4(%%sr1,%3)\n" | 413 | "4: stw %%r1,4(%%sr1,%3)\n" |
| 414 | "5: stw %2,8(%%sr1,%3)\n" | 414 | "5: stw %2,8(%%sr1,%3)\n" |
| 415 | " copy %%r0, %0\n" | 415 | " copy %%r0, %0\n" |
| @@ -596,7 +596,6 @@ void handle_unaligned(struct pt_regs *regs) · 180 unmodified lines | |||
| 596 | ret = ERR_NOTHANDLED; /* "undefined", but lets kill them. */ | 596 | ret = ERR_NOTHANDLED; /* "undefined", but lets kill them. */ |
| 597 | break; | 597 | break; |
| 598 | } | 598 | } |
| 599 | #ifdef CONFIG_PA20 | ||
| 600 | switch (regs->iir & OPCODE2_MASK) | 599 | switch (regs->iir & OPCODE2_MASK) |
| 601 | { | 600 | { |
| 602 | case OPCODE_FLDD_L: | 601 | case OPCODE_FLDD_L: |
| @@ -607,22 +606,23 @@ void handle_unaligned(struct pt_regs *regs) · 4 unmodified lines | |||
| 607 | flop=1; | 606 | flop=1; |
| 608 | ret = emulate_std(regs, R2(regs->iir),1); | 607 | ret = emulate_std(regs, R2(regs->iir),1); |
| 609 | break; | 608 | break; |
| 609 | #ifdef CONFIG_PA20 | ||
| 610 | case OPCODE_LDD_L: | 610 | case OPCODE_LDD_L: |
| 611 | ret = emulate_ldd(regs, R2(regs->iir),0); | 611 | ret = emulate_ldd(regs, R2(regs->iir),0); |
| 612 | break; | 612 | break; |
| 613 | case OPCODE_STD_L: | 613 | case OPCODE_STD_L: |
| 614 | ret = emulate_std(regs, R2(regs->iir),0); | 614 | ret = emulate_std(regs, R2(regs->iir),0); |
| 615 | break; | 615 | break; |
| 616 | } | ||
| 617 | #endif | 616 | #endif |
| 617 | } | ||
| 618 | switch (regs->iir & OPCODE3_MASK) | 618 | switch (regs->iir & OPCODE3_MASK) |
| 619 | { | 619 | { |
| 620 | case OPCODE_FLDW_L: | 620 | case OPCODE_FLDW_L: |
| 621 | flop=1; | 621 | flop=1; |
| 622 | ret = emulate_ldw(regs, R2(regs->iir),0); | 622 | ret = emulate_ldw(regs, R2(regs->iir), 1); |
| 623 | break; | 623 | break; |
| 624 | case OPCODE_LDW_M: | 624 | case OPCODE_LDW_M: |
| 625 | ret = emulate_ldw(regs, R2(regs->iir),1); | 625 | ret = emulate_ldw(regs, R2(regs->iir), 0); |
| 626 | break; | 626 | break; |
| 627 | 627 | ||
| 628 | case OPCODE_FSTW_L: | 628 | case OPCODE_FSTW_L: |
| @@ -346,6 +346,16 @@ u64 ioread64be(const void __iomem *addr) · 345 unmodified lines | |||
| 346 | return *((u64 *)addr); | 346 | return *((u64 *)addr); |
| 347 | } | 347 | } |
| 348 | 348 | ||
| 349 | u64 ioread64_lo_hi(const void __iomem *addr) | ||
| 350 | { | ||
| 351 | u32 low, high; | ||
| 352 | |||
| 353 | low = ioread32(addr); | ||
| 354 | high = ioread32(addr + sizeof(u32)); | ||
| 355 | |||
| 356 | return low + ((u64)high << 32); | ||
| 357 | } | ||
| 358 | |||
| 349 | u64 ioread64_hi_lo(const void __iomem *addr) | 359 | u64 ioread64_hi_lo(const void __iomem *addr) |
| 350 | { | 360 | { |
| 351 | u32 low, high; | 361 | u32 low, high; |
| @@ -419,6 +429,12 @@ void iowrite64be(u64 datum, void __iomem *addr) · 67 unmodified lines | |||
| 419 | } | 429 | } |
| 420 | } | 430 | } |
| 421 | 431 | ||
| 432 | void iowrite64_lo_hi(u64 val, void __iomem *addr) | ||
| 433 | { | ||
| 434 | iowrite32(val, addr); | ||
| 435 | iowrite32(val >> 32, addr + sizeof(u32)); | ||
| 436 | } | ||
| 437 | |||
| 422 | void iowrite64_hi_lo(u64 val, void __iomem *addr) | 438 | void iowrite64_hi_lo(u64 val, void __iomem *addr) |
| 423 | { | 439 | { |
| 424 | iowrite32(val >> 32, addr + sizeof(u32)); | 440 | iowrite32(val >> 32, addr + sizeof(u32)); |
| @@ -530,6 +546,7 @@ EXPORT_SYMBOL(ioread32); · 105 unmodified lines | |||
| 530 | EXPORT_SYMBOL(ioread32be); | 546 | EXPORT_SYMBOL(ioread32be); |
| 531 | EXPORT_SYMBOL(ioread64); | 547 | EXPORT_SYMBOL(ioread64); |
| 532 | EXPORT_SYMBOL(ioread64be); | 548 | EXPORT_SYMBOL(ioread64be); |
| 549 | EXPORT_SYMBOL(ioread64_lo_hi); | ||
| 533 | EXPORT_SYMBOL(ioread64_hi_lo); | 550 | EXPORT_SYMBOL(ioread64_hi_lo); |
| 534 | EXPORT_SYMBOL(iowrite8); | 551 | EXPORT_SYMBOL(iowrite8); |
| 535 | EXPORT_SYMBOL(iowrite16); | 552 | EXPORT_SYMBOL(iowrite16); |
| @@ -538,6 +555,7 @@ EXPORT_SYMBOL(iowrite32); · 2 unmodified lines | |||
| 538 | EXPORT_SYMBOL(iowrite32be); | 555 | EXPORT_SYMBOL(iowrite32be); |
| 539 | EXPORT_SYMBOL(iowrite64); | 556 | EXPORT_SYMBOL(iowrite64); |
| 540 | EXPORT_SYMBOL(iowrite64be); | 557 | EXPORT_SYMBOL(iowrite64be); |
| 558 | EXPORT_SYMBOL(iowrite64_lo_hi); | ||
| 541 | EXPORT_SYMBOL(iowrite64_hi_lo); | 559 | EXPORT_SYMBOL(iowrite64_hi_lo); |
| 542 | EXPORT_SYMBOL(ioread8_rep); | 560 | EXPORT_SYMBOL(ioread8_rep); |
| 543 | EXPORT_SYMBOL(ioread16_rep); | 561 | EXPORT_SYMBOL(ioread16_rep); |
| @@ -337,9 +337,9 @@ static void __init setup_bootmem(void) · 336 unmodified lines | |||
| 337 | 337 | ||
| 338 | static bool kernel_set_to_readonly; | 338 | static bool kernel_set_to_readonly; |
| 339 | 339 | ||
| 340 | static void __init map_pages(unsigned long start_vaddr, | 340 | static void __ref map_pages(unsigned long start_vaddr, |
| 341 | unsigned long start_paddr, unsigned long size, | 341 | unsigned long start_paddr, unsigned long size, |
| 342 | pgprot_t pgprot, int force) | 342 | pgprot_t pgprot, int force) |
| 343 | { | 343 | { |
| 344 | pmd_t *pmd; | 344 | pmd_t *pmd; |
| 345 | pte_t *pg_table; | 345 | pte_t *pg_table; |
| @@ -449,7 +449,7 @@ void __init set_kernel_text_rw(int enable_read_write) · 103 unmodified lines | |||
| 449 | flush_tlb_all(); | 449 | flush_tlb_all(); |
| 450 | } | 450 | } |
| 451 | 451 | ||
| 452 | void __ref free_initmem(void) | 452 | void free_initmem(void) |
| 453 | { | 453 | { |
| 454 | unsigned long init_begin = (unsigned long)__init_begin; | 454 | unsigned long init_begin = (unsigned long)__init_begin; |
| 455 | unsigned long init_end = (unsigned long)__init_end; | 455 | unsigned long init_end = (unsigned long)__init_end; |
| @@ -463,7 +463,6 @@ void __ref free_initmem(void) · 7 unmodified lines | |||
| 463 | /* The init text pages are marked R-X. We have to | 463 | /* The init text pages are marked R-X. We have to |
| 464 | * flush the icache and mark them RW- | 464 | * flush the icache and mark them RW- |
| 465 | * | 465 | * |
| 466 | * This is tricky, because map_pages is in the init section. | ||
| 467 | * Do a dummy remap of the data section first (the data | 466 | * Do a dummy remap of the data section first (the data |
| 468 | * section is already PAGE_KERNEL) to pull in the TLB entries | 467 | * section is already PAGE_KERNEL) to pull in the TLB entries |
| 469 | * for map_kernel */ | 468 | * for map_kernel */ |
| @@ -421,14 +421,14 @@ InstructionTLBMiss: · 420 unmodified lines | |||
| 421 | */ | 421 | */ |
| 422 | /* Get PTE (linux-style) and check access */ | 422 | /* Get PTE (linux-style) and check access */ |
| 423 | mfspr r3,SPRN_IMISS | 423 | mfspr r3,SPRN_IMISS |
| 424 | #ifdef CONFIG_MODULES | 424 | #if defined(CONFIG_MODULES) || defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) |
| 425 | lis r1, TASK_SIZE@h /* check if kernel address */ | 425 | lis r1, TASK_SIZE@h /* check if kernel address */ |
| 426 | cmplw 0,r1,r3 | 426 | cmplw 0,r1,r3 |
| 427 | #endif | 427 | #endif |
| 428 | mfspr r2, SPRN_SDR1 | 428 | mfspr r2, SPRN_SDR1 |
| 429 | li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | _PAGE_USER | 429 | li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | _PAGE_USER |
| 430 | rlwinm r2, r2, 28, 0xfffff000 | 430 | rlwinm r2, r2, 28, 0xfffff000 |
| 431 | #ifdef CONFIG_MODULES | 431 | #if defined(CONFIG_MODULES) || defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) |
| 432 | bgt- 112f | 432 | bgt- 112f |
| 433 | lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */ | 433 | lis r2, (swapper_pg_dir - PAGE_OFFSET)@ha /* if kernel address, use */ |
| 434 | li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC | 434 | li r1,_PAGE_PRESENT | _PAGE_ACCESSED | _PAGE_EXEC |
| @@ -3264,12 +3264,14 @@ void emulate_update_regs(struct pt_regs *regs, struct instruction_op *op) · 3263 unmodified lines | |||
| 3264 | case BARRIER_EIEIO: | 3264 | case BARRIER_EIEIO: |
| 3265 | eieio(); | 3265 | eieio(); |
| 3266 | break; | 3266 | break; |
| 3267 | #ifdef CONFIG_PPC64 | ||
| 3267 | case BARRIER_LWSYNC: | 3268 | case BARRIER_LWSYNC: |
| 3268 | asm volatile("lwsync" : : : "memory"); | 3269 | asm volatile("lwsync" : : : "memory"); |
| 3269 | break; | 3270 | break; |
| 3270 | case BARRIER_PTESYNC: | 3271 | case BARRIER_PTESYNC: |
| 3271 | asm volatile("ptesync" : : : "memory"); | 3272 | asm volatile("ptesync" : : : "memory"); |
| 3272 | break; | 3273 | break; |
| 3274 | #endif | ||
| 3273 | } | 3275 | } |
| 3274 | break; | 3276 | break; |
| 3275 | 3277 | ||
| @@ -50,6 +50,12 @@ riscv-march-$(CONFIG_ARCH_RV32I) := rv32ima · 49 unmodified lines | |||
| 50 | riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima | 50 | riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima |
| 51 | riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd | 51 | riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd |
| 52 | riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c | 52 | riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c |
| 53 | |||
| 54 | # Newer binutils versions default to ISA spec version 20191213 which moves some | ||
| 55 | # instructions from the I extension to the Zicsr and Zifencei extensions. | ||
| 56 | toolchain-need-zicsr-zifencei := $(call cc-option-yn, -march=$(riscv-march-y)_zicsr_zifencei) | ||
| 57 | riscv-march-$(toolchain-need-zicsr-zifencei) := $(riscv-march-y)_zicsr_zifencei | ||
| 58 | |||
| 53 | KBUILD_CFLAGS += -march=$(subst fd,,$(riscv-march-y)) | 59 | KBUILD_CFLAGS += -march=$(subst fd,,$(riscv-march-y)) |
| 54 | KBUILD_AFLAGS += -march=$(riscv-march-y) | 60 | KBUILD_AFLAGS += -march=$(riscv-march-y) |
| 55 | 61 | ||
| @@ -23,7 +23,7 @@ CONFIG_SLOB=y · 22 unmodified lines | |||
| 23 | CONFIG_SOC_CANAAN=y | 23 | CONFIG_SOC_CANAAN=y |
| 24 | CONFIG_SMP=y | 24 | CONFIG_SMP=y |
| 25 | CONFIG_NR_CPUS=2 | 25 | CONFIG_NR_CPUS=2 |
| 26 | CONFIG_CMDLINE="earlycon console=ttySIF0 rootdelay=2 root=/dev/mmcblk0p1 ro" | 26 | CONFIG_CMDLINE="earlycon console=ttySIF0 root=/dev/mmcblk0p1 rootwait ro" |
| 27 | CONFIG_CMDLINE_FORCE=y | 27 | CONFIG_CMDLINE_FORCE=y |
| 28 | # CONFIG_SECCOMP is not set | 28 | # CONFIG_SECCOMP is not set |
| 29 | # CONFIG_STACKPROTECTOR is not set | 29 | # CONFIG_STACKPROTECTOR is not set |
| @@ -51,6 +51,8 @@ obj-$(CONFIG_MODULE_SECTIONS) += module-sections.o · 50 unmodified lines | |||
| 51 | obj-$(CONFIG_FUNCTION_TRACER) += mcount.o ftrace.o | 51 | obj-$(CONFIG_FUNCTION_TRACER) += mcount.o ftrace.o |
| 52 | obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o | 52 | obj-$(CONFIG_DYNAMIC_FTRACE) += mcount-dyn.o |
| 53 | 53 | ||
| 54 | obj-$(CONFIG_TRACE_IRQFLAGS) += trace_irq.o | ||
| 55 | |||
| 54 | obj-$(CONFIG_RISCV_BASE_PMU) += perf_event.o | 56 | obj-$(CONFIG_RISCV_BASE_PMU) += perf_event.o |
| 55 | obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o | 57 | obj-$(CONFIG_PERF_EVENTS) += perf_callchain.o |
| 56 | obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o | 58 | obj-$(CONFIG_HAVE_PERF_REGS) += perf_regs.o |
| @@ -12,6 +12,7 @@ · 11 unmodified lines | |||
| 12 | #include <linux/sched/hotplug.h> | 12 | #include <linux/sched/hotplug.h> |
| 13 | #include <asm/irq.h> | 13 | #include <asm/irq.h> |
| 14 | #include <asm/cpu_ops.h> | 14 | #include <asm/cpu_ops.h> |
| 15 | #include <asm/numa.h> | ||
| 15 | #include <asm/sbi.h> | 16 | #include <asm/sbi.h> |
| 16 | 17 | ||
| 17 | bool cpu_has_hotplug(unsigned int cpu) | 18 | bool cpu_has_hotplug(unsigned int cpu) |
| @@ -40,6 +41,7 @@ int __cpu_disable(void) · 22 unmodified lines | |||
| 40 | return ret; | 41 | return ret; |
| 41 | 42 | ||
| 42 | remove_cpu_topology(cpu); | 43 | remove_cpu_topology(cpu); |
| 44 | numa_remove_cpu(cpu); | ||
| 43 | set_cpu_online(cpu, false); | 45 | set_cpu_online(cpu, false); |
| 44 | irq_migrate_all_off_this_cpu(); | 46 | irq_migrate_all_off_this_cpu(); |
| 45 | 47 | ||
| @@ -108,7 +108,7 @@ _save_context: · 107 unmodified lines | |||
| 108 | .option pop | 108 | .option pop |
| 109 | 109 | ||
| 110 | #ifdef CONFIG_TRACE_IRQFLAGS | 110 | #ifdef CONFIG_TRACE_IRQFLAGS |
| 111 | call trace_hardirqs_off | 111 | call __trace_hardirqs_off |
| 112 | #endif | 112 | #endif |
| 113 | 113 | ||
| 114 | #ifdef CONFIG_CONTEXT_TRACKING | 114 | #ifdef CONFIG_CONTEXT_TRACKING |
| @@ -143,7 +143,7 @@ skip_context_tracking: · 28 unmodified lines | |||
| 143 | li t0, EXC_BREAKPOINT | 143 | li t0, EXC_BREAKPOINT |
| 144 | beq s4, t0, 1f | 144 | beq s4, t0, 1f |
| 145 | #ifdef CONFIG_TRACE_IRQFLAGS | 145 | #ifdef CONFIG_TRACE_IRQFLAGS |
| 146 | call trace_hardirqs_on | 146 | call __trace_hardirqs_on |
| 147 | #endif | 147 | #endif |
| 148 | csrs CSR_STATUS, SR_IE | 148 | csrs CSR_STATUS, SR_IE |
| 149 | 149 | ||
| @@ -234,7 +234,7 @@ ret_from_exception: · 84 unmodified lines | |||
| 234 | REG_L s0, PT_STATUS(sp) | 234 | REG_L s0, PT_STATUS(sp) |
| 235 | csrc CSR_STATUS, SR_IE | 235 | csrc CSR_STATUS, SR_IE |
| 236 | #ifdef CONFIG_TRACE_IRQFLAGS | 236 | #ifdef CONFIG_TRACE_IRQFLAGS |
| 237 | call trace_hardirqs_off | 237 | call __trace_hardirqs_off |
| 238 | #endif | 238 | #endif |
| 239 | #ifdef CONFIG_RISCV_M_MODE | 239 | #ifdef CONFIG_RISCV_M_MODE |
| 240 | /* the MPP value is too large to be used as an immediate arg for addi */ | 240 | /* the MPP value is too large to be used as an immediate arg for addi */ |
| @@ -270,10 +270,10 @@ restore_all: · 29 unmodified lines | |||
| 270 | REG_L s1, PT_STATUS(sp) | 270 | REG_L s1, PT_STATUS(sp) |
| 271 | andi t0, s1, SR_PIE | 271 | andi t0, s1, SR_PIE |
| 272 | beqz t0, 1f | 272 | beqz t0, 1f |
| 273 | call trace_hardirqs_on | 273 | call __trace_hardirqs_on |
| 274 | j 2f | 274 | j 2f |
| 275 | 1: | 275 | 1: |
| 276 | call trace_hardirqs_off | 276 | call __trace_hardirqs_off |
| 277 | 2: | 277 | 2: |
| 278 | #endif | 278 | #endif |
| 279 | REG_L a0, PT_STATUS(sp) | 279 | REG_L a0, PT_STATUS(sp) |
| @@ -22,14 +22,13 @@ · 21 unmodified lines | |||
| 22 | add \reg, \reg, t0 | 22 | add \reg, \reg, t0 |
| 23 | .endm | 23 | .endm |
| 24 | .macro XIP_FIXUP_FLASH_OFFSET reg | 24 | .macro XIP_FIXUP_FLASH_OFFSET reg |
| 25 | la t1, __data_loc | 25 | la t0, __data_loc |
| 26 | li t0, XIP_OFFSET_MASK | 26 | REG_L t1, _xip_phys_offset |
| 27 | and t1, t1, t0 | 27 | sub \reg, \reg, t1 |
| 28 | li t1, XIP_OFFSET | 28 | add \reg, \reg, t0 |
| 29 | sub t0, t0, t1 | ||
| 30 | sub \reg, \reg, t0 | ||
| 31 | .endm | 29 | .endm |
| 32 | _xip_fixup: .dword CONFIG_PHYS_RAM_BASE - CONFIG_XIP_PHYS_ADDR - XIP_OFFSET | 30 | _xip_fixup: .dword CONFIG_PHYS_RAM_BASE - CONFIG_XIP_PHYS_ADDR - XIP_OFFSET |
| 31 | _xip_phys_offset: .dword CONFIG_XIP_PHYS_ADDR + XIP_OFFSET | ||
| 33 | #else | 32 | #else |
| 34 | .macro XIP_FIXUP_OFFSET reg | 33 | .macro XIP_FIXUP_OFFSET reg |
| 35 | .endm | 34 | .endm |
| @@ -5,6 +5,7 @@ · 4 unmodified lines | |||
| 5 | * Copyright (c) 2020 Western Digital Corporation or its affiliates. | 5 | * Copyright (c) 2020 Western Digital Corporation or its affiliates. |
| 6 | */ | 6 | */ |
| 7 | 7 | ||
| 8 | #include <linux/bits.h> | ||
| 8 | #include <linux/init.h> | 9 | #include <linux/init.h> |
| 9 | #include <linux/pm.h> | 10 | #include <linux/pm.h> |
| 10 | #include <linux/reboot.h> | 11 | #include <linux/reboot.h> |
| @@ -85,7 +86,7 @@ static unsigned long __sbi_v01_cpumask_to_hartmask(const struct cpumask *cpu_mas · 74 unmodified lines | |||
| 85 | pr_warn("Unable to send any request to hartid > BITS_PER_LONG for SBI v0.1\n"); | 86 | pr_warn("Unable to send any request to hartid > BITS_PER_LONG for SBI v0.1\n"); |
| 86 | break; | 87 | break; |
| 87 | } | 88 | } |
| 88 | hmask |= 1 << hartid; | 89 | hmask |= BIT(hartid); |
| 89 | } | 90 | } |
| 90 | 91 | ||
| 91 | return hmask; | 92 | return hmask; |
| @@ -160,7 +161,7 @@ static int __sbi_send_ipi_v01(const struct cpumask *cpu_mask) · 68 unmodified lines | |||
| 160 | { | 161 | { |
| 161 | unsigned long hart_mask; | 162 | unsigned long hart_mask; |
| 162 | 163 | ||
| 163 | if (!cpu_mask) | 164 | if (!cpu_mask || cpumask_empty(cpu_mask)) |
| 164 | cpu_mask = cpu_online_mask; | 165 | cpu_mask = cpu_online_mask; |
| 165 | hart_mask = __sbi_v01_cpumask_to_hartmask(cpu_mask); | 166 | hart_mask = __sbi_v01_cpumask_to_hartmask(cpu_mask); |
| 166 | 167 | ||
| @@ -176,7 +177,7 @@ static int __sbi_rfence_v01(int fid, const struct cpumask *cpu_mask, · 9 unmodified lines | |||
| 176 | int result = 0; | 177 | int result = 0; |
| 177 | unsigned long hart_mask; | 178 | unsigned long hart_mask; |
| 178 | 179 | ||
| 179 | if (!cpu_mask) | 180 | if (!cpu_mask || cpumask_empty(cpu_mask)) |
| 180 | cpu_mask = cpu_online_mask; | 181 | cpu_mask = cpu_online_mask; |
| 181 | hart_mask = __sbi_v01_cpumask_to_hartmask(cpu_mask); | 182 | hart_mask = __sbi_v01_cpumask_to_hartmask(cpu_mask); |
| 182 | 183 | ||
| @@ -249,26 +250,37 @@ static void __sbi_set_timer_v02(uint64_t stime_value) · 66 unmodified lines | |||
| 249 | 250 | ||
| 250 | static int __sbi_send_ipi_v02(const struct cpumask *cpu_mask) | 251 | static int __sbi_send_ipi_v02(const struct cpumask *cpu_mask) |
| 251 | { | 252 | { |
| 252 | unsigned long hartid, cpuid, hmask = 0, hbase = 0; | 253 | unsigned long hartid, cpuid, hmask = 0, hbase = 0, htop = 0; |
| 253 | struct sbiret ret = {0}; | 254 | struct sbiret ret = {0}; |
| 254 | int result; | 255 | int result; |
| 255 | 256 | ||
| 256 | if (!cpu_mask) | 257 | if (!cpu_mask || cpumask_empty(cpu_mask)) |
| 257 | cpu_mask = cpu_online_mask; | 258 | cpu_mask = cpu_online_mask; |
| 258 | 259 | ||
| 259 | for_each_cpu(cpuid, cpu_mask) { | 260 | for_each_cpu(cpuid, cpu_mask) { |
| 260 | hartid = cpuid_to_hartid_map(cpuid); | 261 | hartid = cpuid_to_hartid_map(cpuid); |
| 261 | if (hmask && ((hbase + BITS_PER_LONG) <= hartid)) { | 262 | if (hmask) { |
| 262 | ret = sbi_ecall(SBI_EXT_IPI, SBI_EXT_IPI_SEND_IPI, | 263 | if (hartid + BITS_PER_LONG <= htop || |
| 263 | hmask, hbase, 0, 0, 0, 0); | 264 | hbase + BITS_PER_LONG <= hartid) { |
| 264 | if (ret.error) | 265 | ret = sbi_ecall(SBI_EXT_IPI, |
| 265 | goto ecall_failed; | 266 | SBI_EXT_IPI_SEND_IPI, hmask, |
| 266 | hmask = 0; | 267 | hbase, 0, 0, 0, 0); |
| 267 | hbase = 0; | 268 | if (ret.error) |
| 269 | goto ecall_failed; | ||
| 270 | hmask = 0; | ||
| 271 | } else if (hartid < hbase) { | ||
| 272 | /* shift the mask to fit lower hartid */ | ||
| 273 | hmask <<= hbase - hartid; | ||
| 274 | hbase = hartid; | ||
| 275 | } | ||
| 268 | } | 276 | } |
| 269 | if (!hmask) | 277 | if (!hmask) { |
| 270 | hbase = hartid; | 278 | hbase = hartid; |
| 271 | hmask |= 1UL << (hartid - hbase); | 279 | htop = hartid; |
| 280 | } else if (hartid > htop) { | ||
| 281 | htop = hartid; | ||
| 282 | } | ||
| 283 | hmask |= BIT(hartid - hbase); | ||
| 272 | } | 284 | } |
| 273 | 285 | ||
| 274 | if (hmask) { | 286 | if (hmask) { |
| @@ -344,25 +356,35 @@ static int __sbi_rfence_v02(int fid, const struct cpumask *cpu_mask, · 69 unmodified lines | |||
| 344 | unsigned long start, unsigned long size, | 356 | unsigned long start, unsigned long size, |
| 345 | unsigned long arg4, unsigned long arg5) | 357 | unsigned long arg4, unsigned long arg5) |
| 346 | { | 358 | { |
| 347 | unsigned long hartid, cpuid, hmask = 0, hbase = 0; | 359 | unsigned long hartid, cpuid, hmask = 0, hbase = 0, htop = 0; |
| 348 | int result; | 360 | int result; |
| 349 | 361 | ||
| 350 | if (!cpu_mask) | 362 | if (!cpu_mask || cpumask_empty(cpu_mask)) |
| 351 | cpu_mask = cpu_online_mask; | 363 | cpu_mask = cpu_online_mask; |
| 352 | 364 | ||
| 353 | for_each_cpu(cpuid, cpu_mask) { | 365 | for_each_cpu(cpuid, cpu_mask) { |
| 354 | hartid = cpuid_to_hartid_map(cpuid); | 366 | hartid = cpuid_to_hartid_map(cpuid); |
| 355 | if (hmask && ((hbase + BITS_PER_LONG) <= hartid)) { | 367 | if (hmask) { |
| 356 | result = __sbi_rfence_v02_call(fid, hmask, hbase, | 368 | if (hartid + BITS_PER_LONG <= htop || |
| 357 | start, size, arg4, arg5); | 369 | hbase + BITS_PER_LONG <= hartid) { |
| 358 | if (result) | 370 | result = __sbi_rfence_v02_call(fid, hmask, |
| 359 | return result; | 371 | hbase, start, size, arg4, arg5); |
| 360 | hmask = 0; | 372 | if (result) |
| 361 | hbase = 0; | 373 | return result; |
| 374 | hmask = 0; | ||
| 375 | } else if (hartid < hbase) { | ||
| 376 | /* shift the mask to fit lower hartid */ | ||
| 377 | hmask <<= hbase - hartid; | ||
| 378 | hbase = hartid; | ||
| 379 | } | ||
| 362 | } | 380 | } |
| 363 | if (!hmask) | 381 | if (!hmask) { |
| 364 | hbase = hartid; | 382 | hbase = hartid; |
| 365 | hmask |= 1UL << (hartid - hbase); | 383 | htop = hartid; |
| 384 | } else if (hartid > htop) { | ||
| 385 | htop = hartid; | ||
| 386 | } | ||
| 387 | hmask |= BIT(hartid - hbase); | ||
| 366 | } | 388 | } |
| 367 | 389 | ||
| 368 | if (hmask) { | 390 | if (hmask) { |
| @@ -22,15 +22,16 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs, · 21 unmodified lines | |||
| 22 | bool (*fn)(void *, unsigned long), void *arg) | 22 | bool (*fn)(void *, unsigned long), void *arg) |
| 23 | { | 23 | { |
| 24 | unsigned long fp, sp, pc; | 24 | unsigned long fp, sp, pc; |
| 25 | int level = 0; | ||
| 25 | 26 | ||
| 26 | if (regs) { | 27 | if (regs) { |
| 27 | fp = frame_pointer(regs); | 28 | fp = frame_pointer(regs); |
| 28 | sp = user_stack_pointer(regs); | 29 | sp = user_stack_pointer(regs); |
| 29 | pc = instruction_pointer(regs); | 30 | pc = instruction_pointer(regs); |
| 30 | } else if (task == NULL || task == current) { | 31 | } else if (task == NULL || task == current) { |
| 31 | fp = (unsigned long)__builtin_frame_address(1); | 32 | fp = (unsigned long)__builtin_frame_address(0); |
| 32 | sp = (unsigned long)__builtin_frame_address(0); | 33 | sp = sp_in_global; |
| 33 | pc = (unsigned long)__builtin_return_address(0); | 34 | pc = (unsigned long)walk_stackframe; |
| 34 | } else { | 35 | } else { |
| 35 | /* task blocked in __switch_to */ | 36 | /* task blocked in __switch_to */ |
| 36 | fp = task->thread.s[0]; | 37 | fp = task->thread.s[0]; |
| @@ -42,7 +43,7 @@ void notrace walk_stackframe(struct task_struct *task, struct pt_regs *regs, · 5 unmodified lines | |||
| 42 | unsigned long low, high; | 43 | unsigned long low, high; |
| 43 | struct stackframe *frame; | 44 | struct stackframe *frame; |
| 44 | 45 | ||
| 45 | if (unlikely(!__kernel_text_address(pc) || !fn(arg, pc))) | 46 | if (unlikely(!__kernel_text_address(pc) || (level++ >= 1 && !fn(arg, pc)))) |
| 46 | break; | 47 | break; |
| 47 | 48 | ||
| 48 | /* Validate frame pointer */ | 49 | /* Validate frame pointer */ |
| @@ -0,0 +1,27 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0 | ||
| 2 | /* | ||
| 3 | * Copyright (C) 2022 Changbin Du <changbin.du@gmail.com> | ||
| 4 | */ | ||
| 5 | |||
| 6 | #include <linux/irqflags.h> | ||
| 7 | #include <linux/kprobes.h> | ||
| 8 | #include "trace_irq.h" | ||
| 9 | |||
| 10 | /* | ||
| 11 | * trace_hardirqs_on/off require the caller to setup frame pointer properly. | ||
| 12 | * Otherwise, CALLER_ADDR1 might trigger an pagging exception in kernel. | ||
| 13 | * Here we add one extra level so they can be safely called by low | ||
| 14 | * level entry code which $fp is used for other purpose. | ||
| 15 | */ | ||
| 16 | |||
| 17 | void __trace_hardirqs_on(void) | ||
| 18 | { | ||
| 19 | trace_hardirqs_on(); | ||
| 20 | } | ||
| 21 | NOKPROBE_SYMBOL(__trace_hardirqs_on); | ||
| 22 | |||
| 23 | void __trace_hardirqs_off(void) | ||
| 24 | { | ||
| 25 | trace_hardirqs_off(); | ||
| 26 | } | ||
| 27 | NOKPROBE_SYMBOL(__trace_hardirqs_off); | ||
| @@ -0,0 +1,11 @@ | |||
| 1 | /* SPDX-License-Identifier: GPL-2.0 */ | ||
| 2 | /* | ||
| 3 | * Copyright (C) 2022 Changbin Du <changbin.du@gmail.com> | ||
| 4 | */ | ||
| 5 | #ifndef __TRACE_IRQ_H | ||
| 6 | #define __TRACE_IRQ_H | ||
| 7 | |||
| 8 | void __trace_hardirqs_on(void); | ||
| 9 | void __trace_hardirqs_off(void); | ||
| 10 | |||
| 11 | #endif /* __TRACE_IRQ_H */ | ||
| @@ -90,6 +90,7 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) · 89 unmodified lines | |||
| 90 | int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) | 90 | int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) |
| 91 | { | 91 | { |
| 92 | struct kvm_cpu_context *cntx; | 92 | struct kvm_cpu_context *cntx; |
| 93 | struct kvm_vcpu_csr *reset_csr = &vcpu->arch.guest_reset_csr; | ||
| 93 | 94 | ||
| 94 | /* Mark this VCPU never ran */ | 95 | /* Mark this VCPU never ran */ |
| 95 | vcpu->arch.ran_atleast_once = false; | 96 | vcpu->arch.ran_atleast_once = false; |
| @@ -106,6 +107,9 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) · 10 unmodified lines | |||
| 106 | cntx->hstatus |= HSTATUS_SPVP; | 107 | cntx->hstatus |= HSTATUS_SPVP; |
| 107 | cntx->hstatus |= HSTATUS_SPV; | 108 | cntx->hstatus |= HSTATUS_SPV; |
| 108 | 109 | ||
| 110 | /* By default, make CY, TM, and IR counters accessible in VU mode */ | ||
| 111 | reset_csr->scounteren = 0x7; | ||
| 112 | |||
| 109 | /* Setup VCPU timer */ | 113 | /* Setup VCPU timer */ |
| 110 | kvm_riscv_vcpu_timer_init(vcpu); | 114 | kvm_riscv_vcpu_timer_init(vcpu); |
| 111 | 115 | ||
| @@ -699,6 +703,20 @@ static void kvm_riscv_update_hvip(struct kvm_vcpu *vcpu) · 587 unmodified lines | |||
| 699 | csr_write(CSR_HVIP, csr->hvip); | 703 | csr_write(CSR_HVIP, csr->hvip); |
| 700 | } | 704 | } |
| 701 | 705 | ||
| 706 | /* | ||
| 707 | * Actually run the vCPU, entering an RCU extended quiescent state (EQS) while | ||
| 708 | * the vCPU is running. | ||
| 709 | * | ||
| 710 | * This must be noinstr as instrumentation may make use of RCU, and this is not | ||
| 711 | * safe during the EQS. | ||
| 712 | */ | ||
| 713 | static void noinstr kvm_riscv_vcpu_enter_exit(struct kvm_vcpu *vcpu) | ||
| 714 | { | ||
| 715 | guest_state_enter_irqoff(); | ||
| 716 | __kvm_riscv_switch_to(&vcpu->arch); | ||
| 717 | guest_state_exit_irqoff(); | ||
| 718 | } | ||
| 719 | |||
| 702 | int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) | 720 | int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) |
| 703 | { | 721 | { |
| 704 | int ret; | 722 | int ret; |
| @@ -790,9 +808,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 85 unmodified lines | |||
| 790 | continue; | 808 | continue; |
| 791 | } | 809 | } |
| 792 | 810 | ||
| 793 | guest_enter_irqoff(); | 811 | guest_timing_enter_irqoff(); |
| 794 | 812 | ||
| 795 | __kvm_riscv_switch_to(&vcpu->arch); | 813 | kvm_riscv_vcpu_enter_exit(vcpu); |
| 796 | 814 | ||
| 797 | vcpu->mode = OUTSIDE_GUEST_MODE; | 815 | vcpu->mode = OUTSIDE_GUEST_MODE; |
| 798 | vcpu->stat.exits++; | 816 | vcpu->stat.exits++; |
| @@ -812,25 +830,21 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) · 13 unmodified lines | |||
| 812 | kvm_riscv_vcpu_sync_interrupts(vcpu); | 830 | kvm_riscv_vcpu_sync_interrupts(vcpu); |
| 813 | 831 | ||
| 814 | /* | 832 | /* |
| 815 | * We may have taken a host interrupt in VS/VU-mode (i.e. | 833 | * We must ensure that any pending interrupts are taken before |
| 816 | * while executing the guest). This interrupt is still | 834 | * we exit guest timing so that timer ticks are accounted as |
| 817 | * pending, as we haven't serviced it yet! | 835 | * guest time. Transiently unmask interrupts so that any |
| 836 | * pending interrupts are taken. | ||
| 818 | * | 837 | * |
| 819 | * We're now back in HS-mode with interrupts disabled | 838 | * There's no barrier which ensures that pending interrupts are |
| 820 | * so enabling the interrupts now will have the effect | 839 | * recognised, so we just hope that the CPU takes any pending |
| 821 | * of taking the interrupt again, in HS-mode this time. | 840 | * interrupts between the enable and disable. |
| 822 | */ | 841 | */ |
| 823 | local_irq_enable(); | 842 | local_irq_enable(); |
| 843 | local_irq_disable(); | ||
| 824 | 844 | ||
| 825 | /* | 845 | guest_timing_exit_irqoff(); |
| 826 | * We do local_irq_enable() before calling guest_exit() so | 846 | |
| 827 | * that if a timer interrupt hits while running the guest | 847 | local_irq_enable(); |
| 828 | * we account that tick as being spent in the guest. We | ||
| 829 | * enable preemption after calling guest_exit() so that if | ||
| 830 | * we get preempted we make sure ticks after that is not | ||
| 831 | * counted as guest time. | ||
| 832 | */ | ||
| 833 | guest_exit(); | ||
| 834 | 848 | ||
| 835 | preempt_enable(); | 849 | preempt_enable(); |
| 836 | 850 | ||
| @@ -9,6 +9,7 @@ · 8 unmodified lines | |||
| 9 | #include <linux/errno.h> | 9 | #include <linux/errno.h> |
| 10 | #include <linux/err.h> | 10 | #include <linux/err.h> |
| 11 | #include <linux/kvm_host.h> | 11 | #include <linux/kvm_host.h> |
| 12 | #include <linux/version.h> | ||
| 12 | #include <asm/csr.h> | 13 | #include <asm/csr.h> |
| 13 | #include <asm/sbi.h> | 14 | #include <asm/sbi.h> |
| 14 | #include <asm/kvm_vcpu_timer.h> | 15 | #include <asm/kvm_vcpu_timer.h> |
| @@ -32,7 +33,7 @@ static int kvm_sbi_ext_base_handler(struct kvm_vcpu *vcpu, struct kvm_run *run, · 17 unmodified lines | |||
| 32 | *out_val = KVM_SBI_IMPID; | 33 | *out_val = KVM_SBI_IMPID; |
| 33 | break; | 34 | break; |
| 34 | case SBI_EXT_BASE_GET_IMP_VERSION: | 35 | case SBI_EXT_BASE_GET_IMP_VERSION: |
| 35 | *out_val = 0; | 36 | *out_val = LINUX_VERSION_CODE; |
| 36 | break; | 37 | break; |
| 37 | case SBI_EXT_BASE_PROBE_EXT: | 38 | case SBI_EXT_BASE_PROBE_EXT: |
| 38 | if ((cp->a0 >= SBI_EXT_EXPERIMENTAL_START && | 39 | if ((cp->a0 >= SBI_EXT_EXPERIMENTAL_START && |
| @@ -33,7 +33,7 @@ static inline void regs_set_gpr(struct pt_regs *regs, unsigned int offset, · 32 unmodified lines | |||
| 33 | if (unlikely(offset > MAX_REG_OFFSET)) | 33 | if (unlikely(offset > MAX_REG_OFFSET)) |
| 34 | return; | 34 | return; |
| 35 | 35 | ||
| 36 | if (!offset) | 36 | if (offset) |
| 37 | *(unsigned long *)((unsigned long)regs + offset) = val; | 37 | *(unsigned long *)((unsigned long)regs + offset) = val; |
| 38 | } | 38 | } |
| 39 | 39 | ||
| @@ -43,8 +43,8 @@ static bool ex_handler_uaccess_err_zero(const struct exception_table_entry *ex, · 3 unmodified lines | |||
| 43 | int reg_err = FIELD_GET(EX_DATA_REG_ERR, ex->data); | 43 | int reg_err = FIELD_GET(EX_DATA_REG_ERR, ex->data); |
| 44 | int reg_zero = FIELD_GET(EX_DATA_REG_ZERO, ex->data); | 44 | int reg_zero = FIELD_GET(EX_DATA_REG_ZERO, ex->data); |
| 45 | 45 | ||
| 46 | regs_set_gpr(regs, reg_err, -EFAULT); | 46 | regs_set_gpr(regs, reg_err * sizeof(unsigned long), -EFAULT); |
| 47 | regs_set_gpr(regs, reg_zero, 0); | 47 | regs_set_gpr(regs, reg_zero * sizeof(unsigned long), 0); |
| 48 | 48 | ||
| 49 | regs->epc = get_ex_fixup(ex); | 49 | regs->epc = get_ex_fixup(ex); |
| 50 | return true; | 50 | return true; |
| @@ -232,6 +232,7 @@ static pmd_t __maybe_unused early_dtb_pmd[PTRS_PER_PMD] __initdata __aligned(PAG · 231 unmodified lines | |||
| 232 | 232 | ||
| 233 | #ifdef CONFIG_XIP_KERNEL | 233 | #ifdef CONFIG_XIP_KERNEL |
| 234 | #define pt_ops (*(struct pt_alloc_ops *)XIP_FIXUP(&pt_ops)) | 234 | #define pt_ops (*(struct pt_alloc_ops *)XIP_FIXUP(&pt_ops)) |
| 235 | #define riscv_pfn_base (*(unsigned long *)XIP_FIXUP(&riscv_pfn_base)) | ||
| 235 | #define trampoline_pg_dir ((pgd_t *)XIP_FIXUP(trampoline_pg_dir)) | 236 | #define trampoline_pg_dir ((pgd_t *)XIP_FIXUP(trampoline_pg_dir)) |
| 236 | #define fixmap_pte ((pte_t *)XIP_FIXUP(fixmap_pte)) | 237 | #define fixmap_pte ((pte_t *)XIP_FIXUP(fixmap_pte)) |
| 237 | #define early_pg_dir ((pgd_t *)XIP_FIXUP(early_pg_dir)) | 238 | #define early_pg_dir ((pgd_t *)XIP_FIXUP(early_pg_dir)) |
| @@ -522,6 +523,7 @@ static uintptr_t __init best_map_size(phys_addr_t base, phys_addr_t size) · 284 unmodified lines | |||
| 522 | } | 523 | } |
| 523 | 524 | ||
| 524 | #ifdef CONFIG_XIP_KERNEL | 525 | #ifdef CONFIG_XIP_KERNEL |
| 526 | #define phys_ram_base (*(phys_addr_t *)XIP_FIXUP(&phys_ram_base)) | ||
| 525 | extern char _xiprom[], _exiprom[], __data_loc; | 527 | extern char _xiprom[], _exiprom[], __data_loc; |
| 526 | 528 | ||
| 527 | /* called from head.S with MMU off */ | 529 | /* called from head.S with MMU off */ |
| @@ -4667,6 +4667,8 @@ static long kvm_s390_guest_sida_op(struct kvm_vcpu *vcpu, · 4666 unmodified lines | |||
| 4667 | return -EINVAL; | 4667 | return -EINVAL; |
| 4668 | if (mop->size + mop->sida_offset > sida_size(vcpu->arch.sie_block)) | 4668 | if (mop->size + mop->sida_offset > sida_size(vcpu->arch.sie_block)) |
| 4669 | return -E2BIG; | 4669 | return -E2BIG; |
| 4670 | if (!kvm_s390_pv_cpu_is_protected(vcpu)) | ||
| 4671 | return -EINVAL; | ||
| 4670 | 4672 | ||
| 4671 | switch (mop->op) { | 4673 | switch (mop->op) { |
| 4672 | case KVM_S390_MEMOP_SIDA_READ: | 4674 | case KVM_S390_MEMOP_SIDA_READ: |
| @@ -5,9 +5,6 @@ · 4 unmodified lines | |||
| 5 | 5 | ||
| 6 | #include "test_modules.h" | 6 | #include "test_modules.h" |
| 7 | 7 | ||
| 8 | #define DECLARE_RETURN(i) int test_modules_return_ ## i(void) | ||
| 9 | REPEAT_10000(DECLARE_RETURN); | ||
| 10 | |||
| 11 | /* | 8 | /* |
| 12 | * Test that modules with many relocations are loaded properly. | 9 | * Test that modules with many relocations are loaded properly. |
| 13 | */ | 10 | */ |
| @@ -47,4 +47,7 @@ · 46 unmodified lines | |||
| 47 | __REPEAT_10000_1(f, 8); \ | 47 | __REPEAT_10000_1(f, 8); \ |
| 48 | __REPEAT_10000_1(f, 9) | 48 | __REPEAT_10000_1(f, 9) |
| 49 | 49 | ||
| 50 | #define DECLARE_RETURN(i) int test_modules_return_ ## i(void) | ||
| 51 | REPEAT_10000(DECLARE_RETURN); | ||
| 52 | |||
| 50 | #endif | 53 | #endif |
| @@ -18,12 +18,12 @@ · 17 unmodified lines | |||
| 18 | static int crypto_blake2s_update_x86(struct shash_desc *desc, | 18 | static int crypto_blake2s_update_x86(struct shash_desc *desc, |
| 19 | const u8 *in, unsigned int inlen) | 19 | const u8 *in, unsigned int inlen) |
| 20 | { | 20 | { |
| 21 | return crypto_blake2s_update(desc, in, inlen, blake2s_compress); | 21 | return crypto_blake2s_update(desc, in, inlen, false); |
| 22 | } | 22 | } |
| 23 | 23 | ||
| 24 | static int crypto_blake2s_final_x86(struct shash_desc *desc, u8 *out) | 24 | static int crypto_blake2s_final_x86(struct shash_desc *desc, u8 *out) |
| 25 | { | 25 | { |
| 26 | return crypto_blake2s_final(desc, out, blake2s_compress); | 26 | return crypto_blake2s_final(desc, out, false); |
| 27 | } | 27 | } |
| 28 | 28 | ||
| 29 | #define BLAKE2S_ALG(name, driver_name, digest_size) \ | 29 | #define BLAKE2S_ALG(name, driver_name, digest_size) \ |
| @@ -4703,6 +4703,19 @@ static __initconst const struct x86_pmu intel_pmu = { · 4702 unmodified lines | |||
| 4703 | .lbr_read = intel_pmu_lbr_read_64, | 4703 | .lbr_read = intel_pmu_lbr_read_64, |
| 4704 | .lbr_save = intel_pmu_lbr_save, | 4704 | .lbr_save = intel_pmu_lbr_save, |
| 4705 | .lbr_restore = intel_pmu_lbr_restore, | 4705 | .lbr_restore = intel_pmu_lbr_restore, |
| 4706 | |||
| 4707 | /* | ||
| 4708 | * SMM has access to all 4 rings and while traditionally SMM code only | ||
| 4709 | * ran in CPL0, 2021-era firmware is starting to make use of CPL3 in SMM. | ||
| 4710 | * | ||
| 4711 | * Since the EVENTSEL.{USR,OS} CPL filtering makes no distinction | ||
| 4712 | * between SMM or not, this results in what should be pure userspace | ||
| 4713 | * counters including SMM data. | ||
| 4714 | * | ||
| 4715 | * This is a clear privilege issue, therefore globally disable | ||
| 4716 | * counting SMM by default. | ||
| 4717 | */ | ||
| 4718 | .attr_freeze_on_smi = 1, | ||
| 4706 | }; | 4719 | }; |
| 4707 | 4720 | ||
| 4708 | static __init void intel_clovertown_quirk(void) | 4721 | static __init void intel_clovertown_quirk(void) |
| @@ -897,8 +897,9 @@ static void pt_handle_status(struct pt *pt) · 896 unmodified lines | |||
| 897 | * means we are already losing data; need to let the decoder | 897 | * means we are already losing data; need to let the decoder |
| 898 | * know. | 898 | * know. |
| 899 | */ | 899 | */ |
| 900 | if (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries) || | 900 | if (!buf->single && |
| 901 | buf->output_off == pt_buffer_region_size(buf)) { | 901 | (!intel_pt_validate_hw_cap(PT_CAP_topa_multiple_entries) || |
| 902 | buf->output_off == pt_buffer_region_size(buf))) { | ||
| 902 | perf_aux_output_flag(&pt->handle, | 903 | perf_aux_output_flag(&pt->handle, |
| 903 | PERF_AUX_FLAG_TRUNCATED); | 904 | PERF_AUX_FLAG_TRUNCATED); |
| 904 | advance++; | 905 | advance++; |
| @@ -22,7 +22,7 @@ · 21 unmodified lines | |||
| 22 | 22 | ||
| 23 | #ifdef CONFIG_DEBUG_BUGVERBOSE | 23 | #ifdef CONFIG_DEBUG_BUGVERBOSE |
| 24 | 24 | ||
| 25 | #define _BUG_FLAGS(ins, flags) \ | 25 | #define _BUG_FLAGS(ins, flags, extra) \ |
| 26 | do { \ | 26 | do { \ |
| 27 | asm_inline volatile("1:\t" ins "\n" \ | 27 | asm_inline volatile("1:\t" ins "\n" \ |
| 28 | ".pushsection __bug_table,\"aw\"\n" \ | 28 | ".pushsection __bug_table,\"aw\"\n" \ |
| @@ -31,7 +31,8 @@ do { \ · 2 unmodified lines | |||
| 31 | "\t.word %c1" "\t# bug_entry::line\n" \ | 31 | "\t.word %c1" "\t# bug_entry::line\n" \ |
| 32 | "\t.word %c2" "\t# bug_entry::flags\n" \ | 32 | "\t.word %c2" "\t# bug_entry::flags\n" \ |
| 33 | "\t.org 2b+%c3\n" \ | 33 | "\t.org 2b+%c3\n" \ |
| 34 | ".popsection" \ | 34 | ".popsection\n" \ |
| 35 | extra \ | ||
| 35 | : : "i" (__FILE__), "i" (__LINE__), \ | 36 | : : "i" (__FILE__), "i" (__LINE__), \ |
| 36 | "i" (flags), \ | 37 | "i" (flags), \ |
| 37 | "i" (sizeof(struct bug_entry))); \ | 38 | "i" (sizeof(struct bug_entry))); \ |
| @@ -39,14 +40,15 @@ do { \ · 1 unmodified line | |||
| 39 | 40 | ||
| 40 | #else /* !CONFIG_DEBUG_BUGVERBOSE */ | 41 | #else /* !CONFIG_DEBUG_BUGVERBOSE */ |
| 41 | 42 | ||
| 42 | #define _BUG_FLAGS(ins, flags) \ | 43 | #define _BUG_FLAGS(ins, flags, extra) \ |
| 43 | do { \ | 44 | do { \ |
| 44 | asm_inline volatile("1:\t" ins "\n" \ | 45 | asm_inline volatile("1:\t" ins "\n" \ |
| 45 | ".pushsection __bug_table,\"aw\"\n" \ | 46 | ".pushsection __bug_table,\"aw\"\n" \ |
| 46 | "2:\t" __BUG_REL(1b) "\t# bug_entry::bug_addr\n" \ | 47 | "2:\t" __BUG_REL(1b) "\t# bug_entry::bug_addr\n" \ |
| 47 | "\t.word %c0" "\t# bug_entry::flags\n" \ | 48 | "\t.word %c0" "\t# bug_entry::flags\n" \ |
| 48 | "\t.org 2b+%c1\n" \ | 49 | "\t.org 2b+%c1\n" \ |
| 49 | ".popsection" \ | 50 | ".popsection\n" \ |
| 51 | extra \ | ||
| 50 | : : "i" (flags), \ | 52 | : : "i" (flags), \ |
| 51 | "i" (sizeof(struct bug_entry))); \ | 53 | "i" (sizeof(struct bug_entry))); \ |
| 52 | } while (0) | 54 | } while (0) |
| @@ -55,7 +57,7 @@ do { \ · 2 unmodified lines | |||
| 55 | 57 | ||
| 56 | #else | 58 | #else |
| 57 | 59 | ||
| 58 | #define _BUG_FLAGS(ins, flags) asm volatile(ins) | 60 | #define _BUG_FLAGS(ins, flags, extra) asm volatile(ins) |
| 59 | 61 | ||
| 60 | #endif /* CONFIG_GENERIC_BUG */ | 62 | #endif /* CONFIG_GENERIC_BUG */ |
| 61 | 63 | ||
| @@ -63,8 +65,8 @@ do { \ · 1 unmodified line | |||
| 63 | #define BUG() \ | 65 | #define BUG() \ |
| 64 | do { \ | 66 | do { \ |
| 65 | instrumentation_begin(); \ | 67 | instrumentation_begin(); \ |
| 66 | _BUG_FLAGS(ASM_UD2, 0); \ | 68 | _BUG_FLAGS(ASM_UD2, 0, ""); \ |
| 67 | unreachable(); \ | 69 | __builtin_unreachable(); \ |
| 68 | } while (0) | 70 | } while (0) |
| 69 | 71 | ||
| 70 | /* | 72 | /* |
| @@ -75,9 +77,9 @@ do { \ · 4 unmodified lines | |||
| 75 | */ | 77 | */ |
| 76 | #define __WARN_FLAGS(flags) \ | 78 | #define __WARN_FLAGS(flags) \ |
| 77 | do { \ | 79 | do { \ |
| 80 | __auto_type f = BUGFLAG_WARNING|(flags); \ | ||
| 78 | instrumentation_begin(); \ | 81 | instrumentation_begin(); \ |
| 79 | _BUG_FLAGS(ASM_UD2, BUGFLAG_WARNING|(flags)); \ | 82 | _BUG_FLAGS(ASM_UD2, f, ASM_REACHABLE); \ |
| 80 | annotate_reachable(); \ | ||
| 81 | instrumentation_end(); \ | 83 | instrumentation_end(); \ |
| 82 | } while (0) | 84 | } while (0) |
| 83 | 85 | ||
| @@ -82,7 +82,7 @@ KVM_X86_OP_NULL(guest_apic_has_interrupt) · 81 unmodified lines | |||
| 82 | KVM_X86_OP(load_eoi_exitmap) | 82 | KVM_X86_OP(load_eoi_exitmap) |
| 83 | KVM_X86_OP(set_virtual_apic_mode) | 83 | KVM_X86_OP(set_virtual_apic_mode) |
| 84 | KVM_X86_OP_NULL(set_apic_access_page_addr) | 84 | KVM_X86_OP_NULL(set_apic_access_page_addr) |
| 85 | KVM_X86_OP(deliver_posted_interrupt) | 85 | KVM_X86_OP(deliver_interrupt) |
| 86 | KVM_X86_OP_NULL(sync_pir_to_irr) | 86 | KVM_X86_OP_NULL(sync_pir_to_irr) |
| 87 | KVM_X86_OP(set_tss_addr) | 87 | KVM_X86_OP(set_tss_addr) |
| 88 | KVM_X86_OP(set_identity_map_addr) | 88 | KVM_X86_OP(set_identity_map_addr) |
| @@ -703,7 +703,6 @@ struct kvm_vcpu_arch { · 702 unmodified lines | |||
| 703 | struct fpu_guest guest_fpu; | 703 | struct fpu_guest guest_fpu; |
| 704 | 704 | ||
| 705 | u64 xcr0; | 705 | u64 xcr0; |
| 706 | u64 guest_supported_xcr0; | ||
| 707 | 706 | ||
| 708 | struct kvm_pio_request pio; | 707 | struct kvm_pio_request pio; |
| 709 | void *pio_data; | 708 | void *pio_data; |
| @@ -1410,7 +1409,8 @@ struct kvm_x86_ops { · 700 unmodified lines | |||
| 1410 | void (*load_eoi_exitmap)(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap); | 1409 | void (*load_eoi_exitmap)(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap); |
| 1411 | void (*set_virtual_apic_mode)(struct kvm_vcpu *vcpu); | 1410 | void (*set_virtual_apic_mode)(struct kvm_vcpu *vcpu); |
| 1412 | void (*set_apic_access_page_addr)(struct kvm_vcpu *vcpu); | 1411 | void (*set_apic_access_page_addr)(struct kvm_vcpu *vcpu); |
| 1413 | int (*deliver_posted_interrupt)(struct kvm_vcpu *vcpu, int vector); | 1412 | void (*deliver_interrupt)(struct kvm_lapic *apic, int delivery_mode, |
| 1413 | int trig_mode, int vector); | ||
| 1414 | int (*sync_pir_to_irr)(struct kvm_vcpu *vcpu); | 1414 | int (*sync_pir_to_irr)(struct kvm_vcpu *vcpu); |
| 1415 | int (*set_tss_addr)(struct kvm *kvm, unsigned int addr); | 1415 | int (*set_tss_addr)(struct kvm *kvm, unsigned int addr); |
| 1416 | int (*set_identity_map_addr)(struct kvm *kvm, u64 ident_addr); | 1416 | int (*set_identity_map_addr)(struct kvm *kvm, u64 ident_addr); |
| @@ -476,6 +476,7 @@ · 475 unmodified lines | |||
| 476 | #define MSR_AMD64_ICIBSEXTDCTL 0xc001103c | 476 | #define MSR_AMD64_ICIBSEXTDCTL 0xc001103c |
| 477 | #define MSR_AMD64_IBSOPDATA4 0xc001103d | 477 | #define MSR_AMD64_IBSOPDATA4 0xc001103d |
| 478 | #define MSR_AMD64_IBS_REG_COUNT_MAX 8 /* includes MSR_AMD64_IBSBRTARGET */ | 478 | #define MSR_AMD64_IBS_REG_COUNT_MAX 8 /* includes MSR_AMD64_IBSBRTARGET */ |
| 479 | #define MSR_AMD64_SVM_AVIC_DOORBELL 0xc001011b | ||
| 479 | #define MSR_AMD64_VM_PAGE_FLUSH 0xc001011e | 480 | #define MSR_AMD64_VM_PAGE_FLUSH 0xc001011e |
| 480 | #define MSR_AMD64_SEV_ES_GHCB 0xc0010130 | 481 | #define MSR_AMD64_SEV_ES_GHCB 0xc0010130 |
| 481 | #define MSR_AMD64_SEV 0xc0010131 | 482 | #define MSR_AMD64_SEV 0xc0010131 |
| @@ -220,6 +220,42 @@ struct __attribute__ ((__packed__)) vmcb_control_area { · 219 unmodified lines | |||
| 220 | #define SVM_NESTED_CTL_SEV_ENABLE BIT(1) | 220 | #define SVM_NESTED_CTL_SEV_ENABLE BIT(1) |
| 221 | #define SVM_NESTED_CTL_SEV_ES_ENABLE BIT(2) | 221 | #define SVM_NESTED_CTL_SEV_ES_ENABLE BIT(2) |
| 222 | 222 | ||
| 223 | |||
| 224 | /* AVIC */ | ||
| 225 | #define AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK (0xFF) | ||
| 226 | #define AVIC_LOGICAL_ID_ENTRY_VALID_BIT 31 | ||
| 227 | #define AVIC_LOGICAL_ID_ENTRY_VALID_MASK (1 << 31) | ||
| 228 | |||
| 229 | #define AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK (0xFFULL) | ||
| 230 | #define AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK (0xFFFFFFFFFFULL << 12) | ||
| 231 | #define AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK (1ULL << 62) | ||
| 232 | #define AVIC_PHYSICAL_ID_ENTRY_VALID_MASK (1ULL << 63) | ||
| 233 | #define AVIC_PHYSICAL_ID_TABLE_SIZE_MASK (0xFF) | ||
| 234 | |||
| 235 | #define AVIC_DOORBELL_PHYSICAL_ID_MASK (0xFF) | ||
| 236 | |||
| 237 | #define AVIC_UNACCEL_ACCESS_WRITE_MASK 1 | ||
| 238 | #define AVIC_UNACCEL_ACCESS_OFFSET_MASK 0xFF0 | ||
| 239 | #define AVIC_UNACCEL_ACCESS_VECTOR_MASK 0xFFFFFFFF | ||
| 240 | |||
| 241 | enum avic_ipi_failure_cause { | ||
| 242 | AVIC_IPI_FAILURE_INVALID_INT_TYPE, | ||
| 243 | AVIC_IPI_FAILURE_TARGET_NOT_RUNNING, | ||
| 244 | AVIC_IPI_FAILURE_INVALID_TARGET, | ||
| 245 | AVIC_IPI_FAILURE_INVALID_BACKING_PAGE, | ||
| 246 | }; | ||
| 247 | |||
| 248 | |||
| 249 | /* | ||
| 250 | * 0xff is broadcast, so the max index allowed for physical APIC ID | ||
| 251 | * table is 0xfe. APIC IDs above 0xff are reserved. | ||
| 252 | */ | ||
| 253 | #define AVIC_MAX_PHYSICAL_ID_COUNT 0xff | ||
| 254 | |||
| 255 | #define AVIC_HPA_MASK ~((0xFFFULL << 52) | 0xFFF) | ||
| 256 | #define VMCB_AVIC_APIC_BAR_MASK 0xFFFFFFFFFF000ULL | ||
| 257 | |||
| 258 | |||
| 223 | struct vmcb_seg { | 259 | struct vmcb_seg { |
| 224 | u16 selector; | 260 | u16 selector; |
| 225 | u16 attrib; | 261 | u16 attrib; |
| @@ -100,6 +100,13 @@ · 99 unmodified lines | |||
| 100 | /* Memory mapped from other domains has valid IOMMU entries */ | 100 | /* Memory mapped from other domains has valid IOMMU entries */ |
| 101 | #define XEN_HVM_CPUID_IOMMU_MAPPINGS (1u << 2) | 101 | #define XEN_HVM_CPUID_IOMMU_MAPPINGS (1u << 2) |
| 102 | #define XEN_HVM_CPUID_VCPU_ID_PRESENT (1u << 3) /* vcpu id is present in EBX */ | 102 | #define XEN_HVM_CPUID_VCPU_ID_PRESENT (1u << 3) /* vcpu id is present in EBX */ |
| 103 | #define XEN_HVM_CPUID_DOMID_PRESENT (1u << 4) /* domid is present in ECX */ | ||
| 104 | /* | ||
| 105 | * Bits 55:49 from the IO-APIC RTE and bits 11:5 from the MSI address can be | ||
| 106 | * used to store high bits for the Destination ID. This expands the Destination | ||
| 107 | * ID field from 8 to 15 bits, allowing to target APIC IDs up 32768. | ||
| 108 | */ | ||
| 109 | #define XEN_HVM_CPUID_EXT_DEST_ID (1u << 5) | ||
| 103 | 110 | ||
| 104 | /* | 111 | /* |
| 105 | * Leaf 6 (0x40000x05) | 112 | * Leaf 6 (0x40000x05) |
| @@ -43,20 +43,6 @@ static inline uint32_t xen_cpuid_base(void) · 42 unmodified lines | |||
| 43 | return hypervisor_cpuid_base("XenVMMXenVMM", 2); | 43 | return hypervisor_cpuid_base("XenVMMXenVMM", 2); |
| 44 | } | 44 | } |
| 45 | 45 | ||
| 46 | #ifdef CONFIG_XEN | ||
| 47 | extern bool __init xen_hvm_need_lapic(void); | ||
| 48 | |||
| 49 | static inline bool __init xen_x2apic_para_available(void) | ||
| 50 | { | ||
| 51 | return xen_hvm_need_lapic(); | ||
| 52 | } | ||
| 53 | #else | ||
| 54 | static inline bool __init xen_x2apic_para_available(void) | ||
| 55 | { | ||
| 56 | return (xen_cpuid_base() != 0); | ||
| 57 | } | ||
| 58 | #endif | ||
| 59 | |||
| 60 | struct pci_dev; | 46 | struct pci_dev; |
| 61 | 47 | ||
| 62 | #ifdef CONFIG_XEN_PV_DOM0 | 48 | #ifdef CONFIG_XEN_PV_DOM0 |
| @@ -410,6 +410,8 @@ void sgx_encl_release(struct kref *ref) · 409 unmodified lines | |||
| 410 | } | 410 | } |
| 411 | 411 | ||
| 412 | kfree(entry); | 412 | kfree(entry); |
| 413 | /* Invoke scheduler to prevent soft lockups. */ | ||
| 414 | cond_resched(); | ||
| 413 | } | 415 | } |
| 414 | 416 | ||
| 415 | xa_destroy(&encl->page_array); | 417 | xa_destroy(&encl->page_array); |
| @@ -344,10 +344,8 @@ static void sgx_reclaim_pages(void) · 343 unmodified lines | |||
| 344 | { | 344 | { |
| 345 | struct sgx_epc_page *chunk[SGX_NR_TO_SCAN]; | 345 | struct sgx_epc_page *chunk[SGX_NR_TO_SCAN]; |
| 346 | struct sgx_backing backing[SGX_NR_TO_SCAN]; | 346 | struct sgx_backing backing[SGX_NR_TO_SCAN]; |
| 347 | struct sgx_epc_section *section; | ||
| 348 | struct sgx_encl_page *encl_page; | 347 | struct sgx_encl_page *encl_page; |
| 349 | struct sgx_epc_page *epc_page; | 348 | struct sgx_epc_page *epc_page; |
| 350 | struct sgx_numa_node *node; | ||
| 351 | pgoff_t page_index; | 349 | pgoff_t page_index; |
| 352 | int cnt = 0; | 350 | int cnt = 0; |
| 353 | int ret; | 351 | int ret; |
| @@ -418,13 +416,7 @@ skip: · 64 unmodified lines | |||
| 418 | kref_put(&encl_page->encl->refcount, sgx_encl_release); | 416 | kref_put(&encl_page->encl->refcount, sgx_encl_release); |
| 419 | epc_page->flags &= ~SGX_EPC_PAGE_RECLAIMER_TRACKED; | 417 | epc_page->flags &= ~SGX_EPC_PAGE_RECLAIMER_TRACKED; |
| 420 | 418 | ||
| 421 | section = &sgx_epc_sections[epc_page->section]; | 419 | sgx_free_epc_page(epc_page); |
| 422 | node = section->node; | ||
| 423 | |||
| 424 | spin_lock(&node->lock); | ||
| 425 | list_add_tail(&epc_page->list, &node->free_page_list); | ||
| 426 | spin_unlock(&node->lock); | ||
| 427 | atomic_long_inc(&sgx_nr_free_pages); | ||
| 428 | } | 420 | } |
| 429 | } | 421 | } |
| 430 | 422 | ||
| @@ -91,11 +91,9 @@ int xfpregs_set(struct task_struct *target, const struct user_regset *regset, · 90 unmodified lines | |||
| 91 | const void *kbuf, const void __user *ubuf) | 91 | const void *kbuf, const void __user *ubuf) |
| 92 | { | 92 | { |
| 93 | struct fpu *fpu = &target->thread.fpu; | 93 | struct fpu *fpu = &target->thread.fpu; |
| 94 | struct user32_fxsr_struct newstate; | 94 | struct fxregs_state newstate; |
| 95 | int ret; | 95 | int ret; |
| 96 | 96 | ||
| 97 | BUILD_BUG_ON(sizeof(newstate) != sizeof(struct fxregs_state)); | ||
| 98 | |||
| 99 | if (!cpu_feature_enabled(X86_FEATURE_FXSR)) | 97 | if (!cpu_feature_enabled(X86_FEATURE_FXSR)) |
| 100 | return -ENODEV; | 98 | return -ENODEV; |
| 101 | 99 | ||
| @@ -116,9 +114,10 @@ int xfpregs_set(struct task_struct *target, const struct user_regset *regset, · 14 unmodified lines | |||
| 116 | /* Copy the state */ | 114 | /* Copy the state */ |
| 117 | memcpy(&fpu->fpstate->regs.fxsave, &newstate, sizeof(newstate)); | 115 | memcpy(&fpu->fpstate->regs.fxsave, &newstate, sizeof(newstate)); |
| 118 | 116 | ||
| 119 | /* Clear xmm8..15 */ | 117 | /* Clear xmm8..15 for 32-bit callers */ |
| 120 | BUILD_BUG_ON(sizeof(fpu->__fpstate.regs.fxsave.xmm_space) != 16 * 16); | 118 | BUILD_BUG_ON(sizeof(fpu->__fpstate.regs.fxsave.xmm_space) != 16 * 16); |
| 121 | memset(&fpu->fpstate->regs.fxsave.xmm_space[8], 0, 8 * 16); | 119 | if (in_ia32_syscall()) |
| 120 | memset(&fpu->fpstate->regs.fxsave.xmm_space[8*4], 0, 8 * 16); | ||
| 122 | 121 | ||
| 123 | /* Mark FP and SSE as in use when XSAVE is enabled */ | 122 | /* Mark FP and SSE as in use when XSAVE is enabled */ |
| 124 | if (use_xsave()) | 123 | if (use_xsave()) |
| @@ -1558,7 +1558,10 @@ static int fpstate_realloc(u64 xfeatures, unsigned int ksize, · 1557 unmodified lines | |||
| 1558 | fpregs_restore_userregs(); | 1558 | fpregs_restore_userregs(); |
| 1559 | 1559 | ||
| 1560 | newfps->xfeatures = curfps->xfeatures | xfeatures; | 1560 | newfps->xfeatures = curfps->xfeatures | xfeatures; |
| 1561 | newfps->user_xfeatures = curfps->user_xfeatures | xfeatures; | 1561 | |
| 1562 | if (!guest_fpu) | ||
| 1563 | newfps->user_xfeatures = curfps->user_xfeatures | xfeatures; | ||
| 1564 | |||
| 1562 | newfps->xfd = curfps->xfd & ~xfeatures; | 1565 | newfps->xfd = curfps->xfd & ~xfeatures; |
| 1563 | 1566 | ||
| 1564 | /* Do the final updates within the locked region */ | 1567 | /* Do the final updates within the locked region */ |
| @@ -462,19 +462,22 @@ static bool pv_tlb_flush_supported(void) · 461 unmodified lines | |||
| 462 | { | 462 | { |
| 463 | return (kvm_para_has_feature(KVM_FEATURE_PV_TLB_FLUSH) && | 463 | return (kvm_para_has_feature(KVM_FEATURE_PV_TLB_FLUSH) && |
| 464 | !kvm_para_has_hint(KVM_HINTS_REALTIME) && | 464 | !kvm_para_has_hint(KVM_HINTS_REALTIME) && |
| 465 | kvm_para_has_feature(KVM_FEATURE_STEAL_TIME)); | 465 | kvm_para_has_feature(KVM_FEATURE_STEAL_TIME) && |
| 466 | (num_possible_cpus() != 1)); | ||
| 466 | } | 467 | } |
| 467 | 468 | ||
| 468 | static bool pv_ipi_supported(void) | 469 | static bool pv_ipi_supported(void) |
| 469 | { | 470 | { |
| 470 | return kvm_para_has_feature(KVM_FEATURE_PV_SEND_IPI); | 471 | return (kvm_para_has_feature(KVM_FEATURE_PV_SEND_IPI) && |
| 472 | (num_possible_cpus() != 1)); | ||
| 471 | } | 473 | } |
| 472 | 474 | ||
| 473 | static bool pv_sched_yield_supported(void) | 475 | static bool pv_sched_yield_supported(void) |
| 474 | { | 476 | { |
| 475 | return (kvm_para_has_feature(KVM_FEATURE_PV_SCHED_YIELD) && | 477 | return (kvm_para_has_feature(KVM_FEATURE_PV_SCHED_YIELD) && |
| 476 | !kvm_para_has_hint(KVM_HINTS_REALTIME) && | 478 | !kvm_para_has_hint(KVM_HINTS_REALTIME) && |
| 477 | kvm_para_has_feature(KVM_FEATURE_STEAL_TIME)); | 479 | kvm_para_has_feature(KVM_FEATURE_STEAL_TIME) && |
| 480 | (num_possible_cpus() != 1)); | ||
| 478 | } | 481 | } |
| 479 | 482 | ||
| 480 | #define KVM_IPI_CLUSTER_SIZE (2 * BITS_PER_LONG) | 483 | #define KVM_IPI_CLUSTER_SIZE (2 * BITS_PER_LONG) |
| @@ -1224,7 +1224,7 @@ static struct user_regset x86_64_regsets[] __ro_after_init = { · 1223 unmodified lines | |||
| 1224 | }, | 1224 | }, |
| 1225 | [REGSET_FP] = { | 1225 | [REGSET_FP] = { |
| 1226 | .core_note_type = NT_PRFPREG, | 1226 | .core_note_type = NT_PRFPREG, |
| 1227 | .n = sizeof(struct user_i387_struct) / sizeof(long), | 1227 | .n = sizeof(struct fxregs_state) / sizeof(long), |
| 1228 | .size = sizeof(long), .align = sizeof(long), | 1228 | .size = sizeof(long), .align = sizeof(long), |
| 1229 | .active = regset_xregset_fpregs_active, .regset_get = xfpregs_get, .set = xfpregs_set | 1229 | .active = regset_xregset_fpregs_active, .regset_get = xfpregs_get, .set = xfpregs_set |
| 1230 | }, | 1230 | }, |
| @@ -1271,7 +1271,7 @@ static struct user_regset x86_32_regsets[] __ro_after_init = { · 40 unmodified lines | |||
| 1271 | }, | 1271 | }, |
| 1272 | [REGSET_XFP] = { | 1272 | [REGSET_XFP] = { |
| 1273 | .core_note_type = NT_PRXFPREG, | 1273 | .core_note_type = NT_PRXFPREG, |
| 1274 | .n = sizeof(struct user32_fxsr_struct) / sizeof(u32), | 1274 | .n = sizeof(struct fxregs_state) / sizeof(u32), |
| 1275 | .size = sizeof(u32), .align = sizeof(u32), | 1275 | .size = sizeof(u32), .align = sizeof(u32), |
| 1276 | .active = regset_xregset_fpregs_active, .regset_get = xfpregs_get, .set = xfpregs_set | 1276 | .active = regset_xregset_fpregs_active, .regset_get = xfpregs_get, .set = xfpregs_set |
| 1277 | }, | 1277 | }, |
| @@ -1,5 +1,4 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0 | 1 | // SPDX-License-Identifier: GPL-2.0 |
| 2 | #include <linux/dmi.h> | ||
| 3 | #include <linux/ioport.h> | 2 | #include <linux/ioport.h> |
| 4 | #include <asm/e820/api.h> | 3 | #include <asm/e820/api.h> |
| 5 | 4 | ||
| @@ -24,31 +23,11 @@ static void resource_clip(struct resource *res, resource_size_t start, · 18 unmodified lines | |||
| 24 | res->start = end + 1; | 23 | res->start = end + 1; |
| 25 | } | 24 | } |
| 26 | 25 | ||
| 27 | /* | ||
| 28 | * Some BIOS-es contain a bug where they add addresses which map to | ||
| 29 | * system RAM in the PCI host bridge window returned by the ACPI _CRS | ||
| 30 | * method, see commit 4dc2287c1805 ("x86: avoid E820 regions when | ||
| 31 | * allocating address space"). To avoid this Linux by default excludes | ||
| 32 | * E820 reservations when allocating addresses since 2010. | ||
| 33 | * In 2019 some systems have shown-up with E820 reservations which cover | ||
| 34 | * the entire _CRS returned PCI host bridge window, causing all attempts | ||
| 35 | * to assign memory to PCI BARs to fail if Linux uses E820 reservations. | ||
| 36 | * | ||
| 37 | * Ideally Linux would fully stop using E820 reservations, but then | ||
| 38 | * the old systems this was added for will regress. | ||
| 39 | * Instead keep the old behavior for old systems, while ignoring the | ||
| 40 | * E820 reservations for any systems from now on. | ||
| 41 | */ | ||
| 42 | static void remove_e820_regions(struct resource *avail) | 26 | static void remove_e820_regions(struct resource *avail) |
| 43 | { | 27 | { |
| 44 | int i, year = dmi_get_bios_year(); | 28 | int i; |
| 45 | struct e820_entry *entry; | 29 | struct e820_entry *entry; |
| 46 | 30 | ||
| 47 | if (year >= 2018) | ||
| 48 | return; | ||
| 49 | |||
| 50 | pr_info_once("PCI: Removing E820 reservations from host bridge windows\n"); | ||
| 51 | |||
| 52 | for (i = 0; i < e820_table->nr_entries; i++) { | 31 | for (i = 0; i < e820_table->nr_entries; i++) { |
| 53 | entry = &e820_table->entries[i]; | 32 | entry = &e820_table->entries[i]; |
| 54 | 33 | ||
| @@ -282,6 +282,7 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · 281 unmodified lines | |||
| 282 | { | 282 | { |
| 283 | struct kvm_lapic *apic = vcpu->arch.apic; | 283 | struct kvm_lapic *apic = vcpu->arch.apic; |
| 284 | struct kvm_cpuid_entry2 *best; | 284 | struct kvm_cpuid_entry2 *best; |
| 285 | u64 guest_supported_xcr0; | ||
| 285 | 286 | ||
| 286 | best = kvm_find_cpuid_entry(vcpu, 1, 0); | 287 | best = kvm_find_cpuid_entry(vcpu, 1, 0); |
| 287 | if (best && apic) { | 288 | if (best && apic) { |
| @@ -293,9 +294,11 @@ static void kvm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu) · 5 unmodified lines | |||
| 293 | kvm_apic_set_version(vcpu); | 294 | kvm_apic_set_version(vcpu); |
| 294 | } | 295 | } |
| 295 | 296 | ||
| 296 | vcpu->arch.guest_supported_xcr0 = | 297 | guest_supported_xcr0 = |
| 297 | cpuid_get_supported_xcr0(vcpu->arch.cpuid_entries, vcpu->arch.cpuid_nent); | 298 | cpuid_get_supported_xcr0(vcpu->arch.cpuid_entries, vcpu->arch.cpuid_nent); |
| 298 | 299 | ||
| 300 | vcpu->arch.guest_fpu.fpstate->user_xfeatures = guest_supported_xcr0; | ||
| 301 | |||
| 299 | kvm_update_pv_runtime(vcpu); | 302 | kvm_update_pv_runtime(vcpu); |
| 300 | 303 | ||
| 301 | vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu); | 304 | vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu); |
| @@ -554,12 +557,13 @@ void kvm_set_cpu_caps(void) · 252 unmodified lines | |||
| 554 | ); | 557 | ); |
| 555 | 558 | ||
| 556 | kvm_cpu_cap_mask(CPUID_7_0_EBX, | 559 | kvm_cpu_cap_mask(CPUID_7_0_EBX, |
| 557 | F(FSGSBASE) | F(SGX) | F(BMI1) | F(HLE) | F(AVX2) | F(SMEP) | | 560 | F(FSGSBASE) | F(SGX) | F(BMI1) | F(HLE) | F(AVX2) | |
| 558 | F(BMI2) | F(ERMS) | F(INVPCID) | F(RTM) | 0 /*MPX*/ | F(RDSEED) | | 561 | F(FDP_EXCPTN_ONLY) | F(SMEP) | F(BMI2) | F(ERMS) | F(INVPCID) | |
| 559 | F(ADX) | F(SMAP) | F(AVX512IFMA) | F(AVX512F) | F(AVX512PF) | | 562 | F(RTM) | F(ZERO_FCS_FDS) | 0 /*MPX*/ | F(AVX512F) | |
| 560 | F(AVX512ER) | F(AVX512CD) | F(CLFLUSHOPT) | F(CLWB) | F(AVX512DQ) | | 563 | F(AVX512DQ) | F(RDSEED) | F(ADX) | F(SMAP) | F(AVX512IFMA) | |
| 561 | F(SHA_NI) | F(AVX512BW) | F(AVX512VL) | 0 /*INTEL_PT*/ | 564 | F(CLFLUSHOPT) | F(CLWB) | 0 /*INTEL_PT*/ | F(AVX512PF) | |
| 562 | ); | 565 | F(AVX512ER) | F(AVX512CD) | F(SHA_NI) | F(AVX512BW) | |
| 566 | F(AVX512VL)); | ||
| 563 | 567 | ||
| 564 | kvm_cpu_cap_mask(CPUID_7_ECX, | 568 | kvm_cpu_cap_mask(CPUID_7_ECX, |
| 565 | F(AVX512VBMI) | F(LA57) | F(PKU) | 0 /*OSPKE*/ | F(RDPID) | | 569 | F(AVX512VBMI) | F(LA57) | F(PKU) | 0 /*OSPKE*/ | F(RDPID) | |
| @@ -1096,14 +1096,8 @@ static int __apic_accept_irq(struct kvm_lapic *apic, int delivery_mode, · 1095 unmodified lines | |||
| 1096 | apic->regs + APIC_TMR); | 1096 | apic->regs + APIC_TMR); |
| 1097 | } | 1097 | } |
| 1098 | 1098 | ||
| 1099 | if (static_call(kvm_x86_deliver_posted_interrupt)(vcpu, vector)) { | 1099 | static_call(kvm_x86_deliver_interrupt)(apic, delivery_mode, |
| 1100 | kvm_lapic_set_irr(vector, apic); | 1100 | trig_mode, vector); |
| 1101 | kvm_make_request(KVM_REQ_EVENT, vcpu); | ||
| 1102 | kvm_vcpu_kick(vcpu); | ||
| 1103 | } else { | ||
| 1104 | trace_kvm_apicv_accept_irq(vcpu->vcpu_id, delivery_mode, | ||
| 1105 | trig_mode, vector); | ||
| 1106 | } | ||
| 1107 | break; | 1101 | break; |
| 1108 | 1102 | ||
| 1109 | case APIC_DM_REMRD: | 1103 | case APIC_DM_REMRD: |
| @@ -2312,7 +2306,12 @@ void kvm_apic_update_apicv(struct kvm_vcpu *vcpu) · 1202 unmodified lines | |||
| 2312 | apic->irr_pending = true; | 2306 | apic->irr_pending = true; |
| 2313 | apic->isr_count = 1; | 2307 | apic->isr_count = 1; |
| 2314 | } else { | 2308 | } else { |
| 2315 | apic->irr_pending = (apic_search_irr(apic) != -1); | 2309 | /* |
| 2310 | * Don't clear irr_pending, searching the IRR can race with | ||
| 2311 | * updates from the CPU as APICv is still active from hardware's | ||
| 2312 | * perspective. The flag will be cleared as appropriate when | ||
| 2313 | * KVM injects the interrupt. | ||
| 2314 | */ | ||
| 2316 | apic->isr_count = count_vectors(apic->regs + APIC_ISR); | 2315 | apic->isr_count = count_vectors(apic->regs + APIC_ISR); |
| 2317 | } | 2316 | } |
| 2318 | } | 2317 | } |
| @@ -3889,12 +3889,23 @@ static void shadow_page_table_clear_flood(struct kvm_vcpu *vcpu, gva_t addr) · 3888 unmodified lines | |||
| 3889 | walk_shadow_page_lockless_end(vcpu); | 3889 | walk_shadow_page_lockless_end(vcpu); |
| 3890 | } | 3890 | } |
| 3891 | 3891 | ||
| 3892 | static u32 alloc_apf_token(struct kvm_vcpu *vcpu) | ||
| 3893 | { | ||
| 3894 | /* make sure the token value is not 0 */ | ||
| 3895 | u32 id = vcpu->arch.apf.id; | ||
| 3896 | |||
| 3897 | if (id << 12 == 0) | ||
| 3898 | vcpu->arch.apf.id = 1; | ||
| 3899 | |||
| 3900 | return (vcpu->arch.apf.id++ << 12) | vcpu->vcpu_id; | ||
| 3901 | } | ||
| 3902 | |||
| 3892 | static bool kvm_arch_setup_async_pf(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, | 3903 | static bool kvm_arch_setup_async_pf(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, |
| 3893 | gfn_t gfn) | 3904 | gfn_t gfn) |
| 3894 | { | 3905 | { |
| 3895 | struct kvm_arch_async_pf arch; | 3906 | struct kvm_arch_async_pf arch; |
| 3896 | 3907 | ||
| 3897 | arch.token = (vcpu->arch.apf.id++ << 12) | vcpu->vcpu_id; | 3908 | arch.token = alloc_apf_token(vcpu); |
| 3898 | arch.gfn = gfn; | 3909 | arch.gfn = gfn; |
| 3899 | arch.direct_map = vcpu->arch.mmu->direct_map; | 3910 | arch.direct_map = vcpu->arch.mmu->direct_map; |
| 3900 | arch.cr3 = vcpu->arch.mmu->get_guest_pgd(vcpu); | 3911 | arch.cr3 = vcpu->arch.mmu->get_guest_pgd(vcpu); |
| @@ -95,7 +95,7 @@ static void kvm_perf_overflow(struct perf_event *perf_event, · 94 unmodified lines | |||
| 95 | } | 95 | } |
| 96 | 96 | ||
| 97 | static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, | 97 | static void pmc_reprogram_counter(struct kvm_pmc *pmc, u32 type, |
| 98 | unsigned config, bool exclude_user, | 98 | u64 config, bool exclude_user, |
| 99 | bool exclude_kernel, bool intr, | 99 | bool exclude_kernel, bool intr, |
| 100 | bool in_tx, bool in_tx_cp) | 100 | bool in_tx, bool in_tx_cp) |
| 101 | { | 101 | { |
| @@ -181,7 +181,8 @@ static int cmp_u64(const void *a, const void *b) · 79 unmodified lines | |||
| 181 | 181 | ||
| 182 | void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel) | 182 | void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel) |
| 183 | { | 183 | { |
| 184 | unsigned config, type = PERF_TYPE_RAW; | 184 | u64 config; |
| 185 | u32 type = PERF_TYPE_RAW; | ||
| 185 | struct kvm *kvm = pmc->vcpu->kvm; | 186 | struct kvm *kvm = pmc->vcpu->kvm; |
| 186 | struct kvm_pmu_event_filter *filter; | 187 | struct kvm_pmu_event_filter *filter; |
| 187 | bool allow_event = true; | 188 | bool allow_event = true; |
| @@ -220,7 +221,7 @@ void reprogram_gp_counter(struct kvm_pmc *pmc, u64 eventsel) · 32 unmodified lines | |||
| 220 | } | 221 | } |
| 221 | 222 | ||
| 222 | if (type == PERF_TYPE_RAW) | 223 | if (type == PERF_TYPE_RAW) |
| 223 | config = eventsel & X86_RAW_EVENT_MASK; | 224 | config = eventsel & AMD64_RAW_EVENT_MASK; |
| 224 | 225 | ||
| 225 | if (pmc->current_config == eventsel && pmc_resume_counter(pmc)) | 226 | if (pmc->current_config == eventsel && pmc_resume_counter(pmc)) |
| 226 | return; | 227 | return; |
| @@ -27,20 +27,6 @@ · 26 unmodified lines | |||
| 27 | #include "irq.h" | 27 | #include "irq.h" |
| 28 | #include "svm.h" | 28 | #include "svm.h" |
| 29 | 29 | ||
| 30 | #define SVM_AVIC_DOORBELL 0xc001011b | ||
| 31 | |||
| 32 | #define AVIC_HPA_MASK ~((0xFFFULL << 52) | 0xFFF) | ||
| 33 | |||
| 34 | /* | ||
| 35 | * 0xff is broadcast, so the max index allowed for physical APIC ID | ||
| 36 | * table is 0xfe. APIC IDs above 0xff are reserved. | ||
| 37 | */ | ||
| 38 | #define AVIC_MAX_PHYSICAL_ID_COUNT 255 | ||
| 39 | |||
| 40 | #define AVIC_UNACCEL_ACCESS_WRITE_MASK 1 | ||
| 41 | #define AVIC_UNACCEL_ACCESS_OFFSET_MASK 0xFF0 | ||
| 42 | #define AVIC_UNACCEL_ACCESS_VECTOR_MASK 0xFFFFFFFF | ||
| 43 | |||
| 44 | /* AVIC GATAG is encoded using VM and VCPU IDs */ | 30 | /* AVIC GATAG is encoded using VM and VCPU IDs */ |
| 45 | #define AVIC_VCPU_ID_BITS 8 | 31 | #define AVIC_VCPU_ID_BITS 8 |
| 46 | #define AVIC_VCPU_ID_MASK ((1 << AVIC_VCPU_ID_BITS) - 1) | 32 | #define AVIC_VCPU_ID_MASK ((1 << AVIC_VCPU_ID_BITS) - 1) |
| @@ -73,12 +59,6 @@ struct amd_svm_iommu_ir { · 26 unmodified lines | |||
| 73 | void *data; /* Storing pointer to struct amd_ir_data */ | 59 | void *data; /* Storing pointer to struct amd_ir_data */ |
| 74 | }; | 60 | }; |
| 75 | 61 | ||
| 76 | enum avic_ipi_failure_cause { | ||
| 77 | AVIC_IPI_FAILURE_INVALID_INT_TYPE, | ||
| 78 | AVIC_IPI_FAILURE_TARGET_NOT_RUNNING, | ||
| 79 | AVIC_IPI_FAILURE_INVALID_TARGET, | ||
| 80 | AVIC_IPI_FAILURE_INVALID_BACKING_PAGE, | ||
| 81 | }; | ||
| 82 | 62 | ||
| 83 | /* Note: | 63 | /* Note: |
| 84 | * This function is called from IOMMU driver to notify | 64 | * This function is called from IOMMU driver to notify |
| @@ -289,6 +269,22 @@ static int avic_init_backing_page(struct kvm_vcpu *vcpu) · 204 unmodified lines | |||
| 289 | return 0; | 269 | return 0; |
| 290 | } | 270 | } |
| 291 | 271 | ||
| 272 | void avic_ring_doorbell(struct kvm_vcpu *vcpu) | ||
| 273 | { | ||
| 274 | /* | ||
| 275 | * Note, the vCPU could get migrated to a different pCPU at any point, | ||
| 276 | * which could result in signalling the wrong/previous pCPU. But if | ||
| 277 | * that happens the vCPU is guaranteed to do a VMRUN (after being | ||
| 278 | * migrated) and thus will process pending interrupts, i.e. a doorbell | ||
| 279 | * is not needed (and the spurious one is harmless). | ||
| 280 | */ | ||
| 281 | int cpu = READ_ONCE(vcpu->cpu); | ||
| 282 | |||
| 283 | if (cpu != get_cpu()) | ||
| 284 | wrmsrl(MSR_AMD64_SVM_AVIC_DOORBELL, kvm_cpu_get_apicid(cpu)); | ||
| 285 | put_cpu(); | ||
| 286 | } | ||
| 287 | |||
| 292 | static void avic_kick_target_vcpus(struct kvm *kvm, struct kvm_lapic *source, | 288 | static void avic_kick_target_vcpus(struct kvm *kvm, struct kvm_lapic *source, |
| 293 | u32 icrl, u32 icrh) | 289 | u32 icrl, u32 icrh) |
| 294 | { | 290 | { |
| @@ -304,8 +300,13 @@ static void avic_kick_target_vcpus(struct kvm *kvm, struct kvm_lapic *source, · 9 unmodified lines | |||
| 304 | kvm_for_each_vcpu(i, vcpu, kvm) { | 300 | kvm_for_each_vcpu(i, vcpu, kvm) { |
| 305 | if (kvm_apic_match_dest(vcpu, source, icrl & APIC_SHORT_MASK, | 301 | if (kvm_apic_match_dest(vcpu, source, icrl & APIC_SHORT_MASK, |
| 306 | GET_APIC_DEST_FIELD(icrh), | 302 | GET_APIC_DEST_FIELD(icrh), |
| 307 | icrl & APIC_DEST_MASK)) | 303 | icrl & APIC_DEST_MASK)) { |
| 308 | kvm_vcpu_wake_up(vcpu); | 304 | vcpu->arch.apic->irr_pending = true; |
| 305 | svm_complete_interrupt_delivery(vcpu, | ||
| 306 | icrl & APIC_MODE_MASK, | ||
| 307 | icrl & APIC_INT_LEVELTRIG, | ||
| 308 | icrl & APIC_VECTOR_MASK); | ||
| 309 | } | ||
| 309 | } | 310 | } |
| 310 | } | 311 | } |
| 311 | 312 | ||
| @@ -345,8 +346,6 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu) · 33 unmodified lines | |||
| 345 | avic_kick_target_vcpus(vcpu->kvm, apic, icrl, icrh); | 346 | avic_kick_target_vcpus(vcpu->kvm, apic, icrl, icrh); |
| 346 | break; | 347 | break; |
| 347 | case AVIC_IPI_FAILURE_INVALID_TARGET: | 348 | case AVIC_IPI_FAILURE_INVALID_TARGET: |
| 348 | WARN_ONCE(1, "Invalid IPI target: index=%u, vcpu=%d, icr=%#0x:%#0x\n", | ||
| 349 | index, vcpu->vcpu_id, icrh, icrl); | ||
| 350 | break; | 349 | break; |
| 351 | case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE: | 350 | case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE: |
| 352 | WARN_ONCE(1, "Invalid backing page\n"); | 351 | WARN_ONCE(1, "Invalid backing page\n"); |
| @@ -669,52 +668,6 @@ void svm_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap) · 316 unmodified lines | |||
| 669 | return; | 668 | return; |
| 670 | } | 669 | } |
| 671 | 670 | ||
| 672 | int svm_deliver_avic_intr(struct kvm_vcpu *vcpu, int vec) | ||
| 673 | { | ||
| 674 | if (!vcpu->arch.apicv_active) | ||
| 675 | return -1; | ||
| 676 | |||
| 677 | kvm_lapic_set_irr(vec, vcpu->arch.apic); | ||
| 678 | |||
| 679 | /* | ||
| 680 | * Pairs with the smp_mb_*() after setting vcpu->guest_mode in | ||
| 681 | * vcpu_enter_guest() to ensure the write to the vIRR is ordered before | ||
| 682 | * the read of guest_mode, which guarantees that either VMRUN will see | ||
| 683 | * and process the new vIRR entry, or that the below code will signal | ||
| 684 | * the doorbell if the vCPU is already running in the guest. | ||
| 685 | */ | ||
| 686 | smp_mb__after_atomic(); | ||
| 687 | |||
| 688 | /* | ||
| 689 | * Signal the doorbell to tell hardware to inject the IRQ if the vCPU | ||
| 690 | * is in the guest. If the vCPU is not in the guest, hardware will | ||
| 691 | * automatically process AVIC interrupts at VMRUN. | ||
| 692 | */ | ||
| 693 | if (vcpu->mode == IN_GUEST_MODE) { | ||
| 694 | int cpu = READ_ONCE(vcpu->cpu); | ||
| 695 | |||
| 696 | /* | ||
| 697 | * Note, the vCPU could get migrated to a different pCPU at any | ||
| 698 | * point, which could result in signalling the wrong/previous | ||
| 699 | * pCPU. But if that happens the vCPU is guaranteed to do a | ||
| 700 | * VMRUN (after being migrated) and thus will process pending | ||
| 701 | * interrupts, i.e. a doorbell is not needed (and the spurious | ||
| 702 | * one is harmless). | ||
| 703 | */ | ||
| 704 | if (cpu != get_cpu()) | ||
| 705 | wrmsrl(SVM_AVIC_DOORBELL, kvm_cpu_get_apicid(cpu)); | ||
| 706 | put_cpu(); | ||
| 707 | } else { | ||
| 708 | /* | ||
| 709 | * Wake the vCPU if it was blocking. KVM will then detect the | ||
| 710 | * pending IRQ when checking if the vCPU has a wake event. | ||
| 711 | */ | ||
| 712 | kvm_vcpu_wake_up(vcpu); | ||
| 713 | } | ||
| 714 | |||
| 715 | return 0; | ||
| 716 | } | ||
| 717 | |||
| 718 | bool svm_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu) | 671 | bool svm_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu) |
| 719 | { | 672 | { |
| 720 | return false; | 673 | return false; |
| @@ -1457,18 +1457,6 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, · 1456 unmodified lines | |||
| 1457 | !__nested_vmcb_check_save(vcpu, &save_cached)) | 1457 | !__nested_vmcb_check_save(vcpu, &save_cached)) |
| 1458 | goto out_free; | 1458 | goto out_free; |
| 1459 | 1459 | ||
| 1460 | /* | ||
| 1461 | * While the nested guest CR3 is already checked and set by | ||
| 1462 | * KVM_SET_SREGS, it was set when nested state was yet loaded, | ||
| 1463 | * thus MMU might not be initialized correctly. | ||
| 1464 | * Set it again to fix this. | ||
| 1465 | */ | ||
| 1466 | |||
| 1467 | ret = nested_svm_load_cr3(&svm->vcpu, vcpu->arch.cr3, | ||
| 1468 | nested_npt_enabled(svm), false); | ||
| 1469 | if (WARN_ON_ONCE(ret)) | ||
| 1470 | goto out_free; | ||
| 1471 | |||
| 1472 | 1460 | ||
| 1473 | /* | 1461 | /* |
| 1474 | * All checks done, we can enter guest mode. Userspace provides | 1462 | * All checks done, we can enter guest mode. Userspace provides |
| @@ -1494,6 +1482,20 @@ static int svm_set_nested_state(struct kvm_vcpu *vcpu, · 19 unmodified lines | |||
| 1494 | 1482 | ||
| 1495 | svm_switch_vmcb(svm, &svm->nested.vmcb02); | 1483 | svm_switch_vmcb(svm, &svm->nested.vmcb02); |
| 1496 | nested_vmcb02_prepare_control(svm); | 1484 | nested_vmcb02_prepare_control(svm); |
| 1485 | |||
| 1486 | /* | ||
| 1487 | * While the nested guest CR3 is already checked and set by | ||
| 1488 | * KVM_SET_SREGS, it was set when nested state was yet loaded, | ||
| 1489 | * thus MMU might not be initialized correctly. | ||
| 1490 | * Set it again to fix this. | ||
| 1491 | */ | ||
| 1492 | |||
| 1493 | ret = nested_svm_load_cr3(&svm->vcpu, vcpu->arch.cr3, | ||
| 1494 | nested_npt_enabled(svm), false); | ||
| 1495 | if (WARN_ON_ONCE(ret)) | ||
| 1496 | goto out_free; | ||
| 1497 | |||
| 1498 | |||
| 1497 | kvm_make_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu); | 1499 | kvm_make_request(KVM_REQ_GET_NESTED_STATE_PAGES, vcpu); |
| 1498 | ret = 0; | 1500 | ret = 0; |
| 1499 | out_free: | 1501 | out_free: |
| @@ -1585,6 +1585,7 @@ void svm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · 1584 unmodified lines | |||
| 1585 | { | 1585 | { |
| 1586 | struct vcpu_svm *svm = to_svm(vcpu); | 1586 | struct vcpu_svm *svm = to_svm(vcpu); |
| 1587 | u64 hcr0 = cr0; | 1587 | u64 hcr0 = cr0; |
| 1588 | bool old_paging = is_paging(vcpu); | ||
| 1588 | 1589 | ||
| 1589 | #ifdef CONFIG_X86_64 | 1590 | #ifdef CONFIG_X86_64 |
| 1590 | if (vcpu->arch.efer & EFER_LME && !vcpu->arch.guest_state_protected) { | 1591 | if (vcpu->arch.efer & EFER_LME && !vcpu->arch.guest_state_protected) { |
| @@ -1601,8 +1602,11 @@ void svm_set_cr0(struct kvm_vcpu *vcpu, unsigned long cr0) · 10 unmodified lines | |||
| 1601 | #endif | 1602 | #endif |
| 1602 | vcpu->arch.cr0 = cr0; | 1603 | vcpu->arch.cr0 = cr0; |
| 1603 | 1604 | ||
| 1604 | if (!npt_enabled) | 1605 | if (!npt_enabled) { |
| 1605 | hcr0 |= X86_CR0_PG | X86_CR0_WP; | 1606 | hcr0 |= X86_CR0_PG | X86_CR0_WP; |
| 1607 | if (old_paging != is_paging(vcpu)) | ||
| 1608 | svm_set_cr4(vcpu, kvm_read_cr4(vcpu)); | ||
| 1609 | } | ||
| 1606 | 1610 | ||
| 1607 | /* | 1611 | /* |
| 1608 | * re-enable caching here because the QEMU bios | 1612 | * re-enable caching here because the QEMU bios |
| @@ -1646,8 +1650,12 @@ void svm_set_cr4(struct kvm_vcpu *vcpu, unsigned long cr4) · 37 unmodified lines | |||
| 1646 | svm_flush_tlb(vcpu); | 1650 | svm_flush_tlb(vcpu); |
| 1647 | 1651 | ||
| 1648 | vcpu->arch.cr4 = cr4; | 1652 | vcpu->arch.cr4 = cr4; |
| 1649 | if (!npt_enabled) | 1653 | if (!npt_enabled) { |
| 1650 | cr4 |= X86_CR4_PAE; | 1654 | cr4 |= X86_CR4_PAE; |
| 1655 | |||
| 1656 | if (!is_paging(vcpu)) | ||
| 1657 | cr4 &= ~(X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_PKE); | ||
| 1658 | } | ||
| 1651 | cr4 |= host_cr4_mce; | 1659 | cr4 |= host_cr4_mce; |
| 1652 | to_svm(vcpu)->vmcb->save.cr4 = cr4; | 1660 | to_svm(vcpu)->vmcb->save.cr4 = cr4; |
| 1653 | vmcb_mark_dirty(to_svm(vcpu)->vmcb, VMCB_CR); | 1661 | vmcb_mark_dirty(to_svm(vcpu)->vmcb, VMCB_CR); |
| @@ -2685,8 +2693,23 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr) · 1031 unmodified lines | |||
| 2685 | u64 data = msr->data; | 2693 | u64 data = msr->data; |
| 2686 | switch (ecx) { | 2694 | switch (ecx) { |
| 2687 | case MSR_AMD64_TSC_RATIO: | 2695 | case MSR_AMD64_TSC_RATIO: |
| 2688 | if (!msr->host_initiated && !svm->tsc_scaling_enabled) | 2696 | |
| 2689 | return 1; | 2697 | if (!svm->tsc_scaling_enabled) { |
| 2698 | |||
| 2699 | if (!msr->host_initiated) | ||
| 2700 | return 1; | ||
| 2701 | /* | ||
| 2702 | * In case TSC scaling is not enabled, always | ||
| 2703 | * leave this MSR at the default value. | ||
| 2704 | * | ||
| 2705 | * Due to bug in qemu 6.2.0, it would try to set | ||
| 2706 | * this msr to 0 if tsc scaling is not enabled. | ||
| 2707 | * Ignore this value as well. | ||
| 2708 | */ | ||
| 2709 | if (data != 0 && data != svm->tsc_ratio_msr) | ||
| 2710 | return 1; | ||
| 2711 | break; | ||
| 2712 | } | ||
| 2690 | 2713 | ||
| 2691 | if (data & TSC_RATIO_RSVD) | 2714 | if (data & TSC_RATIO_RSVD) |
| 2692 | return 1; | 2715 | return 1; |
| @@ -3291,6 +3314,55 @@ static void svm_set_irq(struct kvm_vcpu *vcpu) · 598 unmodified lines | |||
| 3291 | SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_INTR; | 3314 | SVM_EVTINJ_VALID | SVM_EVTINJ_TYPE_INTR; |
| 3292 | } | 3315 | } |
| 3293 | 3316 | ||
| 3317 | void svm_complete_interrupt_delivery(struct kvm_vcpu *vcpu, int delivery_mode, | ||
| 3318 | int trig_mode, int vector) | ||
| 3319 | { | ||
| 3320 | /* | ||
| 3321 | * vcpu->arch.apicv_active must be read after vcpu->mode. | ||
| 3322 | * Pairs with smp_store_release in vcpu_enter_guest. | ||
| 3323 | */ | ||
| 3324 | bool in_guest_mode = (smp_load_acquire(&vcpu->mode) == IN_GUEST_MODE); | ||
| 3325 | |||
| 3326 | if (!READ_ONCE(vcpu->arch.apicv_active)) { | ||
| 3327 | /* Process the interrupt via inject_pending_event */ | ||
| 3328 | kvm_make_request(KVM_REQ_EVENT, vcpu); | ||
| 3329 | kvm_vcpu_kick(vcpu); | ||
| 3330 | return; | ||
| 3331 | } | ||
| 3332 | |||
| 3333 | trace_kvm_apicv_accept_irq(vcpu->vcpu_id, delivery_mode, trig_mode, vector); | ||
| 3334 | if (in_guest_mode) { | ||
| 3335 | /* | ||
| 3336 | * Signal the doorbell to tell hardware to inject the IRQ. If | ||
| 3337 | * the vCPU exits the guest before the doorbell chimes, hardware | ||
| 3338 | * will automatically process AVIC interrupts at the next VMRUN. | ||
| 3339 | */ | ||
| 3340 | avic_ring_doorbell(vcpu); | ||
| 3341 | } else { | ||
| 3342 | /* | ||
| 3343 | * Wake the vCPU if it was blocking. KVM will then detect the | ||
| 3344 | * pending IRQ when checking if the vCPU has a wake event. | ||
| 3345 | */ | ||
| 3346 | kvm_vcpu_wake_up(vcpu); | ||
| 3347 | } | ||
| 3348 | } | ||
| 3349 | |||
| 3350 | static void svm_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, | ||
| 3351 | int trig_mode, int vector) | ||
| 3352 | { | ||
| 3353 | kvm_lapic_set_irr(vector, apic); | ||
| 3354 | |||
| 3355 | /* | ||
| 3356 | * Pairs with the smp_mb_*() after setting vcpu->guest_mode in | ||
| 3357 | * vcpu_enter_guest() to ensure the write to the vIRR is ordered before | ||
| 3358 | * the read of guest_mode. This guarantees that either VMRUN will see | ||
| 3359 | * and process the new vIRR entry, or that svm_complete_interrupt_delivery | ||
| 3360 | * will signal the doorbell if the CPU has already entered the guest. | ||
| 3361 | */ | ||
| 3362 | smp_mb__after_atomic(); | ||
| 3363 | svm_complete_interrupt_delivery(apic->vcpu, delivery_mode, trig_mode, vector); | ||
| 3364 | } | ||
| 3365 | |||
| 3294 | static void svm_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) | 3366 | static void svm_update_cr8_intercept(struct kvm_vcpu *vcpu, int tpr, int irr) |
| 3295 | { | 3367 | { |
| 3296 | struct vcpu_svm *svm = to_svm(vcpu); | 3368 | struct vcpu_svm *svm = to_svm(vcpu); |
| @@ -3338,11 +3410,13 @@ static int svm_nmi_allowed(struct kvm_vcpu *vcpu, bool for_injection) · 41 unmodified lines | |||
| 3338 | if (svm->nested.nested_run_pending) | 3410 | if (svm->nested.nested_run_pending) |
| 3339 | return -EBUSY; | 3411 | return -EBUSY; |
| 3340 | 3412 | ||
| 3413 | if (svm_nmi_blocked(vcpu)) | ||
| 3414 | return 0; | ||
| 3415 | |||
| 3341 | /* An NMI must not be injected into L2 if it's supposed to VM-Exit. */ | 3416 | /* An NMI must not be injected into L2 if it's supposed to VM-Exit. */ |
| 3342 | if (for_injection && is_guest_mode(vcpu) && nested_exit_on_nmi(svm)) | 3417 | if (for_injection && is_guest_mode(vcpu) && nested_exit_on_nmi(svm)) |
| 3343 | return -EBUSY; | 3418 | return -EBUSY; |
| 3344 | 3419 | return 1; | |
| 3345 | return !svm_nmi_blocked(vcpu); | ||
| 3346 | } | 3420 | } |
| 3347 | 3421 | ||
| 3348 | static bool svm_get_nmi_mask(struct kvm_vcpu *vcpu) | 3422 | static bool svm_get_nmi_mask(struct kvm_vcpu *vcpu) |
| @@ -3394,9 +3468,13 @@ bool svm_interrupt_blocked(struct kvm_vcpu *vcpu) · 45 unmodified lines | |||
| 3394 | static int svm_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection) | 3468 | static int svm_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection) |
| 3395 | { | 3469 | { |
| 3396 | struct vcpu_svm *svm = to_svm(vcpu); | 3470 | struct vcpu_svm *svm = to_svm(vcpu); |
| 3471 | |||
| 3397 | if (svm->nested.nested_run_pending) | 3472 | if (svm->nested.nested_run_pending) |
| 3398 | return -EBUSY; | 3473 | return -EBUSY; |
| 3399 | 3474 | ||
| 3475 | if (svm_interrupt_blocked(vcpu)) | ||
| 3476 | return 0; | ||
| 3477 | |||
| 3400 | /* | 3478 | /* |
| 3401 | * An IRQ must not be injected into L2 if it's supposed to VM-Exit, | 3479 | * An IRQ must not be injected into L2 if it's supposed to VM-Exit, |
| 3402 | * e.g. if the IRQ arrived asynchronously after checking nested events. | 3480 | * e.g. if the IRQ arrived asynchronously after checking nested events. |
| @@ -3404,7 +3482,7 @@ static int svm_interrupt_allowed(struct kvm_vcpu *vcpu, bool for_injection) · 1 unmodified line | |||
| 3404 | if (for_injection && is_guest_mode(vcpu) && nested_exit_on_intr(svm)) | 3482 | if (for_injection && is_guest_mode(vcpu) && nested_exit_on_intr(svm)) |
| 3405 | return -EBUSY; | 3483 | return -EBUSY; |
| 3406 | 3484 | ||
| 3407 | return !svm_interrupt_blocked(vcpu); | 3485 | return 1; |
| 3408 | } | 3486 | } |
| 3409 | 3487 | ||
| 3410 | static void svm_enable_irq_window(struct kvm_vcpu *vcpu) | 3488 | static void svm_enable_irq_window(struct kvm_vcpu *vcpu) |
| @@ -3615,7 +3693,7 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu) · 204 unmodified lines | |||
| 3615 | struct vcpu_svm *svm = to_svm(vcpu); | 3693 | struct vcpu_svm *svm = to_svm(vcpu); |
| 3616 | unsigned long vmcb_pa = svm->current_vmcb->pa; | 3694 | unsigned long vmcb_pa = svm->current_vmcb->pa; |
| 3617 | 3695 | ||
| 3618 | kvm_guest_enter_irqoff(); | 3696 | guest_state_enter_irqoff(); |
| 3619 | 3697 | ||
| 3620 | if (sev_es_guest(vcpu->kvm)) { | 3698 | if (sev_es_guest(vcpu->kvm)) { |
| 3621 | __svm_sev_es_vcpu_run(vmcb_pa); | 3699 | __svm_sev_es_vcpu_run(vmcb_pa); |
| @@ -3635,7 +3713,7 @@ static noinstr void svm_vcpu_enter_exit(struct kvm_vcpu *vcpu) · 13 unmodified lines | |||
| 3635 | vmload(__sme_page_pa(sd->save_area)); | 3713 | vmload(__sme_page_pa(sd->save_area)); |
| 3636 | } | 3714 | } |
| 3637 | 3715 | ||
| 3638 | kvm_guest_exit_irqoff(); | 3716 | guest_state_exit_irqoff(); |
| 3639 | } | 3717 | } |
| 3640 | 3718 | ||
| 3641 | static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) | 3719 | static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu) |
| @@ -4135,11 +4213,14 @@ static int svm_smi_allowed(struct kvm_vcpu *vcpu, bool for_injection) · 493 unmodified lines | |||
| 4135 | if (svm->nested.nested_run_pending) | 4213 | if (svm->nested.nested_run_pending) |
| 4136 | return -EBUSY; | 4214 | return -EBUSY; |
| 4137 | 4215 | ||
| 4216 | if (svm_smi_blocked(vcpu)) | ||
| 4217 | return 0; | ||
| 4218 | |||
| 4138 | /* An SMI must not be injected into L2 if it's supposed to VM-Exit. */ | 4219 | /* An SMI must not be injected into L2 if it's supposed to VM-Exit. */ |
| 4139 | if (for_injection && is_guest_mode(vcpu) && nested_exit_on_smi(svm)) | 4220 | if (for_injection && is_guest_mode(vcpu) && nested_exit_on_smi(svm)) |
| 4140 | return -EBUSY; | 4221 | return -EBUSY; |
| 4141 | 4222 | ||
| 4142 | return !svm_smi_blocked(vcpu); | 4223 | return 1; |
| 4143 | } | 4224 | } |
| 4144 | 4225 | ||
| 4145 | static int svm_enter_smm(struct kvm_vcpu *vcpu, char *smstate) | 4226 | static int svm_enter_smm(struct kvm_vcpu *vcpu, char *smstate) |
| @@ -4233,11 +4314,18 @@ static int svm_leave_smm(struct kvm_vcpu *vcpu, const char *smstate) · 87 unmodified lines | |||
| 4233 | * Enter the nested guest now | 4314 | * Enter the nested guest now |
| 4234 | */ | 4315 | */ |
| 4235 | 4316 | ||
| 4317 | vmcb_mark_all_dirty(svm->vmcb01.ptr); | ||
| 4318 | |||
| 4236 | vmcb12 = map.hva; | 4319 | vmcb12 = map.hva; |
| 4237 | nested_copy_vmcb_control_to_cache(svm, &vmcb12->control); | 4320 | nested_copy_vmcb_control_to_cache(svm, &vmcb12->control); |
| 4238 | nested_copy_vmcb_save_to_cache(svm, &vmcb12->save); | 4321 | nested_copy_vmcb_save_to_cache(svm, &vmcb12->save); |
| 4239 | ret = enter_svm_guest_mode(vcpu, vmcb12_gpa, vmcb12, false); | 4322 | ret = enter_svm_guest_mode(vcpu, vmcb12_gpa, vmcb12, false); |
| 4240 | 4323 | ||
| 4324 | if (ret) | ||
| 4325 | goto unmap_save; | ||
| 4326 | |||
| 4327 | svm->nested.nested_run_pending = 1; | ||
| 4328 | |||
| 4241 | unmap_save: | 4329 | unmap_save: |
| 4242 | kvm_vcpu_unmap(vcpu, &map_save, true); | 4330 | kvm_vcpu_unmap(vcpu, &map_save, true); |
| 4243 | unmap_map: | 4331 | unmap_map: |
| @@ -4545,7 +4633,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { · 301 unmodified lines | |||
| 4545 | .pmu_ops = &amd_pmu_ops, | 4633 | .pmu_ops = &amd_pmu_ops, |
| 4546 | .nested_ops = &svm_nested_ops, | 4634 | .nested_ops = &svm_nested_ops, |
| 4547 | 4635 | ||
| 4548 | .deliver_posted_interrupt = svm_deliver_avic_intr, | 4636 | .deliver_interrupt = svm_deliver_interrupt, |
| 4549 | .dy_apicv_has_pending_interrupt = svm_dy_apicv_has_pending_interrupt, | 4637 | .dy_apicv_has_pending_interrupt = svm_dy_apicv_has_pending_interrupt, |
| 4550 | .update_pi_irte = svm_update_pi_irte, | 4638 | .update_pi_irte = svm_update_pi_irte, |
| 4551 | .setup_mce = svm_setup_mce, | 4639 | .setup_mce = svm_setup_mce, |
| @@ -4622,6 +4710,7 @@ static __init void svm_set_cpu_caps(void) · 70 unmodified lines | |||
| 4622 | /* CPUID 0x80000001 and 0x8000000A (SVM features) */ | 4710 | /* CPUID 0x80000001 and 0x8000000A (SVM features) */ |
| 4623 | if (nested) { | 4711 | if (nested) { |
| 4624 | kvm_cpu_cap_set(X86_FEATURE_SVM); | 4712 | kvm_cpu_cap_set(X86_FEATURE_SVM); |
| 4713 | kvm_cpu_cap_set(X86_FEATURE_VMCBCLEAN); | ||
| 4625 | 4714 | ||
| 4626 | if (nrips) | 4715 | if (nrips) |
| 4627 | kvm_cpu_cap_set(X86_FEATURE_NRIPS); | 4716 | kvm_cpu_cap_set(X86_FEATURE_NRIPS); |
| @@ -489,6 +489,8 @@ void svm_set_gif(struct vcpu_svm *svm, bool value); · 488 unmodified lines | |||
| 489 | int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code); | 489 | int svm_invoke_exit_handler(struct kvm_vcpu *vcpu, u64 exit_code); |
| 490 | void set_msr_interception(struct kvm_vcpu *vcpu, u32 *msrpm, u32 msr, | 490 | void set_msr_interception(struct kvm_vcpu *vcpu, u32 *msrpm, u32 msr, |
| 491 | int read, int write); | 491 | int read, int write); |
| 492 | void svm_complete_interrupt_delivery(struct kvm_vcpu *vcpu, int delivery_mode, | ||
| 493 | int trig_mode, int vec); | ||
| 492 | 494 | ||
| 493 | /* nested.c */ | 495 | /* nested.c */ |
| 494 | 496 | ||
| @@ -556,17 +558,6 @@ extern struct kvm_x86_nested_ops svm_nested_ops; · 61 unmodified lines | |||
| 556 | 558 | ||
| 557 | /* avic.c */ | 559 | /* avic.c */ |
| 558 | 560 | ||
| 559 | #define AVIC_LOGICAL_ID_ENTRY_GUEST_PHYSICAL_ID_MASK (0xFF) | ||
| 560 | #define AVIC_LOGICAL_ID_ENTRY_VALID_BIT 31 | ||
| 561 | #define AVIC_LOGICAL_ID_ENTRY_VALID_MASK (1 << 31) | ||
| 562 | |||
| 563 | #define AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK (0xFFULL) | ||
| 564 | #define AVIC_PHYSICAL_ID_ENTRY_BACKING_PAGE_MASK (0xFFFFFFFFFFULL << 12) | ||
| 565 | #define AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK (1ULL << 62) | ||
| 566 | #define AVIC_PHYSICAL_ID_ENTRY_VALID_MASK (1ULL << 63) | ||
| 567 | |||
| 568 | #define VMCB_AVIC_APIC_BAR_MASK 0xFFFFFFFFFF000ULL | ||
| 569 | |||
| 570 | int avic_ga_log_notifier(u32 ga_tag); | 561 | int avic_ga_log_notifier(u32 ga_tag); |
| 571 | void avic_vm_destroy(struct kvm *kvm); | 562 | void avic_vm_destroy(struct kvm *kvm); |
| 572 | int avic_vm_init(struct kvm *kvm); | 563 | int avic_vm_init(struct kvm *kvm); |
| @@ -583,12 +574,12 @@ bool svm_check_apicv_inhibit_reasons(ulong bit); · 10 unmodified lines | |||
| 583 | void svm_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap); | 574 | void svm_load_eoi_exitmap(struct kvm_vcpu *vcpu, u64 *eoi_exit_bitmap); |
| 584 | void svm_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr); | 575 | void svm_hwapic_irr_update(struct kvm_vcpu *vcpu, int max_irr); |
| 585 | void svm_hwapic_isr_update(struct kvm_vcpu *vcpu, int max_isr); | 576 | void svm_hwapic_isr_update(struct kvm_vcpu *vcpu, int max_isr); |
| 586 | int svm_deliver_avic_intr(struct kvm_vcpu *vcpu, int vec); | ||
| 587 | bool svm_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu); | 577 | bool svm_dy_apicv_has_pending_interrupt(struct kvm_vcpu *vcpu); |
| 588 | int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq, | 578 | int svm_update_pi_irte(struct kvm *kvm, unsigned int host_irq, |
| 589 | uint32_t guest_irq, bool set); | 579 | uint32_t guest_irq, bool set); |
| 590 | void avic_vcpu_blocking(struct kvm_vcpu *vcpu); | 580 | void avic_vcpu_blocking(struct kvm_vcpu *vcpu); |
| 591 | void avic_vcpu_unblocking(struct kvm_vcpu *vcpu); | 581 | void avic_vcpu_unblocking(struct kvm_vcpu *vcpu); |
| 582 | void avic_ring_doorbell(struct kvm_vcpu *vcpu); | ||
| 592 | 583 | ||
| 593 | /* sev.c */ | 584 | /* sev.c */ |
| 594 | 585 | ||
| @@ -4041,6 +4041,21 @@ static int vmx_deliver_posted_interrupt(struct kvm_vcpu *vcpu, int vector) · 4040 unmodified lines | |||
| 4041 | return 0; | 4041 | return 0; |
| 4042 | } | 4042 | } |
| 4043 | 4043 | ||
| 4044 | static void vmx_deliver_interrupt(struct kvm_lapic *apic, int delivery_mode, | ||
| 4045 | int trig_mode, int vector) | ||
| 4046 | { | ||
| 4047 | struct kvm_vcpu *vcpu = apic->vcpu; | ||
| 4048 | |||
| 4049 | if (vmx_deliver_posted_interrupt(vcpu, vector)) { | ||
| 4050 | kvm_lapic_set_irr(vector, apic); | ||
| 4051 | kvm_make_request(KVM_REQ_EVENT, vcpu); | ||
| 4052 | kvm_vcpu_kick(vcpu); | ||
| 4053 | } else { | ||
| 4054 | trace_kvm_apicv_accept_irq(vcpu->vcpu_id, delivery_mode, | ||
| 4055 | trig_mode, vector); | ||
| 4056 | } | ||
| 4057 | } | ||
| 4058 | |||
| 4044 | /* | 4059 | /* |
| 4045 | * Set up the vmcs's constant host-state fields, i.e., host-state fields that | 4060 | * Set up the vmcs's constant host-state fields, i.e., host-state fields that |
| 4046 | * will not change in the lifetime of the guest. | 4061 | * will not change in the lifetime of the guest. |
| @@ -6754,7 +6769,7 @@ static fastpath_t vmx_exit_handlers_fastpath(struct kvm_vcpu *vcpu) · 2707 unmodified lines | |||
| 6754 | static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, | 6769 | static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, |
| 6755 | struct vcpu_vmx *vmx) | 6770 | struct vcpu_vmx *vmx) |
| 6756 | { | 6771 | { |
| 6757 | kvm_guest_enter_irqoff(); | 6772 | guest_state_enter_irqoff(); |
| 6758 | 6773 | ||
| 6759 | /* L1D Flush includes CPU buffer clear to mitigate MDS */ | 6774 | /* L1D Flush includes CPU buffer clear to mitigate MDS */ |
| 6760 | if (static_branch_unlikely(&vmx_l1d_should_flush)) | 6775 | if (static_branch_unlikely(&vmx_l1d_should_flush)) |
| @@ -6770,7 +6785,7 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu, · 9 unmodified lines | |||
| 6770 | 6785 | ||
| 6771 | vcpu->arch.cr2 = native_read_cr2(); | 6786 | vcpu->arch.cr2 = native_read_cr2(); |
| 6772 | 6787 | ||
| 6773 | kvm_guest_exit_irqoff(); | 6788 | guest_state_exit_irqoff(); |
| 6774 | } | 6789 | } |
| 6775 | 6790 | ||
| 6776 | static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) | 6791 | static fastpath_t vmx_vcpu_run(struct kvm_vcpu *vcpu) |
| @@ -7644,6 +7659,7 @@ static int vmx_leave_smm(struct kvm_vcpu *vcpu, const char *smstate) · 867 unmodified lines | |||
| 7644 | if (ret) | 7659 | if (ret) |
| 7645 | return ret; | 7660 | return ret; |
| 7646 | 7661 | ||
| 7662 | vmx->nested.nested_run_pending = 1; | ||
| 7647 | vmx->nested.smm.guest_mode = false; | 7663 | vmx->nested.smm.guest_mode = false; |
| 7648 | } | 7664 | } |
| 7649 | return 0; | 7665 | return 0; |
| @@ -7768,7 +7784,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { · 118 unmodified lines | |||
| 7768 | .hwapic_isr_update = vmx_hwapic_isr_update, | 7784 | .hwapic_isr_update = vmx_hwapic_isr_update, |
| 7769 | .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, | 7785 | .guest_apic_has_interrupt = vmx_guest_apic_has_interrupt, |
| 7770 | .sync_pir_to_irr = vmx_sync_pir_to_irr, | 7786 | .sync_pir_to_irr = vmx_sync_pir_to_irr, |
| 7771 | .deliver_posted_interrupt = vmx_deliver_posted_interrupt, | 7787 | .deliver_interrupt = vmx_deliver_interrupt, |
| 7772 | .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, | 7788 | .dy_apicv_has_pending_interrupt = pi_has_pending_interrupt, |
| 7773 | 7789 | ||
| 7774 | .set_tss_addr = vmx_set_tss_addr, | 7790 | .set_tss_addr = vmx_set_tss_addr, |
| @@ -90,6 +90,8 @@ · 89 unmodified lines | |||
| 90 | u64 __read_mostly kvm_mce_cap_supported = MCG_CTL_P | MCG_SER_P; | 90 | u64 __read_mostly kvm_mce_cap_supported = MCG_CTL_P | MCG_SER_P; |
| 91 | EXPORT_SYMBOL_GPL(kvm_mce_cap_supported); | 91 | EXPORT_SYMBOL_GPL(kvm_mce_cap_supported); |
| 92 | 92 | ||
| 93 | #define ERR_PTR_USR(e) ((void __user *)ERR_PTR(e)) | ||
| 94 | |||
| 93 | #define emul_to_vcpu(ctxt) \ | 95 | #define emul_to_vcpu(ctxt) \ |
| 94 | ((struct kvm_vcpu *)(ctxt)->vcpu) | 96 | ((struct kvm_vcpu *)(ctxt)->vcpu) |
| 95 | 97 | ||
| @@ -982,6 +984,18 @@ void kvm_load_host_xsave_state(struct kvm_vcpu *vcpu) · 886 unmodified lines | |||
| 982 | } | 984 | } |
| 983 | EXPORT_SYMBOL_GPL(kvm_load_host_xsave_state); | 985 | EXPORT_SYMBOL_GPL(kvm_load_host_xsave_state); |
| 984 | 986 | ||
| 987 | static inline u64 kvm_guest_supported_xcr0(struct kvm_vcpu *vcpu) | ||
| 988 | { | ||
| 989 | return vcpu->arch.guest_fpu.fpstate->user_xfeatures; | ||
| 990 | } | ||
| 991 | |||
| 992 | #ifdef CONFIG_X86_64 | ||
| 993 | static inline u64 kvm_guest_supported_xfd(struct kvm_vcpu *vcpu) | ||
| 994 | { | ||
| 995 | return kvm_guest_supported_xcr0(vcpu) & XFEATURE_MASK_USER_DYNAMIC; | ||
| 996 | } | ||
| 997 | #endif | ||
| 998 | |||
| 985 | static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) | 999 | static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) |
| 986 | { | 1000 | { |
| 987 | u64 xcr0 = xcr; | 1001 | u64 xcr0 = xcr; |
| @@ -1001,7 +1015,7 @@ static int __kvm_set_xcr(struct kvm_vcpu *vcpu, u32 index, u64 xcr) · 13 unmodified lines | |||
| 1001 | * saving. However, xcr0 bit 0 is always set, even if the | 1015 | * saving. However, xcr0 bit 0 is always set, even if the |
| 1002 | * emulated CPU does not support XSAVE (see kvm_vcpu_reset()). | 1016 | * emulated CPU does not support XSAVE (see kvm_vcpu_reset()). |
| 1003 | */ | 1017 | */ |
| 1004 | valid_bits = vcpu->arch.guest_supported_xcr0 | XFEATURE_MASK_FP; | 1018 | valid_bits = kvm_guest_supported_xcr0(vcpu) | XFEATURE_MASK_FP; |
| 1005 | if (xcr0 & ~valid_bits) | 1019 | if (xcr0 & ~valid_bits) |
| 1006 | return 1; | 1020 | return 1; |
| 1007 | 1021 | ||
| @@ -2349,10 +2363,12 @@ static u64 compute_guest_tsc(struct kvm_vcpu *vcpu, s64 kernel_ns) · 1341 unmodified lines | |||
| 2349 | return tsc; | 2363 | return tsc; |
| 2350 | } | 2364 | } |
| 2351 | 2365 | ||
| 2366 | #ifdef CONFIG_X86_64 | ||
| 2352 | static inline int gtod_is_based_on_tsc(int mode) | 2367 | static inline int gtod_is_based_on_tsc(int mode) |
| 2353 | { | 2368 | { |
| 2354 | return mode == VDSO_CLOCKMODE_TSC || mode == VDSO_CLOCKMODE_HVCLOCK; | 2369 | return mode == VDSO_CLOCKMODE_TSC || mode == VDSO_CLOCKMODE_HVCLOCK; |
| 2355 | } | 2370 | } |
| 2371 | #endif | ||
| 2356 | 2372 | ||
| 2357 | static void kvm_track_tsc_matching(struct kvm_vcpu *vcpu) | 2373 | static void kvm_track_tsc_matching(struct kvm_vcpu *vcpu) |
| 2358 | { | 2374 | { |
| @@ -3704,8 +3720,7 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · 1345 unmodified lines | |||
| 3704 | !guest_cpuid_has(vcpu, X86_FEATURE_XFD)) | 3720 | !guest_cpuid_has(vcpu, X86_FEATURE_XFD)) |
| 3705 | return 1; | 3721 | return 1; |
| 3706 | 3722 | ||
| 3707 | if (data & ~(XFEATURE_MASK_USER_DYNAMIC & | 3723 | if (data & ~kvm_guest_supported_xfd(vcpu)) |
| 3708 | vcpu->arch.guest_supported_xcr0)) | ||
| 3709 | return 1; | 3724 | return 1; |
| 3710 | 3725 | ||
| 3711 | fpu_update_guest_xfd(&vcpu->arch.guest_fpu, data); | 3726 | fpu_update_guest_xfd(&vcpu->arch.guest_fpu, data); |
| @@ -3715,8 +3730,7 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct msr_data *msr_info) · 3 unmodified lines | |||
| 3715 | !guest_cpuid_has(vcpu, X86_FEATURE_XFD)) | 3730 | !guest_cpuid_has(vcpu, X86_FEATURE_XFD)) |
| 3716 | return 1; | 3731 | return 1; |
| 3717 | 3732 | ||
| 3718 | if (data & ~(XFEATURE_MASK_USER_DYNAMIC & | 3733 | if (data & ~kvm_guest_supported_xfd(vcpu)) |
| 3719 | vcpu->arch.guest_supported_xcr0)) | ||
| 3720 | return 1; | 3734 | return 1; |
| 3721 | 3735 | ||
| 3722 | vcpu->arch.guest_fpu.xfd_err = data; | 3736 | vcpu->arch.guest_fpu.xfd_err = data; |
| @@ -4231,6 +4245,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) · 508 unmodified lines | |||
| 4231 | case KVM_CAP_EXIT_ON_EMULATION_FAILURE: | 4245 | case KVM_CAP_EXIT_ON_EMULATION_FAILURE: |
| 4232 | case KVM_CAP_VCPU_ATTRIBUTES: | 4246 | case KVM_CAP_VCPU_ATTRIBUTES: |
| 4233 | case KVM_CAP_SYS_ATTRIBUTES: | 4247 | case KVM_CAP_SYS_ATTRIBUTES: |
| 4248 | case KVM_CAP_ENABLE_CAP: | ||
| 4234 | r = 1; | 4249 | r = 1; |
| 4235 | break; | 4250 | break; |
| 4236 | case KVM_CAP_EXIT_HYPERCALL: | 4251 | case KVM_CAP_EXIT_HYPERCALL: |
| @@ -4340,7 +4355,7 @@ static inline void __user *kvm_get_attr_addr(struct kvm_device_attr *attr) · 103 unmodified lines | |||
| 4340 | void __user *uaddr = (void __user*)(unsigned long)attr->addr; | 4355 | void __user *uaddr = (void __user*)(unsigned long)attr->addr; |
| 4341 | 4356 | ||
| 4342 | if ((u64)(unsigned long)uaddr != attr->addr) | 4357 | if ((u64)(unsigned long)uaddr != attr->addr) |
| 4343 | return ERR_PTR(-EFAULT); | 4358 | return ERR_PTR_USR(-EFAULT); |
| 4344 | return uaddr; | 4359 | return uaddr; |
| 4345 | } | 4360 | } |
| 4346 | 4361 | ||
| @@ -8940,6 +8955,13 @@ static int kvm_pv_clock_pairing(struct kvm_vcpu *vcpu, gpa_t paddr, · 4593 unmodified lines | |||
| 8940 | if (clock_type != KVM_CLOCK_PAIRING_WALLCLOCK) | 8955 | if (clock_type != KVM_CLOCK_PAIRING_WALLCLOCK) |
| 8941 | return -KVM_EOPNOTSUPP; | 8956 | return -KVM_EOPNOTSUPP; |
| 8942 | 8957 | ||
| 8958 | /* | ||
| 8959 | * When tsc is in permanent catchup mode guests won't be able to use | ||
| 8960 | * pvclock_read_retry loop to get consistent view of pvclock | ||
| 8961 | */ | ||
| 8962 | if (vcpu->arch.tsc_always_catchup) | ||
| 8963 | return -KVM_EOPNOTSUPP; | ||
| 8964 | |||
| 8943 | if (!kvm_get_walltime_and_clockread(&ts, &cycle)) | 8965 | if (!kvm_get_walltime_and_clockread(&ts, &cycle)) |
| 8944 | return -KVM_EOPNOTSUPP; | 8966 | return -KVM_EOPNOTSUPP; |
| 8945 | 8967 | ||
| @@ -9981,7 +10003,9 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · 1035 unmodified lines | |||
| 9981 | * result in virtual interrupt delivery. | 10003 | * result in virtual interrupt delivery. |
| 9982 | */ | 10004 | */ |
| 9983 | local_irq_disable(); | 10005 | local_irq_disable(); |
| 9984 | vcpu->mode = IN_GUEST_MODE; | 10006 | |
| 10007 | /* Store vcpu->apicv_active before vcpu->mode. */ | ||
| 10008 | smp_store_release(&vcpu->mode, IN_GUEST_MODE); | ||
| 9985 | 10009 | ||
| 9986 | srcu_read_unlock(&vcpu->kvm->srcu, vcpu->srcu_idx); | 10010 | srcu_read_unlock(&vcpu->kvm->srcu, vcpu->srcu_idx); |
| 9987 | 10011 | ||
| @@ -10041,6 +10065,8 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · 53 unmodified lines | |||
| 10041 | set_debugreg(0, 7); | 10065 | set_debugreg(0, 7); |
| 10042 | } | 10066 | } |
| 10043 | 10067 | ||
| 10068 | guest_timing_enter_irqoff(); | ||
| 10069 | |||
| 10044 | for (;;) { | 10070 | for (;;) { |
| 10045 | /* | 10071 | /* |
| 10046 | * Assert that vCPU vs. VM APICv state is consistent. An APICv | 10072 | * Assert that vCPU vs. VM APICv state is consistent. An APICv |
| @@ -10125,7 +10151,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu) · 78 unmodified lines | |||
| 10125 | * of accounting via context tracking, but the loss of accuracy is | 10151 | * of accounting via context tracking, but the loss of accuracy is |
| 10126 | * acceptable for all known use cases. | 10152 | * acceptable for all known use cases. |
| 10127 | */ | 10153 | */ |
| 10128 | vtime_account_guest_exit(); | 10154 | guest_timing_exit_irqoff(); |
| 10129 | 10155 | ||
| 10130 | if (lapic_in_kernel(vcpu)) { | 10156 | if (lapic_in_kernel(vcpu)) { |
| 10131 | s64 delta = vcpu->arch.apic->lapic_timer.advance_expire_delta; | 10157 | s64 delta = vcpu->arch.apic->lapic_timer.advance_expire_delta; |
| @@ -11639,8 +11665,6 @@ void kvm_arch_sync_events(struct kvm *kvm) · 1507 unmodified lines | |||
| 11639 | kvm_free_pit(kvm); | 11665 | kvm_free_pit(kvm); |
| 11640 | } | 11666 | } |
| 11641 | 11667 | ||
| 11642 | #define ERR_PTR_USR(e) ((void __user *)ERR_PTR(e)) | ||
| 11643 | |||
| 11644 | /** | 11668 | /** |
| 11645 | * __x86_set_memory_region: Setup KVM internal memory slot | 11669 | * __x86_set_memory_region: Setup KVM internal memory slot |
| 11646 | * | 11670 | * |
| @@ -10,51 +10,6 @@ · 9 unmodified lines | |||
| 10 | 10 | ||
| 11 | void kvm_spurious_fault(void); | 11 | void kvm_spurious_fault(void); |
| 12 | 12 | ||
| 13 | static __always_inline void kvm_guest_enter_irqoff(void) | ||
| 14 | { | ||
| 15 | /* | ||
| 16 | * VMENTER enables interrupts (host state), but the kernel state is | ||
| 17 | * interrupts disabled when this is invoked. Also tell RCU about | ||
| 18 | * it. This is the same logic as for exit_to_user_mode(). | ||
| 19 | * | ||
| 20 | * This ensures that e.g. latency analysis on the host observes | ||
| 21 | * guest mode as interrupt enabled. | ||
| 22 | * | ||
| 23 | * guest_enter_irqoff() informs context tracking about the | ||
| 24 | * transition to guest mode and if enabled adjusts RCU state | ||
| 25 | * accordingly. | ||
| 26 | */ | ||
| 27 | instrumentation_begin(); | ||
| 28 | trace_hardirqs_on_prepare(); | ||
| 29 | lockdep_hardirqs_on_prepare(CALLER_ADDR0); | ||
| 30 | instrumentation_end(); | ||
| 31 | |||
| 32 | guest_enter_irqoff(); | ||
| 33 | lockdep_hardirqs_on(CALLER_ADDR0); | ||
| 34 | } | ||
| 35 | |||
| 36 | static __always_inline void kvm_guest_exit_irqoff(void) | ||
| 37 | { | ||
| 38 | /* | ||
| 39 | * VMEXIT disables interrupts (host state), but tracing and lockdep | ||
| 40 | * have them in state 'on' as recorded before entering guest mode. | ||
| 41 | * Same as enter_from_user_mode(). | ||
| 42 | * | ||
| 43 | * context_tracking_guest_exit() restores host context and reinstates | ||
| 44 | * RCU if enabled and required. | ||
| 45 | * | ||
| 46 | * This needs to be done immediately after VM-Exit, before any code | ||
| 47 | * that might contain tracepoints or call out to the greater world, | ||
| 48 | * e.g. before x86_spec_ctrl_restore_host(). | ||
| 49 | */ | ||
| 50 | lockdep_hardirqs_off(CALLER_ADDR0); | ||
| 51 | context_tracking_guest_exit(); | ||
| 52 | |||
| 53 | instrumentation_begin(); | ||
| 54 | trace_hardirqs_off_finish(); | ||
| 55 | instrumentation_end(); | ||
| 56 | } | ||
| 57 | |||
| 58 | #define KVM_NESTED_VMENTER_CONSISTENCY_CHECK(consistency_check) \ | 13 | #define KVM_NESTED_VMENTER_CONSISTENCY_CHECK(consistency_check) \ |
| 59 | ({ \ | 14 | ({ \ |
| 60 | bool failed = (consistency_check); \ | 15 | bool failed = (consistency_check); \ |
| @@ -133,32 +133,57 @@ static void kvm_xen_update_runstate(struct kvm_vcpu *v, int state) · 132 unmodified lines | |||
| 133 | void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, int state) | 133 | void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, int state) |
| 134 | { | 134 | { |
| 135 | struct kvm_vcpu_xen *vx = &v->arch.xen; | 135 | struct kvm_vcpu_xen *vx = &v->arch.xen; |
| 136 | struct gfn_to_hva_cache *ghc = &vx->runstate_cache; | ||
| 137 | struct kvm_memslots *slots = kvm_memslots(v->kvm); | ||
| 138 | bool atomic = (state == RUNSTATE_runnable); | ||
| 136 | uint64_t state_entry_time; | 139 | uint64_t state_entry_time; |
| 137 | unsigned int offset; | 140 | int __user *user_state; |
| 141 | uint64_t __user *user_times; | ||
| 138 | 142 | ||
| 139 | kvm_xen_update_runstate(v, state); | 143 | kvm_xen_update_runstate(v, state); |
| 140 | 144 | ||
| 141 | if (!vx->runstate_set) | 145 | if (!vx->runstate_set) |
| 142 | return; | 146 | return; |
| 143 | 147 | ||
| 144 | BUILD_BUG_ON(sizeof(struct compat_vcpu_runstate_info) != 0x2c); | 148 | if (unlikely(slots->generation != ghc->generation || kvm_is_error_hva(ghc->hva)) && |
| 149 | kvm_gfn_to_hva_cache_init(v->kvm, ghc, ghc->gpa, ghc->len)) | ||
| 150 | return; | ||
| 151 | |||
| 152 | /* We made sure it fits in a single page */ | ||
| 153 | BUG_ON(!ghc->memslot); | ||
| 154 | |||
| 155 | if (atomic) | ||
| 156 | pagefault_disable(); | ||
| 145 | 157 | ||
| 146 | offset = offsetof(struct compat_vcpu_runstate_info, state_entry_time); | ||
| 147 | #ifdef CONFIG_X86_64 | ||
| 148 | /* | 158 | /* |
| 149 | * The only difference is alignment of uint64_t in 32-bit. | 159 | * The only difference between 32-bit and 64-bit versions of the |
| 150 | * So the first field 'state' is accessed directly using | 160 | * runstate struct us the alignment of uint64_t in 32-bit, which |
| 151 | * offsetof() (where its offset happens to be zero), while the | 161 | * means that the 64-bit version has an additional 4 bytes of |
| 152 | * remaining fields which are all uint64_t, start at 'offset' | 162 | * padding after the first field 'state'. |
| 153 | * which we tweak here by adding 4. | 163 | * |
| 164 | * So we use 'int __user *user_state' to point to the state field, | ||
| 165 | * and 'uint64_t __user *user_times' for runstate_entry_time. So | ||
| 166 | * the actual array of time[] in each state starts at user_times[1]. | ||
| 154 | */ | 167 | */ |
| 168 | BUILD_BUG_ON(offsetof(struct vcpu_runstate_info, state) != 0); | ||
| 169 | BUILD_BUG_ON(offsetof(struct compat_vcpu_runstate_info, state) != 0); | ||
| 170 | user_state = (int __user *)ghc->hva; | ||
| 171 | |||
| 172 | BUILD_BUG_ON(sizeof(struct compat_vcpu_runstate_info) != 0x2c); | ||
| 173 | |||
| 174 | user_times = (uint64_t __user *)(ghc->hva + | ||
| 175 | offsetof(struct compat_vcpu_runstate_info, | ||
| 176 | state_entry_time)); | ||
| 177 | #ifdef CONFIG_X86_64 | ||
| 155 | BUILD_BUG_ON(offsetof(struct vcpu_runstate_info, state_entry_time) != | 178 | BUILD_BUG_ON(offsetof(struct vcpu_runstate_info, state_entry_time) != |
| 156 | offsetof(struct compat_vcpu_runstate_info, state_entry_time) + 4); | 179 | offsetof(struct compat_vcpu_runstate_info, state_entry_time) + 4); |
| 157 | BUILD_BUG_ON(offsetof(struct vcpu_runstate_info, time) != | 180 | BUILD_BUG_ON(offsetof(struct vcpu_runstate_info, time) != |
| 158 | offsetof(struct compat_vcpu_runstate_info, time) + 4); | 181 | offsetof(struct compat_vcpu_runstate_info, time) + 4); |
| 159 | 182 | ||
| 160 | if (v->kvm->arch.xen.long_mode) | 183 | if (v->kvm->arch.xen.long_mode) |
| 161 | offset = offsetof(struct vcpu_runstate_info, state_entry_time); | 184 | user_times = (uint64_t __user *)(ghc->hva + |
| 185 | offsetof(struct vcpu_runstate_info, | ||
| 186 | state_entry_time)); | ||
| 162 | #endif | 187 | #endif |
| 163 | /* | 188 | /* |
| 164 | * First write the updated state_entry_time at the appropriate | 189 | * First write the updated state_entry_time at the appropriate |
| @@ -172,10 +197,8 @@ void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, int state) · 7 unmodified lines | |||
| 172 | BUILD_BUG_ON(sizeof_field(struct compat_vcpu_runstate_info, state_entry_time) != | 197 | BUILD_BUG_ON(sizeof_field(struct compat_vcpu_runstate_info, state_entry_time) != |
| 173 | sizeof(state_entry_time)); | 198 | sizeof(state_entry_time)); |
| 174 | 199 | ||
| 175 | if (kvm_write_guest_offset_cached(v->kvm, &v->arch.xen.runstate_cache, | 200 | if (__put_user(state_entry_time, user_times)) |
| 176 | &state_entry_time, offset, | 201 | goto out; |
| 177 | sizeof(state_entry_time))) | ||
| 178 | return; | ||
| 179 | smp_wmb(); | 202 | smp_wmb(); |
| 180 | 203 | ||
| 181 | /* | 204 | /* |
| @@ -189,11 +212,8 @@ void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, int state) · 7 unmodified lines | |||
| 189 | BUILD_BUG_ON(sizeof_field(struct compat_vcpu_runstate_info, state) != | 212 | BUILD_BUG_ON(sizeof_field(struct compat_vcpu_runstate_info, state) != |
| 190 | sizeof(vx->current_runstate)); | 213 | sizeof(vx->current_runstate)); |
| 191 | 214 | ||
| 192 | if (kvm_write_guest_offset_cached(v->kvm, &v->arch.xen.runstate_cache, | 215 | if (__put_user(vx->current_runstate, user_state)) |
| 193 | &vx->current_runstate, | 216 | goto out; |
| 194 | offsetof(struct vcpu_runstate_info, state), | ||
| 195 | sizeof(vx->current_runstate))) | ||
| 196 | return; | ||
| 197 | 217 | ||
| 198 | /* | 218 | /* |
| 199 | * Write the actual runstate times immediately after the | 219 | * Write the actual runstate times immediately after the |
| @@ -208,24 +228,23 @@ void kvm_xen_update_runstate_guest(struct kvm_vcpu *v, int state) · 8 unmodified lines | |||
| 208 | BUILD_BUG_ON(sizeof_field(struct vcpu_runstate_info, time) != | 228 | BUILD_BUG_ON(sizeof_field(struct vcpu_runstate_info, time) != |
| 209 | sizeof(vx->runstate_times)); | 229 | sizeof(vx->runstate_times)); |
| 210 | 230 | ||
| 211 | if (kvm_write_guest_offset_cached(v->kvm, &v->arch.xen.runstate_cache, | 231 | if (__copy_to_user(user_times + 1, vx->runstate_times, sizeof(vx->runstate_times))) |
| 212 | &vx->runstate_times[0], | 232 | goto out; |
| 213 | offset + sizeof(u64), | ||
| 214 | sizeof(vx->runstate_times))) | ||
| 215 | return; | ||
| 216 | |||
| 217 | smp_wmb(); | 233 | smp_wmb(); |
| 218 | 234 | ||
| 219 | /* | 235 | /* |
| 220 | * Finally, clear the XEN_RUNSTATE_UPDATE bit in the guest's | 236 | * Finally, clear the XEN_RUNSTATE_UPDATE bit in the guest's |
| 221 | * runstate_entry_time field. | 237 | * runstate_entry_time field. |
| 222 | */ | 238 | */ |
| 223 | |||
| 224 | state_entry_time &= ~XEN_RUNSTATE_UPDATE; | 239 | state_entry_time &= ~XEN_RUNSTATE_UPDATE; |
| 225 | if (kvm_write_guest_offset_cached(v->kvm, &v->arch.xen.runstate_cache, | 240 | __put_user(state_entry_time, user_times); |
| 226 | &state_entry_time, offset, | 241 | smp_wmb(); |
| 227 | sizeof(state_entry_time))) | 242 | |
| 228 | return; | 243 | out: |
| 244 | mark_page_dirty_in_slot(v->kvm, ghc->memslot, ghc->gpa >> PAGE_SHIFT); | ||
| 245 | |||
| 246 | if (atomic) | ||
| 247 | pagefault_enable(); | ||
| 229 | } | 248 | } |
| 230 | 249 | ||
| 231 | int __kvm_xen_has_interrupt(struct kvm_vcpu *v) | 250 | int __kvm_xen_has_interrupt(struct kvm_vcpu *v) |
| @@ -443,6 +462,12 @@ int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) · 211 unmodified lines | |||
| 443 | break; | 462 | break; |
| 444 | } | 463 | } |
| 445 | 464 | ||
| 465 | /* It must fit within a single page */ | ||
| 466 | if ((data->u.gpa & ~PAGE_MASK) + sizeof(struct vcpu_info) > PAGE_SIZE) { | ||
| 467 | r = -EINVAL; | ||
| 468 | break; | ||
| 469 | } | ||
| 470 | |||
| 446 | r = kvm_gfn_to_hva_cache_init(vcpu->kvm, | 471 | r = kvm_gfn_to_hva_cache_init(vcpu->kvm, |
| 447 | &vcpu->arch.xen.vcpu_info_cache, | 472 | &vcpu->arch.xen.vcpu_info_cache, |
| 448 | data->u.gpa, | 473 | data->u.gpa, |
| @@ -460,6 +485,12 @@ int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) · 11 unmodified lines | |||
| 460 | break; | 485 | break; |
| 461 | } | 486 | } |
| 462 | 487 | ||
| 488 | /* It must fit within a single page */ | ||
| 489 | if ((data->u.gpa & ~PAGE_MASK) + sizeof(struct pvclock_vcpu_time_info) > PAGE_SIZE) { | ||
| 490 | r = -EINVAL; | ||
| 491 | break; | ||
| 492 | } | ||
| 493 | |||
| 463 | r = kvm_gfn_to_hva_cache_init(vcpu->kvm, | 494 | r = kvm_gfn_to_hva_cache_init(vcpu->kvm, |
| 464 | &vcpu->arch.xen.vcpu_time_info_cache, | 495 | &vcpu->arch.xen.vcpu_time_info_cache, |
| 465 | data->u.gpa, | 496 | data->u.gpa, |
| @@ -481,6 +512,12 @@ int kvm_xen_vcpu_set_attr(struct kvm_vcpu *vcpu, struct kvm_xen_vcpu_attr *data) · 15 unmodified lines | |||
| 481 | break; | 512 | break; |
| 482 | } | 513 | } |
| 483 | 514 | ||
| 515 | /* It must fit within a single page */ | ||
| 516 | if ((data->u.gpa & ~PAGE_MASK) + sizeof(struct vcpu_runstate_info) > PAGE_SIZE) { | ||
| 517 | r = -EINVAL; | ||
| 518 | break; | ||
| 519 | } | ||
| 520 | |||
| 484 | r = kvm_gfn_to_hva_cache_init(vcpu->kvm, | 521 | r = kvm_gfn_to_hva_cache_init(vcpu->kvm, |
| 485 | &vcpu->arch.xen.runstate_cache, | 522 | &vcpu->arch.xen.runstate_cache, |
| 486 | data->u.gpa, | 523 | data->u.gpa, |
| @@ -9,6 +9,7 @@ · 8 unmodified lines | |||
| 9 | #include <xen/events.h> | 9 | #include <xen/events.h> |
| 10 | #include <xen/interface/memory.h> | 10 | #include <xen/interface/memory.h> |
| 11 | 11 | ||
| 12 | #include <asm/apic.h> | ||
| 12 | #include <asm/cpu.h> | 13 | #include <asm/cpu.h> |
| 13 | #include <asm/smp.h> | 14 | #include <asm/smp.h> |
| 14 | #include <asm/io_apic.h> | 15 | #include <asm/io_apic.h> |
| @@ -184,8 +185,7 @@ static int xen_cpu_dead_hvm(unsigned int cpu) · 169 unmodified lines | |||
| 184 | 185 | ||
| 185 | if (xen_have_vector_callback && xen_feature(XENFEAT_hvm_safe_pvclock)) | 186 | if (xen_have_vector_callback && xen_feature(XENFEAT_hvm_safe_pvclock)) |
| 186 | xen_teardown_timer(cpu); | 187 | xen_teardown_timer(cpu); |
| 187 | 188 | return 0; | |
| 188 | return 0; | ||
| 189 | } | 189 | } |
| 190 | 190 | ||
| 191 | static bool no_vector_callback __initdata; | 191 | static bool no_vector_callback __initdata; |
| @@ -242,15 +242,14 @@ static __init int xen_parse_no_vector_callback(char *arg) · 50 unmodified lines | |||
| 242 | } | 242 | } |
| 243 | early_param("xen_no_vector_callback", xen_parse_no_vector_callback); | 243 | early_param("xen_no_vector_callback", xen_parse_no_vector_callback); |
| 244 | 244 | ||
| 245 | bool __init xen_hvm_need_lapic(void) | 245 | static __init bool xen_x2apic_available(void) |
| 246 | { | 246 | { |
| 247 | if (xen_pv_domain()) | 247 | return x2apic_supported(); |
| 248 | return false; | 248 | } |
| 249 | if (!xen_hvm_domain()) | 249 | |
| 250 | return false; | 250 | static bool __init msi_ext_dest_id(void) |
| 251 | if (xen_feature(XENFEAT_hvm_pirqs) && xen_have_vector_callback) | 251 | { |
| 252 | return false; | 252 | return cpuid_eax(xen_cpuid_base() + 4) & XEN_HVM_CPUID_EXT_DEST_ID; |
| 253 | return true; | ||
| 254 | } | 253 | } |
| 255 | 254 | ||
| 256 | static __init void xen_hvm_guest_late_init(void) | 255 | static __init void xen_hvm_guest_late_init(void) |
| @@ -312,9 +311,10 @@ struct hypervisor_x86 x86_hyper_xen_hvm __initdata = { · 55 unmodified lines | |||
| 312 | .detect = xen_platform_hvm, | 311 | .detect = xen_platform_hvm, |
| 313 | .type = X86_HYPER_XEN_HVM, | 312 | .type = X86_HYPER_XEN_HVM, |
| 314 | .init.init_platform = xen_hvm_guest_init, | 313 | .init.init_platform = xen_hvm_guest_init, |
| 315 | .init.x2apic_available = xen_x2apic_para_available, | 314 | .init.x2apic_available = xen_x2apic_available, |
| 316 | .init.init_mem_mapping = xen_hvm_init_mem_mapping, | 315 | .init.init_mem_mapping = xen_hvm_init_mem_mapping, |
| 317 | .init.guest_late_init = xen_hvm_guest_late_init, | 316 | .init.guest_late_init = xen_hvm_guest_late_init, |
| 317 | .init.msi_ext_dest_id = msi_ext_dest_id, | ||
| 318 | .runtime.pin_vcpu = xen_pin_vcpu, | 318 | .runtime.pin_vcpu = xen_pin_vcpu, |
| 319 | .ignore_nopv = true, | 319 | .ignore_nopv = true, |
| 320 | }; | 320 | }; |
| @@ -1341,10 +1341,6 @@ asmlinkage __visible void __init xen_start_kernel(void) · 1340 unmodified lines | |||
| 1341 | 1341 | ||
| 1342 | xen_acpi_sleep_register(); | 1342 | xen_acpi_sleep_register(); |
| 1343 | 1343 | ||
| 1344 | /* Avoid searching for BIOS MP tables */ | ||
| 1345 | x86_init.mpparse.find_smp_config = x86_init_noop; | ||
| 1346 | x86_init.mpparse.get_smp_config = x86_init_uint_noop; | ||
| 1347 | |||
| 1348 | xen_boot_params_init_edd(); | 1344 | xen_boot_params_init_edd(); |
| 1349 | 1345 | ||
| 1350 | #ifdef CONFIG_ACPI | 1346 | #ifdef CONFIG_ACPI |
| @@ -148,28 +148,12 @@ int xen_smp_intr_init_pv(unsigned int cpu) · 147 unmodified lines | |||
| 148 | return rc; | 148 | return rc; |
| 149 | } | 149 | } |
| 150 | 150 | ||
| 151 | static void __init xen_fill_possible_map(void) | 151 | static void __init _get_smp_config(unsigned int early) |
| 152 | { | ||
| 153 | int i, rc; | ||
| 154 | |||
| 155 | if (xen_initial_domain()) | ||
| 156 | return; | ||
| 157 | |||
| 158 | for (i = 0; i < nr_cpu_ids; i++) { | ||
| 159 | rc = HYPERVISOR_vcpu_op(VCPUOP_is_up, i, NULL); | ||
| 160 | if (rc >= 0) { | ||
| 161 | num_processors++; | ||
| 162 | set_cpu_possible(i, true); | ||
| 163 | } | ||
| 164 | } | ||
| 165 | } | ||
| 166 | |||
| 167 | static void __init xen_filter_cpu_maps(void) | ||
| 168 | { | 152 | { |
| 169 | int i, rc; | 153 | int i, rc; |
| 170 | unsigned int subtract = 0; | 154 | unsigned int subtract = 0; |
| 171 | 155 | ||
| 172 | if (!xen_initial_domain()) | 156 | if (early) |
| 173 | return; | 157 | return; |
| 174 | 158 | ||
| 175 | num_processors = 0; | 159 | num_processors = 0; |
| @@ -210,7 +194,6 @@ static void __init xen_pv_smp_prepare_boot_cpu(void) · 34 unmodified lines | |||
| 210 | * sure the old memory can be recycled. */ | 194 | * sure the old memory can be recycled. */ |
| 211 | make_lowmem_page_readwrite(xen_initial_gdt); | 195 | make_lowmem_page_readwrite(xen_initial_gdt); |
| 212 | 196 | ||
| 213 | xen_filter_cpu_maps(); | ||
| 214 | xen_setup_vcpu_info_placement(); | 197 | xen_setup_vcpu_info_placement(); |
| 215 | 198 | ||
| 216 | /* | 199 | /* |
| @@ -476,5 +459,8 @@ static const struct smp_ops xen_smp_ops __initconst = { · 259 unmodified lines | |||
| 476 | void __init xen_smp_init(void) | 459 | void __init xen_smp_init(void) |
| 477 | { | 460 | { |
| 478 | smp_ops = xen_smp_ops; | 461 | smp_ops = xen_smp_ops; |
| 479 | xen_fill_possible_map(); | 462 | |
| 463 | /* Avoid searching for BIOS MP tables */ | ||
| 464 | x86_init.mpparse.find_smp_config = x86_init_noop; | ||
| 465 | x86_init.mpparse.get_smp_config = _get_smp_config; | ||
| 480 | } | 466 | } |
| @@ -57,6 +57,14 @@ void __init xen_init_vga(const struct dom0_vga_console_info *info, size_t size) · 56 unmodified lines | |||
| 57 | screen_info->rsvd_size = info->u.vesa_lfb.rsvd_size; | 57 | screen_info->rsvd_size = info->u.vesa_lfb.rsvd_size; |
| 58 | screen_info->rsvd_pos = info->u.vesa_lfb.rsvd_pos; | 58 | screen_info->rsvd_pos = info->u.vesa_lfb.rsvd_pos; |
| 59 | 59 | ||
| 60 | if (size >= offsetof(struct dom0_vga_console_info, | ||
| 61 | u.vesa_lfb.ext_lfb_base) | ||
| 62 | + sizeof(info->u.vesa_lfb.ext_lfb_base) | ||
| 63 | && info->u.vesa_lfb.ext_lfb_base) { | ||
| 64 | screen_info->ext_lfb_base = info->u.vesa_lfb.ext_lfb_base; | ||
| 65 | screen_info->capabilities |= VIDEO_CAPABILITY_64BIT_BASE; | ||
| 66 | } | ||
| 67 | |||
| 60 | if (info->video_type == XEN_VGATYPE_EFI_LFB) { | 68 | if (info->video_type == XEN_VGATYPE_EFI_LFB) { |
| 61 | screen_info->orig_video_isVGA = VIDEO_TYPE_EFI; | 69 | screen_info->orig_video_isVGA = VIDEO_TYPE_EFI; |
| 62 | break; | 70 | break; |
| @@ -66,14 +74,6 @@ void __init xen_init_vga(const struct dom0_vga_console_info *info, size_t size) · 3 unmodified lines | |||
| 66 | u.vesa_lfb.mode_attrs) | 74 | u.vesa_lfb.mode_attrs) |
| 67 | + sizeof(info->u.vesa_lfb.mode_attrs)) | 75 | + sizeof(info->u.vesa_lfb.mode_attrs)) |
| 68 | screen_info->vesa_attributes = info->u.vesa_lfb.mode_attrs; | 76 | screen_info->vesa_attributes = info->u.vesa_lfb.mode_attrs; |
| 69 | |||
| 70 | if (size >= offsetof(struct dom0_vga_console_info, | ||
| 71 | u.vesa_lfb.ext_lfb_base) | ||
| 72 | + sizeof(info->u.vesa_lfb.ext_lfb_base) | ||
| 73 | && info->u.vesa_lfb.ext_lfb_base) { | ||
| 74 | screen_info->ext_lfb_base = info->u.vesa_lfb.ext_lfb_base; | ||
| 75 | screen_info->capabilities |= VIDEO_CAPABILITY_64BIT_BASE; | ||
| 76 | } | ||
| 77 | break; | 77 | break; |
| 78 | } | 78 | } |
| 79 | } | 79 | } |
| @@ -7018,6 +7018,8 @@ static void bfq_exit_queue(struct elevator_queue *e) · 7017 unmodified lines | |||
| 7018 | spin_unlock_irq(&bfqd->lock); | 7018 | spin_unlock_irq(&bfqd->lock); |
| 7019 | #endif | 7019 | #endif |
| 7020 | 7020 | ||
| 7021 | wbt_enable_default(bfqd->queue); | ||
| 7022 | |||
| 7021 | kfree(bfqd); | 7023 | kfree(bfqd); |
| 7022 | } | 7024 | } |
| 7023 | 7025 | ||
| @@ -373,7 +373,7 @@ void bio_integrity_advance(struct bio *bio, unsigned int bytes_done) · 372 unmodified lines | |||
| 373 | struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk); | 373 | struct blk_integrity *bi = blk_get_integrity(bio->bi_bdev->bd_disk); |
| 374 | unsigned bytes = bio_integrity_bytes(bi, bytes_done >> 9); | 374 | unsigned bytes = bio_integrity_bytes(bi, bytes_done >> 9); |
| 375 | 375 | ||
| 376 | bip->bip_iter.bi_sector += bytes_done >> 9; | 376 | bip->bip_iter.bi_sector += bio_integrity_intervals(bi, bytes_done >> 9); |
| 377 | bvec_iter_advance(bip->bip_vec, &bip->bip_iter, bytes); | 377 | bvec_iter_advance(bip->bip_vec, &bip->bip_iter, bytes); |
| 378 | } | 378 | } |
| 379 | 379 | ||
| @@ -284,13 +284,6 @@ void blk_queue_start_drain(struct request_queue *q) · 283 unmodified lines | |||
| 284 | wake_up_all(&q->mq_freeze_wq); | 284 | wake_up_all(&q->mq_freeze_wq); |
| 285 | } | 285 | } |
| 286 | 286 | ||
| 287 | void blk_set_queue_dying(struct request_queue *q) | ||
| 288 | { | ||
| 289 | blk_queue_flag_set(QUEUE_FLAG_DYING, q); | ||
| 290 | blk_queue_start_drain(q); | ||
| 291 | } | ||
| 292 | EXPORT_SYMBOL_GPL(blk_set_queue_dying); | ||
| 293 | |||
| 294 | /** | 287 | /** |
| 295 | * blk_cleanup_queue - shutdown a request queue | 288 | * blk_cleanup_queue - shutdown a request queue |
| 296 | * @q: request queue to shutdown | 289 | * @q: request queue to shutdown |
| @@ -308,7 +301,8 @@ void blk_cleanup_queue(struct request_queue *q) · 11 unmodified lines | |||
| 308 | WARN_ON_ONCE(blk_queue_registered(q)); | 301 | WARN_ON_ONCE(blk_queue_registered(q)); |
| 309 | 302 | ||
| 310 | /* mark @q DYING, no new request or merges will be allowed afterwards */ | 303 | /* mark @q DYING, no new request or merges will be allowed afterwards */ |
| 311 | blk_set_queue_dying(q); | 304 | blk_queue_flag_set(QUEUE_FLAG_DYING, q); |
| 305 | blk_queue_start_drain(q); | ||
| 312 | 306 | ||
| 313 | blk_queue_flag_set(QUEUE_FLAG_NOMERGES, q); | 307 | blk_queue_flag_set(QUEUE_FLAG_NOMERGES, q); |
| 314 | blk_queue_flag_set(QUEUE_FLAG_NOXMERGES, q); | 308 | blk_queue_flag_set(QUEUE_FLAG_NOXMERGES, q); |
| @@ -446,7 +446,7 @@ static struct bio *bio_copy_kern(struct request_queue *q, void *data, · 445 unmodified lines | |||
| 446 | if (bytes > len) | 446 | if (bytes > len) |
| 447 | bytes = len; | 447 | bytes = len; |
| 448 | 448 | ||
| 449 | page = alloc_page(GFP_NOIO | gfp_mask); | 449 | page = alloc_page(GFP_NOIO | __GFP_ZERO | gfp_mask); |
| 450 | if (!page) | 450 | if (!page) |
| 451 | goto cleanup; | 451 | goto cleanup; |
| 452 | 452 | ||
| @@ -736,6 +736,10 @@ static void blk_complete_request(struct request *req) · 735 unmodified lines | |||
| 736 | 736 | ||
| 737 | /* Completion has already been traced */ | 737 | /* Completion has already been traced */ |
| 738 | bio_clear_flag(bio, BIO_TRACE_COMPLETION); | 738 | bio_clear_flag(bio, BIO_TRACE_COMPLETION); |
| 739 | |||
| 740 | if (req_op(req) == REQ_OP_ZONE_APPEND) | ||
| 741 | bio->bi_iter.bi_sector = req->__sector; | ||
| 742 | |||
| 739 | if (!is_flush) | 743 | if (!is_flush) |
| 740 | bio_endio(bio); | 744 | bio_endio(bio); |
| 741 | bio = next; | 745 | bio = next; |
| @@ -525,8 +525,6 @@ void elv_unregister_queue(struct request_queue *q) · 524 unmodified lines | |||
| 525 | kobject_del(&e->kobj); | 525 | kobject_del(&e->kobj); |
| 526 | 526 | ||
| 527 | e->registered = 0; | 527 | e->registered = 0; |
| 528 | /* Re-enable throttling in case elevator disabled it */ | ||
| 529 | wbt_enable_default(q); | ||
| 530 | } | 528 | } |
| 531 | } | 529 | } |
| 532 | 530 | ||
| @@ -289,6 +289,8 @@ static void blkdev_bio_end_io_async(struct bio *bio) · 288 unmodified lines | |||
| 289 | struct kiocb *iocb = dio->iocb; | 289 | struct kiocb *iocb = dio->iocb; |
| 290 | ssize_t ret; | 290 | ssize_t ret; |
| 291 | 291 | ||
| 292 | WRITE_ONCE(iocb->private, NULL); | ||
| 293 | |||
| 292 | if (likely(!bio->bi_status)) { | 294 | if (likely(!bio->bi_status)) { |
| 293 | ret = dio->size; | 295 | ret = dio->size; |
| 294 | iocb->ki_pos += ret; | 296 | iocb->ki_pos += ret; |
| @@ -566,34 +568,37 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to) · 271 unmodified lines | |||
| 566 | { | 568 | { |
| 567 | struct block_device *bdev = iocb->ki_filp->private_data; | 569 | struct block_device *bdev = iocb->ki_filp->private_data; |
| 568 | loff_t size = bdev_nr_bytes(bdev); | 570 | loff_t size = bdev_nr_bytes(bdev); |
| 569 | size_t count = iov_iter_count(to); | ||
| 570 | loff_t pos = iocb->ki_pos; | 571 | loff_t pos = iocb->ki_pos; |
| 571 | size_t shorted = 0; | 572 | size_t shorted = 0; |
| 572 | ssize_t ret = 0; | 573 | ssize_t ret = 0; |
| 574 | size_t count; | ||
| 573 | 575 | ||
| 574 | if (unlikely(pos + count > size)) { | 576 | if (unlikely(pos + iov_iter_count(to) > size)) { |
| 575 | if (pos >= size) | 577 | if (pos >= size) |
| 576 | return 0; | 578 | return 0; |
| 577 | size -= pos; | 579 | size -= pos; |
| 578 | if (count > size) { | 580 | shorted = iov_iter_count(to) - size; |
| 579 | shorted = count - size; | 581 | iov_iter_truncate(to, size); |
| 580 | iov_iter_truncate(to, size); | ||
| 581 | } | ||
| 582 | } | 582 | } |
| 583 | 583 | ||
| 584 | count = iov_iter_count(to); | ||
| 585 | if (!count) | ||
| 586 | goto reexpand; /* skip atime */ | ||
| 587 | |||
| 584 | if (iocb->ki_flags & IOCB_DIRECT) { | 588 | if (iocb->ki_flags & IOCB_DIRECT) { |
| 585 | struct address_space *mapping = iocb->ki_filp->f_mapping; | 589 | struct address_space *mapping = iocb->ki_filp->f_mapping; |
| 586 | 590 | ||
| 587 | if (iocb->ki_flags & IOCB_NOWAIT) { | 591 | if (iocb->ki_flags & IOCB_NOWAIT) { |
| 588 | if (filemap_range_needs_writeback(mapping, iocb->ki_pos, | 592 | if (filemap_range_needs_writeback(mapping, pos, |
| 589 | iocb->ki_pos + count - 1)) | 593 | pos + count - 1)) { |
| 590 | return -EAGAIN; | 594 | ret = -EAGAIN; |
| 595 | goto reexpand; | ||
| 596 | } | ||
| 591 | } else { | 597 | } else { |
| 592 | ret = filemap_write_and_wait_range(mapping, | 598 | ret = filemap_write_and_wait_range(mapping, pos, |
| 593 | iocb->ki_pos, | 599 | pos + count - 1); |
| 594 | iocb->ki_pos + count - 1); | ||
| 595 | if (ret < 0) | 600 | if (ret < 0) |
| 596 | return ret; | 601 | goto reexpand; |
| 597 | } | 602 | } |
| 598 | 603 | ||
| 599 | file_accessed(iocb->ki_filp); | 604 | file_accessed(iocb->ki_filp); |
| @@ -603,12 +608,14 @@ static ssize_t blkdev_read_iter(struct kiocb *iocb, struct iov_iter *to) · 3 unmodified lines | |||
| 603 | iocb->ki_pos += ret; | 608 | iocb->ki_pos += ret; |
| 604 | count -= ret; | 609 | count -= ret; |
| 605 | } | 610 | } |
| 611 | iov_iter_revert(to, count - iov_iter_count(to)); | ||
| 606 | if (ret < 0 || !count) | 612 | if (ret < 0 || !count) |
| 607 | return ret; | 613 | goto reexpand; |
| 608 | } | 614 | } |
| 609 | 615 | ||
| 610 | ret = filemap_read(iocb, to, ret); | 616 | ret = filemap_read(iocb, to, ret); |
| 611 | 617 | ||
| 618 | reexpand: | ||
| 612 | if (unlikely(shorted)) | 619 | if (unlikely(shorted)) |
| 613 | iov_iter_reexpand(to, iov_iter_count(to) + shorted); | 620 | iov_iter_reexpand(to, iov_iter_count(to) + shorted); |
| 614 | return ret; | 621 | return ret; |
| @@ -548,6 +548,20 @@ out_free_ext_minor: · 547 unmodified lines | |||
| 548 | } | 548 | } |
| 549 | EXPORT_SYMBOL(device_add_disk); | 549 | EXPORT_SYMBOL(device_add_disk); |
| 550 | 550 | ||
| 551 | /** | ||
| 552 | * blk_mark_disk_dead - mark a disk as dead | ||
| 553 | * @disk: disk to mark as dead | ||
| 554 | * | ||
| 555 | * Mark as disk as dead (e.g. surprise removed) and don't accept any new I/O | ||
| 556 | * to this disk. | ||
| 557 | */ | ||
| 558 | void blk_mark_disk_dead(struct gendisk *disk) | ||
| 559 | { | ||
| 560 | set_bit(GD_DEAD, &disk->state); | ||
| 561 | blk_queue_start_drain(disk->queue); | ||
| 562 | } | ||
| 563 | EXPORT_SYMBOL_GPL(blk_mark_disk_dead); | ||
| 564 | |||
| 551 | /** | 565 | /** |
| 552 | * del_gendisk - remove the gendisk | 566 | * del_gendisk - remove the gendisk |
| 553 | * @disk: the struct gendisk to remove | 567 | * @disk: the struct gendisk to remove |
| @@ -25,12 +25,9 @@ struct alg_type_list { · 24 unmodified lines | |||
| 25 | struct list_head list; | 25 | struct list_head list; |
| 26 | }; | 26 | }; |
| 27 | 27 | ||
| 28 | static atomic_long_t alg_memory_allocated; | ||
| 29 | |||
| 30 | static struct proto alg_proto = { | 28 | static struct proto alg_proto = { |
| 31 | .name = "ALG", | 29 | .name = "ALG", |
| 32 | .owner = THIS_MODULE, | 30 | .owner = THIS_MODULE, |
| 33 | .memory_allocated = &alg_memory_allocated, | ||
| 34 | .obj_size = sizeof(struct alg_sock), | 31 | .obj_size = sizeof(struct alg_sock), |
| 35 | }; | 32 | }; |
| 36 | 33 | ||
| @@ -1324,3 +1324,4 @@ module_exit(crypto_algapi_exit); · 1323 unmodified lines | |||
| 1324 | 1324 | ||
| 1325 | MODULE_LICENSE("GPL"); | 1325 | MODULE_LICENSE("GPL"); |
| 1326 | MODULE_DESCRIPTION("Cryptographic algorithms API"); | 1326 | MODULE_DESCRIPTION("Cryptographic algorithms API"); |
| 1327 | MODULE_SOFTDEP("pre: cryptomgr"); | ||
| @@ -643,4 +643,3 @@ EXPORT_SYMBOL_GPL(crypto_req_done); · 642 unmodified lines | |||
| 643 | 643 | ||
| 644 | MODULE_DESCRIPTION("Cryptographic core API"); | 644 | MODULE_DESCRIPTION("Cryptographic core API"); |
| 645 | MODULE_LICENSE("GPL"); | 645 | MODULE_LICENSE("GPL"); |
| 646 | MODULE_SOFTDEP("pre: cryptomgr"); | ||
| @@ -15,12 +15,12 @@ · 14 unmodified lines | |||
| 15 | static int crypto_blake2s_update_generic(struct shash_desc *desc, | 15 | static int crypto_blake2s_update_generic(struct shash_desc *desc, |
| 16 | const u8 *in, unsigned int inlen) | 16 | const u8 *in, unsigned int inlen) |
| 17 | { | 17 | { |
| 18 | return crypto_blake2s_update(desc, in, inlen, blake2s_compress_generic); | 18 | return crypto_blake2s_update(desc, in, inlen, true); |
| 19 | } | 19 | } |
| 20 | 20 | ||
| 21 | static int crypto_blake2s_final_generic(struct shash_desc *desc, u8 *out) | 21 | static int crypto_blake2s_final_generic(struct shash_desc *desc, u8 *out) |
| 22 | { | 22 | { |
| 23 | return crypto_blake2s_final(desc, out, blake2s_compress_generic); | 23 | return crypto_blake2s_final(desc, out, true); |
| 24 | } | 24 | } |
| 25 | 25 | ||
| 26 | #define BLAKE2S_ALG(name, driver_name, digest_size) \ | 26 | #define BLAKE2S_ALG(name, driver_name, digest_size) \ |
| @@ -44,6 +44,7 @@ static struct var_t vars[] = { · 43 unmodified lines | |||
| 44 | { CAPS_START, .u.s = {"[:dv ap 160] " } }, | 44 | { CAPS_START, .u.s = {"[:dv ap 160] " } }, |
| 45 | { CAPS_STOP, .u.s = {"[:dv ap 100 ] " } }, | 45 | { CAPS_STOP, .u.s = {"[:dv ap 100 ] " } }, |
| 46 | { RATE, .u.n = {"[:ra %d] ", 180, 75, 650, 0, 0, NULL } }, | 46 | { RATE, .u.n = {"[:ra %d] ", 180, 75, 650, 0, 0, NULL } }, |
| 47 | { PITCH, .u.n = {"[:dv ap %d] ", 122, 50, 350, 0, 0, NULL } }, | ||
| 47 | { INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } }, | 48 | { INFLECTION, .u.n = {"[:dv pr %d] ", 100, 0, 10000, 0, 0, NULL } }, |
| 48 | { VOL, .u.n = {"[:dv g5 %d] ", 86, 60, 86, 0, 0, NULL } }, | 49 | { VOL, .u.n = {"[:dv g5 %d] ", 86, 60, 86, 0, 0, NULL } }, |
| 49 | { PUNCT, .u.n = {"[:pu %c] ", 0, 0, 2, 0, 0, "nsa" } }, | 50 | { PUNCT, .u.n = {"[:pu %c] ", 0, 0, 2, 0, 0, "nsa" } }, |
| @@ -11,6 +11,7 @@ menuconfig ACPI · 10 unmodified lines | |||
| 11 | depends on ARCH_SUPPORTS_ACPI | 11 | depends on ARCH_SUPPORTS_ACPI |
| 12 | select PNP | 12 | select PNP |
| 13 | select NLS | 13 | select NLS |
| 14 | select CRC32 | ||
| 14 | default y if X86 | 15 | default y if X86 |
| 15 | help | 16 | help |
| 16 | Advanced Configuration and Power Interface (ACPI) support for | 17 | Advanced Configuration and Power Interface (ACPI) support for |
| @@ -1361,9 +1361,17 @@ static void __init arm_smmu_v3_pmcg_init_resources(struct resource *res, · 1360 unmodified lines | |||
| 1361 | res[0].start = pmcg->page0_base_address; | 1361 | res[0].start = pmcg->page0_base_address; |
| 1362 | res[0].end = pmcg->page0_base_address + SZ_4K - 1; | 1362 | res[0].end = pmcg->page0_base_address + SZ_4K - 1; |
| 1363 | res[0].flags = IORESOURCE_MEM; | 1363 | res[0].flags = IORESOURCE_MEM; |
| 1364 | res[1].start = pmcg->page1_base_address; | 1364 | /* |
| 1365 | res[1].end = pmcg->page1_base_address + SZ_4K - 1; | 1365 | * The initial version in DEN0049C lacked a way to describe register |
| 1366 | res[1].flags = IORESOURCE_MEM; | 1366 | * page 1, which makes it broken for most PMCG implementations; in |
| 1367 | * that case, just let the driver fail gracefully if it expects to | ||
| 1368 | * find a second memory resource. | ||
| 1369 | */ | ||
| 1370 | if (node->revision > 0) { | ||
| 1371 | res[1].start = pmcg->page1_base_address; | ||
| 1372 | res[1].end = pmcg->page1_base_address + SZ_4K - 1; | ||
| 1373 | res[1].flags = IORESOURCE_MEM; | ||
| 1374 | } | ||
| 1367 | 1375 | ||
| 1368 | if (pmcg->overflow_gsiv) | 1376 | if (pmcg->overflow_gsiv) |
| 1369 | acpi_iort_register_irq(pmcg->overflow_gsiv, "overflow", | 1377 | acpi_iort_register_irq(pmcg->overflow_gsiv, "overflow", |
| @@ -2065,6 +2065,16 @@ bool acpi_ec_dispatch_gpe(void) · 2064 unmodified lines | |||
| 2065 | if (acpi_any_gpe_status_set(first_ec->gpe)) | 2065 | if (acpi_any_gpe_status_set(first_ec->gpe)) |
| 2066 | return true; | 2066 | return true; |
| 2067 | 2067 | ||
| 2068 | /* | ||
| 2069 | * Cancel the SCI wakeup and process all pending events in case there | ||
| 2070 | * are any wakeup ones in there. | ||
| 2071 | * | ||
| 2072 | * Note that if any non-EC GPEs are active at this point, the SCI will | ||
| 2073 | * retrigger after the rearming in acpi_s2idle_wake(), so no events | ||
| 2074 | * should be missed by canceling the wakeup here. | ||
| 2075 | */ | ||
| 2076 | pm_system_cancel_wakeup(); | ||
| 2077 | |||
| 2068 | /* | 2078 | /* |
| 2069 | * Dispatch the EC GPE in-band, but do not report wakeup in any case | 2079 | * Dispatch the EC GPE in-band, but do not report wakeup in any case |
| 2070 | * to allow the caller to process events properly after that. | 2080 | * to allow the caller to process events properly after that. |
| @@ -96,6 +96,11 @@ static const struct dmi_system_id processor_power_dmi_table[] = { · 95 unmodified lines | |||
| 96 | DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK Computer Inc."), | 96 | DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK Computer Inc."), |
| 97 | DMI_MATCH(DMI_PRODUCT_NAME,"L8400B series Notebook PC")}, | 97 | DMI_MATCH(DMI_PRODUCT_NAME,"L8400B series Notebook PC")}, |
| 98 | (void *)1}, | 98 | (void *)1}, |
| 99 | /* T40 can not handle C3 idle state */ | ||
| 100 | { set_max_cstate, "IBM ThinkPad T40", { | ||
| 101 | DMI_MATCH(DMI_SYS_VENDOR, "IBM"), | ||
| 102 | DMI_MATCH(DMI_PRODUCT_NAME, "23737CU")}, | ||
| 103 | (void *)2}, | ||
| 99 | {}, | 104 | {}, |
| 100 | }; | 105 | }; |
| 101 | 106 | ||
| @@ -736,21 +736,15 @@ bool acpi_s2idle_wake(void) · 735 unmodified lines | |||
| 736 | return true; | 736 | return true; |
| 737 | } | 737 | } |
| 738 | 738 | ||
| 739 | /* Check non-EC GPE wakeups and dispatch the EC GPE. */ | 739 | /* |
| 740 | * Check non-EC GPE wakeups and if there are none, cancel the | ||
| 741 | * SCI-related wakeup and dispatch the EC GPE. | ||
| 742 | */ | ||
| 740 | if (acpi_ec_dispatch_gpe()) { | 743 | if (acpi_ec_dispatch_gpe()) { |
| 741 | pm_pr_dbg("ACPI non-EC GPE wakeup\n"); | 744 | pm_pr_dbg("ACPI non-EC GPE wakeup\n"); |
| 742 | return true; | 745 | return true; |
| 743 | } | 746 | } |
| 744 | 747 | ||
| 745 | /* | ||
| 746 | * Cancel the SCI wakeup and process all pending events in case | ||
| 747 | * there are any wakeup ones in there. | ||
| 748 | * | ||
| 749 | * Note that if any non-EC GPEs are active at this point, the | ||
| 750 | * SCI will retrigger after the rearming below, so no events | ||
| 751 | * should be missed by canceling the wakeup here. | ||
| 752 | */ | ||
| 753 | pm_system_cancel_wakeup(); | ||
| 754 | acpi_os_wait_events_complete(); | 748 | acpi_os_wait_events_complete(); |
| 755 | 749 | ||
| 756 | /* | 750 | /* |
| @@ -764,6 +758,7 @@ bool acpi_s2idle_wake(void) · 7 unmodified lines | |||
| 764 | return true; | 758 | return true; |
| 765 | } | 759 | } |
| 766 | 760 | ||
| 761 | pm_wakeup_clear(acpi_sci_irq); | ||
| 767 | rearm_wake_irq(acpi_sci_irq); | 762 | rearm_wake_irq(acpi_sci_irq); |
| 768 | } | 763 | } |
| 769 | 764 | ||
| @@ -400,7 +400,7 @@ int __init_or_acpilib acpi_table_parse_entries_array( · 399 unmodified lines | |||
| 400 | 400 | ||
| 401 | acpi_get_table(id, instance, &table_header); | 401 | acpi_get_table(id, instance, &table_header); |
| 402 | if (!table_header) { | 402 | if (!table_header) { |
| 403 | pr_warn("%4.4s not present\n", id); | 403 | pr_debug("%4.4s not present\n", id); |
| 404 | return -ENODEV; | 404 | return -ENODEV; |
| 405 | } | 405 | } |
| 406 | 406 | ||
| @@ -424,15 +424,11 @@ static int lps0_device_attach(struct acpi_device *adev, · 423 unmodified lines | |||
| 424 | mem_sleep_current = PM_SUSPEND_TO_IDLE; | 424 | mem_sleep_current = PM_SUSPEND_TO_IDLE; |
| 425 | 425 | ||
| 426 | /* | 426 | /* |
| 427 | * Some Intel based LPS0 systems, like ASUS Zenbook UX430UNR/i7-8550U don't | 427 | * Some LPS0 systems, like ASUS Zenbook UX430UNR/i7-8550U, require the |
| 428 | * use intel-hid or intel-vbtn but require the EC GPE to be enabled while | 428 | * EC GPE to be enabled while suspended for certain wakeup devices to |
| 429 | * suspended for certain wakeup devices to work, so mark it as wakeup-capable. | 429 | * work, so mark it as wakeup-capable. |
| 430 | * | ||
| 431 | * Only enable on !AMD as enabling this universally causes problems for a number | ||
| 432 | * of AMD based systems. | ||
| 433 | */ | 430 | */ |
| 434 | if (!acpi_s2idle_vendor_amd()) | 431 | acpi_ec_mark_gpe_for_wake(); |
| 435 | acpi_ec_mark_gpe_for_wake(); | ||
| 436 | 432 | ||
| 437 | return 0; | 433 | return 0; |
| 438 | } | 434 | } |
| @@ -2007,6 +2007,9 @@ static bool ata_log_supported(struct ata_device *dev, u8 log) · 2006 unmodified lines | |||
| 2007 | { | 2007 | { |
| 2008 | struct ata_port *ap = dev->link->ap; | 2008 | struct ata_port *ap = dev->link->ap; |
| 2009 | 2009 | ||
| 2010 | if (dev->horkage & ATA_HORKAGE_NO_LOG_DIR) | ||
| 2011 | return false; | ||
| 2012 | |||
| 2010 | if (ata_read_log_page(dev, ATA_LOG_DIRECTORY, 0, ap->sector_buf, 1)) | 2013 | if (ata_read_log_page(dev, ATA_LOG_DIRECTORY, 0, ap->sector_buf, 1)) |
| 2011 | return false; | 2014 | return false; |
| 2012 | return get_unaligned_le16(&ap->sector_buf[log * 2]) ? true : false; | 2015 | return get_unaligned_le16(&ap->sector_buf[log * 2]) ? true : false; |
| @@ -2445,23 +2448,21 @@ static void ata_dev_config_cpr(struct ata_device *dev) · 432 unmodified lines | |||
| 2445 | struct ata_cpr_log *cpr_log = NULL; | 2448 | struct ata_cpr_log *cpr_log = NULL; |
| 2446 | u8 *desc, *buf = NULL; | 2449 | u8 *desc, *buf = NULL; |
| 2447 | 2450 | ||
| 2448 | if (!ata_identify_page_supported(dev, | 2451 | if (ata_id_major_version(dev->id) < 11 || |
| 2449 | ATA_LOG_CONCURRENT_POSITIONING_RANGES)) | 2452 | !ata_log_supported(dev, ATA_LOG_CONCURRENT_POSITIONING_RANGES)) |
| 2450 | goto out; | 2453 | goto out; |
| 2451 | 2454 | ||
| 2452 | /* | 2455 | /* |
| 2453 | * Read IDENTIFY DEVICE data log, page 0x47 | 2456 | * Read the concurrent positioning ranges log (0x47). We can have at |
| 2454 | * (concurrent positioning ranges). We can have at most 255 32B range | 2457 | * most 255 32B range descriptors plus a 64B header. |
| 2455 | * descriptors plus a 64B header. | ||
| 2456 | */ | 2458 | */ |
| 2457 | buf_len = (64 + 255 * 32 + 511) & ~511; | 2459 | buf_len = (64 + 255 * 32 + 511) & ~511; |
| 2458 | buf = kzalloc(buf_len, GFP_KERNEL); | 2460 | buf = kzalloc(buf_len, GFP_KERNEL); |
| 2459 | if (!buf) | 2461 | if (!buf) |
| 2460 | goto out; | 2462 | goto out; |
| 2461 | 2463 | ||
| 2462 | err_mask = ata_read_log_page(dev, ATA_LOG_IDENTIFY_DEVICE, | 2464 | err_mask = ata_read_log_page(dev, ATA_LOG_CONCURRENT_POSITIONING_RANGES, |
| 2463 | ATA_LOG_CONCURRENT_POSITIONING_RANGES, | 2465 | 0, buf, buf_len >> 9); |
| 2464 | buf, buf_len >> 9); | ||
| 2465 | if (err_mask) | 2466 | if (err_mask) |
| 2466 | goto out; | 2467 | goto out; |
| 2467 | 2468 | ||
| @@ -4028,6 +4029,7 @@ static const struct ata_blacklist_entry ata_device_blacklist [] = { · 1560 unmodified lines | |||
| 4028 | 4029 | ||
| 4029 | /* devices that don't properly handle TRIM commands */ | 4030 | /* devices that don't properly handle TRIM commands */ |
| 4030 | { "SuperSSpeed S238*", NULL, ATA_HORKAGE_NOTRIM, }, | 4031 | { "SuperSSpeed S238*", NULL, ATA_HORKAGE_NOTRIM, }, |
| 4032 | { "M88V29*", NULL, ATA_HORKAGE_NOTRIM, }, | ||
| 4031 | 4033 | ||
| 4032 | /* | 4034 | /* |
| 4033 | * As defined, the DRAT (Deterministic Read After Trim) and RZAT | 4035 | * As defined, the DRAT (Deterministic Read After Trim) and RZAT |
| @@ -4073,6 +4075,13 @@ static const struct ata_blacklist_entry ata_device_blacklist [] = { · 39 unmodified lines | |||
| 4073 | { "WDC WD3000JD-*", NULL, ATA_HORKAGE_WD_BROKEN_LPM }, | 4075 | { "WDC WD3000JD-*", NULL, ATA_HORKAGE_WD_BROKEN_LPM }, |
| 4074 | { "WDC WD3200JD-*", NULL, ATA_HORKAGE_WD_BROKEN_LPM }, | 4076 | { "WDC WD3200JD-*", NULL, ATA_HORKAGE_WD_BROKEN_LPM }, |
| 4075 | 4077 | ||
| 4078 | /* | ||
| 4079 | * This sata dom device goes on a walkabout when the ATA_LOG_DIRECTORY | ||
| 4080 | * log page is accessed. Ensure we never ask for this log page with | ||
| 4081 | * these devices. | ||
| 4082 | */ | ||
| 4083 | { "SATADOM-ML 3ME", NULL, ATA_HORKAGE_NO_LOG_DIR }, | ||
| 4084 | |||
| 4076 | /* End Marker */ | 4085 | /* End Marker */ |
| 4077 | { } | 4086 | { } |
| 4078 | }; | 4087 | }; |
| @@ -919,6 +919,20 @@ static int hpt37x_init_one(struct pci_dev *dev, const struct pci_device_id *id) · 918 unmodified lines | |||
| 919 | irqmask &= ~0x10; | 919 | irqmask &= ~0x10; |
| 920 | pci_write_config_byte(dev, 0x5a, irqmask); | 920 | pci_write_config_byte(dev, 0x5a, irqmask); |
| 921 | 921 | ||
| 922 | /* | ||
| 923 | * HPT371 chips physically have only one channel, the secondary one, | ||
| 924 | * but the primary channel registers do exist! Go figure... | ||
| 925 | * So, we manually disable the non-existing channel here | ||
| 926 | * (if the BIOS hasn't done this already). | ||
| 927 | */ | ||
| 928 | if (dev->device == PCI_DEVICE_ID_TTI_HPT371) { | ||
| 929 | u8 mcr1; | ||
| 930 | |||
| 931 | pci_read_config_byte(dev, 0x50, &mcr1); | ||
| 932 | mcr1 &= ~0x04; | ||
| 933 | pci_write_config_byte(dev, 0x50, mcr1); | ||
| 934 | } | ||
| 935 | |||
| 922 | /* | 936 | /* |
| 923 | * default to pci clock. make sure MA15/16 are set to output | 937 | * default to pci clock. make sure MA15/16 are set to output |
| 924 | * to prevent drives having problems with 40-pin cables. Needed | 938 | * to prevent drives having problems with 40-pin cables. Needed |
| @@ -950,14 +964,14 @@ static int hpt37x_init_one(struct pci_dev *dev, const struct pci_device_id *id) · 25 unmodified lines | |||
| 950 | 964 | ||
| 951 | if ((freq >> 12) != 0xABCDE) { | 965 | if ((freq >> 12) != 0xABCDE) { |
| 952 | int i; | 966 | int i; |
| 953 | u8 sr; | 967 | u16 sr; |
| 954 | u32 total = 0; | 968 | u32 total = 0; |
| 955 | 969 | ||
| 956 | dev_warn(&dev->dev, "BIOS has not set timing clocks\n"); | 970 | dev_warn(&dev->dev, "BIOS has not set timing clocks\n"); |
| 957 | 971 | ||
| 958 | /* This is the process the HPT371 BIOS is reported to use */ | 972 | /* This is the process the HPT371 BIOS is reported to use */ |
| 959 | for (i = 0; i < 128; i++) { | 973 | for (i = 0; i < 128; i++) { |
| 960 | pci_read_config_byte(dev, 0x78, &sr); | 974 | pci_read_config_word(dev, 0x78, &sr); |
| 961 | total += sr & 0x1FF; | 975 | total += sr & 0x1FF; |
| 962 | udelay(15); | 976 | udelay(15); |
| 963 | } | 977 | } |
| @@ -322,7 +322,7 @@ static void fsl_sata_set_irq_coalescing(struct ata_host *host, · 321 unmodified lines | |||
| 322 | static ssize_t fsl_sata_intr_coalescing_show(struct device *dev, | 322 | static ssize_t fsl_sata_intr_coalescing_show(struct device *dev, |
| 323 | struct device_attribute *attr, char *buf) | 323 | struct device_attribute *attr, char *buf) |
| 324 | { | 324 | { |
| 325 | return sysfs_emit(buf, "%d %d\n", | 325 | return sysfs_emit(buf, "%u %u\n", |
| 326 | intr_coalescing_count, intr_coalescing_ticks); | 326 | intr_coalescing_count, intr_coalescing_ticks); |
| 327 | } | 327 | } |
| 328 | 328 | ||
| @@ -332,10 +332,8 @@ static ssize_t fsl_sata_intr_coalescing_store(struct device *dev, · 3 unmodified lines | |||
| 332 | { | 332 | { |
| 333 | unsigned int coalescing_count, coalescing_ticks; | 333 | unsigned int coalescing_count, coalescing_ticks; |
| 334 | 334 | ||
| 335 | if (sscanf(buf, "%d%d", | 335 | if (sscanf(buf, "%u%u", &coalescing_count, &coalescing_ticks) != 2) { |
| 336 | &coalescing_count, | 336 | dev_err(dev, "fsl-sata: wrong parameter format.\n"); |
| 337 | &coalescing_ticks) != 2) { | ||
| 338 | printk(KERN_ERR "fsl-sata: wrong parameter format.\n"); | ||
| 339 | return -EINVAL; | 337 | return -EINVAL; |
| 340 | } | 338 | } |
| 341 | 339 | ||
| @@ -359,7 +357,7 @@ static ssize_t fsl_sata_rx_watermark_show(struct device *dev, · 17 unmodified lines | |||
| 359 | rx_watermark &= 0x1f; | 357 | rx_watermark &= 0x1f; |
| 360 | spin_unlock_irqrestore(&host->lock, flags); | 358 | spin_unlock_irqrestore(&host->lock, flags); |
| 361 | 359 | ||
| 362 | return sysfs_emit(buf, "%d\n", rx_watermark); | 360 | return sysfs_emit(buf, "%u\n", rx_watermark); |
| 363 | } | 361 | } |
| 364 | 362 | ||
| 365 | static ssize_t fsl_sata_rx_watermark_store(struct device *dev, | 363 | static ssize_t fsl_sata_rx_watermark_store(struct device *dev, |
| @@ -373,8 +371,8 @@ static ssize_t fsl_sata_rx_watermark_store(struct device *dev, · 7 unmodified lines | |||
| 373 | void __iomem *csr_base = host_priv->csr_base; | 371 | void __iomem *csr_base = host_priv->csr_base; |
| 374 | u32 temp; | 372 | u32 temp; |
| 375 | 373 | ||
| 376 | if (sscanf(buf, "%d", &rx_watermark) != 1) { | 374 | if (kstrtouint(buf, 10, &rx_watermark) < 0) { |
| 377 | printk(KERN_ERR "fsl-sata: wrong parameter format.\n"); | 375 | dev_err(dev, "fsl-sata: wrong parameter format.\n"); |
| 378 | return -EINVAL; | 376 | return -EINVAL; |
| 379 | } | 377 | } |
| 380 | 378 | ||
| @@ -382,8 +380,8 @@ static ssize_t fsl_sata_rx_watermark_store(struct device *dev, · 1 unmodified line | |||
| 382 | temp = ioread32(csr_base + TRANSCFG); | 380 | temp = ioread32(csr_base + TRANSCFG); |
| 383 | temp &= 0xffffffe0; | 381 | temp &= 0xffffffe0; |
| 384 | iowrite32(temp | rx_watermark, csr_base + TRANSCFG); | 382 | iowrite32(temp | rx_watermark, csr_base + TRANSCFG); |
| 385 | |||
| 386 | spin_unlock_irqrestore(&host->lock, flags); | 383 | spin_unlock_irqrestore(&host->lock, flags); |
| 384 | |||
| 387 | return strlen(buf); | 385 | return strlen(buf); |
| 388 | } | 386 | } |
| 389 | 387 | ||
| @@ -629,6 +629,9 @@ re_probe: · 628 unmodified lines | |||
| 629 | drv->remove(dev); | 629 | drv->remove(dev); |
| 630 | 630 | ||
| 631 | devres_release_all(dev); | 631 | devres_release_all(dev); |
| 632 | arch_teardown_dma_ops(dev); | ||
| 633 | kfree(dev->dma_range_map); | ||
| 634 | dev->dma_range_map = NULL; | ||
| 632 | driver_sysfs_remove(dev); | 635 | driver_sysfs_remove(dev); |
| 633 | dev->driver = NULL; | 636 | dev->driver = NULL; |
| 634 | dev_set_drvdata(dev, NULL); | 637 | dev_set_drvdata(dev, NULL); |
| @@ -1209,6 +1212,8 @@ static void __device_release_driver(struct device *dev, struct device *parent) · 574 unmodified lines | |||
| 1209 | 1212 | ||
| 1210 | devres_release_all(dev); | 1213 | devres_release_all(dev); |
| 1211 | arch_teardown_dma_ops(dev); | 1214 | arch_teardown_dma_ops(dev); |
| 1215 | kfree(dev->dma_range_map); | ||
| 1216 | dev->dma_range_map = NULL; | ||
| 1212 | dev->driver = NULL; | 1217 | dev->driver = NULL; |
| 1213 | dev_set_drvdata(dev, NULL); | 1218 | dev_set_drvdata(dev, NULL); |
| 1214 | if (dev->pm_domain && dev->pm_domain->dismiss) | 1219 | if (dev->pm_domain && dev->pm_domain->dismiss) |
| @@ -34,7 +34,8 @@ suspend_state_t pm_suspend_target_state; · 33 unmodified lines | |||
| 34 | bool events_check_enabled __read_mostly; | 34 | bool events_check_enabled __read_mostly; |
| 35 | 35 | ||
| 36 | /* First wakeup IRQ seen by the kernel in the last cycle. */ | 36 | /* First wakeup IRQ seen by the kernel in the last cycle. */ |
| 37 | unsigned int pm_wakeup_irq __read_mostly; | 37 | static unsigned int wakeup_irq[2] __read_mostly; |
| 38 | static DEFINE_RAW_SPINLOCK(wakeup_irq_lock); | ||
| 38 | 39 | ||
| 39 | /* If greater than 0 and the system is suspending, terminate the suspend. */ | 40 | /* If greater than 0 and the system is suspending, terminate the suspend. */ |
| 40 | static atomic_t pm_abort_suspend __read_mostly; | 41 | static atomic_t pm_abort_suspend __read_mostly; |
| @@ -942,19 +943,45 @@ void pm_system_cancel_wakeup(void) · 901 unmodified lines | |||
| 942 | atomic_dec_if_positive(&pm_abort_suspend); | 943 | atomic_dec_if_positive(&pm_abort_suspend); |
| 943 | } | 944 | } |
| 944 | 945 | ||
| 945 | void pm_wakeup_clear(bool reset) | 946 | void pm_wakeup_clear(unsigned int irq_number) |
| 946 | { | 947 | { |
| 947 | pm_wakeup_irq = 0; | 948 | raw_spin_lock_irq(&wakeup_irq_lock); |
| 948 | if (reset) | 949 | |
| 950 | if (irq_number && wakeup_irq[0] == irq_number) | ||
| 951 | wakeup_irq[0] = wakeup_irq[1]; | ||
| 952 | else | ||
| 953 | wakeup_irq[0] = 0; | ||
| 954 | |||
| 955 | wakeup_irq[1] = 0; | ||
| 956 | |||
| 957 | raw_spin_unlock_irq(&wakeup_irq_lock); | ||
| 958 | |||
| 959 | if (!irq_number) | ||
| 949 | atomic_set(&pm_abort_suspend, 0); | 960 | atomic_set(&pm_abort_suspend, 0); |
| 950 | } | 961 | } |
| 951 | 962 | ||
| 952 | void pm_system_irq_wakeup(unsigned int irq_number) | 963 | void pm_system_irq_wakeup(unsigned int irq_number) |
| 953 | { | 964 | { |
| 954 | if (pm_wakeup_irq == 0) { | 965 | unsigned long flags; |
| 955 | pm_wakeup_irq = irq_number; | 966 | |
| 967 | raw_spin_lock_irqsave(&wakeup_irq_lock, flags); | ||
| 968 | |||
| 969 | if (wakeup_irq[0] == 0) | ||
| 970 | wakeup_irq[0] = irq_number; | ||
| 971 | else if (wakeup_irq[1] == 0) | ||
| 972 | wakeup_irq[1] = irq_number; | ||
| 973 | else | ||
| 974 | irq_number = 0; | ||
| 975 | |||
| 976 | raw_spin_unlock_irqrestore(&wakeup_irq_lock, flags); | ||
| 977 | |||
| 978 | if (irq_number) | ||
| 956 | pm_system_wakeup(); | 979 | pm_system_wakeup(); |
| 957 | } | 980 | } |
| 981 | |||
| 982 | unsigned int pm_wakeup_irq(void) | ||
| 983 | { | ||
| 984 | return wakeup_irq[0]; | ||
| 958 | } | 985 | } |
| 959 | 986 | ||
| 960 | /** | 987 | /** |
| @@ -189,11 +189,9 @@ static void regmap_irq_sync_unlock(struct irq_data *data) · 188 unmodified lines | |||
| 189 | ret = regmap_write(map, reg, d->mask_buf[i]); | 189 | ret = regmap_write(map, reg, d->mask_buf[i]); |
| 190 | if (d->chip->clear_ack) { | 190 | if (d->chip->clear_ack) { |
| 191 | if (d->chip->ack_invert && !ret) | 191 | if (d->chip->ack_invert && !ret) |
| 192 | ret = regmap_write(map, reg, | 192 | ret = regmap_write(map, reg, UINT_MAX); |
| 193 | d->mask_buf[i]); | ||
| 194 | else if (!ret) | 193 | else if (!ret) |
| 195 | ret = regmap_write(map, reg, | 194 | ret = regmap_write(map, reg, 0); |
| 196 | ~d->mask_buf[i]); | ||
| 197 | } | 195 | } |
| 198 | if (ret != 0) | 196 | if (ret != 0) |
| 199 | dev_err(d->map->dev, "Failed to ack 0x%x: %d\n", | 197 | dev_err(d->map->dev, "Failed to ack 0x%x: %d\n", |
| @@ -556,11 +554,9 @@ static irqreturn_t regmap_irq_thread(int irq, void *d) · 356 unmodified lines | |||
| 556 | data->status_buf[i]); | 554 | data->status_buf[i]); |
| 557 | if (chip->clear_ack) { | 555 | if (chip->clear_ack) { |
| 558 | if (chip->ack_invert && !ret) | 556 | if (chip->ack_invert && !ret) |
| 559 | ret = regmap_write(map, reg, | 557 | ret = regmap_write(map, reg, UINT_MAX); |
| 560 | data->status_buf[i]); | ||
| 561 | else if (!ret) | 558 | else if (!ret) |
| 562 | ret = regmap_write(map, reg, | 559 | ret = regmap_write(map, reg, 0); |
| 563 | ~data->status_buf[i]); | ||
| 564 | } | 560 | } |
| 565 | if (ret != 0) | 561 | if (ret != 0) |
| 566 | dev_err(map->dev, "Failed to ack 0x%x: %d\n", | 562 | dev_err(map->dev, "Failed to ack 0x%x: %d\n", |
| @@ -817,13 +813,9 @@ int regmap_add_irq_chip_fwnode(struct fwnode_handle *fwnode, · 250 unmodified lines | |||
| 817 | d->status_buf[i] & d->mask_buf[i]); | 813 | d->status_buf[i] & d->mask_buf[i]); |
| 818 | if (chip->clear_ack) { | 814 | if (chip->clear_ack) { |
| 819 | if (chip->ack_invert && !ret) | 815 | if (chip->ack_invert && !ret) |
| 820 | ret = regmap_write(map, reg, | 816 | ret = regmap_write(map, reg, UINT_MAX); |
| 821 | (d->status_buf[i] & | ||
| 822 | d->mask_buf[i])); | ||
| 823 | else if (!ret) | 817 | else if (!ret) |
| 824 | ret = regmap_write(map, reg, | 818 | ret = regmap_write(map, reg, 0); |
| 825 | ~(d->status_buf[i] & | ||
| 826 | d->mask_buf[i])); | ||
| 827 | } | 819 | } |
| 828 | if (ret != 0) { | 820 | if (ret != 0) { |
| 829 | dev_err(map->dev, "Failed to ack 0x%x: %d\n", | 821 | dev_err(map->dev, "Failed to ack 0x%x: %d\n", |
| @@ -79,6 +79,7 @@ · 78 unmodified lines | |||
| 79 | #include <linux/ioprio.h> | 79 | #include <linux/ioprio.h> |
| 80 | #include <linux/blk-cgroup.h> | 80 | #include <linux/blk-cgroup.h> |
| 81 | #include <linux/sched/mm.h> | 81 | #include <linux/sched/mm.h> |
| 82 | #include <linux/statfs.h> | ||
| 82 | 83 | ||
| 83 | #include "loop.h" | 84 | #include "loop.h" |
| 84 | 85 | ||
| @@ -774,8 +775,13 @@ static void loop_config_discard(struct loop_device *lo) · 689 unmodified lines | |||
| 774 | granularity = 0; | 775 | granularity = 0; |
| 775 | 776 | ||
| 776 | } else { | 777 | } else { |
| 778 | struct kstatfs sbuf; | ||
| 779 | |||
| 777 | max_discard_sectors = UINT_MAX >> 9; | 780 | max_discard_sectors = UINT_MAX >> 9; |
| 778 | granularity = inode->i_sb->s_blocksize; | 781 | if (!vfs_statfs(&file->f_path, &sbuf)) |
| 782 | granularity = sbuf.f_bsize; | ||
| 783 | else | ||
| 784 | max_discard_sectors = 0; | ||
| 779 | } | 785 | } |
| 780 | 786 | ||
| 781 | if (max_discard_sectors) { | 787 | if (max_discard_sectors) { |
| @@ -1082,7 +1088,7 @@ out_putf: · 300 unmodified lines | |||
| 1082 | return error; | 1088 | return error; |
| 1083 | } | 1089 | } |
| 1084 | 1090 | ||
| 1085 | static void __loop_clr_fd(struct loop_device *lo) | 1091 | static void __loop_clr_fd(struct loop_device *lo, bool release) |
| 1086 | { | 1092 | { |
| 1087 | struct file *filp; | 1093 | struct file *filp; |
| 1088 | gfp_t gfp = lo->old_gfp_mask; | 1094 | gfp_t gfp = lo->old_gfp_mask; |
| @@ -1144,6 +1150,8 @@ static void __loop_clr_fd(struct loop_device *lo) · 55 unmodified lines | |||
| 1144 | /* let user-space know about this change */ | 1150 | /* let user-space know about this change */ |
| 1145 | kobject_uevent(&disk_to_dev(lo->lo_disk)->kobj, KOBJ_CHANGE); | 1151 | kobject_uevent(&disk_to_dev(lo->lo_disk)->kobj, KOBJ_CHANGE); |
| 1146 | mapping_set_gfp_mask(filp->f_mapping, gfp); | 1152 | mapping_set_gfp_mask(filp->f_mapping, gfp); |
| 1153 | /* This is safe: open() is still holding a reference. */ | ||
| 1154 | module_put(THIS_MODULE); | ||
| 1147 | blk_mq_unfreeze_queue(lo->lo_queue); | 1155 | blk_mq_unfreeze_queue(lo->lo_queue); |
| 1148 | 1156 | ||
| 1149 | disk_force_media_change(lo->lo_disk, DISK_EVENT_MEDIA_CHANGE); | 1157 | disk_force_media_change(lo->lo_disk, DISK_EVENT_MEDIA_CHANGE); |
| @@ -1151,52 +1159,44 @@ static void __loop_clr_fd(struct loop_device *lo) · 1 unmodified line | |||
| 1151 | if (lo->lo_flags & LO_FLAGS_PARTSCAN) { | 1159 | if (lo->lo_flags & LO_FLAGS_PARTSCAN) { |
| 1152 | int err; | 1160 | int err; |
| 1153 | 1161 | ||
| 1154 | mutex_lock(&lo->lo_disk->open_mutex); | 1162 | /* |
| 1163 | * open_mutex has been held already in release path, so don't | ||
| 1164 | * acquire it if this function is called in such case. | ||
| 1165 | * | ||
| 1166 | * If the reread partition isn't from release path, lo_refcnt | ||
| 1167 | * must be at least one and it can only become zero when the | ||
| 1168 | * current holder is released. | ||
| 1169 | */ | ||
| 1170 | if (!release) | ||
| 1171 | mutex_lock(&lo->lo_disk->open_mutex); | ||
| 1155 | err = bdev_disk_changed(lo->lo_disk, false); | 1172 | err = bdev_disk_changed(lo->lo_disk, false); |
| 1156 | mutex_unlock(&lo->lo_disk->open_mutex); | 1173 | if (!release) |
| 1174 | mutex_unlock(&lo->lo_disk->open_mutex); | ||
| 1157 | if (err) | 1175 | if (err) |
| 1158 | pr_warn("%s: partition scan of loop%d failed (rc=%d)\n", | 1176 | pr_warn("%s: partition scan of loop%d failed (rc=%d)\n", |
| 1159 | __func__, lo->lo_number, err); | 1177 | __func__, lo->lo_number, err); |
| 1160 | /* Device is gone, no point in returning error */ | 1178 | /* Device is gone, no point in returning error */ |
| 1161 | } | 1179 | } |
| 1162 | 1180 | ||
| 1181 | /* | ||
| 1182 | * lo->lo_state is set to Lo_unbound here after above partscan has | ||
| 1183 | * finished. There cannot be anybody else entering __loop_clr_fd() as | ||
| 1184 | * Lo_rundown state protects us from all the other places trying to | ||
| 1185 | * change the 'lo' device. | ||
| 1186 | */ | ||
| 1163 | lo->lo_flags = 0; | 1187 | lo->lo_flags = 0; |
| 1164 | if (!part_shift) | 1188 | if (!part_shift) |
| 1165 | lo->lo_disk->flags |= GENHD_FL_NO_PART; | 1189 | lo->lo_disk->flags |= GENHD_FL_NO_PART; |
| 1166 | |||
| 1167 | fput(filp); | ||
| 1168 | } | ||
| 1169 | |||
| 1170 | static void loop_rundown_completed(struct loop_device *lo) | ||
| 1171 | { | ||
| 1172 | mutex_lock(&lo->lo_mutex); | 1190 | mutex_lock(&lo->lo_mutex); |
| 1173 | lo->lo_state = Lo_unbound; | 1191 | lo->lo_state = Lo_unbound; |
| 1174 | mutex_unlock(&lo->lo_mutex); | 1192 | mutex_unlock(&lo->lo_mutex); |
| 1175 | module_put(THIS_MODULE); | ||
| 1176 | } | ||
| 1177 | |||
| 1178 | static void loop_rundown_workfn(struct work_struct *work) | ||
| 1179 | { | ||
| 1180 | struct loop_device *lo = container_of(work, struct loop_device, | ||
| 1181 | rundown_work); | ||
| 1182 | struct block_device *bdev = lo->lo_device; | ||
| 1183 | struct gendisk *disk = lo->lo_disk; | ||
| 1184 | |||
| 1185 | __loop_clr_fd(lo); | ||
| 1186 | kobject_put(&bdev->bd_device.kobj); | ||
| 1187 | module_put(disk->fops->owner); | ||
| 1188 | loop_rundown_completed(lo); | ||
| 1189 | } | ||
| 1190 | |||
| 1191 | static void loop_schedule_rundown(struct loop_device *lo) | ||
| 1192 | { | ||
| 1193 | struct block_device *bdev = lo->lo_device; | ||
| 1194 | struct gendisk *disk = lo->lo_disk; | ||
| 1195 | 1193 | ||
| 1196 | __module_get(disk->fops->owner); | 1194 | /* |
| 1197 | kobject_get(&bdev->bd_device.kobj); | 1195 | * Need not hold lo_mutex to fput backing file. Calling fput holding |
| 1198 | INIT_WORK(&lo->rundown_work, loop_rundown_workfn); | 1196 | * lo_mutex triggers a circular lock dependency possibility warning as |
| 1199 | queue_work(system_long_wq, &lo->rundown_work); | 1197 | * fput can take open_mutex which is usually taken before lo_mutex. |
| 1198 | */ | ||
| 1199 | fput(filp); | ||
| 1200 | } | 1200 | } |
| 1201 | 1201 | ||
| 1202 | static int loop_clr_fd(struct loop_device *lo) | 1202 | static int loop_clr_fd(struct loop_device *lo) |
| @@ -1228,8 +1228,7 @@ static int loop_clr_fd(struct loop_device *lo) · 25 unmodified lines | |||
| 1228 | lo->lo_state = Lo_rundown; | 1228 | lo->lo_state = Lo_rundown; |
| 1229 | mutex_unlock(&lo->lo_mutex); | 1229 | mutex_unlock(&lo->lo_mutex); |
| 1230 | 1230 | ||
| 1231 | __loop_clr_fd(lo); | 1231 | __loop_clr_fd(lo, false); |
| 1232 | loop_rundown_completed(lo); | ||
| 1233 | return 0; | 1232 | return 0; |
| 1234 | } | 1233 | } |
| 1235 | 1234 | ||
| @@ -1754,7 +1753,7 @@ static void lo_release(struct gendisk *disk, fmode_t mode) · 518 unmodified lines | |||
| 1754 | * In autoclear mode, stop the loop thread | 1753 | * In autoclear mode, stop the loop thread |
| 1755 | * and remove configuration after last close. | 1754 | * and remove configuration after last close. |
| 1756 | */ | 1755 | */ |
| 1757 | loop_schedule_rundown(lo); | 1756 | __loop_clr_fd(lo, true); |
| 1758 | return; | 1757 | return; |
| 1759 | } else if (lo->lo_state == Lo_bound) { | 1758 | } else if (lo->lo_state == Lo_bound) { |
| 1760 | /* | 1759 | /* |
| @@ -56,7 +56,6 @@ struct loop_device { · 55 unmodified lines | |||
| 56 | struct gendisk *lo_disk; | 56 | struct gendisk *lo_disk; |
| 57 | struct mutex lo_mutex; | 57 | struct mutex lo_mutex; |
| 58 | bool idr_visible; | 58 | bool idr_visible; |
| 59 | struct work_struct rundown_work; | ||
| 60 | }; | 59 | }; |
| 61 | 60 | ||
| 62 | struct loop_cmd { | 61 | struct loop_cmd { |
| @@ -4112,7 +4112,7 @@ static void mtip_pci_remove(struct pci_dev *pdev) · 4111 unmodified lines | |||
| 4112 | "Completion workers still active!\n"); | 4112 | "Completion workers still active!\n"); |
| 4113 | } | 4113 | } |
| 4114 | 4114 | ||
| 4115 | blk_set_queue_dying(dd->queue); | 4115 | blk_mark_disk_dead(dd->disk); |
| 4116 | set_bit(MTIP_DDF_REMOVE_PENDING_BIT, &dd->dd_flag); | 4116 | set_bit(MTIP_DDF_REMOVE_PENDING_BIT, &dd->dd_flag); |
| 4117 | 4117 | ||
| 4118 | /* Clean up the block layer. */ | 4118 | /* Clean up the block layer. */ |
| @@ -7185,7 +7185,7 @@ static ssize_t do_rbd_remove(struct bus_type *bus, · 7184 unmodified lines | |||
| 7185 | * IO to complete/fail. | 7185 | * IO to complete/fail. |
| 7186 | */ | 7186 | */ |
| 7187 | blk_mq_freeze_queue(rbd_dev->disk->queue); | 7187 | blk_mq_freeze_queue(rbd_dev->disk->queue); |
| 7188 | blk_set_queue_dying(rbd_dev->disk->queue); | 7188 | blk_mark_disk_dead(rbd_dev->disk); |
| 7189 | } | 7189 | } |
| 7190 | 7190 | ||
| 7191 | del_gendisk(rbd_dev->disk); | 7191 | del_gendisk(rbd_dev->disk); |
| @@ -2126,7 +2126,7 @@ static void blkfront_closing(struct blkfront_info *info) · 2125 unmodified lines | |||
| 2126 | 2126 | ||
| 2127 | /* No more blkif_request(). */ | 2127 | /* No more blkif_request(). */ |
| 2128 | blk_mq_stop_hw_queues(info->rq); | 2128 | blk_mq_stop_hw_queues(info->rq); |
| 2129 | blk_set_queue_dying(info->rq); | 2129 | blk_mark_disk_dead(info->gd); |
| 2130 | set_capacity(info->gd, 0); | 2130 | set_capacity(info->gd, 0); |
| 2131 | 2131 | ||
| 2132 | for_each_rinfo(info, rinfo, i) { | 2132 | for_each_rinfo(info, rinfo, i) { |
| @@ -366,6 +366,7 @@ static const struct mhi_pci_dev_info mhi_foxconn_sdx55_info = { · 365 unmodified lines | |||
| 366 | .config = &modem_foxconn_sdx55_config, | 366 | .config = &modem_foxconn_sdx55_config, |
| 367 | .bar_num = MHI_PCI_DEFAULT_BAR_NUM, | 367 | .bar_num = MHI_PCI_DEFAULT_BAR_NUM, |
| 368 | .dma_data_width = 32, | 368 | .dma_data_width = 32, |
| 369 | .mru_default = 32768, | ||
| 369 | .sideband_wake = false, | 370 | .sideband_wake = false, |
| 370 | }; | 371 | }; |
| 371 | 372 | ||
| @@ -401,6 +402,7 @@ static const struct mhi_pci_dev_info mhi_mv31_info = { · 29 unmodified lines | |||
| 401 | .config = &modem_mv31_config, | 402 | .config = &modem_mv31_config, |
| 402 | .bar_num = MHI_PCI_DEFAULT_BAR_NUM, | 403 | .bar_num = MHI_PCI_DEFAULT_BAR_NUM, |
| 403 | .dma_data_width = 32, | 404 | .dma_data_width = 32, |
| 405 | .mru_default = 32768, | ||
| 404 | }; | 406 | }; |
| 405 | 407 | ||
| 406 | static const struct mhi_channel_config mhi_sierra_em919x_channels[] = { | 408 | static const struct mhi_channel_config mhi_sierra_em919x_channels[] = { |
| @@ -762,7 +762,7 @@ static bool crng_init_try_arch(struct crng_state *crng) · 761 unmodified lines | |||
| 762 | return arch_init; | 762 | return arch_init; |
| 763 | } | 763 | } |
| 764 | 764 | ||
| 765 | static bool __init crng_init_try_arch_early(struct crng_state *crng) | 765 | static bool __init crng_init_try_arch_early(void) |
| 766 | { | 766 | { |
| 767 | int i; | 767 | int i; |
| 768 | bool arch_init = true; | 768 | bool arch_init = true; |
| @@ -774,7 +774,7 @@ static bool __init crng_init_try_arch_early(struct crng_state *crng) · 5 unmodified lines | |||
| 774 | rv = random_get_entropy(); | 774 | rv = random_get_entropy(); |
| 775 | arch_init = false; | 775 | arch_init = false; |
| 776 | } | 776 | } |
| 777 | crng->state[i] ^= rv; | 777 | primary_crng.state[i] ^= rv; |
| 778 | } | 778 | } |
| 779 | 779 | ||
| 780 | return arch_init; | 780 | return arch_init; |
| @@ -788,22 +788,20 @@ static void crng_initialize_secondary(struct crng_state *crng) · 7 unmodified lines | |||
| 788 | crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1; | 788 | crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1; |
| 789 | } | 789 | } |
| 790 | 790 | ||
| 791 | static void __init crng_initialize_primary(struct crng_state *crng) | 791 | static void __init crng_initialize_primary(void) |
| 792 | { | 792 | { |
| 793 | _extract_entropy(&crng->state[4], sizeof(u32) * 12); | 793 | _extract_entropy(&primary_crng.state[4], sizeof(u32) * 12); |
| 794 | if (crng_init_try_arch_early(crng) && trust_cpu && crng_init < 2) { | 794 | if (crng_init_try_arch_early() && trust_cpu && crng_init < 2) { |
| 795 | invalidate_batched_entropy(); | 795 | invalidate_batched_entropy(); |
| 796 | numa_crng_init(); | 796 | numa_crng_init(); |
| 797 | crng_init = 2; | 797 | crng_init = 2; |
| 798 | pr_notice("crng init done (trusting CPU's manufacturer)\n"); | 798 | pr_notice("crng init done (trusting CPU's manufacturer)\n"); |
| 799 | } | 799 | } |
| 800 | crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1; | 800 | primary_crng.init_time = jiffies - CRNG_RESEED_INTERVAL - 1; |
| 801 | } | 801 | } |
| 802 | 802 | ||
| 803 | static void crng_finalize_init(struct crng_state *crng) | 803 | static void crng_finalize_init(void) |
| 804 | { | 804 | { |
| 805 | if (crng != &primary_crng || crng_init >= 2) | ||
| 806 | return; | ||
| 807 | if (!system_wq) { | 805 | if (!system_wq) { |
| 808 | /* We can't call numa_crng_init until we have workqueues, | 806 | /* We can't call numa_crng_init until we have workqueues, |
| 809 | * so mark this for processing later. */ | 807 | * so mark this for processing later. */ |
| @@ -814,6 +812,7 @@ static void crng_finalize_init(struct crng_state *crng) · 4 unmodified lines | |||
| 814 | invalidate_batched_entropy(); | 812 | invalidate_batched_entropy(); |
| 815 | numa_crng_init(); | 813 | numa_crng_init(); |
| 816 | crng_init = 2; | 814 | crng_init = 2; |
| 815 | crng_need_final_init = false; | ||
| 817 | process_random_ready_list(); | 816 | process_random_ready_list(); |
| 818 | wake_up_interruptible(&crng_init_wait); | 817 | wake_up_interruptible(&crng_init_wait); |
| 819 | kill_fasync(&fasync, SIGIO, POLL_IN); | 818 | kill_fasync(&fasync, SIGIO, POLL_IN); |
| @@ -980,7 +979,8 @@ static void crng_reseed(struct crng_state *crng, bool use_input_pool) · 160 unmodified lines | |||
| 980 | memzero_explicit(&buf, sizeof(buf)); | 979 | memzero_explicit(&buf, sizeof(buf)); |
| 981 | WRITE_ONCE(crng->init_time, jiffies); | 980 | WRITE_ONCE(crng->init_time, jiffies); |
| 982 | spin_unlock_irqrestore(&crng->lock, flags); | 981 | spin_unlock_irqrestore(&crng->lock, flags); |
| 983 | crng_finalize_init(crng); | 982 | if (crng == &primary_crng && crng_init < 2) |
| 983 | crng_finalize_init(); | ||
| 984 | } | 984 | } |
| 985 | 985 | ||
| 986 | static void _extract_crng(struct crng_state *crng, u8 out[CHACHA_BLOCK_SIZE]) | 986 | static void _extract_crng(struct crng_state *crng, u8 out[CHACHA_BLOCK_SIZE]) |
| @@ -1697,8 +1697,8 @@ int __init rand_initialize(void) · 710 unmodified lines | |||
| 1697 | { | 1697 | { |
| 1698 | init_std_data(); | 1698 | init_std_data(); |
| 1699 | if (crng_need_final_init) | 1699 | if (crng_need_final_init) |
| 1700 | crng_finalize_init(&primary_crng); | 1700 | crng_finalize_init(); |
| 1701 | crng_initialize_primary(&primary_crng); | 1701 | crng_initialize_primary(); |
| 1702 | crng_global_init_time = jiffies; | 1702 | crng_global_init_time = jiffies; |
| 1703 | if (ratelimit_disable) { | 1703 | if (ratelimit_disable) { |
| 1704 | urandom_warning.interval = 0; | 1704 | urandom_warning.interval = 0; |
| @@ -1856,7 +1856,10 @@ static long random_ioctl(struct file *f, unsigned int cmd, unsigned long arg) · 151 unmodified lines | |||
| 1856 | */ | 1856 | */ |
| 1857 | if (!capable(CAP_SYS_ADMIN)) | 1857 | if (!capable(CAP_SYS_ADMIN)) |
| 1858 | return -EPERM; | 1858 | return -EPERM; |
| 1859 | input_pool.entropy_count = 0; | 1859 | if (xchg(&input_pool.entropy_count, 0) && random_write_wakeup_bits) { |
| 1860 | wake_up_interruptible(&random_write_wait); | ||
| 1861 | kill_fasync(&fasync, SIGIO, POLL_OUT); | ||
| 1862 | } | ||
| 1860 | return 0; | 1863 | return 0; |
| 1861 | case RNDRESEEDCRNG: | 1864 | case RNDRESEEDCRNG: |
| 1862 | if (!capable(CAP_SYS_ADMIN)) | 1865 | if (!capable(CAP_SYS_ADMIN)) |
| @@ -2205,13 +2208,15 @@ void add_hwgenerator_randomness(const char *buffer, size_t count, · 342 unmodified lines | |||
| 2205 | return; | 2208 | return; |
| 2206 | } | 2209 | } |
| 2207 | 2210 | ||
| 2208 | /* Suspend writing if we're above the trickle threshold. | 2211 | /* Throttle writing if we're above the trickle threshold. |
| 2209 | * We'll be woken up again once below random_write_wakeup_thresh, | 2212 | * We'll be woken up again once below random_write_wakeup_thresh, |
| 2210 | * or when the calling thread is about to terminate. | 2213 | * when the calling thread is about to terminate, or once |
| 2214 | * CRNG_RESEED_INTERVAL has lapsed. | ||
| 2211 | */ | 2215 | */ |
| 2212 | wait_event_interruptible(random_write_wait, | 2216 | wait_event_interruptible_timeout(random_write_wait, |
| 2213 | !system_wq || kthread_should_stop() || | 2217 | !system_wq || kthread_should_stop() || |
| 2214 | POOL_ENTROPY_BITS() <= random_write_wakeup_bits); | 2218 | POOL_ENTROPY_BITS() <= random_write_wakeup_bits, |
| 2219 | CRNG_RESEED_INTERVAL); | ||
| 2215 | mix_pool_bytes(buffer, count); | 2220 | mix_pool_bytes(buffer, count); |
| 2216 | credit_entropy_bits(entropy); | 2221 | credit_entropy_bits(entropy); |
| 2217 | } | 2222 | } |
| @@ -139,11 +139,10 @@ static const struct ingenic_cgu_clk_info jz4725b_cgu_clocks[] = { · 138 unmodified lines | |||
| 139 | }, | 139 | }, |
| 140 | 140 | ||
| 141 | [JZ4725B_CLK_I2S] = { | 141 | [JZ4725B_CLK_I2S] = { |
| 142 | "i2s", CGU_CLK_MUX | CGU_CLK_DIV | CGU_CLK_GATE, | 142 | "i2s", CGU_CLK_MUX | CGU_CLK_DIV, |
| 143 | .parents = { JZ4725B_CLK_EXT, JZ4725B_CLK_PLL_HALF, -1, -1 }, | 143 | .parents = { JZ4725B_CLK_EXT, JZ4725B_CLK_PLL_HALF, -1, -1 }, |
| 144 | .mux = { CGU_REG_CPCCR, 31, 1 }, | 144 | .mux = { CGU_REG_CPCCR, 31, 1 }, |
| 145 | .div = { CGU_REG_I2SCDR, 0, 1, 9, -1, -1, -1 }, | 145 | .div = { CGU_REG_I2SCDR, 0, 1, 9, -1, -1, -1 }, |
| 146 | .gate = { CGU_REG_CLKGR, 6 }, | ||
| 147 | }, | 146 | }, |
| 148 | 147 | ||
| 149 | [JZ4725B_CLK_SPI] = { | 148 | [JZ4725B_CLK_SPI] = { |
| @@ -108,42 +108,6 @@ static const struct clk_parent_data gcc_xo_gpll0_gpll4[] = { · 107 unmodified lines | |||
| 108 | { .hw = &gpll4.clkr.hw }, | 108 | { .hw = &gpll4.clkr.hw }, |
| 109 | }; | 109 | }; |
| 110 | 110 | ||
| 111 | static struct clk_rcg2 system_noc_clk_src = { | ||
| 112 | .cmd_rcgr = 0x0120, | ||
| 113 | .hid_width = 5, | ||
| 114 | .parent_map = gcc_xo_gpll0_map, | ||
| 115 | .clkr.hw.init = &(struct clk_init_data){ | ||
| 116 | .name = "system_noc_clk_src", | ||
| 117 | .parent_data = gcc_xo_gpll0, | ||
| 118 | .num_parents = ARRAY_SIZE(gcc_xo_gpll0), | ||
| 119 | .ops = &clk_rcg2_ops, | ||
| 120 | }, | ||
| 121 | }; | ||
| 122 | |||
| 123 | static struct clk_rcg2 config_noc_clk_src = { | ||
| 124 | .cmd_rcgr = 0x0150, | ||
| 125 | .hid_width = 5, | ||
| 126 | .parent_map = gcc_xo_gpll0_map, | ||
| 127 | .clkr.hw.init = &(struct clk_init_data){ | ||
| 128 | .name = "config_noc_clk_src", | ||
| 129 | .parent_data = gcc_xo_gpll0, | ||
| 130 | .num_parents = ARRAY_SIZE(gcc_xo_gpll0), | ||
| 131 | .ops = &clk_rcg2_ops, | ||
| 132 | }, | ||
| 133 | }; | ||
| 134 | |||
| 135 | static struct clk_rcg2 periph_noc_clk_src = { | ||
| 136 | .cmd_rcgr = 0x0190, | ||
| 137 | .hid_width = 5, | ||
| 138 | .parent_map = gcc_xo_gpll0_map, | ||
| 139 | .clkr.hw.init = &(struct clk_init_data){ | ||
| 140 | .name = "periph_noc_clk_src", | ||
| 141 | .parent_data = gcc_xo_gpll0, | ||
| 142 | .num_parents = ARRAY_SIZE(gcc_xo_gpll0), | ||
| 143 | .ops = &clk_rcg2_ops, | ||
| 144 | }, | ||
| 145 | }; | ||
| 146 | |||
| 147 | static struct freq_tbl ftbl_ufs_axi_clk_src[] = { | 111 | static struct freq_tbl ftbl_ufs_axi_clk_src[] = { |
| 148 | F(50000000, P_GPLL0, 12, 0, 0), | 112 | F(50000000, P_GPLL0, 12, 0, 0), |
| 149 | F(100000000, P_GPLL0, 6, 0, 0), | 113 | F(100000000, P_GPLL0, 6, 0, 0), |
| @@ -1150,8 +1114,6 @@ static struct clk_branch gcc_blsp1_ahb_clk = { · 1000 unmodified lines | |||
| 1150 | .enable_mask = BIT(17), | 1114 | .enable_mask = BIT(17), |
| 1151 | .hw.init = &(struct clk_init_data){ | 1115 | .hw.init = &(struct clk_init_data){ |
| 1152 | .name = "gcc_blsp1_ahb_clk", | 1116 | .name = "gcc_blsp1_ahb_clk", |
| 1153 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 1154 | .num_parents = 1, | ||
| 1155 | .ops = &clk_branch2_ops, | 1117 | .ops = &clk_branch2_ops, |
| 1156 | }, | 1118 | }, |
| 1157 | }, | 1119 | }, |
| @@ -1435,8 +1397,6 @@ static struct clk_branch gcc_blsp2_ahb_clk = { · 277 unmodified lines | |||
| 1435 | .enable_mask = BIT(15), | 1397 | .enable_mask = BIT(15), |
| 1436 | .hw.init = &(struct clk_init_data){ | 1398 | .hw.init = &(struct clk_init_data){ |
| 1437 | .name = "gcc_blsp2_ahb_clk", | 1399 | .name = "gcc_blsp2_ahb_clk", |
| 1438 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 1439 | .num_parents = 1, | ||
| 1440 | .ops = &clk_branch2_ops, | 1400 | .ops = &clk_branch2_ops, |
| 1441 | }, | 1401 | }, |
| 1442 | }, | 1402 | }, |
| @@ -1764,8 +1724,6 @@ static struct clk_branch gcc_lpass_q6_axi_clk = { · 321 unmodified lines | |||
| 1764 | .enable_mask = BIT(0), | 1724 | .enable_mask = BIT(0), |
| 1765 | .hw.init = &(struct clk_init_data){ | 1725 | .hw.init = &(struct clk_init_data){ |
| 1766 | .name = "gcc_lpass_q6_axi_clk", | 1726 | .name = "gcc_lpass_q6_axi_clk", |
| 1767 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 1768 | .num_parents = 1, | ||
| 1769 | .ops = &clk_branch2_ops, | 1727 | .ops = &clk_branch2_ops, |
| 1770 | }, | 1728 | }, |
| 1771 | }, | 1729 | }, |
| @@ -1778,8 +1736,6 @@ static struct clk_branch gcc_mss_q6_bimc_axi_clk = { · 6 unmodified lines | |||
| 1778 | .enable_mask = BIT(0), | 1736 | .enable_mask = BIT(0), |
| 1779 | .hw.init = &(struct clk_init_data){ | 1737 | .hw.init = &(struct clk_init_data){ |
| 1780 | .name = "gcc_mss_q6_bimc_axi_clk", | 1738 | .name = "gcc_mss_q6_bimc_axi_clk", |
| 1781 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 1782 | .num_parents = 1, | ||
| 1783 | .ops = &clk_branch2_ops, | 1739 | .ops = &clk_branch2_ops, |
| 1784 | }, | 1740 | }, |
| 1785 | }, | 1741 | }, |
| @@ -1807,9 +1763,6 @@ static struct clk_branch gcc_pcie_0_cfg_ahb_clk = { · 21 unmodified lines | |||
| 1807 | .enable_mask = BIT(0), | 1763 | .enable_mask = BIT(0), |
| 1808 | .hw.init = &(struct clk_init_data){ | 1764 | .hw.init = &(struct clk_init_data){ |
| 1809 | .name = "gcc_pcie_0_cfg_ahb_clk", | 1765 | .name = "gcc_pcie_0_cfg_ahb_clk", |
| 1810 | .parent_hws = (const struct clk_hw *[]){ &config_noc_clk_src.clkr.hw }, | ||
| 1811 | .num_parents = 1, | ||
| 1812 | .flags = CLK_SET_RATE_PARENT, | ||
| 1813 | .ops = &clk_branch2_ops, | 1766 | .ops = &clk_branch2_ops, |
| 1814 | }, | 1767 | }, |
| 1815 | }, | 1768 | }, |
| @@ -1822,9 +1775,6 @@ static struct clk_branch gcc_pcie_0_mstr_axi_clk = { · 6 unmodified lines | |||
| 1822 | .enable_mask = BIT(0), | 1775 | .enable_mask = BIT(0), |
| 1823 | .hw.init = &(struct clk_init_data){ | 1776 | .hw.init = &(struct clk_init_data){ |
| 1824 | .name = "gcc_pcie_0_mstr_axi_clk", | 1777 | .name = "gcc_pcie_0_mstr_axi_clk", |
| 1825 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 1826 | .num_parents = 1, | ||
| 1827 | .flags = CLK_SET_RATE_PARENT, | ||
| 1828 | .ops = &clk_branch2_ops, | 1778 | .ops = &clk_branch2_ops, |
| 1829 | }, | 1779 | }, |
| 1830 | }, | 1780 | }, |
| @@ -1854,9 +1804,6 @@ static struct clk_branch gcc_pcie_0_slv_axi_clk = { · 23 unmodified lines | |||
| 1854 | .enable_mask = BIT(0), | 1804 | .enable_mask = BIT(0), |
| 1855 | .hw.init = &(struct clk_init_data){ | 1805 | .hw.init = &(struct clk_init_data){ |
| 1856 | .name = "gcc_pcie_0_slv_axi_clk", | 1806 | .name = "gcc_pcie_0_slv_axi_clk", |
| 1857 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 1858 | .num_parents = 1, | ||
| 1859 | .flags = CLK_SET_RATE_PARENT, | ||
| 1860 | .ops = &clk_branch2_ops, | 1807 | .ops = &clk_branch2_ops, |
| 1861 | }, | 1808 | }, |
| 1862 | }, | 1809 | }, |
| @@ -1884,9 +1831,6 @@ static struct clk_branch gcc_pcie_1_cfg_ahb_clk = { · 21 unmodified lines | |||
| 1884 | .enable_mask = BIT(0), | 1831 | .enable_mask = BIT(0), |
| 1885 | .hw.init = &(struct clk_init_data){ | 1832 | .hw.init = &(struct clk_init_data){ |
| 1886 | .name = "gcc_pcie_1_cfg_ahb_clk", | 1833 | .name = "gcc_pcie_1_cfg_ahb_clk", |
| 1887 | .parent_hws = (const struct clk_hw *[]){ &config_noc_clk_src.clkr.hw }, | ||
| 1888 | .num_parents = 1, | ||
| 1889 | .flags = CLK_SET_RATE_PARENT, | ||
| 1890 | .ops = &clk_branch2_ops, | 1834 | .ops = &clk_branch2_ops, |
| 1891 | }, | 1835 | }, |
| 1892 | }, | 1836 | }, |
| @@ -1899,9 +1843,6 @@ static struct clk_branch gcc_pcie_1_mstr_axi_clk = { · 6 unmodified lines | |||
| 1899 | .enable_mask = BIT(0), | 1843 | .enable_mask = BIT(0), |
| 1900 | .hw.init = &(struct clk_init_data){ | 1844 | .hw.init = &(struct clk_init_data){ |
| 1901 | .name = "gcc_pcie_1_mstr_axi_clk", | 1845 | .name = "gcc_pcie_1_mstr_axi_clk", |
| 1902 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 1903 | .num_parents = 1, | ||
| 1904 | .flags = CLK_SET_RATE_PARENT, | ||
| 1905 | .ops = &clk_branch2_ops, | 1846 | .ops = &clk_branch2_ops, |
| 1906 | }, | 1847 | }, |
| 1907 | }, | 1848 | }, |
| @@ -1930,9 +1871,6 @@ static struct clk_branch gcc_pcie_1_slv_axi_clk = { · 22 unmodified lines | |||
| 1930 | .enable_mask = BIT(0), | 1871 | .enable_mask = BIT(0), |
| 1931 | .hw.init = &(struct clk_init_data){ | 1872 | .hw.init = &(struct clk_init_data){ |
| 1932 | .name = "gcc_pcie_1_slv_axi_clk", | 1873 | .name = "gcc_pcie_1_slv_axi_clk", |
| 1933 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 1934 | .num_parents = 1, | ||
| 1935 | .flags = CLK_SET_RATE_PARENT, | ||
| 1936 | .ops = &clk_branch2_ops, | 1874 | .ops = &clk_branch2_ops, |
| 1937 | }, | 1875 | }, |
| 1938 | }, | 1876 | }, |
| @@ -1960,8 +1898,6 @@ static struct clk_branch gcc_pdm_ahb_clk = { · 21 unmodified lines | |||
| 1960 | .enable_mask = BIT(0), | 1898 | .enable_mask = BIT(0), |
| 1961 | .hw.init = &(struct clk_init_data){ | 1899 | .hw.init = &(struct clk_init_data){ |
| 1962 | .name = "gcc_pdm_ahb_clk", | 1900 | .name = "gcc_pdm_ahb_clk", |
| 1963 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 1964 | .num_parents = 1, | ||
| 1965 | .ops = &clk_branch2_ops, | 1901 | .ops = &clk_branch2_ops, |
| 1966 | }, | 1902 | }, |
| 1967 | }, | 1903 | }, |
| @@ -1989,9 +1925,6 @@ static struct clk_branch gcc_sdcc1_ahb_clk = { · 21 unmodified lines | |||
| 1989 | .enable_mask = BIT(0), | 1925 | .enable_mask = BIT(0), |
| 1990 | .hw.init = &(struct clk_init_data){ | 1926 | .hw.init = &(struct clk_init_data){ |
| 1991 | .name = "gcc_sdcc1_ahb_clk", | 1927 | .name = "gcc_sdcc1_ahb_clk", |
| 1992 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 1993 | .num_parents = 1, | ||
| 1994 | .flags = CLK_SET_RATE_PARENT, | ||
| 1995 | .ops = &clk_branch2_ops, | 1928 | .ops = &clk_branch2_ops, |
| 1996 | }, | 1929 | }, |
| 1997 | }, | 1930 | }, |
| @@ -2004,9 +1937,6 @@ static struct clk_branch gcc_sdcc2_ahb_clk = { · 6 unmodified lines | |||
| 2004 | .enable_mask = BIT(0), | 1937 | .enable_mask = BIT(0), |
| 2005 | .hw.init = &(struct clk_init_data){ | 1938 | .hw.init = &(struct clk_init_data){ |
| 2006 | .name = "gcc_sdcc2_ahb_clk", | 1939 | .name = "gcc_sdcc2_ahb_clk", |
| 2007 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 2008 | .num_parents = 1, | ||
| 2009 | .flags = CLK_SET_RATE_PARENT, | ||
| 2010 | .ops = &clk_branch2_ops, | 1940 | .ops = &clk_branch2_ops, |
| 2011 | }, | 1941 | }, |
| 2012 | }, | 1942 | }, |
| @@ -2034,9 +1964,6 @@ static struct clk_branch gcc_sdcc3_ahb_clk = { · 21 unmodified lines | |||
| 2034 | .enable_mask = BIT(0), | 1964 | .enable_mask = BIT(0), |
| 2035 | .hw.init = &(struct clk_init_data){ | 1965 | .hw.init = &(struct clk_init_data){ |
| 2036 | .name = "gcc_sdcc3_ahb_clk", | 1966 | .name = "gcc_sdcc3_ahb_clk", |
| 2037 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 2038 | .num_parents = 1, | ||
| 2039 | .flags = CLK_SET_RATE_PARENT, | ||
| 2040 | .ops = &clk_branch2_ops, | 1967 | .ops = &clk_branch2_ops, |
| 2041 | }, | 1968 | }, |
| 2042 | }, | 1969 | }, |
| @@ -2064,9 +1991,6 @@ static struct clk_branch gcc_sdcc4_ahb_clk = { · 21 unmodified lines | |||
| 2064 | .enable_mask = BIT(0), | 1991 | .enable_mask = BIT(0), |
| 2065 | .hw.init = &(struct clk_init_data){ | 1992 | .hw.init = &(struct clk_init_data){ |
| 2066 | .name = "gcc_sdcc4_ahb_clk", | 1993 | .name = "gcc_sdcc4_ahb_clk", |
| 2067 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 2068 | .num_parents = 1, | ||
| 2069 | .flags = CLK_SET_RATE_PARENT, | ||
| 2070 | .ops = &clk_branch2_ops, | 1994 | .ops = &clk_branch2_ops, |
| 2071 | }, | 1995 | }, |
| 2072 | }, | 1996 | }, |
| @@ -2124,8 +2048,6 @@ static struct clk_branch gcc_tsif_ahb_clk = { · 51 unmodified lines | |||
| 2124 | .enable_mask = BIT(0), | 2048 | .enable_mask = BIT(0), |
| 2125 | .hw.init = &(struct clk_init_data){ | 2049 | .hw.init = &(struct clk_init_data){ |
| 2126 | .name = "gcc_tsif_ahb_clk", | 2050 | .name = "gcc_tsif_ahb_clk", |
| 2127 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 2128 | .num_parents = 1, | ||
| 2129 | .ops = &clk_branch2_ops, | 2051 | .ops = &clk_branch2_ops, |
| 2130 | }, | 2052 | }, |
| 2131 | }, | 2053 | }, |
| @@ -2153,8 +2075,6 @@ static struct clk_branch gcc_ufs_ahb_clk = { · 21 unmodified lines | |||
| 2153 | .enable_mask = BIT(0), | 2075 | .enable_mask = BIT(0), |
| 2154 | .hw.init = &(struct clk_init_data){ | 2076 | .hw.init = &(struct clk_init_data){ |
| 2155 | .name = "gcc_ufs_ahb_clk", | 2077 | .name = "gcc_ufs_ahb_clk", |
| 2156 | .parent_hws = (const struct clk_hw *[]){ &config_noc_clk_src.clkr.hw }, | ||
| 2157 | .num_parents = 1, | ||
| 2158 | .ops = &clk_branch2_ops, | 2078 | .ops = &clk_branch2_ops, |
| 2159 | }, | 2079 | }, |
| 2160 | }, | 2080 | }, |
| @@ -2198,8 +2118,6 @@ static struct clk_branch gcc_ufs_rx_symbol_0_clk = { · 37 unmodified lines | |||
| 2198 | .enable_mask = BIT(0), | 2118 | .enable_mask = BIT(0), |
| 2199 | .hw.init = &(struct clk_init_data){ | 2119 | .hw.init = &(struct clk_init_data){ |
| 2200 | .name = "gcc_ufs_rx_symbol_0_clk", | 2120 | .name = "gcc_ufs_rx_symbol_0_clk", |
| 2201 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 2202 | .num_parents = 1, | ||
| 2203 | .ops = &clk_branch2_ops, | 2121 | .ops = &clk_branch2_ops, |
| 2204 | }, | 2122 | }, |
| 2205 | }, | 2123 | }, |
| @@ -2213,8 +2131,6 @@ static struct clk_branch gcc_ufs_rx_symbol_1_clk = { · 7 unmodified lines | |||
| 2213 | .enable_mask = BIT(0), | 2131 | .enable_mask = BIT(0), |
| 2214 | .hw.init = &(struct clk_init_data){ | 2132 | .hw.init = &(struct clk_init_data){ |
| 2215 | .name = "gcc_ufs_rx_symbol_1_clk", | 2133 | .name = "gcc_ufs_rx_symbol_1_clk", |
| 2216 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 2217 | .num_parents = 1, | ||
| 2218 | .ops = &clk_branch2_ops, | 2134 | .ops = &clk_branch2_ops, |
| 2219 | }, | 2135 | }, |
| 2220 | }, | 2136 | }, |
| @@ -2243,8 +2159,6 @@ static struct clk_branch gcc_ufs_tx_symbol_0_clk = { · 22 unmodified lines | |||
| 2243 | .enable_mask = BIT(0), | 2159 | .enable_mask = BIT(0), |
| 2244 | .hw.init = &(struct clk_init_data){ | 2160 | .hw.init = &(struct clk_init_data){ |
| 2245 | .name = "gcc_ufs_tx_symbol_0_clk", | 2161 | .name = "gcc_ufs_tx_symbol_0_clk", |
| 2246 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 2247 | .num_parents = 1, | ||
| 2248 | .ops = &clk_branch2_ops, | 2162 | .ops = &clk_branch2_ops, |
| 2249 | }, | 2163 | }, |
| 2250 | }, | 2164 | }, |
| @@ -2258,8 +2172,6 @@ static struct clk_branch gcc_ufs_tx_symbol_1_clk = { · 7 unmodified lines | |||
| 2258 | .enable_mask = BIT(0), | 2172 | .enable_mask = BIT(0), |
| 2259 | .hw.init = &(struct clk_init_data){ | 2173 | .hw.init = &(struct clk_init_data){ |
| 2260 | .name = "gcc_ufs_tx_symbol_1_clk", | 2174 | .name = "gcc_ufs_tx_symbol_1_clk", |
| 2261 | .parent_hws = (const struct clk_hw *[]){ &system_noc_clk_src.clkr.hw }, | ||
| 2262 | .num_parents = 1, | ||
| 2263 | .ops = &clk_branch2_ops, | 2175 | .ops = &clk_branch2_ops, |
| 2264 | }, | 2176 | }, |
| 2265 | }, | 2177 | }, |
| @@ -2364,8 +2276,6 @@ static struct clk_branch gcc_usb_hs_ahb_clk = { · 98 unmodified lines | |||
| 2364 | .enable_mask = BIT(0), | 2276 | .enable_mask = BIT(0), |
| 2365 | .hw.init = &(struct clk_init_data){ | 2277 | .hw.init = &(struct clk_init_data){ |
| 2366 | .name = "gcc_usb_hs_ahb_clk", | 2278 | .name = "gcc_usb_hs_ahb_clk", |
| 2367 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 2368 | .num_parents = 1, | ||
| 2369 | .ops = &clk_branch2_ops, | 2279 | .ops = &clk_branch2_ops, |
| 2370 | }, | 2280 | }, |
| 2371 | }, | 2281 | }, |
| @@ -2488,8 +2398,6 @@ static struct clk_branch gcc_boot_rom_ahb_clk = { · 116 unmodified lines | |||
| 2488 | .enable_mask = BIT(10), | 2398 | .enable_mask = BIT(10), |
| 2489 | .hw.init = &(struct clk_init_data){ | 2399 | .hw.init = &(struct clk_init_data){ |
| 2490 | .name = "gcc_boot_rom_ahb_clk", | 2400 | .name = "gcc_boot_rom_ahb_clk", |
| 2491 | .parent_hws = (const struct clk_hw *[]){ &config_noc_clk_src.clkr.hw }, | ||
| 2492 | .num_parents = 1, | ||
| 2493 | .ops = &clk_branch2_ops, | 2401 | .ops = &clk_branch2_ops, |
| 2494 | }, | 2402 | }, |
| 2495 | }, | 2403 | }, |
| @@ -2503,8 +2411,6 @@ static struct clk_branch gcc_prng_ahb_clk = { · 7 unmodified lines | |||
| 2503 | .enable_mask = BIT(13), | 2411 | .enable_mask = BIT(13), |
| 2504 | .hw.init = &(struct clk_init_data){ | 2412 | .hw.init = &(struct clk_init_data){ |
| 2505 | .name = "gcc_prng_ahb_clk", | 2413 | .name = "gcc_prng_ahb_clk", |
| 2506 | .parent_hws = (const struct clk_hw *[]){ &periph_noc_clk_src.clkr.hw }, | ||
| 2507 | .num_parents = 1, | ||
| 2508 | .ops = &clk_branch2_ops, | 2414 | .ops = &clk_branch2_ops, |
| 2509 | }, | 2415 | }, |
| 2510 | }, | 2416 | }, |
| @@ -2547,9 +2453,6 @@ static struct clk_regmap *gcc_msm8994_clocks[] = { · 36 unmodified lines | |||
| 2547 | [GPLL0] = &gpll0.clkr, | 2453 | [GPLL0] = &gpll0.clkr, |
| 2548 | [GPLL4_EARLY] = &gpll4_early.clkr, | 2454 | [GPLL4_EARLY] = &gpll4_early.clkr, |
| 2549 | [GPLL4] = &gpll4.clkr, | 2455 | [GPLL4] = &gpll4.clkr, |
| 2550 | [CONFIG_NOC_CLK_SRC] = &config_noc_clk_src.clkr, | ||
| 2551 | [PERIPH_NOC_CLK_SRC] = &periph_noc_clk_src.clkr, | ||
| 2552 | [SYSTEM_NOC_CLK_SRC] = &system_noc_clk_src.clkr, | ||
| 2553 | [UFS_AXI_CLK_SRC] = &ufs_axi_clk_src.clkr, | 2456 | [UFS_AXI_CLK_SRC] = &ufs_axi_clk_src.clkr, |
| 2554 | [USB30_MASTER_CLK_SRC] = &usb30_master_clk_src.clkr, | 2457 | [USB30_MASTER_CLK_SRC] = &usb30_master_clk_src.clkr, |
| 2555 | [BLSP1_QUP1_I2C_APPS_CLK_SRC] = &blsp1_qup1_i2c_apps_clk_src.clkr, | 2458 | [BLSP1_QUP1_I2C_APPS_CLK_SRC] = &blsp1_qup1_i2c_apps_clk_src.clkr, |
| @@ -2696,6 +2599,15 @@ static struct clk_regmap *gcc_msm8994_clocks[] = { · 140 unmodified lines | |||
| 2696 | [USB_SS_PHY_LDO] = &usb_ss_phy_ldo.clkr, | 2599 | [USB_SS_PHY_LDO] = &usb_ss_phy_ldo.clkr, |
| 2697 | [GCC_BOOT_ROM_AHB_CLK] = &gcc_boot_rom_ahb_clk.clkr, | 2600 | [GCC_BOOT_ROM_AHB_CLK] = &gcc_boot_rom_ahb_clk.clkr, |
| 2698 | [GCC_PRNG_AHB_CLK] = &gcc_prng_ahb_clk.clkr, | 2601 | [GCC_PRNG_AHB_CLK] = &gcc_prng_ahb_clk.clkr, |
| 2602 | |||
| 2603 | /* | ||
| 2604 | * The following clocks should NOT be managed by this driver, but they once were | ||
| 2605 | * mistakengly added. Now they are only here to indicate that they are not defined | ||
| 2606 | * on purpose, even though the names will stay in the header file (for ABI sanity). | ||
| 2607 | */ | ||
| 2608 | [CONFIG_NOC_CLK_SRC] = NULL, | ||
| 2609 | [PERIPH_NOC_CLK_SRC] = NULL, | ||
| 2610 | [SYSTEM_NOC_CLK_SRC] = NULL, | ||
| 2699 | }; | 2611 | }; |
| 2700 | 2612 | ||
| 2701 | static struct gdsc *gcc_msm8994_gdscs[] = { | 2613 | static struct gdsc *gcc_msm8994_gdscs[] = { |
| @@ -241,7 +241,7 @@ static void __init dmtimer_systimer_assign_alwon(void) · 240 unmodified lines | |||
| 241 | bool quirk_unreliable_oscillator = false; | 241 | bool quirk_unreliable_oscillator = false; |
| 242 | 242 | ||
| 243 | /* Quirk unreliable 32 KiHz oscillator with incomplete dts */ | 243 | /* Quirk unreliable 32 KiHz oscillator with incomplete dts */ |
| 244 | if (of_machine_is_compatible("ti,omap3-beagle") || | 244 | if (of_machine_is_compatible("ti,omap3-beagle-ab4") || |
| 245 | of_machine_is_compatible("timll,omap3-devkit8000")) { | 245 | of_machine_is_compatible("timll,omap3-devkit8000")) { |
| 246 | quirk_unreliable_oscillator = true; | 246 | quirk_unreliable_oscillator = true; |
| 247 | counter_32k = -ENODEV; | 247 | counter_32k = -ENODEV; |
| @@ -1518,6 +1518,10 @@ static int cpufreq_online(unsigned int cpu) · 1517 unmodified lines | |||
| 1518 | 1518 | ||
| 1519 | kobject_uevent(&policy->kobj, KOBJ_ADD); | 1519 | kobject_uevent(&policy->kobj, KOBJ_ADD); |
| 1520 | 1520 | ||
| 1521 | /* Callback for handling stuff after policy is ready */ | ||
| 1522 | if (cpufreq_driver->ready) | ||
| 1523 | cpufreq_driver->ready(policy); | ||
| 1524 | |||
| 1521 | if (cpufreq_thermal_control_enabled(cpufreq_driver)) | 1525 | if (cpufreq_thermal_control_enabled(cpufreq_driver)) |
| 1522 | policy->cdev = of_cpufreq_cooling_register(policy); | 1526 | policy->cdev = of_cpufreq_cooling_register(policy); |
| 1523 | 1527 | ||
| @@ -388,7 +388,7 @@ static int qcom_cpufreq_hw_lmh_init(struct cpufreq_policy *policy, int index) · 387 unmodified lines | |||
| 388 | 388 | ||
| 389 | snprintf(data->irq_name, sizeof(data->irq_name), "dcvsh-irq-%u", policy->cpu); | 389 | snprintf(data->irq_name, sizeof(data->irq_name), "dcvsh-irq-%u", policy->cpu); |
| 390 | ret = request_threaded_irq(data->throttle_irq, NULL, qcom_lmh_dcvs_handle_irq, | 390 | ret = request_threaded_irq(data->throttle_irq, NULL, qcom_lmh_dcvs_handle_irq, |
| 391 | IRQF_ONESHOT, data->irq_name, data); | 391 | IRQF_ONESHOT | IRQF_NO_AUTOEN, data->irq_name, data); |
| 392 | if (ret) { | 392 | if (ret) { |
| 393 | dev_err(&pdev->dev, "Error registering %s: %d\n", data->irq_name, ret); | 393 | dev_err(&pdev->dev, "Error registering %s: %d\n", data->irq_name, ret); |
| 394 | return 0; | 394 | return 0; |
| @@ -542,6 +542,14 @@ static int qcom_cpufreq_hw_cpu_exit(struct cpufreq_policy *policy) · 147 unmodified lines | |||
| 542 | return 0; | 542 | return 0; |
| 543 | } | 543 | } |
| 544 | 544 | ||
| 545 | static void qcom_cpufreq_ready(struct cpufreq_policy *policy) | ||
| 546 | { | ||
| 547 | struct qcom_cpufreq_data *data = policy->driver_data; | ||
| 548 | |||
| 549 | if (data->throttle_irq >= 0) | ||
| 550 | enable_irq(data->throttle_irq); | ||
| 551 | } | ||
| 552 | |||
| 545 | static struct freq_attr *qcom_cpufreq_hw_attr[] = { | 553 | static struct freq_attr *qcom_cpufreq_hw_attr[] = { |
| 546 | &cpufreq_freq_attr_scaling_available_freqs, | 554 | &cpufreq_freq_attr_scaling_available_freqs, |
| 547 | &cpufreq_freq_attr_scaling_boost_freqs, | 555 | &cpufreq_freq_attr_scaling_boost_freqs, |
| @@ -561,6 +569,7 @@ static struct cpufreq_driver cpufreq_qcom_hw_driver = { · 13 unmodified lines | |||
| 561 | .fast_switch = qcom_cpufreq_hw_fast_switch, | 569 | .fast_switch = qcom_cpufreq_hw_fast_switch, |
| 562 | .name = "qcom-cpufreq-hw", | 570 | .name = "qcom-cpufreq-hw", |
| 563 | .attr = qcom_cpufreq_hw_attr, | 571 | .attr = qcom_cpufreq_hw_attr, |
| 572 | .ready = qcom_cpufreq_ready, | ||
| 564 | }; | 573 | }; |
| 565 | 574 | ||
| 566 | static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev) | 575 | static int qcom_cpufreq_hw_driver_probe(struct platform_device *pdev) |
| @@ -1753,7 +1753,6 @@ void otx2_cpt_print_uc_dbg_info(struct otx2_cptpf_dev *cptpf) · 1752 unmodified lines | |||
| 1753 | char engs_info[2 * OTX2_CPT_NAME_LENGTH]; | 1753 | char engs_info[2 * OTX2_CPT_NAME_LENGTH]; |
| 1754 | struct otx2_cpt_eng_grp_info *grp; | 1754 | struct otx2_cpt_eng_grp_info *grp; |
| 1755 | struct otx2_cpt_engs_rsvd *engs; | 1755 | struct otx2_cpt_engs_rsvd *engs; |
| 1756 | u32 mask[4]; | ||
| 1757 | int i, j; | 1756 | int i, j; |
| 1758 | 1757 | ||
| 1759 | pr_debug("Engine groups global info"); | 1758 | pr_debug("Engine groups global info"); |
| @@ -1785,6 +1784,8 @@ void otx2_cpt_print_uc_dbg_info(struct otx2_cptpf_dev *cptpf) · 25 unmodified lines | |||
| 1785 | for (j = 0; j < OTX2_CPT_MAX_ETYPES_PER_GRP; j++) { | 1784 | for (j = 0; j < OTX2_CPT_MAX_ETYPES_PER_GRP; j++) { |
| 1786 | engs = &grp->engs[j]; | 1785 | engs = &grp->engs[j]; |
| 1787 | if (engs->type) { | 1786 | if (engs->type) { |
| 1787 | u32 mask[5] = { }; | ||
| 1788 | |||
| 1788 | get_engs_info(grp, engs_info, | 1789 | get_engs_info(grp, engs_info, |
| 1789 | 2 * OTX2_CPT_NAME_LENGTH, j); | 1790 | 2 * OTX2_CPT_NAME_LENGTH, j); |
| 1790 | pr_debug("Slot%d: %s", j, engs_info); | 1791 | pr_debug("Slot%d: %s", j, engs_info); |
| @@ -176,6 +176,20 @@ struct dma_fence_array *dma_fence_array_create(int num_fences, · 175 unmodified lines | |||
| 176 | 176 | ||
| 177 | array->base.error = PENDING_ERROR; | 177 | array->base.error = PENDING_ERROR; |
| 178 | 178 | ||
| 179 | /* | ||
| 180 | * dma_fence_array objects should never contain any other fence | ||
| 181 | * containers or otherwise we run into recursion and potential kernel | ||
| 182 | * stack overflow on operations on the dma_fence_array. | ||
| 183 | * | ||
| 184 | * The correct way of handling this is to flatten out the array by the | ||
| 185 | * caller instead. | ||
| 186 | * | ||
| 187 | * Enforce this here by checking that we don't create a dma_fence_array | ||
| 188 | * with any container inside. | ||
| 189 | */ | ||
| 190 | while (num_fences--) | ||
| 191 | WARN_ON(dma_fence_is_container(fences[num_fences])); | ||
| 192 | |||
| 179 | return array; | 193 | return array; |
| 180 | } | 194 | } |
| 181 | EXPORT_SYMBOL(dma_fence_array_create); | 195 | EXPORT_SYMBOL(dma_fence_array_create); |
| @@ -148,8 +148,7 @@ static bool dma_fence_chain_enable_signaling(struct dma_fence *fence) · 147 unmodified lines | |||
| 148 | 148 | ||
| 149 | dma_fence_get(&head->base); | 149 | dma_fence_get(&head->base); |
| 150 | dma_fence_chain_for_each(fence, &head->base) { | 150 | dma_fence_chain_for_each(fence, &head->base) { |
| 151 | struct dma_fence_chain *chain = to_dma_fence_chain(fence); | 151 | struct dma_fence *f = dma_fence_chain_contained(fence); |
| 152 | struct dma_fence *f = chain ? chain->fence : fence; | ||
| 153 | 152 | ||
| 154 | dma_fence_get(f); | 153 | dma_fence_get(f); |
| 155 | if (!dma_fence_add_callback(f, &head->cb, dma_fence_chain_cb)) { | 154 | if (!dma_fence_add_callback(f, &head->cb, dma_fence_chain_cb)) { |
| @@ -165,8 +164,7 @@ static bool dma_fence_chain_enable_signaling(struct dma_fence *fence) · 9 unmodified lines | |||
| 165 | static bool dma_fence_chain_signaled(struct dma_fence *fence) | 164 | static bool dma_fence_chain_signaled(struct dma_fence *fence) |
| 166 | { | 165 | { |
| 167 | dma_fence_chain_for_each(fence, fence) { | 166 | dma_fence_chain_for_each(fence, fence) { |
| 168 | struct dma_fence_chain *chain = to_dma_fence_chain(fence); | 167 | struct dma_fence *f = dma_fence_chain_contained(fence); |
| 169 | struct dma_fence *f = chain ? chain->fence : fence; | ||
| 170 | 168 | ||
| 171 | if (!dma_fence_is_signaled(f)) { | 169 | if (!dma_fence_is_signaled(f)) { |
| 172 | dma_fence_put(fence); | 170 | dma_fence_put(fence); |
| @@ -254,5 +252,14 @@ void dma_fence_chain_init(struct dma_fence_chain *chain, · 81 unmodified lines | |||
| 254 | 252 | ||
| 255 | dma_fence_init(&chain->base, &dma_fence_chain_ops, | 253 | dma_fence_init(&chain->base, &dma_fence_chain_ops, |
| 256 | &chain->lock, context, seqno); | 254 | &chain->lock, context, seqno); |
| 255 | |||
| 256 | /* | ||
| 257 | * Chaining dma_fence_chain container together is only allowed through | ||
| 258 | * the prev fence and not through the contained fence. | ||
| 259 | * | ||
| 260 | * The correct way of handling this is to flatten out the fence | ||
| 261 | * structure into a dma_fence_array by the caller instead. | ||
| 262 | */ | ||
| 263 | WARN_ON(dma_fence_is_chain(fence)); | ||
| 257 | } | 264 | } |
| 258 | EXPORT_SYMBOL(dma_fence_chain_init); | 265 | EXPORT_SYMBOL(dma_fence_chain_init); |
| @@ -14,6 +14,7 @@ · 13 unmodified lines | |||
| 14 | #include <linux/xarray.h> | 14 | #include <linux/xarray.h> |
| 15 | #include <linux/list.h> | 15 | #include <linux/list.h> |
| 16 | #include <linux/slab.h> | 16 | #include <linux/slab.h> |
| 17 | #include <linux/nospec.h> | ||
| 17 | #include <linux/uaccess.h> | 18 | #include <linux/uaccess.h> |
| 18 | #include <linux/syscalls.h> | 19 | #include <linux/syscalls.h> |
| 19 | #include <linux/dma-heap.h> | 20 | #include <linux/dma-heap.h> |
| @@ -135,6 +136,7 @@ static long dma_heap_ioctl(struct file *file, unsigned int ucmd, · 115 unmodified lines | |||
| 135 | if (nr >= ARRAY_SIZE(dma_heap_ioctl_cmds)) | 136 | if (nr >= ARRAY_SIZE(dma_heap_ioctl_cmds)) |
| 136 | return -EINVAL; | 137 | return -EINVAL; |
| 137 | 138 | ||
| 139 | nr = array_index_nospec(nr, ARRAY_SIZE(dma_heap_ioctl_cmds)); | ||
| 138 | /* Get the kernel ioctl cmd that matches */ | 140 | /* Get the kernel ioctl cmd that matches */ |
| 139 | kcmd = dma_heap_ioctl_cmds[nr]; | 141 | kcmd = dma_heap_ioctl_cmds[nr]; |
| 140 | 142 | ||
| @@ -256,6 +256,11 @@ void dma_resv_add_shared_fence(struct dma_resv *obj, struct dma_fence *fence) · 255 unmodified lines | |||
| 256 | 256 | ||
| 257 | dma_resv_assert_held(obj); | 257 | dma_resv_assert_held(obj); |
| 258 | 258 | ||
| 259 | /* Drivers should not add containers here, instead add each fence | ||
| 260 | * individually. | ||
| 261 | */ | ||
| 262 | WARN_ON(dma_fence_is_container(fence)); | ||
| 263 | |||
| 259 | fobj = dma_resv_shared_list(obj); | 264 | fobj = dma_resv_shared_list(obj); |
| 260 | count = fobj->shared_count; | 265 | count = fobj->shared_count; |
| 261 | 266 | ||
| @@ -323,12 +328,8 @@ void dma_resv_add_excl_fence(struct dma_resv *obj, struct dma_fence *fence) · 61 unmodified lines | |||
| 323 | } | 328 | } |
| 324 | EXPORT_SYMBOL(dma_resv_add_excl_fence); | 329 | EXPORT_SYMBOL(dma_resv_add_excl_fence); |
| 325 | 330 | ||
| 326 | /** | 331 | /* Restart the iterator by initializing all the necessary fields, but not the |
| 327 | * dma_resv_iter_restart_unlocked - restart the unlocked iterator | 332 | * relation to the dma_resv object. */ |
| 328 | * @cursor: The dma_resv_iter object to restart | ||
| 329 | * | ||
| 330 | * Restart the unlocked iteration by initializing the cursor object. | ||
| 331 | */ | ||
| 332 | static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor) | 333 | static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor) |
| 333 | { | 334 | { |
| 334 | cursor->seq = read_seqcount_begin(&cursor->obj->seq); | 335 | cursor->seq = read_seqcount_begin(&cursor->obj->seq); |
| @@ -344,14 +345,7 @@ static void dma_resv_iter_restart_unlocked(struct dma_resv_iter *cursor) · 9 unmodified lines | |||
| 344 | cursor->is_restarted = true; | 345 | cursor->is_restarted = true; |
| 345 | } | 346 | } |
| 346 | 347 | ||
| 347 | /** | 348 | /* Walk to the next not signaled fence and grab a reference to it */ |
| 348 | * dma_resv_iter_walk_unlocked - walk over fences in a dma_resv obj | ||
| 349 | * @cursor: cursor to record the current position | ||
| 350 | * | ||
| 351 | * Return all the fences in the dma_resv object which are not yet signaled. | ||
| 352 | * The returned fence has an extra local reference so will stay alive. | ||
| 353 | * If a concurrent modify is detected the whole iteration is started over again. | ||
| 354 | */ | ||
| 355 | static void dma_resv_iter_walk_unlocked(struct dma_resv_iter *cursor) | 349 | static void dma_resv_iter_walk_unlocked(struct dma_resv_iter *cursor) |
| 356 | { | 350 | { |
| 357 | struct dma_resv *obj = cursor->obj; | 351 | struct dma_resv *obj = cursor->obj; |
| @@ -387,6 +381,12 @@ static void dma_resv_iter_walk_unlocked(struct dma_resv_iter *cursor) · 29 unmodified lines | |||
| 387 | * dma_resv_iter_first_unlocked - first fence in an unlocked dma_resv obj. | 381 | * dma_resv_iter_first_unlocked - first fence in an unlocked dma_resv obj. |
| 388 | * @cursor: the cursor with the current position | 382 | * @cursor: the cursor with the current position |
| 389 | * | 383 | * |
| 384 | * Subsequent fences are iterated with dma_resv_iter_next_unlocked(). | ||
| 385 | * | ||
| 386 | * Beware that the iterator can be restarted. Code which accumulates statistics | ||
| 387 | * or similar needs to check for this with dma_resv_iter_is_restarted(). For | ||
| 388 | * this reason prefer the locked dma_resv_iter_first() whenver possible. | ||
| 389 | * | ||
| 390 | * Returns the first fence from an unlocked dma_resv obj. | 390 | * Returns the first fence from an unlocked dma_resv obj. |
| 391 | */ | 391 | */ |
| 392 | struct dma_fence *dma_resv_iter_first_unlocked(struct dma_resv_iter *cursor) | 392 | struct dma_fence *dma_resv_iter_first_unlocked(struct dma_resv_iter *cursor) |
| @@ -406,6 +406,10 @@ EXPORT_SYMBOL(dma_resv_iter_first_unlocked); · 13 unmodified lines | |||
| 406 | * dma_resv_iter_next_unlocked - next fence in an unlocked dma_resv obj. | 406 | * dma_resv_iter_next_unlocked - next fence in an unlocked dma_resv obj. |
| 407 | * @cursor: the cursor with the current position | 407 | * @cursor: the cursor with the current position |
| 408 | * | 408 | * |
| 409 | * Beware that the iterator can be restarted. Code which accumulates statistics | ||
| 410 | * or similar needs to check for this with dma_resv_iter_is_restarted(). For | ||
| 411 | * this reason prefer the locked dma_resv_iter_next() whenver possible. | ||
| 412 | * | ||
| 409 | * Returns the next fence from an unlocked dma_resv obj. | 413 | * Returns the next fence from an unlocked dma_resv obj. |
| 410 | */ | 414 | */ |
| 411 | struct dma_fence *dma_resv_iter_next_unlocked(struct dma_resv_iter *cursor) | 415 | struct dma_fence *dma_resv_iter_next_unlocked(struct dma_resv_iter *cursor) |
| @@ -431,6 +435,8 @@ EXPORT_SYMBOL(dma_resv_iter_next_unlocked); · 19 unmodified lines | |||
| 431 | * dma_resv_iter_first - first fence from a locked dma_resv object | 435 | * dma_resv_iter_first - first fence from a locked dma_resv object |
| 432 | * @cursor: cursor to record the current position | 436 | * @cursor: cursor to record the current position |
| 433 | * | 437 | * |
| 438 | * Subsequent fences are iterated with dma_resv_iter_next_unlocked(). | ||
| 439 | * | ||
| 434 | * Return the first fence in the dma_resv object while holding the | 440 | * Return the first fence in the dma_resv object while holding the |
| 435 | * &dma_resv.lock. | 441 | * &dma_resv.lock. |
| 436 | */ | 442 | */ |
| @@ -1681,8 +1681,10 @@ static void at_xdmac_tasklet(struct tasklet_struct *t) · 1680 unmodified lines | |||
| 1681 | __func__, atchan->irq_status); | 1681 | __func__, atchan->irq_status); |
| 1682 | 1682 | ||
| 1683 | if (!(atchan->irq_status & AT_XDMAC_CIS_LIS) && | 1683 | if (!(atchan->irq_status & AT_XDMAC_CIS_LIS) && |
| 1684 | !(atchan->irq_status & error_mask)) | 1684 | !(atchan->irq_status & error_mask)) { |
| 1685 | spin_unlock_irq(&atchan->lock); | ||
| 1685 | return; | 1686 | return; |
| 1687 | } | ||
| 1686 | 1688 | ||
| 1687 | if (atchan->irq_status & error_mask) | 1689 | if (atchan->irq_status & error_mask) |
| 1688 | at_xdmac_handle_error(atchan); | 1690 | at_xdmac_handle_error(atchan); |
| @@ -207,7 +207,7 @@ int pt_core_init(struct pt_device *pt) · 206 unmodified lines | |||
| 207 | if (!cmd_q->qbase) { | 207 | if (!cmd_q->qbase) { |
| 208 | dev_err(dev, "unable to allocate command queue\n"); | 208 | dev_err(dev, "unable to allocate command queue\n"); |
| 209 | ret = -ENOMEM; | 209 | ret = -ENOMEM; |
| 210 | goto e_dma_alloc; | 210 | goto e_destroy_pool; |
| 211 | } | 211 | } |
| 212 | 212 | ||
| 213 | cmd_q->qidx = 0; | 213 | cmd_q->qidx = 0; |
| @@ -229,8 +229,10 @@ int pt_core_init(struct pt_device *pt) · 15 unmodified lines | |||
| 229 | 229 | ||
| 230 | /* Request an irq */ | 230 | /* Request an irq */ |
| 231 | ret = request_irq(pt->pt_irq, pt_core_irq_handler, 0, dev_name(pt->dev), pt); | 231 | ret = request_irq(pt->pt_irq, pt_core_irq_handler, 0, dev_name(pt->dev), pt); |
| 232 | if (ret) | 232 | if (ret) { |
| 233 | goto e_pool; | 233 | dev_err(dev, "unable to allocate an IRQ\n"); |
| 234 | goto e_free_dma; | ||
| 235 | } | ||
| 234 | 236 | ||
| 235 | /* Update the device registers with queue information. */ | 237 | /* Update the device registers with queue information. */ |
| 236 | cmd_q->qcontrol &= ~CMD_Q_SIZE; | 238 | cmd_q->qcontrol &= ~CMD_Q_SIZE; |
| @@ -250,21 +252,20 @@ int pt_core_init(struct pt_device *pt) · 13 unmodified lines | |||
| 250 | /* Register the DMA engine support */ | 252 | /* Register the DMA engine support */ |
| 251 | ret = pt_dmaengine_register(pt); | 253 | ret = pt_dmaengine_register(pt); |
| 252 | if (ret) | 254 | if (ret) |
| 253 | goto e_dmaengine; | 255 | goto e_free_irq; |
| 254 | 256 | ||
| 255 | /* Set up debugfs entries */ | 257 | /* Set up debugfs entries */ |
| 256 | ptdma_debugfs_setup(pt); | 258 | ptdma_debugfs_setup(pt); |
| 257 | 259 | ||
| 258 | return 0; | 260 | return 0; |
| 259 | 261 | ||
| 260 | e_dmaengine: | 262 | e_free_irq: |
| 261 | free_irq(pt->pt_irq, pt); | 263 | free_irq(pt->pt_irq, pt); |
| 262 | 264 | ||
| 263 | e_dma_alloc: | 265 | e_free_dma: |
| 264 | dma_free_coherent(dev, cmd_q->qsize, cmd_q->qbase, cmd_q->qbase_dma); | 266 | dma_free_coherent(dev, cmd_q->qsize, cmd_q->qbase, cmd_q->qbase_dma); |
| 265 | 267 | ||
| 266 | e_pool: | 268 | e_destroy_pool: |
| 267 | dev_err(dev, "unable to allocate an IRQ\n"); | ||
| 268 | dma_pool_destroy(pt->cmd_q.dma_pool); | 269 | dma_pool_destroy(pt->cmd_q.dma_pool); |
| 269 | 270 | ||
| 270 | return ret; | 271 | return ret; |
| @@ -1868,8 +1868,13 @@ static int rcar_dmac_probe(struct platform_device *pdev) · 1867 unmodified lines | |||
| 1868 | 1868 | ||
| 1869 | dmac->dev = &pdev->dev; | 1869 | dmac->dev = &pdev->dev; |
| 1870 | platform_set_drvdata(pdev, dmac); | 1870 | platform_set_drvdata(pdev, dmac); |
| 1871 | dma_set_max_seg_size(dmac->dev, RCAR_DMATCR_MASK); | 1871 | ret = dma_set_max_seg_size(dmac->dev, RCAR_DMATCR_MASK); |
| 1872 | dma_set_mask_and_coherent(dmac->dev, DMA_BIT_MASK(40)); | 1872 | if (ret) |
| 1873 | return ret; | ||
| 1874 | |||
| 1875 | ret = dma_set_mask_and_coherent(dmac->dev, DMA_BIT_MASK(40)); | ||
| 1876 | if (ret) | ||
| 1877 | return ret; | ||
| 1873 | 1878 | ||
| 1874 | ret = rcar_dmac_parse_of(&pdev->dev, dmac); | 1879 | ret = rcar_dmac_parse_of(&pdev->dev, dmac); |
| 1875 | if (ret < 0) | 1880 | if (ret < 0) |
| @@ -115,8 +115,10 @@ static dma_cookie_t shdma_tx_submit(struct dma_async_tx_descriptor *tx) · 114 unmodified lines | |||
| 115 | ret = pm_runtime_get(schan->dev); | 115 | ret = pm_runtime_get(schan->dev); |
| 116 | 116 | ||
| 117 | spin_unlock_irq(&schan->chan_lock); | 117 | spin_unlock_irq(&schan->chan_lock); |
| 118 | if (ret < 0) | 118 | if (ret < 0) { |
| 119 | dev_err(schan->dev, "%s(): GET = %d\n", __func__, ret); | 119 | dev_err(schan->dev, "%s(): GET = %d\n", __func__, ret); |
| 120 | pm_runtime_put(schan->dev); | ||
| 121 | } | ||
| 120 | 122 | ||
| 121 | pm_runtime_barrier(schan->dev); | 123 | pm_runtime_barrier(schan->dev); |
| 122 | 124 | ||
| @@ -292,10 +292,12 @@ static int stm32_dmamux_probe(struct platform_device *pdev) · 291 unmodified lines | |||
| 292 | ret = of_dma_router_register(node, stm32_dmamux_route_allocate, | 292 | ret = of_dma_router_register(node, stm32_dmamux_route_allocate, |
| 293 | &stm32_dmamux->dmarouter); | 293 | &stm32_dmamux->dmarouter); |
| 294 | if (ret) | 294 | if (ret) |
| 295 | goto err_clk; | 295 | goto pm_disable; |
| 296 | 296 | ||
| 297 | return 0; | 297 | return 0; |
| 298 | 298 | ||
| 299 | pm_disable: | ||
| 300 | pm_runtime_disable(&pdev->dev); | ||
| 299 | err_clk: | 301 | err_clk: |
| 300 | clk_disable_unprepare(stm32_dmamux->clk); | 302 | clk_disable_unprepare(stm32_dmamux->clk); |
| 301 | 303 | ||
| @@ -350,7 +350,7 @@ static int altr_sdram_probe(struct platform_device *pdev) · 349 unmodified lines | |||
| 350 | if (irq < 0) { | 350 | if (irq < 0) { |
| 351 | edac_printk(KERN_ERR, EDAC_MC, | 351 | edac_printk(KERN_ERR, EDAC_MC, |
| 352 | "No irq %d in DT\n", irq); | 352 | "No irq %d in DT\n", irq); |
| 353 | return -ENODEV; | 353 | return irq; |
| 354 | } | 354 | } |
| 355 | 355 | ||
| 356 | /* Arria10 has a 2nd IRQ */ | 356 | /* Arria10 has a 2nd IRQ */ |
| @@ -215,7 +215,7 @@ void *edac_align_ptr(void **p, unsigned int size, int n_elems) · 214 unmodified lines | |||
| 215 | else | 215 | else |
| 216 | return (char *)ptr; | 216 | return (char *)ptr; |
| 217 | 217 | ||
| 218 | r = (unsigned long)p % align; | 218 | r = (unsigned long)ptr % align; |
| 219 | 219 | ||
| 220 | if (r == 0) | 220 | if (r == 0) |
| 221 | return (char *)ptr; | 221 | return (char *)ptr; |
| @@ -1919,7 +1919,7 @@ static int xgene_edac_probe(struct platform_device *pdev) · 1918 unmodified lines | |||
| 1919 | irq = platform_get_irq_optional(pdev, i); | 1919 | irq = platform_get_irq_optional(pdev, i); |
| 1920 | if (irq < 0) { | 1920 | if (irq < 0) { |
| 1921 | dev_err(&pdev->dev, "No IRQ resource\n"); | 1921 | dev_err(&pdev->dev, "No IRQ resource\n"); |
| 1922 | rc = -EINVAL; | 1922 | rc = irq; |
| 1923 | goto out_err; | 1923 | goto out_err; |
| 1924 | } | 1924 | } |
| 1925 | rc = devm_request_irq(&pdev->dev, irq, | 1925 | rc = devm_request_irq(&pdev->dev, irq, |
| @@ -278,7 +278,8 @@ static int gpio_fwd_get(struct gpio_chip *chip, unsigned int offset) · 277 unmodified lines | |||
| 278 | { | 278 | { |
| 279 | struct gpiochip_fwd *fwd = gpiochip_get_data(chip); | 279 | struct gpiochip_fwd *fwd = gpiochip_get_data(chip); |
| 280 | 280 | ||
| 281 | return gpiod_get_value(fwd->descs[offset]); | 281 | return chip->can_sleep ? gpiod_get_value_cansleep(fwd->descs[offset]) |
| 282 | : gpiod_get_value(fwd->descs[offset]); | ||
| 282 | } | 283 | } |
| 283 | 284 | ||
| 284 | static int gpio_fwd_get_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, | 285 | static int gpio_fwd_get_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, |
| @@ -293,7 +294,10 @@ static int gpio_fwd_get_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, · 8 unmodified lines | |||
| 293 | for_each_set_bit(i, mask, fwd->chip.ngpio) | 294 | for_each_set_bit(i, mask, fwd->chip.ngpio) |
| 294 | descs[j++] = fwd->descs[i]; | 295 | descs[j++] = fwd->descs[i]; |
| 295 | 296 | ||
| 296 | error = gpiod_get_array_value(j, descs, NULL, values); | 297 | if (fwd->chip.can_sleep) |
| 298 | error = gpiod_get_array_value_cansleep(j, descs, NULL, values); | ||
| 299 | else | ||
| 300 | error = gpiod_get_array_value(j, descs, NULL, values); | ||
| 297 | if (error) | 301 | if (error) |
| 298 | return error; | 302 | return error; |
| 299 | 303 | ||
| @@ -328,7 +332,10 @@ static void gpio_fwd_set(struct gpio_chip *chip, unsigned int offset, int value) · 28 unmodified lines | |||
| 328 | { | 332 | { |
| 329 | struct gpiochip_fwd *fwd = gpiochip_get_data(chip); | 333 | struct gpiochip_fwd *fwd = gpiochip_get_data(chip); |
| 330 | 334 | ||
| 331 | gpiod_set_value(fwd->descs[offset], value); | 335 | if (chip->can_sleep) |
| 336 | gpiod_set_value_cansleep(fwd->descs[offset], value); | ||
| 337 | else | ||
| 338 | gpiod_set_value(fwd->descs[offset], value); | ||
| 332 | } | 339 | } |
| 333 | 340 | ||
| 334 | static void gpio_fwd_set_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, | 341 | static void gpio_fwd_set_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, |
| @@ -343,7 +350,10 @@ static void gpio_fwd_set_multiple(struct gpiochip_fwd *fwd, unsigned long *mask, · 8 unmodified lines | |||
| 343 | descs[j++] = fwd->descs[i]; | 350 | descs[j++] = fwd->descs[i]; |
| 344 | } | 351 | } |
| 345 | 352 | ||
| 346 | gpiod_set_array_value(j, descs, NULL, values); | 353 | if (fwd->chip.can_sleep) |
| 354 | gpiod_set_array_value_cansleep(j, descs, NULL, values); | ||
| 355 | else | ||
| 356 | gpiod_set_array_value(j, descs, NULL, values); | ||
| 347 | } | 357 | } |
| 348 | 358 | ||
| 349 | static void gpio_fwd_set_multiple_locked(struct gpio_chip *chip, | 359 | static void gpio_fwd_set_multiple_locked(struct gpio_chip *chip, |
| @@ -410,10 +410,8 @@ static int rockchip_irq_set_type(struct irq_data *d, unsigned int type) · 409 unmodified lines | |||
| 410 | level = rockchip_gpio_readl(bank, bank->gpio_regs->int_type); | 410 | level = rockchip_gpio_readl(bank, bank->gpio_regs->int_type); |
| 411 | polarity = rockchip_gpio_readl(bank, bank->gpio_regs->int_polarity); | 411 | polarity = rockchip_gpio_readl(bank, bank->gpio_regs->int_polarity); |
| 412 | 412 | ||
| 413 | switch (type) { | 413 | if (type == IRQ_TYPE_EDGE_BOTH) { |
| 414 | case IRQ_TYPE_EDGE_BOTH: | ||
| 415 | if (bank->gpio_type == GPIO_TYPE_V2) { | 414 | if (bank->gpio_type == GPIO_TYPE_V2) { |
| 416 | bank->toggle_edge_mode &= ~mask; | ||
| 417 | rockchip_gpio_writel_bit(bank, d->hwirq, 1, | 415 | rockchip_gpio_writel_bit(bank, d->hwirq, 1, |
| 418 | bank->gpio_regs->int_bothedge); | 416 | bank->gpio_regs->int_bothedge); |
| 419 | goto out; | 417 | goto out; |
| @@ -431,30 +429,34 @@ static int rockchip_irq_set_type(struct irq_data *d, unsigned int type) · 11 unmodified lines | |||
| 431 | else | 429 | else |
| 432 | polarity |= mask; | 430 | polarity |= mask; |
| 433 | } | 431 | } |
| 434 | break; | 432 | } else { |
| 435 | case IRQ_TYPE_EDGE_RISING: | 433 | if (bank->gpio_type == GPIO_TYPE_V2) { |
| 436 | bank->toggle_edge_mode &= ~mask; | 434 | rockchip_gpio_writel_bit(bank, d->hwirq, 0, |
| 437 | level |= mask; | 435 | bank->gpio_regs->int_bothedge); |
| 438 | polarity |= mask; | 436 | } else { |
| 439 | break; | 437 | bank->toggle_edge_mode &= ~mask; |
| 440 | case IRQ_TYPE_EDGE_FALLING: | 438 | } |
| 441 | bank->toggle_edge_mode &= ~mask; | 439 | switch (type) { |
| 442 | level |= mask; | 440 | case IRQ_TYPE_EDGE_RISING: |
| 443 | polarity &= ~mask; | 441 | level |= mask; |
| 444 | break; | 442 | polarity |= mask; |
| 445 | case IRQ_TYPE_LEVEL_HIGH: | 443 | break; |
| 446 | bank->toggle_edge_mode &= ~mask; | 444 | case IRQ_TYPE_EDGE_FALLING: |
| 447 | level &= ~mask; | 445 | level |= mask; |
| 448 | polarity |= mask; | 446 | polarity &= ~mask; |
| 449 | break; | 447 | break; |
| 450 | case IRQ_TYPE_LEVEL_LOW: | 448 | case IRQ_TYPE_LEVEL_HIGH: |
| 451 | bank->toggle_edge_mode &= ~mask; | 449 | level &= ~mask; |
| 452 | level &= ~mask; | 450 | polarity |= mask; |
| 453 | polarity &= ~mask; | 451 | break; |
| 454 | break; | 452 | case IRQ_TYPE_LEVEL_LOW: |
| 455 | default: | 453 | level &= ~mask; |
| 456 | ret = -EINVAL; | 454 | polarity &= ~mask; |
| 457 | goto out; | 455 | break; |
| 456 | default: | ||
| 457 | ret = -EINVAL; | ||
| 458 | goto out; | ||
| 459 | } | ||
| 458 | } | 460 | } |
| 459 | 461 | ||
| 460 | rockchip_gpio_writel(bank, level, bank->gpio_regs->int_type); | 462 | rockchip_gpio_writel(bank, level, bank->gpio_regs->int_type); |
| @@ -223,7 +223,7 @@ static int sifive_gpio_probe(struct platform_device *pdev) · 222 unmodified lines | |||
| 223 | NULL, | 223 | NULL, |
| 224 | chip->base + SIFIVE_GPIO_OUTPUT_EN, | 224 | chip->base + SIFIVE_GPIO_OUTPUT_EN, |
| 225 | chip->base + SIFIVE_GPIO_INPUT_EN, | 225 | chip->base + SIFIVE_GPIO_INPUT_EN, |
| 226 | 0); | 226 | BGPIOF_READ_OUTPUT_REG_SET); |
| 227 | if (ret) { | 227 | if (ret) { |
| 228 | dev_err(dev, "unable to init generic GPIO\n"); | 228 | dev_err(dev, "unable to init generic GPIO\n"); |
| 229 | return ret; | 229 | return ret; |
| @@ -570,6 +570,11 @@ static struct gpio_sim_bank *to_gpio_sim_bank(struct config_item *item) · 569 unmodified lines | |||
| 570 | return container_of(group, struct gpio_sim_bank, group); | 570 | return container_of(group, struct gpio_sim_bank, group); |
| 571 | } | 571 | } |
| 572 | 572 | ||
| 573 | static bool gpio_sim_bank_has_label(struct gpio_sim_bank *bank) | ||
| 574 | { | ||
| 575 | return bank->label && *bank->label; | ||
| 576 | } | ||
| 577 | |||
| 573 | static struct gpio_sim_device * | 578 | static struct gpio_sim_device * |
| 574 | gpio_sim_bank_get_device(struct gpio_sim_bank *bank) | 579 | gpio_sim_bank_get_device(struct gpio_sim_bank *bank) |
| 575 | { | 580 | { |
| @@ -770,9 +775,15 @@ static int gpio_sim_add_hogs(struct gpio_sim_device *dev) · 194 unmodified lines | |||
| 770 | * point the device doesn't exist yet and so dev_name() | 775 | * point the device doesn't exist yet and so dev_name() |
| 771 | * is not available. | 776 | * is not available. |
| 772 | */ | 777 | */ |
| 773 | hog->chip_label = kasprintf(GFP_KERNEL, | 778 | if (gpio_sim_bank_has_label(bank)) |
| 774 | "gpio-sim.%u-%s", dev->id, | 779 | hog->chip_label = kstrdup(bank->label, |
| 775 | fwnode_get_name(bank->swnode)); | 780 | GFP_KERNEL); |
| 781 | else | ||
| 782 | hog->chip_label = kasprintf(GFP_KERNEL, | ||
| 783 | "gpio-sim.%u-%s", | ||
| 784 | dev->id, | ||
| 785 | fwnode_get_name( | ||
| 786 | bank->swnode)); | ||
| 776 | if (!hog->chip_label) { | 787 | if (!hog->chip_label) { |
| 777 | gpio_sim_remove_hogs(dev); | 788 | gpio_sim_remove_hogs(dev); |
| 778 | return -ENOMEM; | 789 | return -ENOMEM; |
| @@ -816,7 +827,7 @@ gpio_sim_make_bank_swnode(struct gpio_sim_bank *bank, · 37 unmodified lines | |||
| 816 | 827 | ||
| 817 | properties[prop_idx++] = PROPERTY_ENTRY_U32("ngpios", bank->num_lines); | 828 | properties[prop_idx++] = PROPERTY_ENTRY_U32("ngpios", bank->num_lines); |
| 818 | 829 | ||
| 819 | if (bank->label && (strlen(bank->label) > 0)) | 830 | if (gpio_sim_bank_has_label(bank)) |
| 820 | properties[prop_idx++] = PROPERTY_ENTRY_STRING("gpio-sim,label", | 831 | properties[prop_idx++] = PROPERTY_ENTRY_STRING("gpio-sim,label", |
| 821 | bank->label); | 832 | bank->label); |
| 822 | 833 | ||
| @@ -343,9 +343,12 @@ static int tegra186_gpio_of_xlate(struct gpio_chip *chip, · 342 unmodified lines | |||
| 343 | return offset + pin; | 343 | return offset + pin; |
| 344 | } | 344 | } |
| 345 | 345 | ||
| 346 | #define to_tegra_gpio(x) container_of((x), struct tegra_gpio, gpio) | ||
| 347 | |||
| 346 | static void tegra186_irq_ack(struct irq_data *data) | 348 | static void tegra186_irq_ack(struct irq_data *data) |
| 347 | { | 349 | { |
| 348 | struct tegra_gpio *gpio = irq_data_get_irq_chip_data(data); | 350 | struct gpio_chip *gc = irq_data_get_irq_chip_data(data); |
| 351 | struct tegra_gpio *gpio = to_tegra_gpio(gc); | ||
| 349 | void __iomem *base; | 352 | void __iomem *base; |
| 350 | 353 | ||
| 351 | base = tegra186_gpio_get_base(gpio, data->hwirq); | 354 | base = tegra186_gpio_get_base(gpio, data->hwirq); |
| @@ -357,7 +360,8 @@ static void tegra186_irq_ack(struct irq_data *data) · 5 unmodified lines | |||
| 357 | 360 | ||
| 358 | static void tegra186_irq_mask(struct irq_data *data) | 361 | static void tegra186_irq_mask(struct irq_data *data) |
| 359 | { | 362 | { |
| 360 | struct tegra_gpio *gpio = irq_data_get_irq_chip_data(data); | 363 | struct gpio_chip *gc = irq_data_get_irq_chip_data(data); |
| 364 | struct tegra_gpio *gpio = to_tegra_gpio(gc); | ||
| 361 | void __iomem *base; | 365 | void __iomem *base; |
| 362 | u32 value; | 366 | u32 value; |
| 363 | 367 | ||
| @@ -372,7 +376,8 @@ static void tegra186_irq_mask(struct irq_data *data) · 8 unmodified lines | |||
| 372 | 376 | ||
| 373 | static void tegra186_irq_unmask(struct irq_data *data) | 377 | static void tegra186_irq_unmask(struct irq_data *data) |
| 374 | { | 378 | { |
| 375 | struct tegra_gpio *gpio = irq_data_get_irq_chip_data(data); | 379 | struct gpio_chip *gc = irq_data_get_irq_chip_data(data); |
| 380 | struct tegra_gpio *gpio = to_tegra_gpio(gc); | ||
| 376 | void __iomem *base; | 381 | void __iomem *base; |
| 377 | u32 value; | 382 | u32 value; |
| 378 | 383 | ||
| @@ -387,7 +392,8 @@ static void tegra186_irq_unmask(struct irq_data *data) · 8 unmodified lines | |||
| 387 | 392 | ||
| 388 | static int tegra186_irq_set_type(struct irq_data *data, unsigned int type) | 393 | static int tegra186_irq_set_type(struct irq_data *data, unsigned int type) |
| 389 | { | 394 | { |
| 390 | struct tegra_gpio *gpio = irq_data_get_irq_chip_data(data); | 395 | struct gpio_chip *gc = irq_data_get_irq_chip_data(data); |
| 396 | struct tegra_gpio *gpio = to_tegra_gpio(gc); | ||
| 391 | void __iomem *base; | 397 | void __iomem *base; |
| 392 | u32 value; | 398 | u32 value; |
| 393 | 399 | ||
| @@ -330,7 +330,7 @@ static int linehandle_create(struct gpio_device *gdev, void __user *ip) · 329 unmodified lines | |||
| 330 | goto out_free_lh; | 330 | goto out_free_lh; |
| 331 | } | 331 | } |
| 332 | 332 | ||
| 333 | ret = gpiod_request(desc, lh->label); | 333 | ret = gpiod_request_user(desc, lh->label); |
| 334 | if (ret) | 334 | if (ret) |
| 335 | goto out_free_lh; | 335 | goto out_free_lh; |
| 336 | lh->descs[i] = desc; | 336 | lh->descs[i] = desc; |
| @@ -1378,7 +1378,7 @@ static int linereq_create(struct gpio_device *gdev, void __user *ip) · 1041 unmodified lines | |||
| 1378 | goto out_free_linereq; | 1378 | goto out_free_linereq; |
| 1379 | } | 1379 | } |
| 1380 | 1380 | ||
| 1381 | ret = gpiod_request(desc, lr->label); | 1381 | ret = gpiod_request_user(desc, lr->label); |
| 1382 | if (ret) | 1382 | if (ret) |
| 1383 | goto out_free_linereq; | 1383 | goto out_free_linereq; |
| 1384 | 1384 | ||
| @@ -1764,7 +1764,7 @@ static int lineevent_create(struct gpio_device *gdev, void __user *ip) · 379 unmodified lines | |||
| 1764 | } | 1764 | } |
| 1765 | } | 1765 | } |
| 1766 | 1766 | ||
| 1767 | ret = gpiod_request(desc, le->label); | 1767 | ret = gpiod_request_user(desc, le->label); |
| 1768 | if (ret) | 1768 | if (ret) |
| 1769 | goto out_free_le; | 1769 | goto out_free_le; |
| 1770 | le->desc = desc; | 1770 | le->desc = desc; |
| @@ -475,12 +475,9 @@ static ssize_t export_store(struct class *class, · 474 unmodified lines | |||
| 475 | * they may be undone on its behalf too. | 475 | * they may be undone on its behalf too. |
| 476 | */ | 476 | */ |
| 477 | 477 | ||
| 478 | status = gpiod_request(desc, "sysfs"); | 478 | status = gpiod_request_user(desc, "sysfs"); |
| 479 | if (status) { | 479 | if (status) |
| 480 | if (status == -EPROBE_DEFER) | ||
| 481 | status = -ENODEV; | ||
| 482 | goto done; | 480 | goto done; |
| 483 | } | ||
| 484 | 481 | ||
| 485 | status = gpiod_set_transitory(desc, false); | 482 | status = gpiod_set_transitory(desc, false); |
| 486 | if (!status) { | 483 | if (!status) { |
| @@ -3147,6 +3147,16 @@ int gpiod_to_irq(const struct gpio_desc *desc) · 3146 unmodified lines | |||
| 3147 | 3147 | ||
| 3148 | return retirq; | 3148 | return retirq; |
| 3149 | } | 3149 | } |
| 3150 | #ifdef CONFIG_GPIOLIB_IRQCHIP | ||
| 3151 | if (gc->irq.chip) { | ||
| 3152 | /* | ||
| 3153 | * Avoid race condition with other code, which tries to lookup | ||
| 3154 | * an IRQ before the irqchip has been properly registered, | ||
| 3155 | * i.e. while gpiochip is still being brought up. | ||
| 3156 | */ | ||
| 3157 | return -EPROBE_DEFER; | ||
| 3158 | } | ||
| 3159 | #endif | ||
| 3150 | return -ENXIO; | 3160 | return -ENXIO; |
| 3151 | } | 3161 | } |
| 3152 | EXPORT_SYMBOL_GPL(gpiod_to_irq); | 3162 | EXPORT_SYMBOL_GPL(gpiod_to_irq); |
| @@ -135,6 +135,18 @@ struct gpio_desc { · 134 unmodified lines | |||
| 135 | 135 | ||
| 136 | int gpiod_request(struct gpio_desc *desc, const char *label); | 136 | int gpiod_request(struct gpio_desc *desc, const char *label); |
| 137 | void gpiod_free(struct gpio_desc *desc); | 137 | void gpiod_free(struct gpio_desc *desc); |
| 138 | |||
| 139 | static inline int gpiod_request_user(struct gpio_desc *desc, const char *label) | ||
| 140 | { | ||
| 141 | int ret; | ||
| 142 | |||
| 143 | ret = gpiod_request(desc, label); | ||
| 144 | if (ret == -EPROBE_DEFER) | ||
| 145 | ret = -ENODEV; | ||
| 146 | |||
| 147 | return ret; | ||
| 148 | } | ||
| 149 | |||
| 138 | int gpiod_configure_flags(struct gpio_desc *desc, const char *con_id, | 150 | int gpiod_configure_flags(struct gpio_desc *desc, const char *con_id, |
| 139 | unsigned long lflags, enum gpiod_flags dflags); | 151 | unsigned long lflags, enum gpiod_flags dflags); |
| 140 | int gpio_set_debounce_timeout(struct gpio_desc *desc, unsigned int debounce); | 152 | int gpio_set_debounce_timeout(struct gpio_desc *desc, unsigned int debounce); |
| @@ -71,6 +71,7 @@ config DRM_DEBUG_SELFTEST · 70 unmodified lines | |||
| 71 | select DRM_DP_HELPER | 71 | select DRM_DP_HELPER |
| 72 | select DRM_LIB_RANDOM | 72 | select DRM_LIB_RANDOM |
| 73 | select DRM_KMS_HELPER | 73 | select DRM_KMS_HELPER |
| 74 | select DRM_BUDDY | ||
| 74 | select DRM_EXPORT_FOR_TESTS if m | 75 | select DRM_EXPORT_FOR_TESTS if m |
| 75 | default n | 76 | default n |
| 76 | help | 77 | help |
| @@ -403,6 +404,8 @@ source "drivers/gpu/drm/xlnx/Kconfig" · 326 unmodified lines | |||
| 403 | 404 | ||
| 404 | source "drivers/gpu/drm/gud/Kconfig" | 405 | source "drivers/gpu/drm/gud/Kconfig" |
| 405 | 406 | ||
| 407 | source "drivers/gpu/drm/solomon/Kconfig" | ||
| 408 | |||
| 406 | source "drivers/gpu/drm/sprd/Kconfig" | 409 | source "drivers/gpu/drm/sprd/Kconfig" |
| 407 | 410 | ||
| 408 | config DRM_HYPERV | 411 | config DRM_HYPERV |
| @@ -132,4 +132,5 @@ obj-$(CONFIG_DRM_TIDSS) += tidss/ · 131 unmodified lines | |||
| 132 | obj-y += xlnx/ | 132 | obj-y += xlnx/ |
| 133 | obj-y += gud/ | 133 | obj-y += gud/ |
| 134 | obj-$(CONFIG_DRM_HYPERV) += hyperv/ | 134 | obj-$(CONFIG_DRM_HYPERV) += hyperv/ |
| 135 | obj-y += solomon/ | ||
| 135 | obj-$(CONFIG_DRM_SPRD) += sprd/ | 136 | obj-$(CONFIG_DRM_SPRD) += sprd/ |
| @@ -46,7 +46,7 @@ amdgpu-y += amdgpu_device.o amdgpu_kms.o \ · 45 unmodified lines | |||
| 46 | atom.o amdgpu_fence.o amdgpu_ttm.o amdgpu_object.o amdgpu_gart.o \ | 46 | atom.o amdgpu_fence.o amdgpu_ttm.o amdgpu_object.o amdgpu_gart.o \ |
| 47 | amdgpu_encoders.o amdgpu_display.o amdgpu_i2c.o \ | 47 | amdgpu_encoders.o amdgpu_display.o amdgpu_i2c.o \ |
| 48 | amdgpu_gem.o amdgpu_ring.o \ | 48 | amdgpu_gem.o amdgpu_ring.o \ |
| 49 | amdgpu_cs.o amdgpu_bios.o amdgpu_benchmark.o amdgpu_test.o \ | 49 | amdgpu_cs.o amdgpu_bios.o amdgpu_benchmark.o \ |
| 50 | atombios_dp.o amdgpu_afmt.o amdgpu_trace_points.o \ | 50 | atombios_dp.o amdgpu_afmt.o amdgpu_trace_points.o \ |
| 51 | atombios_encoders.o amdgpu_sa.o atombios_i2c.o \ | 51 | atombios_encoders.o amdgpu_sa.o atombios_i2c.o \ |
| 52 | amdgpu_dma_buf.o amdgpu_vm.o amdgpu_ib.o amdgpu_pll.o \ | 52 | amdgpu_dma_buf.o amdgpu_vm.o amdgpu_ib.o amdgpu_pll.o \ |
| @@ -260,7 +260,7 @@ static int aldebaran_mode2_restore_ip(struct amdgpu_device *adev) · 259 unmodified lines | |||
| 260 | adev->gfx.rlc.funcs->resume(adev); | 260 | adev->gfx.rlc.funcs->resume(adev); |
| 261 | 261 | ||
| 262 | /* Wait for FW reset event complete */ | 262 | /* Wait for FW reset event complete */ |
| 263 | r = smu_wait_for_event(adev, SMU_EVENT_RESET_COMPLETE, 0); | 263 | r = amdgpu_dpm_wait_for_event(adev, SMU_EVENT_RESET_COMPLETE, 0); |
| 264 | if (r) { | 264 | if (r) { |
| 265 | dev_err(adev->dev, | 265 | dev_err(adev->dev, |
| 266 | "Failed to get response from firmware after reset\n"); | 266 | "Failed to get response from firmware after reset\n"); |
| @@ -99,7 +99,6 @@ · 98 unmodified lines | |||
| 99 | #include "amdgpu_gem.h" | 99 | #include "amdgpu_gem.h" |
| 100 | #include "amdgpu_doorbell.h" | 100 | #include "amdgpu_doorbell.h" |
| 101 | #include "amdgpu_amdkfd.h" | 101 | #include "amdgpu_amdkfd.h" |
| 102 | #include "amdgpu_smu.h" | ||
| 103 | #include "amdgpu_discovery.h" | 102 | #include "amdgpu_discovery.h" |
| 104 | #include "amdgpu_mes.h" | 103 | #include "amdgpu_mes.h" |
| 105 | #include "amdgpu_umc.h" | 104 | #include "amdgpu_umc.h" |
| @@ -109,6 +108,7 @@ · 3 unmodified lines | |||
| 109 | #include "amdgpu_smuio.h" | 108 | #include "amdgpu_smuio.h" |
| 110 | #include "amdgpu_fdinfo.h" | 109 | #include "amdgpu_fdinfo.h" |
| 111 | #include "amdgpu_mca.h" | 110 | #include "amdgpu_mca.h" |
| 111 | #include "amdgpu_ras.h" | ||
| 112 | 112 | ||
| 113 | #define MAX_GPU_INSTANCE 16 | 113 | #define MAX_GPU_INSTANCE 16 |
| 114 | 114 | ||
| @@ -155,8 +155,6 @@ extern int amdgpu_vis_vram_limit; · 40 unmodified lines | |||
| 155 | extern int amdgpu_gart_size; | 155 | extern int amdgpu_gart_size; |
| 156 | extern int amdgpu_gtt_size; | 156 | extern int amdgpu_gtt_size; |
| 157 | extern int amdgpu_moverate; | 157 | extern int amdgpu_moverate; |
| 158 | extern int amdgpu_benchmarking; | ||
| 159 | extern int amdgpu_testing; | ||
| 160 | extern int amdgpu_audio; | 158 | extern int amdgpu_audio; |
| 161 | extern int amdgpu_disp_priority; | 159 | extern int amdgpu_disp_priority; |
| 162 | extern int amdgpu_hw_i2c; | 160 | extern int amdgpu_hw_i2c; |
| @@ -197,7 +195,6 @@ extern int amdgpu_emu_mode; · 34 unmodified lines | |||
| 197 | extern uint amdgpu_smu_memory_pool_size; | 195 | extern uint amdgpu_smu_memory_pool_size; |
| 198 | extern int amdgpu_smu_pptable_id; | 196 | extern int amdgpu_smu_pptable_id; |
| 199 | extern uint amdgpu_dc_feature_mask; | 197 | extern uint amdgpu_dc_feature_mask; |
| 200 | extern uint amdgpu_freesync_vid_mode; | ||
| 201 | extern uint amdgpu_dc_debug_mask; | 198 | extern uint amdgpu_dc_debug_mask; |
| 202 | extern uint amdgpu_dm_abm_level; | 199 | extern uint amdgpu_dm_abm_level; |
| 203 | extern int amdgpu_backlight; | 200 | extern int amdgpu_backlight; |
| @@ -214,6 +211,7 @@ extern int amdgpu_mes; · 10 unmodified lines | |||
| 214 | extern int amdgpu_noretry; | 211 | extern int amdgpu_noretry; |
| 215 | extern int amdgpu_force_asic_type; | 212 | extern int amdgpu_force_asic_type; |
| 216 | extern int amdgpu_smartshift_bias; | 213 | extern int amdgpu_smartshift_bias; |
| 214 | extern int amdgpu_use_xgmi_p2p; | ||
| 217 | #ifdef CONFIG_HSA_AMD | 215 | #ifdef CONFIG_HSA_AMD |
| 218 | extern int sched_policy; | 216 | extern int sched_policy; |
| 219 | extern bool debug_evictions; | 217 | extern bool debug_evictions; |
| @@ -373,7 +371,8 @@ int amdgpu_device_ip_block_add(struct amdgpu_device *adev, · 153 unmodified lines | |||
| 373 | */ | 371 | */ |
| 374 | bool amdgpu_get_bios(struct amdgpu_device *adev); | 372 | bool amdgpu_get_bios(struct amdgpu_device *adev); |
| 375 | bool amdgpu_read_bios(struct amdgpu_device *adev); | 373 | bool amdgpu_read_bios(struct amdgpu_device *adev); |
| 376 | 374 | bool amdgpu_soc15_read_bios_from_rom(struct amdgpu_device *adev, | |
| 375 | u8 *bios, u32 length_bytes); | ||
| 377 | /* | 376 | /* |
| 378 | * Clocks | 377 | * Clocks |
| 379 | */ | 378 | */ |
| @@ -586,13 +585,7 @@ void amdgpu_device_wb_free(struct amdgpu_device *adev, u32 wb); · 206 unmodified lines | |||
| 586 | /* | 585 | /* |
| 587 | * Benchmarking | 586 | * Benchmarking |
| 588 | */ | 587 | */ |
| 589 | void amdgpu_benchmark(struct amdgpu_device *adev, int test_number); | 588 | int amdgpu_benchmark(struct amdgpu_device *adev, int test_number); |
| 590 | |||
| 591 | |||
| 592 | /* | ||
| 593 | * Testing | ||
| 594 | */ | ||
| 595 | void amdgpu_test_moves(struct amdgpu_device *adev); | ||
| 596 | 589 | ||
| 597 | /* | 590 | /* |
| 598 | * ASIC specific register table accessible by UMD | 591 | * ASIC specific register table accessible by UMD |
| @@ -771,6 +764,8 @@ struct amd_powerplay { · 172 unmodified lines | |||
| 771 | const struct amd_pm_funcs *pp_funcs; | 764 | const struct amd_pm_funcs *pp_funcs; |
| 772 | }; | 765 | }; |
| 773 | 766 | ||
| 767 | struct ip_discovery_top; | ||
| 768 | |||
| 774 | /* polaris10 kickers */ | 769 | /* polaris10 kickers */ |
| 775 | #define ASICID_IS_P20(did, rid) (((did == 0x67DF) && \ | 770 | #define ASICID_IS_P20(did, rid) (((did == 0x67DF) && \ |
| 776 | ((rid == 0xE3) || \ | 771 | ((rid == 0xE3) || \ |
| @@ -813,6 +808,8 @@ struct amd_powerplay { · 36 unmodified lines | |||
| 813 | #define AMDGPU_RESET_MAGIC_NUM 64 | 808 | #define AMDGPU_RESET_MAGIC_NUM 64 |
| 814 | #define AMDGPU_MAX_DF_PERFMONS 4 | 809 | #define AMDGPU_MAX_DF_PERFMONS 4 |
| 815 | #define AMDGPU_PRODUCT_NAME_LEN 64 | 810 | #define AMDGPU_PRODUCT_NAME_LEN 64 |
| 811 | struct amdgpu_reset_domain; | ||
| 812 | |||
| 816 | struct amdgpu_device { | 813 | struct amdgpu_device { |
| 817 | struct device *dev; | 814 | struct device *dev; |
| 818 | struct pci_dev *pdev; | 815 | struct pci_dev *pdev; |
| @@ -950,12 +947,6 @@ struct amdgpu_device { · 131 unmodified lines | |||
| 950 | 947 | ||
| 951 | /* powerplay */ | 948 | /* powerplay */ |
| 952 | struct amd_powerplay powerplay; | 949 | struct amd_powerplay powerplay; |
| 953 | bool pp_force_state_enabled; | ||
| 954 | |||
| 955 | /* smu */ | ||
| 956 | struct smu_context smu; | ||
| 957 | |||
| 958 | /* dpm */ | ||
| 959 | struct amdgpu_pm pm; | 950 | struct amdgpu_pm pm; |
| 960 | u32 cg_flags; | 951 | u32 cg_flags; |
| 961 | u32 pg_flags; | 952 | u32 pg_flags; |
| @@ -1054,9 +1045,7 @@ struct amdgpu_device { · 92 unmodified lines | |||
| 1054 | bool in_s4; | 1045 | bool in_s4; |
| 1055 | bool in_s0ix; | 1046 | bool in_s0ix; |
| 1056 | 1047 | ||
| 1057 | atomic_t in_gpu_reset; | ||
| 1058 | enum pp_mp1_state mp1_state; | 1048 | enum pp_mp1_state mp1_state; |
| 1059 | struct rw_semaphore reset_sem; | ||
| 1060 | struct amdgpu_doorbell_index doorbell_index; | 1049 | struct amdgpu_doorbell_index doorbell_index; |
| 1061 | 1050 | ||
| 1062 | struct mutex notifier_lock; | 1051 | struct mutex notifier_lock; |
| @@ -1100,6 +1089,18 @@ struct amdgpu_device { · 37 unmodified lines | |||
| 1100 | uint32_t ip_versions[MAX_HWIP][HWIP_MAX_INSTANCE]; | 1089 | uint32_t ip_versions[MAX_HWIP][HWIP_MAX_INSTANCE]; |
| 1101 | 1090 | ||
| 1102 | bool ram_is_direct_mapped; | 1091 | bool ram_is_direct_mapped; |
| 1092 | |||
| 1093 | struct list_head ras_list; | ||
| 1094 | |||
| 1095 | struct ip_discovery_top *ip_top; | ||
| 1096 | |||
| 1097 | struct amdgpu_reset_domain *reset_domain; | ||
| 1098 | |||
| 1099 | struct mutex benchmark_mutex; | ||
| 1100 | |||
| 1101 | /* reset dump register */ | ||
| 1102 | uint32_t *reset_dump_reg_list; | ||
| 1103 | int num_regs; | ||
| 1103 | }; | 1104 | }; |
| 1104 | 1105 | ||
| 1105 | static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) | 1106 | static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) |
| @@ -1293,9 +1294,12 @@ bool amdgpu_device_has_job_running(struct amdgpu_device *adev); · 187 unmodified lines | |||
| 1293 | bool amdgpu_device_should_recover_gpu(struct amdgpu_device *adev); | 1294 | bool amdgpu_device_should_recover_gpu(struct amdgpu_device *adev); |
| 1294 | int amdgpu_device_gpu_recover(struct amdgpu_device *adev, | 1295 | int amdgpu_device_gpu_recover(struct amdgpu_device *adev, |
| 1295 | struct amdgpu_job* job); | 1296 | struct amdgpu_job* job); |
| 1297 | int amdgpu_device_gpu_recover_imp(struct amdgpu_device *adev, | ||
| 1298 | struct amdgpu_job *job); | ||
| 1296 | void amdgpu_device_pci_config_reset(struct amdgpu_device *adev); | 1299 | void amdgpu_device_pci_config_reset(struct amdgpu_device *adev); |
| 1297 | int amdgpu_device_pci_reset(struct amdgpu_device *adev); | 1300 | int amdgpu_device_pci_reset(struct amdgpu_device *adev); |
| 1298 | bool amdgpu_device_need_post(struct amdgpu_device *adev); | 1301 | bool amdgpu_device_need_post(struct amdgpu_device *adev); |
| 1302 | bool amdgpu_device_should_use_aspm(struct amdgpu_device *adev); | ||
| 1299 | 1303 | ||
| 1300 | void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes, | 1304 | void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes, |
| 1301 | u64 num_vis_bytes); | 1305 | u64 num_vis_bytes); |
| @@ -1321,6 +1325,10 @@ void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, · 19 unmodified lines | |||
| 1321 | struct amdgpu_ring *ring); | 1325 | struct amdgpu_ring *ring); |
| 1322 | 1326 | ||
| 1323 | void amdgpu_device_halt(struct amdgpu_device *adev); | 1327 | void amdgpu_device_halt(struct amdgpu_device *adev); |
| 1328 | u32 amdgpu_device_pcie_port_rreg(struct amdgpu_device *adev, | ||
| 1329 | u32 reg); | ||
| 1330 | void amdgpu_device_pcie_port_wreg(struct amdgpu_device *adev, | ||
| 1331 | u32 reg, u32 v); | ||
| 1324 | 1332 | ||
| 1325 | /* atpx handler */ | 1333 | /* atpx handler */ |
| 1326 | #if defined(CONFIG_VGA_SWITCHEROO) | 1334 | #if defined(CONFIG_VGA_SWITCHEROO) |
| @@ -1408,12 +1416,10 @@ int amdgpu_acpi_smart_shift_update(struct drm_device *dev, enum amdgpu_ss ss_sta · 81 unmodified lines | |||
| 1408 | int amdgpu_acpi_pcie_notify_device_ready(struct amdgpu_device *adev); | 1416 | int amdgpu_acpi_pcie_notify_device_ready(struct amdgpu_device *adev); |
| 1409 | 1417 | ||
| 1410 | void amdgpu_acpi_get_backlight_caps(struct amdgpu_dm_backlight_caps *caps); | 1418 | void amdgpu_acpi_get_backlight_caps(struct amdgpu_dm_backlight_caps *caps); |
| 1411 | bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev); | ||
| 1412 | void amdgpu_acpi_detect(void); | 1419 | void amdgpu_acpi_detect(void); |
| 1413 | #else | 1420 | #else |
| 1414 | static inline int amdgpu_acpi_init(struct amdgpu_device *adev) { return 0; } | 1421 | static inline int amdgpu_acpi_init(struct amdgpu_device *adev) { return 0; } |
| 1415 | static inline void amdgpu_acpi_fini(struct amdgpu_device *adev) { } | 1422 | static inline void amdgpu_acpi_fini(struct amdgpu_device *adev) { } |
| 1416 | static inline bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev) { return false; } | ||
| 1417 | static inline void amdgpu_acpi_detect(void) { } | 1423 | static inline void amdgpu_acpi_detect(void) { } |
| 1418 | static inline bool amdgpu_acpi_is_power_shift_control_supported(void) { return false; } | 1424 | static inline bool amdgpu_acpi_is_power_shift_control_supported(void) { return false; } |
| 1419 | static inline int amdgpu_acpi_power_shift_control(struct amdgpu_device *adev, | 1425 | static inline int amdgpu_acpi_power_shift_control(struct amdgpu_device *adev, |
| @@ -1422,6 +1428,14 @@ static inline int amdgpu_acpi_smart_shift_update(struct drm_device *dev, · 2 unmodified lines | |||
| 1422 | enum amdgpu_ss ss_state) { return 0; } | 1428 | enum amdgpu_ss ss_state) { return 0; } |
| 1423 | #endif | 1429 | #endif |
| 1424 | 1430 | ||
| 1431 | #if defined(CONFIG_ACPI) && defined(CONFIG_SUSPEND) | ||
| 1432 | bool amdgpu_acpi_is_s3_active(struct amdgpu_device *adev); | ||
| 1433 | bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev); | ||
| 1434 | #else | ||
| 1435 | static inline bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev) { return false; } | ||
| 1436 | static inline bool amdgpu_acpi_is_s3_active(struct amdgpu_device *adev) { return false; } | ||
| 1437 | #endif | ||
| 1438 | |||
| 1425 | int amdgpu_cs_find_mapping(struct amdgpu_cs_parser *parser, | 1439 | int amdgpu_cs_find_mapping(struct amdgpu_cs_parser *parser, |
| 1426 | uint64_t addr, struct amdgpu_bo **bo, | 1440 | uint64_t addr, struct amdgpu_bo **bo, |
| 1427 | struct amdgpu_bo_va_mapping **mapping); | 1441 | struct amdgpu_bo_va_mapping **mapping); |
| @@ -1452,6 +1466,15 @@ int amdgpu_device_set_cg_state(struct amdgpu_device *adev, · 24 unmodified lines | |||
| 1452 | int amdgpu_device_set_pg_state(struct amdgpu_device *adev, | 1466 | int amdgpu_device_set_pg_state(struct amdgpu_device *adev, |
| 1453 | enum amd_powergating_state state); | 1467 | enum amd_powergating_state state); |
| 1454 | 1468 | ||
| 1469 | static inline bool amdgpu_device_has_timeouts_enabled(struct amdgpu_device *adev) | ||
| 1470 | { | ||
| 1471 | return amdgpu_gpu_recovery != 0 && | ||
| 1472 | adev->gfx_timeout != MAX_SCHEDULE_TIMEOUT && | ||
| 1473 | adev->compute_timeout != MAX_SCHEDULE_TIMEOUT && | ||
| 1474 | adev->sdma_timeout != MAX_SCHEDULE_TIMEOUT && | ||
| 1475 | adev->video_timeout != MAX_SCHEDULE_TIMEOUT; | ||
| 1476 | } | ||
| 1477 | |||
| 1455 | #include "amdgpu_object.h" | 1478 | #include "amdgpu_object.h" |
| 1456 | 1479 | ||
| 1457 | static inline bool amdgpu_is_tmz(struct amdgpu_device *adev) | 1480 | static inline bool amdgpu_is_tmz(struct amdgpu_device *adev) |
| @@ -1459,8 +1482,6 @@ static inline bool amdgpu_is_tmz(struct amdgpu_device *adev) · 1 unmodified line | |||
| 1459 | return adev->gmc.tmz_enabled; | 1482 | return adev->gmc.tmz_enabled; |
| 1460 | } | 1483 | } |
| 1461 | 1484 | ||
| 1462 | static inline int amdgpu_in_reset(struct amdgpu_device *adev) | 1485 | int amdgpu_in_reset(struct amdgpu_device *adev); |
| 1463 | { | 1486 | |
| 1464 | return atomic_read(&adev->in_gpu_reset); | ||
| 1465 | } | ||
| 1466 | #endif | 1487 | #endif |
| @@ -1031,6 +1031,20 @@ void amdgpu_acpi_detect(void) · 1030 unmodified lines | |||
| 1031 | } | 1031 | } |
| 1032 | } | 1032 | } |
| 1033 | 1033 | ||
| 1034 | #if IS_ENABLED(CONFIG_SUSPEND) | ||
| 1035 | /** | ||
| 1036 | * amdgpu_acpi_is_s3_active | ||
| 1037 | * | ||
| 1038 | * @adev: amdgpu_device_pointer | ||
| 1039 | * | ||
| 1040 | * returns true if supported, false if not. | ||
| 1041 | */ | ||
| 1042 | bool amdgpu_acpi_is_s3_active(struct amdgpu_device *adev) | ||
| 1043 | { | ||
| 1044 | return !(adev->flags & AMD_IS_APU) || | ||
| 1045 | (pm_suspend_target_state == PM_SUSPEND_MEM); | ||
| 1046 | } | ||
| 1047 | |||
| 1034 | /** | 1048 | /** |
| 1035 | * amdgpu_acpi_is_s0ix_active | 1049 | * amdgpu_acpi_is_s0ix_active |
| 1036 | * | 1050 | * |
| @@ -1040,11 +1054,24 @@ void amdgpu_acpi_detect(void) · 3 unmodified lines | |||
| 1040 | */ | 1054 | */ |
| 1041 | bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev) | 1055 | bool amdgpu_acpi_is_s0ix_active(struct amdgpu_device *adev) |
| 1042 | { | 1056 | { |
| 1043 | #if IS_ENABLED(CONFIG_AMD_PMC) && IS_ENABLED(CONFIG_SUSPEND) | 1057 | if (!(adev->flags & AMD_IS_APU) || |
| 1044 | if (acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0) { | 1058 | (pm_suspend_target_state != PM_SUSPEND_TO_IDLE)) |
| 1045 | if (adev->flags & AMD_IS_APU) | 1059 | return false; |
| 1046 | return pm_suspend_target_state == PM_SUSPEND_TO_IDLE; | 1060 | |
| 1061 | if (!(acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0)) { | ||
| 1062 | dev_warn_once(adev->dev, | ||
| 1063 | "Power consumption will be higher as BIOS has not been configured for suspend-to-idle.\n" | ||
| 1064 | "To use suspend-to-idle change the sleep mode in BIOS setup.\n"); | ||
| 1065 | return false; | ||
| 1047 | } | 1066 | } |
| 1048 | #endif | 1067 | |
| 1068 | #if !IS_ENABLED(CONFIG_AMD_PMC) | ||
| 1069 | dev_warn_once(adev->dev, | ||
| 1070 | "Power consumption will be higher as the kernel has not been compiled with CONFIG_AMD_PMC.\n"); | ||
| 1049 | return false; | 1071 | return false; |
| 1072 | #else | ||
| 1073 | return true; | ||
| 1074 | #endif /* CONFIG_AMD_PMC */ | ||
| 1050 | } | 1075 | } |
| 1076 | |||
| 1077 | #endif /* CONFIG_SUSPEND */ | ||
| @@ -131,6 +131,7 @@ struct amdkfd_process_info { · 130 unmodified lines | |||
| 131 | atomic_t evicted_bos; | 131 | atomic_t evicted_bos; |
| 132 | struct delayed_work restore_userptr_work; | 132 | struct delayed_work restore_userptr_work; |
| 133 | struct pid *pid; | 133 | struct pid *pid; |
| 134 | bool block_mmu_notifications; | ||
| 134 | }; | 135 | }; |
| 135 | 136 | ||
| 136 | int amdgpu_amdkfd_init(void); | 137 | int amdgpu_amdkfd_init(void); |
| @@ -268,7 +269,7 @@ uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *drm_priv); · 131 unmodified lines | |||
| 268 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( | 269 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( |
| 269 | struct amdgpu_device *adev, uint64_t va, uint64_t size, | 270 | struct amdgpu_device *adev, uint64_t va, uint64_t size, |
| 270 | void *drm_priv, struct kgd_mem **mem, | 271 | void *drm_priv, struct kgd_mem **mem, |
| 271 | uint64_t *offset, uint32_t flags); | 272 | uint64_t *offset, uint32_t flags, bool criu_resume); |
| 272 | int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( | 273 | int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( |
| 273 | struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv, | 274 | struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv, |
| 274 | uint64_t *size); | 275 | uint64_t *size); |
| @@ -297,6 +298,10 @@ int amdgpu_amdkfd_get_tile_config(struct amdgpu_device *adev, · 22 unmodified lines | |||
| 297 | struct tile_config *config); | 298 | struct tile_config *config); |
| 298 | void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev, | 299 | void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev, |
| 299 | bool reset); | 300 | bool reset); |
| 301 | bool amdgpu_amdkfd_bo_mapped_to_dev(struct amdgpu_device *adev, struct kgd_mem *mem); | ||
| 302 | void amdgpu_amdkfd_block_mmu_notifications(void *p); | ||
| 303 | int amdgpu_amdkfd_criu_resume(void *p); | ||
| 304 | |||
| 300 | #if IS_ENABLED(CONFIG_HSA_AMD) | 305 | #if IS_ENABLED(CONFIG_HSA_AMD) |
| 301 | void amdgpu_amdkfd_gpuvm_init_mem_limits(void); | 306 | void amdgpu_amdkfd_gpuvm_init_mem_limits(void); |
| 302 | void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev, | 307 | void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev, |
| @@ -37,10 +37,7 @@ const struct kfd2kgd_calls aldebaran_kfd2kgd = { · 36 unmodified lines | |||
| 37 | .hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied, | 37 | .hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied, |
| 38 | .hqd_destroy = kgd_gfx_v9_hqd_destroy, | 38 | .hqd_destroy = kgd_gfx_v9_hqd_destroy, |
| 39 | .hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy, | 39 | .hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy, |
| 40 | .address_watch_disable = kgd_gfx_v9_address_watch_disable, | ||
| 41 | .address_watch_execute = kgd_gfx_v9_address_watch_execute, | ||
| 42 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, | 40 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, |
| 43 | .address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset, | ||
| 44 | .get_atc_vmid_pasid_mapping_info = | 41 | .get_atc_vmid_pasid_mapping_info = |
| 45 | kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, | 42 | kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, |
| 46 | .set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base, | 43 | .set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base, |
| @@ -289,10 +289,7 @@ const struct kfd2kgd_calls arcturus_kfd2kgd = { · 288 unmodified lines | |||
| 289 | .hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied, | 289 | .hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied, |
| 290 | .hqd_destroy = kgd_gfx_v9_hqd_destroy, | 290 | .hqd_destroy = kgd_gfx_v9_hqd_destroy, |
| 291 | .hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy, | 291 | .hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy, |
| 292 | .address_watch_disable = kgd_gfx_v9_address_watch_disable, | ||
| 293 | .address_watch_execute = kgd_gfx_v9_address_watch_execute, | ||
| 294 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, | 292 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, |
| 295 | .address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset, | ||
| 296 | .get_atc_vmid_pasid_mapping_info = | 293 | .get_atc_vmid_pasid_mapping_info = |
| 297 | kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, | 294 | kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, |
| 298 | .set_vm_context_page_table_base = | 295 | .set_vm_context_page_table_base = |
| @@ -671,20 +671,6 @@ static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, · 670 unmodified lines | |||
| 671 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | 671 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 672 | } | 672 | } |
| 673 | 673 | ||
| 674 | static int kgd_address_watch_disable(struct amdgpu_device *adev) | ||
| 675 | { | ||
| 676 | return 0; | ||
| 677 | } | ||
| 678 | |||
| 679 | static int kgd_address_watch_execute(struct amdgpu_device *adev, | ||
| 680 | unsigned int watch_point_id, | ||
| 681 | uint32_t cntl_val, | ||
| 682 | uint32_t addr_hi, | ||
| 683 | uint32_t addr_lo) | ||
| 684 | { | ||
| 685 | return 0; | ||
| 686 | } | ||
| 687 | |||
| 688 | static int kgd_wave_control_execute(struct amdgpu_device *adev, | 674 | static int kgd_wave_control_execute(struct amdgpu_device *adev, |
| 689 | uint32_t gfx_index_val, | 675 | uint32_t gfx_index_val, |
| 690 | uint32_t sq_cmd) | 676 | uint32_t sq_cmd) |
| @@ -709,13 +695,6 @@ static int kgd_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines | |||
| 709 | return 0; | 695 | return 0; |
| 710 | } | 696 | } |
| 711 | 697 | ||
| 712 | static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev, | ||
| 713 | unsigned int watch_point_id, | ||
| 714 | unsigned int reg_offset) | ||
| 715 | { | ||
| 716 | return 0; | ||
| 717 | } | ||
| 718 | |||
| 719 | static void set_vm_context_page_table_base(struct amdgpu_device *adev, | 698 | static void set_vm_context_page_table_base(struct amdgpu_device *adev, |
| 720 | uint32_t vmid, uint64_t page_table_base) | 699 | uint32_t vmid, uint64_t page_table_base) |
| 721 | { | 700 | { |
| @@ -767,10 +746,7 @@ const struct kfd2kgd_calls gfx_v10_kfd2kgd = { · 45 unmodified lines | |||
| 767 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | 746 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, |
| 768 | .hqd_destroy = kgd_hqd_destroy, | 747 | .hqd_destroy = kgd_hqd_destroy, |
| 769 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | 748 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, |
| 770 | .address_watch_disable = kgd_address_watch_disable, | ||
| 771 | .address_watch_execute = kgd_address_watch_execute, | ||
| 772 | .wave_control_execute = kgd_wave_control_execute, | 749 | .wave_control_execute = kgd_wave_control_execute, |
| 773 | .address_watch_get_offset = kgd_address_watch_get_offset, | ||
| 774 | .get_atc_vmid_pasid_mapping_info = | 750 | .get_atc_vmid_pasid_mapping_info = |
| 775 | get_atc_vmid_pasid_mapping_info, | 751 | get_atc_vmid_pasid_mapping_info, |
| 776 | .set_vm_context_page_table_base = set_vm_context_page_table_base, | 752 | .set_vm_context_page_table_base = set_vm_context_page_table_base, |
| @@ -582,21 +582,6 @@ static int hqd_sdma_destroy_v10_3(struct amdgpu_device *adev, void *mqd, · 581 unmodified lines | |||
| 582 | return 0; | 582 | return 0; |
| 583 | } | 583 | } |
| 584 | 584 | ||
| 585 | |||
| 586 | static int address_watch_disable_v10_3(struct amdgpu_device *adev) | ||
| 587 | { | ||
| 588 | return 0; | ||
| 589 | } | ||
| 590 | |||
| 591 | static int address_watch_execute_v10_3(struct amdgpu_device *adev, | ||
| 592 | unsigned int watch_point_id, | ||
| 593 | uint32_t cntl_val, | ||
| 594 | uint32_t addr_hi, | ||
| 595 | uint32_t addr_lo) | ||
| 596 | { | ||
| 597 | return 0; | ||
| 598 | } | ||
| 599 | |||
| 600 | static int wave_control_execute_v10_3(struct amdgpu_device *adev, | 585 | static int wave_control_execute_v10_3(struct amdgpu_device *adev, |
| 601 | uint32_t gfx_index_val, | 586 | uint32_t gfx_index_val, |
| 602 | uint32_t sq_cmd) | 587 | uint32_t sq_cmd) |
| @@ -621,13 +606,6 @@ static int wave_control_execute_v10_3(struct amdgpu_device *adev, · 18 unmodified lines | |||
| 621 | return 0; | 606 | return 0; |
| 622 | } | 607 | } |
| 623 | 608 | ||
| 624 | static uint32_t address_watch_get_offset_v10_3(struct amdgpu_device *adev, | ||
| 625 | unsigned int watch_point_id, | ||
| 626 | unsigned int reg_offset) | ||
| 627 | { | ||
| 628 | return 0; | ||
| 629 | } | ||
| 630 | |||
| 631 | static void set_vm_context_page_table_base_v10_3(struct amdgpu_device *adev, | 609 | static void set_vm_context_page_table_base_v10_3(struct amdgpu_device *adev, |
| 632 | uint32_t vmid, uint64_t page_table_base) | 610 | uint32_t vmid, uint64_t page_table_base) |
| 633 | { | 611 | { |
| @@ -809,10 +787,7 @@ const struct kfd2kgd_calls gfx_v10_3_kfd2kgd = { · 175 unmodified lines | |||
| 809 | .hqd_sdma_is_occupied = hqd_sdma_is_occupied_v10_3, | 787 | .hqd_sdma_is_occupied = hqd_sdma_is_occupied_v10_3, |
| 810 | .hqd_destroy = hqd_destroy_v10_3, | 788 | .hqd_destroy = hqd_destroy_v10_3, |
| 811 | .hqd_sdma_destroy = hqd_sdma_destroy_v10_3, | 789 | .hqd_sdma_destroy = hqd_sdma_destroy_v10_3, |
| 812 | .address_watch_disable = address_watch_disable_v10_3, | ||
| 813 | .address_watch_execute = address_watch_execute_v10_3, | ||
| 814 | .wave_control_execute = wave_control_execute_v10_3, | 790 | .wave_control_execute = wave_control_execute_v10_3, |
| 815 | .address_watch_get_offset = address_watch_get_offset_v10_3, | ||
| 816 | .get_atc_vmid_pasid_mapping_info = NULL, | 791 | .get_atc_vmid_pasid_mapping_info = NULL, |
| 817 | .set_vm_context_page_table_base = set_vm_context_page_table_base_v10_3, | 792 | .set_vm_context_page_table_base = set_vm_context_page_table_base_v10_3, |
| 818 | .program_trap_handler_settings = program_trap_handler_settings_v10_3, | 793 | .program_trap_handler_settings = program_trap_handler_settings_v10_3, |
| @@ -45,43 +45,6 @@ enum { · 44 unmodified lines | |||
| 45 | MAX_WATCH_ADDRESSES = 4 | 45 | MAX_WATCH_ADDRESSES = 4 |
| 46 | }; | 46 | }; |
| 47 | 47 | ||
| 48 | enum { | ||
| 49 | ADDRESS_WATCH_REG_ADDR_HI = 0, | ||
| 50 | ADDRESS_WATCH_REG_ADDR_LO, | ||
| 51 | ADDRESS_WATCH_REG_CNTL, | ||
| 52 | ADDRESS_WATCH_REG_MAX | ||
| 53 | }; | ||
| 54 | |||
| 55 | /* not defined in the CI/KV reg file */ | ||
| 56 | enum { | ||
| 57 | ADDRESS_WATCH_REG_CNTL_ATC_BIT = 0x10000000UL, | ||
| 58 | ADDRESS_WATCH_REG_CNTL_DEFAULT_MASK = 0x00FFFFFF, | ||
| 59 | ADDRESS_WATCH_REG_ADDLOW_MASK_EXTENSION = 0x03000000, | ||
| 60 | /* extend the mask to 26 bits to match the low address field */ | ||
| 61 | ADDRESS_WATCH_REG_ADDLOW_SHIFT = 6, | ||
| 62 | ADDRESS_WATCH_REG_ADDHIGH_MASK = 0xFFFF | ||
| 63 | }; | ||
| 64 | |||
| 65 | static const uint32_t watchRegs[MAX_WATCH_ADDRESSES * ADDRESS_WATCH_REG_MAX] = { | ||
| 66 | mmTCP_WATCH0_ADDR_H, mmTCP_WATCH0_ADDR_L, mmTCP_WATCH0_CNTL, | ||
| 67 | mmTCP_WATCH1_ADDR_H, mmTCP_WATCH1_ADDR_L, mmTCP_WATCH1_CNTL, | ||
| 68 | mmTCP_WATCH2_ADDR_H, mmTCP_WATCH2_ADDR_L, mmTCP_WATCH2_CNTL, | ||
| 69 | mmTCP_WATCH3_ADDR_H, mmTCP_WATCH3_ADDR_L, mmTCP_WATCH3_CNTL | ||
| 70 | }; | ||
| 71 | |||
| 72 | union TCP_WATCH_CNTL_BITS { | ||
| 73 | struct { | ||
| 74 | uint32_t mask:24; | ||
| 75 | uint32_t vmid:4; | ||
| 76 | uint32_t atc:1; | ||
| 77 | uint32_t mode:2; | ||
| 78 | uint32_t valid:1; | ||
| 79 | } bitfields, bits; | ||
| 80 | uint32_t u32All; | ||
| 81 | signed int i32All; | ||
| 82 | float f32All; | ||
| 83 | }; | ||
| 84 | |||
| 85 | static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe, | 48 | static void lock_srbm(struct amdgpu_device *adev, uint32_t mec, uint32_t pipe, |
| 86 | uint32_t queue, uint32_t vmid) | 49 | uint32_t queue, uint32_t vmid) |
| 87 | { | 50 | { |
| @@ -529,55 +492,6 @@ static int kgd_hqd_sdma_destroy(struct amdgpu_device *adev, void *mqd, · 441 unmodified lines | |||
| 529 | return 0; | 492 | return 0; |
| 530 | } | 493 | } |
| 531 | 494 | ||
| 532 | static int kgd_address_watch_disable(struct amdgpu_device *adev) | ||
| 533 | { | ||
| 534 | union TCP_WATCH_CNTL_BITS cntl; | ||
| 535 | unsigned int i; | ||
| 536 | |||
| 537 | cntl.u32All = 0; | ||
| 538 | |||
| 539 | cntl.bitfields.valid = 0; | ||
| 540 | cntl.bitfields.mask = ADDRESS_WATCH_REG_CNTL_DEFAULT_MASK; | ||
| 541 | cntl.bitfields.atc = 1; | ||
| 542 | |||
| 543 | /* Turning off this address until we set all the registers */ | ||
| 544 | for (i = 0; i < MAX_WATCH_ADDRESSES; i++) | ||
| 545 | WREG32(watchRegs[i * ADDRESS_WATCH_REG_MAX + | ||
| 546 | ADDRESS_WATCH_REG_CNTL], cntl.u32All); | ||
| 547 | |||
| 548 | return 0; | ||
| 549 | } | ||
| 550 | |||
| 551 | static int kgd_address_watch_execute(struct amdgpu_device *adev, | ||
| 552 | unsigned int watch_point_id, | ||
| 553 | uint32_t cntl_val, | ||
| 554 | uint32_t addr_hi, | ||
| 555 | uint32_t addr_lo) | ||
| 556 | { | ||
| 557 | union TCP_WATCH_CNTL_BITS cntl; | ||
| 558 | |||
| 559 | cntl.u32All = cntl_val; | ||
| 560 | |||
| 561 | /* Turning off this watch point until we set all the registers */ | ||
| 562 | cntl.bitfields.valid = 0; | ||
| 563 | WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + | ||
| 564 | ADDRESS_WATCH_REG_CNTL], cntl.u32All); | ||
| 565 | |||
| 566 | WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + | ||
| 567 | ADDRESS_WATCH_REG_ADDR_HI], addr_hi); | ||
| 568 | |||
| 569 | WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + | ||
| 570 | ADDRESS_WATCH_REG_ADDR_LO], addr_lo); | ||
| 571 | |||
| 572 | /* Enable the watch point */ | ||
| 573 | cntl.bitfields.valid = 1; | ||
| 574 | |||
| 575 | WREG32(watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + | ||
| 576 | ADDRESS_WATCH_REG_CNTL], cntl.u32All); | ||
| 577 | |||
| 578 | return 0; | ||
| 579 | } | ||
| 580 | |||
| 581 | static int kgd_wave_control_execute(struct amdgpu_device *adev, | 495 | static int kgd_wave_control_execute(struct amdgpu_device *adev, |
| 582 | uint32_t gfx_index_val, | 496 | uint32_t gfx_index_val, |
| 583 | uint32_t sq_cmd) | 497 | uint32_t sq_cmd) |
| @@ -602,13 +516,6 @@ static int kgd_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines | |||
| 602 | return 0; | 516 | return 0; |
| 603 | } | 517 | } |
| 604 | 518 | ||
| 605 | static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev, | ||
| 606 | unsigned int watch_point_id, | ||
| 607 | unsigned int reg_offset) | ||
| 608 | { | ||
| 609 | return watchRegs[watch_point_id * ADDRESS_WATCH_REG_MAX + reg_offset]; | ||
| 610 | } | ||
| 611 | |||
| 612 | static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, | 519 | static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, |
| 613 | uint8_t vmid, uint16_t *p_pasid) | 520 | uint8_t vmid, uint16_t *p_pasid) |
| 614 | { | 521 | { |
| @@ -665,10 +572,7 @@ const struct kfd2kgd_calls gfx_v7_kfd2kgd = { · 50 unmodified lines | |||
| 665 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | 572 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, |
| 666 | .hqd_destroy = kgd_hqd_destroy, | 573 | .hqd_destroy = kgd_hqd_destroy, |
| 667 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | 574 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, |
| 668 | .address_watch_disable = kgd_address_watch_disable, | ||
| 669 | .address_watch_execute = kgd_address_watch_execute, | ||
| 670 | .wave_control_execute = kgd_wave_control_execute, | 575 | .wave_control_execute = kgd_wave_control_execute, |
| 671 | .address_watch_get_offset = kgd_address_watch_get_offset, | ||
| 672 | .get_atc_vmid_pasid_mapping_info = get_atc_vmid_pasid_mapping_info, | 576 | .get_atc_vmid_pasid_mapping_info = get_atc_vmid_pasid_mapping_info, |
| 673 | .set_scratch_backing_va = set_scratch_backing_va, | 577 | .set_scratch_backing_va = set_scratch_backing_va, |
| 674 | .set_vm_context_page_table_base = set_vm_context_page_table_base, | 578 | .set_vm_context_page_table_base = set_vm_context_page_table_base, |
| @@ -538,20 +538,6 @@ static bool get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, · 537 unmodified lines | |||
| 538 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | 538 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 539 | } | 539 | } |
| 540 | 540 | ||
| 541 | static int kgd_address_watch_disable(struct amdgpu_device *adev) | ||
| 542 | { | ||
| 543 | return 0; | ||
| 544 | } | ||
| 545 | |||
| 546 | static int kgd_address_watch_execute(struct amdgpu_device *adev, | ||
| 547 | unsigned int watch_point_id, | ||
| 548 | uint32_t cntl_val, | ||
| 549 | uint32_t addr_hi, | ||
| 550 | uint32_t addr_lo) | ||
| 551 | { | ||
| 552 | return 0; | ||
| 553 | } | ||
| 554 | |||
| 555 | static int kgd_wave_control_execute(struct amdgpu_device *adev, | 541 | static int kgd_wave_control_execute(struct amdgpu_device *adev, |
| 556 | uint32_t gfx_index_val, | 542 | uint32_t gfx_index_val, |
| 557 | uint32_t sq_cmd) | 543 | uint32_t sq_cmd) |
| @@ -576,13 +562,6 @@ static int kgd_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines | |||
| 576 | return 0; | 562 | return 0; |
| 577 | } | 563 | } |
| 578 | 564 | ||
| 579 | static uint32_t kgd_address_watch_get_offset(struct amdgpu_device *adev, | ||
| 580 | unsigned int watch_point_id, | ||
| 581 | unsigned int reg_offset) | ||
| 582 | { | ||
| 583 | return 0; | ||
| 584 | } | ||
| 585 | |||
| 586 | static void set_scratch_backing_va(struct amdgpu_device *adev, | 565 | static void set_scratch_backing_va(struct amdgpu_device *adev, |
| 587 | uint64_t va, uint32_t vmid) | 566 | uint64_t va, uint32_t vmid) |
| 588 | { | 567 | { |
| @@ -614,10 +593,7 @@ const struct kfd2kgd_calls gfx_v8_kfd2kgd = { · 25 unmodified lines | |||
| 614 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | 593 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, |
| 615 | .hqd_destroy = kgd_hqd_destroy, | 594 | .hqd_destroy = kgd_hqd_destroy, |
| 616 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | 595 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, |
| 617 | .address_watch_disable = kgd_address_watch_disable, | ||
| 618 | .address_watch_execute = kgd_address_watch_execute, | ||
| 619 | .wave_control_execute = kgd_wave_control_execute, | 596 | .wave_control_execute = kgd_wave_control_execute, |
| 620 | .address_watch_get_offset = kgd_address_watch_get_offset, | ||
| 621 | .get_atc_vmid_pasid_mapping_info = | 597 | .get_atc_vmid_pasid_mapping_info = |
| 622 | get_atc_vmid_pasid_mapping_info, | 598 | get_atc_vmid_pasid_mapping_info, |
| 623 | .set_scratch_backing_va = set_scratch_backing_va, | 599 | .set_scratch_backing_va = set_scratch_backing_va, |
| @@ -622,20 +622,6 @@ bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, · 621 unmodified lines | |||
| 622 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); | 622 | return !!(value & ATC_VMID0_PASID_MAPPING__VALID_MASK); |
| 623 | } | 623 | } |
| 624 | 624 | ||
| 625 | int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev) | ||
| 626 | { | ||
| 627 | return 0; | ||
| 628 | } | ||
| 629 | |||
| 630 | int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev, | ||
| 631 | unsigned int watch_point_id, | ||
| 632 | uint32_t cntl_val, | ||
| 633 | uint32_t addr_hi, | ||
| 634 | uint32_t addr_lo) | ||
| 635 | { | ||
| 636 | return 0; | ||
| 637 | } | ||
| 638 | |||
| 639 | int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, | 625 | int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, |
| 640 | uint32_t gfx_index_val, | 626 | uint32_t gfx_index_val, |
| 641 | uint32_t sq_cmd) | 627 | uint32_t sq_cmd) |
| @@ -660,13 +646,6 @@ int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, · 18 unmodified lines | |||
| 660 | return 0; | 646 | return 0; |
| 661 | } | 647 | } |
| 662 | 648 | ||
| 663 | uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev, | ||
| 664 | unsigned int watch_point_id, | ||
| 665 | unsigned int reg_offset) | ||
| 666 | { | ||
| 667 | return 0; | ||
| 668 | } | ||
| 669 | |||
| 670 | void kgd_gfx_v9_set_vm_context_page_table_base(struct amdgpu_device *adev, | 649 | void kgd_gfx_v9_set_vm_context_page_table_base(struct amdgpu_device *adev, |
| 671 | uint32_t vmid, uint64_t page_table_base) | 650 | uint32_t vmid, uint64_t page_table_base) |
| 672 | { | 651 | { |
| @@ -736,7 +715,7 @@ static void get_wave_count(struct amdgpu_device *adev, int queue_idx, · 63 unmodified lines | |||
| 736 | * process whose pasid is provided as a parameter. The process could have ZERO | 715 | * process whose pasid is provided as a parameter. The process could have ZERO |
| 737 | * or more queues running and submitting waves to compute units. | 716 | * or more queues running and submitting waves to compute units. |
| 738 | * | 717 | * |
| 739 | * @kgd: Handle of device from which to get number of waves in flight | 718 | * @adev: Handle of device from which to get number of waves in flight |
| 740 | * @pasid: Identifies the process for which this query call is invoked | 719 | * @pasid: Identifies the process for which this query call is invoked |
| 741 | * @pasid_wave_cnt: Output parameter updated with number of waves in flight that | 720 | * @pasid_wave_cnt: Output parameter updated with number of waves in flight that |
| 742 | * belong to process with given pasid | 721 | * belong to process with given pasid |
| @@ -745,7 +724,7 @@ static void get_wave_count(struct amdgpu_device *adev, int queue_idx, · 2 unmodified lines | |||
| 745 | * | 724 | * |
| 746 | * Note: It's possible that the device has too many queues (oversubscription) | 725 | * Note: It's possible that the device has too many queues (oversubscription) |
| 747 | * in which case a VMID could be remapped to a different PASID. This could lead | 726 | * in which case a VMID could be remapped to a different PASID. This could lead |
| 748 | * to an iaccurate wave count. Following is a high-level sequence: | 727 | * to an inaccurate wave count. Following is a high-level sequence: |
| 749 | * Time T1: vmid = getVmid(); vmid is associated with Pasid P1 | 728 | * Time T1: vmid = getVmid(); vmid is associated with Pasid P1 |
| 750 | * Time T2: passId = getPasId(vmid); vmid is associated with Pasid P2 | 729 | * Time T2: passId = getPasId(vmid); vmid is associated with Pasid P2 |
| 751 | * In the sequence above wave count obtained from time T1 will be incorrectly | 730 | * In the sequence above wave count obtained from time T1 will be incorrectly |
| @@ -888,10 +867,7 @@ const struct kfd2kgd_calls gfx_v9_kfd2kgd = { · 136 unmodified lines | |||
| 888 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, | 867 | .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied, |
| 889 | .hqd_destroy = kgd_gfx_v9_hqd_destroy, | 868 | .hqd_destroy = kgd_gfx_v9_hqd_destroy, |
| 890 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, | 869 | .hqd_sdma_destroy = kgd_hqd_sdma_destroy, |
| 891 | .address_watch_disable = kgd_gfx_v9_address_watch_disable, | ||
| 892 | .address_watch_execute = kgd_gfx_v9_address_watch_execute, | ||
| 893 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, | 870 | .wave_control_execute = kgd_gfx_v9_wave_control_execute, |
| 894 | .address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset, | ||
| 895 | .get_atc_vmid_pasid_mapping_info = | 871 | .get_atc_vmid_pasid_mapping_info = |
| 896 | kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, | 872 | kgd_gfx_v9_get_atc_vmid_pasid_mapping_info, |
| 897 | .set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base, | 873 | .set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base, |
| @@ -46,19 +46,9 @@ int kgd_gfx_v9_hqd_destroy(struct amdgpu_device *adev, void *mqd, · 45 unmodified lines | |||
| 46 | enum kfd_preempt_type reset_type, | 46 | enum kfd_preempt_type reset_type, |
| 47 | unsigned int utimeout, uint32_t pipe_id, | 47 | unsigned int utimeout, uint32_t pipe_id, |
| 48 | uint32_t queue_id); | 48 | uint32_t queue_id); |
| 49 | int kgd_gfx_v9_address_watch_disable(struct amdgpu_device *adev); | ||
| 50 | int kgd_gfx_v9_address_watch_execute(struct amdgpu_device *adev, | ||
| 51 | unsigned int watch_point_id, | ||
| 52 | uint32_t cntl_val, | ||
| 53 | uint32_t addr_hi, | ||
| 54 | uint32_t addr_lo); | ||
| 55 | int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, | 49 | int kgd_gfx_v9_wave_control_execute(struct amdgpu_device *adev, |
| 56 | uint32_t gfx_index_val, | 50 | uint32_t gfx_index_val, |
| 57 | uint32_t sq_cmd); | 51 | uint32_t sq_cmd); |
| 58 | uint32_t kgd_gfx_v9_address_watch_get_offset(struct amdgpu_device *adev, | ||
| 59 | unsigned int watch_point_id, | ||
| 60 | unsigned int reg_offset); | ||
| 61 | |||
| 62 | bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, | 52 | bool kgd_gfx_v9_get_atc_vmid_pasid_mapping_info(struct amdgpu_device *adev, |
| 63 | uint8_t vmid, uint16_t *p_pasid); | 53 | uint8_t vmid, uint16_t *p_pasid); |
| 64 | 54 | ||
| @@ -121,7 +121,7 @@ static size_t amdgpu_amdkfd_acc_size(uint64_t size) · 120 unmodified lines | |||
| 121 | } | 121 | } |
| 122 | 122 | ||
| 123 | /** | 123 | /** |
| 124 | * @amdgpu_amdkfd_reserve_mem_limit() - Decrease available memory by size | 124 | * amdgpu_amdkfd_reserve_mem_limit() - Decrease available memory by size |
| 125 | * of buffer including any reserved for control structures | 125 | * of buffer including any reserved for control structures |
| 126 | * | 126 | * |
| 127 | * @adev: Device to which allocated BO belongs to | 127 | * @adev: Device to which allocated BO belongs to |
| @@ -778,7 +778,7 @@ unwind: · 650 unmodified lines | |||
| 778 | continue; | 778 | continue; |
| 779 | if (attachment[i]->bo_va) { | 779 | if (attachment[i]->bo_va) { |
| 780 | amdgpu_bo_reserve(bo[i], true); | 780 | amdgpu_bo_reserve(bo[i], true); |
| 781 | amdgpu_vm_bo_rmv(adev, attachment[i]->bo_va); | 781 | amdgpu_vm_bo_del(adev, attachment[i]->bo_va); |
| 782 | amdgpu_bo_unreserve(bo[i]); | 782 | amdgpu_bo_unreserve(bo[i]); |
| 783 | list_del(&attachment[i]->list); | 783 | list_del(&attachment[i]->list); |
| 784 | } | 784 | } |
| @@ -795,7 +795,7 @@ static void kfd_mem_detach(struct kfd_mem_attachment *attachment) · 10 unmodified lines | |||
| 795 | 795 | ||
| 796 | pr_debug("\t remove VA 0x%llx in entry %p\n", | 796 | pr_debug("\t remove VA 0x%llx in entry %p\n", |
| 797 | attachment->va, attachment); | 797 | attachment->va, attachment); |
| 798 | amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); | 798 | amdgpu_vm_bo_del(attachment->adev, attachment->bo_va); |
| 799 | drm_gem_object_put(&bo->tbo.base); | 799 | drm_gem_object_put(&bo->tbo.base); |
| 800 | list_del(&attachment->list); | 800 | list_del(&attachment->list); |
| 801 | kfree(attachment); | 801 | kfree(attachment); |
| @@ -842,7 +842,8 @@ static void remove_kgd_mem_from_kfd_bo_list(struct kgd_mem *mem, · 40 unmodified lines | |||
| 842 | * | 842 | * |
| 843 | * Returns 0 for success, negative errno for errors. | 843 | * Returns 0 for success, negative errno for errors. |
| 844 | */ | 844 | */ |
| 845 | static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr) | 845 | static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr, |
| 846 | bool criu_resume) | ||
| 846 | { | 847 | { |
| 847 | struct amdkfd_process_info *process_info = mem->process_info; | 848 | struct amdkfd_process_info *process_info = mem->process_info; |
| 848 | struct amdgpu_bo *bo = mem->bo; | 849 | struct amdgpu_bo *bo = mem->bo; |
| @@ -864,6 +865,18 @@ static int init_user_pages(struct kgd_mem *mem, uint64_t user_addr) · 15 unmodified lines | |||
| 864 | goto out; | 865 | goto out; |
| 865 | } | 866 | } |
| 866 | 867 | ||
| 868 | if (criu_resume) { | ||
| 869 | /* | ||
| 870 | * During a CRIU restore operation, the userptr buffer objects | ||
| 871 | * will be validated in the restore_userptr_work worker at a | ||
| 872 | * later stage when it is scheduled by another ioctl called by | ||
| 873 | * CRIU master process for the target pid for restore. | ||
| 874 | */ | ||
| 875 | atomic_inc(&mem->invalid); | ||
| 876 | mutex_unlock(&process_info->lock); | ||
| 877 | return 0; | ||
| 878 | } | ||
| 879 | |||
| 867 | ret = amdgpu_ttm_tt_get_user_pages(bo, bo->tbo.ttm->pages); | 880 | ret = amdgpu_ttm_tt_get_user_pages(bo, bo->tbo.ttm->pages); |
| 868 | if (ret) { | 881 | if (ret) { |
| 869 | pr_err("%s: Failed to get user pages: %d\n", __func__, ret); | 882 | pr_err("%s: Failed to get user pages: %d\n", __func__, ret); |
| @@ -1452,10 +1465,39 @@ uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *drm_priv) · 582 unmodified lines | |||
| 1452 | return avm->pd_phys_addr; | 1465 | return avm->pd_phys_addr; |
| 1453 | } | 1466 | } |
| 1454 | 1467 | ||
| 1468 | void amdgpu_amdkfd_block_mmu_notifications(void *p) | ||
| 1469 | { | ||
| 1470 | struct amdkfd_process_info *pinfo = (struct amdkfd_process_info *)p; | ||
| 1471 | |||
| 1472 | mutex_lock(&pinfo->lock); | ||
| 1473 | WRITE_ONCE(pinfo->block_mmu_notifications, true); | ||
| 1474 | mutex_unlock(&pinfo->lock); | ||
| 1475 | } | ||
| 1476 | |||
| 1477 | int amdgpu_amdkfd_criu_resume(void *p) | ||
| 1478 | { | ||
| 1479 | int ret = 0; | ||
| 1480 | struct amdkfd_process_info *pinfo = (struct amdkfd_process_info *)p; | ||
| 1481 | |||
| 1482 | mutex_lock(&pinfo->lock); | ||
| 1483 | pr_debug("scheduling work\n"); | ||
| 1484 | atomic_inc(&pinfo->evicted_bos); | ||
| 1485 | if (!READ_ONCE(pinfo->block_mmu_notifications)) { | ||
| 1486 | ret = -EINVAL; | ||
| 1487 | goto out_unlock; | ||
| 1488 | } | ||
| 1489 | WRITE_ONCE(pinfo->block_mmu_notifications, false); | ||
| 1490 | schedule_delayed_work(&pinfo->restore_userptr_work, 0); | ||
| 1491 | |||
| 1492 | out_unlock: | ||
| 1493 | mutex_unlock(&pinfo->lock); | ||
| 1494 | return ret; | ||
| 1495 | } | ||
| 1496 | |||
| 1455 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( | 1497 | int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( |
| 1456 | struct amdgpu_device *adev, uint64_t va, uint64_t size, | 1498 | struct amdgpu_device *adev, uint64_t va, uint64_t size, |
| 1457 | void *drm_priv, struct kgd_mem **mem, | 1499 | void *drm_priv, struct kgd_mem **mem, |
| 1458 | uint64_t *offset, uint32_t flags) | 1500 | uint64_t *offset, uint32_t flags, bool criu_resume) |
| 1459 | { | 1501 | { |
| 1460 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); | 1502 | struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); |
| 1461 | enum ttm_bo_type bo_type = ttm_bo_type_device; | 1503 | enum ttm_bo_type bo_type = ttm_bo_type_device; |
| @@ -1558,7 +1600,8 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( · 96 unmodified lines | |||
| 1558 | add_kgd_mem_to_kfd_bo_list(*mem, avm->process_info, user_addr); | 1600 | add_kgd_mem_to_kfd_bo_list(*mem, avm->process_info, user_addr); |
| 1559 | 1601 | ||
| 1560 | if (user_addr) { | 1602 | if (user_addr) { |
| 1561 | ret = init_user_pages(*mem, user_addr); | 1603 | pr_debug("creating userptr BO for user_addr = %llu\n", user_addr); |
| 1604 | ret = init_user_pages(*mem, user_addr, criu_resume); | ||
| 1562 | if (ret) | 1605 | if (ret) |
| 1563 | goto allocate_init_user_pages_failed; | 1606 | goto allocate_init_user_pages_failed; |
| 1564 | } else if (flags & (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL | | 1607 | } else if (flags & (KFD_IOC_ALLOC_MEM_FLAGS_DOORBELL | |
| @@ -1813,12 +1856,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( · 248 unmodified lines | |||
| 1813 | true); | 1856 | true); |
| 1814 | ret = unreserve_bo_and_vms(&ctx, false, false); | 1857 | ret = unreserve_bo_and_vms(&ctx, false, false); |
| 1815 | 1858 | ||
| 1816 | /* Only apply no TLB flush on Aldebaran to | ||
| 1817 | * workaround regressions on other Asics. | ||
| 1818 | */ | ||
| 1819 | if (table_freed && (adev->asic_type != CHIP_ALDEBARAN)) | ||
| 1820 | *table_freed = true; | ||
| 1821 | |||
| 1822 | goto out; | 1859 | goto out; |
| 1823 | 1860 | ||
| 1824 | out_unreserve: | 1861 | out_unreserve: |
| @@ -2068,6 +2105,10 @@ int amdgpu_amdkfd_evict_userptr(struct kgd_mem *mem, · 243 unmodified lines | |||
| 2068 | int evicted_bos; | 2105 | int evicted_bos; |
| 2069 | int r = 0; | 2106 | int r = 0; |
| 2070 | 2107 | ||
| 2108 | /* Do not process MMU notifications until stage-4 IOCTL is received */ | ||
| 2109 | if (READ_ONCE(process_info->block_mmu_notifications)) | ||
| 2110 | return 0; | ||
| 2111 | |||
| 2071 | atomic_inc(&mem->invalid); | 2112 | atomic_inc(&mem->invalid); |
| 2072 | evicted_bos = atomic_inc_return(&process_info->evicted_bos); | 2113 | evicted_bos = atomic_inc_return(&process_info->evicted_bos); |
| 2073 | if (evicted_bos == 1) { | 2114 | if (evicted_bos == 1) { |
| @@ -2635,3 +2676,14 @@ int amdgpu_amdkfd_get_tile_config(struct amdgpu_device *adev, · 561 unmodified lines | |||
| 2635 | 2676 | ||
| 2636 | return 0; | 2677 | return 0; |
| 2637 | } | 2678 | } |
| 2679 | |||
| 2680 | bool amdgpu_amdkfd_bo_mapped_to_dev(struct amdgpu_device *adev, struct kgd_mem *mem) | ||
| 2681 | { | ||
| 2682 | struct kfd_mem_attachment *entry; | ||
| 2683 | |||
| 2684 | list_for_each_entry(entry, &mem->attachments, list) { | ||
| 2685 | if (entry->is_mapped && entry->adev == adev) | ||
| 2686 | return true; | ||
| 2687 | } | ||
| 2688 | return false; | ||
| 2689 | } | ||
| @@ -1083,6 +1083,7 @@ int amdgpu_atombios_get_clock_dividers(struct amdgpu_device *adev, · 1082 unmodified lines | |||
| 1083 | return 0; | 1083 | return 0; |
| 1084 | } | 1084 | } |
| 1085 | 1085 | ||
| 1086 | #ifdef CONFIG_DRM_AMDGPU_SI | ||
| 1086 | int amdgpu_atombios_get_memory_pll_dividers(struct amdgpu_device *adev, | 1087 | int amdgpu_atombios_get_memory_pll_dividers(struct amdgpu_device *adev, |
| 1087 | u32 clock, | 1088 | u32 clock, |
| 1088 | bool strobe_mode, | 1089 | bool strobe_mode, |
| @@ -1503,6 +1504,7 @@ int amdgpu_atombios_init_mc_reg_table(struct amdgpu_device *adev, · 414 unmodified lines | |||
| 1503 | } | 1504 | } |
| 1504 | return -EINVAL; | 1505 | return -EINVAL; |
| 1505 | } | 1506 | } |
| 1507 | #endif | ||
| 1506 | 1508 | ||
| 1507 | bool amdgpu_atombios_has_gpu_virtualization_table(struct amdgpu_device *adev) | 1509 | bool amdgpu_atombios_has_gpu_virtualization_table(struct amdgpu_device *adev) |
| 1508 | { | 1510 | { |
| @@ -160,6 +160,7 @@ int amdgpu_atombios_get_clock_dividers(struct amdgpu_device *adev, · 159 unmodified lines | |||
| 160 | bool strobe_mode, | 160 | bool strobe_mode, |
| 161 | struct atom_clock_dividers *dividers); | 161 | struct atom_clock_dividers *dividers); |
| 162 | 162 | ||
| 163 | #ifdef CONFIG_DRM_AMDGPU_SI | ||
| 163 | int amdgpu_atombios_get_memory_pll_dividers(struct amdgpu_device *adev, | 164 | int amdgpu_atombios_get_memory_pll_dividers(struct amdgpu_device *adev, |
| 164 | u32 clock, | 165 | u32 clock, |
| 165 | bool strobe_mode, | 166 | bool strobe_mode, |
| @@ -179,6 +180,17 @@ int amdgpu_atombios_get_voltage_table(struct amdgpu_device *adev, · 13 unmodified lines | |||
| 179 | int amdgpu_atombios_init_mc_reg_table(struct amdgpu_device *adev, | 180 | int amdgpu_atombios_init_mc_reg_table(struct amdgpu_device *adev, |
| 180 | u8 module_index, | 181 | u8 module_index, |
| 181 | struct atom_mc_reg_table *reg_table); | 182 | struct atom_mc_reg_table *reg_table); |
| 183 | int amdgpu_atombios_get_max_vddc(struct amdgpu_device *adev, u8 voltage_type, | ||
| 184 | u16 voltage_id, u16 *voltage); | ||
| 185 | int amdgpu_atombios_get_leakage_vddc_based_on_leakage_idx(struct amdgpu_device *adev, | ||
| 186 | u16 *voltage, | ||
| 187 | u16 leakage_idx); | ||
| 188 | void amdgpu_atombios_get_default_voltages(struct amdgpu_device *adev, | ||
| 189 | u16 *vddc, u16 *vddci, u16 *mvdd); | ||
| 190 | int amdgpu_atombios_get_svi2_info(struct amdgpu_device *adev, | ||
| 191 | u8 voltage_type, | ||
| 192 | u8 *svd_gpio_id, u8 *svc_gpio_id); | ||
| 193 | #endif | ||
| 182 | 194 | ||
| 183 | bool amdgpu_atombios_has_gpu_virtualization_table(struct amdgpu_device *adev); | 195 | bool amdgpu_atombios_has_gpu_virtualization_table(struct amdgpu_device *adev); |
| 184 | 196 | ||
| @@ -190,21 +202,11 @@ void amdgpu_atombios_scratch_regs_set_backlight_level(struct amdgpu_device *adev · 5 unmodified lines | |||
| 190 | bool amdgpu_atombios_scratch_need_asic_init(struct amdgpu_device *adev); | 202 | bool amdgpu_atombios_scratch_need_asic_init(struct amdgpu_device *adev); |
| 191 | 203 | ||
| 192 | void amdgpu_atombios_copy_swap(u8 *dst, u8 *src, u8 num_bytes, bool to_le); | 204 | void amdgpu_atombios_copy_swap(u8 *dst, u8 *src, u8 num_bytes, bool to_le); |
| 193 | int amdgpu_atombios_get_max_vddc(struct amdgpu_device *adev, u8 voltage_type, | ||
| 194 | u16 voltage_id, u16 *voltage); | ||
| 195 | int amdgpu_atombios_get_leakage_vddc_based_on_leakage_idx(struct amdgpu_device *adev, | ||
| 196 | u16 *voltage, | ||
| 197 | u16 leakage_idx); | ||
| 198 | void amdgpu_atombios_get_default_voltages(struct amdgpu_device *adev, | ||
| 199 | u16 *vddc, u16 *vddci, u16 *mvdd); | ||
| 200 | int amdgpu_atombios_get_clock_dividers(struct amdgpu_device *adev, | 205 | int amdgpu_atombios_get_clock_dividers(struct amdgpu_device *adev, |
| 201 | u8 clock_type, | 206 | u8 clock_type, |
| 202 | u32 clock, | 207 | u32 clock, |
| 203 | bool strobe_mode, | 208 | bool strobe_mode, |
| 204 | struct atom_clock_dividers *dividers); | 209 | struct atom_clock_dividers *dividers); |
| 205 | int amdgpu_atombios_get_svi2_info(struct amdgpu_device *adev, | ||
| 206 | u8 voltage_type, | ||
| 207 | u8 *svd_gpio_id, u8 *svc_gpio_id); | ||
| 208 | 210 | ||
| 209 | int amdgpu_atombios_get_data_table(struct amdgpu_device *adev, | 211 | int amdgpu_atombios_get_data_table(struct amdgpu_device *adev, |
| 210 | uint32_t table, | 212 | uint32_t table, |
| @@ -29,14 +29,13 @@ · 28 unmodified lines | |||
| 29 | #define AMDGPU_BENCHMARK_COMMON_MODES_N 17 | 29 | #define AMDGPU_BENCHMARK_COMMON_MODES_N 17 |
| 30 | 30 | ||
| 31 | static int amdgpu_benchmark_do_move(struct amdgpu_device *adev, unsigned size, | 31 | static int amdgpu_benchmark_do_move(struct amdgpu_device *adev, unsigned size, |
| 32 | uint64_t saddr, uint64_t daddr, int n) | 32 | uint64_t saddr, uint64_t daddr, int n, s64 *time_ms) |
| 33 | { | 33 | { |
| 34 | unsigned long start_jiffies; | 34 | ktime_t stime, etime; |
| 35 | unsigned long end_jiffies; | ||
| 36 | struct dma_fence *fence; | 35 | struct dma_fence *fence; |
| 37 | int i, r; | 36 | int i, r; |
| 38 | 37 | ||
| 39 | start_jiffies = jiffies; | 38 | stime = ktime_get(); |
| 40 | for (i = 0; i < n; i++) { | 39 | for (i = 0; i < n; i++) { |
| 41 | struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; | 40 | struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; |
| 42 | r = amdgpu_copy_buffer(ring, saddr, daddr, size, NULL, &fence, | 41 | r = amdgpu_copy_buffer(ring, saddr, daddr, size, NULL, &fence, |
| @@ -48,120 +47,81 @@ static int amdgpu_benchmark_do_move(struct amdgpu_device *adev, unsigned size, · 5 unmodified lines | |||
| 48 | if (r) | 47 | if (r) |
| 49 | goto exit_do_move; | 48 | goto exit_do_move; |
| 50 | } | 49 | } |
| 51 | end_jiffies = jiffies; | ||
| 52 | r = jiffies_to_msecs(end_jiffies - start_jiffies); | ||
| 53 | 50 | ||
| 54 | exit_do_move: | 51 | exit_do_move: |
| 52 | etime = ktime_get(); | ||
| 53 | *time_ms = ktime_ms_delta(etime, stime); | ||
| 54 | |||
| 55 | return r; | 55 | return r; |
| 56 | } | 56 | } |
| 57 | 57 | ||
| 58 | 58 | ||
| 59 | static void amdgpu_benchmark_log_results(int n, unsigned size, | 59 | static void amdgpu_benchmark_log_results(struct amdgpu_device *adev, |
| 60 | unsigned int time, | 60 | int n, unsigned size, |
| 61 | s64 time_ms, | ||
| 61 | unsigned sdomain, unsigned ddomain, | 62 | unsigned sdomain, unsigned ddomain, |
| 62 | char *kind) | 63 | char *kind) |
| 63 | { | 64 | { |
| 64 | unsigned int throughput = (n * (size >> 10)) / time; | 65 | s64 throughput = (n * (size >> 10)); |
| 65 | DRM_INFO("amdgpu: %s %u bo moves of %u kB from" | 66 | |
| 66 | " %d to %d in %u ms, throughput: %u Mb/s or %u MB/s\n", | 67 | throughput = div64_s64(throughput, time_ms); |
| 67 | kind, n, size >> 10, sdomain, ddomain, time, | 68 | |
| 69 | dev_info(adev->dev, "amdgpu: %s %u bo moves of %u kB from" | ||
| 70 | " %d to %d in %lld ms, throughput: %lld Mb/s or %lld MB/s\n", | ||
| 71 | kind, n, size >> 10, sdomain, ddomain, time_ms, | ||
| 68 | throughput * 8, throughput); | 72 | throughput * 8, throughput); |
| 69 | } | 73 | } |
| 70 | 74 | ||
| 71 | static void amdgpu_benchmark_move(struct amdgpu_device *adev, unsigned size, | 75 | static int amdgpu_benchmark_move(struct amdgpu_device *adev, unsigned size, |
| 72 | unsigned sdomain, unsigned ddomain) | 76 | unsigned sdomain, unsigned ddomain) |
| 73 | { | 77 | { |
| 74 | struct amdgpu_bo *dobj = NULL; | 78 | struct amdgpu_bo *dobj = NULL; |
| 75 | struct amdgpu_bo *sobj = NULL; | 79 | struct amdgpu_bo *sobj = NULL; |
| 76 | struct amdgpu_bo_param bp; | ||
| 77 | uint64_t saddr, daddr; | 80 | uint64_t saddr, daddr; |
| 81 | s64 time_ms; | ||
| 78 | int r, n; | 82 | int r, n; |
| 79 | int time; | ||
| 80 | |||
| 81 | memset(&bp, 0, sizeof(bp)); | ||
| 82 | bp.size = size; | ||
| 83 | bp.byte_align = PAGE_SIZE; | ||
| 84 | bp.domain = sdomain; | ||
| 85 | bp.flags = 0; | ||
| 86 | bp.type = ttm_bo_type_kernel; | ||
| 87 | bp.resv = NULL; | ||
| 88 | bp.bo_ptr_size = sizeof(struct amdgpu_bo); | ||
| 89 | 83 | ||
| 90 | n = AMDGPU_BENCHMARK_ITERATIONS; | 84 | n = AMDGPU_BENCHMARK_ITERATIONS; |
| 91 | r = amdgpu_bo_create(adev, &bp, &sobj); | 85 | |
| 92 | if (r) { | 86 | r = amdgpu_bo_create_kernel(adev, size, |
| 93 | goto out_cleanup; | 87 | PAGE_SIZE, sdomain, |
| 94 | } | 88 | &sobj, |
| 95 | r = amdgpu_bo_reserve(sobj, false); | 89 | &saddr, |
| 96 | if (unlikely(r != 0)) | 90 | NULL); |
| 97 | goto out_cleanup; | 91 | if (r) |
| 98 | r = amdgpu_bo_pin(sobj, sdomain); | ||
| 99 | if (r) { | ||
| 100 | amdgpu_bo_unreserve(sobj); | ||
| 101 | goto out_cleanup; | ||
| 102 | } | ||
| 103 | r = amdgpu_ttm_alloc_gart(&sobj->tbo); | ||
| 104 | amdgpu_bo_unreserve(sobj); | ||
| 105 | if (r) { | ||
| 106 | goto out_cleanup; | ||
| 107 | } | ||
| 108 | saddr = amdgpu_bo_gpu_offset(sobj); | ||
| 109 | bp.domain = ddomain; | ||
| 110 | r = amdgpu_bo_create(adev, &bp, &dobj); | ||
| 111 | if (r) { | ||
| 112 | goto out_cleanup; | ||
| 113 | } | ||
| 114 | r = amdgpu_bo_reserve(dobj, false); | ||
| 115 | if (unlikely(r != 0)) | ||
| 116 | goto out_cleanup; | ||
| 117 | r = amdgpu_bo_pin(dobj, ddomain); | ||
| 118 | if (r) { | ||
| 119 | amdgpu_bo_unreserve(sobj); | ||
| 120 | goto out_cleanup; | 92 | goto out_cleanup; |
| 121 | } | 93 | r = amdgpu_bo_create_kernel(adev, size, |
| 122 | r = amdgpu_ttm_alloc_gart(&dobj->tbo); | 94 | PAGE_SIZE, ddomain, |
| 123 | amdgpu_bo_unreserve(dobj); | 95 | &dobj, |
| 124 | if (r) { | 96 | &daddr, |
| 97 | NULL); | ||
| 98 | if (r) | ||
| 125 | goto out_cleanup; | 99 | goto out_cleanup; |
| 126 | } | ||
| 127 | daddr = amdgpu_bo_gpu_offset(dobj); | ||
| 128 | 100 | ||
| 129 | if (adev->mman.buffer_funcs) { | 101 | if (adev->mman.buffer_funcs) { |
| 130 | time = amdgpu_benchmark_do_move(adev, size, saddr, daddr, n); | 102 | r = amdgpu_benchmark_do_move(adev, size, saddr, daddr, n, &time_ms); |
| 131 | if (time < 0) | 103 | if (r) |
| 132 | goto out_cleanup; | 104 | goto out_cleanup; |
| 133 | if (time > 0) | 105 | else |
| 134 | amdgpu_benchmark_log_results(n, size, time, | 106 | amdgpu_benchmark_log_results(adev, n, size, time_ms, |
| 135 | sdomain, ddomain, "dma"); | 107 | sdomain, ddomain, "dma"); |
| 136 | } | 108 | } |
| 137 | 109 | ||
| 138 | out_cleanup: | 110 | out_cleanup: |
| 139 | /* Check error value now. The value can be overwritten when clean up.*/ | 111 | /* Check error value now. The value can be overwritten when clean up.*/ |
| 140 | if (r) { | 112 | if (r < 0) |
| 141 | DRM_ERROR("Error while benchmarking BO move.\n"); | 113 | dev_info(adev->dev, "Error while benchmarking BO move.\n"); |
| 142 | } | ||
| 143 | 114 | ||
| 144 | if (sobj) { | 115 | if (sobj) |
| 145 | r = amdgpu_bo_reserve(sobj, true); | 116 | amdgpu_bo_free_kernel(&sobj, &saddr, NULL); |
| 146 | if (likely(r == 0)) { | 117 | if (dobj) |
| 147 | amdgpu_bo_unpin(sobj); | 118 | amdgpu_bo_free_kernel(&dobj, &daddr, NULL); |
| 148 | amdgpu_bo_unreserve(sobj); | 119 | return r; |
| 149 | } | ||
| 150 | amdgpu_bo_unref(&sobj); | ||
| 151 | } | ||
| 152 | if (dobj) { | ||
| 153 | r = amdgpu_bo_reserve(dobj, true); | ||
| 154 | if (likely(r == 0)) { | ||
| 155 | amdgpu_bo_unpin(dobj); | ||
| 156 | amdgpu_bo_unreserve(dobj); | ||
| 157 | } | ||
| 158 | amdgpu_bo_unref(&dobj); | ||
| 159 | } | ||
| 160 | } | 120 | } |
| 161 | 121 | ||
| 162 | void amdgpu_benchmark(struct amdgpu_device *adev, int test_number) | 122 | int amdgpu_benchmark(struct amdgpu_device *adev, int test_number) |
| 163 | { | 123 | { |
| 164 | int i; | 124 | int i, r; |
| 165 | static const int common_modes[AMDGPU_BENCHMARK_COMMON_MODES_N] = { | 125 | static const int common_modes[AMDGPU_BENCHMARK_COMMON_MODES_N] = { |
| 166 | 640 * 480 * 4, | 126 | 640 * 480 * 4, |
| 167 | 720 * 480 * 4, | 127 | 720 * 480 * 4, |
| @@ -182,63 +142,119 @@ void amdgpu_benchmark(struct amdgpu_device *adev, int test_number) · 14 unmodified lines | |||
| 182 | 1920 * 1200 * 4 | 142 | 1920 * 1200 * 4 |
| 183 | }; | 143 | }; |
| 184 | 144 | ||
| 145 | mutex_lock(&adev->benchmark_mutex); | ||
| 185 | switch (test_number) { | 146 | switch (test_number) { |
| 186 | case 1: | 147 | case 1: |
| 148 | dev_info(adev->dev, | ||
| 149 | "benchmark test: %d (simple test, VRAM to GTT and GTT to VRAM)\n", | ||
| 150 | test_number); | ||
| 187 | /* simple test, VRAM to GTT and GTT to VRAM */ | 151 | /* simple test, VRAM to GTT and GTT to VRAM */ |
| 188 | amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_GTT, | 152 | r = amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_GTT, |
| 189 | AMDGPU_GEM_DOMAIN_VRAM); | 153 | AMDGPU_GEM_DOMAIN_VRAM); |
| 190 | amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_VRAM, | 154 | if (r) |
| 191 | AMDGPU_GEM_DOMAIN_GTT); | 155 | goto done; |
| 156 | r = amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_VRAM, | ||
| 157 | AMDGPU_GEM_DOMAIN_GTT); | ||
| 158 | if (r) | ||
| 159 | goto done; | ||
| 192 | break; | 160 | break; |
| 193 | case 2: | 161 | case 2: |
| 162 | dev_info(adev->dev, | ||
| 163 | "benchmark test: %d (simple test, VRAM to VRAM)\n", | ||
| 164 | test_number); | ||
| 194 | /* simple test, VRAM to VRAM */ | 165 | /* simple test, VRAM to VRAM */ |
| 195 | amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_VRAM, | 166 | r = amdgpu_benchmark_move(adev, 1024*1024, AMDGPU_GEM_DOMAIN_VRAM, |
| 196 | AMDGPU_GEM_DOMAIN_VRAM); | 167 | AMDGPU_GEM_DOMAIN_VRAM); |
| 168 | if (r) | ||
| 169 | goto done; | ||
| 197 | break; | 170 | break; |
| 198 | case 3: | 171 | case 3: |
| 172 | dev_info(adev->dev, | ||
| 173 | "benchmark test: %d (GTT to VRAM, buffer size sweep, powers of 2)\n", | ||
| 174 | test_number); | ||
| 199 | /* GTT to VRAM, buffer size sweep, powers of 2 */ | 175 | /* GTT to VRAM, buffer size sweep, powers of 2 */ |
| 200 | for (i = 1; i <= 16384; i <<= 1) | 176 | for (i = 1; i <= 16384; i <<= 1) { |
| 201 | amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, | 177 | r = amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, |
| 202 | AMDGPU_GEM_DOMAIN_GTT, | 178 | AMDGPU_GEM_DOMAIN_GTT, |
| 203 | AMDGPU_GEM_DOMAIN_VRAM); | 179 | AMDGPU_GEM_DOMAIN_VRAM); |
| 180 | if (r) | ||
| 181 | goto done; | ||
| 182 | } | ||
| 204 | break; | 183 | break; |
| 205 | case 4: | 184 | case 4: |
| 185 | dev_info(adev->dev, | ||
| 186 | "benchmark test: %d (VRAM to GTT, buffer size sweep, powers of 2)\n", | ||
| 187 | test_number); | ||
| 206 | /* VRAM to GTT, buffer size sweep, powers of 2 */ | 188 | /* VRAM to GTT, buffer size sweep, powers of 2 */ |
| 207 | for (i = 1; i <= 16384; i <<= 1) | 189 | for (i = 1; i <= 16384; i <<= 1) { |
| 208 | amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, | 190 | r = amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, |
| 209 | AMDGPU_GEM_DOMAIN_VRAM, | 191 | AMDGPU_GEM_DOMAIN_VRAM, |
| 210 | AMDGPU_GEM_DOMAIN_GTT); | 192 | AMDGPU_GEM_DOMAIN_GTT); |
| 193 | if (r) | ||
| 194 | goto done; | ||
| 195 | } | ||
| 211 | break; | 196 | break; |
| 212 | case 5: | 197 | case 5: |
| 198 | dev_info(adev->dev, | ||
| 199 | "benchmark test: %d (VRAM to VRAM, buffer size sweep, powers of 2)\n", | ||
| 200 | test_number); | ||
| 213 | /* VRAM to VRAM, buffer size sweep, powers of 2 */ | 201 | /* VRAM to VRAM, buffer size sweep, powers of 2 */ |
| 214 | for (i = 1; i <= 16384; i <<= 1) | 202 | for (i = 1; i <= 16384; i <<= 1) { |
| 215 | amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, | 203 | r = amdgpu_benchmark_move(adev, i * AMDGPU_GPU_PAGE_SIZE, |
| 216 | AMDGPU_GEM_DOMAIN_VRAM, | 204 | AMDGPU_GEM_DOMAIN_VRAM, |
| 217 | AMDGPU_GEM_DOMAIN_VRAM); | 205 | AMDGPU_GEM_DOMAIN_VRAM); |
| 206 | if (r) | ||
| 207 | goto done; | ||
| 208 | } | ||
| 218 | break; | 209 | break; |
| 219 | case 6: | 210 | case 6: |
| 211 | dev_info(adev->dev, | ||
| 212 | "benchmark test: %d (GTT to VRAM, buffer size sweep, common modes)\n", | ||
| 213 | test_number); | ||
| 220 | /* GTT to VRAM, buffer size sweep, common modes */ | 214 | /* GTT to VRAM, buffer size sweep, common modes */ |
| 221 | for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) | 215 | for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) { |
| 222 | amdgpu_benchmark_move(adev, common_modes[i], | 216 | r = amdgpu_benchmark_move(adev, common_modes[i], |
| 223 | AMDGPU_GEM_DOMAIN_GTT, | 217 | AMDGPU_GEM_DOMAIN_GTT, |
| 224 | AMDGPU_GEM_DOMAIN_VRAM); | 218 | AMDGPU_GEM_DOMAIN_VRAM); |
| 219 | if (r) | ||
| 220 | goto done; | ||
| 221 | } | ||
| 225 | break; | 222 | break; |
| 226 | case 7: | 223 | case 7: |
| 224 | dev_info(adev->dev, | ||
| 225 | "benchmark test: %d (VRAM to GTT, buffer size sweep, common modes)\n", | ||
| 226 | test_number); | ||
| 227 | /* VRAM to GTT, buffer size sweep, common modes */ | 227 | /* VRAM to GTT, buffer size sweep, common modes */ |
| 228 | for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) | 228 | for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) { |
| 229 | amdgpu_benchmark_move(adev, common_modes[i], | 229 | r = amdgpu_benchmark_move(adev, common_modes[i], |
| 230 | AMDGPU_GEM_DOMAIN_VRAM, | 230 | AMDGPU_GEM_DOMAIN_VRAM, |
| 231 | AMDGPU_GEM_DOMAIN_GTT); | 231 | AMDGPU_GEM_DOMAIN_GTT); |
| 232 | if (r) | ||
| 233 | goto done; | ||
| 234 | } | ||
| 232 | break; | 235 | break; |
| 233 | case 8: | 236 | case 8: |
| 237 | dev_info(adev->dev, | ||
| 238 | "benchmark test: %d (VRAM to VRAM, buffer size sweep, common modes)\n", | ||
| 239 | test_number); | ||
| 234 | /* VRAM to VRAM, buffer size sweep, common modes */ | 240 | /* VRAM to VRAM, buffer size sweep, common modes */ |
| 235 | for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) | 241 | for (i = 0; i < AMDGPU_BENCHMARK_COMMON_MODES_N; i++) { |
| 236 | amdgpu_benchmark_move(adev, common_modes[i], | 242 | r = amdgpu_benchmark_move(adev, common_modes[i], |
| 237 | AMDGPU_GEM_DOMAIN_VRAM, | 243 | AMDGPU_GEM_DOMAIN_VRAM, |
| 238 | AMDGPU_GEM_DOMAIN_VRAM); | 244 | AMDGPU_GEM_DOMAIN_VRAM); |
| 245 | if (r) | ||
| 246 | goto done; | ||
| 247 | } | ||
| 239 | break; | 248 | break; |
| 240 | 249 | ||
| 241 | default: | 250 | default: |
| 242 | DRM_ERROR("Unknown benchmark\n"); | 251 | dev_info(adev->dev, "Unknown benchmark %d\n", test_number); |
| 252 | r = -EINVAL; | ||
| 253 | break; | ||
| 243 | } | 254 | } |
| 255 | |||
| 256 | done: | ||
| 257 | mutex_unlock(&adev->benchmark_mutex); | ||
| 258 | |||
| 259 | return r; | ||
| 244 | } | 260 | } |
| @@ -464,3 +464,41 @@ success: · 463 unmodified lines | |||
| 464 | adev->is_atom_fw = (adev->asic_type >= CHIP_VEGA10) ? true : false; | 464 | adev->is_atom_fw = (adev->asic_type >= CHIP_VEGA10) ? true : false; |
| 465 | return true; | 465 | return true; |
| 466 | } | 466 | } |
| 467 | |||
| 468 | /* helper function for soc15 and onwards to read bios from rom */ | ||
| 469 | bool amdgpu_soc15_read_bios_from_rom(struct amdgpu_device *adev, | ||
| 470 | u8 *bios, u32 length_bytes) | ||
| 471 | { | ||
| 472 | u32 *dw_ptr; | ||
| 473 | u32 i, length_dw; | ||
| 474 | u32 rom_index_offset; | ||
| 475 | u32 rom_data_offset; | ||
| 476 | |||
| 477 | if (bios == NULL) | ||
| 478 | return false; | ||
| 479 | if (length_bytes == 0) | ||
| 480 | return false; | ||
| 481 | /* APU vbios image is part of sbios image */ | ||
| 482 | if (adev->flags & AMD_IS_APU) | ||
| 483 | return false; | ||
| 484 | if (!adev->smuio.funcs || | ||
| 485 | !adev->smuio.funcs->get_rom_index_offset || | ||
| 486 | !adev->smuio.funcs->get_rom_data_offset) | ||
| 487 | return false; | ||
| 488 | |||
| 489 | dw_ptr = (u32 *)bios; | ||
| 490 | length_dw = ALIGN(length_bytes, 4) / 4; | ||
| 491 | |||
| 492 | rom_index_offset = | ||
| 493 | adev->smuio.funcs->get_rom_index_offset(adev); | ||
| 494 | rom_data_offset = | ||
| 495 | adev->smuio.funcs->get_rom_data_offset(adev); | ||
| 496 | |||
| 497 | /* set rom index to 0 */ | ||
| 498 | WREG32(rom_index_offset, 0); | ||
| 499 | /* read out the rom data */ | ||
| 500 | for (i = 0; i < length_dw; i++) | ||
| 501 | dw_ptr[i] = RREG32(rom_data_offset); | ||
| 502 | |||
| 503 | return true; | ||
| 504 | } | ||
| @@ -127,8 +127,6 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs · 126 unmodified lines | |||
| 127 | goto free_chunk; | 127 | goto free_chunk; |
| 128 | } | 128 | } |
| 129 | 129 | ||
| 130 | mutex_lock(&p->ctx->lock); | ||
| 131 | |||
| 132 | /* skip guilty context job */ | 130 | /* skip guilty context job */ |
| 133 | if (atomic_read(&p->ctx->guilty) == 1) { | 131 | if (atomic_read(&p->ctx->guilty) == 1) { |
| 134 | ret = -ECANCELED; | 132 | ret = -ECANCELED; |
| @@ -314,7 +312,7 @@ static void amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev, · 179 unmodified lines | |||
| 314 | } | 312 | } |
| 315 | 313 | ||
| 316 | total_vram = adev->gmc.real_vram_size - atomic64_read(&adev->vram_pin_size); | 314 | total_vram = adev->gmc.real_vram_size - atomic64_read(&adev->vram_pin_size); |
| 317 | used_vram = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr); | 315 | used_vram = ttm_resource_manager_usage(&adev->mman.vram_mgr.manager); |
| 318 | free_vram = used_vram >= total_vram ? 0 : total_vram - used_vram; | 316 | free_vram = used_vram >= total_vram ? 0 : total_vram - used_vram; |
| 319 | 317 | ||
| 320 | spin_lock(&adev->mm_stats.lock); | 318 | spin_lock(&adev->mm_stats.lock); |
| @@ -585,6 +583,16 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, · 264 unmodified lines | |||
| 585 | } | 583 | } |
| 586 | } | 584 | } |
| 587 | 585 | ||
| 586 | /* Move fence waiting after getting reservation lock of | ||
| 587 | * PD root. Then there is no need on a ctx mutex lock. | ||
| 588 | */ | ||
| 589 | r = amdgpu_ctx_wait_prev_fence(p->ctx, p->entity); | ||
| 590 | if (unlikely(r != 0)) { | ||
| 591 | if (r != -ERESTARTSYS) | ||
| 592 | DRM_ERROR("amdgpu_ctx_wait_prev_fence failed.\n"); | ||
| 593 | goto error_validate; | ||
| 594 | } | ||
| 595 | |||
| 588 | amdgpu_cs_get_threshold_for_moves(p->adev, &p->bytes_moved_threshold, | 596 | amdgpu_cs_get_threshold_for_moves(p->adev, &p->bytes_moved_threshold, |
| 589 | &p->bytes_moved_vis_threshold); | 597 | &p->bytes_moved_vis_threshold); |
| 590 | p->bytes_moved = 0; | 598 | p->bytes_moved = 0; |
| @@ -700,7 +708,6 @@ static void amdgpu_cs_parser_fini(struct amdgpu_cs_parser *parser, int error, · 109 unmodified lines | |||
| 700 | dma_fence_put(parser->fence); | 708 | dma_fence_put(parser->fence); |
| 701 | 709 | ||
| 702 | if (parser->ctx) { | 710 | if (parser->ctx) { |
| 703 | mutex_unlock(&parser->ctx->lock); | ||
| 704 | amdgpu_ctx_put(parser->ctx); | 711 | amdgpu_ctx_put(parser->ctx); |
| 705 | } | 712 | } |
| 706 | if (parser->bo_list) | 713 | if (parser->bo_list) |
| @@ -944,7 +951,7 @@ static int amdgpu_cs_ib_fill(struct amdgpu_device *adev, · 237 unmodified lines | |||
| 944 | if (parser->job->uf_addr && ring->funcs->no_user_fence) | 951 | if (parser->job->uf_addr && ring->funcs->no_user_fence) |
| 945 | return -EINVAL; | 952 | return -EINVAL; |
| 946 | 953 | ||
| 947 | return amdgpu_ctx_wait_prev_fence(parser->ctx, parser->entity); | 954 | return 0; |
| 948 | } | 955 | } |
| 949 | 956 | ||
| 950 | static int amdgpu_cs_process_fence_dep(struct amdgpu_cs_parser *p, | 957 | static int amdgpu_cs_process_fence_dep(struct amdgpu_cs_parser *p, |
| @@ -1360,7 +1367,6 @@ int amdgpu_cs_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 409 unmodified lines | |||
| 1360 | goto out; | 1367 | goto out; |
| 1361 | 1368 | ||
| 1362 | r = amdgpu_cs_submit(&parser, cs); | 1369 | r = amdgpu_cs_submit(&parser, cs); |
| 1363 | |||
| 1364 | out: | 1370 | out: |
| 1365 | amdgpu_cs_parser_fini(&parser, r, reserved_buffers); | 1371 | amdgpu_cs_parser_fini(&parser, r, reserved_buffers); |
| 1366 | 1372 | ||
| @@ -1506,6 +1512,7 @@ int amdgpu_cs_fence_to_handle_ioctl(struct drm_device *dev, void *data, · 139 unmodified lines | |||
| 1506 | return 0; | 1512 | return 0; |
| 1507 | 1513 | ||
| 1508 | default: | 1514 | default: |
| 1515 | dma_fence_put(fence); | ||
| 1509 | return -EINVAL; | 1516 | return -EINVAL; |
| 1510 | } | 1517 | } |
| 1511 | } | 1518 | } |
| @@ -98,7 +98,7 @@ int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 97 unmodified lines | |||
| 98 | 98 | ||
| 99 | if (r) { | 99 | if (r) { |
| 100 | DRM_ERROR("failed to do bo_map on static CSA, err=%d\n", r); | 100 | DRM_ERROR("failed to do bo_map on static CSA, err=%d\n", r); |
| 101 | amdgpu_vm_bo_rmv(adev, *bo_va); | 101 | amdgpu_vm_bo_del(adev, *bo_va); |
| 102 | ttm_eu_backoff_reservation(&ticket, &list); | 102 | ttm_eu_backoff_reservation(&ticket, &list); |
| 103 | return r; | 103 | return r; |
| 104 | } | 104 | } |
| @@ -230,13 +230,13 @@ static int amdgpu_ctx_init(struct amdgpu_device *adev, · 229 unmodified lines | |||
| 230 | 230 | ||
| 231 | kref_init(&ctx->refcount); | 231 | kref_init(&ctx->refcount); |
| 232 | spin_lock_init(&ctx->ring_lock); | 232 | spin_lock_init(&ctx->ring_lock); |
| 233 | mutex_init(&ctx->lock); | ||
| 234 | 233 | ||
| 235 | ctx->reset_counter = atomic_read(&adev->gpu_reset_counter); | 234 | ctx->reset_counter = atomic_read(&adev->gpu_reset_counter); |
| 236 | ctx->reset_counter_query = ctx->reset_counter; | 235 | ctx->reset_counter_query = ctx->reset_counter; |
| 237 | ctx->vram_lost_counter = atomic_read(&adev->vram_lost_counter); | 236 | ctx->vram_lost_counter = atomic_read(&adev->vram_lost_counter); |
| 238 | ctx->init_priority = priority; | 237 | ctx->init_priority = priority; |
| 239 | ctx->override_priority = AMDGPU_CTX_PRIORITY_UNSET; | 238 | ctx->override_priority = AMDGPU_CTX_PRIORITY_UNSET; |
| 239 | ctx->stable_pstate = AMDGPU_CTX_STABLE_PSTATE_NONE; | ||
| 240 | 240 | ||
| 241 | return 0; | 241 | return 0; |
| 242 | } | 242 | } |
| @@ -255,6 +255,86 @@ static void amdgpu_ctx_fini_entity(struct amdgpu_ctx_entity *entity) · 12 unmodified lines | |||
| 255 | kfree(entity); | 255 | kfree(entity); |
| 256 | } | 256 | } |
| 257 | 257 | ||
| 258 | static int amdgpu_ctx_get_stable_pstate(struct amdgpu_ctx *ctx, | ||
| 259 | u32 *stable_pstate) | ||
| 260 | { | ||
| 261 | struct amdgpu_device *adev = ctx->adev; | ||
| 262 | enum amd_dpm_forced_level current_level; | ||
| 263 | |||
| 264 | if (!ctx) | ||
| 265 | return -EINVAL; | ||
| 266 | |||
| 267 | current_level = amdgpu_dpm_get_performance_level(adev); | ||
| 268 | |||
| 269 | switch (current_level) { | ||
| 270 | case AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD: | ||
| 271 | *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_STANDARD; | ||
| 272 | break; | ||
| 273 | case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK: | ||
| 274 | *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_MIN_SCLK; | ||
| 275 | break; | ||
| 276 | case AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK: | ||
| 277 | *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_MIN_MCLK; | ||
| 278 | break; | ||
| 279 | case AMD_DPM_FORCED_LEVEL_PROFILE_PEAK: | ||
| 280 | *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_PEAK; | ||
| 281 | break; | ||
| 282 | default: | ||
| 283 | *stable_pstate = AMDGPU_CTX_STABLE_PSTATE_NONE; | ||
| 284 | break; | ||
| 285 | } | ||
| 286 | return 0; | ||
| 287 | } | ||
| 288 | |||
| 289 | static int amdgpu_ctx_set_stable_pstate(struct amdgpu_ctx *ctx, | ||
| 290 | u32 stable_pstate) | ||
| 291 | { | ||
| 292 | struct amdgpu_device *adev = ctx->adev; | ||
| 293 | enum amd_dpm_forced_level level; | ||
| 294 | int r; | ||
| 295 | |||
| 296 | if (!ctx) | ||
| 297 | return -EINVAL; | ||
| 298 | |||
| 299 | mutex_lock(&adev->pm.stable_pstate_ctx_lock); | ||
| 300 | if (adev->pm.stable_pstate_ctx && adev->pm.stable_pstate_ctx != ctx) { | ||
| 301 | r = -EBUSY; | ||
| 302 | goto done; | ||
| 303 | } | ||
| 304 | |||
| 305 | switch (stable_pstate) { | ||
| 306 | case AMDGPU_CTX_STABLE_PSTATE_NONE: | ||
| 307 | level = AMD_DPM_FORCED_LEVEL_AUTO; | ||
| 308 | break; | ||
| 309 | case AMDGPU_CTX_STABLE_PSTATE_STANDARD: | ||
| 310 | level = AMD_DPM_FORCED_LEVEL_PROFILE_STANDARD; | ||
| 311 | break; | ||
| 312 | case AMDGPU_CTX_STABLE_PSTATE_MIN_SCLK: | ||
| 313 | level = AMD_DPM_FORCED_LEVEL_PROFILE_MIN_SCLK; | ||
| 314 | break; | ||
| 315 | case AMDGPU_CTX_STABLE_PSTATE_MIN_MCLK: | ||
| 316 | level = AMD_DPM_FORCED_LEVEL_PROFILE_MIN_MCLK; | ||
| 317 | break; | ||
| 318 | case AMDGPU_CTX_STABLE_PSTATE_PEAK: | ||
| 319 | level = AMD_DPM_FORCED_LEVEL_PROFILE_PEAK; | ||
| 320 | break; | ||
| 321 | default: | ||
| 322 | r = -EINVAL; | ||
| 323 | goto done; | ||
| 324 | } | ||
| 325 | |||
| 326 | r = amdgpu_dpm_force_performance_level(adev, level); | ||
| 327 | |||
| 328 | if (level == AMD_DPM_FORCED_LEVEL_AUTO) | ||
| 329 | adev->pm.stable_pstate_ctx = NULL; | ||
| 330 | else | ||
| 331 | adev->pm.stable_pstate_ctx = ctx; | ||
| 332 | done: | ||
| 333 | mutex_unlock(&adev->pm.stable_pstate_ctx_lock); | ||
| 334 | |||
| 335 | return r; | ||
| 336 | } | ||
| 337 | |||
| 258 | static void amdgpu_ctx_fini(struct kref *ref) | 338 | static void amdgpu_ctx_fini(struct kref *ref) |
| 259 | { | 339 | { |
| 260 | struct amdgpu_ctx *ctx = container_of(ref, struct amdgpu_ctx, refcount); | 340 | struct amdgpu_ctx *ctx = container_of(ref, struct amdgpu_ctx, refcount); |
| @@ -270,8 +350,7 @@ static void amdgpu_ctx_fini(struct kref *ref) · 9 unmodified lines | |||
| 270 | ctx->entities[i][j] = NULL; | 350 | ctx->entities[i][j] = NULL; |
| 271 | } | 351 | } |
| 272 | } | 352 | } |
| 273 | 353 | amdgpu_ctx_set_stable_pstate(ctx, AMDGPU_CTX_STABLE_PSTATE_NONE); | |
| 274 | mutex_destroy(&ctx->lock); | ||
| 275 | kfree(ctx); | 354 | kfree(ctx); |
| 276 | } | 355 | } |
| 277 | 356 | ||
| @@ -467,11 +546,41 @@ static int amdgpu_ctx_query2(struct amdgpu_device *adev, · 189 unmodified lines | |||
| 467 | return 0; | 546 | return 0; |
| 468 | } | 547 | } |
| 469 | 548 | ||
| 549 | |||
| 550 | |||
| 551 | static int amdgpu_ctx_stable_pstate(struct amdgpu_device *adev, | ||
| 552 | struct amdgpu_fpriv *fpriv, uint32_t id, | ||
| 553 | bool set, u32 *stable_pstate) | ||
| 554 | { | ||
| 555 | struct amdgpu_ctx *ctx; | ||
| 556 | struct amdgpu_ctx_mgr *mgr; | ||
| 557 | int r; | ||
| 558 | |||
| 559 | if (!fpriv) | ||
| 560 | return -EINVAL; | ||
| 561 | |||
| 562 | mgr = &fpriv->ctx_mgr; | ||
| 563 | mutex_lock(&mgr->lock); | ||
| 564 | ctx = idr_find(&mgr->ctx_handles, id); | ||
| 565 | if (!ctx) { | ||
| 566 | mutex_unlock(&mgr->lock); | ||
| 567 | return -EINVAL; | ||
| 568 | } | ||
| 569 | |||
| 570 | if (set) | ||
| 571 | r = amdgpu_ctx_set_stable_pstate(ctx, *stable_pstate); | ||
| 572 | else | ||
| 573 | r = amdgpu_ctx_get_stable_pstate(ctx, stable_pstate); | ||
| 574 | |||
| 575 | mutex_unlock(&mgr->lock); | ||
| 576 | return r; | ||
| 577 | } | ||
| 578 | |||
| 470 | int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, | 579 | int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, |
| 471 | struct drm_file *filp) | 580 | struct drm_file *filp) |
| 472 | { | 581 | { |
| 473 | int r; | 582 | int r; |
| 474 | uint32_t id; | 583 | uint32_t id, stable_pstate; |
| 475 | int32_t priority; | 584 | int32_t priority; |
| 476 | 585 | ||
| 477 | union drm_amdgpu_ctx *args = data; | 586 | union drm_amdgpu_ctx *args = data; |
| @@ -500,6 +609,21 @@ int amdgpu_ctx_ioctl(struct drm_device *dev, void *data, · 22 unmodified lines | |||
| 500 | case AMDGPU_CTX_OP_QUERY_STATE2: | 609 | case AMDGPU_CTX_OP_QUERY_STATE2: |
| 501 | r = amdgpu_ctx_query2(adev, fpriv, id, &args->out); | 610 | r = amdgpu_ctx_query2(adev, fpriv, id, &args->out); |
| 502 | break; | 611 | break; |
| 612 | case AMDGPU_CTX_OP_GET_STABLE_PSTATE: | ||
| 613 | if (args->in.flags) | ||
| 614 | return -EINVAL; | ||
| 615 | r = amdgpu_ctx_stable_pstate(adev, fpriv, id, false, &stable_pstate); | ||
| 616 | if (!r) | ||
| 617 | args->out.pstate.flags = stable_pstate; | ||
| 618 | break; | ||
| 619 | case AMDGPU_CTX_OP_SET_STABLE_PSTATE: | ||
| 620 | if (args->in.flags & ~AMDGPU_CTX_STABLE_PSTATE_FLAGS_MASK) | ||
| 621 | return -EINVAL; | ||
| 622 | stable_pstate = args->in.flags & AMDGPU_CTX_STABLE_PSTATE_FLAGS_MASK; | ||
| 623 | if (stable_pstate > AMDGPU_CTX_STABLE_PSTATE_PEAK) | ||
| 624 | return -EINVAL; | ||
| 625 | r = amdgpu_ctx_stable_pstate(adev, fpriv, id, true, &stable_pstate); | ||
| 626 | break; | ||
| 503 | default: | 627 | default: |
| 504 | return -EINVAL; | 628 | return -EINVAL; |
| 505 | } | 629 | } |
| @@ -49,10 +49,10 @@ struct amdgpu_ctx { · 48 unmodified lines | |||
| 49 | bool preamble_presented; | 49 | bool preamble_presented; |
| 50 | int32_t init_priority; | 50 | int32_t init_priority; |
| 51 | int32_t override_priority; | 51 | int32_t override_priority; |
| 52 | struct mutex lock; | ||
| 53 | atomic_t guilty; | 52 | atomic_t guilty; |
| 54 | unsigned long ras_counter_ce; | 53 | unsigned long ras_counter_ce; |
| 55 | unsigned long ras_counter_ue; | 54 | unsigned long ras_counter_ue; |
| 55 | uint32_t stable_pstate; | ||
| 56 | }; | 56 | }; |
| 57 | 57 | ||
| 58 | struct amdgpu_ctx_mgr { | 58 | struct amdgpu_ctx_mgr { |
| @@ -37,6 +37,8 @@ · 36 unmodified lines | |||
| 37 | #include "amdgpu_fw_attestation.h" | 37 | #include "amdgpu_fw_attestation.h" |
| 38 | #include "amdgpu_umr.h" | 38 | #include "amdgpu_umr.h" |
| 39 | 39 | ||
| 40 | #include "amdgpu_reset.h" | ||
| 41 | |||
| 40 | #if defined(CONFIG_DEBUG_FS) | 42 | #if defined(CONFIG_DEBUG_FS) |
| 41 | 43 | ||
| 42 | /** | 44 | /** |
| @@ -728,7 +730,7 @@ static ssize_t amdgpu_debugfs_gca_config_read(struct file *f, char __user *buf, · 685 unmodified lines | |||
| 728 | return -ENOMEM; | 730 | return -ENOMEM; |
| 729 | 731 | ||
| 730 | /* version, increment each time something is added */ | 732 | /* version, increment each time something is added */ |
| 731 | config[no_regs++] = 3; | 733 | config[no_regs++] = 4; |
| 732 | config[no_regs++] = adev->gfx.config.max_shader_engines; | 734 | config[no_regs++] = adev->gfx.config.max_shader_engines; |
| 733 | config[no_regs++] = adev->gfx.config.max_tile_pipes; | 735 | config[no_regs++] = adev->gfx.config.max_tile_pipes; |
| 734 | config[no_regs++] = adev->gfx.config.max_cu_per_sh; | 736 | config[no_regs++] = adev->gfx.config.max_cu_per_sh; |
| @@ -768,6 +770,9 @@ static ssize_t amdgpu_debugfs_gca_config_read(struct file *f, char __user *buf, · 33 unmodified lines | |||
| 768 | config[no_regs++] = adev->pdev->subsystem_device; | 770 | config[no_regs++] = adev->pdev->subsystem_device; |
| 769 | config[no_regs++] = adev->pdev->subsystem_vendor; | 771 | config[no_regs++] = adev->pdev->subsystem_vendor; |
| 770 | 772 | ||
| 773 | /* rev==4 APU flag */ | ||
| 774 | config[no_regs++] = adev->flags & AMD_IS_APU ? 1 : 0; | ||
| 775 | |||
| 771 | while (size && (*pos < no_regs * 4)) { | 776 | while (size && (*pos < no_regs * 4)) { |
| 772 | uint32_t value; | 777 | uint32_t value; |
| 773 | 778 | ||
| @@ -1120,8 +1125,10 @@ static ssize_t amdgpu_debugfs_gfxoff_read(struct file *f, char __user *buf, · 346 unmodified lines | |||
| 1120 | return -EINVAL; | 1125 | return -EINVAL; |
| 1121 | 1126 | ||
| 1122 | r = pm_runtime_get_sync(adev_to_drm(adev)->dev); | 1127 | r = pm_runtime_get_sync(adev_to_drm(adev)->dev); |
| 1123 | if (r < 0) | 1128 | if (r < 0) { |
| 1129 | pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); | ||
| 1124 | return r; | 1130 | return r; |
| 1131 | } | ||
| 1125 | 1132 | ||
| 1126 | while (size) { | 1133 | while (size) { |
| 1127 | uint32_t value; | 1134 | uint32_t value; |
| @@ -1279,7 +1286,7 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · 151 unmodified lines | |||
| 1279 | } | 1286 | } |
| 1280 | 1287 | ||
| 1281 | /* Avoid accidently unparking the sched thread during GPU reset */ | 1288 | /* Avoid accidently unparking the sched thread during GPU reset */ |
| 1282 | r = down_write_killable(&adev->reset_sem); | 1289 | r = down_write_killable(&adev->reset_domain->sem); |
| 1283 | if (r) | 1290 | if (r) |
| 1284 | return r; | 1291 | return r; |
| 1285 | 1292 | ||
| @@ -1308,7 +1315,7 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · 22 unmodified lines | |||
| 1308 | kthread_unpark(ring->sched.thread); | 1315 | kthread_unpark(ring->sched.thread); |
| 1309 | } | 1316 | } |
| 1310 | 1317 | ||
| 1311 | up_write(&adev->reset_sem); | 1318 | up_write(&adev->reset_domain->sem); |
| 1312 | 1319 | ||
| 1313 | pm_runtime_mark_last_busy(dev->dev); | 1320 | pm_runtime_mark_last_busy(dev->dev); |
| 1314 | pm_runtime_put_autosuspend(dev->dev); | 1321 | pm_runtime_put_autosuspend(dev->dev); |
| @@ -1357,6 +1364,25 @@ static int amdgpu_debugfs_evict_gtt(void *data, u64 *val) · 42 unmodified lines | |||
| 1357 | return 0; | 1364 | return 0; |
| 1358 | } | 1365 | } |
| 1359 | 1366 | ||
| 1367 | static int amdgpu_debugfs_benchmark(void *data, u64 val) | ||
| 1368 | { | ||
| 1369 | struct amdgpu_device *adev = (struct amdgpu_device *)data; | ||
| 1370 | struct drm_device *dev = adev_to_drm(adev); | ||
| 1371 | int r; | ||
| 1372 | |||
| 1373 | r = pm_runtime_get_sync(dev->dev); | ||
| 1374 | if (r < 0) { | ||
| 1375 | pm_runtime_put_autosuspend(dev->dev); | ||
| 1376 | return r; | ||
| 1377 | } | ||
| 1378 | |||
| 1379 | r = amdgpu_benchmark(adev, val); | ||
| 1380 | |||
| 1381 | pm_runtime_mark_last_busy(dev->dev); | ||
| 1382 | pm_runtime_put_autosuspend(dev->dev); | ||
| 1383 | |||
| 1384 | return r; | ||
| 1385 | } | ||
| 1360 | 1386 | ||
| 1361 | static int amdgpu_debugfs_vm_info_show(struct seq_file *m, void *unused) | 1387 | static int amdgpu_debugfs_vm_info_show(struct seq_file *m, void *unused) |
| 1362 | { | 1388 | { |
| @@ -1393,6 +1419,8 @@ DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_evict_vram_fops, amdgpu_debugfs_evict_vram, · 30 unmodified lines | |||
| 1393 | NULL, "%lld\n"); | 1419 | NULL, "%lld\n"); |
| 1394 | DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_evict_gtt_fops, amdgpu_debugfs_evict_gtt, | 1420 | DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_evict_gtt_fops, amdgpu_debugfs_evict_gtt, |
| 1395 | NULL, "%lld\n"); | 1421 | NULL, "%lld\n"); |
| 1422 | DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_benchmark_fops, NULL, amdgpu_debugfs_benchmark, | ||
| 1423 | "%lld\n"); | ||
| 1396 | 1424 | ||
| 1397 | static void amdgpu_ib_preempt_fences_swap(struct amdgpu_ring *ring, | 1425 | static void amdgpu_ib_preempt_fences_swap(struct amdgpu_ring *ring, |
| 1398 | struct dma_fence **fences) | 1426 | struct dma_fence **fences) |
| @@ -1517,7 +1545,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) · 118 unmodified lines | |||
| 1517 | return -ENOMEM; | 1545 | return -ENOMEM; |
| 1518 | 1546 | ||
| 1519 | /* Avoid accidently unparking the sched thread during GPU reset */ | 1547 | /* Avoid accidently unparking the sched thread during GPU reset */ |
| 1520 | r = down_read_killable(&adev->reset_sem); | 1548 | r = down_read_killable(&adev->reset_domain->sem); |
| 1521 | if (r) | 1549 | if (r) |
| 1522 | goto pro_end; | 1550 | goto pro_end; |
| 1523 | 1551 | ||
| @@ -1560,7 +1588,7 @@ failure: · 36 unmodified lines | |||
| 1560 | /* restart the scheduler */ | 1588 | /* restart the scheduler */ |
| 1561 | kthread_unpark(ring->sched.thread); | 1589 | kthread_unpark(ring->sched.thread); |
| 1562 | 1590 | ||
| 1563 | up_read(&adev->reset_sem); | 1591 | up_read(&adev->reset_domain->sem); |
| 1564 | 1592 | ||
| 1565 | ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, resched); | 1593 | ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, resched); |
| 1566 | 1594 | ||
| @@ -1585,22 +1613,25 @@ static int amdgpu_debugfs_sclk_set(void *data, u64 val) · 18 unmodified lines | |||
| 1585 | return ret; | 1613 | return ret; |
| 1586 | } | 1614 | } |
| 1587 | 1615 | ||
| 1588 | if (is_support_sw_smu(adev)) { | 1616 | ret = amdgpu_dpm_get_dpm_freq_range(adev, PP_SCLK, &min_freq, &max_freq); |
| 1589 | ret = smu_get_dpm_freq_range(&adev->smu, SMU_SCLK, &min_freq, &max_freq); | 1617 | if (ret == -EOPNOTSUPP) { |
| 1590 | if (ret || val > max_freq || val < min_freq) | 1618 | ret = 0; |
| 1591 | return -EINVAL; | 1619 | goto out; |
| 1592 | ret = smu_set_soft_freq_range(&adev->smu, SMU_SCLK, (uint32_t)val, (uint32_t)val); | ||
| 1593 | } else { | ||
| 1594 | return 0; | ||
| 1595 | } | 1620 | } |
| 1621 | if (ret || val > max_freq || val < min_freq) { | ||
| 1622 | ret = -EINVAL; | ||
| 1623 | goto out; | ||
| 1624 | } | ||
| 1625 | |||
| 1626 | ret = amdgpu_dpm_set_soft_freq_range(adev, PP_SCLK, (uint32_t)val, (uint32_t)val); | ||
| 1627 | if (ret) | ||
| 1628 | ret = -EINVAL; | ||
| 1596 | 1629 | ||
| 1630 | out: | ||
| 1597 | pm_runtime_mark_last_busy(adev_to_drm(adev)->dev); | 1631 | pm_runtime_mark_last_busy(adev_to_drm(adev)->dev); |
| 1598 | pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); | 1632 | pm_runtime_put_autosuspend(adev_to_drm(adev)->dev); |
| 1599 | 1633 | ||
| 1600 | if (ret) | 1634 | return ret; |
| 1601 | return -EINVAL; | ||
| 1602 | |||
| 1603 | return 0; | ||
| 1604 | } | 1635 | } |
| 1605 | 1636 | ||
| 1606 | DEFINE_DEBUGFS_ATTRIBUTE(fops_ib_preempt, NULL, | 1637 | DEFINE_DEBUGFS_ATTRIBUTE(fops_ib_preempt, NULL, |
| @@ -1609,6 +1640,86 @@ DEFINE_DEBUGFS_ATTRIBUTE(fops_ib_preempt, NULL, · 2 unmodified lines | |||
| 1609 | DEFINE_DEBUGFS_ATTRIBUTE(fops_sclk_set, NULL, | 1640 | DEFINE_DEBUGFS_ATTRIBUTE(fops_sclk_set, NULL, |
| 1610 | amdgpu_debugfs_sclk_set, "%llu\n"); | 1641 | amdgpu_debugfs_sclk_set, "%llu\n"); |
| 1611 | 1642 | ||
| 1643 | static ssize_t amdgpu_reset_dump_register_list_read(struct file *f, | ||
| 1644 | char __user *buf, size_t size, loff_t *pos) | ||
| 1645 | { | ||
| 1646 | struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(f)->i_private; | ||
| 1647 | char reg_offset[12]; | ||
| 1648 | int i, ret, len = 0; | ||
| 1649 | |||
| 1650 | if (*pos) | ||
| 1651 | return 0; | ||
| 1652 | |||
| 1653 | memset(reg_offset, 0, 12); | ||
| 1654 | ret = down_read_killable(&adev->reset_domain->sem); | ||
| 1655 | if (ret) | ||
| 1656 | return ret; | ||
| 1657 | |||
| 1658 | for (i = 0; i < adev->num_regs; i++) { | ||
| 1659 | sprintf(reg_offset, "0x%x\n", adev->reset_dump_reg_list[i]); | ||
| 1660 | up_read(&adev->reset_domain->sem); | ||
| 1661 | if (copy_to_user(buf + len, reg_offset, strlen(reg_offset))) | ||
| 1662 | return -EFAULT; | ||
| 1663 | |||
| 1664 | len += strlen(reg_offset); | ||
| 1665 | ret = down_read_killable(&adev->reset_domain->sem); | ||
| 1666 | if (ret) | ||
| 1667 | return ret; | ||
| 1668 | } | ||
| 1669 | |||
| 1670 | up_read(&adev->reset_domain->sem); | ||
| 1671 | *pos += len; | ||
| 1672 | |||
| 1673 | return len; | ||
| 1674 | } | ||
| 1675 | |||
| 1676 | static ssize_t amdgpu_reset_dump_register_list_write(struct file *f, | ||
| 1677 | const char __user *buf, size_t size, loff_t *pos) | ||
| 1678 | { | ||
| 1679 | struct amdgpu_device *adev = (struct amdgpu_device *)file_inode(f)->i_private; | ||
| 1680 | char reg_offset[11]; | ||
| 1681 | uint32_t *tmp; | ||
| 1682 | int ret, i = 0, len = 0; | ||
| 1683 | |||
| 1684 | do { | ||
| 1685 | memset(reg_offset, 0, 11); | ||
| 1686 | if (copy_from_user(reg_offset, buf + len, | ||
| 1687 | min(10, ((int)size-len)))) { | ||
| 1688 | ret = -EFAULT; | ||
| 1689 | goto error_free; | ||
| 1690 | } | ||
| 1691 | |||
| 1692 | tmp = krealloc_array(tmp, i + 1, sizeof(uint32_t), GFP_KERNEL); | ||
| 1693 | if (sscanf(reg_offset, "%X %n", &tmp[i], &ret) != 1) { | ||
| 1694 | ret = -EINVAL; | ||
| 1695 | goto error_free; | ||
| 1696 | } | ||
| 1697 | |||
| 1698 | len += ret; | ||
| 1699 | i++; | ||
| 1700 | } while (len < size); | ||
| 1701 | |||
| 1702 | ret = down_write_killable(&adev->reset_domain->sem); | ||
| 1703 | if (ret) | ||
| 1704 | goto error_free; | ||
| 1705 | |||
| 1706 | swap(adev->reset_dump_reg_list, tmp); | ||
| 1707 | adev->num_regs = i; | ||
| 1708 | up_write(&adev->reset_domain->sem); | ||
| 1709 | ret = size; | ||
| 1710 | |||
| 1711 | error_free: | ||
| 1712 | kfree(tmp); | ||
| 1713 | return ret; | ||
| 1714 | } | ||
| 1715 | |||
| 1716 | static const struct file_operations amdgpu_reset_dump_register_list = { | ||
| 1717 | .owner = THIS_MODULE, | ||
| 1718 | .read = amdgpu_reset_dump_register_list_read, | ||
| 1719 | .write = amdgpu_reset_dump_register_list_write, | ||
| 1720 | .llseek = default_llseek | ||
| 1721 | }; | ||
| 1722 | |||
| 1612 | int amdgpu_debugfs_init(struct amdgpu_device *adev) | 1723 | int amdgpu_debugfs_init(struct amdgpu_device *adev) |
| 1613 | { | 1724 | { |
| 1614 | struct dentry *root = adev_to_drm(adev)->primary->debugfs_root; | 1725 | struct dentry *root = adev_to_drm(adev)->primary->debugfs_root; |
| @@ -1675,6 +1786,10 @@ int amdgpu_debugfs_init(struct amdgpu_device *adev) · 60 unmodified lines | |||
| 1675 | &amdgpu_debugfs_test_ib_fops); | 1786 | &amdgpu_debugfs_test_ib_fops); |
| 1676 | debugfs_create_file("amdgpu_vm_info", 0444, root, adev, | 1787 | debugfs_create_file("amdgpu_vm_info", 0444, root, adev, |
| 1677 | &amdgpu_debugfs_vm_info_fops); | 1788 | &amdgpu_debugfs_vm_info_fops); |
| 1789 | debugfs_create_file("amdgpu_benchmark", 0200, root, adev, | ||
| 1790 | &amdgpu_benchmark_fops); | ||
| 1791 | debugfs_create_file("amdgpu_reset_dump_register_list", 0644, root, adev, | ||
| 1792 | &amdgpu_reset_dump_register_list); | ||
| 1678 | 1793 | ||
| 1679 | adev->debugfs_vbios_blob.data = adev->bios; | 1794 | adev->debugfs_vbios_blob.data = adev->bios; |
| 1680 | adev->debugfs_vbios_blob.size = adev->bios_size; | 1795 | adev->debugfs_vbios_blob.size = adev->bios_size; |
| @@ -31,6 +31,7 @@ · 30 unmodified lines | |||
| 31 | #include <linux/console.h> | 31 | #include <linux/console.h> |
| 32 | #include <linux/slab.h> | 32 | #include <linux/slab.h> |
| 33 | #include <linux/iommu.h> | 33 | #include <linux/iommu.h> |
| 34 | #include <linux/pci.h> | ||
| 34 | 35 | ||
| 35 | #include <drm/drm_atomic_helper.h> | 36 | #include <drm/drm_atomic_helper.h> |
| 36 | #include <drm/drm_probe_helper.h> | 37 | #include <drm/drm_probe_helper.h> |
| @@ -55,7 +56,6 @@ · 18 unmodified lines | |||
| 55 | #include "soc15.h" | 56 | #include "soc15.h" |
| 56 | #include "nv.h" | 57 | #include "nv.h" |
| 57 | #include "bif/bif_4_1_d.h" | 58 | #include "bif/bif_4_1_d.h" |
| 58 | #include <linux/pci.h> | ||
| 59 | #include <linux/firmware.h> | 59 | #include <linux/firmware.h> |
| 60 | #include "amdgpu_vf_error.h" | 60 | #include "amdgpu_vf_error.h" |
| 61 | 61 | ||
| @@ -88,6 +88,8 @@ MODULE_FIRMWARE("amdgpu/vangogh_gpu_info.bin"); · 26 unmodified lines | |||
| 88 | MODULE_FIRMWARE("amdgpu/yellow_carp_gpu_info.bin"); | 88 | MODULE_FIRMWARE("amdgpu/yellow_carp_gpu_info.bin"); |
| 89 | 89 | ||
| 90 | #define AMDGPU_RESUME_MS 2000 | 90 | #define AMDGPU_RESUME_MS 2000 |
| 91 | #define AMDGPU_MAX_RETRY_LIMIT 2 | ||
| 92 | #define AMDGPU_RETRY_SRIOV_RESET(r) ((r) == -EBUSY || (r) == -ETIMEDOUT || (r) == -EINVAL) | ||
| 91 | 93 | ||
| 92 | const char *amdgpu_asic_name[] = { | 94 | const char *amdgpu_asic_name[] = { |
| 93 | "TAHITI", | 95 | "TAHITI", |
| @@ -424,10 +426,10 @@ bool amdgpu_device_skip_hw_access(struct amdgpu_device *adev) · 330 unmodified lines | |||
| 424 | * the lock. | 426 | * the lock. |
| 425 | */ | 427 | */ |
| 426 | if (in_task()) { | 428 | if (in_task()) { |
| 427 | if (down_read_trylock(&adev->reset_sem)) | 429 | if (down_read_trylock(&adev->reset_domain->sem)) |
| 428 | up_read(&adev->reset_sem); | 430 | up_read(&adev->reset_domain->sem); |
| 429 | else | 431 | else |
| 430 | lockdep_assert_held(&adev->reset_sem); | 432 | lockdep_assert_held(&adev->reset_domain->sem); |
| 431 | } | 433 | } |
| 432 | #endif | 434 | #endif |
| 433 | return false; | 435 | return false; |
| @@ -453,9 +455,9 @@ uint32_t amdgpu_device_rreg(struct amdgpu_device *adev, · 19 unmodified lines | |||
| 453 | if ((reg * 4) < adev->rmmio_size) { | 455 | if ((reg * 4) < adev->rmmio_size) { |
| 454 | if (!(acc_flags & AMDGPU_REGS_NO_KIQ) && | 456 | if (!(acc_flags & AMDGPU_REGS_NO_KIQ) && |
| 455 | amdgpu_sriov_runtime(adev) && | 457 | amdgpu_sriov_runtime(adev) && |
| 456 | down_read_trylock(&adev->reset_sem)) { | 458 | down_read_trylock(&adev->reset_domain->sem)) { |
| 457 | ret = amdgpu_kiq_rreg(adev, reg); | 459 | ret = amdgpu_kiq_rreg(adev, reg); |
| 458 | up_read(&adev->reset_sem); | 460 | up_read(&adev->reset_domain->sem); |
| 459 | } else { | 461 | } else { |
| 460 | ret = readl(((void __iomem *)adev->rmmio) + (reg * 4)); | 462 | ret = readl(((void __iomem *)adev->rmmio) + (reg * 4)); |
| 461 | } | 463 | } |
| @@ -538,9 +540,9 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, · 76 unmodified lines | |||
| 538 | if ((reg * 4) < adev->rmmio_size) { | 540 | if ((reg * 4) < adev->rmmio_size) { |
| 539 | if (!(acc_flags & AMDGPU_REGS_NO_KIQ) && | 541 | if (!(acc_flags & AMDGPU_REGS_NO_KIQ) && |
| 540 | amdgpu_sriov_runtime(adev) && | 542 | amdgpu_sriov_runtime(adev) && |
| 541 | down_read_trylock(&adev->reset_sem)) { | 543 | down_read_trylock(&adev->reset_domain->sem)) { |
| 542 | amdgpu_kiq_wreg(adev, reg, v); | 544 | amdgpu_kiq_wreg(adev, reg, v); |
| 543 | up_read(&adev->reset_sem); | 545 | up_read(&adev->reset_domain->sem); |
| 544 | } else { | 546 | } else { |
| 545 | writel(v, ((void __iomem *)adev->rmmio) + (reg * 4)); | 547 | writel(v, ((void __iomem *)adev->rmmio) + (reg * 4)); |
| 546 | } | 548 | } |
| @@ -554,7 +556,11 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, · 7 unmodified lines | |||
| 554 | /** | 556 | /** |
| 555 | * amdgpu_mm_wreg_mmio_rlc - write register either with direct/indirect mmio or with RLC path if in range | 557 | * amdgpu_mm_wreg_mmio_rlc - write register either with direct/indirect mmio or with RLC path if in range |
| 556 | * | 558 | * |
| 557 | * this function is invoked only the debugfs register access | 559 | * @adev: amdgpu_device pointer |
| 560 | * @reg: mmio/rlc register | ||
| 561 | * @v: value to write | ||
| 562 | * | ||
| 563 | * this function is invoked only for the debugfs register access | ||
| 558 | */ | 564 | */ |
| 559 | void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, | 565 | void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, |
| 560 | uint32_t reg, uint32_t v) | 566 | uint32_t reg, uint32_t v) |
| @@ -566,7 +572,7 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, · 5 unmodified lines | |||
| 566 | adev->gfx.rlc.funcs && | 572 | adev->gfx.rlc.funcs && |
| 567 | adev->gfx.rlc.funcs->is_rlcg_access_range) { | 573 | adev->gfx.rlc.funcs->is_rlcg_access_range) { |
| 568 | if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg)) | 574 | if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg)) |
| 569 | return adev->gfx.rlc.funcs->sriov_wreg(adev, reg, v, 0, 0); | 575 | return amdgpu_sriov_wreg(adev, reg, v, 0, 0); |
| 570 | } else if ((reg * 4) >= adev->rmmio_size) { | 576 | } else if ((reg * 4) >= adev->rmmio_size) { |
| 571 | adev->pcie_wreg(adev, reg * 4, v); | 577 | adev->pcie_wreg(adev, reg * 4, v); |
| 572 | } else { | 578 | } else { |
| @@ -1312,6 +1318,31 @@ bool amdgpu_device_need_post(struct amdgpu_device *adev) · 739 unmodified lines | |||
| 1312 | return true; | 1318 | return true; |
| 1313 | } | 1319 | } |
| 1314 | 1320 | ||
| 1321 | /** | ||
| 1322 | * amdgpu_device_should_use_aspm - check if the device should program ASPM | ||
| 1323 | * | ||
| 1324 | * @adev: amdgpu_device pointer | ||
| 1325 | * | ||
| 1326 | * Confirm whether the module parameter and pcie bridge agree that ASPM should | ||
| 1327 | * be set for this device. | ||
| 1328 | * | ||
| 1329 | * Returns true if it should be used or false if not. | ||
| 1330 | */ | ||
| 1331 | bool amdgpu_device_should_use_aspm(struct amdgpu_device *adev) | ||
| 1332 | { | ||
| 1333 | switch (amdgpu_aspm) { | ||
| 1334 | case -1: | ||
| 1335 | break; | ||
| 1336 | case 0: | ||
| 1337 | return false; | ||
| 1338 | case 1: | ||
| 1339 | return true; | ||
| 1340 | default: | ||
| 1341 | return false; | ||
| 1342 | } | ||
| 1343 | return pcie_aspm_enabled(adev->pdev); | ||
| 1344 | } | ||
| 1345 | |||
| 1315 | /* if we get transitioned to only one device, take VGA back */ | 1346 | /* if we get transitioned to only one device, take VGA back */ |
| 1316 | /** | 1347 | /** |
| 1317 | * amdgpu_device_vga_set_decode - enable/disable vga decode | 1348 | * amdgpu_device_vga_set_decode - enable/disable vga decode |
| @@ -1446,7 +1477,8 @@ static int amdgpu_device_init_apu_flags(struct amdgpu_device *adev) · 128 unmodified lines | |||
| 1446 | case CHIP_YELLOW_CARP: | 1477 | case CHIP_YELLOW_CARP: |
| 1447 | break; | 1478 | break; |
| 1448 | case CHIP_CYAN_SKILLFISH: | 1479 | case CHIP_CYAN_SKILLFISH: |
| 1449 | if (adev->pdev->device == 0x13FE) | 1480 | if ((adev->pdev->device == 0x13FE) || |
| 1481 | (adev->pdev->device == 0x143F)) | ||
| 1450 | adev->apu_flags |= AMD_APU_IS_CYAN_SKILLFISH2; | 1482 | adev->apu_flags |= AMD_APU_IS_CYAN_SKILLFISH2; |
| 1451 | break; | 1483 | break; |
| 1452 | default: | 1484 | default: |
| @@ -1507,6 +1539,11 @@ static int amdgpu_device_check_arguments(struct amdgpu_device *adev) · 54 unmodified lines | |||
| 1507 | amdgpu_sched_hw_submission = roundup_pow_of_two(amdgpu_sched_hw_submission); | 1539 | amdgpu_sched_hw_submission = roundup_pow_of_two(amdgpu_sched_hw_submission); |
| 1508 | } | 1540 | } |
| 1509 | 1541 | ||
| 1542 | if (amdgpu_reset_method < -1 || amdgpu_reset_method > 4) { | ||
| 1543 | dev_warn(adev->dev, "invalid option for reset method, reverting to default\n"); | ||
| 1544 | amdgpu_reset_method = -1; | ||
| 1545 | } | ||
| 1546 | |||
| 1510 | amdgpu_device_check_smu_prv_buffer_size(adev); | 1547 | amdgpu_device_check_smu_prv_buffer_size(adev); |
| 1511 | 1548 | ||
| 1512 | amdgpu_device_check_vm_size(adev); | 1549 | amdgpu_device_check_vm_size(adev); |
| @@ -2073,6 +2110,8 @@ out: · 560 unmodified lines | |||
| 2073 | */ | 2110 | */ |
| 2074 | static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) | 2111 | static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) |
| 2075 | { | 2112 | { |
| 2113 | struct drm_device *dev = adev_to_drm(adev); | ||
| 2114 | struct pci_dev *parent; | ||
| 2076 | int i, r; | 2115 | int i, r; |
| 2077 | 2116 | ||
| 2078 | amdgpu_device_enable_virtual_display(adev); | 2117 | amdgpu_device_enable_virtual_display(adev); |
| @@ -2137,6 +2176,16 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev) · 58 unmodified lines | |||
| 2137 | break; | 2176 | break; |
| 2138 | } | 2177 | } |
| 2139 | 2178 | ||
| 2179 | if (amdgpu_has_atpx() && | ||
| 2180 | (amdgpu_is_atpx_hybrid() || | ||
| 2181 | amdgpu_has_atpx_dgpu_power_cntl()) && | ||
| 2182 | ((adev->flags & AMD_IS_APU) == 0) && | ||
| 2183 | !pci_is_thunderbolt_attached(to_pci_dev(dev->dev))) | ||
| 2184 | adev->flags |= AMD_IS_PX; | ||
| 2185 | |||
| 2186 | parent = pci_upstream_bridge(adev->pdev); | ||
| 2187 | adev->has_pr3 = parent ? pci_pr3_present(parent) : false; | ||
| 2188 | |||
| 2140 | amdgpu_amdkfd_device_probe(adev); | 2189 | amdgpu_amdkfd_device_probe(adev); |
| 2141 | 2190 | ||
| 2142 | adev->pm.pp_feature = amdgpu_pp_feature_mask; | 2191 | adev->pm.pp_feature = amdgpu_pp_feature_mask; |
| @@ -2287,6 +2336,49 @@ static int amdgpu_device_fw_loading(struct amdgpu_device *adev) · 144 unmodified lines | |||
| 2287 | return r; | 2336 | return r; |
| 2288 | } | 2337 | } |
| 2289 | 2338 | ||
| 2339 | static int amdgpu_device_init_schedulers(struct amdgpu_device *adev) | ||
| 2340 | { | ||
| 2341 | long timeout; | ||
| 2342 | int r, i; | ||
| 2343 | |||
| 2344 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | ||
| 2345 | struct amdgpu_ring *ring = adev->rings[i]; | ||
| 2346 | |||
| 2347 | /* No need to setup the GPU scheduler for rings that don't need it */ | ||
| 2348 | if (!ring || ring->no_scheduler) | ||
| 2349 | continue; | ||
| 2350 | |||
| 2351 | switch (ring->funcs->type) { | ||
| 2352 | case AMDGPU_RING_TYPE_GFX: | ||
| 2353 | timeout = adev->gfx_timeout; | ||
| 2354 | break; | ||
| 2355 | case AMDGPU_RING_TYPE_COMPUTE: | ||
| 2356 | timeout = adev->compute_timeout; | ||
| 2357 | break; | ||
| 2358 | case AMDGPU_RING_TYPE_SDMA: | ||
| 2359 | timeout = adev->sdma_timeout; | ||
| 2360 | break; | ||
| 2361 | default: | ||
| 2362 | timeout = adev->video_timeout; | ||
| 2363 | break; | ||
| 2364 | } | ||
| 2365 | |||
| 2366 | r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, | ||
| 2367 | ring->num_hw_submission, amdgpu_job_hang_limit, | ||
| 2368 | timeout, adev->reset_domain->wq, | ||
| 2369 | ring->sched_score, ring->name, | ||
| 2370 | adev->dev); | ||
| 2371 | if (r) { | ||
| 2372 | DRM_ERROR("Failed to create scheduler on ring %s.\n", | ||
| 2373 | ring->name); | ||
| 2374 | return r; | ||
| 2375 | } | ||
| 2376 | } | ||
| 2377 | |||
| 2378 | return 0; | ||
| 2379 | } | ||
| 2380 | |||
| 2381 | |||
| 2290 | /** | 2382 | /** |
| 2291 | * amdgpu_device_ip_init - run init for hardware IPs | 2383 | * amdgpu_device_ip_init - run init for hardware IPs |
| 2292 | * | 2384 | * |
| @@ -2398,8 +2490,28 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev) · 105 unmodified lines | |||
| 2398 | if (r) | 2490 | if (r) |
| 2399 | goto init_failed; | 2491 | goto init_failed; |
| 2400 | 2492 | ||
| 2401 | if (adev->gmc.xgmi.num_physical_nodes > 1) | 2493 | /** |
| 2402 | amdgpu_xgmi_add_device(adev); | 2494 | * In case of XGMI grab extra reference for reset domain for this device |
| 2495 | */ | ||
| 2496 | if (adev->gmc.xgmi.num_physical_nodes > 1) { | ||
| 2497 | if (amdgpu_xgmi_add_device(adev) == 0) { | ||
| 2498 | struct amdgpu_hive_info *hive = amdgpu_get_xgmi_hive(adev); | ||
| 2499 | |||
| 2500 | if (!hive->reset_domain || | ||
| 2501 | !amdgpu_reset_get_reset_domain(hive->reset_domain)) { | ||
| 2502 | r = -ENOENT; | ||
| 2503 | goto init_failed; | ||
| 2504 | } | ||
| 2505 | |||
| 2506 | /* Drop the early temporary reset domain we created for device */ | ||
| 2507 | amdgpu_reset_put_reset_domain(adev->reset_domain); | ||
| 2508 | adev->reset_domain = hive->reset_domain; | ||
| 2509 | } | ||
| 2510 | } | ||
| 2511 | |||
| 2512 | r = amdgpu_device_init_schedulers(adev); | ||
| 2513 | if (r) | ||
| 2514 | goto init_failed; | ||
| 2403 | 2515 | ||
| 2404 | /* Don't init kfd if whole hive need to be reset during init */ | 2516 | /* Don't init kfd if whole hive need to be reset during init */ |
| 2405 | if (!adev->gmc.xgmi.pending_reset) | 2517 | if (!adev->gmc.xgmi.pending_reset) |
| @@ -2610,6 +2722,12 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) · 204 unmodified lines | |||
| 2610 | adev->ip_blocks[i].status.late_initialized = true; | 2722 | adev->ip_blocks[i].status.late_initialized = true; |
| 2611 | } | 2723 | } |
| 2612 | 2724 | ||
| 2725 | r = amdgpu_ras_late_init(adev); | ||
| 2726 | if (r) { | ||
| 2727 | DRM_ERROR("amdgpu_ras_late_init failed %d", r); | ||
| 2728 | return r; | ||
| 2729 | } | ||
| 2730 | |||
| 2613 | amdgpu_ras_set_error_query_ready(adev, true); | 2731 | amdgpu_ras_set_error_query_ready(adev, true); |
| 2614 | 2732 | ||
| 2615 | amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE); | 2733 | amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE); |
| @@ -2624,7 +2742,7 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) · 8 unmodified lines | |||
| 2624 | /* For passthrough configuration on arcturus and aldebaran, enable special handling SBR */ | 2742 | /* For passthrough configuration on arcturus and aldebaran, enable special handling SBR */ |
| 2625 | if (amdgpu_passthrough(adev) && ((adev->asic_type == CHIP_ARCTURUS && adev->gmc.xgmi.num_physical_nodes > 1)|| | 2743 | if (amdgpu_passthrough(adev) && ((adev->asic_type == CHIP_ARCTURUS && adev->gmc.xgmi.num_physical_nodes > 1)|| |
| 2626 | adev->asic_type == CHIP_ALDEBARAN )) | 2744 | adev->asic_type == CHIP_ALDEBARAN )) |
| 2627 | smu_handle_passthrough_sbr(&adev->smu, true); | 2745 | amdgpu_dpm_handle_passthrough_sbr(adev, true); |
| 2628 | 2746 | ||
| 2629 | if (adev->gmc.xgmi.num_physical_nodes > 1) { | 2747 | if (adev->gmc.xgmi.num_physical_nodes > 1) { |
| 2630 | mutex_lock(&mgpu_info.mutex); | 2748 | mutex_lock(&mgpu_info.mutex); |
| @@ -2708,11 +2826,11 @@ static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) · 77 unmodified lines | |||
| 2708 | } | 2826 | } |
| 2709 | } | 2827 | } |
| 2710 | 2828 | ||
| 2711 | amdgpu_amdkfd_suspend(adev, false); | ||
| 2712 | |||
| 2713 | amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); | 2829 | amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE); |
| 2714 | amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); | 2830 | amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE); |
| 2715 | 2831 | ||
| 2832 | amdgpu_amdkfd_suspend(adev, false); | ||
| 2833 | |||
| 2716 | /* Workaroud for ASICs need to disable SMC first */ | 2834 | /* Workaroud for ASICs need to disable SMC first */ |
| 2717 | amdgpu_device_smu_fini_early(adev); | 2835 | amdgpu_device_smu_fini_early(adev); |
| 2718 | 2836 | ||
| @@ -2881,7 +2999,7 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev) · 162 unmodified lines | |||
| 2881 | int i, r; | 2999 | int i, r; |
| 2882 | 3000 | ||
| 2883 | if (adev->in_s0ix) | 3001 | if (adev->in_s0ix) |
| 2884 | amdgpu_gfx_state_change_set(adev, sGpuChangeState_D3Entry); | 3002 | amdgpu_dpm_gfx_state_change(adev, sGpuChangeState_D3Entry); |
| 2885 | 3003 | ||
| 2886 | for (i = adev->num_ip_blocks - 1; i >= 0; i--) { | 3004 | for (i = adev->num_ip_blocks - 1; i >= 0; i--) { |
| 2887 | if (!adev->ip_blocks[i].status.valid) | 3005 | if (!adev->ip_blocks[i].status.valid) |
| @@ -3307,9 +3425,9 @@ static void amdgpu_device_xgmi_reset_func(struct work_struct *__work) · 419 unmodified lines | |||
| 3307 | if (adev->asic_reset_res) | 3425 | if (adev->asic_reset_res) |
| 3308 | goto fail; | 3426 | goto fail; |
| 3309 | 3427 | ||
| 3310 | if (adev->mmhub.ras_funcs && | 3428 | if (adev->mmhub.ras && adev->mmhub.ras->ras_block.hw_ops && |
| 3311 | adev->mmhub.ras_funcs->reset_ras_error_count) | 3429 | adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count) |
| 3312 | adev->mmhub.ras_funcs->reset_ras_error_count(adev); | 3430 | adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count(adev); |
| 3313 | } else { | 3431 | } else { |
| 3314 | 3432 | ||
| 3315 | task_barrier_full(&hive->tb); | 3433 | task_barrier_full(&hive->tb); |
| @@ -3493,12 +3611,12 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 177 unmodified lines | |||
| 3493 | mutex_init(&adev->mn_lock); | 3611 | mutex_init(&adev->mn_lock); |
| 3494 | mutex_init(&adev->virt.vf_errors.lock); | 3612 | mutex_init(&adev->virt.vf_errors.lock); |
| 3495 | hash_init(adev->mn_hash); | 3613 | hash_init(adev->mn_hash); |
| 3496 | atomic_set(&adev->in_gpu_reset, 0); | ||
| 3497 | init_rwsem(&adev->reset_sem); | ||
| 3498 | mutex_init(&adev->psp.mutex); | 3614 | mutex_init(&adev->psp.mutex); |
| 3499 | mutex_init(&adev->notifier_lock); | 3615 | mutex_init(&adev->notifier_lock); |
| 3616 | mutex_init(&adev->pm.stable_pstate_ctx_lock); | ||
| 3617 | mutex_init(&adev->benchmark_mutex); | ||
| 3500 | 3618 | ||
| 3501 | amdgpu_device_init_apu_flags(adev); | 3619 | amdgpu_device_init_apu_flags(adev); |
| 3502 | 3620 | ||
| 3503 | r = amdgpu_device_check_arguments(adev); | 3621 | r = amdgpu_device_check_arguments(adev); |
| 3504 | if (r) | 3622 | if (r) |
| @@ -3519,6 +3637,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 14 unmodified lines | |||
| 3519 | 3637 | ||
| 3520 | INIT_LIST_HEAD(&adev->reset_list); | 3638 | INIT_LIST_HEAD(&adev->reset_list); |
| 3521 | 3639 | ||
| 3640 | INIT_LIST_HEAD(&adev->ras_list); | ||
| 3641 | |||
| 3522 | INIT_DELAYED_WORK(&adev->delayed_init_work, | 3642 | INIT_DELAYED_WORK(&adev->delayed_init_work, |
| 3523 | amdgpu_device_delayed_init_work_handler); | 3643 | amdgpu_device_delayed_init_work_handler); |
| 3524 | INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work, | 3644 | INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work, |
| @@ -3577,6 +3697,15 @@ int amdgpu_device_init(struct amdgpu_device *adev, · 52 unmodified lines | |||
| 3577 | return r; | 3697 | return r; |
| 3578 | } | 3698 | } |
| 3579 | 3699 | ||
| 3700 | /* | ||
| 3701 | * Reset domain needs to be present early, before XGMI hive discovered | ||
| 3702 | * (if any) and intitialized to use reset sem and in_gpu reset flag | ||
| 3703 | * early on during init. | ||
| 3704 | */ | ||
| 3705 | adev->reset_domain = amdgpu_reset_create_reset_domain(SINGLE_DEVICE ,"amdgpu-reset-dev"); | ||
| 3706 | if (!adev->reset_domain) | ||
| 3707 | return -ENOMEM; | ||
| 3708 | |||
| 3580 | /* early init functions */ | 3709 | /* early init functions */ |
| 3581 | r = amdgpu_device_ip_early_init(adev); | 3710 | r = amdgpu_device_ip_early_init(adev); |
| 3582 | if (r) | 3711 | if (r) |
| @@ -3749,19 +3878,6 @@ fence_driver_init: · 166 unmodified lines | |||
| 3749 | } else | 3878 | } else |
| 3750 | adev->ucode_sysfs_en = true; | 3879 | adev->ucode_sysfs_en = true; |
| 3751 | 3880 | ||
| 3752 | if ((amdgpu_testing & 1)) { | ||
| 3753 | if (adev->accel_working) | ||
| 3754 | amdgpu_test_moves(adev); | ||
| 3755 | else | ||
| 3756 | DRM_INFO("amdgpu: acceleration disabled, skipping move tests\n"); | ||
| 3757 | } | ||
| 3758 | if (amdgpu_benchmarking) { | ||
| 3759 | if (adev->accel_working) | ||
| 3760 | amdgpu_benchmark(adev, amdgpu_benchmarking); | ||
| 3761 | else | ||
| 3762 | DRM_INFO("amdgpu: acceleration disabled, skipping benchmarks\n"); | ||
| 3763 | } | ||
| 3764 | |||
| 3765 | /* | 3881 | /* |
| 3766 | * Register gpu instance before amdgpu_device_enable_mgpu_fan_boost. | 3882 | * Register gpu instance before amdgpu_device_enable_mgpu_fan_boost. |
| 3767 | * Otherwise the mgpu fan boost feature will be skipped due to the | 3883 | * Otherwise the mgpu fan boost feature will be skipped due to the |
| @@ -3953,6 +4069,9 @@ void amdgpu_device_fini_sw(struct amdgpu_device *adev) · 185 unmodified lines | |||
| 3953 | if (adev->mman.discovery_bin) | 4069 | if (adev->mman.discovery_bin) |
| 3954 | amdgpu_discovery_fini(adev); | 4070 | amdgpu_discovery_fini(adev); |
| 3955 | 4071 | ||
| 4072 | amdgpu_reset_put_reset_domain(adev->reset_domain); | ||
| 4073 | adev->reset_domain = NULL; | ||
| 4074 | |||
| 3956 | kfree(adev->pci_state); | 4075 | kfree(adev->pci_state); |
| 3957 | 4076 | ||
| 3958 | } | 4077 | } |
| @@ -4044,7 +4163,7 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon) · 85 unmodified lines | |||
| 4044 | return 0; | 4163 | return 0; |
| 4045 | 4164 | ||
| 4046 | if (adev->in_s0ix) | 4165 | if (adev->in_s0ix) |
| 4047 | amdgpu_gfx_state_change_set(adev, sGpuChangeState_D0Entry); | 4166 | amdgpu_dpm_gfx_state_change(adev, sGpuChangeState_D0Entry); |
| 4048 | 4167 | ||
| 4049 | /* post card */ | 4168 | /* post card */ |
| 4050 | if (amdgpu_device_need_post(adev)) { | 4169 | if (amdgpu_device_need_post(adev)) { |
| @@ -4347,7 +4466,9 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev, · 296 unmodified lines | |||
| 4347 | { | 4466 | { |
| 4348 | int r; | 4467 | int r; |
| 4349 | struct amdgpu_hive_info *hive = NULL; | 4468 | struct amdgpu_hive_info *hive = NULL; |
| 4469 | int retry_limit = 0; | ||
| 4350 | 4470 | ||
| 4471 | retry: | ||
| 4351 | amdgpu_amdkfd_pre_reset(adev); | 4472 | amdgpu_amdkfd_pre_reset(adev); |
| 4352 | 4473 | ||
| 4353 | amdgpu_amdkfd_pre_reset(adev); | 4474 | amdgpu_amdkfd_pre_reset(adev); |
| @@ -4396,6 +4517,14 @@ error: · 42 unmodified lines | |||
| 4396 | } | 4517 | } |
| 4397 | amdgpu_virt_release_full_gpu(adev, true); | 4518 | amdgpu_virt_release_full_gpu(adev, true); |
| 4398 | 4519 | ||
| 4520 | if (AMDGPU_RETRY_SRIOV_RESET(r)) { | ||
| 4521 | if (retry_limit < AMDGPU_MAX_RETRY_LIMIT) { | ||
| 4522 | retry_limit++; | ||
| 4523 | goto retry; | ||
| 4524 | } else | ||
| 4525 | DRM_ERROR("GPU reset retry is beyond the retry limit\n"); | ||
| 4526 | } | ||
| 4527 | |||
| 4399 | return r; | 4528 | return r; |
| 4400 | } | 4529 | } |
| 4401 | 4530 | ||
| @@ -4587,6 +4716,22 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, · 185 unmodified lines | |||
| 4587 | return r; | 4716 | return r; |
| 4588 | } | 4717 | } |
| 4589 | 4718 | ||
| 4719 | static int amdgpu_reset_reg_dumps(struct amdgpu_device *adev) | ||
| 4720 | { | ||
| 4721 | uint32_t reg_value; | ||
| 4722 | int i; | ||
| 4723 | |||
| 4724 | lockdep_assert_held(&adev->reset_domain->sem); | ||
| 4725 | dump_stack(); | ||
| 4726 | |||
| 4727 | for (i = 0; i < adev->num_regs; i++) { | ||
| 4728 | reg_value = RREG32(adev->reset_dump_reg_list[i]); | ||
| 4729 | trace_amdgpu_reset_reg_dumps(adev->reset_dump_reg_list[i], reg_value); | ||
| 4730 | } | ||
| 4731 | |||
| 4732 | return 0; | ||
| 4733 | } | ||
| 4734 | |||
| 4590 | int amdgpu_do_asic_reset(struct list_head *device_list_handle, | 4735 | int amdgpu_do_asic_reset(struct list_head *device_list_handle, |
| 4591 | struct amdgpu_reset_context *reset_context) | 4736 | struct amdgpu_reset_context *reset_context) |
| 4592 | { | 4737 | { |
| @@ -4597,6 +4742,7 @@ int amdgpu_do_asic_reset(struct list_head *device_list_handle, · 4 unmodified lines | |||
| 4597 | /* Try reset handler method first */ | 4742 | /* Try reset handler method first */ |
| 4598 | tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device, | 4743 | tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device, |
| 4599 | reset_list); | 4744 | reset_list); |
| 4745 | amdgpu_reset_reg_dumps(tmp_adev); | ||
| 4600 | r = amdgpu_reset_perform_reset(tmp_adev, reset_context); | 4746 | r = amdgpu_reset_perform_reset(tmp_adev, reset_context); |
| 4601 | /* If reset handler not implemented, continue; otherwise return */ | 4747 | /* If reset handler not implemented, continue; otherwise return */ |
| 4602 | if (r == -ENOSYS) | 4748 | if (r == -ENOSYS) |
| @@ -4645,9 +4791,9 @@ int amdgpu_do_asic_reset(struct list_head *device_list_handle, · 42 unmodified lines | |||
| 4645 | 4791 | ||
| 4646 | if (!r && amdgpu_ras_intr_triggered()) { | 4792 | if (!r && amdgpu_ras_intr_triggered()) { |
| 4647 | list_for_each_entry(tmp_adev, device_list_handle, reset_list) { | 4793 | list_for_each_entry(tmp_adev, device_list_handle, reset_list) { |
| 4648 | if (tmp_adev->mmhub.ras_funcs && | 4794 | if (tmp_adev->mmhub.ras && tmp_adev->mmhub.ras->ras_block.hw_ops && |
| 4649 | tmp_adev->mmhub.ras_funcs->reset_ras_error_count) | 4795 | tmp_adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count) |
| 4650 | tmp_adev->mmhub.ras_funcs->reset_ras_error_count(tmp_adev); | 4796 | tmp_adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count(tmp_adev); |
| 4651 | } | 4797 | } |
| 4652 | 4798 | ||
| 4653 | amdgpu_ras_intr_cleared(); | 4799 | amdgpu_ras_intr_cleared(); |
| @@ -4754,17 +4900,8 @@ end: · 100 unmodified lines | |||
| 4754 | return r; | 4900 | return r; |
| 4755 | } | 4901 | } |
| 4756 | 4902 | ||
| 4757 | static bool amdgpu_device_lock_adev(struct amdgpu_device *adev, | 4903 | static void amdgpu_device_set_mp1_state(struct amdgpu_device *adev) |
| 4758 | struct amdgpu_hive_info *hive) | ||
| 4759 | { | 4904 | { |
| 4760 | if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0) | ||
| 4761 | return false; | ||
| 4762 | |||
| 4763 | if (hive) { | ||
| 4764 | down_write_nest_lock(&adev->reset_sem, &hive->hive_lock); | ||
| 4765 | } else { | ||
| 4766 | down_write(&adev->reset_sem); | ||
| 4767 | } | ||
| 4768 | 4905 | ||
| 4769 | switch (amdgpu_asic_reset_method(adev)) { | 4906 | switch (amdgpu_asic_reset_method(adev)) { |
| 4770 | case AMD_RESET_METHOD_MODE1: | 4907 | case AMD_RESET_METHOD_MODE1: |
| @@ -4777,56 +4914,12 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev, · 6 unmodified lines | |||
| 4777 | adev->mp1_state = PP_MP1_STATE_NONE; | 4914 | adev->mp1_state = PP_MP1_STATE_NONE; |
| 4778 | break; | 4915 | break; |
| 4779 | } | 4916 | } |
| 4780 | |||
| 4781 | return true; | ||
| 4782 | } | 4917 | } |
| 4783 | 4918 | ||
| 4784 | static void amdgpu_device_unlock_adev(struct amdgpu_device *adev) | 4919 | static void amdgpu_device_unset_mp1_state(struct amdgpu_device *adev) |
| 4785 | { | 4920 | { |
| 4786 | amdgpu_vf_error_trans_all(adev); | 4921 | amdgpu_vf_error_trans_all(adev); |
| 4787 | adev->mp1_state = PP_MP1_STATE_NONE; | 4922 | adev->mp1_state = PP_MP1_STATE_NONE; |
| 4788 | atomic_set(&adev->in_gpu_reset, 0); | ||
| 4789 | up_write(&adev->reset_sem); | ||
| 4790 | } | ||
| 4791 | |||
| 4792 | /* | ||
| 4793 | * to lockup a list of amdgpu devices in a hive safely, if not a hive | ||
| 4794 | * with multiple nodes, it will be similar as amdgpu_device_lock_adev. | ||
| 4795 | * | ||
| 4796 | * unlock won't require roll back. | ||
| 4797 | */ | ||
| 4798 | static int amdgpu_device_lock_hive_adev(struct amdgpu_device *adev, struct amdgpu_hive_info *hive) | ||
| 4799 | { | ||
| 4800 | struct amdgpu_device *tmp_adev = NULL; | ||
| 4801 | |||
| 4802 | if (!amdgpu_sriov_vf(adev) && (adev->gmc.xgmi.num_physical_nodes > 1)) { | ||
| 4803 | if (!hive) { | ||
| 4804 | dev_err(adev->dev, "Hive is NULL while device has multiple xgmi nodes"); | ||
| 4805 | return -ENODEV; | ||
| 4806 | } | ||
| 4807 | list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) { | ||
| 4808 | if (!amdgpu_device_lock_adev(tmp_adev, hive)) | ||
| 4809 | goto roll_back; | ||
| 4810 | } | ||
| 4811 | } else if (!amdgpu_device_lock_adev(adev, hive)) | ||
| 4812 | return -EAGAIN; | ||
| 4813 | |||
| 4814 | return 0; | ||
| 4815 | roll_back: | ||
| 4816 | if (!list_is_first(&tmp_adev->gmc.xgmi.head, &hive->device_list)) { | ||
| 4817 | /* | ||
| 4818 | * if the lockup iteration break in the middle of a hive, | ||
| 4819 | * it may means there may has a race issue, | ||
| 4820 | * or a hive device locked up independently. | ||
| 4821 | * we may be in trouble and may not, so will try to roll back | ||
| 4822 | * the lock and give out a warnning. | ||
| 4823 | */ | ||
| 4824 | dev_warn(tmp_adev->dev, "Hive lock iteration broke in the middle. Rolling back to unlock"); | ||
| 4825 | list_for_each_entry_continue_reverse(tmp_adev, &hive->device_list, gmc.xgmi.head) { | ||
| 4826 | amdgpu_device_unlock_adev(tmp_adev); | ||
| 4827 | } | ||
| 4828 | } | ||
| 4829 | return -EAGAIN; | ||
| 4830 | } | 4923 | } |
| 4831 | 4924 | ||
| 4832 | static void amdgpu_device_resume_display_audio(struct amdgpu_device *adev) | 4925 | static void amdgpu_device_resume_display_audio(struct amdgpu_device *adev) |
| @@ -4960,7 +5053,7 @@ retry: · 127 unmodified lines | |||
| 4960 | } | 5053 | } |
| 4961 | 5054 | ||
| 4962 | /** | 5055 | /** |
| 4963 | * amdgpu_device_gpu_recover - reset the asic and recover scheduler | 5056 | * amdgpu_device_gpu_recover_imp - reset the asic and recover scheduler |
| 4964 | * | 5057 | * |
| 4965 | * @adev: amdgpu_device pointer | 5058 | * @adev: amdgpu_device pointer |
| 4966 | * @job: which job trigger hang | 5059 | * @job: which job trigger hang |
| @@ -4970,7 +5063,7 @@ retry: · 3 unmodified lines | |||
| 4970 | * Returns 0 for success or an error on failure. | 5063 | * Returns 0 for success or an error on failure. |
| 4971 | */ | 5064 | */ |
| 4972 | 5065 | ||
| 4973 | int amdgpu_device_gpu_recover(struct amdgpu_device *adev, | 5066 | int amdgpu_device_gpu_recover_imp(struct amdgpu_device *adev, |
| 4974 | struct amdgpu_job *job) | 5067 | struct amdgpu_job *job) |
| 4975 | { | 5068 | { |
| 4976 | struct list_head device_list, *device_list_handle = NULL; | 5069 | struct list_head device_list, *device_list_handle = NULL; |
| @@ -5004,26 +5097,10 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 27 unmodified lines | |||
| 5004 | dev_info(adev->dev, "GPU %s begin!\n", | 5097 | dev_info(adev->dev, "GPU %s begin!\n", |
| 5005 | need_emergency_restart ? "jobs stop":"reset"); | 5098 | need_emergency_restart ? "jobs stop":"reset"); |
| 5006 | 5099 | ||
| 5007 | /* | ||
| 5008 | * Here we trylock to avoid chain of resets executing from | ||
| 5009 | * either trigger by jobs on different adevs in XGMI hive or jobs on | ||
| 5010 | * different schedulers for same device while this TO handler is running. | ||
| 5011 | * We always reset all schedulers for device and all devices for XGMI | ||
| 5012 | * hive so that should take care of them too. | ||
| 5013 | */ | ||
| 5014 | if (!amdgpu_sriov_vf(adev)) | 5100 | if (!amdgpu_sriov_vf(adev)) |
| 5015 | hive = amdgpu_get_xgmi_hive(adev); | 5101 | hive = amdgpu_get_xgmi_hive(adev); |
| 5016 | if (hive) { | 5102 | if (hive) |
| 5017 | if (atomic_cmpxchg(&hive->in_reset, 0, 1) != 0) { | ||
| 5018 | DRM_INFO("Bailing on TDR for s_job:%llx, hive: %llx as another already in progress", | ||
| 5019 | job ? job->base.id : -1, hive->hive_id); | ||
| 5020 | amdgpu_put_xgmi_hive(hive); | ||
| 5021 | if (job && job->vm) | ||
| 5022 | drm_sched_increase_karma(&job->base); | ||
| 5023 | return 0; | ||
| 5024 | } | ||
| 5025 | mutex_lock(&hive->hive_lock); | 5103 | mutex_lock(&hive->hive_lock); |
| 5026 | } | ||
| 5027 | 5104 | ||
| 5028 | reset_context.method = AMD_RESET_METHOD_NONE; | 5105 | reset_context.method = AMD_RESET_METHOD_NONE; |
| 5029 | reset_context.reset_req_dev = adev; | 5106 | reset_context.reset_req_dev = adev; |
| @@ -5031,22 +5108,6 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 1 unmodified line | |||
| 5031 | reset_context.hive = hive; | 5108 | reset_context.hive = hive; |
| 5032 | clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); | 5109 | clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags); |
| 5033 | 5110 | ||
| 5034 | /* | ||
| 5035 | * lock the device before we try to operate the linked list | ||
| 5036 | * if didn't get the device lock, don't touch the linked list since | ||
| 5037 | * others may iterating it. | ||
| 5038 | */ | ||
| 5039 | r = amdgpu_device_lock_hive_adev(adev, hive); | ||
| 5040 | if (r) { | ||
| 5041 | dev_info(adev->dev, "Bailing on TDR for s_job:%llx, as another already in progress", | ||
| 5042 | job ? job->base.id : -1); | ||
| 5043 | |||
| 5044 | /* even we skipped this reset, still need to set the job to guilty */ | ||
| 5045 | if (job && job->vm) | ||
| 5046 | drm_sched_increase_karma(&job->base); | ||
| 5047 | goto skip_recovery; | ||
| 5048 | } | ||
| 5049 | |||
| 5050 | /* | 5111 | /* |
| 5051 | * Build list of devices to reset. | 5112 | * Build list of devices to reset. |
| 5052 | * In case we are in XGMI hive mode, resort the device list | 5113 | * In case we are in XGMI hive mode, resort the device list |
| @@ -5064,8 +5125,16 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev, · 11 unmodified lines | |||
| 5064 | device_list_handle = &device_list; | 5125 | device_list_handle = &device_list; |
| 5065 | } | 5126 | } |
| 5066 | 5127 | ||
| 5128 | /* We need to lock reset domain only once both for XGMI and single device */ | ||
| 5129 | tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device, | ||
| 5130 | reset_list); | ||
| 5131 | amdgpu_device_lock_reset_domain(tmp_adev->reset_domain); | ||
| 5132 | |||
| 5067 | /* block all schedulers and reset given job's ring */ | 5133 | /* block all schedulers and reset given job's ring */ |
| 5068 | list_for_each_entry(tmp_adev, device_list_handle, reset_list) { | 5134 | list_for_each_entry(tmp_adev, device_list_handle, reset_list) { |
| 5135 | |||
| 5136 | amdgpu_device_set_mp1_state(tmp_adev); | ||
| 5137 | |||
| 5069 | /* | 5138 | /* |
| 5070 | * Try to put the audio codec into suspend state | 5139 | * Try to put the audio codec into suspend state |
| 5071 | * before gpu reset started. | 5140 | * before gpu reset started. |
| @@ -5187,6 +5256,9 @@ skip_hw_reset: · 115 unmodified lines | |||
| 5187 | drm_helper_resume_force_mode(adev_to_drm(tmp_adev)); | 5256 | drm_helper_resume_force_mode(adev_to_drm(tmp_adev)); |
| 5188 | } | 5257 | } |
| 5189 | 5258 | ||
| 5259 | if (tmp_adev->asic_reset_res) | ||
| 5260 | r = tmp_adev->asic_reset_res; | ||
| 5261 | |||
| 5190 | tmp_adev->asic_reset_res = 0; | 5262 | tmp_adev->asic_reset_res = 0; |
| 5191 | 5263 | ||
| 5192 | if (r) { | 5264 | if (r) { |
| @@ -5214,21 +5286,55 @@ skip_sched_resume: · 21 unmodified lines | |||
| 5214 | 5286 | ||
| 5215 | if (audio_suspended) | 5287 | if (audio_suspended) |
| 5216 | amdgpu_device_resume_display_audio(tmp_adev); | 5288 | amdgpu_device_resume_display_audio(tmp_adev); |
| 5217 | amdgpu_device_unlock_adev(tmp_adev); | 5289 | |
| 5290 | amdgpu_device_unset_mp1_state(tmp_adev); | ||
| 5218 | } | 5291 | } |
| 5219 | 5292 | ||
| 5220 | skip_recovery: | 5293 | tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device, |
| 5294 | reset_list); | ||
| 5295 | amdgpu_device_unlock_reset_domain(tmp_adev->reset_domain); | ||
| 5296 | |||
| 5221 | if (hive) { | 5297 | if (hive) { |
| 5222 | atomic_set(&hive->in_reset, 0); | ||
| 5223 | mutex_unlock(&hive->hive_lock); | 5298 | mutex_unlock(&hive->hive_lock); |
| 5224 | amdgpu_put_xgmi_hive(hive); | 5299 | amdgpu_put_xgmi_hive(hive); |
| 5225 | } | 5300 | } |
| 5226 | 5301 | ||
| 5227 | if (r && r != -EAGAIN) | 5302 | if (r) |
| 5228 | dev_info(adev->dev, "GPU reset end with ret = %d\n", r); | 5303 | dev_info(adev->dev, "GPU reset end with ret = %d\n", r); |
| 5229 | return r; | 5304 | return r; |
| 5230 | } | 5305 | } |
| 5231 | 5306 | ||
| 5307 | struct amdgpu_recover_work_struct { | ||
| 5308 | struct work_struct base; | ||
| 5309 | struct amdgpu_device *adev; | ||
| 5310 | struct amdgpu_job *job; | ||
| 5311 | int ret; | ||
| 5312 | }; | ||
| 5313 | |||
| 5314 | static void amdgpu_device_queue_gpu_recover_work(struct work_struct *work) | ||
| 5315 | { | ||
| 5316 | struct amdgpu_recover_work_struct *recover_work = container_of(work, struct amdgpu_recover_work_struct, base); | ||
| 5317 | |||
| 5318 | recover_work->ret = amdgpu_device_gpu_recover_imp(recover_work->adev, recover_work->job); | ||
| 5319 | } | ||
| 5320 | /* | ||
| 5321 | * Serialize gpu recover into reset domain single threaded wq | ||
| 5322 | */ | ||
| 5323 | int amdgpu_device_gpu_recover(struct amdgpu_device *adev, | ||
| 5324 | struct amdgpu_job *job) | ||
| 5325 | { | ||
| 5326 | struct amdgpu_recover_work_struct work = {.adev = adev, .job = job}; | ||
| 5327 | |||
| 5328 | INIT_WORK(&work.base, amdgpu_device_queue_gpu_recover_work); | ||
| 5329 | |||
| 5330 | if (!amdgpu_reset_domain_schedule(adev->reset_domain, &work.base)) | ||
| 5331 | return -EAGAIN; | ||
| 5332 | |||
| 5333 | flush_work(&work.base); | ||
| 5334 | |||
| 5335 | return work.ret; | ||
| 5336 | } | ||
| 5337 | |||
| 5232 | /** | 5338 | /** |
| 5233 | * amdgpu_device_get_pcie_info - fence pcie info about the PCIE slot | 5339 | * amdgpu_device_get_pcie_info - fence pcie info about the PCIE slot |
| 5234 | * | 5340 | * |
| @@ -5416,20 +5522,6 @@ int amdgpu_device_baco_exit(struct drm_device *dev) · 181 unmodified lines | |||
| 5416 | return 0; | 5522 | return 0; |
| 5417 | } | 5523 | } |
| 5418 | 5524 | ||
| 5419 | static void amdgpu_cancel_all_tdr(struct amdgpu_device *adev) | ||
| 5420 | { | ||
| 5421 | int i; | ||
| 5422 | |||
| 5423 | for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { | ||
| 5424 | struct amdgpu_ring *ring = adev->rings[i]; | ||
| 5425 | |||
| 5426 | if (!ring || !ring->sched.thread) | ||
| 5427 | continue; | ||
| 5428 | |||
| 5429 | cancel_delayed_work_sync(&ring->sched.work_tdr); | ||
| 5430 | } | ||
| 5431 | } | ||
| 5432 | |||
| 5433 | /** | 5525 | /** |
| 5434 | * amdgpu_pci_error_detected - Called when a PCI error is detected. | 5526 | * amdgpu_pci_error_detected - Called when a PCI error is detected. |
| 5435 | * @pdev: PCI device struct | 5527 | * @pdev: PCI device struct |
| @@ -5460,14 +5552,11 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_sta · 24 unmodified lines | |||
| 5460 | /* Fatal error, prepare for slot reset */ | 5552 | /* Fatal error, prepare for slot reset */ |
| 5461 | case pci_channel_io_frozen: | 5553 | case pci_channel_io_frozen: |
| 5462 | /* | 5554 | /* |
| 5463 | * Cancel and wait for all TDRs in progress if failing to | 5555 | * Locking adev->reset_domain->sem will prevent any external access |
| 5464 | * set adev->in_gpu_reset in amdgpu_device_lock_adev | ||
| 5465 | * | ||
| 5466 | * Locking adev->reset_sem will prevent any external access | ||
| 5467 | * to GPU during PCI error recovery | 5556 | * to GPU during PCI error recovery |
| 5468 | */ | 5557 | */ |
| 5469 | while (!amdgpu_device_lock_adev(adev, NULL)) | 5558 | amdgpu_device_lock_reset_domain(adev->reset_domain); |
| 5470 | amdgpu_cancel_all_tdr(adev); | 5559 | amdgpu_device_set_mp1_state(adev); |
| 5471 | 5560 | ||
| 5472 | /* | 5561 | /* |
| 5473 | * Block any work scheduling as we do for regular GPU reset | 5562 | * Block any work scheduling as we do for regular GPU reset |
| @@ -5574,7 +5663,8 @@ out: · 100 unmodified lines | |||
| 5574 | DRM_INFO("PCIe error recovery succeeded\n"); | 5663 | DRM_INFO("PCIe error recovery succeeded\n"); |
| 5575 | } else { | 5664 | } else { |
| 5576 | DRM_ERROR("PCIe error recovery failed, err:%d", r); | 5665 | DRM_ERROR("PCIe error recovery failed, err:%d", r); |
| 5577 | amdgpu_device_unlock_adev(adev); | 5666 | amdgpu_device_unset_mp1_state(adev); |
| 5667 | amdgpu_device_unlock_reset_domain(adev->reset_domain); | ||
| 5578 | } | 5668 | } |
| 5579 | 5669 | ||
| 5580 | return r ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED; | 5670 | return r ? PCI_ERS_RESULT_DISCONNECT : PCI_ERS_RESULT_RECOVERED; |
| @@ -5611,7 +5701,8 @@ void amdgpu_pci_resume(struct pci_dev *pdev) · 30 unmodified lines | |||
| 5611 | drm_sched_start(&ring->sched, true); | 5701 | drm_sched_start(&ring->sched, true); |
| 5612 | } | 5702 | } |
| 5613 | 5703 | ||
| 5614 | amdgpu_device_unlock_adev(adev); | 5704 | amdgpu_device_unset_mp1_state(adev); |
| 5705 | amdgpu_device_unlock_reset_domain(adev->reset_domain); | ||
| 5615 | } | 5706 | } |
| 5616 | 5707 | ||
| 5617 | bool amdgpu_device_cache_pci_state(struct pci_dev *pdev) | 5708 | bool amdgpu_device_cache_pci_state(struct pci_dev *pdev) |
| @@ -5688,6 +5779,11 @@ void amdgpu_device_invalidate_hdp(struct amdgpu_device *adev, · 70 unmodified lines | |||
| 5688 | amdgpu_asic_invalidate_hdp(adev, ring); | 5779 | amdgpu_asic_invalidate_hdp(adev, ring); |
| 5689 | } | 5780 | } |
| 5690 | 5781 | ||
| 5782 | int amdgpu_in_reset(struct amdgpu_device *adev) | ||
| 5783 | { | ||
| 5784 | return atomic_read(&adev->reset_domain->in_gpu_reset); | ||
| 5785 | } | ||
| 5786 | |||
| 5691 | /** | 5787 | /** |
| 5692 | * amdgpu_device_halt() - bring hardware to some kind of halt state | 5788 | * amdgpu_device_halt() - bring hardware to some kind of halt state |
| 5693 | * | 5789 | * |
| @@ -5726,3 +5822,36 @@ void amdgpu_device_halt(struct amdgpu_device *adev) · 32 unmodified lines | |||
| 5726 | pci_disable_device(pdev); | 5822 | pci_disable_device(pdev); |
| 5727 | pci_wait_for_pending_transaction(pdev); | 5823 | pci_wait_for_pending_transaction(pdev); |
| 5728 | } | 5824 | } |
| 5825 | |||
| 5826 | u32 amdgpu_device_pcie_port_rreg(struct amdgpu_device *adev, | ||
| 5827 | u32 reg) | ||
| 5828 | { | ||
| 5829 | unsigned long flags, address, data; | ||
| 5830 | u32 r; | ||
| 5831 | |||
| 5832 | address = adev->nbio.funcs->get_pcie_port_index_offset(adev); | ||
| 5833 | data = adev->nbio.funcs->get_pcie_port_data_offset(adev); | ||
| 5834 | |||
| 5835 | spin_lock_irqsave(&adev->pcie_idx_lock, flags); | ||
| 5836 | WREG32(address, reg * 4); | ||
| 5837 | (void)RREG32(address); | ||
| 5838 | r = RREG32(data); | ||
| 5839 | spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); | ||
| 5840 | return r; | ||
| 5841 | } | ||
| 5842 | |||
| 5843 | void amdgpu_device_pcie_port_wreg(struct amdgpu_device *adev, | ||
| 5844 | u32 reg, u32 v) | ||
| 5845 | { | ||
| 5846 | unsigned long flags, address, data; | ||
| 5847 | |||
| 5848 | address = adev->nbio.funcs->get_pcie_port_index_offset(adev); | ||
| 5849 | data = adev->nbio.funcs->get_pcie_port_data_offset(adev); | ||
| 5850 | |||
| 5851 | spin_lock_irqsave(&adev->pcie_idx_lock, flags); | ||
| 5852 | WREG32(address, reg * 4); | ||
| 5853 | (void)RREG32(address); | ||
| 5854 | WREG32(data, v); | ||
| 5855 | (void)RREG32(data); | ||
| 5856 | spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); | ||
| 5857 | } | ||
| @@ -360,8 +360,11 @@ out: · 359 unmodified lines | |||
| 360 | return r; | 360 | return r; |
| 361 | } | 361 | } |
| 362 | 362 | ||
| 363 | static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev); | ||
| 364 | |||
| 363 | void amdgpu_discovery_fini(struct amdgpu_device *adev) | 365 | void amdgpu_discovery_fini(struct amdgpu_device *adev) |
| 364 | { | 366 | { |
| 367 | amdgpu_discovery_sysfs_fini(adev); | ||
| 365 | kfree(adev->mman.discovery_bin); | 368 | kfree(adev->mman.discovery_bin); |
| 366 | adev->mman.discovery_bin = NULL; | 369 | adev->mman.discovery_bin = NULL; |
| 367 | } | 370 | } |
| @@ -382,6 +385,578 @@ static int amdgpu_discovery_validate_ip(const struct ip *ip) · 14 unmodified lines | |||
| 382 | return 0; | 385 | return 0; |
| 383 | } | 386 | } |
| 384 | 387 | ||
| 388 | static void amdgpu_discovery_read_harvest_bit_per_ip(struct amdgpu_device *adev, | ||
| 389 | uint32_t *vcn_harvest_count) | ||
| 390 | { | ||
| 391 | struct binary_header *bhdr; | ||
| 392 | struct ip_discovery_header *ihdr; | ||
| 393 | struct die_header *dhdr; | ||
| 394 | struct ip *ip; | ||
| 395 | uint16_t die_offset, ip_offset, num_dies, num_ips; | ||
| 396 | int i, j; | ||
| 397 | |||
| 398 | bhdr = (struct binary_header *)adev->mman.discovery_bin; | ||
| 399 | ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin + | ||
| 400 | le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset)); | ||
| 401 | num_dies = le16_to_cpu(ihdr->num_dies); | ||
| 402 | |||
| 403 | /* scan harvest bit of all IP data structures */ | ||
| 404 | for (i = 0; i < num_dies; i++) { | ||
| 405 | die_offset = le16_to_cpu(ihdr->die_info[i].die_offset); | ||
| 406 | dhdr = (struct die_header *)(adev->mman.discovery_bin + die_offset); | ||
| 407 | num_ips = le16_to_cpu(dhdr->num_ips); | ||
| 408 | ip_offset = die_offset + sizeof(*dhdr); | ||
| 409 | |||
| 410 | for (j = 0; j < num_ips; j++) { | ||
| 411 | ip = (struct ip *)(adev->mman.discovery_bin + ip_offset); | ||
| 412 | |||
| 413 | if (amdgpu_discovery_validate_ip(ip)) | ||
| 414 | goto next_ip; | ||
| 415 | |||
| 416 | if (le16_to_cpu(ip->harvest) == 1) { | ||
| 417 | switch (le16_to_cpu(ip->hw_id)) { | ||
| 418 | case VCN_HWID: | ||
| 419 | (*vcn_harvest_count)++; | ||
| 420 | if (ip->number_instance == 0) | ||
| 421 | adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN0; | ||
| 422 | else | ||
| 423 | adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN1; | ||
| 424 | break; | ||
| 425 | case DMU_HWID: | ||
| 426 | adev->harvest_ip_mask |= AMD_HARVEST_IP_DMU_MASK; | ||
| 427 | break; | ||
| 428 | default: | ||
| 429 | break; | ||
| 430 | } | ||
| 431 | } | ||
| 432 | next_ip: | ||
| 433 | ip_offset += sizeof(*ip) + 4 * (ip->num_base_address - 1); | ||
| 434 | } | ||
| 435 | } | ||
| 436 | } | ||
| 437 | |||
| 438 | static void amdgpu_discovery_read_from_harvest_table(struct amdgpu_device *adev, | ||
| 439 | uint32_t *vcn_harvest_count) | ||
| 440 | { | ||
| 441 | struct binary_header *bhdr; | ||
| 442 | struct harvest_table *harvest_info; | ||
| 443 | int i; | ||
| 444 | |||
| 445 | bhdr = (struct binary_header *)adev->mman.discovery_bin; | ||
| 446 | harvest_info = (struct harvest_table *)(adev->mman.discovery_bin + | ||
| 447 | le16_to_cpu(bhdr->table_list[HARVEST_INFO].offset)); | ||
| 448 | for (i = 0; i < 32; i++) { | ||
| 449 | if (le16_to_cpu(harvest_info->list[i].hw_id) == 0) | ||
| 450 | break; | ||
| 451 | |||
| 452 | switch (le16_to_cpu(harvest_info->list[i].hw_id)) { | ||
| 453 | case VCN_HWID: | ||
| 454 | (*vcn_harvest_count)++; | ||
| 455 | if (harvest_info->list[i].number_instance == 0) | ||
| 456 | adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN0; | ||
| 457 | else | ||
| 458 | adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN1; | ||
| 459 | break; | ||
| 460 | case DMU_HWID: | ||
| 461 | adev->harvest_ip_mask |= AMD_HARVEST_IP_DMU_MASK; | ||
| 462 | break; | ||
| 463 | default: | ||
| 464 | break; | ||
| 465 | } | ||
| 466 | } | ||
| 467 | } | ||
| 468 | |||
| 469 | /* ================================================== */ | ||
| 470 | |||
| 471 | struct ip_hw_instance { | ||
| 472 | struct kobject kobj; /* ip_discovery/die/#die/#hw_id/#instance/<attrs...> */ | ||
| 473 | |||
| 474 | int hw_id; | ||
| 475 | u8 num_instance; | ||
| 476 | u8 major, minor, revision; | ||
| 477 | u8 harvest; | ||
| 478 | |||
| 479 | int num_base_addresses; | ||
| 480 | u32 base_addr[]; | ||
| 481 | }; | ||
| 482 | |||
| 483 | struct ip_hw_id { | ||
| 484 | struct kset hw_id_kset; /* ip_discovery/die/#die/#hw_id/, contains ip_hw_instance */ | ||
| 485 | int hw_id; | ||
| 486 | }; | ||
| 487 | |||
| 488 | struct ip_die_entry { | ||
| 489 | struct kset ip_kset; /* ip_discovery/die/#die/, contains ip_hw_id */ | ||
| 490 | u16 num_ips; | ||
| 491 | }; | ||
| 492 | |||
| 493 | /* -------------------------------------------------- */ | ||
| 494 | |||
| 495 | struct ip_hw_instance_attr { | ||
| 496 | struct attribute attr; | ||
| 497 | ssize_t (*show)(struct ip_hw_instance *ip_hw_instance, char *buf); | ||
| 498 | }; | ||
| 499 | |||
| 500 | static ssize_t hw_id_show(struct ip_hw_instance *ip_hw_instance, char *buf) | ||
| 501 | { | ||
| 502 | return sysfs_emit(buf, "%d\n", ip_hw_instance->hw_id); | ||
| 503 | } | ||
| 504 | |||
| 505 | static ssize_t num_instance_show(struct ip_hw_instance *ip_hw_instance, char *buf) | ||
| 506 | { | ||
| 507 | return sysfs_emit(buf, "%d\n", ip_hw_instance->num_instance); | ||
| 508 | } | ||
| 509 | |||
| 510 | static ssize_t major_show(struct ip_hw_instance *ip_hw_instance, char *buf) | ||
| 511 | { | ||
| 512 | return sysfs_emit(buf, "%d\n", ip_hw_instance->major); | ||
| 513 | } | ||
| 514 | |||
| 515 | static ssize_t minor_show(struct ip_hw_instance *ip_hw_instance, char *buf) | ||
| 516 | { | ||
| 517 | return sysfs_emit(buf, "%d\n", ip_hw_instance->minor); | ||
| 518 | } | ||
| 519 | |||
| 520 | static ssize_t revision_show(struct ip_hw_instance *ip_hw_instance, char *buf) | ||
| 521 | { | ||
| 522 | return sysfs_emit(buf, "%d\n", ip_hw_instance->revision); | ||
| 523 | } | ||
| 524 | |||
| 525 | static ssize_t harvest_show(struct ip_hw_instance *ip_hw_instance, char *buf) | ||
| 526 | { | ||
| 527 | return sysfs_emit(buf, "0x%01X\n", ip_hw_instance->harvest); | ||
| 528 | } | ||
| 529 | |||
| 530 | static ssize_t num_base_addresses_show(struct ip_hw_instance *ip_hw_instance, char *buf) | ||
| 531 | { | ||
| 532 | return sysfs_emit(buf, "%d\n", ip_hw_instance->num_base_addresses); | ||
| 533 | } | ||
| 534 | |||
| 535 | static ssize_t base_addr_show(struct ip_hw_instance *ip_hw_instance, char *buf) | ||
| 536 | { | ||
| 537 | ssize_t res, at; | ||
| 538 | int ii; | ||
| 539 | |||
| 540 | for (res = at = ii = 0; ii < ip_hw_instance->num_base_addresses; ii++) { | ||
| 541 | /* Here we satisfy the condition that, at + size <= PAGE_SIZE. | ||
| 542 | */ | ||
| 543 | if (at + 12 > PAGE_SIZE) | ||
| 544 | break; | ||
| 545 | res = sysfs_emit_at(buf, at, "0x%08X\n", | ||
| 546 | ip_hw_instance->base_addr[ii]); | ||
| 547 | if (res <= 0) | ||
| 548 | break; | ||
| 549 | at += res; | ||
| 550 | } | ||
| 551 | |||
| 552 | return res < 0 ? res : at; | ||
| 553 | } | ||
| 554 | |||
| 555 | static struct ip_hw_instance_attr ip_hw_attr[] = { | ||
| 556 | __ATTR_RO(hw_id), | ||
| 557 | __ATTR_RO(num_instance), | ||
| 558 | __ATTR_RO(major), | ||
| 559 | __ATTR_RO(minor), | ||
| 560 | __ATTR_RO(revision), | ||
| 561 | __ATTR_RO(harvest), | ||
| 562 | __ATTR_RO(num_base_addresses), | ||
| 563 | __ATTR_RO(base_addr), | ||
| 564 | }; | ||
| 565 | |||
| 566 | static struct attribute *ip_hw_instance_attrs[ARRAY_SIZE(ip_hw_attr) + 1]; | ||
| 567 | ATTRIBUTE_GROUPS(ip_hw_instance); | ||
| 568 | |||
| 569 | #define to_ip_hw_instance(x) container_of(x, struct ip_hw_instance, kobj) | ||
| 570 | #define to_ip_hw_instance_attr(x) container_of(x, struct ip_hw_instance_attr, attr) | ||
| 571 | |||
| 572 | static ssize_t ip_hw_instance_attr_show(struct kobject *kobj, | ||
| 573 | struct attribute *attr, | ||
| 574 | char *buf) | ||
| 575 | { | ||
| 576 | struct ip_hw_instance *ip_hw_instance = to_ip_hw_instance(kobj); | ||
| 577 | struct ip_hw_instance_attr *ip_hw_attr = to_ip_hw_instance_attr(attr); | ||
| 578 | |||
| 579 | if (!ip_hw_attr->show) | ||
| 580 | return -EIO; | ||
| 581 | |||
| 582 | return ip_hw_attr->show(ip_hw_instance, buf); | ||
| 583 | } | ||
| 584 | |||
| 585 | static const struct sysfs_ops ip_hw_instance_sysfs_ops = { | ||
| 586 | .show = ip_hw_instance_attr_show, | ||
| 587 | }; | ||
| 588 | |||
| 589 | static void ip_hw_instance_release(struct kobject *kobj) | ||
| 590 | { | ||
| 591 | struct ip_hw_instance *ip_hw_instance = to_ip_hw_instance(kobj); | ||
| 592 | |||
| 593 | kfree(ip_hw_instance); | ||
| 594 | } | ||
| 595 | |||
| 596 | static struct kobj_type ip_hw_instance_ktype = { | ||
| 597 | .release = ip_hw_instance_release, | ||
| 598 | .sysfs_ops = &ip_hw_instance_sysfs_ops, | ||
| 599 | .default_groups = ip_hw_instance_groups, | ||
| 600 | }; | ||
| 601 | |||
| 602 | /* -------------------------------------------------- */ | ||
| 603 | |||
| 604 | #define to_ip_hw_id(x) container_of(to_kset(x), struct ip_hw_id, hw_id_kset) | ||
| 605 | |||
| 606 | static void ip_hw_id_release(struct kobject *kobj) | ||
| 607 | { | ||
| 608 | struct ip_hw_id *ip_hw_id = to_ip_hw_id(kobj); | ||
| 609 | |||
| 610 | if (!list_empty(&ip_hw_id->hw_id_kset.list)) | ||
| 611 | DRM_ERROR("ip_hw_id->hw_id_kset is not empty"); | ||
| 612 | kfree(ip_hw_id); | ||
| 613 | } | ||
| 614 | |||
| 615 | static struct kobj_type ip_hw_id_ktype = { | ||
| 616 | .release = ip_hw_id_release, | ||
| 617 | .sysfs_ops = &kobj_sysfs_ops, | ||
| 618 | }; | ||
| 619 | |||
| 620 | /* -------------------------------------------------- */ | ||
| 621 | |||
| 622 | static void die_kobj_release(struct kobject *kobj); | ||
| 623 | static void ip_disc_release(struct kobject *kobj); | ||
| 624 | |||
| 625 | struct ip_die_entry_attribute { | ||
| 626 | struct attribute attr; | ||
| 627 | ssize_t (*show)(struct ip_die_entry *ip_die_entry, char *buf); | ||
| 628 | }; | ||
| 629 | |||
| 630 | #define to_ip_die_entry_attr(x) container_of(x, struct ip_die_entry_attribute, attr) | ||
| 631 | |||
| 632 | static ssize_t num_ips_show(struct ip_die_entry *ip_die_entry, char *buf) | ||
| 633 | { | ||
| 634 | return sysfs_emit(buf, "%d\n", ip_die_entry->num_ips); | ||
| 635 | } | ||
| 636 | |||
| 637 | /* If there are more ip_die_entry attrs, other than the number of IPs, | ||
| 638 | * we can make this intro an array of attrs, and then initialize | ||
| 639 | * ip_die_entry_attrs in a loop. | ||
| 640 | */ | ||
| 641 | static struct ip_die_entry_attribute num_ips_attr = | ||
| 642 | __ATTR_RO(num_ips); | ||
| 643 | |||
| 644 | static struct attribute *ip_die_entry_attrs[] = { | ||
| 645 | &num_ips_attr.attr, | ||
| 646 | NULL, | ||
| 647 | }; | ||
| 648 | ATTRIBUTE_GROUPS(ip_die_entry); /* ip_die_entry_groups */ | ||
| 649 | |||
| 650 | #define to_ip_die_entry(x) container_of(to_kset(x), struct ip_die_entry, ip_kset) | ||
| 651 | |||
| 652 | static ssize_t ip_die_entry_attr_show(struct kobject *kobj, | ||
| 653 | struct attribute *attr, | ||
| 654 | char *buf) | ||
| 655 | { | ||
| 656 | struct ip_die_entry_attribute *ip_die_entry_attr = to_ip_die_entry_attr(attr); | ||
| 657 | struct ip_die_entry *ip_die_entry = to_ip_die_entry(kobj); | ||
| 658 | |||
| 659 | if (!ip_die_entry_attr->show) | ||
| 660 | return -EIO; | ||
| 661 | |||
| 662 | return ip_die_entry_attr->show(ip_die_entry, buf); | ||
| 663 | } | ||
| 664 | |||
| 665 | static void ip_die_entry_release(struct kobject *kobj) | ||
| 666 | { | ||
| 667 | struct ip_die_entry *ip_die_entry = to_ip_die_entry(kobj); | ||
| 668 | |||
| 669 | if (!list_empty(&ip_die_entry->ip_kset.list)) | ||
| 670 | DRM_ERROR("ip_die_entry->ip_kset is not empty"); | ||
| 671 | kfree(ip_die_entry); | ||
| 672 | } | ||
| 673 | |||
| 674 | static const struct sysfs_ops ip_die_entry_sysfs_ops = { | ||
| 675 | .show = ip_die_entry_attr_show, | ||
| 676 | }; | ||
| 677 | |||
| 678 | static struct kobj_type ip_die_entry_ktype = { | ||
| 679 | .release = ip_die_entry_release, | ||
| 680 | .sysfs_ops = &ip_die_entry_sysfs_ops, | ||
| 681 | .default_groups = ip_die_entry_groups, | ||
| 682 | }; | ||
| 683 | |||
| 684 | static struct kobj_type die_kobj_ktype = { | ||
| 685 | .release = die_kobj_release, | ||
| 686 | .sysfs_ops = &kobj_sysfs_ops, | ||
| 687 | }; | ||
| 688 | |||
| 689 | static struct kobj_type ip_discovery_ktype = { | ||
| 690 | .release = ip_disc_release, | ||
| 691 | .sysfs_ops = &kobj_sysfs_ops, | ||
| 692 | }; | ||
| 693 | |||
| 694 | struct ip_discovery_top { | ||
| 695 | struct kobject kobj; /* ip_discovery/ */ | ||
| 696 | struct kset die_kset; /* ip_discovery/die/, contains ip_die_entry */ | ||
| 697 | struct amdgpu_device *adev; | ||
| 698 | }; | ||
| 699 | |||
| 700 | static void die_kobj_release(struct kobject *kobj) | ||
| 701 | { | ||
| 702 | struct ip_discovery_top *ip_top = container_of(to_kset(kobj), | ||
| 703 | struct ip_discovery_top, | ||
| 704 | die_kset); | ||
| 705 | if (!list_empty(&ip_top->die_kset.list)) | ||
| 706 | DRM_ERROR("ip_top->die_kset is not empty"); | ||
| 707 | } | ||
| 708 | |||
| 709 | static void ip_disc_release(struct kobject *kobj) | ||
| 710 | { | ||
| 711 | struct ip_discovery_top *ip_top = container_of(kobj, struct ip_discovery_top, | ||
| 712 | kobj); | ||
| 713 | struct amdgpu_device *adev = ip_top->adev; | ||
| 714 | |||
| 715 | adev->ip_top = NULL; | ||
| 716 | kfree(ip_top); | ||
| 717 | } | ||
| 718 | |||
| 719 | static int amdgpu_discovery_sysfs_ips(struct amdgpu_device *adev, | ||
| 720 | struct ip_die_entry *ip_die_entry, | ||
| 721 | const size_t _ip_offset, const int num_ips) | ||
| 722 | { | ||
| 723 | int ii, jj, kk, res; | ||
| 724 | |||
| 725 | DRM_DEBUG("num_ips:%d", num_ips); | ||
| 726 | |||
| 727 | /* Find all IPs of a given HW ID, and add their instance to | ||
| 728 | * #die/#hw_id/#instance/<attributes> | ||
| 729 | */ | ||
| 730 | for (ii = 0; ii < HW_ID_MAX; ii++) { | ||
| 731 | struct ip_hw_id *ip_hw_id = NULL; | ||
| 732 | size_t ip_offset = _ip_offset; | ||
| 733 | |||
| 734 | for (jj = 0; jj < num_ips; jj++) { | ||
| 735 | struct ip *ip; | ||
| 736 | struct ip_hw_instance *ip_hw_instance; | ||
| 737 | |||
| 738 | ip = (struct ip *)(adev->mman.discovery_bin + ip_offset); | ||
| 739 | if (amdgpu_discovery_validate_ip(ip) || | ||
| 740 | le16_to_cpu(ip->hw_id) != ii) | ||
| 741 | goto next_ip; | ||
| 742 | |||
| 743 | DRM_DEBUG("match:%d @ ip_offset:%zu", ii, ip_offset); | ||
| 744 | |||
| 745 | /* We have a hw_id match; register the hw | ||
| 746 | * block if not yet registered. | ||
| 747 | */ | ||
| 748 | if (!ip_hw_id) { | ||
| 749 | ip_hw_id = kzalloc(sizeof(*ip_hw_id), GFP_KERNEL); | ||
| 750 | if (!ip_hw_id) | ||
| 751 | return -ENOMEM; | ||
| 752 | ip_hw_id->hw_id = ii; | ||
| 753 | |||
| 754 | kobject_set_name(&ip_hw_id->hw_id_kset.kobj, "%d", ii); | ||
| 755 | ip_hw_id->hw_id_kset.kobj.kset = &ip_die_entry->ip_kset; | ||
| 756 | ip_hw_id->hw_id_kset.kobj.ktype = &ip_hw_id_ktype; | ||
| 757 | res = kset_register(&ip_hw_id->hw_id_kset); | ||
| 758 | if (res) { | ||
| 759 | DRM_ERROR("Couldn't register ip_hw_id kset"); | ||
| 760 | kfree(ip_hw_id); | ||
| 761 | return res; | ||
| 762 | } | ||
| 763 | if (hw_id_names[ii]) { | ||
| 764 | res = sysfs_create_link(&ip_die_entry->ip_kset.kobj, | ||
| 765 | &ip_hw_id->hw_id_kset.kobj, | ||
| 766 | hw_id_names[ii]); | ||
| 767 | if (res) { | ||
| 768 | DRM_ERROR("Couldn't create IP link %s in IP Die:%s\n", | ||
| 769 | hw_id_names[ii], | ||
| 770 | kobject_name(&ip_die_entry->ip_kset.kobj)); | ||
| 771 | } | ||
| 772 | } | ||
| 773 | } | ||
| 774 | |||
| 775 | /* Now register its instance. | ||
| 776 | */ | ||
| 777 | ip_hw_instance = kzalloc(struct_size(ip_hw_instance, | ||
| 778 | base_addr, | ||
| 779 | ip->num_base_address), | ||
| 780 | GFP_KERNEL); | ||
| 781 | if (!ip_hw_instance) { | ||
| 782 | DRM_ERROR("no memory for ip_hw_instance"); | ||
| 783 | return -ENOMEM; | ||
| 784 | } | ||
| 785 | ip_hw_instance->hw_id = le16_to_cpu(ip->hw_id); /* == ii */ | ||
| 786 | ip_hw_instance->num_instance = ip->number_instance; | ||
| 787 | ip_hw_instance->major = ip->major; | ||
| 788 | ip_hw_instance->minor = ip->minor; | ||
| 789 | ip_hw_instance->revision = ip->revision; | ||
| 790 | ip_hw_instance->harvest = ip->harvest; | ||
| 791 | ip_hw_instance->num_base_addresses = ip->num_base_address; | ||
| 792 | |||
| 793 | for (kk = 0; kk < ip_hw_instance->num_base_addresses; kk++) | ||
| 794 | ip_hw_instance->base_addr[kk] = ip->base_address[kk]; | ||
| 795 | |||
| 796 | kobject_init(&ip_hw_instance->kobj, &ip_hw_instance_ktype); | ||
| 797 | ip_hw_instance->kobj.kset = &ip_hw_id->hw_id_kset; | ||
| 798 | res = kobject_add(&ip_hw_instance->kobj, NULL, | ||
| 799 | "%d", ip_hw_instance->num_instance); | ||
| 800 | next_ip: | ||
| 801 | ip_offset += sizeof(*ip) + 4 * (ip->num_base_address - 1); | ||
| 802 | } | ||
| 803 | } | ||
| 804 | |||
| 805 | return 0; | ||
| 806 | } | ||
| 807 | |||
| 808 | static int amdgpu_discovery_sysfs_recurse(struct amdgpu_device *adev) | ||
| 809 | { | ||
| 810 | struct binary_header *bhdr; | ||
| 811 | struct ip_discovery_header *ihdr; | ||
| 812 | struct die_header *dhdr; | ||
| 813 | struct kset *die_kset = &adev->ip_top->die_kset; | ||
| 814 | u16 num_dies, die_offset, num_ips; | ||
| 815 | size_t ip_offset; | ||
| 816 | int ii, res; | ||
| 817 | |||
| 818 | bhdr = (struct binary_header *)adev->mman.discovery_bin; | ||
| 819 | ihdr = (struct ip_discovery_header *)(adev->mman.discovery_bin + | ||
| 820 | le16_to_cpu(bhdr->table_list[IP_DISCOVERY].offset)); | ||
| 821 | num_dies = le16_to_cpu(ihdr->num_dies); | ||
| 822 | |||
| 823 | DRM_DEBUG("number of dies: %d\n", num_dies); | ||
| 824 | |||
| 825 | for (ii = 0; ii < num_dies; ii++) { | ||
| 826 | struct ip_die_entry *ip_die_entry; | ||
| 827 | |||
| 828 | die_offset = le16_to_cpu(ihdr->die_info[ii].die_offset); | ||
| 829 | dhdr = (struct die_header *)(adev->mman.discovery_bin + die_offset); | ||
| 830 | num_ips = le16_to_cpu(dhdr->num_ips); | ||
| 831 | ip_offset = die_offset + sizeof(*dhdr); | ||
| 832 | |||
| 833 | /* Add the die to the kset. | ||
| 834 | * | ||
| 835 | * dhdr->die_id == ii, which was checked in | ||
| 836 | * amdgpu_discovery_reg_base_init(). | ||
| 837 | */ | ||
| 838 | |||
| 839 | ip_die_entry = kzalloc(sizeof(*ip_die_entry), GFP_KERNEL); | ||
| 840 | if (!ip_die_entry) | ||
| 841 | return -ENOMEM; | ||
| 842 | |||
| 843 | ip_die_entry->num_ips = num_ips; | ||
| 844 | |||
| 845 | kobject_set_name(&ip_die_entry->ip_kset.kobj, "%d", le16_to_cpu(dhdr->die_id)); | ||
| 846 | ip_die_entry->ip_kset.kobj.kset = die_kset; | ||
| 847 | ip_die_entry->ip_kset.kobj.ktype = &ip_die_entry_ktype; | ||
| 848 | res = kset_register(&ip_die_entry->ip_kset); | ||
| 849 | if (res) { | ||
| 850 | DRM_ERROR("Couldn't register ip_die_entry kset"); | ||
| 851 | kfree(ip_die_entry); | ||
| 852 | return res; | ||
| 853 | } | ||
| 854 | |||
| 855 | amdgpu_discovery_sysfs_ips(adev, ip_die_entry, ip_offset, num_ips); | ||
| 856 | } | ||
| 857 | |||
| 858 | return 0; | ||
| 859 | } | ||
| 860 | |||
| 861 | static int amdgpu_discovery_sysfs_init(struct amdgpu_device *adev) | ||
| 862 | { | ||
| 863 | struct kset *die_kset; | ||
| 864 | int res, ii; | ||
| 865 | |||
| 866 | adev->ip_top = kzalloc(sizeof(*adev->ip_top), GFP_KERNEL); | ||
| 867 | if (!adev->ip_top) | ||
| 868 | return -ENOMEM; | ||
| 869 | |||
| 870 | adev->ip_top->adev = adev; | ||
| 871 | |||
| 872 | res = kobject_init_and_add(&adev->ip_top->kobj, &ip_discovery_ktype, | ||
| 873 | &adev->dev->kobj, "ip_discovery"); | ||
| 874 | if (res) { | ||
| 875 | DRM_ERROR("Couldn't init and add ip_discovery/"); | ||
| 876 | goto Err; | ||
| 877 | } | ||
| 878 | |||
| 879 | die_kset = &adev->ip_top->die_kset; | ||
| 880 | kobject_set_name(&die_kset->kobj, "%s", "die"); | ||
| 881 | die_kset->kobj.parent = &adev->ip_top->kobj; | ||
| 882 | die_kset->kobj.ktype = &die_kobj_ktype; | ||
| 883 | res = kset_register(&adev->ip_top->die_kset); | ||
| 884 | if (res) { | ||
| 885 | DRM_ERROR("Couldn't register die_kset"); | ||
| 886 | goto Err; | ||
| 887 | } | ||
| 888 | |||
| 889 | for (ii = 0; ii < ARRAY_SIZE(ip_hw_attr); ii++) | ||
| 890 | ip_hw_instance_attrs[ii] = &ip_hw_attr[ii].attr; | ||
| 891 | ip_hw_instance_attrs[ii] = NULL; | ||
| 892 | |||
| 893 | res = amdgpu_discovery_sysfs_recurse(adev); | ||
| 894 | |||
| 895 | return res; | ||
| 896 | Err: | ||
| 897 | kobject_put(&adev->ip_top->kobj); | ||
| 898 | return res; | ||
| 899 | } | ||
| 900 | |||
| 901 | /* -------------------------------------------------- */ | ||
| 902 | |||
| 903 | #define list_to_kobj(el) container_of(el, struct kobject, entry) | ||
| 904 | |||
| 905 | static void amdgpu_discovery_sysfs_ip_hw_free(struct ip_hw_id *ip_hw_id) | ||
| 906 | { | ||
| 907 | struct list_head *el, *tmp; | ||
| 908 | struct kset *hw_id_kset; | ||
| 909 | |||
| 910 | hw_id_kset = &ip_hw_id->hw_id_kset; | ||
| 911 | spin_lock(&hw_id_kset->list_lock); | ||
| 912 | list_for_each_prev_safe(el, tmp, &hw_id_kset->list) { | ||
| 913 | list_del_init(el); | ||
| 914 | spin_unlock(&hw_id_kset->list_lock); | ||
| 915 | /* kobject is embedded in ip_hw_instance */ | ||
| 916 | kobject_put(list_to_kobj(el)); | ||
| 917 | spin_lock(&hw_id_kset->list_lock); | ||
| 918 | } | ||
| 919 | spin_unlock(&hw_id_kset->list_lock); | ||
| 920 | kobject_put(&ip_hw_id->hw_id_kset.kobj); | ||
| 921 | } | ||
| 922 | |||
| 923 | static void amdgpu_discovery_sysfs_die_free(struct ip_die_entry *ip_die_entry) | ||
| 924 | { | ||
| 925 | struct list_head *el, *tmp; | ||
| 926 | struct kset *ip_kset; | ||
| 927 | |||
| 928 | ip_kset = &ip_die_entry->ip_kset; | ||
| 929 | spin_lock(&ip_kset->list_lock); | ||
| 930 | list_for_each_prev_safe(el, tmp, &ip_kset->list) { | ||
| 931 | list_del_init(el); | ||
| 932 | spin_unlock(&ip_kset->list_lock); | ||
| 933 | amdgpu_discovery_sysfs_ip_hw_free(to_ip_hw_id(list_to_kobj(el))); | ||
| 934 | spin_lock(&ip_kset->list_lock); | ||
| 935 | } | ||
| 936 | spin_unlock(&ip_kset->list_lock); | ||
| 937 | kobject_put(&ip_die_entry->ip_kset.kobj); | ||
| 938 | } | ||
| 939 | |||
| 940 | static void amdgpu_discovery_sysfs_fini(struct amdgpu_device *adev) | ||
| 941 | { | ||
| 942 | struct list_head *el, *tmp; | ||
| 943 | struct kset *die_kset; | ||
| 944 | |||
| 945 | die_kset = &adev->ip_top->die_kset; | ||
| 946 | spin_lock(&die_kset->list_lock); | ||
| 947 | list_for_each_prev_safe(el, tmp, &die_kset->list) { | ||
| 948 | list_del_init(el); | ||
| 949 | spin_unlock(&die_kset->list_lock); | ||
| 950 | amdgpu_discovery_sysfs_die_free(to_ip_die_entry(list_to_kobj(el))); | ||
| 951 | spin_lock(&die_kset->list_lock); | ||
| 952 | } | ||
| 953 | spin_unlock(&die_kset->list_lock); | ||
| 954 | kobject_put(&adev->ip_top->die_kset.kobj); | ||
| 955 | kobject_put(&adev->ip_top->kobj); | ||
| 956 | } | ||
| 957 | |||
| 958 | /* ================================================== */ | ||
| 959 | |||
| 385 | int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev) | 960 | int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev) |
| 386 | { | 961 | { |
| 387 | struct binary_header *bhdr; | 962 | struct binary_header *bhdr; |
| @@ -492,6 +1067,8 @@ next_ip: · 104 unmodified lines | |||
| 492 | } | 1067 | } |
| 493 | } | 1068 | } |
| 494 | 1069 | ||
| 1070 | amdgpu_discovery_sysfs_init(adev); | ||
| 1071 | |||
| 495 | return 0; | 1072 | return 0; |
| 496 | } | 1073 | } |
| 497 | 1074 | ||
| @@ -545,32 +1122,26 @@ int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id, int n · 47 unmodified lines | |||
| 545 | 1122 | ||
| 546 | void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev) | 1123 | void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev) |
| 547 | { | 1124 | { |
| 548 | struct binary_header *bhdr; | 1125 | int vcn_harvest_count = 0; |
| 549 | struct harvest_table *harvest_info; | ||
| 550 | int i, vcn_harvest_count = 0; | ||
| 551 | |||
| 552 | bhdr = (struct binary_header *)adev->mman.discovery_bin; | ||
| 553 | harvest_info = (struct harvest_table *)(adev->mman.discovery_bin + | ||
| 554 | le16_to_cpu(bhdr->table_list[HARVEST_INFO].offset)); | ||
| 555 | |||
| 556 | for (i = 0; i < 32; i++) { | ||
| 557 | if (le16_to_cpu(harvest_info->list[i].hw_id) == 0) | ||
| 558 | break; | ||
| 559 | 1126 | ||
| 560 | switch (le16_to_cpu(harvest_info->list[i].hw_id)) { | 1127 | /* |
| 561 | case VCN_HWID: | 1128 | * Harvest table does not fit Navi1x and legacy GPUs, |
| 562 | vcn_harvest_count++; | 1129 | * so read harvest bit per IP data structure to set |
| 563 | if (harvest_info->list[i].number_instance == 0) | 1130 | * harvest configuration. |
| 564 | adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN0; | 1131 | */ |
| 565 | else | 1132 | if (adev->ip_versions[GC_HWIP][0] < IP_VERSION(10, 2, 0)) { |
| 566 | adev->vcn.harvest_config |= AMDGPU_VCN_HARVEST_VCN1; | 1133 | if ((adev->pdev->device == 0x731E && |
| 567 | break; | 1134 | (adev->pdev->revision == 0xC6 || |
| 568 | case DMU_HWID: | 1135 | adev->pdev->revision == 0xC7)) || |
| 569 | adev->harvest_ip_mask |= AMD_HARVEST_IP_DMU_MASK; | 1136 | (adev->pdev->device == 0x7340 && |
| 570 | break; | 1137 | adev->pdev->revision == 0xC9) || |
| 571 | default: | 1138 | (adev->pdev->device == 0x7360 && |
| 572 | break; | 1139 | adev->pdev->revision == 0xC7)) |
| 573 | } | 1140 | amdgpu_discovery_read_harvest_bit_per_ip(adev, |
| 1141 | &vcn_harvest_count); | ||
| 1142 | } else { | ||
| 1143 | amdgpu_discovery_read_from_harvest_table(adev, | ||
| 1144 | &vcn_harvest_count); | ||
| 574 | } | 1145 | } |
| 575 | 1146 | ||
| 576 | amdgpu_discovery_harvest_config_quirk(adev); | 1147 | amdgpu_discovery_harvest_config_quirk(adev); |
| @@ -674,12 +1245,15 @@ static int amdgpu_discovery_set_common_ip_blocks(struct amdgpu_device *adev) · 97 unmodified lines | |||
| 674 | case IP_VERSION(10, 1, 1): | 1245 | case IP_VERSION(10, 1, 1): |
| 675 | case IP_VERSION(10, 1, 2): | 1246 | case IP_VERSION(10, 1, 2): |
| 676 | case IP_VERSION(10, 1, 3): | 1247 | case IP_VERSION(10, 1, 3): |
| 1248 | case IP_VERSION(10, 1, 4): | ||
| 677 | case IP_VERSION(10, 3, 0): | 1249 | case IP_VERSION(10, 3, 0): |
| 678 | case IP_VERSION(10, 3, 1): | 1250 | case IP_VERSION(10, 3, 1): |
| 679 | case IP_VERSION(10, 3, 2): | 1251 | case IP_VERSION(10, 3, 2): |
| 680 | case IP_VERSION(10, 3, 3): | 1252 | case IP_VERSION(10, 3, 3): |
| 681 | case IP_VERSION(10, 3, 4): | 1253 | case IP_VERSION(10, 3, 4): |
| 682 | case IP_VERSION(10, 3, 5): | 1254 | case IP_VERSION(10, 3, 5): |
| 1255 | case IP_VERSION(10, 3, 6): | ||
| 1256 | case IP_VERSION(10, 3, 7): | ||
| 683 | amdgpu_device_ip_block_add(adev, &nv_common_ip_block); | 1257 | amdgpu_device_ip_block_add(adev, &nv_common_ip_block); |
| 684 | break; | 1258 | break; |
| 685 | default: | 1259 | default: |
| @@ -709,12 +1283,15 @@ static int amdgpu_discovery_set_gmc_ip_blocks(struct amdgpu_device *adev) · 23 unmodified lines | |||
| 709 | case IP_VERSION(10, 1, 1): | 1283 | case IP_VERSION(10, 1, 1): |
| 710 | case IP_VERSION(10, 1, 2): | 1284 | case IP_VERSION(10, 1, 2): |
| 711 | case IP_VERSION(10, 1, 3): | 1285 | case IP_VERSION(10, 1, 3): |
| 1286 | case IP_VERSION(10, 1, 4): | ||
| 712 | case IP_VERSION(10, 3, 0): | 1287 | case IP_VERSION(10, 3, 0): |
| 713 | case IP_VERSION(10, 3, 1): | 1288 | case IP_VERSION(10, 3, 1): |
| 714 | case IP_VERSION(10, 3, 2): | 1289 | case IP_VERSION(10, 3, 2): |
| 715 | case IP_VERSION(10, 3, 3): | 1290 | case IP_VERSION(10, 3, 3): |
| 716 | case IP_VERSION(10, 3, 4): | 1291 | case IP_VERSION(10, 3, 4): |
| 717 | case IP_VERSION(10, 3, 5): | 1292 | case IP_VERSION(10, 3, 5): |
| 1293 | case IP_VERSION(10, 3, 6): | ||
| 1294 | case IP_VERSION(10, 3, 7): | ||
| 718 | amdgpu_device_ip_block_add(adev, &gmc_v10_0_ip_block); | 1295 | amdgpu_device_ip_block_add(adev, &gmc_v10_0_ip_block); |
| 719 | break; | 1296 | break; |
| 720 | default: | 1297 | default: |
| @@ -790,6 +1367,8 @@ static int amdgpu_discovery_set_psp_ip_blocks(struct amdgpu_device *adev) · 69 unmodified lines | |||
| 790 | case IP_VERSION(13, 0, 1): | 1367 | case IP_VERSION(13, 0, 1): |
| 791 | case IP_VERSION(13, 0, 2): | 1368 | case IP_VERSION(13, 0, 2): |
| 792 | case IP_VERSION(13, 0, 3): | 1369 | case IP_VERSION(13, 0, 3): |
| 1370 | case IP_VERSION(13, 0, 5): | ||
| 1371 | case IP_VERSION(13, 0, 8): | ||
| 793 | amdgpu_device_ip_block_add(adev, &psp_v13_0_ip_block); | 1372 | amdgpu_device_ip_block_add(adev, &psp_v13_0_ip_block); |
| 794 | break; | 1373 | break; |
| 795 | default: | 1374 | default: |
| @@ -831,6 +1410,8 @@ static int amdgpu_discovery_set_smu_ip_blocks(struct amdgpu_device *adev) · 35 unmodified lines | |||
| 831 | case IP_VERSION(13, 0, 1): | 1410 | case IP_VERSION(13, 0, 1): |
| 832 | case IP_VERSION(13, 0, 2): | 1411 | case IP_VERSION(13, 0, 2): |
| 833 | case IP_VERSION(13, 0, 3): | 1412 | case IP_VERSION(13, 0, 3): |
| 1413 | case IP_VERSION(13, 0, 5): | ||
| 1414 | case IP_VERSION(13, 0, 8): | ||
| 834 | amdgpu_device_ip_block_add(adev, &smu_v13_0_ip_block); | 1415 | amdgpu_device_ip_block_add(adev, &smu_v13_0_ip_block); |
| 835 | break; | 1416 | break; |
| 836 | default: | 1417 | default: |
| @@ -846,8 +1427,14 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev) · 9 unmodified lines | |||
| 846 | { | 1427 | { |
| 847 | if (adev->enable_virtual_display || amdgpu_sriov_vf(adev)) { | 1428 | if (adev->enable_virtual_display || amdgpu_sriov_vf(adev)) { |
| 848 | amdgpu_device_ip_block_add(adev, &amdgpu_vkms_ip_block); | 1429 | amdgpu_device_ip_block_add(adev, &amdgpu_vkms_ip_block); |
| 1430 | return 0; | ||
| 1431 | } | ||
| 1432 | |||
| 1433 | if (!amdgpu_device_has_dc_support(adev)) | ||
| 1434 | return 0; | ||
| 1435 | |||
| 849 | #if defined(CONFIG_DRM_AMD_DC) | 1436 | #if defined(CONFIG_DRM_AMD_DC) |
| 850 | } else if (adev->ip_versions[DCE_HWIP][0]) { | 1437 | if (adev->ip_versions[DCE_HWIP][0]) { |
| 851 | switch (adev->ip_versions[DCE_HWIP][0]) { | 1438 | switch (adev->ip_versions[DCE_HWIP][0]) { |
| 852 | case IP_VERSION(1, 0, 0): | 1439 | case IP_VERSION(1, 0, 0): |
| 853 | case IP_VERSION(1, 0, 1): | 1440 | case IP_VERSION(1, 0, 1): |
| @@ -861,6 +1448,8 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev) · 7 unmodified lines | |||
| 861 | case IP_VERSION(3, 0, 1): | 1448 | case IP_VERSION(3, 0, 1): |
| 862 | case IP_VERSION(3, 1, 2): | 1449 | case IP_VERSION(3, 1, 2): |
| 863 | case IP_VERSION(3, 1, 3): | 1450 | case IP_VERSION(3, 1, 3): |
| 1451 | case IP_VERSION(3, 1, 5): | ||
| 1452 | case IP_VERSION(3, 1, 6): | ||
| 864 | amdgpu_device_ip_block_add(adev, &dm_ip_block); | 1453 | amdgpu_device_ip_block_add(adev, &dm_ip_block); |
| 865 | break; | 1454 | break; |
| 866 | default: | 1455 | default: |
| @@ -882,8 +1471,8 @@ static int amdgpu_discovery_set_display_ip_blocks(struct amdgpu_device *adev) · 15 unmodified lines | |||
| 882 | adev->ip_versions[DCI_HWIP][0]); | 1471 | adev->ip_versions[DCI_HWIP][0]); |
| 883 | return -EINVAL; | 1472 | return -EINVAL; |
| 884 | } | 1473 | } |
| 885 | #endif | ||
| 886 | } | 1474 | } |
| 1475 | #endif | ||
| 887 | return 0; | 1476 | return 0; |
| 888 | } | 1477 | } |
| 889 | 1478 | ||
| @@ -904,12 +1493,15 @@ static int amdgpu_discovery_set_gc_ip_blocks(struct amdgpu_device *adev) · 14 unmodified lines | |||
| 904 | case IP_VERSION(10, 1, 2): | 1493 | case IP_VERSION(10, 1, 2): |
| 905 | case IP_VERSION(10, 1, 1): | 1494 | case IP_VERSION(10, 1, 1): |
| 906 | case IP_VERSION(10, 1, 3): | 1495 | case IP_VERSION(10, 1, 3): |
| 1496 | case IP_VERSION(10, 1, 4): | ||
| 907 | case IP_VERSION(10, 3, 0): | 1497 | case IP_VERSION(10, 3, 0): |
| 908 | case IP_VERSION(10, 3, 2): | 1498 | case IP_VERSION(10, 3, 2): |
| 909 | case IP_VERSION(10, 3, 1): | 1499 | case IP_VERSION(10, 3, 1): |
| 910 | case IP_VERSION(10, 3, 4): | 1500 | case IP_VERSION(10, 3, 4): |
| 911 | case IP_VERSION(10, 3, 5): | 1501 | case IP_VERSION(10, 3, 5): |
| 1502 | case IP_VERSION(10, 3, 6): | ||
| 912 | case IP_VERSION(10, 3, 3): | 1503 | case IP_VERSION(10, 3, 3): |
| 1504 | case IP_VERSION(10, 3, 7): | ||
| 913 | amdgpu_device_ip_block_add(adev, &gfx_v10_0_ip_block); | 1505 | amdgpu_device_ip_block_add(adev, &gfx_v10_0_ip_block); |
| 914 | break; | 1506 | break; |
| 915 | default: | 1507 | default: |
| @@ -944,8 +1536,10 @@ static int amdgpu_discovery_set_sdma_ip_blocks(struct amdgpu_device *adev) · 28 unmodified lines | |||
| 944 | case IP_VERSION(5, 2, 2): | 1536 | case IP_VERSION(5, 2, 2): |
| 945 | case IP_VERSION(5, 2, 4): | 1537 | case IP_VERSION(5, 2, 4): |
| 946 | case IP_VERSION(5, 2, 5): | 1538 | case IP_VERSION(5, 2, 5): |
| 1539 | case IP_VERSION(5, 2, 6): | ||
| 947 | case IP_VERSION(5, 2, 3): | 1540 | case IP_VERSION(5, 2, 3): |
| 948 | case IP_VERSION(5, 2, 1): | 1541 | case IP_VERSION(5, 2, 1): |
| 1542 | case IP_VERSION(5, 2, 7): | ||
| 949 | amdgpu_device_ip_block_add(adev, &sdma_v5_2_ip_block); | 1543 | amdgpu_device_ip_block_add(adev, &sdma_v5_2_ip_block); |
| 950 | break; | 1544 | break; |
| 951 | default: | 1545 | default: |
| @@ -1012,6 +1606,7 @@ static int amdgpu_discovery_set_mm_ip_blocks(struct amdgpu_device *adev) · 60 unmodified lines | |||
| 1012 | case IP_VERSION(3, 0, 0): | 1606 | case IP_VERSION(3, 0, 0): |
| 1013 | case IP_VERSION(3, 0, 16): | 1607 | case IP_VERSION(3, 0, 16): |
| 1014 | case IP_VERSION(3, 1, 1): | 1608 | case IP_VERSION(3, 1, 1): |
| 1609 | case IP_VERSION(3, 1, 2): | ||
| 1015 | case IP_VERSION(3, 0, 2): | 1610 | case IP_VERSION(3, 0, 2): |
| 1016 | case IP_VERSION(3, 0, 192): | 1611 | case IP_VERSION(3, 0, 192): |
| 1017 | amdgpu_device_ip_block_add(adev, &vcn_v3_0_ip_block); | 1612 | amdgpu_device_ip_block_add(adev, &vcn_v3_0_ip_block); |
| @@ -1038,12 +1633,14 @@ static int amdgpu_discovery_set_mes_ip_blocks(struct amdgpu_device *adev) · 20 unmodified lines | |||
| 1038 | case IP_VERSION(10, 1, 1): | 1633 | case IP_VERSION(10, 1, 1): |
| 1039 | case IP_VERSION(10, 1, 2): | 1634 | case IP_VERSION(10, 1, 2): |
| 1040 | case IP_VERSION(10, 1, 3): | 1635 | case IP_VERSION(10, 1, 3): |
| 1636 | case IP_VERSION(10, 1, 4): | ||
| 1041 | case IP_VERSION(10, 3, 0): | 1637 | case IP_VERSION(10, 3, 0): |
| 1042 | case IP_VERSION(10, 3, 1): | 1638 | case IP_VERSION(10, 3, 1): |
| 1043 | case IP_VERSION(10, 3, 2): | 1639 | case IP_VERSION(10, 3, 2): |
| 1044 | case IP_VERSION(10, 3, 3): | 1640 | case IP_VERSION(10, 3, 3): |
| 1045 | case IP_VERSION(10, 3, 4): | 1641 | case IP_VERSION(10, 3, 4): |
| 1046 | case IP_VERSION(10, 3, 5): | 1642 | case IP_VERSION(10, 3, 5): |
| 1643 | case IP_VERSION(10, 3, 6): | ||
| 1047 | amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block); | 1644 | amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block); |
| 1048 | break; | 1645 | break; |
| 1049 | default: | 1646 | default: |
| @@ -1217,11 +1814,6 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 167 unmodified lines | |||
| 1217 | return -EINVAL; | 1814 | return -EINVAL; |
| 1218 | 1815 | ||
| 1219 | amdgpu_discovery_harvest_ip(adev); | 1816 | amdgpu_discovery_harvest_ip(adev); |
| 1220 | |||
| 1221 | if (!adev->mman.discovery_bin) { | ||
| 1222 | DRM_ERROR("ip discovery uninitialized\n"); | ||
| 1223 | return -EINVAL; | ||
| 1224 | } | ||
| 1225 | break; | 1817 | break; |
| 1226 | } | 1818 | } |
| 1227 | 1819 | ||
| @@ -1242,6 +1834,7 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 14 unmodified lines | |||
| 1242 | case IP_VERSION(10, 1, 1): | 1834 | case IP_VERSION(10, 1, 1): |
| 1243 | case IP_VERSION(10, 1, 2): | 1835 | case IP_VERSION(10, 1, 2): |
| 1244 | case IP_VERSION(10, 1, 3): | 1836 | case IP_VERSION(10, 1, 3): |
| 1837 | case IP_VERSION(10, 1, 4): | ||
| 1245 | case IP_VERSION(10, 3, 0): | 1838 | case IP_VERSION(10, 3, 0): |
| 1246 | case IP_VERSION(10, 3, 2): | 1839 | case IP_VERSION(10, 3, 2): |
| 1247 | case IP_VERSION(10, 3, 4): | 1840 | case IP_VERSION(10, 3, 4): |
| @@ -1254,10 +1847,32 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 6 unmodified lines | |||
| 1254 | case IP_VERSION(10, 3, 3): | 1847 | case IP_VERSION(10, 3, 3): |
| 1255 | adev->family = AMDGPU_FAMILY_YC; | 1848 | adev->family = AMDGPU_FAMILY_YC; |
| 1256 | break; | 1849 | break; |
| 1850 | case IP_VERSION(10, 3, 6): | ||
| 1851 | adev->family = AMDGPU_FAMILY_GC_10_3_6; | ||
| 1852 | break; | ||
| 1853 | case IP_VERSION(10, 3, 7): | ||
| 1854 | adev->family = AMDGPU_FAMILY_GC_10_3_7; | ||
| 1855 | break; | ||
| 1257 | default: | 1856 | default: |
| 1258 | return -EINVAL; | 1857 | return -EINVAL; |
| 1259 | } | 1858 | } |
| 1260 | 1859 | ||
| 1860 | switch (adev->ip_versions[GC_HWIP][0]) { | ||
| 1861 | case IP_VERSION(9, 1, 0): | ||
| 1862 | case IP_VERSION(9, 2, 2): | ||
| 1863 | case IP_VERSION(9, 3, 0): | ||
| 1864 | case IP_VERSION(10, 1, 3): | ||
| 1865 | case IP_VERSION(10, 1, 4): | ||
| 1866 | case IP_VERSION(10, 3, 1): | ||
| 1867 | case IP_VERSION(10, 3, 3): | ||
| 1868 | case IP_VERSION(10, 3, 6): | ||
| 1869 | case IP_VERSION(10, 3, 7): | ||
| 1870 | adev->flags |= AMD_IS_APU; | ||
| 1871 | break; | ||
| 1872 | default: | ||
| 1873 | break; | ||
| 1874 | } | ||
| 1875 | |||
| 1261 | if (adev->ip_versions[XGMI_HWIP][0] == IP_VERSION(4, 8, 0)) | 1876 | if (adev->ip_versions[XGMI_HWIP][0] == IP_VERSION(4, 8, 0)) |
| 1262 | adev->gmc.xgmi.supported = true; | 1877 | adev->gmc.xgmi.supported = true; |
| 1263 | 1878 | ||
| @@ -1285,7 +1900,9 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 21 unmodified lines | |||
| 1285 | break; | 1900 | break; |
| 1286 | case IP_VERSION(7, 2, 0): | 1901 | case IP_VERSION(7, 2, 0): |
| 1287 | case IP_VERSION(7, 2, 1): | 1902 | case IP_VERSION(7, 2, 1): |
| 1903 | case IP_VERSION(7, 3, 0): | ||
| 1288 | case IP_VERSION(7, 5, 0): | 1904 | case IP_VERSION(7, 5, 0): |
| 1905 | case IP_VERSION(7, 5, 1): | ||
| 1289 | adev->nbio.funcs = &nbio_v7_2_funcs; | 1906 | adev->nbio.funcs = &nbio_v7_2_funcs; |
| 1290 | adev->nbio.hdp_flush_reg = &nbio_v7_2_hdp_flush_reg; | 1907 | adev->nbio.hdp_flush_reg = &nbio_v7_2_hdp_flush_reg; |
| 1291 | break; | 1908 | break; |
| @@ -1368,6 +1985,8 @@ int amdgpu_discovery_set_ip_blocks(struct amdgpu_device *adev) · 76 unmodified lines | |||
| 1368 | case IP_VERSION(11, 0, 11): | 1985 | case IP_VERSION(11, 0, 11): |
| 1369 | case IP_VERSION(11, 5, 0): | 1986 | case IP_VERSION(11, 5, 0): |
| 1370 | case IP_VERSION(13, 0, 1): | 1987 | case IP_VERSION(13, 0, 1): |
| 1988 | case IP_VERSION(13, 0, 9): | ||
| 1989 | case IP_VERSION(13, 0, 10): | ||
| 1371 | adev->smuio.funcs = &smuio_v11_0_6_funcs; | 1990 | adev->smuio.funcs = &smuio_v11_0_6_funcs; |
| 1372 | break; | 1991 | break; |
| 1373 | case IP_VERSION(13, 0, 2): | 1992 | case IP_VERSION(13, 0, 2): |
| @@ -506,28 +506,9 @@ uint32_t amdgpu_display_supported_domains(struct amdgpu_device *adev, · 505 unmodified lines | |||
| 506 | */ | 506 | */ |
| 507 | if ((bo_flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC) && | 507 | if ((bo_flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC) && |
| 508 | amdgpu_bo_support_uswc(bo_flags) && | 508 | amdgpu_bo_support_uswc(bo_flags) && |
| 509 | amdgpu_device_asic_has_dc_support(adev->asic_type)) { | 509 | amdgpu_device_asic_has_dc_support(adev->asic_type) && |
| 510 | switch (adev->asic_type) { | 510 | adev->mode_info.gpu_vm_support) |
| 511 | case CHIP_CARRIZO: | 511 | domain |= AMDGPU_GEM_DOMAIN_GTT; |
| 512 | case CHIP_STONEY: | ||
| 513 | domain |= AMDGPU_GEM_DOMAIN_GTT; | ||
| 514 | break; | ||
| 515 | case CHIP_RAVEN: | ||
| 516 | /* enable S/G on PCO and RV2 */ | ||
| 517 | if ((adev->apu_flags & AMD_APU_IS_RAVEN2) || | ||
| 518 | (adev->apu_flags & AMD_APU_IS_PICASSO)) | ||
| 519 | domain |= AMDGPU_GEM_DOMAIN_GTT; | ||
| 520 | break; | ||
| 521 | case CHIP_RENOIR: | ||
| 522 | case CHIP_VANGOGH: | ||
| 523 | case CHIP_YELLOW_CARP: | ||
| 524 | domain |= AMDGPU_GEM_DOMAIN_GTT; | ||
| 525 | break; | ||
| 526 | |||
| 527 | default: | ||
| 528 | break; | ||
| 529 | } | ||
| 530 | } | ||
| 531 | #endif | 512 | #endif |
| 532 | 513 | ||
| 533 | return domain; | 514 | return domain; |
| @@ -956,7 +937,7 @@ static int amdgpu_display_verify_sizes(struct amdgpu_framebuffer *rfb) · 422 unmodified lines | |||
| 956 | int ret; | 937 | int ret; |
| 957 | unsigned int i, block_width, block_height, block_size_log2; | 938 | unsigned int i, block_width, block_height, block_size_log2; |
| 958 | 939 | ||
| 959 | if (!rfb->base.dev->mode_config.allow_fb_modifiers) | 940 | if (rfb->base.dev->mode_config.fb_modifiers_not_supported) |
| 960 | return 0; | 941 | return 0; |
| 961 | 942 | ||
| 962 | for (i = 0; i < format_info->num_planes; ++i) { | 943 | for (i = 0; i < format_info->num_planes; ++i) { |
| @@ -1143,7 +1124,7 @@ int amdgpu_display_framebuffer_init(struct drm_device *dev, · 180 unmodified lines | |||
| 1143 | if (ret) | 1124 | if (ret) |
| 1144 | return ret; | 1125 | return ret; |
| 1145 | 1126 | ||
| 1146 | if (!dev->mode_config.allow_fb_modifiers) { | 1127 | if (dev->mode_config.fb_modifiers_not_supported && !adev->enable_virtual_display) { |
| 1147 | drm_WARN_ONCE(dev, adev->family >= AMDGPU_FAMILY_AI, | 1128 | drm_WARN_ONCE(dev, adev->family >= AMDGPU_FAMILY_AI, |
| 1148 | "GFX9+ requires FB check based on format modifier\n"); | 1129 | "GFX9+ requires FB check based on format modifier\n"); |
| 1149 | ret = check_tiling_flags_gfx6(rfb); | 1130 | ret = check_tiling_flags_gfx6(rfb); |
| @@ -1151,7 +1132,7 @@ int amdgpu_display_framebuffer_init(struct drm_device *dev, · 1 unmodified line | |||
| 1151 | return ret; | 1132 | return ret; |
| 1152 | } | 1133 | } |
| 1153 | 1134 | ||
| 1154 | if (dev->mode_config.allow_fb_modifiers && | 1135 | if (!dev->mode_config.fb_modifiers_not_supported && |
| 1155 | !(rfb->base.flags & DRM_MODE_FB_MODIFIERS)) { | 1136 | !(rfb->base.flags & DRM_MODE_FB_MODIFIERS)) { |
| 1156 | ret = convert_tiling_flags_to_modifier(rfb); | 1137 | ret = convert_tiling_flags_to_modifier(rfb); |
| 1157 | if (ret) { | 1138 | if (ret) { |
| @@ -99,9 +99,10 @@ · 98 unmodified lines | |||
| 99 | * - 3.42.0 - Add 16bpc fixed point display support | 99 | * - 3.42.0 - Add 16bpc fixed point display support |
| 100 | * - 3.43.0 - Add device hot plug/unplug support | 100 | * - 3.43.0 - Add device hot plug/unplug support |
| 101 | * - 3.44.0 - DCN3 supports DCC independent block settings: !64B && 128B, 64B && 128B | 101 | * - 3.44.0 - DCN3 supports DCC independent block settings: !64B && 128B, 64B && 128B |
| 102 | * - 3.45.0 - Add context ioctl stable pstate interface | ||
| 102 | */ | 103 | */ |
| 103 | #define KMS_DRIVER_MAJOR 3 | 104 | #define KMS_DRIVER_MAJOR 3 |
| 104 | #define KMS_DRIVER_MINOR 44 | 105 | #define KMS_DRIVER_MINOR 45 |
| 105 | #define KMS_DRIVER_PATCHLEVEL 0 | 106 | #define KMS_DRIVER_PATCHLEVEL 0 |
| 106 | 107 | ||
| 107 | int amdgpu_vram_limit; | 108 | int amdgpu_vram_limit; |
| @@ -109,8 +110,6 @@ int amdgpu_vis_vram_limit; · 1 unmodified line | |||
| 109 | int amdgpu_gart_size = -1; /* auto */ | 110 | int amdgpu_gart_size = -1; /* auto */ |
| 110 | int amdgpu_gtt_size = -1; /* auto */ | 111 | int amdgpu_gtt_size = -1; /* auto */ |
| 111 | int amdgpu_moverate = -1; /* auto */ | 112 | int amdgpu_moverate = -1; /* auto */ |
| 112 | int amdgpu_benchmarking; | ||
| 113 | int amdgpu_testing; | ||
| 114 | int amdgpu_audio = -1; | 113 | int amdgpu_audio = -1; |
| 115 | int amdgpu_disp_priority; | 114 | int amdgpu_disp_priority; |
| 116 | int amdgpu_hw_i2c; | 115 | int amdgpu_hw_i2c; |
| @@ -174,10 +173,10 @@ int amdgpu_mes; · 57 unmodified lines | |||
| 174 | int amdgpu_noretry = -1; | 173 | int amdgpu_noretry = -1; |
| 175 | int amdgpu_force_asic_type = -1; | 174 | int amdgpu_force_asic_type = -1; |
| 176 | int amdgpu_tmz = -1; /* auto */ | 175 | int amdgpu_tmz = -1; /* auto */ |
| 177 | uint amdgpu_freesync_vid_mode; | ||
| 178 | int amdgpu_reset_method = -1; /* auto */ | 176 | int amdgpu_reset_method = -1; /* auto */ |
| 179 | int amdgpu_num_kcq = -1; | 177 | int amdgpu_num_kcq = -1; |
| 180 | int amdgpu_smartshift_bias; | 178 | int amdgpu_smartshift_bias; |
| 179 | int amdgpu_use_xgmi_p2p = 1; | ||
| 181 | 180 | ||
| 182 | static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work); | 181 | static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work); |
| 183 | 182 | ||
| @@ -231,20 +230,6 @@ module_param_named(gttsize, amdgpu_gtt_size, int, 0600); · 47 unmodified lines | |||
| 231 | MODULE_PARM_DESC(moverate, "Maximum buffer migration rate in MB/s. (32, 64, etc., -1=auto, 0=1=disabled)"); | 230 | MODULE_PARM_DESC(moverate, "Maximum buffer migration rate in MB/s. (32, 64, etc., -1=auto, 0=1=disabled)"); |
| 232 | module_param_named(moverate, amdgpu_moverate, int, 0600); | 231 | module_param_named(moverate, amdgpu_moverate, int, 0600); |
| 233 | 232 | ||
| 234 | /** | ||
| 235 | * DOC: benchmark (int) | ||
| 236 | * Run benchmarks. The default is 0 (Skip benchmarks). | ||
| 237 | */ | ||
| 238 | MODULE_PARM_DESC(benchmark, "Run benchmark"); | ||
| 239 | module_param_named(benchmark, amdgpu_benchmarking, int, 0444); | ||
| 240 | |||
| 241 | /** | ||
| 242 | * DOC: test (int) | ||
| 243 | * Test BO GTT->VRAM and VRAM->GTT GPU copies. The default is 0 (Skip test, only set 1 to run test). | ||
| 244 | */ | ||
| 245 | MODULE_PARM_DESC(test, "Run tests"); | ||
| 246 | module_param_named(test, amdgpu_testing, int, 0444); | ||
| 247 | |||
| 248 | /** | 233 | /** |
| 249 | * DOC: audio (int) | 234 | * DOC: audio (int) |
| 250 | * Set HDMI/DPAudio. Only affects non-DC display handling. The default is -1 (Enabled), set 0 to disabled it. | 235 | * Set HDMI/DPAudio. Only affects non-DC display handling. The default is -1 (Enabled), set 0 to disabled it. |
| @@ -667,6 +652,13 @@ MODULE_PARM_DESC(force_asic_type, · 416 unmodified lines | |||
| 667 | "A non negative value used to specify the asic type for all supported GPUs"); | 652 | "A non negative value used to specify the asic type for all supported GPUs"); |
| 668 | module_param_named(force_asic_type, amdgpu_force_asic_type, int, 0444); | 653 | module_param_named(force_asic_type, amdgpu_force_asic_type, int, 0444); |
| 669 | 654 | ||
| 655 | /** | ||
| 656 | * DOC: use_xgmi_p2p (int) | ||
| 657 | * Enables/disables XGMI P2P interface (0 = disable, 1 = enable). | ||
| 658 | */ | ||
| 659 | MODULE_PARM_DESC(use_xgmi_p2p, | ||
| 660 | "Enable XGMI P2P interface (0 = disable; 1 = enable (default))"); | ||
| 661 | module_param_named(use_xgmi_p2p, amdgpu_use_xgmi_p2p, int, 0444); | ||
| 670 | 662 | ||
| 671 | 663 | ||
| 672 | #ifdef CONFIG_HSA_AMD | 664 | #ifdef CONFIG_HSA_AMD |
| @@ -740,7 +732,7 @@ MODULE_PARM_DESC(debug_largebar, · 67 unmodified lines | |||
| 740 | * systems with a broken CRAT table. | 732 | * systems with a broken CRAT table. |
| 741 | * | 733 | * |
| 742 | * Default is auto (according to asic type, iommu_v2, and crat table, to decide | 734 | * Default is auto (according to asic type, iommu_v2, and crat table, to decide |
| 743 | * whehter use CRAT) | 735 | * whether use CRAT) |
| 744 | */ | 736 | */ |
| 745 | int ignore_crat; | 737 | int ignore_crat; |
| 746 | module_param(ignore_crat, int, 0444); | 738 | module_param(ignore_crat, int, 0444); |
| @@ -843,37 +835,11 @@ module_param_named(backlight, amdgpu_backlight, bint, 0444); · 96 unmodified lines | |||
| 843 | MODULE_PARM_DESC(tmz, "Enable TMZ feature (-1 = auto (default), 0 = off, 1 = on)"); | 835 | MODULE_PARM_DESC(tmz, "Enable TMZ feature (-1 = auto (default), 0 = off, 1 = on)"); |
| 844 | module_param_named(tmz, amdgpu_tmz, int, 0444); | 836 | module_param_named(tmz, amdgpu_tmz, int, 0444); |
| 845 | 837 | ||
| 846 | /** | ||
| 847 | * DOC: freesync_video (uint) | ||
| 848 | * Enable the optimization to adjust front porch timing to achieve seamless | ||
| 849 | * mode change experience when setting a freesync supported mode for which full | ||
| 850 | * modeset is not needed. | ||
| 851 | * | ||
| 852 | * The Display Core will add a set of modes derived from the base FreeSync | ||
| 853 | * video mode into the corresponding connector's mode list based on commonly | ||
| 854 | * used refresh rates and VRR range of the connected display, when users enable | ||
| 855 | * this feature. From the userspace perspective, they can see a seamless mode | ||
| 856 | * change experience when the change between different refresh rates under the | ||
| 857 | * same resolution. Additionally, userspace applications such as Video playback | ||
| 858 | * can read this modeset list and change the refresh rate based on the video | ||
| 859 | * frame rate. Finally, the userspace can also derive an appropriate mode for a | ||
| 860 | * particular refresh rate based on the FreeSync Mode and add it to the | ||
| 861 | * connector's mode list. | ||
| 862 | * | ||
| 863 | * Note: This is an experimental feature. | ||
| 864 | * | ||
| 865 | * The default value: 0 (off). | ||
| 866 | */ | ||
| 867 | MODULE_PARM_DESC( | ||
| 868 | freesync_video, | ||
| 869 | "Enable freesync modesetting optimization feature (0 = off (default), 1 = on)"); | ||
| 870 | module_param_named(freesync_video, amdgpu_freesync_vid_mode, uint, 0444); | ||
| 871 | |||
| 872 | /** | 838 | /** |
| 873 | * DOC: reset_method (int) | 839 | * DOC: reset_method (int) |
| 874 | * GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco, 5 = pci) | 840 | * GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco) |
| 875 | */ | 841 | */ |
| 876 | MODULE_PARM_DESC(reset_method, "GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco/bamaco, 5 = pci)"); | 842 | MODULE_PARM_DESC(reset_method, "GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco/bamaco)"); |
| 877 | module_param_named(reset_method, amdgpu_reset_method, int, 0444); | 843 | module_param_named(reset_method, amdgpu_reset_method, int, 0444); |
| 878 | 844 | ||
| 879 | /** | 845 | /** |
| @@ -1942,13 +1908,14 @@ static const struct pci_device_id pciidlist[] = { · 1062 unmodified lines | |||
| 1942 | {0x1002, 0x73FF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_DIMGREY_CAVEFISH}, | 1908 | {0x1002, 0x73FF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_DIMGREY_CAVEFISH}, |
| 1943 | 1909 | ||
| 1944 | /* Aldebaran */ | 1910 | /* Aldebaran */ |
| 1945 | {0x1002, 0x7408, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT}, | 1911 | {0x1002, 0x7408, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN}, |
| 1946 | {0x1002, 0x740C, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT}, | 1912 | {0x1002, 0x740C, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN}, |
| 1947 | {0x1002, 0x740F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT}, | 1913 | {0x1002, 0x740F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN}, |
| 1948 | {0x1002, 0x7410, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT}, | 1914 | {0x1002, 0x7410, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN}, |
| 1949 | 1915 | ||
| 1950 | /* CYAN_SKILLFISH */ | 1916 | /* CYAN_SKILLFISH */ |
| 1951 | {0x1002, 0x13FE, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_CYAN_SKILLFISH|AMD_IS_APU}, | 1917 | {0x1002, 0x13FE, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_CYAN_SKILLFISH|AMD_IS_APU}, |
| 1918 | {0x1002, 0x143F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_CYAN_SKILLFISH|AMD_IS_APU}, | ||
| 1952 | 1919 | ||
| 1953 | /* BEIGE_GOBY */ | 1920 | /* BEIGE_GOBY */ |
| 1954 | {0x1002, 0x7420, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_BEIGE_GOBY}, | 1921 | {0x1002, 0x7420, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_BEIGE_GOBY}, |
| @@ -1994,6 +1961,28 @@ static bool amdgpu_is_fw_framebuffer(resource_size_t base, · 39 unmodified lines | |||
| 1994 | return found; | 1961 | return found; |
| 1995 | } | 1962 | } |
| 1996 | 1963 | ||
| 1964 | static void amdgpu_get_secondary_funcs(struct amdgpu_device *adev) | ||
| 1965 | { | ||
| 1966 | struct pci_dev *p = NULL; | ||
| 1967 | int i; | ||
| 1968 | |||
| 1969 | /* 0 - GPU | ||
| 1970 | * 1 - audio | ||
| 1971 | * 2 - USB | ||
| 1972 | * 3 - UCSI | ||
| 1973 | */ | ||
| 1974 | for (i = 1; i < 4; i++) { | ||
| 1975 | p = pci_get_domain_bus_and_slot(pci_domain_nr(adev->pdev->bus), | ||
| 1976 | adev->pdev->bus->number, i); | ||
| 1977 | if (p) { | ||
| 1978 | pm_runtime_get_sync(&p->dev); | ||
| 1979 | pm_runtime_mark_last_busy(&p->dev); | ||
| 1980 | pm_runtime_put_autosuspend(&p->dev); | ||
| 1981 | pci_dev_put(p); | ||
| 1982 | } | ||
| 1983 | } | ||
| 1984 | } | ||
| 1985 | |||
| 1997 | static int amdgpu_pci_probe(struct pci_dev *pdev, | 1986 | static int amdgpu_pci_probe(struct pci_dev *pdev, |
| 1998 | const struct pci_device_id *ent) | 1987 | const struct pci_device_id *ent) |
| 1999 | { | 1988 | { |
| @@ -2011,6 +2000,9 @@ static int amdgpu_pci_probe(struct pci_dev *pdev, · 11 unmodified lines | |||
| 2011 | return -ENODEV; | 2000 | return -ENODEV; |
| 2012 | } | 2001 | } |
| 2013 | 2002 | ||
| 2003 | if (amdgpu_aspm == -1 && !pcie_aspm_enabled(pdev)) | ||
| 2004 | amdgpu_aspm = 0; | ||
| 2005 | |||
| 2014 | if (amdgpu_virtual_display || | 2006 | if (amdgpu_virtual_display || |
| 2015 | amdgpu_device_asic_has_dc_support(flags & AMD_ASIC_MASK)) | 2007 | amdgpu_device_asic_has_dc_support(flags & AMD_ASIC_MASK)) |
| 2016 | supports_atomic = true; | 2008 | supports_atomic = true; |
| @@ -2126,6 +2118,48 @@ retry_init: · 109 unmodified lines | |||
| 2126 | if (ret) | 2118 | if (ret) |
| 2127 | DRM_ERROR("Creating debugfs files failed (%d).\n", ret); | 2119 | DRM_ERROR("Creating debugfs files failed (%d).\n", ret); |
| 2128 | 2120 | ||
| 2121 | if (adev->runpm) { | ||
| 2122 | /* only need to skip on ATPX */ | ||
| 2123 | if (amdgpu_device_supports_px(ddev)) | ||
| 2124 | dev_pm_set_driver_flags(ddev->dev, DPM_FLAG_NO_DIRECT_COMPLETE); | ||
| 2125 | /* we want direct complete for BOCO */ | ||
| 2126 | if (amdgpu_device_supports_boco(ddev)) | ||
| 2127 | dev_pm_set_driver_flags(ddev->dev, DPM_FLAG_SMART_PREPARE | | ||
| 2128 | DPM_FLAG_SMART_SUSPEND | | ||
| 2129 | DPM_FLAG_MAY_SKIP_RESUME); | ||
| 2130 | pm_runtime_use_autosuspend(ddev->dev); | ||
| 2131 | pm_runtime_set_autosuspend_delay(ddev->dev, 5000); | ||
| 2132 | |||
| 2133 | pm_runtime_allow(ddev->dev); | ||
| 2134 | |||
| 2135 | pm_runtime_mark_last_busy(ddev->dev); | ||
| 2136 | pm_runtime_put_autosuspend(ddev->dev); | ||
| 2137 | |||
| 2138 | /* | ||
| 2139 | * For runpm implemented via BACO, PMFW will handle the | ||
| 2140 | * timing for BACO in and out: | ||
| 2141 | * - put ASIC into BACO state only when both video and | ||
| 2142 | * audio functions are in D3 state. | ||
| 2143 | * - pull ASIC out of BACO state when either video or | ||
| 2144 | * audio function is in D0 state. | ||
| 2145 | * Also, at startup, PMFW assumes both functions are in | ||
| 2146 | * D0 state. | ||
| 2147 | * | ||
| 2148 | * So if snd driver was loaded prior to amdgpu driver | ||
| 2149 | * and audio function was put into D3 state, there will | ||
| 2150 | * be no PMFW-aware D-state transition(D0->D3) on runpm | ||
| 2151 | * suspend. Thus the BACO will be not correctly kicked in. | ||
| 2152 | * | ||
| 2153 | * Via amdgpu_get_secondary_funcs(), the audio dev is put | ||
| 2154 | * into D0 state. Then there will be a PMFW-aware D-state | ||
| 2155 | * transition(D0->D3) on runpm suspend. | ||
| 2156 | */ | ||
| 2157 | if (amdgpu_device_supports_baco(ddev) && | ||
| 2158 | !(adev->flags & AMD_IS_APU) && | ||
| 2159 | (adev->asic_type >= CHIP_NAVI10)) | ||
| 2160 | amdgpu_get_secondary_funcs(adev); | ||
| 2161 | } | ||
| 2162 | |||
| 2129 | return 0; | 2163 | return 0; |
| 2130 | 2164 | ||
| 2131 | err_pci: | 2165 | err_pci: |
| @@ -2137,8 +2171,15 @@ static void · 5 unmodified lines | |||
| 2137 | amdgpu_pci_remove(struct pci_dev *pdev) | 2171 | amdgpu_pci_remove(struct pci_dev *pdev) |
| 2138 | { | 2172 | { |
| 2139 | struct drm_device *dev = pci_get_drvdata(pdev); | 2173 | struct drm_device *dev = pci_get_drvdata(pdev); |
| 2174 | struct amdgpu_device *adev = drm_to_adev(dev); | ||
| 2140 | 2175 | ||
| 2141 | drm_dev_unplug(dev); | 2176 | drm_dev_unplug(dev); |
| 2177 | |||
| 2178 | if (adev->runpm) { | ||
| 2179 | pm_runtime_get_sync(dev->dev); | ||
| 2180 | pm_runtime_forbid(dev->dev); | ||
| 2181 | } | ||
| 2182 | |||
| 2142 | amdgpu_driver_unload_kms(dev); | 2183 | amdgpu_driver_unload_kms(dev); |
| 2143 | 2184 | ||
| 2144 | /* | 2185 | /* |
| @@ -2246,13 +2287,20 @@ static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work) · 101 unmodified lines | |||
| 2246 | static int amdgpu_pmops_prepare(struct device *dev) | 2287 | static int amdgpu_pmops_prepare(struct device *dev) |
| 2247 | { | 2288 | { |
| 2248 | struct drm_device *drm_dev = dev_get_drvdata(dev); | 2289 | struct drm_device *drm_dev = dev_get_drvdata(dev); |
| 2290 | struct amdgpu_device *adev = drm_to_adev(drm_dev); | ||
| 2249 | 2291 | ||
| 2250 | /* Return a positive number here so | 2292 | /* Return a positive number here so |
| 2251 | * DPM_FLAG_SMART_SUSPEND works properly | 2293 | * DPM_FLAG_SMART_SUSPEND works properly |
| 2252 | */ | 2294 | */ |
| 2253 | if (amdgpu_device_supports_boco(drm_dev)) | 2295 | if (amdgpu_device_supports_boco(drm_dev)) |
| 2254 | return pm_runtime_suspended(dev) && | 2296 | return pm_runtime_suspended(dev); |
| 2255 | pm_suspend_via_firmware(); | 2297 | |
| 2298 | /* if we will not support s3 or s2i for the device | ||
| 2299 | * then skip suspend | ||
| 2300 | */ | ||
| 2301 | if (!amdgpu_acpi_is_s0ix_active(adev) && | ||
| 2302 | !amdgpu_acpi_is_s3_active(adev)) | ||
| 2303 | return 1; | ||
| 2256 | 2304 | ||
| 2257 | return 0; | 2305 | return 0; |
| 2258 | } | 2306 | } |
| @@ -446,24 +446,18 @@ int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring, · 445 unmodified lines | |||
| 446 | * for the requested ring. | 446 | * for the requested ring. |
| 447 | * | 447 | * |
| 448 | * @ring: ring to init the fence driver on | 448 | * @ring: ring to init the fence driver on |
| 449 | * @num_hw_submission: number of entries on the hardware queue | ||
| 450 | * @sched_score: optional score atomic shared with other schedulers | ||
| 451 | * | 449 | * |
| 452 | * Init the fence driver for the requested ring (all asics). | 450 | * Init the fence driver for the requested ring (all asics). |
| 453 | * Helper function for amdgpu_fence_driver_init(). | 451 | * Helper function for amdgpu_fence_driver_init(). |
| 454 | */ | 452 | */ |
| 455 | int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, | 453 | int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring) |
| 456 | unsigned num_hw_submission, | ||
| 457 | atomic_t *sched_score) | ||
| 458 | { | 454 | { |
| 459 | struct amdgpu_device *adev = ring->adev; | 455 | struct amdgpu_device *adev = ring->adev; |
| 460 | long timeout; | ||
| 461 | int r; | ||
| 462 | 456 | ||
| 463 | if (!adev) | 457 | if (!adev) |
| 464 | return -EINVAL; | 458 | return -EINVAL; |
| 465 | 459 | ||
| 466 | if (!is_power_of_2(num_hw_submission)) | 460 | if (!is_power_of_2(ring->num_hw_submission)) |
| 467 | return -EINVAL; | 461 | return -EINVAL; |
| 468 | 462 | ||
| 469 | ring->fence_drv.cpu_addr = NULL; | 463 | ring->fence_drv.cpu_addr = NULL; |
| @@ -474,41 +468,14 @@ int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, · 4 unmodified lines | |||
| 474 | 468 | ||
| 475 | timer_setup(&ring->fence_drv.fallback_timer, amdgpu_fence_fallback, 0); | 469 | timer_setup(&ring->fence_drv.fallback_timer, amdgpu_fence_fallback, 0); |
| 476 | 470 | ||
| 477 | ring->fence_drv.num_fences_mask = num_hw_submission * 2 - 1; | 471 | ring->fence_drv.num_fences_mask = ring->num_hw_submission * 2 - 1; |
| 478 | spin_lock_init(&ring->fence_drv.lock); | 472 | spin_lock_init(&ring->fence_drv.lock); |
| 479 | ring->fence_drv.fences = kcalloc(num_hw_submission * 2, sizeof(void *), | 473 | ring->fence_drv.fences = kcalloc(ring->num_hw_submission * 2, sizeof(void *), |
| 480 | GFP_KERNEL); | 474 | GFP_KERNEL); |
| 475 | |||
| 481 | if (!ring->fence_drv.fences) | 476 | if (!ring->fence_drv.fences) |
| 482 | return -ENOMEM; | 477 | return -ENOMEM; |
| 483 | 478 | ||
| 484 | /* No need to setup the GPU scheduler for rings that don't need it */ | ||
| 485 | if (ring->no_scheduler) | ||
| 486 | return 0; | ||
| 487 | |||
| 488 | switch (ring->funcs->type) { | ||
| 489 | case AMDGPU_RING_TYPE_GFX: | ||
| 490 | timeout = adev->gfx_timeout; | ||
| 491 | break; | ||
| 492 | case AMDGPU_RING_TYPE_COMPUTE: | ||
| 493 | timeout = adev->compute_timeout; | ||
| 494 | break; | ||
| 495 | case AMDGPU_RING_TYPE_SDMA: | ||
| 496 | timeout = adev->sdma_timeout; | ||
| 497 | break; | ||
| 498 | default: | ||
| 499 | timeout = adev->video_timeout; | ||
| 500 | break; | ||
| 501 | } | ||
| 502 | |||
| 503 | r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, | ||
| 504 | num_hw_submission, amdgpu_job_hang_limit, | ||
| 505 | timeout, NULL, sched_score, ring->name); | ||
| 506 | if (r) { | ||
| 507 | DRM_ERROR("Failed to create scheduler on ring %s.\n", | ||
| 508 | ring->name); | ||
| 509 | return r; | ||
| 510 | } | ||
| 511 | |||
| 512 | return 0; | 479 | return 0; |
| 513 | } | 480 | } |
| 514 | 481 | ||
| @@ -30,7 +30,6 @@ · 29 unmodified lines | |||
| 30 | #include "amdgpu_eeprom.h" | 30 | #include "amdgpu_eeprom.h" |
| 31 | 31 | ||
| 32 | #define FRU_EEPROM_MADDR 0x60000 | 32 | #define FRU_EEPROM_MADDR 0x60000 |
| 33 | #define I2C_PRODUCT_INFO_OFFSET 0xC0 | ||
| 34 | 33 | ||
| 35 | static bool is_fru_eeprom_supported(struct amdgpu_device *adev) | 34 | static bool is_fru_eeprom_supported(struct amdgpu_device *adev) |
| 36 | { | 35 | { |
| @@ -40,7 +39,13 @@ static bool is_fru_eeprom_supported(struct amdgpu_device *adev) · 3 unmodified lines | |||
| 40 | */ | 39 | */ |
| 41 | struct atom_context *atom_ctx = adev->mode_info.atom_context; | 40 | struct atom_context *atom_ctx = adev->mode_info.atom_context; |
| 42 | 41 | ||
| 43 | /* VBIOS is of the format ###-DXXXYY-##. For SKU identification, | 42 | /* The i2c access is blocked on VF |
| 43 | * TODO: Need other way to get the info | ||
| 44 | */ | ||
| 45 | if (amdgpu_sriov_vf(adev)) | ||
| 46 | return false; | ||
| 47 | |||
| 48 | /* VBIOS is of the format ###-DXXXYYYY-##. For SKU identification, | ||
| 44 | * we can use just the "DXXX" portion. If there were more models, we | 49 | * we can use just the "DXXX" portion. If there were more models, we |
| 45 | * could convert the 3 characters to a hex integer and use a switch | 50 | * could convert the 3 characters to a hex integer and use a switch |
| 46 | * for ease/speed/readability. For now, 2 string comparisons are | 51 | * for ease/speed/readability. For now, 2 string comparisons are |
| @@ -59,17 +64,24 @@ static bool is_fru_eeprom_supported(struct amdgpu_device *adev) · 12 unmodified lines | |||
| 59 | case CHIP_ALDEBARAN: | 64 | case CHIP_ALDEBARAN: |
| 60 | /* All Aldebaran SKUs have the FRU */ | 65 | /* All Aldebaran SKUs have the FRU */ |
| 61 | return true; | 66 | return true; |
| 67 | case CHIP_SIENNA_CICHLID: | ||
| 68 | if (strnstr(atom_ctx->vbios_version, "D603", | ||
| 69 | sizeof(atom_ctx->vbios_version))) | ||
| 70 | return true; | ||
| 71 | else | ||
| 72 | return false; | ||
| 62 | default: | 73 | default: |
| 63 | return false; | 74 | return false; |
| 64 | } | 75 | } |
| 65 | } | 76 | } |
| 66 | 77 | ||
| 67 | static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr, | 78 | static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr, |
| 68 | unsigned char *buff) | 79 | unsigned char *buf, size_t buf_size) |
| 69 | { | 80 | { |
| 70 | int ret, size; | 81 | int ret; |
| 82 | u8 size; | ||
| 71 | 83 | ||
| 72 | ret = amdgpu_eeprom_read(&adev->pm.smu_i2c, addrptr, buff, 1); | 84 | ret = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addrptr, buf, 1); |
| 73 | if (ret < 1) { | 85 | if (ret < 1) { |
| 74 | DRM_WARN("FRU: Failed to get size field"); | 86 | DRM_WARN("FRU: Failed to get size field"); |
| 75 | return ret; | 87 | return ret; |
| @@ -78,9 +90,11 @@ static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr, · 2 unmodified lines | |||
| 78 | /* The size returned by the i2c requires subtraction of 0xC0 since the | 90 | /* The size returned by the i2c requires subtraction of 0xC0 since the |
| 79 | * size apparently always reports as 0xC0+actual size. | 91 | * size apparently always reports as 0xC0+actual size. |
| 80 | */ | 92 | */ |
| 81 | size = buff[0] - I2C_PRODUCT_INFO_OFFSET; | 93 | size = buf[0] & 0x3F; |
| 94 | size = min_t(size_t, size, buf_size); | ||
| 82 | 95 | ||
| 83 | ret = amdgpu_eeprom_read(&adev->pm.smu_i2c, addrptr + 1, buff, size); | 96 | ret = amdgpu_eeprom_read(adev->pm.fru_eeprom_i2c_bus, addrptr + 1, |
| 97 | buf, size); | ||
| 84 | if (ret < 1) { | 98 | if (ret < 1) { |
| 85 | DRM_WARN("FRU: Failed to get data field"); | 99 | DRM_WARN("FRU: Failed to get data field"); |
| 86 | return ret; | 100 | return ret; |
| @@ -91,19 +105,15 @@ static int amdgpu_fru_read_eeprom(struct amdgpu_device *adev, uint32_t addrptr, · 4 unmodified lines | |||
| 91 | 105 | ||
| 92 | int amdgpu_fru_get_product_info(struct amdgpu_device *adev) | 106 | int amdgpu_fru_get_product_info(struct amdgpu_device *adev) |
| 93 | { | 107 | { |
| 94 | unsigned char buff[AMDGPU_PRODUCT_NAME_LEN+2]; | 108 | unsigned char buf[AMDGPU_PRODUCT_NAME_LEN]; |
| 95 | u32 addrptr; | 109 | u32 addrptr; |
| 96 | int size, len; | 110 | int size, len; |
| 97 | int offset = 2; | ||
| 98 | 111 | ||
| 99 | if (!is_fru_eeprom_supported(adev)) | 112 | if (!is_fru_eeprom_supported(adev)) |
| 100 | return 0; | 113 | return 0; |
| 101 | 114 | ||
| 102 | if (adev->asic_type == CHIP_ALDEBARAN) | ||
| 103 | offset = 0; | ||
| 104 | |||
| 105 | /* If algo exists, it means that the i2c_adapter's initialized */ | 115 | /* If algo exists, it means that the i2c_adapter's initialized */ |
| 106 | if (!adev->pm.smu_i2c.algo) { | 116 | if (!adev->pm.fru_eeprom_i2c_bus || !adev->pm.fru_eeprom_i2c_bus->algo) { |
| 107 | DRM_WARN("Cannot access FRU, EEPROM accessor not initialized"); | 117 | DRM_WARN("Cannot access FRU, EEPROM accessor not initialized"); |
| 108 | return -ENODEV; | 118 | return -ENODEV; |
| 109 | } | 119 | } |
| @@ -121,7 +131,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 11 unmodified lines | |||
| 121 | * and the language field, so just start from 0xb, manufacturer size | 131 | * and the language field, so just start from 0xb, manufacturer size |
| 122 | */ | 132 | */ |
| 123 | addrptr = FRU_EEPROM_MADDR + 0xb; | 133 | addrptr = FRU_EEPROM_MADDR + 0xb; |
| 124 | size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | 134 | size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); |
| 125 | if (size < 1) { | 135 | if (size < 1) { |
| 126 | DRM_ERROR("Failed to read FRU Manufacturer, ret:%d", size); | 136 | DRM_ERROR("Failed to read FRU Manufacturer, ret:%d", size); |
| 127 | return -EINVAL; | 137 | return -EINVAL; |
| @@ -131,7 +141,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 3 unmodified lines | |||
| 131 | * size field being 1 byte. This pattern continues below. | 141 | * size field being 1 byte. This pattern continues below. |
| 132 | */ | 142 | */ |
| 133 | addrptr += size + 1; | 143 | addrptr += size + 1; |
| 134 | size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | 144 | size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); |
| 135 | if (size < 1) { | 145 | if (size < 1) { |
| 136 | DRM_ERROR("Failed to read FRU product name, ret:%d", size); | 146 | DRM_ERROR("Failed to read FRU product name, ret:%d", size); |
| 137 | return -EINVAL; | 147 | return -EINVAL; |
| @@ -143,12 +153,11 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 5 unmodified lines | |||
| 143 | AMDGPU_PRODUCT_NAME_LEN); | 153 | AMDGPU_PRODUCT_NAME_LEN); |
| 144 | len = AMDGPU_PRODUCT_NAME_LEN - 1; | 154 | len = AMDGPU_PRODUCT_NAME_LEN - 1; |
| 145 | } | 155 | } |
| 146 | /* Start at 2 due to buff using fields 0 and 1 for the address */ | 156 | memcpy(adev->product_name, buf, len); |
| 147 | memcpy(adev->product_name, &buff[offset], len); | ||
| 148 | adev->product_name[len] = '\0'; | 157 | adev->product_name[len] = '\0'; |
| 149 | 158 | ||
| 150 | addrptr += size + 1; | 159 | addrptr += size + 1; |
| 151 | size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | 160 | size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); |
| 152 | if (size < 1) { | 161 | if (size < 1) { |
| 153 | DRM_ERROR("Failed to read FRU product number, ret:%d", size); | 162 | DRM_ERROR("Failed to read FRU product number, ret:%d", size); |
| 154 | return -EINVAL; | 163 | return -EINVAL; |
| @@ -162,11 +171,11 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 7 unmodified lines | |||
| 162 | DRM_WARN("FRU Product Number is larger than 16 characters. This is likely a mistake"); | 171 | DRM_WARN("FRU Product Number is larger than 16 characters. This is likely a mistake"); |
| 163 | len = sizeof(adev->product_number) - 1; | 172 | len = sizeof(adev->product_number) - 1; |
| 164 | } | 173 | } |
| 165 | memcpy(adev->product_number, &buff[offset], len); | 174 | memcpy(adev->product_number, buf, len); |
| 166 | adev->product_number[len] = '\0'; | 175 | adev->product_number[len] = '\0'; |
| 167 | 176 | ||
| 168 | addrptr += size + 1; | 177 | addrptr += size + 1; |
| 169 | size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | 178 | size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); |
| 170 | 179 | ||
| 171 | if (size < 1) { | 180 | if (size < 1) { |
| 172 | DRM_ERROR("Failed to read FRU product version, ret:%d", size); | 181 | DRM_ERROR("Failed to read FRU product version, ret:%d", size); |
| @@ -174,7 +183,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 1 unmodified line | |||
| 174 | } | 183 | } |
| 175 | 184 | ||
| 176 | addrptr += size + 1; | 185 | addrptr += size + 1; |
| 177 | size = amdgpu_fru_read_eeprom(adev, addrptr, buff); | 186 | size = amdgpu_fru_read_eeprom(adev, addrptr, buf, sizeof(buf)); |
| 178 | 187 | ||
| 179 | if (size < 1) { | 188 | if (size < 1) { |
| 180 | DRM_ERROR("Failed to read FRU serial number, ret:%d", size); | 189 | DRM_ERROR("Failed to read FRU serial number, ret:%d", size); |
| @@ -189,7 +198,7 @@ int amdgpu_fru_get_product_info(struct amdgpu_device *adev) · 8 unmodified lines | |||
| 189 | DRM_WARN("FRU Serial Number is larger than 16 characters. This is likely a mistake"); | 198 | DRM_WARN("FRU Serial Number is larger than 16 characters. This is likely a mistake"); |
| 190 | len = sizeof(adev->serial) - 1; | 199 | len = sizeof(adev->serial) - 1; |
| 191 | } | 200 | } |
| 192 | memcpy(adev->serial, &buff[offset], len); | 201 | memcpy(adev->serial, buf, len); |
| 193 | adev->serial[len] = '\0'; | 202 | adev->serial[len] = '\0'; |
| 194 | 203 | ||
| 195 | return 0; | 204 | return 0; |
| @@ -150,7 +150,7 @@ void amdgpu_gart_table_vram_free(struct amdgpu_device *adev) · 149 unmodified lines | |||
| 150 | * replaces them with the dummy page (all asics). | 150 | * replaces them with the dummy page (all asics). |
| 151 | * Returns 0 for success, -EINVAL for failure. | 151 | * Returns 0 for success, -EINVAL for failure. |
| 152 | */ | 152 | */ |
| 153 | int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, | 153 | void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, |
| 154 | int pages) | 154 | int pages) |
| 155 | { | 155 | { |
| 156 | unsigned t; | 156 | unsigned t; |
| @@ -161,13 +161,11 @@ int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, · 4 unmodified lines | |||
| 161 | uint64_t flags = 0; | 161 | uint64_t flags = 0; |
| 162 | int idx; | 162 | int idx; |
| 163 | 163 | ||
| 164 | if (!adev->gart.ready) { | 164 | if (!adev->gart.ptr) |
| 165 | WARN(1, "trying to unbind memory from uninitialized GART !\n"); | 165 | return; |
| 166 | return -EINVAL; | ||
| 167 | } | ||
| 168 | 166 | ||
| 169 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) | 167 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) |
| 170 | return 0; | 168 | return; |
| 171 | 169 | ||
| 172 | t = offset / AMDGPU_GPU_PAGE_SIZE; | 170 | t = offset / AMDGPU_GPU_PAGE_SIZE; |
| 173 | p = t / AMDGPU_GPU_PAGES_IN_CPU_PAGE; | 171 | p = t / AMDGPU_GPU_PAGES_IN_CPU_PAGE; |
| @@ -188,7 +186,6 @@ int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, · 14 unmodified lines | |||
| 188 | amdgpu_gmc_flush_gpu_tlb(adev, 0, i, 0); | 186 | amdgpu_gmc_flush_gpu_tlb(adev, 0, i, 0); |
| 189 | 187 | ||
| 190 | drm_dev_exit(idx); | 188 | drm_dev_exit(idx); |
| 191 | return 0; | ||
| 192 | } | 189 | } |
| 193 | 190 | ||
| 194 | /** | 191 | /** |
| @@ -204,7 +201,7 @@ int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, · 9 unmodified lines | |||
| 204 | * Map the dma_addresses into GART entries (all asics). | 201 | * Map the dma_addresses into GART entries (all asics). |
| 205 | * Returns 0 for success, -EINVAL for failure. | 202 | * Returns 0 for success, -EINVAL for failure. |
| 206 | */ | 203 | */ |
| 207 | int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, | 204 | void amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, |
| 208 | int pages, dma_addr_t *dma_addr, uint64_t flags, | 205 | int pages, dma_addr_t *dma_addr, uint64_t flags, |
| 209 | void *dst) | 206 | void *dst) |
| 210 | { | 207 | { |
| @@ -212,13 +209,8 @@ int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, · 1 unmodified line | |||
| 212 | unsigned i, j, t; | 209 | unsigned i, j, t; |
| 213 | int idx; | 210 | int idx; |
| 214 | 211 | ||
| 215 | if (!adev->gart.ready) { | ||
| 216 | WARN(1, "trying to bind memory to uninitialized GART !\n"); | ||
| 217 | return -EINVAL; | ||
| 218 | } | ||
| 219 | |||
| 220 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) | 212 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) |
| 221 | return 0; | 213 | return; |
| 222 | 214 | ||
| 223 | t = offset / AMDGPU_GPU_PAGE_SIZE; | 215 | t = offset / AMDGPU_GPU_PAGE_SIZE; |
| 224 | 216 | ||
| @@ -230,7 +222,6 @@ int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, · 5 unmodified lines | |||
| 230 | } | 222 | } |
| 231 | } | 223 | } |
| 232 | drm_dev_exit(idx); | 224 | drm_dev_exit(idx); |
| 233 | return 0; | ||
| 234 | } | 225 | } |
| 235 | 226 | ||
| 236 | /** | 227 | /** |
| @@ -246,20 +237,14 @@ int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, · 9 unmodified lines | |||
| 246 | * (all asics). | 237 | * (all asics). |
| 247 | * Returns 0 for success, -EINVAL for failure. | 238 | * Returns 0 for success, -EINVAL for failure. |
| 248 | */ | 239 | */ |
| 249 | int amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset, | 240 | void amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset, |
| 250 | int pages, dma_addr_t *dma_addr, | 241 | int pages, dma_addr_t *dma_addr, |
| 251 | uint64_t flags) | 242 | uint64_t flags) |
| 252 | { | 243 | { |
| 253 | if (!adev->gart.ready) { | ||
| 254 | WARN(1, "trying to bind memory to uninitialized GART !\n"); | ||
| 255 | return -EINVAL; | ||
| 256 | } | ||
| 257 | |||
| 258 | if (!adev->gart.ptr) | 244 | if (!adev->gart.ptr) |
| 259 | return 0; | 245 | return; |
| 260 | 246 | ||
| 261 | return amdgpu_gart_map(adev, offset, pages, dma_addr, flags, | 247 | amdgpu_gart_map(adev, offset, pages, dma_addr, flags, adev->gart.ptr); |
| 262 | adev->gart.ptr); | ||
| 263 | } | 248 | } |
| 264 | 249 | ||
| 265 | /** | 250 | /** |
| @@ -274,6 +259,9 @@ void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev) · 8 unmodified lines | |||
| 274 | { | 259 | { |
| 275 | int i; | 260 | int i; |
| 276 | 261 | ||
| 262 | if (!adev->gart.ptr) | ||
| 263 | return; | ||
| 264 | |||
| 277 | mb(); | 265 | mb(); |
| 278 | amdgpu_device_flush_hdp(adev, NULL); | 266 | amdgpu_device_flush_hdp(adev, NULL); |
| 279 | for (i = 0; i < adev->num_vmhubs; i++) | 267 | for (i = 0; i < adev->num_vmhubs; i++) |
| @@ -46,7 +46,6 @@ struct amdgpu_gart { · 45 unmodified lines | |||
| 46 | unsigned num_gpu_pages; | 46 | unsigned num_gpu_pages; |
| 47 | unsigned num_cpu_pages; | 47 | unsigned num_cpu_pages; |
| 48 | unsigned table_size; | 48 | unsigned table_size; |
| 49 | bool ready; | ||
| 50 | 49 | ||
| 51 | /* Asic default pte flags */ | 50 | /* Asic default pte flags */ |
| 52 | uint64_t gart_pte_flags; | 51 | uint64_t gart_pte_flags; |
| @@ -58,12 +57,12 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); · 5 unmodified lines | |||
| 58 | void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); | 57 | void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); |
| 59 | int amdgpu_gart_init(struct amdgpu_device *adev); | 58 | int amdgpu_gart_init(struct amdgpu_device *adev); |
| 60 | void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); | 59 | void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); |
| 61 | int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, | 60 | void amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, |
| 62 | int pages); | 61 | int pages); |
| 63 | int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, | 62 | void amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, |
| 64 | int pages, dma_addr_t *dma_addr, uint64_t flags, | 63 | int pages, dma_addr_t *dma_addr, uint64_t flags, |
| 65 | void *dst); | 64 | void *dst); |
| 66 | int amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset, | 65 | void amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset, |
| 67 | int pages, dma_addr_t *dma_addr, uint64_t flags); | 66 | int pages, dma_addr_t *dma_addr, uint64_t flags); |
| 68 | void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev); | 67 | void amdgpu_gart_invalidate_tlb(struct amdgpu_device *adev); |
| 69 | #endif | 68 | #endif |
| @@ -222,7 +222,7 @@ static void amdgpu_gem_object_close(struct drm_gem_object *obj, · 221 unmodified lines | |||
| 222 | if (!bo_va || --bo_va->ref_count) | 222 | if (!bo_va || --bo_va->ref_count) |
| 223 | goto out_unlock; | 223 | goto out_unlock; |
| 224 | 224 | ||
| 225 | amdgpu_vm_bo_rmv(adev, bo_va); | 225 | amdgpu_vm_bo_del(adev, bo_va); |
| 226 | if (!amdgpu_vm_ready(vm)) | 226 | if (!amdgpu_vm_ready(vm)) |
| 227 | goto out_unlock; | 227 | goto out_unlock; |
| 228 | 228 | ||
| @@ -615,72 +615,40 @@ int amdgpu_get_gfx_off_status(struct amdgpu_device *adev, uint32_t *value) · 614 unmodified lines | |||
| 615 | 615 | ||
| 616 | mutex_lock(&adev->gfx.gfx_off_mutex); | 616 | mutex_lock(&adev->gfx.gfx_off_mutex); |
| 617 | 617 | ||
| 618 | r = smu_get_status_gfxoff(adev, value); | 618 | r = amdgpu_dpm_get_status_gfxoff(adev, value); |
| 619 | 619 | ||
| 620 | mutex_unlock(&adev->gfx.gfx_off_mutex); | 620 | mutex_unlock(&adev->gfx.gfx_off_mutex); |
| 621 | 621 | ||
| 622 | return r; | 622 | return r; |
| 623 | } | 623 | } |
| 624 | 624 | ||
| 625 | int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev) | 625 | int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block) |
| 626 | { | 626 | { |
| 627 | int r; | 627 | int r; |
| 628 | struct ras_fs_if fs_info = { | 628 | r = amdgpu_ras_block_late_init(adev, ras_block); |
| 629 | .sysfs_name = "gfx_err_count", | ||
| 630 | }; | ||
| 631 | struct ras_ih_if ih_info = { | ||
| 632 | .cb = amdgpu_gfx_process_ras_data_cb, | ||
| 633 | }; | ||
| 634 | |||
| 635 | if (!adev->gfx.ras_if) { | ||
| 636 | adev->gfx.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | ||
| 637 | if (!adev->gfx.ras_if) | ||
| 638 | return -ENOMEM; | ||
| 639 | adev->gfx.ras_if->block = AMDGPU_RAS_BLOCK__GFX; | ||
| 640 | adev->gfx.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 641 | adev->gfx.ras_if->sub_block_index = 0; | ||
| 642 | } | ||
| 643 | fs_info.head = ih_info.head = *adev->gfx.ras_if; | ||
| 644 | r = amdgpu_ras_late_init(adev, adev->gfx.ras_if, | ||
| 645 | &fs_info, &ih_info); | ||
| 646 | if (r) | 629 | if (r) |
| 647 | goto free; | 630 | return r; |
| 648 | 631 | ||
| 649 | if (amdgpu_ras_is_supported(adev, adev->gfx.ras_if->block)) { | 632 | if (amdgpu_ras_is_supported(adev, ras_block->block)) { |
| 650 | if (!amdgpu_persistent_edc_harvesting_supported(adev)) | 633 | if (!amdgpu_persistent_edc_harvesting_supported(adev)) |
| 651 | amdgpu_ras_reset_error_status(adev, AMDGPU_RAS_BLOCK__GFX); | 634 | amdgpu_ras_reset_error_status(adev, AMDGPU_RAS_BLOCK__GFX); |
| 652 | 635 | ||
| 653 | r = amdgpu_irq_get(adev, &adev->gfx.cp_ecc_error_irq, 0); | 636 | r = amdgpu_irq_get(adev, &adev->gfx.cp_ecc_error_irq, 0); |
| 654 | if (r) | 637 | if (r) |
| 655 | goto late_fini; | 638 | goto late_fini; |
| 656 | } else { | ||
| 657 | /* free gfx ras_if if ras is not supported */ | ||
| 658 | r = 0; | ||
| 659 | goto free; | ||
| 660 | } | 639 | } |
| 661 | 640 | ||
| 662 | return 0; | 641 | return 0; |
| 663 | late_fini: | 642 | late_fini: |
| 664 | amdgpu_ras_late_fini(adev, adev->gfx.ras_if, &ih_info); | 643 | amdgpu_ras_block_late_fini(adev, ras_block); |
| 665 | free: | ||
| 666 | kfree(adev->gfx.ras_if); | ||
| 667 | adev->gfx.ras_if = NULL; | ||
| 668 | return r; | 644 | return r; |
| 669 | } | 645 | } |
| 670 | 646 | ||
| 671 | void amdgpu_gfx_ras_fini(struct amdgpu_device *adev) | 647 | void amdgpu_gfx_ras_fini(struct amdgpu_device *adev) |
| 672 | { | 648 | { |
| 673 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX) && | 649 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX) && |
| 674 | adev->gfx.ras_if) { | 650 | adev->gfx.ras_if) |
| 675 | struct ras_common_if *ras_if = adev->gfx.ras_if; | 651 | amdgpu_ras_block_late_fini(adev, adev->gfx.ras_if); |
| 676 | struct ras_ih_if ih_info = { | ||
| 677 | .head = *ras_if, | ||
| 678 | .cb = amdgpu_gfx_process_ras_data_cb, | ||
| 679 | }; | ||
| 680 | |||
| 681 | amdgpu_ras_late_fini(adev, ras_if, &ih_info); | ||
| 682 | kfree(ras_if); | ||
| 683 | } | ||
| 684 | } | 652 | } |
| 685 | 653 | ||
| 686 | int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, | 654 | int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, |
| @@ -695,9 +663,9 @@ int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, · 8 unmodified lines | |||
| 695 | */ | 663 | */ |
| 696 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) { | 664 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) { |
| 697 | kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); | 665 | kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); |
| 698 | if (adev->gfx.ras_funcs && | 666 | if (adev->gfx.ras && adev->gfx.ras->ras_block.hw_ops && |
| 699 | adev->gfx.ras_funcs->query_ras_error_count) | 667 | adev->gfx.ras->ras_block.hw_ops->query_ras_error_count) |
| 700 | adev->gfx.ras_funcs->query_ras_error_count(adev, err_data); | 668 | adev->gfx.ras->ras_block.hw_ops->query_ras_error_count(adev, err_data); |
| 701 | amdgpu_ras_reset_gpu(adev); | 669 | amdgpu_ras_reset_gpu(adev); |
| 702 | } | 670 | } |
| 703 | return AMDGPU_RAS_SUCCESS; | 671 | return AMDGPU_RAS_SUCCESS; |
| @@ -852,19 +820,3 @@ int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev) · 148 unmodified lines | |||
| 852 | } | 820 | } |
| 853 | return amdgpu_num_kcq; | 821 | return amdgpu_num_kcq; |
| 854 | } | 822 | } |
| 855 | |||
| 856 | /* amdgpu_gfx_state_change_set - Handle gfx power state change set | ||
| 857 | * @adev: amdgpu_device pointer | ||
| 858 | * @state: gfx power state(1 -sGpuChangeState_D0Entry and 2 -sGpuChangeState_D3Entry) | ||
| 859 | * | ||
| 860 | */ | ||
| 861 | |||
| 862 | void amdgpu_gfx_state_change_set(struct amdgpu_device *adev, enum gfx_change_state state) | ||
| 863 | { | ||
| 864 | mutex_lock(&adev->pm.mutex); | ||
| 865 | if (adev->powerplay.pp_funcs && | ||
| 866 | adev->powerplay.pp_funcs->gfx_state_change_set) | ||
| 867 | ((adev)->powerplay.pp_funcs->gfx_state_change_set( | ||
| 868 | (adev)->powerplay.pp_handle, state)); | ||
| 869 | mutex_unlock(&adev->pm.mutex); | ||
| 870 | } | ||
| @@ -31,6 +31,7 @@ · 30 unmodified lines | |||
| 31 | #include "amdgpu_ring.h" | 31 | #include "amdgpu_ring.h" |
| 32 | #include "amdgpu_rlc.h" | 32 | #include "amdgpu_rlc.h" |
| 33 | #include "soc15.h" | 33 | #include "soc15.h" |
| 34 | #include "amdgpu_ras.h" | ||
| 34 | 35 | ||
| 35 | /* GFX current status */ | 36 | /* GFX current status */ |
| 36 | #define AMDGPU_GFX_NORMAL_MODE 0x00000000L | 37 | #define AMDGPU_GFX_NORMAL_MODE 0x00000000L |
| @@ -47,12 +48,6 @@ enum amdgpu_gfx_pipe_priority { · 10 unmodified lines | |||
| 47 | AMDGPU_GFX_PIPE_PRIO_HIGH = AMDGPU_RING_PRIO_2 | 48 | AMDGPU_GFX_PIPE_PRIO_HIGH = AMDGPU_RING_PRIO_2 |
| 48 | }; | 49 | }; |
| 49 | 50 | ||
| 50 | /* Argument for PPSMC_MSG_GpuChangeState */ | ||
| 51 | enum gfx_change_state { | ||
| 52 | sGpuChangeState_D0Entry = 1, | ||
| 53 | sGpuChangeState_D3Entry, | ||
| 54 | }; | ||
| 55 | |||
| 56 | #define AMDGPU_GFX_QUEUE_PRIORITY_MINIMUM 0 | 51 | #define AMDGPU_GFX_QUEUE_PRIORITY_MINIMUM 0 |
| 57 | #define AMDGPU_GFX_QUEUE_PRIORITY_MAXIMUM 15 | 52 | #define AMDGPU_GFX_QUEUE_PRIORITY_MAXIMUM 15 |
| 58 | 53 | ||
| @@ -204,16 +199,8 @@ struct amdgpu_cu_info { · 145 unmodified lines | |||
| 204 | uint32_t bitmap[4][4]; | 199 | uint32_t bitmap[4][4]; |
| 205 | }; | 200 | }; |
| 206 | 201 | ||
| 207 | struct amdgpu_gfx_ras_funcs { | 202 | struct amdgpu_gfx_ras { |
| 208 | int (*ras_late_init)(struct amdgpu_device *adev); | 203 | struct amdgpu_ras_block_object ras_block; |
| 209 | void (*ras_fini)(struct amdgpu_device *adev); | ||
| 210 | int (*ras_error_inject)(struct amdgpu_device *adev, | ||
| 211 | void *inject_if); | ||
| 212 | int (*query_ras_error_count)(struct amdgpu_device *adev, | ||
| 213 | void *ras_error_status); | ||
| 214 | void (*reset_ras_error_count)(struct amdgpu_device *adev); | ||
| 215 | void (*query_ras_error_status)(struct amdgpu_device *adev); | ||
| 216 | void (*reset_ras_error_status)(struct amdgpu_device *adev); | ||
| 217 | void (*enable_watchdog_timer)(struct amdgpu_device *adev); | 204 | void (*enable_watchdog_timer)(struct amdgpu_device *adev); |
| 218 | }; | 205 | }; |
| 219 | 206 | ||
| @@ -337,7 +324,7 @@ struct amdgpu_gfx { · 117 unmodified lines | |||
| 337 | 324 | ||
| 338 | /*ras */ | 325 | /*ras */ |
| 339 | struct ras_common_if *ras_if; | 326 | struct ras_common_if *ras_if; |
| 340 | const struct amdgpu_gfx_ras_funcs *ras_funcs; | 327 | struct amdgpu_gfx_ras *ras; |
| 341 | }; | 328 | }; |
| 342 | 329 | ||
| 343 | #define amdgpu_gfx_get_gpu_clock_counter(adev) (adev)->gfx.funcs->get_gpu_clock_counter((adev)) | 330 | #define amdgpu_gfx_get_gpu_clock_counter(adev) (adev)->gfx.funcs->get_gpu_clock_counter((adev)) |
| @@ -399,7 +386,7 @@ bool amdgpu_gfx_is_me_queue_enabled(struct amdgpu_device *adev, int me, · 55 unmodified lines | |||
| 399 | int pipe, int queue); | 386 | int pipe, int queue); |
| 400 | void amdgpu_gfx_off_ctrl(struct amdgpu_device *adev, bool enable); | 387 | void amdgpu_gfx_off_ctrl(struct amdgpu_device *adev, bool enable); |
| 401 | int amdgpu_get_gfx_off_status(struct amdgpu_device *adev, uint32_t *value); | 388 | int amdgpu_get_gfx_off_status(struct amdgpu_device *adev, uint32_t *value); |
| 402 | int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev); | 389 | int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block); |
| 403 | void amdgpu_gfx_ras_fini(struct amdgpu_device *adev); | 390 | void amdgpu_gfx_ras_fini(struct amdgpu_device *adev); |
| 404 | int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, | 391 | int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev, |
| 405 | void *err_data, | 392 | void *err_data, |
| @@ -410,5 +397,4 @@ int amdgpu_gfx_cp_ecc_error_irq(struct amdgpu_device *adev, · 4 unmodified lines | |||
| 410 | uint32_t amdgpu_kiq_rreg(struct amdgpu_device *adev, uint32_t reg); | 397 | uint32_t amdgpu_kiq_rreg(struct amdgpu_device *adev, uint32_t reg); |
| 411 | void amdgpu_kiq_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v); | 398 | void amdgpu_kiq_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v); |
| 412 | int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev); | 399 | int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev); |
| 413 | void amdgpu_gfx_state_change_set(struct amdgpu_device *adev, enum gfx_change_state state); | ||
| 414 | #endif | 400 | #endif |
| @@ -436,82 +436,35 @@ void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr, · 435 unmodified lines | |||
| 436 | } while (fault->timestamp < tmp); | 436 | } while (fault->timestamp < tmp); |
| 437 | } | 437 | } |
| 438 | 438 | ||
| 439 | int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev) | 439 | int amdgpu_gmc_ras_early_init(struct amdgpu_device *adev) |
| 440 | { | 440 | { |
| 441 | int r; | 441 | if (!adev->gmc.xgmi.connected_to_cpu) { |
| 442 | 442 | adev->gmc.xgmi.ras = &xgmi_ras; | |
| 443 | if (adev->umc.ras_funcs && | 443 | amdgpu_ras_register_ras_block(adev, &adev->gmc.xgmi.ras->ras_block); |
| 444 | adev->umc.ras_funcs->ras_late_init) { | 444 | adev->gmc.xgmi.ras_if = &adev->gmc.xgmi.ras->ras_block.ras_comm; |
| 445 | r = adev->umc.ras_funcs->ras_late_init(adev); | ||
| 446 | if (r) | ||
| 447 | return r; | ||
| 448 | } | ||
| 449 | |||
| 450 | if (adev->mmhub.ras_funcs && | ||
| 451 | adev->mmhub.ras_funcs->ras_late_init) { | ||
| 452 | r = adev->mmhub.ras_funcs->ras_late_init(adev); | ||
| 453 | if (r) | ||
| 454 | return r; | ||
| 455 | } | ||
| 456 | |||
| 457 | if (!adev->gmc.xgmi.connected_to_cpu) | ||
| 458 | adev->gmc.xgmi.ras_funcs = &xgmi_ras_funcs; | ||
| 459 | |||
| 460 | if (adev->gmc.xgmi.ras_funcs && | ||
| 461 | adev->gmc.xgmi.ras_funcs->ras_late_init) { | ||
| 462 | r = adev->gmc.xgmi.ras_funcs->ras_late_init(adev); | ||
| 463 | if (r) | ||
| 464 | return r; | ||
| 465 | } | ||
| 466 | |||
| 467 | if (adev->hdp.ras_funcs && | ||
| 468 | adev->hdp.ras_funcs->ras_late_init) { | ||
| 469 | r = adev->hdp.ras_funcs->ras_late_init(adev); | ||
| 470 | if (r) | ||
| 471 | return r; | ||
| 472 | } | ||
| 473 | |||
| 474 | if (adev->mca.mp0.ras_funcs && | ||
| 475 | adev->mca.mp0.ras_funcs->ras_late_init) { | ||
| 476 | r = adev->mca.mp0.ras_funcs->ras_late_init(adev); | ||
| 477 | if (r) | ||
| 478 | return r; | ||
| 479 | } | ||
| 480 | |||
| 481 | if (adev->mca.mp1.ras_funcs && | ||
| 482 | adev->mca.mp1.ras_funcs->ras_late_init) { | ||
| 483 | r = adev->mca.mp1.ras_funcs->ras_late_init(adev); | ||
| 484 | if (r) | ||
| 485 | return r; | ||
| 486 | } | 445 | } |
| 487 | 446 | ||
| 488 | if (adev->mca.mpio.ras_funcs && | 447 | return 0; |
| 489 | adev->mca.mpio.ras_funcs->ras_late_init) { | 448 | } |
| 490 | r = adev->mca.mpio.ras_funcs->ras_late_init(adev); | ||
| 491 | if (r) | ||
| 492 | return r; | ||
| 493 | } | ||
| 494 | 449 | ||
| 450 | int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev) | ||
| 451 | { | ||
| 495 | return 0; | 452 | return 0; |
| 496 | } | 453 | } |
| 497 | 454 | ||
| 498 | void amdgpu_gmc_ras_fini(struct amdgpu_device *adev) | 455 | void amdgpu_gmc_ras_fini(struct amdgpu_device *adev) |
| 499 | { | 456 | { |
| 500 | if (adev->umc.ras_funcs && | 457 | if (adev->umc.ras && adev->umc.ras->ras_block.ras_fini) |
| 501 | adev->umc.ras_funcs->ras_fini) | 458 | adev->umc.ras->ras_block.ras_fini(adev); |
| 502 | adev->umc.ras_funcs->ras_fini(adev); | ||
| 503 | 459 | ||
| 504 | if (adev->mmhub.ras_funcs && | 460 | if (adev->mmhub.ras && adev->mmhub.ras->ras_block.ras_fini) |
| 505 | adev->mmhub.ras_funcs->ras_fini) | 461 | adev->mmhub.ras->ras_block.ras_fini(adev); |
| 506 | adev->mmhub.ras_funcs->ras_fini(adev); | ||
| 507 | 462 | ||
| 508 | if (adev->gmc.xgmi.ras_funcs && | 463 | if (adev->gmc.xgmi.ras && adev->gmc.xgmi.ras->ras_block.ras_fini) |
| 509 | adev->gmc.xgmi.ras_funcs->ras_fini) | 464 | adev->gmc.xgmi.ras->ras_block.ras_fini(adev); |
| 510 | adev->gmc.xgmi.ras_funcs->ras_fini(adev); | ||
| 511 | 465 | ||
| 512 | if (adev->hdp.ras_funcs && | 466 | if (adev->hdp.ras && adev->hdp.ras->ras_block.ras_fini) |
| 513 | adev->hdp.ras_funcs->ras_fini) | 467 | adev->hdp.ras->ras_block.ras_fini(adev); |
| 514 | adev->hdp.ras_funcs->ras_fini(adev); | ||
| 515 | } | 468 | } |
| 516 | 469 | ||
| 517 | /* | 470 | /* |
| @@ -584,6 +537,7 @@ void amdgpu_gmc_tmz_set(struct amdgpu_device *adev) · 66 unmodified lines | |||
| 584 | case CHIP_NAVI12: | 537 | case CHIP_NAVI12: |
| 585 | case CHIP_VANGOGH: | 538 | case CHIP_VANGOGH: |
| 586 | case CHIP_YELLOW_CARP: | 539 | case CHIP_YELLOW_CARP: |
| 540 | case CHIP_IP_DISCOVERY: | ||
| 587 | /* Don't enable it by default yet. | 541 | /* Don't enable it by default yet. |
| 588 | */ | 542 | */ |
| 589 | if (amdgpu_tmz < 1) { | 543 | if (amdgpu_tmz < 1) { |
| @@ -831,3 +785,49 @@ void amdgpu_gmc_get_reserved_allocation(struct amdgpu_device *adev) · 241 unmodified lines | |||
| 831 | break; | 785 | break; |
| 832 | } | 786 | } |
| 833 | } | 787 | } |
| 788 | |||
| 789 | int amdgpu_gmc_vram_checking(struct amdgpu_device *adev) | ||
| 790 | { | ||
| 791 | struct amdgpu_bo *vram_bo = NULL; | ||
| 792 | uint64_t vram_gpu = 0; | ||
| 793 | void *vram_ptr = NULL; | ||
| 794 | |||
| 795 | int ret, size = 0x100000; | ||
| 796 | uint8_t cptr[10]; | ||
| 797 | |||
| 798 | ret = amdgpu_bo_create_kernel(adev, size, PAGE_SIZE, | ||
| 799 | AMDGPU_GEM_DOMAIN_VRAM, | ||
| 800 | &vram_bo, | ||
| 801 | &vram_gpu, | ||
| 802 | &vram_ptr); | ||
| 803 | if (ret) | ||
| 804 | return ret; | ||
| 805 | |||
| 806 | memset(vram_ptr, 0x86, size); | ||
| 807 | memset(cptr, 0x86, 10); | ||
| 808 | |||
| 809 | /** | ||
| 810 | * Check the start, the mid, and the end of the memory if the content of | ||
| 811 | * each byte is the pattern "0x86". If yes, we suppose the vram bo is | ||
| 812 | * workable. | ||
| 813 | * | ||
| 814 | * Note: If check the each byte of whole 1M bo, it will cost too many | ||
| 815 | * seconds, so here, we just pick up three parts for emulation. | ||
| 816 | */ | ||
| 817 | ret = memcmp(vram_ptr, cptr, 10); | ||
| 818 | if (ret) | ||
| 819 | return ret; | ||
| 820 | |||
| 821 | ret = memcmp(vram_ptr + (size / 2), cptr, 10); | ||
| 822 | if (ret) | ||
| 823 | return ret; | ||
| 824 | |||
| 825 | ret = memcmp(vram_ptr + size - 10, cptr, 10); | ||
| 826 | if (ret) | ||
| 827 | return ret; | ||
| 828 | |||
| 829 | amdgpu_bo_free_kernel(&vram_bo, &vram_gpu, | ||
| 830 | &vram_ptr); | ||
| 831 | |||
| 832 | return 0; | ||
| 833 | } | ||
| @@ -29,6 +29,7 @@ · 28 unmodified lines | |||
| 29 | #include <linux/types.h> | 29 | #include <linux/types.h> |
| 30 | 30 | ||
| 31 | #include "amdgpu_irq.h" | 31 | #include "amdgpu_irq.h" |
| 32 | #include "amdgpu_ras.h" | ||
| 32 | 33 | ||
| 33 | /* VA hole for 48bit addresses on Vega10 */ | 34 | /* VA hole for 48bit addresses on Vega10 */ |
| 34 | #define AMDGPU_GMC_HOLE_START 0x0000800000000000ULL | 35 | #define AMDGPU_GMC_HOLE_START 0x0000800000000000ULL |
| @@ -135,12 +136,8 @@ struct amdgpu_gmc_funcs { · 100 unmodified lines | |||
| 135 | unsigned int (*get_vbios_fb_size)(struct amdgpu_device *adev); | 136 | unsigned int (*get_vbios_fb_size)(struct amdgpu_device *adev); |
| 136 | }; | 137 | }; |
| 137 | 138 | ||
| 138 | struct amdgpu_xgmi_ras_funcs { | 139 | struct amdgpu_xgmi_ras { |
| 139 | int (*ras_late_init)(struct amdgpu_device *adev); | 140 | struct amdgpu_ras_block_object ras_block; |
| 140 | void (*ras_fini)(struct amdgpu_device *adev); | ||
| 141 | int (*query_ras_error_count)(struct amdgpu_device *adev, | ||
| 142 | void *ras_error_status); | ||
| 143 | void (*reset_ras_error_count)(struct amdgpu_device *adev); | ||
| 144 | }; | 141 | }; |
| 145 | 142 | ||
| 146 | struct amdgpu_xgmi { | 143 | struct amdgpu_xgmi { |
| @@ -159,7 +156,7 @@ struct amdgpu_xgmi { · 12 unmodified lines | |||
| 159 | struct ras_common_if *ras_if; | 156 | struct ras_common_if *ras_if; |
| 160 | bool connected_to_cpu; | 157 | bool connected_to_cpu; |
| 161 | bool pending_reset; | 158 | bool pending_reset; |
| 162 | const struct amdgpu_xgmi_ras_funcs *ras_funcs; | 159 | struct amdgpu_xgmi_ras *ras; |
| 163 | }; | 160 | }; |
| 164 | 161 | ||
| 165 | struct amdgpu_gmc { | 162 | struct amdgpu_gmc { |
| @@ -321,6 +318,7 @@ bool amdgpu_gmc_filter_faults(struct amdgpu_device *adev, · 155 unmodified lines | |||
| 321 | uint16_t pasid, uint64_t timestamp); | 318 | uint16_t pasid, uint64_t timestamp); |
| 322 | void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr, | 319 | void amdgpu_gmc_filter_faults_remove(struct amdgpu_device *adev, uint64_t addr, |
| 323 | uint16_t pasid); | 320 | uint16_t pasid); |
| 321 | int amdgpu_gmc_ras_early_init(struct amdgpu_device *adev); | ||
| 324 | int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev); | 322 | int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev); |
| 325 | void amdgpu_gmc_ras_fini(struct amdgpu_device *adev); | 323 | void amdgpu_gmc_ras_fini(struct amdgpu_device *adev); |
| 326 | int amdgpu_gmc_allocate_vm_inv_eng(struct amdgpu_device *adev); | 324 | int amdgpu_gmc_allocate_vm_inv_eng(struct amdgpu_device *adev); |
| @@ -339,4 +337,5 @@ void amdgpu_gmc_init_pdb0(struct amdgpu_device *adev); · 12 unmodified lines | |||
| 339 | uint64_t amdgpu_gmc_vram_mc2pa(struct amdgpu_device *adev, uint64_t mc_addr); | 337 | uint64_t amdgpu_gmc_vram_mc2pa(struct amdgpu_device *adev, uint64_t mc_addr); |
| 340 | uint64_t amdgpu_gmc_vram_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo); | 338 | uint64_t amdgpu_gmc_vram_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo); |
| 341 | uint64_t amdgpu_gmc_vram_cpu_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo); | 339 | uint64_t amdgpu_gmc_vram_cpu_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo); |
| 340 | int amdgpu_gmc_vram_checking(struct amdgpu_device *adev); | ||
| 342 | #endif | 341 | #endif |
| @@ -60,7 +60,7 @@ static ssize_t amdgpu_mem_info_gtt_total_show(struct device *dev, · 59 unmodified lines | |||
| 60 | struct ttm_resource_manager *man; | 60 | struct ttm_resource_manager *man; |
| 61 | 61 | ||
| 62 | man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); | 62 | man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); |
| 63 | return sysfs_emit(buf, "%llu\n", man->size * PAGE_SIZE); | 63 | return sysfs_emit(buf, "%llu\n", man->size); |
| 64 | } | 64 | } |
| 65 | 65 | ||
| 66 | /** | 66 | /** |
| @@ -77,8 +77,9 @@ static ssize_t amdgpu_mem_info_gtt_used_show(struct device *dev, · 10 unmodified lines | |||
| 77 | { | 77 | { |
| 78 | struct drm_device *ddev = dev_get_drvdata(dev); | 78 | struct drm_device *ddev = dev_get_drvdata(dev); |
| 79 | struct amdgpu_device *adev = drm_to_adev(ddev); | 79 | struct amdgpu_device *adev = drm_to_adev(ddev); |
| 80 | struct ttm_resource_manager *man = &adev->mman.gtt_mgr.manager; | ||
| 80 | 81 | ||
| 81 | return sysfs_emit(buf, "%llu\n", amdgpu_gtt_mgr_usage(&adev->mman.gtt_mgr)); | 82 | return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man)); |
| 82 | } | 83 | } |
| 83 | 84 | ||
| 84 | static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, | 85 | static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, |
| @@ -130,20 +131,17 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, · 45 unmodified lines | |||
| 130 | struct amdgpu_gtt_node *node; | 131 | struct amdgpu_gtt_node *node; |
| 131 | int r; | 132 | int r; |
| 132 | 133 | ||
| 133 | if (!(place->flags & TTM_PL_FLAG_TEMPORARY) && | ||
| 134 | atomic64_add_return(num_pages, &mgr->used) > man->size) { | ||
| 135 | atomic64_sub(num_pages, &mgr->used); | ||
| 136 | return -ENOSPC; | ||
| 137 | } | ||
| 138 | |||
| 139 | node = kzalloc(struct_size(node, base.mm_nodes, 1), GFP_KERNEL); | 134 | node = kzalloc(struct_size(node, base.mm_nodes, 1), GFP_KERNEL); |
| 140 | if (!node) { | 135 | if (!node) |
| 141 | r = -ENOMEM; | 136 | return -ENOMEM; |
| 142 | goto err_out; | ||
| 143 | } | ||
| 144 | 137 | ||
| 145 | node->tbo = tbo; | 138 | node->tbo = tbo; |
| 146 | ttm_resource_init(tbo, place, &node->base.base); | 139 | ttm_resource_init(tbo, place, &node->base.base); |
| 140 | if (!(place->flags & TTM_PL_FLAG_TEMPORARY) && | ||
| 141 | ttm_resource_manager_usage(man) > man->size) { | ||
| 142 | r = -ENOSPC; | ||
| 143 | goto err_free; | ||
| 144 | } | ||
| 147 | 145 | ||
| 148 | if (place->lpfn) { | 146 | if (place->lpfn) { |
| 149 | spin_lock(&mgr->lock); | 147 | spin_lock(&mgr->lock); |
| @@ -169,11 +167,6 @@ static int amdgpu_gtt_mgr_new(struct ttm_resource_manager *man, · 19 unmodified lines | |||
| 169 | err_free: | 167 | err_free: |
| 170 | ttm_resource_fini(man, &node->base.base); | 168 | ttm_resource_fini(man, &node->base.base); |
| 171 | kfree(node); | 169 | kfree(node); |
| 172 | |||
| 173 | err_out: | ||
| 174 | if (!(place->flags & TTM_PL_FLAG_TEMPORARY)) | ||
| 175 | atomic64_sub(num_pages, &mgr->used); | ||
| 176 | |||
| 177 | return r; | 170 | return r; |
| 178 | } | 171 | } |
| 179 | 172 | ||
| @@ -196,25 +189,10 @@ static void amdgpu_gtt_mgr_del(struct ttm_resource_manager *man, · 16 unmodified lines | |||
| 196 | drm_mm_remove_node(&node->base.mm_nodes[0]); | 189 | drm_mm_remove_node(&node->base.mm_nodes[0]); |
| 197 | spin_unlock(&mgr->lock); | 190 | spin_unlock(&mgr->lock); |
| 198 | 191 | ||
| 199 | if (!(res->placement & TTM_PL_FLAG_TEMPORARY)) | ||
| 200 | atomic64_sub(res->num_pages, &mgr->used); | ||
| 201 | |||
| 202 | ttm_resource_fini(man, res); | 192 | ttm_resource_fini(man, res); |
| 203 | kfree(node); | 193 | kfree(node); |
| 204 | } | 194 | } |
| 205 | 195 | ||
| 206 | /** | ||
| 207 | * amdgpu_gtt_mgr_usage - return usage of GTT domain | ||
| 208 | * | ||
| 209 | * @mgr: amdgpu_gtt_mgr pointer | ||
| 210 | * | ||
| 211 | * Return how many bytes are used in the GTT domain | ||
| 212 | */ | ||
| 213 | uint64_t amdgpu_gtt_mgr_usage(struct amdgpu_gtt_mgr *mgr) | ||
| 214 | { | ||
| 215 | return atomic64_read(&mgr->used) * PAGE_SIZE; | ||
| 216 | } | ||
| 217 | |||
| 218 | /** | 196 | /** |
| 219 | * amdgpu_gtt_mgr_recover - re-init gart | 197 | * amdgpu_gtt_mgr_recover - re-init gart |
| 220 | * | 198 | * |
| @@ -222,26 +200,21 @@ uint64_t amdgpu_gtt_mgr_usage(struct amdgpu_gtt_mgr *mgr) · 1 unmodified line | |||
| 222 | * | 200 | * |
| 223 | * Re-init the gart for each known BO in the GTT. | 201 | * Re-init the gart for each known BO in the GTT. |
| 224 | */ | 202 | */ |
| 225 | int amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr) | 203 | void amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr) |
| 226 | { | 204 | { |
| 227 | struct amdgpu_gtt_node *node; | 205 | struct amdgpu_gtt_node *node; |
| 228 | struct drm_mm_node *mm_node; | 206 | struct drm_mm_node *mm_node; |
| 229 | struct amdgpu_device *adev; | 207 | struct amdgpu_device *adev; |
| 230 | int r = 0; | ||
| 231 | 208 | ||
| 232 | adev = container_of(mgr, typeof(*adev), mman.gtt_mgr); | 209 | adev = container_of(mgr, typeof(*adev), mman.gtt_mgr); |
| 233 | spin_lock(&mgr->lock); | 210 | spin_lock(&mgr->lock); |
| 234 | drm_mm_for_each_node(mm_node, &mgr->mm) { | 211 | drm_mm_for_each_node(mm_node, &mgr->mm) { |
| 235 | node = container_of(mm_node, typeof(*node), base.mm_nodes[0]); | 212 | node = container_of(mm_node, typeof(*node), base.mm_nodes[0]); |
| 236 | r = amdgpu_ttm_recover_gart(node->tbo); | 213 | amdgpu_ttm_recover_gart(node->tbo); |
| 237 | if (r) | ||
| 238 | break; | ||
| 239 | } | 214 | } |
| 240 | spin_unlock(&mgr->lock); | 215 | spin_unlock(&mgr->lock); |
| 241 | 216 | ||
| 242 | amdgpu_gart_invalidate_tlb(adev); | 217 | amdgpu_gart_invalidate_tlb(adev); |
| 243 | |||
| 244 | return r; | ||
| 245 | } | 218 | } |
| 246 | 219 | ||
| 247 | /** | 220 | /** |
| @@ -260,9 +233,6 @@ static void amdgpu_gtt_mgr_debug(struct ttm_resource_manager *man, · 12 unmodified lines | |||
| 260 | spin_lock(&mgr->lock); | 233 | spin_lock(&mgr->lock); |
| 261 | drm_mm_print(&mgr->mm, printer); | 234 | drm_mm_print(&mgr->mm, printer); |
| 262 | spin_unlock(&mgr->lock); | 235 | spin_unlock(&mgr->lock); |
| 263 | |||
| 264 | drm_printf(printer, "man size:%llu pages, gtt used:%llu pages\n", | ||
| 265 | man->size, atomic64_read(&mgr->used)); | ||
| 266 | } | 236 | } |
| 267 | 237 | ||
| 268 | static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func = { | 238 | static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func = { |
| @@ -288,14 +258,12 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) · 19 unmodified lines | |||
| 288 | man->use_tt = true; | 258 | man->use_tt = true; |
| 289 | man->func = &amdgpu_gtt_mgr_func; | 259 | man->func = &amdgpu_gtt_mgr_func; |
| 290 | 260 | ||
| 291 | ttm_resource_manager_init(man, &adev->mman.bdev, | 261 | ttm_resource_manager_init(man, &adev->mman.bdev, gtt_size); |
| 292 | gtt_size >> PAGE_SHIFT); | ||
| 293 | 262 | ||
| 294 | start = AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS; | 263 | start = AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS; |
| 295 | size = (adev->gmc.gart_size >> PAGE_SHIFT) - start; | 264 | size = (adev->gmc.gart_size >> PAGE_SHIFT) - start; |
| 296 | drm_mm_init(&mgr->mm, start, size); | 265 | drm_mm_init(&mgr->mm, start, size); |
| 297 | spin_lock_init(&mgr->lock); | 266 | spin_lock_init(&mgr->lock); |
| 298 | atomic64_set(&mgr->used, 0); | ||
| 299 | 267 | ||
| 300 | ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); | 268 | ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); |
| 301 | ttm_resource_manager_set_used(man, true); | 269 | ttm_resource_manager_set_used(man, true); |
| @@ -24,45 +24,9 @@ · 23 unmodified lines | |||
| 24 | #include "amdgpu.h" | 24 | #include "amdgpu.h" |
| 25 | #include "amdgpu_ras.h" | 25 | #include "amdgpu_ras.h" |
| 26 | 26 | ||
| 27 | int amdgpu_hdp_ras_late_init(struct amdgpu_device *adev) | ||
| 28 | { | ||
| 29 | int r; | ||
| 30 | struct ras_ih_if ih_info = { | ||
| 31 | .cb = NULL, | ||
| 32 | }; | ||
| 33 | struct ras_fs_if fs_info = { | ||
| 34 | .sysfs_name = "hdp_err_count", | ||
| 35 | }; | ||
| 36 | |||
| 37 | if (!adev->hdp.ras_if) { | ||
| 38 | adev->hdp.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | ||
| 39 | if (!adev->hdp.ras_if) | ||
| 40 | return -ENOMEM; | ||
| 41 | adev->hdp.ras_if->block = AMDGPU_RAS_BLOCK__HDP; | ||
| 42 | adev->hdp.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 43 | adev->hdp.ras_if->sub_block_index = 0; | ||
| 44 | } | ||
| 45 | ih_info.head = fs_info.head = *adev->hdp.ras_if; | ||
| 46 | r = amdgpu_ras_late_init(adev, adev->hdp.ras_if, | ||
| 47 | &fs_info, &ih_info); | ||
| 48 | if (r || !amdgpu_ras_is_supported(adev, adev->hdp.ras_if->block)) { | ||
| 49 | kfree(adev->hdp.ras_if); | ||
| 50 | adev->hdp.ras_if = NULL; | ||
| 51 | } | ||
| 52 | |||
| 53 | return r; | ||
| 54 | } | ||
| 55 | |||
| 56 | void amdgpu_hdp_ras_fini(struct amdgpu_device *adev) | 27 | void amdgpu_hdp_ras_fini(struct amdgpu_device *adev) |
| 57 | { | 28 | { |
| 58 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__HDP) && | 29 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__HDP) && |
| 59 | adev->hdp.ras_if) { | 30 | adev->hdp.ras_if) |
| 60 | struct ras_common_if *ras_if = adev->hdp.ras_if; | 31 | amdgpu_ras_block_late_fini(adev, adev->hdp.ras_if); |
| 61 | struct ras_ih_if ih_info = { | ||
| 62 | .cb = NULL, | ||
| 63 | }; | ||
| 64 | |||
| 65 | amdgpu_ras_late_fini(adev, ras_if, &ih_info); | ||
| 66 | kfree(ras_if); | ||
| 67 | } | ||
| 68 | } | 32 | } |
| @@ -22,13 +22,10 @@ · 21 unmodified lines | |||
| 22 | */ | 22 | */ |
| 23 | #ifndef __AMDGPU_HDP_H__ | 23 | #ifndef __AMDGPU_HDP_H__ |
| 24 | #define __AMDGPU_HDP_H__ | 24 | #define __AMDGPU_HDP_H__ |
| 25 | #include "amdgpu_ras.h" | ||
| 25 | 26 | ||
| 26 | struct amdgpu_hdp_ras_funcs { | 27 | struct amdgpu_hdp_ras { |
| 27 | int (*ras_late_init)(struct amdgpu_device *adev); | 28 | struct amdgpu_ras_block_object ras_block; |
| 28 | void (*ras_fini)(struct amdgpu_device *adev); | ||
| 29 | void (*query_ras_error_count)(struct amdgpu_device *adev, | ||
| 30 | void *ras_error_status); | ||
| 31 | void (*reset_ras_error_count)(struct amdgpu_device *adev); | ||
| 32 | }; | 29 | }; |
| 33 | 30 | ||
| 34 | struct amdgpu_hdp_funcs { | 31 | struct amdgpu_hdp_funcs { |
| @@ -43,9 +40,9 @@ struct amdgpu_hdp_funcs { · 8 unmodified lines | |||
| 43 | struct amdgpu_hdp { | 40 | struct amdgpu_hdp { |
| 44 | struct ras_common_if *ras_if; | 41 | struct ras_common_if *ras_if; |
| 45 | const struct amdgpu_hdp_funcs *funcs; | 42 | const struct amdgpu_hdp_funcs *funcs; |
| 46 | const struct amdgpu_hdp_ras_funcs *ras_funcs; | 43 | struct amdgpu_hdp_ras *ras; |
| 47 | }; | 44 | }; |
| 48 | 45 | ||
| 49 | int amdgpu_hdp_ras_late_init(struct amdgpu_device *adev); | 46 | int amdgpu_hdp_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block); |
| 50 | void amdgpu_hdp_ras_fini(struct amdgpu_device *adev); | 47 | void amdgpu_hdp_ras_fini(struct amdgpu_device *adev); |
| 51 | #endif /* __AMDGPU_HDP_H__ */ | 48 | #endif /* __AMDGPU_HDP_H__ */ |
| @@ -199,13 +199,13 @@ static irqreturn_t amdgpu_irq_handler(int irq, void *arg) · 198 unmodified lines | |||
| 199 | * ack the interrupt if it is there | 199 | * ack the interrupt if it is there |
| 200 | */ | 200 | */ |
| 201 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF)) { | 201 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF)) { |
| 202 | if (adev->nbio.ras_funcs && | 202 | if (adev->nbio.ras && |
| 203 | adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring) | 203 | adev->nbio.ras->handle_ras_controller_intr_no_bifring) |
| 204 | adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring(adev); | 204 | adev->nbio.ras->handle_ras_controller_intr_no_bifring(adev); |
| 205 | 205 | ||
| 206 | if (adev->nbio.ras_funcs && | 206 | if (adev->nbio.ras && |
| 207 | adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring) | 207 | adev->nbio.ras->handle_ras_err_event_athub_intr_no_bifring) |
| 208 | adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring(adev); | 208 | adev->nbio.ras->handle_ras_err_event_athub_intr_no_bifring(adev); |
| 209 | } | 209 | } |
| 210 | 210 | ||
| 211 | return ret; | 211 | return ret; |
| @@ -37,6 +37,7 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) · 36 unmodified lines | |||
| 37 | struct amdgpu_task_info ti; | 37 | struct amdgpu_task_info ti; |
| 38 | struct amdgpu_device *adev = ring->adev; | 38 | struct amdgpu_device *adev = ring->adev; |
| 39 | int idx; | 39 | int idx; |
| 40 | int r; | ||
| 40 | 41 | ||
| 41 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) { | 42 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) { |
| 42 | DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s", | 43 | DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s", |
| @@ -63,7 +64,9 @@ static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job) · 20 unmodified lines | |||
| 63 | ti.process_name, ti.tgid, ti.task_name, ti.pid); | 64 | ti.process_name, ti.tgid, ti.task_name, ti.pid); |
| 64 | 65 | ||
| 65 | if (amdgpu_device_should_recover_gpu(ring->adev)) { | 66 | if (amdgpu_device_should_recover_gpu(ring->adev)) { |
| 66 | amdgpu_device_gpu_recover(ring->adev, job); | 67 | r = amdgpu_device_gpu_recover_imp(ring->adev, job); |
| 68 | if (r) | ||
| 69 | DRM_ERROR("GPU Recovery Failed: %d\n", r); | ||
| 67 | } else { | 70 | } else { |
| 68 | drm_sched_suspend_timeout(&ring->sched); | 71 | drm_sched_suspend_timeout(&ring->sched); |
| 69 | if (amdgpu_sriov_vf(adev)) | 72 | if (amdgpu_sriov_vf(adev)) |
| @@ -87,11 +87,6 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) · 86 unmodified lines | |||
| 87 | if (adev->rmmio == NULL) | 87 | if (adev->rmmio == NULL) |
| 88 | return; | 88 | return; |
| 89 | 89 | ||
| 90 | if (adev->runpm) { | ||
| 91 | pm_runtime_get_sync(dev->dev); | ||
| 92 | pm_runtime_forbid(dev->dev); | ||
| 93 | } | ||
| 94 | |||
| 95 | if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DRV_UNLOAD)) | 90 | if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DRV_UNLOAD)) |
| 96 | DRM_WARN("smart shift update failed\n"); | 91 | DRM_WARN("smart shift update failed\n"); |
| 97 | 92 | ||
| @@ -124,22 +119,6 @@ void amdgpu_register_gpu_instance(struct amdgpu_device *adev) · 26 unmodified lines | |||
| 124 | mutex_unlock(&mgpu_info.mutex); | 119 | mutex_unlock(&mgpu_info.mutex); |
| 125 | } | 120 | } |
| 126 | 121 | ||
| 127 | static void amdgpu_get_audio_func(struct amdgpu_device *adev) | ||
| 128 | { | ||
| 129 | struct pci_dev *p = NULL; | ||
| 130 | |||
| 131 | p = pci_get_domain_bus_and_slot(pci_domain_nr(adev->pdev->bus), | ||
| 132 | adev->pdev->bus->number, 1); | ||
| 133 | if (p) { | ||
| 134 | pm_runtime_get_sync(&p->dev); | ||
| 135 | |||
| 136 | pm_runtime_mark_last_busy(&p->dev); | ||
| 137 | pm_runtime_put_autosuspend(&p->dev); | ||
| 138 | |||
| 139 | pci_dev_put(p); | ||
| 140 | } | ||
| 141 | } | ||
| 142 | |||
| 143 | /** | 122 | /** |
| 144 | * amdgpu_driver_load_kms - Main load function for KMS. | 123 | * amdgpu_driver_load_kms - Main load function for KMS. |
| 145 | * | 124 | * |
| @@ -152,21 +131,10 @@ static void amdgpu_get_audio_func(struct amdgpu_device *adev) · 6 unmodified lines | |||
| 152 | int amdgpu_driver_load_kms(struct amdgpu_device *adev, unsigned long flags) | 131 | int amdgpu_driver_load_kms(struct amdgpu_device *adev, unsigned long flags) |
| 153 | { | 132 | { |
| 154 | struct drm_device *dev; | 133 | struct drm_device *dev; |
| 155 | struct pci_dev *parent; | ||
| 156 | int r, acpi_status; | 134 | int r, acpi_status; |
| 157 | 135 | ||
| 158 | dev = adev_to_drm(adev); | 136 | dev = adev_to_drm(adev); |
| 159 | 137 | ||
| 160 | if (amdgpu_has_atpx() && | ||
| 161 | (amdgpu_is_atpx_hybrid() || | ||
| 162 | amdgpu_has_atpx_dgpu_power_cntl()) && | ||
| 163 | ((flags & AMD_IS_APU) == 0) && | ||
| 164 | !pci_is_thunderbolt_attached(to_pci_dev(dev->dev))) | ||
| 165 | flags |= AMD_IS_PX; | ||
| 166 | |||
| 167 | parent = pci_upstream_bridge(adev->pdev); | ||
| 168 | adev->has_pr3 = parent ? pci_pr3_present(parent) : false; | ||
| 169 | |||
| 170 | /* amdgpu_device_init should report only fatal error | 138 | /* amdgpu_device_init should report only fatal error |
| 171 | * like memory allocation failure or iomapping failure, | 139 | * like memory allocation failure or iomapping failure, |
| 172 | * or memory manager initialization failure, it must | 140 | * or memory manager initialization failure, it must |
| @@ -224,58 +192,12 @@ int amdgpu_driver_load_kms(struct amdgpu_device *adev, unsigned long flags) · 51 unmodified lines | |||
| 224 | if (acpi_status) | 192 | if (acpi_status) |
| 225 | dev_dbg(dev->dev, "Error during ACPI methods call\n"); | 193 | dev_dbg(dev->dev, "Error during ACPI methods call\n"); |
| 226 | 194 | ||
| 227 | if (adev->runpm) { | ||
| 228 | /* only need to skip on ATPX */ | ||
| 229 | if (amdgpu_device_supports_px(dev)) | ||
| 230 | dev_pm_set_driver_flags(dev->dev, DPM_FLAG_NO_DIRECT_COMPLETE); | ||
| 231 | /* we want direct complete for BOCO */ | ||
| 232 | if (amdgpu_device_supports_boco(dev)) | ||
| 233 | dev_pm_set_driver_flags(dev->dev, DPM_FLAG_SMART_PREPARE | | ||
| 234 | DPM_FLAG_SMART_SUSPEND | | ||
| 235 | DPM_FLAG_MAY_SKIP_RESUME); | ||
| 236 | pm_runtime_use_autosuspend(dev->dev); | ||
| 237 | pm_runtime_set_autosuspend_delay(dev->dev, 5000); | ||
| 238 | |||
| 239 | pm_runtime_allow(dev->dev); | ||
| 240 | |||
| 241 | pm_runtime_mark_last_busy(dev->dev); | ||
| 242 | pm_runtime_put_autosuspend(dev->dev); | ||
| 243 | |||
| 244 | /* | ||
| 245 | * For runpm implemented via BACO, PMFW will handle the | ||
| 246 | * timing for BACO in and out: | ||
| 247 | * - put ASIC into BACO state only when both video and | ||
| 248 | * audio functions are in D3 state. | ||
| 249 | * - pull ASIC out of BACO state when either video or | ||
| 250 | * audio function is in D0 state. | ||
| 251 | * Also, at startup, PMFW assumes both functions are in | ||
| 252 | * D0 state. | ||
| 253 | * | ||
| 254 | * So if snd driver was loaded prior to amdgpu driver | ||
| 255 | * and audio function was put into D3 state, there will | ||
| 256 | * be no PMFW-aware D-state transition(D0->D3) on runpm | ||
| 257 | * suspend. Thus the BACO will be not correctly kicked in. | ||
| 258 | * | ||
| 259 | * Via amdgpu_get_audio_func(), the audio dev is put | ||
| 260 | * into D0 state. Then there will be a PMFW-aware D-state | ||
| 261 | * transition(D0->D3) on runpm suspend. | ||
| 262 | */ | ||
| 263 | if (amdgpu_device_supports_baco(dev) && | ||
| 264 | !(adev->flags & AMD_IS_APU) && | ||
| 265 | (adev->asic_type >= CHIP_NAVI10)) | ||
| 266 | amdgpu_get_audio_func(adev); | ||
| 267 | } | ||
| 268 | |||
| 269 | if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DRV_LOAD)) | 195 | if (amdgpu_acpi_smart_shift_update(dev, AMDGPU_SS_DRV_LOAD)) |
| 270 | DRM_WARN("smart shift update failed\n"); | 196 | DRM_WARN("smart shift update failed\n"); |
| 271 | 197 | ||
| 272 | out: | 198 | out: |
| 273 | if (r) { | 199 | if (r) |
| 274 | /* balance pm_runtime_get_sync in amdgpu_driver_unload_kms */ | ||
| 275 | if (adev->rmmio && adev->runpm) | ||
| 276 | pm_runtime_put_noidle(dev->dev); | ||
| 277 | amdgpu_driver_unload_kms(dev); | 200 | amdgpu_driver_unload_kms(dev); |
| 278 | } | ||
| 279 | 201 | ||
| 280 | return r; | 202 | return r; |
| 281 | } | 203 | } |
| @@ -406,6 +328,10 @@ static int amdgpu_firmware_info(struct drm_amdgpu_info_firmware *fw_info, · 124 unmodified lines | |||
| 406 | fw_info->ver = adev->psp.toc.fw_version; | 328 | fw_info->ver = adev->psp.toc.fw_version; |
| 407 | fw_info->feature = adev->psp.toc.feature_version; | 329 | fw_info->feature = adev->psp.toc.feature_version; |
| 408 | break; | 330 | break; |
| 331 | case AMDGPU_INFO_FW_CAP: | ||
| 332 | fw_info->ver = adev->psp.cap_fw_version; | ||
| 333 | fw_info->feature = adev->psp.cap_feature_version; | ||
| 334 | break; | ||
| 409 | default: | 335 | default: |
| 410 | return -EINVAL; | 336 | return -EINVAL; |
| 411 | } | 337 | } |
| @@ -678,13 +604,13 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 266 unmodified lines | |||
| 678 | ui64 = atomic64_read(&adev->num_vram_cpu_page_faults); | 604 | ui64 = atomic64_read(&adev->num_vram_cpu_page_faults); |
| 679 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; | 605 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; |
| 680 | case AMDGPU_INFO_VRAM_USAGE: | 606 | case AMDGPU_INFO_VRAM_USAGE: |
| 681 | ui64 = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr); | 607 | ui64 = ttm_resource_manager_usage(&adev->mman.vram_mgr.manager); |
| 682 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; | 608 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; |
| 683 | case AMDGPU_INFO_VIS_VRAM_USAGE: | 609 | case AMDGPU_INFO_VIS_VRAM_USAGE: |
| 684 | ui64 = amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr); | 610 | ui64 = amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr); |
| 685 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; | 611 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; |
| 686 | case AMDGPU_INFO_GTT_USAGE: | 612 | case AMDGPU_INFO_GTT_USAGE: |
| 687 | ui64 = amdgpu_gtt_mgr_usage(&adev->mman.gtt_mgr); | 613 | ui64 = ttm_resource_manager_usage(&adev->mman.gtt_mgr.manager); |
| 688 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; | 614 | return copy_to_user(out, &ui64, min(size, 8u)) ? -EFAULT : 0; |
| 689 | case AMDGPU_INFO_GDS_CONFIG: { | 615 | case AMDGPU_INFO_GDS_CONFIG: { |
| 690 | struct drm_amdgpu_info_gds gds_info; | 616 | struct drm_amdgpu_info_gds gds_info; |
| @@ -716,14 +642,17 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 25 unmodified lines | |||
| 716 | case AMDGPU_INFO_MEMORY: { | 642 | case AMDGPU_INFO_MEMORY: { |
| 717 | struct drm_amdgpu_memory_info mem; | 643 | struct drm_amdgpu_memory_info mem; |
| 718 | struct ttm_resource_manager *gtt_man = | 644 | struct ttm_resource_manager *gtt_man = |
| 719 | ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); | 645 | &adev->mman.gtt_mgr.manager; |
| 646 | struct ttm_resource_manager *vram_man = | ||
| 647 | &adev->mman.vram_mgr.manager; | ||
| 648 | |||
| 720 | memset(&mem, 0, sizeof(mem)); | 649 | memset(&mem, 0, sizeof(mem)); |
| 721 | mem.vram.total_heap_size = adev->gmc.real_vram_size; | 650 | mem.vram.total_heap_size = adev->gmc.real_vram_size; |
| 722 | mem.vram.usable_heap_size = adev->gmc.real_vram_size - | 651 | mem.vram.usable_heap_size = adev->gmc.real_vram_size - |
| 723 | atomic64_read(&adev->vram_pin_size) - | 652 | atomic64_read(&adev->vram_pin_size) - |
| 724 | AMDGPU_VM_RESERVED_VRAM; | 653 | AMDGPU_VM_RESERVED_VRAM; |
| 725 | mem.vram.heap_usage = | 654 | mem.vram.heap_usage = |
| 726 | amdgpu_vram_mgr_usage(&adev->mman.vram_mgr); | 655 | ttm_resource_manager_usage(vram_man); |
| 727 | mem.vram.max_allocation = mem.vram.usable_heap_size * 3 / 4; | 656 | mem.vram.max_allocation = mem.vram.usable_heap_size * 3 / 4; |
| 728 | 657 | ||
| 729 | mem.cpu_accessible_vram.total_heap_size = | 658 | mem.cpu_accessible_vram.total_heap_size = |
| @@ -741,8 +670,7 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp) · 11 unmodified lines | |||
| 741 | mem.gtt.total_heap_size *= PAGE_SIZE; | 670 | mem.gtt.total_heap_size *= PAGE_SIZE; |
| 742 | mem.gtt.usable_heap_size = mem.gtt.total_heap_size - | 671 | mem.gtt.usable_heap_size = mem.gtt.total_heap_size - |
| 743 | atomic64_read(&adev->gart_pin_size); | 672 | atomic64_read(&adev->gart_pin_size); |
| 744 | mem.gtt.heap_usage = | 673 | mem.gtt.heap_usage = ttm_resource_manager_usage(gtt_man); |
| 745 | amdgpu_gtt_mgr_usage(&adev->mman.gtt_mgr); | ||
| 746 | mem.gtt.max_allocation = mem.gtt.usable_heap_size * 3 / 4; | 674 | mem.gtt.max_allocation = mem.gtt.usable_heap_size * 3 / 4; |
| 747 | 675 | ||
| 748 | return copy_to_user(out, &mem, | 676 | return copy_to_user(out, &mem, |
| @@ -1268,18 +1196,20 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, · 519 unmodified lines | |||
| 1268 | if (amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_VCE) != NULL) | 1196 | if (amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_VCE) != NULL) |
| 1269 | amdgpu_vce_free_handles(adev, file_priv); | 1197 | amdgpu_vce_free_handles(adev, file_priv); |
| 1270 | 1198 | ||
| 1271 | amdgpu_vm_bo_rmv(adev, fpriv->prt_va); | ||
| 1272 | |||
| 1273 | if (amdgpu_mcbp || amdgpu_sriov_vf(adev)) { | 1199 | if (amdgpu_mcbp || amdgpu_sriov_vf(adev)) { |
| 1274 | /* TODO: how to handle reserve failure */ | 1200 | /* TODO: how to handle reserve failure */ |
| 1275 | BUG_ON(amdgpu_bo_reserve(adev->virt.csa_obj, true)); | 1201 | BUG_ON(amdgpu_bo_reserve(adev->virt.csa_obj, true)); |
| 1276 | amdgpu_vm_bo_rmv(adev, fpriv->csa_va); | 1202 | amdgpu_vm_bo_del(adev, fpriv->csa_va); |
| 1277 | fpriv->csa_va = NULL; | 1203 | fpriv->csa_va = NULL; |
| 1278 | amdgpu_bo_unreserve(adev->virt.csa_obj); | 1204 | amdgpu_bo_unreserve(adev->virt.csa_obj); |
| 1279 | } | 1205 | } |
| 1280 | 1206 | ||
| 1281 | pasid = fpriv->vm.pasid; | 1207 | pasid = fpriv->vm.pasid; |
| 1282 | pd = amdgpu_bo_ref(fpriv->vm.root.bo); | 1208 | pd = amdgpu_bo_ref(fpriv->vm.root.bo); |
| 1209 | if (!WARN_ON(amdgpu_bo_reserve(pd, true))) { | ||
| 1210 | amdgpu_vm_bo_del(adev, fpriv->prt_va); | ||
| 1211 | amdgpu_bo_unreserve(pd); | ||
| 1212 | } | ||
| 1283 | 1213 | ||
| 1284 | amdgpu_ctx_mgr_fini(&fpriv->ctx_mgr); | 1214 | amdgpu_ctx_mgr_fini(&fpriv->ctx_mgr); |
| 1285 | amdgpu_vm_fini(adev, &fpriv->vm); | 1215 | amdgpu_vm_fini(adev, &fpriv->vm); |
| @@ -1427,8 +1357,7 @@ static int amdgpu_debugfs_firmware_info_show(struct seq_file *m, void *unused) · 141 unmodified lines | |||
| 1427 | struct drm_amdgpu_info_firmware fw_info; | 1357 | struct drm_amdgpu_info_firmware fw_info; |
| 1428 | struct drm_amdgpu_query_fw query_fw; | 1358 | struct drm_amdgpu_query_fw query_fw; |
| 1429 | struct atom_context *ctx = adev->mode_info.atom_context; | 1359 | struct atom_context *ctx = adev->mode_info.atom_context; |
| 1430 | uint8_t smu_minor, smu_debug; | 1360 | uint8_t smu_program, smu_major, smu_minor, smu_debug; |
| 1431 | uint16_t smu_major; | ||
| 1432 | int ret, i; | 1361 | int ret, i; |
| 1433 | 1362 | ||
| 1434 | static const char *ta_fw_name[TA_FW_TYPE_MAX_INDEX] = { | 1363 | static const char *ta_fw_name[TA_FW_TYPE_MAX_INDEX] = { |
| @@ -1574,11 +1503,12 @@ static int amdgpu_debugfs_firmware_info_show(struct seq_file *m, void *unused) · 139 unmodified lines | |||
| 1574 | ret = amdgpu_firmware_info(&fw_info, &query_fw, adev); | 1503 | ret = amdgpu_firmware_info(&fw_info, &query_fw, adev); |
| 1575 | if (ret) | 1504 | if (ret) |
| 1576 | return ret; | 1505 | return ret; |
| 1577 | smu_major = (fw_info.ver >> 16) & 0xffff; | 1506 | smu_program = (fw_info.ver >> 24) & 0xff; |
| 1507 | smu_major = (fw_info.ver >> 16) & 0xff; | ||
| 1578 | smu_minor = (fw_info.ver >> 8) & 0xff; | 1508 | smu_minor = (fw_info.ver >> 8) & 0xff; |
| 1579 | smu_debug = (fw_info.ver >> 0) & 0xff; | 1509 | smu_debug = (fw_info.ver >> 0) & 0xff; |
| 1580 | seq_printf(m, "SMC feature version: %u, firmware version: 0x%08x (%d.%d.%d)\n", | 1510 | seq_printf(m, "SMC feature version: %u, program: %d, firmware version: 0x%08x (%d.%d.%d)\n", |
| 1581 | fw_info.feature, fw_info.ver, smu_major, smu_minor, smu_debug); | 1511 | fw_info.feature, smu_program, fw_info.ver, smu_major, smu_minor, smu_debug); |
| 1582 | 1512 | ||
| 1583 | /* SDMA */ | 1513 | /* SDMA */ |
| 1584 | query_fw.fw_type = AMDGPU_INFO_FW_SDMA; | 1514 | query_fw.fw_type = AMDGPU_INFO_FW_SDMA; |
| @@ -1623,6 +1553,16 @@ static int amdgpu_debugfs_firmware_info_show(struct seq_file *m, void *unused) · 38 unmodified lines | |||
| 1623 | seq_printf(m, "TOC feature version: %u, firmware version: 0x%08x\n", | 1553 | seq_printf(m, "TOC feature version: %u, firmware version: 0x%08x\n", |
| 1624 | fw_info.feature, fw_info.ver); | 1554 | fw_info.feature, fw_info.ver); |
| 1625 | 1555 | ||
| 1556 | /* CAP */ | ||
| 1557 | if (adev->psp.cap_fw) { | ||
| 1558 | query_fw.fw_type = AMDGPU_INFO_FW_CAP; | ||
| 1559 | ret = amdgpu_firmware_info(&fw_info, &query_fw, adev); | ||
| 1560 | if (ret) | ||
| 1561 | return ret; | ||
| 1562 | seq_printf(m, "CAP feature version: %u, firmware version: 0x%08x\n", | ||
| 1563 | fw_info.feature, fw_info.ver); | ||
| 1564 | } | ||
| 1565 | |||
| 1626 | seq_printf(m, "VBIOS version: %s\n", ctx->vbios_version); | 1566 | seq_printf(m, "VBIOS version: %s\n", ctx->vbios_version); |
| 1627 | 1567 | ||
| 1628 | return 0; | 1568 | return 0; |
| @@ -71,47 +71,8 @@ void amdgpu_mca_query_ras_error_count(struct amdgpu_device *adev, · 70 unmodified lines | |||
| 71 | amdgpu_mca_reset_error_count(adev, mc_status_addr); | 71 | amdgpu_mca_reset_error_count(adev, mc_status_addr); |
| 72 | } | 72 | } |
| 73 | 73 | ||
| 74 | int amdgpu_mca_ras_late_init(struct amdgpu_device *adev, | ||
| 75 | struct amdgpu_mca_ras *mca_dev) | ||
| 76 | { | ||
| 77 | int r; | ||
| 78 | struct ras_ih_if ih_info = { | ||
| 79 | .cb = NULL, | ||
| 80 | }; | ||
| 81 | struct ras_fs_if fs_info = { | ||
| 82 | .sysfs_name = mca_dev->ras_funcs->sysfs_name, | ||
| 83 | }; | ||
| 84 | |||
| 85 | if (!mca_dev->ras_if) { | ||
| 86 | mca_dev->ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | ||
| 87 | if (!mca_dev->ras_if) | ||
| 88 | return -ENOMEM; | ||
| 89 | mca_dev->ras_if->block = mca_dev->ras_funcs->ras_block; | ||
| 90 | mca_dev->ras_if->sub_block_index = mca_dev->ras_funcs->ras_sub_block; | ||
| 91 | mca_dev->ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 92 | } | ||
| 93 | ih_info.head = fs_info.head = *mca_dev->ras_if; | ||
| 94 | r = amdgpu_ras_late_init(adev, mca_dev->ras_if, | ||
| 95 | &fs_info, &ih_info); | ||
| 96 | if (r || !amdgpu_ras_is_supported(adev, mca_dev->ras_if->block)) { | ||
| 97 | kfree(mca_dev->ras_if); | ||
| 98 | mca_dev->ras_if = NULL; | ||
| 99 | } | ||
| 100 | |||
| 101 | return r; | ||
| 102 | } | ||
| 103 | |||
| 104 | void amdgpu_mca_ras_fini(struct amdgpu_device *adev, | 74 | void amdgpu_mca_ras_fini(struct amdgpu_device *adev, |
| 105 | struct amdgpu_mca_ras *mca_dev) | 75 | struct amdgpu_mca_ras *mca_dev) |
| 106 | { | 76 | { |
| 107 | struct ras_ih_if ih_info = { | 77 | amdgpu_ras_block_late_fini(adev, mca_dev->ras_if); |
| 108 | .cb = NULL, | ||
| 109 | }; | ||
| 110 | |||
| 111 | if (!mca_dev->ras_if) | ||
| 112 | return; | ||
| 113 | |||
| 114 | amdgpu_ras_late_fini(adev, mca_dev->ras_if, &ih_info); | ||
| 115 | kfree(mca_dev->ras_if); | ||
| 116 | mca_dev->ras_if = NULL; | ||
| 117 | } | 78 | } |
| @@ -21,21 +21,13 @@ · 20 unmodified lines | |||
| 21 | #ifndef __AMDGPU_MCA_H__ | 21 | #ifndef __AMDGPU_MCA_H__ |
| 22 | #define __AMDGPU_MCA_H__ | 22 | #define __AMDGPU_MCA_H__ |
| 23 | 23 | ||
| 24 | struct amdgpu_mca_ras_funcs { | 24 | struct amdgpu_mca_ras_block { |
| 25 | int (*ras_late_init)(struct amdgpu_device *adev); | 25 | struct amdgpu_ras_block_object ras_block; |
| 26 | void (*ras_fini)(struct amdgpu_device *adev); | ||
| 27 | void (*query_ras_error_count)(struct amdgpu_device *adev, | ||
| 28 | void *ras_error_status); | ||
| 29 | void (*query_ras_error_address)(struct amdgpu_device *adev, | ||
| 30 | void *ras_error_status); | ||
| 31 | uint32_t ras_block; | ||
| 32 | uint32_t ras_sub_block; | ||
| 33 | const char* sysfs_name; | ||
| 34 | }; | 26 | }; |
| 35 | 27 | ||
| 36 | struct amdgpu_mca_ras { | 28 | struct amdgpu_mca_ras { |
| 37 | struct ras_common_if *ras_if; | 29 | struct ras_common_if *ras_if; |
| 38 | const struct amdgpu_mca_ras_funcs *ras_funcs; | 30 | struct amdgpu_mca_ras_block *ras; |
| 39 | }; | 31 | }; |
| 40 | 32 | ||
| 41 | struct amdgpu_mca_funcs { | 33 | struct amdgpu_mca_funcs { |
| @@ -64,9 +56,6 @@ void amdgpu_mca_query_ras_error_count(struct amdgpu_device *adev, · 22 unmodified lines | |||
| 64 | uint64_t mc_status_addr, | 56 | uint64_t mc_status_addr, |
| 65 | void *ras_error_status); | 57 | void *ras_error_status); |
| 66 | 58 | ||
| 67 | int amdgpu_mca_ras_late_init(struct amdgpu_device *adev, | ||
| 68 | struct amdgpu_mca_ras *mca_dev); | ||
| 69 | |||
| 70 | void amdgpu_mca_ras_fini(struct amdgpu_device *adev, | 59 | void amdgpu_mca_ras_fini(struct amdgpu_device *adev, |
| 71 | struct amdgpu_mca_ras *mca_dev); | 60 | struct amdgpu_mca_ras *mca_dev); |
| 72 | 61 | ||
| @@ -24,45 +24,9 @@ · 23 unmodified lines | |||
| 24 | #include "amdgpu.h" | 24 | #include "amdgpu.h" |
| 25 | #include "amdgpu_ras.h" | 25 | #include "amdgpu_ras.h" |
| 26 | 26 | ||
| 27 | int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev) | ||
| 28 | { | ||
| 29 | int r; | ||
| 30 | struct ras_ih_if ih_info = { | ||
| 31 | .cb = NULL, | ||
| 32 | }; | ||
| 33 | struct ras_fs_if fs_info = { | ||
| 34 | .sysfs_name = "mmhub_err_count", | ||
| 35 | }; | ||
| 36 | |||
| 37 | if (!adev->mmhub.ras_if) { | ||
| 38 | adev->mmhub.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | ||
| 39 | if (!adev->mmhub.ras_if) | ||
| 40 | return -ENOMEM; | ||
| 41 | adev->mmhub.ras_if->block = AMDGPU_RAS_BLOCK__MMHUB; | ||
| 42 | adev->mmhub.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 43 | adev->mmhub.ras_if->sub_block_index = 0; | ||
| 44 | } | ||
| 45 | ih_info.head = fs_info.head = *adev->mmhub.ras_if; | ||
| 46 | r = amdgpu_ras_late_init(adev, adev->mmhub.ras_if, | ||
| 47 | &fs_info, &ih_info); | ||
| 48 | if (r || !amdgpu_ras_is_supported(adev, adev->mmhub.ras_if->block)) { | ||
| 49 | kfree(adev->mmhub.ras_if); | ||
| 50 | adev->mmhub.ras_if = NULL; | ||
| 51 | } | ||
| 52 | |||
| 53 | return r; | ||
| 54 | } | ||
| 55 | |||
| 56 | void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev) | 27 | void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev) |
| 57 | { | 28 | { |
| 58 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__MMHUB) && | 29 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__MMHUB) && |
| 59 | adev->mmhub.ras_if) { | 30 | adev->mmhub.ras_if) |
| 60 | struct ras_common_if *ras_if = adev->mmhub.ras_if; | 31 | amdgpu_ras_block_late_fini(adev, adev->mmhub.ras_if); |
| 61 | struct ras_ih_if ih_info = { | ||
| 62 | .cb = NULL, | ||
| 63 | }; | ||
| 64 | |||
| 65 | amdgpu_ras_late_fini(adev, ras_if, &ih_info); | ||
| 66 | kfree(ras_if); | ||
| 67 | } | ||
| 68 | } | 32 | } |
| @@ -21,14 +21,8 @@ · 20 unmodified lines | |||
| 21 | #ifndef __AMDGPU_MMHUB_H__ | 21 | #ifndef __AMDGPU_MMHUB_H__ |
| 22 | #define __AMDGPU_MMHUB_H__ | 22 | #define __AMDGPU_MMHUB_H__ |
| 23 | 23 | ||
| 24 | struct amdgpu_mmhub_ras_funcs { | 24 | struct amdgpu_mmhub_ras { |
| 25 | int (*ras_late_init)(struct amdgpu_device *adev); | 25 | struct amdgpu_ras_block_object ras_block; |
| 26 | void (*ras_fini)(struct amdgpu_device *adev); | ||
| 27 | void (*query_ras_error_count)(struct amdgpu_device *adev, | ||
| 28 | void *ras_error_status); | ||
| 29 | void (*query_ras_error_status)(struct amdgpu_device *adev); | ||
| 30 | void (*reset_ras_error_count)(struct amdgpu_device *adev); | ||
| 31 | void (*reset_ras_error_status)(struct amdgpu_device *adev); | ||
| 32 | }; | 26 | }; |
| 33 | 27 | ||
| 34 | struct amdgpu_mmhub_funcs { | 28 | struct amdgpu_mmhub_funcs { |
| @@ -50,10 +44,9 @@ struct amdgpu_mmhub_funcs { · 15 unmodified lines | |||
| 50 | struct amdgpu_mmhub { | 44 | struct amdgpu_mmhub { |
| 51 | struct ras_common_if *ras_if; | 45 | struct ras_common_if *ras_if; |
| 52 | const struct amdgpu_mmhub_funcs *funcs; | 46 | const struct amdgpu_mmhub_funcs *funcs; |
| 53 | const struct amdgpu_mmhub_ras_funcs *ras_funcs; | 47 | struct amdgpu_mmhub_ras *ras; |
| 54 | }; | 48 | }; |
| 55 | 49 | ||
| 56 | int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev); | ||
| 57 | void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev); | 50 | void amdgpu_mmhub_ras_fini(struct amdgpu_device *adev); |
| 58 | #endif | 51 | #endif |
| 59 | 52 | ||
| @@ -341,6 +341,7 @@ struct amdgpu_mode_info { · 340 unmodified lines | |||
| 341 | int num_crtc; /* number of crtcs */ | 341 | int num_crtc; /* number of crtcs */ |
| 342 | int num_hpd; /* number of hpd pins */ | 342 | int num_hpd; /* number of hpd pins */ |
| 343 | int num_dig; /* number of dig blocks */ | 343 | int num_dig; /* number of dig blocks */ |
| 344 | bool gpu_vm_support; /* supports display from GTT */ | ||
| 344 | int disp_priority; | 345 | int disp_priority; |
| 345 | const struct amdgpu_display_funcs *funcs; | 346 | const struct amdgpu_display_funcs *funcs; |
| 346 | const enum drm_plane_type *plane_type; | 347 | const enum drm_plane_type *plane_type; |
| @@ -22,61 +22,31 @@ · 21 unmodified lines | |||
| 22 | #include "amdgpu.h" | 22 | #include "amdgpu.h" |
| 23 | #include "amdgpu_ras.h" | 23 | #include "amdgpu_ras.h" |
| 24 | 24 | ||
| 25 | int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev) | 25 | int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block) |
| 26 | { | 26 | { |
| 27 | int r; | 27 | int r; |
| 28 | struct ras_ih_if ih_info = { | 28 | r = amdgpu_ras_block_late_init(adev, ras_block); |
| 29 | .cb = NULL, | ||
| 30 | }; | ||
| 31 | struct ras_fs_if fs_info = { | ||
| 32 | .sysfs_name = "pcie_bif_err_count", | ||
| 33 | }; | ||
| 34 | |||
| 35 | if (!adev->nbio.ras_if) { | ||
| 36 | adev->nbio.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | ||
| 37 | if (!adev->nbio.ras_if) | ||
| 38 | return -ENOMEM; | ||
| 39 | adev->nbio.ras_if->block = AMDGPU_RAS_BLOCK__PCIE_BIF; | ||
| 40 | adev->nbio.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 41 | adev->nbio.ras_if->sub_block_index = 0; | ||
| 42 | } | ||
| 43 | ih_info.head = fs_info.head = *adev->nbio.ras_if; | ||
| 44 | r = amdgpu_ras_late_init(adev, adev->nbio.ras_if, | ||
| 45 | &fs_info, &ih_info); | ||
| 46 | if (r) | 29 | if (r) |
| 47 | goto free; | 30 | return r; |
| 48 | 31 | ||
| 49 | if (amdgpu_ras_is_supported(adev, adev->nbio.ras_if->block)) { | 32 | if (amdgpu_ras_is_supported(adev, ras_block->block)) { |
| 50 | r = amdgpu_irq_get(adev, &adev->nbio.ras_controller_irq, 0); | 33 | r = amdgpu_irq_get(adev, &adev->nbio.ras_controller_irq, 0); |
| 51 | if (r) | 34 | if (r) |
| 52 | goto late_fini; | 35 | goto late_fini; |
| 53 | r = amdgpu_irq_get(adev, &adev->nbio.ras_err_event_athub_irq, 0); | 36 | r = amdgpu_irq_get(adev, &adev->nbio.ras_err_event_athub_irq, 0); |
| 54 | if (r) | 37 | if (r) |
| 55 | goto late_fini; | 38 | goto late_fini; |
| 56 | } else { | ||
| 57 | r = 0; | ||
| 58 | goto free; | ||
| 59 | } | 39 | } |
| 60 | 40 | ||
| 61 | return 0; | 41 | return 0; |
| 62 | late_fini: | 42 | late_fini: |
| 63 | amdgpu_ras_late_fini(adev, adev->nbio.ras_if, &ih_info); | 43 | amdgpu_ras_block_late_fini(adev, ras_block); |
| 64 | free: | ||
| 65 | kfree(adev->nbio.ras_if); | ||
| 66 | adev->nbio.ras_if = NULL; | ||
| 67 | return r; | 44 | return r; |
| 68 | } | 45 | } |
| 69 | 46 | ||
| 70 | void amdgpu_nbio_ras_fini(struct amdgpu_device *adev) | 47 | void amdgpu_nbio_ras_fini(struct amdgpu_device *adev) |
| 71 | { | 48 | { |
| 72 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF) && | 49 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF) && |
| 73 | adev->nbio.ras_if) { | 50 | adev->nbio.ras_if) |
| 74 | struct ras_common_if *ras_if = adev->nbio.ras_if; | 51 | amdgpu_ras_block_late_fini(adev, adev->nbio.ras_if); |
| 75 | struct ras_ih_if ih_info = { | ||
| 76 | .cb = NULL, | ||
| 77 | }; | ||
| 78 | |||
| 79 | amdgpu_ras_late_fini(adev, ras_if, &ih_info); | ||
| 80 | kfree(ras_if); | ||
| 81 | } | ||
| 82 | } | 52 | } |
| @@ -47,15 +47,12 @@ struct nbio_hdp_flush_reg { · 46 unmodified lines | |||
| 47 | u32 ref_and_mask_sdma7; | 47 | u32 ref_and_mask_sdma7; |
| 48 | }; | 48 | }; |
| 49 | 49 | ||
| 50 | struct amdgpu_nbio_ras_funcs { | 50 | struct amdgpu_nbio_ras { |
| 51 | struct amdgpu_ras_block_object ras_block; | ||
| 51 | void (*handle_ras_controller_intr_no_bifring)(struct amdgpu_device *adev); | 52 | void (*handle_ras_controller_intr_no_bifring)(struct amdgpu_device *adev); |
| 52 | void (*handle_ras_err_event_athub_intr_no_bifring)(struct amdgpu_device *adev); | 53 | void (*handle_ras_err_event_athub_intr_no_bifring)(struct amdgpu_device *adev); |
| 53 | int (*init_ras_controller_interrupt)(struct amdgpu_device *adev); | 54 | int (*init_ras_controller_interrupt)(struct amdgpu_device *adev); |
| 54 | int (*init_ras_err_event_athub_interrupt)(struct amdgpu_device *adev); | 55 | int (*init_ras_err_event_athub_interrupt)(struct amdgpu_device *adev); |
| 55 | void (*query_ras_error_count)(struct amdgpu_device *adev, | ||
| 56 | void *ras_error_status); | ||
| 57 | int (*ras_late_init)(struct amdgpu_device *adev); | ||
| 58 | void (*ras_fini)(struct amdgpu_device *adev); | ||
| 59 | }; | 56 | }; |
| 60 | 57 | ||
| 61 | struct amdgpu_nbio_funcs { | 58 | struct amdgpu_nbio_funcs { |
| @@ -104,9 +101,9 @@ struct amdgpu_nbio { · 42 unmodified lines | |||
| 104 | struct amdgpu_irq_src ras_err_event_athub_irq; | 101 | struct amdgpu_irq_src ras_err_event_athub_irq; |
| 105 | struct ras_common_if *ras_if; | 102 | struct ras_common_if *ras_if; |
| 106 | const struct amdgpu_nbio_funcs *funcs; | 103 | const struct amdgpu_nbio_funcs *funcs; |
| 107 | const struct amdgpu_nbio_ras_funcs *ras_funcs; | 104 | struct amdgpu_nbio_ras *ras; |
| 108 | }; | 105 | }; |
| 109 | 106 | ||
| 110 | int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev); | 107 | int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block); |
| 111 | void amdgpu_nbio_ras_fini(struct amdgpu_device *adev); | 108 | void amdgpu_nbio_ras_fini(struct amdgpu_device *adev); |
| 112 | #endif | 109 | #endif |
| @@ -451,7 +451,7 @@ static bool amdgpu_bo_validate_size(struct amdgpu_device *adev, · 450 unmodified lines | |||
| 451 | if (domain & AMDGPU_GEM_DOMAIN_GTT) { | 451 | if (domain & AMDGPU_GEM_DOMAIN_GTT) { |
| 452 | man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); | 452 | man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT); |
| 453 | 453 | ||
| 454 | if (size < (man->size << PAGE_SHIFT)) | 454 | if (size < man->size) |
| 455 | return true; | 455 | return true; |
| 456 | else | 456 | else |
| 457 | goto fail; | 457 | goto fail; |
| @@ -460,7 +460,7 @@ static bool amdgpu_bo_validate_size(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 460 | if (domain & AMDGPU_GEM_DOMAIN_VRAM) { | 460 | if (domain & AMDGPU_GEM_DOMAIN_VRAM) { |
| 461 | man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM); | 461 | man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM); |
| 462 | 462 | ||
| 463 | if (size < (man->size << PAGE_SHIFT)) | 463 | if (size < man->size) |
| 464 | return true; | 464 | return true; |
| 465 | else | 465 | else |
| 466 | goto fail; | 466 | goto fail; |
| @@ -575,6 +575,9 @@ int amdgpu_bo_create(struct amdgpu_device *adev, · 108 unmodified lines | |||
| 575 | if (!amdgpu_bo_support_uswc(bo->flags)) | 575 | if (!amdgpu_bo_support_uswc(bo->flags)) |
| 576 | bo->flags &= ~AMDGPU_GEM_CREATE_CPU_GTT_USWC; | 576 | bo->flags &= ~AMDGPU_GEM_CREATE_CPU_GTT_USWC; |
| 577 | 577 | ||
| 578 | if (adev->ras_enabled) | ||
| 579 | bo->flags |= AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE; | ||
| 580 | |||
| 578 | bo->tbo.bdev = &adev->mman.bdev; | 581 | bo->tbo.bdev = &adev->mman.bdev; |
| 579 | if (bp->domain & (AMDGPU_GEM_DOMAIN_GWS | AMDGPU_GEM_DOMAIN_OA | | 582 | if (bp->domain & (AMDGPU_GEM_DOMAIN_GWS | AMDGPU_GEM_DOMAIN_OA | |
| 580 | AMDGPU_GEM_DOMAIN_GDS)) | 583 | AMDGPU_GEM_DOMAIN_GDS)) |
| @@ -1303,7 +1306,8 @@ void amdgpu_bo_release_notify(struct ttm_buffer_object *bo) · 722 unmodified lines | |||
| 1303 | !(abo->flags & AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE)) | 1306 | !(abo->flags & AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE)) |
| 1304 | return; | 1307 | return; |
| 1305 | 1308 | ||
| 1306 | dma_resv_lock(bo->base.resv, NULL); | 1309 | if (WARN_ON_ONCE(!dma_resv_trylock(bo->base.resv))) |
| 1310 | return; | ||
| 1307 | 1311 | ||
| 1308 | r = amdgpu_fill_buffer(abo, AMDGPU_POISON, bo->base.resv, &fence); | 1312 | r = amdgpu_fill_buffer(abo, AMDGPU_POISON, bo->base.resv, &fence); |
| 1309 | if (!WARN_ON(r)) { | 1313 | if (!WARN_ON(r)) { |
| @@ -25,12 +25,6 @@ · 24 unmodified lines | |||
| 25 | 25 | ||
| 26 | #include "amdgpu.h" | 26 | #include "amdgpu.h" |
| 27 | 27 | ||
| 28 | static inline struct amdgpu_preempt_mgr * | ||
| 29 | to_preempt_mgr(struct ttm_resource_manager *man) | ||
| 30 | { | ||
| 31 | return container_of(man, struct amdgpu_preempt_mgr, manager); | ||
| 32 | } | ||
| 33 | |||
| 34 | /** | 28 | /** |
| 35 | * DOC: mem_info_preempt_used | 29 | * DOC: mem_info_preempt_used |
| 36 | * | 30 | * |
| @@ -45,10 +39,9 @@ static ssize_t mem_info_preempt_used_show(struct device *dev, · 8 unmodified lines | |||
| 45 | { | 39 | { |
| 46 | struct drm_device *ddev = dev_get_drvdata(dev); | 40 | struct drm_device *ddev = dev_get_drvdata(dev); |
| 47 | struct amdgpu_device *adev = drm_to_adev(ddev); | 41 | struct amdgpu_device *adev = drm_to_adev(ddev); |
| 48 | struct ttm_resource_manager *man; | 42 | struct ttm_resource_manager *man = &adev->mman.preempt_mgr; |
| 49 | 43 | ||
| 50 | man = ttm_manager_type(&adev->mman.bdev, AMDGPU_PL_PREEMPT); | 44 | return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man)); |
| 51 | return sysfs_emit(buf, "%llu\n", amdgpu_preempt_mgr_usage(man)); | ||
| 52 | } | 45 | } |
| 53 | 46 | ||
| 54 | static DEVICE_ATTR_RO(mem_info_preempt_used); | 47 | static DEVICE_ATTR_RO(mem_info_preempt_used); |
| @@ -68,16 +61,12 @@ static int amdgpu_preempt_mgr_new(struct ttm_resource_manager *man, · 13 unmodified lines | |||
| 68 | const struct ttm_place *place, | 61 | const struct ttm_place *place, |
| 69 | struct ttm_resource **res) | 62 | struct ttm_resource **res) |
| 70 | { | 63 | { |
| 71 | struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man); | ||
| 72 | |||
| 73 | *res = kzalloc(sizeof(**res), GFP_KERNEL); | 64 | *res = kzalloc(sizeof(**res), GFP_KERNEL); |
| 74 | if (!*res) | 65 | if (!*res) |
| 75 | return -ENOMEM; | 66 | return -ENOMEM; |
| 76 | 67 | ||
| 77 | ttm_resource_init(tbo, place, *res); | 68 | ttm_resource_init(tbo, place, *res); |
| 78 | (*res)->start = AMDGPU_BO_INVALID_OFFSET; | 69 | (*res)->start = AMDGPU_BO_INVALID_OFFSET; |
| 79 | |||
| 80 | atomic64_add((*res)->num_pages, &mgr->used); | ||
| 81 | return 0; | 70 | return 0; |
| 82 | } | 71 | } |
| 83 | 72 | ||
| @@ -92,49 +81,13 @@ static int amdgpu_preempt_mgr_new(struct ttm_resource_manager *man, · 8 unmodified lines | |||
| 92 | static void amdgpu_preempt_mgr_del(struct ttm_resource_manager *man, | 81 | static void amdgpu_preempt_mgr_del(struct ttm_resource_manager *man, |
| 93 | struct ttm_resource *res) | 82 | struct ttm_resource *res) |
| 94 | { | 83 | { |
| 95 | struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man); | ||
| 96 | |||
| 97 | atomic64_sub(res->num_pages, &mgr->used); | ||
| 98 | ttm_resource_fini(man, res); | 84 | ttm_resource_fini(man, res); |
| 99 | kfree(res); | 85 | kfree(res); |
| 100 | } | 86 | } |
| 101 | 87 | ||
| 102 | /** | ||
| 103 | * amdgpu_preempt_mgr_usage - return usage of PREEMPT domain | ||
| 104 | * | ||
| 105 | * @man: TTM memory type manager | ||
| 106 | * | ||
| 107 | * Return how many bytes are used in the GTT domain | ||
| 108 | */ | ||
| 109 | uint64_t amdgpu_preempt_mgr_usage(struct ttm_resource_manager *man) | ||
| 110 | { | ||
| 111 | struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man); | ||
| 112 | s64 result = atomic64_read(&mgr->used); | ||
| 113 | |||
| 114 | return (result > 0 ? result : 0) * PAGE_SIZE; | ||
| 115 | } | ||
| 116 | |||
| 117 | /** | ||
| 118 | * amdgpu_preempt_mgr_debug - dump VRAM table | ||
| 119 | * | ||
| 120 | * @man: TTM memory type manager | ||
| 121 | * @printer: DRM printer to use | ||
| 122 | * | ||
| 123 | * Dump the table content using printk. | ||
| 124 | */ | ||
| 125 | static void amdgpu_preempt_mgr_debug(struct ttm_resource_manager *man, | ||
| 126 | struct drm_printer *printer) | ||
| 127 | { | ||
| 128 | struct amdgpu_preempt_mgr *mgr = to_preempt_mgr(man); | ||
| 129 | |||
| 130 | drm_printf(printer, "man size:%llu pages, preempt used:%lld pages\n", | ||
| 131 | man->size, (u64)atomic64_read(&mgr->used)); | ||
| 132 | } | ||
| 133 | |||
| 134 | static const struct ttm_resource_manager_func amdgpu_preempt_mgr_func = { | 88 | static const struct ttm_resource_manager_func amdgpu_preempt_mgr_func = { |
| 135 | .alloc = amdgpu_preempt_mgr_new, | 89 | .alloc = amdgpu_preempt_mgr_new, |
| 136 | .free = amdgpu_preempt_mgr_del, | 90 | .free = amdgpu_preempt_mgr_del, |
| 137 | .debug = amdgpu_preempt_mgr_debug | ||
| 138 | }; | 91 | }; |
| 139 | 92 | ||
| 140 | /** | 93 | /** |
| @@ -146,8 +99,7 @@ static const struct ttm_resource_manager_func amdgpu_preempt_mgr_func = { · 5 unmodified lines | |||
| 146 | */ | 99 | */ |
| 147 | int amdgpu_preempt_mgr_init(struct amdgpu_device *adev) | 100 | int amdgpu_preempt_mgr_init(struct amdgpu_device *adev) |
| 148 | { | 101 | { |
| 149 | struct amdgpu_preempt_mgr *mgr = &adev->mman.preempt_mgr; | 102 | struct ttm_resource_manager *man = &adev->mman.preempt_mgr; |
| 150 | struct ttm_resource_manager *man = &mgr->manager; | ||
| 151 | int ret; | 103 | int ret; |
| 152 | 104 | ||
| 153 | man->use_tt = true; | 105 | man->use_tt = true; |
| @@ -155,16 +107,13 @@ int amdgpu_preempt_mgr_init(struct amdgpu_device *adev) · 1 unmodified line | |||
| 155 | 107 | ||
| 156 | ttm_resource_manager_init(man, &adev->mman.bdev, (1 << 30)); | 108 | ttm_resource_manager_init(man, &adev->mman.bdev, (1 << 30)); |
| 157 | 109 | ||
| 158 | atomic64_set(&mgr->used, 0); | ||
| 159 | |||
| 160 | ret = device_create_file(adev->dev, &dev_attr_mem_info_preempt_used); | 110 | ret = device_create_file(adev->dev, &dev_attr_mem_info_preempt_used); |
| 161 | if (ret) { | 111 | if (ret) { |
| 162 | DRM_ERROR("Failed to create device file mem_info_preempt_used\n"); | 112 | DRM_ERROR("Failed to create device file mem_info_preempt_used\n"); |
| 163 | return ret; | 113 | return ret; |
| 164 | } | 114 | } |
| 165 | 115 | ||
| 166 | ttm_set_driver_manager(&adev->mman.bdev, AMDGPU_PL_PREEMPT, | 116 | ttm_set_driver_manager(&adev->mman.bdev, AMDGPU_PL_PREEMPT, man); |
| 167 | &mgr->manager); | ||
| 168 | ttm_resource_manager_set_used(man, true); | 117 | ttm_resource_manager_set_used(man, true); |
| 169 | return 0; | 118 | return 0; |
| 170 | } | 119 | } |
| @@ -179,8 +128,7 @@ int amdgpu_preempt_mgr_init(struct amdgpu_device *adev) · 8 unmodified lines | |||
| 179 | */ | 128 | */ |
| 180 | void amdgpu_preempt_mgr_fini(struct amdgpu_device *adev) | 129 | void amdgpu_preempt_mgr_fini(struct amdgpu_device *adev) |
| 181 | { | 130 | { |
| 182 | struct amdgpu_preempt_mgr *mgr = &adev->mman.preempt_mgr; | 131 | struct ttm_resource_manager *man = &adev->mman.preempt_mgr; |
| 183 | struct ttm_resource_manager *man = &mgr->manager; | ||
| 184 | int ret; | 132 | int ret; |
| 185 | 133 | ||
| 186 | ttm_resource_manager_set_used(man, false); | 134 | ttm_resource_manager_set_used(man, false); |
| @@ -133,6 +133,8 @@ static int psp_early_init(void *handle) · 132 unmodified lines | |||
| 133 | break; | 133 | break; |
| 134 | case IP_VERSION(13, 0, 1): | 134 | case IP_VERSION(13, 0, 1): |
| 135 | case IP_VERSION(13, 0, 3): | 135 | case IP_VERSION(13, 0, 3): |
| 136 | case IP_VERSION(13, 0, 5): | ||
| 137 | case IP_VERSION(13, 0, 8): | ||
| 136 | psp_v13_0_set_psp_funcs(psp); | 138 | psp_v13_0_set_psp_funcs(psp); |
| 137 | psp->autoload_supported = true; | 139 | psp->autoload_supported = true; |
| 138 | break; | 140 | break; |
| @@ -259,6 +261,32 @@ static bool psp_get_runtime_db_entry(struct amdgpu_device *adev, · 120 unmodified lines | |||
| 259 | return ret; | 261 | return ret; |
| 260 | } | 262 | } |
| 261 | 263 | ||
| 264 | static int psp_init_sriov_microcode(struct psp_context *psp) | ||
| 265 | { | ||
| 266 | struct amdgpu_device *adev = psp->adev; | ||
| 267 | int ret = 0; | ||
| 268 | |||
| 269 | switch (adev->ip_versions[MP0_HWIP][0]) { | ||
| 270 | case IP_VERSION(9, 0, 0): | ||
| 271 | ret = psp_init_cap_microcode(psp, "vega10"); | ||
| 272 | break; | ||
| 273 | case IP_VERSION(11, 0, 9): | ||
| 274 | ret = psp_init_cap_microcode(psp, "navi12"); | ||
| 275 | break; | ||
| 276 | case IP_VERSION(11, 0, 7): | ||
| 277 | ret = psp_init_cap_microcode(psp, "sienna_cichlid"); | ||
| 278 | break; | ||
| 279 | case IP_VERSION(13, 0, 2): | ||
| 280 | ret = psp_init_ta_microcode(psp, "aldebaran"); | ||
| 281 | break; | ||
| 282 | default: | ||
| 283 | BUG(); | ||
| 284 | break; | ||
| 285 | } | ||
| 286 | |||
| 287 | return ret; | ||
| 288 | } | ||
| 289 | |||
| 262 | static int psp_sw_init(void *handle) | 290 | static int psp_sw_init(void *handle) |
| 263 | { | 291 | { |
| 264 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 292 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| @@ -273,19 +301,13 @@ static int psp_sw_init(void *handle) · 8 unmodified lines | |||
| 273 | ret = -ENOMEM; | 301 | ret = -ENOMEM; |
| 274 | } | 302 | } |
| 275 | 303 | ||
| 276 | if (!amdgpu_sriov_vf(adev)) { | 304 | if (amdgpu_sriov_vf(adev)) |
| 305 | ret = psp_init_sriov_microcode(psp); | ||
| 306 | else | ||
| 277 | ret = psp_init_microcode(psp); | 307 | ret = psp_init_microcode(psp); |
| 278 | if (ret) { | 308 | if (ret) { |
| 279 | DRM_ERROR("Failed to load psp firmware!\n"); | 309 | DRM_ERROR("Failed to load psp firmware!\n"); |
| 280 | return ret; | 310 | return ret; |
| 281 | } | ||
| 282 | } else if (amdgpu_sriov_vf(adev) && | ||
| 283 | adev->ip_versions[MP0_HWIP][0] == IP_VERSION(13, 0, 2)) { | ||
| 284 | ret = psp_init_ta_microcode(psp, "aldebaran"); | ||
| 285 | if (ret) { | ||
| 286 | DRM_ERROR("Failed to initialize ta microcode!\n"); | ||
| 287 | return ret; | ||
| 288 | } | ||
| 289 | } | 311 | } |
| 290 | 312 | ||
| 291 | memset(&boot_cfg_entry, 0, sizeof(boot_cfg_entry)); | 313 | memset(&boot_cfg_entry, 0, sizeof(boot_cfg_entry)); |
| @@ -353,6 +375,10 @@ static int psp_sw_fini(void *handle) · 61 unmodified lines | |||
| 353 | release_firmware(psp->ta_fw); | 375 | release_firmware(psp->ta_fw); |
| 354 | psp->ta_fw = NULL; | 376 | psp->ta_fw = NULL; |
| 355 | } | 377 | } |
| 378 | if (adev->psp.cap_fw) { | ||
| 379 | release_firmware(psp->cap_fw); | ||
| 380 | psp->cap_fw = NULL; | ||
| 381 | } | ||
| 356 | 382 | ||
| 357 | if (adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 0) || | 383 | if (adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 0) || |
| 358 | adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 7)) | 384 | adev->ip_versions[MP0_HWIP][0] == IP_VERSION(11, 0, 7)) |
| @@ -491,7 +517,10 @@ psp_cmd_submit_buf(struct psp_context *psp, · 132 unmodified lines | |||
| 491 | DRM_WARN("psp gfx command %s(0x%X) failed and response status is (0x%X)\n", | 517 | DRM_WARN("psp gfx command %s(0x%X) failed and response status is (0x%X)\n", |
| 492 | psp_gfx_cmd_name(psp->cmd_buf_mem->cmd_id), psp->cmd_buf_mem->cmd_id, | 518 | psp_gfx_cmd_name(psp->cmd_buf_mem->cmd_id), psp->cmd_buf_mem->cmd_id, |
| 493 | psp->cmd_buf_mem->resp.status); | 519 | psp->cmd_buf_mem->resp.status); |
| 494 | if (!timeout) { | 520 | /* If we load CAP FW, PSP must return 0 under SRIOV |
| 521 | * also return failure in case of timeout | ||
| 522 | */ | ||
| 523 | if ((ucode && (ucode->ucode_id == AMDGPU_UCODE_ID_CAP)) || !timeout) { | ||
| 495 | ret = -EINVAL; | 524 | ret = -EINVAL; |
| 496 | goto exit; | 525 | goto exit; |
| 497 | } | 526 | } |
| @@ -914,19 +943,15 @@ static void psp_prep_ta_load_cmd_buf(struct psp_gfx_cmd_resp *cmd, · 416 unmodified lines | |||
| 914 | static int psp_ta_init_shared_buf(struct psp_context *psp, | 943 | static int psp_ta_init_shared_buf(struct psp_context *psp, |
| 915 | struct ta_mem_context *mem_ctx) | 944 | struct ta_mem_context *mem_ctx) |
| 916 | { | 945 | { |
| 917 | int ret; | ||
| 918 | |||
| 919 | /* | 946 | /* |
| 920 | * Allocate 16k memory aligned to 4k from Frame Buffer (local | 947 | * Allocate 16k memory aligned to 4k from Frame Buffer (local |
| 921 | * physical) for ta to host memory | 948 | * physical) for ta to host memory |
| 922 | */ | 949 | */ |
| 923 | ret = amdgpu_bo_create_kernel(psp->adev, mem_ctx->shared_mem_size, | 950 | return amdgpu_bo_create_kernel(psp->adev, mem_ctx->shared_mem_size, |
| 924 | PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM, | 951 | PAGE_SIZE, AMDGPU_GEM_DOMAIN_VRAM, |
| 925 | &mem_ctx->shared_bo, | 952 | &mem_ctx->shared_bo, |
| 926 | &mem_ctx->shared_mc_addr, | 953 | &mem_ctx->shared_mc_addr, |
| 927 | &mem_ctx->shared_buf); | 954 | &mem_ctx->shared_buf); |
| 928 | |||
| 929 | return ret; | ||
| 930 | } | 955 | } |
| 931 | 956 | ||
| 932 | static void psp_ta_free_shared_buf(struct ta_mem_context *mem_ctx) | 957 | static void psp_ta_free_shared_buf(struct ta_mem_context *mem_ctx) |
| @@ -1308,6 +1333,11 @@ static void psp_ras_ta_check_status(struct psp_context *psp) · 375 unmodified lines | |||
| 1308 | break; | 1333 | break; |
| 1309 | case TA_RAS_STATUS__SUCCESS: | 1334 | case TA_RAS_STATUS__SUCCESS: |
| 1310 | break; | 1335 | break; |
| 1336 | case TA_RAS_STATUS__TEE_ERROR_ACCESS_DENIED: | ||
| 1337 | if (ras_cmd->cmd_id == TA_RAS_COMMAND__TRIGGER_ERROR) | ||
| 1338 | dev_warn(psp->adev->dev, | ||
| 1339 | "RAS WARNING: Inject error to critical region is not allowed\n"); | ||
| 1340 | break; | ||
| 1311 | default: | 1341 | default: |
| 1312 | dev_warn(psp->adev->dev, | 1342 | dev_warn(psp->adev->dev, |
| 1313 | "RAS WARNING: ras status = 0x%X\n", ras_cmd->ras_status); | 1343 | "RAS WARNING: ras status = 0x%X\n", ras_cmd->ras_status); |
| @@ -1520,7 +1550,9 @@ int psp_ras_trigger_error(struct psp_context *psp, · 206 unmodified lines | |||
| 1520 | if (amdgpu_ras_intr_triggered()) | 1550 | if (amdgpu_ras_intr_triggered()) |
| 1521 | return 0; | 1551 | return 0; |
| 1522 | 1552 | ||
| 1523 | if (ras_cmd->ras_status) | 1553 | if (ras_cmd->ras_status == TA_RAS_STATUS__TEE_ERROR_ACCESS_DENIED) |
| 1554 | return -EACCES; | ||
| 1555 | else if (ras_cmd->ras_status) | ||
| 1524 | return -EINVAL; | 1556 | return -EINVAL; |
| 1525 | 1557 | ||
| 1526 | return 0; | 1558 | return 0; |
| @@ -2051,6 +2083,9 @@ static int psp_get_fw_type(struct amdgpu_firmware_info *ucode, · 524 unmodified lines | |||
| 2051 | enum psp_gfx_fw_type *type) | 2083 | enum psp_gfx_fw_type *type) |
| 2052 | { | 2084 | { |
| 2053 | switch (ucode->ucode_id) { | 2085 | switch (ucode->ucode_id) { |
| 2086 | case AMDGPU_UCODE_ID_CAP: | ||
| 2087 | *type = GFX_FW_TYPE_CAP; | ||
| 2088 | break; | ||
| 2054 | case AMDGPU_UCODE_ID_SDMA0: | 2089 | case AMDGPU_UCODE_ID_SDMA0: |
| 2055 | *type = GFX_FW_TYPE_SDMA0; | 2090 | *type = GFX_FW_TYPE_SDMA0; |
| 2056 | break; | 2091 | break; |
| @@ -3217,6 +3252,58 @@ out: · 1160 unmodified lines | |||
| 3217 | return err; | 3252 | return err; |
| 3218 | } | 3253 | } |
| 3219 | 3254 | ||
| 3255 | int psp_init_cap_microcode(struct psp_context *psp, | ||
| 3256 | const char *chip_name) | ||
| 3257 | { | ||
| 3258 | struct amdgpu_device *adev = psp->adev; | ||
| 3259 | char fw_name[PSP_FW_NAME_LEN]; | ||
| 3260 | const struct psp_firmware_header_v1_0 *cap_hdr_v1_0; | ||
| 3261 | struct amdgpu_firmware_info *info = NULL; | ||
| 3262 | int err = 0; | ||
| 3263 | |||
| 3264 | if (!chip_name) { | ||
| 3265 | dev_err(adev->dev, "invalid chip name for cap microcode\n"); | ||
| 3266 | return -EINVAL; | ||
| 3267 | } | ||
| 3268 | |||
| 3269 | if (!amdgpu_sriov_vf(adev)) { | ||
| 3270 | dev_err(adev->dev, "cap microcode should only be loaded under SRIOV\n"); | ||
| 3271 | return -EINVAL; | ||
| 3272 | } | ||
| 3273 | |||
| 3274 | snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_cap.bin", chip_name); | ||
| 3275 | err = request_firmware(&adev->psp.cap_fw, fw_name, adev->dev); | ||
| 3276 | if (err) { | ||
| 3277 | dev_warn(adev->dev, "cap microcode does not exist, skip\n"); | ||
| 3278 | err = 0; | ||
| 3279 | goto out; | ||
| 3280 | } | ||
| 3281 | |||
| 3282 | err = amdgpu_ucode_validate(adev->psp.cap_fw); | ||
| 3283 | if (err) { | ||
| 3284 | dev_err(adev->dev, "fail to initialize cap microcode\n"); | ||
| 3285 | goto out; | ||
| 3286 | } | ||
| 3287 | |||
| 3288 | info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CAP]; | ||
| 3289 | info->ucode_id = AMDGPU_UCODE_ID_CAP; | ||
| 3290 | info->fw = adev->psp.cap_fw; | ||
| 3291 | cap_hdr_v1_0 = (const struct psp_firmware_header_v1_0 *) | ||
| 3292 | adev->psp.cap_fw->data; | ||
| 3293 | adev->firmware.fw_size += ALIGN( | ||
| 3294 | le32_to_cpu(cap_hdr_v1_0->header.ucode_size_bytes), PAGE_SIZE); | ||
| 3295 | adev->psp.cap_fw_version = le32_to_cpu(cap_hdr_v1_0->header.ucode_version); | ||
| 3296 | adev->psp.cap_feature_version = le32_to_cpu(cap_hdr_v1_0->sos.fw_version); | ||
| 3297 | adev->psp.cap_ucode_size = le32_to_cpu(cap_hdr_v1_0->header.ucode_size_bytes); | ||
| 3298 | |||
| 3299 | return 0; | ||
| 3300 | |||
| 3301 | out: | ||
| 3302 | release_firmware(adev->psp.cap_fw); | ||
| 3303 | adev->psp.cap_fw = NULL; | ||
| 3304 | return err; | ||
| 3305 | } | ||
| 3306 | |||
| 3220 | static int psp_set_clockgating_state(void *handle, | 3307 | static int psp_set_clockgating_state(void *handle, |
| 3221 | enum amd_clockgating_state state) | 3308 | enum amd_clockgating_state state) |
| 3222 | { | 3309 | { |
| @@ -306,6 +306,9 @@ struct psp_context · 305 unmodified lines | |||
| 306 | /* toc firmware */ | 306 | /* toc firmware */ |
| 307 | const struct firmware *toc_fw; | 307 | const struct firmware *toc_fw; |
| 308 | 308 | ||
| 309 | /* cap firmware */ | ||
| 310 | const struct firmware *cap_fw; | ||
| 311 | |||
| 309 | /* fence buffer */ | 312 | /* fence buffer */ |
| 310 | struct amdgpu_bo *fence_buf_bo; | 313 | struct amdgpu_bo *fence_buf_bo; |
| 311 | uint64_t fence_buf_mc_addr; | 314 | uint64_t fence_buf_mc_addr; |
| @@ -327,6 +330,10 @@ struct psp_context · 15 unmodified lines | |||
| 327 | const struct firmware *ta_fw; | 330 | const struct firmware *ta_fw; |
| 328 | uint32_t ta_fw_version; | 331 | uint32_t ta_fw_version; |
| 329 | 332 | ||
| 333 | uint32_t cap_fw_version; | ||
| 334 | uint32_t cap_feature_version; | ||
| 335 | uint32_t cap_ucode_size; | ||
| 336 | |||
| 330 | struct ta_context asd_context; | 337 | struct ta_context asd_context; |
| 331 | struct psp_xgmi_context xgmi_context; | 338 | struct psp_xgmi_context xgmi_context; |
| 332 | struct psp_ras_context ras_context; | 339 | struct psp_ras_context ras_context; |
| @@ -440,6 +447,8 @@ int psp_init_sos_microcode(struct psp_context *psp, · 107 unmodified lines | |||
| 440 | const char *chip_name); | 447 | const char *chip_name); |
| 441 | int psp_init_ta_microcode(struct psp_context *psp, | 448 | int psp_init_ta_microcode(struct psp_context *psp, |
| 442 | const char *chip_name); | 449 | const char *chip_name); |
| 450 | int psp_init_cap_microcode(struct psp_context *psp, | ||
| 451 | const char *chip_name); | ||
| 443 | int psp_get_fw_attestation_records_addr(struct psp_context *psp, | 452 | int psp_get_fw_attestation_records_addr(struct psp_context *psp, |
| 444 | uint64_t *output_ptr); | 453 | uint64_t *output_ptr); |
| 445 | 454 | ||
| @@ -75,6 +75,13 @@ const char *ras_mca_block_string[] = { · 74 unmodified lines | |||
| 75 | "mca_iohc", | 75 | "mca_iohc", |
| 76 | }; | 76 | }; |
| 77 | 77 | ||
| 78 | struct amdgpu_ras_block_list { | ||
| 79 | /* ras block link */ | ||
| 80 | struct list_head node; | ||
| 81 | |||
| 82 | struct amdgpu_ras_block_object *ras_obj; | ||
| 83 | }; | ||
| 84 | |||
| 78 | const char *get_ras_block_str(struct ras_common_if *ras_block) | 85 | const char *get_ras_block_str(struct ras_common_if *ras_block) |
| 79 | { | 86 | { |
| 80 | if (!ras_block) | 87 | if (!ras_block) |
| @@ -89,6 +96,9 @@ const char *get_ras_block_str(struct ras_common_if *ras_block) · 8 unmodified lines | |||
| 89 | return ras_block_string[ras_block->block]; | 96 | return ras_block_string[ras_block->block]; |
| 90 | } | 97 | } |
| 91 | 98 | ||
| 99 | #define ras_block_str(_BLOCK_) \ | ||
| 100 | (((_BLOCK_) < ARRAY_SIZE(ras_block_string)) ? ras_block_string[_BLOCK_] : "Out Of Range") | ||
| 101 | |||
| 92 | #define ras_err_str(i) (ras_error_string[ffs(i)]) | 102 | #define ras_err_str(i) (ras_error_string[ffs(i)]) |
| 93 | 103 | ||
| 94 | #define RAS_DEFAULT_FLAGS (AMDGPU_RAS_FLAG_INIT_BY_VBIOS) | 104 | #define RAS_DEFAULT_FLAGS (AMDGPU_RAS_FLAG_INIT_BY_VBIOS) |
| @@ -155,14 +165,9 @@ static int amdgpu_reserve_page_direct(struct amdgpu_device *adev, uint64_t addre · 60 unmodified lines | |||
| 155 | } | 165 | } |
| 156 | 166 | ||
| 157 | memset(&err_rec, 0x0, sizeof(struct eeprom_table_record)); | 167 | memset(&err_rec, 0x0, sizeof(struct eeprom_table_record)); |
| 158 | |||
| 159 | err_rec.address = address; | ||
| 160 | err_rec.retired_page = address >> AMDGPU_GPU_PAGE_SHIFT; | ||
| 161 | err_rec.ts = (uint64_t)ktime_get_real_seconds(); | ||
| 162 | err_rec.err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE; | ||
| 163 | |||
| 164 | err_data.err_addr = &err_rec; | 168 | err_data.err_addr = &err_rec; |
| 165 | err_data.err_addr_cnt = 1; | 169 | amdgpu_umc_fill_error_record(&err_data, address, |
| 170 | (address >> AMDGPU_GPU_PAGE_SHIFT), 0, 0); | ||
| 166 | 171 | ||
| 167 | if (amdgpu_bad_page_threshold != 0) { | 172 | if (amdgpu_bad_page_threshold != 0) { |
| 168 | amdgpu_ras_add_bad_pages(adev, err_data.err_addr, | 173 | amdgpu_ras_add_bad_pages(adev, err_data.err_addr, |
| @@ -452,7 +457,7 @@ static ssize_t amdgpu_ras_debugfs_ctrl_write(struct file *f, · 283 unmodified lines | |||
| 452 | } | 457 | } |
| 453 | 458 | ||
| 454 | if (ret) | 459 | if (ret) |
| 455 | return -EINVAL; | 460 | return ret; |
| 456 | 461 | ||
| 457 | return size; | 462 | return size; |
| 458 | } | 463 | } |
| @@ -866,30 +871,47 @@ static int amdgpu_ras_enable_all_features(struct amdgpu_device *adev, · 407 unmodified lines | |||
| 866 | } | 871 | } |
| 867 | /* feature ctl end */ | 872 | /* feature ctl end */ |
| 868 | 873 | ||
| 874 | static int amdgpu_ras_block_match_default(struct amdgpu_ras_block_object *block_obj, | ||
| 875 | enum amdgpu_ras_block block) | ||
| 876 | { | ||
| 877 | if (!block_obj) | ||
| 878 | return -EINVAL; | ||
| 879 | |||
| 880 | if (block_obj->ras_comm.block == block) | ||
| 881 | return 0; | ||
| 882 | |||
| 883 | return -EINVAL; | ||
| 884 | } | ||
| 869 | 885 | ||
| 870 | static void amdgpu_ras_mca_query_error_status(struct amdgpu_device *adev, | 886 | static struct amdgpu_ras_block_object *amdgpu_ras_get_ras_block(struct amdgpu_device *adev, |
| 871 | struct ras_common_if *ras_block, | 887 | enum amdgpu_ras_block block, uint32_t sub_block_index) |
| 872 | struct ras_err_data *err_data) | ||
| 873 | { | 888 | { |
| 874 | switch (ras_block->sub_block_index) { | 889 | struct amdgpu_ras_block_list *node, *tmp; |
| 875 | case AMDGPU_RAS_MCA_BLOCK__MP0: | 890 | struct amdgpu_ras_block_object *obj; |
| 876 | if (adev->mca.mp0.ras_funcs && | 891 | |
| 877 | adev->mca.mp0.ras_funcs->query_ras_error_count) | 892 | if (block >= AMDGPU_RAS_BLOCK__LAST) |
| 878 | adev->mca.mp0.ras_funcs->query_ras_error_count(adev, &err_data); | 893 | return NULL; |
| 879 | break; | 894 | |
| 880 | case AMDGPU_RAS_MCA_BLOCK__MP1: | 895 | if (!amdgpu_ras_is_supported(adev, block)) |
| 881 | if (adev->mca.mp1.ras_funcs && | 896 | return NULL; |
| 882 | adev->mca.mp1.ras_funcs->query_ras_error_count) | 897 | |
| 883 | adev->mca.mp1.ras_funcs->query_ras_error_count(adev, &err_data); | 898 | list_for_each_entry_safe(node, tmp, &adev->ras_list, node) { |
| 884 | break; | 899 | if (!node->ras_obj) { |
| 885 | case AMDGPU_RAS_MCA_BLOCK__MPIO: | 900 | dev_warn(adev->dev, "Warning: abnormal ras list node.\n"); |
| 886 | if (adev->mca.mpio.ras_funcs && | 901 | continue; |
| 887 | adev->mca.mpio.ras_funcs->query_ras_error_count) | 902 | } |
| 888 | adev->mca.mpio.ras_funcs->query_ras_error_count(adev, &err_data); | 903 | |
| 889 | break; | 904 | obj = node->ras_obj; |
| 890 | default: | 905 | if (obj->ras_block_match) { |
| 891 | break; | 906 | if (obj->ras_block_match(obj, block, sub_block_index) == 0) |
| 907 | return obj; | ||
| 908 | } else { | ||
| 909 | if (amdgpu_ras_block_match_default(obj, block) == 0) | ||
| 910 | return obj; | ||
| 911 | } | ||
| 892 | } | 912 | } |
| 913 | |||
| 914 | return NULL; | ||
| 893 | } | 915 | } |
| 894 | 916 | ||
| 895 | static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_data *err_data) | 917 | static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_data *err_data) |
| @@ -901,26 +923,26 @@ static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_d · 5 unmodified lines | |||
| 901 | * choosing right query method according to | 923 | * choosing right query method according to |
| 902 | * whether smu support query error information | 924 | * whether smu support query error information |
| 903 | */ | 925 | */ |
| 904 | ret = smu_get_ecc_info(&adev->smu, (void *)&(ras->umc_ecc)); | 926 | ret = amdgpu_dpm_get_ecc_info(adev, (void *)&(ras->umc_ecc)); |
| 905 | if (ret == -EOPNOTSUPP) { | 927 | if (ret == -EOPNOTSUPP) { |
| 906 | if (adev->umc.ras_funcs && | 928 | if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && |
| 907 | adev->umc.ras_funcs->query_ras_error_count) | 929 | adev->umc.ras->ras_block.hw_ops->query_ras_error_count) |
| 908 | adev->umc.ras_funcs->query_ras_error_count(adev, err_data); | 930 | adev->umc.ras->ras_block.hw_ops->query_ras_error_count(adev, err_data); |
| 909 | 931 | ||
| 910 | /* umc query_ras_error_address is also responsible for clearing | 932 | /* umc query_ras_error_address is also responsible for clearing |
| 911 | * error status | 933 | * error status |
| 912 | */ | 934 | */ |
| 913 | if (adev->umc.ras_funcs && | 935 | if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && |
| 914 | adev->umc.ras_funcs->query_ras_error_address) | 936 | adev->umc.ras->ras_block.hw_ops->query_ras_error_address) |
| 915 | adev->umc.ras_funcs->query_ras_error_address(adev, err_data); | 937 | adev->umc.ras->ras_block.hw_ops->query_ras_error_address(adev, err_data); |
| 916 | } else if (!ret) { | 938 | } else if (!ret) { |
| 917 | if (adev->umc.ras_funcs && | 939 | if (adev->umc.ras && |
| 918 | adev->umc.ras_funcs->ecc_info_query_ras_error_count) | 940 | adev->umc.ras->ecc_info_query_ras_error_count) |
| 919 | adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, err_data); | 941 | adev->umc.ras->ecc_info_query_ras_error_count(adev, err_data); |
| 920 | 942 | ||
| 921 | if (adev->umc.ras_funcs && | 943 | if (adev->umc.ras && |
| 922 | adev->umc.ras_funcs->ecc_info_query_ras_error_address) | 944 | adev->umc.ras->ecc_info_query_ras_error_address) |
| 923 | adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, err_data); | 945 | adev->umc.ras->ecc_info_query_ras_error_address(adev, err_data); |
| 924 | } | 946 | } |
| 925 | } | 947 | } |
| 926 | 948 | ||
| @@ -928,62 +950,32 @@ static void amdgpu_ras_get_ecc_info(struct amdgpu_device *adev, struct ras_err_d · 1 unmodified line | |||
| 928 | int amdgpu_ras_query_error_status(struct amdgpu_device *adev, | 950 | int amdgpu_ras_query_error_status(struct amdgpu_device *adev, |
| 929 | struct ras_query_if *info) | 951 | struct ras_query_if *info) |
| 930 | { | 952 | { |
| 953 | struct amdgpu_ras_block_object *block_obj = NULL; | ||
| 931 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head); | 954 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head); |
| 932 | struct ras_err_data err_data = {0, 0, 0, NULL}; | 955 | struct ras_err_data err_data = {0, 0, 0, NULL}; |
| 933 | int i; | ||
| 934 | 956 | ||
| 935 | if (!obj) | 957 | if (!obj) |
| 936 | return -EINVAL; | 958 | return -EINVAL; |
| 937 | 959 | ||
| 938 | switch (info->head.block) { | 960 | if (info->head.block == AMDGPU_RAS_BLOCK__UMC) { |
| 939 | case AMDGPU_RAS_BLOCK__UMC: | ||
| 940 | amdgpu_ras_get_ecc_info(adev, &err_data); | 961 | amdgpu_ras_get_ecc_info(adev, &err_data); |
| 941 | break; | 962 | } else { |
| 942 | case AMDGPU_RAS_BLOCK__SDMA: | 963 | block_obj = amdgpu_ras_get_ras_block(adev, info->head.block, 0); |
| 943 | if (adev->sdma.funcs->query_ras_error_count) { | 964 | if (!block_obj || !block_obj->hw_ops) { |
| 944 | for (i = 0; i < adev->sdma.num_instances; i++) | 965 | dev_dbg_once(adev->dev, "%s doesn't config RAS function\n", |
| 945 | adev->sdma.funcs->query_ras_error_count(adev, i, | 966 | get_ras_block_str(&info->head)); |
| 946 | &err_data); | 967 | return -EINVAL; |
| 947 | } | 968 | } |
| 948 | break; | 969 | |
| 949 | case AMDGPU_RAS_BLOCK__GFX: | 970 | if (block_obj->hw_ops->query_ras_error_count) |
| 950 | if (adev->gfx.ras_funcs && | 971 | block_obj->hw_ops->query_ras_error_count(adev, &err_data); |
| 951 | adev->gfx.ras_funcs->query_ras_error_count) | 972 | |
| 952 | adev->gfx.ras_funcs->query_ras_error_count(adev, &err_data); | 973 | if ((info->head.block == AMDGPU_RAS_BLOCK__SDMA) || |
| 953 | 974 | (info->head.block == AMDGPU_RAS_BLOCK__GFX) || | |
| 954 | if (adev->gfx.ras_funcs && | 975 | (info->head.block == AMDGPU_RAS_BLOCK__MMHUB)) { |
| 955 | adev->gfx.ras_funcs->query_ras_error_status) | 976 | if (block_obj->hw_ops->query_ras_error_status) |
| 956 | adev->gfx.ras_funcs->query_ras_error_status(adev); | 977 | block_obj->hw_ops->query_ras_error_status(adev); |
| 957 | break; | 978 | } |
| 958 | case AMDGPU_RAS_BLOCK__MMHUB: | ||
| 959 | if (adev->mmhub.ras_funcs && | ||
| 960 | adev->mmhub.ras_funcs->query_ras_error_count) | ||
| 961 | adev->mmhub.ras_funcs->query_ras_error_count(adev, &err_data); | ||
| 962 | |||
| 963 | if (adev->mmhub.ras_funcs && | ||
| 964 | adev->mmhub.ras_funcs->query_ras_error_status) | ||
| 965 | adev->mmhub.ras_funcs->query_ras_error_status(adev); | ||
| 966 | break; | ||
| 967 | case AMDGPU_RAS_BLOCK__PCIE_BIF: | ||
| 968 | if (adev->nbio.ras_funcs && | ||
| 969 | adev->nbio.ras_funcs->query_ras_error_count) | ||
| 970 | adev->nbio.ras_funcs->query_ras_error_count(adev, &err_data); | ||
| 971 | break; | ||
| 972 | case AMDGPU_RAS_BLOCK__XGMI_WAFL: | ||
| 973 | if (adev->gmc.xgmi.ras_funcs && | ||
| 974 | adev->gmc.xgmi.ras_funcs->query_ras_error_count) | ||
| 975 | adev->gmc.xgmi.ras_funcs->query_ras_error_count(adev, &err_data); | ||
| 976 | break; | ||
| 977 | case AMDGPU_RAS_BLOCK__HDP: | ||
| 978 | if (adev->hdp.ras_funcs && | ||
| 979 | adev->hdp.ras_funcs->query_ras_error_count) | ||
| 980 | adev->hdp.ras_funcs->query_ras_error_count(adev, &err_data); | ||
| 981 | break; | ||
| 982 | case AMDGPU_RAS_BLOCK__MCA: | ||
| 983 | amdgpu_ras_mca_query_error_status(adev, &info->head, &err_data); | ||
| 984 | break; | ||
| 985 | default: | ||
| 986 | break; | ||
| 987 | } | 979 | } |
| 988 | 980 | ||
| 989 | obj->err_data.ue_count += err_data.ue_count; | 981 | obj->err_data.ue_count += err_data.ue_count; |
| @@ -1040,68 +1032,27 @@ int amdgpu_ras_query_error_status(struct amdgpu_device *adev, · 50 unmodified lines | |||
| 1040 | int amdgpu_ras_reset_error_status(struct amdgpu_device *adev, | 1032 | int amdgpu_ras_reset_error_status(struct amdgpu_device *adev, |
| 1041 | enum amdgpu_ras_block block) | 1033 | enum amdgpu_ras_block block) |
| 1042 | { | 1034 | { |
| 1035 | struct amdgpu_ras_block_object *block_obj = amdgpu_ras_get_ras_block(adev, block, 0); | ||
| 1036 | |||
| 1043 | if (!amdgpu_ras_is_supported(adev, block)) | 1037 | if (!amdgpu_ras_is_supported(adev, block)) |
| 1044 | return -EINVAL; | 1038 | return -EINVAL; |
| 1045 | 1039 | ||
| 1046 | switch (block) { | 1040 | if (!block_obj || !block_obj->hw_ops) { |
| 1047 | case AMDGPU_RAS_BLOCK__GFX: | 1041 | dev_dbg_once(adev->dev, "%s doesn't config RAS function\n", |
| 1048 | if (adev->gfx.ras_funcs && | 1042 | ras_block_str(block)); |
| 1049 | adev->gfx.ras_funcs->reset_ras_error_count) | 1043 | return -EINVAL; |
| 1050 | adev->gfx.ras_funcs->reset_ras_error_count(adev); | ||
| 1051 | |||
| 1052 | if (adev->gfx.ras_funcs && | ||
| 1053 | adev->gfx.ras_funcs->reset_ras_error_status) | ||
| 1054 | adev->gfx.ras_funcs->reset_ras_error_status(adev); | ||
| 1055 | break; | ||
| 1056 | case AMDGPU_RAS_BLOCK__MMHUB: | ||
| 1057 | if (adev->mmhub.ras_funcs && | ||
| 1058 | adev->mmhub.ras_funcs->reset_ras_error_count) | ||
| 1059 | adev->mmhub.ras_funcs->reset_ras_error_count(adev); | ||
| 1060 | |||
| 1061 | if (adev->mmhub.ras_funcs && | ||
| 1062 | adev->mmhub.ras_funcs->reset_ras_error_status) | ||
| 1063 | adev->mmhub.ras_funcs->reset_ras_error_status(adev); | ||
| 1064 | break; | ||
| 1065 | case AMDGPU_RAS_BLOCK__SDMA: | ||
| 1066 | if (adev->sdma.funcs->reset_ras_error_count) | ||
| 1067 | adev->sdma.funcs->reset_ras_error_count(adev); | ||
| 1068 | break; | ||
| 1069 | case AMDGPU_RAS_BLOCK__HDP: | ||
| 1070 | if (adev->hdp.ras_funcs && | ||
| 1071 | adev->hdp.ras_funcs->reset_ras_error_count) | ||
| 1072 | adev->hdp.ras_funcs->reset_ras_error_count(adev); | ||
| 1073 | break; | ||
| 1074 | default: | ||
| 1075 | break; | ||
| 1076 | } | 1044 | } |
| 1077 | 1045 | ||
| 1078 | return 0; | 1046 | if (block_obj->hw_ops->reset_ras_error_count) |
| 1079 | } | 1047 | block_obj->hw_ops->reset_ras_error_count(adev); |
| 1080 | |||
| 1081 | /* Trigger XGMI/WAFL error */ | ||
| 1082 | static int amdgpu_ras_error_inject_xgmi(struct amdgpu_device *adev, | ||
| 1083 | struct ta_ras_trigger_error_input *block_info) | ||
| 1084 | { | ||
| 1085 | int ret; | ||
| 1086 | 1048 | ||
| 1087 | if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_DISALLOW)) | 1049 | if ((block == AMDGPU_RAS_BLOCK__GFX) || |
| 1088 | dev_warn(adev->dev, "Failed to disallow df cstate"); | 1050 | (block == AMDGPU_RAS_BLOCK__MMHUB)) { |
| 1089 | 1051 | if (block_obj->hw_ops->reset_ras_error_status) | |
| 1090 | if (amdgpu_dpm_allow_xgmi_power_down(adev, false)) | 1052 | block_obj->hw_ops->reset_ras_error_status(adev); |
| 1091 | dev_warn(adev->dev, "Failed to disallow XGMI power down"); | 1053 | } |
| 1092 | |||
| 1093 | ret = psp_ras_trigger_error(&adev->psp, block_info); | ||
| 1094 | |||
| 1095 | if (amdgpu_ras_intr_triggered()) | ||
| 1096 | return ret; | ||
| 1097 | |||
| 1098 | if (amdgpu_dpm_allow_xgmi_power_down(adev, true)) | ||
| 1099 | dev_warn(adev->dev, "Failed to allow XGMI power down"); | ||
| 1100 | |||
| 1101 | if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_ALLOW)) | ||
| 1102 | dev_warn(adev->dev, "Failed to allow df cstate"); | ||
| 1103 | 1054 | ||
| 1104 | return ret; | 1055 | return 0; |
| 1105 | } | 1056 | } |
| 1106 | 1057 | ||
| 1107 | /* wrapper of psp_ras_trigger_error */ | 1058 | /* wrapper of psp_ras_trigger_error */ |
| @@ -1116,11 +1067,20 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 8 unmodified lines | |||
| 1116 | .address = info->address, | 1067 | .address = info->address, |
| 1117 | .value = info->value, | 1068 | .value = info->value, |
| 1118 | }; | 1069 | }; |
| 1119 | int ret = 0; | 1070 | int ret = -EINVAL; |
| 1071 | struct amdgpu_ras_block_object *block_obj = amdgpu_ras_get_ras_block(adev, | ||
| 1072 | info->head.block, | ||
| 1073 | info->head.sub_block_index); | ||
| 1120 | 1074 | ||
| 1121 | if (!obj) | 1075 | if (!obj) |
| 1122 | return -EINVAL; | 1076 | return -EINVAL; |
| 1123 | 1077 | ||
| 1078 | if (!block_obj || !block_obj->hw_ops) { | ||
| 1079 | dev_dbg_once(adev->dev, "%s doesn't config RAS function\n", | ||
| 1080 | get_ras_block_str(&info->head)); | ||
| 1081 | return -EINVAL; | ||
| 1082 | } | ||
| 1083 | |||
| 1124 | /* Calculate XGMI relative offset */ | 1084 | /* Calculate XGMI relative offset */ |
| 1125 | if (adev->gmc.xgmi.num_physical_nodes > 1) { | 1085 | if (adev->gmc.xgmi.num_physical_nodes > 1) { |
| 1126 | block_info.address = | 1086 | block_info.address = |
| @@ -1128,28 +1088,15 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 1 unmodified line | |||
| 1128 | block_info.address); | 1088 | block_info.address); |
| 1129 | } | 1089 | } |
| 1130 | 1090 | ||
| 1131 | switch (info->head.block) { | 1091 | if (info->head.block == AMDGPU_RAS_BLOCK__GFX) { |
| 1132 | case AMDGPU_RAS_BLOCK__GFX: | 1092 | if (block_obj->hw_ops->ras_error_inject) |
| 1133 | if (adev->gfx.ras_funcs && | 1093 | ret = block_obj->hw_ops->ras_error_inject(adev, info); |
| 1134 | adev->gfx.ras_funcs->ras_error_inject) | 1094 | } else { |
| 1135 | ret = adev->gfx.ras_funcs->ras_error_inject(adev, info); | 1095 | /* If defined special ras_error_inject(e.g: xgmi), implement special ras_error_inject */ |
| 1136 | else | 1096 | if (block_obj->hw_ops->ras_error_inject) |
| 1137 | ret = -EINVAL; | 1097 | ret = block_obj->hw_ops->ras_error_inject(adev, &block_info); |
| 1138 | break; | 1098 | else /*If not defined .ras_error_inject, use default ras_error_inject*/ |
| 1139 | case AMDGPU_RAS_BLOCK__UMC: | 1099 | ret = psp_ras_trigger_error(&adev->psp, &block_info); |
| 1140 | case AMDGPU_RAS_BLOCK__SDMA: | ||
| 1141 | case AMDGPU_RAS_BLOCK__MMHUB: | ||
| 1142 | case AMDGPU_RAS_BLOCK__PCIE_BIF: | ||
| 1143 | case AMDGPU_RAS_BLOCK__MCA: | ||
| 1144 | ret = psp_ras_trigger_error(&adev->psp, &block_info); | ||
| 1145 | break; | ||
| 1146 | case AMDGPU_RAS_BLOCK__XGMI_WAFL: | ||
| 1147 | ret = amdgpu_ras_error_inject_xgmi(adev, &block_info); | ||
| 1148 | break; | ||
| 1149 | default: | ||
| 1150 | dev_info(adev->dev, "%s error injection is not supported yet\n", | ||
| 1151 | get_ras_block_str(&info->head)); | ||
| 1152 | ret = -EINVAL; | ||
| 1153 | } | 1100 | } |
| 1154 | 1101 | ||
| 1155 | if (ret) | 1102 | if (ret) |
| @@ -1329,18 +1276,17 @@ static int amdgpu_ras_sysfs_remove_feature_node(struct amdgpu_device *adev) · 173 unmodified lines | |||
| 1329 | } | 1276 | } |
| 1330 | 1277 | ||
| 1331 | int amdgpu_ras_sysfs_create(struct amdgpu_device *adev, | 1278 | int amdgpu_ras_sysfs_create(struct amdgpu_device *adev, |
| 1332 | struct ras_fs_if *head) | 1279 | struct ras_common_if *head) |
| 1333 | { | 1280 | { |
| 1334 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, &head->head); | 1281 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, head); |
| 1335 | 1282 | ||
| 1336 | if (!obj || obj->attr_inuse) | 1283 | if (!obj || obj->attr_inuse) |
| 1337 | return -EINVAL; | 1284 | return -EINVAL; |
| 1338 | 1285 | ||
| 1339 | get_obj(obj); | 1286 | get_obj(obj); |
| 1340 | 1287 | ||
| 1341 | memcpy(obj->fs_data.sysfs_name, | 1288 | snprintf(obj->fs_data.sysfs_name, sizeof(obj->fs_data.sysfs_name), |
| 1342 | head->sysfs_name, | 1289 | "%s_err_count", head->name); |
| 1343 | sizeof(obj->fs_data.sysfs_name)); | ||
| 1344 | 1290 | ||
| 1345 | obj->sysfs_attr = (struct device_attribute){ | 1291 | obj->sysfs_attr = (struct device_attribute){ |
| 1346 | .attr = { | 1292 | .attr = { |
| @@ -1647,9 +1593,9 @@ int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, · 300 unmodified lines | |||
| 1647 | } | 1593 | } |
| 1648 | 1594 | ||
| 1649 | int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, | 1595 | int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, |
| 1650 | struct ras_ih_if *info) | 1596 | struct ras_common_if *head) |
| 1651 | { | 1597 | { |
| 1652 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head); | 1598 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, head); |
| 1653 | struct ras_ih_data *data; | 1599 | struct ras_ih_data *data; |
| 1654 | 1600 | ||
| 1655 | if (!obj) | 1601 | if (!obj) |
| @@ -1669,24 +1615,27 @@ int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, · 13 unmodified lines | |||
| 1669 | } | 1615 | } |
| 1670 | 1616 | ||
| 1671 | int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev, | 1617 | int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev, |
| 1672 | struct ras_ih_if *info) | 1618 | struct ras_common_if *head) |
| 1673 | { | 1619 | { |
| 1674 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head); | 1620 | struct ras_manager *obj = amdgpu_ras_find_obj(adev, head); |
| 1675 | struct ras_ih_data *data; | 1621 | struct ras_ih_data *data; |
| 1622 | struct amdgpu_ras_block_object *ras_obj; | ||
| 1676 | 1623 | ||
| 1677 | if (!obj) { | 1624 | if (!obj) { |
| 1678 | /* in case we registe the IH before enable ras feature */ | 1625 | /* in case we registe the IH before enable ras feature */ |
| 1679 | obj = amdgpu_ras_create_obj(adev, &info->head); | 1626 | obj = amdgpu_ras_create_obj(adev, head); |
| 1680 | if (!obj) | 1627 | if (!obj) |
| 1681 | return -EINVAL; | 1628 | return -EINVAL; |
| 1682 | } else | 1629 | } else |
| 1683 | get_obj(obj); | 1630 | get_obj(obj); |
| 1684 | 1631 | ||
| 1632 | ras_obj = container_of(head, struct amdgpu_ras_block_object, ras_comm); | ||
| 1633 | |||
| 1685 | data = &obj->ih_data; | 1634 | data = &obj->ih_data; |
| 1686 | /* add the callback.etc */ | 1635 | /* add the callback.etc */ |
| 1687 | *data = (struct ras_ih_data) { | 1636 | *data = (struct ras_ih_data) { |
| 1688 | .inuse = 0, | 1637 | .inuse = 0, |
| 1689 | .cb = info->cb, | 1638 | .cb = ras_obj->ras_cb, |
| 1690 | .element_size = sizeof(struct amdgpu_iv_entry), | 1639 | .element_size = sizeof(struct amdgpu_iv_entry), |
| 1691 | .rptr = 0, | 1640 | .rptr = 0, |
| 1692 | .wptr = 0, | 1641 | .wptr = 0, |
| @@ -1715,10 +1664,7 @@ static int amdgpu_ras_interrupt_remove_all(struct amdgpu_device *adev) · 22 unmodified lines | |||
| 1715 | struct ras_manager *obj, *tmp; | 1664 | struct ras_manager *obj, *tmp; |
| 1716 | 1665 | ||
| 1717 | list_for_each_entry_safe(obj, tmp, &con->head, node) { | 1666 | list_for_each_entry_safe(obj, tmp, &con->head, node) { |
| 1718 | struct ras_ih_if info = { | 1667 | amdgpu_ras_interrupt_remove_handler(adev, &obj->head); |
| 1719 | .head = obj->head, | ||
| 1720 | }; | ||
| 1721 | amdgpu_ras_interrupt_remove_handler(adev, &info); | ||
| 1722 | } | 1668 | } |
| 1723 | 1669 | ||
| 1724 | return 0; | 1670 | return 0; |
| @@ -1766,24 +1712,28 @@ static void amdgpu_ras_log_on_err_counter(struct amdgpu_device *adev) · 41 unmodified lines | |||
| 1766 | static void amdgpu_ras_error_status_query(struct amdgpu_device *adev, | 1712 | static void amdgpu_ras_error_status_query(struct amdgpu_device *adev, |
| 1767 | struct ras_query_if *info) | 1713 | struct ras_query_if *info) |
| 1768 | { | 1714 | { |
| 1715 | struct amdgpu_ras_block_object *block_obj; | ||
| 1769 | /* | 1716 | /* |
| 1770 | * Only two block need to query read/write | 1717 | * Only two block need to query read/write |
| 1771 | * RspStatus at current state | 1718 | * RspStatus at current state |
| 1772 | */ | 1719 | */ |
| 1773 | switch (info->head.block) { | 1720 | if ((info->head.block != AMDGPU_RAS_BLOCK__GFX) && |
| 1774 | case AMDGPU_RAS_BLOCK__GFX: | 1721 | (info->head.block != AMDGPU_RAS_BLOCK__MMHUB)) |
| 1775 | if (adev->gfx.ras_funcs && | 1722 | return; |
| 1776 | adev->gfx.ras_funcs->query_ras_error_status) | 1723 | |
| 1777 | adev->gfx.ras_funcs->query_ras_error_status(adev); | 1724 | block_obj = amdgpu_ras_get_ras_block(adev, |
| 1778 | break; | 1725 | info->head.block, |
| 1779 | case AMDGPU_RAS_BLOCK__MMHUB: | 1726 | info->head.sub_block_index); |
| 1780 | if (adev->mmhub.ras_funcs && | 1727 | |
| 1781 | adev->mmhub.ras_funcs->query_ras_error_status) | 1728 | if (!block_obj || !block_obj->hw_ops) { |
| 1782 | adev->mmhub.ras_funcs->query_ras_error_status(adev); | 1729 | dev_dbg_once(adev->dev, "%s doesn't config RAS function\n", |
| 1783 | break; | 1730 | get_ras_block_str(&info->head)); |
| 1784 | default: | 1731 | return; |
| 1785 | break; | ||
| 1786 | } | 1732 | } |
| 1733 | |||
| 1734 | if (block_obj->hw_ops->query_ras_error_status) | ||
| 1735 | block_obj->hw_ops->query_ras_error_status(adev); | ||
| 1736 | |||
| 1787 | } | 1737 | } |
| 1788 | 1738 | ||
| 1789 | static void amdgpu_ras_query_err_status(struct amdgpu_device *adev) | 1739 | static void amdgpu_ras_query_err_status(struct amdgpu_device *adev) |
| @@ -2141,8 +2091,7 @@ int amdgpu_ras_recovery_init(struct amdgpu_device *adev) · 351 unmodified lines | |||
| 2141 | if (ret) | 2091 | if (ret) |
| 2142 | goto free; | 2092 | goto free; |
| 2143 | 2093 | ||
| 2144 | if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num) | 2094 | amdgpu_dpm_send_hbm_bad_pages_num(adev, con->eeprom_control.ras_num_recs); |
| 2145 | adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs); | ||
| 2146 | } | 2095 | } |
| 2147 | 2096 | ||
| 2148 | #ifdef CONFIG_X86_MCE_AMD | 2097 | #ifdef CONFIG_X86_MCE_AMD |
| @@ -2348,24 +2297,27 @@ int amdgpu_ras_init(struct amdgpu_device *adev) · 199 unmodified lines | |||
| 2348 | case CHIP_VEGA20: | 2297 | case CHIP_VEGA20: |
| 2349 | case CHIP_ARCTURUS: | 2298 | case CHIP_ARCTURUS: |
| 2350 | case CHIP_ALDEBARAN: | 2299 | case CHIP_ALDEBARAN: |
| 2351 | if (!adev->gmc.xgmi.connected_to_cpu) | 2300 | if (!adev->gmc.xgmi.connected_to_cpu) { |
| 2352 | adev->nbio.ras_funcs = &nbio_v7_4_ras_funcs; | 2301 | adev->nbio.ras = &nbio_v7_4_ras; |
| 2302 | amdgpu_ras_register_ras_block(adev, &adev->nbio.ras->ras_block); | ||
| 2303 | adev->nbio.ras_if = &adev->nbio.ras->ras_block.ras_comm; | ||
| 2304 | } | ||
| 2353 | break; | 2305 | break; |
| 2354 | default: | 2306 | default: |
| 2355 | /* nbio ras is not available */ | 2307 | /* nbio ras is not available */ |
| 2356 | break; | 2308 | break; |
| 2357 | } | 2309 | } |
| 2358 | 2310 | ||
| 2359 | if (adev->nbio.ras_funcs && | 2311 | if (adev->nbio.ras && |
| 2360 | adev->nbio.ras_funcs->init_ras_controller_interrupt) { | 2312 | adev->nbio.ras->init_ras_controller_interrupt) { |
| 2361 | r = adev->nbio.ras_funcs->init_ras_controller_interrupt(adev); | 2313 | r = adev->nbio.ras->init_ras_controller_interrupt(adev); |
| 2362 | if (r) | 2314 | if (r) |
| 2363 | goto release_con; | 2315 | goto release_con; |
| 2364 | } | 2316 | } |
| 2365 | 2317 | ||
| 2366 | if (adev->nbio.ras_funcs && | 2318 | if (adev->nbio.ras && |
| 2367 | adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt) { | 2319 | adev->nbio.ras->init_ras_err_event_athub_interrupt) { |
| 2368 | r = adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt(adev); | 2320 | r = adev->nbio.ras->init_ras_err_event_athub_interrupt(adev); |
| 2369 | if (r) | 2321 | if (r) |
| 2370 | goto release_con; | 2322 | goto release_con; |
| 2371 | } | 2323 | } |
| @@ -2377,12 +2329,12 @@ int amdgpu_ras_init(struct amdgpu_device *adev) · 5 unmodified lines | |||
| 2377 | } | 2329 | } |
| 2378 | else if (adev->df.funcs && | 2330 | else if (adev->df.funcs && |
| 2379 | adev->df.funcs->query_ras_poison_mode && | 2331 | adev->df.funcs->query_ras_poison_mode && |
| 2380 | adev->umc.ras_funcs && | 2332 | adev->umc.ras && |
| 2381 | adev->umc.ras_funcs->query_ras_poison_mode) { | 2333 | adev->umc.ras->query_ras_poison_mode) { |
| 2382 | df_poison = | 2334 | df_poison = |
| 2383 | adev->df.funcs->query_ras_poison_mode(adev); | 2335 | adev->df.funcs->query_ras_poison_mode(adev); |
| 2384 | umc_poison = | 2336 | umc_poison = |
| 2385 | adev->umc.ras_funcs->query_ras_poison_mode(adev); | 2337 | adev->umc.ras->query_ras_poison_mode(adev); |
| 2386 | /* Only poison is set in both DF and UMC, we can support it */ | 2338 | /* Only poison is set in both DF and UMC, we can support it */ |
| 2387 | if (df_poison && umc_poison) | 2339 | if (df_poison && umc_poison) |
| 2388 | con->poison_supported = true; | 2340 | con->poison_supported = true; |
| @@ -2445,11 +2397,10 @@ bool amdgpu_ras_is_poison_mode_supported(struct amdgpu_device *adev) · 56 unmodified lines | |||
| 2445 | } | 2397 | } |
| 2446 | 2398 | ||
| 2447 | /* helper function to handle common stuff in ip late init phase */ | 2399 | /* helper function to handle common stuff in ip late init phase */ |
| 2448 | int amdgpu_ras_late_init(struct amdgpu_device *adev, | 2400 | int amdgpu_ras_block_late_init(struct amdgpu_device *adev, |
| 2449 | struct ras_common_if *ras_block, | 2401 | struct ras_common_if *ras_block) |
| 2450 | struct ras_fs_if *fs_info, | ||
| 2451 | struct ras_ih_if *ih_info) | ||
| 2452 | { | 2402 | { |
| 2403 | struct amdgpu_ras_block_object *ras_obj = NULL; | ||
| 2453 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); | 2404 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); |
| 2454 | unsigned long ue_count, ce_count; | 2405 | unsigned long ue_count, ce_count; |
| 2455 | int r; | 2406 | int r; |
| @@ -2477,15 +2428,16 @@ int amdgpu_ras_late_init(struct amdgpu_device *adev, · 21 unmodified lines | |||
| 2477 | if (adev->in_suspend || amdgpu_in_reset(adev)) | 2428 | if (adev->in_suspend || amdgpu_in_reset(adev)) |
| 2478 | return 0; | 2429 | return 0; |
| 2479 | 2430 | ||
| 2480 | if (ih_info->cb) { | 2431 | ras_obj = container_of(ras_block, struct amdgpu_ras_block_object, ras_comm); |
| 2481 | r = amdgpu_ras_interrupt_add_handler(adev, ih_info); | 2432 | if (ras_obj->ras_cb) { |
| 2433 | r = amdgpu_ras_interrupt_add_handler(adev, ras_block); | ||
| 2482 | if (r) | 2434 | if (r) |
| 2483 | goto interrupt; | 2435 | goto cleanup; |
| 2484 | } | 2436 | } |
| 2485 | 2437 | ||
| 2486 | r = amdgpu_ras_sysfs_create(adev, fs_info); | 2438 | r = amdgpu_ras_sysfs_create(adev, ras_block); |
| 2487 | if (r) | 2439 | if (r) |
| 2488 | goto sysfs; | 2440 | goto interrupt; |
| 2489 | 2441 | ||
| 2490 | /* Those are the cached values at init. | 2442 | /* Those are the cached values at init. |
| 2491 | */ | 2443 | */ |
| @@ -2495,27 +2447,34 @@ int amdgpu_ras_late_init(struct amdgpu_device *adev, · 3 unmodified lines | |||
| 2495 | } | 2447 | } |
| 2496 | 2448 | ||
| 2497 | return 0; | 2449 | return 0; |
| 2498 | cleanup: | 2450 | |
| 2499 | amdgpu_ras_sysfs_remove(adev, ras_block); | ||
| 2500 | sysfs: | ||
| 2501 | if (ih_info->cb) | ||
| 2502 | amdgpu_ras_interrupt_remove_handler(adev, ih_info); | ||
| 2503 | interrupt: | 2451 | interrupt: |
| 2452 | if (ras_obj->ras_cb) | ||
| 2453 | amdgpu_ras_interrupt_remove_handler(adev, ras_block); | ||
| 2454 | cleanup: | ||
| 2504 | amdgpu_ras_feature_enable(adev, ras_block, 0); | 2455 | amdgpu_ras_feature_enable(adev, ras_block, 0); |
| 2505 | return r; | 2456 | return r; |
| 2506 | } | 2457 | } |
| 2507 | 2458 | ||
| 2459 | static int amdgpu_ras_block_late_init_default(struct amdgpu_device *adev, | ||
| 2460 | struct ras_common_if *ras_block) | ||
| 2461 | { | ||
| 2462 | return amdgpu_ras_block_late_init(adev, ras_block); | ||
| 2463 | } | ||
| 2464 | |||
| 2508 | /* helper function to remove ras fs node and interrupt handler */ | 2465 | /* helper function to remove ras fs node and interrupt handler */ |
| 2509 | void amdgpu_ras_late_fini(struct amdgpu_device *adev, | 2466 | void amdgpu_ras_block_late_fini(struct amdgpu_device *adev, |
| 2510 | struct ras_common_if *ras_block, | 2467 | struct ras_common_if *ras_block) |
| 2511 | struct ras_ih_if *ih_info) | ||
| 2512 | { | 2468 | { |
| 2513 | if (!ras_block || !ih_info) | 2469 | struct amdgpu_ras_block_object *ras_obj; |
| 2470 | if (!ras_block) | ||
| 2514 | return; | 2471 | return; |
| 2515 | 2472 | ||
| 2516 | amdgpu_ras_sysfs_remove(adev, ras_block); | 2473 | amdgpu_ras_sysfs_remove(adev, ras_block); |
| 2517 | if (ih_info->cb) | 2474 | |
| 2518 | amdgpu_ras_interrupt_remove_handler(adev, ih_info); | 2475 | ras_obj = container_of(ras_block, struct amdgpu_ras_block_object, ras_comm); |
| 2476 | if (ras_obj->ras_cb) | ||
| 2477 | amdgpu_ras_interrupt_remove_handler(adev, ras_block); | ||
| 2519 | } | 2478 | } |
| 2520 | 2479 | ||
| 2521 | /* do some init work after IP late init as dependence. | 2480 | /* do some init work after IP late init as dependence. |
| @@ -2568,6 +2527,33 @@ void amdgpu_ras_suspend(struct amdgpu_device *adev) · 46 unmodified lines | |||
| 2568 | amdgpu_ras_disable_all_features(adev, 1); | 2527 | amdgpu_ras_disable_all_features(adev, 1); |
| 2569 | } | 2528 | } |
| 2570 | 2529 | ||
| 2530 | int amdgpu_ras_late_init(struct amdgpu_device *adev) | ||
| 2531 | { | ||
| 2532 | struct amdgpu_ras_block_list *node, *tmp; | ||
| 2533 | struct amdgpu_ras_block_object *obj; | ||
| 2534 | int r; | ||
| 2535 | |||
| 2536 | list_for_each_entry_safe(node, tmp, &adev->ras_list, node) { | ||
| 2537 | if (!node->ras_obj) { | ||
| 2538 | dev_warn(adev->dev, "Warning: abnormal ras list node.\n"); | ||
| 2539 | continue; | ||
| 2540 | } | ||
| 2541 | |||
| 2542 | obj = node->ras_obj; | ||
| 2543 | if (obj->ras_late_init) { | ||
| 2544 | r = obj->ras_late_init(adev, &obj->ras_comm); | ||
| 2545 | if (r) { | ||
| 2546 | dev_err(adev->dev, "%s failed to execute ras_late_init! ret:%d\n", | ||
| 2547 | obj->ras_comm.name, r); | ||
| 2548 | return r; | ||
| 2549 | } | ||
| 2550 | } else | ||
| 2551 | amdgpu_ras_block_late_init_default(adev, &obj->ras_comm); | ||
| 2552 | } | ||
| 2553 | |||
| 2554 | return 0; | ||
| 2555 | } | ||
| 2556 | |||
| 2571 | /* do some fini work before IP fini as dependence */ | 2557 | /* do some fini work before IP fini as dependence */ |
| 2572 | int amdgpu_ras_pre_fini(struct amdgpu_device *adev) | 2558 | int amdgpu_ras_pre_fini(struct amdgpu_device *adev) |
| 2573 | { | 2559 | { |
| @@ -2585,6 +2571,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) · 11 unmodified lines | |||
| 2585 | 2571 | ||
| 2586 | int amdgpu_ras_fini(struct amdgpu_device *adev) | 2572 | int amdgpu_ras_fini(struct amdgpu_device *adev) |
| 2587 | { | 2573 | { |
| 2574 | struct amdgpu_ras_block_list *ras_node, *tmp; | ||
| 2588 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); | 2575 | struct amdgpu_ras *con = amdgpu_ras_get_context(adev); |
| 2589 | 2576 | ||
| 2590 | if (!adev->ras_enabled || !con) | 2577 | if (!adev->ras_enabled || !con) |
| @@ -2603,6 +2590,12 @@ int amdgpu_ras_fini(struct amdgpu_device *adev) · 12 unmodified lines | |||
| 2603 | amdgpu_ras_set_context(adev, NULL); | 2590 | amdgpu_ras_set_context(adev, NULL); |
| 2604 | kfree(con); | 2591 | kfree(con); |
| 2605 | 2592 | ||
| 2593 | /* Clear ras blocks from ras_list and free ras block list node */ | ||
| 2594 | list_for_each_entry_safe(ras_node, tmp, &adev->ras_list, node) { | ||
| 2595 | list_del(&ras_node->node); | ||
| 2596 | kfree(ras_node); | ||
| 2597 | } | ||
| 2598 | |||
| 2606 | return 0; | 2599 | return 0; |
| 2607 | } | 2600 | } |
| 2608 | 2601 | ||
| @@ -2717,8 +2710,6 @@ static int amdgpu_bad_page_notifier(struct notifier_block *nb, · 108 unmodified lines | |||
| 2717 | dev_info(adev->dev, "Uncorrectable error detected in UMC inst: %d, chan_idx: %d", | 2710 | dev_info(adev->dev, "Uncorrectable error detected in UMC inst: %d, chan_idx: %d", |
| 2718 | umc_inst, ch_inst); | 2711 | umc_inst, ch_inst); |
| 2719 | 2712 | ||
| 2720 | memset(&err_rec, 0x0, sizeof(struct eeprom_table_record)); | ||
| 2721 | |||
| 2722 | /* | 2713 | /* |
| 2723 | * Translate UMC channel address to Physical address | 2714 | * Translate UMC channel address to Physical address |
| 2724 | */ | 2715 | */ |
| @@ -2730,16 +2721,10 @@ static int amdgpu_bad_page_notifier(struct notifier_block *nb, · 5 unmodified lines | |||
| 2730 | ADDR_OF_256B_BLOCK(channel_index) | | 2721 | ADDR_OF_256B_BLOCK(channel_index) | |
| 2731 | OFFSET_IN_256B_BLOCK(m->addr); | 2722 | OFFSET_IN_256B_BLOCK(m->addr); |
| 2732 | 2723 | ||
| 2733 | err_rec.address = m->addr; | 2724 | memset(&err_rec, 0x0, sizeof(struct eeprom_table_record)); |
| 2734 | err_rec.retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT; | ||
| 2735 | err_rec.ts = (uint64_t)ktime_get_real_seconds(); | ||
| 2736 | err_rec.err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE; | ||
| 2737 | err_rec.cu = 0; | ||
| 2738 | err_rec.mem_channel = channel_index; | ||
| 2739 | err_rec.mcumc_id = umc_inst; | ||
| 2740 | |||
| 2741 | err_data.err_addr = &err_rec; | 2725 | err_data.err_addr = &err_rec; |
| 2742 | err_data.err_addr_cnt = 1; | 2726 | amdgpu_umc_fill_error_record(&err_data, m->addr, |
| 2727 | retired_page, channel_index, umc_inst); | ||
| 2743 | 2728 | ||
| 2744 | if (amdgpu_bad_page_threshold != 0) { | 2729 | if (amdgpu_bad_page_threshold != 0) { |
| 2745 | amdgpu_ras_add_bad_pages(adev, err_data.err_addr, | 2730 | amdgpu_ras_add_bad_pages(adev, err_data.err_addr, |
| @@ -2777,3 +2762,63 @@ static void amdgpu_register_bad_pages_mca_notifier(struct amdgpu_device *adev) · 31 unmodified lines | |||
| 2777 | } | 2762 | } |
| 2778 | } | 2763 | } |
| 2779 | #endif | 2764 | #endif |
| 2765 | |||
| 2766 | struct amdgpu_ras *amdgpu_ras_get_context(struct amdgpu_device *adev) | ||
| 2767 | { | ||
| 2768 | if (!adev) | ||
| 2769 | return NULL; | ||
| 2770 | |||
| 2771 | return adev->psp.ras_context.ras; | ||
| 2772 | } | ||
| 2773 | |||
| 2774 | int amdgpu_ras_set_context(struct amdgpu_device *adev, struct amdgpu_ras *ras_con) | ||
| 2775 | { | ||
| 2776 | if (!adev) | ||
| 2777 | return -EINVAL; | ||
| 2778 | |||
| 2779 | adev->psp.ras_context.ras = ras_con; | ||
| 2780 | return 0; | ||
| 2781 | } | ||
| 2782 | |||
| 2783 | /* check if ras is supported on block, say, sdma, gfx */ | ||
| 2784 | int amdgpu_ras_is_supported(struct amdgpu_device *adev, | ||
| 2785 | unsigned int block) | ||
| 2786 | { | ||
| 2787 | struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | ||
| 2788 | |||
| 2789 | if (block >= AMDGPU_RAS_BLOCK_COUNT) | ||
| 2790 | return 0; | ||
| 2791 | return ras && (adev->ras_enabled & (1 << block)); | ||
| 2792 | } | ||
| 2793 | |||
| 2794 | int amdgpu_ras_reset_gpu(struct amdgpu_device *adev) | ||
| 2795 | { | ||
| 2796 | struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | ||
| 2797 | |||
| 2798 | if (atomic_cmpxchg(&ras->in_recovery, 0, 1) == 0) | ||
| 2799 | schedule_work(&ras->recovery_work); | ||
| 2800 | return 0; | ||
| 2801 | } | ||
| 2802 | |||
| 2803 | |||
| 2804 | /* Register each ip ras block into amdgpu ras */ | ||
| 2805 | int amdgpu_ras_register_ras_block(struct amdgpu_device *adev, | ||
| 2806 | struct amdgpu_ras_block_object *ras_block_obj) | ||
| 2807 | { | ||
| 2808 | struct amdgpu_ras_block_list *ras_node; | ||
| 2809 | if (!adev || !ras_block_obj) | ||
| 2810 | return -EINVAL; | ||
| 2811 | |||
| 2812 | if (!amdgpu_ras_asic_supported(adev)) | ||
| 2813 | return 0; | ||
| 2814 | |||
| 2815 | ras_node = kzalloc(sizeof(*ras_node), GFP_KERNEL); | ||
| 2816 | if (!ras_node) | ||
| 2817 | return -ENOMEM; | ||
| 2818 | |||
| 2819 | INIT_LIST_HEAD(&ras_node->node); | ||
| 2820 | ras_node->ras_obj = ras_block_obj; | ||
| 2821 | list_add_tail(&ras_node->node, &adev->ras_list); | ||
| 2822 | |||
| 2823 | return 0; | ||
| 2824 | } | ||
| @@ -26,11 +26,11 @@ · 25 unmodified lines | |||
| 26 | 26 | ||
| 27 | #include <linux/debugfs.h> | 27 | #include <linux/debugfs.h> |
| 28 | #include <linux/list.h> | 28 | #include <linux/list.h> |
| 29 | #include "amdgpu.h" | ||
| 30 | #include "amdgpu_psp.h" | ||
| 31 | #include "ta_ras_if.h" | 29 | #include "ta_ras_if.h" |
| 32 | #include "amdgpu_ras_eeprom.h" | 30 | #include "amdgpu_ras_eeprom.h" |
| 33 | 31 | ||
| 32 | struct amdgpu_iv_entry; | ||
| 33 | |||
| 34 | #define AMDGPU_RAS_FLAG_INIT_BY_VBIOS (0x1 << 0) | 34 | #define AMDGPU_RAS_FLAG_INIT_BY_VBIOS (0x1 << 0) |
| 35 | 35 | ||
| 36 | enum amdgpu_ras_block { | 36 | enum amdgpu_ras_block { |
| @@ -484,6 +484,27 @@ struct ras_debug_if { · 447 unmodified lines | |||
| 484 | }; | 484 | }; |
| 485 | int op; | 485 | int op; |
| 486 | }; | 486 | }; |
| 487 | |||
| 488 | struct amdgpu_ras_block_object { | ||
| 489 | struct ras_common_if ras_comm; | ||
| 490 | |||
| 491 | int (*ras_block_match)(struct amdgpu_ras_block_object *block_obj, | ||
| 492 | enum amdgpu_ras_block block, uint32_t sub_block_index); | ||
| 493 | int (*ras_late_init)(struct amdgpu_device *adev, struct ras_common_if *ras_block); | ||
| 494 | void (*ras_fini)(struct amdgpu_device *adev); | ||
| 495 | ras_ih_cb ras_cb; | ||
| 496 | const struct amdgpu_ras_block_hw_ops *hw_ops; | ||
| 497 | }; | ||
| 498 | |||
| 499 | struct amdgpu_ras_block_hw_ops { | ||
| 500 | int (*ras_error_inject)(struct amdgpu_device *adev, void *inject_if); | ||
| 501 | void (*query_ras_error_count)(struct amdgpu_device *adev, void *ras_error_status); | ||
| 502 | void (*query_ras_error_status)(struct amdgpu_device *adev); | ||
| 503 | void (*query_ras_error_address)(struct amdgpu_device *adev, void *ras_error_status); | ||
| 504 | void (*reset_ras_error_count)(struct amdgpu_device *adev); | ||
| 505 | void (*reset_ras_error_status)(struct amdgpu_device *adev); | ||
| 506 | }; | ||
| 507 | |||
| 487 | /* work flow | 508 | /* work flow |
| 488 | * vbios | 509 | * vbios |
| 489 | * 1: ras feature enable (enabled by default) | 510 | * 1: ras feature enable (enabled by default) |
| @@ -498,19 +519,6 @@ struct ras_debug_if { · 8 unmodified lines | |||
| 498 | * 8: feature disable | 519 | * 8: feature disable |
| 499 | */ | 520 | */ |
| 500 | 521 | ||
| 501 | #define amdgpu_ras_get_context(adev) ((adev)->psp.ras_context.ras) | ||
| 502 | #define amdgpu_ras_set_context(adev, ras_con) ((adev)->psp.ras_context.ras = (ras_con)) | ||
| 503 | |||
| 504 | /* check if ras is supported on block, say, sdma, gfx */ | ||
| 505 | static inline int amdgpu_ras_is_supported(struct amdgpu_device *adev, | ||
| 506 | unsigned int block) | ||
| 507 | { | ||
| 508 | struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | ||
| 509 | |||
| 510 | if (block >= AMDGPU_RAS_BLOCK_COUNT) | ||
| 511 | return 0; | ||
| 512 | return ras && (adev->ras_enabled & (1 << block)); | ||
| 513 | } | ||
| 514 | 522 | ||
| 515 | int amdgpu_ras_recovery_init(struct amdgpu_device *adev); | 523 | int amdgpu_ras_recovery_init(struct amdgpu_device *adev); |
| 516 | 524 | ||
| @@ -527,15 +535,6 @@ int amdgpu_ras_add_bad_pages(struct amdgpu_device *adev, · 10 unmodified lines | |||
| 527 | 535 | ||
| 528 | int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev); | 536 | int amdgpu_ras_save_bad_pages(struct amdgpu_device *adev); |
| 529 | 537 | ||
| 530 | static inline int amdgpu_ras_reset_gpu(struct amdgpu_device *adev) | ||
| 531 | { | ||
| 532 | struct amdgpu_ras *ras = amdgpu_ras_get_context(adev); | ||
| 533 | |||
| 534 | if (atomic_cmpxchg(&ras->in_recovery, 0, 1) == 0) | ||
| 535 | schedule_work(&ras->recovery_work); | ||
| 536 | return 0; | ||
| 537 | } | ||
| 538 | |||
| 539 | static inline enum ta_ras_block | 538 | static inline enum ta_ras_block |
| 540 | amdgpu_ras_block_to_ta(enum amdgpu_ras_block block) { | 539 | amdgpu_ras_block_to_ta(enum amdgpu_ras_block block) { |
| 541 | switch (block) { | 540 | switch (block) { |
| @@ -596,15 +595,15 @@ amdgpu_ras_error_to_ta(enum amdgpu_ras_error_type error) { · 54 unmodified lines | |||
| 596 | 595 | ||
| 597 | /* called in ip_init and ip_fini */ | 596 | /* called in ip_init and ip_fini */ |
| 598 | int amdgpu_ras_init(struct amdgpu_device *adev); | 597 | int amdgpu_ras_init(struct amdgpu_device *adev); |
| 598 | int amdgpu_ras_late_init(struct amdgpu_device *adev); | ||
| 599 | int amdgpu_ras_fini(struct amdgpu_device *adev); | 599 | int amdgpu_ras_fini(struct amdgpu_device *adev); |
| 600 | int amdgpu_ras_pre_fini(struct amdgpu_device *adev); | 600 | int amdgpu_ras_pre_fini(struct amdgpu_device *adev); |
| 601 | int amdgpu_ras_late_init(struct amdgpu_device *adev, | 601 | |
| 602 | struct ras_common_if *ras_block, | 602 | int amdgpu_ras_block_late_init(struct amdgpu_device *adev, |
| 603 | struct ras_fs_if *fs_info, | 603 | struct ras_common_if *ras_block); |
| 604 | struct ras_ih_if *ih_info); | 604 | |
| 605 | void amdgpu_ras_late_fini(struct amdgpu_device *adev, | 605 | void amdgpu_ras_block_late_fini(struct amdgpu_device *adev, |
| 606 | struct ras_common_if *ras_block, | 606 | struct ras_common_if *ras_block); |
| 607 | struct ras_ih_if *ih_info); | ||
| 608 | 607 | ||
| 609 | int amdgpu_ras_feature_enable(struct amdgpu_device *adev, | 608 | int amdgpu_ras_feature_enable(struct amdgpu_device *adev, |
| 610 | struct ras_common_if *head, bool enable); | 609 | struct ras_common_if *head, bool enable); |
| @@ -613,7 +612,7 @@ int amdgpu_ras_feature_enable_on_boot(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 613 | struct ras_common_if *head, bool enable); | 612 | struct ras_common_if *head, bool enable); |
| 614 | 613 | ||
| 615 | int amdgpu_ras_sysfs_create(struct amdgpu_device *adev, | 614 | int amdgpu_ras_sysfs_create(struct amdgpu_device *adev, |
| 616 | struct ras_fs_if *head); | 615 | struct ras_common_if *head); |
| 617 | 616 | ||
| 618 | int amdgpu_ras_sysfs_remove(struct amdgpu_device *adev, | 617 | int amdgpu_ras_sysfs_remove(struct amdgpu_device *adev, |
| 619 | struct ras_common_if *head); | 618 | struct ras_common_if *head); |
| @@ -630,10 +629,10 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev, · 10 unmodified lines | |||
| 630 | struct ras_inject_if *info); | 629 | struct ras_inject_if *info); |
| 631 | 630 | ||
| 632 | int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev, | 631 | int amdgpu_ras_interrupt_add_handler(struct amdgpu_device *adev, |
| 633 | struct ras_ih_if *info); | 632 | struct ras_common_if *head); |
| 634 | 633 | ||
| 635 | int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, | 634 | int amdgpu_ras_interrupt_remove_handler(struct amdgpu_device *adev, |
| 636 | struct ras_ih_if *info); | 635 | struct ras_common_if *head); |
| 637 | 636 | ||
| 638 | int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, | 637 | int amdgpu_ras_interrupt_dispatch(struct amdgpu_device *adev, |
| 639 | struct ras_dispatch_if *info); | 638 | struct ras_dispatch_if *info); |
| @@ -667,4 +666,14 @@ const char *get_ras_block_str(struct ras_common_if *ras_block); · 27 unmodified lines | |||
| 667 | 666 | ||
| 668 | bool amdgpu_ras_is_poison_mode_supported(struct amdgpu_device *adev); | 667 | bool amdgpu_ras_is_poison_mode_supported(struct amdgpu_device *adev); |
| 669 | 668 | ||
| 669 | int amdgpu_ras_is_supported(struct amdgpu_device *adev, unsigned int block); | ||
| 670 | |||
| 671 | int amdgpu_ras_reset_gpu(struct amdgpu_device *adev); | ||
| 672 | |||
| 673 | struct amdgpu_ras* amdgpu_ras_get_context(struct amdgpu_device *adev); | ||
| 674 | |||
| 675 | int amdgpu_ras_set_context(struct amdgpu_device *adev, struct amdgpu_ras *ras_con); | ||
| 676 | |||
| 677 | int amdgpu_ras_register_ras_block(struct amdgpu_device *adev, | ||
| 678 | struct amdgpu_ras_block_object *ras_block_obj); | ||
| 670 | #endif | 679 | #endif |
| @@ -31,6 +31,8 @@ · 30 unmodified lines | |||
| 31 | #include <linux/debugfs.h> | 31 | #include <linux/debugfs.h> |
| 32 | #include <linux/uaccess.h> | 32 | #include <linux/uaccess.h> |
| 33 | 33 | ||
| 34 | #include "amdgpu_reset.h" | ||
| 35 | |||
| 34 | #define EEPROM_I2C_MADDR_VEGA20 0x0 | 36 | #define EEPROM_I2C_MADDR_VEGA20 0x0 |
| 35 | #define EEPROM_I2C_MADDR_ARCTURUS 0x40000 | 37 | #define EEPROM_I2C_MADDR_ARCTURUS 0x40000 |
| 36 | #define EEPROM_I2C_MADDR_ARCTURUS_D342 0x0 | 38 | #define EEPROM_I2C_MADDR_ARCTURUS_D342 0x0 |
| @@ -193,12 +195,12 @@ static int __write_table_header(struct amdgpu_ras_eeprom_control *control) · 156 unmodified lines | |||
| 193 | __encode_table_header_to_buf(&control->tbl_hdr, buf); | 195 | __encode_table_header_to_buf(&control->tbl_hdr, buf); |
| 194 | 196 | ||
| 195 | /* i2c may be unstable in gpu reset */ | 197 | /* i2c may be unstable in gpu reset */ |
| 196 | down_read(&adev->reset_sem); | 198 | down_read(&adev->reset_domain->sem); |
| 197 | res = amdgpu_eeprom_write(&adev->pm.smu_i2c, | 199 | res = amdgpu_eeprom_write(adev->pm.ras_eeprom_i2c_bus, |
| 198 | control->i2c_address + | 200 | control->i2c_address + |
| 199 | control->ras_header_offset, | 201 | control->ras_header_offset, |
| 200 | buf, RAS_TABLE_HEADER_SIZE); | 202 | buf, RAS_TABLE_HEADER_SIZE); |
| 201 | up_read(&adev->reset_sem); | 203 | up_read(&adev->reset_domain->sem); |
| 202 | 204 | ||
| 203 | if (res < 0) { | 205 | if (res < 0) { |
| 204 | DRM_ERROR("Failed to write EEPROM table header:%d", res); | 206 | DRM_ERROR("Failed to write EEPROM table header:%d", res); |
| @@ -263,6 +265,7 @@ static int amdgpu_ras_eeprom_correct_header_tag( · 58 unmodified lines | |||
| 263 | */ | 265 | */ |
| 264 | int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control) | 266 | int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control) |
| 265 | { | 267 | { |
| 268 | struct amdgpu_device *adev = to_amdgpu_device(control); | ||
| 266 | struct amdgpu_ras_eeprom_table_header *hdr = &control->tbl_hdr; | 269 | struct amdgpu_ras_eeprom_table_header *hdr = &control->tbl_hdr; |
| 267 | u8 csum; | 270 | u8 csum; |
| 268 | int res; | 271 | int res; |
| @@ -282,6 +285,8 @@ int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control) · 13 unmodified lines | |||
| 282 | control->ras_num_recs = 0; | 285 | control->ras_num_recs = 0; |
| 283 | control->ras_fri = 0; | 286 | control->ras_fri = 0; |
| 284 | 287 | ||
| 288 | amdgpu_dpm_send_hbm_bad_pages_num(adev, control->ras_num_recs); | ||
| 289 | |||
| 285 | amdgpu_ras_debugfs_set_ret_size(control); | 290 | amdgpu_ras_debugfs_set_ret_size(control); |
| 286 | 291 | ||
| 287 | mutex_unlock(&control->ras_tbl_mutex); | 292 | mutex_unlock(&control->ras_tbl_mutex); |
| @@ -387,13 +392,13 @@ static int __amdgpu_ras_eeprom_write(struct amdgpu_ras_eeprom_control *control, · 99 unmodified lines | |||
| 387 | int res; | 392 | int res; |
| 388 | 393 | ||
| 389 | /* i2c may be unstable in gpu reset */ | 394 | /* i2c may be unstable in gpu reset */ |
| 390 | down_read(&adev->reset_sem); | 395 | down_read(&adev->reset_domain->sem); |
| 391 | buf_size = num * RAS_TABLE_RECORD_SIZE; | 396 | buf_size = num * RAS_TABLE_RECORD_SIZE; |
| 392 | res = amdgpu_eeprom_write(&adev->pm.smu_i2c, | 397 | res = amdgpu_eeprom_write(adev->pm.ras_eeprom_i2c_bus, |
| 393 | control->i2c_address + | 398 | control->i2c_address + |
| 394 | RAS_INDEX_TO_OFFSET(control, fri), | 399 | RAS_INDEX_TO_OFFSET(control, fri), |
| 395 | buf, buf_size); | 400 | buf, buf_size); |
| 396 | up_read(&adev->reset_sem); | 401 | up_read(&adev->reset_domain->sem); |
| 397 | if (res < 0) { | 402 | if (res < 0) { |
| 398 | DRM_ERROR("Writing %d EEPROM table records error:%d", | 403 | DRM_ERROR("Writing %d EEPROM table records error:%d", |
| 399 | num, res); | 404 | num, res); |
| @@ -547,12 +552,12 @@ amdgpu_ras_eeprom_update_header(struct amdgpu_ras_eeprom_control *control) · 147 unmodified lines | |||
| 547 | goto Out; | 552 | goto Out; |
| 548 | } | 553 | } |
| 549 | 554 | ||
| 550 | down_read(&adev->reset_sem); | 555 | down_read(&adev->reset_domain->sem); |
| 551 | res = amdgpu_eeprom_read(&adev->pm.smu_i2c, | 556 | res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus, |
| 552 | control->i2c_address + | 557 | control->i2c_address + |
| 553 | control->ras_record_offset, | 558 | control->ras_record_offset, |
| 554 | buf, buf_size); | 559 | buf, buf_size); |
| 555 | up_read(&adev->reset_sem); | 560 | up_read(&adev->reset_domain->sem); |
| 556 | if (res < 0) { | 561 | if (res < 0) { |
| 557 | DRM_ERROR("EEPROM failed reading records:%d\n", | 562 | DRM_ERROR("EEPROM failed reading records:%d\n", |
| 558 | res); | 563 | res); |
| @@ -642,13 +647,13 @@ static int __amdgpu_ras_eeprom_read(struct amdgpu_ras_eeprom_control *control, · 83 unmodified lines | |||
| 642 | int res; | 647 | int res; |
| 643 | 648 | ||
| 644 | /* i2c may be unstable in gpu reset */ | 649 | /* i2c may be unstable in gpu reset */ |
| 645 | down_read(&adev->reset_sem); | 650 | down_read(&adev->reset_domain->sem); |
| 646 | buf_size = num * RAS_TABLE_RECORD_SIZE; | 651 | buf_size = num * RAS_TABLE_RECORD_SIZE; |
| 647 | res = amdgpu_eeprom_read(&adev->pm.smu_i2c, | 652 | res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus, |
| 648 | control->i2c_address + | 653 | control->i2c_address + |
| 649 | RAS_INDEX_TO_OFFSET(control, fri), | 654 | RAS_INDEX_TO_OFFSET(control, fri), |
| 650 | buf, buf_size); | 655 | buf, buf_size); |
| 651 | up_read(&adev->reset_sem); | 656 | up_read(&adev->reset_domain->sem); |
| 652 | if (res < 0) { | 657 | if (res < 0) { |
| 653 | DRM_ERROR("Reading %d EEPROM table records error:%d", | 658 | DRM_ERROR("Reading %d EEPROM table records error:%d", |
| 654 | num, res); | 659 | num, res); |
| @@ -1009,7 +1014,7 @@ static int __verify_ras_table_checksum(struct amdgpu_ras_eeprom_control *control · 354 unmodified lines | |||
| 1009 | return -ENOMEM; | 1014 | return -ENOMEM; |
| 1010 | } | 1015 | } |
| 1011 | 1016 | ||
| 1012 | res = amdgpu_eeprom_read(&adev->pm.smu_i2c, | 1017 | res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus, |
| 1013 | control->i2c_address + | 1018 | control->i2c_address + |
| 1014 | control->ras_header_offset, | 1019 | control->ras_header_offset, |
| 1015 | buf, buf_size); | 1020 | buf, buf_size); |
| @@ -1045,7 +1050,7 @@ int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control, · 29 unmodified lines | |||
| 1045 | return 0; | 1050 | return 0; |
| 1046 | 1051 | ||
| 1047 | /* Verify i2c adapter is initialized */ | 1052 | /* Verify i2c adapter is initialized */ |
| 1048 | if (!adev->pm.smu_i2c.algo) | 1053 | if (!adev->pm.ras_eeprom_i2c_bus || !adev->pm.ras_eeprom_i2c_bus->algo) |
| 1049 | return -ENOENT; | 1054 | return -ENOENT; |
| 1050 | 1055 | ||
| 1051 | if (!__get_eeprom_i2c_addr(adev, control)) | 1056 | if (!__get_eeprom_i2c_addr(adev, control)) |
| @@ -1057,7 +1062,7 @@ int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control, · 5 unmodified lines | |||
| 1057 | mutex_init(&control->ras_tbl_mutex); | 1062 | mutex_init(&control->ras_tbl_mutex); |
| 1058 | 1063 | ||
| 1059 | /* Read the table header from EEPROM address */ | 1064 | /* Read the table header from EEPROM address */ |
| 1060 | res = amdgpu_eeprom_read(&adev->pm.smu_i2c, | 1065 | res = amdgpu_eeprom_read(adev->pm.ras_eeprom_i2c_bus, |
| 1061 | control->i2c_address + control->ras_header_offset, | 1066 | control->i2c_address + control->ras_header_offset, |
| 1062 | buf, RAS_TABLE_HEADER_SIZE); | 1067 | buf, RAS_TABLE_HEADER_SIZE); |
| 1063 | if (res < RAS_TABLE_HEADER_SIZE) { | 1068 | if (res < RAS_TABLE_HEADER_SIZE) { |
| @@ -96,3 +96,59 @@ int amdgpu_reset_perform_reset(struct amdgpu_device *adev, · 95 unmodified lines | |||
| 96 | return reset_handler->restore_hwcontext(adev->reset_cntl, | 96 | return reset_handler->restore_hwcontext(adev->reset_cntl, |
| 97 | reset_context); | 97 | reset_context); |
| 98 | } | 98 | } |
| 99 | |||
| 100 | |||
| 101 | void amdgpu_reset_destroy_reset_domain(struct kref *ref) | ||
| 102 | { | ||
| 103 | struct amdgpu_reset_domain *reset_domain = container_of(ref, | ||
| 104 | struct amdgpu_reset_domain, | ||
| 105 | refcount); | ||
| 106 | if (reset_domain->wq) | ||
| 107 | destroy_workqueue(reset_domain->wq); | ||
| 108 | |||
| 109 | kvfree(reset_domain); | ||
| 110 | } | ||
| 111 | |||
| 112 | struct amdgpu_reset_domain *amdgpu_reset_create_reset_domain(enum amdgpu_reset_domain_type type, | ||
| 113 | char *wq_name) | ||
| 114 | { | ||
| 115 | struct amdgpu_reset_domain *reset_domain; | ||
| 116 | |||
| 117 | reset_domain = kvzalloc(sizeof(struct amdgpu_reset_domain), GFP_KERNEL); | ||
| 118 | if (!reset_domain) { | ||
| 119 | DRM_ERROR("Failed to allocate amdgpu_reset_domain!"); | ||
| 120 | return NULL; | ||
| 121 | } | ||
| 122 | |||
| 123 | reset_domain->type = type; | ||
| 124 | kref_init(&reset_domain->refcount); | ||
| 125 | |||
| 126 | reset_domain->wq = create_singlethread_workqueue(wq_name); | ||
| 127 | if (!reset_domain->wq) { | ||
| 128 | DRM_ERROR("Failed to allocate wq for amdgpu_reset_domain!"); | ||
| 129 | amdgpu_reset_put_reset_domain(reset_domain); | ||
| 130 | return NULL; | ||
| 131 | |||
| 132 | } | ||
| 133 | |||
| 134 | atomic_set(&reset_domain->in_gpu_reset, 0); | ||
| 135 | init_rwsem(&reset_domain->sem); | ||
| 136 | |||
| 137 | return reset_domain; | ||
| 138 | } | ||
| 139 | |||
| 140 | void amdgpu_device_lock_reset_domain(struct amdgpu_reset_domain *reset_domain) | ||
| 141 | { | ||
| 142 | atomic_set(&reset_domain->in_gpu_reset, 1); | ||
| 143 | down_write(&reset_domain->sem); | ||
| 144 | } | ||
| 145 | |||
| 146 | |||
| 147 | void amdgpu_device_unlock_reset_domain(struct amdgpu_reset_domain *reset_domain) | ||
| 148 | { | ||
| 149 | atomic_set(&reset_domain->in_gpu_reset, 0); | ||
| 150 | up_write(&reset_domain->sem); | ||
| 151 | } | ||
| 152 | |||
| 153 | |||
| 154 | |||
| @@ -70,6 +70,21 @@ struct amdgpu_reset_control { · 69 unmodified lines | |||
| 70 | void (*async_reset)(struct work_struct *work); | 70 | void (*async_reset)(struct work_struct *work); |
| 71 | }; | 71 | }; |
| 72 | 72 | ||
| 73 | |||
| 74 | enum amdgpu_reset_domain_type { | ||
| 75 | SINGLE_DEVICE, | ||
| 76 | XGMI_HIVE | ||
| 77 | }; | ||
| 78 | |||
| 79 | struct amdgpu_reset_domain { | ||
| 80 | struct kref refcount; | ||
| 81 | struct workqueue_struct *wq; | ||
| 82 | enum amdgpu_reset_domain_type type; | ||
| 83 | struct rw_semaphore sem; | ||
| 84 | atomic_t in_gpu_reset; | ||
| 85 | }; | ||
| 86 | |||
| 87 | |||
| 73 | int amdgpu_reset_init(struct amdgpu_device *adev); | 88 | int amdgpu_reset_init(struct amdgpu_device *adev); |
| 74 | int amdgpu_reset_fini(struct amdgpu_device *adev); | 89 | int amdgpu_reset_fini(struct amdgpu_device *adev); |
| 75 | 90 | ||
| @@ -82,4 +97,29 @@ int amdgpu_reset_perform_reset(struct amdgpu_device *adev, · 6 unmodified lines | |||
| 82 | int amdgpu_reset_add_handler(struct amdgpu_reset_control *reset_ctl, | 97 | int amdgpu_reset_add_handler(struct amdgpu_reset_control *reset_ctl, |
| 83 | struct amdgpu_reset_handler *handler); | 98 | struct amdgpu_reset_handler *handler); |
| 84 | 99 | ||
| 100 | struct amdgpu_reset_domain *amdgpu_reset_create_reset_domain(enum amdgpu_reset_domain_type type, | ||
| 101 | char *wq_name); | ||
| 102 | |||
| 103 | void amdgpu_reset_destroy_reset_domain(struct kref *ref); | ||
| 104 | |||
| 105 | static inline bool amdgpu_reset_get_reset_domain(struct amdgpu_reset_domain *domain) | ||
| 106 | { | ||
| 107 | return kref_get_unless_zero(&domain->refcount) != 0; | ||
| 108 | } | ||
| 109 | |||
| 110 | static inline void amdgpu_reset_put_reset_domain(struct amdgpu_reset_domain *domain) | ||
| 111 | { | ||
| 112 | kref_put(&domain->refcount, amdgpu_reset_destroy_reset_domain); | ||
| 113 | } | ||
| 114 | |||
| 115 | static inline bool amdgpu_reset_domain_schedule(struct amdgpu_reset_domain *domain, | ||
| 116 | struct work_struct *work) | ||
| 117 | { | ||
| 118 | return queue_work(domain->wq, work); | ||
| 119 | } | ||
| 120 | |||
| 121 | void amdgpu_device_lock_reset_domain(struct amdgpu_reset_domain *reset_domain); | ||
| 122 | |||
| 123 | void amdgpu_device_unlock_reset_domain(struct amdgpu_reset_domain *reset_domain); | ||
| 124 | |||
| 85 | #endif | 125 | #endif |
| @@ -191,8 +191,9 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, · 190 unmodified lines | |||
| 191 | ring->adev = adev; | 191 | ring->adev = adev; |
| 192 | ring->idx = adev->num_rings++; | 192 | ring->idx = adev->num_rings++; |
| 193 | adev->rings[ring->idx] = ring; | 193 | adev->rings[ring->idx] = ring; |
| 194 | r = amdgpu_fence_driver_init_ring(ring, sched_hw_submission, | 194 | ring->num_hw_submission = sched_hw_submission; |
| 195 | sched_score); | 195 | ring->sched_score = sched_score; |
| 196 | r = amdgpu_fence_driver_init_ring(ring); | ||
| 196 | if (r) | 197 | if (r) |
| 197 | return r; | 198 | return r; |
| 198 | } | 199 | } |
| @@ -114,9 +114,7 @@ struct amdgpu_fence_driver { · 113 unmodified lines | |||
| 114 | void amdgpu_fence_driver_clear_job_fences(struct amdgpu_ring *ring); | 114 | void amdgpu_fence_driver_clear_job_fences(struct amdgpu_ring *ring); |
| 115 | void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring); | 115 | void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring); |
| 116 | 116 | ||
| 117 | int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, | 117 | int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring); |
| 118 | unsigned num_hw_submission, | ||
| 119 | atomic_t *sched_score); | ||
| 120 | int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring, | 118 | int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring, |
| 121 | struct amdgpu_irq_src *irq_src, | 119 | struct amdgpu_irq_src *irq_src, |
| 122 | unsigned irq_type); | 120 | unsigned irq_type); |
| @@ -251,6 +249,8 @@ struct amdgpu_ring { · 128 unmodified lines | |||
| 251 | bool has_compute_vm_bug; | 249 | bool has_compute_vm_bug; |
| 252 | bool no_scheduler; | 250 | bool no_scheduler; |
| 253 | int hw_prio; | 251 | int hw_prio; |
| 252 | unsigned num_hw_submission; | ||
| 253 | atomic_t *sched_score; | ||
| 254 | }; | 254 | }; |
| 255 | 255 | ||
| 256 | #define amdgpu_ring_parse_cs(r, p, ib) ((r)->funcs->parse_cs((p), (ib))) | 256 | #define amdgpu_ring_parse_cs(r, p, ib) ((r)->funcs->parse_cs((p), (ib))) |
| @@ -127,11 +127,19 @@ struct amdgpu_rlc_funcs { · 126 unmodified lines | |||
| 127 | void (*reset)(struct amdgpu_device *adev); | 127 | void (*reset)(struct amdgpu_device *adev); |
| 128 | void (*start)(struct amdgpu_device *adev); | 128 | void (*start)(struct amdgpu_device *adev); |
| 129 | void (*update_spm_vmid)(struct amdgpu_device *adev, unsigned vmid); | 129 | void (*update_spm_vmid)(struct amdgpu_device *adev, unsigned vmid); |
| 130 | void (*sriov_wreg)(struct amdgpu_device *adev, u32 offset, u32 v, u32 acc_flags, u32 hwip); | ||
| 131 | u32 (*sriov_rreg)(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip); | ||
| 132 | bool (*is_rlcg_access_range)(struct amdgpu_device *adev, uint32_t reg); | 130 | bool (*is_rlcg_access_range)(struct amdgpu_device *adev, uint32_t reg); |
| 133 | }; | 131 | }; |
| 134 | 132 | ||
| 133 | struct amdgpu_rlcg_reg_access_ctrl { | ||
| 134 | uint32_t scratch_reg0; | ||
| 135 | uint32_t scratch_reg1; | ||
| 136 | uint32_t scratch_reg2; | ||
| 137 | uint32_t scratch_reg3; | ||
| 138 | uint32_t grbm_cntl; | ||
| 139 | uint32_t grbm_idx; | ||
| 140 | uint32_t spare_int; | ||
| 141 | }; | ||
| 142 | |||
| 135 | struct amdgpu_rlc { | 143 | struct amdgpu_rlc { |
| 136 | /* for power gating */ | 144 | /* for power gating */ |
| 137 | struct amdgpu_bo *save_restore_obj; | 145 | struct amdgpu_bo *save_restore_obj; |
| @@ -191,6 +199,10 @@ struct amdgpu_rlc { · 53 unmodified lines | |||
| 191 | struct amdgpu_bo *rlc_toc_bo; | 199 | struct amdgpu_bo *rlc_toc_bo; |
| 192 | uint64_t rlc_toc_gpu_addr; | 200 | uint64_t rlc_toc_gpu_addr; |
| 193 | void *rlc_toc_buf; | 201 | void *rlc_toc_buf; |
| 202 | |||
| 203 | bool rlcg_reg_access_supported; | ||
| 204 | /* registers for rlcg indirect reg access */ | ||
| 205 | struct amdgpu_rlcg_reg_access_ctrl reg_access_ctrl; | ||
| 194 | }; | 206 | }; |
| 195 | 207 | ||
| 196 | void amdgpu_gfx_rlc_enter_safe_mode(struct amdgpu_device *adev); | 208 | void amdgpu_gfx_rlc_enter_safe_mode(struct amdgpu_device *adev); |
| @@ -87,71 +87,35 @@ uint64_t amdgpu_sdma_get_csa_mc_addr(struct amdgpu_ring *ring, · 86 unmodified lines | |||
| 87 | } | 87 | } |
| 88 | 88 | ||
| 89 | int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev, | 89 | int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev, |
| 90 | void *ras_ih_info) | 90 | struct ras_common_if *ras_block) |
| 91 | { | 91 | { |
| 92 | int r, i; | 92 | int r, i; |
| 93 | struct ras_ih_if *ih_info = (struct ras_ih_if *)ras_ih_info; | ||
| 94 | struct ras_fs_if fs_info = { | ||
| 95 | .sysfs_name = "sdma_err_count", | ||
| 96 | }; | ||
| 97 | |||
| 98 | if (!ih_info) | ||
| 99 | return -EINVAL; | ||
| 100 | 93 | ||
| 101 | if (!adev->sdma.ras_if) { | 94 | r = amdgpu_ras_block_late_init(adev, ras_block); |
| 102 | adev->sdma.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | ||
| 103 | if (!adev->sdma.ras_if) | ||
| 104 | return -ENOMEM; | ||
| 105 | adev->sdma.ras_if->block = AMDGPU_RAS_BLOCK__SDMA; | ||
| 106 | adev->sdma.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 107 | adev->sdma.ras_if->sub_block_index = 0; | ||
| 108 | } | ||
| 109 | fs_info.head = ih_info->head = *adev->sdma.ras_if; | ||
| 110 | |||
| 111 | r = amdgpu_ras_late_init(adev, adev->sdma.ras_if, | ||
| 112 | &fs_info, ih_info); | ||
| 113 | if (r) | 95 | if (r) |
| 114 | goto free; | 96 | return r; |
| 115 | 97 | ||
| 116 | if (amdgpu_ras_is_supported(adev, adev->sdma.ras_if->block)) { | 98 | if (amdgpu_ras_is_supported(adev, ras_block->block)) { |
| 117 | for (i = 0; i < adev->sdma.num_instances; i++) { | 99 | for (i = 0; i < adev->sdma.num_instances; i++) { |
| 118 | r = amdgpu_irq_get(adev, &adev->sdma.ecc_irq, | 100 | r = amdgpu_irq_get(adev, &adev->sdma.ecc_irq, |
| 119 | AMDGPU_SDMA_IRQ_INSTANCE0 + i); | 101 | AMDGPU_SDMA_IRQ_INSTANCE0 + i); |
| 120 | if (r) | 102 | if (r) |
| 121 | goto late_fini; | 103 | goto late_fini; |
| 122 | } | 104 | } |
| 123 | } else { | ||
| 124 | r = 0; | ||
| 125 | goto free; | ||
| 126 | } | 105 | } |
| 127 | 106 | ||
| 128 | return 0; | 107 | return 0; |
| 129 | 108 | ||
| 130 | late_fini: | 109 | late_fini: |
| 131 | amdgpu_ras_late_fini(adev, adev->sdma.ras_if, ih_info); | 110 | amdgpu_ras_block_late_fini(adev, ras_block); |
| 132 | free: | ||
| 133 | kfree(adev->sdma.ras_if); | ||
| 134 | adev->sdma.ras_if = NULL; | ||
| 135 | return r; | 111 | return r; |
| 136 | } | 112 | } |
| 137 | 113 | ||
| 138 | void amdgpu_sdma_ras_fini(struct amdgpu_device *adev) | 114 | void amdgpu_sdma_ras_fini(struct amdgpu_device *adev) |
| 139 | { | 115 | { |
| 140 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__SDMA) && | 116 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__SDMA) && |
| 141 | adev->sdma.ras_if) { | 117 | adev->sdma.ras_if) |
| 142 | struct ras_common_if *ras_if = adev->sdma.ras_if; | 118 | amdgpu_ras_block_late_fini(adev, adev->sdma.ras_if); |
| 143 | struct ras_ih_if ih_info = { | ||
| 144 | .head = *ras_if, | ||
| 145 | /* the cb member will not be used by | ||
| 146 | * amdgpu_ras_interrupt_remove_handler, init it only | ||
| 147 | * to cheat the check in ras_late_fini | ||
| 148 | */ | ||
| 149 | .cb = amdgpu_sdma_process_ras_data_cb, | ||
| 150 | }; | ||
| 151 | |||
| 152 | amdgpu_ras_late_fini(adev, ras_if, &ih_info); | ||
| 153 | kfree(ras_if); | ||
| 154 | } | ||
| 155 | } | 119 | } |
| 156 | 120 | ||
| 157 | int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev, | 121 | int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev, |
| @@ -23,6 +23,7 @@ · 22 unmodified lines | |||
| 23 | 23 | ||
| 24 | #ifndef __AMDGPU_SDMA_H__ | 24 | #ifndef __AMDGPU_SDMA_H__ |
| 25 | #define __AMDGPU_SDMA_H__ | 25 | #define __AMDGPU_SDMA_H__ |
| 26 | #include "amdgpu_ras.h" | ||
| 26 | 27 | ||
| 27 | /* max number of IP instances */ | 28 | /* max number of IP instances */ |
| 28 | #define AMDGPU_MAX_SDMA_INSTANCES 8 | 29 | #define AMDGPU_MAX_SDMA_INSTANCES 8 |
| @@ -50,13 +51,8 @@ struct amdgpu_sdma_instance { · 21 unmodified lines | |||
| 50 | bool burst_nop; | 51 | bool burst_nop; |
| 51 | }; | 52 | }; |
| 52 | 53 | ||
| 53 | struct amdgpu_sdma_ras_funcs { | 54 | struct amdgpu_sdma_ras { |
| 54 | int (*ras_late_init)(struct amdgpu_device *adev, | 55 | struct amdgpu_ras_block_object ras_block; |
| 55 | void *ras_ih_info); | ||
| 56 | void (*ras_fini)(struct amdgpu_device *adev); | ||
| 57 | int (*query_ras_error_count)(struct amdgpu_device *adev, | ||
| 58 | uint32_t instance, void *ras_error_status); | ||
| 59 | void (*reset_ras_error_count)(struct amdgpu_device *adev); | ||
| 60 | }; | 56 | }; |
| 61 | 57 | ||
| 62 | struct amdgpu_sdma { | 58 | struct amdgpu_sdma { |
| @@ -73,7 +69,7 @@ struct amdgpu_sdma { · 10 unmodified lines | |||
| 73 | uint32_t srbm_soft_reset; | 69 | uint32_t srbm_soft_reset; |
| 74 | bool has_page_queue; | 70 | bool has_page_queue; |
| 75 | struct ras_common_if *ras_if; | 71 | struct ras_common_if *ras_if; |
| 76 | const struct amdgpu_sdma_ras_funcs *funcs; | 72 | struct amdgpu_sdma_ras *ras; |
| 77 | }; | 73 | }; |
| 78 | 74 | ||
| 79 | /* | 75 | /* |
| @@ -121,7 +117,7 @@ amdgpu_sdma_get_instance_from_ring(struct amdgpu_ring *ring); · 41 unmodified lines | |||
| 121 | int amdgpu_sdma_get_index_from_ring(struct amdgpu_ring *ring, uint32_t *index); | 117 | int amdgpu_sdma_get_index_from_ring(struct amdgpu_ring *ring, uint32_t *index); |
| 122 | uint64_t amdgpu_sdma_get_csa_mc_addr(struct amdgpu_ring *ring, unsigned vmid); | 118 | uint64_t amdgpu_sdma_get_csa_mc_addr(struct amdgpu_ring *ring, unsigned vmid); |
| 123 | int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev, | 119 | int amdgpu_sdma_ras_late_init(struct amdgpu_device *adev, |
| 124 | void *ras_ih_info); | 120 | struct ras_common_if *ras_block); |
| 125 | void amdgpu_sdma_ras_fini(struct amdgpu_device *adev); | 121 | void amdgpu_sdma_ras_fini(struct amdgpu_device *adev); |
| 126 | int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev, | 122 | int amdgpu_sdma_process_ras_data_cb(struct amdgpu_device *adev, |
| 127 | void *err_data, | 123 | void *err_data, |
| @@ -261,10 +261,9 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct amdgpu_sync *sync, · 260 unmodified lines | |||
| 261 | 261 | ||
| 262 | dma_resv_for_each_fence(&cursor, resv, true, f) { | 262 | dma_resv_for_each_fence(&cursor, resv, true, f) { |
| 263 | dma_fence_chain_for_each(f, f) { | 263 | dma_fence_chain_for_each(f, f) { |
| 264 | struct dma_fence_chain *chain = to_dma_fence_chain(f); | 264 | struct dma_fence *tmp = dma_fence_chain_contained(f); |
| 265 | 265 | ||
| 266 | if (amdgpu_sync_test_fence(adev, mode, owner, chain ? | 266 | if (amdgpu_sync_test_fence(adev, mode, owner, tmp)) { |
| 267 | chain->fence : f)) { | ||
| 268 | r = amdgpu_sync_fence(sync, f); | 267 | r = amdgpu_sync_fence(sync, f); |
| 269 | dma_fence_put(f); | 268 | dma_fence_put(f); |
| 270 | if (r) | 269 | if (r) |
| @@ -1,250 +0,0 @@ | |||
| 1 | // SPDX-License-Identifier: GPL-2.0 OR MIT | ||
| 2 | /* | ||
| 3 | * Copyright 2009 VMware, Inc. | ||
| 4 | * | ||
| 5 | * Permission is hereby granted, free of charge, to any person obtaining a | ||
| 6 | * copy of this software and associated documentation files (the "Software"), | ||
| 7 | * to deal in the Software without restriction, including without limitation | ||
| 8 | * the rights to use, copy, modify, merge, publish, distribute, sublicense, | ||
| 9 | * and/or sell copies of the Software, and to permit persons to whom the | ||
| 10 | * Software is furnished to do so, subject to the following conditions: | ||
| 11 | * | ||
| 12 | * The above copyright notice and this permission notice shall be included in | ||
| 13 | * all copies or substantial portions of the Software. | ||
| 14 | * | ||
| 15 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
| 16 | * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
| 17 | * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL | ||
| 18 | * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR | ||
| 19 | * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, | ||
| 20 | * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR | ||
| 21 | * OTHER DEALINGS IN THE SOFTWARE. | ||
| 22 | * | ||
| 23 | * Authors: Michel Dänzer | ||
| 24 | */ | ||
| 25 | |||
| 26 | #include <drm/amdgpu_drm.h> | ||
| 27 | #include "amdgpu.h" | ||
| 28 | #include "amdgpu_uvd.h" | ||
| 29 | #include "amdgpu_vce.h" | ||
| 30 | |||
| 31 | /* Test BO GTT->VRAM and VRAM->GTT GPU copies across the whole GTT aperture */ | ||
| 32 | static void amdgpu_do_test_moves(struct amdgpu_device *adev) | ||
| 33 | { | ||
| 34 | struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; | ||
| 35 | struct amdgpu_bo *vram_obj = NULL; | ||
| 36 | struct amdgpu_bo **gtt_obj = NULL; | ||
| 37 | struct amdgpu_bo_param bp; | ||
| 38 | uint64_t gart_addr, vram_addr; | ||
| 39 | unsigned n, size; | ||
| 40 | int i, r; | ||
| 41 | |||
| 42 | size = 1024 * 1024; | ||
| 43 | |||
| 44 | /* Number of tests = | ||
| 45 | * (Total GTT - gart_pin_size - (2 transfer windows for buffer moves)) / test size | ||
| 46 | */ | ||
| 47 | n = adev->gmc.gart_size - atomic64_read(&adev->gart_pin_size); | ||
| 48 | n -= AMDGPU_GTT_MAX_TRANSFER_SIZE * AMDGPU_GTT_NUM_TRANSFER_WINDOWS * | ||
| 49 | AMDGPU_GPU_PAGE_SIZE; | ||
| 50 | n /= size; | ||
| 51 | |||
| 52 | gtt_obj = kcalloc(n, sizeof(*gtt_obj), GFP_KERNEL); | ||
| 53 | if (!gtt_obj) { | ||
| 54 | DRM_ERROR("Failed to allocate %d pointers\n", n); | ||
| 55 | r = 1; | ||
| 56 | goto out_cleanup; | ||
| 57 | } | ||
| 58 | memset(&bp, 0, sizeof(bp)); | ||
| 59 | bp.size = size; | ||
| 60 | bp.byte_align = PAGE_SIZE; | ||
| 61 | bp.domain = AMDGPU_GEM_DOMAIN_VRAM; | ||
| 62 | bp.flags = 0; | ||
| 63 | bp.type = ttm_bo_type_kernel; | ||
| 64 | bp.resv = NULL; | ||
| 65 | bp.bo_ptr_size = sizeof(struct amdgpu_bo); | ||
| 66 | |||
| 67 | r = amdgpu_bo_create(adev, &bp, &vram_obj); | ||
| 68 | if (r) { | ||
| 69 | DRM_ERROR("Failed to create VRAM object\n"); | ||
| 70 | goto out_cleanup; | ||
| 71 | } | ||
| 72 | r = amdgpu_bo_reserve(vram_obj, false); | ||
| 73 | if (unlikely(r != 0)) | ||
| 74 | goto out_unref; | ||
| 75 | r = amdgpu_bo_pin(vram_obj, AMDGPU_GEM_DOMAIN_VRAM); | ||
| 76 | if (r) { | ||
| 77 | DRM_ERROR("Failed to pin VRAM object\n"); | ||
| 78 | goto out_unres; | ||
| 79 | } | ||
| 80 | vram_addr = amdgpu_bo_gpu_offset(vram_obj); | ||
| 81 | for (i = 0; i < n; i++) { | ||
| 82 | void *gtt_map, *vram_map; | ||
| 83 | void **gart_start, **gart_end; | ||
| 84 | void **vram_start, **vram_end; | ||
| 85 | struct dma_fence *fence = NULL; | ||
| 86 | |||
| 87 | bp.domain = AMDGPU_GEM_DOMAIN_GTT; | ||
| 88 | r = amdgpu_bo_create(adev, &bp, gtt_obj + i); | ||
| 89 | if (r) { | ||
| 90 | DRM_ERROR("Failed to create GTT object %d\n", i); | ||
| 91 | goto out_lclean; | ||
| 92 | } | ||
| 93 | |||
| 94 | r = amdgpu_bo_reserve(gtt_obj[i], false); | ||
| 95 | if (unlikely(r != 0)) | ||
| 96 | goto out_lclean_unref; | ||
| 97 | r = amdgpu_bo_pin(gtt_obj[i], AMDGPU_GEM_DOMAIN_GTT); | ||
| 98 | if (r) { | ||
| 99 | DRM_ERROR("Failed to pin GTT object %d\n", i); | ||
| 100 | goto out_lclean_unres; | ||
| 101 | } | ||
| 102 | r = amdgpu_ttm_alloc_gart(>t_obj[i]->tbo); | ||
| 103 | if (r) { | ||
| 104 | DRM_ERROR("%p bind failed\n", gtt_obj[i]); | ||
| 105 | goto out_lclean_unpin; | ||
| 106 | } | ||
| 107 | gart_addr = amdgpu_bo_gpu_offset(gtt_obj[i]); | ||
| 108 | |||
| 109 | r = amdgpu_bo_kmap(gtt_obj[i], >t_map); | ||
| 110 | if (r) { | ||
| 111 | DRM_ERROR("Failed to map GTT object %d\n", i); | ||
| 112 | goto out_lclean_unpin; | ||
| 113 | } | ||
| 114 | |||
| 115 | for (gart_start = gtt_map, gart_end = gtt_map + size; | ||
| 116 | gart_start < gart_end; | ||
| 117 | gart_start++) | ||
| 118 | *gart_start = gart_start; | ||
| 119 | |||
| 120 | amdgpu_bo_kunmap(gtt_obj[i]); | ||
| 121 | |||
| 122 | r = amdgpu_copy_buffer(ring, gart_addr, vram_addr, | ||
| 123 | size, NULL, &fence, false, false, false); | ||
| 124 | |||
| 125 | if (r) { | ||
| 126 | DRM_ERROR("Failed GTT->VRAM copy %d\n", i); | ||
| 127 | goto out_lclean_unpin; | ||
| 128 | } | ||
| 129 | |||
| 130 | r = dma_fence_wait(fence, false); | ||
| 131 | if (r) { | ||
| 132 | DRM_ERROR("Failed to wait for GTT->VRAM fence %d\n", i); | ||
| 133 | goto out_lclean_unpin; | ||
| 134 | } | ||
| 135 | |||
| 136 | dma_fence_put(fence); | ||
| 137 | fence = NULL; | ||
| 138 | |||
| 139 | r = amdgpu_bo_kmap(vram_obj, &vram_map); | ||
| 140 | if (r) { | ||
| 141 | DRM_ERROR("Failed to map VRAM object after copy %d\n", i); | ||
| 142 | goto out_lclean_unpin; | ||
| 143 | } | ||
| 144 | |||
| 145 | for (gart_start = gtt_map, gart_end = gtt_map + size, | ||
| 146 | vram_start = vram_map, vram_end = vram_map + size; | ||
| 147 | vram_start < vram_end; | ||
| 148 | gart_start++, vram_start++) { | ||
| 149 | if (*vram_start != gart_start) { | ||
| 150 | DRM_ERROR("Incorrect GTT->VRAM copy %d: Got 0x%p, " | ||
| 151 | "expected 0x%p (GTT/VRAM offset " | ||
| 152 | "0x%16llx/0x%16llx)\n", | ||
| 153 | i, *vram_start, gart_start, | ||
| 154 | (unsigned long long) | ||
| 155 | (gart_addr - adev->gmc.gart_start + | ||
| 156 | (void *)gart_start - gtt_map), | ||
| 157 | (unsigned long long) | ||
| 158 | (vram_addr - adev->gmc.vram_start + | ||
| 159 | (void *)gart_start - gtt_map)); | ||
| 160 | amdgpu_bo_kunmap(vram_obj); | ||
| 161 | goto out_lclean_unpin; | ||
| 162 | } | ||
| 163 | *vram_start = vram_start; | ||
| 164 | } | ||
| 165 | |||
| 166 | amdgpu_bo_kunmap(vram_obj); | ||
| 167 | |||
| 168 | r = amdgpu_copy_buffer(ring, vram_addr, gart_addr, | ||
| 169 | size, NULL, &fence, false, false, false); | ||
| 170 | |||
| 171 | if (r) { | ||
| 172 | DRM_ERROR("Failed VRAM->GTT copy %d\n", i); | ||
| 173 | goto out_lclean_unpin; | ||
| 174 | } | ||
| 175 | |||
| 176 | r = dma_fence_wait(fence, false); | ||
| 177 | if (r) { | ||
| 178 | DRM_ERROR("Failed to wait for VRAM->GTT fence %d\n", i); | ||
| 179 | goto out_lclean_unpin; | ||
| 180 | } | ||
| 181 | |||
| 182 | dma_fence_put(fence); | ||
| 183 | fence = NULL; | ||
| 184 | |||
| 185 | r = amdgpu_bo_kmap(gtt_obj[i], >t_map); | ||
| 186 | if (r) { | ||
| 187 | DRM_ERROR("Failed to map GTT object after copy %d\n", i); | ||
| 188 | goto out_lclean_unpin; | ||
| 189 | } | ||
| 190 | |||
| 191 | for (gart_start = gtt_map, gart_end = gtt_map + size, | ||
| 192 | vram_start = vram_map, vram_end = vram_map + size; | ||
| 193 | gart_start < gart_end; | ||
| 194 | gart_start++, vram_start++) { | ||
| 195 | if (*gart_start != vram_start) { | ||
| 196 | DRM_ERROR("Incorrect VRAM->GTT copy %d: Got 0x%p, " | ||
| 197 | "expected 0x%p (VRAM/GTT offset " | ||
| 198 | "0x%16llx/0x%16llx)\n", | ||
| 199 | i, *gart_start, vram_start, | ||
| 200 | (unsigned long long) | ||
| 201 | (vram_addr - adev->gmc.vram_start + | ||
| 202 | (void *)vram_start - vram_map), | ||
| 203 | (unsigned long long) | ||
| 204 | (gart_addr - adev->gmc.gart_start + | ||
| 205 | (void *)vram_start - vram_map)); | ||
| 206 | amdgpu_bo_kunmap(gtt_obj[i]); | ||
| 207 | goto out_lclean_unpin; | ||
| 208 | } | ||
| 209 | } | ||
| 210 | |||
| 211 | amdgpu_bo_kunmap(gtt_obj[i]); | ||
| 212 | |||
| 213 | DRM_INFO("Tested GTT->VRAM and VRAM->GTT copy for GTT offset 0x%llx\n", | ||
| 214 | gart_addr - adev->gmc.gart_start); | ||
| 215 | continue; | ||
| 216 | |||
| 217 | out_lclean_unpin: | ||
| 218 | amdgpu_bo_unpin(gtt_obj[i]); | ||
| 219 | out_lclean_unres: | ||
| 220 | amdgpu_bo_unreserve(gtt_obj[i]); | ||
| 221 | out_lclean_unref: | ||
| 222 | amdgpu_bo_unref(>t_obj[i]); | ||
| 223 | out_lclean: | ||
| 224 | for (--i; i >= 0; --i) { | ||
| 225 | amdgpu_bo_unpin(gtt_obj[i]); | ||
| 226 | amdgpu_bo_unreserve(gtt_obj[i]); | ||
| 227 | amdgpu_bo_unref(>t_obj[i]); | ||
| 228 | } | ||
| 229 | if (fence) | ||
| 230 | dma_fence_put(fence); | ||
| 231 | break; | ||
| 232 | } | ||
| 233 | |||
| 234 | amdgpu_bo_unpin(vram_obj); | ||
| 235 | out_unres: | ||
| 236 | amdgpu_bo_unreserve(vram_obj); | ||
| 237 | out_unref: | ||
| 238 | amdgpu_bo_unref(&vram_obj); | ||
| 239 | out_cleanup: | ||
| 240 | kfree(gtt_obj); | ||
| 241 | if (r) { | ||
| 242 | pr_warn("Error while testing BO move\n"); | ||
| 243 | } | ||
| 244 | } | ||
| 245 | |||
| 246 | void amdgpu_test_moves(struct amdgpu_device *adev) | ||
| 247 | { | ||
| 248 | if (adev->mman.buffer_funcs) | ||
| 249 | amdgpu_do_test_moves(adev); | ||
| 250 | } | ||
| @@ -358,11 +358,10 @@ TRACE_EVENT(amdgpu_vm_update_ptes, · 357 unmodified lines | |||
| 358 | } | 358 | } |
| 359 | ), | 359 | ), |
| 360 | TP_printk("pid:%u vm_ctx:0x%llx start:0x%010llx end:0x%010llx," | 360 | TP_printk("pid:%u vm_ctx:0x%llx start:0x%010llx end:0x%010llx," |
| 361 | " flags:0x%llx, incr:%llu, dst:\n%s%s", __entry->pid, | 361 | " flags:0x%llx, incr:%llu, dst:\n%s", __entry->pid, |
| 362 | __entry->vm_ctx, __entry->start, __entry->end, | 362 | __entry->vm_ctx, __entry->start, __entry->end, |
| 363 | __entry->flags, __entry->incr, __print_array( | 363 | __entry->flags, __entry->incr, __print_array( |
| 364 | __get_dynamic_array(dst), min(__entry->nptes, 32u), 8), | 364 | __get_dynamic_array(dst), __entry->nptes, 8)) |
| 365 | __entry->nptes > 32 ? "..." : "") | ||
| 366 | ); | 365 | ); |
| 367 | 366 | ||
| 368 | TRACE_EVENT(amdgpu_vm_set_ptes, | 367 | TRACE_EVENT(amdgpu_vm_set_ptes, |
| @@ -537,6 +536,22 @@ TRACE_EVENT(amdgpu_ib_pipe_sync, · 168 unmodified lines | |||
| 537 | __entry->seqno) | 536 | __entry->seqno) |
| 538 | ); | 537 | ); |
| 539 | 538 | ||
| 539 | TRACE_EVENT(amdgpu_reset_reg_dumps, | ||
| 540 | TP_PROTO(uint32_t address, uint32_t value), | ||
| 541 | TP_ARGS(address, value), | ||
| 542 | TP_STRUCT__entry( | ||
| 543 | __field(uint32_t, address) | ||
| 544 | __field(uint32_t, value) | ||
| 545 | ), | ||
| 546 | TP_fast_assign( | ||
| 547 | __entry->address = address; | ||
| 548 | __entry->value = value; | ||
| 549 | ), | ||
| 550 | TP_printk("amdgpu register dump 0x%x: 0x%x", | ||
| 551 | __entry->address, | ||
| 552 | __entry->value) | ||
| 553 | ); | ||
| 554 | |||
| 540 | #undef AMDGPU_JOB_GET_TIMELINE_NAME | 555 | #undef AMDGPU_JOB_GET_TIMELINE_NAME |
| 541 | #endif | 556 | #endif |
| 542 | 557 | ||
| @@ -50,6 +50,7 @@ · 49 unmodified lines | |||
| 50 | #include <drm/ttm/ttm_range_manager.h> | 50 | #include <drm/ttm/ttm_range_manager.h> |
| 51 | 51 | ||
| 52 | #include <drm/amdgpu_drm.h> | 52 | #include <drm/amdgpu_drm.h> |
| 53 | #include <drm/drm_drv.h> | ||
| 53 | 54 | ||
| 54 | #include "amdgpu.h" | 55 | #include "amdgpu.h" |
| 55 | #include "amdgpu_object.h" | 56 | #include "amdgpu_object.h" |
| @@ -170,10 +171,10 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, · 114 unmodified lines | |||
| 170 | * @bo: buffer object to map | 171 | * @bo: buffer object to map |
| 171 | * @mem: memory object to map | 172 | * @mem: memory object to map |
| 172 | * @mm_cur: range to map | 173 | * @mm_cur: range to map |
| 173 | * @num_pages: number of pages to map | ||
| 174 | * @window: which GART window to use | 174 | * @window: which GART window to use |
| 175 | * @ring: DMA ring to use for the copy | 175 | * @ring: DMA ring to use for the copy |
| 176 | * @tmz: if we should setup a TMZ enabled mapping | 176 | * @tmz: if we should setup a TMZ enabled mapping |
| 177 | * @size: in number of bytes to map, out number of bytes mapped | ||
| 177 | * @addr: resulting address inside the MC address space | 178 | * @addr: resulting address inside the MC address space |
| 178 | * | 179 | * |
| 179 | * Setup one of the GART windows to access a specific piece of memory or return | 180 | * Setup one of the GART windows to access a specific piece of memory or return |
| @@ -182,15 +183,14 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, · 2 unmodified lines | |||
| 182 | static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, | 183 | static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, |
| 183 | struct ttm_resource *mem, | 184 | struct ttm_resource *mem, |
| 184 | struct amdgpu_res_cursor *mm_cur, | 185 | struct amdgpu_res_cursor *mm_cur, |
| 185 | unsigned num_pages, unsigned window, | 186 | unsigned window, struct amdgpu_ring *ring, |
| 186 | struct amdgpu_ring *ring, bool tmz, | 187 | bool tmz, uint64_t *size, uint64_t *addr) |
| 187 | uint64_t *addr) | ||
| 188 | { | 188 | { |
| 189 | struct amdgpu_device *adev = ring->adev; | 189 | struct amdgpu_device *adev = ring->adev; |
| 190 | struct amdgpu_job *job; | 190 | unsigned offset, num_pages, num_dw, num_bytes; |
| 191 | unsigned num_dw, num_bytes; | ||
| 192 | struct dma_fence *fence; | ||
| 193 | uint64_t src_addr, dst_addr; | 191 | uint64_t src_addr, dst_addr; |
| 192 | struct dma_fence *fence; | ||
| 193 | struct amdgpu_job *job; | ||
| 194 | void *cpu_addr; | 194 | void *cpu_addr; |
| 195 | uint64_t flags; | 195 | uint64_t flags; |
| 196 | unsigned int i; | 196 | unsigned int i; |
| @@ -198,7 +198,9 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 1 unmodified line | |||
| 198 | 198 | ||
| 199 | BUG_ON(adev->mman.buffer_funcs->copy_max_bytes < | 199 | BUG_ON(adev->mman.buffer_funcs->copy_max_bytes < |
| 200 | AMDGPU_GTT_MAX_TRANSFER_SIZE * 8); | 200 | AMDGPU_GTT_MAX_TRANSFER_SIZE * 8); |
| 201 | BUG_ON(mem->mem_type == AMDGPU_PL_PREEMPT); | 201 | |
| 202 | if (WARN_ON(mem->mem_type == AMDGPU_PL_PREEMPT)) | ||
| 203 | return -EINVAL; | ||
| 202 | 204 | ||
| 203 | /* Map only what can't be accessed directly */ | 205 | /* Map only what can't be accessed directly */ |
| 204 | if (!tmz && mem->start != AMDGPU_BO_INVALID_OFFSET) { | 206 | if (!tmz && mem->start != AMDGPU_BO_INVALID_OFFSET) { |
| @@ -207,10 +209,22 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 2 unmodified lines | |||
| 207 | return 0; | 209 | return 0; |
| 208 | } | 210 | } |
| 209 | 211 | ||
| 212 | |||
| 213 | /* | ||
| 214 | * If start begins at an offset inside the page, then adjust the size | ||
| 215 | * and addr accordingly | ||
| 216 | */ | ||
| 217 | offset = mm_cur->start & ~PAGE_MASK; | ||
| 218 | |||
| 219 | num_pages = PFN_UP(*size + offset); | ||
| 220 | num_pages = min_t(uint32_t, num_pages, AMDGPU_GTT_MAX_TRANSFER_SIZE); | ||
| 221 | |||
| 222 | *size = min(*size, (uint64_t)num_pages * PAGE_SIZE - offset); | ||
| 223 | |||
| 210 | *addr = adev->gmc.gart_start; | 224 | *addr = adev->gmc.gart_start; |
| 211 | *addr += (u64)window * AMDGPU_GTT_MAX_TRANSFER_SIZE * | 225 | *addr += (u64)window * AMDGPU_GTT_MAX_TRANSFER_SIZE * |
| 212 | AMDGPU_GPU_PAGE_SIZE; | 226 | AMDGPU_GPU_PAGE_SIZE; |
| 213 | *addr += mm_cur->start & ~PAGE_MASK; | 227 | *addr += offset; |
| 214 | 228 | ||
| 215 | num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8); | 229 | num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8); |
| 216 | num_bytes = num_pages * 8 * AMDGPU_GPU_PAGES_IN_CPU_PAGE; | 230 | num_bytes = num_pages * 8 * AMDGPU_GPU_PAGES_IN_CPU_PAGE; |
| @@ -241,10 +255,7 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 24 unmodified lines | |||
| 241 | dma_addr_t *dma_addr; | 255 | dma_addr_t *dma_addr; |
| 242 | 256 | ||
| 243 | dma_addr = &bo->ttm->dma_address[mm_cur->start >> PAGE_SHIFT]; | 257 | dma_addr = &bo->ttm->dma_address[mm_cur->start >> PAGE_SHIFT]; |
| 244 | r = amdgpu_gart_map(adev, 0, num_pages, dma_addr, flags, | 258 | amdgpu_gart_map(adev, 0, num_pages, dma_addr, flags, cpu_addr); |
| 245 | cpu_addr); | ||
| 246 | if (r) | ||
| 247 | goto error_free; | ||
| 248 | } else { | 259 | } else { |
| 249 | dma_addr_t dma_address; | 260 | dma_addr_t dma_address; |
| 250 | 261 | ||
| @@ -252,11 +263,8 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo, · 1 unmodified line | |||
| 252 | dma_address += adev->vm_manager.vram_base_offset; | 263 | dma_address += adev->vm_manager.vram_base_offset; |
| 253 | 264 | ||
| 254 | for (i = 0; i < num_pages; ++i) { | 265 | for (i = 0; i < num_pages; ++i) { |
| 255 | r = amdgpu_gart_map(adev, i << PAGE_SHIFT, 1, | 266 | amdgpu_gart_map(adev, i << PAGE_SHIFT, 1, &dma_address, |
| 256 | &dma_address, flags, cpu_addr); | 267 | flags, cpu_addr); |
| 257 | if (r) | ||
| 258 | goto error_free; | ||
| 259 | |||
| 260 | dma_address += PAGE_SIZE; | 268 | dma_address += PAGE_SIZE; |
| 261 | } | 269 | } |
| 262 | } | 270 | } |
| @@ -297,9 +305,6 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev, · 34 unmodified lines | |||
| 297 | struct dma_resv *resv, | 305 | struct dma_resv *resv, |
| 298 | struct dma_fence **f) | 306 | struct dma_fence **f) |
| 299 | { | 307 | { |
| 300 | const uint32_t GTT_MAX_BYTES = (AMDGPU_GTT_MAX_TRANSFER_SIZE * | ||
| 301 | AMDGPU_GPU_PAGE_SIZE); | ||
| 302 | |||
| 303 | struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; | 308 | struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; |
| 304 | struct amdgpu_res_cursor src_mm, dst_mm; | 309 | struct amdgpu_res_cursor src_mm, dst_mm; |
| 305 | struct dma_fence *fence = NULL; | 310 | struct dma_fence *fence = NULL; |
| @@ -315,29 +320,20 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev, · 9 unmodified lines | |||
| 315 | 320 | ||
| 316 | mutex_lock(&adev->mman.gtt_window_lock); | 321 | mutex_lock(&adev->mman.gtt_window_lock); |
| 317 | while (src_mm.remaining) { | 322 | while (src_mm.remaining) { |
| 318 | uint32_t src_page_offset = src_mm.start & ~PAGE_MASK; | 323 | uint64_t from, to, cur_size; |
| 319 | uint32_t dst_page_offset = dst_mm.start & ~PAGE_MASK; | ||
| 320 | struct dma_fence *next; | 324 | struct dma_fence *next; |
| 321 | uint32_t cur_size; | ||
| 322 | uint64_t from, to; | ||
| 323 | 325 | ||
| 324 | /* Copy size cannot exceed GTT_MAX_BYTES. So if src or dst | 326 | /* Never copy more than 256MiB at once to avoid a timeout */ |
| 325 | * begins at an offset, then adjust the size accordingly | 327 | cur_size = min3(src_mm.size, dst_mm.size, 256ULL << 20); |
| 326 | */ | ||
| 327 | cur_size = max(src_page_offset, dst_page_offset); | ||
| 328 | cur_size = min(min3(src_mm.size, dst_mm.size, size), | ||
| 329 | (uint64_t)(GTT_MAX_BYTES - cur_size)); | ||
| 330 | 328 | ||
| 331 | /* Map src to window 0 and dst to window 1. */ | 329 | /* Map src to window 0 and dst to window 1. */ |
| 332 | r = amdgpu_ttm_map_buffer(src->bo, src->mem, &src_mm, | 330 | r = amdgpu_ttm_map_buffer(src->bo, src->mem, &src_mm, |
| 333 | PFN_UP(cur_size + src_page_offset), | 331 | 0, ring, tmz, &cur_size, &from); |
| 334 | 0, ring, tmz, &from); | ||
| 335 | if (r) | 332 | if (r) |
| 336 | goto error; | 333 | goto error; |
| 337 | 334 | ||
| 338 | r = amdgpu_ttm_map_buffer(dst->bo, dst->mem, &dst_mm, | 335 | r = amdgpu_ttm_map_buffer(dst->bo, dst->mem, &dst_mm, |
| 339 | PFN_UP(cur_size + dst_page_offset), | 336 | 1, ring, tmz, &cur_size, &to); |
| 340 | 1, ring, tmz, &to); | ||
| 341 | if (r) | 337 | if (r) |
| 342 | goto error; | 338 | goto error; |
| 343 | 339 | ||
| @@ -396,8 +392,7 @@ static int amdgpu_move_blit(struct ttm_buffer_object *bo, · 52 unmodified lines | |||
| 396 | (abo->flags & AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE)) { | 392 | (abo->flags & AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE)) { |
| 397 | struct dma_fence *wipe_fence = NULL; | 393 | struct dma_fence *wipe_fence = NULL; |
| 398 | 394 | ||
| 399 | r = amdgpu_fill_buffer(ttm_to_amdgpu_bo(bo), AMDGPU_POISON, | 395 | r = amdgpu_fill_buffer(abo, AMDGPU_POISON, NULL, &wipe_fence); |
| 400 | NULL, &wipe_fence); | ||
| 401 | if (r) { | 396 | if (r) { |
| 402 | goto error; | 397 | goto error; |
| 403 | } else if (wipe_fence) { | 398 | } else if (wipe_fence) { |
| @@ -821,14 +816,13 @@ static void amdgpu_ttm_tt_unpin_userptr(struct ttm_device *bdev, · 417 unmodified lines | |||
| 821 | #endif | 816 | #endif |
| 822 | } | 817 | } |
| 823 | 818 | ||
| 824 | static int amdgpu_ttm_gart_bind(struct amdgpu_device *adev, | 819 | static void amdgpu_ttm_gart_bind(struct amdgpu_device *adev, |
| 825 | struct ttm_buffer_object *tbo, | 820 | struct ttm_buffer_object *tbo, |
| 826 | uint64_t flags) | 821 | uint64_t flags) |
| 827 | { | 822 | { |
| 828 | struct amdgpu_bo *abo = ttm_to_amdgpu_bo(tbo); | 823 | struct amdgpu_bo *abo = ttm_to_amdgpu_bo(tbo); |
| 829 | struct ttm_tt *ttm = tbo->ttm; | 824 | struct ttm_tt *ttm = tbo->ttm; |
| 830 | struct amdgpu_ttm_tt *gtt = (void *)ttm; | 825 | struct amdgpu_ttm_tt *gtt = (void *)ttm; |
| 831 | int r; | ||
| 832 | 826 | ||
| 833 | if (amdgpu_bo_encrypted(abo)) | 827 | if (amdgpu_bo_encrypted(abo)) |
| 834 | flags |= AMDGPU_PTE_TMZ; | 828 | flags |= AMDGPU_PTE_TMZ; |
| @@ -836,10 +830,8 @@ static int amdgpu_ttm_gart_bind(struct amdgpu_device *adev, · 1 unmodified line | |||
| 836 | if (abo->flags & AMDGPU_GEM_CREATE_CP_MQD_GFX9) { | 830 | if (abo->flags & AMDGPU_GEM_CREATE_CP_MQD_GFX9) { |
| 837 | uint64_t page_idx = 1; | 831 | uint64_t page_idx = 1; |
| 838 | 832 | ||
| 839 | r = amdgpu_gart_bind(adev, gtt->offset, page_idx, | 833 | amdgpu_gart_bind(adev, gtt->offset, page_idx, |
| 840 | gtt->ttm.dma_address, flags); | 834 | gtt->ttm.dma_address, flags); |
| 841 | if (r) | ||
| 842 | goto gart_bind_fail; | ||
| 843 | 835 | ||
| 844 | /* The memory type of the first page defaults to UC. Now | 836 | /* The memory type of the first page defaults to UC. Now |
| 845 | * modify the memory type to NC from the second page of | 837 | * modify the memory type to NC from the second page of |
| @@ -848,21 +840,13 @@ static int amdgpu_ttm_gart_bind(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 848 | flags &= ~AMDGPU_PTE_MTYPE_VG10_MASK; | 840 | flags &= ~AMDGPU_PTE_MTYPE_VG10_MASK; |
| 849 | flags |= AMDGPU_PTE_MTYPE_VG10(AMDGPU_MTYPE_NC); | 841 | flags |= AMDGPU_PTE_MTYPE_VG10(AMDGPU_MTYPE_NC); |
| 850 | 842 | ||
| 851 | r = amdgpu_gart_bind(adev, | 843 | amdgpu_gart_bind(adev, gtt->offset + (page_idx << PAGE_SHIFT), |
| 852 | gtt->offset + (page_idx << PAGE_SHIFT), | 844 | ttm->num_pages - page_idx, |
| 853 | ttm->num_pages - page_idx, | 845 | &(gtt->ttm.dma_address[page_idx]), flags); |
| 854 | &(gtt->ttm.dma_address[page_idx]), flags); | ||
| 855 | } else { | 846 | } else { |
| 856 | r = amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages, | 847 | amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages, |
| 857 | gtt->ttm.dma_address, flags); | 848 | gtt->ttm.dma_address, flags); |
| 858 | } | 849 | } |
| 859 | |||
| 860 | gart_bind_fail: | ||
| 861 | if (r) | ||
| 862 | DRM_ERROR("failed to bind %u pages at 0x%08llX\n", | ||
| 863 | ttm->num_pages, gtt->offset); | ||
| 864 | |||
| 865 | return r; | ||
| 866 | } | 850 | } |
| 867 | 851 | ||
| 868 | /* | 852 | /* |
| @@ -878,7 +862,7 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev, · 9 unmodified lines | |||
| 878 | struct amdgpu_device *adev = amdgpu_ttm_adev(bdev); | 862 | struct amdgpu_device *adev = amdgpu_ttm_adev(bdev); |
| 879 | struct amdgpu_ttm_tt *gtt = (void*)ttm; | 863 | struct amdgpu_ttm_tt *gtt = (void*)ttm; |
| 880 | uint64_t flags; | 864 | uint64_t flags; |
| 881 | int r = 0; | 865 | int r; |
| 882 | 866 | ||
| 883 | if (!bo_mem) | 867 | if (!bo_mem) |
| 884 | return -EINVAL; | 868 | return -EINVAL; |
| @@ -925,14 +909,10 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev, · 40 unmodified lines | |||
| 925 | 909 | ||
| 926 | /* bind pages into GART page tables */ | 910 | /* bind pages into GART page tables */ |
| 927 | gtt->offset = (u64)bo_mem->start << PAGE_SHIFT; | 911 | gtt->offset = (u64)bo_mem->start << PAGE_SHIFT; |
| 928 | r = amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages, | 912 | amdgpu_gart_bind(adev, gtt->offset, ttm->num_pages, |
| 929 | gtt->ttm.dma_address, flags); | 913 | gtt->ttm.dma_address, flags); |
| 930 | |||
| 931 | if (r) | ||
| 932 | DRM_ERROR("failed to bind %u pages at 0x%08llX\n", | ||
| 933 | ttm->num_pages, gtt->offset); | ||
| 934 | gtt->bound = true; | 914 | gtt->bound = true; |
| 935 | return r; | 915 | return 0; |
| 936 | } | 916 | } |
| 937 | 917 | ||
| 938 | /* | 918 | /* |
| @@ -982,12 +962,7 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo) · 43 unmodified lines | |||
| 982 | 962 | ||
| 983 | /* Bind pages */ | 963 | /* Bind pages */ |
| 984 | gtt->offset = (u64)tmp->start << PAGE_SHIFT; | 964 | gtt->offset = (u64)tmp->start << PAGE_SHIFT; |
| 985 | r = amdgpu_ttm_gart_bind(adev, bo, flags); | 965 | amdgpu_ttm_gart_bind(adev, bo, flags); |
| 986 | if (unlikely(r)) { | ||
| 987 | ttm_resource_free(bo, &tmp); | ||
| 988 | return r; | ||
| 989 | } | ||
| 990 | |||
| 991 | amdgpu_gart_invalidate_tlb(adev); | 966 | amdgpu_gart_invalidate_tlb(adev); |
| 992 | ttm_resource_free(bo, &bo->resource); | 967 | ttm_resource_free(bo, &bo->resource); |
| 993 | ttm_bo_assign_mem(bo, tmp); | 968 | ttm_bo_assign_mem(bo, tmp); |
| @@ -1001,19 +976,16 @@ int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo) · 7 unmodified lines | |||
| 1001 | * Called by amdgpu_gtt_mgr_recover() from amdgpu_device_reset() to | 976 | * Called by amdgpu_gtt_mgr_recover() from amdgpu_device_reset() to |
| 1002 | * rebind GTT pages during a GPU reset. | 977 | * rebind GTT pages during a GPU reset. |
| 1003 | */ | 978 | */ |
| 1004 | int amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo) | 979 | void amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo) |
| 1005 | { | 980 | { |
| 1006 | struct amdgpu_device *adev = amdgpu_ttm_adev(tbo->bdev); | 981 | struct amdgpu_device *adev = amdgpu_ttm_adev(tbo->bdev); |
| 1007 | uint64_t flags; | 982 | uint64_t flags; |
| 1008 | int r; | ||
| 1009 | 983 | ||
| 1010 | if (!tbo->ttm) | 984 | if (!tbo->ttm) |
| 1011 | return 0; | 985 | return; |
| 1012 | 986 | ||
| 1013 | flags = amdgpu_ttm_tt_pte_flags(adev, tbo->ttm, tbo->resource); | 987 | flags = amdgpu_ttm_tt_pte_flags(adev, tbo->ttm, tbo->resource); |
| 1014 | r = amdgpu_ttm_gart_bind(adev, tbo, flags); | 988 | amdgpu_ttm_gart_bind(adev, tbo, flags); |
| 1015 | |||
| 1016 | return r; | ||
| 1017 | } | 989 | } |
| 1018 | 990 | ||
| 1019 | /* | 991 | /* |
| @@ -1027,7 +999,6 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev, · 7 unmodified lines | |||
| 1027 | { | 999 | { |
| 1028 | struct amdgpu_device *adev = amdgpu_ttm_adev(bdev); | 1000 | struct amdgpu_device *adev = amdgpu_ttm_adev(bdev); |
| 1029 | struct amdgpu_ttm_tt *gtt = (void *)ttm; | 1001 | struct amdgpu_ttm_tt *gtt = (void *)ttm; |
| 1030 | int r; | ||
| 1031 | 1002 | ||
| 1032 | /* if the pages have userptr pinning then clear that first */ | 1003 | /* if the pages have userptr pinning then clear that first */ |
| 1033 | if (gtt->userptr) { | 1004 | if (gtt->userptr) { |
| @@ -1047,10 +1018,7 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev, · 13 unmodified lines | |||
| 1047 | return; | 1018 | return; |
| 1048 | 1019 | ||
| 1049 | /* unbind shouldn't be done for GDS/GWS/OA in ttm_bo_clean_mm */ | 1020 | /* unbind shouldn't be done for GDS/GWS/OA in ttm_bo_clean_mm */ |
| 1050 | r = amdgpu_gart_unbind(adev, gtt->offset, ttm->num_pages); | 1021 | amdgpu_gart_unbind(adev, gtt->offset, ttm->num_pages); |
| 1051 | if (r) | ||
| 1052 | DRM_ERROR("failed to unbind %u pages at 0x%08llX\n", | ||
| 1053 | gtt->ttm.num_pages, gtt->offset); | ||
| 1054 | gtt->bound = false; | 1022 | gtt->bound = false; |
| 1055 | } | 1023 | } |
| 1056 | 1024 | ||
| @@ -1168,6 +1136,26 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev, · 111 unmodified lines | |||
| 1168 | return ttm_pool_free(&adev->mman.bdev.pool, ttm); | 1136 | return ttm_pool_free(&adev->mman.bdev.pool, ttm); |
| 1169 | } | 1137 | } |
| 1170 | 1138 | ||
| 1139 | /** | ||
| 1140 | * amdgpu_ttm_tt_get_userptr - Return the userptr GTT ttm_tt for the current | ||
| 1141 | * task | ||
| 1142 | * | ||
| 1143 | * @tbo: The ttm_buffer_object that contains the userptr | ||
| 1144 | * @user_addr: The returned value | ||
| 1145 | */ | ||
| 1146 | int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo, | ||
| 1147 | uint64_t *user_addr) | ||
| 1148 | { | ||
| 1149 | struct amdgpu_ttm_tt *gtt; | ||
| 1150 | |||
| 1151 | if (!tbo->ttm) | ||
| 1152 | return -EINVAL; | ||
| 1153 | |||
| 1154 | gtt = (void *)tbo->ttm; | ||
| 1155 | *user_addr = gtt->userptr; | ||
| 1156 | return 0; | ||
| 1157 | } | ||
| 1158 | |||
| 1171 | /** | 1159 | /** |
| 1172 | * amdgpu_ttm_tt_set_userptr - Initialize userptr GTT ttm_tt for the current | 1160 | * amdgpu_ttm_tt_set_userptr - Initialize userptr GTT ttm_tt for the current |
| 1173 | * task | 1161 | * task |
| @@ -1433,6 +1421,63 @@ static void amdgpu_ttm_vram_mm_access(struct amdgpu_device *adev, loff_t pos, · 259 unmodified lines | |||
| 1433 | } | 1421 | } |
| 1434 | } | 1422 | } |
| 1435 | 1423 | ||
| 1424 | static int amdgpu_ttm_access_memory_sdma(struct ttm_buffer_object *bo, | ||
| 1425 | unsigned long offset, void *buf, int len, int write) | ||
| 1426 | { | ||
| 1427 | struct amdgpu_bo *abo = ttm_to_amdgpu_bo(bo); | ||
| 1428 | struct amdgpu_device *adev = amdgpu_ttm_adev(abo->tbo.bdev); | ||
| 1429 | struct amdgpu_res_cursor src_mm; | ||
| 1430 | struct amdgpu_job *job; | ||
| 1431 | struct dma_fence *fence; | ||
| 1432 | uint64_t src_addr, dst_addr; | ||
| 1433 | unsigned int num_dw; | ||
| 1434 | int r, idx; | ||
| 1435 | |||
| 1436 | if (len != PAGE_SIZE) | ||
| 1437 | return -EINVAL; | ||
| 1438 | |||
| 1439 | if (!adev->mman.sdma_access_ptr) | ||
| 1440 | return -EACCES; | ||
| 1441 | |||
| 1442 | if (!drm_dev_enter(adev_to_drm(adev), &idx)) | ||
| 1443 | return -ENODEV; | ||
| 1444 | |||
| 1445 | if (write) | ||
| 1446 | memcpy(adev->mman.sdma_access_ptr, buf, len); | ||
| 1447 | |||
| 1448 | num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8); | ||
| 1449 | r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, AMDGPU_IB_POOL_DELAYED, &job); | ||
| 1450 | if (r) | ||
| 1451 | goto out; | ||
| 1452 | |||
| 1453 | amdgpu_res_first(abo->tbo.resource, offset, len, &src_mm); | ||
| 1454 | src_addr = amdgpu_ttm_domain_start(adev, bo->resource->mem_type) + src_mm.start; | ||
| 1455 | dst_addr = amdgpu_bo_gpu_offset(adev->mman.sdma_access_bo); | ||
| 1456 | if (write) | ||
| 1457 | swap(src_addr, dst_addr); | ||
| 1458 | |||
| 1459 | amdgpu_emit_copy_buffer(adev, &job->ibs[0], src_addr, dst_addr, PAGE_SIZE, false); | ||
| 1460 | |||
| 1461 | amdgpu_ring_pad_ib(adev->mman.buffer_funcs_ring, &job->ibs[0]); | ||
| 1462 | WARN_ON(job->ibs[0].length_dw > num_dw); | ||
| 1463 | |||
| 1464 | r = amdgpu_job_submit(job, &adev->mman.entity, AMDGPU_FENCE_OWNER_UNDEFINED, &fence); | ||
| 1465 | if (r) { | ||
| 1466 | amdgpu_job_free(job); | ||
| 1467 | goto out; | ||
| 1468 | } | ||
| 1469 | |||
| 1470 | if (!dma_fence_wait_timeout(fence, false, adev->sdma_timeout)) | ||
| 1471 | r = -ETIMEDOUT; | ||
| 1472 | dma_fence_put(fence); | ||
| 1473 | |||
| 1474 | if (!(r || write)) | ||
| 1475 | memcpy(buf, adev->mman.sdma_access_ptr, len); | ||
| 1476 | out: | ||
| 1477 | drm_dev_exit(idx); | ||
| 1478 | return r; | ||
| 1479 | } | ||
| 1480 | |||
| 1436 | /** | 1481 | /** |
| 1437 | * amdgpu_ttm_access_memory - Read or Write memory that backs a buffer object. | 1482 | * amdgpu_ttm_access_memory - Read or Write memory that backs a buffer object. |
| 1438 | * | 1483 | * |
| @@ -1457,6 +1502,10 @@ static int amdgpu_ttm_access_memory(struct ttm_buffer_object *bo, · 18 unmodified lines | |||
| 1457 | if (bo->resource->mem_type != TTM_PL_VRAM) | 1502 | if (bo->resource->mem_type != TTM_PL_VRAM) |
| 1458 | return -EIO; | 1503 | return -EIO; |
| 1459 | 1504 | ||
| 1505 | if (amdgpu_device_has_timeouts_enabled(adev) && | ||
| 1506 | !amdgpu_ttm_access_memory_sdma(bo, offset, buf, len, write)) | ||
| 1507 | return len; | ||
| 1508 | |||
| 1460 | amdgpu_res_first(bo->resource, offset, len, &cursor); | 1509 | amdgpu_res_first(bo->resource, offset, len, &cursor); |
| 1461 | while (cursor.remaining) { | 1510 | while (cursor.remaining) { |
| 1462 | size_t count, size = cursor.size; | 1511 | size_t count, size = cursor.size; |
| @@ -1797,6 +1846,12 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) · 334 unmodified lines | |||
| 1797 | return r; | 1846 | return r; |
| 1798 | } | 1847 | } |
| 1799 | 1848 | ||
| 1849 | if (amdgpu_bo_create_kernel(adev, PAGE_SIZE, PAGE_SIZE, | ||
| 1850 | AMDGPU_GEM_DOMAIN_GTT, | ||
| 1851 | &adev->mman.sdma_access_bo, NULL, | ||
| 1852 | &adev->mman.sdma_access_ptr)) | ||
| 1853 | DRM_WARN("Debug VRAM access will use slowpath MM access\n"); | ||
| 1854 | |||
| 1800 | return 0; | 1855 | return 0; |
| 1801 | } | 1856 | } |
| 1802 | 1857 | ||
| @@ -1818,6 +1873,8 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) · 15 unmodified lines | |||
| 1818 | if (adev->mman.stolen_reserved_size) | 1873 | if (adev->mman.stolen_reserved_size) |
| 1819 | amdgpu_bo_free_kernel(&adev->mman.stolen_reserved_memory, | 1874 | amdgpu_bo_free_kernel(&adev->mman.stolen_reserved_memory, |
| 1820 | NULL, NULL); | 1875 | NULL, NULL); |
| 1876 | amdgpu_bo_free_kernel(&adev->mman.sdma_access_bo, NULL, | ||
| 1877 | &adev->mman.sdma_access_ptr); | ||
| 1821 | amdgpu_ttm_fw_reserve_vram_fini(adev); | 1878 | amdgpu_ttm_fw_reserve_vram_fini(adev); |
| 1822 | 1879 | ||
| 1823 | if (drm_dev_enter(adev_to_drm(adev), &idx)) { | 1880 | if (drm_dev_enter(adev_to_drm(adev), &idx)) { |
| @@ -1884,27 +1941,59 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) · 60 unmodified lines | |||
| 1884 | size = adev->gmc.real_vram_size; | 1941 | size = adev->gmc.real_vram_size; |
| 1885 | else | 1942 | else |
| 1886 | size = adev->gmc.visible_vram_size; | 1943 | size = adev->gmc.visible_vram_size; |
| 1887 | man->size = size >> PAGE_SHIFT; | 1944 | man->size = size; |
| 1888 | adev->mman.buffer_funcs_enabled = enable; | 1945 | adev->mman.buffer_funcs_enabled = enable; |
| 1889 | } | 1946 | } |
| 1890 | 1947 | ||
| 1948 | static int amdgpu_ttm_prepare_job(struct amdgpu_device *adev, | ||
| 1949 | bool direct_submit, | ||
| 1950 | unsigned int num_dw, | ||
| 1951 | struct dma_resv *resv, | ||
| 1952 | bool vm_needs_flush, | ||
| 1953 | struct amdgpu_job **job) | ||
| 1954 | { | ||
| 1955 | enum amdgpu_ib_pool_type pool = direct_submit ? | ||
| 1956 | AMDGPU_IB_POOL_DIRECT : | ||
| 1957 | AMDGPU_IB_POOL_DELAYED; | ||
| 1958 | int r; | ||
| 1959 | |||
| 1960 | r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, pool, job); | ||
| 1961 | if (r) | ||
| 1962 | return r; | ||
| 1963 | |||
| 1964 | if (vm_needs_flush) { | ||
| 1965 | (*job)->vm_pd_addr = amdgpu_gmc_pd_addr(adev->gmc.pdb0_bo ? | ||
| 1966 | adev->gmc.pdb0_bo : | ||
| 1967 | adev->gart.bo); | ||
| 1968 | (*job)->vm_needs_flush = true; | ||
| 1969 | } | ||
| 1970 | if (resv) { | ||
| 1971 | r = amdgpu_sync_resv(adev, &(*job)->sync, resv, | ||
| 1972 | AMDGPU_SYNC_ALWAYS, | ||
| 1973 | AMDGPU_FENCE_OWNER_UNDEFINED); | ||
| 1974 | if (r) { | ||
| 1975 | DRM_ERROR("sync failed (%d).\n", r); | ||
| 1976 | amdgpu_job_free(*job); | ||
| 1977 | return r; | ||
| 1978 | } | ||
| 1979 | } | ||
| 1980 | return 0; | ||
| 1981 | } | ||
| 1982 | |||
| 1891 | int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset, | 1983 | int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset, |
| 1892 | uint64_t dst_offset, uint32_t byte_count, | 1984 | uint64_t dst_offset, uint32_t byte_count, |
| 1893 | struct dma_resv *resv, | 1985 | struct dma_resv *resv, |
| 1894 | struct dma_fence **fence, bool direct_submit, | 1986 | struct dma_fence **fence, bool direct_submit, |
| 1895 | bool vm_needs_flush, bool tmz) | 1987 | bool vm_needs_flush, bool tmz) |
| 1896 | { | 1988 | { |
| 1897 | enum amdgpu_ib_pool_type pool = direct_submit ? AMDGPU_IB_POOL_DIRECT : | ||
| 1898 | AMDGPU_IB_POOL_DELAYED; | ||
| 1899 | struct amdgpu_device *adev = ring->adev; | 1989 | struct amdgpu_device *adev = ring->adev; |
| 1990 | unsigned num_loops, num_dw; | ||
| 1900 | struct amdgpu_job *job; | 1991 | struct amdgpu_job *job; |
| 1901 | |||
| 1902 | uint32_t max_bytes; | 1992 | uint32_t max_bytes; |
| 1903 | unsigned num_loops, num_dw; | ||
| 1904 | unsigned i; | 1993 | unsigned i; |
| 1905 | int r; | 1994 | int r; |
| 1906 | 1995 | ||
| 1907 | if (direct_submit && !ring->sched.ready) { | 1996 | if (!direct_submit && !ring->sched.ready) { |
| 1908 | DRM_ERROR("Trying to move memory with ring turned off.\n"); | 1997 | DRM_ERROR("Trying to move memory with ring turned off.\n"); |
| 1909 | return -EINVAL; | 1998 | return -EINVAL; |
| 1910 | } | 1999 | } |
| @@ -1912,26 +2001,11 @@ int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset, · 1 unmodified line | |||
| 1912 | max_bytes = adev->mman.buffer_funcs->copy_max_bytes; | 2001 | max_bytes = adev->mman.buffer_funcs->copy_max_bytes; |
| 1913 | num_loops = DIV_ROUND_UP(byte_count, max_bytes); | 2002 | num_loops = DIV_ROUND_UP(byte_count, max_bytes); |
| 1914 | num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->copy_num_dw, 8); | 2003 | num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->copy_num_dw, 8); |
| 1915 | 2004 | r = amdgpu_ttm_prepare_job(adev, direct_submit, num_dw, | |
| 1916 | r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, pool, &job); | 2005 | resv, vm_needs_flush, &job); |
| 1917 | if (r) | 2006 | if (r) |
| 1918 | return r; | 2007 | return r; |
| 1919 | 2008 | ||
| 1920 | if (vm_needs_flush) { | ||
| 1921 | job->vm_pd_addr = amdgpu_gmc_pd_addr(adev->gmc.pdb0_bo ? | ||
| 1922 | adev->gmc.pdb0_bo : adev->gart.bo); | ||
| 1923 | job->vm_needs_flush = true; | ||
| 1924 | } | ||
| 1925 | if (resv) { | ||
| 1926 | r = amdgpu_sync_resv(adev, &job->sync, resv, | ||
| 1927 | AMDGPU_SYNC_ALWAYS, | ||
| 1928 | AMDGPU_FENCE_OWNER_UNDEFINED); | ||
| 1929 | if (r) { | ||
| 1930 | DRM_ERROR("sync failed (%d).\n", r); | ||
| 1931 | goto error_free; | ||
| 1932 | } | ||
| 1933 | } | ||
| 1934 | |||
| 1935 | for (i = 0; i < num_loops; i++) { | 2009 | for (i = 0; i < num_loops; i++) { |
| 1936 | uint32_t cur_size_in_bytes = min(byte_count, max_bytes); | 2010 | uint32_t cur_size_in_bytes = min(byte_count, max_bytes); |
| 1937 | 2011 | ||
| @@ -1961,77 +2035,35 @@ error_free: · 23 unmodified lines | |||
| 1961 | return r; | 2035 | return r; |
| 1962 | } | 2036 | } |
| 1963 | 2037 | ||
| 1964 | int amdgpu_fill_buffer(struct amdgpu_bo *bo, | 2038 | static int amdgpu_ttm_fill_mem(struct amdgpu_ring *ring, uint32_t src_data, |
| 1965 | uint32_t src_data, | 2039 | uint64_t dst_addr, uint32_t byte_count, |
| 1966 | struct dma_resv *resv, | 2040 | struct dma_resv *resv, |
| 1967 | struct dma_fence **fence) | 2041 | struct dma_fence **fence, |
| 2042 | bool vm_needs_flush) | ||
| 1968 | { | 2043 | { |
| 1969 | struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | 2044 | struct amdgpu_device *adev = ring->adev; |
| 1970 | uint32_t max_bytes = adev->mman.buffer_funcs->fill_max_bytes; | ||
| 1971 | struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; | ||
| 1972 | |||
| 1973 | struct amdgpu_res_cursor cursor; | ||
| 1974 | unsigned int num_loops, num_dw; | 2045 | unsigned int num_loops, num_dw; |
| 1975 | uint64_t num_bytes; | ||
| 1976 | |||
| 1977 | struct amdgpu_job *job; | 2046 | struct amdgpu_job *job; |
| 2047 | uint32_t max_bytes; | ||
| 2048 | unsigned int i; | ||
| 1978 | int r; | 2049 | int r; |
| 1979 | 2050 | ||
| 1980 | if (!adev->mman.buffer_funcs_enabled) { | 2051 | max_bytes = adev->mman.buffer_funcs->fill_max_bytes; |
| 1981 | DRM_ERROR("Trying to clear memory with ring turned off.\n"); | 2052 | num_loops = DIV_ROUND_UP_ULL(byte_count, max_bytes); |
| 1982 | return -EINVAL; | 2053 | num_dw = ALIGN(num_loops * adev->mman.buffer_funcs->fill_num_dw, 8); |
| 1983 | } | 2054 | r = amdgpu_ttm_prepare_job(adev, false, num_dw, resv, vm_needs_flush, |
| 1984 | 2055 | &job); | |
| 1985 | if (bo->tbo.resource->mem_type == AMDGPU_PL_PREEMPT) { | ||
| 1986 | DRM_ERROR("Trying to clear preemptible memory.\n"); | ||
| 1987 | return -EINVAL; | ||
| 1988 | } | ||
| 1989 | |||
| 1990 | if (bo->tbo.resource->mem_type == TTM_PL_TT) { | ||
| 1991 | r = amdgpu_ttm_alloc_gart(&bo->tbo); | ||
| 1992 | if (r) | ||
| 1993 | return r; | ||
| 1994 | } | ||
| 1995 | |||
| 1996 | num_bytes = bo->tbo.resource->num_pages << PAGE_SHIFT; | ||
| 1997 | num_loops = 0; | ||
| 1998 | |||
| 1999 | amdgpu_res_first(bo->tbo.resource, 0, num_bytes, &cursor); | ||
| 2000 | while (cursor.remaining) { | ||
| 2001 | num_loops += DIV_ROUND_UP_ULL(cursor.size, max_bytes); | ||
| 2002 | amdgpu_res_next(&cursor, cursor.size); | ||
| 2003 | } | ||
| 2004 | num_dw = num_loops * adev->mman.buffer_funcs->fill_num_dw; | ||
| 2005 | |||
| 2006 | /* for IB padding */ | ||
| 2007 | num_dw += 64; | ||
| 2008 | |||
| 2009 | r = amdgpu_job_alloc_with_ib(adev, num_dw * 4, AMDGPU_IB_POOL_DELAYED, | ||
| 2010 | &job); | ||
| 2011 | if (r) | 2056 | if (r) |
| 2012 | return r; | 2057 | return r; |
| 2013 | 2058 | ||
| 2014 | if (resv) { | 2059 | for (i = 0; i < num_loops; i++) { |
| 2015 | r = amdgpu_sync_resv(adev, &job->sync, resv, | 2060 | uint32_t cur_size = min(byte_count, max_bytes); |
| 2016 | AMDGPU_SYNC_ALWAYS, | ||
| 2017 | AMDGPU_FENCE_OWNER_UNDEFINED); | ||
| 2018 | if (r) { | ||
| 2019 | DRM_ERROR("sync failed (%d).\n", r); | ||
| 2020 | goto error_free; | ||
| 2021 | } | ||
| 2022 | } | ||
| 2023 | |||
| 2024 | amdgpu_res_first(bo->tbo.resource, 0, num_bytes, &cursor); | ||
| 2025 | while (cursor.remaining) { | ||
| 2026 | uint32_t cur_size = min_t(uint64_t, cursor.size, max_bytes); | ||
| 2027 | uint64_t dst_addr = cursor.start; | ||
| 2028 | 2061 | ||
| 2029 | dst_addr += amdgpu_ttm_domain_start(adev, | ||
| 2030 | bo->tbo.resource->mem_type); | ||
| 2031 | amdgpu_emit_fill_buffer(adev, &job->ibs[0], src_data, dst_addr, | 2062 | amdgpu_emit_fill_buffer(adev, &job->ibs[0], src_data, dst_addr, |
| 2032 | cur_size); | 2063 | cur_size); |
| 2033 | 2064 | ||
| 2034 | amdgpu_res_next(&cursor, cur_size); | 2065 | dst_addr += cur_size; |
| 2066 | byte_count -= cur_size; | ||
| 2035 | } | 2067 | } |
| 2036 | 2068 | ||
| 2037 | amdgpu_ring_pad_ib(ring, &job->ibs[0]); | 2069 | amdgpu_ring_pad_ib(ring, &job->ibs[0]); |
| @@ -2048,6 +2080,55 @@ error_free: · 10 unmodified lines | |||
| 2048 | return r; | 2080 | return r; |
| 2049 | } | 2081 | } |
| 2050 | 2082 | ||
| 2083 | int amdgpu_fill_buffer(struct amdgpu_bo *bo, | ||
| 2084 | uint32_t src_data, | ||
| 2085 | struct dma_resv *resv, | ||
| 2086 | struct dma_fence **f) | ||
| 2087 | { | ||
| 2088 | struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev); | ||
| 2089 | struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; | ||
| 2090 | struct dma_fence *fence = NULL; | ||
| 2091 | struct amdgpu_res_cursor dst; | ||
| 2092 | int r; | ||
| 2093 | |||
| 2094 | if (!adev->mman.buffer_funcs_enabled) { | ||
| 2095 | DRM_ERROR("Trying to clear memory with ring turned off.\n"); | ||
| 2096 | return -EINVAL; | ||
| 2097 | } | ||
| 2098 | |||
| 2099 | amdgpu_res_first(bo->tbo.resource, 0, amdgpu_bo_size(bo), &dst); | ||
| 2100 | |||
| 2101 | mutex_lock(&adev->mman.gtt_window_lock); | ||
| 2102 | while (dst.remaining) { | ||
| 2103 | struct dma_fence *next; | ||
| 2104 | uint64_t cur_size, to; | ||
| 2105 | |||
| 2106 | /* Never fill more than 256MiB at once to avoid timeouts */ | ||
| 2107 | cur_size = min(dst.size, 256ULL << 20); | ||
| 2108 | |||
| 2109 | r = amdgpu_ttm_map_buffer(&bo->tbo, bo->tbo.resource, &dst, | ||
| 2110 | 1, ring, false, &cur_size, &to); | ||
| 2111 | if (r) | ||
| 2112 | goto error; | ||
| 2113 | |||
| 2114 | r = amdgpu_ttm_fill_mem(ring, src_data, to, cur_size, resv, | ||
| 2115 | &next, true); | ||
| 2116 | if (r) | ||
| 2117 | goto error; | ||
| 2118 | |||
| 2119 | dma_fence_put(fence); | ||
| 2120 | fence = next; | ||
| 2121 | |||
| 2122 | amdgpu_res_next(&dst, cur_size); | ||
| 2123 | } | ||
| 2124 | error: | ||
| 2125 | mutex_unlock(&adev->mman.gtt_window_lock); | ||
| 2126 | if (f) | ||
| 2127 | *f = dma_fence_get(fence); | ||
| 2128 | dma_fence_put(fence); | ||
| 2129 | return r; | ||
| 2130 | } | ||
| 2131 | |||
| 2051 | /** | 2132 | /** |
| 2052 | * amdgpu_ttm_evict_resources - evict memory buffers | 2133 | * amdgpu_ttm_evict_resources - evict memory buffers |
| 2053 | * @adev: amdgpu device object | 2134 | * @adev: amdgpu device object |
| @@ -44,7 +44,6 @@ struct amdgpu_vram_mgr { · 43 unmodified lines | |||
| 44 | spinlock_t lock; | 44 | spinlock_t lock; |
| 45 | struct list_head reservations_pending; | 45 | struct list_head reservations_pending; |
| 46 | struct list_head reserved_pages; | 46 | struct list_head reserved_pages; |
| 47 | atomic64_t usage; | ||
| 48 | atomic64_t vis_usage; | 47 | atomic64_t vis_usage; |
| 49 | }; | 48 | }; |
| 50 | 49 | ||
| @@ -52,12 +51,6 @@ struct amdgpu_gtt_mgr { · 1 unmodified line | |||
| 52 | struct ttm_resource_manager manager; | 51 | struct ttm_resource_manager manager; |
| 53 | struct drm_mm mm; | 52 | struct drm_mm mm; |
| 54 | spinlock_t lock; | 53 | spinlock_t lock; |
| 55 | atomic64_t used; | ||
| 56 | }; | ||
| 57 | |||
| 58 | struct amdgpu_preempt_mgr { | ||
| 59 | struct ttm_resource_manager manager; | ||
| 60 | atomic64_t used; | ||
| 61 | }; | 54 | }; |
| 62 | 55 | ||
| 63 | struct amdgpu_mman { | 56 | struct amdgpu_mman { |
| @@ -76,7 +69,7 @@ struct amdgpu_mman { · 12 unmodified lines | |||
| 76 | 69 | ||
| 77 | struct amdgpu_vram_mgr vram_mgr; | 70 | struct amdgpu_vram_mgr vram_mgr; |
| 78 | struct amdgpu_gtt_mgr gtt_mgr; | 71 | struct amdgpu_gtt_mgr gtt_mgr; |
| 79 | struct amdgpu_preempt_mgr preempt_mgr; | 72 | struct ttm_resource_manager preempt_mgr; |
| 80 | 73 | ||
| 81 | uint64_t stolen_vga_size; | 74 | uint64_t stolen_vga_size; |
| 82 | struct amdgpu_bo *stolen_vga_memory; | 75 | struct amdgpu_bo *stolen_vga_memory; |
| @@ -98,6 +91,10 @@ struct amdgpu_mman { · 15 unmodified lines | |||
| 98 | u64 fw_vram_usage_size; | 91 | u64 fw_vram_usage_size; |
| 99 | struct amdgpu_bo *fw_vram_usage_reserved_bo; | 92 | struct amdgpu_bo *fw_vram_usage_reserved_bo; |
| 100 | void *fw_vram_usage_va; | 93 | void *fw_vram_usage_va; |
| 94 | |||
| 95 | /* PAGE_SIZE'd BO for process memory r/w over SDMA. */ | ||
| 96 | struct amdgpu_bo *sdma_access_bo; | ||
| 97 | void *sdma_access_ptr; | ||
| 101 | }; | 98 | }; |
| 102 | 99 | ||
| 103 | struct amdgpu_copy_mem { | 100 | struct amdgpu_copy_mem { |
| @@ -114,8 +111,7 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev); · 10 unmodified lines | |||
| 114 | void amdgpu_vram_mgr_fini(struct amdgpu_device *adev); | 111 | void amdgpu_vram_mgr_fini(struct amdgpu_device *adev); |
| 115 | 112 | ||
| 116 | bool amdgpu_gtt_mgr_has_gart_addr(struct ttm_resource *mem); | 113 | bool amdgpu_gtt_mgr_has_gart_addr(struct ttm_resource *mem); |
| 117 | uint64_t amdgpu_gtt_mgr_usage(struct amdgpu_gtt_mgr *mgr); | 114 | void amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr); |
| 118 | int amdgpu_gtt_mgr_recover(struct amdgpu_gtt_mgr *mgr); | ||
| 119 | 115 | ||
| 120 | uint64_t amdgpu_preempt_mgr_usage(struct ttm_resource_manager *man); | 116 | uint64_t amdgpu_preempt_mgr_usage(struct ttm_resource_manager *man); |
| 121 | 117 | ||
| @@ -129,7 +125,6 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev, · 7 unmodified lines | |||
| 129 | void amdgpu_vram_mgr_free_sgt(struct device *dev, | 125 | void amdgpu_vram_mgr_free_sgt(struct device *dev, |
| 130 | enum dma_data_direction dir, | 126 | enum dma_data_direction dir, |
| 131 | struct sg_table *sgt); | 127 | struct sg_table *sgt); |
| 132 | uint64_t amdgpu_vram_mgr_usage(struct amdgpu_vram_mgr *mgr); | ||
| 133 | uint64_t amdgpu_vram_mgr_vis_usage(struct amdgpu_vram_mgr *mgr); | 128 | uint64_t amdgpu_vram_mgr_vis_usage(struct amdgpu_vram_mgr *mgr); |
| 134 | int amdgpu_vram_mgr_reserve_range(struct amdgpu_vram_mgr *mgr, | 129 | int amdgpu_vram_mgr_reserve_range(struct amdgpu_vram_mgr *mgr, |
| 135 | uint64_t start, uint64_t size); | 130 | uint64_t start, uint64_t size); |
| @@ -158,7 +153,7 @@ int amdgpu_fill_buffer(struct amdgpu_bo *bo, · 22 unmodified lines | |||
| 158 | struct dma_fence **fence); | 153 | struct dma_fence **fence); |
| 159 | 154 | ||
| 160 | int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo); | 155 | int amdgpu_ttm_alloc_gart(struct ttm_buffer_object *bo); |
| 161 | int amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo); | 156 | void amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo); |
| 162 | uint64_t amdgpu_ttm_domain_start(struct amdgpu_device *adev, uint32_t type); | 157 | uint64_t amdgpu_ttm_domain_start(struct amdgpu_device *adev, uint32_t type); |
| 163 | 158 | ||
| 164 | #if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR) | 159 | #if IS_ENABLED(CONFIG_DRM_AMDGPU_USERPTR) |
| @@ -177,6 +172,8 @@ static inline bool amdgpu_ttm_tt_get_user_pages_done(struct ttm_tt *ttm) · 12 unmodified lines | |||
| 177 | #endif | 172 | #endif |
| 178 | 173 | ||
| 179 | void amdgpu_ttm_tt_set_user_pages(struct ttm_tt *ttm, struct page **pages); | 174 | void amdgpu_ttm_tt_set_user_pages(struct ttm_tt *ttm, struct page **pages); |
| 175 | int amdgpu_ttm_tt_get_userptr(const struct ttm_buffer_object *tbo, | ||
| 176 | uint64_t *user_addr); | ||
| 180 | int amdgpu_ttm_tt_set_userptr(struct ttm_buffer_object *bo, | 177 | int amdgpu_ttm_tt_set_userptr(struct ttm_buffer_object *bo, |
| 181 | uint64_t addr, uint32_t flags); | 178 | uint64_t addr, uint32_t flags); |
| 182 | bool amdgpu_ttm_tt_has_userptr(struct ttm_tt *ttm); | 179 | bool amdgpu_ttm_tt_has_userptr(struct ttm_tt *ttm); |
| @@ -378,6 +378,7 @@ enum AMDGPU_UCODE_ID { · 377 unmodified lines | |||
| 378 | AMDGPU_UCODE_ID_VCN0_RAM, | 378 | AMDGPU_UCODE_ID_VCN0_RAM, |
| 379 | AMDGPU_UCODE_ID_VCN1_RAM, | 379 | AMDGPU_UCODE_ID_VCN1_RAM, |
| 380 | AMDGPU_UCODE_ID_DMCUB, | 380 | AMDGPU_UCODE_ID_DMCUB, |
| 381 | AMDGPU_UCODE_ID_CAP, | ||
| 381 | AMDGPU_UCODE_ID_MAXIMUM, | 382 | AMDGPU_UCODE_ID_MAXIMUM, |
| 382 | }; | 383 | }; |
| 383 | 384 | ||
| @@ -21,7 +21,7 @@ · 20 unmodified lines | |||
| 21 | * | 21 | * |
| 22 | */ | 22 | */ |
| 23 | 23 | ||
| 24 | #include "amdgpu_ras.h" | 24 | #include "amdgpu.h" |
| 25 | 25 | ||
| 26 | static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, | 26 | static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, |
| 27 | void *ras_error_status, | 27 | void *ras_error_status, |
| @@ -33,14 +33,14 @@ static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, · 5 unmodified lines | |||
| 33 | int ret = 0; | 33 | int ret = 0; |
| 34 | 34 | ||
| 35 | kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); | 35 | kgd2kfd_set_sram_ecc_flag(adev->kfd.dev); |
| 36 | ret = smu_get_ecc_info(&adev->smu, (void *)&(con->umc_ecc)); | 36 | ret = amdgpu_dpm_get_ecc_info(adev, (void *)&(con->umc_ecc)); |
| 37 | if (ret == -EOPNOTSUPP) { | 37 | if (ret == -EOPNOTSUPP) { |
| 38 | if (adev->umc.ras_funcs && | 38 | if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && |
| 39 | adev->umc.ras_funcs->query_ras_error_count) | 39 | adev->umc.ras->ras_block.hw_ops->query_ras_error_count) |
| 40 | adev->umc.ras_funcs->query_ras_error_count(adev, ras_error_status); | 40 | adev->umc.ras->ras_block.hw_ops->query_ras_error_count(adev, ras_error_status); |
| 41 | 41 | ||
| 42 | if (adev->umc.ras_funcs && | 42 | if (adev->umc.ras && adev->umc.ras->ras_block.hw_ops && |
| 43 | adev->umc.ras_funcs->query_ras_error_address && | 43 | adev->umc.ras->ras_block.hw_ops->query_ras_error_address && |
| 44 | adev->umc.max_ras_err_cnt_per_query) { | 44 | adev->umc.max_ras_err_cnt_per_query) { |
| 45 | err_data->err_addr = | 45 | err_data->err_addr = |
| 46 | kcalloc(adev->umc.max_ras_err_cnt_per_query, | 46 | kcalloc(adev->umc.max_ras_err_cnt_per_query, |
| @@ -56,15 +56,15 @@ static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, · 9 unmodified lines | |||
| 56 | /* umc query_ras_error_address is also responsible for clearing | 56 | /* umc query_ras_error_address is also responsible for clearing |
| 57 | * error status | 57 | * error status |
| 58 | */ | 58 | */ |
| 59 | adev->umc.ras_funcs->query_ras_error_address(adev, ras_error_status); | 59 | adev->umc.ras->ras_block.hw_ops->query_ras_error_address(adev, ras_error_status); |
| 60 | } | 60 | } |
| 61 | } else if (!ret) { | 61 | } else if (!ret) { |
| 62 | if (adev->umc.ras_funcs && | 62 | if (adev->umc.ras && |
| 63 | adev->umc.ras_funcs->ecc_info_query_ras_error_count) | 63 | adev->umc.ras->ecc_info_query_ras_error_count) |
| 64 | adev->umc.ras_funcs->ecc_info_query_ras_error_count(adev, ras_error_status); | 64 | adev->umc.ras->ecc_info_query_ras_error_count(adev, ras_error_status); |
| 65 | 65 | ||
| 66 | if (adev->umc.ras_funcs && | 66 | if (adev->umc.ras && |
| 67 | adev->umc.ras_funcs->ecc_info_query_ras_error_address && | 67 | adev->umc.ras->ecc_info_query_ras_error_address && |
| 68 | adev->umc.max_ras_err_cnt_per_query) { | 68 | adev->umc.max_ras_err_cnt_per_query) { |
| 69 | err_data->err_addr = | 69 | err_data->err_addr = |
| 70 | kcalloc(adev->umc.max_ras_err_cnt_per_query, | 70 | kcalloc(adev->umc.max_ras_err_cnt_per_query, |
| @@ -80,7 +80,7 @@ static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, · 9 unmodified lines | |||
| 80 | /* umc query_ras_error_address is also responsible for clearing | 80 | /* umc query_ras_error_address is also responsible for clearing |
| 81 | * error status | 81 | * error status |
| 82 | */ | 82 | */ |
| 83 | adev->umc.ras_funcs->ecc_info_query_ras_error_address(adev, ras_error_status); | 83 | adev->umc.ras->ecc_info_query_ras_error_address(adev, ras_error_status); |
| 84 | } | 84 | } |
| 85 | } | 85 | } |
| 86 | 86 | ||
| @@ -96,8 +96,7 @@ static int amdgpu_umc_do_page_retirement(struct amdgpu_device *adev, · 9 unmodified lines | |||
| 96 | err_data->err_addr_cnt); | 96 | err_data->err_addr_cnt); |
| 97 | amdgpu_ras_save_bad_pages(adev); | 97 | amdgpu_ras_save_bad_pages(adev); |
| 98 | 98 | ||
| 99 | if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->send_hbm_bad_pages_num) | 99 | amdgpu_dpm_send_hbm_bad_pages_num(adev, con->eeprom_control.ras_num_recs); |
| 100 | adev->smu.ppt_funcs->send_hbm_bad_pages_num(&adev->smu, con->eeprom_control.ras_num_recs); | ||
| 101 | } | 100 | } |
| 102 | 101 | ||
| 103 | if (reset) | 102 | if (reset) |
| @@ -130,76 +129,44 @@ int amdgpu_umc_poison_handler(struct amdgpu_device *adev, · 26 unmodified lines | |||
| 130 | return ret; | 129 | return ret; |
| 131 | } | 130 | } |
| 132 | 131 | ||
| 133 | static int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | 132 | int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, |
| 134 | void *ras_error_status, | 133 | void *ras_error_status, |
| 135 | struct amdgpu_iv_entry *entry) | 134 | struct amdgpu_iv_entry *entry) |
| 136 | { | 135 | { |
| 137 | return amdgpu_umc_do_page_retirement(adev, ras_error_status, entry, true); | 136 | return amdgpu_umc_do_page_retirement(adev, ras_error_status, entry, true); |
| 138 | } | 137 | } |
| 139 | 138 | ||
| 140 | int amdgpu_umc_ras_late_init(struct amdgpu_device *adev) | 139 | int amdgpu_umc_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block) |
| 141 | { | 140 | { |
| 142 | int r; | 141 | int r; |
| 143 | struct ras_fs_if fs_info = { | ||
| 144 | .sysfs_name = "umc_err_count", | ||
| 145 | }; | ||
| 146 | struct ras_ih_if ih_info = { | ||
| 147 | .cb = amdgpu_umc_process_ras_data_cb, | ||
| 148 | }; | ||
| 149 | |||
| 150 | if (!adev->umc.ras_if) { | ||
| 151 | adev->umc.ras_if = | ||
| 152 | kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | ||
| 153 | if (!adev->umc.ras_if) | ||
| 154 | return -ENOMEM; | ||
| 155 | adev->umc.ras_if->block = AMDGPU_RAS_BLOCK__UMC; | ||
| 156 | adev->umc.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 157 | adev->umc.ras_if->sub_block_index = 0; | ||
| 158 | } | ||
| 159 | ih_info.head = fs_info.head = *adev->umc.ras_if; | ||
| 160 | 142 | ||
| 161 | r = amdgpu_ras_late_init(adev, adev->umc.ras_if, | 143 | r = amdgpu_ras_block_late_init(adev, ras_block); |
| 162 | &fs_info, &ih_info); | ||
| 163 | if (r) | 144 | if (r) |
| 164 | goto free; | 145 | return r; |
| 165 | 146 | ||
| 166 | if (amdgpu_ras_is_supported(adev, adev->umc.ras_if->block)) { | 147 | if (amdgpu_ras_is_supported(adev, ras_block->block)) { |
| 167 | r = amdgpu_irq_get(adev, &adev->gmc.ecc_irq, 0); | 148 | r = amdgpu_irq_get(adev, &adev->gmc.ecc_irq, 0); |
| 168 | if (r) | 149 | if (r) |
| 169 | goto late_fini; | 150 | goto late_fini; |
| 170 | } else { | ||
| 171 | r = 0; | ||
| 172 | goto free; | ||
| 173 | } | 151 | } |
| 174 | 152 | ||
| 175 | /* ras init of specific umc version */ | 153 | /* ras init of specific umc version */ |
| 176 | if (adev->umc.ras_funcs && | 154 | if (adev->umc.ras && |
| 177 | adev->umc.ras_funcs->err_cnt_init) | 155 | adev->umc.ras->err_cnt_init) |
| 178 | adev->umc.ras_funcs->err_cnt_init(adev); | 156 | adev->umc.ras->err_cnt_init(adev); |
| 179 | 157 | ||
| 180 | return 0; | 158 | return 0; |
| 181 | 159 | ||
| 182 | late_fini: | 160 | late_fini: |
| 183 | amdgpu_ras_late_fini(adev, adev->umc.ras_if, &ih_info); | 161 | amdgpu_ras_block_late_fini(adev, ras_block); |
| 184 | free: | ||
| 185 | kfree(adev->umc.ras_if); | ||
| 186 | adev->umc.ras_if = NULL; | ||
| 187 | return r; | 162 | return r; |
| 188 | } | 163 | } |
| 189 | 164 | ||
| 190 | void amdgpu_umc_ras_fini(struct amdgpu_device *adev) | 165 | void amdgpu_umc_ras_fini(struct amdgpu_device *adev) |
| 191 | { | 166 | { |
| 192 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__UMC) && | 167 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__UMC) && |
| 193 | adev->umc.ras_if) { | 168 | adev->umc.ras_if) |
| 194 | struct ras_common_if *ras_if = adev->umc.ras_if; | 169 | amdgpu_ras_block_late_fini(adev, adev->umc.ras_if); |
| 195 | struct ras_ih_if ih_info = { | ||
| 196 | .head = *ras_if, | ||
| 197 | .cb = amdgpu_umc_process_ras_data_cb, | ||
| 198 | }; | ||
| 199 | |||
| 200 | amdgpu_ras_late_fini(adev, ras_if, &ih_info); | ||
| 201 | kfree(ras_if); | ||
| 202 | } | ||
| 203 | } | 170 | } |
| 204 | 171 | ||
| 205 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, | 172 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, |
| @@ -219,3 +186,24 @@ int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, · 13 unmodified lines | |||
| 219 | amdgpu_ras_interrupt_dispatch(adev, &ih_data); | 186 | amdgpu_ras_interrupt_dispatch(adev, &ih_data); |
| 220 | return 0; | 187 | return 0; |
| 221 | } | 188 | } |
| 189 | |||
| 190 | void amdgpu_umc_fill_error_record(struct ras_err_data *err_data, | ||
| 191 | uint64_t err_addr, | ||
| 192 | uint64_t retired_page, | ||
| 193 | uint32_t channel_index, | ||
| 194 | uint32_t umc_inst) | ||
| 195 | { | ||
| 196 | struct eeprom_table_record *err_rec = | ||
| 197 | &err_data->err_addr[err_data->err_addr_cnt]; | ||
| 198 | |||
| 199 | err_rec->address = err_addr; | ||
| 200 | /* page frame address is saved */ | ||
| 201 | err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT; | ||
| 202 | err_rec->ts = (uint64_t)ktime_get_real_seconds(); | ||
| 203 | err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE; | ||
| 204 | err_rec->cu = 0; | ||
| 205 | err_rec->mem_channel = channel_index; | ||
| 206 | err_rec->mcumc_id = umc_inst; | ||
| 207 | |||
| 208 | err_data->err_addr_cnt++; | ||
| 209 | } | ||
| @@ -20,6 +20,7 @@ · 19 unmodified lines | |||
| 20 | */ | 20 | */ |
| 21 | #ifndef __AMDGPU_UMC_H__ | 21 | #ifndef __AMDGPU_UMC_H__ |
| 22 | #define __AMDGPU_UMC_H__ | 22 | #define __AMDGPU_UMC_H__ |
| 23 | #include "amdgpu_ras.h" | ||
| 23 | 24 | ||
| 24 | /* | 25 | /* |
| 25 | * (addr / 256) * 4096, the higher 26 bits in ErrorAddr | 26 | * (addr / 256) * 4096, the higher 26 bits in ErrorAddr |
| @@ -40,14 +41,9 @@ · 14 unmodified lines | |||
| 40 | #define LOOP_UMC_CH_INST(ch_inst) for ((ch_inst) = 0; (ch_inst) < adev->umc.channel_inst_num; (ch_inst)++) | 41 | #define LOOP_UMC_CH_INST(ch_inst) for ((ch_inst) = 0; (ch_inst) < adev->umc.channel_inst_num; (ch_inst)++) |
| 41 | #define LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) LOOP_UMC_INST((umc_inst)) LOOP_UMC_CH_INST((ch_inst)) | 42 | #define LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) LOOP_UMC_INST((umc_inst)) LOOP_UMC_CH_INST((ch_inst)) |
| 42 | 43 | ||
| 43 | struct amdgpu_umc_ras_funcs { | 44 | struct amdgpu_umc_ras { |
| 45 | struct amdgpu_ras_block_object ras_block; | ||
| 44 | void (*err_cnt_init)(struct amdgpu_device *adev); | 46 | void (*err_cnt_init)(struct amdgpu_device *adev); |
| 45 | int (*ras_late_init)(struct amdgpu_device *adev); | ||
| 46 | void (*ras_fini)(struct amdgpu_device *adev); | ||
| 47 | void (*query_ras_error_count)(struct amdgpu_device *adev, | ||
| 48 | void *ras_error_status); | ||
| 49 | void (*query_ras_error_address)(struct amdgpu_device *adev, | ||
| 50 | void *ras_error_status); | ||
| 51 | bool (*query_ras_poison_mode)(struct amdgpu_device *adev); | 47 | bool (*query_ras_poison_mode)(struct amdgpu_device *adev); |
| 52 | void (*ecc_info_query_ras_error_count)(struct amdgpu_device *adev, | 48 | void (*ecc_info_query_ras_error_count)(struct amdgpu_device *adev, |
| 53 | void *ras_error_status); | 49 | void *ras_error_status); |
| @@ -73,10 +69,10 @@ struct amdgpu_umc { · 19 unmodified lines | |||
| 73 | struct ras_common_if *ras_if; | 69 | struct ras_common_if *ras_if; |
| 74 | 70 | ||
| 75 | const struct amdgpu_umc_funcs *funcs; | 71 | const struct amdgpu_umc_funcs *funcs; |
| 76 | const struct amdgpu_umc_ras_funcs *ras_funcs; | 72 | struct amdgpu_umc_ras *ras; |
| 77 | }; | 73 | }; |
| 78 | 74 | ||
| 79 | int amdgpu_umc_ras_late_init(struct amdgpu_device *adev); | 75 | int amdgpu_umc_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block); |
| 80 | void amdgpu_umc_ras_fini(struct amdgpu_device *adev); | 76 | void amdgpu_umc_ras_fini(struct amdgpu_device *adev); |
| 81 | int amdgpu_umc_poison_handler(struct amdgpu_device *adev, | 77 | int amdgpu_umc_poison_handler(struct amdgpu_device *adev, |
| 82 | void *ras_error_status, | 78 | void *ras_error_status, |
| @@ -84,4 +80,13 @@ int amdgpu_umc_poison_handler(struct amdgpu_device *adev, · 1 unmodified line | |||
| 84 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, | 80 | int amdgpu_umc_process_ecc_irq(struct amdgpu_device *adev, |
| 85 | struct amdgpu_irq_src *source, | 81 | struct amdgpu_irq_src *source, |
| 86 | struct amdgpu_iv_entry *entry); | 82 | struct amdgpu_iv_entry *entry); |
| 83 | void amdgpu_umc_fill_error_record(struct ras_err_data *err_data, | ||
| 84 | uint64_t err_addr, | ||
| 85 | uint64_t retired_page, | ||
| 86 | uint32_t channel_index, | ||
| 87 | uint32_t umc_inst); | ||
| 88 | |||
| 89 | int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev, | ||
| 90 | void *ras_error_status, | ||
| 91 | struct amdgpu_iv_entry *entry); | ||
| 87 | #endif | 92 | #endif |
| @@ -834,6 +834,7 @@ static int amdgpu_uvd_cs_msg(struct amdgpu_uvd_cs_ctx *ctx, · 833 unmodified lines | |||
| 834 | handle = msg[2]; | 834 | handle = msg[2]; |
| 835 | 835 | ||
| 836 | if (handle == 0) { | 836 | if (handle == 0) { |
| 837 | amdgpu_bo_kunmap(bo); | ||
| 837 | DRM_ERROR("Invalid UVD handle!\n"); | 838 | DRM_ERROR("Invalid UVD handle!\n"); |
| 838 | return -EINVAL; | 839 | return -EINVAL; |
| 839 | } | 840 | } |
| @@ -892,6 +893,7 @@ static int amdgpu_uvd_cs_msg(struct amdgpu_uvd_cs_ctx *ctx, · 52 unmodified lines | |||
| 892 | DRM_ERROR("Illegal UVD message type (%d)!\n", msg_type); | 893 | DRM_ERROR("Illegal UVD message type (%d)!\n", msg_type); |
| 893 | } | 894 | } |
| 894 | 895 | ||
| 896 | amdgpu_bo_kunmap(bo); | ||
| 895 | return -EINVAL; | 897 | return -EINVAL; |
| 896 | } | 898 | } |
| 897 | 899 | ||
| @@ -51,6 +51,7 @@ · 50 unmodified lines | |||
| 51 | #define FIRMWARE_ALDEBARAN "amdgpu/aldebaran_vcn.bin" | 51 | #define FIRMWARE_ALDEBARAN "amdgpu/aldebaran_vcn.bin" |
| 52 | #define FIRMWARE_BEIGE_GOBY "amdgpu/beige_goby_vcn.bin" | 52 | #define FIRMWARE_BEIGE_GOBY "amdgpu/beige_goby_vcn.bin" |
| 53 | #define FIRMWARE_YELLOW_CARP "amdgpu/yellow_carp_vcn.bin" | 53 | #define FIRMWARE_YELLOW_CARP "amdgpu/yellow_carp_vcn.bin" |
| 54 | #define FIRMWARE_VCN_3_1_2 "amdgpu/vcn_3_1_2_vcn.bin" | ||
| 54 | 55 | ||
| 55 | MODULE_FIRMWARE(FIRMWARE_RAVEN); | 56 | MODULE_FIRMWARE(FIRMWARE_RAVEN); |
| 56 | MODULE_FIRMWARE(FIRMWARE_PICASSO); | 57 | MODULE_FIRMWARE(FIRMWARE_PICASSO); |
| @@ -68,6 +69,7 @@ MODULE_FIRMWARE(FIRMWARE_VANGOGH); · 11 unmodified lines | |||
| 68 | MODULE_FIRMWARE(FIRMWARE_DIMGREY_CAVEFISH); | 69 | MODULE_FIRMWARE(FIRMWARE_DIMGREY_CAVEFISH); |
| 69 | MODULE_FIRMWARE(FIRMWARE_BEIGE_GOBY); | 70 | MODULE_FIRMWARE(FIRMWARE_BEIGE_GOBY); |
| 70 | MODULE_FIRMWARE(FIRMWARE_YELLOW_CARP); | 71 | MODULE_FIRMWARE(FIRMWARE_YELLOW_CARP); |
| 72 | MODULE_FIRMWARE(FIRMWARE_VCN_3_1_2); | ||
| 71 | 73 | ||
| 72 | static void amdgpu_vcn_idle_work_handler(struct work_struct *work); | 74 | static void amdgpu_vcn_idle_work_handler(struct work_struct *work); |
| 73 | 75 | ||
| @@ -165,6 +167,12 @@ int amdgpu_vcn_sw_init(struct amdgpu_device *adev) · 91 unmodified lines | |||
| 165 | (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG)) | 167 | (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG)) |
| 166 | adev->vcn.indirect_sram = true; | 168 | adev->vcn.indirect_sram = true; |
| 167 | break; | 169 | break; |
| 170 | case IP_VERSION(3, 1, 2): | ||
| 171 | fw_name = FIRMWARE_VCN_3_1_2; | ||
| 172 | if ((adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) && | ||
| 173 | (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG)) | ||
| 174 | adev->vcn.indirect_sram = true; | ||
| 175 | break; | ||
| 168 | default: | 176 | default: |
| 169 | return -EINVAL; | 177 | return -EINVAL; |
| 170 | } | 178 | } |
| @@ -575,8 +575,10 @@ static int amdgpu_virt_write_vf2pf_data(struct amdgpu_device *adev) · 574 unmodified lines | |||
| 575 | vf2pf_info->driver_cert = 0; | 575 | vf2pf_info->driver_cert = 0; |
| 576 | vf2pf_info->os_info.all = 0; | 576 | vf2pf_info->os_info.all = 0; |
| 577 | 577 | ||
| 578 | vf2pf_info->fb_usage = amdgpu_vram_mgr_usage(&adev->mman.vram_mgr) >> 20; | 578 | vf2pf_info->fb_usage = |
| 579 | vf2pf_info->fb_vis_usage = amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr) >> 20; | 579 | ttm_resource_manager_usage(&adev->mman.vram_mgr.manager) >> 20; |
| 580 | vf2pf_info->fb_vis_usage = | ||
| 581 | amdgpu_vram_mgr_vis_usage(&adev->mman.vram_mgr) >> 20; | ||
| 580 | vf2pf_info->fb_size = adev->gmc.real_vram_size >> 20; | 582 | vf2pf_info->fb_size = adev->gmc.real_vram_size >> 20; |
| 581 | vf2pf_info->fb_vis_size = adev->gmc.visible_vram_size >> 20; | 583 | vf2pf_info->fb_vis_size = adev->gmc.visible_vram_size >> 20; |
| 582 | 584 | ||
| @@ -820,3 +822,148 @@ void amdgpu_virt_update_sriov_video_codec(struct amdgpu_device *adev, · 237 unmodified lines | |||
| 820 | } | 822 | } |
| 821 | } | 823 | } |
| 822 | } | 824 | } |
| 825 | |||
| 826 | static bool amdgpu_virt_get_rlcg_reg_access_flag(struct amdgpu_device *adev, | ||
| 827 | u32 acc_flags, u32 hwip, | ||
| 828 | bool write, u32 *rlcg_flag) | ||
| 829 | { | ||
| 830 | bool ret = false; | ||
| 831 | |||
| 832 | switch (hwip) { | ||
| 833 | case GC_HWIP: | ||
| 834 | if (amdgpu_sriov_reg_indirect_gc(adev)) { | ||
| 835 | *rlcg_flag = | ||
| 836 | write ? AMDGPU_RLCG_GC_WRITE : AMDGPU_RLCG_GC_READ; | ||
| 837 | ret = true; | ||
| 838 | /* only in new version, AMDGPU_REGS_NO_KIQ and | ||
| 839 | * AMDGPU_REGS_RLC are enabled simultaneously */ | ||
| 840 | } else if ((acc_flags & AMDGPU_REGS_RLC) && | ||
| 841 | !(acc_flags & AMDGPU_REGS_NO_KIQ) && write) { | ||
| 842 | *rlcg_flag = AMDGPU_RLCG_GC_WRITE_LEGACY; | ||
| 843 | ret = true; | ||
| 844 | } | ||
| 845 | break; | ||
| 846 | case MMHUB_HWIP: | ||
| 847 | if (amdgpu_sriov_reg_indirect_mmhub(adev) && | ||
| 848 | (acc_flags & AMDGPU_REGS_RLC) && write) { | ||
| 849 | *rlcg_flag = AMDGPU_RLCG_MMHUB_WRITE; | ||
| 850 | ret = true; | ||
| 851 | } | ||
| 852 | break; | ||
| 853 | default: | ||
| 854 | break; | ||
| 855 | } | ||
| 856 | return ret; | ||
| 857 | } | ||
| 858 | |||
| 859 | static u32 amdgpu_virt_rlcg_reg_rw(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag) | ||
| 860 | { | ||
| 861 | struct amdgpu_rlcg_reg_access_ctrl *reg_access_ctrl; | ||
| 862 | uint32_t timeout = 50000; | ||
| 863 | uint32_t i, tmp; | ||
| 864 | uint32_t ret = 0; | ||
| 865 | static void *scratch_reg0; | ||
| 866 | static void *scratch_reg1; | ||
| 867 | static void *scratch_reg2; | ||
| 868 | static void *scratch_reg3; | ||
| 869 | static void *spare_int; | ||
| 870 | |||
| 871 | if (!adev->gfx.rlc.rlcg_reg_access_supported) { | ||
| 872 | dev_err(adev->dev, | ||
| 873 | "indirect registers access through rlcg is not available\n"); | ||
| 874 | return 0; | ||
| 875 | } | ||
| 876 | |||
| 877 | reg_access_ctrl = &adev->gfx.rlc.reg_access_ctrl; | ||
| 878 | scratch_reg0 = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->scratch_reg0; | ||
| 879 | scratch_reg1 = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->scratch_reg1; | ||
| 880 | scratch_reg2 = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->scratch_reg2; | ||
| 881 | scratch_reg3 = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->scratch_reg3; | ||
| 882 | if (reg_access_ctrl->spare_int) | ||
| 883 | spare_int = (void __iomem *)adev->rmmio + 4 * reg_access_ctrl->spare_int; | ||
| 884 | |||
| 885 | if (offset == reg_access_ctrl->grbm_cntl) { | ||
| 886 | /* if the target reg offset is grbm_cntl, write to scratch_reg2 */ | ||
| 887 | writel(v, scratch_reg2); | ||
| 888 | writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); | ||
| 889 | } else if (offset == reg_access_ctrl->grbm_idx) { | ||
| 890 | /* if the target reg offset is grbm_idx, write to scratch_reg3 */ | ||
| 891 | writel(v, scratch_reg3); | ||
| 892 | writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); | ||
| 893 | } else { | ||
| 894 | /* | ||
| 895 | * SCRATCH_REG0 = read/write value | ||
| 896 | * SCRATCH_REG1[30:28] = command | ||
| 897 | * SCRATCH_REG1[19:0] = address in dword | ||
| 898 | * SCRATCH_REG1[26:24] = Error reporting | ||
| 899 | */ | ||
| 900 | writel(v, scratch_reg0); | ||
| 901 | writel((offset | flag), scratch_reg1); | ||
| 902 | if (reg_access_ctrl->spare_int) | ||
| 903 | writel(1, spare_int); | ||
| 904 | |||
| 905 | for (i = 0; i < timeout; i++) { | ||
| 906 | tmp = readl(scratch_reg1); | ||
| 907 | if (!(tmp & AMDGPU_RLCG_SCRATCH1_ADDRESS_MASK)) | ||
| 908 | break; | ||
| 909 | udelay(10); | ||
| 910 | } | ||
| 911 | |||
| 912 | if (i >= timeout) { | ||
| 913 | if (amdgpu_sriov_rlcg_error_report_enabled(adev)) { | ||
| 914 | if (tmp & AMDGPU_RLCG_VFGATE_DISABLED) { | ||
| 915 | dev_err(adev->dev, | ||
| 916 | "vfgate is disabled, rlcg failed to program reg: 0x%05x\n", offset); | ||
| 917 | } else if (tmp & AMDGPU_RLCG_WRONG_OPERATION_TYPE) { | ||
| 918 | dev_err(adev->dev, | ||
| 919 | "wrong operation type, rlcg failed to program reg: 0x%05x\n", offset); | ||
| 920 | } else if (tmp & AMDGPU_RLCG_REG_NOT_IN_RANGE) { | ||
| 921 | dev_err(adev->dev, | ||
| 922 | "regiser is not in range, rlcg failed to program reg: 0x%05x\n", offset); | ||
| 923 | } else { | ||
| 924 | dev_err(adev->dev, | ||
| 925 | "unknown error type, rlcg failed to program reg: 0x%05x\n", offset); | ||
| 926 | } | ||
| 927 | } else { | ||
| 928 | dev_err(adev->dev, | ||
| 929 | "timeout: rlcg faled to program reg: 0x%05x\n", offset); | ||
| 930 | } | ||
| 931 | } | ||
| 932 | } | ||
| 933 | |||
| 934 | ret = readl(scratch_reg0); | ||
| 935 | return ret; | ||
| 936 | } | ||
| 937 | |||
| 938 | void amdgpu_sriov_wreg(struct amdgpu_device *adev, | ||
| 939 | u32 offset, u32 value, | ||
| 940 | u32 acc_flags, u32 hwip) | ||
| 941 | { | ||
| 942 | u32 rlcg_flag; | ||
| 943 | |||
| 944 | if (!amdgpu_sriov_runtime(adev) && | ||
| 945 | amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, true, &rlcg_flag)) { | ||
| 946 | amdgpu_virt_rlcg_reg_rw(adev, offset, value, rlcg_flag); | ||
| 947 | return; | ||
| 948 | } | ||
| 949 | |||
| 950 | if (acc_flags & AMDGPU_REGS_NO_KIQ) | ||
| 951 | WREG32_NO_KIQ(offset, value); | ||
| 952 | else | ||
| 953 | WREG32(offset, value); | ||
| 954 | } | ||
| 955 | |||
| 956 | u32 amdgpu_sriov_rreg(struct amdgpu_device *adev, | ||
| 957 | u32 offset, u32 acc_flags, u32 hwip) | ||
| 958 | { | ||
| 959 | u32 rlcg_flag; | ||
| 960 | |||
| 961 | if (!amdgpu_sriov_runtime(adev) && | ||
| 962 | amdgpu_virt_get_rlcg_reg_access_flag(adev, acc_flags, hwip, false, &rlcg_flag)) | ||
| 963 | return amdgpu_virt_rlcg_reg_rw(adev, offset, 0, rlcg_flag); | ||
| 964 | |||
| 965 | if (acc_flags & AMDGPU_REGS_NO_KIQ) | ||
| 966 | return RREG32_NO_KIQ(offset); | ||
| 967 | else | ||
| 968 | return RREG32(offset); | ||
| 969 | } | ||
| @@ -32,6 +32,19 @@ · 31 unmodified lines | |||
| 32 | #define AMDGPU_PASSTHROUGH_MODE (1 << 3) /* thw whole GPU is pass through for VM */ | 32 | #define AMDGPU_PASSTHROUGH_MODE (1 << 3) /* thw whole GPU is pass through for VM */ |
| 33 | #define AMDGPU_SRIOV_CAPS_RUNTIME (1 << 4) /* is out of full access mode */ | 33 | #define AMDGPU_SRIOV_CAPS_RUNTIME (1 << 4) /* is out of full access mode */ |
| 34 | 34 | ||
| 35 | /* flags for indirect register access path supported by rlcg for sriov */ | ||
| 36 | #define AMDGPU_RLCG_GC_WRITE_LEGACY (0x8 << 28) | ||
| 37 | #define AMDGPU_RLCG_GC_WRITE (0x0 << 28) | ||
| 38 | #define AMDGPU_RLCG_GC_READ (0x1 << 28) | ||
| 39 | #define AMDGPU_RLCG_MMHUB_WRITE (0x2 << 28) | ||
| 40 | |||
| 41 | /* error code for indirect register access path supported by rlcg for sriov */ | ||
| 42 | #define AMDGPU_RLCG_VFGATE_DISABLED 0x4000000 | ||
| 43 | #define AMDGPU_RLCG_WRONG_OPERATION_TYPE 0x2000000 | ||
| 44 | #define AMDGPU_RLCG_REG_NOT_IN_RANGE 0x1000000 | ||
| 45 | |||
| 46 | #define AMDGPU_RLCG_SCRATCH1_ADDRESS_MASK 0xFFFFF | ||
| 47 | |||
| 35 | /* all asic after AI use this offset */ | 48 | /* all asic after AI use this offset */ |
| 36 | #define mmRCC_IOV_FUNC_IDENTIFIER 0xDE5 | 49 | #define mmRCC_IOV_FUNC_IDENTIFIER 0xDE5 |
| 37 | /* tonga/fiji use this offset */ | 50 | /* tonga/fiji use this offset */ |
| @@ -275,13 +288,18 @@ struct amdgpu_video_codec_info; · 237 unmodified lines | |||
| 275 | (amdgpu_sriov_vf((adev)) && \ | 288 | (amdgpu_sriov_vf((adev)) && \ |
| 276 | ((adev)->virt.reg_access & (AMDGIM_FEATURE_GC_REG_RLC_EN))) | 289 | ((adev)->virt.reg_access & (AMDGIM_FEATURE_GC_REG_RLC_EN))) |
| 277 | 290 | ||
| 291 | #define amdgpu_sriov_rlcg_error_report_enabled(adev) \ | ||
| 292 | (amdgpu_sriov_reg_indirect_mmhub(adev) || amdgpu_sriov_reg_indirect_gc(adev)) | ||
| 293 | |||
| 278 | #define amdgpu_passthrough(adev) \ | 294 | #define amdgpu_passthrough(adev) \ |
| 279 | ((adev)->virt.caps & AMDGPU_PASSTHROUGH_MODE) | 295 | ((adev)->virt.caps & AMDGPU_PASSTHROUGH_MODE) |
| 280 | 296 | ||
| 281 | static inline bool is_virtual_machine(void) | 297 | static inline bool is_virtual_machine(void) |
| 282 | { | 298 | { |
| 283 | #ifdef CONFIG_X86 | 299 | #if defined(CONFIG_X86) |
| 284 | return boot_cpu_has(X86_FEATURE_HYPERVISOR); | 300 | return boot_cpu_has(X86_FEATURE_HYPERVISOR); |
| 301 | #elif defined(CONFIG_ARM64) | ||
| 302 | return !is_kernel_in_hyp_mode(); | ||
| 285 | #else | 303 | #else |
| 286 | return false; | 304 | return false; |
| 287 | #endif | 305 | #endif |
| @@ -293,7 +311,6 @@ static inline bool is_virtual_machine(void) · 5 unmodified lines | |||
| 293 | ((!amdgpu_in_reset(adev)) && adev->virt.tdr_debug) | 311 | ((!amdgpu_in_reset(adev)) && adev->virt.tdr_debug) |
| 294 | #define amdgpu_sriov_is_normal(adev) \ | 312 | #define amdgpu_sriov_is_normal(adev) \ |
| 295 | ((!amdgpu_in_reset(adev)) && (!adev->virt.tdr_debug)) | 313 | ((!amdgpu_in_reset(adev)) && (!adev->virt.tdr_debug)) |
| 296 | |||
| 297 | bool amdgpu_virt_mmio_blocked(struct amdgpu_device *adev); | 314 | bool amdgpu_virt_mmio_blocked(struct amdgpu_device *adev); |
| 298 | void amdgpu_virt_init_setting(struct amdgpu_device *adev); | 315 | void amdgpu_virt_init_setting(struct amdgpu_device *adev); |
| 299 | void amdgpu_virt_kiq_reg_write_reg_wait(struct amdgpu_device *adev, | 316 | void amdgpu_virt_kiq_reg_write_reg_wait(struct amdgpu_device *adev, |
| @@ -321,4 +338,9 @@ enum amdgpu_sriov_vf_mode amdgpu_virt_get_sriov_vf_mode(struct amdgpu_device *ad · 21 unmodified lines | |||
| 321 | void amdgpu_virt_update_sriov_video_codec(struct amdgpu_device *adev, | 338 | void amdgpu_virt_update_sriov_video_codec(struct amdgpu_device *adev, |
| 322 | struct amdgpu_video_codec_info *encode, uint32_t encode_array_size, | 339 | struct amdgpu_video_codec_info *encode, uint32_t encode_array_size, |
| 323 | struct amdgpu_video_codec_info *decode, uint32_t decode_array_size); | 340 | struct amdgpu_video_codec_info *decode, uint32_t decode_array_size); |
| 341 | void amdgpu_sriov_wreg(struct amdgpu_device *adev, | ||
| 342 | u32 offset, u32 value, | ||
| 343 | u32 acc_flags, u32 hwip); | ||
| 344 | u32 amdgpu_sriov_rreg(struct amdgpu_device *adev, | ||
| 345 | u32 offset, u32 acc_flags, u32 hwip); | ||
| 324 | #endif | 346 | #endif |
| @@ -391,7 +391,6 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev, · 390 unmodified lines | |||
| 391 | int index) | 391 | int index) |
| 392 | { | 392 | { |
| 393 | struct drm_plane *plane; | 393 | struct drm_plane *plane; |
| 394 | uint64_t modifiers[] = {DRM_FORMAT_MOD_LINEAR, DRM_FORMAT_MOD_INVALID}; | ||
| 395 | int ret; | 394 | int ret; |
| 396 | 395 | ||
| 397 | plane = kzalloc(sizeof(*plane), GFP_KERNEL); | 396 | plane = kzalloc(sizeof(*plane), GFP_KERNEL); |
| @@ -402,7 +401,7 @@ static struct drm_plane *amdgpu_vkms_plane_init(struct drm_device *dev, · 4 unmodified lines | |||
| 402 | &amdgpu_vkms_plane_funcs, | 401 | &amdgpu_vkms_plane_funcs, |
| 403 | amdgpu_vkms_formats, | 402 | amdgpu_vkms_formats, |
| 404 | ARRAY_SIZE(amdgpu_vkms_formats), | 403 | ARRAY_SIZE(amdgpu_vkms_formats), |
| 405 | modifiers, type, NULL); | 404 | NULL, type, NULL); |
| 406 | if (ret) { | 405 | if (ret) { |
| 407 | kfree(plane); | 406 | kfree(plane); |
| 408 | return ERR_PTR(ret); | 407 | return ERR_PTR(ret); |
| @@ -375,6 +375,8 @@ static void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, · 374 unmodified lines | |||
| 375 | if (bo->tbo.base.resv != vm->root.bo->tbo.base.resv) | 375 | if (bo->tbo.base.resv != vm->root.bo->tbo.base.resv) |
| 376 | return; | 376 | return; |
| 377 | 377 | ||
| 378 | dma_resv_assert_held(vm->root.bo->tbo.base.resv); | ||
| 379 | |||
| 378 | vm->bulk_moveable = false; | 380 | vm->bulk_moveable = false; |
| 379 | if (bo->tbo.type == ttm_bo_type_kernel && bo->parent) | 381 | if (bo->tbo.type == ttm_bo_type_kernel && bo->parent) |
| 380 | amdgpu_vm_bo_relocated(base); | 382 | amdgpu_vm_bo_relocated(base); |
| @@ -768,11 +770,16 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, · 387 unmodified lines | |||
| 768 | * Check if all VM PDs/PTs are ready for updates | 770 | * Check if all VM PDs/PTs are ready for updates |
| 769 | * | 771 | * |
| 770 | * Returns: | 772 | * Returns: |
| 771 | * True if eviction list is empty. | 773 | * True if VM is not evicting. |
| 772 | */ | 774 | */ |
| 773 | bool amdgpu_vm_ready(struct amdgpu_vm *vm) | 775 | bool amdgpu_vm_ready(struct amdgpu_vm *vm) |
| 774 | { | 776 | { |
| 775 | return list_empty(&vm->evicted); | 777 | bool ret; |
| 778 | |||
| 779 | amdgpu_vm_eviction_lock(vm); | ||
| 780 | ret = !vm->evicting; | ||
| 781 | amdgpu_vm_eviction_unlock(vm); | ||
| 782 | return ret; | ||
| 776 | } | 783 | } |
| 777 | 784 | ||
| 778 | /** | 785 | /** |
| @@ -1634,7 +1641,7 @@ static int amdgpu_vm_update_ptes(struct amdgpu_vm_update_params *params, · 855 unmodified lines | |||
| 1634 | nptes = max(nptes, 1u); | 1641 | nptes = max(nptes, 1u); |
| 1635 | 1642 | ||
| 1636 | trace_amdgpu_vm_update_ptes(params, frag_start, upd_end, | 1643 | trace_amdgpu_vm_update_ptes(params, frag_start, upd_end, |
| 1637 | nptes, dst, incr, upd_flags, | 1644 | min(nptes, 32u), dst, incr, upd_flags, |
| 1638 | vm->task_info.pid, | 1645 | vm->task_info.pid, |
| 1639 | vm->immediate.fence_context); | 1646 | vm->immediate.fence_context); |
| 1640 | amdgpu_vm_update_flags(params, to_amdgpu_bo_vm(pt), | 1647 | amdgpu_vm_update_flags(params, to_amdgpu_bo_vm(pt), |
| @@ -2257,6 +2264,7 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev, · 616 unmodified lines | |||
| 2257 | if (!bo) | 2264 | if (!bo) |
| 2258 | return bo_va; | 2265 | return bo_va; |
| 2259 | 2266 | ||
| 2267 | dma_resv_assert_held(bo->tbo.base.resv); | ||
| 2260 | if (amdgpu_dmabuf_is_xgmi_accessible(adev, bo)) { | 2268 | if (amdgpu_dmabuf_is_xgmi_accessible(adev, bo)) { |
| 2261 | bo_va->is_xgmi = true; | 2269 | bo_va->is_xgmi = true; |
| 2262 | /* Power up XGMI if it can be potentially used */ | 2270 | /* Power up XGMI if it can be potentially used */ |
| @@ -2634,7 +2642,7 @@ void amdgpu_vm_bo_trace_cs(struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket) · 371 unmodified lines | |||
| 2634 | } | 2642 | } |
| 2635 | 2643 | ||
| 2636 | /** | 2644 | /** |
| 2637 | * amdgpu_vm_bo_rmv - remove a bo to a specific vm | 2645 | * amdgpu_vm_bo_del - remove a bo from a specific vm |
| 2638 | * | 2646 | * |
| 2639 | * @adev: amdgpu_device pointer | 2647 | * @adev: amdgpu_device pointer |
| 2640 | * @bo_va: requested bo_va | 2648 | * @bo_va: requested bo_va |
| @@ -2643,7 +2651,7 @@ void amdgpu_vm_bo_trace_cs(struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket) · 2 unmodified lines | |||
| 2643 | * | 2651 | * |
| 2644 | * Object have to be reserved! | 2652 | * Object have to be reserved! |
| 2645 | */ | 2653 | */ |
| 2646 | void amdgpu_vm_bo_rmv(struct amdgpu_device *adev, | 2654 | void amdgpu_vm_bo_del(struct amdgpu_device *adev, |
| 2647 | struct amdgpu_bo_va *bo_va) | 2655 | struct amdgpu_bo_va *bo_va) |
| 2648 | { | 2656 | { |
| 2649 | struct amdgpu_bo_va_mapping *mapping, *next; | 2657 | struct amdgpu_bo_va_mapping *mapping, *next; |
| @@ -2651,7 +2659,10 @@ void amdgpu_vm_bo_rmv(struct amdgpu_device *adev, · 1 unmodified line | |||
| 2651 | struct amdgpu_vm *vm = bo_va->base.vm; | 2659 | struct amdgpu_vm *vm = bo_va->base.vm; |
| 2652 | struct amdgpu_vm_bo_base **base; | 2660 | struct amdgpu_vm_bo_base **base; |
| 2653 | 2661 | ||
| 2662 | dma_resv_assert_held(vm->root.bo->tbo.base.resv); | ||
| 2663 | |||
| 2654 | if (bo) { | 2664 | if (bo) { |
| 2665 | dma_resv_assert_held(bo->tbo.base.resv); | ||
| 2655 | if (bo->tbo.base.resv == vm->root.bo->tbo.base.resv) | 2666 | if (bo->tbo.base.resv == vm->root.bo->tbo.base.resv) |
| 2656 | vm->bulk_moveable = false; | 2667 | vm->bulk_moveable = false; |
| 2657 | 2668 | ||
| @@ -435,7 +435,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev, · 434 unmodified lines | |||
| 435 | struct amdgpu_bo_va_mapping *amdgpu_vm_bo_lookup_mapping(struct amdgpu_vm *vm, | 435 | struct amdgpu_bo_va_mapping *amdgpu_vm_bo_lookup_mapping(struct amdgpu_vm *vm, |
| 436 | uint64_t addr); | 436 | uint64_t addr); |
| 437 | void amdgpu_vm_bo_trace_cs(struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket); | 437 | void amdgpu_vm_bo_trace_cs(struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket); |
| 438 | void amdgpu_vm_bo_rmv(struct amdgpu_device *adev, | 438 | void amdgpu_vm_bo_del(struct amdgpu_device *adev, |
| 439 | struct amdgpu_bo_va *bo_va); | 439 | struct amdgpu_bo_va *bo_va); |
| 440 | void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint32_t min_vm_size, | 440 | void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint32_t min_vm_size, |
| 441 | uint32_t fragment_size_default, unsigned max_level, | 441 | uint32_t fragment_size_default, unsigned max_level, |
| @@ -96,9 +96,9 @@ static ssize_t amdgpu_mem_info_vram_used_show(struct device *dev, · 95 unmodified lines | |||
| 96 | { | 96 | { |
| 97 | struct drm_device *ddev = dev_get_drvdata(dev); | 97 | struct drm_device *ddev = dev_get_drvdata(dev); |
| 98 | struct amdgpu_device *adev = drm_to_adev(ddev); | 98 | struct amdgpu_device *adev = drm_to_adev(ddev); |
| 99 | struct ttm_resource_manager *man = &adev->mman.vram_mgr.manager; | ||
| 99 | 100 | ||
| 100 | return sysfs_emit(buf, "%llu\n", | 101 | return sysfs_emit(buf, "%llu\n", ttm_resource_manager_usage(man)); |
| 101 | amdgpu_vram_mgr_usage(&adev->mman.vram_mgr)); | ||
| 102 | } | 102 | } |
| 103 | 103 | ||
| 104 | /** | 104 | /** |
| @@ -253,7 +253,9 @@ static void amdgpu_vram_mgr_do_reserve(struct ttm_resource_manager *man) · 148 unmodified lines | |||
| 253 | 253 | ||
| 254 | vis_usage = amdgpu_vram_mgr_vis_size(adev, &rsv->mm_node); | 254 | vis_usage = amdgpu_vram_mgr_vis_size(adev, &rsv->mm_node); |
| 255 | atomic64_add(vis_usage, &mgr->vis_usage); | 255 | atomic64_add(vis_usage, &mgr->vis_usage); |
| 256 | atomic64_add(rsv->mm_node.size << PAGE_SHIFT, &mgr->usage); | 256 | spin_lock(&man->bdev->lru_lock); |
| 257 | man->usage += rsv->mm_node.size << PAGE_SHIFT; | ||
| 258 | spin_unlock(&man->bdev->lru_lock); | ||
| 257 | list_move(&rsv->node, &mgr->reserved_pages); | 259 | list_move(&rsv->node, &mgr->reserved_pages); |
| 258 | } | 260 | } |
| 259 | } | 261 | } |
| @@ -281,7 +283,7 @@ int amdgpu_vram_mgr_reserve_range(struct amdgpu_vram_mgr *mgr, · 21 unmodified lines | |||
| 281 | rsv->mm_node.size = size >> PAGE_SHIFT; | 283 | rsv->mm_node.size = size >> PAGE_SHIFT; |
| 282 | 284 | ||
| 283 | spin_lock(&mgr->lock); | 285 | spin_lock(&mgr->lock); |
| 284 | list_add_tail(&mgr->reservations_pending, &rsv->node); | 286 | list_add_tail(&rsv->node, &mgr->reservations_pending); |
| 285 | amdgpu_vram_mgr_do_reserve(&mgr->manager); | 287 | amdgpu_vram_mgr_do_reserve(&mgr->manager); |
| 286 | spin_unlock(&mgr->lock); | 288 | spin_unlock(&mgr->lock); |
| 287 | 289 | ||
| @@ -378,19 +380,13 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, · 90 unmodified lines | |||
| 378 | 380 | ||
| 379 | lpfn = place->lpfn; | 381 | lpfn = place->lpfn; |
| 380 | if (!lpfn) | 382 | if (!lpfn) |
| 381 | lpfn = man->size; | 383 | lpfn = man->size >> PAGE_SHIFT; |
| 382 | 384 | ||
| 383 | max_bytes = adev->gmc.mc_vram_size; | 385 | max_bytes = adev->gmc.mc_vram_size; |
| 384 | if (tbo->type != ttm_bo_type_kernel) | 386 | if (tbo->type != ttm_bo_type_kernel) |
| 385 | max_bytes -= AMDGPU_VM_RESERVED_VRAM; | 387 | max_bytes -= AMDGPU_VM_RESERVED_VRAM; |
| 386 | 388 | ||
| 387 | /* bail out quickly if there's likely not enough VRAM for this BO */ | ||
| 388 | mem_bytes = tbo->base.size; | 389 | mem_bytes = tbo->base.size; |
| 389 | if (atomic64_add_return(mem_bytes, &mgr->usage) > max_bytes) { | ||
| 390 | r = -ENOSPC; | ||
| 391 | goto error_sub; | ||
| 392 | } | ||
| 393 | |||
| 394 | if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { | 390 | if (place->flags & TTM_PL_FLAG_CONTIGUOUS) { |
| 395 | pages_per_node = ~0ul; | 391 | pages_per_node = ~0ul; |
| 396 | num_nodes = 1; | 392 | num_nodes = 1; |
| @@ -408,13 +404,17 @@ static int amdgpu_vram_mgr_new(struct ttm_resource_manager *man, · 11 unmodified lines | |||
| 408 | 404 | ||
| 409 | node = kvmalloc(struct_size(node, mm_nodes, num_nodes), | 405 | node = kvmalloc(struct_size(node, mm_nodes, num_nodes), |
| 410 | GFP_KERNEL | __GFP_ZERO); | 406 | GFP_KERNEL | __GFP_ZERO); |
| 411 | if (!node) { | 407 | if (!node) |
| 412 | r = -ENOMEM; | 408 | return -ENOMEM; |
| 413 | goto error_sub; | ||
| 414 | } | ||
| 415 | 409 | ||
| 416 | ttm_resource_init(tbo, place, &node->base); | 410 | ttm_resource_init(tbo, place, &node->base); |
| 417 | 411 | ||
| 412 | /* bail out quickly if there's likely not enough VRAM for this BO */ | ||
| 413 | if (ttm_resource_manager_usage(man) > max_bytes) { | ||
| 414 | r = -ENOSPC; | ||
| 415 | goto error_fini; | ||
| 416 | } | ||
| 417 | |||
| 418 | mode = DRM_MM_INSERT_BEST; | 418 | mode = DRM_MM_INSERT_BEST; |
| 419 | if (place->flags & TTM_PL_FLAG_TOPDOWN) | 419 | if (place->flags & TTM_PL_FLAG_TOPDOWN) |
| 420 | mode = DRM_MM_INSERT_HIGH; | 420 | mode = DRM_MM_INSERT_HIGH; |
| @@ -472,11 +472,10 @@ error_free: · 51 unmodified lines | |||
| 472 | while (i--) | 472 | while (i--) |
| 473 | drm_mm_remove_node(&node->mm_nodes[i]); | 473 | drm_mm_remove_node(&node->mm_nodes[i]); |
| 474 | spin_unlock(&mgr->lock); | 474 | spin_unlock(&mgr->lock); |
| 475 | error_fini: | ||
| 475 | ttm_resource_fini(man, &node->base); | 476 | ttm_resource_fini(man, &node->base); |
| 476 | kvfree(node); | 477 | kvfree(node); |
| 477 | 478 | ||
| 478 | error_sub: | ||
| 479 | atomic64_sub(mem_bytes, &mgr->usage); | ||
| 480 | return r; | 479 | return r; |
| 481 | } | 480 | } |
| 482 | 481 | ||
| @@ -494,7 +493,7 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, · 11 unmodified lines | |||
| 494 | struct ttm_range_mgr_node *node = to_ttm_range_mgr_node(res); | 493 | struct ttm_range_mgr_node *node = to_ttm_range_mgr_node(res); |
| 495 | struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); | 494 | struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); |
| 496 | struct amdgpu_device *adev = to_amdgpu_device(mgr); | 495 | struct amdgpu_device *adev = to_amdgpu_device(mgr); |
| 497 | uint64_t usage = 0, vis_usage = 0; | 496 | uint64_t vis_usage = 0; |
| 498 | unsigned i, pages; | 497 | unsigned i, pages; |
| 499 | 498 | ||
| 500 | spin_lock(&mgr->lock); | 499 | spin_lock(&mgr->lock); |
| @@ -503,13 +502,11 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man, · 2 unmodified lines | |||
| 503 | struct drm_mm_node *mm = &node->mm_nodes[i]; | 502 | struct drm_mm_node *mm = &node->mm_nodes[i]; |
| 504 | 503 | ||
| 505 | drm_mm_remove_node(mm); | 504 | drm_mm_remove_node(mm); |
| 506 | usage += mm->size << PAGE_SHIFT; | ||
| 507 | vis_usage += amdgpu_vram_mgr_vis_size(adev, mm); | 505 | vis_usage += amdgpu_vram_mgr_vis_size(adev, mm); |
| 508 | } | 506 | } |
| 509 | amdgpu_vram_mgr_do_reserve(man); | 507 | amdgpu_vram_mgr_do_reserve(man); |
| 510 | spin_unlock(&mgr->lock); | 508 | spin_unlock(&mgr->lock); |
| 511 | 509 | ||
| 512 | atomic64_sub(usage, &mgr->usage); | ||
| 513 | atomic64_sub(vis_usage, &mgr->vis_usage); | 510 | atomic64_sub(vis_usage, &mgr->vis_usage); |
| 514 | 511 | ||
| 515 | ttm_resource_fini(man, res); | 512 | ttm_resource_fini(man, res); |
| @@ -627,18 +624,6 @@ void amdgpu_vram_mgr_free_sgt(struct device *dev, · 111 unmodified lines | |||
| 627 | kfree(sgt); | 624 | kfree(sgt); |
| 628 | } | 625 | } |
| 629 | 626 | ||
| 630 | /** | ||
| 631 | * amdgpu_vram_mgr_usage - how many bytes are used in this domain | ||
| 632 | * | ||
| 633 | * @mgr: amdgpu_vram_mgr pointer | ||
| 634 | * | ||
| 635 | * Returns how many bytes are used in this domain. | ||
| 636 | */ | ||
| 637 | uint64_t amdgpu_vram_mgr_usage(struct amdgpu_vram_mgr *mgr) | ||
| 638 | { | ||
| 639 | return atomic64_read(&mgr->usage); | ||
| 640 | } | ||
| 641 | |||
| 642 | /** | 627 | /** |
| 643 | * amdgpu_vram_mgr_vis_usage - how many bytes are used in the visible part | 628 | * amdgpu_vram_mgr_vis_usage - how many bytes are used in the visible part |
| 644 | * | 629 | * |
| @@ -664,13 +649,12 @@ static void amdgpu_vram_mgr_debug(struct ttm_resource_manager *man, · 19 unmodified lines | |||
| 664 | { | 649 | { |
| 665 | struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); | 650 | struct amdgpu_vram_mgr *mgr = to_vram_mgr(man); |
| 666 | 651 | ||
| 652 | drm_printf(printer, " vis usage:%llu\n", | ||
| 653 | amdgpu_vram_mgr_vis_usage(mgr)); | ||
| 654 | |||
| 667 | spin_lock(&mgr->lock); | 655 | spin_lock(&mgr->lock); |
| 668 | drm_mm_print(&mgr->mm, printer); | 656 | drm_mm_print(&mgr->mm, printer); |
| 669 | spin_unlock(&mgr->lock); | 657 | spin_unlock(&mgr->lock); |
| 670 | |||
| 671 | drm_printf(printer, "man size:%llu pages, ram usage:%lluMB, vis usage:%lluMB\n", | ||
| 672 | man->size, amdgpu_vram_mgr_usage(mgr) >> 20, | ||
| 673 | amdgpu_vram_mgr_vis_usage(mgr) >> 20); | ||
| 674 | } | 658 | } |
| 675 | 659 | ||
| 676 | static const struct ttm_resource_manager_func amdgpu_vram_mgr_func = { | 660 | static const struct ttm_resource_manager_func amdgpu_vram_mgr_func = { |
| @@ -692,11 +676,11 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) · 15 unmodified lines | |||
| 692 | struct ttm_resource_manager *man = &mgr->manager; | 676 | struct ttm_resource_manager *man = &mgr->manager; |
| 693 | 677 | ||
| 694 | ttm_resource_manager_init(man, &adev->mman.bdev, | 678 | ttm_resource_manager_init(man, &adev->mman.bdev, |
| 695 | adev->gmc.real_vram_size >> PAGE_SHIFT); | 679 | adev->gmc.real_vram_size); |
| 696 | 680 | ||
| 697 | man->func = &amdgpu_vram_mgr_func; | 681 | man->func = &amdgpu_vram_mgr_func; |
| 698 | 682 | ||
| 699 | drm_mm_init(&mgr->mm, 0, man->size); | 683 | drm_mm_init(&mgr->mm, 0, man->size >> PAGE_SHIFT); |
| 700 | spin_lock_init(&mgr->lock); | 684 | spin_lock_init(&mgr->lock); |
| 701 | INIT_LIST_HEAD(&mgr->reservations_pending); | 685 | INIT_LIST_HEAD(&mgr->reservations_pending); |
| 702 | INIT_LIST_HEAD(&mgr->reserved_pages); | 686 | INIT_LIST_HEAD(&mgr->reserved_pages); |
| @@ -32,6 +32,8 @@ · 31 unmodified lines | |||
| 32 | #include "wafl/wafl2_4_0_0_smn.h" | 32 | #include "wafl/wafl2_4_0_0_smn.h" |
| 33 | #include "wafl/wafl2_4_0_0_sh_mask.h" | 33 | #include "wafl/wafl2_4_0_0_sh_mask.h" |
| 34 | 34 | ||
| 35 | #include "amdgpu_reset.h" | ||
| 36 | |||
| 35 | #define smnPCS_XGMI23_PCS_ERROR_STATUS 0x11a01210 | 37 | #define smnPCS_XGMI23_PCS_ERROR_STATUS 0x11a01210 |
| 36 | #define smnPCS_XGMI3X16_PCS_ERROR_STATUS 0x11a0020c | 38 | #define smnPCS_XGMI3X16_PCS_ERROR_STATUS 0x11a0020c |
| 37 | #define smnPCS_GOPX1_PCS_ERROR_STATUS 0x12200210 | 39 | #define smnPCS_GOPX1_PCS_ERROR_STATUS 0x12200210 |
| @@ -227,6 +229,9 @@ static void amdgpu_xgmi_hive_release(struct kobject *kobj) · 189 unmodified lines | |||
| 227 | struct amdgpu_hive_info *hive = container_of( | 229 | struct amdgpu_hive_info *hive = container_of( |
| 228 | kobj, struct amdgpu_hive_info, kobj); | 230 | kobj, struct amdgpu_hive_info, kobj); |
| 229 | 231 | ||
| 232 | amdgpu_reset_put_reset_domain(hive->reset_domain); | ||
| 233 | hive->reset_domain = NULL; | ||
| 234 | |||
| 230 | mutex_destroy(&hive->hive_lock); | 235 | mutex_destroy(&hive->hive_lock); |
| 231 | kfree(hive); | 236 | kfree(hive); |
| 232 | } | 237 | } |
| @@ -398,15 +403,35 @@ struct amdgpu_hive_info *amdgpu_get_xgmi_hive(struct amdgpu_device *adev) · 165 unmodified lines | |||
| 398 | goto pro_end; | 403 | goto pro_end; |
| 399 | } | 404 | } |
| 400 | 405 | ||
| 406 | /** | ||
| 407 | * Avoid recreating reset domain when hive is reconstructed for the case | ||
| 408 | * of reset the devices in the XGMI hive during probe for SRIOV | ||
| 409 | * See https://www.spinics.net/lists/amd-gfx/msg58836.html | ||
| 410 | */ | ||
| 411 | if (adev->reset_domain->type != XGMI_HIVE) { | ||
| 412 | hive->reset_domain = amdgpu_reset_create_reset_domain(XGMI_HIVE, "amdgpu-reset-hive"); | ||
| 413 | if (!hive->reset_domain) { | ||
| 414 | dev_err(adev->dev, "XGMI: failed initializing reset domain for xgmi hive\n"); | ||
| 415 | ret = -ENOMEM; | ||
| 416 | kobject_put(&hive->kobj); | ||
| 417 | kfree(hive); | ||
| 418 | hive = NULL; | ||
| 419 | goto pro_end; | ||
| 420 | } | ||
| 421 | } else { | ||
| 422 | amdgpu_reset_get_reset_domain(adev->reset_domain); | ||
| 423 | hive->reset_domain = adev->reset_domain; | ||
| 424 | } | ||
| 425 | |||
| 401 | hive->hive_id = adev->gmc.xgmi.hive_id; | 426 | hive->hive_id = adev->gmc.xgmi.hive_id; |
| 402 | INIT_LIST_HEAD(&hive->device_list); | 427 | INIT_LIST_HEAD(&hive->device_list); |
| 403 | INIT_LIST_HEAD(&hive->node); | 428 | INIT_LIST_HEAD(&hive->node); |
| 404 | mutex_init(&hive->hive_lock); | 429 | mutex_init(&hive->hive_lock); |
| 405 | atomic_set(&hive->in_reset, 0); | ||
| 406 | atomic_set(&hive->number_devices, 0); | 430 | atomic_set(&hive->number_devices, 0); |
| 407 | task_barrier_init(&hive->tb); | 431 | task_barrier_init(&hive->tb); |
| 408 | hive->pstate = AMDGPU_XGMI_PSTATE_UNKNOWN; | 432 | hive->pstate = AMDGPU_XGMI_PSTATE_UNKNOWN; |
| 409 | hive->hi_req_gpu = NULL; | 433 | hive->hi_req_gpu = NULL; |
| 434 | |||
| 410 | /* | 435 | /* |
| 411 | * hive pstate on boot is high in vega20 so we have to go to low | 436 | * hive pstate on boot is high in vega20 so we have to go to low |
| 412 | * pstate on after boot. | 437 | * pstate on after boot. |
| @@ -732,53 +757,22 @@ int amdgpu_xgmi_remove_device(struct amdgpu_device *adev) · 319 unmodified lines | |||
| 732 | return psp_xgmi_terminate(&adev->psp); | 757 | return psp_xgmi_terminate(&adev->psp); |
| 733 | } | 758 | } |
| 734 | 759 | ||
| 735 | static int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev) | 760 | static int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev, struct ras_common_if *ras_block) |
| 736 | { | 761 | { |
| 737 | int r; | ||
| 738 | struct ras_ih_if ih_info = { | ||
| 739 | .cb = NULL, | ||
| 740 | }; | ||
| 741 | struct ras_fs_if fs_info = { | ||
| 742 | .sysfs_name = "xgmi_wafl_err_count", | ||
| 743 | }; | ||
| 744 | |||
| 745 | if (!adev->gmc.xgmi.supported || | 762 | if (!adev->gmc.xgmi.supported || |
| 746 | adev->gmc.xgmi.num_physical_nodes == 0) | 763 | adev->gmc.xgmi.num_physical_nodes == 0) |
| 747 | return 0; | 764 | return 0; |
| 748 | 765 | ||
| 749 | adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev); | 766 | adev->gmc.xgmi.ras->ras_block.hw_ops->reset_ras_error_count(adev); |
| 750 | |||
| 751 | if (!adev->gmc.xgmi.ras_if) { | ||
| 752 | adev->gmc.xgmi.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL); | ||
| 753 | if (!adev->gmc.xgmi.ras_if) | ||
| 754 | return -ENOMEM; | ||
| 755 | adev->gmc.xgmi.ras_if->block = AMDGPU_RAS_BLOCK__XGMI_WAFL; | ||
| 756 | adev->gmc.xgmi.ras_if->type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 757 | adev->gmc.xgmi.ras_if->sub_block_index = 0; | ||
| 758 | } | ||
| 759 | ih_info.head = fs_info.head = *adev->gmc.xgmi.ras_if; | ||
| 760 | r = amdgpu_ras_late_init(adev, adev->gmc.xgmi.ras_if, | ||
| 761 | &fs_info, &ih_info); | ||
| 762 | if (r || !amdgpu_ras_is_supported(adev, adev->gmc.xgmi.ras_if->block)) { | ||
| 763 | kfree(adev->gmc.xgmi.ras_if); | ||
| 764 | adev->gmc.xgmi.ras_if = NULL; | ||
| 765 | } | ||
| 766 | 767 | ||
| 767 | return r; | 768 | return amdgpu_ras_block_late_init(adev, ras_block); |
| 768 | } | 769 | } |
| 769 | 770 | ||
| 770 | static void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev) | 771 | static void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev) |
| 771 | { | 772 | { |
| 772 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL) && | 773 | if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL) && |
| 773 | adev->gmc.xgmi.ras_if) { | 774 | adev->gmc.xgmi.ras_if) |
| 774 | struct ras_common_if *ras_if = adev->gmc.xgmi.ras_if; | 775 | amdgpu_ras_block_late_fini(adev, adev->gmc.xgmi.ras_if); |
| 775 | struct ras_ih_if ih_info = { | ||
| 776 | .cb = NULL, | ||
| 777 | }; | ||
| 778 | |||
| 779 | amdgpu_ras_late_fini(adev, ras_if, &ih_info); | ||
| 780 | kfree(ras_if); | ||
| 781 | } | ||
| 782 | } | 776 | } |
| 783 | 777 | ||
| 784 | uint64_t amdgpu_xgmi_get_relative_phy_addr(struct amdgpu_device *adev, | 778 | uint64_t amdgpu_xgmi_get_relative_phy_addr(struct amdgpu_device *adev, |
| @@ -865,7 +859,7 @@ static int amdgpu_xgmi_query_pcs_error_status(struct amdgpu_device *adev, · 80 unmodified lines | |||
| 865 | return 0; | 859 | return 0; |
| 866 | } | 860 | } |
| 867 | 861 | ||
| 868 | static int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev, | 862 | static void amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev, |
| 869 | void *ras_error_status) | 863 | void *ras_error_status) |
| 870 | { | 864 | { |
| 871 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | 865 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| @@ -874,7 +868,7 @@ static int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 874 | uint32_t ue_cnt = 0, ce_cnt = 0; | 868 | uint32_t ue_cnt = 0, ce_cnt = 0; |
| 875 | 869 | ||
| 876 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL)) | 870 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL)) |
| 877 | return -EINVAL; | 871 | return ; |
| 878 | 872 | ||
| 879 | err_data->ue_count = 0; | 873 | err_data->ue_count = 0; |
| 880 | err_data->ce_count = 0; | 874 | err_data->ce_count = 0; |
| @@ -940,17 +934,54 @@ static int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev, · 59 unmodified lines | |||
| 940 | break; | 934 | break; |
| 941 | } | 935 | } |
| 942 | 936 | ||
| 943 | adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev); | 937 | adev->gmc.xgmi.ras->ras_block.hw_ops->reset_ras_error_count(adev); |
| 944 | 938 | ||
| 945 | err_data->ue_count += ue_cnt; | 939 | err_data->ue_count += ue_cnt; |
| 946 | err_data->ce_count += ce_cnt; | 940 | err_data->ce_count += ce_cnt; |
| 941 | } | ||
| 947 | 942 | ||
| 948 | return 0; | 943 | /* Trigger XGMI/WAFL error */ |
| 944 | static int amdgpu_ras_error_inject_xgmi(struct amdgpu_device *adev, void *inject_if) | ||
| 945 | { | ||
| 946 | int ret = 0; | ||
| 947 | struct ta_ras_trigger_error_input *block_info = | ||
| 948 | (struct ta_ras_trigger_error_input *)inject_if; | ||
| 949 | |||
| 950 | if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_DISALLOW)) | ||
| 951 | dev_warn(adev->dev, "Failed to disallow df cstate"); | ||
| 952 | |||
| 953 | if (amdgpu_dpm_allow_xgmi_power_down(adev, false)) | ||
| 954 | dev_warn(adev->dev, "Failed to disallow XGMI power down"); | ||
| 955 | |||
| 956 | ret = psp_ras_trigger_error(&adev->psp, block_info); | ||
| 957 | |||
| 958 | if (amdgpu_ras_intr_triggered()) | ||
| 959 | return ret; | ||
| 960 | |||
| 961 | if (amdgpu_dpm_allow_xgmi_power_down(adev, true)) | ||
| 962 | dev_warn(adev->dev, "Failed to allow XGMI power down"); | ||
| 963 | |||
| 964 | if (amdgpu_dpm_set_df_cstate(adev, DF_CSTATE_ALLOW)) | ||
| 965 | dev_warn(adev->dev, "Failed to allow df cstate"); | ||
| 966 | |||
| 967 | return ret; | ||
| 949 | } | 968 | } |
| 950 | 969 | ||
| 951 | const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs = { | 970 | struct amdgpu_ras_block_hw_ops xgmi_ras_hw_ops = { |
| 952 | .ras_late_init = amdgpu_xgmi_ras_late_init, | ||
| 953 | .ras_fini = amdgpu_xgmi_ras_fini, | ||
| 954 | .query_ras_error_count = amdgpu_xgmi_query_ras_error_count, | 971 | .query_ras_error_count = amdgpu_xgmi_query_ras_error_count, |
| 955 | .reset_ras_error_count = amdgpu_xgmi_reset_ras_error_count, | 972 | .reset_ras_error_count = amdgpu_xgmi_reset_ras_error_count, |
| 973 | .ras_error_inject = amdgpu_ras_error_inject_xgmi, | ||
| 974 | }; | ||
| 975 | |||
| 976 | struct amdgpu_xgmi_ras xgmi_ras = { | ||
| 977 | .ras_block = { | ||
| 978 | .ras_comm = { | ||
| 979 | .name = "xgmi_wafl", | ||
| 980 | .block = AMDGPU_RAS_BLOCK__XGMI_WAFL, | ||
| 981 | .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | ||
| 982 | }, | ||
| 983 | .hw_ops = &xgmi_ras_hw_ops, | ||
| 984 | .ras_late_init = amdgpu_xgmi_ras_late_init, | ||
| 985 | .ras_fini = amdgpu_xgmi_ras_fini, | ||
| 986 | }, | ||
| 956 | }; | 987 | }; |
| @@ -24,7 +24,7 @@ · 23 unmodified lines | |||
| 24 | 24 | ||
| 25 | #include <drm/task_barrier.h> | 25 | #include <drm/task_barrier.h> |
| 26 | #include "amdgpu_psp.h" | 26 | #include "amdgpu_psp.h" |
| 27 | 27 | #include "amdgpu_ras.h" | |
| 28 | 28 | ||
| 29 | struct amdgpu_hive_info { | 29 | struct amdgpu_hive_info { |
| 30 | struct kobject kobj; | 30 | struct kobject kobj; |
| @@ -33,7 +33,6 @@ struct amdgpu_hive_info { · 2 unmodified lines | |||
| 33 | struct list_head node; | 33 | struct list_head node; |
| 34 | atomic_t number_devices; | 34 | atomic_t number_devices; |
| 35 | struct mutex hive_lock; | 35 | struct mutex hive_lock; |
| 36 | atomic_t in_reset; | ||
| 37 | int hi_req_count; | 36 | int hi_req_count; |
| 38 | struct amdgpu_device *hi_req_gpu; | 37 | struct amdgpu_device *hi_req_gpu; |
| 39 | struct task_barrier tb; | 38 | struct task_barrier tb; |
| @@ -42,6 +41,8 @@ struct amdgpu_hive_info { · 2 unmodified lines | |||
| 42 | AMDGPU_XGMI_PSTATE_MAX_VEGA20, | 41 | AMDGPU_XGMI_PSTATE_MAX_VEGA20, |
| 43 | AMDGPU_XGMI_PSTATE_UNKNOWN | 42 | AMDGPU_XGMI_PSTATE_UNKNOWN |
| 44 | } pstate; | 43 | } pstate; |
| 44 | |||
| 45 | struct amdgpu_reset_domain *reset_domain; | ||
| 45 | }; | 46 | }; |
| 46 | 47 | ||
| 47 | struct amdgpu_pcs_ras_field { | 48 | struct amdgpu_pcs_ras_field { |
| @@ -50,7 +51,7 @@ struct amdgpu_pcs_ras_field { · 2 unmodified lines | |||
| 50 | uint32_t pcs_err_shift; | 51 | uint32_t pcs_err_shift; |
| 51 | }; | 52 | }; |
| 52 | 53 | ||
| 53 | extern const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs; | 54 | extern struct amdgpu_xgmi_ras xgmi_ras; |
| 54 | struct amdgpu_hive_info *amdgpu_get_xgmi_hive(struct amdgpu_device *adev); | 55 | struct amdgpu_hive_info *amdgpu_get_xgmi_hive(struct amdgpu_device *adev); |
| 55 | void amdgpu_put_xgmi_hive(struct amdgpu_hive_info *hive); | 56 | void amdgpu_put_xgmi_hive(struct amdgpu_hive_info *hive); |
| 56 | int amdgpu_xgmi_update_topology(struct amdgpu_hive_info *hive, struct amdgpu_device *adev); | 57 | int amdgpu_xgmi_update_topology(struct amdgpu_hive_info *hive, struct amdgpu_device *adev); |
| @@ -66,7 +67,8 @@ uint64_t amdgpu_xgmi_get_relative_phy_addr(struct amdgpu_device *adev, · 9 unmodified lines | |||
| 66 | static inline bool amdgpu_xgmi_same_hive(struct amdgpu_device *adev, | 67 | static inline bool amdgpu_xgmi_same_hive(struct amdgpu_device *adev, |
| 67 | struct amdgpu_device *bo_adev) | 68 | struct amdgpu_device *bo_adev) |
| 68 | { | 69 | { |
| 69 | return (adev != bo_adev && | 70 | return (amdgpu_use_xgmi_p2p && |
| 71 | adev != bo_adev && | ||
| 70 | adev->gmc.xgmi.hive_id && | 72 | adev->gmc.xgmi.hive_id && |
| 71 | adev->gmc.xgmi.hive_id == bo_adev->gmc.xgmi.hive_id); | 73 | adev->gmc.xgmi.hive_id == bo_adev->gmc.xgmi.hive_id); |
| 72 | } | 74 | } |
| @@ -68,12 +68,13 @@ int athub_v1_0_set_clockgating(struct amdgpu_device *adev, · 67 unmodified lines | |||
| 68 | if (amdgpu_sriov_vf(adev)) | 68 | if (amdgpu_sriov_vf(adev)) |
| 69 | return 0; | 69 | return 0; |
| 70 | 70 | ||
| 71 | switch (adev->asic_type) { | 71 | switch (adev->ip_versions[ATHUB_HWIP][0]) { |
| 72 | case CHIP_VEGA10: | 72 | case IP_VERSION(9, 0, 0): |
| 73 | case CHIP_VEGA12: | 73 | case IP_VERSION(9, 1, 0): |
| 74 | case CHIP_VEGA20: | 74 | case IP_VERSION(9, 2, 0): |
| 75 | case CHIP_RAVEN: | 75 | case IP_VERSION(9, 3, 0): |
| 76 | case CHIP_RENOIR: | 76 | case IP_VERSION(9, 4, 0): |
| 77 | case IP_VERSION(1, 5, 0): | ||
| 77 | athub_update_medium_grain_clock_gating(adev, | 78 | athub_update_medium_grain_clock_gating(adev, |
| 78 | state == AMD_CG_STATE_GATE); | 79 | state == AMD_CG_STATE_GATE); |
| 79 | athub_update_medium_grain_light_sleep(adev, | 80 | athub_update_medium_grain_light_sleep(adev, |
| @@ -78,6 +78,7 @@ int athub_v2_0_set_clockgating(struct amdgpu_device *adev, · 77 unmodified lines | |||
| 78 | return 0; | 78 | return 0; |
| 79 | 79 | ||
| 80 | switch (adev->ip_versions[ATHUB_HWIP][0]) { | 80 | switch (adev->ip_versions[ATHUB_HWIP][0]) { |
| 81 | case IP_VERSION(1, 3, 1): | ||
| 81 | case IP_VERSION(2, 0, 0): | 82 | case IP_VERSION(2, 0, 0): |
| 82 | case IP_VERSION(2, 0, 2): | 83 | case IP_VERSION(2, 0, 2): |
| 83 | athub_v2_0_update_medium_grain_clock_gating(adev, | 84 | athub_v2_0_update_medium_grain_clock_gating(adev, |
| @@ -74,6 +74,7 @@ int athub_v2_1_set_clockgating(struct amdgpu_device *adev, · 73 unmodified lines | |||
| 74 | case IP_VERSION(2, 1, 0): | 74 | case IP_VERSION(2, 1, 0): |
| 75 | case IP_VERSION(2, 1, 1): | 75 | case IP_VERSION(2, 1, 1): |
| 76 | case IP_VERSION(2, 1, 2): | 76 | case IP_VERSION(2, 1, 2): |
| 77 | case IP_VERSION(2, 4, 0): | ||
| 77 | athub_v2_1_update_medium_grain_clock_gating(adev, state == AMD_CG_STATE_GATE); | 78 | athub_v2_1_update_medium_grain_clock_gating(adev, state == AMD_CG_STATE_GATE); |
| 78 | athub_v2_1_update_medium_grain_light_sleep(adev, state == AMD_CG_STATE_GATE); | 79 | athub_v2_1_update_medium_grain_light_sleep(adev, state == AMD_CG_STATE_GATE); |
| 79 | break; | 80 | break; |
| @@ -25,6 +25,8 @@ · 24 unmodified lines | |||
| 25 | #include <linux/module.h> | 25 | #include <linux/module.h> |
| 26 | #include <linux/sched.h> | 26 | #include <linux/sched.h> |
| 27 | #include <linux/slab.h> | 27 | #include <linux/slab.h> |
| 28 | #include <linux/string_helpers.h> | ||
| 29 | |||
| 28 | #include <asm/unaligned.h> | 30 | #include <asm/unaligned.h> |
| 29 | 31 | ||
| 30 | #include <drm/drm_util.h> | 32 | #include <drm/drm_util.h> |
| @@ -740,7 +742,7 @@ static void atom_op_jump(atom_exec_context *ctx, int *ptr, int arg) · 709 unmodified lines | |||
| 740 | break; | 742 | break; |
| 741 | } | 743 | } |
| 742 | if (arg != ATOM_COND_ALWAYS) | 744 | if (arg != ATOM_COND_ALWAYS) |
| 743 | SDEBUG(" taken: %s\n", execute ? "yes" : "no"); | 745 | SDEBUG(" taken: %s\n", str_yes_no(execute)); |
| 744 | SDEBUG(" target: 0x%04X\n", target); | 746 | SDEBUG(" target: 0x%04X\n", target); |
| 745 | if (execute) { | 747 | if (execute) { |
| 746 | if (ctx->last_jump == (ctx->start + target)) { | 748 | if (ctx->last_jump == (ctx->start + target)) { |
| @@ -1719,7 +1719,7 @@ static void cik_program_aspm(struct amdgpu_device *adev) · 1718 unmodified lines | |||
| 1719 | bool disable_l0s = false, disable_l1 = false, disable_plloff_in_l1 = false; | 1719 | bool disable_l0s = false, disable_l1 = false, disable_plloff_in_l1 = false; |
| 1720 | bool disable_clkreq = false; | 1720 | bool disable_clkreq = false; |
| 1721 | 1721 | ||
| 1722 | if (amdgpu_aspm == 0) | 1722 | if (!amdgpu_device_should_use_aspm(adev)) |
| 1723 | return; | 1723 | return; |
| 1724 | 1724 | ||
| 1725 | if (pci_is_root_bus(adev->pdev->bus)) | 1725 | if (pci_is_root_bus(adev->pdev->bus)) |
| @@ -2532,7 +2532,7 @@ static void dce_v10_0_crtc_dpms(struct drm_crtc *crtc, int mode) · 2531 unmodified lines | |||
| 2532 | break; | 2532 | break; |
| 2533 | } | 2533 | } |
| 2534 | /* adjust pm to dpms */ | 2534 | /* adjust pm to dpms */ |
| 2535 | amdgpu_pm_compute_clocks(adev); | 2535 | amdgpu_dpm_compute_clocks(adev); |
| 2536 | } | 2536 | } |
| 2537 | 2537 | ||
| 2538 | static void dce_v10_0_crtc_prepare(struct drm_crtc *crtc) | 2538 | static void dce_v10_0_crtc_prepare(struct drm_crtc *crtc) |
| @@ -2798,6 +2798,8 @@ static int dce_v10_0_sw_init(void *handle) · 259 unmodified lines | |||
| 2798 | adev_to_drm(adev)->mode_config.preferred_depth = 24; | 2798 | adev_to_drm(adev)->mode_config.preferred_depth = 24; |
| 2799 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; | 2799 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; |
| 2800 | 2800 | ||
| 2801 | adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true; | ||
| 2802 | |||
| 2801 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; | 2803 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; |
| 2802 | 2804 | ||
| 2803 | r = amdgpu_display_modeset_create_props(adev); | 2805 | r = amdgpu_display_modeset_create_props(adev); |
| @@ -2608,7 +2608,7 @@ static void dce_v11_0_crtc_dpms(struct drm_crtc *crtc, int mode) · 2607 unmodified lines | |||
| 2608 | break; | 2608 | break; |
| 2609 | } | 2609 | } |
| 2610 | /* adjust pm to dpms */ | 2610 | /* adjust pm to dpms */ |
| 2611 | amdgpu_pm_compute_clocks(adev); | 2611 | amdgpu_dpm_compute_clocks(adev); |
| 2612 | } | 2612 | } |
| 2613 | 2613 | ||
| 2614 | static void dce_v11_0_crtc_prepare(struct drm_crtc *crtc) | 2614 | static void dce_v11_0_crtc_prepare(struct drm_crtc *crtc) |
| @@ -2916,6 +2916,8 @@ static int dce_v11_0_sw_init(void *handle) · 301 unmodified lines | |||
| 2916 | adev_to_drm(adev)->mode_config.preferred_depth = 24; | 2916 | adev_to_drm(adev)->mode_config.preferred_depth = 24; |
| 2917 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; | 2917 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; |
| 2918 | 2918 | ||
| 2919 | adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true; | ||
| 2920 | |||
| 2919 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; | 2921 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; |
| 2920 | 2922 | ||
| 2921 | r = amdgpu_display_modeset_create_props(adev); | 2923 | r = amdgpu_display_modeset_create_props(adev); |
| @@ -2424,7 +2424,7 @@ static void dce_v6_0_crtc_dpms(struct drm_crtc *crtc, int mode) · 2423 unmodified lines | |||
| 2424 | break; | 2424 | break; |
| 2425 | } | 2425 | } |
| 2426 | /* adjust pm to dpms */ | 2426 | /* adjust pm to dpms */ |
| 2427 | amdgpu_pm_compute_clocks(adev); | 2427 | amdgpu_dpm_compute_clocks(adev); |
| 2428 | } | 2428 | } |
| 2429 | 2429 | ||
| 2430 | static void dce_v6_0_crtc_prepare(struct drm_crtc *crtc) | 2430 | static void dce_v6_0_crtc_prepare(struct drm_crtc *crtc) |
| @@ -2674,6 +2674,7 @@ static int dce_v6_0_sw_init(void *handle) · 243 unmodified lines | |||
| 2674 | adev_to_drm(adev)->mode_config.max_height = 16384; | 2674 | adev_to_drm(adev)->mode_config.max_height = 16384; |
| 2675 | adev_to_drm(adev)->mode_config.preferred_depth = 24; | 2675 | adev_to_drm(adev)->mode_config.preferred_depth = 24; |
| 2676 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; | 2676 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; |
| 2677 | adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true; | ||
| 2677 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; | 2678 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; |
| 2678 | 2679 | ||
| 2679 | r = amdgpu_display_modeset_create_props(adev); | 2680 | r = amdgpu_display_modeset_create_props(adev); |
| @@ -2433,7 +2433,7 @@ static void dce_v8_0_crtc_dpms(struct drm_crtc *crtc, int mode) · 2432 unmodified lines | |||
| 2433 | break; | 2433 | break; |
| 2434 | } | 2434 | } |
| 2435 | /* adjust pm to dpms */ | 2435 | /* adjust pm to dpms */ |
| 2436 | amdgpu_pm_compute_clocks(adev); | 2436 | amdgpu_dpm_compute_clocks(adev); |
| 2437 | } | 2437 | } |
| 2438 | 2438 | ||
| 2439 | static void dce_v8_0_crtc_prepare(struct drm_crtc *crtc) | 2439 | static void dce_v8_0_crtc_prepare(struct drm_crtc *crtc) |
| @@ -2695,6 +2695,8 @@ static int dce_v8_0_sw_init(void *handle) · 255 unmodified lines | |||
| 2695 | adev_to_drm(adev)->mode_config.preferred_depth = 24; | 2695 | adev_to_drm(adev)->mode_config.preferred_depth = 24; |
| 2696 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; | 2696 | adev_to_drm(adev)->mode_config.prefer_shadow = 1; |
| 2697 | 2697 | ||
| 2698 | adev_to_drm(adev)->mode_config.fb_modifiers_not_supported = true; | ||
| 2699 | |||
| 2698 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; | 2700 | adev_to_drm(adev)->mode_config.fb_base = adev->gmc.aper_base; |
| 2699 | 2701 | ||
| 2700 | r = amdgpu_display_modeset_create_props(adev); | 2702 | r = amdgpu_display_modeset_create_props(adev); |
| @@ -458,7 +458,7 @@ static int df_v3_6_pmc_add_cntr(struct amdgpu_device *adev, · 457 unmodified lines | |||
| 458 | 458 | ||
| 459 | #define DEFERRED_ARM_MASK (1 << 31) | 459 | #define DEFERRED_ARM_MASK (1 << 31) |
| 460 | static int df_v3_6_pmc_set_deferred(struct amdgpu_device *adev, | 460 | static int df_v3_6_pmc_set_deferred(struct amdgpu_device *adev, |
| 461 | int counter_idx, uint64_t config, | 461 | uint64_t config, int counter_idx, |
| 462 | bool is_deferred) | 462 | bool is_deferred) |
| 463 | { | 463 | { |
| 464 | 464 | ||
| @@ -476,8 +476,8 @@ static int df_v3_6_pmc_set_deferred(struct amdgpu_device *adev, · 11 unmodified lines | |||
| 476 | } | 476 | } |
| 477 | 477 | ||
| 478 | static bool df_v3_6_pmc_is_deferred(struct amdgpu_device *adev, | 478 | static bool df_v3_6_pmc_is_deferred(struct amdgpu_device *adev, |
| 479 | int counter_idx, | 479 | uint64_t config, |
| 480 | uint64_t config) | 480 | int counter_idx) |
| 481 | { | 481 | { |
| 482 | return (df_v3_6_pmc_has_counter(adev, config, counter_idx) && | 482 | return (df_v3_6_pmc_has_counter(adev, config, counter_idx) && |
| 483 | (adev->df_perfmon_config_assign_mask[counter_idx] | 483 | (adev->df_perfmon_config_assign_mask[counter_idx] |
| @@ -56,10 +56,6 @@ · 55 unmodified lines | |||
| 56 | #define GFX10_NUM_GFX_RINGS_Sienna_Cichlid 1 | 56 | #define GFX10_NUM_GFX_RINGS_Sienna_Cichlid 1 |
| 57 | #define GFX10_MEC_HPD_SIZE 2048 | 57 | #define GFX10_MEC_HPD_SIZE 2048 |
| 58 | 58 | ||
| 59 | #define RLCG_VFGATE_DISABLED 0x4000000 | ||
| 60 | #define RLCG_WRONG_OPERATION_TYPE 0x2000000 | ||
| 61 | #define RLCG_NOT_IN_RANGE 0x1000000 | ||
| 62 | |||
| 63 | #define F32_CE_PROGRAM_RAM_SIZE 65536 | 59 | #define F32_CE_PROGRAM_RAM_SIZE 65536 |
| 64 | #define RLCG_UCODE_LOADING_START_ADDRESS 0x00002000L | 60 | #define RLCG_UCODE_LOADING_START_ADDRESS 0x00002000L |
| 65 | 61 | ||
| @@ -110,6 +106,12 @@ · 44 unmodified lines | |||
| 110 | #define mmGOLDEN_TSC_COUNT_UPPER_Vangogh_BASE_IDX 1 | 106 | #define mmGOLDEN_TSC_COUNT_UPPER_Vangogh_BASE_IDX 1 |
| 111 | #define mmGOLDEN_TSC_COUNT_LOWER_Vangogh 0x0026 | 107 | #define mmGOLDEN_TSC_COUNT_LOWER_Vangogh 0x0026 |
| 112 | #define mmGOLDEN_TSC_COUNT_LOWER_Vangogh_BASE_IDX 1 | 108 | #define mmGOLDEN_TSC_COUNT_LOWER_Vangogh_BASE_IDX 1 |
| 109 | |||
| 110 | #define mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6 0x002d | ||
| 111 | #define mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6_BASE_IDX 1 | ||
| 112 | #define mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6 0x002e | ||
| 113 | #define mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6_BASE_IDX 1 | ||
| 114 | |||
| 113 | #define mmSPI_CONFIG_CNTL_1_Vangogh 0x2441 | 115 | #define mmSPI_CONFIG_CNTL_1_Vangogh 0x2441 |
| 114 | #define mmSPI_CONFIG_CNTL_1_Vangogh_BASE_IDX 1 | 116 | #define mmSPI_CONFIG_CNTL_1_Vangogh_BASE_IDX 1 |
| 115 | #define mmVGT_TF_MEMORY_BASE_HI_Vangogh 0x2261 | 117 | #define mmVGT_TF_MEMORY_BASE_HI_Vangogh 0x2261 |
| @@ -180,14 +182,6 @@ · 64 unmodified lines | |||
| 180 | #define mmRLC_SPARE_INT_0_Sienna_Cichlid 0x4ca5 | 182 | #define mmRLC_SPARE_INT_0_Sienna_Cichlid 0x4ca5 |
| 181 | #define mmRLC_SPARE_INT_0_Sienna_Cichlid_BASE_IDX 1 | 183 | #define mmRLC_SPARE_INT_0_Sienna_Cichlid_BASE_IDX 1 |
| 182 | 184 | ||
| 183 | #define GFX_RLCG_GC_WRITE_OLD (0x8 << 28) | ||
| 184 | #define GFX_RLCG_GC_WRITE (0x0 << 28) | ||
| 185 | #define GFX_RLCG_GC_READ (0x1 << 28) | ||
| 186 | #define GFX_RLCG_MMHUB_WRITE (0x2 << 28) | ||
| 187 | |||
| 188 | #define RLCG_ERROR_REPORT_ENABLED(adev) \ | ||
| 189 | (amdgpu_sriov_reg_indirect_mmhub(adev) || amdgpu_sriov_reg_indirect_gc(adev)) | ||
| 190 | |||
| 191 | MODULE_FIRMWARE("amdgpu/navi10_ce.bin"); | 185 | MODULE_FIRMWARE("amdgpu/navi10_ce.bin"); |
| 192 | MODULE_FIRMWARE("amdgpu/navi10_pfp.bin"); | 186 | MODULE_FIRMWARE("amdgpu/navi10_pfp.bin"); |
| 193 | MODULE_FIRMWARE("amdgpu/navi10_me.bin"); | 187 | MODULE_FIRMWARE("amdgpu/navi10_me.bin"); |
| @@ -270,6 +264,20 @@ MODULE_FIRMWARE("amdgpu/cyan_skillfish2_mec.bin"); · 76 unmodified lines | |||
| 270 | MODULE_FIRMWARE("amdgpu/cyan_skillfish2_mec2.bin"); | 264 | MODULE_FIRMWARE("amdgpu/cyan_skillfish2_mec2.bin"); |
| 271 | MODULE_FIRMWARE("amdgpu/cyan_skillfish2_rlc.bin"); | 265 | MODULE_FIRMWARE("amdgpu/cyan_skillfish2_rlc.bin"); |
| 272 | 266 | ||
| 267 | MODULE_FIRMWARE("amdgpu/gc_10_3_6_ce.bin"); | ||
| 268 | MODULE_FIRMWARE("amdgpu/gc_10_3_6_pfp.bin"); | ||
| 269 | MODULE_FIRMWARE("amdgpu/gc_10_3_6_me.bin"); | ||
| 270 | MODULE_FIRMWARE("amdgpu/gc_10_3_6_mec.bin"); | ||
| 271 | MODULE_FIRMWARE("amdgpu/gc_10_3_6_mec2.bin"); | ||
| 272 | MODULE_FIRMWARE("amdgpu/gc_10_3_6_rlc.bin"); | ||
| 273 | |||
| 274 | MODULE_FIRMWARE("amdgpu/gc_10_3_7_ce.bin"); | ||
| 275 | MODULE_FIRMWARE("amdgpu/gc_10_3_7_pfp.bin"); | ||
| 276 | MODULE_FIRMWARE("amdgpu/gc_10_3_7_me.bin"); | ||
| 277 | MODULE_FIRMWARE("amdgpu/gc_10_3_7_mec.bin"); | ||
| 278 | MODULE_FIRMWARE("amdgpu/gc_10_3_7_mec2.bin"); | ||
| 279 | MODULE_FIRMWARE("amdgpu/gc_10_3_7_rlc.bin"); | ||
| 280 | |||
| 273 | static const struct soc15_reg_golden golden_settings_gc_10_1[] = | 281 | static const struct soc15_reg_golden golden_settings_gc_10_1[] = |
| 274 | { | 282 | { |
| 275 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmCB_HW_CONTROL_4, 0xffffffff, 0x00400014), | 283 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmCB_HW_CONTROL_4, 0xffffffff, 0x00400014), |
| @@ -1463,143 +1471,6 @@ static const struct soc15_reg_golden golden_settings_gc_10_1_2[] = · 1187 unmodified lines | |||
| 1463 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00c00000) | 1471 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00c00000) |
| 1464 | }; | 1472 | }; |
| 1465 | 1473 | ||
| 1466 | static bool gfx_v10_get_rlcg_flag(struct amdgpu_device *adev, u32 acc_flags, u32 hwip, | ||
| 1467 | int write, u32 *rlcg_flag) | ||
| 1468 | { | ||
| 1469 | switch (hwip) { | ||
| 1470 | case GC_HWIP: | ||
| 1471 | if (amdgpu_sriov_reg_indirect_gc(adev)) { | ||
| 1472 | *rlcg_flag = write ? GFX_RLCG_GC_WRITE : GFX_RLCG_GC_READ; | ||
| 1473 | |||
| 1474 | return true; | ||
| 1475 | /* only in new version, AMDGPU_REGS_NO_KIQ and AMDGPU_REGS_RLC enabled simultaneously */ | ||
| 1476 | } else if ((acc_flags & AMDGPU_REGS_RLC) && !(acc_flags & AMDGPU_REGS_NO_KIQ)) { | ||
| 1477 | *rlcg_flag = GFX_RLCG_GC_WRITE_OLD; | ||
| 1478 | |||
| 1479 | return true; | ||
| 1480 | } | ||
| 1481 | |||
| 1482 | break; | ||
| 1483 | case MMHUB_HWIP: | ||
| 1484 | if (amdgpu_sriov_reg_indirect_mmhub(adev) && | ||
| 1485 | (acc_flags & AMDGPU_REGS_RLC) && write) { | ||
| 1486 | *rlcg_flag = GFX_RLCG_MMHUB_WRITE; | ||
| 1487 | return true; | ||
| 1488 | } | ||
| 1489 | |||
| 1490 | break; | ||
| 1491 | default: | ||
| 1492 | DRM_DEBUG("Not program register by RLCG\n"); | ||
| 1493 | } | ||
| 1494 | |||
| 1495 | return false; | ||
| 1496 | } | ||
| 1497 | |||
| 1498 | static u32 gfx_v10_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag) | ||
| 1499 | { | ||
| 1500 | static void *scratch_reg0; | ||
| 1501 | static void *scratch_reg1; | ||
| 1502 | static void *scratch_reg2; | ||
| 1503 | static void *scratch_reg3; | ||
| 1504 | static void *spare_int; | ||
| 1505 | static uint32_t grbm_cntl; | ||
| 1506 | static uint32_t grbm_idx; | ||
| 1507 | uint32_t i = 0; | ||
| 1508 | uint32_t retries = 50000; | ||
| 1509 | u32 ret = 0; | ||
| 1510 | u32 tmp; | ||
| 1511 | |||
| 1512 | scratch_reg0 = adev->rmmio + | ||
| 1513 | (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0) * 4; | ||
| 1514 | scratch_reg1 = adev->rmmio + | ||
| 1515 | (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1) * 4; | ||
| 1516 | scratch_reg2 = adev->rmmio + | ||
| 1517 | (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG2) * 4; | ||
| 1518 | scratch_reg3 = adev->rmmio + | ||
| 1519 | (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3) * 4; | ||
| 1520 | |||
| 1521 | if (adev->ip_versions[GC_HWIP][0] >= IP_VERSION(10, 3, 0)) { | ||
| 1522 | spare_int = adev->rmmio + | ||
| 1523 | (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_0_Sienna_Cichlid_BASE_IDX] | ||
| 1524 | + mmRLC_SPARE_INT_0_Sienna_Cichlid) * 4; | ||
| 1525 | } else { | ||
| 1526 | spare_int = adev->rmmio + | ||
| 1527 | (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT) * 4; | ||
| 1528 | } | ||
| 1529 | |||
| 1530 | grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL; | ||
| 1531 | grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX; | ||
| 1532 | |||
| 1533 | if (offset == grbm_cntl || offset == grbm_idx) { | ||
| 1534 | if (offset == grbm_cntl) | ||
| 1535 | writel(v, scratch_reg2); | ||
| 1536 | else if (offset == grbm_idx) | ||
| 1537 | writel(v, scratch_reg3); | ||
| 1538 | |||
| 1539 | writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); | ||
| 1540 | } else { | ||
| 1541 | writel(v, scratch_reg0); | ||
| 1542 | writel(offset | flag, scratch_reg1); | ||
| 1543 | writel(1, spare_int); | ||
| 1544 | |||
| 1545 | for (i = 0; i < retries; i++) { | ||
| 1546 | tmp = readl(scratch_reg1); | ||
| 1547 | if (!(tmp & flag)) | ||
| 1548 | break; | ||
| 1549 | |||
| 1550 | udelay(10); | ||
| 1551 | } | ||
| 1552 | |||
| 1553 | if (i >= retries) { | ||
| 1554 | if (RLCG_ERROR_REPORT_ENABLED(adev)) { | ||
| 1555 | if (tmp & RLCG_VFGATE_DISABLED) | ||
| 1556 | pr_err("The vfgate is disabled, program reg:0x%05x failed!\n", offset); | ||
| 1557 | else if (tmp & RLCG_WRONG_OPERATION_TYPE) | ||
| 1558 | pr_err("Wrong operation type, program reg:0x%05x failed!\n", offset); | ||
| 1559 | else if (tmp & RLCG_NOT_IN_RANGE) | ||
| 1560 | pr_err("The register is not in range, program reg:0x%05x failed!\n", offset); | ||
| 1561 | else | ||
| 1562 | pr_err("Unknown error type, program reg:0x%05x failed!\n", offset); | ||
| 1563 | } else | ||
| 1564 | pr_err("timeout: rlcg program reg:0x%05x failed!\n", offset); | ||
| 1565 | } | ||
| 1566 | } | ||
| 1567 | |||
| 1568 | ret = readl(scratch_reg0); | ||
| 1569 | |||
| 1570 | return ret; | ||
| 1571 | } | ||
| 1572 | |||
| 1573 | static void gfx_v10_sriov_wreg(struct amdgpu_device *adev, u32 offset, u32 value, u32 acc_flags, u32 hwip) | ||
| 1574 | { | ||
| 1575 | u32 rlcg_flag; | ||
| 1576 | |||
| 1577 | if (!amdgpu_sriov_runtime(adev) && | ||
| 1578 | gfx_v10_get_rlcg_flag(adev, acc_flags, hwip, 1, &rlcg_flag)) { | ||
| 1579 | gfx_v10_rlcg_rw(adev, offset, value, rlcg_flag); | ||
| 1580 | return; | ||
| 1581 | } | ||
| 1582 | |||
| 1583 | if (acc_flags & AMDGPU_REGS_NO_KIQ) | ||
| 1584 | WREG32_NO_KIQ(offset, value); | ||
| 1585 | else | ||
| 1586 | WREG32(offset, value); | ||
| 1587 | } | ||
| 1588 | |||
| 1589 | static u32 gfx_v10_sriov_rreg(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip) | ||
| 1590 | { | ||
| 1591 | u32 rlcg_flag; | ||
| 1592 | |||
| 1593 | if (!amdgpu_sriov_runtime(adev) && | ||
| 1594 | gfx_v10_get_rlcg_flag(adev, acc_flags, hwip, 0, &rlcg_flag)) | ||
| 1595 | return gfx_v10_rlcg_rw(adev, offset, 0, rlcg_flag); | ||
| 1596 | |||
| 1597 | if (acc_flags & AMDGPU_REGS_NO_KIQ) | ||
| 1598 | return RREG32_NO_KIQ(offset); | ||
| 1599 | else | ||
| 1600 | return RREG32(offset); | ||
| 1601 | } | ||
| 1602 | |||
| 1603 | static const struct soc15_reg_golden golden_settings_gc_10_1_nv14[] = | 1474 | static const struct soc15_reg_golden golden_settings_gc_10_1_nv14[] = |
| 1604 | { | 1475 | { |
| 1605 | /* Pending on emulation bring up */ | 1476 | /* Pending on emulation bring up */ |
| @@ -3557,6 +3428,57 @@ static const struct soc15_reg_golden golden_settings_gc_10_0_cyan_skillfish[] = · 1951 unmodified lines | |||
| 3557 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0x00800000, 0x00800000) | 3428 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0x00800000, 0x00800000) |
| 3558 | }; | 3429 | }; |
| 3559 | 3430 | ||
| 3431 | static const struct soc15_reg_golden golden_settings_gc_10_3_6[] = | ||
| 3432 | { | ||
| 3433 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmCGTT_SPI_CS_CLK_CTRL, 0xff7f0fff, 0x78000100), | ||
| 3434 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmCH_PIPE_STEER, 0x000000ff, 0x00000044), | ||
| 3435 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmCPF_GCR_CNTL, 0x0007ffff, 0x0000c200), | ||
| 3436 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG3, 0xffffffff, 0x00000280), | ||
| 3437 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG4, 0xffffffff, 0x00800000), | ||
| 3438 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGB_ADDR_CONFIG, 0x0c1807ff, 0x00000042), | ||
| 3439 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCR_GENERAL_CNTL, 0x1ff1ffff, 0x00000500), | ||
| 3440 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL1_PIPE_STEER, 0x000000ff, 0x00000044), | ||
| 3441 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_0, 0x77777777, 0x32103210), | ||
| 3442 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_1, 0x77777777, 0x32103210), | ||
| 3443 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2A_ADDR_MATCH_MASK, 0xffffffff, 0xfffffff3), | ||
| 3444 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_ADDR_MATCH_MASK, 0xffffffff, 0xfffffff3), | ||
| 3445 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CM_CTRL1, 0xff8fff0f, 0x580f1008), | ||
| 3446 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CTRL3, 0xf7ffffff, 0x00f80988), | ||
| 3447 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmLDS_CONFIG, 0x000001ff, 0x00000020), | ||
| 3448 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_CL_ENHANCE, 0xf17fffff, 0x01200007), | ||
| 3449 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_BINNER_TIMEOUT_COUNTER, 0xffffffff, 0x00000800), | ||
| 3450 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_ENHANCE_2, 0xffffffbf, 0x00000820), | ||
| 3451 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmSQG_CONFIG, 0x000017ff, 0x00001000), | ||
| 3452 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmSX_DEBUG_1, 0xffffff7f, 0x00010020), | ||
| 3453 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmTA_CNTL_AUX, 0xfff7ffff, 0x01030000), | ||
| 3454 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00100000) | ||
| 3455 | }; | ||
| 3456 | |||
| 3457 | static const struct soc15_reg_golden golden_settings_gc_10_3_7[] = { | ||
| 3458 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmCGTT_SPI_CS_CLK_CTRL, 0xff7f0fff, 0x78000100), | ||
| 3459 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmCH_PIPE_STEER, 0x000000ff, 0x000000e4), | ||
| 3460 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmCPF_GCR_CNTL, 0x0007ffff, 0x0000c200), | ||
| 3461 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG3, 0xffffffff, 0x00000280), | ||
| 3462 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG4, 0xffffffff, 0x00800000), | ||
| 3463 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGB_ADDR_CONFIG, 0x0c1807ff, 0x00000041), | ||
| 3464 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCR_GENERAL_CNTL, 0x1ff1ffff, 0x00000500), | ||
| 3465 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL1_PIPE_STEER, 0x000000ff, 0x000000e4), | ||
| 3466 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_0, 0x77777777, 0x32103210), | ||
| 3467 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2_PIPE_STEER_1, 0x77777777, 0x32103210), | ||
| 3468 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2A_ADDR_MATCH_MASK, 0xffffffff, 0xffffffff), | ||
| 3469 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_ADDR_MATCH_MASK, 0xffffffff, 0xffffffff), | ||
| 3470 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CM_CTRL1, 0xff8fff0f, 0x580f1008), | ||
| 3471 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_CTRL3, 0xf7ffffff, 0x00f80988), | ||
| 3472 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmLDS_CONFIG, 0x000001ff, 0x00000020), | ||
| 3473 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_CL_ENHANCE, 0xf000003f, 0x01200007), | ||
| 3474 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_BINNER_TIMEOUT_COUNTER, 0xffffffff, 0x00000800), | ||
| 3475 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmPA_SC_ENHANCE_2, 0xffffffbf, 0x00000820), | ||
| 3476 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmSQG_CONFIG, 0x000017ff, 0x00001000), | ||
| 3477 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmSX_DEBUG_1, 0xffffff7f, 0x00010020), | ||
| 3478 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmTA_CNTL_AUX, 0xfff7ffff, 0x01030000), | ||
| 3479 | SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00100000) | ||
| 3480 | }; | ||
| 3481 | |||
| 3560 | #define DEFAULT_SH_MEM_CONFIG \ | 3482 | #define DEFAULT_SH_MEM_CONFIG \ |
| 3561 | ((SH_MEM_ADDRESS_MODE_64 << SH_MEM_CONFIG__ADDRESS_MODE__SHIFT) | \ | 3483 | ((SH_MEM_ADDRESS_MODE_64 << SH_MEM_CONFIG__ADDRESS_MODE__SHIFT) | \ |
| 3562 | (SH_MEM_ALIGNMENT_MODE_UNALIGNED << SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT) | \ | 3484 | (SH_MEM_ALIGNMENT_MODE_UNALIGNED << SH_MEM_CONFIG__ALIGNMENT_MODE__SHIFT) | \ |
| @@ -3790,10 +3712,21 @@ static void gfx_v10_0_init_golden_registers(struct amdgpu_device *adev) · 227 unmodified lines | |||
| 3790 | (const u32)ARRAY_SIZE(golden_settings_gc_10_3_5)); | 3712 | (const u32)ARRAY_SIZE(golden_settings_gc_10_3_5)); |
| 3791 | break; | 3713 | break; |
| 3792 | case IP_VERSION(10, 1, 3): | 3714 | case IP_VERSION(10, 1, 3): |
| 3715 | case IP_VERSION(10, 1, 4): | ||
| 3793 | soc15_program_register_sequence(adev, | 3716 | soc15_program_register_sequence(adev, |
| 3794 | golden_settings_gc_10_0_cyan_skillfish, | 3717 | golden_settings_gc_10_0_cyan_skillfish, |
| 3795 | (const u32)ARRAY_SIZE(golden_settings_gc_10_0_cyan_skillfish)); | 3718 | (const u32)ARRAY_SIZE(golden_settings_gc_10_0_cyan_skillfish)); |
| 3796 | break; | 3719 | break; |
| 3720 | case IP_VERSION(10, 3, 6): | ||
| 3721 | soc15_program_register_sequence(adev, | ||
| 3722 | golden_settings_gc_10_3_6, | ||
| 3723 | (const u32)ARRAY_SIZE(golden_settings_gc_10_3_6)); | ||
| 3724 | break; | ||
| 3725 | case IP_VERSION(10, 3, 7): | ||
| 3726 | soc15_program_register_sequence(adev, | ||
| 3727 | golden_settings_gc_10_3_7, | ||
| 3728 | (const u32)ARRAY_SIZE(golden_settings_gc_10_3_7)); | ||
| 3729 | break; | ||
| 3797 | default: | 3730 | default: |
| 3798 | break; | 3731 | break; |
| 3799 | } | 3732 | } |
| @@ -3968,6 +3901,7 @@ static void gfx_v10_0_check_fw_write_wait(struct amdgpu_device *adev) · 168 unmodified lines | |||
| 3968 | case IP_VERSION(10, 1, 2): | 3901 | case IP_VERSION(10, 1, 2): |
| 3969 | case IP_VERSION(10, 1, 1): | 3902 | case IP_VERSION(10, 1, 1): |
| 3970 | case IP_VERSION(10, 1, 3): | 3903 | case IP_VERSION(10, 1, 3): |
| 3904 | case IP_VERSION(10, 1, 4): | ||
| 3971 | if ((adev->gfx.me_fw_version >= 0x00000046) && | 3905 | if ((adev->gfx.me_fw_version >= 0x00000046) && |
| 3972 | (adev->gfx.me_feature_version >= 27) && | 3906 | (adev->gfx.me_feature_version >= 27) && |
| 3973 | (adev->gfx.pfp_fw_version >= 0x00000068) && | 3907 | (adev->gfx.pfp_fw_version >= 0x00000068) && |
| @@ -3981,7 +3915,9 @@ static void gfx_v10_0_check_fw_write_wait(struct amdgpu_device *adev) · 7 unmodified lines | |||
| 3981 | case IP_VERSION(10, 3, 1): | 3915 | case IP_VERSION(10, 3, 1): |
| 3982 | case IP_VERSION(10, 3, 4): | 3916 | case IP_VERSION(10, 3, 4): |
| 3983 | case IP_VERSION(10, 3, 5): | 3917 | case IP_VERSION(10, 3, 5): |
| 3918 | case IP_VERSION(10, 3, 6): | ||
| 3984 | case IP_VERSION(10, 3, 3): | 3919 | case IP_VERSION(10, 3, 3): |
| 3920 | case IP_VERSION(10, 3, 7): | ||
| 3985 | adev->gfx.cp_fw_write_wait = true; | 3921 | adev->gfx.cp_fw_write_wait = true; |
| 3986 | break; | 3922 | break; |
| 3987 | default: | 3923 | default: |
| @@ -4102,12 +4038,19 @@ static int gfx_v10_0_init_microcode(struct amdgpu_device *adev) · 114 unmodified lines | |||
| 4102 | case IP_VERSION(10, 3, 3): | 4038 | case IP_VERSION(10, 3, 3): |
| 4103 | chip_name = "yellow_carp"; | 4039 | chip_name = "yellow_carp"; |
| 4104 | break; | 4040 | break; |
| 4041 | case IP_VERSION(10, 3, 6): | ||
| 4042 | chip_name = "gc_10_3_6"; | ||
| 4043 | break; | ||
| 4105 | case IP_VERSION(10, 1, 3): | 4044 | case IP_VERSION(10, 1, 3): |
| 4045 | case IP_VERSION(10, 1, 4): | ||
| 4106 | if (adev->apu_flags & AMD_APU_IS_CYAN_SKILLFISH2) | 4046 | if (adev->apu_flags & AMD_APU_IS_CYAN_SKILLFISH2) |
| 4107 | chip_name = "cyan_skillfish2"; | 4047 | chip_name = "cyan_skillfish2"; |
| 4108 | else | 4048 | else |
| 4109 | chip_name = "cyan_skillfish"; | 4049 | chip_name = "cyan_skillfish"; |
| 4110 | break; | 4050 | break; |
| 4051 | case IP_VERSION(10, 3, 7): | ||
| 4052 | chip_name = "gc_10_3_7"; | ||
| 4053 | break; | ||
| 4111 | default: | 4054 | default: |
| 4112 | BUG(); | 4055 | BUG(); |
| 4113 | } | 4056 | } |
| @@ -4448,6 +4391,30 @@ static void gfx_v10_0_rlc_fini(struct amdgpu_device *adev) · 334 unmodified lines | |||
| 4448 | (void **)&adev->gfx.rlc.cp_table_ptr); | 4391 | (void **)&adev->gfx.rlc.cp_table_ptr); |
| 4449 | } | 4392 | } |
| 4450 | 4393 | ||
| 4394 | static void gfx_v10_0_init_rlcg_reg_access_ctrl(struct amdgpu_device *adev) | ||
| 4395 | { | ||
| 4396 | struct amdgpu_rlcg_reg_access_ctrl *reg_access_ctrl; | ||
| 4397 | |||
| 4398 | reg_access_ctrl = &adev->gfx.rlc.reg_access_ctrl; | ||
| 4399 | reg_access_ctrl->scratch_reg0 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG0); | ||
| 4400 | reg_access_ctrl->scratch_reg1 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG1); | ||
| 4401 | reg_access_ctrl->scratch_reg2 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG2); | ||
| 4402 | reg_access_ctrl->scratch_reg3 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG3); | ||
| 4403 | reg_access_ctrl->grbm_cntl = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_CNTL); | ||
| 4404 | reg_access_ctrl->grbm_idx = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX); | ||
| 4405 | switch (adev->ip_versions[GC_HWIP][0]) { | ||
| 4406 | case IP_VERSION(10, 3, 0): | ||
| 4407 | reg_access_ctrl->spare_int = | ||
| 4408 | SOC15_REG_OFFSET(GC, 0, mmRLC_SPARE_INT_0_Sienna_Cichlid); | ||
| 4409 | break; | ||
| 4410 | default: | ||
| 4411 | reg_access_ctrl->spare_int = | ||
| 4412 | SOC15_REG_OFFSET(GC, 0, mmRLC_SPARE_INT); | ||
| 4413 | break; | ||
| 4414 | } | ||
| 4415 | adev->gfx.rlc.rlcg_reg_access_supported = true; | ||
| 4416 | } | ||
| 4417 | |||
| 4451 | static int gfx_v10_0_rlc_init(struct amdgpu_device *adev) | 4418 | static int gfx_v10_0_rlc_init(struct amdgpu_device *adev) |
| 4452 | { | 4419 | { |
| 4453 | const struct cs_section_def *cs_data; | 4420 | const struct cs_section_def *cs_data; |
| @@ -4468,6 +4435,7 @@ static int gfx_v10_0_rlc_init(struct amdgpu_device *adev) · 14 unmodified lines | |||
| 4468 | if (adev->gfx.rlc.funcs->update_spm_vmid) | 4435 | if (adev->gfx.rlc.funcs->update_spm_vmid) |
| 4469 | adev->gfx.rlc.funcs->update_spm_vmid(adev, 0xf); | 4436 | adev->gfx.rlc.funcs->update_spm_vmid(adev, 0xf); |
| 4470 | 4437 | ||
| 4438 | |||
| 4471 | return 0; | 4439 | return 0; |
| 4472 | } | 4440 | } |
| 4473 | 4441 | ||
| @@ -4678,7 +4646,9 @@ static void gfx_v10_0_gpu_early_init(struct amdgpu_device *adev) · 204 unmodified lines | |||
| 4678 | case IP_VERSION(10, 3, 1): | 4646 | case IP_VERSION(10, 3, 1): |
| 4679 | case IP_VERSION(10, 3, 4): | 4647 | case IP_VERSION(10, 3, 4): |
| 4680 | case IP_VERSION(10, 3, 5): | 4648 | case IP_VERSION(10, 3, 5): |
| 4649 | case IP_VERSION(10, 3, 6): | ||
| 4681 | case IP_VERSION(10, 3, 3): | 4650 | case IP_VERSION(10, 3, 3): |
| 4651 | case IP_VERSION(10, 3, 7): | ||
| 4682 | adev->gfx.config.max_hw_contexts = 8; | 4652 | adev->gfx.config.max_hw_contexts = 8; |
| 4683 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; | 4653 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 4684 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; | 4654 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -4689,6 +4659,7 @@ static void gfx_v10_0_gpu_early_init(struct amdgpu_device *adev) · 4 unmodified lines | |||
| 4689 | 1 << REG_GET_FIELD(gb_addr_config, GB_ADDR_CONFIG, NUM_PKRS); | 4659 | 1 << REG_GET_FIELD(gb_addr_config, GB_ADDR_CONFIG, NUM_PKRS); |
| 4690 | break; | 4660 | break; |
| 4691 | case IP_VERSION(10, 1, 3): | 4661 | case IP_VERSION(10, 1, 3): |
| 4662 | case IP_VERSION(10, 1, 4): | ||
| 4692 | adev->gfx.config.max_hw_contexts = 8; | 4663 | adev->gfx.config.max_hw_contexts = 8; |
| 4693 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; | 4664 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 4694 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; | 4665 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -4801,6 +4772,7 @@ static int gfx_v10_0_sw_init(void *handle) · 106 unmodified lines | |||
| 4801 | case IP_VERSION(10, 1, 1): | 4772 | case IP_VERSION(10, 1, 1): |
| 4802 | case IP_VERSION(10, 1, 2): | 4773 | case IP_VERSION(10, 1, 2): |
| 4803 | case IP_VERSION(10, 1, 3): | 4774 | case IP_VERSION(10, 1, 3): |
| 4775 | case IP_VERSION(10, 1, 4): | ||
| 4804 | adev->gfx.me.num_me = 1; | 4776 | adev->gfx.me.num_me = 1; |
| 4805 | adev->gfx.me.num_pipe_per_me = 1; | 4777 | adev->gfx.me.num_pipe_per_me = 1; |
| 4806 | adev->gfx.me.num_queue_per_pipe = 1; | 4778 | adev->gfx.me.num_queue_per_pipe = 1; |
| @@ -4813,7 +4785,9 @@ static int gfx_v10_0_sw_init(void *handle) · 6 unmodified lines | |||
| 4813 | case IP_VERSION(10, 3, 1): | 4785 | case IP_VERSION(10, 3, 1): |
| 4814 | case IP_VERSION(10, 3, 4): | 4786 | case IP_VERSION(10, 3, 4): |
| 4815 | case IP_VERSION(10, 3, 5): | 4787 | case IP_VERSION(10, 3, 5): |
| 4788 | case IP_VERSION(10, 3, 6): | ||
| 4816 | case IP_VERSION(10, 3, 3): | 4789 | case IP_VERSION(10, 3, 3): |
| 4790 | case IP_VERSION(10, 3, 7): | ||
| 4817 | adev->gfx.me.num_me = 1; | 4791 | adev->gfx.me.num_me = 1; |
| 4818 | adev->gfx.me.num_pipe_per_me = 1; | 4792 | adev->gfx.me.num_pipe_per_me = 1; |
| 4819 | adev->gfx.me.num_queue_per_pipe = 1; | 4793 | adev->gfx.me.num_queue_per_pipe = 1; |
| @@ -4865,10 +4839,14 @@ static int gfx_v10_0_sw_init(void *handle) · 45 unmodified lines | |||
| 4865 | if (r) | 4839 | if (r) |
| 4866 | return r; | 4840 | return r; |
| 4867 | 4841 | ||
| 4868 | r = gfx_v10_0_rlc_init(adev); | 4842 | if (adev->gfx.rlc.funcs) { |
| 4869 | if (r) { | 4843 | if (adev->gfx.rlc.funcs->init) { |
| 4870 | DRM_ERROR("Failed to init rlc BOs!\n"); | 4844 | r = adev->gfx.rlc.funcs->init(adev); |
| 4871 | return r; | 4845 | if (r) { |
| 4846 | dev_err(adev->dev, "Failed to init rlc BOs!\n"); | ||
| 4847 | return r; | ||
| 4848 | } | ||
| 4849 | } | ||
| 4872 | } | 4850 | } |
| 4873 | 4851 | ||
| 4874 | r = gfx_v10_0_mec_init(adev); | 4852 | r = gfx_v10_0_mec_init(adev); |
| @@ -5047,7 +5025,8 @@ static void gfx_v10_0_setup_rb(struct amdgpu_device *adev) · 172 unmodified lines | |||
| 5047 | for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) { | 5025 | for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) { |
| 5048 | bitmap = i * adev->gfx.config.max_sh_per_se + j; | 5026 | bitmap = i * adev->gfx.config.max_sh_per_se + j; |
| 5049 | if (((adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 0)) || | 5027 | if (((adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 0)) || |
| 5050 | (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3))) && | 5028 | (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) || |
| 5029 | (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 6))) && | ||
| 5051 | ((gfx_v10_3_get_disabled_sa(adev) >> bitmap) & 1)) | 5030 | ((gfx_v10_3_get_disabled_sa(adev) >> bitmap) & 1)) |
| 5052 | continue; | 5031 | continue; |
| 5053 | gfx_v10_0_select_se_sh(adev, i, j, 0xffffffff); | 5032 | gfx_v10_0_select_se_sh(adev, i, j, 0xffffffff); |
| @@ -6321,7 +6300,9 @@ static void gfx_v10_0_cp_gfx_set_doorbell(struct amdgpu_device *adev, · 1267 unmodified lines | |||
| 6321 | case IP_VERSION(10, 3, 1): | 6300 | case IP_VERSION(10, 3, 1): |
| 6322 | case IP_VERSION(10, 3, 4): | 6301 | case IP_VERSION(10, 3, 4): |
| 6323 | case IP_VERSION(10, 3, 5): | 6302 | case IP_VERSION(10, 3, 5): |
| 6303 | case IP_VERSION(10, 3, 6): | ||
| 6324 | case IP_VERSION(10, 3, 3): | 6304 | case IP_VERSION(10, 3, 3): |
| 6305 | case IP_VERSION(10, 3, 7): | ||
| 6325 | tmp = REG_SET_FIELD(0, CP_RB_DOORBELL_RANGE_LOWER, | 6306 | tmp = REG_SET_FIELD(0, CP_RB_DOORBELL_RANGE_LOWER, |
| 6326 | DOORBELL_RANGE_LOWER_Sienna_Cichlid, ring->doorbell_index); | 6307 | DOORBELL_RANGE_LOWER_Sienna_Cichlid, ring->doorbell_index); |
| 6327 | WREG32_SOC15(GC, 0, mmCP_RB_DOORBELL_RANGE_LOWER, tmp); | 6308 | WREG32_SOC15(GC, 0, mmCP_RB_DOORBELL_RANGE_LOWER, tmp); |
| @@ -6458,7 +6439,9 @@ static void gfx_v10_0_cp_compute_enable(struct amdgpu_device *adev, bool enable) · 130 unmodified lines | |||
| 6458 | case IP_VERSION(10, 3, 1): | 6439 | case IP_VERSION(10, 3, 1): |
| 6459 | case IP_VERSION(10, 3, 4): | 6440 | case IP_VERSION(10, 3, 4): |
| 6460 | case IP_VERSION(10, 3, 5): | 6441 | case IP_VERSION(10, 3, 5): |
| 6442 | case IP_VERSION(10, 3, 6): | ||
| 6461 | case IP_VERSION(10, 3, 3): | 6443 | case IP_VERSION(10, 3, 3): |
| 6444 | case IP_VERSION(10, 3, 7): | ||
| 6462 | WREG32_SOC15(GC, 0, mmCP_MEC_CNTL_Sienna_Cichlid, 0); | 6445 | WREG32_SOC15(GC, 0, mmCP_MEC_CNTL_Sienna_Cichlid, 0); |
| 6463 | break; | 6446 | break; |
| 6464 | default: | 6447 | default: |
| @@ -6472,7 +6455,9 @@ static void gfx_v10_0_cp_compute_enable(struct amdgpu_device *adev, bool enable) · 7 unmodified lines | |||
| 6472 | case IP_VERSION(10, 3, 1): | 6455 | case IP_VERSION(10, 3, 1): |
| 6473 | case IP_VERSION(10, 3, 4): | 6456 | case IP_VERSION(10, 3, 4): |
| 6474 | case IP_VERSION(10, 3, 5): | 6457 | case IP_VERSION(10, 3, 5): |
| 6458 | case IP_VERSION(10, 3, 6): | ||
| 6475 | case IP_VERSION(10, 3, 3): | 6459 | case IP_VERSION(10, 3, 3): |
| 6460 | case IP_VERSION(10, 3, 7): | ||
| 6476 | WREG32_SOC15(GC, 0, mmCP_MEC_CNTL_Sienna_Cichlid, | 6461 | WREG32_SOC15(GC, 0, mmCP_MEC_CNTL_Sienna_Cichlid, |
| 6477 | (CP_MEC_CNTL__MEC_ME1_HALT_MASK | | 6462 | (CP_MEC_CNTL__MEC_ME1_HALT_MASK | |
| 6478 | CP_MEC_CNTL__MEC_ME2_HALT_MASK)); | 6463 | CP_MEC_CNTL__MEC_ME2_HALT_MASK)); |
| @@ -6570,6 +6555,7 @@ static void gfx_v10_0_kiq_setting(struct amdgpu_ring *ring) · 91 unmodified lines | |||
| 6570 | case IP_VERSION(10, 3, 1): | 6555 | case IP_VERSION(10, 3, 1): |
| 6571 | case IP_VERSION(10, 3, 4): | 6556 | case IP_VERSION(10, 3, 4): |
| 6572 | case IP_VERSION(10, 3, 5): | 6557 | case IP_VERSION(10, 3, 5): |
| 6558 | case IP_VERSION(10, 3, 6): | ||
| 6573 | case IP_VERSION(10, 3, 3): | 6559 | case IP_VERSION(10, 3, 3): |
| 6574 | tmp = RREG32_SOC15(GC, 0, mmRLC_CP_SCHEDULERS_Sienna_Cichlid); | 6560 | tmp = RREG32_SOC15(GC, 0, mmRLC_CP_SCHEDULERS_Sienna_Cichlid); |
| 6575 | tmp &= 0xffffff00; | 6561 | tmp &= 0xffffff00; |
| @@ -7300,6 +7286,8 @@ static bool gfx_v10_0_check_grbm_cam_remapping(struct amdgpu_device *adev) · 724 unmodified lines | |||
| 7300 | break; | 7286 | break; |
| 7301 | case IP_VERSION(10, 3, 1): | 7287 | case IP_VERSION(10, 3, 1): |
| 7302 | case IP_VERSION(10, 3, 3): | 7288 | case IP_VERSION(10, 3, 3): |
| 7289 | case IP_VERSION(10, 3, 6): | ||
| 7290 | case IP_VERSION(10, 3, 7): | ||
| 7303 | return true; | 7291 | return true; |
| 7304 | default: | 7292 | default: |
| 7305 | data = RREG32_SOC15(GC, 0, mmVGT_ESGS_RING_SIZE); | 7293 | data = RREG32_SOC15(GC, 0, mmVGT_ESGS_RING_SIZE); |
| @@ -7334,7 +7322,9 @@ static void gfx_v10_0_setup_grbm_cam_remapping(struct amdgpu_device *adev) · 28 unmodified lines | |||
| 7334 | case IP_VERSION(10, 3, 1): | 7322 | case IP_VERSION(10, 3, 1): |
| 7335 | case IP_VERSION(10, 3, 4): | 7323 | case IP_VERSION(10, 3, 4): |
| 7336 | case IP_VERSION(10, 3, 5): | 7324 | case IP_VERSION(10, 3, 5): |
| 7325 | case IP_VERSION(10, 3, 6): | ||
| 7337 | case IP_VERSION(10, 3, 3): | 7326 | case IP_VERSION(10, 3, 3): |
| 7327 | case IP_VERSION(10, 3, 7): | ||
| 7338 | /* mmVGT_TF_RING_SIZE_UMD -> mmVGT_TF_RING_SIZE */ | 7328 | /* mmVGT_TF_RING_SIZE_UMD -> mmVGT_TF_RING_SIZE */ |
| 7339 | data = (SOC15_REG_OFFSET(GC, 0, mmVGT_TF_RING_SIZE_UMD) << | 7329 | data = (SOC15_REG_OFFSET(GC, 0, mmVGT_TF_RING_SIZE_UMD) << |
| 7340 | GRBM_CAM_DATA__CAM_ADDR__SHIFT) | | 7330 | GRBM_CAM_DATA__CAM_ADDR__SHIFT) | |
| @@ -7654,6 +7644,7 @@ static int gfx_v10_0_soft_reset(void *handle) · 313 unmodified lines | |||
| 7654 | case IP_VERSION(10, 3, 1): | 7644 | case IP_VERSION(10, 3, 1): |
| 7655 | case IP_VERSION(10, 3, 4): | 7645 | case IP_VERSION(10, 3, 4): |
| 7656 | case IP_VERSION(10, 3, 5): | 7646 | case IP_VERSION(10, 3, 5): |
| 7647 | case IP_VERSION(10, 3, 6): | ||
| 7657 | case IP_VERSION(10, 3, 3): | 7648 | case IP_VERSION(10, 3, 3): |
| 7658 | if (REG_GET_FIELD(tmp, GRBM_STATUS2, RLC_BUSY_Sienna_Cichlid)) | 7649 | if (REG_GET_FIELD(tmp, GRBM_STATUS2, RLC_BUSY_Sienna_Cichlid)) |
| 7659 | grbm_soft_reset = REG_SET_FIELD(grbm_soft_reset, | 7650 | grbm_soft_reset = REG_SET_FIELD(grbm_soft_reset, |
| @@ -7721,6 +7712,21 @@ static uint64_t gfx_v10_0_get_gpu_clock_counter(struct amdgpu_device *adev) · 61 unmodified lines | |||
| 7721 | preempt_enable(); | 7712 | preempt_enable(); |
| 7722 | clock = clock_lo | (clock_hi << 32ULL); | 7713 | clock = clock_lo | (clock_hi << 32ULL); |
| 7723 | break; | 7714 | break; |
| 7715 | case IP_VERSION(10, 3, 6): | ||
| 7716 | preempt_disable(); | ||
| 7717 | clock_hi = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6); | ||
| 7718 | clock_lo = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6); | ||
| 7719 | hi_check = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER_GC_10_3_6); | ||
| 7720 | /* The SMUIO TSC clock frequency is 100MHz, which sets 32-bit carry over | ||
| 7721 | * roughly every 42 seconds. | ||
| 7722 | */ | ||
| 7723 | if (hi_check != clock_hi) { | ||
| 7724 | clock_lo = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_LOWER_GC_10_3_6); | ||
| 7725 | clock_hi = hi_check; | ||
| 7726 | } | ||
| 7727 | preempt_enable(); | ||
| 7728 | clock = clock_lo | (clock_hi << 32ULL); | ||
| 7729 | break; | ||
| 7724 | default: | 7730 | default: |
| 7725 | preempt_disable(); | 7731 | preempt_disable(); |
| 7726 | clock_hi = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER); | 7732 | clock_hi = RREG32_SOC15_NO_KIQ(SMUIO, 0, mmGOLDEN_TSC_COUNT_UPPER); |
| @@ -7778,6 +7784,7 @@ static int gfx_v10_0_early_init(void *handle) · 51 unmodified lines | |||
| 7778 | case IP_VERSION(10, 1, 1): | 7784 | case IP_VERSION(10, 1, 1): |
| 7779 | case IP_VERSION(10, 1, 2): | 7785 | case IP_VERSION(10, 1, 2): |
| 7780 | case IP_VERSION(10, 1, 3): | 7786 | case IP_VERSION(10, 1, 3): |
| 7787 | case IP_VERSION(10, 1, 4): | ||
| 7781 | adev->gfx.num_gfx_rings = GFX10_NUM_GFX_RINGS_NV1X; | 7788 | adev->gfx.num_gfx_rings = GFX10_NUM_GFX_RINGS_NV1X; |
| 7782 | break; | 7789 | break; |
| 7783 | case IP_VERSION(10, 3, 0): | 7790 | case IP_VERSION(10, 3, 0): |
| @@ -7785,7 +7792,9 @@ static int gfx_v10_0_early_init(void *handle) · 1 unmodified line | |||
| 7785 | case IP_VERSION(10, 3, 1): | 7792 | case IP_VERSION(10, 3, 1): |
| 7786 | case IP_VERSION(10, 3, 4): | 7793 | case IP_VERSION(10, 3, 4): |
| 7787 | case IP_VERSION(10, 3, 5): | 7794 | case IP_VERSION(10, 3, 5): |
| 7795 | case IP_VERSION(10, 3, 6): | ||
| 7788 | case IP_VERSION(10, 3, 3): | 7796 | case IP_VERSION(10, 3, 3): |
| 7797 | case IP_VERSION(10, 3, 7): | ||
| 7789 | adev->gfx.num_gfx_rings = GFX10_NUM_GFX_RINGS_Sienna_Cichlid; | 7798 | adev->gfx.num_gfx_rings = GFX10_NUM_GFX_RINGS_Sienna_Cichlid; |
| 7790 | break; | 7799 | break; |
| 7791 | default: | 7800 | default: |
| @@ -7801,6 +7810,9 @@ static int gfx_v10_0_early_init(void *handle) · 9 unmodified lines | |||
| 7801 | gfx_v10_0_set_gds_init(adev); | 7810 | gfx_v10_0_set_gds_init(adev); |
| 7802 | gfx_v10_0_set_rlc_funcs(adev); | 7811 | gfx_v10_0_set_rlc_funcs(adev); |
| 7803 | 7812 | ||
| 7813 | /* init rlcg reg access ctrl */ | ||
| 7814 | gfx_v10_0_init_rlcg_reg_access_ctrl(adev); | ||
| 7815 | |||
| 7804 | return 0; | 7816 | return 0; |
| 7805 | } | 7817 | } |
| 7806 | 7818 | ||
| @@ -7843,6 +7855,7 @@ static void gfx_v10_0_set_safe_mode(struct amdgpu_device *adev) · 36 unmodified lines | |||
| 7843 | case IP_VERSION(10, 3, 1): | 7855 | case IP_VERSION(10, 3, 1): |
| 7844 | case IP_VERSION(10, 3, 4): | 7856 | case IP_VERSION(10, 3, 4): |
| 7845 | case IP_VERSION(10, 3, 5): | 7857 | case IP_VERSION(10, 3, 5): |
| 7858 | case IP_VERSION(10, 3, 6): | ||
| 7846 | case IP_VERSION(10, 3, 3): | 7859 | case IP_VERSION(10, 3, 3): |
| 7847 | WREG32_SOC15(GC, 0, mmRLC_SAFE_MODE_Sienna_Cichlid, data); | 7860 | WREG32_SOC15(GC, 0, mmRLC_SAFE_MODE_Sienna_Cichlid, data); |
| 7848 | 7861 | ||
| @@ -7879,6 +7892,7 @@ static void gfx_v10_0_unset_safe_mode(struct amdgpu_device *adev) · 30 unmodified lines | |||
| 7879 | case IP_VERSION(10, 3, 1): | 7892 | case IP_VERSION(10, 3, 1): |
| 7880 | case IP_VERSION(10, 3, 4): | 7893 | case IP_VERSION(10, 3, 4): |
| 7881 | case IP_VERSION(10, 3, 5): | 7894 | case IP_VERSION(10, 3, 5): |
| 7895 | case IP_VERSION(10, 3, 6): | ||
| 7882 | case IP_VERSION(10, 3, 3): | 7896 | case IP_VERSION(10, 3, 3): |
| 7883 | WREG32_SOC15(GC, 0, mmRLC_SAFE_MODE_Sienna_Cichlid, data); | 7897 | WREG32_SOC15(GC, 0, mmRLC_SAFE_MODE_Sienna_Cichlid, data); |
| 7884 | break; | 7898 | break; |
| @@ -8333,6 +8347,7 @@ static void gfx_v10_cntl_power_gating(struct amdgpu_device *adev, bool enable) · 448 unmodified lines | |||
| 8333 | switch (adev->ip_versions[GC_HWIP][0]) { | 8347 | switch (adev->ip_versions[GC_HWIP][0]) { |
| 8334 | case IP_VERSION(10, 3, 1): | 8348 | case IP_VERSION(10, 3, 1): |
| 8335 | case IP_VERSION(10, 3, 3): | 8349 | case IP_VERSION(10, 3, 3): |
| 8350 | case IP_VERSION(10, 3, 6): | ||
| 8336 | data = 0x4E20 & RLC_PG_DELAY_3__CGCG_ACTIVE_BEFORE_CGPG_MASK_Vangogh; | 8351 | data = 0x4E20 & RLC_PG_DELAY_3__CGCG_ACTIVE_BEFORE_CGPG_MASK_Vangogh; |
| 8337 | WREG32_SOC15(GC, 0, mmRLC_PG_DELAY_3, data); | 8352 | WREG32_SOC15(GC, 0, mmRLC_PG_DELAY_3, data); |
| 8338 | break; | 8353 | break; |
| @@ -8377,8 +8392,6 @@ static const struct amdgpu_rlc_funcs gfx_v10_0_rlc_funcs_sriov = { · 38 unmodified lines | |||
| 8377 | .reset = gfx_v10_0_rlc_reset, | 8392 | .reset = gfx_v10_0_rlc_reset, |
| 8378 | .start = gfx_v10_0_rlc_start, | 8393 | .start = gfx_v10_0_rlc_start, |
| 8379 | .update_spm_vmid = gfx_v10_0_update_spm_vmid, | 8394 | .update_spm_vmid = gfx_v10_0_update_spm_vmid, |
| 8380 | .sriov_wreg = gfx_v10_sriov_wreg, | ||
| 8381 | .sriov_rreg = gfx_v10_sriov_rreg, | ||
| 8382 | .is_rlcg_access_range = gfx_v10_0_is_rlcg_access_range, | 8395 | .is_rlcg_access_range = gfx_v10_0_is_rlcg_access_range, |
| 8383 | }; | 8396 | }; |
| 8384 | 8397 | ||
| @@ -8403,6 +8416,7 @@ static int gfx_v10_0_set_powergating_state(void *handle, · 18 unmodified lines | |||
| 8403 | break; | 8416 | break; |
| 8404 | case IP_VERSION(10, 3, 1): | 8417 | case IP_VERSION(10, 3, 1): |
| 8405 | case IP_VERSION(10, 3, 3): | 8418 | case IP_VERSION(10, 3, 3): |
| 8419 | case IP_VERSION(10, 3, 6): | ||
| 8406 | gfx_v10_cntl_pg(adev, enable); | 8420 | gfx_v10_cntl_pg(adev, enable); |
| 8407 | amdgpu_gfx_off_ctrl(adev, enable); | 8421 | amdgpu_gfx_off_ctrl(adev, enable); |
| 8408 | break; | 8422 | break; |
| @@ -8429,6 +8443,7 @@ static int gfx_v10_0_set_clockgating_state(void *handle, · 20 unmodified lines | |||
| 8429 | case IP_VERSION(10, 3, 1): | 8443 | case IP_VERSION(10, 3, 1): |
| 8430 | case IP_VERSION(10, 3, 4): | 8444 | case IP_VERSION(10, 3, 4): |
| 8431 | case IP_VERSION(10, 3, 5): | 8445 | case IP_VERSION(10, 3, 5): |
| 8446 | case IP_VERSION(10, 3, 6): | ||
| 8432 | case IP_VERSION(10, 3, 3): | 8447 | case IP_VERSION(10, 3, 3): |
| 8433 | gfx_v10_0_update_gfx_clock_gating(adev, | 8448 | gfx_v10_0_update_gfx_clock_gating(adev, |
| 8434 | state == AMD_CG_STATE_GATE); | 8449 | state == AMD_CG_STATE_GATE); |
| @@ -9537,11 +9552,14 @@ static void gfx_v10_0_set_rlc_funcs(struct amdgpu_device *adev) · 1102 unmodified lines | |||
| 9537 | case IP_VERSION(10, 1, 10): | 9552 | case IP_VERSION(10, 1, 10): |
| 9538 | case IP_VERSION(10, 1, 1): | 9553 | case IP_VERSION(10, 1, 1): |
| 9539 | case IP_VERSION(10, 1, 3): | 9554 | case IP_VERSION(10, 1, 3): |
| 9555 | case IP_VERSION(10, 1, 4): | ||
| 9540 | case IP_VERSION(10, 3, 2): | 9556 | case IP_VERSION(10, 3, 2): |
| 9541 | case IP_VERSION(10, 3, 1): | 9557 | case IP_VERSION(10, 3, 1): |
| 9542 | case IP_VERSION(10, 3, 4): | 9558 | case IP_VERSION(10, 3, 4): |
| 9543 | case IP_VERSION(10, 3, 5): | 9559 | case IP_VERSION(10, 3, 5): |
| 9560 | case IP_VERSION(10, 3, 6): | ||
| 9544 | case IP_VERSION(10, 3, 3): | 9561 | case IP_VERSION(10, 3, 3): |
| 9562 | case IP_VERSION(10, 3, 7): | ||
| 9545 | adev->gfx.rlc.funcs = &gfx_v10_0_rlc_funcs; | 9563 | adev->gfx.rlc.funcs = &gfx_v10_0_rlc_funcs; |
| 9546 | break; | 9564 | break; |
| 9547 | case IP_VERSION(10, 1, 2): | 9565 | case IP_VERSION(10, 1, 2): |
| @@ -9634,7 +9652,9 @@ static int gfx_v10_0_get_cu_info(struct amdgpu_device *adev, · 86 unmodified lines | |||
| 9634 | for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) { | 9652 | for (j = 0; j < adev->gfx.config.max_sh_per_se; j++) { |
| 9635 | bitmap = i * adev->gfx.config.max_sh_per_se + j; | 9653 | bitmap = i * adev->gfx.config.max_sh_per_se + j; |
| 9636 | if (((adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 0)) || | 9654 | if (((adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 0)) || |
| 9637 | (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3))) && | 9655 | (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) || |
| 9656 | (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 6)) || | ||
| 9657 | (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 7))) && | ||
| 9638 | ((gfx_v10_3_get_disabled_sa(adev) >> bitmap) & 1)) | 9658 | ((gfx_v10_3_get_disabled_sa(adev) >> bitmap) & 1)) |
| 9639 | continue; | 9659 | continue; |
| 9640 | mask = 1; | 9660 | mask = 1; |
| @@ -63,13 +63,6 @@ · 62 unmodified lines | |||
| 63 | #define mmGCEA_PROBE_MAP 0x070c | 63 | #define mmGCEA_PROBE_MAP 0x070c |
| 64 | #define mmGCEA_PROBE_MAP_BASE_IDX 0 | 64 | #define mmGCEA_PROBE_MAP_BASE_IDX 0 |
| 65 | 65 | ||
| 66 | #define GFX9_RLCG_GC_WRITE_OLD (0x8 << 28) | ||
| 67 | #define GFX9_RLCG_GC_WRITE (0x0 << 28) | ||
| 68 | #define GFX9_RLCG_GC_READ (0x1 << 28) | ||
| 69 | #define GFX9_RLCG_VFGATE_DISABLED 0x4000000 | ||
| 70 | #define GFX9_RLCG_WRONG_OPERATION_TYPE 0x2000000 | ||
| 71 | #define GFX9_RLCG_NOT_IN_RANGE 0x1000000 | ||
| 72 | |||
| 73 | MODULE_FIRMWARE("amdgpu/vega10_ce.bin"); | 66 | MODULE_FIRMWARE("amdgpu/vega10_ce.bin"); |
| 74 | MODULE_FIRMWARE("amdgpu/vega10_pfp.bin"); | 67 | MODULE_FIRMWARE("amdgpu/vega10_pfp.bin"); |
| 75 | MODULE_FIRMWARE("amdgpu/vega10_me.bin"); | 68 | MODULE_FIRMWARE("amdgpu/vega10_me.bin"); |
| @@ -746,128 +739,6 @@ static const u32 GFX_RLC_SRM_INDEX_CNTL_DATA_OFFSETS[] = · 670 unmodified lines | |||
| 746 | mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0, | 739 | mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0, |
| 747 | }; | 740 | }; |
| 748 | 741 | ||
| 749 | static u32 gfx_v9_0_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag) | ||
| 750 | { | ||
| 751 | static void *scratch_reg0; | ||
| 752 | static void *scratch_reg1; | ||
| 753 | static void *scratch_reg2; | ||
| 754 | static void *scratch_reg3; | ||
| 755 | static void *spare_int; | ||
| 756 | static uint32_t grbm_cntl; | ||
| 757 | static uint32_t grbm_idx; | ||
| 758 | uint32_t i = 0; | ||
| 759 | uint32_t retries = 50000; | ||
| 760 | u32 ret = 0; | ||
| 761 | u32 tmp; | ||
| 762 | |||
| 763 | scratch_reg0 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0)*4; | ||
| 764 | scratch_reg1 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1)*4; | ||
| 765 | scratch_reg2 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG2_BASE_IDX] + mmSCRATCH_REG2)*4; | ||
| 766 | scratch_reg3 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG3_BASE_IDX] + mmSCRATCH_REG3)*4; | ||
| 767 | spare_int = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT)*4; | ||
| 768 | |||
| 769 | grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL; | ||
| 770 | grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX; | ||
| 771 | |||
| 772 | if (offset == grbm_cntl || offset == grbm_idx) { | ||
| 773 | if (offset == grbm_cntl) | ||
| 774 | writel(v, scratch_reg2); | ||
| 775 | else if (offset == grbm_idx) | ||
| 776 | writel(v, scratch_reg3); | ||
| 777 | |||
| 778 | writel(v, ((void __iomem *)adev->rmmio) + (offset * 4)); | ||
| 779 | } else { | ||
| 780 | /* | ||
| 781 | * SCRATCH_REG0 = read/write value | ||
| 782 | * SCRATCH_REG1[30:28] = command | ||
| 783 | * SCRATCH_REG1[19:0] = address in dword | ||
| 784 | * SCRATCH_REG1[26:24] = Error reporting | ||
| 785 | */ | ||
| 786 | writel(v, scratch_reg0); | ||
| 787 | writel(offset | flag, scratch_reg1); | ||
| 788 | writel(1, spare_int); | ||
| 789 | |||
| 790 | for (i = 0; i < retries; i++) { | ||
| 791 | tmp = readl(scratch_reg1); | ||
| 792 | if (!(tmp & flag)) | ||
| 793 | break; | ||
| 794 | |||
| 795 | udelay(10); | ||
| 796 | } | ||
| 797 | |||
| 798 | if (i >= retries) { | ||
| 799 | if (amdgpu_sriov_reg_indirect_gc(adev)) { | ||
| 800 | if (tmp & GFX9_RLCG_VFGATE_DISABLED) | ||
| 801 | pr_err("The vfgate is disabled, program reg:0x%05x failed!\n", offset); | ||
| 802 | else if (tmp & GFX9_RLCG_WRONG_OPERATION_TYPE) | ||
| 803 | pr_err("Wrong operation type, program reg:0x%05x failed!\n", offset); | ||
| 804 | else if (tmp & GFX9_RLCG_NOT_IN_RANGE) | ||
| 805 | pr_err("The register is not in range, program reg:0x%05x failed!\n", offset); | ||
| 806 | else | ||
| 807 | pr_err("Unknown error type, program reg:0x%05x failed!\n", offset); | ||
| 808 | } else | ||
| 809 | pr_err("timeout: rlcg program reg:0x%05x failed!\n", offset); | ||
| 810 | } | ||
| 811 | } | ||
| 812 | |||
| 813 | ret = readl(scratch_reg0); | ||
| 814 | |||
| 815 | return ret; | ||
| 816 | } | ||
| 817 | |||
| 818 | static bool gfx_v9_0_get_rlcg_flag(struct amdgpu_device *adev, u32 acc_flags, u32 hwip, | ||
| 819 | int write, u32 *rlcg_flag) | ||
| 820 | { | ||
| 821 | |||
| 822 | switch (hwip) { | ||
| 823 | case GC_HWIP: | ||
| 824 | if (amdgpu_sriov_reg_indirect_gc(adev)) { | ||
| 825 | *rlcg_flag = write ? GFX9_RLCG_GC_WRITE : GFX9_RLCG_GC_READ; | ||
| 826 | |||
| 827 | return true; | ||
| 828 | /* only in new version, AMDGPU_REGS_NO_KIQ and AMDGPU_REGS_RLC enabled simultaneously */ | ||
| 829 | } else if ((acc_flags & AMDGPU_REGS_RLC) && !(acc_flags & AMDGPU_REGS_NO_KIQ) && write) { | ||
| 830 | *rlcg_flag = GFX9_RLCG_GC_WRITE_OLD; | ||
| 831 | return true; | ||
| 832 | } | ||
| 833 | |||
| 834 | break; | ||
| 835 | default: | ||
| 836 | return false; | ||
| 837 | } | ||
| 838 | |||
| 839 | return false; | ||
| 840 | } | ||
| 841 | |||
| 842 | static u32 gfx_v9_0_sriov_rreg(struct amdgpu_device *adev, u32 offset, u32 acc_flags, u32 hwip) | ||
| 843 | { | ||
| 844 | u32 rlcg_flag; | ||
| 845 | |||
| 846 | if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 0, &rlcg_flag)) | ||
| 847 | return gfx_v9_0_rlcg_rw(adev, offset, 0, rlcg_flag); | ||
| 848 | |||
| 849 | if (acc_flags & AMDGPU_REGS_NO_KIQ) | ||
| 850 | return RREG32_NO_KIQ(offset); | ||
| 851 | else | ||
| 852 | return RREG32(offset); | ||
| 853 | } | ||
| 854 | |||
| 855 | static void gfx_v9_0_sriov_wreg(struct amdgpu_device *adev, u32 offset, | ||
| 856 | u32 value, u32 acc_flags, u32 hwip) | ||
| 857 | { | ||
| 858 | u32 rlcg_flag; | ||
| 859 | |||
| 860 | if (!amdgpu_sriov_runtime(adev) && gfx_v9_0_get_rlcg_flag(adev, acc_flags, hwip, 1, &rlcg_flag)) { | ||
| 861 | gfx_v9_0_rlcg_rw(adev, offset, value, rlcg_flag); | ||
| 862 | return; | ||
| 863 | } | ||
| 864 | |||
| 865 | if (acc_flags & AMDGPU_REGS_NO_KIQ) | ||
| 866 | WREG32_NO_KIQ(offset, value); | ||
| 867 | else | ||
| 868 | WREG32(offset, value); | ||
| 869 | } | ||
| 870 | |||
| 871 | #define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042 | 742 | #define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042 |
| 872 | #define VEGA12_GB_ADDR_CONFIG_GOLDEN 0x24104041 | 743 | #define VEGA12_GB_ADDR_CONFIG_GOLDEN 0x24104041 |
| 873 | #define RAVEN_GB_ADDR_CONFIG_GOLDEN 0x24000042 | 744 | #define RAVEN_GB_ADDR_CONFIG_GOLDEN 0x24000042 |
| @@ -882,7 +753,7 @@ static int gfx_v9_0_get_cu_info(struct amdgpu_device *adev, · 8 unmodified lines | |||
| 882 | static uint64_t gfx_v9_0_get_gpu_clock_counter(struct amdgpu_device *adev); | 753 | static uint64_t gfx_v9_0_get_gpu_clock_counter(struct amdgpu_device *adev); |
| 883 | static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring); | 754 | static void gfx_v9_0_ring_emit_de_meta(struct amdgpu_ring *ring); |
| 884 | static u64 gfx_v9_0_ring_get_rptr_compute(struct amdgpu_ring *ring); | 755 | static u64 gfx_v9_0_ring_get_rptr_compute(struct amdgpu_ring *ring); |
| 885 | static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, | 756 | static void gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, |
| 886 | void *ras_error_status); | 757 | void *ras_error_status); |
| 887 | static int gfx_v9_0_ras_error_inject(struct amdgpu_device *adev, | 758 | static int gfx_v9_0_ras_error_inject(struct amdgpu_device *adev, |
| 888 | void *inject_if); | 759 | void *inject_if); |
| @@ -2008,6 +1879,21 @@ static int gfx_v9_0_cp_jump_table_num(struct amdgpu_device *adev) · 1119 unmodified lines | |||
| 2008 | return 4; | 1879 | return 4; |
| 2009 | } | 1880 | } |
| 2010 | 1881 | ||
| 1882 | static void gfx_v9_0_init_rlcg_reg_access_ctrl(struct amdgpu_device *adev) | ||
| 1883 | { | ||
| 1884 | struct amdgpu_rlcg_reg_access_ctrl *reg_access_ctrl; | ||
| 1885 | |||
| 1886 | reg_access_ctrl = &adev->gfx.rlc.reg_access_ctrl; | ||
| 1887 | reg_access_ctrl->scratch_reg0 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG0); | ||
| 1888 | reg_access_ctrl->scratch_reg1 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG1); | ||
| 1889 | reg_access_ctrl->scratch_reg2 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG2); | ||
| 1890 | reg_access_ctrl->scratch_reg3 = SOC15_REG_OFFSET(GC, 0, mmSCRATCH_REG3); | ||
| 1891 | reg_access_ctrl->grbm_cntl = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_CNTL); | ||
| 1892 | reg_access_ctrl->grbm_idx = SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX); | ||
| 1893 | reg_access_ctrl->spare_int = SOC15_REG_OFFSET(GC, 0, mmRLC_SPARE_INT); | ||
| 1894 | adev->gfx.rlc.rlcg_reg_access_supported = true; | ||
| 1895 | } | ||
| 1896 | |||
| 2011 | static int gfx_v9_0_rlc_init(struct amdgpu_device *adev) | 1897 | static int gfx_v9_0_rlc_init(struct amdgpu_device *adev) |
| 2012 | { | 1898 | { |
| 2013 | const struct cs_section_def *cs_data; | 1899 | const struct cs_section_def *cs_data; |
| @@ -2197,12 +2083,16 @@ static const struct amdgpu_gfx_funcs gfx_v9_0_gfx_funcs = { · 183 unmodified lines | |||
| 2197 | .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q, | 2083 | .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q, |
| 2198 | }; | 2084 | }; |
| 2199 | 2085 | ||
| 2200 | static const struct amdgpu_gfx_ras_funcs gfx_v9_0_ras_funcs = { | 2086 | const struct amdgpu_ras_block_hw_ops gfx_v9_0_ras_ops = { |
| 2201 | .ras_late_init = amdgpu_gfx_ras_late_init, | 2087 | .ras_error_inject = &gfx_v9_0_ras_error_inject, |
| 2202 | .ras_fini = amdgpu_gfx_ras_fini, | 2088 | .query_ras_error_count = &gfx_v9_0_query_ras_error_count, |
| 2203 | .ras_error_inject = &gfx_v9_0_ras_error_inject, | 2089 | .reset_ras_error_count = &gfx_v9_0_reset_ras_error_count, |
| 2204 | .query_ras_error_count = &gfx_v9_0_query_ras_error_count, | 2090 | }; |
| 2205 | .reset_ras_error_count = &gfx_v9_0_reset_ras_error_count, | 2091 | |
| 2092 | static struct amdgpu_gfx_ras gfx_v9_0_ras = { | ||
| 2093 | .ras_block = { | ||
| 2094 | .hw_ops = &gfx_v9_0_ras_ops, | ||
| 2095 | }, | ||
| 2206 | }; | 2096 | }; |
| 2207 | 2097 | ||
| 2208 | static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) | 2098 | static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) |
| @@ -2231,7 +2121,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 22 unmodified lines | |||
| 2231 | DRM_INFO("fix gfx.config for vega12\n"); | 2121 | DRM_INFO("fix gfx.config for vega12\n"); |
| 2232 | break; | 2122 | break; |
| 2233 | case IP_VERSION(9, 4, 0): | 2123 | case IP_VERSION(9, 4, 0): |
| 2234 | adev->gfx.ras_funcs = &gfx_v9_0_ras_funcs; | 2124 | adev->gfx.ras = &gfx_v9_0_ras; |
| 2235 | adev->gfx.config.max_hw_contexts = 8; | 2125 | adev->gfx.config.max_hw_contexts = 8; |
| 2236 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; | 2126 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 2237 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; | 2127 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -2258,7 +2148,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 20 unmodified lines | |||
| 2258 | gb_addr_config = RAVEN_GB_ADDR_CONFIG_GOLDEN; | 2148 | gb_addr_config = RAVEN_GB_ADDR_CONFIG_GOLDEN; |
| 2259 | break; | 2149 | break; |
| 2260 | case IP_VERSION(9, 4, 1): | 2150 | case IP_VERSION(9, 4, 1): |
| 2261 | adev->gfx.ras_funcs = &gfx_v9_4_ras_funcs; | 2151 | adev->gfx.ras = &gfx_v9_4_ras; |
| 2262 | adev->gfx.config.max_hw_contexts = 8; | 2152 | adev->gfx.config.max_hw_contexts = 8; |
| 2263 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; | 2153 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 2264 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; | 2154 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -2279,7 +2169,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 14 unmodified lines | |||
| 2279 | gb_addr_config |= 0x22010042; | 2169 | gb_addr_config |= 0x22010042; |
| 2280 | break; | 2170 | break; |
| 2281 | case IP_VERSION(9, 4, 2): | 2171 | case IP_VERSION(9, 4, 2): |
| 2282 | adev->gfx.ras_funcs = &gfx_v9_4_2_ras_funcs; | 2172 | adev->gfx.ras = &gfx_v9_4_2_ras; |
| 2283 | adev->gfx.config.max_hw_contexts = 8; | 2173 | adev->gfx.config.max_hw_contexts = 8; |
| 2284 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; | 2174 | adev->gfx.config.sc_prim_fifo_size_frontend = 0x20; |
| 2285 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; | 2175 | adev->gfx.config.sc_prim_fifo_size_backend = 0x100; |
| @@ -2298,6 +2188,31 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev) · 12 unmodified lines | |||
| 2298 | break; | 2188 | break; |
| 2299 | } | 2189 | } |
| 2300 | 2190 | ||
| 2191 | if (adev->gfx.ras) { | ||
| 2192 | err = amdgpu_ras_register_ras_block(adev, &adev->gfx.ras->ras_block); | ||
| 2193 | if (err) { | ||
| 2194 | DRM_ERROR("Failed to register gfx ras block!\n"); | ||
| 2195 | return err; | ||
| 2196 | } | ||
| 2197 | |||
| 2198 | strcpy(adev->gfx.ras->ras_block.ras_comm.name, "gfx"); | ||
| 2199 | adev->gfx.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__GFX; | ||
| 2200 | adev->gfx.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 2201 | adev->gfx.ras_if = &adev->gfx.ras->ras_block.ras_comm; | ||
| 2202 | |||
| 2203 | /* If not define special ras_late_init function, use gfx default ras_late_init */ | ||
| 2204 | if (!adev->gfx.ras->ras_block.ras_late_init) | ||
| 2205 | adev->gfx.ras->ras_block.ras_late_init = amdgpu_gfx_ras_late_init; | ||
| 2206 | |||
| 2207 | /* If not define special ras_fini function, use gfx default ras_fini */ | ||
| 2208 | if (!adev->gfx.ras->ras_block.ras_fini) | ||
| 2209 | adev->gfx.ras->ras_block.ras_fini = amdgpu_gfx_ras_fini; | ||
| 2210 | |||
| 2211 | /* If not defined special ras_cb function, use default ras_cb */ | ||
| 2212 | if (!adev->gfx.ras->ras_block.ras_cb) | ||
| 2213 | adev->gfx.ras->ras_block.ras_cb = amdgpu_gfx_process_ras_data_cb; | ||
| 2214 | } | ||
| 2215 | |||
| 2301 | adev->gfx.config.gb_addr_config = gb_addr_config; | 2216 | adev->gfx.config.gb_addr_config = gb_addr_config; |
| 2302 | 2217 | ||
| 2303 | adev->gfx.config.gb_addr_config_fields.num_pipes = 1 << | 2218 | adev->gfx.config.gb_addr_config_fields.num_pipes = 1 << |
| @@ -2434,10 +2349,14 @@ static int gfx_v9_0_sw_init(void *handle) · 130 unmodified lines | |||
| 2434 | return r; | 2349 | return r; |
| 2435 | } | 2350 | } |
| 2436 | 2351 | ||
| 2437 | r = adev->gfx.rlc.funcs->init(adev); | 2352 | if (adev->gfx.rlc.funcs) { |
| 2438 | if (r) { | 2353 | if (adev->gfx.rlc.funcs->init) { |
| 2439 | DRM_ERROR("Failed to init rlc BOs!\n"); | 2354 | r = adev->gfx.rlc.funcs->init(adev); |
| 2440 | return r; | 2355 | if (r) { |
| 2356 | dev_err(adev->dev, "Failed to init rlc BOs!\n"); | ||
| 2357 | return r; | ||
| 2358 | } | ||
| 2359 | } | ||
| 2441 | } | 2360 | } |
| 2442 | 2361 | ||
| 2443 | r = gfx_v9_0_mec_init(adev); | 2362 | r = gfx_v9_0_mec_init(adev); |
| @@ -2513,9 +2432,8 @@ static int gfx_v9_0_sw_fini(void *handle) · 69 unmodified lines | |||
| 2513 | int i; | 2432 | int i; |
| 2514 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 2433 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 2515 | 2434 | ||
| 2516 | if (adev->gfx.ras_funcs && | 2435 | if (adev->gfx.ras && adev->gfx.ras->ras_block.ras_fini) |
| 2517 | adev->gfx.ras_funcs->ras_fini) | 2436 | adev->gfx.ras->ras_block.ras_fini(adev); |
| 2518 | adev->gfx.ras_funcs->ras_fini(adev); | ||
| 2519 | 2437 | ||
| 2520 | for (i = 0; i < adev->gfx.num_gfx_rings; i++) | 2438 | for (i = 0; i < adev->gfx.num_gfx_rings; i++) |
| 2521 | amdgpu_ring_fini(&adev->gfx.gfx_ring[i]); | 2439 | amdgpu_ring_fini(&adev->gfx.gfx_ring[i]); |
| @@ -4840,6 +4758,9 @@ static int gfx_v9_0_early_init(void *handle) · 2318 unmodified lines | |||
| 4840 | gfx_v9_0_set_gds_init(adev); | 4758 | gfx_v9_0_set_gds_init(adev); |
| 4841 | gfx_v9_0_set_rlc_funcs(adev); | 4759 | gfx_v9_0_set_rlc_funcs(adev); |
| 4842 | 4760 | ||
| 4761 | /* init rlcg reg access ctrl */ | ||
| 4762 | gfx_v9_0_init_rlcg_reg_access_ctrl(adev); | ||
| 4763 | |||
| 4843 | return 0; | 4764 | return 0; |
| 4844 | } | 4765 | } |
| 4845 | 4766 | ||
| @@ -4870,16 +4791,9 @@ static int gfx_v9_0_ecc_late_init(void *handle) · 24 unmodified lines | |||
| 4870 | if (r) | 4791 | if (r) |
| 4871 | return r; | 4792 | return r; |
| 4872 | 4793 | ||
| 4873 | if (adev->gfx.ras_funcs && | 4794 | if (adev->gfx.ras && |
| 4874 | adev->gfx.ras_funcs->ras_late_init) { | 4795 | adev->gfx.ras->enable_watchdog_timer) |
| 4875 | r = adev->gfx.ras_funcs->ras_late_init(adev); | 4796 | adev->gfx.ras->enable_watchdog_timer(adev); |
| 4876 | if (r) | ||
| 4877 | return r; | ||
| 4878 | } | ||
| 4879 | |||
| 4880 | if (adev->gfx.ras_funcs && | ||
| 4881 | adev->gfx.ras_funcs->enable_watchdog_timer) | ||
| 4882 | adev->gfx.ras_funcs->enable_watchdog_timer(adev); | ||
| 4883 | 4797 | ||
| 4884 | return 0; | 4798 | return 0; |
| 4885 | } | 4799 | } |
| @@ -5250,8 +5164,6 @@ static const struct amdgpu_rlc_funcs gfx_v9_0_rlc_funcs = { · 364 unmodified lines | |||
| 5250 | .reset = gfx_v9_0_rlc_reset, | 5164 | .reset = gfx_v9_0_rlc_reset, |
| 5251 | .start = gfx_v9_0_rlc_start, | 5165 | .start = gfx_v9_0_rlc_start, |
| 5252 | .update_spm_vmid = gfx_v9_0_update_spm_vmid, | 5166 | .update_spm_vmid = gfx_v9_0_update_spm_vmid, |
| 5253 | .sriov_wreg = gfx_v9_0_sriov_wreg, | ||
| 5254 | .sriov_rreg = gfx_v9_0_sriov_rreg, | ||
| 5255 | .is_rlcg_access_range = gfx_v9_0_is_rlcg_access_range, | 5167 | .is_rlcg_access_range = gfx_v9_0_is_rlcg_access_range, |
| 5256 | }; | 5168 | }; |
| 5257 | 5169 | ||
| @@ -6819,7 +6731,7 @@ static void gfx_v9_0_reset_ras_error_count(struct amdgpu_device *adev) · 1561 unmodified lines | |||
| 6819 | WREG32_SOC15(GC, 0, mmATC_L2_CACHE_4K_EDC_INDEX, 255); | 6731 | WREG32_SOC15(GC, 0, mmATC_L2_CACHE_4K_EDC_INDEX, 255); |
| 6820 | } | 6732 | } |
| 6821 | 6733 | ||
| 6822 | static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, | 6734 | static void gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, |
| 6823 | void *ras_error_status) | 6735 | void *ras_error_status) |
| 6824 | { | 6736 | { |
| 6825 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | 6737 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| @@ -6828,7 +6740,7 @@ static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 6828 | uint32_t reg_value; | 6740 | uint32_t reg_value; |
| 6829 | 6741 | ||
| 6830 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) | 6742 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) |
| 6831 | return -EINVAL; | 6743 | return; |
| 6832 | 6744 | ||
| 6833 | err_data->ue_count = 0; | 6745 | err_data->ue_count = 0; |
| 6834 | err_data->ce_count = 0; | 6746 | err_data->ce_count = 0; |
| @@ -6857,8 +6769,6 @@ static int gfx_v9_0_query_ras_error_count(struct amdgpu_device *adev, · 22 unmodified lines | |||
| 6857 | mutex_unlock(&adev->grbm_idx_mutex); | 6769 | mutex_unlock(&adev->grbm_idx_mutex); |
| 6858 | 6770 | ||
| 6859 | gfx_v9_0_query_utc_edc_status(adev, err_data); | 6771 | gfx_v9_0_query_utc_edc_status(adev, err_data); |
| 6860 | |||
| 6861 | return 0; | ||
| 6862 | } | 6772 | } |
| 6863 | 6773 | ||
| 6864 | static void gfx_v9_0_emit_mem_sync(struct amdgpu_ring *ring) | 6774 | static void gfx_v9_0_emit_mem_sync(struct amdgpu_ring *ring) |
| @@ -863,7 +863,7 @@ static int gfx_v9_4_ras_error_count(struct amdgpu_device *adev, · 862 unmodified lines | |||
| 863 | return 0; | 863 | return 0; |
| 864 | } | 864 | } |
| 865 | 865 | ||
| 866 | static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, | 866 | static void gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, |
| 867 | void *ras_error_status) | 867 | void *ras_error_status) |
| 868 | { | 868 | { |
| 869 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | 869 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| @@ -872,7 +872,7 @@ static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 872 | uint32_t reg_value; | 872 | uint32_t reg_value; |
| 873 | 873 | ||
| 874 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) | 874 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) |
| 875 | return -EINVAL; | 875 | return; |
| 876 | 876 | ||
| 877 | err_data->ue_count = 0; | 877 | err_data->ue_count = 0; |
| 878 | err_data->ce_count = 0; | 878 | err_data->ce_count = 0; |
| @@ -903,7 +903,6 @@ static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev, · 24 unmodified lines | |||
| 903 | 903 | ||
| 904 | gfx_v9_4_query_utc_edc_status(adev, err_data); | 904 | gfx_v9_4_query_utc_edc_status(adev, err_data); |
| 905 | 905 | ||
| 906 | return 0; | ||
| 907 | } | 906 | } |
| 908 | 907 | ||
| 909 | static void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev) | 908 | static void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev) |
| @@ -1029,11 +1028,16 @@ static void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev) · 119 unmodified lines | |||
| 1029 | mutex_unlock(&adev->grbm_idx_mutex); | 1028 | mutex_unlock(&adev->grbm_idx_mutex); |
| 1030 | } | 1029 | } |
| 1031 | 1030 | ||
| 1032 | const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs = { | 1031 | |
| 1033 | .ras_late_init = amdgpu_gfx_ras_late_init, | 1032 | const struct amdgpu_ras_block_hw_ops gfx_v9_4_ras_ops = { |
| 1034 | .ras_fini = amdgpu_gfx_ras_fini, | 1033 | .ras_error_inject = &gfx_v9_4_ras_error_inject, |
| 1035 | .ras_error_inject = &gfx_v9_4_ras_error_inject, | 1034 | .query_ras_error_count = &gfx_v9_4_query_ras_error_count, |
| 1036 | .query_ras_error_count = &gfx_v9_4_query_ras_error_count, | 1035 | .reset_ras_error_count = &gfx_v9_4_reset_ras_error_count, |
| 1037 | .reset_ras_error_count = &gfx_v9_4_reset_ras_error_count, | 1036 | .query_ras_error_status = &gfx_v9_4_query_ras_error_status, |
| 1038 | .query_ras_error_status = &gfx_v9_4_query_ras_error_status, | 1037 | }; |
| 1038 | |||
| 1039 | struct amdgpu_gfx_ras gfx_v9_4_ras = { | ||
| 1040 | .ras_block = { | ||
| 1041 | .hw_ops = &gfx_v9_4_ras_ops, | ||
| 1042 | }, | ||
| 1039 | }; | 1043 | }; |
| @@ -24,6 +24,6 @@ · 23 unmodified lines | |||
| 24 | #ifndef __GFX_V9_4_H__ | 24 | #ifndef __GFX_V9_4_H__ |
| 25 | #define __GFX_V9_4_H__ | 25 | #define __GFX_V9_4_H__ |
| 26 | 26 | ||
| 27 | extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs; | 27 | extern struct amdgpu_gfx_ras gfx_v9_4_ras; |
| 28 | 28 | ||
| 29 | #endif /* __GFX_V9_4_H__ */ | 29 | #endif /* __GFX_V9_4_H__ */ |
| @@ -1641,14 +1641,14 @@ static int gfx_v9_4_2_query_utc_edc_count(struct amdgpu_device *adev, · 1640 unmodified lines | |||
| 1641 | return 0; | 1641 | return 0; |
| 1642 | } | 1642 | } |
| 1643 | 1643 | ||
| 1644 | static int gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev, | 1644 | static void gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev, |
| 1645 | void *ras_error_status) | 1645 | void *ras_error_status) |
| 1646 | { | 1646 | { |
| 1647 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | 1647 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| 1648 | uint32_t sec_count = 0, ded_count = 0; | 1648 | uint32_t sec_count = 0, ded_count = 0; |
| 1649 | 1649 | ||
| 1650 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) | 1650 | if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) |
| 1651 | return -EINVAL; | 1651 | return; |
| 1652 | 1652 | ||
| 1653 | err_data->ue_count = 0; | 1653 | err_data->ue_count = 0; |
| 1654 | err_data->ce_count = 0; | 1654 | err_data->ce_count = 0; |
| @@ -1661,7 +1661,6 @@ static int gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev, · 6 unmodified lines | |||
| 1661 | err_data->ce_count += sec_count; | 1661 | err_data->ce_count += sec_count; |
| 1662 | err_data->ue_count += ded_count; | 1662 | err_data->ue_count += ded_count; |
| 1663 | 1663 | ||
| 1664 | return 0; | ||
| 1665 | } | 1664 | } |
| 1666 | 1665 | ||
| 1667 | static void gfx_v9_4_2_reset_utc_err_status(struct amdgpu_device *adev) | 1666 | static void gfx_v9_4_2_reset_utc_err_status(struct amdgpu_device *adev) |
| @@ -1931,13 +1930,17 @@ static void gfx_v9_4_2_reset_sq_timeout_status(struct amdgpu_device *adev) · 263 unmodified lines | |||
| 1931 | mutex_unlock(&adev->grbm_idx_mutex); | 1930 | mutex_unlock(&adev->grbm_idx_mutex); |
| 1932 | } | 1931 | } |
| 1933 | 1932 | ||
| 1934 | const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs = { | 1933 | struct amdgpu_ras_block_hw_ops gfx_v9_4_2_ras_ops = { |
| 1935 | .ras_late_init = amdgpu_gfx_ras_late_init, | 1934 | .ras_error_inject = &gfx_v9_4_2_ras_error_inject, |
| 1936 | .ras_fini = amdgpu_gfx_ras_fini, | 1935 | .query_ras_error_count = &gfx_v9_4_2_query_ras_error_count, |
| 1937 | .ras_error_inject = &gfx_v9_4_2_ras_error_inject, | 1936 | .reset_ras_error_count = &gfx_v9_4_2_reset_ras_error_count, |
| 1938 | .query_ras_error_count = &gfx_v9_4_2_query_ras_error_count, | 1937 | .query_ras_error_status = &gfx_v9_4_2_query_ras_error_status, |
| 1939 | .reset_ras_error_count = &gfx_v9_4_2_reset_ras_error_count, | 1938 | .reset_ras_error_status = &gfx_v9_4_2_reset_ras_error_status, |
| 1940 | .query_ras_error_status = &gfx_v9_4_2_query_ras_error_status, | 1939 | }; |
| 1941 | .reset_ras_error_status = &gfx_v9_4_2_reset_ras_error_status, | 1940 | |
| 1941 | struct amdgpu_gfx_ras gfx_v9_4_2_ras = { | ||
| 1942 | .ras_block = { | ||
| 1943 | .hw_ops = &gfx_v9_4_2_ras_ops, | ||
| 1944 | }, | ||
| 1942 | .enable_watchdog_timer = &gfx_v9_4_2_enable_watchdog_timer, | 1945 | .enable_watchdog_timer = &gfx_v9_4_2_enable_watchdog_timer, |
| 1943 | }; | 1946 | }; |
| @@ -31,6 +31,6 @@ void gfx_v9_4_2_init_golden_registers(struct amdgpu_device *adev, · 30 unmodified lines | |||
| 31 | void gfx_v9_4_2_set_power_brake_sequence(struct amdgpu_device *adev); | 31 | void gfx_v9_4_2_set_power_brake_sequence(struct amdgpu_device *adev); |
| 32 | int gfx_v9_4_2_do_edc_gpr_workarounds(struct amdgpu_device *adev); | 32 | int gfx_v9_4_2_do_edc_gpr_workarounds(struct amdgpu_device *adev); |
| 33 | 33 | ||
| 34 | extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs; | 34 | extern struct amdgpu_gfx_ras gfx_v9_4_2_ras; |
| 35 | 35 | ||
| 36 | #endif /* __GFX_V9_4_2_H__ */ | 36 | #endif /* __GFX_V9_4_2_H__ */ |
| @@ -543,7 +543,9 @@ static void gfxhub_v2_1_utcl2_harvest(struct amdgpu_device *adev) · 542 unmodified lines | |||
| 543 | adev->gfx.config.max_sh_per_se * | 543 | adev->gfx.config.max_sh_per_se * |
| 544 | adev->gfx.config.max_shader_engines); | 544 | adev->gfx.config.max_shader_engines); |
| 545 | 545 | ||
| 546 | if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 3, 3)) { | 546 | switch (adev->ip_versions[GC_HWIP][0]) { |
| 547 | case IP_VERSION(10, 3, 1): | ||
| 548 | case IP_VERSION(10, 3, 3): | ||
| 547 | /* Get SA disabled bitmap from eFuse setting */ | 549 | /* Get SA disabled bitmap from eFuse setting */ |
| 548 | efuse_setting = RREG32_SOC15(GC, 0, mmCC_GC_SA_UNIT_DISABLE); | 550 | efuse_setting = RREG32_SOC15(GC, 0, mmCC_GC_SA_UNIT_DISABLE); |
| 549 | efuse_setting &= CC_GC_SA_UNIT_DISABLE__SA_DISABLE_MASK; | 551 | efuse_setting &= CC_GC_SA_UNIT_DISABLE__SA_DISABLE_MASK; |
| @@ -566,6 +568,9 @@ static void gfxhub_v2_1_utcl2_harvest(struct amdgpu_device *adev) · 16 unmodified lines | |||
| 566 | disabled_sa = tmp; | 568 | disabled_sa = tmp; |
| 567 | 569 | ||
| 568 | WREG32_SOC15(GC, 0, mmGCUTCL2_HARVEST_BYPASS_GROUPS_YELLOW_CARP, disabled_sa); | 570 | WREG32_SOC15(GC, 0, mmGCUTCL2_HARVEST_BYPASS_GROUPS_YELLOW_CARP, disabled_sa); |
| 571 | break; | ||
| 572 | default: | ||
| 573 | break; | ||
| 569 | } | 574 | } |
| 570 | } | 575 | } |
| 571 | 576 | ||
| @@ -48,6 +48,8 @@ · 47 unmodified lines | |||
| 48 | #include "athub_v2_0.h" | 48 | #include "athub_v2_0.h" |
| 49 | #include "athub_v2_1.h" | 49 | #include "athub_v2_1.h" |
| 50 | 50 | ||
| 51 | #include "amdgpu_reset.h" | ||
| 52 | |||
| 51 | #if 0 | 53 | #if 0 |
| 52 | static const struct soc15_reg_golden golden_settings_navi10_hdp[] = | 54 | static const struct soc15_reg_golden golden_settings_navi10_hdp[] = |
| 53 | { | 55 | { |
| @@ -328,7 +330,7 @@ static void gmc_v10_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 274 unmodified lines | |||
| 328 | */ | 330 | */ |
| 329 | if (adev->gfx.kiq.ring.sched.ready && | 331 | if (adev->gfx.kiq.ring.sched.ready && |
| 330 | (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev)) && | 332 | (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev)) && |
| 331 | down_read_trylock(&adev->reset_sem)) { | 333 | down_read_trylock(&adev->reset_domain->sem)) { |
| 332 | struct amdgpu_vmhub *hub = &adev->vmhub[vmhub]; | 334 | struct amdgpu_vmhub *hub = &adev->vmhub[vmhub]; |
| 333 | const unsigned eng = 17; | 335 | const unsigned eng = 17; |
| 334 | u32 inv_req = hub->vmhub_funcs->get_invalidate_req(vmid, flush_type); | 336 | u32 inv_req = hub->vmhub_funcs->get_invalidate_req(vmid, flush_type); |
| @@ -338,7 +340,7 @@ static void gmc_v10_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 3 unmodified lines | |||
| 338 | amdgpu_virt_kiq_reg_write_reg_wait(adev, req, ack, inv_req, | 340 | amdgpu_virt_kiq_reg_write_reg_wait(adev, req, ack, inv_req, |
| 339 | 1 << vmid); | 341 | 1 << vmid); |
| 340 | 342 | ||
| 341 | up_read(&adev->reset_sem); | 343 | up_read(&adev->reset_domain->sem); |
| 342 | return; | 344 | return; |
| 343 | } | 345 | } |
| 344 | 346 | ||
| @@ -664,11 +666,31 @@ static void gmc_v10_0_set_umc_funcs(struct amdgpu_device *adev) · 319 unmodified lines | |||
| 664 | adev->umc.umc_inst_num = UMC_V8_7_UMC_INSTANCE_NUM; | 666 | adev->umc.umc_inst_num = UMC_V8_7_UMC_INSTANCE_NUM; |
| 665 | adev->umc.channel_offs = UMC_V8_7_PER_CHANNEL_OFFSET_SIENNA; | 667 | adev->umc.channel_offs = UMC_V8_7_PER_CHANNEL_OFFSET_SIENNA; |
| 666 | adev->umc.channel_idx_tbl = &umc_v8_7_channel_idx_tbl[0][0]; | 668 | adev->umc.channel_idx_tbl = &umc_v8_7_channel_idx_tbl[0][0]; |
| 667 | adev->umc.ras_funcs = &umc_v8_7_ras_funcs; | 669 | adev->umc.ras = &umc_v8_7_ras; |
| 668 | break; | 670 | break; |
| 669 | default: | 671 | default: |
| 670 | break; | 672 | break; |
| 671 | } | 673 | } |
| 674 | if (adev->umc.ras) { | ||
| 675 | amdgpu_ras_register_ras_block(adev, &adev->umc.ras->ras_block); | ||
| 676 | |||
| 677 | strcpy(adev->umc.ras->ras_block.ras_comm.name, "umc"); | ||
| 678 | adev->umc.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__UMC; | ||
| 679 | adev->umc.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 680 | adev->umc.ras_if = &adev->umc.ras->ras_block.ras_comm; | ||
| 681 | |||
| 682 | /* If don't define special ras_late_init function, use default ras_late_init */ | ||
| 683 | if (!adev->umc.ras->ras_block.ras_late_init) | ||
| 684 | adev->umc.ras->ras_block.ras_late_init = amdgpu_umc_ras_late_init; | ||
| 685 | |||
| 686 | /* If don't define special ras_fini function, use default ras_fini */ | ||
| 687 | if (!adev->umc.ras->ras_block.ras_fini) | ||
| 688 | adev->umc.ras->ras_block.ras_fini = amdgpu_umc_ras_fini; | ||
| 689 | |||
| 690 | /* If not defined special ras_cb function, use default ras_cb */ | ||
| 691 | if (!adev->umc.ras->ras_block.ras_cb) | ||
| 692 | adev->umc.ras->ras_block.ras_cb = amdgpu_umc_process_ras_data_cb; | ||
| 693 | } | ||
| 672 | } | 694 | } |
| 673 | 695 | ||
| 674 | 696 | ||
| @@ -677,6 +699,7 @@ static void gmc_v10_0_set_mmhub_funcs(struct amdgpu_device *adev) · 2 unmodified lines | |||
| 677 | switch (adev->ip_versions[MMHUB_HWIP][0]) { | 699 | switch (adev->ip_versions[MMHUB_HWIP][0]) { |
| 678 | case IP_VERSION(2, 3, 0): | 700 | case IP_VERSION(2, 3, 0): |
| 679 | case IP_VERSION(2, 4, 0): | 701 | case IP_VERSION(2, 4, 0): |
| 702 | case IP_VERSION(2, 4, 1): | ||
| 680 | adev->mmhub.funcs = &mmhub_v2_3_funcs; | 703 | adev->mmhub.funcs = &mmhub_v2_3_funcs; |
| 681 | break; | 704 | break; |
| 682 | default: | 705 | default: |
| @@ -693,7 +716,9 @@ static void gmc_v10_0_set_gfxhub_funcs(struct amdgpu_device *adev) · 10 unmodified lines | |||
| 693 | case IP_VERSION(10, 3, 1): | 716 | case IP_VERSION(10, 3, 1): |
| 694 | case IP_VERSION(10, 3, 4): | 717 | case IP_VERSION(10, 3, 4): |
| 695 | case IP_VERSION(10, 3, 5): | 718 | case IP_VERSION(10, 3, 5): |
| 719 | case IP_VERSION(10, 3, 6): | ||
| 696 | case IP_VERSION(10, 3, 3): | 720 | case IP_VERSION(10, 3, 3): |
| 721 | case IP_VERSION(10, 3, 7): | ||
| 697 | adev->gfxhub.funcs = &gfxhub_v2_1_funcs; | 722 | adev->gfxhub.funcs = &gfxhub_v2_1_funcs; |
| 698 | break; | 723 | break; |
| 699 | default: | 724 | default: |
| @@ -705,6 +730,7 @@ static void gmc_v10_0_set_gfxhub_funcs(struct amdgpu_device *adev) · 5 unmodified lines | |||
| 705 | 730 | ||
| 706 | static int gmc_v10_0_early_init(void *handle) | 731 | static int gmc_v10_0_early_init(void *handle) |
| 707 | { | 732 | { |
| 733 | int r; | ||
| 708 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 734 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 709 | 735 | ||
| 710 | gmc_v10_0_set_mmhub_funcs(adev); | 736 | gmc_v10_0_set_mmhub_funcs(adev); |
| @@ -720,6 +746,10 @@ static int gmc_v10_0_early_init(void *handle) · 9 unmodified lines | |||
| 720 | adev->gmc.private_aperture_end = | 746 | adev->gmc.private_aperture_end = |
| 721 | adev->gmc.private_aperture_start + (4ULL << 30) - 1; | 747 | adev->gmc.private_aperture_start + (4ULL << 30) - 1; |
| 722 | 748 | ||
| 749 | r = amdgpu_gmc_ras_early_init(adev); | ||
| 750 | if (r) | ||
| 751 | return r; | ||
| 752 | |||
| 723 | return 0; | 753 | return 0; |
| 724 | } | 754 | } |
| 725 | 755 | ||
| @@ -862,12 +892,15 @@ static int gmc_v10_0_sw_init(void *handle) · 136 unmodified lines | |||
| 862 | case IP_VERSION(10, 1, 1): | 892 | case IP_VERSION(10, 1, 1): |
| 863 | case IP_VERSION(10, 1, 2): | 893 | case IP_VERSION(10, 1, 2): |
| 864 | case IP_VERSION(10, 1, 3): | 894 | case IP_VERSION(10, 1, 3): |
| 895 | case IP_VERSION(10, 1, 4): | ||
| 865 | case IP_VERSION(10, 3, 0): | 896 | case IP_VERSION(10, 3, 0): |
| 866 | case IP_VERSION(10, 3, 2): | 897 | case IP_VERSION(10, 3, 2): |
| 867 | case IP_VERSION(10, 3, 1): | 898 | case IP_VERSION(10, 3, 1): |
| 868 | case IP_VERSION(10, 3, 4): | 899 | case IP_VERSION(10, 3, 4): |
| 869 | case IP_VERSION(10, 3, 5): | 900 | case IP_VERSION(10, 3, 5): |
| 901 | case IP_VERSION(10, 3, 6): | ||
| 870 | case IP_VERSION(10, 3, 3): | 902 | case IP_VERSION(10, 3, 3): |
| 903 | case IP_VERSION(10, 3, 7): | ||
| 871 | adev->num_vmhubs = 2; | 904 | adev->num_vmhubs = 2; |
| 872 | /* | 905 | /* |
| 873 | * To fulfill 4-level page support, | 906 | * To fulfill 4-level page support, |
| @@ -986,14 +1019,7 @@ static int gmc_v10_0_gart_enable(struct amdgpu_device *adev) · 112 unmodified lines | |||
| 986 | return -EINVAL; | 1019 | return -EINVAL; |
| 987 | } | 1020 | } |
| 988 | 1021 | ||
| 989 | if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev)) | 1022 | amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); |
| 990 | goto skip_pin_bo; | ||
| 991 | |||
| 992 | r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | ||
| 993 | if (r) | ||
| 994 | return r; | ||
| 995 | |||
| 996 | skip_pin_bo: | ||
| 997 | r = adev->gfxhub.funcs->gart_enable(adev); | 1023 | r = adev->gfxhub.funcs->gart_enable(adev); |
| 998 | if (r) | 1024 | if (r) |
| 999 | return r; | 1025 | return r; |
| @@ -1019,8 +1045,6 @@ skip_pin_bo: · 19 unmodified lines | |||
| 1019 | (unsigned)(adev->gmc.gart_size >> 20), | 1045 | (unsigned)(adev->gmc.gart_size >> 20), |
| 1020 | (unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo)); | 1046 | (unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo)); |
| 1021 | 1047 | ||
| 1022 | adev->gart.ready = true; | ||
| 1023 | |||
| 1024 | return 0; | 1048 | return 0; |
| 1025 | } | 1049 | } |
| 1026 | 1050 | ||
| @@ -1043,6 +1067,12 @@ static int gmc_v10_0_hw_init(void *handle) · 16 unmodified lines | |||
| 1043 | if (r) | 1067 | if (r) |
| 1044 | return r; | 1068 | return r; |
| 1045 | 1069 | ||
| 1070 | if (amdgpu_emu_mode == 1) { | ||
| 1071 | r = amdgpu_gmc_vram_checking(adev); | ||
| 1072 | if (r) | ||
| 1073 | return r; | ||
| 1074 | } | ||
| 1075 | |||
| 1046 | if (adev->umc.funcs && adev->umc.funcs->init_registers) | 1076 | if (adev->umc.funcs && adev->umc.funcs->init_registers) |
| 1047 | adev->umc.funcs->init_registers(adev); | 1077 | adev->umc.funcs->init_registers(adev); |
| 1048 | 1078 | ||
| @@ -1140,6 +1170,10 @@ static void gmc_v10_0_get_clockgating_state(void *handle, u32 *flags) · 91 unmodified lines | |||
| 1140 | { | 1170 | { |
| 1141 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 1171 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1142 | 1172 | ||
| 1173 | if (adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 1, 3) || | ||
| 1174 | adev->ip_versions[GC_HWIP][0] == IP_VERSION(10, 1, 4)) | ||
| 1175 | return; | ||
| 1176 | |||
| 1143 | adev->mmhub.funcs->get_clockgating(adev, flags); | 1177 | adev->mmhub.funcs->get_clockgating(adev, flags); |
| 1144 | 1178 | ||
| 1145 | if (adev->ip_versions[ATHUB_HWIP][0] >= IP_VERSION(2, 1, 0)) | 1179 | if (adev->ip_versions[ATHUB_HWIP][0] >= IP_VERSION(2, 1, 0)) |
| @@ -469,16 +469,14 @@ static void gmc_v6_0_set_prt(struct amdgpu_device *adev, bool enable) · 468 unmodified lines | |||
| 469 | static int gmc_v6_0_gart_enable(struct amdgpu_device *adev) | 469 | static int gmc_v6_0_gart_enable(struct amdgpu_device *adev) |
| 470 | { | 470 | { |
| 471 | uint64_t table_addr; | 471 | uint64_t table_addr; |
| 472 | int r, i; | ||
| 473 | u32 field; | 472 | u32 field; |
| 473 | int i; | ||
| 474 | 474 | ||
| 475 | if (adev->gart.bo == NULL) { | 475 | if (adev->gart.bo == NULL) { |
| 476 | dev_err(adev->dev, "No VRAM object for PCIE GART.\n"); | 476 | dev_err(adev->dev, "No VRAM object for PCIE GART.\n"); |
| 477 | return -EINVAL; | 477 | return -EINVAL; |
| 478 | } | 478 | } |
| 479 | r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | 479 | amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); |
| 480 | if (r) | ||
| 481 | return r; | ||
| 482 | 480 | ||
| 483 | table_addr = amdgpu_bo_gpu_offset(adev->gart.bo); | 481 | table_addr = amdgpu_bo_gpu_offset(adev->gart.bo); |
| 484 | 482 | ||
| @@ -558,7 +556,6 @@ static int gmc_v6_0_gart_enable(struct amdgpu_device *adev) · 73 unmodified lines | |||
| 558 | dev_info(adev->dev, "PCIE GART of %uM enabled (table at 0x%016llX).\n", | 556 | dev_info(adev->dev, "PCIE GART of %uM enabled (table at 0x%016llX).\n", |
| 559 | (unsigned)(adev->gmc.gart_size >> 20), | 557 | (unsigned)(adev->gmc.gart_size >> 20), |
| 560 | (unsigned long long)table_addr); | 558 | (unsigned long long)table_addr); |
| 561 | adev->gart.ready = true; | ||
| 562 | return 0; | 559 | return 0; |
| 563 | } | 560 | } |
| 564 | 561 | ||
| @@ -922,7 +919,10 @@ static int gmc_v6_0_hw_init(void *handle) · 357 unmodified lines | |||
| 922 | if (r) | 919 | if (r) |
| 923 | return r; | 920 | return r; |
| 924 | 921 | ||
| 925 | return r; | 922 | if (amdgpu_emu_mode == 1) |
| 923 | return amdgpu_gmc_vram_checking(adev); | ||
| 924 | else | ||
| 925 | return r; | ||
| 926 | } | 926 | } |
| 927 | 927 | ||
| 928 | static int gmc_v6_0_hw_fini(void *handle) | 928 | static int gmc_v6_0_hw_fini(void *handle) |
| @@ -613,17 +613,14 @@ static void gmc_v7_0_set_prt(struct amdgpu_device *adev, bool enable) · 612 unmodified lines | |||
| 613 | static int gmc_v7_0_gart_enable(struct amdgpu_device *adev) | 613 | static int gmc_v7_0_gart_enable(struct amdgpu_device *adev) |
| 614 | { | 614 | { |
| 615 | uint64_t table_addr; | 615 | uint64_t table_addr; |
| 616 | int r, i; | ||
| 617 | u32 tmp, field; | 616 | u32 tmp, field; |
| 617 | int i; | ||
| 618 | 618 | ||
| 619 | if (adev->gart.bo == NULL) { | 619 | if (adev->gart.bo == NULL) { |
| 620 | dev_err(adev->dev, "No VRAM object for PCIE GART.\n"); | 620 | dev_err(adev->dev, "No VRAM object for PCIE GART.\n"); |
| 621 | return -EINVAL; | 621 | return -EINVAL; |
| 622 | } | 622 | } |
| 623 | r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | 623 | amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); |
| 624 | if (r) | ||
| 625 | return r; | ||
| 626 | |||
| 627 | table_addr = amdgpu_bo_gpu_offset(adev->gart.bo); | 624 | table_addr = amdgpu_bo_gpu_offset(adev->gart.bo); |
| 628 | 625 | ||
| 629 | /* Setup TLB control */ | 626 | /* Setup TLB control */ |
| @@ -712,7 +709,6 @@ static int gmc_v7_0_gart_enable(struct amdgpu_device *adev) · 82 unmodified lines | |||
| 712 | DRM_INFO("PCIE GART of %uM enabled (table at 0x%016llX).\n", | 709 | DRM_INFO("PCIE GART of %uM enabled (table at 0x%016llX).\n", |
| 713 | (unsigned)(adev->gmc.gart_size >> 20), | 710 | (unsigned)(adev->gmc.gart_size >> 20), |
| 714 | (unsigned long long)table_addr); | 711 | (unsigned long long)table_addr); |
| 715 | adev->gart.ready = true; | ||
| 716 | return 0; | 712 | return 0; |
| 717 | } | 713 | } |
| 718 | 714 | ||
| @@ -1111,7 +1107,10 @@ static int gmc_v7_0_hw_init(void *handle) · 392 unmodified lines | |||
| 1111 | if (r) | 1107 | if (r) |
| 1112 | return r; | 1108 | return r; |
| 1113 | 1109 | ||
| 1114 | return r; | 1110 | if (amdgpu_emu_mode == 1) |
| 1111 | return amdgpu_gmc_vram_checking(adev); | ||
| 1112 | else | ||
| 1113 | return r; | ||
| 1115 | } | 1114 | } |
| 1116 | 1115 | ||
| 1117 | static int gmc_v7_0_hw_fini(void *handle) | 1116 | static int gmc_v7_0_hw_fini(void *handle) |
| @@ -837,17 +837,14 @@ static void gmc_v8_0_set_prt(struct amdgpu_device *adev, bool enable) · 836 unmodified lines | |||
| 837 | static int gmc_v8_0_gart_enable(struct amdgpu_device *adev) | 837 | static int gmc_v8_0_gart_enable(struct amdgpu_device *adev) |
| 838 | { | 838 | { |
| 839 | uint64_t table_addr; | 839 | uint64_t table_addr; |
| 840 | int r, i; | ||
| 841 | u32 tmp, field; | 840 | u32 tmp, field; |
| 841 | int i; | ||
| 842 | 842 | ||
| 843 | if (adev->gart.bo == NULL) { | 843 | if (adev->gart.bo == NULL) { |
| 844 | dev_err(adev->dev, "No VRAM object for PCIE GART.\n"); | 844 | dev_err(adev->dev, "No VRAM object for PCIE GART.\n"); |
| 845 | return -EINVAL; | 845 | return -EINVAL; |
| 846 | } | 846 | } |
| 847 | r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | 847 | amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); |
| 848 | if (r) | ||
| 849 | return r; | ||
| 850 | |||
| 851 | table_addr = amdgpu_bo_gpu_offset(adev->gart.bo); | 848 | table_addr = amdgpu_bo_gpu_offset(adev->gart.bo); |
| 852 | 849 | ||
| 853 | /* Setup TLB control */ | 850 | /* Setup TLB control */ |
| @@ -953,7 +950,6 @@ static int gmc_v8_0_gart_enable(struct amdgpu_device *adev) · 99 unmodified lines | |||
| 953 | DRM_INFO("PCIE GART of %uM enabled (table at 0x%016llX).\n", | 950 | DRM_INFO("PCIE GART of %uM enabled (table at 0x%016llX).\n", |
| 954 | (unsigned)(adev->gmc.gart_size >> 20), | 951 | (unsigned)(adev->gmc.gart_size >> 20), |
| 955 | (unsigned long long)table_addr); | 952 | (unsigned long long)table_addr); |
| 956 | adev->gart.ready = true; | ||
| 957 | return 0; | 953 | return 0; |
| 958 | } | 954 | } |
| 959 | 955 | ||
| @@ -1242,7 +1238,10 @@ static int gmc_v8_0_hw_init(void *handle) · 282 unmodified lines | |||
| 1242 | if (r) | 1238 | if (r) |
| 1243 | return r; | 1239 | return r; |
| 1244 | 1240 | ||
| 1245 | return r; | 1241 | if (amdgpu_emu_mode == 1) |
| 1242 | return amdgpu_gmc_vram_checking(adev); | ||
| 1243 | else | ||
| 1244 | return r; | ||
| 1246 | } | 1245 | } |
| 1247 | 1246 | ||
| 1248 | static int gmc_v8_0_hw_fini(void *handle) | 1247 | static int gmc_v8_0_hw_fini(void *handle) |
| @@ -62,6 +62,8 @@ · 61 unmodified lines | |||
| 62 | #include "amdgpu_ras.h" | 62 | #include "amdgpu_ras.h" |
| 63 | #include "amdgpu_xgmi.h" | 63 | #include "amdgpu_xgmi.h" |
| 64 | 64 | ||
| 65 | #include "amdgpu_reset.h" | ||
| 66 | |||
| 65 | /* add these here since we already include dce12 headers and these are for DCN */ | 67 | /* add these here since we already include dce12 headers and these are for DCN */ |
| 66 | #define mmHUBP0_DCSURF_PRI_VIEWPORT_DIMENSION 0x055d | 68 | #define mmHUBP0_DCSURF_PRI_VIEWPORT_DIMENSION 0x055d |
| 67 | #define mmHUBP0_DCSURF_PRI_VIEWPORT_DIMENSION_BASE_IDX 2 | 69 | #define mmHUBP0_DCSURF_PRI_VIEWPORT_DIMENSION_BASE_IDX 2 |
| @@ -787,13 +789,13 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid, · 719 unmodified lines | |||
| 787 | */ | 789 | */ |
| 788 | if (adev->gfx.kiq.ring.sched.ready && | 790 | if (adev->gfx.kiq.ring.sched.ready && |
| 789 | (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev)) && | 791 | (amdgpu_sriov_runtime(adev) || !amdgpu_sriov_vf(adev)) && |
| 790 | down_read_trylock(&adev->reset_sem)) { | 792 | down_read_trylock(&adev->reset_domain->sem)) { |
| 791 | uint32_t req = hub->vm_inv_eng0_req + hub->eng_distance * eng; | 793 | uint32_t req = hub->vm_inv_eng0_req + hub->eng_distance * eng; |
| 792 | uint32_t ack = hub->vm_inv_eng0_ack + hub->eng_distance * eng; | 794 | uint32_t ack = hub->vm_inv_eng0_ack + hub->eng_distance * eng; |
| 793 | 795 | ||
| 794 | amdgpu_virt_kiq_reg_write_reg_wait(adev, req, ack, inv_req, | 796 | amdgpu_virt_kiq_reg_write_reg_wait(adev, req, ack, inv_req, |
| 795 | 1 << vmid); | 797 | 1 << vmid); |
| 796 | up_read(&adev->reset_sem); | 798 | up_read(&adev->reset_domain->sem); |
| 797 | return; | 799 | return; |
| 798 | } | 800 | } |
| 799 | 801 | ||
| @@ -900,7 +902,7 @@ static int gmc_v9_0_flush_gpu_tlb_pasid(struct amdgpu_device *adev, · 100 unmodified lines | |||
| 900 | if (amdgpu_in_reset(adev)) | 902 | if (amdgpu_in_reset(adev)) |
| 901 | return -EIO; | 903 | return -EIO; |
| 902 | 904 | ||
| 903 | if (ring->sched.ready && down_read_trylock(&adev->reset_sem)) { | 905 | if (ring->sched.ready && down_read_trylock(&adev->reset_domain->sem)) { |
| 904 | /* Vega20+XGMI caches PTEs in TC and TLB. Add a | 906 | /* Vega20+XGMI caches PTEs in TC and TLB. Add a |
| 905 | * heavy-weight TLB flush (type 2), which flushes | 907 | * heavy-weight TLB flush (type 2), which flushes |
| 906 | * both. Due to a race condition with concurrent | 908 | * both. Due to a race condition with concurrent |
| @@ -927,7 +929,7 @@ static int gmc_v9_0_flush_gpu_tlb_pasid(struct amdgpu_device *adev, · 20 unmodified lines | |||
| 927 | if (r) { | 929 | if (r) { |
| 928 | amdgpu_ring_undo(ring); | 930 | amdgpu_ring_undo(ring); |
| 929 | spin_unlock(&adev->gfx.kiq.ring_lock); | 931 | spin_unlock(&adev->gfx.kiq.ring_lock); |
| 930 | up_read(&adev->reset_sem); | 932 | up_read(&adev->reset_domain->sem); |
| 931 | return -ETIME; | 933 | return -ETIME; |
| 932 | } | 934 | } |
| 933 | 935 | ||
| @@ -936,10 +938,10 @@ static int gmc_v9_0_flush_gpu_tlb_pasid(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 936 | r = amdgpu_fence_wait_polling(ring, seq, adev->usec_timeout); | 938 | r = amdgpu_fence_wait_polling(ring, seq, adev->usec_timeout); |
| 937 | if (r < 1) { | 939 | if (r < 1) { |
| 938 | dev_err(adev->dev, "wait for kiq fence error: %ld.\n", r); | 940 | dev_err(adev->dev, "wait for kiq fence error: %ld.\n", r); |
| 939 | up_read(&adev->reset_sem); | 941 | up_read(&adev->reset_domain->sem); |
| 940 | return -ETIME; | 942 | return -ETIME; |
| 941 | } | 943 | } |
| 942 | up_read(&adev->reset_sem); | 944 | up_read(&adev->reset_domain->sem); |
| 943 | return 0; | 945 | return 0; |
| 944 | } | 946 | } |
| 945 | 947 | ||
| @@ -1202,7 +1204,7 @@ static void gmc_v9_0_set_umc_funcs(struct amdgpu_device *adev) · 256 unmodified lines | |||
| 1202 | adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM; | 1204 | adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM; |
| 1203 | adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_VG20; | 1205 | adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_VG20; |
| 1204 | adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0]; | 1206 | adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0]; |
| 1205 | adev->umc.ras_funcs = &umc_v6_1_ras_funcs; | 1207 | adev->umc.ras = &umc_v6_1_ras; |
| 1206 | break; | 1208 | break; |
| 1207 | case IP_VERSION(6, 1, 2): | 1209 | case IP_VERSION(6, 1, 2): |
| 1208 | adev->umc.max_ras_err_cnt_per_query = UMC_V6_1_TOTAL_CHANNEL_NUM; | 1210 | adev->umc.max_ras_err_cnt_per_query = UMC_V6_1_TOTAL_CHANNEL_NUM; |
| @@ -1210,15 +1212,16 @@ static void gmc_v9_0_set_umc_funcs(struct amdgpu_device *adev) · 1 unmodified line | |||
| 1210 | adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM; | 1212 | adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM; |
| 1211 | adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_ARCT; | 1213 | adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_ARCT; |
| 1212 | adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0]; | 1214 | adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0]; |
| 1213 | adev->umc.ras_funcs = &umc_v6_1_ras_funcs; | 1215 | adev->umc.ras = &umc_v6_1_ras; |
| 1214 | break; | 1216 | break; |
| 1215 | case IP_VERSION(6, 7, 0): | 1217 | case IP_VERSION(6, 7, 0): |
| 1216 | adev->umc.max_ras_err_cnt_per_query = UMC_V6_7_TOTAL_CHANNEL_NUM; | 1218 | adev->umc.max_ras_err_cnt_per_query = |
| 1219 | UMC_V6_7_TOTAL_CHANNEL_NUM * UMC_V6_7_BAD_PAGE_NUM_PER_CHANNEL; | ||
| 1217 | adev->umc.channel_inst_num = UMC_V6_7_CHANNEL_INSTANCE_NUM; | 1220 | adev->umc.channel_inst_num = UMC_V6_7_CHANNEL_INSTANCE_NUM; |
| 1218 | adev->umc.umc_inst_num = UMC_V6_7_UMC_INSTANCE_NUM; | 1221 | adev->umc.umc_inst_num = UMC_V6_7_UMC_INSTANCE_NUM; |
| 1219 | adev->umc.channel_offs = UMC_V6_7_PER_CHANNEL_OFFSET; | 1222 | adev->umc.channel_offs = UMC_V6_7_PER_CHANNEL_OFFSET; |
| 1220 | if (!adev->gmc.xgmi.connected_to_cpu) | 1223 | if (!adev->gmc.xgmi.connected_to_cpu) |
| 1221 | adev->umc.ras_funcs = &umc_v6_7_ras_funcs; | 1224 | adev->umc.ras = &umc_v6_7_ras; |
| 1222 | if (1 & adev->smuio.funcs->get_die_id(adev)) | 1225 | if (1 & adev->smuio.funcs->get_die_id(adev)) |
| 1223 | adev->umc.channel_idx_tbl = &umc_v6_7_channel_idx_tbl_first[0][0]; | 1226 | adev->umc.channel_idx_tbl = &umc_v6_7_channel_idx_tbl_first[0][0]; |
| 1224 | else | 1227 | else |
| @@ -1227,6 +1230,27 @@ static void gmc_v9_0_set_umc_funcs(struct amdgpu_device *adev) · 2 unmodified lines | |||
| 1227 | default: | 1230 | default: |
| 1228 | break; | 1231 | break; |
| 1229 | } | 1232 | } |
| 1233 | |||
| 1234 | if (adev->umc.ras) { | ||
| 1235 | amdgpu_ras_register_ras_block(adev, &adev->umc.ras->ras_block); | ||
| 1236 | |||
| 1237 | strcpy(adev->umc.ras->ras_block.ras_comm.name, "umc"); | ||
| 1238 | adev->umc.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__UMC; | ||
| 1239 | adev->umc.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 1240 | adev->umc.ras_if = &adev->umc.ras->ras_block.ras_comm; | ||
| 1241 | |||
| 1242 | /* If don't define special ras_late_init function, use default ras_late_init */ | ||
| 1243 | if (!adev->umc.ras->ras_block.ras_late_init) | ||
| 1244 | adev->umc.ras->ras_block.ras_late_init = amdgpu_umc_ras_late_init; | ||
| 1245 | |||
| 1246 | /* If don't define special ras_fini function, use default ras_fini */ | ||
| 1247 | if (!adev->umc.ras->ras_block.ras_fini) | ||
| 1248 | adev->umc.ras->ras_block.ras_fini = amdgpu_umc_ras_fini; | ||
| 1249 | |||
| 1250 | /* If not defined special ras_cb function, use default ras_cb */ | ||
| 1251 | if (!adev->umc.ras->ras_block.ras_cb) | ||
| 1252 | adev->umc.ras->ras_block.ras_cb = amdgpu_umc_process_ras_data_cb; | ||
| 1253 | } | ||
| 1230 | } | 1254 | } |
| 1231 | 1255 | ||
| 1232 | static void gmc_v9_0_set_mmhub_funcs(struct amdgpu_device *adev) | 1256 | static void gmc_v9_0_set_mmhub_funcs(struct amdgpu_device *adev) |
| @@ -1248,18 +1272,31 @@ static void gmc_v9_0_set_mmhub_ras_funcs(struct amdgpu_device *adev) · 15 unmodified lines | |||
| 1248 | { | 1272 | { |
| 1249 | switch (adev->ip_versions[MMHUB_HWIP][0]) { | 1273 | switch (adev->ip_versions[MMHUB_HWIP][0]) { |
| 1250 | case IP_VERSION(9, 4, 0): | 1274 | case IP_VERSION(9, 4, 0): |
| 1251 | adev->mmhub.ras_funcs = &mmhub_v1_0_ras_funcs; | 1275 | adev->mmhub.ras = &mmhub_v1_0_ras; |
| 1252 | break; | 1276 | break; |
| 1253 | case IP_VERSION(9, 4, 1): | 1277 | case IP_VERSION(9, 4, 1): |
| 1254 | adev->mmhub.ras_funcs = &mmhub_v9_4_ras_funcs; | 1278 | adev->mmhub.ras = &mmhub_v9_4_ras; |
| 1255 | break; | 1279 | break; |
| 1256 | case IP_VERSION(9, 4, 2): | 1280 | case IP_VERSION(9, 4, 2): |
| 1257 | adev->mmhub.ras_funcs = &mmhub_v1_7_ras_funcs; | 1281 | adev->mmhub.ras = &mmhub_v1_7_ras; |
| 1258 | break; | 1282 | break; |
| 1259 | default: | 1283 | default: |
| 1260 | /* mmhub ras is not available */ | 1284 | /* mmhub ras is not available */ |
| 1261 | break; | 1285 | break; |
| 1262 | } | 1286 | } |
| 1287 | |||
| 1288 | if (adev->mmhub.ras) { | ||
| 1289 | amdgpu_ras_register_ras_block(adev, &adev->mmhub.ras->ras_block); | ||
| 1290 | |||
| 1291 | strcpy(adev->mmhub.ras->ras_block.ras_comm.name, "mmhub"); | ||
| 1292 | adev->mmhub.ras->ras_block.ras_comm.block = AMDGPU_RAS_BLOCK__MMHUB; | ||
| 1293 | adev->mmhub.ras->ras_block.ras_comm.type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE; | ||
| 1294 | adev->mmhub.ras_if = &adev->mmhub.ras->ras_block.ras_comm; | ||
| 1295 | |||
| 1296 | /* If don't define special ras_fini function, use default ras_fini */ | ||
| 1297 | if (!adev->mmhub.ras->ras_block.ras_fini) | ||
| 1298 | adev->mmhub.ras->ras_block.ras_fini = amdgpu_mmhub_ras_fini; | ||
| 1299 | } | ||
| 1263 | } | 1300 | } |
| 1264 | 1301 | ||
| 1265 | static void gmc_v9_0_set_gfxhub_funcs(struct amdgpu_device *adev) | 1302 | static void gmc_v9_0_set_gfxhub_funcs(struct amdgpu_device *adev) |
| @@ -1269,7 +1306,9 @@ static void gmc_v9_0_set_gfxhub_funcs(struct amdgpu_device *adev) · 3 unmodified lines | |||
| 1269 | 1306 | ||
| 1270 | static void gmc_v9_0_set_hdp_ras_funcs(struct amdgpu_device *adev) | 1307 | static void gmc_v9_0_set_hdp_ras_funcs(struct amdgpu_device *adev) |
| 1271 | { | 1308 | { |
| 1272 | adev->hdp.ras_funcs = &hdp_v4_0_ras_funcs; | 1309 | adev->hdp.ras = &hdp_v4_0_ras; |
| 1310 | amdgpu_ras_register_ras_block(adev, &adev->hdp.ras->ras_block); | ||
| 1311 | adev->hdp.ras_if = &adev->hdp.ras->ras_block.ras_comm; | ||
| 1273 | } | 1312 | } |
| 1274 | 1313 | ||
| 1275 | static void gmc_v9_0_set_mca_funcs(struct amdgpu_device *adev) | 1314 | static void gmc_v9_0_set_mca_funcs(struct amdgpu_device *adev) |
| @@ -1287,6 +1326,7 @@ static void gmc_v9_0_set_mca_funcs(struct amdgpu_device *adev) · 11 unmodified lines | |||
| 1287 | 1326 | ||
| 1288 | static int gmc_v9_0_early_init(void *handle) | 1327 | static int gmc_v9_0_early_init(void *handle) |
| 1289 | { | 1328 | { |
| 1329 | int r; | ||
| 1290 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 1330 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1291 | 1331 | ||
| 1292 | /* ARCT and VEGA20 don't have XGMI defined in their IP discovery tables */ | 1332 | /* ARCT and VEGA20 don't have XGMI defined in their IP discovery tables */ |
| @@ -1316,6 +1356,10 @@ static int gmc_v9_0_early_init(void *handle) · 23 unmodified lines | |||
| 1316 | adev->gmc.private_aperture_end = | 1356 | adev->gmc.private_aperture_end = |
| 1317 | adev->gmc.private_aperture_start + (4ULL << 30) - 1; | 1357 | adev->gmc.private_aperture_start + (4ULL << 30) - 1; |
| 1318 | 1358 | ||
| 1359 | r = amdgpu_gmc_ras_early_init(adev); | ||
| 1360 | if (r) | ||
| 1361 | return r; | ||
| 1362 | |||
| 1319 | return 0; | 1363 | return 0; |
| 1320 | } | 1364 | } |
| 1321 | 1365 | ||
| @@ -1342,13 +1386,13 @@ static int gmc_v9_0_late_init(void *handle) · 20 unmodified lines | |||
| 1342 | } | 1386 | } |
| 1343 | 1387 | ||
| 1344 | if (!amdgpu_persistent_edc_harvesting_supported(adev)) { | 1388 | if (!amdgpu_persistent_edc_harvesting_supported(adev)) { |
| 1345 | if (adev->mmhub.ras_funcs && | 1389 | if (adev->mmhub.ras && adev->mmhub.ras->ras_block.hw_ops && |
| 1346 | adev->mmhub.ras_funcs->reset_ras_error_count) | 1390 | adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count) |
| 1347 | adev->mmhub.ras_funcs->reset_ras_error_count(adev); | 1391 | adev->mmhub.ras->ras_block.hw_ops->reset_ras_error_count(adev); |
| 1348 | 1392 | ||
| 1349 | if (adev->hdp.ras_funcs && | 1393 | if (adev->hdp.ras && adev->hdp.ras->ras_block.hw_ops && |
| 1350 | adev->hdp.ras_funcs->reset_ras_error_count) | 1394 | adev->hdp.ras->ras_block.hw_ops->reset_ras_error_count) |
| 1351 | adev->hdp.ras_funcs->reset_ras_error_count(adev); | 1395 | adev->hdp.ras->ras_block.hw_ops->reset_ras_error_count(adev); |
| 1352 | } | 1396 | } |
| 1353 | 1397 | ||
| 1354 | r = amdgpu_gmc_ras_late_init(adev); | 1398 | r = amdgpu_gmc_ras_late_init(adev); |
| @@ -1752,14 +1796,7 @@ static int gmc_v9_0_gart_enable(struct amdgpu_device *adev) · 397 unmodified lines | |||
| 1752 | return -EINVAL; | 1796 | return -EINVAL; |
| 1753 | } | 1797 | } |
| 1754 | 1798 | ||
| 1755 | if (amdgpu_sriov_vf(adev) && amdgpu_in_reset(adev)) | 1799 | amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); |
| 1756 | goto skip_pin_bo; | ||
| 1757 | |||
| 1758 | r = amdgpu_gtt_mgr_recover(&adev->mman.gtt_mgr); | ||
| 1759 | if (r) | ||
| 1760 | return r; | ||
| 1761 | |||
| 1762 | skip_pin_bo: | ||
| 1763 | r = adev->gfxhub.funcs->gart_enable(adev); | 1800 | r = adev->gfxhub.funcs->gart_enable(adev); |
| 1764 | if (r) | 1801 | if (r) |
| 1765 | return r; | 1802 | return r; |
| @@ -1776,7 +1813,6 @@ skip_pin_bo: · 10 unmodified lines | |||
| 1776 | DRM_INFO("PTB located at 0x%016llX\n", | 1813 | DRM_INFO("PTB located at 0x%016llX\n", |
| 1777 | (unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo)); | 1814 | (unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo)); |
| 1778 | 1815 | ||
| 1779 | adev->gart.ready = true; | ||
| 1780 | return 0; | 1816 | return 0; |
| 1781 | } | 1817 | } |
| 1782 | 1818 | ||
| @@ -1784,7 +1820,7 @@ static int gmc_v9_0_hw_init(void *handle) · 1 unmodified line | |||
| 1784 | { | 1820 | { |
| 1785 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 1821 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1786 | bool value; | 1822 | bool value; |
| 1787 | int i; | 1823 | int i, r; |
| 1788 | 1824 | ||
| 1789 | /* The sequence of these two function calls matters.*/ | 1825 | /* The sequence of these two function calls matters.*/ |
| 1790 | gmc_v9_0_init_golden_registers(adev); | 1826 | gmc_v9_0_init_golden_registers(adev); |
| @@ -1819,7 +1855,14 @@ static int gmc_v9_0_hw_init(void *handle) · 28 unmodified lines | |||
| 1819 | if (adev->umc.funcs && adev->umc.funcs->init_registers) | 1855 | if (adev->umc.funcs && adev->umc.funcs->init_registers) |
| 1820 | adev->umc.funcs->init_registers(adev); | 1856 | adev->umc.funcs->init_registers(adev); |
| 1821 | 1857 | ||
| 1822 | return gmc_v9_0_gart_enable(adev); | 1858 | r = gmc_v9_0_gart_enable(adev); |
| 1859 | if (r) | ||
| 1860 | return r; | ||
| 1861 | |||
| 1862 | if (amdgpu_emu_mode == 1) | ||
| 1863 | return amdgpu_gmc_vram_checking(adev); | ||
| 1864 | else | ||
| 1865 | return r; | ||
| 1823 | } | 1866 | } |
| 1824 | 1867 | ||
| 1825 | /** | 1868 | /** |
| @@ -146,17 +146,30 @@ static void hdp_v4_0_init_registers(struct amdgpu_device *adev) · 145 unmodified lines | |||
| 146 | 146 | ||
| 147 | WREG32_FIELD15(HDP, 0, HDP_MISC_CNTL, FLUSH_INVALIDATE_CACHE, 1); | 147 | WREG32_FIELD15(HDP, 0, HDP_MISC_CNTL, FLUSH_INVALIDATE_CACHE, 1); |
| 148 | 148 | ||
| 149 | if (adev->ip_versions[HDP_HWIP][0] == IP_VERSION(4, 4, 0)) | ||
| 150 | WREG32_FIELD15(HDP, 0, HDP_MISC_CNTL, READ_BUFFER_WATERMARK, 2); | ||
| 151 | |||
| 149 | WREG32_SOC15(HDP, 0, mmHDP_NONSURFACE_BASE, (adev->gmc.vram_start >> 8)); | 152 | WREG32_SOC15(HDP, 0, mmHDP_NONSURFACE_BASE, (adev->gmc.vram_start >> 8)); |
| 150 | WREG32_SOC15(HDP, 0, mmHDP_NONSURFACE_BASE_HI, (adev->gmc.vram_start >> 40)); | 153 | WREG32_SOC15(HDP, 0, mmHDP_NONSURFACE_BASE_HI, (adev->gmc.vram_start >> 40)); |
| 151 | } | 154 | } |
| 152 | 155 | ||
| 153 | const struct amdgpu_hdp_ras_funcs hdp_v4_0_ras_funcs = { | 156 | struct amdgpu_ras_block_hw_ops hdp_v4_0_ras_hw_ops = { |
| 154 | .ras_late_init = amdgpu_hdp_ras_late_init, | ||
| 155 | .ras_fini = amdgpu_hdp_ras_fini, | ||
| 156 | .query_ras_error_count = hdp_v4_0_query_ras_error_count, | 157 | .query_ras_error_count = hdp_v4_0_query_ras_error_count, |
| 157 | .reset_ras_error_count = hdp_v4_0_reset_ras_error_count, | 158 | .reset_ras_error_count = hdp_v4_0_reset_ras_error_count, |
| 158 | }; | 159 | }; |
| 159 | 160 | ||
| 161 | struct amdgpu_hdp_ras hdp_v4_0_ras = { | ||
| 162 | .ras_block = { | ||
| 163 | .ras_comm = { | ||
| 164 | .name = "hdp", | ||
| 165 | .block = AMDGPU_RAS_BLOCK__HDP, | ||
| 166 | .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | ||
| 167 | }, | ||
| 168 | .hw_ops = &hdp_v4_0_ras_hw_ops, | ||
| 169 | .ras_fini = amdgpu_hdp_ras_fini, | ||
| 170 | }, | ||
| 171 | }; | ||
| 172 | |||
| 160 | const struct amdgpu_hdp_funcs hdp_v4_0_funcs = { | 173 | const struct amdgpu_hdp_funcs hdp_v4_0_funcs = { |
| 161 | .flush_hdp = hdp_v4_0_flush_hdp, | 174 | .flush_hdp = hdp_v4_0_flush_hdp, |
| 162 | .invalidate_hdp = hdp_v4_0_invalidate_hdp, | 175 | .invalidate_hdp = hdp_v4_0_invalidate_hdp, |
| @@ -27,6 +27,6 @@ · 26 unmodified lines | |||
| 27 | #include "soc15_common.h" | 27 | #include "soc15_common.h" |
| 28 | 28 | ||
| 29 | extern const struct amdgpu_hdp_funcs hdp_v4_0_funcs; | 29 | extern const struct amdgpu_hdp_funcs hdp_v4_0_funcs; |
| 30 | extern const struct amdgpu_hdp_ras_funcs hdp_v4_0_ras_funcs; | 30 | extern struct amdgpu_hdp_ras hdp_v4_0_ras; |
| 31 | 31 | ||
| 32 | #endif | 32 | #endif |
| @@ -50,11 +50,16 @@ static int jpeg_v3_0_early_init(void *handle) · 49 unmodified lines | |||
| 50 | { | 50 | { |
| 51 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 51 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 52 | 52 | ||
| 53 | if (adev->asic_type != CHIP_YELLOW_CARP) { | 53 | u32 harvest; |
| 54 | u32 harvest = RREG32_SOC15(JPEG, 0, mmCC_UVD_HARVESTING); | ||
| 55 | 54 | ||
| 55 | switch (adev->ip_versions[UVD_HWIP][0]) { | ||
| 56 | case IP_VERSION(3, 1, 1): | ||
| 57 | break; | ||
| 58 | default: | ||
| 59 | harvest = RREG32_SOC15(JPEG, 0, mmCC_UVD_HARVESTING); | ||
| 56 | if (harvest & CC_UVD_HARVESTING__UVD_DISABLE_MASK) | 60 | if (harvest & CC_UVD_HARVESTING__UVD_DISABLE_MASK) |
| 57 | return -ENOENT; | 61 | return -ENOENT; |
| 62 | break; | ||
| 58 | } | 63 | } |
| 59 | 64 | ||
| 60 | adev->jpeg.num_jpeg_inst = 1; | 65 | adev->jpeg.num_jpeg_inst = 1; |
| @@ -37,24 +37,42 @@ static void mca_v3_0_mp0_query_ras_error_count(struct amdgpu_device *adev, · 36 unmodified lines | |||
| 37 | ras_error_status); | 37 | ras_error_status); |
| 38 | } | 38 | } |
| 39 | 39 | ||
| 40 | static int mca_v3_0_mp0_ras_late_init(struct amdgpu_device *adev) | 40 | static void mca_v3_0_mp0_ras_fini(struct amdgpu_device *adev) |
| 41 | { | 41 | { |
| 42 | return amdgpu_mca_ras_late_init(adev, &adev->mca.mp0); | 42 | amdgpu_mca_ras_fini(adev, &adev->mca.mp0); |
| 43 | } | 43 | } |
| 44 | 44 | ||
| 45 | static void mca_v3_0_mp0_ras_fini(struct amdgpu_device *adev) | 45 | static int mca_v3_0_ras_block_match(struct amdgpu_ras_block_object *block_obj, |
| 46 | enum amdgpu_ras_block block, uint32_t sub_block_index) | ||
| 46 | { | 47 | { |
| 47 | amdgpu_mca_ras_fini(adev, &adev->mca.mp0); | 48 | if (!block_obj) |
| 49 | return -EINVAL; | ||
| 50 | |||
| 51 | if ((block_obj->ras_comm.block == block) && | ||
| 52 | (block_obj->ras_comm.sub_block_index == sub_block_index)) { | ||
| 53 | return 0; | ||
| 54 | } | ||
| 55 | |||
| 56 | return -EINVAL; | ||
| 48 | } | 57 | } |
| 49 | 58 | ||
| 50 | const struct amdgpu_mca_ras_funcs mca_v3_0_mp0_ras_funcs = { | 59 | const struct amdgpu_ras_block_hw_ops mca_v3_0_mp0_hw_ops = { |
| 51 | .ras_late_init = mca_v3_0_mp0_ras_late_init, | ||
| 52 | .ras_fini = mca_v3_0_mp0_ras_fini, | ||
| 53 | .query_ras_error_count = mca_v3_0_mp0_query_ras_error_count, | 60 | .query_ras_error_count = mca_v3_0_mp0_query_ras_error_count, |
| 54 | .query_ras_error_address = NULL, | 61 | .query_ras_error_address = NULL, |
| 55 | .ras_block = AMDGPU_RAS_BLOCK__MCA, | 62 | }; |
| 56 | .ras_sub_block = AMDGPU_RAS_MCA_BLOCK__MP0, | 63 | |
| 57 | .sysfs_name = "mp0_err_count", | 64 | struct amdgpu_mca_ras_block mca_v3_0_mp0_ras = { |
| 65 | .ras_block = { | ||
| 66 | .ras_comm = { | ||
| 67 | .block = AMDGPU_RAS_BLOCK__MCA, | ||
| 68 | .sub_block_index = AMDGPU_RAS_MCA_BLOCK__MP0, | ||
| 69 | .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | ||
| 70 | .name = "mp0", | ||
| 71 | }, | ||
| 72 | .hw_ops = &mca_v3_0_mp0_hw_ops, | ||
| 73 | .ras_block_match = mca_v3_0_ras_block_match, | ||
| 74 | .ras_fini = mca_v3_0_mp0_ras_fini, | ||
| 75 | }, | ||
| 58 | }; | 76 | }; |
| 59 | 77 | ||
| 60 | static void mca_v3_0_mp1_query_ras_error_count(struct amdgpu_device *adev, | 78 | static void mca_v3_0_mp1_query_ras_error_count(struct amdgpu_device *adev, |
| @@ -65,24 +83,28 @@ static void mca_v3_0_mp1_query_ras_error_count(struct amdgpu_device *adev, · 4 unmodified lines | |||
| 65 | ras_error_status); | 83 | ras_error_status); |
| 66 | } | 84 | } |
| 67 | 85 | ||
| 68 | static int mca_v3_0_mp1_ras_late_init(struct amdgpu_device *adev) | ||
| 69 | { | ||
| 70 | return amdgpu_mca_ras_late_init(adev, &adev->mca.mp1); | ||
| 71 | } | ||
| 72 | |||
| 73 | static void mca_v3_0_mp1_ras_fini(struct amdgpu_device *adev) | 86 | static void mca_v3_0_mp1_ras_fini(struct amdgpu_device *adev) |
| 74 | { | 87 | { |
| 75 | amdgpu_mca_ras_fini(adev, &adev->mca.mp1); | 88 | amdgpu_mca_ras_fini(adev, &adev->mca.mp1); |
| 76 | } | 89 | } |
| 77 | 90 | ||
| 78 | const struct amdgpu_mca_ras_funcs mca_v3_0_mp1_ras_funcs = { | 91 | const struct amdgpu_ras_block_hw_ops mca_v3_0_mp1_hw_ops = { |
| 79 | .ras_late_init = mca_v3_0_mp1_ras_late_init, | ||
| 80 | .ras_fini = mca_v3_0_mp1_ras_fini, | ||
| 81 | .query_ras_error_count = mca_v3_0_mp1_query_ras_error_count, | 92 | .query_ras_error_count = mca_v3_0_mp1_query_ras_error_count, |
| 82 | .query_ras_error_address = NULL, | 93 | .query_ras_error_address = NULL, |
| 83 | .ras_block = AMDGPU_RAS_BLOCK__MCA, | 94 | }; |
| 84 | .ras_sub_block = AMDGPU_RAS_MCA_BLOCK__MP1, | 95 | |
| 85 | .sysfs_name = "mp1_err_count", | 96 | struct amdgpu_mca_ras_block mca_v3_0_mp1_ras = { |
| 97 | .ras_block = { | ||
| 98 | .ras_comm = { | ||
| 99 | .block = AMDGPU_RAS_BLOCK__MCA, | ||
| 100 | .sub_block_index = AMDGPU_RAS_MCA_BLOCK__MP1, | ||
| 101 | .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | ||
| 102 | .name = "mp1", | ||
| 103 | }, | ||
| 104 | .hw_ops = &mca_v3_0_mp1_hw_ops, | ||
| 105 | .ras_block_match = mca_v3_0_ras_block_match, | ||
| 106 | .ras_fini = mca_v3_0_mp1_ras_fini, | ||
| 107 | }, | ||
| 86 | }; | 108 | }; |
| 87 | 109 | ||
| 88 | static void mca_v3_0_mpio_query_ras_error_count(struct amdgpu_device *adev, | 110 | static void mca_v3_0_mpio_query_ras_error_count(struct amdgpu_device *adev, |
| @@ -93,24 +115,28 @@ static void mca_v3_0_mpio_query_ras_error_count(struct amdgpu_device *adev, · 4 unmodified lines | |||
| 93 | ras_error_status); | 115 | ras_error_status); |
| 94 | } | 116 | } |
| 95 | 117 | ||
| 96 | static int mca_v3_0_mpio_ras_late_init(struct amdgpu_device *adev) | ||
| 97 | { | ||
| 98 | return amdgpu_mca_ras_late_init(adev, &adev->mca.mpio); | ||
| 99 | } | ||
| 100 | |||
| 101 | static void mca_v3_0_mpio_ras_fini(struct amdgpu_device *adev) | 118 | static void mca_v3_0_mpio_ras_fini(struct amdgpu_device *adev) |
| 102 | { | 119 | { |
| 103 | amdgpu_mca_ras_fini(adev, &adev->mca.mpio); | 120 | amdgpu_mca_ras_fini(adev, &adev->mca.mpio); |
| 104 | } | 121 | } |
| 105 | 122 | ||
| 106 | const struct amdgpu_mca_ras_funcs mca_v3_0_mpio_ras_funcs = { | 123 | const struct amdgpu_ras_block_hw_ops mca_v3_0_mpio_hw_ops = { |
| 107 | .ras_late_init = mca_v3_0_mpio_ras_late_init, | ||
| 108 | .ras_fini = mca_v3_0_mpio_ras_fini, | ||
| 109 | .query_ras_error_count = mca_v3_0_mpio_query_ras_error_count, | 124 | .query_ras_error_count = mca_v3_0_mpio_query_ras_error_count, |
| 110 | .query_ras_error_address = NULL, | 125 | .query_ras_error_address = NULL, |
| 111 | .ras_block = AMDGPU_RAS_BLOCK__MCA, | 126 | }; |
| 112 | .ras_sub_block = AMDGPU_RAS_MCA_BLOCK__MPIO, | 127 | |
| 113 | .sysfs_name = "mpio_err_count", | 128 | struct amdgpu_mca_ras_block mca_v3_0_mpio_ras = { |
| 129 | .ras_block = { | ||
| 130 | .ras_comm = { | ||
| 131 | .block = AMDGPU_RAS_BLOCK__MCA, | ||
| 132 | .sub_block_index = AMDGPU_RAS_MCA_BLOCK__MPIO, | ||
| 133 | .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | ||
| 134 | .name = "mpio", | ||
| 135 | }, | ||
| 136 | .hw_ops = &mca_v3_0_mpio_hw_ops, | ||
| 137 | .ras_block_match = mca_v3_0_ras_block_match, | ||
| 138 | .ras_fini = mca_v3_0_mpio_ras_fini, | ||
| 139 | }, | ||
| 114 | }; | 140 | }; |
| 115 | 141 | ||
| 116 | 142 | ||
| @@ -118,9 +144,15 @@ static void mca_v3_0_init(struct amdgpu_device *adev) · 1 unmodified line | |||
| 118 | { | 144 | { |
| 119 | struct amdgpu_mca *mca = &adev->mca; | 145 | struct amdgpu_mca *mca = &adev->mca; |
| 120 | 146 | ||
| 121 | mca->mp0.ras_funcs = &mca_v3_0_mp0_ras_funcs; | 147 | mca->mp0.ras = &mca_v3_0_mp0_ras; |
| 122 | mca->mp1.ras_funcs = &mca_v3_0_mp1_ras_funcs; | 148 | mca->mp1.ras = &mca_v3_0_mp1_ras; |
| 123 | mca->mpio.ras_funcs = &mca_v3_0_mpio_ras_funcs; | 149 | mca->mpio.ras = &mca_v3_0_mpio_ras; |
| 150 | amdgpu_ras_register_ras_block(adev, &mca->mp0.ras->ras_block); | ||
| 151 | amdgpu_ras_register_ras_block(adev, &mca->mp1.ras->ras_block); | ||
| 152 | amdgpu_ras_register_ras_block(adev, &mca->mpio.ras->ras_block); | ||
| 153 | mca->mp0.ras_if = &mca->mp0.ras->ras_block.ras_comm; | ||
| 154 | mca->mp1.ras_if = &mca->mp1.ras->ras_block.ras_comm; | ||
| 155 | mca->mpio.ras_if = &mca->mpio.ras->ras_block.ras_comm; | ||
| 124 | } | 156 | } |
| 125 | 157 | ||
| 126 | const struct amdgpu_mca_funcs mca_v3_0_funcs = { | 158 | const struct amdgpu_mca_funcs mca_v3_0_funcs = { |
| @@ -774,13 +774,17 @@ static void mmhub_v1_0_reset_ras_error_count(struct amdgpu_device *adev) · 773 unmodified lines | |||
| 774 | } | 774 | } |
| 775 | } | 775 | } |
| 776 | 776 | ||
| 777 | const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs = { | 777 | struct amdgpu_ras_block_hw_ops mmhub_v1_0_ras_hw_ops = { |
| 778 | .ras_late_init = amdgpu_mmhub_ras_late_init, | ||
| 779 | .ras_fini = amdgpu_mmhub_ras_fini, | ||
| 780 | .query_ras_error_count = mmhub_v1_0_query_ras_error_count, | 778 | .query_ras_error_count = mmhub_v1_0_query_ras_error_count, |
| 781 | .reset_ras_error_count = mmhub_v1_0_reset_ras_error_count, | 779 | .reset_ras_error_count = mmhub_v1_0_reset_ras_error_count, |
| 782 | }; | 780 | }; |
| 783 | 781 | ||
| 782 | struct amdgpu_mmhub_ras mmhub_v1_0_ras = { | ||
| 783 | .ras_block = { | ||
| 784 | .hw_ops = &mmhub_v1_0_ras_hw_ops, | ||
| 785 | }, | ||
| 786 | }; | ||
| 787 | |||
| 784 | const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs = { | 788 | const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs = { |
| 785 | .get_fb_location = mmhub_v1_0_get_fb_location, | 789 | .get_fb_location = mmhub_v1_0_get_fb_location, |
| 786 | .init = mmhub_v1_0_init, | 790 | .init = mmhub_v1_0_init, |
| @@ -24,6 +24,6 @@ · 23 unmodified lines | |||
| 24 | #define __MMHUB_V1_0_H__ | 24 | #define __MMHUB_V1_0_H__ |
| 25 | 25 | ||
| 26 | extern const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs; | 26 | extern const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs; |
| 27 | extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs; | 27 | extern struct amdgpu_mmhub_ras mmhub_v1_0_ras; |
| 28 | 28 | ||
| 29 | #endif | 29 | #endif |
| @@ -1321,15 +1321,19 @@ static void mmhub_v1_7_reset_ras_error_status(struct amdgpu_device *adev) · 1320 unmodified lines | |||
| 1321 | } | 1321 | } |
| 1322 | } | 1322 | } |
| 1323 | 1323 | ||
| 1324 | const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs = { | 1324 | struct amdgpu_ras_block_hw_ops mmhub_v1_7_ras_hw_ops = { |
| 1325 | .ras_late_init = amdgpu_mmhub_ras_late_init, | ||
| 1326 | .ras_fini = amdgpu_mmhub_ras_fini, | ||
| 1327 | .query_ras_error_count = mmhub_v1_7_query_ras_error_count, | 1325 | .query_ras_error_count = mmhub_v1_7_query_ras_error_count, |
| 1328 | .reset_ras_error_count = mmhub_v1_7_reset_ras_error_count, | 1326 | .reset_ras_error_count = mmhub_v1_7_reset_ras_error_count, |
| 1329 | .query_ras_error_status = mmhub_v1_7_query_ras_error_status, | 1327 | .query_ras_error_status = mmhub_v1_7_query_ras_error_status, |
| 1330 | .reset_ras_error_status = mmhub_v1_7_reset_ras_error_status, | 1328 | .reset_ras_error_status = mmhub_v1_7_reset_ras_error_status, |
| 1331 | }; | 1329 | }; |
| 1332 | 1330 | ||
| 1331 | struct amdgpu_mmhub_ras mmhub_v1_7_ras = { | ||
| 1332 | .ras_block = { | ||
| 1333 | .hw_ops = &mmhub_v1_7_ras_hw_ops, | ||
| 1334 | }, | ||
| 1335 | }; | ||
| 1336 | |||
| 1333 | const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs = { | 1337 | const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs = { |
| 1334 | .get_fb_location = mmhub_v1_7_get_fb_location, | 1338 | .get_fb_location = mmhub_v1_7_get_fb_location, |
| 1335 | .init = mmhub_v1_7_init, | 1339 | .init = mmhub_v1_7_init, |
| @@ -24,6 +24,6 @@ · 23 unmodified lines | |||
| 24 | #define __MMHUB_V1_7_H__ | 24 | #define __MMHUB_V1_7_H__ |
| 25 | 25 | ||
| 26 | extern const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs; | 26 | extern const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs; |
| 27 | extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs; | 27 | extern struct amdgpu_mmhub_ras mmhub_v1_7_ras; |
| 28 | 28 | ||
| 29 | #endif | 29 | #endif |
| @@ -93,6 +93,7 @@ mmhub_v2_3_print_l2_protection_fault_status(struct amdgpu_device *adev, · 92 unmodified lines | |||
| 93 | switch (adev->ip_versions[MMHUB_HWIP][0]) { | 93 | switch (adev->ip_versions[MMHUB_HWIP][0]) { |
| 94 | case IP_VERSION(2, 3, 0): | 94 | case IP_VERSION(2, 3, 0): |
| 95 | case IP_VERSION(2, 4, 0): | 95 | case IP_VERSION(2, 4, 0): |
| 96 | case IP_VERSION(2, 4, 1): | ||
| 96 | mmhub_cid = mmhub_client_ids_vangogh[cid][rw]; | 97 | mmhub_cid = mmhub_client_ids_vangogh[cid][rw]; |
| 97 | break; | 98 | break; |
| 98 | default: | 99 | default: |
| @@ -1655,14 +1655,18 @@ static void mmhub_v9_4_query_ras_error_status(struct amdgpu_device *adev) · 1654 unmodified lines | |||
| 1655 | } | 1655 | } |
| 1656 | } | 1656 | } |
| 1657 | 1657 | ||
| 1658 | const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs = { | 1658 | const struct amdgpu_ras_block_hw_ops mmhub_v9_4_ras_hw_ops = { |
| 1659 | .ras_late_init = amdgpu_mmhub_ras_late_init, | ||
| 1660 | .ras_fini = amdgpu_mmhub_ras_fini, | ||
| 1661 | .query_ras_error_count = mmhub_v9_4_query_ras_error_count, | 1659 | .query_ras_error_count = mmhub_v9_4_query_ras_error_count, |
| 1662 | .reset_ras_error_count = mmhub_v9_4_reset_ras_error_count, | 1660 | .reset_ras_error_count = mmhub_v9_4_reset_ras_error_count, |
| 1663 | .query_ras_error_status = mmhub_v9_4_query_ras_error_status, | 1661 | .query_ras_error_status = mmhub_v9_4_query_ras_error_status, |
| 1664 | }; | 1662 | }; |
| 1665 | 1663 | ||
| 1664 | struct amdgpu_mmhub_ras mmhub_v9_4_ras = { | ||
| 1665 | .ras_block = { | ||
| 1666 | .hw_ops = &mmhub_v9_4_ras_hw_ops, | ||
| 1667 | }, | ||
| 1668 | }; | ||
| 1669 | |||
| 1666 | const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs = { | 1670 | const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs = { |
| 1667 | .get_fb_location = mmhub_v9_4_get_fb_location, | 1671 | .get_fb_location = mmhub_v9_4_get_fb_location, |
| 1668 | .init = mmhub_v9_4_init, | 1672 | .init = mmhub_v9_4_init, |
| @@ -24,6 +24,6 @@ · 23 unmodified lines | |||
| 24 | #define __MMHUB_V9_4_H__ | 24 | #define __MMHUB_V9_4_H__ |
| 25 | 25 | ||
| 26 | extern const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs; | 26 | extern const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs; |
| 27 | extern const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs; | 27 | extern struct amdgpu_mmhub_ras mmhub_v9_4_ras; |
| 28 | 28 | ||
| 29 | #endif | 29 | #endif |
| @@ -32,6 +32,8 @@ · 31 unmodified lines | |||
| 32 | #include "soc15_common.h" | 32 | #include "soc15_common.h" |
| 33 | #include "mxgpu_ai.h" | 33 | #include "mxgpu_ai.h" |
| 34 | 34 | ||
| 35 | #include "amdgpu_reset.h" | ||
| 36 | |||
| 35 | static void xgpu_ai_mailbox_send_ack(struct amdgpu_device *adev) | 37 | static void xgpu_ai_mailbox_send_ack(struct amdgpu_device *adev) |
| 36 | { | 38 | { |
| 37 | WREG8(AI_MAIBOX_CONTROL_RCV_OFFSET_BYTE, 2); | 39 | WREG8(AI_MAIBOX_CONTROL_RCV_OFFSET_BYTE, 2); |
| @@ -257,10 +259,10 @@ static void xgpu_ai_mailbox_flr_work(struct work_struct *work) · 219 unmodified lines | |||
| 257 | * otherwise the mailbox msg will be ruined/reseted by | 259 | * otherwise the mailbox msg will be ruined/reseted by |
| 258 | * the VF FLR. | 260 | * the VF FLR. |
| 259 | */ | 261 | */ |
| 260 | if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0) | 262 | if (atomic_cmpxchg(&adev->reset_domain->in_gpu_reset, 0, 1) != 0) |
| 261 | return; | 263 | return; |
| 262 | 264 | ||
| 263 | down_write(&adev->reset_sem); | 265 | down_write(&adev->reset_domain->sem); |
| 264 | 266 | ||
| 265 | amdgpu_virt_fini_data_exchange(adev); | 267 | amdgpu_virt_fini_data_exchange(adev); |
| 266 | 268 | ||
| @@ -275,14 +277,14 @@ static void xgpu_ai_mailbox_flr_work(struct work_struct *work) · 8 unmodified lines | |||
| 275 | } while (timeout > 1); | 277 | } while (timeout > 1); |
| 276 | 278 | ||
| 277 | flr_done: | 279 | flr_done: |
| 278 | atomic_set(&adev->in_gpu_reset, 0); | 280 | atomic_set(&adev->reset_domain->in_gpu_reset, 0); |
| 279 | up_write(&adev->reset_sem); | 281 | up_write(&adev->reset_domain->sem); |
| 280 | 282 | ||
| 281 | /* Trigger recovery for world switch failure if no TDR */ | 283 | /* Trigger recovery for world switch failure if no TDR */ |
| 282 | if (amdgpu_device_should_recover_gpu(adev) | 284 | if (amdgpu_device_should_recover_gpu(adev) |
| 283 | && (!amdgpu_device_has_job_running(adev) || | 285 | && (!amdgpu_device_has_job_running(adev) || |
| 284 | adev->sdma_timeout == MAX_SCHEDULE_TIMEOUT)) | 286 | adev->sdma_timeout == MAX_SCHEDULE_TIMEOUT)) |
| 285 | amdgpu_device_gpu_recover(adev, NULL); | 287 | amdgpu_device_gpu_recover_imp(adev, NULL); |
| 286 | } | 288 | } |
| 287 | 289 | ||
| 288 | static int xgpu_ai_set_mailbox_rcv_irq(struct amdgpu_device *adev, | 290 | static int xgpu_ai_set_mailbox_rcv_irq(struct amdgpu_device *adev, |
| @@ -307,8 +309,11 @@ static int xgpu_ai_mailbox_rcv_irq(struct amdgpu_device *adev, · 18 unmodified lines | |||
| 307 | 309 | ||
| 308 | switch (event) { | 310 | switch (event) { |
| 309 | case IDH_FLR_NOTIFICATION: | 311 | case IDH_FLR_NOTIFICATION: |
| 310 | if (amdgpu_sriov_runtime(adev)) | 312 | if (amdgpu_sriov_runtime(adev) && !amdgpu_in_reset(adev)) |
| 311 | schedule_work(&adev->virt.flr_work); | 313 | WARN_ONCE(!amdgpu_reset_domain_schedule(adev->reset_domain, |
| 314 | &adev->virt.flr_work), | ||
| 315 | "Failed to queue work! at %s", | ||
| 316 | __func__); | ||
| 312 | break; | 317 | break; |
| 313 | case IDH_QUERY_ALIVE: | 318 | case IDH_QUERY_ALIVE: |
| 314 | xgpu_ai_mailbox_send_ack(adev); | 319 | xgpu_ai_mailbox_send_ack(adev); |
| @@ -31,6 +31,8 @@ · 30 unmodified lines | |||
| 31 | #include "soc15_common.h" | 31 | #include "soc15_common.h" |
| 32 | #include "mxgpu_nv.h" | 32 | #include "mxgpu_nv.h" |
| 33 | 33 | ||
| 34 | #include "amdgpu_reset.h" | ||
| 35 | |||
| 34 | static void xgpu_nv_mailbox_send_ack(struct amdgpu_device *adev) | 36 | static void xgpu_nv_mailbox_send_ack(struct amdgpu_device *adev) |
| 35 | { | 37 | { |
| 36 | WREG8(NV_MAIBOX_CONTROL_RCV_OFFSET_BYTE, 2); | 38 | WREG8(NV_MAIBOX_CONTROL_RCV_OFFSET_BYTE, 2); |
| @@ -281,10 +283,10 @@ static void xgpu_nv_mailbox_flr_work(struct work_struct *work) · 244 unmodified lines | |||
| 281 | * otherwise the mailbox msg will be ruined/reseted by | 283 | * otherwise the mailbox msg will be ruined/reseted by |
| 282 | * the VF FLR. | 284 | * the VF FLR. |
| 283 | */ | 285 | */ |
| 284 | if (atomic_cmpxchg(&adev->in_gpu_reset, 0, 1) != 0) | 286 | if (atomic_cmpxchg(&adev->reset_domain->in_gpu_reset, 0, 1) != 0) |
| 285 | return; | 287 | return; |
| 286 | 288 | ||
| 287 | down_write(&adev->reset_sem); | 289 | down_write(&adev->reset_domain->sem); |
| 288 | 290 | ||
| 289 | amdgpu_virt_fini_data_exchange(adev); | 291 | amdgpu_virt_fini_data_exchange(adev); |
| 290 | 292 | ||
| @@ -299,8 +301,8 @@ static void xgpu_nv_mailbox_flr_work(struct work_struct *work) · 8 unmodified lines | |||
| 299 | } while (timeout > 1); | 301 | } while (timeout > 1); |
| 300 | 302 | ||
| 301 | flr_done: | 303 | flr_done: |
| 302 | atomic_set(&adev->in_gpu_reset, 0); | 304 | atomic_set(&adev->reset_domain->in_gpu_reset, 0); |
| 303 | up_write(&adev->reset_sem); | 305 | up_write(&adev->reset_domain->sem); |
| 304 | 306 | ||
| 305 | /* Trigger recovery for world switch failure if no TDR */ | 307 | /* Trigger recovery for world switch failure if no TDR */ |
| 306 | if (amdgpu_device_should_recover_gpu(adev) | 308 | if (amdgpu_device_should_recover_gpu(adev) |
| @@ -309,7 +311,7 @@ flr_done: · 2 unmodified lines | |||
| 309 | adev->gfx_timeout == MAX_SCHEDULE_TIMEOUT || | 311 | adev->gfx_timeout == MAX_SCHEDULE_TIMEOUT || |
| 310 | adev->compute_timeout == MAX_SCHEDULE_TIMEOUT || | 312 | adev->compute_timeout == MAX_SCHEDULE_TIMEOUT || |
| 311 | adev->video_timeout == MAX_SCHEDULE_TIMEOUT)) | 313 | adev->video_timeout == MAX_SCHEDULE_TIMEOUT)) |
| 312 | amdgpu_device_gpu_recover(adev, NULL); | 314 | amdgpu_device_gpu_recover_imp(adev, NULL); |
| 313 | } | 315 | } |
| 314 | 316 | ||
| 315 | static int xgpu_nv_set_mailbox_rcv_irq(struct amdgpu_device *adev, | 317 | static int xgpu_nv_set_mailbox_rcv_irq(struct amdgpu_device *adev, |
| @@ -337,8 +339,11 @@ static int xgpu_nv_mailbox_rcv_irq(struct amdgpu_device *adev, · 21 unmodified lines | |||
| 337 | 339 | ||
| 338 | switch (event) { | 340 | switch (event) { |
| 339 | case IDH_FLR_NOTIFICATION: | 341 | case IDH_FLR_NOTIFICATION: |
| 340 | if (amdgpu_sriov_runtime(adev)) | 342 | if (amdgpu_sriov_runtime(adev) && !amdgpu_in_reset(adev)) |
| 341 | schedule_work(&adev->virt.flr_work); | 343 | WARN_ONCE(!amdgpu_reset_domain_schedule(adev->reset_domain, |
| 344 | &adev->virt.flr_work), | ||
| 345 | "Failed to queue work! at %s", | ||
| 346 | __func__); | ||
| 342 | break; | 347 | break; |
| 343 | /* READY_TO_ACCESS_GPU is fetched by kernel polling, IRQ can ignore | 348 | /* READY_TO_ACCESS_GPU is fetched by kernel polling, IRQ can ignore |
| 344 | * it byfar since that polling thread will handle it, | 349 | * it byfar since that polling thread will handle it, |
| @@ -42,6 +42,8 @@ · 41 unmodified lines | |||
| 42 | #include "smu/smu_7_1_3_d.h" | 42 | #include "smu/smu_7_1_3_d.h" |
| 43 | #include "mxgpu_vi.h" | 43 | #include "mxgpu_vi.h" |
| 44 | 44 | ||
| 45 | #include "amdgpu_reset.h" | ||
| 46 | |||
| 45 | /* VI golden setting */ | 47 | /* VI golden setting */ |
| 46 | static const u32 xgpu_fiji_mgcg_cgcg_init[] = { | 48 | static const u32 xgpu_fiji_mgcg_cgcg_init[] = { |
| 47 | mmRLC_CGTT_MGCG_OVERRIDE, 0xffffffff, 0xffffffff, | 49 | mmRLC_CGTT_MGCG_OVERRIDE, 0xffffffff, 0xffffffff, |
| @@ -521,7 +523,7 @@ static void xgpu_vi_mailbox_flr_work(struct work_struct *work) · 473 unmodified lines | |||
| 521 | 523 | ||
| 522 | /* Trigger recovery due to world switch failure */ | 524 | /* Trigger recovery due to world switch failure */ |
| 523 | if (amdgpu_device_should_recover_gpu(adev)) | 525 | if (amdgpu_device_should_recover_gpu(adev)) |
| 524 | amdgpu_device_gpu_recover(adev, NULL); | 526 | amdgpu_device_gpu_recover_imp(adev, NULL); |
| 525 | } | 527 | } |
| 526 | 528 | ||
| 527 | static int xgpu_vi_set_mailbox_rcv_irq(struct amdgpu_device *adev, | 529 | static int xgpu_vi_set_mailbox_rcv_irq(struct amdgpu_device *adev, |
| @@ -544,14 +546,17 @@ static int xgpu_vi_mailbox_rcv_irq(struct amdgpu_device *adev, · 16 unmodified lines | |||
| 544 | { | 546 | { |
| 545 | int r; | 547 | int r; |
| 546 | 548 | ||
| 547 | /* trigger gpu-reset by hypervisor only if TDR disbaled */ | 549 | /* trigger gpu-reset by hypervisor only if TDR disabled */ |
| 548 | if (!amdgpu_gpu_recovery) { | 550 | if (!amdgpu_gpu_recovery) { |
| 549 | /* see what event we get */ | 551 | /* see what event we get */ |
| 550 | r = xgpu_vi_mailbox_rcv_msg(adev, IDH_FLR_NOTIFICATION); | 552 | r = xgpu_vi_mailbox_rcv_msg(adev, IDH_FLR_NOTIFICATION); |
| 551 | 553 | ||
| 552 | /* only handle FLR_NOTIFY now */ | 554 | /* only handle FLR_NOTIFY now */ |
| 553 | if (!r) | 555 | if (!r && !amdgpu_in_reset(adev)) |
| 554 | schedule_work(&adev->virt.flr_work); | 556 | WARN_ONCE(!amdgpu_reset_domain_schedule(adev->reset_domain, |
| 557 | &adev->virt.flr_work), | ||
| 558 | "Failed to queue work! at %s", | ||
| 559 | __func__); | ||
| 555 | } | 560 | } |
| 556 | 561 | ||
| 557 | return 0; | 562 | return 0; |
| @@ -59,10 +59,16 @@ static u32 nbio_v7_2_get_rev_id(struct amdgpu_device *adev) · 58 unmodified lines | |||
| 59 | { | 59 | { |
| 60 | u32 tmp; | 60 | u32 tmp; |
| 61 | 61 | ||
| 62 | if (adev->asic_type == CHIP_YELLOW_CARP) | 62 | switch (adev->ip_versions[NBIO_HWIP][0]) { |
| 63 | case IP_VERSION(7, 2, 1): | ||
| 64 | case IP_VERSION(7, 3, 0): | ||
| 65 | case IP_VERSION(7, 5, 0): | ||
| 63 | tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_YC); | 66 | tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0_YC); |
| 64 | else | 67 | break; |
| 68 | default: | ||
| 65 | tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0); | 69 | tmp = RREG32_SOC15(NBIO, 0, regRCC_STRAP0_RCC_DEV0_EPF0_STRAP0); |
| 70 | break; | ||
| 71 | } | ||
| 66 | 72 | ||
| 67 | tmp &= RCC_STRAP0_RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0_MASK; | 73 | tmp &= RCC_STRAP0_RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0_MASK; |
| 68 | tmp >>= RCC_STRAP0_RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0__SHIFT; | 74 | tmp >>= RCC_STRAP0_RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0__SHIFT; |
| @@ -72,20 +78,26 @@ static u32 nbio_v7_2_get_rev_id(struct amdgpu_device *adev) · 3 unmodified lines | |||
| 72 | 78 | ||
| 73 | static void nbio_v7_2_mc_access_enable(struct amdgpu_device *adev, bool enable) | 79 | static void nbio_v7_2_mc_access_enable(struct amdgpu_device *adev, bool enable) |
| 74 | { | 80 | { |
| 75 | if (enable) | 81 | switch (adev->ip_versions[NBIO_HWIP][0]) { |
| 76 | if (adev->asic_type == CHIP_YELLOW_CARP) | 82 | case IP_VERSION(7, 2, 1): |
| 83 | case IP_VERSION(7, 3, 0): | ||
| 84 | case IP_VERSION(7, 5, 0): | ||
| 85 | if (enable) | ||
| 77 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC, | 86 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC, |
| 78 | BIF_BX0_BIF_FB_EN__FB_READ_EN_MASK | | 87 | BIF_BX0_BIF_FB_EN__FB_READ_EN_MASK | |
| 79 | BIF_BX0_BIF_FB_EN__FB_WRITE_EN_MASK); | 88 | BIF_BX0_BIF_FB_EN__FB_WRITE_EN_MASK); |
| 80 | else | 89 | else |
| 90 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC, 0); | ||
| 91 | break; | ||
| 92 | default: | ||
| 93 | if (enable) | ||
| 81 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN, | 94 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN, |
| 82 | BIF_BX0_BIF_FB_EN__FB_READ_EN_MASK | | 95 | BIF_BX0_BIF_FB_EN__FB_READ_EN_MASK | |
| 83 | BIF_BX0_BIF_FB_EN__FB_WRITE_EN_MASK); | 96 | BIF_BX0_BIF_FB_EN__FB_WRITE_EN_MASK); |
| 84 | else | ||
| 85 | if (adev->asic_type == CHIP_YELLOW_CARP) | ||
| 86 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN_YC, 0); | ||
| 87 | else | 97 | else |
| 88 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN, 0); | 98 | WREG32_SOC15(NBIO, 0, regBIF_BX0_BIF_FB_EN, 0); |
| 99 | break; | ||
| 100 | } | ||
| 89 | } | 101 | } |
| 90 | 102 | ||
| 91 | static u32 nbio_v7_2_get_memsize(struct amdgpu_device *adev) | 103 | static u32 nbio_v7_2_get_memsize(struct amdgpu_device *adev) |
| @@ -250,7 +262,10 @@ static void nbio_v7_2_update_medium_grain_light_sleep(struct amdgpu_device *adev · 158 unmodified lines | |||
| 250 | { | 262 | { |
| 251 | uint32_t def, data; | 263 | uint32_t def, data; |
| 252 | 264 | ||
| 253 | if (adev->asic_type == CHIP_YELLOW_CARP) { | 265 | switch (adev->ip_versions[NBIO_HWIP][0]) { |
| 266 | case IP_VERSION(7, 2, 1): | ||
| 267 | case IP_VERSION(7, 3, 0): | ||
| 268 | case IP_VERSION(7, 5, 0): | ||
| 254 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2)); | 269 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2)); |
| 255 | if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS)) | 270 | if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS)) |
| 256 | data |= PCIE_CNTL2__SLV_MEM_LS_EN_MASK; | 271 | data |= PCIE_CNTL2__SLV_MEM_LS_EN_MASK; |
| @@ -260,8 +275,8 @@ static void nbio_v7_2_update_medium_grain_light_sleep(struct amdgpu_device *adev · 3 unmodified lines | |||
| 260 | if (def != data) | 275 | if (def != data) |
| 261 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2), data); | 276 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2), data); |
| 262 | 277 | ||
| 263 | data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_TX_POWER_CTRL_1)); | 278 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, |
| 264 | def = data; | 279 | regBIF1_PCIE_TX_POWER_CTRL_1)); |
| 265 | if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS)) | 280 | if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS)) |
| 266 | data |= (BIF1_PCIE_TX_POWER_CTRL_1__MST_MEM_LS_EN_MASK | | 281 | data |= (BIF1_PCIE_TX_POWER_CTRL_1__MST_MEM_LS_EN_MASK | |
| 267 | BIF1_PCIE_TX_POWER_CTRL_1__REPLAY_MEM_LS_EN_MASK); | 282 | BIF1_PCIE_TX_POWER_CTRL_1__REPLAY_MEM_LS_EN_MASK); |
| @@ -272,7 +287,8 @@ static void nbio_v7_2_update_medium_grain_light_sleep(struct amdgpu_device *adev · 4 unmodified lines | |||
| 272 | if (def != data) | 287 | if (def != data) |
| 273 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_TX_POWER_CTRL_1), | 288 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_TX_POWER_CTRL_1), |
| 274 | data); | 289 | data); |
| 275 | } else { | 290 | break; |
| 291 | default: | ||
| 276 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2)); | 292 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2)); |
| 277 | if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS)) | 293 | if (enable && (adev->cg_flags & AMD_CG_SUPPORT_BIF_LS)) |
| 278 | data |= (PCIE_CNTL2__SLV_MEM_LS_EN_MASK | | 294 | data |= (PCIE_CNTL2__SLV_MEM_LS_EN_MASK | |
| @@ -285,6 +301,7 @@ static void nbio_v7_2_update_medium_grain_light_sleep(struct amdgpu_device *adev · 6 unmodified lines | |||
| 285 | 301 | ||
| 286 | if (def != data) | 302 | if (def != data) |
| 287 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2), data); | 303 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CNTL2), data); |
| 304 | break; | ||
| 288 | } | 305 | } |
| 289 | } | 306 | } |
| 290 | 307 | ||
| @@ -352,7 +369,10 @@ const struct nbio_hdp_flush_reg nbio_v7_2_hdp_flush_reg = { · 61 unmodified lines | |||
| 352 | static void nbio_v7_2_init_registers(struct amdgpu_device *adev) | 369 | static void nbio_v7_2_init_registers(struct amdgpu_device *adev) |
| 353 | { | 370 | { |
| 354 | uint32_t def, data; | 371 | uint32_t def, data; |
| 355 | if (adev->asic_type == CHIP_YELLOW_CARP) { | 372 | switch (adev->ip_versions[NBIO_HWIP][0]) { |
| 373 | case IP_VERSION(7, 2, 1): | ||
| 374 | case IP_VERSION(7, 3, 0): | ||
| 375 | case IP_VERSION(7, 5, 0): | ||
| 356 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_MST_CTRL_3)); | 376 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_MST_CTRL_3)); |
| 357 | data = REG_SET_FIELD(data, BIF1_PCIE_MST_CTRL_3, | 377 | data = REG_SET_FIELD(data, BIF1_PCIE_MST_CTRL_3, |
| 358 | CI_SWUS_MAX_READ_REQUEST_SIZE_MODE, 1); | 378 | CI_SWUS_MAX_READ_REQUEST_SIZE_MODE, 1); |
| @@ -361,7 +381,8 @@ static void nbio_v7_2_init_registers(struct amdgpu_device *adev) · 2 unmodified lines | |||
| 361 | 381 | ||
| 362 | if (def != data) | 382 | if (def != data) |
| 363 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_MST_CTRL_3), data); | 383 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regBIF1_PCIE_MST_CTRL_3), data); |
| 364 | } else { | 384 | break; |
| 385 | default: | ||
| 365 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CONFIG_CNTL)); | 386 | def = data = RREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CONFIG_CNTL)); |
| 366 | data = REG_SET_FIELD(data, PCIE_CONFIG_CNTL, | 387 | data = REG_SET_FIELD(data, PCIE_CONFIG_CNTL, |
| 367 | CI_SWUS_MAX_READ_REQUEST_SIZE_MODE, 1); | 388 | CI_SWUS_MAX_READ_REQUEST_SIZE_MODE, 1); |
| @@ -370,6 +391,7 @@ static void nbio_v7_2_init_registers(struct amdgpu_device *adev) · 2 unmodified lines | |||
| 370 | 391 | ||
| 371 | if (def != data) | 392 | if (def != data) |
| 372 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CONFIG_CNTL), data); | 393 | WREG32_PCIE_PORT(SOC15_REG_OFFSET(NBIO, 0, regPCIE_CONFIG_CNTL), data); |
| 394 | break; | ||
| 373 | } | 395 | } |
| 374 | 396 | ||
| 375 | if (amdgpu_sriov_vf(adev)) | 397 | if (amdgpu_sriov_vf(adev)) |
| @@ -658,16 +658,28 @@ static void nbio_v7_4_enable_doorbell_interrupt(struct amdgpu_device *adev, · 657 unmodified lines | |||
| 658 | DOORBELL_INTERRUPT_DISABLE, enable ? 0 : 1); | 658 | DOORBELL_INTERRUPT_DISABLE, enable ? 0 : 1); |
| 659 | } | 659 | } |
| 660 | 660 | ||
| 661 | const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs = { | 661 | const struct amdgpu_ras_block_hw_ops nbio_v7_4_ras_hw_ops = { |
| 662 | .query_ras_error_count = nbio_v7_4_query_ras_error_count, | ||
| 663 | }; | ||
| 664 | |||
| 665 | struct amdgpu_nbio_ras nbio_v7_4_ras = { | ||
| 666 | .ras_block = { | ||
| 667 | .ras_comm = { | ||
| 668 | .name = "pcie_bif", | ||
| 669 | .block = AMDGPU_RAS_BLOCK__PCIE_BIF, | ||
| 670 | .type = AMDGPU_RAS_ERROR__MULTI_UNCORRECTABLE, | ||
| 671 | }, | ||
| 672 | .hw_ops = &nbio_v7_4_ras_hw_ops, | ||
| 673 | .ras_late_init = amdgpu_nbio_ras_late_init, | ||
| 674 | .ras_fini = amdgpu_nbio_ras_fini, | ||
| 675 | }, | ||
| 662 | .handle_ras_controller_intr_no_bifring = nbio_v7_4_handle_ras_controller_intr_no_bifring, | 676 | .handle_ras_controller_intr_no_bifring = nbio_v7_4_handle_ras_controller_intr_no_bifring, |
| 663 | .handle_ras_err_event_athub_intr_no_bifring = nbio_v7_4_handle_ras_err_event_athub_intr_no_bifring, | 677 | .handle_ras_err_event_athub_intr_no_bifring = nbio_v7_4_handle_ras_err_event_athub_intr_no_bifring, |
| 664 | .init_ras_controller_interrupt = nbio_v7_4_init_ras_controller_interrupt, | 678 | .init_ras_controller_interrupt = nbio_v7_4_init_ras_controller_interrupt, |
| 665 | .init_ras_err_event_athub_interrupt = nbio_v7_4_init_ras_err_event_athub_interrupt, | 679 | .init_ras_err_event_athub_interrupt = nbio_v7_4_init_ras_err_event_athub_interrupt, |
| 666 | .query_ras_error_count = nbio_v7_4_query_ras_error_count, | ||
| 667 | .ras_late_init = amdgpu_nbio_ras_late_init, | ||
| 668 | .ras_fini = amdgpu_nbio_ras_fini, | ||
| 669 | }; | 680 | }; |
| 670 | 681 | ||
| 682 | |||
| 671 | static void nbio_v7_4_program_ltr(struct amdgpu_device *adev) | 683 | static void nbio_v7_4_program_ltr(struct amdgpu_device *adev) |
| 672 | { | 684 | { |
| 673 | uint32_t def, data; | 685 | uint32_t def, data; |
| @@ -29,6 +29,6 @@ · 28 unmodified lines | |||
| 29 | extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg; | 29 | extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg; |
| 30 | extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg_ald; | 30 | extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg_ald; |
| 31 | extern const struct amdgpu_nbio_funcs nbio_v7_4_funcs; | 31 | extern const struct amdgpu_nbio_funcs nbio_v7_4_funcs; |
| 32 | extern const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs; | 32 | extern struct amdgpu_nbio_ras nbio_v7_4_ras; |
| 33 | 33 | ||
| 34 | #endif | 34 | #endif |
| @@ -204,6 +204,7 @@ static int nv_query_video_codecs(struct amdgpu_device *adev, bool encode, · 203 unmodified lines | |||
| 204 | *codecs = &sc_video_codecs_decode; | 204 | *codecs = &sc_video_codecs_decode; |
| 205 | return 0; | 205 | return 0; |
| 206 | case IP_VERSION(3, 1, 1): | 206 | case IP_VERSION(3, 1, 1): |
| 207 | case IP_VERSION(3, 1, 2): | ||
| 207 | if (encode) | 208 | if (encode) |
| 208 | *codecs = &nv_video_codecs_encode; | 209 | *codecs = &nv_video_codecs_encode; |
| 209 | else | 210 | else |
| @@ -258,21 +259,6 @@ static u64 nv_pcie_rreg64(struct amdgpu_device *adev, u32 reg) · 48 unmodified lines | |||
| 258 | return amdgpu_device_indirect_rreg64(adev, address, data, reg); | 259 | return amdgpu_device_indirect_rreg64(adev, address, data, reg); |
| 259 | } | 260 | } |
| 260 | 261 | ||
| 261 | static u32 nv_pcie_port_rreg(struct amdgpu_device *adev, u32 reg) | ||
| 262 | { | ||
| 263 | unsigned long flags, address, data; | ||
| 264 | u32 r; | ||
| 265 | address = adev->nbio.funcs->get_pcie_port_index_offset(adev); | ||
| 266 | data = adev->nbio.funcs->get_pcie_port_data_offset(adev); | ||
| 267 | |||
| 268 | spin_lock_irqsave(&adev->pcie_idx_lock, flags); | ||
| 269 | WREG32(address, reg * 4); | ||
| 270 | (void)RREG32(address); | ||
| 271 | r = RREG32(data); | ||
| 272 | spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); | ||
| 273 | return r; | ||
| 274 | } | ||
| 275 | |||
| 276 | static void nv_pcie_wreg64(struct amdgpu_device *adev, u32 reg, u64 v) | 262 | static void nv_pcie_wreg64(struct amdgpu_device *adev, u32 reg, u64 v) |
| 277 | { | 263 | { |
| 278 | unsigned long address, data; | 264 | unsigned long address, data; |
| @@ -283,21 +269,6 @@ static void nv_pcie_wreg64(struct amdgpu_device *adev, u32 reg, u64 v) · 4 unmodified lines | |||
| 283 | amdgpu_device_indirect_wreg64(adev, address, data, reg, v); | 269 | amdgpu_device_indirect_wreg64(adev, address, data, reg, v); |
| 284 | } | 270 | } |
| 285 | 271 | ||
| 286 | static void nv_pcie_port_wreg(struct amdgpu_device *adev, u32 reg, u32 v) | ||
| 287 | { | ||
| 288 | unsigned long flags, address, data; | ||
| 289 | |||
| 290 | address = adev->nbio.funcs->get_pcie_port_index_offset(adev); | ||
| 291 | data = adev->nbio.funcs->get_pcie_port_data_offset(adev); | ||
| 292 | |||
| 293 | spin_lock_irqsave(&adev->pcie_idx_lock, flags); | ||
| 294 | WREG32(address, reg * 4); | ||
| 295 | (void)RREG32(address); | ||
| 296 | WREG32(data, v); | ||
| 297 | (void)RREG32(data); | ||
| 298 | spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); | ||
| 299 | } | ||
| 300 | |||
| 301 | static u32 nv_didt_rreg(struct amdgpu_device *adev, u32 reg) | 272 | static u32 nv_didt_rreg(struct amdgpu_device *adev, u32 reg) |
| 302 | { | 273 | { |
| 303 | unsigned long flags, address, data; | 274 | unsigned long flags, address, data; |
| @@ -360,38 +331,6 @@ static bool nv_read_disabled_bios(struct amdgpu_device *adev) · 56 unmodified lines | |||
| 360 | return false; | 331 | return false; |
| 361 | } | 332 | } |
| 362 | 333 | ||
| 363 | static bool nv_read_bios_from_rom(struct amdgpu_device *adev, | ||
| 364 | u8 *bios, u32 length_bytes) | ||
| 365 | { | ||
| 366 | u32 *dw_ptr; | ||
| 367 | u32 i, length_dw; | ||
| 368 | u32 rom_index_offset, rom_data_offset; | ||
| 369 | |||
| 370 | if (bios == NULL) | ||
| 371 | return false; | ||
| 372 | if (length_bytes == 0) | ||
| 373 | return false; | ||
| 374 | /* APU vbios image is part of sbios image */ | ||
| 375 | if (adev->flags & AMD_IS_APU) | ||
| 376 | return false; | ||
| 377 | |||
| 378 | dw_ptr = (u32 *)bios; | ||
| 379 | length_dw = ALIGN(length_bytes, 4) / 4; | ||
| 380 | |||
| 381 | rom_index_offset = | ||
| 382 | adev->smuio.funcs->get_rom_index_offset(adev); | ||
| 383 | rom_data_offset = | ||
| 384 | adev->smuio.funcs->get_rom_data_offset(adev); | ||
| 385 | |||
| 386 | /* set rom index to 0 */ | ||
| 387 | WREG32(rom_index_offset, 0); | ||
| 388 | /* read out the rom data */ | ||
| 389 | for (i = 0; i < length_dw; i++) | ||
| 390 | dw_ptr[i] = RREG32(rom_data_offset); | ||
| 391 | |||
| 392 | return true; | ||
| 393 | } | ||
| 394 | |||
| 395 | static struct soc15_allowed_register_entry nv_allowed_read_registers[] = { | 334 | static struct soc15_allowed_register_entry nv_allowed_read_registers[] = { |
| 396 | { SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS)}, | 335 | { SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS)}, |
| 397 | { SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS2)}, | 336 | { SOC15_REG_ENTRY(GC, 0, mmGRBM_STATUS2)}, |
| @@ -515,6 +454,8 @@ nv_asic_reset_method(struct amdgpu_device *adev) · 117 unmodified lines | |||
| 515 | case IP_VERSION(11, 5, 0): | 454 | case IP_VERSION(11, 5, 0): |
| 516 | case IP_VERSION(13, 0, 1): | 455 | case IP_VERSION(13, 0, 1): |
| 517 | case IP_VERSION(13, 0, 3): | 456 | case IP_VERSION(13, 0, 3): |
| 457 | case IP_VERSION(13, 0, 5): | ||
| 458 | case IP_VERSION(13, 0, 8): | ||
| 518 | return AMD_RESET_METHOD_MODE2; | 459 | return AMD_RESET_METHOD_MODE2; |
| 519 | case IP_VERSION(11, 0, 7): | 460 | case IP_VERSION(11, 0, 7): |
| 520 | case IP_VERSION(11, 0, 11): | 461 | case IP_VERSION(11, 0, 11): |
| @@ -584,7 +525,7 @@ static void nv_pcie_gen3_enable(struct amdgpu_device *adev) · 63 unmodified lines | |||
| 584 | 525 | ||
| 585 | static void nv_program_aspm(struct amdgpu_device *adev) | 526 | static void nv_program_aspm(struct amdgpu_device *adev) |
| 586 | { | 527 | { |
| 587 | if (!amdgpu_aspm) | 528 | if (!amdgpu_device_should_use_aspm(adev)) |
| 588 | return; | 529 | return; |
| 589 | 530 | ||
| 590 | if (!(adev->flags & AMD_IS_APU) && | 531 | if (!(adev->flags & AMD_IS_APU) && |
| @@ -699,7 +640,8 @@ static int nv_update_umd_stable_pstate(struct amdgpu_device *adev, · 108 unmodified lines | |||
| 699 | adev->gfx.funcs->update_perfmon_mgcg(adev, !enter); | 640 | adev->gfx.funcs->update_perfmon_mgcg(adev, !enter); |
| 700 | 641 | ||
| 701 | if (!(adev->flags & AMD_IS_APU) && | 642 | if (!(adev->flags & AMD_IS_APU) && |
| 702 | (adev->nbio.funcs->enable_aspm)) | 643 | (adev->nbio.funcs->enable_aspm) && |
| 644 | amdgpu_device_should_use_aspm(adev)) | ||
| 703 | adev->nbio.funcs->enable_aspm(adev, !enter); | 645 | adev->nbio.funcs->enable_aspm(adev, !enter); |
| 704 | 646 | ||
| 705 | return 0; | 647 | return 0; |
| @@ -708,7 +650,7 @@ static int nv_update_umd_stable_pstate(struct amdgpu_device *adev, · 2 unmodified lines | |||
| 708 | static const struct amdgpu_asic_funcs nv_asic_funcs = | 650 | static const struct amdgpu_asic_funcs nv_asic_funcs = |
| 709 | { | 651 | { |
| 710 | .read_disabled_bios = &nv_read_disabled_bios, | 652 | .read_disabled_bios = &nv_read_disabled_bios, |
| 711 | .read_bios_from_rom = &nv_read_bios_from_rom, | 653 | .read_bios_from_rom = &amdgpu_soc15_read_bios_from_rom, |
| 712 | .read_register = &nv_read_register, | 654 | .read_register = &nv_read_register, |
| 713 | .reset = &nv_asic_reset, | 655 | .reset = &nv_asic_reset, |
| 714 | .reset_method = &nv_asic_reset_method, | 656 | .reset_method = &nv_asic_reset_method, |
| @@ -742,8 +684,8 @@ static int nv_common_early_init(void *handle) · 27 unmodified lines | |||
| 742 | adev->pcie_wreg = &nv_pcie_wreg; | 684 | adev->pcie_wreg = &nv_pcie_wreg; |
| 743 | adev->pcie_rreg64 = &nv_pcie_rreg64; | 685 | adev->pcie_rreg64 = &nv_pcie_rreg64; |
| 744 | adev->pcie_wreg64 = &nv_pcie_wreg64; | 686 | adev->pcie_wreg64 = &nv_pcie_wreg64; |
| 745 | adev->pciep_rreg = &nv_pcie_port_rreg; | 687 | adev->pciep_rreg = amdgpu_device_pcie_port_rreg; |
| 746 | adev->pciep_wreg = &nv_pcie_port_wreg; | 688 | adev->pciep_wreg = amdgpu_device_pcie_port_wreg; |
| 747 | 689 | ||
| 748 | /* TODO: will add them during VCN v2 implementation */ | 690 | /* TODO: will add them during VCN v2 implementation */ |
| 749 | adev->uvd_ctx_rreg = NULL; | 691 | adev->uvd_ctx_rreg = NULL; |
| @@ -964,10 +906,52 @@ static int nv_common_early_init(void *handle) · 214 unmodified lines | |||
| 964 | adev->external_rev_id = adev->rev_id + 0x01; | 906 | adev->external_rev_id = adev->rev_id + 0x01; |
| 965 | break; | 907 | break; |
| 966 | case IP_VERSION(10, 1, 3): | 908 | case IP_VERSION(10, 1, 3): |
| 909 | case IP_VERSION(10, 1, 4): | ||
| 967 | adev->cg_flags = 0; | 910 | adev->cg_flags = 0; |
| 968 | adev->pg_flags = 0; | 911 | adev->pg_flags = 0; |
| 969 | adev->external_rev_id = adev->rev_id + 0x82; | 912 | adev->external_rev_id = adev->rev_id + 0x82; |
| 970 | break; | 913 | break; |
| 914 | case IP_VERSION(10, 3, 6): | ||
| 915 | adev->cg_flags = AMD_CG_SUPPORT_GFX_MGCG | | ||
| 916 | AMD_CG_SUPPORT_GFX_MGLS | | ||
| 917 | AMD_CG_SUPPORT_GFX_CGCG | | ||
| 918 | AMD_CG_SUPPORT_GFX_CGLS | | ||
| 919 | AMD_CG_SUPPORT_GFX_3D_CGCG | | ||
| 920 | AMD_CG_SUPPORT_GFX_3D_CGLS | | ||
| 921 | AMD_CG_SUPPORT_GFX_RLC_LS | | ||
| 922 | AMD_CG_SUPPORT_GFX_CP_LS | | ||
| 923 | AMD_CG_SUPPORT_GFX_FGCG | | ||
| 924 | AMD_CG_SUPPORT_MC_MGCG | | ||
| 925 | AMD_CG_SUPPORT_MC_LS | | ||
| 926 | AMD_CG_SUPPORT_SDMA_LS | | ||
| 927 | AMD_CG_SUPPORT_HDP_MGCG | | ||
| 928 | AMD_CG_SUPPORT_HDP_LS | | ||
| 929 | AMD_CG_SUPPORT_ATHUB_MGCG | | ||
| 930 | AMD_CG_SUPPORT_ATHUB_LS | | ||
| 931 | AMD_CG_SUPPORT_IH_CG | | ||
| 932 | AMD_CG_SUPPORT_VCN_MGCG | | ||
| 933 | AMD_CG_SUPPORT_JPEG_MGCG; | ||
| 934 | adev->pg_flags = AMD_PG_SUPPORT_GFX_PG | | ||
| 935 | AMD_PG_SUPPORT_VCN | | ||
| 936 | AMD_PG_SUPPORT_VCN_DPG | | ||
| 937 | AMD_PG_SUPPORT_JPEG; | ||
| 938 | adev->external_rev_id = adev->rev_id + 0x01; | ||
| 939 | break; | ||
| 940 | case IP_VERSION(10, 3, 7): | ||
| 941 | adev->cg_flags = AMD_CG_SUPPORT_GFX_MGCG | | ||
| 942 | AMD_CG_SUPPORT_GFX_MGLS | | ||
| 943 | AMD_CG_SUPPORT_GFX_CGCG | | ||
| 944 | AMD_CG_SUPPORT_GFX_CGLS | | ||
| 945 | AMD_CG_SUPPORT_GFX_3D_CGCG | | ||
| 946 | AMD_CG_SUPPORT_GFX_3D_CGLS | | ||
| 947 | AMD_CG_SUPPORT_GFX_RLC_LS | | ||
| 948 | AMD_CG_SUPPORT_GFX_CP_LS | | ||
| 949 | AMD_CG_SUPPORT_GFX_FGCG; | ||
| 950 | adev->pg_flags = AMD_PG_SUPPORT_VCN | | ||
| 951 | AMD_PG_SUPPORT_VCN_DPG | | ||
| 952 | AMD_PG_SUPPORT_JPEG; | ||
| 953 | adev->external_rev_id = adev->rev_id + 0x01; | ||
| 954 | break; | ||
| 971 | default: | 955 | default: |
| 972 | /* FIXME: not supported yet */ | 956 | /* FIXME: not supported yet */ |
| 973 | return -EINVAL; | 957 | return -EINVAL; |
| @@ -258,6 +258,7 @@ enum psp_gfx_fw_type { · 257 unmodified lines | |||
| 258 | GFX_FW_TYPE_SDMA6 = 56, /* SDMA6 MI */ | 258 | GFX_FW_TYPE_SDMA6 = 56, /* SDMA6 MI */ |
| 259 | GFX_FW_TYPE_SDMA7 = 57, /* SDMA7 MI */ | 259 | GFX_FW_TYPE_SDMA7 = 57, /* SDMA7 MI */ |
| 260 | GFX_FW_TYPE_VCN1 = 58, /* VCN1 MI */ | 260 | GFX_FW_TYPE_VCN1 = 58, /* VCN1 MI */ |
| 261 | GFX_FW_TYPE_CAP = 62, /* CAP_FW */ | ||
| 261 | GFX_FW_TYPE_REG_LIST = 67, /* REG_LIST MI */ | 262 | GFX_FW_TYPE_REG_LIST = 67, /* REG_LIST MI */ |
| 262 | GFX_FW_TYPE_MAX | 263 | GFX_FW_TYPE_MAX |
| 263 | }; | 264 | }; |
| @@ -53,11 +53,13 @@ MODULE_FIRMWARE("amdgpu/navi14_ta.bin"); · 52 unmodified lines | |||
| 53 | MODULE_FIRMWARE("amdgpu/navi12_sos.bin"); | 53 | MODULE_FIRMWARE("amdgpu/navi12_sos.bin"); |
| 54 | MODULE_FIRMWARE("amdgpu/navi12_asd.bin"); | 54 | MODULE_FIRMWARE("amdgpu/navi12_asd.bin"); |
| 55 | MODULE_FIRMWARE("amdgpu/navi12_ta.bin"); | 55 | MODULE_FIRMWARE("amdgpu/navi12_ta.bin"); |
| 56 | MODULE_FIRMWARE("amdgpu/navi12_cap.bin"); | ||
| 56 | MODULE_FIRMWARE("amdgpu/arcturus_sos.bin"); | 57 | MODULE_FIRMWARE("amdgpu/arcturus_sos.bin"); |
| 57 | MODULE_FIRMWARE("amdgpu/arcturus_asd.bin"); | 58 | MODULE_FIRMWARE("amdgpu/arcturus_asd.bin"); |
| 58 | MODULE_FIRMWARE("amdgpu/arcturus_ta.bin"); | 59 | MODULE_FIRMWARE("amdgpu/arcturus_ta.bin"); |
| 59 | MODULE_FIRMWARE("amdgpu/sienna_cichlid_sos.bin"); | 60 | MODULE_FIRMWARE("amdgpu/sienna_cichlid_sos.bin"); |
| 60 | MODULE_FIRMWARE("amdgpu/sienna_cichlid_ta.bin"); | 61 | MODULE_FIRMWARE("amdgpu/sienna_cichlid_ta.bin"); |
| 62 | MODULE_FIRMWARE("amdgpu/sienna_cichlid_cap.bin"); | ||
| 61 | MODULE_FIRMWARE("amdgpu/navy_flounder_sos.bin"); | 63 | MODULE_FIRMWARE("amdgpu/navy_flounder_sos.bin"); |
| 62 | MODULE_FIRMWARE("amdgpu/navy_flounder_ta.bin"); | 64 | MODULE_FIRMWARE("amdgpu/navy_flounder_ta.bin"); |
| 63 | MODULE_FIRMWARE("amdgpu/vangogh_asd.bin"); | 65 | MODULE_FIRMWARE("amdgpu/vangogh_asd.bin"); |
| @@ -177,8 +179,6 @@ static int psp_v11_0_init_microcode(struct psp_context *psp) · 113 unmodified lines | |||
| 177 | err = psp_init_asd_microcode(psp, chip_name); | 179 | err = psp_init_asd_microcode(psp, chip_name); |
| 178 | if (err) | 180 | if (err) |
| 179 | return err; | 181 | return err; |
| 180 | if (amdgpu_sriov_vf(adev)) | ||
| 181 | break; | ||
| 182 | snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_ta.bin", chip_name); | 182 | snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_ta.bin", chip_name); |
| 183 | err = request_firmware(&adev->psp.ta_fw, fw_name, adev->dev); | 183 | err = request_firmware(&adev->psp.ta_fw, fw_name, adev->dev); |
| 184 | if (err) { | 184 | if (err) { |
| @@ -34,6 +34,12 @@ MODULE_FIRMWARE("amdgpu/aldebaran_ta.bin"); · 33 unmodified lines | |||
| 34 | MODULE_FIRMWARE("amdgpu/yellow_carp_asd.bin"); | 34 | MODULE_FIRMWARE("amdgpu/yellow_carp_asd.bin"); |
| 35 | MODULE_FIRMWARE("amdgpu/yellow_carp_toc.bin"); | 35 | MODULE_FIRMWARE("amdgpu/yellow_carp_toc.bin"); |
| 36 | MODULE_FIRMWARE("amdgpu/yellow_carp_ta.bin"); | 36 | MODULE_FIRMWARE("amdgpu/yellow_carp_ta.bin"); |
| 37 | MODULE_FIRMWARE("amdgpu/psp_13_0_5_asd.bin"); | ||
| 38 | MODULE_FIRMWARE("amdgpu/psp_13_0_5_toc.bin"); | ||
| 39 | MODULE_FIRMWARE("amdgpu/psp_13_0_5_ta.bin"); | ||
| 40 | MODULE_FIRMWARE("amdgpu/psp_13_0_8_asd.bin"); | ||
| 41 | MODULE_FIRMWARE("amdgpu/psp_13_0_8_toc.bin"); | ||
| 42 | MODULE_FIRMWARE("amdgpu/psp_13_0_8_ta.bin"); | ||
| 37 | 43 | ||
| 38 | /* For large FW files the time to complete can be very long */ | 44 | /* For large FW files the time to complete can be very long */ |
| 39 | #define USBC_PD_POLLING_LIMIT_S 240 | 45 | #define USBC_PD_POLLING_LIMIT_S 240 |
| @@ -55,6 +61,12 @@ static int psp_v13_0_init_microcode(struct psp_context *psp) · 15 unmodified lines | |||
| 55 | case IP_VERSION(13, 0, 3): | 61 | case IP_VERSION(13, 0, 3): |
| 56 | chip_name = "yellow_carp"; | 62 | chip_name = "yellow_carp"; |
| 57 | break; | 63 | break; |
| 64 | case IP_VERSION(13, 0, 5): | ||
| 65 | chip_name = "psp_13_0_5"; | ||
| 66 | break; | ||
| 67 | case IP_VERSION(13, 0, 8): | ||
| 68 | chip_name = "psp_13_0_8"; | ||
| 69 | break; | ||
| 58 | default: | 70 | default: |
| 59 | BUG(); | 71 | BUG(); |
| 60 | } | 72 | } |
| @@ -69,6 +81,8 @@ static int psp_v13_0_init_microcode(struct psp_context *psp) · 8 unmodified lines | |||
| 69 | break; | 81 | break; |
| 70 | case IP_VERSION(13, 0, 1): | 82 | case IP_VERSION(13, 0, 1): |
| 71 | case IP_VERSION(13, 0, 3): | 83 | case IP_VERSION(13, 0, 3): |
| 84 | case IP_VERSION(13, 0, 5): | ||
| 85 | case IP_VERSION(13, 0, 8): | ||
| 72 | err = psp_init_asd_microcode(psp, chip_name); | 86 | err = psp_init_asd_microcode(psp, chip_name); |
| 73 | if (err) | 87 | if (err) |
| 74 | return err; | 88 | return err; |
| @@ -44,6 +44,7 @@ · 43 unmodified lines | |||
| 44 | 44 | ||
| 45 | MODULE_FIRMWARE("amdgpu/vega10_sos.bin"); | 45 | MODULE_FIRMWARE("amdgpu/vega10_sos.bin"); |
| 46 | MODULE_FIRMWARE("amdgpu/vega10_asd.bin"); | 46 | MODULE_FIRMWARE("amdgpu/vega10_asd.bin"); |
| 47 | MODULE_FIRMWARE("amdgpu/vega10_cap.bin"); | ||
| 47 | MODULE_FIRMWARE("amdgpu/vega12_sos.bin"); | 48 | MODULE_FIRMWARE("amdgpu/vega12_sos.bin"); |
| 48 | MODULE_FIRMWARE("amdgpu/vega12_asd.bin"); | 49 | MODULE_FIRMWARE("amdgpu/vega12_asd.bin"); |
| 49 | 50 | ||
| @@ -1885,22 +1885,16 @@ static int sdma_v4_0_process_ras_data_cb(struct amdgpu_device *adev, · 1884 unmodified lines | |||
| 1885 | static int sdma_v4_0_late_init(void *handle) | 1885 | static int sdma_v4_0_late_init(void *handle) |
| 1886 | { | 1886 | { |
| 1887 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 1887 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 1888 | struct ras_ih_if ih_info = { | ||
| 1889 | .cb = sdma_v4_0_process_ras_data_cb, | ||
| 1890 | }; | ||
| 1891 | 1888 | ||
| 1892 | sdma_v4_0_setup_ulv(adev); | 1889 | sdma_v4_0_setup_ulv(adev); |
| 1893 | 1890 | ||
| 1894 | if (!amdgpu_persistent_edc_harvesting_supported(adev)) { | 1891 | if (!amdgpu_persistent_edc_harvesting_supported(adev)) { |
| 1895 | if (adev->sdma.funcs && | 1892 | if (adev->sdma.ras && adev->sdma.ras->ras_block.hw_ops && |
| 1896 | adev->sdma.funcs->reset_ras_error_count) | 1893 | adev->sdma.ras->ras_block.hw_ops->reset_ras_error_count) |
| 1897 | adev->sdma.funcs->reset_ras_error_count(adev); | 1894 | adev->sdma.ras->ras_block.hw_ops->reset_ras_error_count(adev); |
| 1898 | } | 1895 | } |
| 1899 | 1896 | ||
| 1900 | if (adev->sdma.funcs && adev->sdma.funcs->ras_late_init) | 1897 | return 0; |
| 1901 | return adev->sdma.funcs->ras_late_init(adev, &ih_info); | ||
| 1902 | else | ||
| 1903 | return 0; | ||
| 1904 | } | 1898 | } |
| 1905 | 1899 | ||
| 1906 | static int sdma_v4_0_sw_init(void *handle) | 1900 | static int sdma_v4_0_sw_init(void *handle) |
| @@ -2001,8 +1995,9 @@ static int sdma_v4_0_sw_fini(void *handle) · 94 unmodified lines | |||
| 2001 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 1995 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 2002 | int i; | 1996 | int i; |
| 2003 | 1997 | ||
| 2004 | if (adev->sdma.funcs && adev->sdma.funcs->ras_fini) | 1998 | if (adev->sdma.ras && adev->sdma.ras->ras_block.hw_ops && |
| 2005 | adev->sdma.funcs->ras_fini(adev); | 1999 | adev->sdma.ras->ras_block.ras_fini) |
| 2000 | adev->sdma.ras->ras_block.ras_fini(adev); | ||
| 2006 | 2001 | ||
| 2007 | for (i = 0; i < adev->sdma.num_instances; i++) { | 2002 | for (i = 0; i < adev->sdma.num_instances; i++) { |
| 2008 | amdgpu_ring_fini(&adev->sdma.instance[i].ring); | 2003 | amdgpu_ring_fini(&adev->sdma.instance[i].ring); |
| @@ -2057,6 +2052,10 @@ static int sdma_v4_0_suspend(void *handle) · 48 unmodified lines | |||
| 2057 | { | 2052 | { |
| 2058 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 2053 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 2059 | 2054 | ||
| 2055 | /* SMU saves SDMA state for us */ | ||
| 2056 | if (adev->in_s0ix) | ||
| 2057 | return 0; | ||
| 2058 | |||
| 2060 | return sdma_v4_0_hw_fini(adev); | 2059 | return sdma_v4_0_hw_fini(adev); |
| 2061 | } | 2060 | } |
| 2062 | 2061 | ||
| @@ -2064,6 +2063,10 @@ static int sdma_v4_0_resume(void *handle) · 1 unmodified line | |||
| 2064 | { | 2063 | { |
| 2065 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; | 2064 | struct amdgpu_device *adev = (struct amdgpu_device *)handle; |
| 2066 | 2065 | ||
| 2066 | /* SMU restores SDMA state for us */ | ||
| 2067 | if (adev->in_s0ix) | ||
| 2068 | return 0; | ||
| 2069 | |||
| 2067 | return sdma_v4_0_hw_init(adev); | 2070 | return sdma_v4_0_hw_init(adev); |
| 2068 | } | 2071 | } |
| 2069 | 2072 | ||
| @@ -2740,7 +2743,7 @@ static void sdma_v4_0_get_ras_error_count(uint32_t value, · 670 unmodified lines | |||
| 2740 | } | 2743 | } |
| 2741 | } | 2744 | } |
| 2742 | 2745 | ||
| 2743 | static int sdma_v4_0_query_ras_error_count(struct amdgpu_device *adev, | 2746 | static int sdma_v4_0_query_ras_error_count_by_instance(struct amdgpu_device *adev, |
| 2744 | uint32_t instance, void *ras_error_status) | 2747 | uint32_t instance, void *ras_error_status) |
| 2745 | { | 2748 | { |
| 2746 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; | 2749 | struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status; |
| @@ -2762,6 +2765,18 @@ static int sdma_v4_0_query_ras_error_count(struct amdgpu_device *adev, · 15 unmodified lines | |||
| 2762 | return 0; | 2765 | return 0; |
| 2763 | }; | 2766 | }; |
| 2764 | 2767 | ||
| 2768 | static void sdma_v4_0_query_ras_error_count(struct amdgpu_device *adev, void *ras_error_status) | ||
| 2769 | { | ||
| 2770 | int i = 0; | ||
| 2771 | |||
| 2772 | for (i = 0; i < adev->sdma.num_instances; i++) { | ||
| 2773 | if (sdma_v4_0_query_ras_error_count_by_instance(adev, i, ras_error_status)) { | ||
| 2774 | dev_err(adev->dev, "Query ras error count failed in SDMA%d\n", i); | ||
| 2775 | return; | ||
| 2776 | } | ||
| 2777 | } | ||
| 2778 | } | ||
| 2779 | |||
| 2765 | static void sdma_v4_0_reset_ras_error_count(struct amdgpu_device *adev) | 2780 | static void sdma_v4_0_reset_ras_error_count(struct amdgpu_device *adev) |
| 2766 | { | 2781 | { |
| 2767 | int i; | 2782 | int i; |
| @@ -2773,26 +2788,52 @@ static void sdma_v4_0_reset_ras_error_count(struct amdgpu_device *adev) · 5 unmodified lines | |||
| 2773 | } | 2788 | } |
| 2774 | } | 2789 | } |
| 2775 | 2790 | ||
| 2776 | static const struct amdgpu_sdma_ras_funcs sdma_v4_0_ras_funcs = { | 2791 | const struct amdgpu_ras_block_hw_ops sdma_v4_0_ras_hw_ops = { |
| 2777 | .ras_late_init = amdgpu_sdma_ras_late_init, | ||
| 2778 | .ras_fini = amdgpu_sdma_ras_fini, | ||
| 2779 | .query_ras_error_count = sdma_v4_0_query_ras_error_count, | 2792 | .query_ras_error_count = sdma_v4_0_query_ras_error_count, |
| 2780 | .reset_ras_error_count = sdma_v4_0_reset_ras_error_count, | 2793 | .reset_ras_error_count = sdma_v4_0_reset_ras_error_count, |
| 2781 | }; | 2794 | }; |
| 2782 | 2795 | ||
| 2796 | static struct amdgpu_sdma_ras sdma_v4_0_ras = { | ||
| 2797 | .ras_block = { | ||
| 2798 | .hw_ops = &sdma_v4_0_ras_hw_ops, | ||
| 2799 | .ras_cb = sdma_v4_0_process_ras_data_cb, | ||
| 2800 | }, | ||
| 2801 | }; | ||
| 2802 | |||
| 2783 | static void sdma_v4_0_set_ras_funcs(struct amdgpu_device *adev) | 2803 | static void sdma_v4_0_set_ras_funcs(struct amdgpu_device *adev) |
| 2784 | { | 2804 | { |
| 2785 | switch (adev->ip_versions[SDMA0_HWIP][0]) { | 2805 | switch (adev->ip_versions[SDMA0_HWIP][0]) { |
| 2786 | case IP_VERSION(4, 2, 0): | 2806 | case IP_VERSION(4, 2, 0): |
| 2787 | case IP_VERSION(4, 2, 2): | 2807 | case IP_VERSION(4, 2, 2): |
| 2788 | adev->sdma.funcs = &sdma_v4_0_ras_funcs; | 2808 | adev->sdma.ras = &sdma_v4_0_ras; |
| 2789 | break; | 2809 | break; |
| 2790 | case IP_VERSION(4, 4, 0): | 2810 | case IP_VERSION(4, 4, 0): |
| 2791 | adev->sdma.funcs = &sdma_v4_4_ras_funcs; | 2811 | adev->sdma.ras = &sdma_v4_4_ras; |
| 2792 | break; | 2812 | break; |
| 2793 | default: | 2813 | default: |
| 2794 | break; | 2814 | break; |
| 2795 | } | 2815 | } |
24 more lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 71 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 246 lines aren’t shown here.
This diff is large: its 13 lines aren’t shown here.
This diff is large: its 137 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 384 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 235 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 2085 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 846 lines aren’t shown here.
This diff is large: its 231 lines aren’t shown here.
This diff is large: its 159 lines aren’t shown here.
This diff is large: its 294 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 448 lines aren’t shown here.
This diff is large: its 112 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 474 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 37 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 80 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 61 lines aren’t shown here.
This diff is large: its 197 lines aren’t shown here.
This diff is large: its 242 lines aren’t shown here.
This diff is large: its 254 lines aren’t shown here.
This diff is large: its 231 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 93 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 19 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 350 lines aren’t shown here.
This diff is large: its 207 lines aren’t shown here.
This diff is large: its 422 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 601 lines aren’t shown here.
This diff is large: its 53 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 425 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 337 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 369 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 487 lines aren’t shown here.
This diff is large: its 94 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 23 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 87 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 618 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 315 lines aren’t shown here.
This diff is large: its 127 lines aren’t shown here.
This diff is large: its 716 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 313 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 166 lines aren’t shown here.
This diff is large: its 1592 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 2589 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 918 lines aren’t shown here.
This diff is large: its 280 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 164 lines aren’t shown here.
This diff is large: its 269 lines aren’t shown here.
This diff is large: its 67 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 73 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 320 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 29 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 90 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 26 lines aren’t shown here.
This diff is large: its 359 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 256 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 30 lines aren’t shown here.
This diff is large: its 265 lines aren’t shown here.
This diff is large: its 83 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 7 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 2305 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 57 lines aren’t shown here.
This diff is large: its 2307 lines aren’t shown here.
This diff is large: its 43 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 39 lines aren’t shown here.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This diff is large: its 59 lines aren’t shown here.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
This diff is large: its 358 lines aren’t shown here.
This diff is large: its 33 lines aren’t shown here.
This diff is large: its 88 lines aren’t shown here.
This diff is large: its 8 lines aren’t shown here.
This diff is large: its 45 lines aren’t shown here.
This diff is large: its 17 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 261 lines aren’t shown here.
This diff is large: its 32 lines aren’t shown here.
This diff is large: its 375 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 21 lines aren’t shown here.
This diff is large: its 62 lines aren’t shown here.
This diff is large: its 91 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 95 lines aren’t shown here.
This diff is large: its 41 lines aren’t shown here.
This diff is large: its 50 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 12 lines aren’t shown here.
This diff is large: its 28 lines aren’t shown here.
This diff is large: its 20 lines aren’t shown here.
This diff is large: its 439 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 31 lines aren’t shown here.
This diff is large: its 138 lines aren’t shown here.
This diff is large: its 54 lines aren’t shown here.
This diff is large: its 52 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 255 lines aren’t shown here.
This diff is large: its 36 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 44 lines aren’t shown here.
This diff is large: its 35 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 130 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 69 lines aren’t shown here.
This diff is large: its 63 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 25 lines aren’t shown here.
This diff is large: its 11 lines aren’t shown here.
This diff is large: its 34 lines aren’t shown here.
This diff is large: its 9 lines aren’t shown here.
This diff is large: its 18 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 16 lines aren’t shown here.
This diff is large: its 15 lines aren’t shown here.
This diff is large: its 66 lines aren’t shown here.
This diff is large: its 10 lines aren’t shown here.
This diff is large: its 58 lines aren’t shown here.
This diff is large: its 24 lines aren’t shown here.
This diff is large: its 15192 lines aren’t shown here.
This diff is larger than a page shows: the files after these aren’t listed. Clone the repository to see all of it.